Ravens PHP Scripts: Forums
 

 

View next topic
View previous topic
Post new topic   Reply to topic    Ravens PHP Scripts And Web Hosting Forum Index -> NukeSentinel™
Author Message
hitwalker
Sells PC To Pay For Divorce


Joined:
Posts: 5661

PostPosted: Mon Oct 18, 2004 5:09 am Reply with quote

if noticed that sentinel 2.1.0 just blocked the visits from Mediapartners-Google/2.1

all visits blocked as :

Abuse-Admin
User Agent: Mediapartners-Google/2.1
Query String: admin.php?op=ABConfigFilter
Query String: admin.php?op=ABBlockedIP&min=0&column=date&direction=desc&sip=
Query String: admin.php?op=ABConfigAuthor


any way to prevent this ?

John
 
View user's profile Send private message
Raven
Site Admin/Owner


Joined: Aug 27, 2002
Posts: 17086

PostPosted: Mon Oct 18, 2004 6:40 am Reply with quote

John,

If you're using v2.1.0 you can protect IP ranges and that one is a protected range by default, I believe. If not, you can add the IP range in the Protected Range Panel. However, if you are using v2.0.2 then see this post to protect the range Only registered users can see links on this board! Get registered or login!
 
View user's profile Send private message
Nukeum66
Life Cycles Becoming CPU Cycles


Joined: Jul 30, 2003
Posts: 551
Location: Neurotic, State, USA

PostPosted: Mon Oct 18, 2004 6:42 am Reply with quote

Try adding this to your robots.txt

Code:
User-agent: Mediapartners-Google*

Disallow: admin.php

_________________
Scott Johnson MIS Ubuntu/Linux 11.10 
View user's profile Send private message Visit poster's website
hitwalker
PostPosted: Mon Oct 18, 2004 6:51 am Reply with quote

hi thanks for the reply,

the site in question is a subdomain,so the allowed is infact in the root so there shouldnt be that problem.
and for raven....i installed the 2.1.0 version.
there wasnt any version installed yet so ....
but i dont know exactly what you mean that google should already be excluded ?
But should the protection from google show as default?...cause it doesnt show as there isnt any IP's that are excluded....
And as i was typing it get banned another 2 times.... Sad
 
Raven
PostPosted: Mon Oct 18, 2004 6:55 am Reply with quote

In the Protected Range menu you should see a couple of entries like:

66.249.64.0 66.249.79.255

That should encompass that bot as I use it too. If that range is not in there, or if the range you have is different, just add it.
 
hitwalker
PostPosted: Mon Oct 18, 2004 7:12 am Reply with quote

well i looked and when you use the function to show the excluded range its empty,so yes i entered the full range now.
so will see what happens now...
thanks.
 
Raven
PostPosted: Mon Oct 18, 2004 7:19 am Reply with quote

Not Excluded Range, Protected Range.
 
hitwalker
PostPosted: Mon Oct 18, 2004 7:26 am Reply with quote

yes oops sorry...
i changed it...
see if it will work...
google just got banned again....
 
Raven
PostPosted: Mon Oct 18, 2004 8:07 am Reply with quote

Try this in your robots.txt

User-agent: *
Disallow: admin.php
Disallow: /admin/
Disallow: /images/
Disallow: /includes/
Disallow: /themes/
Disallow: /blocks/
Disallow: /modules/
Disallow: /language/
Disallow: /abuse/
Disallow: /cgi-bin/
Disallow: /curl/
Disallow: /db/
Disallow: /docs/
Disallow: /gcf/
Disallow: /public/
 
hitwalker
PostPosted: Mon Oct 18, 2004 8:22 am Reply with quote

ok thanks raven will add that also...
but i dont understand why it bans google now.
is that related to the new version of sentinel ?
On my nuke site i also use the 2.1.0 and isnt banning google.
so this is kinda weird behaviour...
 
Raven
PostPosted: Mon Oct 18, 2004 8:33 am Reply with quote

Under 2.0.2 on my site, google and media partners would get banned when they tried to go down the admin path. Once I modified my robots.txt and put that range in, I have not had any problems. It is weird, I agree.
 
hitwalker
PostPosted: Mon Oct 18, 2004 9:02 am Reply with quote

Well the difference between the 2 sites is nothing...both are now 2.1.0
Website 1 isnt banning google ,btw it never did...(even before the forbidden directories were added to the robot.txt

Website 2 is or was banning google and did that 30 minutes after install...
So for me trying to understand this it seems weird cause both are in the same situation and both respond and act differently.


As for the crawler story...like many others i have been crawled a million times by the msn bot and this would probably help someone cause this is what msn maild me some time ago....


Code:


User-agent: Mediapartners-Google*
Disallow:
User-agent: *
Disallow: /cgi-bin/
Disallow: /images/
User-Agent: msnbot
Crawl-Delay: 200
Disallow: /cgi-bin/
Disallow: /images/
 
This will tell MSNBot to wait 200 seconds between each request it makes to your server.



With this they offer a solution to slow down the msn request.
I didnt used it and completely banned msn from even entering the site.
But perhaps someone want to use this,but i havent seen it before so thats why i post it...[/code]
 
blith
Client


Joined: Jul 18, 2003
Posts: 977

PostPosted: Mon Oct 18, 2004 9:52 am Reply with quote

Raven, what do you have under disallow for User-agent: Mediapartners-Google*
 
View user's profile Send private message Visit poster's website
Raven
PostPosted: Mon Oct 18, 2004 11:24 am Reply with quote

blith wrote:
Raven, what do you have under disallow for User-agent: Mediapartners-Google*
That is my entire robots.txt file. I don't have any problems.
 
blith
PostPosted: Mon Oct 18, 2004 11:27 am Reply with quote

I ask becasue I went 2.1 IP tracking and found the googlebots in all the stuff I just uploaded!! They were all over the admin folder! And private messages and everything else I thought I had disallowed
 
Display posts from previous:       
Post new topic   Reply to topic    Ravens PHP Scripts And Web Hosting Forum Index -> NukeSentinel™

View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2007 phpBB Group
All times are GMT - 6 Hours
 
Forums ©