Is this a Google bot, or an intruder?

captureroftyme · New Member Joined: Jan 08, 2005 Posts: 19

Hey all, i just got sentinel installed and it already blocked an ip from trying to get info from my admin section. The ip is 66.249.66.39 , I looked it up and it says google, but is this a google bot or an intruder?
If its a google bot, I DO want them to check my site, not sure about my admin stuff though. But if this is a normal thing, like google spiders, I want it to be able to index my site, so how do I unblock it?

manunkind · Posted: Sat Jan 08, 2005 10:56 am

That is Google. What do you have in your robots.txt file?

captureroftyme · New Member Joined: Jan 08, 2005 Posts: 19

This is a copy of my robots.txt file

User-agent: Mediapartners-Google*
Disallow:
User-agent: *
Disallow: admin.php
Disallow: /admin/
Disallow: /images/
Disallow: /includes/
Disallow: /themes/
Disallow: /blocks/
Disallow: /modules/
Disallow: /language/

Can I get rid of the robot.txt file completely, I mean I know I can, but what will be the result?

Nuke_Royal · New Member Joined: Nov 20, 2004 Posts: 16

that they end up spidering your admin panel lol
and you get nice direct hits to your admin...
but hey it is traffic....

captureroftyme · New Member Joined: Jan 08, 2005 Posts: 19

Great, TY. Now how do i unban that google bot?

captureroftyme · New Member Joined: Jan 08, 2005 Posts: 19

Sentinel just caught another one trying to access my admin panel, but they were trying to look in the flazh nav box I have. Is this a normal thing, I mean should they be doing this and is it dangerous to my site?
The new IP it caught was 66.249.65.165. If this is normal for search engines, then how do I unban them?

TheosEleos · Posted: Thu Jan 13, 2005 12:19 pm

Go into Sentinel and the blocked IP menu. Choose display blocked IP's and find your ip and remove it.

If you are also writing to .htaccess you will need to remove the ip from there too.

captureroftyme · New Member Joined: Jan 08, 2005 Posts: 19

Ok, I was able to unban the google bots, but it just keeps banning them. I added googles ip range to the protected list and it STILL bans them! How do I keep it from banning the google bots? I need them to be able to spider my site.

captureroftyme · New Member Joined: Jan 08, 2005 Posts: 19

Where can I find an uninstall script, or directions. If this thing is going to ban google everytime they try to index me, and there is no way to unban them and KEEP them unbanned, then I dont want it.
Please tell me how to uninstall sentinel.

Raven · Posted: Sun Jan 16, 2005 11:04 pm

You do not need to uninstall NukeSentinel. Google and Yahoo come stock and standard protected ranges. In your protected range admin panel, just add any ranges that you need. If those ranges are already in there, then you have to make sure that you don't have the IP's in the ban table nor in .htaccess.

captureroftyme · New Member Joined: Jan 08, 2005 Posts: 19

I added them to the protected range, removed them from the banned ip list, and removed them from the htaccess file. But, it still keeps banning them.

Raven · Posted: Mon Jan 17, 2005 4:40 am

If you are not using NukeSentinel v2.1.3, please upgrade and see if you still have the problem.

captureroftyme · New Member Joined: Jan 08, 2005 Posts: 19

I am using v2.1.3 and I got this morning and all 3 of the google bots were banned AGAIN! Please tell me how to keep them from being banned or tell me how to uninstall this thing.

truckerclock · New Member Joined: Jan 18, 2005 Posts: 7

I am getting the same thing. I have to turn off the admin ban to allow google through. I have admin.php blocked in the .htaccess file but somehow they get through and get banned. I hope there is a resolution for this.

Raven · Posted: Tue Jan 18, 2005 5:13 am

captureroftyme wrote:

I am using v2.1.3 and I got this morning and all 3 of the google bots were banned AGAIN! Please tell me how to keep them from being banned or tell me how to uninstall this thing.

Reverse your installation steps.

Raven · Posted: Tue Jan 18, 2005 5:19 am

truckerclock wrote:

I am getting the same thing. I have to turn off the admin ban to allow google through. I have admin.php blocked in the .htaccess file but somehow they get through and get banned. I hope there is a resolution for this.

Have you made sure that your robots.txt has, at a minimum, these lines:

User-agent: *
Disallow: admin.php
Disallow: /admin/

truckerclock · New Member Joined: Jan 18, 2005 Posts: 7

It has

Code:

User-agent: Mediapartners-Google*
Disallow:

User-agent: *
Disallow: /admin
Disallow: admin.php

Is that right?

Raven · Posted: Tue Jan 18, 2005 8:30 am

I had problems with these lines:
User-agent: Mediapartners-Google*
Disallow:

That is your problem, imo. You are telling Google that it has access to everything at your site. remove these lines:

Code:

User-agent: Mediapartners-Google*
Disallow:

That should fix it. You could also do

Code:

User-agent: Mediapartners-Google*
Disallow: /admin
Disallow: admin.php

truckerclock · New Member Joined: Jan 18, 2005 Posts: 7

Thanks for the quick reply. I have removed those two lines. I should know within an hour or so if that fixes it.

captureroftyme · New Member Joined: Jan 08, 2005 Posts: 19

I finally got it to stop banning them. I had added googles ip range to the protected ip range once but it didnt register or something. I added it again and it has been 3 days since it has banned anyone

truckerclock · New Member Joined: Jan 18, 2005 Posts: 7

I removed the two lines specified by raven from the robots.txt file and added the IP's and have been okay also since.

Racine · New Member Joined: Aug 07, 2004 Posts: 8

Raven wrote:

You do not need to uninstall NukeSentinel. Google and Yahoo come stock and standard protected ranges. In your protected range admin panel, just add any ranges that you need. If those ranges are already in there, then you have to make sure that you don't have the IP's in the ban table nor in .htaccess.

I just installed Nuke Sentinel 2.1.1. I was referred to this post, as Google is blocked from "harvesting" news from my site.

I didn't have anything in the Protested Range menu. So I added some IP ranges which I thought weere googlebots. But, should there already be IP ranges in the protected IP Range?

Is there a way to add them using the install scripts? I ran an "new install" of Sentinel, when I was really upgrading. Is this why I don't have the ranges in there?

sixonetonoffun · Posted: Tue Feb 08, 2005 11:15 am

I've been kind of searching for those ranges but the best collection I have found on the net is this list of googlebot ips

Only registered users can see links on this board!
Get registered or login to the forums!

psynegy · New Member Joined: Mar 02, 2005 Posts: 1

I use Gmail (google e-mail) and i recived a link to a website with Sentinal installed and it banned me automatically. And now I cannot acess this website, unless I get a response from the owner of this website. Does this help at all? It may be that gmail uses a file to link to the website that the link points to therefore acting as a possible security threat as it probably uses the method; post. Please tell me if this the link to your website so I can do a test email me the link at

Only registered users can see links on this board!
Get registered or login to the forums!