crawler14.googlebot.com

Posted: Sat Jul 17, 2004 4:38 pm

Hi Raven,

My site has havy traffic from a searchbot of Google crawler 10 to 15.. and are continusly on my site. With the Ip-Tracking module I can see that four different crawlers are online at the same time and for a very long time.

Could this have something to do with Sentinel configurations, I have tried some but without any succes.

Fred Wink

BTW
Your Custumdistro 7.3 however runs like a charm... thanks for your hard work and help.

Posted: Sat Jul 17, 2004 4:41 pm

I would say this is normal behaviour for the googlebots. What excactly is the problem?

Site Admin/Owner Joined: Aug 27, 2002 Posts: 17088

Sentinel does NOT influence crawlers. That's the price you pay for being GT'd or from having gotten their attention. That's a good thing Wink

Sentinel, otoh, keeps the unwanted bunch away.

Posted: Sat Jul 17, 2004 4:58 pm

I already whas on a high level, so maybe because of the changes I'am crawled more often. They are still on my site, and for a bot this is a longtime I gues.

Thanks,

Fred

Posted: Sat Jul 17, 2004 5:50 pm

I have had them on my site for 3 days straight, so, who knows?

Posted: Sat Jul 17, 2004 6:24 pm

They work in mysterious ways for sure. Big gaps between each page view.

Hangin' Around Joined: May 30, 2004 Posts: 46

We have anywhere from 10-15 of them on the site everyday.