Ravens PHP Scripts: Forums
 

 

View next topic
View previous topic
Post new topic   Reply to topic    Ravens PHP Scripts And Web Hosting Forum Index -> General/Other Stuff
Author Message
Tizwit
Involved
Involved


Joined: Aug 29, 2004
Posts: 324
Location: New Mexico

PostPosted: Sat Oct 01, 2005 5:10 pm Reply with quote

I was wondering if anyone knows who runs these search bots:

inktomisearch.com

I get many of them at a time on the site.

_________________
Brian Only registered users can see links on this board! Get registered or login!
Helping the Children in the NM Children's Hospital 
View user's profile Send private message Visit poster's website
Susann
Moderator


Joined: Dec 19, 2004
Posts: 3191
Location: Germany:Moderator German NukeSentinel Support

PostPosted: Sat Oct 01, 2005 6:00 pm Reply with quote

Thats Yahoo.If you check your logfiles you´ll find something like this:

GET /robots.txt HTTP/1.0" 200 5235 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; Only registered users can see links on this board! Get registered or login!"

Slurp is sometimes a problem. But you can use also the crawl delay for slurp.

If you wish to get in the index of msn and some other search engines you should not ban slurp. Smile Just read the FAQ for Webmaster at yahoo.
 
View user's profile Send private message
Tizwit
PostPosted: Sat Oct 01, 2005 6:06 pm Reply with quote

thanx.. Its just annoying. I have been hit by almost a 1000 since this site has opened.. 300 yahoo, 300 msn, 300 google. and 100 others.

my diabetesbytes site will have 20-30 google bots at once on the site..

very annoying
 
Susann
PostPosted: Sat Oct 01, 2005 6:22 pm Reply with quote

20-30 google bots is normal. But some people hate slurp, because of his behavior.I have no problems with slurp. There are other bots very rarely bots and bots who ignore the robots.txt. and these bots are a problem.
 
Display posts from previous:       
Post new topic   Reply to topic    Ravens PHP Scripts And Web Hosting Forum Index -> General/Other Stuff

View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2007 phpBB Group
All times are GMT - 6 Hours
 
Forums ©