| Author |
Message |
mrix
Client
Joined: Dec 04, 2004
Posts: 757
|
 Posted:
Mon May 16, 2005 2:09 am |
 
|
Hello all, I have always used the default Robots.txt file and never make any changes to it, I was wondering if I should be making changes to it to help in search engines etc or does it as default do its job ok?
Cheers
mrix |
|
|
 
|
|
Susann
Moderator
Joined: Dec 19, 2004
Posts: 3143
Location: Germany:Moderator German NukeSentinel Support
|
| Posted:
Mon May 16, 2005 4:50 am |
|
You can check your robots.txt here:
Itīs a reall good robots.txt syntax checker. |
Last edited by Susann on Mon May 16, 2005 4:59 am; edited 1 time in total |
|
|
|
|
mrix
Client
Joined: Dec 04, 2004
Posts: 757
|
| Posted:
Mon May 16, 2005 4:56 am |
|
Hi I tried that test out and had one error which said
Disallow: admin.php
We advise you to start a file/directory name with a leading slash char (Example: /private.html).
dont understand it to be honest?
thanks for your help
mrix
p.s
my robots.txt looks like this
User-agent: Mediapartners-Google*
Disallow:
User-agent: *
Disallow: admin.php
Disallow: /admin/
Disallow: /images/
Disallow: /includes/
Disallow: /themes/
Disallow: /blocks/
Disallow: /modules/
Disallow: /language/
actually this is the standard robots.txt file for 7.6 and 7.7? is it wrong? |
|
|
|
|
|
Susann
Moderator
Joined: Dec 19, 2004
Posts: 3143
Location: Germany:Moderator German NukeSentinel Support
|
| Posted:
Mon May 16, 2005 5:04 am |
|
Try this for admin:
User-agent: *
Disallow: /admin.php
Disallow: /admin/ |
|
|
|
|
|
mrix
Client
Joined: Dec 04, 2004
Posts: 757
|
| Posted:
Mon May 16, 2005 5:56 am |
|
Hello again, I have tried that test again with your changes and all is pasted fine, I cant understand though why would the makers of phpnuke would have as default the robots.txt the other way etc?
thanks again
mrix |
|
|
|
|
|
Susann
Moderator
Joined: Dec 19, 2004
Posts: 3143
Location: Germany:Moderator German NukeSentinel Support
|
| Posted:
Mon May 16, 2005 6:46 am |
|
I donīt know. But i checked my Nuke 6.5 severall times with different robots.text checkers before and this one is the best of all. There are also no errors, if u use crawl-delay for some bots. |
|
|
|
|
|
Dauthus
Worker
Joined: Oct 07, 2003
Posts: 211
|
| Posted:
Tue May 17, 2005 4:59 pm |
|
For some good info on robots.txt, look at this one:
Also, here is a really good set of tutorials:
|
|
|
|
|
|
|
|
|
|
![[Valid RSS]](http://www.ravenphpscripts.com/images/valid-rss.png "Validate my RSS feed"){kind=small}
:: 
::
