everything shows

express · Client Joined: Jan 03, 2007 Posts: 94

Hi,

I have had my RN php site up for a month now. Today while I was fiddling around I typed my url into the web browser. All is good. Then I clicked on "more results from this site" and I mean every single module, block, gcalendar event, etc. kept showing up on google. Even user accounts. I shut the site down immediately but there is still stuff now on google (users account stuff) that does not need to be public.

nuken · Posted: Sat Jan 02, 2010 6:15 pm

Is your members list module visible to guests? If so, make it only viewable to members.

express · Client Joined: Jan 03, 2007 Posts: 94

Yes I rechecked that and it was set to public. I changed it and others to registered only. What do I need to do besides that so JUST me domain name will show on Google and nothing else.
Thanks,
Express

nuken · Posted: Sat Jan 02, 2010 6:48 pm

If it is viewable to guests, it is viewable to google. If you don't want search engines indexing your site, you could ban their ip ranges. You could also redirect them to a separate page with whatever info you want them to see.

express · Client Joined: Jan 03, 2007 Posts: 94

I vaguely remember a couple of years ago something said in here about possibly putting something more restrictive in the robots.txt file to disallow all from crawling site.
Also, how do I stop the sites from "caching" pages of my site?

edited...

I found these 2 scripts but do not know where they should go.

<meta name="robots" content="noarchive">

<meta name="robots" content="noindex">

nuken · Posted: Sun Jan 03, 2010 11:13 pm

Only registered users can see links on this board!
Get registered or login to the forums!

express · Client Joined: Jan 03, 2007 Posts: 94

Well I am still not having success. I installed the meta tags into the header.php file. I then went to google webmaster tools and had them remove the urls/cahced pages that I did not want indexed. I made sure that my modules/blocks/etc in my site were for registered users only.

here is my robots file

User-agent: *
Crawl-delay: 5
Disallow: /abuse/
Disallow: /admin/
Disallow: /blocks/
Disallow: /cgi-bin/
Disallow: /db/
Disallow: /images/
Disallow: /includes/
Disallow: /language/
Disallow: /modules/
Disallow: /ShortLinks/
Disallow: /themes/
Disallow: /admin.php
Disallow: /config.php
Disallow: /modules.php

Now when I do a search on google of my site, it still is pulling up pages from parts of my site that I do not want up.

All I want showing when someone searches my site is my main url such as

Only registered users can see links on this board!
Get registered or login to the forums!

Nothing else. What I am doing wrong?