everything shows

Client Joined: Jan 03, 2007 Posts: 94

Hi,

I have had my RN php site up for a month now. Today while I was fiddling around I typed my url into the web browser. All is good. Then I clicked on "more results from this site" and I mean every single module, block, gcalendar event, etc. kept showing up on google. Even user accounts. I shut the site down immediately but there is still stuff now on google (users account stuff) that does not need to be public.

Posted: Sat Jan 02, 2010 6:15 pm

Is your members list module visible to guests? If so, make it only viewable to members.

Posted: Sat Jan 02, 2010 6:43 pm

Yes I rechecked that and it was set to public. I changed it and others to registered only. What do I need to do besides that so JUST me domain name will show on Google and nothing else.
Thanks,
Express

Posted: Sat Jan 02, 2010 6:48 pm

If it is viewable to guests, it is viewable to google. If you don't want search engines indexing your site, you could ban their ip ranges. You could also redirect them to a separate page with whatever info you want them to see.

Posted: Sun Jan 03, 2010 10:51 am

I vaguely remember a couple of years ago something said in here about possibly putting something more restrictive in the robots.txt file to disallow all from crawling site.
Also, how do I stop the sites from "caching" pages of my site?

edited...

I found these 2 scripts but do not know where they should go.

<meta name="robots" content="noarchive">

<meta name="robots" content="noindex">

Posted: Sun Jan 03, 2010 11:13 pm

[ Only registered users can see links on this board! Get registered or login! ]

Posted: Wed Jan 13, 2010 5:02 pm

Well I am still not having success. I installed the meta tags into the header.php file. I then went to google webmaster tools and had them remove the urls/cahced pages that I did not want indexed. I made sure that my modules/blocks/etc in my site were for registered users only.

here is my robots file

User-agent: *
Crawl-delay: 5
Disallow: /abuse/
Disallow: /admin/
Disallow: /blocks/
Disallow: /cgi-bin/
Disallow: /db/
Disallow: /images/
Disallow: /includes/
Disallow: /language/
Disallow: /modules/
Disallow: /ShortLinks/
Disallow: /themes/
Disallow: /admin.php
Disallow: /config.php
Disallow: /modules.php

Now when I do a search on google of my site, it still is pulling up pages from parts of my site that I do not want up.

All I want showing when someone searches my site is my main url such as [ Only registered users can see links on this board! Get registered or login! ] Nothing else. What I am doing wrong?