Ravens PHP Scripts: Forums
 

 

View next topic
View previous topic
Post new topic   Reply to topic    Ravens PHP Scripts And Web Hosting Forum Index -> General/Other Stuff
Author Message
storebuilder
PHP-Portal Project


Joined: Mar 09, 2004
Posts: 169
Location: Telford UK

PostPosted: Sat Sep 11, 2004 12:24 pm Reply with quote

Raven, can you tell me how many times googlebot visits your site on a daily basis?

It should be around the 200+ mark but I don't see your forum posts becoming visible in google yet.

Just a rough figure will do.
 
View user's profile Send private message Visit poster's website MSN Messenger
storebuilder
PostPosted: Sat Sep 11, 2004 12:35 pm Reply with quote

I just checked your listings in google and hardly anything has changed listings wise since you implemented dynamic titles.

Have you accidentally blocked googlebots from spidering your site?
 
Raven
Site Admin/Owner


Joined: Aug 27, 2002
Posts: 17086

PostPosted: Sat Sep 11, 2004 12:38 pm Reply with quote

Not to my knowledge. Do you want me to send you my robots.txt file? I sent it to you once but I will if you want.
 
View user's profile Send private message
storebuilder
PostPosted: Sat Sep 11, 2004 12:46 pm Reply with quote

Have you got an easy way of tracking visits?

I use msanalysis Only registered users can see links on this board! Get registered or login!

So you can see google visited 600+ times today.

There is something definitely amiss with your google listings. I think you suspected that?
 
Raven
PostPosted: Sat Sep 11, 2004 12:49 pm Reply with quote

Yes but I don't know what it is. I only show 104! What IP range does it use?
 
Raven
PostPosted: Sat Sep 11, 2004 1:05 pm Reply with quote

I made a change to my robots.txt. Can you email me yours?
 
storebuilder
PostPosted: Sat Sep 11, 2004 1:12 pm Reply with quote

Crawler 7 is 64.68.81.148 this is a new crawler that only turned up recently.

the main crawlers that you are missing are the freshbot ones 64.68.62.* - I think somehere, and probably not in your robots.txt but maybe in sentinel or your.htaccess you are denying them access.

I have been looking everyday for your post to turn up "where is audioslaved", not only because we want to know where he is but also because it's a measure of how your site is being indexed.

No one else (I would think) has the phrase "where is audioslaved" in their page title so once you see your site appear no 1 in google for this we are heading in the right direction.

RavensScripts
 
storebuilder
PostPosted: Sat Sep 11, 2004 1:17 pm Reply with quote

Mine is vanilla

User-agent: *
Disallow: admin.php
Disallow: /admin/
Disallow: /images/
Disallow: /includes/
Disallow: /themes/
Disallow: /blocks/
Disallow: /modules/
Disallow: /language/
 
Raven
PostPosted: Sat Sep 11, 2004 1:18 pm Reply with quote

Well, I'm not blocking the IP anywhere, so I'll just modify my robots.txt and see what happens.
 
Muffin
Client


Joined: Apr 10, 2004
Posts: 649
Location: UK

PostPosted: Sat Sep 11, 2004 1:19 pm Reply with quote

Did you put this code in your robots.txt file?

Quote:
User-agent: Mediapartners-Google*


before the disallows?

That helps.

Apologies if I'm teaching granny how to suck eggs here.
Embarassed

_________________
Classic Mini rules the bends & bends the rules!
[img] 
View user's profile Send private message
Raven
PostPosted: Sat Sep 11, 2004 1:20 pm Reply with quote

Yep
 
Raven
PostPosted: Sat Sep 11, 2004 1:22 pm Reply with quote

I am blocking these headers:
DELETE
SEARCH
TRACE

Do you know if Google uses these?
 
storebuilder
PostPosted: Sat Sep 11, 2004 1:28 pm Reply with quote

Can you check your logs to see if 64.68.62.* is accessing your site.
 
Raven
PostPosted: Sat Sep 11, 2004 1:30 pm Reply with quote

I'm sending you an email!
 
storebuilder
PostPosted: Sat Sep 11, 2004 1:39 pm Reply with quote

I don't know how you do this?

I am blocking these headers:
DELETE
SEARCH
TRACE

Blocking these headers where?
 
Raven
PostPosted: Sat Sep 11, 2004 3:06 pm Reply with quote

In NukeSentinel. Do you know if Google uses the Search?
 
Raven
PostPosted: Sat Sep 11, 2004 4:48 pm Reply with quote

storebuilder wrote:
Have you got an easy way of tracking visits?

I use msanalysis Only registered users can see links on this board! Get registered or login!

So you can see google visited 600+ times today.

There is something definitely amiss with your google listings. I think you suspected that?
Your site is unreachable right now.

Edit: Must be the same problem(s) you've been having with your host - it's up and down.
 
storebuilder
PostPosted: Sun Sep 12, 2004 9:26 am Reply with quote

Oh I see. No. I don't know.
 
Raven
PostPosted: Sun Sep 12, 2004 10:52 am Reply with quote

I have been reading all morning about 'googlebot no longer visits my site' and the general concensus is that 'who knows why'? It's very frustrating as it used to hammer my site and now is alomost never here. I cannot thing of anything that I have done or haven't done.
 
Muffin
PostPosted: Sun Sep 12, 2004 5:05 pm Reply with quote

I think if the content stays basically the same which it does here, as it's a nuke and sentinel support site that once they know that that is the content they no longer need to visit so often.

It's not like a news site where the content is changing daily or weekly or even monthly.

Just a guess. I mean if you posted an article up about a different topic like naturism or clockwork rabbits they'd notice it was new content and pay the site more visits again.

(hope ya dont mind me butting in)
 
storebuilder
PostPosted: Mon Sep 13, 2004 1:09 pm Reply with quote

Nice guess muffin, but wide of the mark in this case.

One of the reasons for rewriting the url's so that they appear as HTML is that they carry more importance than database driven pages.

If you were a search engine and someone had 2,000 html pages which were updated regularly then you would figure that someone is putting a lot of effort into that site and so it must be important.

In any case there is a considerable amount of activity in the forums here and as google loves topical content it should be indexing it immediately.

To highlight what the problem is with this site then look at this post Only registered users can see links on this board! Get registered or login! and spot the datestamp and then do this search Only registered users can see links on this board! Get registered or login! - comparatively my forums are a graveyard as I'm still building traffic in a very competitive field.

By combining googletap with a page title mod you are able to get new content listed in a search engine friendly way very quickly.

The funny thing is that you are getting 1,000 visits from msnbot daily and I get none whereas I get 1,000 from google and none from msn - so there's another missing piece in the jigsaw.

I've heard of other people who have experienced googlebot drying up on them so I'm off to see the wizard...
 
storebuilder
PostPosted: Mon Sep 13, 2004 1:31 pm Reply with quote

This is a guess - you are being penalised for duplicate content.

If you do a site:ravenphpscripts.com then the return is a list of pages with no descriptions. Just an option for "similar pages".

Because you have had the same page title for every single page then it's been considered duplicate content and so the googlebot's have dried up.

I found one post indexed since you did the dynamic titles mod and so here's an example search Only registered users can see links on this board! Get registered or login! to bring it up.

Maybe at this point it's best to have some patience and see if the bots start coming back to index your "new" content.
 
Raven
PostPosted: Mon Sep 13, 2004 1:36 pm Reply with quote

Thanks Tony and let me know if the 'Wizard' comes up with anything!
 
storebuilder
PostPosted: Mon Sep 13, 2004 1:50 pm Reply with quote

Oh he's good, he's really good Wink
 
SmackDaddy
Involved
Involved


Joined: Jun 02, 2004
Posts: 268
Location: Englewood, OH

PostPosted: Mon Sep 13, 2004 3:05 pm Reply with quote

I had some links in my downloads and weblinks which had parenthesis in them -- I changed these to brackets after Sentinel was installed when I realized people hitting those pages where getting banned. Unfortunately, for some reason, a googlebot has been banned before as a result of one of those pages, and it's still cached in Google's engine. Twice since, I have had to remove a googlebot from my banned list for accessing the cached page......was really weird, and the last time I only noticed this when I realized google wasn't spidering my site as much as it used to......

_________________
Smack out...
Only registered users can see links on this board! Get registered or login!

Only registered users can see links on this board! Get registered or login! 
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:       
Post new topic   Reply to topic    Ravens PHP Scripts And Web Hosting Forum Index -> General/Other Stuff

View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2007 phpBB Group
All times are GMT - 6 Hours
 
Forums ©