Ravens PHP Scripts: Forums
 

 

View next topic
View previous topic
Post new topic   Reply to topic    Ravens PHP Scripts And Web Hosting Forum Index -> General/Other Stuff
Author Message
sixonetonoffun
Spouse Contemplates Divorce


Joined: Jan 02, 2003
Posts: 2496

PostPosted: Thu Sep 16, 2004 8:10 am Reply with quote

I've noticed NC results are suffering from a lack of descriptions often lately too. I wonder if this is to do with efforts to crawl dynamic sites deeper?

storebuilder from a statistical stand point your view of compression may be true. But from a performance stand point every site should take advantage of compression and caching where ever possible and dynamic sites especially. On shared hosting it can make a huge increase in speed by reducing the number of database queries required to load a page. I've switched over to cached gzipped pages to anyone who isn't logged in and am caching several blocks even to users who are logged in. Even though these are just "junk sites" I really wouldn't want to go without either.

Now here is an anomaly to explain look at the first result for this search: Only registered users can see links on this board! Get registered or login!

The url is nukeresources but the description and cached link goes back to my site. The actual link goes back to nukeresources download page.

_________________
[b][size=5]openSUSE 11.4-x86 | Linux 2.6.37.1-1.2desktop i686 | KDE: 4.6.41>=4.7 | XFCE 4.8 | AMD Athlon(tm) XP 3000+ | MSI K7N2 Delta-L | 3GB Black Diamond DDR
| GeForce 6200@433Mhz 512MB | Xorg 1.9.3 | NVIDIA 270.30[/size:2b8 
View user's profile Send private message
sixonetonoffun
PostPosted: Thu Sep 16, 2004 8:14 am Reply with quote

Oh wait I get it now its the actual download link. So it redirects back to my site lol DOH!
 
SmackDaddy
Involved
Involved


Joined: Jun 02, 2004
Posts: 268
Location: Englewood, OH

PostPosted: Thu Sep 16, 2004 8:16 am Reply with quote

I know it's a simple thing which people can say they've checked, but I think maybe some of you might want to make sure none of the IPs blocked by Sentinel have been googlebots I think the IPs are the 64.68.*.* range? But then again, my site hasn't been hit by a googlebot in 24 hours and usually they hit several times daily......or.....maybe it was just me overlooking my sentinel hits from google bots crawling cached pages and getting banned and me not realizing until days later (not good)...... where's the "loser" smiley with the L on the forehead?!?! *grin*

_________________
Smack out...
Only registered users can see links on this board! Get registered or login!

Only registered users can see links on this board! Get registered or login! 
View user's profile Send private message Send e-mail Visit poster's website
storebuilder
PHP-Portal Project


Joined: Mar 09, 2004
Posts: 169
Location: Telford UK

PostPosted: Thu Sep 16, 2004 9:36 am Reply with quote

At this point can we objectively say that the following is true:

The fiapple theme has the correct stylesheet info giving "storytitle" at the start of a story and is the default theme which a search engine would see.

If your descriptions in google carry the words

Quote:
Donate o Meter. Help Support RavenPhpScripts and NukeSentinel™. Make donations with
PayPal! Donat-o-Meter Stats. September´s Goal: $300.00. Due Date: Sep 30. ...



Which they do - then google can read the page and we can rule out GZip. Because it is reading the first block on the page. If it couldn't see through gzip then it wouldn't see this text?

Do we agree that it can't be the stylesheet and can't be Gzip?
 
View user's profile Send private message Visit poster's website MSN Messenger
Raven
Site Admin/Owner


Joined: Aug 27, 2002
Posts: 17086

PostPosted: Thu Sep 16, 2004 9:45 am Reply with quote

IMO, yes.
 
View user's profile Send private message
storebuilder
PostPosted: Thu Sep 16, 2004 12:34 pm Reply with quote

In which case can I suggest you make a simple HTML page with a page title called "googlebot woes give me the blues" or a page title of your choice - give it something obscure so that it will easily be a number one page.

Match it with a header

Oh bollŁ$%^ it's quicker to make it Wink

Quote:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>googlebot woes give me the blues</title>
<meta name="description" content="googlebot woes give me the blues">
<meta name="keywords" content="googlebot blues, freshbot">
</head>

<body>
<h1>Googlebot woes give me the blues
</h1>
<p>&nbsp;</p>
<p>They sure do those googlebot woes,</p>
<p>got me woeing into my beer,</p>
<p>woe, woe, is those googlebot blues,</p>
<p>gotta bring them right back, my dear. </p>
<p>Now I know you are from kentucky right? But I mixed it up with Kansas for this one Wink. Just pluck E, D, F. Wink. </p>
<p>&nbsp;</p>
<a href="http://www.ravenphpscripts.com">Back to index page
</a>
<p>&nbsp;</p>
<p>&nbsp; </p>
</body>
</html>


Now paste this into the root directory of your site and link to it from your home page.

Then we'll see if it gets indexed properly and start apportioning blame from there Wink
 
storebuilder
PostPosted: Thu Sep 16, 2004 1:12 pm Reply with quote

I see what you mean sixone. But it doesn't explain why the "descriptions" should deteriorate does it?

Deep crawling is just an expression for indexing the whole site. Wheras the freshbot indexes new content daily. So I don't see the correlation.

Let's see what Raven get's by putting up the HTML page and then we can all chew on the fat. Does that seem ok?


Last edited by storebuilder on Thu Sep 16, 2004 1:20 pm; edited 1 time in total 
Raven
PostPosted: Thu Sep 16, 2004 1:18 pm Reply with quote

I placed the link in the Partners block Wink
 
storebuilder
PostPosted: Thu Sep 16, 2004 1:31 pm Reply with quote

Yep - you have put me up on the cross fairly and squarely. Laughing

Where is Chatserv? He normally comes and says "oh, you do this...." Very Happy
 
sixonetonoffun
PostPosted: Thu Sep 16, 2004 3:50 pm Reply with quote

I think we're all concerned with the forums getting crawled since its the most active area of so many sites. Glad we have the 2 of ya on the job here! Wink
 
storebuilder
PostPosted: Mon Sep 20, 2004 6:00 am Reply with quote

I was just doing a search for a "center block links" when I noticed that the top result was ravenphpscripts and the title and description was displayed properly.

I did some more searching and started seeing your pages appearing in the top results all over!!!

Whatever it is that you have removed has done the trick.
 
storebuilder
PostPosted: Mon Sep 20, 2004 6:05 am Reply with quote

module copyrights not working

this search shows that your forum posts are now being indexed - looks like some major traffic is going to be headed your way!
 
Raven
PostPosted: Mon Sep 20, 2004 6:10 am Reply with quote

In that one, yes. But in the Old Articles block, when I searched for 'Chatserv has achieved fame in print', I'm not seeing that, exactly. Here's what came back
Code:
General purpose self-explanatory file with news headlines % ...... gif %% ChatterBox 2004 /modules.php?name=News&file=article&sid=515 2004-09-13 05:38:25 Raven Add-Ons 0 addon.gif %% Chatserv has achieved fame in print! ... 

ravenphpscripts.com/ultramode.txt - 2k - Cached - Similar pages
Is that what you would expect? And I still don't see the forums getting indexed or am I missing it?
 
Raven
PostPosted: Mon Sep 20, 2004 6:14 am Reply with quote

storebuilder wrote:
module copyrights not working

this search shows that your forum posts are now being indexed - looks like some major traffic is going to be headed your way!
Our posts must have crossed Smile I'll keep looking.
 
storebuilder
PostPosted: Mon Sep 20, 2004 6:24 am Reply with quote

Right - but you just need to have a bit of patience. The pages that you get crawled today will appear in the index in a couple of days time, but effectively your whole site needs to be reindexed so it's going to happen over a month or so.
 
storebuilder
PostPosted: Mon Sep 20, 2004 6:25 am Reply with quote

also - the example that I quoted above is a forum post - and has only been indexed since you changed to dynamic titles.
 
storebuilder
PostPosted: Mon Sep 20, 2004 6:27 am Reply with quote

And a search for this Only registered users can see links on this board! Get registered or login!

Shows you up in spot 9. That shows you are being indexed correctly.
 
Raven
PostPosted: Mon Sep 20, 2004 6:30 am Reply with quote

Dance-Y
 
storebuilder
PostPosted: Mon Sep 20, 2004 6:39 am Reply with quote

Dance-Stick ROTFL
 
Raven
PostPosted: Tue Sep 21, 2004 2:50 pm Reply with quote

Almost 800 times today Smile) And, a search for 'Googlebot woes give me the blues' comes up #1. I'm happy.

Edit: 865 times so far Wink
 
Muffin
Client


Joined: Apr 10, 2004
Posts: 649
Location: UK

PostPosted: Tue Sep 21, 2004 6:11 pm Reply with quote

After reading thru this thread with great interest, would it be possible for Storebuilder to put in one post what the rest of us have to do to get so many hits from the google bots, cos as the threads progressed it's got a bit confusing.

Not quite sure what dynamic titles are for a start or where to change them lol Sorry I'm only learning.

Thanks.

_________________
Classic Mini rules the bends & bends the rules!
[img] 
View user's profile Send private message
storebuilder
PostPosted: Fri Sep 24, 2004 11:33 am Reply with quote

Raven, did you decide what it was that was doing the damage?
 
Raven
PostPosted: Fri Sep 24, 2004 12:24 pm Reply with quote

No. I made so many changes. But the final change I made was to remove the google ads. However, before that I had added title tags to almost everything. So, somewhere amongst dynamic titles, theme changes, removing google ads, adding title tags, and offerring a blood sacrifice during the last full moon, it got fixed ROTFL
 
storebuilder
PostPosted: Fri Sep 24, 2004 12:35 pm Reply with quote

Yep - that's just what I would expect from a thoughtful, logical, programmer thinker. Laughing

Have you still got GZip turned off? Cause that's where my money goes....
 
Raven
PostPosted: Fri Sep 24, 2004 12:39 pm Reply with quote

Nope, it's on afaik.
 
Display posts from previous:       
Post new topic   Reply to topic    Ravens PHP Scripts And Web Hosting Forum Index -> General/Other Stuff

View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2007 phpBB Group
All times are GMT - 6 Hours
 
Forums ©