Ravens PHP Scripts: Forums
 

 

View next topic
View previous topic
Post new topic   Reply to topic    Ravens PHP Scripts And Web Hosting Forum Index -> NukeSentinel™
Author Message
bugsy
Worker
Worker


Joined: May 08, 2007
Posts: 130

PostPosted: Tue Sep 04, 2007 3:16 am Reply with quote

Hi

Our site was loading fine properly at Only registered users can see links on this board! Get registered or login! with Force Nuke URL option turned on in Sentinel and site was also indexed properly in Google and was also getting a good amount of hits.

Few days back we have modified the .htaccess to do a permanent 301 redirect to Only registered users can see links on this board! Get registered or login!.. and turned off the Force Nuke URL option and all pages are loading properly but unfortunately hits became almost half.

So while trying to check whether pages are indexed properly in google search i came across this in the first result field in google search
Quote:
Press Booth: You have been blocked
You have been blocked from entering this site. You have attempted to bypass the Filter System on this site. All of the following information has been ... Only registered users can see links on this board! Get registered or login! - 2k -
you can see this for yourself here Only registered users can see links on this board! Get registered or login!

So does it mean due to the redirect or some reason Sentinel has blocked the google bot from indexing www redirected pages? If so then what is the work around it?

Thanks in advance
 
View user's profile Send private message Visit poster's website
kguske
Site Admin


Joined: Jun 04, 2004
Posts: 6383

PostPosted: Tue Sep 04, 2007 4:27 am Reply with quote

Beware of attacks that spoof Googlebot's IP address. They are specifically designed to harm your search engine rankings. You can protect the Googlebot IP range and attacks like that will still be blocked, even though they aren't added to the blocked IP table or htaccess.

_________________
I google, therefore I exist...
Only registered users can see links on this board! Get registered or login!
 
View user's profile Send private message
montego
Site Admin


Joined: Aug 29, 2004
Posts: 9449
Location: Arizona

PostPosted: Tue Sep 04, 2007 7:20 am Reply with quote

bugsy, I don't understand why you would have gone from press.xtvworld.com to Only registered users can see links on this board! Get registered or login! especially if you were already indexed with just "press.". I didn't think that was what you were trying to do in your original thread on the topic. Of course it is going to take time to re-index for the "www.press". Sad

Did you also tell GoogleBot to only cache now the "www.press" version? You do this by logging into your web master account there.

Personally, I would have left it at "press." and redirected all the "www.press" to what was well indexed. Sad

_________________
Only registered users can see links on this board! Get registered or login!
Only registered users can see links on this board! Get registered or login! 
View user's profile Send private message Visit poster's website
bugsy
PostPosted: Tue Sep 04, 2007 8:04 am Reply with quote

ksgnuke - I checked Sentinel Panel and it has indeed blocked the Google Bot - at least its written googlebot 2.1! In fact it blocked Yahoo Slurp too. I have cleared both now. As for the .htaccess i found out though its chmod 666 Sentinel is not writing the IPs there - and i am getting lots of ip blocked email messages but they are showing neither in Sentinel Panel or .htaccess!

Montego - I know it was kinda stupid on my part! I tried a 301 redirect to Only registered users can see links on this board! Get registered or login! from all Only registered users can see links on this board! Get registered or login! queries and that fell on a redirect loop so decided to have a go with Only registered users can see links on this board! Get registered or login! - what i didn't foresee is Sentinel will block off Google Bot and Yahoo Slurp and even AdSense Bot who were trying to follow the 301 redirect!

So to make matters less complicated what i again have done is removed the redirect and again forcing nuke url to Only registered users can see links on this board! Get registered or login! like as before and hoping things will cool down a bit

speedtype ! !
 
montego
PostPosted: Wed Sep 05, 2007 5:50 am Reply with quote

What do you mean by "fell on a redirect loop"?

I would still log onto your Google web master account and tell Google to only index your now NON-www pages... Have you done this?
 
bugsy
PostPosted: Wed Sep 05, 2007 6:06 am Reply with quote

Ya in Google SiteMap panel i have now given Only registered users can see links on this board! Get registered or login! as the preference for crawling over www - but again the sitemap contains non GTapped URLs and Google Search is actually crawling the GTapped URLs..so i am kinda feeding two kind of information to google anyway ... which i do not think is good..

That loop is the message i got in the browser that the site is redirecting to a page that will keep on redirecting in a loop. I was trying to redirect Only registered users can see links on this board! Get registered or login! to Only registered users can see links on this board! Get registered or login!

But on second thoughts even if i do 301 that way again chances are there Sentinel will block bots who will try crawl the redirects..why i do not know....any idea on why .htaccess is not writing the IPs?...I did check that there is a space at the bottom of the .htaccess file..and chmod is 666 as instructed..
 
montego
PostPosted: Thu Sep 06, 2007 6:37 am Reply with quote

Quote:

That loop is the message i got in the browser that the site is redirecting to a page that will keep on redirecting in a loop.


That sounds to me like your rewrite rule wasn't "tight" enough. There should be no looping.

Quote:

But on second thoughts even if i do 301 that way again chances are there Sentinel will block bots who will try crawl the redirects..why i do not know.


I just can't see how or why this is happening. Makes no sense to me. Would have to ask the author of NukeSentinel that one (BobMarion).

Quote:

any idea on why .htaccess is not writing the IPs?...I did check that there is a space at the bottom of the .htaccess file..and chmod is 666 as instructed


There are numerous threads here which talk about all the common reasons why. I don't want to extend this thread into a different topic which has already been addressed many times. Feel free to post a new thread under NukeSentinel 2.5.x if you cannot find your solution, but please do search first as there is a wealth of info here and multiple identical threads / solutions just makes it all the harder for others to find their answers. Thanks.
 
bugsy
PostPosted: Thu Sep 06, 2007 8:27 am Reply with quote

Clarifying Loop:

Ya i might have used a bad re write rule..I think i used something like this
Code:
RewriteEngine On

RewriteCond %{HTTP_HOST} ^www\.press\.xtvworld\.com
RewriteRule ^(.*)$ http://press.xtvworld.com/$1 [R=301]


As for sentinel blocking google and yahoo bots - it can be for other reasons too but it happened just after putting in effect the redirect to Only registered users can see links on this board! Get registered or login!

Here are the messages i received on the block
Code:
User ID: Anonymous (1)

Reason: Abuse-Filter
--------------------
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Query String: Only registered users can see links on this board! Get registered or login!
Forwarded For: none
Client IP: none
Remote Address: xx.xxx.xx.xx
Remote Port: XXXXX
Request Method: GET

---------------------------------------------------------------------

User ID: Anonymous (1)
Reason: Abuse-Filter
--------------------
User Agent: Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Query String: Only registered users can see links on this board! Get registered or login!
Forwarded For: none
Client IP: none
Remote Address: XX.X.XX.XXX
Remote Port: XXXXX
Request Method: GET

-------------------------------------------------------------------

User ID: Anonymous (1)
Reason: Abuse-Filter
--------------------
User Agent: Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Query String: Only registered users can see links on this board! Get registered or login!
Forwarded For: none
Client IP: none
Remote Address: XX.X.XX.XXX
Remote Port: XXXXX
Request Method: GET


As for the .htaccess ya i will check on the forum....actually i was looking for some kinda solution other that loosening the permission more on the .htaccess as one hacker once replaced some of my files which were chmod 777! So thats one chmod i am weary of! Smile
 
montego
PostPosted: Fri Sep 07, 2007 6:33 am Reply with quote

bugsy, well, not sure because that rewrite rule looks reasonable with possibly one exception. Have you tried it with [L,R=301] instead of just the [R=301]? Well, you probably don't want to do that now given your original concern.

However, I still do not see how the redirect could be causing those blocks. What IS concerning to me are the following characters that I see in your URL strings:

%2520
%2526

Unfortunately, I am not sure how those are getting there. Have you added these IP addresses already to your NS Protected IPs or Ranges? If not, then I would expect you to still see these blocks even now. However, if you have added them, then you may be covering up the fact that the issue still exists.

.htaccess does NOT need 777. 666 should work just fine.

BTW, having a file be 777 permissions does not automatically mean someone can just arbitrarily overwrite it from the web. Something within your site's code has to allow that to happen... i.e., another hole.
 
bugsy
PostPosted: Fri Sep 07, 2007 11:37 am Reply with quote

No have not tried [L,R=301] will test it on some other site. In fact I also do not know how they evolved those urls..will add the IPs in Sentinel and so far it has not blocked Google/Yahoo Bot again after clearing the ips..

That hacker somehow managed to hack into lots of 777 folders both nuke and mambo and also changed files which were chmod 777. How he did that i don't know but did manage to get the issue sorted out later - and i am positive any 777 i leave lying around even a cache folder he will hack it! Looked like part of some hacking syndicate or something!!
 
montego
PostPosted: Fri Sep 07, 2007 6:54 pm Reply with quote

The point about 777 that I am making is that just having 777 on a file or directory doesn't, in of itself, expose it for being hacked (updated). What had likely happened is that you had another hole in your system which allowed the hacker to then upload a file or execute a PHP or shell command which then took advantage of the 777 permissions.

Yes, it is best to not be this wide open if you don't know exactly what you have on your system and how secure it is. Things like Attachment mods, galleries, vwar, chat, etc. which allow file system level commands to be executed are the notorious "holes".
 
bugsy
PostPosted: Sat Sep 08, 2007 12:52 pm Reply with quote

Ya i got your point but I am not exactly aware where the hole lies ...i believe upgrading to later versions will take care of lots of issues in one go as they will be more secured anyway!
 
Display posts from previous:       
Post new topic   Reply to topic    Ravens PHP Scripts And Web Hosting Forum Index -> NukeSentinel™

View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2007 phpBB Group
All times are GMT - 6 Hours
 
Forums ©