HowtoInstall folder

Posted: Mon Sep 20, 2010 12:32 pm

I'm getting big overages with my host provider Evil or Very Mad

and trying to remove and delete anything that is not necessary. on my usage report or as the call it GPU usage reports, this file gets accessed a lot or a top file use, method "GET"

/HowToInstall/community/images/vpreview/modules.php

Is it necessary to keep the HowToInstall folder in the Raven installation?
Any other tricks you guys do to keep resources down?

Thanks in advance Very Happy

Posted: Mon Sep 20, 2010 3:13 pm

That file and all the directories you referenced don't even exist in the distribution. To the best of my knowledge there have never even been directories like that (the community/images/vpreview stuff). You might want to compare a fresh download of the distribution with your file structure ... I suspect there may be a lot of unneeded stuff on your server.

Modules.php does exist in the root directory of your distribution (usually named html) and would get accessed a lot. But I'd start by taking a close look at my file layout.

Posted: Mon Sep 20, 2010 4:17 pm

Ok, while I don't have anything like that on my directory. This is an example of one of the files & scripts that it's taking up some resource:
[ Only registered users can see links on this board! Get registered or login! ]

That file does not even exist??? when you click the link however something tries to pull up. How is this possible?

html/HowToInstall/ and everything in it folders/files needed for Raven?

jc

Posted: Mon Sep 20, 2010 8:22 pm

HowToInstall has largely been replaced by the Wiki. In fact, the HowToInstall now points to the Wiki. If you have anything in there besides a QuickStart and index.html then you have outdated files ... at least for the latest Ravennuke. First I'd figure out how the extra files got there in the first place, then I'd get rid of them. To answer your qeustion directly, HowToInstall is not needed ... it just points to a Wiki page at this point.

Posted: Mon Sep 20, 2010 10:08 pm

fkelly, thank you very much! i had old files from previous install in there!

Best,
jc

Posted: Tue Sep 21, 2010 12:31 am

But why or what in Raven script make the link come up? and why is this link taking up resources. I did remove unwanted files.
[ Only registered users can see links on this board! Get registered or login! ]

jc

Posted: Tue Sep 21, 2010 7:41 am

There is nothing in the Ravennuke script that would point you to the link you listed. You might want to do some sort of global search through your files to see what is pointing to /community/images/vpreview ... and then get rid of it. If you have the files on a PC, most PHP editors can do such a global search and I think that on hosting services CPANEL would look through your files for such a string.

Posted: Tue Sep 21, 2010 5:24 pm

sorry for my ignorance, but why would it not point you to a 404 page or something like this if the page does not even exist. here it is with raven site
[ Only registered users can see links on this board! Get registered or login! ]

obviously the files don't exist yet the string url will act like it's pulling up a site?
I have a bunch of /index.php/HowtoInstall/ url strings taking up resource for what ever reason

jc Confused

Posted: Wed Sep 22, 2010 8:22 am

I'm not saying the page doesn't exist, obviously it does and it looks like it is pulling up a dated copy of Raven's home page. It is just that it should not exist or at least that it is not needed. You don't need anything inside of /HowToInstall except what's in the distribution (an index.html and a QuickStart page that points to the RNWIKI). The rest can be deleted. But I'd be curious about why they are there in the first place. They didn't get there by accident.

Posted: Wed Sep 22, 2010 11:47 am

You do not need the HowToInstall folder period. The only reason it is there is to point people to the wiki when they are installing. I do not even upload it to a site.

NeapolitanWorld, I could have sworn I posted in this topic yesterday about the 404 error page. It is a known issue when there is more than one sub-folder in a url that does not exist the error handler does not work right.

Posted: Wed Sep 22, 2010 6:00 pm

Palbin, cool thanks Very Happy

also at fkelly for your time. yes i think the error handler is in this case not working.

Posted: Sat Oct 02, 2010 12:16 pm

Ok, so not sure if to start a new thread but decided to post because its the same problem with the "GPU" on my server, or Grid Performance Unit hitting overages. I am getting a lot of usage from 404 pages on my site to links that are from the the Raven site, the string look like this or very similar:

neapolitanworld.com/HTTP:/www.neapolitanworld.com/community/forum-1.html

the actual correct link is: [ Only registered users can see links on this board! Get registered or login! ]

what script could be actually doing this? any ideas anyone, not sure of where to even start.

Posted: Sat Oct 02, 2010 12:59 pm

I think it could be shortlinks, but I could be wrong.

Posted: Sat Oct 02, 2010 2:42 pm

my thinking also, perhaps my menu? also, that large [ Only registered users can see links on this board! Get registered or login! ] is in the settings for external links. i just removed my 404 handler and at least not get hit with them...no 404 page for now but no biggy

Posted: Tue Oct 05, 2010 10:30 am

Based on what I have read so far and the capitalised HTTP I think the problem here is that you are using the Sommaire Menu block and something is interfering with it.
If I remember correctly when creating manual links in Sommaire it processes fully qualified links using HTTP and http differently. One is processed by Sommaire as an internal link the other as an external link

Smile

Posted: Wed Oct 06, 2010 12:26 am

Guardian2003, yea i was thinking the menu but did not remember the name of it lols. removing the 404 worked like a charm, about 99% of my 404 usage is gone. no 404 page oh well... now i still have the original problem from the thread...and my overages will continue. any idea how to prevent these from taking up any resource? honestly i'm just puzzled as to how or why these url strings are taking up any resource. i disabled the tego shortlinks but these url string still pull up so i don't think it's that.

Quote:

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]
[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]
[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]

[ Only registered users can see links on this board! Get registered or login! ]
[ Only registered users can see links on this board! Get registered or login! ]

Posted: Wed Oct 06, 2010 1:16 am

Sure, if you have a list of the offending url's you can permanently block them with htaccess, which would then allow you to put your 404 page back.
I mentioned this problem in a forum here some years ago where none existent url's where actually seen in a browser (the page is usually text content without any styling) and in some cases the none existent pages even get indexed by search engines because of the rather stupid way that Yahoo (Inktomi) behaves.

When Yahoo indexes your site it actually makes up random url's as it goes along to test for correct 404 and 403 operation so if they aren't being handled correctly, it sees a valid link and hey presto, a none existent url string is indexed. I had this problem with my first phpnuke (not RavenNuke) site - the end result being massive over usage of memory and processor resources.

The first thing I would do is stop using the Error module (if you are using it) - mail me and I can send you some replacement stuff.
Then I would probably try and rule out the Sommaire menu - if you need a 'pretty' menu, hard code one in a new block and see how it goes.
Keep lists of all the suspect 404 url strings and cross reference them in Nuke Sentinel, it's possible you are getting hit by the same referrer or IP - it's easier to block those but other wise your going to have to add those strings to htaccess to block them - a painfully long but worthwhile process..

Posted: Wed Oct 06, 2010 8:43 am

Humm ... reading Guardian's post gives me a couple of ideas you could try. First, I'd look at IP tracking (display tracked IP's) in NS and see if you can see where these strings are coming from. Block the IP's that are issuing them. If that makes the problem go away ... fine.

If not, I'm wondering if you used the string blocker in NS to block say "images/vpreview" whether that would take care of the situation. This would, if I am not mistaken, block any IP that sends the string "images/vpreview" at you and would save you from having to manually code your htaccess file. Might as well use the tools we have available.

Posted: Wed Oct 06, 2010 12:42 pm

Guardian2003, thanks for your help! error module is not being used. Once I removed the 404 handler all 404 usage went away. Once I have time I'll replace the menu, kinda like it Embarassed

fkelly, I tried the string blocker both "/images/vpreview" and as well as entire string and no luck Sad

I used the IP display tracker and go this for top IP's the last numbers being the number of hits "18998" on the first for example:
IP Address Last Viewed Hits Functions
Anonymous 67.195.115.42 (us) United States 2010-10-06 @ 10:16:44 18998
Anonymous 66.249.71.21 (us) United States 2010-10-05 @ 06:01:18 14349
Anonymous 66.249.67.41 (us) United States 2010-10-06 @ 10:17:44 11854
Anonymous 77.88.29.246 (ru) Russian Federation 2010-10-06 @ 09:05:30 7486
Anonymous 65.55.25.150 (us) United States 2010-10-06 @ 08:48:33 7476
Anonymous 65.55.3.169 (us) United States 2010-10-02 @ 06:02:45 6919
Anonymous 79.142.66.55 (it) Italy 2010-10-06 @ 06:24:34 4707
Anonymous 66.249.71.73 (us) United States 2010-09-29 @ 11:51:29 3975
Anonymous 66.249.71.117 (us) United States 2010-09-30 @ 06:43:29 3413
NeapolitanWorld 7x.xxx. (us) United States 2010-10-06 @ 09:02:37 1259

I have accessed my raw log file and have a couple of examples from a couple of those top IPs:

66.249.67.41 - - [06/Oct/2010:08:00:00 -0700] "GET /neapolitanworld.com/community/index.php/HowToInstall/HowToInstall/images/community/images/vpreview/modules.php?name=coppermine&file=displayimage&album=12&pid=17196 HTTP/1.1" 200 16697 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

66.249.67.41 - - [06/Oct/2010:08:00:43 -0700] "GET /neapolitanworld.com/community/index.php/HowToInstall/HowToInstall/images/community/images/vpreview/modules.php?name=coppermine&file=displayimage&album=253&pid=10340 HTTP/1.1" 200 16617 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

67.195.115.42 - - [06/Oct/2010:08:05:22 -0700] "GET /neapolitanworld.com/community/ftopict-3772.html HTTP/1.0" 200 6811 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; [ Only registered users can see links on this board! Get registered or login! ]

Looks like the google bot on the first 2? Guardian I think I might have that same problem that you did with your nuke site. What did you do to fix? also if anyone knows how to block a particular url string with .htaccess file that would be appreciated. Googled it and tried to find it here but was not able to find Embarassed

thanks!
jc

Posted: Wed Oct 06, 2010 1:52 pm

btw i do have a robots.txt file to disallow the HowToInstall/ folder.
should I try this on my robots.txt file:

Disallow: /community/index.php/HowToInstall/HowToInstall/

not sure because i have that string of index.php

Posted: Wed Oct 06, 2010 2:03 pm

fkelly wonders why the string blocker didn't work. First of all, in configuration for NS do you have the string blocker turned on? Second, I'm guessing you entered "/images/vpreview" or some such in the string menu. Third, are you running as admin when testing. If so, NS may bypass testing your string. You may need to test as a non-admin and also from an IP that is not protected.

Posted: Wed Oct 06, 2010 2:39 pm

fkelly, thanks for your input in this. very appreciated!

Here are the strings I added:

HowToInstall/HowToInstall/

images/vpreview
[ Only registered users can see links on this board! Get registered or login! ]

When I enter that last one, full string I get the same thing, the site trying to open the url. Not sure if that's suppose to continue to happen with trying to black a string?

jc

Posted: Wed Oct 06, 2010 2:44 pm

yea, i do have it on because just tried to add:
/images/vpreview
and banned my self....lols, which table on the DB do I go un-ban myself...lols?

Posted: Wed Oct 06, 2010 2:59 pm

You need to delete the IP from your htaccess file (it will be at or near the bottom of the list) and also from the nsnst_banned_ips table.

The site will try to open the url because it tries to do that before it gets to NukeSentinel. But then the IP address that's requesting that url will get banned and won't be able to get back in. So, unless you have a whole lot of IP's hitting at that url it should take care of the situation. I think ... someone who understands servers better than I can chime in but I think that if an IP comes in and is in the htaccess deny from area it won't use many resources but will get rejected from the outset.

Posted: Wed Oct 06, 2010 3:40 pm

The only problem with using NS for this though is that if the referrer was genuinely Google or another search engine, you're going to end up banning them.