Block malicious visitors with 404 page?

Site Admin Joined: Jun 04, 2004 Posts: 6432

Looking through some logs, and noticed that there are several attempts to find certain files (e.g. profile.php, wp-content). This is clearly a bot looking to identify sites that could be compromised:

Code:

[Sun Dec 07 04:21:21 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/editor


[Sun Dec 07 04:21:21 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/scripts


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/inc


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/inc


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/editor


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] client denied by server configuration: /zzz/public_html/admin/fckeditor


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/include


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/system


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/common


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/js


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/manage


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/fckeditor


[Sun Dec 07 04:21:18 2014] [error] [client 180.173.196.145] File does not exist: /zzz/public_html/includes/fckeditor

I saw this interesting article on a tool for blocking bad bots which uses an approach similar to Project Honeypot, and thought of using a 404 page script to unobtrusively block bad bots that might respect robots.txt directives. Basically, it would include a list of files checked by bad bots (not including files used by RavenNuke, for example). A 403 page script could automatically block them (though this could also block admins who forget their password, possibly).

You could use a massive (and growing) htaccess list of bad user agents like this, but like IP addresses, user agents can be spoofed. You could also use a non-htaccess, php-based approach like ZB Block , but that doesn't protect non-PHP pages.

I haven't looked at ZB Block in a while, but noticed that it claims to have been tested with RavenNuke, which is great - another option! However, since it checks the same kinds of things as NukeSentinel (QUERY, POST, IP, Hostname, Referer, and User Agent), with the exception of Hostname, it seems that it would be redundant for RavenNuke with a similarly configured list of IPs, referers and user agents (though ZB Block supports blocking of top level domains like .cn - not sure how that works).

Several questions:

Would a missing file / 404 honeypot be useful?
If so, suggestions of for identifying and maintaining the list of files to trigger a block?
What is your experience with ZB Block or other tools / approaches for blocking malicious traffic / bad bots?

On a side note, there is an interesting new (to me) tool to check uploads within a PHP script: phpMussel (it uses ClamAV approach, hence the name). This might have some overlap with Guardian's great Site Guardian tool, but it only works when a file is uploaded through a PHP script (i.e. it can't detected files that were hacked through other means.

Posted: Wed Dec 10, 2014 12:27 pm

Maybe it's as simple as adding ZB Block to a php 404 page...

Fail2ban analyzes log files to block bad visitors, but that's after the fact (or scheduled) so it wouldn't catch the vulnerability scanners in real time.

Site Admin Joined: Aug 22, 2007 Posts: 1772

Sounds interesting. Have also seen these visits in the logs and have always banned IPs. ZBlock I'll give a try.

Posted: Sun Dec 14, 2014 8:28 am

It really is (almost) as simple as adding ZB Block to a php 404 page: it required a tweak to enable testing and identification of the request, along with some custom signatures to address the targeted vulnerabilities.

I'm looking at the Internet Storm Center's Collaborative 404 Error Page Reporting, which has a logging function and provide statistics on the top 404 targets, i.e. the top vulnerability targets. I'm not sure how to simplify the identification of targets, and ISC's report of popular 404 targets isn't working at the moment.

I hope to have a Beta version of the resulting effort soon and feedback can enhance the custom signatures.

Posted: Wed Dec 17, 2014 2:20 am

Someone mentioned naughty people doing naughty things to websites?
Site Guardian can detect any NEW file uploaded to a website (or deleted for that matter) as well as any pre-exisiting file that has been replaced. It also helps by sniffing around for code that is commonly used to conduct attacks but it would be nice to enhance it's features a bit more.

I have been going around and around on this for years but the ICS collaborative 404 page is new to me so I'll have to take a look at it (for reference if nothing else) but it shouldn't be hard to integrate it into RavenCMS as we already have a fairly powerful error handler (@gkuske see 'rcmsroot/errors.php' and if it's possible to seperate out visitors (including bots) looking for specific files to exploit I think we should do it.

Posted: Wed Dec 17, 2014 5:09 am

Hi Guardian,

Some of the guiding principles on this:
- applicable to any website that supports PHP
- don't require access to database to increase performance and works independently of MySQL problems

I have a basic package that needs some finishing touches (installation instructions and HTML compliance checking), but I'm inclined to release a Beta here for active users to evaluate.

The concept could be integrated into Site Guardian (or simply included with it), though it does require ZB Block which would be separate installation. Since I support some custom (non-CMS) PHP sites, I'd love to see a stand-alone version of Site Guardian, but I'd rather see progress on RavenCMS.

Posted: Tue Jan 20, 2015 1:49 pm

I'd like to test it if it is possible. Let me know and I will install it asap.

Posted: Wed Jan 21, 2015 9:54 pm

I've been testing some enhancements (and a fix to a fix Smile

) for nukeSPAM, but hope to return to the 404 solution soon (it's very close, but needs some finishing touches).

Since the 404 project reporting function has been unavailable for some time, it am planning to disable that capability (a shame really, because I think it's very useful). But the use of ZBBlock with some vulnerability signatures added (and room to add more), should be very useful enhancement to Site Guardian, NukeSentinel(tm), nukeSPAM(tm), RYNA domain blocking (with a possible enhancement here, too), and other tools. The best part is that these tools are, for the most part, unobtrusive to non-malicious visitors...

The nukeSPAM enhancements involve some of the techniques used by spammers who take advantage of Gmail's inbox capabilities to circumvent detection. These attempts will be blocked without requiring relatively expensive (and, in some cases, limited) calls to public databases...check nukeseo.com for more details.