Discussion: Filtering

Site Admin/Owner Joined: Aug 27, 2002 Posts: 17088

technocrat wrote:

<snip> I am not saying it was a waste of time <snip>

Sorry, I didn't mean to come across like that. My comments were more for the public that wasn't involved and not for those that were.

Posted: Sun Oct 01, 2006 11:32 am

Just thought I would drop in and show a function I got to secure against sql attacks, thought it might help and a little feedback would be usefull.

function secure_variable($var){

$insecure_var=$var;
$pattern=array('delete','insert','update','select','join');
$pattern1=array('into','from','where','union', 'table','_');

$final=0;
foreach($pattern as $pattern){
$result=eregi($pattern,$insecure_var);
if ($result==1){
$final=1;
}
}

if($final==1){
foreach($pattern1 as $pattern1){
$result=eregi($pattern1,$insecure_var);
if ($result==1){
$finalf=1;
}
}
}

if($finalf==1){
return false;
}else{
return true;
}
}

For those who are new and dont know, If you type $something=secure_variable($yourvariabletobesecured);

$something will return either true-it is secured of false- its not secure.
then you can make an if statement to kill the process. Works well for letting usernames contain words like delete, or insert
It checks to make sure it contains sql keywords too, then it makes sure it contains other subcharacters or words for sql statements. so basically, insert is ok, delete is ok but delete from is not. or insert into is not, into and from is ok as well, but nothing combined.

Posted: Sun Oct 01, 2006 12:35 pm

I assume you run this against all $_GET, $_POST, $_REQUEST, and $_GLOBAL arrays on every request?

Posted: Sun Oct 01, 2006 1:35 pm

Correct, Any input, the script itself is left alone, anything hard coded wouldnt need to be checked but anything that gets Grabbed would.

I know this doesnt pertain to nuke itself but would considering its a function designed to filter out potential hacks, be best placed here.

Posted: Mon Oct 02, 2006 9:32 am

There are a few problems with your code. First using clike comments would bypass that unless they are stripped out. Second looping through each patter like that is pretty slow.

Take a look at: [ Only registered users can see links on this board! Get registered or login! ]

Posted: Mon Oct 02, 2006 3:52 pm

ok so basically I left some things out, but from your example It looks like it strips the code in the event it finds it, looks great but I wonder what would happen if I use a username like delete or Insert. Im Just trying to be as thorough as possible.

And it loops twice and only twice on each input you set it to. Honestly not even a .01 sec time lose.

I stayed away from looping it within eachother, but I definatley like your approach, Just wish that you could without a doubt use inputs that did not get changed or strip out the contents if it wasnt defined an sql attack. Thanks for the input though.

Posted: Mon Oct 02, 2006 4:11 pm

It would not matter unless they used union first. So uniondelete would get it to fire. Other than that it should not trip.

Yeah the time loss is little but I am all for as much efficacy code as possible. A few 1/2 seconds here, a few there and you have a long load time. So I keep trying to make things as quick as possible. But yeah it would not make that much difference as it is.

Posted: Tue Oct 03, 2006 8:30 pm

ahh I see, I agree about the time load but one problem with your first statement, uniondelete would NOT work.

it would find delete in the string and then it would find union, I just tested it to make sure thats correct. But it found delete in the string then proceeded, even if its not the first word in the string it will find it. then itll search the string again completely for the other words.

Posted: Fri Nov 10, 2006 6:44 pm

This is an interesting thread, one that I couldn't really participate in at the time it was going on because I was still pretty green. I am now working on a module and want to address the filtering issue. I want to use something that will work on a variety of modern nukes. I'm using 7.9 right now (yes, yes, I know...), but I hope to "upgrade" to the new RavenNuke when it comes out. The filter() function is what I see in the 7.9 code, and as I understand it, that was added by the chatserv patches...? Anyway, I was helping someone else who was running 7.8, and I gave him a block of mine to use, but it did weird things because he had no filter() function. I'm not real thrilled with the filter function anyway, so I've studied this thread. I think I am going to try and use this:
[ Only registered users can see links on this board! Get registered or login! ]

It was mentioned a few times in this thread. I think it will suit my needs just fine.

I think everyone agrees that Nuke is lacking the infrastructure for decent filtering, and that it is pretty important. Lots of people have all kinds of ideas...here is where we need someone to just make a decision and go with something. Maybe that won't happen until a fork...?

Anyway, I'll report back on how my experiences with this filter class went.

Posted: Fri Nov 10, 2006 8:47 pm

I personally am working on a funtion that will strip the tags out, html, sql, and numbers all simply designed to strip out anything, now this function will work in reverse, Instead of it being function(html) to strip out all html tags, It will actually work like this function(html) will strip everything but ALLOWED html. this includes sql statements and a check to make sure it fits html standard syntax. Instead of having to write more then one set of filters for every input, this one will do it all. Time is my issue, lack of to be exact lol.

Posted: Fri Nov 10, 2006 11:34 pm

Actually, the filter() function was FB's project for 7.9

Unfortunately it is a catch-all hack that doesn't really enhance the security of the system. It just adds extra overhead and does not properly manage many conditions.

Posted: Fri Nov 10, 2006 11:39 pm

evaders99 wrote:

Actually, the filter() function was FB's project for 7.9

Unfortunately it is a catch-all hack that doesn't really enhance the security of the system. It just adds extra overhead and does not properly manage many conditions.

Ah, thanks for the back-story. Yeah it's a mess.

Posted: Mon Nov 13, 2006 3:40 pm

This has been an informative thread and I'd like to take it a bit further.

First I'll ask a question: do not the majority (I won't say all but the vast majority) of security exposures come from form submissions? Yeah, people can put a onion attack up in the address bar and I suppose there are other attacks possible but the main thing we need to "filter" is form input. And, since we can't trust that the input comes from an actual form that we've developed ... it could come from one simulating one of our forms ... we need to "filter" the input once it arrives as POSTED and before we do anything else with it. I should qualify that to say we also need to filter GET input where for instance they make a choice on a screen and we send that on as a GET.

So now how do we make sure that every form gets filtered. Well: NukeSentinel gives us one approach, after the form is submitted modules.php calls mainfile which in turn calls Sentinel which in turn scrutinizes the post strings for anything illegitimate. Now if the module author also filtered the form input that also gets executed after Sentinel gets done with it (part of what 64bit so aptly called Fractured filtering).

So now we have a situation where, for instance with the wysiwyg editor and kses.php there is extremely sophisticated filtering built into wysiwyg but we are doubling up by hitting it with NS filters too. Am I right? I know it's not exactly doubling up because the filters are different. And of course there's filtering functions still in mainfile which are called at various points.

Now, I've looked at various "classes" that can be used to build forms and their validation and I'm sure we could use some code derived from some of these classes, if not use the classes intact. There's just no reason for the Nuke community to be sitting here writing "original" regular expressions when the code already exists and has been tested extensively.

But this still leaves some questions in my mind about how we'd implement this and make sure that it's consistent across RN or PHP-Portal or 3CMS or whatever we call it. So how about this idea: suppose we develop a MYSQL table, and call it nuke_forms. In that table we'd have a form name and form fields, and their type (text, select, textarea, radio, submit etc.) and the validation criteria we want to apply and the text description of the field and maybe the order that it comes in the form. Suppose then when the form is called it is built "dynamically" by reading the table and generating the html code for the form "automagically".

I suppose we'd have to separate the validation criteria into pre and post processing areas ... and have the pre ones done via Jabascript libraries and the post ones done by PHP classes or functions that we'd maintain in Nuke. The criteria would have to be pretty sophisticated to allow for differing amounts and levels of html ... and we'd build a form for building the form where the criteria would be shown in a select or drop down box with choices like "nohtml" "only formatting html" "everything but scripd tags" or whatever we decided on.

In addition to building the form dynamically from the forms table, then we'd build the POST processing part of it dynamically. So when we went into the section of code that reads:

$var1 = $_POST['formvar1'] we'd determine what filter to "inject" from the table ... as per ...
$var1 = filterx($_POST['formvar'];

and to make sure that all the form elements are processed we'd probably do a foreach $_POST or whatever that syntax is.

So now we've got a form builder and a form validator built into Nuke and we can assure that any form that uses them gets validated appropriately. Next what I'd propose is that we set up the POST processing part of Sentinel to read the form table first. If the form is in that table then Sentinel knows it's been validated and skips POST processing on it. One duplicate filter removed while forms which aren't "registered" still have Sentinel security applied. Next we take the great RN team and begin converting the forms that are already in Nuke over to this approach. The great thing is that this can be done incrementally and we can be sure that add-on modules are still filtered by Sentinel. And the bad thing is that this would be tedious and time consuming work. And another good thing is that it would only need to done once. And another good thing is that we could probably make the forms HTML4.01 and CSS compatible as we go.

But I guess I said enough for this afternoon.

Posted: Mon Nov 13, 2006 4:49 pm

Really you should backup one step. The first thing that really should be done is the removal of registered globals. They cause many of the problems in Nuke.

The next step is the validation and type checking which SHOULD have been done to any input. What I mean is for example the news is fetched by the var $sid, or $_GET['sid']. This SHOULD only be an int and if I remember correctly is only valid as a GET. So you should use something like:

$sid = (isset($_GET['sid']) && ((int)isset($_GET['sid']) == isset($_GET['sid'])) ? $_GET['sid'] : 0;

If you really wanted to do one better you can use a class to fetch and validate the input. I wrote one for v3 of Evo and posted here somewhere at one point I think.

The you can expand on that class to check for HTML if it supposed to be text, and filter out the junk.