| Author |
Message |
floppydrivez
Involved
Joined: Feb 26, 2006
Posts: 337
Location: Jackson, Mississippi
|
 Posted:
Wed Jul 25, 2007 8:02 am |
 
|
Maybe not the correct forum, but anyway I am trying to check the content of a page using file_get_contents($url). Sentinel keeps blocking me. Why is this? |
|
|
    
 |
|
Gremmie
Former Moderator in Good Standing
Joined: Apr 06, 2006
Posts: 2415
Location: Iowa, USA
|
| Posted:
Wed Jul 25, 2007 8:58 am |
|
Could you be more specific about what you are doing? Sentinel examines $_GET and $_POST and doesn't have anything to do with file_get_contents() per se. But if you were passing the $url to a script that might do it.... |
|
|
|
|
|
floppydrivez
Involved
Joined: Feb 26, 2006
Posts: 337
Location: Jackson, Mississippi
|
| Posted:
Wed Jul 25, 2007 9:01 am |
|
Simply trying to check if a link exist on another page.
| Code: | function checklink($url, $mydomain){
$url = strtolower($url);
if (strstr(file_get_contents($url), $mydomain)) {
$link = "linked";
} else {
$link = "not linked";
}
return $link;
} |
|
|
|
|
|
|
Gremmie
Former Moderator in Good Standing
Joined: Apr 06, 2006
Posts: 2415
Location: Iowa, USA
|
| Posted:
Wed Jul 25, 2007 9:58 am |
|
How is checklink() being invoked? Are you calling it after passing $url or $mydomain in $_GET or $_POST?
How do you know it is file_get_contents()?
What does Sentinel say on the ban screen for the reason? What are the $_GET & $_POST values on the ban screen? |
|
|
|
|
|
floppydrivez
Involved
Joined: Feb 26, 2006
Posts: 337
Location: Jackson, Mississippi
|
| Posted:
Wed Jul 25, 2007 10:14 am |
|
No, I am not using $_GET or $_POST which leaves file_get_contents 
| Code: | | checklink($usersite, $nukeurl); |
I know its file_get_contents because I stripped out everything from the function echo'd the output. BTW $usersite is just user_website from the database.
This is what I get when I did so nothing special
| Quote: | You have attempted to access this site with an invalid User Agent.
If you think this is a mistake you can contact the site webmaster at .
Be SURE to include the following information in any email!
User Agent: none
Remote Address: 72.249.25.225
Client IP: none
Forwarded For: none |
Just curious if there are some extra steps I should take to allow that site to access the other. |
|
|
|
|
|
evaders99
Former Moderator in Good Standing
Joined: Apr 30, 2004
Posts: 3221
|
| Posted:
Wed Jul 25, 2007 10:21 am |
|
"invalid User Agent" seems to be the key here. You may need to use the cURL functions to do a proper HTTP retreival and spoof the User Agent |
|
|
|
|
|
floppydrivez
Involved
Joined: Feb 26, 2006
Posts: 337
Location: Jackson, Mississippi
|
| Posted:
Wed Jul 25, 2007 10:21 am |
|
SGremmie, I guess my real question is how can bypass that invalid user agent bit. |
|
|
|
|
|
Gremmie
Former Moderator in Good Standing
Joined: Apr 06, 2006
Posts: 2415
Location: Iowa, USA
|
| Posted:
Wed Jul 25, 2007 10:26 am |
|
Oh I get it now....lol....sorry...
As evaders suggests, you might have to form up a more proper HTTP request.
Or if the site you are querying is under your control, you can modify Sentinel to remove that invalid user agent check. There was a recent thread about how to do this. I hope Bob adds a flag for that like the invalid IP check flag in a future version. |
|
|
|
|
|
floppydrivez
Involved
Joined: Feb 26, 2006
Posts: 337
Location: Jackson, Mississippi
|
| Posted:
Wed Jul 25, 2007 11:22 am |
|
Anyone a curl pro? I see lots of options to use in my function but I honestly have no clue what they mean. I am simply guessing and guessing.
| Code: | function checklink($url, $mydomain){
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_HEADER, true);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_FRESH_CONNECT, true);
$contents = curl_exec ($ch);
curl_close ($ch);
$arrText = array();
$arrText = explode("\n", $contents);
if (strpos ($contents, $mydomain) != FALSE) {
return true;
} else {
return false;
}
} |
|
|
|
|
|
|
floppydrivez
Involved
Joined: Feb 26, 2006
Posts: 337
Location: Jackson, Mississippi
|
| Posted:
Wed Jul 25, 2007 11:27 am |
|
I can see now I got some learning to do. |
|
|
|
|
|
floppydrivez
Involved
Joined: Feb 26, 2006
Posts: 337
Location: Jackson, Mississippi
|
| Posted:
Wed Jul 25, 2007 11:50 am |
|
Chalk one up to the manual for some help.
This needs some refining. I still don't understand some of it. Anyway works like a champ.
| Code: | function checklink($url, $mydomain){
$agent = "CTLinks";
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_HEADER, 0);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_FRESH_CONNECT, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
$contents = curl_exec ($ch);
curl_close ($ch);
$arrText = array();
$arrText = explode("\n", $contents);
if (strpos ($contents, $mydomain) != FALSE) {
return true;
} else {
return false;
}
} |
Don't forget to set the agent string to something relevant. Like I said, it works but I haven't put it through a tough beta yet. |
|
|
|
|
|
montego
Site Admin
Joined: Aug 29, 2004
Posts: 9133
Location: Arizona
|
| Posted:
Wed Jul 25, 2007 7:08 pm |
|
| floppydrivez wrote: | | I can see now I got some learning to do. |
Me too. 
Thanks Evaders! |
|
|
|
|
|
floppydrivez
Involved
Joined: Feb 26, 2006
Posts: 337
Location: Jackson, Mississippi
|
| Posted:
Wed Jul 25, 2007 7:13 pm |
|
This has actually turned out to be very useful. Its for sure worth the read of what is capable with cURL. The limit here is a server must have it installed.
|
|
|
|
|
|
evaders99
Former Moderator in Good Standing
Joined: Apr 30, 2004
Posts: 3221
|
| Posted:
Wed Jul 25, 2007 8:16 pm |
|
cURL is pretty fun stuff. Sadly it allows the hackers to do the same as you, HTTP requests that look pretty legitimate... fortunately I've not seen massive abuse of it yet. |
|
|
|
|
|
|
|
|
|
![[Valid RSS]](http://www.ravenphpscripts.com/images/valid-rss.png "Validate my RSS feed"){kind=small}
:: 
::
