1,105,332 Community Members

Proxy Checking Problem

Member Avatar
Resentful
Junior Poster in Training
55 posts since Dec 2010
Reputation Points: 0 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

Hello, everyone.

I've been attempting to make a proper proxy checker to check for HTTP proxies for a while, yet I've been unable to find one that correctly finds working proxies. The bulk of my code is below.

    $ch = curl_init(); //initizlize and set url
    curl_setopt ($ch, CURLOPT_URL, "http://google.com");
    curl_setopt($ch, CURLOPT_HEADER, 1); //show headers
    curl_setopt($ch, CURLOPT_HTTPGET,1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt($ch, CURLOPT_HEADER, FALSE); 
    curl_setopt($ch, CURLOPT_VERBOSE, TRUE);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
    curl_setopt($ch, CURLOPT_TIMEOUT, 5);
    curl_setopt($ch, CURLOPT_FAILONERROR, TRUE);
    curl_setopt($ch, CURLOPT_PROXY, trim($proxy)); 

    $data = curl_exec($ch);

The code above does indeed work, but when testing the list on other reputable proxy checkers, the majority of the list fails(95%+). Are there any changes I could make to enhance the number of correct proxies it finds?

Thanks in advance,

Resentful

Member Avatar
veedeoo
Veteran Poster
1,008 posts since Oct 2011
Reputation Points: 314 [?]
Q&As Helped to Solve: 164 [?]
Skill Endorsements: 17 [?]
 
0
 

Hi,

Updated: check this proxy API. It looks pretty easy to use.

Member Avatar
Resentful
Junior Poster in Training
55 posts since Dec 2010
Reputation Points: 0 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

Thanks for the reply. I looked at that site earlier. There seems to be several good APIs out there, but my site would be in trouble if there was a downtime on another site. I would love to have the checking on my server due to the resources I currently pay for.

Member Avatar
veedeoo
Veteran Poster
1,008 posts since Oct 2011
Reputation Points: 314 [?]
Q&As Helped to Solve: 164 [?]
Skill Endorsements: 17 [?]
 
0
 

ok, you need to provide us with a sample IP:PORT. AT least three of them ..please don't give me more than 5. I get lazy right away..

like this 00.00.00.00:126 where: 126 is port and the zeros before it are part of the IP.

Member Avatar
Resentful
Junior Poster in Training
55 posts since Dec 2010
Reputation Points: 0 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

Here's three random IPs. I'm not sure if they are valid or not.

210.161.156.151:80
202.134.187.141:80
42.62.5.106:31849

Thanks,

Resentful

Member Avatar
veedeoo
Veteran Poster
1,008 posts since Oct 2011
Reputation Points: 314 [?]
Q&As Helped to Solve: 164 [?]
Skill Endorsements: 17 [?]
 
1
 

ok, I tried them and other IP's I found on other sites with similar service, but for some reason cURL would fail most of the time.

For the random IPs above, I've got 2 dead ones and one 403 with redirection. Codes I used is the same as your code above except I added this..

    curl_setopt($ch,CURLOPT_HTTPPROXYTUNNEL, 1);

We can also check if the cURL is reading something by passing the fetched data to a PHP function called stripos or even better pregmatch.

    if (preg_match("/html/i", $data)) {
        echo 'IP is alive';
} else {
    echo 'IP is dead';
}

## I have seen some other guys do it like this

  $countX = stripos($data,'</html>');   
  if($countX > 0 ){

  echo 'It is alive';

  }

  else{

  echo 'Not available';

  }

There is a class written by Stanislav in PHPclass.org that looks pretty promising. Another one is written by Alexey , this may need minor work, but it is looking pretty good because of the Ping functions, instead of sending the cURL right away.

UPDATE: I forgot mention there is a PHP exec that can be very helpful in knowing if the IP is alive or not. If you run this code..

        function GetPing($ip=NULL) {
             if(empty($ip)) {$ip = $_SERVER['REMOTE_ADDR'];}
             if(getenv("OS")=="Windows_NT") {
              $exec = exec("ping -n 3 -l 64 ".$ip);
              return end(explode(" ", $exec ));
             }
             else {
              $exec = exec("ping -c 3 -s 64 -t 64 ".$ip);
              $array = explode("/", end(explode("=", $exec )) );
              return ceil($array[1]) . 'ms';
             }
            }

            echo GetPing('42.62.5.106');

That should give you at least something, now, you can use the integer response of the ping function above to eliminate the dead ones from the live one.

Although the ping function will not really check on the port, it will confirm that the IP is up and running. Run the ping function first for the IP and then run cURL to see if there is a redirect. Otherwise the positive ping result should be sufficient. The above IP returned 176 ms in ping function, but would return 302 with redirect with cURL.

Reference ping.

Member Avatar
Resentful
Junior Poster in Training
55 posts since Dec 2010
Reputation Points: 0 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

I've fiddled around with some of the options above before without much success. I tried it again and it seems to be working quite well.

Thanks for taking the time to help me out with this. I've been working on this project for nearly a year.

Greatly thankful,

Resentful

Question Answered as of 1 Year Ago by veedeoo
You
This question has already been solved: Start a new discussion instead
Post:
Start New Discussion
Tags Related to this Article