Proxy Checking Problem

[Resentful]

Hello, everyone.

I've been attempting to make a proper proxy checker to check for HTTP proxies for a while, yet I've been unable to find one that correctly finds working proxies. The bulk of my code is below.

    $ch = curl_init(); //initizlize and set url
    curl_setopt ($ch, CURLOPT_URL, "http://google.com");
    curl_setopt($ch, CURLOPT_HEADER, 1); //show headers
    curl_setopt($ch, CURLOPT_HTTPGET,1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt($ch, CURLOPT_HEADER, FALSE); 
    curl_setopt($ch, CURLOPT_VERBOSE, TRUE);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
    curl_setopt($ch, CURLOPT_TIMEOUT, 5);
    curl_setopt($ch, CURLOPT_FAILONERROR, TRUE);
    curl_setopt($ch, CURLOPT_PROXY, trim($proxy)); 

    $data = curl_exec($ch);

The code above does indeed work, but when testing the list on other reputable proxy checkers, the majority of the list fails(95%+). Are there any changes I could make to enhance the number of correct proxies it finds?

Thanks in advance,

Resentful

[veedeoo]

Hi,

Updated: check this proxy API. It looks pretty easy to use.

[Resentful]

Thanks for the reply. I looked at that site earlier. There seems to be several good APIs out there, but my site would be in trouble if there was a downtime on another site. I would love to have the checking on my server due to the resources I currently pay for.

[veedeoo]

ok, you need to provide us with a sample IP:PORT. AT least three of them ..please don't give me more than 5. I get lazy right away..

like this 00.00.00.00:126 where: 126 is port and the zeros before it are part of the IP.

[Resentful]

Here's three random IPs. I'm not sure if they are valid or not.

210.161.156.151:80
202.134.187.141:80
42.62.5.106:31849

Thanks,

Resentful

[veedeoo]

ok, I tried them and other IP's I found on other sites with similar service, but for some reason cURL would fail most of the time.

For the random IPs above, I've got 2 dead ones and one 403 with redirection. Codes I used is the same as your code above except I added this..

    curl_setopt($ch,CURLOPT_HTTPPROXYTUNNEL, 1);

We can also check if the cURL is reading something by passing the fetched data to a PHP function called stripos or even better pregmatch.

    if (preg_match("/html/i", $data)) {
        echo 'IP is alive';
} else {
    echo 'IP is dead';
}

## I have seen some other guys do it like this

  $countX = stripos($data,'</html>');   
  if($countX > 0 ){

  echo 'It is alive';

  }

  else{

  echo 'Not available';

  }

There is a class written by Stanislav in PHPclass.org that looks pretty promising. Another one is written by Alexey , this may need minor work, but it is looking pretty good because of the Ping functions, instead of sending the cURL right away.

UPDATE: I forgot mention there is a PHP exec that can be very helpful in knowing if the IP is alive or not. If you run this code..

        function GetPing($ip=NULL) {
             if(empty($ip)) {$ip = $_SERVER['REMOTE_ADDR'];}
             if(getenv("OS")=="Windows_NT") {
              $exec = exec("ping -n 3 -l 64 ".$ip);
              return end(explode(" ", $exec ));
             }
             else {
              $exec = exec("ping -c 3 -s 64 -t 64 ".$ip);
              $array = explode("/", end(explode("=", $exec )) );
              return ceil($array[1]) . 'ms';
             }
            }

            echo GetPing('42.62.5.106');

That should give you at least something, now, you can use the integer response of the ping function above to eliminate the dead ones from the live one.

Although the ping function will not really check on the port, it will confirm that the IP is up and running. Run the ping function first for the IP and then run cURL to see if there is a redirect. Otherwise the positive ping result should be sufficient. The above IP returned 176 ms in ping function, but would return 302 with redirect with cURL.

Reference ping.

[Resentful]

I've fiddled around with some of the options above before without much success. I tried it again and it seems to be working quite well.

Thanks for taking the time to help me out with this. I've been working on this project for nearly a year.

Greatly thankful,

Resentful