A
A
Alexey2016-03-23 14:49:28
PHP
Alexey, 2016-03-23 14:49:28

Checking Socks5 proxy through multi curl does not work well. And HTTP proxies are perfect. Why is this happening?

Hello everyone.
I have a virtual VPS server.
nginx + php fpm
100Mbps
_
I need to always have fresh proxies.
Since I take public proxies, I need to check them often. For these purposes, Multi curl was chosen.
I wrote a class in PHP and check the proxy like this:

$cmh = curl_multi_init();

foreach($proxies as $k=>$v){
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, 'http://'.$domain."/?".mt_rand(0,999));
        curl_setopt($ch, CURLOPT_NOBODY, true);
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
        curl_setopt($ch, CURLOPT_USERAGENT , "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.152 Safari/537.36");
        curl_setopt($ch, CURLOPT_TIMEOUT, $this->timeout);
        curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);
        curl_setopt($ch, CURLOPT_PROXY, $v);
        curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);

        curl_multi_add_handle($cmh, $ch);
        
        $tasks[$v] = $ch;

      }


do {
  $mrc = curl_multi_exec($cmh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);

  while ($active && $mrc == CURLM_OK) {
        if (curl_multi_select($cmh) != -1) {
          do {
            $mrc = curl_multi_exec($cmh, $active);
            $info = curl_multi_info_read($cmh);
            
            if ($info['msg'] == CURLMSG_DONE) {
              
              $ch = $info['handle'];
              $getinfo = curl_getinfo($ch);
              $retcode = $getinfo['http_code'];
              
              $proxy = array_search($ch, $tasks);
              $phptime = microtime(1)-$startphptime;
              $totaltime = $getinfo['total_time'];
              // пока проверяю только код ответа, потом буду проверять сам ответ
              if($retcode==200){
                $this->good[] = array($proxy,$retcode,$phptime,$totaltime);
              }else{
                $this->bad[] = array($proxy,$retcode,$phptime,$totaltime);
                
              }
              
            
            }
            usleep(100);
          } while ($mrc == CURLM_CALL_MULTI_PERFORM);
        }
        usleep(100);
      }


When I put 1-2 proxies for verification, everything works well. I put more than 10 - a lot of either non-working ones, or those from which the answer did not come for a very long time.
Moreover, the very proxies that are supposedly "long" or non-working according to the multicurl - turn out to be working and very fast when checked manually or by one curl request (without a multicurl).

Set timeouts. with a small timeout, the proxies are not collected at all, and the multicurl simply exits.
I
thought I wrote something wrong.
But I decided to check the HTTP\S proxy.
removed
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);

Launched and Oh, miracle.
Everything worked and 200 proxies were checked in 20 seconds (the time of the longest request)

---
Checked Sox5 again - the result is still terrible.
It seems that all the same, sox5 proxies somehow get in line, and there is no multithreading. And if you limit it with a timeout, not a single multicurl thread will be executed at all.
I also checked both on the server and on Denver - the result is
---
Maybe someone knows why this happens?
And tell me what I'm doing wrong.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Dimonchik, 2016-03-23
@dimonchik2013

Moreover, the very proxies that are supposedly "long" or non-working according to the multicurl - turn out to be working and very fast when checked manually or by one curl request (without a multicurl)

is it about SOCKS proxy?

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question