U
U
user0name02018-08-27 16:39:05
PHP
user0name0, 2018-08-27 16:39:05

Why doesn't curl inside foreach work?

I am writing a parser using curl. The script gets a group of links from the desired page (using curl and regular expressions) and writes them to the $match array, then the following code:

foreach ($match[0] as $url) {
//echo $url: - проверка. Выводит url.
$curl = curl_init();
curl_setopt($curl, CURLOPT_PROXY, "127.0.0.1:9050"); // Использую Tor-прокси.
curl_setopt($curl, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); 
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($curl, CURLOPT_HEADER, 0);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($curl, CURLOPT_URL, 'https://'.$url);  // Если писать вручную, без foreach, то работает.
$urlcont = curl_exec($curl); 
var_dump($urlcont); }
All links are working, but var_dump outputs string(0) "". And displays not false - there are no errors.
Interestingly, if you take any link from the array and insert it outside the foreach into CURLOPT_URL, then everything works fine. The problem arises when iterating over an array.
I've tried setting delays, using multiCURL, sending useragent, using cookies - zero.
URL like site.com/index.php?page=post&id=43 - I thought it was a post-request and even sent it via CURLOPT_POSTFIELDS, but not again.

Answer the question

In order to leave comments, you need to log in

3 answer(s)
U
user0name0, 2018-08-27
@user0name0

Replaced:
curl_setopt($curl, CURLOPT_URL, 'https://'.$url);
To:
curl_setopt($curl, CURLOPT_URL, htmlspecialchars_decode ('https://'.$url));
I don’t know why and how the links are encoded, but the main thing is that now everything works.

D
Dmitry Dart, 2018-08-27
@gobananas

There is not enough data, you need to see the structure of the array at least. In general, I say in advance - everything works, look for an error. PS Make it inside for for the sake of interest, it works there, I checked it )) $match[0]

N
Nick Sdk, 2018-08-27
@lidacriss

curl_setopt($curl, CURLOPT_URL, 'https://'.$url;

it should probably be at least
curl_setopt($curl, CURLOPT_URL, 'https://' . $url);

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question