Answer the question
In order to leave comments, you need to log in
PHP How to make parsing with pause?
Good afternoon.
Today we thought about parsing car categories from one of the sites, for which we wrote a simple parser:
<?php
function cURL_AutonavigatorRu($level = false, $model_id = false){
#http://www.autonavigator.ru
$ch = curl_init('http://www.autonavigator.ru/dispatcher.pl');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.104 Safari/537.36");
$headers = array
(
'Accept: application/json',
'Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4',
'Accept-Encoding: gzip,deflate',
'Accept-Charset: windows-1251,utf-8;q=0.7,*;q=0.7'
);
curl_setopt($ch, CURLOPT_HTTPHEADER,$headers);
curl_setopt($ch, CURLOPT_REFERER, "http://www.autonavigator.ru/my/offer_add/");
if($level == '1'){
curl_setopt($ch, CURLOPT_POSTFIELDS, 'class=list&method=make&show_all=1&vehicle=car&type=used');
}
elseif($level == '2' && $model_id){
curl_setopt($ch, CURLOPT_POSTFIELDS, 'class=list&method=model&show_all=1&vehicle=car&type=used&make_id='.$model_id);
}
elseif($level == '3' && $model_id){
curl_setopt($ch, CURLOPT_POSTFIELDS, 'class=list&method=modif&show_all=1&model_id='.$model_id);
}
else{
curl_setopt($ch, CURLOPT_POSTFIELDS, 'class=list&method=modif&show_all=1&model_id='.$model_id);
}
curl_setopt($ch, CURLOPT_ENCODING , "gzip");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
curl_close($ch);
$json = json_decode(iconv("windows-1251","utf-8",$result), true);
return $json['list'];
}
$ArrAuto = cURL_AutonavigatorRu('1');
foreach($ArrAuto as $auto) {
echo $auto['value'].'<br>';
//sleep(2);
$AllModif1 = cURL_AutonavigatorRu('2',$auto["id"]);
var_dump($AllModif1);
echo '<br><br>----------------------------------<br><br>';
}
?>
Answer the question
In order to leave comments, you need to log in
The question and the description of the question do not match.
For the first sleep, for the second set_time_limit
set_time_limit( int $seconds );
more details at:
php.net/manual/ru/function.set-time-limit.php
This function does not work if PHP is running in safe mode.
The function must be specified 1 time at the very beginning of the script.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question