Answer the question
In order to leave comments, you need to log in
Am I parsing data optimally?
The bottom line is this, a request comes to the VK bot, the bot contacts the desired page via CURL, and through PHPQuery pulls out the text (it is small, 2-3 lines), and throws this text to the VK person. But for some reason this text is thrown for a very long time ..
Here I refer to the desired site in the function
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36');
$res = curl_exec($ch);
return $res;
function parse($val) {
global $number;
$document = phpQuery::newDocument(curlit($val, $number)); // тут передаю страницу на которую заходить
$findelement = $document->find('.ml p'); // ищу в классе .ml параграф (это единственный текст в том классе)
$text; // переменная для текста
foreach($findelement as $element) {
$pq = pq($element);
$text = $pq->text(); // сюда записываю ту строчку текста
}
return $text;
}
Answer the question
In order to leave comments, you need to log in
The delay is not due to the fact that you are parsing the data the wrong way, but because of the cURL request. I'm sure it's been running for so long. And it does not depend on you. Actually, what prevents you from taking and measuring the execution time of a particular piece of code?
So determine which step takes longer - the curl step or the phpQuery step.
Other than that I would replace phpQuery with pure xpath.
Plus, just in case, I would add
But this is not for speed.
better get html block of text through regular expression in $parseText
and frame it in html tags
and push everything in phpQuery
, it will be easier for the parser to build a tree and find the selector you need.
or try nokogiri differs in that it does not build a tree, but recursively goes through all nodes similarly works like XmlReader
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question