Answer the question
In order to leave comments, you need to log in
How to bypass empty node in SymfonyDomCrawler?
Hello. There was a task to parse the popular site www.medium.com. If someone was sitting or sitting there, then they probably noticed that under each article there is a number of claps. The task is to get article titles, links to them and the number of claps under them for future article growth analytics. After digging into the markup, I noticed that there are tags for the number of claps when there is a quantity itself. In other words, if no one clapped, then it is empty - there is no markup. This is where my parser (which, by the way, I write using Symfony DomCrawler) stumbles.
Here is the code:
public function saveData($url)
{
$client = new Client();
$result = $client->request('GET', $url);
$html = '' . $result->getBody();
$crawler = new Crawler($html);
$this->configs = require 'configs.php';
if (!empty($this)) {
$this->database = new Database($this->configs['db']);
}
if (!empty($crawler)) {
$crawler->filter('div[class="postArticle postArticle--short js-postArticle js-trackedPost"]')->each(function (Crawler $node, $i) {
$title = $node->filter('h3')->text();
$link = $node->filter('a[data-action="open-post"]')->attr('href');
$star = !empty($rate = $node->filter('button[data-action="show-recommends"]')) ? $rate->text() : 0;
if (!empty($this->database)) {
$this->database->query("INSERT INTO programming (title, link, rate) VALUES (:title, :link, :rate)", [
'title' => $title,
'link' => $link,
'rate' => (int)$star
]);
}
});
}
}
Answer the question
In order to leave comments, you need to log in
$node->filter returns a Crawler, it has a count() method which returns the number of node.
Accordingly, after $node->filter(...), you need to check the number
$recommends = $node->filter('button[data-action="show-recommends"]');
$rate = $recommends->count() ? $recomments->text() : 0;
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question