K
K
kafkiansky2018-08-14 22:44:41
symfony
kafkiansky, 2018-08-14 22:44:41

How to bypass empty node in SymfonyDomCrawler?

Hello. There was a task to parse the popular site www.medium.com. If someone was sitting or sitting there, then they probably noticed that under each article there is a number of claps. The task is to get article titles, links to them and the number of claps under them for future article growth analytics. After digging into the markup, I noticed that there are tags for the number of claps when there is a quantity itself. In other words, if no one clapped, then it is empty - there is no markup. This is where my parser (which, by the way, I write using Symfony DomCrawler) stumbles.
Here is the code:

public function saveData($url)
    {
        $client = new Client();
        $result = $client->request('GET', $url);
        $html = '' . $result->getBody();
        $crawler = new Crawler($html);
        $this->configs = require 'configs.php';
        if (!empty($this)) {
            $this->database = new Database($this->configs['db']);
        }
        if (!empty($crawler)) {
            $crawler->filter('div[class="postArticle postArticle--short js-postArticle js-trackedPost"]')->each(function (Crawler $node, $i) {
                $title = $node->filter('h3')->text();
                $link = $node->filter('a[data-action="open-post"]')->attr('href');
                $star = !empty($rate = $node->filter('button[data-action="show-recommends"]')) ? $rate->text() : 0;
                if (!empty($this->database)) {
                    $this->database->query("INSERT INTO programming (title, link, rate) VALUES (:title, :link, :rate)", [
                       'title' => $title,
                       'link' => $link,
                       'rate' => (int)$star
                    ]);
                }
            });
        }
    }

I can't figure out how to process a node if there is no markup. Now the error is displayed The current node list is empty. I want to set the rating of this article to zero if there is no markup, which is logical. But I can't figure out how to do it. The current logic doesn't work.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
V
Vladislav Vlastovskiy, 2018-08-14
@neliudov

$node->filter returns a Crawler, it has a count() method which returns the number of node.
Accordingly, after $node->filter(...), you need to check the number

$recommends = $node->filter('button[data-action="show-recommends"]');
$rate = $recommends->count() ? $recomments->text() : 0;

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question