R
R
rbvadik2014-07-29 00:47:52
PHP
rbvadik, 2014-07-29 00:47:52

How to build a recursive function?

Task:
On the first page, catch all links like "//a[@class='foo']". Next, you need to go through each of them and parse the references "//a[@class='foo']" again, and so on, as long as "//a[@class='foo']" exists.

$dom = new DOMDocument();
$dom->loadHTMLFile($url);
$xpath = new DOMXpath($dom);
$pattern = "//a[@class='foo']";

if ($xpath->query($pattern)->length == 0) {
}
else{
  foreach($xpath->query($pattern) as $cat) {
      echo  $cat->nodeValue . " " . $cat->getAttribute("href") . "\n";
    }
}


I understand that I need to build a recursive function, but I can't do it.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
Sergey, 2014-07-29
@rbvadik

Enter a link queue. Once the page has been rendered, add all of the following links to the queue. In a loop, read the links, download and parse them, and so on until the queue runs out. No recursion is needed here.
As well as an additional profit, you can select several links from the queue and use multi curl to download documents in several streams.

Y
Yakov Akulov, 2014-07-29
@jakulov

To understand recursion, you need to understand recursion

function parse_recursive($url) {
 $dom = new DOMDocument();
 $dom->loadHTMLFile($url); 
 $xpath = new DOMXpath($dom); 
 $pattern = "//a[@class='foo']";
 if ($xpath->query($pattern)->length == 0) {
   return;
 }
 else{
     foreach($xpath->query($pattern) as $cat) {
   parse_recursive($cat->getAttribute("href"));
     }
 }
}

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question