Answer the question
In order to leave comments, you need to log in
Use phantomjs or loop through a million pages?
I want to save data from a site that contains elements that do not have a direct address in the attributes, for example, pagers, the "More" button (popup). Just cURL and htmlsimpledomparser won't be enough to simulate a click.
At the same time, I know the addresses of the final pages
that look like /index.php?id=1362
there is a valid html-title (unfortunately all pages return 200 code, and there is no other way to distinguish them).
Do you think it's a good idea not to be smart with phantomjs and go through a million pages like a tank?
I will run the script from the desktop from the console.
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question