Answer the question
In order to leave comments, you need to log in
What language to use for a dynamic site parser?
There is a site with data that is loaded when the page is scrolled or when buttons like "show more" are pressed
Loading via Ajax, but the received data is processed by scripts and displayed according to a certain algorithm.
How can such content be parsed?
Tried through VBS. An excellent solution with imitations of button presses and page scrolling in InternetExplorer.Application.
But this is only on Windows. How can such tasks be solved in JavaScript or Nodejs.
If possible - give examples
Yes! And further! Sites are https!
Answer the question
In order to leave comments, you need to log in
Maybe Selenium will work?
www.seleniumhq.org
habrahabr.ru/post/186496
If I understand the question correctly, then you need to find the url to which the ajax request is sent on the site (F10 > Network) that loads the content, and send the same post ajax to it only with your data (page=1, page=2 or offset =20, offset=40...)
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question