How to parse dynamic content where different request data (AJAX POST)?

Question

Python

While developing the parser, I ran into the problem of parsing dynamic pages, or rather, following pagination links that look like in the screenshot:


And the trace is passed as a request. data:

After clicking on the page number, the content on the page is replaced, the address in the browser does not change, while the pager selectors have not quite logical names, so it is impossible to determine their range.

UPD: if it matters. The answer is a piece of ready-made HTML, which completely replaces the data table I need along with pagers.

To be honest, I don’t quite understand how to bypass such pagination. I tried to google, looked at articles, lessons, but everywhere they talk about parsing simpler AJAX requests, where it is much easier to compose requests for parsing.

Answer 1

Have you tried to track what requests are being generated (at least in the browser's developer console - the Network tab) and what responses are coming to them?

Answer 2

Try parsing dynamic sites using phantomjs.
stackoverflow.com/questions/13287490/is-there-a-wa...