Parsing HTML generated by JS

Question

Perl

If there are confusing tricky pages, the data on which appears after the execution of the JS code, then how are they usually parsed?

Answer 1

They look where JS (ajax) went,
what it added to the page
and what it changed, and get the result.

                With uv. Your K.O.

Answer 2

You can use some kind of Selenium and do everything in the browser (slowly, but very reliably).
You can, as mentioned above, mimic the behavior of js in your parser (quickly, but sometimes quite difficult).
See what the task is.

Answer 3

You have a pearl in the tags. As I would do on sharpe. You can use the ready-made webkit engine, ie. There the browser will do all the hard work for us, I think this is the easiest way. For pearls, there may also be appropriate wrappers. You can use the JS interpreter for sharp, there are many of them, but you need to figure out how to integrate it. In general it is necessary to look at a specific task.

Answer 4

They look at the JS code, where it sends data, and how it receives it, and parse where it sent/received.