How to parse via requests/bs4, a site generated via JS?

Question

Python

Good afternoon.
I wanted to parse the site using requests/bs4. But the site is generated in JS, and the results of parsing through requests / bs4, does not give out :( (although Content-Type: text/html).
If I parse through selenium, then everything is OK, I get the necessary information.
Actually, the question is, can sites on js , parse through requests/bs4 and not through selenium?If so, how?

Answer 1

No, you can't, because requests simply gets the content of the page (without executing js, of course), and bs4 parses the html. Selenium can only be replaced by PhantomJS.

Answer 2

https://github.com/puppeteer/puppeteer

Answer 3

Make a direct request to js, ​​often info will be returned in json, which is convenient for parsing