How to parse content generated by JS in Python. What do you advise?

Question

Python

I'm new to this business, so I'll ask you not to throw stones at me right away)

Today, wherever you look, JS is everywhere. Which actually significantly complicates parsing, at least in Python for sure.
Two libraries: Requests in conjunction with bs4 is not enough. Or I don’t know something, in which case I would be grateful if you poke your nose where it should be (in the documentation).

Actually the question is: how and with what help is it best to parse certain content from pages that are generated by JS? Thanks

in advance for your replies.

Answer 1

Or I don’t know something, in which case I would be grateful if you poke your nose

you need to poke your nose into Google, at every step they write about selenium
PS looked at the previous question, but you have already been poked into it, it means it's useless.

Answer 2

selenium

Answer 3

Depends on content. Either selenium or, if you need to parse what the page loads via the API, then you can intercept the request to the API and write your own implementation on requests\aiohttp\ to taste.

Answer 4

support. Selenium is not always needed (and for a long time, sometimes). First, study how the server gives content (or attach a link to the service). Sometimes, you just need to dig into how requests leave and how they return. And no Selenium is needed.

Answer 5

there are slightly lighter tools with JS processing, but they are all more complicated, so Selenium

Answer 6

Selenium and more