A
A
Alikhan2021-10-19 21:31:18
Python
Alikhan, 2021-10-19 21:31:18

How to interpret JavaScript when requesting requests?

Faced one problem. When parsing a site, you need to get some elements that are loaded onto the page via JavaScript. But requests in python doesn't interpret JavaScript, just the whole page. How can I make requests also interpret JavaScript?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
V
Vindicar, 2021-10-19
Mulaev @AlikhanPython

No way. requests is not intended for this. You need a full browser, and code to manage it.
Those. selenium package.
EDIT: AlexBoss rightly pointed out below - if the site is not trying too hard to weed out bots, then you can try to identify the requests through which the loading of the interesting parts of the page is performed, and execute them yourself. This will make it possible to get by with requests alone, but this is not always easy to do.

A
AlexBoss, 2021-10-19
@AlexBoss

Do not listen to the expert above. Through selenium it will be easier, but you can see what kind of request is coming and throw it separately, you just need to find the dependency in the transmitted data and generate the request.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question