D
D
Dmitry Gogolenko2022-02-02 13:12:57
Wget
Dmitry Gogolenko, 2022-02-02 13:12:57

How to download a site with lazy lood?

I want to download a site that loads as I scroll. Tried downloading it with wget, Web Copy and WinHTTrack. A semi-empty index.html and one folder are saved. How can I download this site? Not by hand)

Answer the question

In order to leave comments, you need to log in

1 answer(s)
M
Michael, 2022-02-02
@gogolb

The bottom line is that the browser uses JavaScript to render the page.
If you need to download only data, then find the request that the browser sends to the backend and download the data with it (there will be JSON or something similar) you can do exactly that, I’m not sure about firefox) in automatic mode and can simulate any user actions in it. See chromedriver. If I am not confusing anything, then Selenium can do this.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question