How to load the entire page of a website in python3?

Question

Python

You need to download the entire page of the photo album from VK. Now I do this: response = urllib.request.urlopen('url').read()
But in this case the whole page is not loaded, only the first 30-60 photos, because the rest of the photos are loaded while scrolling the page. I was told that it was loading some kind of javascript script. Googled how to run it with python, found only some ways with selenium. Can this be done in an easier way? Can't use photo. GetAlbumById from at VK because I enter it as a community

Answer 1

Open the developer panel, look at which address and what request is sent at the time of loading new images, emulate the request. tutorial

Answer 2

Open the debugger in the browser, the network tab - xhr, scroll the page and see what requests go to the server at the same time: https://www.screencast.com/t/rZfK9NMRcz
Deal with these requests and just play them yourself in python or whatever generally. Selenium is not needed here, a cannon on sparrows.

Answer 3

I didn’t do it, but ..
https://vk.com/dev/photos
It seems like the urls will return photos, pull them out from these urls ...
Well, or pick xhr