How to scrape dynamic websites?

Question

Scrapy

Good day, there is a need to download the first 2000 photos in one request from Google images, the scrapy framework is used
. I have no development experience in this framework, so I ask for help:
How to scrape dynamic sites?
After all, only the first few dozen photos are loaded on Google images, the rest are loaded as the page scrolls
. How can I get around this?

Answer 1

1) You need to study what happens on the page when you scroll it. It is very possible that simple requests are sent that can be reproduced using scrapy. Just examine the contents of the console.
2) There are tools that mimic the work of a regular browser (including js execution, page scrolling, etc.). For example:
-Puppeteer
-Splash
-Selenium (it can be controlled from python)
3) Creative process))))