How to parse online stores to monitor prices?

Question

Parsing

You need to regularly parse online stores to monitor prices. There are about 10 sites, and there are 500,000 products. If you parse 1 page every 1 second, then a maximum of 84,400 products can be crawled per day. It will take at least 6 days to crawl all the products. Would it be appropriate to increase the number requests and will it not be an extra load on the site? Sites do not provide xml catalogs and api. How do price monitoring services deal with this? Maybe someone has come across this or knows where you can read about it?

Answer 1

https://habr.com/ru/post/428814/
Suitable?