How not to crash the server when parsing 10k+ pages?

Question

JavaScript

Good afternoon. I wrote a parser that threw a request and parsed the contents, but this option eventually became a lump in my throat. captchas appeared, the question arose to lick everything at once, started working, put the server down for 40 seconds, I don’t want problems and I wasn’t chasing this goal, so please tell me what should I do? Is it possible to parse 100 pages per day, but there are 10k+ of them, the customer is not satisfied with weeks of parsing, or parse at intervals of 50 pages per 10 minutes?

Thank you.

Answer 1

Parse about one page per second, so the donor will not fall. 10k in 3 hours sparse.