How to think over the logic of work?

Question

Multithreading

There is a goal that I have been struggling with for a long time, I need to understand how to implement such a system and whether it is even possible in terms of resource costs.

There is a site on which there are hashes.
The goal is to send the hash to the script for verification so that it collects all its incoming hashes (let's say there are 50 of them), we save them in the database.
After that, you need to take each received hash in the same way and collect its incoming hashes and add it to the database in the same way.
Thus, we went through all 50 hashes and got (let's say each also has 50) and we get 2500.

And thus, for a long time, we will not come to a certain condition that will stop this process.
It seems that everything sounds easy, but I have been struggling with the implementation for a long time.

What exactly are the problems?
1. Proxy (one proxy is enough to check about 1400 hashes), there can be more than 700,000 of them.
2. Speed, what is most important, I tried to make this system on asynchronous libraries (probably this is bad), and the check takes about three days, which is very bad, because it needs to be reduced to at least half an hour, it is because of this that I came here , because I don’t know how to compose it so that it instantly takes all the hashes from the database and starts checking right away, in streams? But how to make it right.

I'm not asking for ready-made solutions to such a problem, I'm only asking for help and advice on how to correctly create the structure of such a difficult check, depending on speed. What to use and how best to compose.

Answer 1

1) paid normal proxies
2) another data source