K
K
Kotaro Hiba2020-11-19 22:20:32
Node.js
Kotaro Hiba, 2020-11-19 22:20:32

How to calculate what load can be given to iron?

Is it possible to calculate what load can be given on a particular hardware.
What do I mean, for example, I have a scraper that will scrape just a huge number of pages, and if you do everything in one thread, it can work for 20 hours.
Let's say you can speed up this process by creating additional workers.

And now the question arises, how to calculate how many workers a PC can pull out or how complex operations it can process.
1. I do not know the capacity of the PC on which the script will run
2. Ideally, give the user the opportunity to choose how many resources he will allocate for work.
If there is any method of calculation or lit-ra on this occasion, please share.

At the moment, I came up with only 1 method, see how many processor cores the user has and run the same number of workers, though I don’t know how correct this is :)

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question