What is the best way to run a large number of operations frequently in NodeJS?

Question

linux

There is a NodeJS project in which, according to the schedule (usual setInterval), about 200 operations of the same type are launched every minute. Operations are launched at intervals of 0.1 seconds, but they take 0.5 - 5 seconds to complete. All operations must be completed in the shortest possible time, so wait until the 1st one is completed and start the 2nd one, etc. not an option at all. Also, I can't really increase the operation start interval (even 0.1 sec is too big for me).
An example scenario of operations: get some data via the API > upload an image from your server > process it > save > send the image to another service via their API.
At the moment, the server is coping, but at the time of startup, the CPU and memory load indicators tend to the maximum value for the server.
Characteristics of "server":
Actually a question. Is such an approach with the launch of operations acceptable in a node, or can it be done more intelligently and optimally?

Answer 1

And what kind of operations, i/o?
to run in parallel, for example, you can use the async package
async.parallel([
function(callback) {
setTimeout(function() {
callback(null, 'one');
}, 200);
},
function(callback) {
setTimeout(function( ) {
callback(null, 'two');
}, 100);
}
],
// optional callback
function(err, results) {
// the results array will equal ['one','two'] even though
// the second function had a shorter timeout.
});

Answer 2

run the entire batch in a loop without a pause, using asynchronous functions to the maximum (do not wait for the previous one to finish before starting the next one), then there will probably be an acceleration from the fact that asynchronous operations, such as receiving via API or writing to disk, will free the main thread actually for node actions, getting from

Answer 3

And what for to start them through an interval any?
There are 200 similar tasks.
A promise is made for each task, and as a result, an array of 200 promises is obtained.
Sometimes these promises are done with Promise.all() and that's it. If necessary, the result of the execution of all promises is processed.
If you want to control the number of simultaneous requests over the network, then you can use
http.globalAgent
https://nodejs.org/api/http.html#http_http_globalagent
https://nodejs.org/api/http.html#http_agent_maxsockets