B
B
bro-dev2018-03-15 16:05:02
JavaScript
bro-dev, 2018-03-15 16:05:02

Heavy calculations on a node, how to distribute in time?

The server was raised with express, in the same script there are separate calculations that last 20 seconds, and so the problem is that during their time the server completely stops responding. The problem is not exactly understanding asynchrony, any route does not respond.
The task is not difficult to get subscribers from 2 VK groups and subtracts, if the groups are 200000 each, then it already thinks for 20 seconds, although it would seem that the algorithm is a simple 2 nested odds. At the time of obtaining subscribers, it is also a long process, but all the rules work, that is, the network is not loaded.
here is the mb algorithm, it has a problem and can be made more optimal

spoiler
//user_ids массив первой группы 200к идшников и так же banIds вторая группа
var tar = [];
        for (let i = 0; i < grdb.user_ids.length; i++) {
            var exist = false;
            for (banId of banIds) {
                if (banId == grdb.user_ids[i]) {
                    exist = true;
                    break;
                }
            }
            if (!exist) tar.push(grdb.user_ids[i]);
        }

Answer the question

In order to leave comments, you need to log in

2 answer(s)
K
Konstantin Kitmanov, 2018-03-15
@k12th

O(n*n) is just not a simple algorithm...
To the point: move this heavy cycle into a separate process using cluster or stupidly child_process . Or write an addon in C++ or Rust .

V
Vladimir Skibin, 2018-03-15
@megafax

Move this processing to a separate process / script that will listen to tcp / http incoming connections and process the data coming into it (your cycle itself) and already communicate with it from the main application. The bottom line is that your event loop is not interrupted on this cycle and until it finishes working, it will not give up control. Those. it is necessary to artificially cause the interruption of this cycle. You can wrap it in promises + setTimout(..., 0) but this is an obvious crutch.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question