How to overcome 100% cpu usage in meteor?

Question

JavaScript

Good afternoon! We are sawing the application on meteor. We have a function to generate xls file from our database.
The generation process is divided into 4 parts:

1. loading from the database of documents;
   
2. processing of documents for unloading;
   
3. file generation;
   
4. giving the file to the user;
   

We encountered problems at several stages. So, in order:

1. for some reason, with a large request (20k records of 1kb each), for some reason the processor is very stressed (and it is meteor that eats it). Have not been able to decide yet - I hung up a temporary crutch through .rawCollection, everything is ok there.
   
2. When processing documents, each iteration takes about 30ms, totaling 20k * 30 = 10 minutes. At this time, a loop runs through all the records and converts them into the desired form. At this point, the download hangs at 100% and control is rarely transferred to the main thread (although fibers are used in meteor, and the codes are executed in a separate fiber. From this I conclude that everything is very strange). At the moment, a crutch hangs in the form of setTimeout for 30ms every 10 records. This increases the generation speed, but at least reduces the constant load on the server and allows requests from other users to be processed.
   
3. File generation is performed by a third-party package and also loads the system, although everything works faster here (we use the latest version of excel4node).
   
4. Everything seems to be ok here.
   If anyone can help me figure out at least one of the points, I will be very grateful.

Answer 1

there is still no answer :(
1. .rawCollection was put on bold requests
2, 3 - generation of xls files was moved to a separate process. Thus, it does not affect the general flow.

Answer 2

Where is the code? What DB (probably mongo?)?
20MB is not so much, but not a little, depending on what else is being done with them in the process.

Answer 3

Tried to separate data acquisition using limit and offset. Faced a similar problem. on the local machine, when inserting about 10k records, it reached a maximum of 3k, and then hung up. whereas on VDS, I easily inserted 100k