C
C
Caretaker2018-06-06 19:34:36
Node.js
Caretaker, 2018-06-06 19:34:36

Incomprehensible metamorphoses with memory usage by the NodeJS service?

Greetings. Need help from a specialist.
In a nutshell - there is a small service, in some way a simplified "radish bike". Keeps in its memory the values ​​​​of the data transferred to it "by a string key", uses the lifetime for this data (this is all the radish can do), notifies connected clients about changes in the data to which these clients are subscribed (this is partially the radish can) + the option to synchronize everything a data block in case of reconnection and a few other little things, purely service ones (just those that the radish is not able to do).
I noticed that the service has been working stably for some time and without problems with a fairly large stream of incoming data (500 keys are poured per second, in the values ​​​​of which structures are 1000 elements long, each element is a small structure of 5-10 string fields). Memory consumption at this time falls within the range of 90-110 MB ...
But at some point, memory usage suddenly (within just a few minutes) shoots up to 1.5-1.9 GB, and from that moment minor dullness becomes noticeable. I inserted monitoring into the code - it removes and logs memory consumption through "process.memoryUsage()". And this is what I saw:
5b180bb0be217405052196.jpeg
At some point, memory usage for all fields first rises sharply, then drops even more sharply to "normal" values, but the total consumption does not decrease. Accordingly, three questions:

  • what kind of avalanche-like increase in memory is this, when the stream of incoming data does not change, its volume is a constant (I did it especially for the test)
  • why, when memory is freed, its consumption by the process does not decrease (actually, what was it at all)
  • where to dig to eliminate this situation (I’ve been trying for the hundredth time to look in the direction of the radish, but some of the features of this module do not fit into the radish, and you can’t refuse them - you have to redo a lot in other services)

Thanks in advance

Answer the question

In order to leave comments, you need to log in

2 answer(s)
C
Caretaker, 2019-02-21
@zuart

Smoking manuals led to a detailed study of the principles of work of cleaning memory from garbage, etc. nonsense. Further, a number of code optimizations, cuts and a way to clarify the problem - some have been eliminated, only "leaks" in the WS module remained ...

P
profaller, 2018-06-07
@profaller

You need to turn to the battle of psychics. They won't help you here.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question