V
V
Viktor Vsk2014-07-24 00:22:54
ruby
Viktor Vsk, 2014-07-24 00:22:54

How to properly use (manage) memory in a Delayed job?

It seems to be a fundamental question. Delayed job - despite its disadvantages and advantages, sidekiq and resque seem to be the most popular and frequently used, but how to deal with a trivial problem, it's impossible to figure it out.
One of the features of a delayed job in comparison with analogues is that it works in a permanent process and does not use forks. At the same time, a feature of Ruby is that its garbage collector works spontaneously and does not return memory to the system, but simply marks it as free. Therefore, advice usually says that it is better to restart processes.
As a result, such a problem with the DJ: it does not matter, performing simple or complex tasks, the memory slowly takes up everything that is possible. The errors are not in the code, because real tasks, just creating hashes, for example, and the subsequent assignment of nil and GC.start lead to one result - the memory occupied by the delayed_job process grows.
If you restart the process during the execution of the task, then the task will terminate and will never be completed (the next ones will work without problems)
Of course, you can try to restart the process after each operation, as well as restart the failed ones ... But somehow this is very crutch, plus now it seems that this scenario (gradual increase in memory) will befall any task, how then is it correct to use a DJ?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
B
bmalets, 2015-04-09
@bmalets

How about the god utility ?
There you can set the allowed amount of memory used.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question