resque task queue. One big one or several small ones?

Question

Ruby on Rails

The situation is something like this.
There is a rails application, Resque is used for queues.
Every hour it is necessary to carry out some kind of synchronization, let's say 1000 records. For each, data is updated, recalculations are carried out, etc. This happens through a queue.
If you do it through one task, then it takes a long time and if the synchronization for one of the elements falls, then restarting the task will not be very correct. since it will start updating all records first. If you start synchronization for each element through a separate task, then restarting becomes more convenient, but the load on the processor increases by 3 times.
Maybe someone came across a similar one or can give theoretical advice in which direction it is better to dig. Perhaps it can be optimized through the settings. I will still dig myself when I have time, but if someone can give advice, I will be grateful.

Answer 1

The load increases because tasks begin to be processed in parallel, each in a new process.
The ideal option is to move to Sidekiq - it is also much faster in itself and if necessary, you can adjust the number of tasks that are performed at the same time.
If this is not possible, then you can run one big task, catch a synchronization drop through begin rescue so as not to disrupt the overall process. For all caught, run a separate task. If there are not many of them, then the load will increase only for the time of their processing.