Q
Q
quest20172017-05-31 11:01:04
Algorithms
quest2017, 2017-05-31 11:01:04

How to clear the queue?

Suppose we have several servers with certain queues in which records about events are accumulated.
For simplicity, let's assume that these are tables in databases to which records are added.
We can find out about the fact that a new record has arrived, if we wish, we can find out how many records are not processed on each server.
The next question is: how would you set the workers on these records?
1) created a pool of workers on each server that would periodically or on a signal climb for records?
2) would you create a pool of workers on third-party machines that would learn about new records through some kind of queue like rabbitmq?
3) how would you decide how many workers you need to set on new records so that on the one hand there are not enough of them, on the other, so that hundreds of workers do not rush after one record?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
L
lxfr, 2017-05-31
@lxfr

It would be better if you indicated how much data is involved.
A table with a thousand rows is one thing, and completely different methods in bigdata.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question