O
O
oowl2015-04-01 13:00:10
Message Queues
oowl, 2015-04-01 13:00:10

How to configure RabbitMQ to uniformly execute different types of tasks?

There are users in the system, their number is unlimited and unknown. Each user can create a queue of tasks. There are a limited number of workers that must perform user tasks evenly.
For simplicity, let's take one worker and one system user who created 3 tasks.

user_id=1, task_id=1
user_id=1, task_id=2
user_id=1, task_id=3

One worker tasks will be executed sequentially. And it's all right.
Now suppose that the second user created his three tasks a millisecond later. In total, the worker will parse them in that order.
user_id=1, task_id=1
user_id=1, task_id=2
user_id=1, task_id=3
user_id=2, task_id=4
user_id=2, task_id=5
user_id=2, task_id=6

It is necessary to achieve behavior when, in the presence of many users, their tasks are performed evenly.
user_id=1, task_id=1
user_id=2, task_id=4
user_id=1, task_id=2
user_id=2, task_id=5
user_id=1, task_id=3
user_id=2, task_id=6

I tried to do this using the topic exchange, specifying the route key as task.USERID, then subscribed to task.*, but this resulted in the same task being delivered to all workers.
Perhaps I do not understand correctly how it works, or is a similar class of tasks solved differently?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
I
Igor, 2015-04-15
@KorroLion

It seems to me that Rabbit assumes sequential execution of tasks without options. This solution is for highload in the first place.
If distribution is important, then the workers work for a long time, which means the queue is not multi-million. What prevents to make this queue in Mysql, for example?

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question