A
A
Alexey2020-01-23 21:19:12
Message Queues
Alexey, 2020-01-23 21:19:12

How to process tasks in groups?

Hello! I just started learning RabbitMq and I don't understand a lot, so please don't kick me.
Let's say I have 100 thousand users who need to send push notifications, for example, when some event was added, well, the post was added in the end (just for example).
And now, a post flies into the system, which means I need to add a task to the queue that will send 100 thousand push messages to users. But push messages can only be sent 1,000 at a time. So I need to do the same task 100 times. And here I have a problem:
If I send a message '{post_id: 123}': to the 'send_push' queue, the consumer will receive it, select 1 thousand users from the database, generate a push notification, send it to Google Cloud Messages, save the last processed user somewhere on the disk (so that with it to start sending the next batch of push) and that's all... This will clear the queue.
How can I make the queue run until it sends a push to all users?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
I
Ilya, 2020-01-24
@sarapinit

For example, queue not just one message, but a message per job. That is, the message will contain a post_id and a list of user_ids, which will contain 1000 IDs.
It will turn out 100 messages, you will rake them one by one.

R
Roman Mirilaczvili, 2020-01-24
@2ord

If I send a message '{post_id: 123}': to the 'send_push' queue, the consumer will receive it, select 1 thousand users from the database
And what prevents to take from a DB on 1000 in a cycle ?

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question