G
G
grulex2017-06-23 12:55:22
RabbitMQ
grulex, 2017-06-23 12:55:22

How to get rid of duplicate messages in RabbitMq queue?

Good afternoon.
There are two services. It is necessary that one service, when an entity changes, inform another service about it.
Now the exchange takes place using complex bicycles and storing a queue in MySql. The plan is to do this with RabbitMQ, but there are some issues.
The first service changes the data quite often (maybe dozens of times per minute), and the second service receives this data every 2 minutes.
If we put each entity change in the RabbitMQ queue, then when the queue is processed by the second service, the same element will be processed many times.
How can you implement uniqueness at the time the queue is processed?
Reading the entire queue and grouping elements in cycles is expensive and somehow a crutch.
If you read one element at a time, process it, and when reading the next one, check whether it was processed, and if it was, just mark it as read, data may be lost when we processed the first element, and after that an update occurred.
Is there any mechanism in RabbitMQ that solves this problem?
If there are any other suggestions without writing their bikes, it would also be interesting to hear.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
M
Maxim Fedorov, 2017-06-23
@grulex

Rabbit does not allow you to replace one message with another or otherwise edit a message. The only option is to filter messages at the consumer level

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question