Why does rabbitmq have such poor performance?

Question

Python

We have a microservice architecture on the project. We use the RPC pattern (tornado + pika). For each result, a queue is created with an ID in rabbitmq. And with a large number of queues (the more queues, the stronger), performance drops, and the number of workers does not matter. Somewhere at 50RPS, rabbitmq gets stuck, although the workers are executing commands. We tested the same thing with Redis, just put the result into it with the command id, it takes out about 400RPS. Are we preparing rabbitmq incorrectly, or is it such a specificity? Maybe there are some better solutions for implementing rpc in a microservice architecture?

Answer 1

In Google, according to the words "rabbitmq" and "rpc", a link to the official documentation on Direct reply-to is issued , have you tried such a mechanism? Creating/deleting a queue for the sake of getting one response is very expensive. 50/400rpc is nothing at all, even for RabbitMQ (of course, everything depends on allocated resources, disk storage, etc.).

Answer 2

Something tells you that you have a strange architecture...

Answer 3

It slows down due to the fact that a queue is created for each response. rabbitmq advises using direct reply to to improve performance https://pika.readthedocs.io/en/0.12.0/examples/dir...

Answer 4

You can try clustering for RabbitMQ and play around with the settings.
There is also an option to use Apache Kafka - a more powerful and high-performance solution.