Why does logstash lose logs?

Question

Python

I need your help!
Now there is such a structure:
1) Redis, acting as a message broker
2) Logstash, as a log deliverer to a file
3) Sanic with 12 workers as a router service. Takes the request coming into it, sends it to the Redis queue, sends it to Sanic, receives the response, sends it to the Redis queue. 4 )
Sanic with 12 workers as a request handler from a

router
on router. Then in the file generated by logstash, in theory, there should be 2000 entries, where pairs with the same id. BUT! in fact, some of the records are missing, some are dubbed. The confusion is complete.

What could be the problem?
Please tell me how to create an architecture that can handle multiple requests at the same time?

Answer 1

Clickhouse
and with Elastic - give him more memory
much more