How do payment systems solve data safety issues when using queues?

Question

Message Queues

Almost all payment systems are highly loaded services. Almost always an attribute of such a project are message queues for asynchronous work with data. Queues typically work with data in RAM. But the server hosting the message broker may crash unexpectedly. Data safety in this case can be ensured by their unloading to the file system. But on the other hand, active work with the file system will significantly reduce performance.

How do the developers of such projects do it? Do they take risks and keep data only in RAM, compromise and periodically upload data to the file system, or do everything in the most reliable way, despite the overhead?

Answer 1

Queues usually work with data in RAM

Here is the whole problem of having this issue. No and no again. Almost no one works directly from memory, and in general there are several approaches to implementation. Of course, HA uses persistence approaches similar to Kafka. As for payment systems, everything works differently there, absolutely. Well, that is, queues are still present, but transactionality is not achieved at their expense

Answer 2

The original question covered too many different topics and is a bit more complicated than just choosing between storing data in memory and on disk. Still, as far as I understand the question in the architecture of queues in a highly loaded system?
If so, then first, you need to use a queue consisting of a cluster of nodes.
Secondly, you need to decide what is more important for your system: cluster consistency or availability.
For example, with RabbitMQ, you can choose which way to optimize.