How to organize recording of application events to remote storage?

Question

Java

Task: organize RealTime monitoring of real-time system processes.
Stack:

• java
  
• ejb,
  
• hibernate
  
• postgres
  

Features: long transactions, load ~100rps
Scenario:
Events (~100rps ) occur in the application that change the entity (entity set) in the database (Postgres).
These events (1) must be sent to a remote server (2) , processed there (deserialized, checked and prepared for recording) (3) , and reflect changes in the storage (4) in order to make RealTime monitoring (javascript) in a separate application , middleware).
Requirements:

• Events are transmitted and processed in the order they occur
  
• Event delivery guaranteed
  
• Next, you need to scale horizontally.
  
• Need Schemaless solution
  

I’ll make a reservation right away that they tried to solve it “infrastructurally”, by replicating the database, but the results are not satisfactory, the replica is 5-10 minutes behind, due to the amount of data and not a very good channel.
Actually open questions:

1. Format and protocol : What format to send (binary,json,bson) , what "serializer" to use (avro,thrift,jakson,protobuf) ?
   
2. Transfer : What to use as a transport (rabbitmq, kaffka, thrift rcp, http)?
   
3. Processing : How to "low latency" process events, what to use (akka,desruptor,plain java)?
   
4. Storage : What kind of storage to use?
   

___________________________________________________________________________
So far, the following vision of architecture has been formed:

• Format: json
  
• Tranport: Apache Kaffka
  
• Processing : Akka(java)
  
• Storage : MongoDB
  

Scenario :

1. an event occurs
   
2. Using CDI-events, when a transaction is committed, the event is serialized to JSON
   
3. Further, the event is published in the Kaffka swamp
   
4. The remote server, Akka, takes the message from the topic, performs checks
   
5. Next, the event is stored in Mongo
   

Dear colleagues, I need advice and criticism of this decision, or alternative architecture options.
Has anyone had experience with similar issues.
Thanks in advance!

Answer 1

Well, what exactly is holding you back? Sampling from the database, transferring, processing or updating the "storage"?
Here, there can be no universal solution. In some cases, you can neglect the intermediate storage until the completion of the "long" transaction, in some it is not very good.
Try to store all the intermediate data about the event in the key-value base of the redis type.