How to build the application architecture correctly?

Question

go

Implemented a client-server application, Client - intercepts a network packet from the queue changes source_ip . dst_ip , inserts it back into the queue and the packet goes to the server, which in turn changes src_ip and dst_ip to the original ones and sends everything to the interface that looks at the Internet. Data to the server about the initial destination of the packet comes through Nats, in the opposite direction the packet from the Internet goes the same way, the only difference is that the client still stores the data in the cache, and by record it sets from whom the client originally came.
I’m interested in such a moment, now there are two queues in the client and the server, both queues forward packets, but in different directions, I want to bring the whole thing back to normal, fasten a pool of workers for processing packets to each queue, what is the right way to divide it into microservices? each queue as a separate application? It was enough to write an application of the mind, but how to properly organize everything in this gap, tell me where to dig. What additional tools to choose? redis for an in-memory database, or will it be faster to store everything in standard hash tables?

Answer 1

It's all mixed up, but I'll try to answer. I didn’t understand at all what the essence of this infrastructure is, but oh well.
As I understand it, your client-server interaction is implemented through queues on top of the nuts.
It is not clear why your client stores data, although the server should do this. Perhaps there is some substitution of terms in this.
We will call it this way: there is the first service that replaces src_ip and dst_ip in a packet from the queue, and sends it to another queue.
There is a second service that also replaces src_ip and dst_ip in a packet from the queue, but already sends them to the network directly upon completion.
There is a task in the first service to store some persistent data.
Here's how I would do it.
It is better to make all services stateless so that you can run as many replicas as you need and not be afraid that the data will get mixed up somewhere. You can, of course, run one and try to pull everything out on it and store data in the RAM, but the RAM is not cheap, and with one application you can run into certain restrictions on a heavy load.
So, I would make several services:
- service 1, receiving packets from queue 1, changing them using data from the database (and writing to the database), and writing the result to queue 2
- service 2, reading from queue 2, changing the data, and sending them to the Internet (I don’t understand why they called it a server, if there is more logic there, then it makes sense to write to queue 3, from which the third service will read and only send packets to the Internet)
- optionally service 3, reading from queue 3 and sending data to the Internet
As well as infrastructure:
- nats
- database (what suits / likes more, well, I would take radish or mongo as the most elementary ones)
At the same time, you can scale each service (launch replicas) to handle the stresses. This can be done in Kuber, or you can do it somehow, it's up to you.
But in general it looks like a heavily overcomplicated garbage, this can be done in one application with three worker pools and two channels.