How do modern messengers work?

Question

Software design

Good afternoon, I want to understand the architecture of modern instant messengers, like Telegram. I'm specifically interested in the part with storing and returning messages, because I can't understand how the server can scan the database in such a short time, find the messages of a specific user and give them away. I'm also interested in the encryption side, especially when e2e encryption is not used, that is, how to monitor security / leaks, thanks in advance for your answer.

Answer 1

how the server in such a short time is able to go through the database, find the messages of a particular user and give them back

I would do it a little differently, for example, the user sends a message that the server receives.
If the recipient of the message is also connected, the server simultaneously delivers the message to the recipient, due to this we have fast message delivery, and in parallel saves the message to the database.
As for the fast loading of the message history, then you need to save the messages in such a way that the messages of one user are always on the same server (if sharding is used). Then the usual SELECT from the database by user_id will work quite quickly even on a huge database. Tables can also be partitioned to load the latest messages in history even faster.
As for security, if you don't use e2e encryption, you can alternatively use regular RSA keys. For example, we generate two keys on the server, send the open one to the client, he encrypts the message with it and sends the message to the server. You decrypt it on the server using the private key. To send a message to a client, you can do the same. The client also generates two RSA keys and sends the public key to the server. When the server needs to deliver a message to the client, it encrypts the message with the public key sent to it by the client and sends it to him.
I am also interested to know other options for solving this problem, I will follow this topic :)
A good addition for storing messages and organizing search from Ilya
The load distribution is solved by sharding - it turns out many small databases instead of one huge one. Most likely, each message is divided into words/parts of words and stored in a word-message_id type search index, and such an index is built for each user and is also sharded. When searching, first we get the identifiers of suitable messages from the search index, then we unload the messages from the database with messages.
The addition from Stalker_RED
is not only "messages from one user were always on the same server" but messages from one chat / channel / group (including chats in which there are only two participants). The same goes for building an index.

Answer 2

I can't figure out how the server can go through the database in such a short time, find the messages of a particular user and return them.

select by id is the fastest, in any database, what exactly do you not understand?
he does not assemble into one chat, the application requests as many chats as needed