S
S
StynuBlizz2017-05-27 06:40:56
Database
StynuBlizz, 2017-05-27 06:40:56

How is the event feed implemented in popular services?

How is the storage of the event feed (meaning the storage of information about the events of the people I subscribe to) in the database, in popular services (Insta, yotube, facebook, vk, etc.) implemented?
I heard about two implementation methods:
1.Storing all events in one table, something like:
AUTHOR_ID | POST
001 | ...
002 | ...
005 | ...
2. Storing an individual tape for each user (everything is also stored in one table, but only each event is duplicated for each user) something like:
USER_ID | AUTHORID | POST
001 | 002 | ....
001 | 006 | ....
002 | 001 | ....
I initially chose the second option, but the problem is that you need to store the number of video views, and for this, with each video view, you will need to change the number of views in each duplicate post entry (there was a VIEW_COUNT column for the number of views in this table).
In general, how is it all done in popular services?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
I
Ivan Koryukov, 2017-05-27
@StynuBlizz

Why do you not like the first option and what are the advantages of the second?
Logic dictates that duplicating a post for every user who subscribes to it is absurd.
Are you saying that if I make a post in a group with 100k subscribers, then I immediately create 100k records in the database?
The number of posts in the posts table will be several orders of magnitude larger than in the first option, and this will only slow down the selection.
However, the giants are unlikely to use the first method just like that. There are billions of social media posts. They cannot be stored in the same table so that a simple

select * from posts join subscribe sub on(...) where sub.user_id=123
produced results in a reasonable amount of time.
Most likely, some daunting indexes are built there, some smart caches are used, the database is distributed so that hundreds of machines search at the same time, but each machine in its own piece of the table...
Don't worry. Do as you see fit, but be aware that as the number of users grows, you will have to rewrite everything again, and when you enter the world level, most likely you will already have a team of specialists, and you will rewrite everything from scratch a couple of times using other technologies.

R
RidgeA, 2017-05-27
@RidgeA

Look for the report Arsen Kostenko How Twitter Works, for example https://www.youtube.com/watch?v=tlPkM0lxMKc
In short, the feed for each user is calculated in advance.
But

Don't sweat it. Do as you see fit, but be aware that as the number of users grows, you will have to rewrite everything again, and when you enter the world level, most likely you will already have a team of specialists, and you will rewrite everything from scratch a couple of times using other technologies.
- 100%

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question