E
E
Egor Skorokhodov2019-03-26 18:18:43
PostgreSQL
Egor Skorokhodov, 2019-03-26 18:18:43

How to properly store such data?

It is necessary to store the user entity with N fields, where N is a previously unknown number, 1<=N<=2000. Many fields will be repeated (full name, email, phone number), but their presence is not guaranteed. There is a need at any time to get all existing field types in user.
The upper ceiling of the planned scaling is several million records.
High sample flexibility is required.
The question arose in choosing a database:
Relational database, PostgreSQL. Storage options:
1. user table: (id + JSON field) + separate table for storing the list of fields. To work this business, as I understand it, will be very, very slow.
2. table user: (id) + table field(primary key, name, value, user_id) wrapped with foreign keys on top. Problems: data redundancy -- if a million users have a full name, the field table will have a million fields of the form:
[ 999 | Full name | Ivan Ivanovich | 555 ], ... , [ 9999 | Full name | Petr Petrovich | 666], ... .
3. user table: (id) + field table (id, name) + field_value table (user_id, field_id, value). Pros: minimal data redundancy, very easy to get a list of all fields. Cons: the number of rows in the field_value table will tend to count(user) * count(field), which with a million user and a thousand field will equal a billion. Is that too much?
Document-oriented database (mongo)
Pros: The storage model is ideal for our data.
Cons: in general, I read a large number of horror stories about mongu, the attitude is slightly biased; lack of experience; fear in the future of encountering too unrelated data. (I would like to be able to painlessly associate data with other entities (products / tags / etc)
Actually, there is an understanding that a mistake now can be very costly in the future, so I ask for advice. Is it worth using one of the listed or unnoticed methods with sql- base, or should you start learning Mongo?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
V
vanyamba-electronics, 2019-03-26
@egor_sc

ID (UNSIGNED INTEGER) AUTOINCREMENT, PRIMARY KEY
USER_ID (UNSIGNED INTEGER)
RECORD_ID (UNSIGNED INTEGER)
FIELD_ID (UNSIGNED INTEGER)
FIELD_VALUE (VARCHAR)

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question