D
D
Dmitry2019-07-15 22:04:13
Database design
Dmitry, 2019-07-15 22:04:13

What database should be used to store the history of changes?

It is required to store the history of all changes in the database (about 50 tables, from 3 to 60 columns in each). In order not to litter the main database, the history of changes is transferred to the historical database, for each table separately, with the structure "date | user | ip | id | field | was | became | request".
Read requests are rare, such as "pull all changes by ID", "find who changed the value A to B".
Now mysql+myrocks with compression enabled is used for the historical base. The maximum amount of data in one table is 500'000'000 records, which is about a billion in total. Several tens to hundreds of thousands of records are added daily.
In general, it works ok, but the reliability is questionable - there have already been cases when, when trying to make a change in the structure of one table, the entire database lay down, irretrievably damaging files on the disk.
What would you recommend to use in this case? Redshift? Greenplum? Tarantool?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
I
Ivan Shumov, 2019-07-15
@inoise

The main thing is to take a non-relational database with horizontal scaling - MongoDB, DynamoDb or something like that. You need to have no disk limit, filter by key, sort and no limit on the size of the value.
PS if you have a base when you change the structure of the table, then you either do not observe versioning or the problem is in the absence of tests

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question