Answer the question
In order to leave comments, you need to log in
How to store a lot of data compactly and securely?
There is a table A. It is necessary to store about a million 4-byte integers (~ 4Mb) for each new record for about a day, and then delete (an array of numbers, not records in A). Let there be 1000 new records per day. The most important requirement for these arrays of numbers is that they must be quickly written and read into the database in one piece of 4MB. Those. element-by-element reading/recording and even more so there will be no selection with conditions.
The most optimal solution that comes to mind is to simply store it as binary data in regular files on disk. But then the solidity of the database is lost, problems with the consistency of the database may arise. Is there any way to force postgres to efficiently store an array of a million int32 without O(1_000_000) memory overhead, but only O(1)? And just as quickly read this data from the database as if you are reading from a file without a single seek() ?
Answer the question
In order to leave comments, you need to log in
Store securely - this is in DBMS that support ACID. Array data, when (de)serialized in an application, can be stored as a BLOB.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question