P
P
PendalF892018-11-04 16:12:18
MySQL
PendalF89, 2018-11-04 16:12:18

How to store a large number of json files?

Hello!
It is required to store approximately 60 million json files on the server and access them from MySQL.
At the moment, they are all in zip archives and links to them are in the MySQL database (used by the project).
I ran into the maximum number of small files that Ubuntu can hold.
The essence of the question: how best to organize storage with maximum savings on disk space?
Options that came to my mind:

  1. We store JSON directly in MySQL. This is convenient, but the database will grow and take up a lot of space.
  2. Store JSON in MongDB, and store a reference to id in MySQL. This is less convenient, but maybe MongoDB compresses files better?
  3. Store zip file directly in MySQL. Never did that.
  4. Store multiple JSON files in a zip file. Inconvenient, additional complexity in implementation.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
Anton Kucherov, 2018-11-04
@PendalF89

Perhaps the best solution would be to actually store them in MongoDB.
Moreover, it supports data compression , and in addition, it becomes possible to easily manipulate them, search for information in them, index, update, etc.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question