Fast blob for storing files with compression

Question

Qt

Some data format is needed to store (readonly) a large number of files.

The essence of the problem is this: there are many hundreds of thousands of small, but highly compressible files. You need to pack them into blob archives, so that later you can quickly pull out files if necessary. Now zip is used, but there is a suspicion that there are more efficient solutions.

Yes, all this is needed for Qt, just C ++ will come down to the extreme.

Answer 1

In my backup, though in PHP, I used zip without compression (I wrote my own implementation, otherwise the ready-made libraries were very stupid), as a container, and the entire stream was compressed with gzip. It turns out something like a Rarovsky solid archive, it compresses better and faster than in a regular zip, and quite fast navigation is possible (since there is a table with all the contents at the end of the zip). Plus, this archive is perfectly readable by any software that can work with archives. Although, of course, it is inferior to the usual zip in terms of delivery speed, but here it is already necessary to prioritize, which is more important than the speed of packaging or the speed of delivery.

Answer 2

Lots of small read-only files? already answered a similar question:
habrahabr.ru/qa/10694/#answer_46206