1
1
12rbah2021-01-18 11:05:23
Data archiving
12rbah, 2021-01-18 11:05:23

Is it possible to speed up archiving/unarchiving by parallelization??

As far as I know, archiving / unzipping can often run into HDD / SSD, so I wondered if performance would increase if the standard algorithm for zip was changed so that archiving / unzipping could be done in parallel?

Answer the question

In order to leave comments, you need to log in

5 answer(s)
A
Adamos, 2021-01-18
@Adamos

7z started supporting multiprocessing of archives ten years ago. Its own algorithm is heavier than zip, but it can also unpack zips.
7z -mmt={number of threads}

K
kalapanga, 2021-01-18
@kalapanga

7-zip has an option to manage multithreading. You can test or even look at the source.

D
Dmitry, 2021-01-18
@dmtrrr

Parallel computing speeds up CPU bound tasks.

S
Saboteur, 2021-01-18
@saboteur_kiev

pigz is a multithreaded archiver for the gzip algorithm.
winrar and 7zip are also capable of multithreading

M
Mercury13, 2021-01-18
@Mercury13

If you use a standard compression library (for example, ZLIB), it will be problematic. You need to at least write your own compression with blackjack and multithreading.
However, it is possible to parallelize unzipping and interpreting what is unzipped, and this helps. Checked (wrote the save in a simple block format, packed in ZIP).
Archiving on ZLIB with the highest degree of compression rests not on the SSD, but on the processor. Also checked. For a balance between compression and the rest of the functionality, I set the compression ratio to four.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question