R
R
Roman Rakzin2017-07-01 17:51:54
File systems
Roman Rakzin, 2017-07-01 17:51:54

What file and folder structure to choose if there are a lot of files?

How is the structure of a file system with a large number of files organized for the fastest access to any file?
As I understand it, the option to keep everything in one folder with a large number of files will work very slowly. Or is it not?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
R
RidgeA, 2017-07-01
@RidgeA

As I understand it, the option to keep everything in one folder with a large number of files will work very slowly. Or is it not?

So.
how much is this?
Alternatively, you can solve it within the framework of a regular file system, but spread the files into several folders.
For example, take MD5 from the filename, some number of characters from the end of the hash and put the file in a subfolder with that name.
If you take 1 character, all files will be distributed over 16 folders (if you take from the hex representation of the hash)
If 2 - 256 each
. Etc.
If this does not suit you for some reason, then you need to look towards distributed file systems

N
nirvimel, 2017-07-02
@nirvimel

When the number of files reaches millions in one directory, then the issue is not only performance, here you can face much more serious problems: see How best to organize a digital library?

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question