Z
Z
ZxAnderson2010-10-08 14:47:25
File systems
ZxAnderson, 2010-10-08 14:47:25

Too many pictures in one directory?

On the server, images are issued using the nginx server.
How many images can be placed in 1 directory for the ext3 file system, so that the time of random access to files does not increase and there is no unnecessary additional load due to searching for files in a directory with a huge number of files?
Accordingly, the same question about directories: what is the best number of subdirectories in a directory?
What restrictions on the lengths of directory and file names should be imposed so that the search time does not increase, what restrictions on nesting levels?
The same question is of interest (but not for practical, but for educational purposes) for the reiserfs, ext4, xfs, jfs file systems ... And which of these file systems will better cope with such a task?

Answer the question

In order to leave comments, you need to log in

5 answer(s)
B
Bambre, 2010-10-08
@Bambre

In ancient times, a way was invented to store many files according to this principle:
1) it was: /files/abcdefg.gif
2) it became: /files/a/b/c/abcdefg.gif
I have never seen a nesting depth of more than 3 in practice, as well as delays. Here is the first recipe - there were delays, but then there was still ext2 and several thousand files in the folder. Unfortunately, I do not remember the exact figures due to the prescription of years.

F
fozzy, 2010-10-08
@fozzy

for small files reiserfs is better, for the rest ext*
As for storage, I agree with Bambr. Just take a look in the direction of the gravefs

@
@mgyk, 2010-10-08
_

The maximum number of files in one directory for ext3, in my opinion, is about 32k.
To place files evenly across subdirectories, this scheme works best. m5sum(file) = a1b2c3… Then we place the file in /a/1b/filename… we get for 100k files 23-25 ​​files in the directory

S
sendmail, 2010-10-08
@sendmail

just to access up to 5-10 thousand you can not bathe

A
Anton Korzunov, 2010-10-09
@kashey

used to store in the front-cache (two gigs) + 4 gigs of pictures (700k chtuk) in $1/$2/md5(name) where $1-$2 are the first two and two second characters of the md5 name.
Now I store in the cache and mysql.
The reason is simple - when it was necessary to move to another server, daddy's archiving took ... 6 hours

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question