The dependence of the speed of the server on the number of files in the folder?

P

pel0tkin2011-01-15 13:57:52

*nix-like systems

pel0tkin, 2011-01-15 13:57:52

Guru * nix systems, tell me how the speed of the site depends on the number of files in folders?

I have a website with 10K images in one folder and the number is growing fast.
Is it worth it to think about sharing the image storage?

Reply

Answer the question

In order to leave comments, you need to log in

7 answer(s)

O

osdyng, 2011-01-15
@osdyng

Depends on the file system. For ext3/ext4 a large number of files is not a problem.
It is necessary to share, because. it will be more convenient to backup, maintain and develop.

N

nill, 2011-01-15
@nill

The file system is loaded from the total number of files on the disk (not significantly) and the frequency of access to them, the number of files in the folder is on her drum.

E

Eugene, 2011-01-15
@Agent_J

Be sure to share. As the number of files in one directory grows, you yourself will notice slow access to a specific file by its full path, create a couple of levels of subcategories so as not to run into this problem.
And at the expense of the hoster's control panel, the situation is slightly different. The panel scans the contents of your folder, taking into account the number of files, this takes some time.

P

Puma Thailand, 2011-01-15
@opium

It is necessary to divide, in a million when the folder is polled, the lag will be visible.

K

Karen Kratyan, 2011-01-15
@kratkar

If I'm not mistaken - in nix systems - everything is files: folders, ports, devices, etc. Therefore, she does not care. Files are stored on disk, regardless of folders.
But I would recommend separating by folders. Imagine that you will work (view, delete, etc.) with a list of 50k+ files, for example.

D

durach, 2011-01-15
@durach

For some reason, in a project with a VERY large number of files, we divided it into folders of the form instead of the name abcdefgh => a/b/c/abcdefgh. Our CTO said that it works faster this way. Although I have not tested in practice.
What I checked, then going into the folder with the same MC with 10K files is not so easy ...

A

admin4eg, 2011-01-15
@admin4eg

I divide by date.
/images/2010/1/
/images/2011/10/
etc. I'm afraid to count how many files I have there, but I have already exhausted the possibility of a terabyte screw by inode by 50%