What file system and with what settings is best to use for a very large number of small files?

Question

File systems

Only 60k files are expected on the server disk. 15k folders.
Structure: 4k folders, each contains about 3750 folders, each contains 4 files.
Files are initially 15-20 bytes. Over time, they will grow by 0.6 kilobytes per month for about 30 months - a disk per 1 terabyte.
Tested at the moment xfs and ext3 - what now stood on the server on two disks.
ext3 - block size 4kb, the disk is much more loaded with data.
xfs - block size 2kB Write
speed:
ext3 in 5 hours 20 minutes - ~18.400.000 files and folders created.
xfs in 5 hours 20 minutes - ~14.200.000 files and folders created.
The more data is written, the faster the write speed drops. For xfs, this point is significantly worse than ext3, although ext3 also does not behave very well.
According to the speed of the first reading of 50k folders using php functions opendir and readdir:
ext3 - ~ 800 seconds
xfs - ~ 1200 seconds
According to the speed of reading the same 50k folders the second and subsequent times:
ext3 - 5 seconds
xfs - 47 seconds
But that's only 50k folders. If there are 15k folders, then with ext3 the first complete reading of all folders will take about 66 hours. Need for ~12. And I'm afraid that re-reading won't be as fast as with 50k folders, because that much data won't be stored in the cache (or where is it stored?) all the time (or will it?). Even if the repeated and subsequent readings will take place very quickly, then in the case of a server reboot, you again need to spend 66 hours on the first reading. So it doesn't fit.
ext4 will run even faster than ext3 as I understand it. But it is unlikely to be able to read initially all 15k folders in ~ 12 hours using the php functions opendir and readdir.
How can you recommend something? Is ext4 much faster in writing/READING than ext3?
What file system to put to test, with what settings? Maybe there are other solutions to the problem of slow reading?

Answer 1

tormented and tested on all available fs, except for the raiser, since he died,
I had to stop on ext4, there are no special alternatives for small files, at least shove them into the database, but if you add them, this is not particularly suitable
, I also recommend reducing and making three levels instead of two folders
, the structure can also be stored in the database and not polling folders for the presence of other folders in them, that is, accessing files directly
for formatting and mounting, I now use something like this
mkfs -t ext4 -m 0 -O dir_index, filetype / dev/vg0/lv0
tune2fs -o journal_data_writeback /dev/vg0/lv0
noatime,nodiratime,noacl,data=writeback,commit=15
performance still leaves much to be desired, I just want to sit down and write a database for small files with awesome performance

Answer 2

ext4 is definitely better. formatting options -T small for small
reiserfs files, you can also try
testing on virtual disks located in files (in the same file to eliminate the effect of fragmentation)
dd if=/dev/zero of=/home/user/ext4test.img bs=1M count=10000
mkfs.ext4 -T small /home/user/ext4test.img
mkdir /mnt/ext4test
mount /home/user/ext4test.img /mnt/ext4test
test