How to make a simple horizontal sharding for the file system?

Question

linux

Data is stored in the file system in folders of several levels. The whole idis translated into the "address" of the folder, for example, /data/AF/BF/C4/06/where there are several binary files with data sets for this id.
Hosted by an inexpensive provider, where it is more profitable to take another box than to increase the disk in an existing one. As it grew, I had to connect several new servers and transfer some of the trees to them. The top level folders are now split like this:

00–3F – основной сервер
40–7F – 1-й дополнительный
80–FF – 2-й дополнительный

Everything was done in a hurry, so I just connected the folders with additional. servers via NFS and instead of some folders, /data/there are now symlinks to NFS mounts. Everything works ok.
Now I want to use the computational abilities of the add. servers, running workers or backups on them.
In principle, you can symmetrically mount folders from “remote” servers on them and, in the same way, use symlinks to collect a complete set of folders 00–FFin the working data folder.
Will such a scaling of the project be “correct”? Each of the N servers will have N-1 mounts from the rest.
Once I tried to do this - almost everything worked, but "artifacts" appeared, which I didn’t figure out: when creating a new sub-sub-folder on one of the servers, cyclic symlinks appeared if the name of the folder being created coincided with the name of the parent.
For example, it was required to create /data/AF/C7/C7/56/(from php, mkdir()with $recursive = true). /data/AF/C7already existed, but instead of a new folder nested in it C7, a symlink to the parent folder was created C7, and the script crashed with a critical error about the impossibility of creating a directory. I still don't understand why this occurs.
Maybe you should look at other solutions for file data storage and horizontal sharding? Hadoop's HDFS seems like overkill for my little task.

Answer 1

https://www.youtube.com/watch?v=z0jQFoW_QgM

Answer 2

Perhaps Gluster will suit you.