How to remake the storage system of a highly loaded site?

Question

PHP

Good evening!
I have a fairly large portal that has been operating since 2010.
At the moment, more than 2,000,000,000 images have been uploaded to the server, and 1 server with a database is used to store them, where paths to servers with images are stored, as well as 4 file servers where these images are stored.
The path to images is formed and stored in the database in the following form:

// s3.domain.com/56/a89/1f1/c7f/56a891f1c7f2451a9e1dd66141334a5d.jpg

The site itself is written in PHP, I started developing it back in 2009, and since then a lot of things have been added and changed, but the site architecture itself is currently very inconvenient, and it’s hard for me to maintain it and add new features, so I decided to rewrite it again with taking into account new technologies and experience gained.
For several days I have been developing a new way of storing images, loading and distributing them, I want to add replication to provide better fault tolerance (now all pictures are stored in one place, and if something happens to the disk, they will be lost).
There are few articles on the Internet on organizing the architecture for storing such a huge number of images, I will be glad for any information on this topic.
Are there any software servers? So that, for example, from PHP you can do something like

// сохранить в файловый сервер изображение, который загрузил пользователь.
// сейчас это изображение лежит во временной папке
$filePath = $fileServer->saveFile("path/to/tmp/folder/user_file.jpg");

// filePath теперь хранит в себе что-то вроде
// http://s20.domain.com/uGDthECHrWU/s46-c-k-no/image.jpg
// нужно сохранить этот путь в бд, чтобы потом отдавать его пользователям.
$db->savePathToUserFile($filePath);

In other words, is there any specialized software that will organize the storage of files so that I don't have to organize the paths to the files myself, give them names, etc.?

Answer 1

Food for thought: one , two

Answer 2

Distributed FS is everything. As I see it, from the free options, Ceph is ideal for you. But it is relatively free, as there are some server requirements. More precisely, according to their number. Plus, if the speed of access to data is important (by itself), then you won’t get by with all sorts of primitive Fiber Channels here. But this is no longer a feature of the FS, but simply common sense. The FS itself will function fine even on FastEthernet.

Answer 3

https://www.insight-it.ru/highload/
Choose any as an example.

For several days I have been developing a new way of storing images, loading and distributing them, I want to add replication to provide better fault tolerance (now all pictures are stored in one place, and if something happens to the disk, they will be lost).
There are few articles on the Internet on organizing the architecture for storing such a huge number of images, I will be glad for any information on this topic.

Than S3 does not suit?
Well, take 3-5 VDS. Make between them Ceph+ObjectStorage.
You will need a normal local communication channel between them.
If the connection between servers is bad, then take Yandex Elliptics.