How to store files, photos, videos?

Question

Data storage

Yes Yes! I know that such questions have already been asked more than a dozen times, but after reading a few I still don’t understand where and how to store files, photos, videos. Let's say I need to store the user's avatar. How can I do it? In the database to store a link to the photo, but where to store the photo itself?

Answer 1

The simplest and most correct one is in the file as it is, for ease of maintenance in the file system, the container of which is separated from everything else (a separate partition or disk is better, but in case of extreme poverty, a file that can be mounted in any fs, for example, in windows it is . vhd / .vhdx images, and in linux it’s just a disk image in raw format), why a separate device? - to simplify maintenance when there will be a million files... on linux, I strongly recommend using cow fs (btrfs/zfs/xfs) for the sake of built-in free transactions based on snapshots, plus incremental backups to do efficiently.
Keep the rest of the logic in the database, the file name can itself be an identifier, i.e. You can not even start the corresponding table in the database, but it's better to start.
In many cases, it is recommended not to store many (tens of thousands) files in one directory, hence the method immediately suggests itself - if the file name consists of numbers (or hex or even base64, to choose from), then breaking it into fixed-length parts, you can use them as directory names , the last part is the name of the actual file in them (such as /datastore/0d6a/011fc/0012.jpeg).
Advantages
- for the web, files can be very effectively given by the web server (if necessary, access rights are resolved by basic auth or temporary symlinks, xs which is more expensive)
- a huge number of utilities can work with files, search, copy, edit, view .. directly when As with any other storage, you first have to extract the file from there, i.e. you will need the appropriate
tool
- probably the only noticeable one - the file system is not integrated with the sql database, which means that if there were any problems, for example, they deleted the file in the database, but due to an error or failure of some kind, the file was not deleted on the disk, and it remained so hanging, detecting this, on especially large storages is expensive
- if files are scattered across subdirectories (name - file identifier), then this operation itself is not free and, with a large number of nestings, increases the load on the processor and the overhead on the disk, slightly but noticeably on large archives (this is the price for a free key-value index), in some situations you can avoid this by storing files in one directory, but a million files can create problems for familiar utilities.
ps well, or here qna.habr.ru/q/10694/#answer_46206