Choosing a file system for a video server?

Question

linux

Hello. The bottom line: there is a site with films, on which the films themselves are stored, and, accordingly, are played. There is a problem with the load on the disk, because of this, the speed of reading and uploading files is much reduced. Worth RAID 0 LVM, volume 2x2TB. How to be? In which direction to dig? What would you do? Thank you.

Answer 1

First, think - why do you need RAID 0?
In this situation, it is not only useless, but also harmful.
Zero raid of two disks is able to almost double the speed of linear reading . You do not have linear reading, but a lot of random queries are obtained under such a load, so there is practically no sense from it. Two independent disks on which films are scattered in this situation will work faster!
Look at the statistics - how evenly do users watch videos, which files are more popular, which are less?
If there are pronounced leaders in download views, then add an SSD drive and store the most popular videos on it.
In this case, there will be no need to spend money on a huge SSD, and requests will be scattered over three disks, and the bulk of requests will fall on SSDs with "hot" data.
If it is not possible to allocate "hot data" and all videos are viewed equally often, you can go by simply increasing the number of disks.
For example, instead of 2hdd*2Tb, use 4hdd*1Tb - and without any raids. The load in this case will be distributed more or less evenly between the four disks.
The ideal file system is XFS.

Answer 2

Well, about XFS (and even more so SSD) you have already been written ... I'll come up a little from the other side. As usual, I suspect that there is a web server that distributes this video to the site, and on the client side there is a browser with flash or the like. In other words, you are streaming video. What does the player do on the client side besides playing the video? And he also caches it! How does he cache it? And these bastards cache for how many resources they have, using the entire disk space and the entire bandwidth !!!
I will explain on my fingers, the client will watch a video with a bitrate of 3000 kilobits, and this video is cached for him at the speed of the client connection (100 megabits for example) and will continue to be cached until the entire file is downloaded completely. Now we have 10 clients that immediately gobbled up all the server resources and bandwidth, for a short time of course, but users arrive, for example, once every 3 seconds, and now they don’t receive any resources at all until the first 10 download. Now let's imagine that the first 10 customers watched only the first 3 minutes of the video and switched to another one. But we then gave them a two-hour video for these three minutes!
What to do? Yes, it's very simple, limit the speed per client to double the maximum bitrate. This can be done both with iptables/tc , and with policies on nginx, for example. Well, either give it in HLS format or the like, or install video server software, which in general is almost the same thing ...
That's just even in the example there is (for a reason) www.nginxtips.com/how-to-limit-nginx -download speed

location ^~ /videos/ {
...
limit_rate_after 1m;
limit_rate 150k;
...
}

PS. And yes, encode video in CBR for distribution. VBR is not very suitable for this ...
Well, it seems that all the military secrets have been revealed ... Well, in addition, go to HLS!

Answer 3

As for FS, it is worth looking towards XFS, of course. And look at the SSD, the hard drive will always be slower than the SSD.

Answer 4

The obvious opinion of Windows
is the linear read speed of the disk, there is the read speed of random clusters.
According to linear reading, hard drives give out normal drives up to 150MB / s,
which blocks the gigabyte Internet channel.
On random queries, the disk speed is only a few megabytes per second due to mechanics (limited number of heads, low disk rotation speed, long time for positioning heads on the desired cluster). Number of IOPS up to 200 operations / s
With a large number of users, the bottleneck is IOPS.
What they do under Windows
a) increase the size of the FS cluster to the maximum (but not higher than 64-128kB)
b) increase the read cache in RAM or on a separate SSD
c) switch to SAS SSD and so on.
Before doing anything, decide on your IOPS loads and then it will be clear whether switching to a FS is enough, adding RAM for the cache, or only an SSD is needed
In general, read optimizations are ineffective, it is better to change hardware.

Answer 5

1. dig in the direction of the link catalog.
Firstly, even an ordinary SATA and even an IDE disk is faster than a 1GB communication channel. and even more so with large file sizes.
so the narrowest metso you have is the channel itself.
As for the raid ,
what does it show iotop -oka
under load?
problems with the speed of "file search" AS A RULE AT A LARGE NUMBER OF SMALL FILES AND NOT the other way around!

Answer 6

How about ZFS or Windows Storage Spaces with SSD cache?