Why can't the nginx proxy on the local network handle a minimum load of 100 people?

Question

Nginx

There is a server on ubuntu It stands as a proxy cache server for the main server where the video files for distribution are mainly located.
Here is the proxy config

config

proxy_cache_path /var/www/html/cache levels=1:2:2 keys_zone=cachessd:100m max_size=400G inactive=12h use_temp_path=off;
proxy_cache_key "$uri$slice_range";
limit_speed_zone speed_zone_one $binary_remote_addr 10m;
server {
listen *:80;
root /var/www/html;
location ~* \.mp4$ {
proxy_pass http://10.0.0.1:80;
proxy_set_header Host d3.kinohit.uz;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
limit_speed speed_zone_one 700k;
slice 5m;
proxy_set_header Range $slice_range;
proxy_cache_valid 200 206 1d;
proxy_cache cachessd;
proxy_cache_key $uri$slice_range;
output_buffers 1 64k;
directio 512;
aio threads;
}
}

This server is connected via 1gbit ethernet to a file server directly on the subnet at the second ip 10.0.0.1
Server characteristics: Core i3 2 cores, 16GB RAM and 2 ssds of 480gb each on one system on another given cache
this server has a two port network card Tp-Link one port for local connection with file servers and the second for accessing the network there is a channel of 800Mbps
When there are no people on the server, everything works fine and video playback starts within 2-3 seconds, the videos are not large, the average size is 400mb
As you can see, the speed for the user using nginx module limit_speed.
As soon as at least 100 people enter the server, the start of the video is delayed up to 10 seconds, although if you watch through ATOP, then the load on the ssd is no more than 15% and the load on the HDD disk on the file server is no more than 5% why can the speed slow down with such a small amount people, no errors appear in nginx CPU when 100 people are loaded no more than 5-10% and the RAM is generally free. Including if the video starts rewinding, then this is also delayed by 5 seconds and at first it can freeze a couple of times for 1-2 seconds.
Used different config options.

nginx.conf config

user www-data;
worker_processes 8;
pid /run/nginx.pid;
worker_rlimit_nofile 200000 ;
events {
worker_connections 3000;
use epoll;
accept_mutex off;
multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 10s;
types_hash_max_size 1024;
# server_tokens off;
# open_file_cache max=200000 inactive=1d;
# open_file_cache_valid 1d;
# open_file_cache_min_uses 2;
# open_file_cache_errors on;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
#access_log /var/log/nginx/access.log;
#error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip off;
# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rs$
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}

What could be the problem? At first I thought that it was just a rather weak server, but after looking at atop it became clear that half of the possible is not being used. And the file server itself gives mp4 as standard, it does not limit the speed, aio on, output_buffers 1 1m and everything else has not been done there, and if you start the video bypassing the cache server, that is, directly through the file server, then the video loads instantly and does not freeze.

Answer 1

lazy to check here
but usually a tap on Nginx - User does not affect / is not a tap on Proxy - Nginx
is secondly
and firstly: 1000/100 = 10 Mbps per user is already kind of narrow:
by byte per pixel - picture 320 * 240 yes for 24 frames = 1843200 bytes = 1.8 MB
no, well, it's clear that mpeg compresses there, this is not a stream, etc. etc.
but physics is physics