Why is the server under low load slow to respond and ping grows to 80ms?

Question

System administration

Good evening, tell me who can come across. There is a cache server for another server with video. Videos began to load disks, so they put another server next to them, made a separate interface of 1Gbps, and through Nginx proxy_cache proxy_pass gives files to the client.
So, at the moment of not even the highest load, the ping grows immediately to 80ms and the files are given more slowly. The external interface that has Internet access also has a 1 Gbit channel (It was decided to make 2 interfaces so as not to load one channel on proxy_pass to the local server).
Why can the ping on the server grow so much. The Internet channel is consumed on average by 200Mbps and the local channel is 250Mbps, that is, only 20-25% of the load and such problems.
Who faced? Where to dig, what could be the matter?
The network adapter costs the most common Tplink through 2 ports.
The server itself is
Intel Core i5-4670 RAM
16 GB
The system lies on the SSD and the cache is written on 4 other ssd disks

Answer 1

Why is the server under low load slow to respond and ping grows to 80ms?

In general, at first glance, the situation seems normal. With such a noticeable load, the ping should inevitably increase.
It's one thing when there is no load - I sent a request, and immediately received a response, and under load - I sent a request, this request stood in line, and returned with a noticeable delay. The queue is a nuisance. What's in the supermarket, what's online.

The Internet channel is consumed on average by 200Mbps and the local channel is 250Mbps, that is, only 20-25% of the load and such problems.

Everything is not so simple here with a load of 25%.
Just imagine - there is a section of the road, with a capacity of 100 thousand cars per day. This means that it can comfortably pass no more than 100 thousand cars a day without problems and traffic jams.
In reality, an average of about 25,000 cars pass through it per day - a load of 25%. Everything is great.
Do you think there are traffic jams in this area?
Of course there is! And very big.
It's simple - in a day it can pass 100 thousand cars, in an hour - 100/24 ​​= 4.1 thousand cars.
In the morning at rush hour, 10,000 cars try to pass this section, which is more than twice its capacity, and a long traffic jam forms in which people stand for an hour.
In the evening the situation is the same, the same 10 thousand cars are trying to pass it in an hour.
And in the remaining 22 hours, another 5,000 cars pass freely and comfortably through this section.
As a result, we have an average route load of 25% and wild traffic jams and a clear bandwidth overload.
It's the same in the network - you look at the average load per second. It is equal to 25% like everything is beautiful.
In fact, you have huge queues there in hundreds of milliseconds alternating with an absolutely empty channel.
Conclusion - on a gigabit channel you can drive up to 1 gigabit, without increasing the ping (standing in queues) only in the case of an absolutely uniform load on the channel.
And since the load is evenly rare, the ping will grow with any noticeable load, because there will be queues.

Answer 2

1. Look at the routes: apparently, the ping still goes where it should not have gone at all (multicast storm or the like).
2. Turn off the interfaces on the network card alternately and see if there are any changes.
3. Try to replace the adapter with a normal one.