How to determine the load source on the server?

Question

Nginx

There is a VPS under CentOS with splash installed. Periodically, the server starts to slow down, but at the same time:
1. The processor is not loaded (less than a percent is used).
2. Memory is free (more than 700 meters), no swap.
3. A bunch of free nginx (as a frontend) and apache (as a backend).
4. Mysql does not seem to slow down.
5. No crowns are configured for the brake time.

Load average grows up to 7 and after some time falls smoothly to values ​​less than 1. Growth occurs at any time of the day or night. At these moments, there are no prerequisites for an increase in load - single requests that work out with a bang at other times.

We have already optimized the project code, set up opcode caches, connected the static cache.

We sinned against the neighbors on the physical server, but, firstly, we are not sure that they can influence our LA, and secondly, the hoster assures that they have nothing to do with it (although there have already been cases when our projects went down under ddos their sites).

In which direction to dig? What generally can be done?

Answer 1

rddtools if it does not give an answer, what slows down, it will help to find a pattern when this happens.

Answer 2

give at least the output of top at the time of this download.

Answer 3

change host. disks.

Answer 4

What do you want from VDS? - this is a communal apartment on the server. Your project may slow down due to neighbors.

Answer 5

Disk operations?
From observations of my VPS (CentOS + ISPmanager) - LA increases greatly during backups. tar raises LA to 3-4 on large amounts of data, rsync raises LA to 1-2 at the time of file matching, rsync has almost no effect on LA during the actual file transfer. The rest of the time, when backups are not taken, LA does not rise above 0.7. So maybe you just have a static cache that increases LA.
CentOS+Plesk is not in your VPS hosting center by any chance? The discs are very slow...

Answer 6

The same problem, hosting at the rate 1024: www.1gb.ru/price_vds_hv.php
The page can be generated in 0.01 or 5.5 seconds - the hoster says that this is all because of the disks, but only the original file is read from the disk and 4 classes included, 0.7 GB free memory, less than 5% load.

Answer 7

usually latency in CPU or HDD, check with vmstat (procs r-CPU, b-HDD)
for FreeBSD -> see processes that consume
CPU resources #top -I
HDD #top -m io

Answer 8

look at wa, how to catch it. most likely the hard sins.

Answer 9

Once there was such a problem - it turned out that the hardware raid was dead and started to fail. Moreover, either he scored WA to the eyeballs, and everything began to slow down terribly, or even without iowait it slowed down. Replaced the raid - everything worked fine.

Answer 10

Something doesn’t agree with me:
“The processor is not loaded (less than a percent is used)”
and
“Load average grows to 7”
Or are you talking about “less than a percent” for one process in the top?
For a heavy load, see the top, and pay attention to the "%wa" parameter in the third line, it should be very small, no more than one percent. This parameter is responsible for waiting for the processor for incoming data, there can be two reasons - either reading from the disk is actually slow (the first post is here, and comments to it), or it is waiting for data from the network. In any case, you can't do without a top.

Answer 11

If nothing is loaded, then how did you know that the server was "slowing down"?

Answer 12

as an option - maybe there is a software raid?