Kvm virtualization and cgroups policy?

Question

Virtualization

There is a node with KVM virtualization. Connection with SolusVM. I noticed that with large numbers of VPS servers on the node (More than 50), all free memory on the node starts to go into the cache. Also completely clogged SWAP. But really, there is still a lot of free memory.

At such moments, some servers start to fall with an error from cgroups about lack of memory on the VPS server itself. As if the limit was exceeded, and the VPS is turned off. Tried updating everything to the latest versions.

I also had the same problem on other nodes, but somehow it cured itself after cleaning the swap. Clearing the swap via swapoff -a also crashes the VPS.

I tried to disable cgroups, this leads to the fact that after rebooting the VPS gives an error that the cgroups folder was not found.

Has anyone experienced this?

Answer 1

servers per node ( More than 50 ), start all free memory

Lord, guide these people to the right path.
I hope you have a raid10 from an SSD there?
If you didn’t understand the subtle hint, then: 50 virtual machines for kvm is a lot. Too much. The system does not have time to write data to disk (just not enough iops). Therefore, the entire cache is kept in memory and cannot be flushed to disk.