Why is the server response time different on the same sites?

Question

PHP

ubuntu server 16.04 LTS + nginx + php-fpm + mysql. Management via ispmanager
Clean installation. Well, in the sense that's just everything installed, by default.
Next, I create 10 domains on it with the same wordpress sites.
I measure the server response time on all sites from the same static page (the page is not loaded with scripts).
Several sites (4 pcs, let's call them "fast") have a minimum server response time, let's say about 100 ms. The rest (6pcs - let's call them "long") are exactly twice as long - about 200 ms.
It happens that when the server is rebooted, one of the "fast" sites starts to respond as "long", which is solved again by rebooting. It also happens that one of the "long" sites responds as "fast", but this is rare.
The difference between the sites is only in the size of the database, the first 4 sites have a database of 100mb each, and the rest have 10mb each.
All tests were carried out both in the field (when the sites are online) and on a new server, which users do not yet know about.
What could it be?