Which cluster solution to choose?

Question

linux

Please advise how best to organize a cluster for a site (bitrix), when the main load falls on disks (recording) and this cannot be avoided and nothing can be changed in the code at the moment, but the problem needs to be solved.
At the moment, RAID (hardware) with high-speed disks is used, and everything seems to be working, but the CPU and memory are idle, I want to move to clustering in the cloud to adjust the server power depending on consumption. There will be two servers for the front and two for the base, but the speed of the disks everywhere the providers have tested is not even close to the iron RAID, because in the raid its RAM is high-speed and the site starts to blunt without it.
The question is whether there are any caching solutions for the file system, maybe some network file systems that will work faster than SSD, the load is not constant on the disks, but spasmodic, but the site slows down precisely because of the disk access time.

Answer 1

Unfortunately, you didn’t describe your Bitrix or B24, so I’m writing on both at once, the difference is
not iotop -okasignificant won't even come close to perfect on nvme. and 2 GB per second is enough for your eyes. at least it is 1 two orders of magnitude greater than the channel speed.
3. The number of processors will not add to the performance of Bitrix, it will be added only by the frequency on the stone. The number will increase only if your server is bad. In a regular load LA \u003d 3-10, there will be no benefit from the number of stones.
4. As well as from the RAM on the latest versions of php bitrix satl, there is quite a bit of memory, and 16 is really with a margin, including the database cache.
5. Raising even the entire site into the cache memory will not give it a noticeable increase, so your iron raid will again give nothing. Since 99% of the time these are queries to the database, and the return of statics and both cannot be increased by this.
6. What to take
1. Bitrix VM can assemble a cluster, but you need to know Linux well, you need a file there. The last time I did this is https://www.mindomo.com/mindmap/-nebo-2c548660f4b4... As you can see, the cluster has a disproportionate dynamics towards increasing complexity.
2. Kubernetas is your choice and Yandex cloud (not to be confused with the disk)
The Bitrix site itself has long been in kubernetes.
It is now provided in the country by 4 companies
Google
Yandex
Selectel
Mali
If you don’t immediately pull kubertenas, then take the Yandex cloud and just raise the vps on it, it will be easy to scale it both horizontally and vertically. (although if you gave your hardware configuration it could be discussed in more detail)
7. Try the latest generation of gems plus nvme - maybe it will just solve your whole problem.
8. Most likely it’s easier for you to go the following way now
1. Upgrade php to 7.4 (the increase depending on the current version, if it’s still 5.6 then it’s huge, if it’s within 7 then up to 30%
2. Cache optimization in bitirx, as a rule it done simply through w. here the increase can be up to 2000% with ease
3. Transferring the heaviest selections to pure D7 Yes, you will have to rebuild the site a little, but if there are a large number of properties and bindings, then you won’t get anywhere (from 30 K products), the increase in this scenario will be up to 20000%
4. If there are very large data arrays, throw them out in hiload.
This approach has its main advantage, but in some places serious temporary injections may be required, but you do not increase the complexity of the project, which greatly affects its stability (maintaining a cluster is more difficult and more expensive than just a server, moreover, many times) at least these are monitoring systems and monitoring monitoring . And as a rule, the optimization of the site itself on Bitrix is ​​frankly 2. easier to do just that.
If you show the site it will be easier.
Remember that up to 50K a day, the site easily lives on 1 server and this is with 300K products.
It is easier to optimize the code itself, increase the complexity and "non-standard" solution. Yes, the latter will be sooner or later, but better late, it is financially cheaper.
If you have an ordinary HDD, since b24 is very gluttonous, then everything is simpler.
Connect the main folders of the upload to Yandex cloud or selectel cloud, you can do it right in Bitrix. As practice shows, this is an apload mail in tasks, and so on. Everything in its folders, as a rule, is 1-10 documents or photos, which easily rises from the cloud without problems, there is no need to keep this entire garbage package on the server itself.
If you have a hetzner (I would recommend it to you under b24 because the most powerful hardware for adequate money), then you can simply buy a disk (cloud) and mount it via fstab in an upload, 10 TB is enough for your eyes. yes, and core 9 and the last ryzens are cheap there, and under b24 this is what the doctor ordered.
Here I wrote in more detail
https://klondike-studio.ru/standards/trebovaniya-k...

Answer 2

Raid can be expanded almost indefinitely. The rest won't work for you. Load probably on MySQL? I have not seen anything similar to sharding or replication in B24.

Answer 3

the speed of disks everywhere at providers how many tested even close to not be compared with iron RAID

What drive did you test on AWS?
Try EBS Provisioned IOPS SSD ( https://aws.amazon.com/ebs/features/ ).

Answer 4

but idle CPU and memory

+

because in the raid your RAM is fast and the site starts to get stupid without it

With such symptoms, it is not necessary to build up iron, but to understand why the front clogs the database so much. It is necessary to turn on the profiler, see what requests come to the database and the plan for their execution.