What type of link aggregation to use for maximum throughput?

Question

linux

Salute comrades!
We have 3 servers with 2x1G LAN and ciscome 3400
Gluster replika 3 will be deployed on the servers, which will be accessed by the same servers (storage for VMs)
Share your experience: which type of aggregation is best suited for maximum throughput in this case (balance-rr or LACP) and does it make sense to use sharding on glusterfs (Disks with VMs will be from 500 to 100 GB)

Answer 1

It is unlikely that balancing will help you in such a configuration. Extra configuration and maintenance, but there will be no profit. Balancing helps a lot when you have many clients connecting - then their total traffic is spread over several physical channels. But still, not a single client will receive a speed greater than the speed of one interface, i.e. 1G in your case.
The reverse situation is also possible - there are few clients, but they go to a bunch of addresses (a typical option is office Internet). Then LAG can be disposed of TOTALLY more completely. But again, each client will not get more than the speed of one interface. Since the packets of one connection are not spread over several interfaces.
In such a situation, I would probably configure it statically so that for each server both partners are available through a dedicated interface. That is:
- server1 -- connects to server2 through NIC1, and to server3 -- through NIC2
- server2 -- connects to server 1 through its NIC1, to server3 -- through its NIC2
- server3 -- connects to server 1 through own NIC1, to server2 -- via NIC2
There will be less overhead for hash calculation in balancing.

Answer 2

Oh.
1) Leave the glaster. Use CEPH. Don't make our mistakes, it's not extensible in practice, and there were problems with performance. Moreover, they will use it as an image repository, here CEPH is an order of magnitude better, it immediately gives a block device.
2) Aggregation - LACP, firstly it is a standard, secondly, it is supported on switches.
3) And even better - two networks, one for clients, the second for glaster / ceph, and do not be stingy, put 10Gb or Infiniband.