Connecting nodes in a cluster. Physical connection, how and with what?

Question

Computer networks

I study server clustering, purely for self-development. There was a question. What is the best way to connect nodes in a cluster and with a common network? Ethernet is slow and inexpensive, infiniband is fast and expensive. What other possibilities are there for connecting nodes?

Answer 1

Well, it determines the bandwidth you need. 10gb Ethernet is now relatively cheap, almost at the price of 1gb, 40gb is already more expensive. Here people write that 56gb Infiniband came out cheaper than Ethernet.
Yes, and Mellanox's latest generation cards are hybrid with Ethernet and Infiniband at the same time.
So I have a small cluster (mariadb galera / glusterfs) of 4 servers that lives on 1gb without any problems. Now we are migrating to 10Gbit due to virtualization, 1Gbit becomes a very bottleneck when migrating a machine to another server

Answer 2

ethernet and jumboframes are very fast, but
the question here is more what speeds do you need, few people need to drive a stream from each 100 gigabit server