How to get a connection between Linux servers 2 Gbps through switches?

J

juffinhalli2015-05-18 11:47:02

Computer networks

juffinhalli, 2015-05-18 11:47:02

auto bond0
iface bond0 inet static
        address  192.168.88.xx
        netmask  255.255.255.0
        slaves eth0 eth1
        bond_miimon 100
        bond_mode 802.3ad
        bond_xmit_hash_policy layer3+4

Task:
to get a connection between Linux servers 2 Gbit / s (multi-threaded) through the switches
My actions:
To check, I connect the servers directly to each other and run the test in 2 threads:

[email protected]:~# iperf -c 192.168.88.1 -P 2
------------------------------------------------------------
Client connecting to 192.168.88.1, TCP port 5001
TCP window size: 19.6 KByte (default)
------------------------------------------------------------
[  4] local 192.168.88.2 port 41924 connected with 192.168.88.1 port 5001
[  3] local 192.168.88.2 port 41923 connected with 192.168.88.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  1.10 GBytes   941 Mbits/sec
[  3]  0.0-10.0 sec  1.10 GBytes   941 Mbits/sec
[SUM]  0.0-10.0 sec  2.19 GBytes  1.88 Gbits/sec
[email protected]:~#

Disconnecting any one cable does not interrupt the connection, only the speed drops.
Changing bond_xmit_hash_policy to layer2 or layer2+3 does not increase speed, only failover.
Next, I turn on LACP on the switches, throw cables and check

[email protected]:~# iperf -c 192.168.88.1 -P 2
------------------------------------------------------------
Client connecting to 192.168.88.1, TCP port 5001
TCP window size: 19.6 KByte (default)
------------------------------------------------------------
[  4] local 192.168.88.2 port 41926 connected with 192.168.88.1 port 5001
[  3] local 192.168.88.2 port 41925 connected with 192.168.88.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 9.6 sec   536 MBytes   468 Mbits/sec
[  4]  0.0-10.0 sec   587 MBytes   492 Mbits/sec
[SUM]  0.0-10.0 sec  1.10 GBytes   941 Mbits/sec

I check again: disconnecting any one cable does not interrupt the connection, the speed is the same 941 Mbits / sec.
It turns out that LACP works on the switches, but only half - only fault tolerance is provided.
There is no possibility to change hash_policy on switches.
Questions:
1. Did Netgear deceive me by declaring full support for IEEE 802.3ad (because only fault tolerance is provided)?
2. In linux, in the kernel module, the IEEE 802.3ad standard is not fully implemented (since an increase in speed is only possible with a direct connection)?
3) How to still get a 2 Gbit / s connection between Linux servers (multi-threaded) through the switches.
PS Ready
to order other PPS Cisco switches I can't afford.
PPPS By multi-stream I mean data transfer in two streams in the same direction

Reply

Answer the question

In order to leave comments, you need to log in

1 answer(s)

A

Andrew, 2015-05-18
@juffinhalli

If you need to drive connections only between 2 servers via the resulting link, then the balance-rr mode will help (bond_xmit_hash_policy can do it) - it divides traffic not by addresses/macs/etc, but sequentially sends each next packet here and there.
Fundamental question: what is the port standard in network cards and switches? 1000Base-T or 1000Base-TX ?
The fact is that the TX standard gives gigabit in each direction, one link, when T means gigabit in the sum of the incoming / outgoing directions. Setevukha can be TX, but I saw only a few switches that support TX, and they were expensive.
So the joke is that the T standard is a bottleneck, and if the switches do not support TX (and according to tests, I assume that this is the case), then there will never be 2 gigabits there and 2 gigabits back at the same time.
So here are the answers:
1. Perhaps not, you must first look at the type of ports and make sure that the "plug" can be exactly in this
2. Completely. As a rule, if there are many connections, then layer2+3, layer3+4, etc. - they work great and try to balance traffic as smoothly as possible, however, with one computer (and especially on one connection that generates iperf!) There is very little sense in them, so only round-robin will normally balance this type of traffic.
UPD:
I looked, in this model, the ports are "IEEE 802.3ab 1000BASE-T Gigabit Ethernet", i.e. for each gigabit will be in the sum of directions. In this case, I recommend temporarily forgetting about the full 2 gigabits and checking iperf traffic not at the same time, but first in one direction, then in the other direction, and at the same time monitor the uniformity of the links loading (from the point of view of the switch), by turning it on from the servers round robin. If the traffic on both channels goes evenly, then the problem is only in the port type. So you need to look for a switch with support for 1000BASE-TX