How do you test 10gb links?

Question

linux

I actually try iperf on ubuntu 11.04 x32

iperf -t 100 -c 192.168.20.2
- Client connecting to 192.168.20.2, TCP port 5001
TCP window size: 16.0 KByte (default)
- [ 3] local 192.168.20.5 port 57490 connected with 192.168. 20.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-100.0 sec 67.3 GBytes 5.78 Gbits/sec



CPU load is not big

top — 20:22:33 up 4:17, 2 users, load average: 0.00, 0.01, 0.05
Tasks: 104 total, 1 running, 103 sleeping, 0 stopped, 0 zombie
Cpu0: 0.5%us, 13.0%
sy , 0.0%ni, 77.3%id, 0.0%wa, 0.0%hi, 9.2%si, 0.0%st %hi, 4.9%si, 0.0%st
Cpu2: 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3: 0.5%us, 0.0 %sy, 0.0%ni, 99.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 8214952k
total, 3356800k used, 4858152k free, 26140k buffers
3219448k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2226 root 20 0 19740 852 720 S 46 0.0 0:12.53 iperf
1 root 20 0 2904 1752 1252 S 0 0.0 0 :00.94 init 2 root 20
0 0 0 0 S 0 0.0 0:00.00 0 S 0 0.0 0:00.43 kworker/u:0 On the receiving side freenas 0.8 iperf -s 192.168.20.2 top last pid: 4432; load averages: 1.22, 0.38, 0.15 up 0+02:09:18 20:23:23 48 processes: 1 running, 47 sleeping CPU: 1.6% user, 0.0% nice, 44.1% system, 25.0% interrupt, 29.3% idle Mem: 60M Active, 30M Inact, 284M Wired, 3404K Cache, 116M Buf, 7495M Free Swap: 38G Total, 38G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 4103 root 4 76 0 15760K 2604K sbwait 1 1:09 94.19% iperf





2326 root 7 44 0 68112K 10176K ucond 1 0:01 0.00% collectd 2240
root 6 76 0 87696K 38200K uwait 3 0:00 0.00 % 44 0 11780K 2772K select 0 0:00 0.00% ntpd 1948 root 1 44 0 36916K 6048K select 1 0:00 0.00% nmbd On both machines, the percentage does not seem to rest, and the speed does not show more than 5.8 Gigabit. Networks from this article habrahabr.ru/blogs/hardware/120393/ What could be the problem?

Answer 1

I once suffered for a very long time with speed on an optical card. Did not rise above 2.8 gigabits in any way. And then I suddenly discovered that my mtu is 1500. After increasing it to 6000, everything took off.

Answer 2

try iperf with -P5 switch

Answer 3

>>Need performance without jumboframes Why
do you need 10G?

Answer 4

> Cpu0: 0.5%us, 13.0%sy, 0.0%ni, 77.3%id, 0.0%wa, 0.0%hi, 9.2%si, 0.0%st
> Cpu1: 0.0%us, 7.8%sy, 0.0%ni, 87.4 %id, 0.0%wa, 0.0%hi, 4.9%si, 0.0%st
> Cpu2: 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0 %st
> Cpu3: 0.5%us, 0.0%sy, 0.0%ni, 99.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
check irqbalance

Answer 5

I see no reason to even try without jumbo-frames.
Don't forget about MTU.
There is no place to test it now, but iperf showed something around 9.3Gbps

Answer 6

First, look at lspci -vvvv, what is the TPS on the piece of iron. Many 10G network cards come with such a skinny PCIE that a real 10G cannot, in principle, pull, and 10G in the environment can only be seen from them on special synthetics with a large offload.