Speed ​​limited to 1Gb on Esxi on 10Gb cards, why?

Question

Computer networks

Good day =)
The essence of the problem. There are 2 servers
1) HP DL360 Gen9 ( Installed in LOM 10Gb SPF + HP520 ) // ESXi 6.7 HP custom
2) HP DL380p Gen8 ( Installed PCI-E HP NC552SFP ) // ESXi 6.5 HP custom
Switch Mikrotik CRS 305 , this case is connected like this DAC

on disks:
On the first one there is RAID 1+0 on 6 SAS 300gb 15k disks
On the second one, 3 arrays, also RAID 1+0 on 600gb 10k, different configurations from 6 to 10 disks.

Everything seems to be beautiful, but here's the problem. Speed ​​no more than 1Gbit on cards.
Checked through iperf , through vmotion and map monitoring on Esxi and tick .. No more than 1Gbit.

Now according to the
settings Switch settings on hypervisors



Network cards on hosts (same on the second one)


iPerf statistics, top run on 1Gbit card, second run on 10Gbit, third run on 10Gbit In 2 streams.


by tick. Everything seems to be simple here too. 2 ports, they are in their bridge, 10gbit links I



don't know where to dig ... I tried to set MTU = 9000 (there are manuals on the network), it does not help.

Everything was done for vMotion ... But even as it is not joyful.

Answer 1

For some reason, the Hardware offload on the bridge port sfp-sfpplus1 was not activated, so the data was switched by the switch processor, and not by the hardware switch chip. As far as I remember, H.W. offload in Mikrotik switches only works for the first created bridge, and you have SFP + ports in bidge2, this may be the reason. If ether1 is used only for management, try removing it from the bridge and using it independently, and transfer the SFP + ports to bridge1.

Answer 2

There may be several problems, here are just some of the options:

• Bandwidth vswitch
  
• Network card drivers installed incorrectly
  
• Copper cable with pickups or even damaged
  

In such cases, you need to check in UDP mode, setting the speed to 10G in the key, and watch, including packet drop.
Set the speed hard to 10GE, disable auto-detection.
We need to localize the problem.

1. Exclude Mikrotik - connect two servers directly and check the speed between them.
   
2. Exclude vmware - load servers with pure Linux, drive speed between them.
   
3. Exclude DAC - connect with two regular SFPs and an optical cord.
   

PS In your case, it looks like your Mikrotik does not pass more gigabit (see Ethernet Test). The fact that the ports are 10GE is not a fact that the device will miss these 10GE.