Why is the LSI-9260-8i slow on Linux?

Question

linux

There is the above controller. Assembled RAID-10 of 6 disks. On Windows, it gives out about 2500 IOPS (tested with fio, one thread randomly writes, the second randomly reads). We loaded it into linux, and there it’s not even possible to get close to such numbers.
Here is the output of fio from linux:

readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.0.14
Starting 2 processes
^Cbs: 2 (f=2): [rw] [0.0% done] [700K/564K/0K /s] [175 /141 /0  iops] [eta 12d:03h:43m:38s]
fio: terminating on signal 2
Jobs: 2 (f=2): [rw] [0.0% done] [580K/488K/0K /s] [145 /122 /0  iops] [eta 12d:04h:41m:14s]
readtest: (groupid=0, jobs=1): err= 0: pid=12357: Thu Apr  3 18:02:40 2014
  read : io=24256KB, bw=653721 B/s, iops=159 , runt= 37995msec
    slat (usec): min=5 , max=35 , avg=12.02, stdev= 1.39
    clat (msec): min=3 , max=1575 , avg=200.44, stdev=147.75
     lat (msec): min=3 , max=1575 , avg=200.45, stdev=147.75
    clat percentiles (msec):
     |  1.00th=[   12],  5.00th=[   25], 10.00th=[   38], 20.00th=[   70],
     | 30.00th=[  143], 40.00th=[  176], 50.00th=[  196], 60.00th=[  215],
     | 70.00th=[  231], 80.00th=[  251], 90.00th=[  351], 95.00th=[  461],
     | 99.00th=[  824], 99.50th=[  914], 99.90th=[ 1156], 99.95th=[ 1287],
     | 99.99th=[ 1582]
    bw (KB/s)  : min=  517, max=  814, per=100.00%, avg=640.89, stdev=84.32
    lat (msec) : 4=0.02%, 10=0.74%, 20=2.47%, 50=11.20%, 100=10.77%
    lat (msec) : 250=54.34%, 500=16.46%, 750=2.85%, 1000=0.84%, 2000=0.31%
  cpu          : usr=0.13%, sys=0.26%, ctx=6064, majf=0, minf=57
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.3%, 32=99.5%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=6064/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=12358: Thu Apr  3 18:02:40 2014
  write: io=21240KB, bw=571775 B/s, iops=139 , runt= 38039msec
    slat (usec): min=5 , max=110 , avg=13.73, stdev= 2.02
    clat (msec): min=5 , max=1319 , avg=229.15, stdev=170.27
     lat (msec): min=5 , max=1319 , avg=229.17, stdev=170.27
    clat percentiles (msec):
     |  1.00th=[   15],  5.00th=[   30], 10.00th=[   45], 20.00th=[   82],
     | 30.00th=[  159], 40.00th=[  186], 50.00th=[  206], 60.00th=[  227],
     | 70.00th=[  249], 80.00th=[  289], 90.00th=[  437], 95.00th=[  594],
     | 99.00th=[  857], 99.50th=[  930], 99.90th=[ 1106], 99.95th=[ 1156],
     | 99.99th=[ 1319]
    bw (KB/s)  : min=  415, max=  717, per=100.00%, avg=562.44, stdev=77.41
    lat (msec) : 10=0.30%, 20=2.11%, 50=9.62%, 100=10.43%, 250=48.12%
    lat (msec) : 500=22.60%, 750=4.73%, 1000=1.69%, 2000=0.40%
  cpu          : usr=0.08%, sys=0.26%, ctx=5311, majf=0, minf=25
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.3%, 32=99.4%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=5310/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=24256KB, aggrb=638KB/s, minb=638KB/s, maxb=638KB/s, mint=37995msec, maxt=37995msec
  WRITE: io=21240KB, aggrb=558KB/s, minb=558KB/s, maxb=558KB/s, mint=38039msec, maxt=38039msec

Disk stats (read/write):
  sdc: ios=6034/5295, merge=0/0, ticks=1204946/1207353, in_queue=2423520, util=99.77%

I really want to understand why.

Answer 1

I advise you to read, perhaps a mistake in testing.
habrahabr.ru/post/154235
Also, 2500 IOPS seems like a strange figure, a lot for disks, not enough for SSDs, apparently this is a cache. In my humble opinion it should be around 700-900 for 6 sata 7200 drives.
I advise you to set io as a deadline scheduler, it should increase performance a little.