ZFS raidz limits rebuild speed?

Question

ZFS

There are freenas with raidz(1) of 9 disks 4tb + 1 under hotspare .
Used for cifs balls.
For the test, I threw 5 TB of data on the volume and decided to check how long the rebuild would last.
I pulled 1 disk and realized that the rebuilding time would be incredibly huge.
over the past 5 hours, only about 2% have counted.
At the same time, the read speed from disks is around 1 MB, and the replacement recording is the same.

and now it has sunk.
scan: resilver in progress since Mon Nov 30 17:11:04 2015
189G scanned out of 5.74T at 8.87M/s, 182h38m to go
21.0G resilvered, 3.22% done
No one is accessing the repository at the moment.
What could justify such behavior?
Maybe in the zfs settings there is some kind of limiter that affects the speed of reading from disks when rebuilding the array?

Answer 1

Googled it myself

# ZFS tuning
#         Mostly taken from:  http://broken.net/uncategorized/zfs-performance-tuning-for-scrubs-and-resilvers/
vfs.zfs.l2arc_write_boost=160000000             # Set the L2ARC warmup writes to 160 MBps
vfs.zfs.l2arc_write_max=320000000               # Set the L2ARC writes to 320 MBps
vfs.zfs.resilver_delay=0                        # Prioritise resilver over normal writes (default 2)
vfs.zfs.scrub_delay=0                           # Prioritise scrub    over normal writes (default 4)
vfs.zfs.top_maxinflight=128                     # Up the number of in-flight I/O (default 32)
vfs.zfs.resilver_min_time_ms=5000               # Up the length of time a resilver process takes in each TXG (default 3000)
vfs.zfs.vdev.max_pending=24                     # Set the queue depth (number of I/O) for each vdev
  #-->                                          # Set this really high.  Then monitor the L(q) column in gstat under load.
  #-->                                          # Set it to just slightly lower than the highest number you see.

add to sysctl.conf
# service sysctl restart
The speed increased by 10 times, and everything rested on the CPU.

scan: resilver in progress since Mon Nov 30 17:11:04 2015
        5.36T scanned out of 5.74T at 101M/s, 1h7m to go
        609G resilvered, 93.25% done