What and how to correctly measure the performance of storage systems?

Question

RAID

Colleagues, good day.
We bought a new storage system for 12 disks, attended to performance tests in different raids and a different number of disks.
I hooked it up via ISCSI to the ESXi host, made a virtual machine with a disk on the storage system and started testing it.
I use the same methods as for regular HDD/SSD, i.e., IoMeter, PCMARK (Stor. test), Passmark, Crystal Disk mark, HD Tune.
Tried scripts from MS.
The meter settings are as follows: 20/80 write / read, 8KB block, queue max 1 IO (Burst Length), random access.
The results are strange, the number of IOPS almost does not change between raid 10 on 12 heads and on 4 heads. Exactly, as well as between 10 for 4 heads and 5 for 12 heads. All the time I see a figure of about 200 IOPs, plus or minus.
Please tell me how and with what it is correct to measure iops in my case?

Answer 1

IO depth = 1 is about nothing, it doesn't happen in real life. Even one consumer will generate IO operations in several threads. 4 workers in IOmeter with IO depth = 8 - this is closer to the point, and then you will see different performance with different types of RAIDs.

Answer 2

Colleagues, thanks for the answers.
I ran the test a little differently, using Crystal Disk Mark.
There's a bigger queue and more threads.
But, anyway, the difference is not what I expected to see:
For example, I can't understand, the difference in writing random 4KB in 8 threads with queue 8 between 4 and 12 disks in a raid is only 30%. Is it a measurement (method) error or is the number of heads not directly proportional to performance?
The first column shows the number of disks, speed, the number of Logical Volume, the number of Logical Drive and the type of raid.
Next, by column (Calculation in MB/s):
SEQ Q32R1 sequential, 1 thread, 32
4KB deep Q8T8 Random, 4KB block, 8 threads, 8
4KB deep Q32T1 Random, 4KB block, 1 thread, 32 deep
4KB Q1T1 Random, 4KB block, 1 thread, depth 1
I'm guessing it's about queue depth.

Answer 3

Negative top margin for the image, only red block overflow to be visible