== sanity test 398b: DIO and buffer IO race ============== 03:39:35 (1713512375)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.090402 s, 139 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=15457: Fri Apr 19 03:39:38 2024
   read: IOPS=341, BW=1365KiB/s (1398kB/s)(1480KiB/1084msec)
    clat (usec): min=3, max=22326, avg=1576.70, stdev=2265.95
     lat (usec): min=4, max=22327, avg=1577.93, stdev=2267.48
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[  404], 50.00th=[ 1205], 60.00th=[ 1680],
     | 70.00th=[ 2057], 80.00th=[ 2507], 90.00th=[ 3261], 95.00th=[ 5407],
     | 99.00th=[ 9372], 99.50th=[20841], 99.90th=[22414], 99.95th=[22414],
     | 99.99th=[22414]
   bw (  KiB/s): min=  872, max= 1680, per=23.46%, avg=1276.00, stdev=571.34, samples=2
   iops        : min=  218, max=  420, avg=319.00, stdev=142.84, samples=2
  write: IOPS=367, BW=1469KiB/s (1504kB/s)(1592KiB/1084msec)
    clat (usec): min=85, max=30802, avg=1218.31, stdev=2098.64
     lat (usec): min=86, max=30803, avg=1220.16, stdev=2098.28
    clat percentiles (usec):
     |  1.00th=[   94],  5.00th=[  105], 10.00th=[  115], 20.00th=[  143],
     | 30.00th=[  184], 40.00th=[  255], 50.00th=[  734], 60.00th=[  947],
     | 70.00th=[ 1237], 80.00th=[ 1860], 90.00th=[ 2999], 95.00th=[ 4015],
     | 99.00th=[ 8717], 99.50th=[10552], 99.90th=[30802], 99.95th=[30802],
     | 99.99th=[30802]
   bw (  KiB/s): min=  904, max= 1880, per=24.33%, avg=1392.00, stdev=690.14, samples=2
   iops        : min=  226, max=  470, avg=348.00, stdev=172.53, samples=2
  lat (usec)   : 4=0.26%, 10=11.85%, 20=5.86%, 50=0.39%, 100=1.95%
  lat (usec)   : 250=19.53%, 500=2.99%, 750=3.52%, 1000=7.42%
  lat (msec)   : 2=21.74%, 4=18.23%, 10=5.73%, 20=0.13%, 50=0.39%
  cpu          : usr=0.00%, sys=18.28%, ctx=957, majf=0, minf=36
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15458: Fri Apr 19 03:39:38 2024
   read: IOPS=326, BW=1304KiB/s (1336kB/s)(1436KiB/1101msec)
    clat (usec): min=3, max=40487, avg=1689.37, stdev=3029.04
     lat (usec): min=4, max=40488, avg=1690.34, stdev=3029.12
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   15], 40.00th=[  889], 50.00th=[ 1401], 60.00th=[ 1745],
     | 70.00th=[ 2147], 80.00th=[ 2474], 90.00th=[ 3195], 95.00th=[ 4293],
     | 99.00th=[ 9765], 99.50th=[30278], 99.90th=[40633], 99.95th=[40633],
     | 99.99th=[40633]
   bw (  KiB/s): min=  872, max= 1544, per=22.21%, avg=1208.00, stdev=475.18, samples=2
   iops        : min=  218, max=  386, avg=302.00, stdev=118.79, samples=2
  write: IOPS=371, BW=1486KiB/s (1522kB/s)(1636KiB/1101msec)
    clat (usec): min=89, max=13237, avg=1181.80, stdev=1619.62
     lat (usec): min=90, max=13238, avg=1182.43, stdev=1619.71
    clat percentiles (usec):
     |  1.00th=[   96],  5.00th=[  110], 10.00th=[  118], 20.00th=[  137],
     | 30.00th=[  167], 40.00th=[  237], 50.00th=[  742], 60.00th=[  930],
     | 70.00th=[ 1270], 80.00th=[ 1860], 90.00th=[ 2933], 95.00th=[ 4178],
     | 99.00th=[ 7898], 99.50th=[ 8586], 99.90th=[13173], 99.95th=[13173],
     | 99.99th=[13173]
   bw (  KiB/s): min= 1000, max= 1736, per=23.91%, avg=1368.00, stdev=520.43, samples=2
   iops        : min=  250, max=  434, avg=342.00, stdev=130.11, samples=2
  lat (usec)   : 4=0.26%, 10=10.81%, 20=4.43%, 50=0.91%, 100=1.43%
  lat (usec)   : 250=20.57%, 500=2.47%, 750=3.52%, 1000=8.33%
  lat (msec)   : 2=22.14%, 4=18.88%, 10=5.60%, 20=0.39%, 50=0.26%
  cpu          : usr=0.09%, sys=18.45%, ctx=1040, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15459: Fri Apr 19 03:39:38 2024
   read: IOPS=363, BW=1454KiB/s (1488kB/s)(1564KiB/1076msec)
    clat (usec): min=3, max=29310, avg=1679.61, stdev=2427.55
     lat (usec): min=3, max=29312, avg=1680.26, stdev=2427.66
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   14], 40.00th=[  955], 50.00th=[ 1369], 60.00th=[ 1762],
     | 70.00th=[ 2114], 80.00th=[ 2540], 90.00th=[ 3556], 95.00th=[ 4686],
     | 99.00th=[12518], 99.50th=[16188], 99.90th=[29230], 99.95th=[29230],
     | 99.99th=[29230]
   bw (  KiB/s): min=  888, max= 1824, per=24.94%, avg=1356.00, stdev=661.85, samples=2
   iops        : min=  222, max=  456, avg=339.00, stdev=165.46, samples=2
  write: IOPS=350, BW=1401KiB/s (1435kB/s)(1508KiB/1076msec)
    clat (usec): min=80, max=15178, avg=1089.65, stdev=1490.38
     lat (usec): min=80, max=15180, avg=1090.83, stdev=1490.12
    clat percentiles (usec):
     |  1.00th=[   95],  5.00th=[  108], 10.00th=[  116], 20.00th=[  135],
     | 30.00th=[  174], 40.00th=[  231], 50.00th=[  660], 60.00th=[  898],
     | 70.00th=[ 1221], 80.00th=[ 1860], 90.00th=[ 2540], 95.00th=[ 3589],
     | 99.00th=[ 6718], 99.50th=[ 9503], 99.90th=[15139], 99.95th=[15139],
     | 99.99th=[15139]
   bw (  KiB/s): min=  944, max= 1720, per=23.28%, avg=1332.00, stdev=548.71, samples=2
   iops        : min=  236, max=  430, avg=333.00, stdev=137.18, samples=2
  lat (usec)   : 4=0.26%, 10=10.94%, 20=5.99%, 50=0.13%, 100=0.91%
  lat (usec)   : 250=20.44%, 500=3.12%, 750=3.26%, 1000=6.77%
  lat (msec)   : 2=22.66%, 4=19.79%, 10=4.95%, 20=0.65%, 50=0.13%
  cpu          : usr=0.56%, sys=17.58%, ctx=969, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15460: Fri Apr 19 03:39:38 2024
   read: IOPS=346, BW=1387KiB/s (1421kB/s)(1508KiB/1087msec)
    clat (usec): min=3, max=36053, avg=1705.81, stdev=2928.69
     lat (usec): min=4, max=36054, avg=1707.83, stdev=2929.77
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   13], 40.00th=[  709], 50.00th=[ 1352], 60.00th=[ 1893],
     | 70.00th=[ 2212], 80.00th=[ 2638], 90.00th=[ 3523], 95.00th=[ 4424],
     | 99.00th=[10683], 99.50th=[29230], 99.90th=[35914], 99.95th=[35914],
     | 99.99th=[35914]
   bw (  KiB/s): min= 1040, max= 1520, per=23.54%, avg=1280.00, stdev=339.41, samples=2
   iops        : min=  260, max=  380, avg=320.00, stdev=84.85, samples=2
  write: IOPS=359, BW=1439KiB/s (1473kB/s)(1564KiB/1087msec)
    clat (usec): min=87, max=7519, avg=1110.60, stdev=1245.03
     lat (usec): min=87, max=7520, avg=1112.22, stdev=1245.23
    clat percentiles (usec):
     |  1.00th=[   94],  5.00th=[  105], 10.00th=[  113], 20.00th=[  141],
     | 30.00th=[  186], 40.00th=[  338], 50.00th=[  775], 60.00th=[  996],
     | 70.00th=[ 1270], 80.00th=[ 1958], 90.00th=[ 2671], 95.00th=[ 3785],
     | 99.00th=[ 6390], 99.50th=[ 6521], 99.90th=[ 7504], 99.95th=[ 7504],
     | 99.99th=[ 7504]
   bw (  KiB/s): min= 1048, max= 1624, per=23.35%, avg=1336.00, stdev=407.29, samples=2
   iops        : min=  262, max=  406, avg=334.00, stdev=101.82, samples=2
  lat (usec)   : 4=0.52%, 10=9.90%, 20=7.29%, 50=0.78%, 100=1.04%
  lat (usec)   : 250=18.49%, 500=3.12%, 750=3.91%, 1000=7.29%
  lat (msec)   : 2=20.05%, 4=22.53%, 10=4.56%, 20=0.26%, 50=0.26%
  cpu          : usr=0.64%, sys=18.69%, ctx=1012, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5439KiB/s (5569kB/s), 1304KiB/s-1454KiB/s (1336kB/s-1488kB/s), io=5988KiB (6132kB), run=1076-1101msec
  WRITE: bw=5722KiB/s (5859kB/s), 1401KiB/s-1486KiB/s (1435kB/s-1522kB/s), io=6300KiB (6451kB), run=1076-1101msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=15453: Fri Apr 19 03:39:38 2024
   read: IOPS=219, BW=877KiB/s (898kB/s)(1480KiB/1687msec)
    clat (usec): min=551, max=24578, avg=1598.51, stdev=2129.80
     lat (usec): min=552, max=24580, avg=1599.15, stdev=2129.86
    clat percentiles (usec):
     |  1.00th=[  594],  5.00th=[  652], 10.00th=[  685], 20.00th=[  742],
     | 30.00th=[  791], 40.00th=[  840], 50.00th=[  930], 60.00th=[ 1037],
     | 70.00th=[ 1172], 80.00th=[ 1795], 90.00th=[ 3458], 95.00th=[ 4752],
     | 99.00th=[12387], 99.50th=[17695], 99.90th=[24511], 99.95th=[24511],
     | 99.99th=[24511]
   bw (  KiB/s): min=  496, max= 1168, per=22.77%, avg=808.00, stdev=338.56, samples=3
   iops        : min=  124, max=  292, avg=202.00, stdev=84.64, samples=3
  write: IOPS=235, BW=944KiB/s (966kB/s)(1592KiB/1687msec)
    clat (usec): min=1004, max=15906, avg=2704.84, stdev=1632.48
     lat (usec): min=1005, max=15907, avg=2705.65, stdev=1632.58
    clat percentiles (usec):
     |  1.00th=[ 1139],  5.00th=[ 1287], 10.00th=[ 1450], 20.00th=[ 1713],
     | 30.00th=[ 1860], 40.00th=[ 1975], 50.00th=[ 2147], 60.00th=[ 2376],
     | 70.00th=[ 2769], 80.00th=[ 3294], 90.00th=[ 4883], 95.00th=[ 6063],
     | 99.00th=[ 9896], 99.50th=[10683], 99.90th=[15926], 99.95th=[15926],
     | 99.99th=[15926]
   bw (  KiB/s): min=  528, max= 1376, per=23.71%, avg=885.33, stdev=439.44, samples=3
   iops        : min=  132, max=  344, avg=221.33, stdev=109.86, samples=3
  lat (usec)   : 750=10.81%, 1000=16.54%
  lat (msec)   : 2=34.64%, 4=27.73%, 10=9.37%, 20=0.78%, 50=0.13%
  cpu          : usr=0.77%, sys=6.17%, ctx=827, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15454: Fri Apr 19 03:39:38 2024
   read: IOPS=214, BW=859KiB/s (880kB/s)(1436KiB/1671msec)
    clat (usec): min=466, max=25942, avg=1409.40, stdev=1698.13
     lat (usec): min=467, max=25944, avg=1410.01, stdev=1698.18
    clat percentiles (usec):
     |  1.00th=[  529],  5.00th=[  611], 10.00th=[  668], 20.00th=[  725],
     | 30.00th=[  783], 40.00th=[  832], 50.00th=[  914], 60.00th=[  996],
     | 70.00th=[ 1221], 80.00th=[ 1631], 90.00th=[ 2671], 95.00th=[ 4113],
     | 99.00th=[ 5669], 99.50th=[ 7308], 99.90th=[25822], 99.95th=[25822],
     | 99.99th=[25822]
   bw (  KiB/s): min=  464, max= 1296, per=22.77%, avg=808.00, stdev=434.29, samples=3
   iops        : min=  116, max=  324, avg=202.00, stdev=108.57, samples=3
  write: IOPS=244, BW=979KiB/s (1003kB/s)(1636KiB/1671msec)
    clat (usec): min=1086, max=27888, avg=2823.75, stdev=2331.20
     lat (usec): min=1086, max=27889, avg=2824.69, stdev=2331.20
    clat percentiles (usec):
     |  1.00th=[ 1156],  5.00th=[ 1369], 10.00th=[ 1483], 20.00th=[ 1729],
     | 30.00th=[ 1844], 40.00th=[ 1958], 50.00th=[ 2114], 60.00th=[ 2343],
     | 70.00th=[ 2769], 80.00th=[ 3392], 90.00th=[ 4686], 95.00th=[ 6063],
     | 99.00th=[13173], 99.50th=[17171], 99.90th=[27919], 99.95th=[27919],
     | 99.99th=[27919]
   bw (  KiB/s): min=  552, max= 1336, per=24.50%, avg=914.67, stdev=395.28, samples=3
   iops        : min=  138, max=  334, avg=228.67, stdev=98.82, samples=3
  lat (usec)   : 500=0.13%, 750=10.94%, 1000=17.06%
  lat (msec)   : 2=34.11%, 4=27.86%, 10=8.98%, 20=0.65%, 50=0.26%
  cpu          : usr=0.60%, sys=6.17%, ctx=851, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15455: Fri Apr 19 03:39:38 2024
   read: IOPS=234, BW=939KiB/s (961kB/s)(1564KiB/1666msec)
    clat (usec): min=553, max=22423, avg=1536.01, stdev=1668.47
     lat (usec): min=554, max=22424, avg=1536.63, stdev=1668.48
    clat percentiles (usec):
     |  1.00th=[  562],  5.00th=[  644], 10.00th=[  701], 20.00th=[  766],
     | 30.00th=[  824], 40.00th=[  873], 50.00th=[  930], 60.00th=[ 1029],
     | 70.00th=[ 1270], 80.00th=[ 1893], 90.00th=[ 3359], 95.00th=[ 4293],
     | 99.00th=[ 7963], 99.50th=[ 8094], 99.90th=[22414], 99.95th=[22414],
     | 99.99th=[22414]
   bw (  KiB/s): min=  464, max= 1296, per=24.27%, avg=861.33, stdev=417.25, samples=3
   iops        : min=  116, max=  324, avg=215.33, stdev=104.31, samples=3
  write: IOPS=226, BW=905KiB/s (927kB/s)(1508KiB/1666msec)
    clat (usec): min=1071, max=22002, avg=2775.37, stdev=2016.44
     lat (usec): min=1071, max=22004, avg=2776.37, stdev=2016.47
    clat percentiles (usec):
     |  1.00th=[ 1156],  5.00th=[ 1319], 10.00th=[ 1532], 20.00th=[ 1729],
     | 30.00th=[ 1844], 40.00th=[ 1975], 50.00th=[ 2114], 60.00th=[ 2343],
     | 70.00th=[ 2769], 80.00th=[ 3556], 90.00th=[ 4555], 95.00th=[ 5800],
     | 99.00th=[12649], 99.50th=[19006], 99.90th=[21890], 99.95th=[21890],
     | 99.99th=[21890]
   bw (  KiB/s): min=  552, max= 1288, per=22.85%, avg=853.33, stdev=385.69, samples=3
   iops        : min=  138, max=  322, avg=213.33, stdev=96.42, samples=3
  lat (usec)   : 750=9.24%, 1000=19.40%
  lat (msec)   : 2=33.07%, 4=27.47%, 10=10.16%, 20=0.39%, 50=0.26%
  cpu          : usr=0.42%, sys=6.37%, ctx=832, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15456: Fri Apr 19 03:39:38 2024
   read: IOPS=225, BW=901KiB/s (923kB/s)(1508KiB/1673msec)
    clat (usec): min=497, max=27572, avg=1466.49, stdev=1792.09
     lat (usec): min=498, max=27572, avg=1467.11, stdev=1792.10
    clat percentiles (usec):
     |  1.00th=[  529],  5.00th=[  611], 10.00th=[  660], 20.00th=[  725],
     | 30.00th=[  783], 40.00th=[  857], 50.00th=[  930], 60.00th=[ 1004],
     | 70.00th=[ 1172], 80.00th=[ 1680], 90.00th=[ 3032], 95.00th=[ 4293],
     | 99.00th=[ 6325], 99.50th=[ 7111], 99.90th=[27657], 99.95th=[27657],
     | 99.99th=[27657]
   bw (  KiB/s): min=  471, max= 1224, per=23.58%, avg=837.00, stdev=376.94, samples=3
   iops        : min=  117, max=  306, avg=209.00, stdev=94.60, samples=3
  write: IOPS=233, BW=935KiB/s (957kB/s)(1564KiB/1673msec)
    clat (usec): min=1099, max=34946, avg=2821.03, stdev=2730.76
     lat (usec): min=1099, max=34947, avg=2821.94, stdev=2730.83
    clat percentiles (usec):
     |  1.00th=[ 1123],  5.00th=[ 1287], 10.00th=[ 1483], 20.00th=[ 1713],
     | 30.00th=[ 1827], 40.00th=[ 1942], 50.00th=[ 2089], 60.00th=[ 2278],
     | 70.00th=[ 2606], 80.00th=[ 3556], 90.00th=[ 4555], 95.00th=[ 5800],
     | 99.00th=[10028], 99.50th=[33162], 99.90th=[34866], 99.95th=[34866],
     | 99.99th=[34866]
   bw (  KiB/s): min=  510, max= 1368, per=23.62%, avg=882.00, stdev=440.21, samples=3
   iops        : min=  127, max=  342, avg=220.33, stdev=110.26, samples=3
  lat (usec)   : 500=0.13%, 750=12.11%, 1000=17.32%
  lat (msec)   : 2=32.94%, 4=26.69%, 10=10.16%, 20=0.26%, 50=0.39%
  cpu          : usr=0.60%, sys=6.04%, ctx=830, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3549KiB/s (3635kB/s), 859KiB/s-939KiB/s (880kB/s-961kB/s), io=5988KiB (6132kB), run=1666-1687msec
  WRITE: bw=3734KiB/s (3824kB/s), 905KiB/s-979KiB/s (927kB/s-1003kB/s), io=6300KiB (6451kB), run=1666-1687msec