== sanity test 398b: DIO and buffer IO race ============== 05:54:27 (1713347667)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0759879 s, 166 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=20986: Wed Apr 17 05:54:31 2024
   read: IOPS=198, BW=794KiB/s (813kB/s)(1480KiB/1865msec)
    clat (usec): min=3, max=28744, avg=1422.99, stdev=3391.71
     lat (usec): min=3, max=28744, avg=1423.63, stdev=3391.69
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   13], 50.00th=[   23], 60.00th=[  734],
     | 70.00th=[ 1483], 80.00th=[ 2114], 90.00th=[ 3064], 95.00th=[ 5014],
     | 99.00th=[22414], 99.50th=[26608], 99.90th=[28705], 99.95th=[28705],
     | 99.99th=[28705]
   bw (  KiB/s): min=  688, max=  984, per=26.09%, avg=808.00, stdev=155.74, samples=3
   iops        : min=  172, max=  246, avg=202.00, stdev=38.94, samples=3
  write: IOPS=213, BW=854KiB/s (874kB/s)(1592KiB/1865msec)
    clat (usec): min=91, max=32701, avg=3316.95, stdev=5124.01
     lat (usec): min=92, max=32702, avg=3317.57, stdev=5124.01
    clat percentiles (usec):
     |  1.00th=[   97],  5.00th=[  111], 10.00th=[  120], 20.00th=[  147],
     | 30.00th=[  196], 40.00th=[  668], 50.00th=[ 1205], 60.00th=[ 1762],
     | 70.00th=[ 2474], 80.00th=[ 5997], 90.00th=[10290], 95.00th=[14615],
     | 99.00th=[22152], 99.50th=[25297], 99.90th=[32637], 99.95th=[32637],
     | 99.99th=[32637]
   bw (  KiB/s): min=  712, max= 1088, per=27.41%, avg=893.33, stdev=188.35, samples=3
   iops        : min=  178, max=  272, avg=223.33, stdev=47.09, samples=3
  lat (usec)   : 4=0.52%, 10=13.28%, 20=8.59%, 50=4.30%, 100=1.82%
  lat (usec)   : 250=17.71%, 500=1.69%, 750=2.34%, 1000=4.30%
  lat (msec)   : 2=15.23%, 4=15.23%, 10=8.20%, 20=5.08%, 50=1.69%
  cpu          : usr=0.27%, sys=8.42%, ctx=839, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20987: Wed Apr 17 05:54:31 2024
   read: IOPS=198, BW=792KiB/s (811kB/s)(1436KiB/1813msec)
    clat (usec): min=3, max=37875, avg=1382.72, stdev=3807.97
     lat (usec): min=4, max=37876, avg=1383.25, stdev=3808.01
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    6], 10.00th=[    8], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   15], 50.00th=[   25], 60.00th=[  963],
     | 70.00th=[ 1483], 80.00th=[ 2024], 90.00th=[ 2933], 95.00th=[ 4490],
     | 99.00th=[23462], 99.50th=[35390], 99.90th=[38011], 99.95th=[38011],
     | 99.99th=[38011]
   bw (  KiB/s): min=  752, max= 1008, per=27.10%, avg=839.33, stdev=146.10, samples=3
   iops        : min=  188, max=  252, avg=209.67, stdev=36.67, samples=3
  write: IOPS=225, BW=902KiB/s (924kB/s)(1636KiB/1813msec)
    clat (usec): min=97, max=30067, avg=3181.72, stdev=5358.43
     lat (usec): min=98, max=30068, avg=3182.34, stdev=5358.44
    clat percentiles (usec):
     |  1.00th=[  101],  5.00th=[  114], 10.00th=[  120], 20.00th=[  145],
     | 30.00th=[  196], 40.00th=[  416], 50.00th=[ 1020], 60.00th=[ 1549],
     | 70.00th=[ 2245], 80.00th=[ 3818], 90.00th=[10945], 95.00th=[15139],
     | 99.00th=[23462], 99.50th=[27919], 99.90th=[30016], 99.95th=[30016],
     | 99.99th=[30016]
   bw (  KiB/s): min=  814, max= 1136, per=28.95%, avg=943.33, stdev=170.09, samples=3
   iops        : min=  203, max=  284, avg=235.67, stdev=42.71, samples=3
  lat (usec)   : 4=0.26%, 10=13.02%, 20=8.59%, 50=3.52%, 100=0.78%
  lat (usec)   : 250=19.53%, 500=2.21%, 750=1.56%, 1000=5.86%
  lat (msec)   : 2=17.58%, 4=14.58%, 10=5.60%, 20=4.95%, 50=1.95%
  cpu          : usr=0.28%, sys=10.04%, ctx=831, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20988: Wed Apr 17 05:54:31 2024
   read: IOPS=202, BW=809KiB/s (829kB/s)(1564KiB/1933msec)
    clat (usec): min=3, max=25946, avg=1374.22, stdev=3186.24
     lat (usec): min=3, max=25946, avg=1374.72, stdev=3186.23
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    5], 10.00th=[    8], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   13], 50.00th=[   19], 60.00th=[  938],
     | 70.00th=[ 1582], 80.00th=[ 2073], 90.00th=[ 2966], 95.00th=[ 4228],
     | 99.00th=[23200], 99.50th=[25297], 99.90th=[25822], 99.95th=[25822],
     | 99.99th=[25822]
   bw (  KiB/s): min=  648, max=  944, per=24.97%, avg=773.33, stdev=153.12, samples=3
   iops        : min=  162, max=  236, avg=193.33, stdev=38.28, samples=3
  write: IOPS=195, BW=780KiB/s (799kB/s)(1508KiB/1933msec)
    clat (usec): min=89, max=47976, avg=3666.89, stdev=6675.90
     lat (usec): min=89, max=47978, avg=3667.56, stdev=6675.95
    clat percentiles (usec):
     |  1.00th=[   90],  5.00th=[  109], 10.00th=[  119], 20.00th=[  133],
     | 30.00th=[  174], 40.00th=[  355], 50.00th=[ 1074], 60.00th=[ 1696],
     | 70.00th=[ 2311], 80.00th=[ 4686], 90.00th=[12125], 95.00th=[17433],
     | 99.00th=[32113], 99.50th=[47449], 99.90th=[47973], 99.95th=[47973],
     | 99.99th=[47973]
   bw (  KiB/s): min=  672, max=  880, per=23.65%, avg=770.67, stdev=104.41, samples=3
   iops        : min=  168, max=  220, avg=192.67, stdev=26.10, samples=3
  lat (usec)   : 4=0.39%, 10=14.84%, 20=10.42%, 50=2.47%, 100=1.82%
  lat (usec)   : 250=18.36%, 500=0.52%, 750=1.30%, 1000=4.56%
  lat (msec)   : 2=17.32%, 4=14.84%, 10=5.47%, 20=5.21%, 50=2.47%
  cpu          : usr=0.26%, sys=7.45%, ctx=807, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20989: Wed Apr 17 05:54:31 2024
   read: IOPS=197, BW=792KiB/s (811kB/s)(1508KiB/1905msec)
    clat (usec): min=3, max=29388, avg=1059.63, stdev=2142.89
     lat (usec): min=4, max=29388, avg=1060.36, stdev=2143.06
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    5], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   15], 50.00th=[   23], 60.00th=[  873],
     | 70.00th=[ 1598], 80.00th=[ 2114], 90.00th=[ 2704], 95.00th=[ 3294],
     | 99.00th=[ 8094], 99.50th=[15139], 99.90th=[29492], 99.95th=[29492],
     | 99.99th=[29492]
   bw (  KiB/s): min=  584, max=  984, per=25.23%, avg=781.33, stdev=200.05, samples=3
   iops        : min=  146, max=  246, avg=195.33, stdev=50.01, samples=3
  write: IOPS=205, BW=821KiB/s (841kB/s)(1564KiB/1905msec)
    clat (usec): min=81, max=45716, avg=3799.57, stdev=6604.28
     lat (usec): min=82, max=45717, avg=3800.18, stdev=6604.33
    clat percentiles (usec):
     |  1.00th=[   93],  5.00th=[  108], 10.00th=[  117], 20.00th=[  133],
     | 30.00th=[  165], 40.00th=[  269], 50.00th=[  922], 60.00th=[ 1696],
     | 70.00th=[ 2376], 80.00th=[ 5669], 90.00th=[13435], 95.00th=[17695],
     | 99.00th=[28705], 99.50th=[43254], 99.90th=[45876], 99.95th=[45876],
     | 99.99th=[45876]
   bw (  KiB/s): min=  752, max=  952, per=25.94%, avg=845.33, stdev=100.66, samples=3
   iops        : min=  188, max=  238, avg=211.33, stdev=25.17, samples=3
  lat (usec)   : 4=0.26%, 10=13.28%, 20=9.51%, 50=4.04%, 100=1.69%
  lat (usec)   : 250=19.53%, 500=1.69%, 750=1.69%, 1000=4.95%
  lat (msec)   : 2=15.62%, 4=14.71%, 10=4.82%, 20=5.99%, 50=2.21%
  cpu          : usr=0.16%, sys=7.72%, ctx=833, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3098KiB/s (3172kB/s), 792KiB/s-809KiB/s (811kB/s-829kB/s), io=5988KiB (6132kB), run=1813-1933msec
  WRITE: bw=3259KiB/s (3337kB/s), 780KiB/s-902KiB/s (799kB/s-924kB/s), io=6300KiB (6451kB), run=1813-1933msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=20982: Wed Apr 17 05:54:38 2024
   read: IOPS=42, BW=171KiB/s (175kB/s)(1480KiB/8654msec)
    clat (usec): min=615, max=35309, avg=2115.60, stdev=3876.55
     lat (usec): min=616, max=35309, avg=2116.29, stdev=3876.56
    clat percentiles (usec):
     |  1.00th=[  717],  5.00th=[  799], 10.00th=[  832], 20.00th=[  898],
     | 30.00th=[  947], 40.00th=[ 1037], 50.00th=[ 1139], 60.00th=[ 1287],
     | 70.00th=[ 1450], 80.00th=[ 1663], 90.00th=[ 2474], 95.00th=[ 7439],
     | 99.00th=[20317], 99.50th=[33817], 99.90th=[35390], 99.95th=[35390],
     | 99.99th=[35390]
   bw (  KiB/s): min=  104, max=  271, per=24.82%, avg=168.76, stdev=46.66, samples=17
   iops        : min=   26, max=   67, avg=42.06, stdev=11.56, samples=17
  write: IOPS=45, BW=184KiB/s (188kB/s)(1592KiB/8654msec)
    clat (usec): min=10744, max=51754, avg=19707.62, stdev=6137.54
     lat (usec): min=10745, max=51755, avg=19708.48, stdev=6137.54
    clat percentiles (usec):
     |  1.00th=[11207],  5.00th=[12256], 10.00th=[12911], 20.00th=[14222],
     | 30.00th=[15664], 40.00th=[17695], 50.00th=[19268], 60.00th=[20579],
     | 70.00th=[22152], 80.00th=[23462], 90.00th=[26084], 95.00th=[28967],
     | 99.00th=[45351], 99.50th=[48497], 99.90th=[51643], 99.95th=[51643],
     | 99.99th=[51643]
   bw (  KiB/s): min=  112, max=  224, per=25.64%, avg=183.35, stdev=35.76, samples=17
   iops        : min=   28, max=   56, avg=45.71, stdev= 8.89, samples=17
  lat (usec)   : 750=1.04%, 1000=16.80%
  lat (msec)   : 2=23.70%, 4=3.26%, 10=1.17%, 20=29.43%, 50=24.48%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.07%, sys=1.36%, ctx=814, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20983: Wed Apr 17 05:54:38 2024
   read: IOPS=40, BW=163KiB/s (167kB/s)(1436KiB/8801msec)
    clat (usec): min=684, max=46647, avg=1982.99, stdev=3692.15
     lat (usec): min=684, max=46647, avg=1984.11, stdev=3692.07
    clat percentiles (usec):
     |  1.00th=[  717],  5.00th=[  791], 10.00th=[  824], 20.00th=[  889],
     | 30.00th=[  938], 40.00th=[ 1012], 50.00th=[ 1139], 60.00th=[ 1287],
     | 70.00th=[ 1450], 80.00th=[ 1663], 90.00th=[ 2278], 95.00th=[ 7570],
     | 99.00th=[14877], 99.50th=[28705], 99.90th=[46400], 99.95th=[46400],
     | 99.99th=[46400]
   bw (  KiB/s): min=   72, max=  256, per=23.88%, avg=162.35, stdev=48.81, samples=17
   iops        : min=   18, max=   64, avg=40.59, stdev=12.20, samples=17
  write: IOPS=46, BW=186KiB/s (190kB/s)(1636KiB/8801msec)
    clat (usec): min=10229, max=54858, avg=19711.14, stdev=5945.29
     lat (usec): min=10230, max=54860, avg=19711.99, stdev=5945.31
    clat percentiles (usec):
     |  1.00th=[11076],  5.00th=[11863], 10.00th=[12780], 20.00th=[14222],
     | 30.00th=[15926], 40.00th=[18482], 50.00th=[19792], 60.00th=[20841],
     | 70.00th=[21890], 80.00th=[23462], 90.00th=[25560], 95.00th=[28705],
     | 99.00th=[40109], 99.50th=[47449], 99.90th=[54789], 99.95th=[54789],
     | 99.99th=[54789]
   bw (  KiB/s): min=  112, max=  208, per=25.60%, avg=183.06, stdev=25.12, samples=17
   iops        : min=   28, max=   52, avg=45.76, stdev= 6.28, samples=17
  lat (usec)   : 750=1.30%, 1000=16.41%
  lat (msec)   : 2=22.40%, 4=3.65%, 10=1.30%, 20=28.65%, 50=26.17%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.06%, sys=1.38%, ctx=820, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20984: Wed Apr 17 05:54:38 2024
   read: IOPS=46, BW=186KiB/s (191kB/s)(1564KiB/8405msec)
    clat (usec): min=674, max=47091, avg=2032.88, stdev=3780.80
     lat (usec): min=675, max=47091, avg=2033.63, stdev=3780.78
    clat percentiles (usec):
     |  1.00th=[  717],  5.00th=[  783], 10.00th=[  832], 20.00th=[  898],
     | 30.00th=[  947], 40.00th=[ 1020], 50.00th=[ 1156], 60.00th=[ 1287],
     | 70.00th=[ 1401], 80.00th=[ 1614], 90.00th=[ 2442], 95.00th=[ 5604],
     | 99.00th=[21103], 99.50th=[24511], 99.90th=[46924], 99.95th=[46924],
     | 99.99th=[46924]
   bw (  KiB/s): min=   96, max=  312, per=27.85%, avg=189.38, stdev=73.09, samples=16
   iops        : min=   24, max=   78, avg=47.25, stdev=18.29, samples=16
  write: IOPS=44, BW=179KiB/s (184kB/s)(1508KiB/8405msec)
    clat (usec): min=10917, max=60538, avg=20155.71, stdev=6320.05
     lat (usec): min=10917, max=60538, avg=20156.56, stdev=6320.08
    clat percentiles (usec):
     |  1.00th=[11076],  5.00th=[12256], 10.00th=[13042], 20.00th=[14353],
     | 30.00th=[16188], 40.00th=[18744], 50.00th=[20055], 60.00th=[21365],
     | 70.00th=[22676], 80.00th=[23987], 90.00th=[26346], 95.00th=[29492],
     | 99.00th=[49546], 99.50th=[52691], 99.90th=[60556], 99.95th=[60556],
     | 99.99th=[60556]
   bw (  KiB/s): min=  120, max=  208, per=24.88%, avg=177.88, stdev=28.48, samples=16
   iops        : min=   30, max=   52, avg=44.38, stdev= 7.08, samples=16
  lat (usec)   : 750=1.56%, 1000=17.84%
  lat (msec)   : 2=24.22%, 4=3.65%, 10=1.56%, 20=26.56%, 50=24.35%
  lat (msec)   : 100=0.26%
  cpu          : usr=0.06%, sys=1.37%, ctx=808, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20985: Wed Apr 17 05:54:38 2024
   read: IOPS=43, BW=173KiB/s (177kB/s)(1508KiB/8732msec)
    clat (usec): min=656, max=29121, avg=2150.19, stdev=3338.24
     lat (usec): min=656, max=29122, avg=2150.84, stdev=3338.24
    clat percentiles (usec):
     |  1.00th=[  725],  5.00th=[  775], 10.00th=[  816], 20.00th=[  881],
     | 30.00th=[  930], 40.00th=[ 1012], 50.00th=[ 1156], 60.00th=[ 1287],
     | 70.00th=[ 1467], 80.00th=[ 1860], 90.00th=[ 3556], 95.00th=[10421],
     | 99.00th=[19530], 99.50th=[21365], 99.90th=[29230], 99.95th=[29230],
     | 99.99th=[29230]
   bw (  KiB/s): min=   80, max=  240, per=24.90%, avg=169.35, stdev=44.60, samples=17
   iops        : min=   20, max=   60, avg=42.29, stdev=11.11, samples=17
  write: IOPS=44, BW=179KiB/s (183kB/s)(1564KiB/8732msec)
    clat (usec): min=10785, max=65541, avg=20195.08, stdev=6932.41
     lat (usec): min=10786, max=65542, avg=20195.95, stdev=6932.45
    clat percentiles (usec):
     |  1.00th=[11469],  5.00th=[12256], 10.00th=[12780], 20.00th=[13829],
     | 30.00th=[15926], 40.00th=[18744], 50.00th=[20055], 60.00th=[21103],
     | 70.00th=[22414], 80.00th=[23725], 90.00th=[25822], 95.00th=[32113],
     | 99.00th=[47449], 99.50th=[53216], 99.90th=[65799], 99.95th=[65799],
     | 99.99th=[65799]
   bw (  KiB/s): min=  128, max=  216, per=24.74%, avg=176.88, stdev=27.23, samples=17
   iops        : min=   32, max=   54, avg=44.18, stdev= 6.78, samples=17
  lat (usec)   : 750=1.17%, 1000=17.97%
  lat (msec)   : 2=21.22%, 4=4.04%, 10=2.21%, 20=27.08%, 50=26.04%
  lat (msec)   : 100=0.26%
  cpu          : usr=0.08%, sys=1.36%, ctx=811, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=680KiB/s (697kB/s), 163KiB/s-186KiB/s (167kB/s-191kB/s), io=5988KiB (6132kB), run=8405-8801msec
  WRITE: bw=716KiB/s (733kB/s), 179KiB/s-186KiB/s (183kB/s-190kB/s), io=6300KiB (6451kB), run=8405-8801msec