== sanity test 398b: DIO and buffer IO race ============== 12:41:39 (1713372099) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB) copied, 1.24516 s, 40.4 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=31244: Wed Apr 17 12:41:54 2024 read: IOPS=123, BW=493KiB/s (505kB/s)(5992KiB/12153msec) clat (usec): min=48, max=165874, avg=3801.45, stdev=5246.86 lat (usec): min=48, max=165876, avg=3802.53, stdev=5246.71 clat percentiles (usec): | 1.00th=[ 53], 5.00th=[ 69], 10.00th=[ 73], 20.00th=[ 153], | 30.00th=[ 2311], 40.00th=[ 2704], 50.00th=[ 3130], 60.00th=[ 3752], | 70.00th=[ 4686], 80.00th=[ 5800], 90.00th=[ 7504], 95.00th=[ 9110], | 99.00th=[ 12387], 99.50th=[ 15139], 99.90th=[ 39060], 99.95th=[166724], | 99.99th=[166724] bw ( KiB/s): min= 248, max= 760, per=24.74%, avg=488.63, stdev=115.62, samples=24 iops : min= 62, max= 190, avg=122.12, stdev=28.92, samples=24 write: IOPS=129, BW=518KiB/s (530kB/s)(6296KiB/12153msec) clat (usec): min=447, max=78000, avg=4036.91, stdev=4318.24 lat (usec): min=448, max=78000, avg=4037.65, stdev=4318.23 clat percentiles (usec): | 1.00th=[ 486], 5.00th=[ 519], 10.00th=[ 545], 20.00th=[ 603], | 30.00th=[ 775], 40.00th=[ 1500], 50.00th=[ 3195], 60.00th=[ 4146], | 70.00th=[ 5145], 80.00th=[ 6915], 90.00th=[ 9372], 95.00th=[11600], | 99.00th=[16581], 99.50th=[20841], 99.90th=[33817], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 296, max= 840, per=24.79%, avg=512.62, stdev=116.81, samples=24 iops : min= 74, max= 210, avg=128.12, stdev=29.21, samples=24 lat (usec) : 50=0.13%, 100=8.79%, 250=1.20%, 500=1.43%, 750=13.70% lat (usec) : 1000=2.93% lat (msec) : 2=5.40%, 4=27.25%, 10=33.56%, 20=5.14%, 50=0.39% lat (msec) : 100=0.03%, 250=0.03% cpu : usr=0.17%, sys=24.74%, ctx=6575, majf=0, minf=36 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=31246: Wed Apr 17 12:41:54 2024 read: IOPS=122, BW=491KiB/s (503kB/s)(5940KiB/12104msec) clat (usec): min=46, max=59056, avg=3724.00, stdev=3290.56 lat (usec): min=47, max=59056, avg=3724.61, stdev=3290.51 clat percentiles (usec): | 1.00th=[ 50], 5.00th=[ 69], 10.00th=[ 73], 20.00th=[ 251], | 30.00th=[ 2376], 40.00th=[ 2802], 50.00th=[ 3228], 60.00th=[ 3884], | 70.00th=[ 4817], 80.00th=[ 5735], 90.00th=[ 7242], 95.00th=[ 8455], | 99.00th=[11863], 99.50th=[15270], 99.90th=[39060], 99.95th=[58983], | 99.99th=[58983] bw ( KiB/s): min= 256, max= 672, per=24.86%, avg=490.96, stdev=85.82, samples=24 iops : min= 64, max= 168, avg=122.71, stdev=21.47, samples=24 write: IOPS=131, BW=524KiB/s (537kB/s)(6348KiB/12104msec) clat (usec): min=462, max=161538, avg=4078.58, stdev=5619.36 lat (usec): min=463, max=161539, avg=4079.45, stdev=5619.33 clat percentiles (usec): | 1.00th=[ 494], 5.00th=[ 523], 10.00th=[ 553], 20.00th=[ 611], | 30.00th=[ 857], 40.00th=[ 1713], 50.00th=[ 3195], 60.00th=[ 4080], | 70.00th=[ 5080], 80.00th=[ 6652], 90.00th=[ 9110], 95.00th=[ 10945], | 99.00th=[ 15926], 99.50th=[ 21103], 99.90th=[ 58983], 99.95th=[162530], | 99.99th=[162530] bw ( KiB/s): min= 336, max= 784, per=25.30%, avg=523.25, stdev=99.71, samples=24 iops : min= 84, max= 196, avg=130.79, stdev=24.92, samples=24 lat (usec) : 50=0.36%, 100=8.20%, 250=1.07%, 500=0.98%, 750=13.09% lat (usec) : 1000=3.45% lat (msec) : 2=5.53%, 4=27.34%, 10=34.96%, 20=4.62%, 50=0.29% lat (msec) : 100=0.07%, 250=0.03% cpu : usr=0.17%, sys=25.30%, ctx=6651, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=31248: Wed Apr 17 12:41:54 2024 read: IOPS=124, BW=497KiB/s (509kB/s)(6028KiB/12138msec) clat (usec): min=46, max=161786, avg=3728.88, stdev=5169.50 lat (usec): min=47, max=161787, avg=3730.93, stdev=5174.22 clat percentiles (usec): | 1.00th=[ 50], 5.00th=[ 65], 10.00th=[ 72], 20.00th=[ 93], | 30.00th=[ 2245], 40.00th=[ 2638], 50.00th=[ 3130], 60.00th=[ 3687], | 70.00th=[ 4621], 80.00th=[ 5997], 90.00th=[ 7635], 95.00th=[ 8979], | 99.00th=[ 12649], 99.50th=[ 19530], 99.90th=[ 30278], 99.95th=[162530], | 99.99th=[162530] bw ( KiB/s): min= 312, max= 672, per=25.13%, avg=496.29, stdev=98.44, samples=24 iops : min= 78, max= 168, avg=124.04, stdev=24.62, samples=24 write: IOPS=128, BW=516KiB/s (528kB/s)(6260KiB/12138msec) clat (usec): min=418, max=71126, avg=4095.35, stdev=4353.39 lat (usec): min=419, max=71127, avg=4096.49, stdev=4353.52 clat percentiles (usec): | 1.00th=[ 490], 5.00th=[ 523], 10.00th=[ 553], 20.00th=[ 627], | 30.00th=[ 848], 40.00th=[ 1893], 50.00th=[ 3425], 60.00th=[ 4146], | 70.00th=[ 5276], 80.00th=[ 6652], 90.00th=[ 8979], 95.00th=[11469], | 99.00th=[16319], 99.50th=[17695], 99.90th=[50070], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 336, max= 640, per=24.87%, avg=514.29, stdev=93.11, samples=24 iops : min= 84, max= 160, avg=128.54, stdev=23.29, samples=24 lat (usec) : 50=0.42%, 100=9.86%, 250=1.40%, 500=1.07%, 750=12.70% lat (usec) : 1000=3.19% lat (msec) : 2=4.85%, 4=27.18%, 10=33.98%, 20=4.88%, 50=0.36% lat (msec) : 100=0.07%, 250=0.03% cpu : usr=0.21%, sys=24.60%, ctx=6575, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=31250: Wed Apr 17 12:41:54 2024 read: IOPS=125, BW=501KiB/s (513kB/s)(6052KiB/12079msec) clat (usec): min=45, max=161522, avg=3727.70, stdev=5076.84 lat (usec): min=45, max=161523, avg=3728.48, stdev=5076.87 clat percentiles (usec): | 1.00th=[ 50], 5.00th=[ 66], 10.00th=[ 72], 20.00th=[ 103], | 30.00th=[ 2311], 40.00th=[ 2704], 50.00th=[ 3163], 60.00th=[ 3818], | 70.00th=[ 4555], 80.00th=[ 5800], 90.00th=[ 7439], 95.00th=[ 8848], | 99.00th=[ 11994], 99.50th=[ 16188], 99.90th=[ 38536], 99.95th=[162530], | 99.99th=[162530] bw ( KiB/s): min= 296, max= 736, per=25.34%, avg=500.54, stdev=97.92, samples=24 iops : min= 74, max= 184, avg=125.08, stdev=24.49, samples=24 write: IOPS=129, BW=516KiB/s (529kB/s)(6236KiB/12079msec) clat (usec): min=446, max=66272, avg=4065.44, stdev=4192.59 lat (usec): min=446, max=66272, avg=4066.12, stdev=4192.61 clat percentiles (usec): | 1.00th=[ 498], 5.00th=[ 523], 10.00th=[ 545], 20.00th=[ 635], | 30.00th=[ 865], 40.00th=[ 1745], 50.00th=[ 3294], 60.00th=[ 4178], | 70.00th=[ 5342], 80.00th=[ 6849], 90.00th=[ 9241], 95.00th=[11076], | 99.00th=[15270], 99.50th=[17695], 99.90th=[51643], 99.95th=[66323], | 99.99th=[66323] bw ( KiB/s): min= 360, max= 664, per=24.93%, avg=515.54, stdev=87.98, samples=24 iops : min= 90, max= 166, avg=128.83, stdev=22.01, samples=24 lat (usec) : 50=0.59%, 100=9.18%, 250=1.01%, 500=0.94%, 750=12.60% lat (usec) : 1000=3.09% lat (msec) : 2=5.92%, 4=26.76%, 10=34.93%, 20=4.69%, 50=0.20% lat (msec) : 100=0.07%, 250=0.03% cpu : usr=0.11%, sys=25.24%, ctx=6362, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1976KiB/s (2023kB/s), 491KiB/s-501KiB/s (503kB/s-513kB/s), io=23.4MiB (24.6MB), run=12079-12153msec WRITE: bw=2069KiB/s (2118kB/s), 516KiB/s-524KiB/s (528kB/s-537kB/s), io=24.6MiB (25.7MB), run=12079-12153msec rand-rw: (groupid=0, jobs=1): err= 0: pid=31245: Wed Apr 17 12:42:22 2024 read: IOPS=37, BW=150KiB/s (154kB/s)(5992KiB/39916msec) clat (usec): min=1494, max=166041, avg=3661.62, stdev=5071.59 lat (usec): min=1494, max=166043, avg=3662.29, stdev=5071.62 clat percentiles (usec): | 1.00th=[ 1713], 5.00th=[ 1827], 10.00th=[ 1909], 20.00th=[ 2008], | 30.00th=[ 2114], 40.00th=[ 2245], 50.00th=[ 2442], 60.00th=[ 2868], | 70.00th=[ 3589], 80.00th=[ 4178], 90.00th=[ 6456], 95.00th=[ 9372], | 99.00th=[ 16319], 99.50th=[ 18744], 99.90th=[ 35914], 99.95th=[166724], | 99.99th=[166724] bw ( KiB/s): min= 56, max= 311, per=25.08%, avg=149.95, stdev=59.82, samples=79 iops : min= 14, max= 77, avg=37.39, stdev=14.96, samples=79 write: IOPS=39, BW=158KiB/s (162kB/s)(6296KiB/39916msec) clat (usec): min=11717, max=79864, avg=21811.76, stdev=7823.81 lat (usec): min=11717, max=79865, avg=21812.55, stdev=7823.88 clat percentiles (usec): | 1.00th=[12387], 5.00th=[13304], 10.00th=[13960], 20.00th=[15139], | 30.00th=[16712], 40.00th=[19006], 50.00th=[20841], 60.00th=[22152], | 70.00th=[23462], 80.00th=[25560], 90.00th=[32900], 95.00th=[38536], | 99.00th=[48497], 99.50th=[52691], 99.90th=[58459], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 55, max= 216, per=25.13%, avg=157.34, stdev=41.61, samples=79 iops : min= 13, max= 54, avg=39.24, stdev=10.45, samples=79 lat (msec) : 2=9.34%, 4=28.65%, 10=8.76%, 20=25.13%, 50=27.77% lat (msec) : 100=0.33%, 250=0.03% cpu : usr=0.10%, sys=4.28%, ctx=3551, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=31247: Wed Apr 17 12:42:22 2024 read: IOPS=37, BW=148KiB/s (152kB/s)(5940KiB/40015msec) clat (usec): min=1556, max=164182, avg=3754.41, stdev=5130.77 lat (usec): min=1557, max=164183, avg=3755.05, stdev=5130.79 clat percentiles (usec): | 1.00th=[ 1713], 5.00th=[ 1844], 10.00th=[ 1926], 20.00th=[ 2040], | 30.00th=[ 2147], 40.00th=[ 2278], 50.00th=[ 2507], 60.00th=[ 2933], | 70.00th=[ 3621], 80.00th=[ 4293], 90.00th=[ 6783], 95.00th=[ 9503], | 99.00th=[ 14877], 99.50th=[ 19006], 99.90th=[ 43779], 99.95th=[164627], | 99.99th=[164627] bw ( KiB/s): min= 16, max= 287, per=24.85%, avg=148.61, stdev=57.94, samples=79 iops : min= 4, max= 71, avg=37.11, stdev=14.47, samples=79 write: IOPS=39, BW=159KiB/s (162kB/s)(6348KiB/40015msec) clat (usec): min=11983, max=79512, avg=21637.65, stdev=7527.41 lat (usec): min=11984, max=79513, avg=21638.40, stdev=7527.44 clat percentiles (usec): | 1.00th=[12518], 5.00th=[13435], 10.00th=[14091], 20.00th=[15139], | 30.00th=[16909], 40.00th=[18744], 50.00th=[20579], 60.00th=[22152], | 70.00th=[23462], 80.00th=[25560], 90.00th=[31327], 95.00th=[36963], | 99.00th=[47449], 99.50th=[49021], 99.90th=[74974], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 64, max= 200, per=25.23%, avg=157.92, stdev=39.62, samples=79 iops : min= 16, max= 50, avg=39.44, stdev= 9.93, samples=79 lat (msec) : 2=8.04%, 4=28.84%, 10=9.34%, 20=25.91%, 50=27.67% lat (msec) : 100=0.16%, 250=0.03% cpu : usr=0.09%, sys=4.28%, ctx=3542, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=31249: Wed Apr 17 12:42:22 2024 read: IOPS=37, BW=150KiB/s (154kB/s)(6028KiB/40121msec) clat (usec): min=1461, max=64246, avg=3826.15, stdev=3452.80 lat (usec): min=1461, max=64246, avg=3826.81, stdev=3452.77 clat percentiles (usec): | 1.00th=[ 1663], 5.00th=[ 1844], 10.00th=[ 1926], 20.00th=[ 2024], | 30.00th=[ 2147], 40.00th=[ 2311], 50.00th=[ 2540], 60.00th=[ 3130], | 70.00th=[ 3752], 80.00th=[ 4621], 90.00th=[ 7308], 95.00th=[10290], | 99.00th=[14615], 99.50th=[17433], 99.90th=[39584], 99.95th=[64226], | 99.99th=[64226] bw ( KiB/s): min= 32, max= 288, per=25.12%, avg=150.20, stdev=56.55, samples=80 iops : min= 8, max= 72, avg=37.48, stdev=14.18, samples=80 write: IOPS=39, BW=156KiB/s (160kB/s)(6260KiB/40121msec) clat (msec): min=11, max=187, avg=21.89, stdev= 8.73 lat (msec): min=11, max=187, avg=21.89, stdev= 8.73 clat percentiles (msec): | 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 16], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 21], 60.00th=[ 23], | 70.00th=[ 24], 80.00th=[ 26], 90.00th=[ 32], 95.00th=[ 39], | 99.00th=[ 48], 99.50th=[ 56], 99.90th=[ 75], 99.95th=[ 188], | 99.99th=[ 188] bw ( KiB/s): min= 47, max= 208, per=24.88%, avg=155.75, stdev=42.07, samples=80 iops : min= 11, max= 52, avg=38.86, stdev=10.58, samples=80 lat (msec) : 2=8.76%, 4=27.73%, 10=9.99%, 20=25.39%, 50=27.80% lat (msec) : 100=0.29%, 250=0.03% cpu : usr=0.07%, sys=4.33%, ctx=3623, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=31251: Wed Apr 17 12:42:22 2024 read: IOPS=38, BW=152KiB/s (156kB/s)(6052KiB/39787msec) clat (usec): min=1519, max=44001, avg=3695.67, stdev=3273.11 lat (usec): min=1520, max=44001, avg=3696.50, stdev=3273.10 clat percentiles (usec): | 1.00th=[ 1696], 5.00th=[ 1844], 10.00th=[ 1909], 20.00th=[ 2008], | 30.00th=[ 2114], 40.00th=[ 2245], 50.00th=[ 2442], 60.00th=[ 2868], | 70.00th=[ 3589], 80.00th=[ 4293], 90.00th=[ 6980], 95.00th=[10159], | 99.00th=[17171], 99.50th=[21890], 99.90th=[39584], 99.95th=[43779], | 99.99th=[43779] bw ( KiB/s): min= 16, max= 280, per=25.34%, avg=151.54, stdev=57.45, samples=79 iops : min= 4, max= 70, avg=37.85, stdev=14.39, samples=79 write: IOPS=39, BW=157KiB/s (160kB/s)(6236KiB/39787msec) clat (msec): min=11, max=184, avg=21.87, stdev= 8.58 lat (msec): min=11, max=184, avg=21.87, stdev= 8.58 clat percentiles (msec): | 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 15], 20.00th=[ 16], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 21], 60.00th=[ 23], | 70.00th=[ 24], 80.00th=[ 26], 90.00th=[ 33], 95.00th=[ 39], | 99.00th=[ 47], 99.50th=[ 51], 99.90th=[ 69], 99.95th=[ 186], | 99.99th=[ 186] bw ( KiB/s): min= 56, max= 200, per=24.98%, avg=156.41, stdev=40.46, samples=79 iops : min= 14, max= 50, avg=39.06, stdev=10.15, samples=79 lat (msec) : 2=9.60%, 4=28.39%, 10=8.76%, 20=25.07%, 50=27.96% lat (msec) : 100=0.20%, 250=0.03% cpu : usr=0.08%, sys=4.32%, ctx=3587, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=598KiB/s (613kB/s), 148KiB/s-152KiB/s (152kB/s-156kB/s), io=23.4MiB (24.6MB), run=39787-40121msec WRITE: bw=627KiB/s (642kB/s), 156KiB/s-159KiB/s (160kB/s-162kB/s), io=24.6MiB (25.7MB), run=39787-40121msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes