-----============= acceptance-small: sanityn ============----- Thu Apr 18 20:16:10 EDT 2024 excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 20:16:13 (1713485773) === Starting client oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Started clients oleg426-client.virtnet: 192.168.204.126@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg426-client.virtnet: executing check_config_client /mnt/lustre oleg426-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg426-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800aaa91800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b6095000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aaa91800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b6095000.idle_timeout=debug disable quota as required oleg426-server: oleg426-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 20:16:19 (1713485779) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109914 s, 95.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6897] debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 20:16:20 (1713485780) PASS 0 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 20:16:24 (1713485784) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 20:16:27 (1713485787) -rw-r--r-- 1 root root 0 Apr 18 20:16 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 20:16:30 (1713485790) -rw-r--r-- 1 root root 0 Apr 18 20:16 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 20:16:33 (1713485793) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 20:16:36 (1713485796) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2e: check chmod on root is propagated to others ========================================================== 20:16:39 (1713485799) total 1 -rwxrwxrwx 1 root root 0 Apr 18 20:16 f2a -rwxrwxrwx 1 root root 0 Apr 18 20:16 f2b total 1 -rwxrwxrwx 1 root root 0 Apr 18 20:16 f2a -rwxrwxrwx 1 root root 0 Apr 18 20:16 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00238223 s, 215 kB/s PASS 2e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 20:16:42 (1713485802) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2g: check blocks update on sync write ==== 20:16:44 (1713485804) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0994148 s, 21.1 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 20:16:48 (1713485808) PASS 3 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 4: fstat validation on multiple mount points ==================================================================== 20:16:50 (1713485810) PASS 4 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 20:16:54 (1713485814) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 6: remove of open file on other node ============================================================================ 20:16:57 (1713485817) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 7: remove of open directory on other node ======================================================================= 20:17:00 (1713485820) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 8: remove of open special file on other node ==================================================================== 20:17:03 (1713485823) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 20:17:06 (1713485826) PASS 9a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 9b: append to striped sparse file ======== 20:17:09 (1713485829) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114716 s, 91.4 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.00765613 s, 0.4 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 20:17:12 (1713485832) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00166074 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0042552 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00370605 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00517116 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00362906 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00391875 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0027668 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00386287 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00270815 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00237514 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00383096 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00284341 s, 0.4 kB/s PASS 10a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 20:17:16 (1713485836) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00287928 s, 1.1 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00591268 s, 693 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000201515 s, 15.2 MB/s PASS 10b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 11: execution of file opened for write should return error ============================================================== 20:17:19 (1713485839) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 20:17:22 (1713485842) warning: '-runtime' deprecated, use '-t runtime' instead - link 3157 (time 1713485852.69 total 10.00 last 315.64) - link 6406 (time 1713485862.69 total 20.00 last 324.82) - link 9604 (time 1713485872.69 total 30.01 last 319.77) - link 12629 (time 1713485882.70 total 40.01 last 302.39) - link 16015 (time 1713485892.70 total 50.01 last 338.59) - link 19363 (time 1713485902.70 total 60.01 last 334.74) - link 22512 (time 1713485912.70 total 70.01 last 314.80) - link 25592 (time 1713485922.70 total 80.02 last 307.93) - link 28805 (time 1713485932.70 total 90.02 last 321.29) - link 30000 (time 1713485936.46 total 93.77 last 318.40) total: 31946 link in 100.00 seconds: 319.46 ops/second using seed 3993523567 running for 100 seconds - stat 10000 (time 1713485846 ; total 3 ; last 3) - stat 20000 (time 1713485850 ; total 7 ; last 4) - stat 30000 (time 1713485853 ; total 10 ; last 3) - stat 40000 (time 1713485856 ; total 13 ; last 3) - stat 50000 (time 1713485859 ; total 16 ; last 3) - stat 60000 (time 1713485862 ; total 19 ; last 3) - stat 70000 (time 1713485865 ; total 22 ; last 3) - stat 80000 (time 1713485868 ; total 25 ; last 3) - stat 90000 (time 1713485872 ; total 29 ; last 4) - stat 100000 (time 1713485875 ; total 32 ; last 3) - stat 110000 (time 1713485879 ; total 36 ; last 4) - stat 120000 (time 1713485882 ; total 39 ; last 3) - stat 130000 (time 1713485885 ; total 42 ; last 3) - stat 140000 (time 1713485888 ; total 45 ; last 3) - stat 150000 (time 1713485891 ; total 48 ; last 3) - stat 160000 (time 1713485897 ; total 54 ; last 6) - stat 170000 (time 1713485906 ; total 63 ; last 9) - stat 180000 (time 1713485916 ; total 73 ; last 10) - stat 190000 (time 1713485926 ; total 83 ; last 10) - stat 199937 (time 1713485937 ; total 94 ; last 11) - stat 200000 (time 1713485937 ; total 94 ; last 0) total: 206278 stats in 100 seconds: 2062.780029 stats/second - unlinked 0 (time 1713485893 ; total 0 ; last 0) - unlinked 10000 (time 1713485943 ; total 50 ; last 50) - unlinked 20000 (time 1713485966 ; total 73 ; last 23) - unlinked 30000 (time 1713485988 ; total 95 ; last 22) unlink(/mnt/lustre2/lockdir/lockfile31946) error: No such file or directory total: 31945 unlinks in 99 seconds: 322.676758 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (23384) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (23387) - No such process PASS 12 (151s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 13: test directory page revocation ======= 20:19:55 (1713485995) f13 f13 PASS 13 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 20:19:58 (1713485998) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4697: 24991 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 20:20:01 (1713486001) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4697: 25730 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 20:20:04 (1713486004) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 26467 Terminated $DIR1/$tdir/sleep 60 PASS 14b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 20:20:06 (1713486006) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 27205 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14d: chmod of executing file is still possible ================================================================== 20:20:09 (1713486009) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 27944 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 15: test out-of-space with multiple writers ===================================================================== 20:20:13 (1713486013) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7519232 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7519232kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16a: 500 iterations of dual-mount fsx ==== 20:20:24 (1713486024) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.121309 s, 69.2 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.12758 s, 65.8 MB/s Chance of close/open is 1 in 50 Seed set to 6025 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1713486026.096249 write 0x853d48 thru 0x863368 (0xf621 bytes) 000002[0] 1713486026.101720 read 0x75aa4a thru 0x7622c0 (0x7877 bytes) 000003[1] 1713486026.110551 write 0x254e20 thru 0x257d06 (0x2ee7 bytes) 000006[1] 1713486026.128585 read 0x397b1b thru 0x399c05 (0x20eb bytes) truncating to largest ever: 0x7cb8c2 000010[0] 1713486026.142822 trunc from 0x863369 to 0x7cb8c1 (0x97aa7 bytes) 000011[0] 1713486026.151945 mapwrite 0x2d79b9 thru 0x2e4f41 (0xd589 bytes) 000012[0] 1713486026.182967 mapwrite 0x342c97 thru 0x34cb87 (0x9ef1 bytes) 000015[1] 1713486026.213564 read 0x2a9dce thru 0x2b130a (0x753d bytes) 000017[0] 1713486026.235168 mapread 0x22e17e thru 0x233035 (0x4eb8 bytes) 000019[1] 1713486026.246349 write 0x036717 thru 0x044bb8 (0xe4a2 bytes) 000021[1] 1713486026.260113 mapwrite 0x0f4fa1 thru 0x0f8763 (0x37c3 bytes) 000023[0] 1713486026.291338 write 0x9a0c70 thru 0x9b02c3 (0xf654 bytes) 000026[0] 1713486026.298978 trunc from 0x9b02c4 to 0x0d2407 (0x8ddebc bytes) 000030[0] 1713486026.315938 mapwrite 0x8706a8 thru 0x877be0 (0x7539 bytes) 000031[1] 1713486026.341824 write 0x366059 thru 0x370e93 (0xae3b bytes) 000032[0] 1713486026.348659 trunc from 0x877be1 to 0x62ce02 (0x24adde bytes) 000033[1] 1713486026.356479 write 0x8bcdb1 thru 0x8c01b7 (0x3407 bytes) 000035[1] 1713486026.365132 read 0x2e1810 thru 0x2efb3d (0xe32e bytes) 000036[0] 1713486026.388303 write 0x34b0ac thru 0x35ab24 (0xfa79 bytes) 000041[0] 1713486026.401892 mapwrite 0x24987b thru 0x24e223 (0x49a9 bytes) 000043[1] 1713486026.426320 read 0x5353ae thru 0x544740 (0xf393 bytes) 000045[0] 1713486026.446149 trunc from 0x8c01b8 to 0x6fe86f (0x1c1948 bytes) 000048[1] 1713486026.458731 trunc from 0x6fe870 to 0x08ce5e (0x671a11 bytes) 000049[0] 1713486026.468559 trunc from 0x08ce5f to 0x791eb0 (0x705052 bytes) 000050[1] 1713486026.478201 write 0x3b6997 thru 0x3c5653 (0xecbd bytes) 000054[0] 1713486026.508657 mapread 0x1a0a14 thru 0x1ab309 (0xa8f6 bytes) 000056[1] 1713486026.524814 trunc from 0x791eb1 to 0x139654 (0x65885c bytes) 000059[0] 1713486026.541268 mapread 0x09188f thru 0x0937f9 (0x1f6b bytes) 000060[1] 1713486026.552707 mapread 0x09c9ba thru 0x09f215 (0x285c bytes) 000063[1] 1713486026.559951 mapread 0x050b55 thru 0x050da3 (0x24f bytes) 000066[1] 1713486026.567151 mapread 0x0d3f39 thru 0x0dce24 (0x8eec bytes) 000073[1] 1713486026.587054 mapread 0x02ce63 thru 0x03ae92 (0xe030 bytes) 000074[0] 1713486026.603486 mapwrite 0x4a984f thru 0x4ad83c (0x3fee bytes) 000075[1] 1713486026.630243 mapwrite 0x2fe67c thru 0x2ff39e (0xd23 bytes) 000078[1] 1713486026.657442 mapread 0x28e927 thru 0x295e18 (0x74f2 bytes) 000082[1] 1713486026.677099 write 0x0550b5 thru 0x05513c (0x88 bytes) 000084[1] 1713486026.683417 trunc from 0x4ad83d to 0x1675ec (0x346250 bytes) 000085[0] 1713486026.691953 read 0x0cfe94 thru 0x0d3f0b (0x4078 bytes) 000087[0] 1713486026.706867 mapwrite 0x715aaa thru 0x724b5a (0xf0b1 bytes) 000088[0] 1713486026.746604 trunc from 0x724b5b to 0x0b6f36 (0x66dc24 bytes) 000094[0] 1713486026.767747 trunc from 0x0b6f37 to 0x203075 (0x14c13f bytes) 000095[1] 1713486026.772092 mapwrite 0x666c6e thru 0x6747e0 (0xdb73 bytes) 000096[1] 1713486026.805988 mapwrite 0x2b4d7a thru 0x2c1c74 (0xcefb bytes) 000097[1] 1713486026.833955 mapwrite 0x95c2bd thru 0x9613b3 (0x50f7 bytes) 000098[0] 1713486026.863340 mapwrite 0x3e6d8d thru 0x3eb413 (0x4687 bytes) 000099[1] 1713486026.888283 mapread 0x3bd540 thru 0x3c7cb7 (0xa778 bytes) 000102[1] 1713486026.907009 read 0x3c0f63 thru 0x3c4d4b (0x3de9 bytes) 000103[0] 1713486026.909819 read 0x2d5e74 thru 0x2e3ebe (0xe04b bytes) 000106[0] 1713486026.930472 read 0x4e1547 thru 0x4ec98b (0xb445 bytes) 000107[1] 1713486026.953578 trunc from 0x9613b4 to 0x1a3d1a (0x7bd699 bytes) 000108[1] 1713486026.966491 read 0x1534c6 thru 0x15fc49 (0xc784 bytes) 000110[1] 1713486026.973886 mapread 0x00e43e thru 0x012dd3 (0x4996 bytes) 000112[1] 1713486026.974405 mapwrite 0x982471 thru 0x98ff6d (0xdafd bytes) 000113[0] 1713486027.016142 read 0x34e94f thru 0x35d3fc (0xeaae bytes) 000114[1] 1713486027.037950 write 0x51bd44 thru 0x52bc3b (0xfef8 bytes) 000117[1] 1713486027.039707 write 0x0ec9d1 thru 0x0f7f9f (0xb5cf bytes) 000118[1] 1713486027.048988 write 0x2fcafd thru 0x309dad (0xd2b1 bytes) 000121[0] 1713486027.057578 mapwrite 0x3842aa thru 0x38cc12 (0x8969 bytes) 000123[0] 1713486027.087926 read 0x1d25ff thru 0x1d58e9 (0x32eb bytes) 000124[0] 1713486027.104992 read 0x441e60 thru 0x443226 (0x13c7 bytes) 000125[0] 1713486027.123912 write 0x80ada1 thru 0x80bae8 (0xd48 bytes) truncating to largest ever: 0x96ae2e 000128[1] 1713486027.137778 trunc from 0x98ff6e to 0x96ae2d (0x25140 bytes) 000129[0] 1713486027.153559 write 0x1c11ac thru 0x1c1e43 (0xc98 bytes) 000130[1] 1713486027.158875 mapread 0x6a4041 thru 0x6b353e (0xf4fe bytes) 000132[1] 1713486027.179756 mapread 0x4316f3 thru 0x43457e (0x2e8c bytes) 000136[1] 1713486027.195589 mapread 0x8d2a73 thru 0x8d7dec (0x537a bytes) 000138[1] 1713486027.209057 write 0x8e6cd6 thru 0x8ed499 (0x67c4 bytes) 000140[1] 1713486027.216478 read 0x309523 thru 0x3182d9 (0xedb7 bytes) 000147[1] 1713486027.254204 write 0x6b1ad7 thru 0x6b1e11 (0x33b bytes) 000148[0] 1713486027.254593 read 0x4a1238 thru 0x4a3519 (0x22e2 bytes) 000152[0] 1713486027.267051 write 0x3630fc thru 0x36ee79 (0xbd7e bytes) 000154[1] 1713486027.299175 mapwrite 0x0920bf thru 0x09438d (0x22cf bytes) 000155[1] 1713486027.330504 mapread 0x180811 thru 0x18c369 (0xbb59 bytes) 000156[0] 1713486027.347477 read 0x17e9d8 thru 0x1818ed (0x2f16 bytes) 000157[1] 1713486027.369579 write 0x9e2c44 thru 0x9f1cf2 (0xf0af bytes) 000158[1] 1713486027.382756 read 0x1377d7 thru 0x145970 (0xe19a bytes) 000159[0] 1713486027.410418 mapwrite 0x67e489 thru 0x68d52c (0xf0a4 bytes) 000160[0] 1713486027.466613 mapread 0x09bf51 thru 0x0ab151 (0xf201 bytes) 000163[0] 1713486027.498222 write 0x1e513d thru 0x1e9f93 (0x4e57 bytes) 000164[0] 1713486027.505637 trunc from 0x9f1cf3 to 0x92ee35 (0xc2ebd bytes) 000165[1] 1713486027.513358 write 0x93971b thru 0x941800 (0x80e6 bytes) 000166[1] 1713486027.522162 read 0x27bc9a thru 0x27c5ab (0x912 bytes) 000167[1] 1713486027.541427 read 0x4f025c thru 0x4f4e30 (0x4bd5 bytes) 000168[1] 1713486027.560077 mapwrite 0x10a720 thru 0x118517 (0xddf8 bytes) 000171[0] 1713486027.584236 mapwrite 0x63f9e1 thru 0x643b64 (0x4184 bytes) 000173[1] 1713486027.616015 mapwrite 0x0d4569 thru 0x0e3a74 (0xf50c bytes) 000174[0] 1713486027.636112 mapread 0x19b9e9 thru 0x19d804 (0x1e1c bytes) 000176[0] 1713486027.649223 trunc from 0x941801 to 0x178a3b (0x7c8dc5 bytes) 000178[1] 1713486027.661171 write 0x4dfa4c thru 0x4e028b (0x840 bytes) 000183[0] 1713486027.679014 trunc from 0x4e028c to 0x362911 (0x17d97a bytes) 000185[0] 1713486027.691649 mapread 0x27a19e thru 0x2845e5 (0xa448 bytes) 000186[0] 1713486027.698290 read 0x1e388a thru 0x1f1b19 (0xe290 bytes) 000187[1] 1713486027.700010 mapread 0x1d8ec5 thru 0x1de806 (0x5942 bytes) 000188[1] 1713486027.711993 mapread 0x354b33 thru 0x35ca85 (0x7f53 bytes) 000192[1] 1713486027.729184 mapwrite 0x8682e2 thru 0x86952a (0x1249 bytes) 000197[1] 1713486027.757134 read 0x673aba thru 0x67d5b2 (0x9af9 bytes) 000199[0] 1713486027.782531 read 0x4864c1 thru 0x4901b6 (0x9cf6 bytes) 000200[0] 1713486027.795089 trunc from 0x86952b to 0x5a31e8 (0x2c6342 bytes) 000201[0] 1713486027.804591 read 0x2a7118 thru 0x2a7c39 (0xb22 bytes) 000203[1] 1713486027.825726 write 0x1734ec thru 0x17c9a8 (0x94bd bytes) 000204[1] 1713486027.836543 trunc from 0x5a31e9 to 0x91e337 (0x37b14f bytes) 000207[0] 1713486027.847149 read 0x0aeb89 thru 0x0bc16d (0xd5e5 bytes) 000208[0] 1713486027.865614 trunc from 0x91e338 to 0x31441a (0x609f1d bytes) 000209[1] 1713486027.879840 write 0x32ebcf thru 0x333206 (0x4638 bytes) 000212[1] 1713486027.892131 write 0x7fa9b4 thru 0x7fbc9d (0x12ea bytes) 000213[1] 1713486027.898181 read 0x621de7 thru 0x625a8e (0x3ca8 bytes) 000214[0] 1713486027.911258 write 0x6cf997 thru 0x6d24a2 (0x2b0c bytes) 000220[1] 1713486027.950553 trunc from 0x7fbc9e to 0x8d4d42 (0xd90a5 bytes) 000221[1] 1713486027.963768 mapread 0x8577cf thru 0x85cc36 (0x5468 bytes) truncating to largest ever: 0x96feca 000222[0] 1713486027.987622 trunc from 0x8d4d43 to 0x96fec9 (0x9b187 bytes) 000224[0] 1713486028.009073 write 0x560f17 thru 0x56e555 (0xd63f bytes) 000225[1] 1713486028.015660 trunc from 0x96feca to 0x6da649 (0x295880 bytes) 000226[0] 1713486028.029199 write 0x3755c1 thru 0x37fc32 (0xa672 bytes) 000228[0] 1713486028.039454 trunc from 0x6da64a to 0x2e3f89 (0x3f66c0 bytes) 000229[1] 1713486028.045160 write 0x5bcf72 thru 0x5bd2f2 (0x381 bytes) 000230[0] 1713486028.050546 mapread 0x2ba544 thru 0x2c3f05 (0x99c2 bytes) 000232[0] 1713486028.069644 mapwrite 0x074d4a thru 0x078fae (0x4265 bytes) 000235[1] 1713486028.096283 read 0x475c30 thru 0x47fae0 (0x9eb1 bytes) 000236[0] 1713486028.106890 trunc from 0x5bd2f3 to 0x96cbe4 (0x3af8f2 bytes) 000240[1] 1713486028.118202 mapread 0x6e5933 thru 0x6f34db (0xdba9 bytes) 000241[1] 1713486028.133810 write 0x64aede thru 0x64f8fa (0x4a1d bytes) 000244[0] 1713486028.145087 write 0x3c22be thru 0x3c99a1 (0x76e4 bytes) 000247[0] 1713486028.155261 write 0x1d1fa0 thru 0x1d73bb (0x541c bytes) 000249[0] 1713486028.163279 mapread 0x7e5766 thru 0x7f5603 (0xfe9e bytes) 000251[0] 1713486028.185071 read 0x38eea8 thru 0x3992a9 (0xa402 bytes) 000256[1] 1713486028.217858 mapwrite 0x9524d5 thru 0x95f08f (0xcbbb bytes) 000257[1] 1713486028.279086 write 0x87fc4b thru 0x885664 (0x5a1a bytes) 000258[0] 1713486028.282419 mapwrite 0x84f6b0 thru 0x85bb9d (0xc4ee bytes) 000259[1] 1713486028.320072 write 0x23fe60 thru 0x24d7c1 (0xd962 bytes) 000260[1] 1713486028.327156 mapread 0x47f344 thru 0x4897d8 (0xa495 bytes) 000262[1] 1713486028.332472 write 0x066ef1 thru 0x06830d (0x141d bytes) 000264[0] 1713486028.334675 write 0x9392fd thru 0x941127 (0x7e2b bytes) 000265[1] 1713486028.343425 trunc from 0x96cbe5 to 0x4a6a45 (0x4c619f bytes) 000266[0] 1713486028.356050 write 0x5da72f thru 0x5ddde0 (0x36b2 bytes) 000267[0] 1713486028.361071 mapread 0x120596 thru 0x12fc67 (0xf6d2 bytes) 000268[0] 1713486028.379690 mapwrite 0x4a664e thru 0x4b654e (0xff01 bytes) 000270[1] 1713486028.415869 read 0x0c87d7 thru 0x0d6daf (0xe5d9 bytes) 000271[1] 1713486028.430107 mapread 0x272c24 thru 0x2759fc (0x2dd9 bytes) 000275[1] 1713486028.434672 mapread 0x46848d thru 0x46f3ad (0x6f21 bytes) 000277[1] 1713486028.448518 read 0x52bb95 thru 0x533629 (0x7a95 bytes) 000278[1] 1713486028.448749 mapwrite 0x40dbd2 thru 0x418ea0 (0xb2cf bytes) 000280[0] 1713486028.491713 write 0x6f75a1 thru 0x6f7ae9 (0x549 bytes) 000281[0] 1713486028.495561 write 0x443ec4 thru 0x44ce6f (0x8fac bytes) 000284[0] 1713486028.501065 write 0x6cc29f thru 0x6d77a9 (0xb50b bytes) 000287[1] 1713486028.504075 trunc from 0x6f7aea to 0x2602d3 (0x497816 bytes) 000294[1] 1713486028.526869 mapread 0x1c6aae thru 0x1cdcea (0x723d bytes) 000295[1] 1713486028.536284 write 0x475657 thru 0x4828f6 (0xd2a0 bytes) 000296[1] 1713486028.537239 trunc from 0x4828f7 to 0x4e2471 (0x5fb7b bytes) 000300[0] 1713486028.551235 mapwrite 0x5c0d8d thru 0x5cdb01 (0xcd75 bytes) 000303[1] 1713486028.590296 mapwrite 0x672148 thru 0x6733e9 (0x12a2 bytes) 000304[0] 1713486028.613901 write 0x3ac06d thru 0x3af1fa (0x318e bytes) 000305[1] 1713486028.620236 write 0x50bddd thru 0x51a207 (0xe42b bytes) 000306[0] 1713486028.626095 mapread 0x14a490 thru 0x158f14 (0xea85 bytes) 000307[0] 1713486028.643938 write 0x2b6f07 thru 0x2bd0b1 (0x61ab bytes) 000313[0] 1713486028.663664 mapwrite 0x375078 thru 0x37d14d (0x80d6 bytes) 000314[1] 1713486028.689077 write 0x7edc62 thru 0x7f9f39 (0xc2d8 bytes) 000315[0] 1713486028.692883 mapread 0x713082 thru 0x71fb75 (0xcaf4 bytes) 000316[1] 1713486028.712928 read 0x781c12 thru 0x783f85 (0x2374 bytes) 000319[0] 1713486028.719699 write 0x6d540a thru 0x6d88b0 (0x34a7 bytes) 000320[1] 1713486028.723688 mapread 0x637ae6 thru 0x637ea6 (0x3c1 bytes) 000321[0] 1713486028.731842 trunc from 0x7f9f3a to 0x2af0d5 (0x54ae64 bytes) 000323[0] 1713486028.741583 read 0x1f9816 thru 0x202d3c (0x9527 bytes) truncating to largest ever: 0x9fb66a 000324[0] 1713486028.755239 trunc from 0x2af0d6 to 0x9fb669 (0x74c594 bytes) 000328[0] 1713486028.764658 mapread 0x7f4c07 thru 0x7f897a (0x3d74 bytes) 000329[0] 1713486028.785118 mapread 0x7cb4cb thru 0x7cc915 (0x144b bytes) 000331[1] 1713486028.785611 trunc from 0x9fb66a to 0x4a1d83 (0x5598e6 bytes) 000332[1] 1713486028.798753 read 0x477ca8 thru 0x47a138 (0x2491 bytes) 000335[1] 1713486028.814241 read 0x26d436 thru 0x273d5e (0x6929 bytes) 000340[1] 1713486028.826578 trunc from 0x4a1d84 to 0x31df50 (0x183e33 bytes) 000342[1] 1713486028.832946 read 0x0e059a thru 0x0ef045 (0xeaac bytes) 000344[1] 1713486028.841092 trunc from 0x31df51 to 0x6d6816 (0x3b88c6 bytes) 000345[0] 1713486028.846591 write 0x357ffc thru 0x359592 (0x1597 bytes) 000346[1] 1713486028.857201 read 0x520547 thru 0x5301dd (0xfc97 bytes) 000347[1] 1713486028.873438 mapread 0x0a841a thru 0x0b36b9 (0xb2a0 bytes) 000348[0] 1713486028.888839 trunc from 0x6d6817 to 0x3cd985 (0x308e91 bytes) 000349[1] 1713486028.903145 write 0x12ba41 thru 0x136d34 (0xb2f4 bytes) 000351[0] 1713486028.914495 read 0x120475 thru 0x1272d8 (0x6e64 bytes) 000352[1] 1713486028.933088 read 0x1d38e0 thru 0x1e1d0e (0xe42f bytes) 000353[1] 1713486028.951451 read 0x2ca901 thru 0x2cf487 (0x4b87 bytes) 000354[1] 1713486028.954582 mapwrite 0x83b98c thru 0x83f290 (0x3905 bytes) 000361[0] 1713486029.018866 mapwrite 0x0fd51e thru 0x0fe4fc (0xfdf bytes) 000362[1] 1713486029.044824 mapread 0x04e77e thru 0x04ffba (0x183d bytes) 000363[0] 1713486029.053126 trunc from 0x83f291 to 0x4d40bd (0x36b1d3 bytes) 000364[1] 1713486029.066761 trunc from 0x4d40be to 0x092a0e (0x4416af bytes) 000365[0] 1713486029.075579 read 0x07cd32 thru 0x08a912 (0xdbe1 bytes) 000366[0] 1713486029.082468 mapread 0x032124 thru 0x03e53c (0xc419 bytes) 000367[1] 1713486029.087052 trunc from 0x092a0f to 0x540ff5 (0x4ae5e7 bytes) 000368[0] 1713486029.096431 read 0x4e2181 thru 0x4f1021 (0xeea1 bytes) 000369[0] 1713486029.112546 trunc from 0x540ff6 to 0x17e64d (0x3c29a8 bytes) 000371[0] 1713486029.124928 mapwrite 0x7fdfd0 thru 0x800ba8 (0x2bd9 bytes) 000374[0] 1713486029.177244 mapread 0x38c6e1 thru 0x395896 (0x91b6 bytes) 000376[1] 1713486029.183534 trunc from 0x800ba9 to 0x5ba7ef (0x2463b9 bytes) 000379[1] 1713486029.201293 write 0x072460 thru 0x0733eb (0xf8c bytes) 000380[1] 1713486029.203406 read 0x42ba52 thru 0x4306e1 (0x4c90 bytes) 000383[1] 1713486029.223062 mapwrite 0x732df4 thru 0x73ce8b (0xa098 bytes) 000389[0] 1713486029.260078 read 0x48f14b thru 0x499cb1 (0xab67 bytes) 000390[1] 1713486029.275246 read 0x2c7927 thru 0x2d10cb (0x97a5 bytes) 000392[0] 1713486029.281089 write 0x7c390b thru 0x7cd882 (0x9f78 bytes) 000393[0] 1713486029.285855 mapwrite 0x4e2c84 thru 0x4e4f52 (0x22cf bytes) 000396[1] 1713486029.309459 mapwrite 0x3c71b1 thru 0x3d0d50 (0x9ba0 bytes) 000398[0] 1713486029.327165 write 0x620f51 thru 0x623509 (0x25b9 bytes) 000401[0] 1713486029.337605 read 0x476da6 thru 0x47b8d6 (0x4b31 bytes) 000403[1] 1713486029.349137 write 0x41cd88 thru 0x420995 (0x3c0e bytes) 000406[1] 1713486029.360259 mapwrite 0x4d48c9 thru 0x4d6cb3 (0x23eb bytes) 000411[1] 1713486029.391577 mapwrite 0x7b31ce thru 0x7b6e3c (0x3c6f bytes) 000412[0] 1713486029.405824 write 0x935853 thru 0x944252 (0xea00 bytes) 000415[1] 1713486029.418997 write 0x2c8966 thru 0x2d30c2 (0xa75d bytes) 000418[0] 1713486029.428289 read 0x5add89 thru 0x5bb351 (0xd5c9 bytes) 000419[0] 1713486029.446824 trunc from 0x944253 to 0x54ef5d (0x3f52f5 bytes) 000426[0] 1713486029.468995 write 0x7ce831 thru 0x7d815f (0x992f bytes) 000429[1] 1713486029.478946 read 0x67443d thru 0x67d89c (0x9460 bytes) 000431[0] 1713486029.493011 write 0x570fc2 thru 0x579907 (0x8946 bytes) 000432[0] 1713486029.514600 mapread 0x4c16c4 thru 0x4c98d2 (0x820f bytes) 000433[1] 1713486029.517882 write 0x0aab0a thru 0x0ac904 (0x1dfb bytes) 000434[0] 1713486029.536120 read 0x47555b thru 0x47a076 (0x4b1c bytes) 000435[1] 1713486029.539098 write 0x3e5d3b thru 0x3e7f3d (0x2203 bytes) 000436[0] 1713486029.542881 read 0x2e70aa thru 0x2eb057 (0x3fae bytes) 000438[0] 1713486029.577109 mapread 0x49ccc4 thru 0x4ab732 (0xea6f bytes) 000441[1] 1713486029.583896 read 0x318c3e thru 0x322f22 (0xa2e5 bytes) 000443[0] 1713486029.603672 trunc from 0x7d8160 to 0x16d571 (0x66abee bytes) 000444[1] 1713486029.618809 mapread 0x102f43 thru 0x10f7fc (0xc8ba bytes) 000448[0] 1713486029.643734 write 0x5b3749 thru 0x5bf36d (0xbc25 bytes) 000450[0] 1713486029.648265 mapread 0x3ea528 thru 0x3f09af (0x6488 bytes) 000451[0] 1713486029.657870 read 0x13efa9 thru 0x1403da (0x1432 bytes) 000456[1] 1713486029.671496 mapread 0x12adaa thru 0x134ad2 (0x9d29 bytes) 000458[0] 1713486029.686747 mapwrite 0x076dba thru 0x08069b (0x98e2 bytes) 000459[0] 1713486029.724728 mapwrite 0x38a529 thru 0x39762c (0xd104 bytes) 000460[1] 1713486029.754864 read 0x15283e thru 0x15be47 (0x960a bytes) 000462[0] 1713486029.776471 read 0x3d5caa thru 0x3de6f2 (0x8a49 bytes) 000463[1] 1713486029.796695 trunc from 0x5bf36e to 0x5df252 (0x1fee5 bytes) 000464[1] 1713486029.814005 write 0x96fdeb thru 0x9729bb (0x2bd1 bytes) 000467[0] 1713486029.824349 trunc from 0x9729bc to 0x380f56 (0x5f1a65 bytes) 000469[0] 1713486029.836157 write 0x004898 thru 0x009e99 (0x5602 bytes) 000472[1] 1713486029.841971 trunc from 0x380f57 to 0x7c894a (0x4479f4 bytes) 000477[1] 1713486029.858672 trunc from 0x7c894b to 0x5824ec (0x24645e bytes) 000479[0] 1713486029.866412 mapwrite 0x0ee659 thru 0x0fd651 (0xeff9 bytes) 000480[1] 1713486029.906800 write 0x545896 thru 0x54ccec (0x7457 bytes) 000481[0] 1713486029.912343 read 0x3b65ef thru 0x3bfa31 (0x9443 bytes) 000482[1] 1713486029.915678 mapread 0x45706f thru 0x46366e (0xc600 bytes) 000483[1] 1713486029.933084 write 0x71f0bf thru 0x729455 (0xa397 bytes) 000486[0] 1713486029.942439 mapwrite 0x8b97ec thru 0x8bfb3e (0x6353 bytes) 000491[1] 1713486030.001172 read 0x1a55b1 thru 0x1b32b5 (0xdd05 bytes) 000492[1] 1713486030.022544 mapread 0x6652a4 thru 0x66726d (0x1fca bytes) 000493[0] 1713486030.047426 write 0x0de234 thru 0x0e2251 (0x401e bytes) 000496[1] 1713486030.061496 mapread 0x40c579 thru 0x4168cf (0xa357 bytes) 000500[0] 1713486030.068490 read 0x422a4d thru 0x42b487 (0x8a3b bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 6030 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x20b9ae 000001[1] 1713486030.298214 trunc from 00000000 to 0x20b9ad (0x20b9ae bytes) 000002[0] 1713486030.303638 write 0x704000 thru 0x70e896 (0xa897 bytes) 000004[0] 1713486030.308096 mapread 0x0be000 thru 0x0be561 (0x562 bytes) 000005[1] 1713486030.309635 mapread 0x164000 thru 0x16947b (0x547c bytes) truncating to largest ever: 0x3bf6c3 000006[0] 1713486030.319861 trunc from 0x70e897 to 0x3bf6c2 (0x34f1d4 bytes) 000010[1] 1713486030.334527 write_OD 0x579000 thru 0x587fff (0xf000 bytes) 000012[0] 1713486030.349906 write 0x745000 thru 0x74ee1a (0x9e1b bytes) 000016[0] 1713486030.355190 write 0x230000 thru 0x2306f3 (0x6f4 bytes) truncating to largest ever: 0x4c708b 000017[1] 1713486030.356594 trunc from 0x74ee1b to 0x4c708a (0x287d90 bytes) 000018[1] 1713486030.364505 read_OD 0x44d000 thru 0x451fff (0x5000 bytes) 000019[1] 1713486030.368196 write_OD 0x999000 thru 0x99dfff (0x5000 bytes) 000020[1] 1713486030.384890 mapwrite 0x166000 thru 0x171a9f (0xbaa0 bytes) 000021[1] 1713486030.415449 write_OD 0x107000 thru 0x114fff (0xe000 bytes) 000022[1] 1713486030.429273 read_OD 0x8e3000 thru 0x8ebfff (0x9000 bytes) 000023[1] 1713486030.431300 mapwrite 0x297000 thru 0x2a58b5 (0xe8b6 bytes) 000024[0] 1713486030.481227 mapwrite 0x371000 thru 0x3751d6 (0x41d7 bytes) 000025[1] 1713486030.509464 mapread 0x68e000 thru 0x6969dc (0x89dd bytes) 000030[1] 1713486030.522583 read_OD 0x64d000 thru 0x651fff (0x5000 bytes) 000034[0] 1713486030.524864 mapread 0x8cb000 thru 0x8d8f21 (0xdf22 bytes) truncating to largest ever: 0x5a77b1 000036[0] 1713486030.530258 trunc from 0x99e000 to 0x5a77b0 (0x3f684f bytes) 000037[1] 1713486030.537689 trunc from 0x5a77b1 to 0x49d505 (0x10a2ab bytes) 000038[1] 1713486030.544222 mapread 0x423000 thru 0x428a92 (0x5a93 bytes) 000040[0] 1713486030.560100 write 0x1f7000 thru 0x1feeac (0x7ead bytes) 000043[1] 1713486030.571392 read_OD 0x33b000 thru 0x33ffff (0x5000 bytes) 000046[1] 1713486030.578589 mapread 0x27a000 thru 0x2848e0 (0xa8e1 bytes) 000047[0] 1713486030.593044 write 0x8ca000 thru 0x8d9d6a (0xfd6b bytes) 000048[1] 1713486030.602149 mapwrite 0x7b9000 thru 0x7c5291 (0xc292 bytes) 000050[0] 1713486030.633596 mapread 0x1d7000 thru 0x1df82b (0x882c bytes) 000051[0] 1713486030.643106 trunc from 0x8d9d6b to 0x214efb (0x6c4e6f bytes) 000053[1] 1713486030.648085 mapread 0x1b7000 thru 0x1bc917 (0x5918 bytes) 000064[1] 1713486030.658573 mapread 0x0e6000 thru 0x0f3a6f (0xda70 bytes) 000065[1] 1713486030.670371 read_OD 0x1d1000 thru 0x1d1fff (0x1000 bytes) 000070[0] 1713486030.671802 mapwrite 0x83d000 thru 0x83e0e9 (0x10ea bytes) 000071[0] 1713486030.694611 trunc from 0x83e0ea to 0x22059a (0x61db4f bytes) 000075[0] 1713486030.706938 mapwrite 0x8fb000 thru 0x90a1c7 (0xf1c8 bytes) 000077[1] 1713486030.731965 mapwrite 0x158000 thru 0x162444 (0xa445 bytes) 000078[1] 1713486030.771175 mapwrite 0x2dd000 thru 0x2ec20b (0xf20c bytes) 000079[1] 1713486030.789410 mapread 0x74d000 thru 0x74f4ba (0x24bb bytes) 000080[1] 1713486030.791960 mapwrite 0x3e6000 thru 0x3eaed7 (0x4ed8 bytes) 000081[0] 1713486030.803064 trunc from 0x90a1c8 to 0x0bf71d (0x84aaaa bytes) 000086[1] 1713486030.819497 write_OD 0x65c000 thru 0x668fff (0xd000 bytes) 000087[0] 1713486030.834831 read 0x22f000 thru 0x23bc8b (0xcc8c bytes) 000088[0] 1713486030.836987 trunc from 0x669000 to 0x374546 (0x2f4ab9 bytes) 000089[0] 1713486030.842494 write 0x2f7000 thru 0x301ec0 (0xaec1 bytes) 000090[0] 1713486030.847816 trunc from 0x374547 to 0x0a607e (0x2ce4c8 bytes) 000091[1] 1713486030.852016 read_OD 0x09d000 thru 0x0a4fff (0x8000 bytes) truncating to largest ever: 0x6ec65f 000092[0] 1713486030.857346 trunc from 0x0a607f to 0x6ec65e (0x6465e0 bytes) 000093[1] 1713486030.862998 read_OD 0x58a000 thru 0x58cfff (0x3000 bytes) 000096[1] 1713486030.869402 trunc from 0x6ec65f to 0x379796 (0x372ec8 bytes) 000101[0] 1713486030.884569 write 0x759000 thru 0x765c27 (0xcc28 bytes) 000103[1] 1713486030.890837 mapwrite 0x768000 thru 0x76b6a9 (0x36aa bytes) 000104[1] 1713486030.914388 mapwrite 0x580000 thru 0x58ac60 (0xac61 bytes) 000105[1] 1713486030.938570 read_OD 0x4e2000 thru 0x4effff (0xe000 bytes) 000107[1] 1713486030.943059 read_OD 0x762000 thru 0x769fff (0x8000 bytes) 000108[1] 1713486030.945543 trunc from 0x76b6aa to 0x6b3d47 (0xb7962 bytes) 000113[0] 1713486030.958082 mapread 0x2e0000 thru 0x2ea240 (0xa241 bytes) 000122[0] 1713486030.969846 trunc from 0x6b3d48 to 0x2945c2 (0x41f785 bytes) 000125[0] 1713486030.982763 write 0x4f8000 thru 0x4fd1af (0x51b0 bytes) 000126[0] 1713486030.986540 mapread 0x4ba000 thru 0x4bea96 (0x4a97 bytes) truncating to largest ever: 0x9f52e5 000127[1] 1713486030.993267 trunc from 0x4fd1b0 to 0x9f52e4 (0x4f8135 bytes) 000128[0] 1713486031.000925 mapwrite 0x44b000 thru 0x4577bf (0xc7c0 bytes) 000131[0] 1713486031.027590 mapwrite 0x44f000 thru 0x45a322 (0xb323 bytes) 000137[1] 1713486031.050005 mapread 0x1c9000 thru 0x1d2275 (0x9276 bytes) 000138[0] 1713486031.088761 write 0x311000 thru 0x312358 (0x1359 bytes) 000139[1] 1713486031.100625 mapread 0x6d0000 thru 0x6d2df1 (0x2df2 bytes) 000143[1] 1713486031.110240 write_OD 0x16b000 thru 0x170fff (0x6000 bytes) 000144[0] 1713486031.125553 mapwrite 0x039000 thru 0x044e3c (0xbe3d bytes) 000145[1] 1713486031.149675 mapwrite 0x3f7000 thru 0x3fa705 (0x3706 bytes) 000146[0] 1713486031.172401 trunc from 0x9f52e5 to 0x70910d (0x2ec1d7 bytes) 000149[0] 1713486031.185402 trunc from 0x70910e to 0x3e7880 (0x32188d bytes) 000150[1] 1713486031.191054 mapwrite 0x03e000 thru 0x03f325 (0x1326 bytes) 000151[0] 1713486031.216171 mapread 0x388000 thru 0x395140 (0xd141 bytes) 000153[0] 1713486031.227678 read 0x22f000 thru 0x23cf54 (0xdf55 bytes) 000156[0] 1713486031.245150 read 0x0ff000 thru 0x10730d (0x830e bytes) 000157[0] 1713486031.245890 mapwrite 0x2be000 thru 0x2c2c8c (0x4c8d bytes) 000158[0] 1713486031.290757 mapread 0x326000 thru 0x32b675 (0x5676 bytes) 000160[1] 1713486031.297008 mapwrite 0x0b1000 thru 0x0b2df6 (0x1df7 bytes) 000161[0] 1713486031.318630 mapread 0x339000 thru 0x33d6f1 (0x46f2 bytes) 000162[0] 1713486031.327651 trunc from 0x3e7881 to 0x1c4fee (0x222892 bytes) 000164[1] 1713486031.334270 read_OD 0x009000 thru 0x00ffff (0x7000 bytes) 000165[0] 1713486031.341256 mapwrite 0x447000 thru 0x44cc0d (0x5c0e bytes) 000166[0] 1713486031.373334 write 0x11c000 thru 0x11c2a3 (0x2a4 bytes) 000167[1] 1713486031.378264 trunc from 0x44cc0e to 0x5a8828 (0x15bc1b bytes) 000168[0] 1713486031.385424 write 0x6ae000 thru 0x6baf55 (0xcf56 bytes) 000171[1] 1713486031.395643 trunc from 0x6baf56 to 0x603e8a (0xb70cb bytes) 000172[1] 1713486031.403703 read_OD 0x598000 thru 0x59cfff (0x5000 bytes) 000174[1] 1713486031.408868 mapwrite 0x649000 thru 0x64b4e1 (0x24e2 bytes) 000175[0] 1713486031.451041 mapwrite 0x763000 thru 0x771c03 (0xec04 bytes) 000176[0] 1713486031.487906 read 0x5e3000 thru 0x5ed29c (0xa29d bytes) 000177[1] 1713486031.491950 mapwrite 0x963000 thru 0x96841e (0x541f bytes) 000182[0] 1713486031.534947 write 0x245000 thru 0x248d14 (0x3d15 bytes) 000191[0] 1713486031.561133 mapwrite 0x630000 thru 0x6366db (0x66dc bytes) 000193[0] 1713486031.600221 write 0x8a3000 thru 0x8ae507 (0xb508 bytes) 000194[0] 1713486031.600954 mapwrite 0x828000 thru 0x83195d (0x995e bytes) 000196[1] 1713486031.618654 mapwrite 0x588000 thru 0x596e1f (0xee20 bytes) 000197[1] 1713486031.658743 write_OD 0x51b000 thru 0x527fff (0xd000 bytes) 000200[0] 1713486031.674527 mapread 0x71b000 thru 0x71ba0b (0xa0c bytes) 000201[1] 1713486031.675878 read_OD 0x0f8000 thru 0x0fafff (0x3000 bytes) 000202[0] 1713486031.677026 mapread 0x207000 thru 0x207740 (0x741 bytes) 000204[0] 1713486031.678153 mapread 0x52c000 thru 0x52e4c1 (0x24c2 bytes) 000207[0] 1713486031.680925 trunc from 0x96841f to 0x087064 (0x8e13ba bytes) 000208[1] 1713486031.686475 mapwrite 0x56f000 thru 0x57a517 (0xb518 bytes) 000209[1] 1713486031.725509 mapread 0x4b7000 thru 0x4bb9e1 (0x49e2 bytes) 000211[0] 1713486031.729645 mapread 0x56c000 thru 0x56ca7b (0xa7c bytes) 000214[0] 1713486031.738684 mapread 0x0d5000 thru 0x0e4b39 (0xfb3a bytes) 000216[0] 1713486031.753398 read 0x4d2000 thru 0x4d86a2 (0x66a3 bytes) 000218[1] 1713486031.771280 read_OD 0x3fa000 thru 0x3fffff (0x6000 bytes) 000219[1] 1713486031.773152 mapwrite 0x218000 thru 0x220a9a (0x8a9b bytes) 000229[1] 1713486031.818519 trunc from 0x57a518 to 0x70f043 (0x194b2c bytes) 000230[0] 1713486031.822775 write 0x69a000 thru 0x6a506a (0xb06b bytes) 000231[1] 1713486031.835232 mapwrite 0x34c000 thru 0x3543db (0x83dc bytes) 000232[1] 1713486031.874580 write_OD 0x5ed000 thru 0x5f2fff (0x6000 bytes) 000234[0] 1713486031.890503 mapread 0x0e6000 thru 0x0eca27 (0x6a28 bytes) 000236[0] 1713486031.901114 mapread 0x0e0000 thru 0x0e3975 (0x3976 bytes) 000239[0] 1713486031.905192 mapwrite 0x073000 thru 0x074ac1 (0x1ac2 bytes) 000241[1] 1713486031.924211 write_OD 0x663000 thru 0x666fff (0x4000 bytes) 000242[1] 1713486031.936659 mapwrite 0x74a000 thru 0x7567cf (0xc7d0 bytes) 000247[1] 1713486031.964708 mapwrite 0x876000 thru 0x8841b0 (0xe1b1 bytes) 000248[0] 1713486031.989882 read 0x1c0000 thru 0x1c106c (0x106d bytes) 000249[1] 1713486032.016492 write_OD 0x803000 thru 0x804fff (0x2000 bytes) 000251[1] 1713486032.035886 mapread 0x7a1000 thru 0x7ae0c4 (0xd0c5 bytes) 000252[0] 1713486032.049822 write 0x980000 thru 0x98ad37 (0xad38 bytes) 000253[1] 1713486032.052690 mapwrite 0x51b000 thru 0x5203d3 (0x53d4 bytes) 000254[0] 1713486032.088008 mapwrite 0x348000 thru 0x34b229 (0x322a bytes) 000259[1] 1713486032.118603 trunc from 0x98ad38 to 0x636dd0 (0x353f67 bytes) 000263[0] 1713486032.136898 write 0x28d000 thru 0x28ddeb (0xdec bytes) 000267[0] 1713486032.149156 mapread 0x4ec000 thru 0x4efa66 (0x3a67 bytes) 000268[0] 1713486032.155971 mapwrite 0x688000 thru 0x6904df (0x84e0 bytes) 000269[1] 1713486032.184859 mapread 0x5ae000 thru 0x5b528e (0x728f bytes) 000270[0] 1713486032.197495 mapwrite 0x607000 thru 0x61619c (0xf19d bytes) 000271[1] 1713486032.228267 mapwrite 0x2f7000 thru 0x2fd359 (0x635a bytes) 000275[0] 1713486032.260590 trunc from 0x6904e0 to 0x470f81 (0x21f55e bytes) 000276[0] 1713486032.266207 trunc from 0x470f82 to 0x916e0c (0x4a5e8b bytes) 000279[0] 1713486032.276597 mapwrite 0x7a1000 thru 0x7aae51 (0x9e52 bytes) 000280[0] 1713486032.302455 mapwrite 0x05f000 thru 0x06e953 (0xf954 bytes) 000281[0] 1713486032.335962 read 0x6ab000 thru 0x6b929a (0xe29b bytes) 000283[1] 1713486032.369740 mapwrite 0x1df000 thru 0x1ed3e4 (0xe3e5 bytes) 000284[0] 1713486032.410911 trunc from 0x916e0d to 0x90f41e (0x79ee bytes) 000289[1] 1713486032.420575 mapwrite 0x00e000 thru 0x010c26 (0x2c27 bytes) 000290[1] 1713486032.436980 mapwrite 0x3d7000 thru 0x3dd566 (0x6567 bytes) 000291[1] 1713486032.458441 trunc from 0x90f41f to 0x4fb88b (0x413b93 bytes) 000292[1] 1713486032.462472 read_OD 0x419000 thru 0x41cfff (0x4000 bytes) 000296[1] 1713486032.468485 trunc from 0x4fb88c to 0x1d48ec (0x326f9f bytes) 000299[1] 1713486032.484249 mapwrite 0x141000 thru 0x145270 (0x4271 bytes) 000300[1] 1713486032.509513 mapread 0x192000 thru 0x19fefd (0xdefe bytes) 000301[0] 1713486032.512552 write 0x819000 thru 0x825bea (0xcbeb bytes) 000303[0] 1713486032.523169 trunc from 0x825beb to 0x0e7885 (0x73e365 bytes) 000304[1] 1713486032.530069 write_OD 0x18b000 thru 0x18dfff (0x3000 bytes) 000305[1] 1713486032.543024 trunc from 0x18e000 to 0x1a18e5 (0x138e6 bytes) 000306[0] 1713486032.550134 mapread 0x149000 thru 0x14cf3d (0x3f3e bytes) 000307[0] 1713486032.559002 read 0x154000 thru 0x15616f (0x2170 bytes) 000314[1] 1713486032.568758 trunc from 0x1a18e6 to 0x286b59 (0xe5274 bytes) 000315[1] 1713486032.576442 write_OD 0x6fc000 thru 0x703fff (0x8000 bytes) 000317[1] 1713486032.591528 write_OD 0x2a1000 thru 0x2abfff (0xb000 bytes) 000318[1] 1713486032.606824 write_OD 0x890000 thru 0x898fff (0x9000 bytes) 000322[0] 1713486032.624438 mapwrite 0x780000 thru 0x7812eb (0x12ec bytes) 000324[0] 1713486032.662719 write 0x0fd000 thru 0x102bd7 (0x5bd8 bytes) 000325[1] 1713486032.666881 read_OD 0x4f6000 thru 0x4fbfff (0x6000 bytes) 000326[1] 1713486032.678499 mapwrite 0x4c1000 thru 0x4c1f24 (0xf25 bytes) 000328[0] 1713486032.720136 mapwrite 0x55b000 thru 0x55ffec (0x4fed bytes) 000330[1] 1713486032.750298 trunc from 0x899000 to 0x4d4d9a (0x3c4265 bytes) 000334[1] 1713486032.764010 mapwrite 0x7a0000 thru 0x7a21e5 (0x21e6 bytes) 000336[1] 1713486032.789326 write_OD 0x332000 thru 0x338fff (0x7000 bytes) 000337[0] 1713486032.803151 read 0x4e7000 thru 0x4f11b2 (0xa1b3 bytes) 000339[0] 1713486032.828544 read 0x1b1000 thru 0x1b6146 (0x5147 bytes) 000344[1] 1713486032.829385 mapread 0x304000 thru 0x308b3a (0x4b3b bytes) 000346[0] 1713486032.834383 mapread 0x788000 thru 0x789e73 (0x1e74 bytes) 000347[1] 1713486032.834690 mapwrite 0x2a0000 thru 0x2af1b9 (0xf1ba bytes) 000349[0] 1713486032.874382 mapwrite 0x6ca000 thru 0x6d33c6 (0x93c7 bytes) 000350[0] 1713486032.914711 mapwrite 0x7c8000 thru 0x7ca05f (0x2060 bytes) 000351[1] 1713486032.934087 mapwrite 0x376000 thru 0x376e98 (0xe99 bytes) 000352[1] 1713486032.956487 trunc from 0x7ca060 to 0x59c2c1 (0x22dd9e bytes) 000355[0] 1713486032.968744 trunc from 0x59c2c2 to 0x57ebf0 (0x1d6d1 bytes) 000356[0] 1713486032.975618 read 0x413000 thru 0x419d39 (0x6d3a bytes) 000357[0] 1713486032.994462 mapwrite 0x966000 thru 0x96fd80 (0x9d81 bytes) 000358[1] 1713486033.027388 trunc from 0x96fd81 to 0x55ee07 (0x410f79 bytes) 000359[0] 1713486033.043926 mapread 0x10f000 thru 0x11ace4 (0xbce5 bytes) 000362[0] 1713486033.062667 mapread 0x2ba000 thru 0x2ba39f (0x3a0 bytes) 000367[0] 1713486033.065155 mapread 0x23d000 thru 0x24c5fe (0xf5ff bytes) 000369[1] 1713486033.082712 mapread 0x30c000 thru 0x3126e4 (0x66e5 bytes) 000373[1] 1713486033.090052 mapwrite 0x6ee000 thru 0x6f39a8 (0x59a9 bytes) 000374[1] 1713486033.116699 write_OD 0x6b7000 thru 0x6befff (0x8000 bytes) 000376[1] 1713486033.130982 mapwrite 0x8af000 thru 0x8b06af (0x16b0 bytes) 000377[1] 1713486033.150367 write_OD 0x8eb000 thru 0x8f6fff (0xc000 bytes) 000378[1] 1713486033.163390 mapwrite 0x1cd000 thru 0x1cfc32 (0x2c33 bytes) 000379[1] 1713486033.187334 read_OD 0x45b000 thru 0x461fff (0x7000 bytes) 000380[0] 1713486033.191096 mapwrite 0x9ca000 thru 0x9d08b1 (0x68b2 bytes) 000381[0] 1713486033.212576 write 0x59b000 thru 0x59e8f0 (0x38f1 bytes) 000383[0] 1713486033.214399 mapread 0x352000 thru 0x35f87b (0xd87c bytes) 000388[0] 1713486033.233488 mapread 0x5e7000 thru 0x5ee526 (0x7527 bytes) 000392[1] 1713486033.249303 mapread 0x066000 thru 0x066214 (0x215 bytes) 000393[1] 1713486033.282330 write_OD 0x8e7000 thru 0x8f0fff (0xa000 bytes) 000394[1] 1713486033.303282 trunc from 0x9d08b2 to 0x3b14d8 (0x61f3d9 bytes) 000396[1] 1713486033.312404 trunc from 0x3b14d9 to 0x92f44f (0x57df77 bytes) 000398[0] 1713486033.321066 mapwrite 0x882000 thru 0x88c2a0 (0xa2a1 bytes) 000399[0] 1713486033.352220 read 0x2cc000 thru 0x2dba23 (0xfa24 bytes) 000407[1] 1713486033.388424 write_OD 0x0bf000 thru 0x0c9fff (0xb000 bytes) 000409[1] 1713486033.410906 mapread 0x1a0000 thru 0x1af7f9 (0xf7fa bytes) 000411[1] 1713486033.425847 mapread 0x26e000 thru 0x2748e5 (0x68e6 bytes) 000412[1] 1713486033.433012 write_OD 0x408000 thru 0x40afff (0x3000 bytes) 000414[1] 1713486033.446534 write_OD 0x3ed000 thru 0x3effff (0x3000 bytes) 000415[1] 1713486033.460553 mapwrite 0x2de000 thru 0x2e9989 (0xb98a bytes) 000416[1] 1713486033.487292 mapread 0x6ba000 thru 0x6c4e22 (0xae23 bytes) 000419[0] 1713486033.503780 trunc from 0x92f450 to 0x0f0a82 (0x83e9cd bytes) 000422[0] 1713486033.513884 mapwrite 0x46e000 thru 0x4703f7 (0x23f8 bytes) 000423[1] 1713486033.534490 read_OD 0x35c000 thru 0x367fff (0xc000 bytes) 000425[1] 1713486033.539818 write_OD 0x86e000 thru 0x87afff (0xd000 bytes) 000427[1] 1713486033.554588 trunc from 0x87b000 to 0x82a4e2 (0x50b1d bytes) 000428[0] 1713486033.562464 mapread 0x141000 thru 0x146f42 (0x5f43 bytes) 000429[1] 1713486033.572878 write_OD 0x86e000 thru 0x875fff (0x8000 bytes) 000430[1] 1713486033.585580 mapwrite 0x0cf000 thru 0x0d614b (0x714c bytes) 000431[0] 1713486033.620218 write 0x9e0000 thru 0x9ebe53 (0xbe54 bytes) 000432[0] 1713486033.628899 mapwrite 0x20f000 thru 0x21e2b9 (0xf2ba bytes) 000435[1] 1713486033.659589 write_OD 0x4bd000 thru 0x4c9fff (0xd000 bytes) 000439[0] 1713486033.674068 mapread 0x59f000 thru 0x5a0486 (0x1487 bytes) 000443[0] 1713486033.676835 mapwrite 0x6f0000 thru 0x6fcf4f (0xcf50 bytes) 000445[0] 1713486033.706150 trunc from 0x9ebe54 to 0x849495 (0x1a29be bytes) 000447[1] 1713486033.710831 trunc from 0x849496 to 0x1c7ae6 (0x6819af bytes) 000448[1] 1713486033.719698 read_OD 0x175000 thru 0x182fff (0xe000 bytes) 000449[0] 1713486033.723443 mapread 0x0bd000 thru 0x0c3124 (0x6125 bytes) 000451[1] 1713486033.730246 trunc from 0x1c7ae7 to 0x5cd065 (0x40557f bytes) 000454[0] 1713486033.744473 mapread 0x51f000 thru 0x524b83 (0x5b84 bytes) 000456[1] 1713486033.754876 write_OD 0x91a000 thru 0x922fff (0x9000 bytes) 000458[1] 1713486033.769021 read_OD 0x682000 thru 0x68efff (0xd000 bytes) 000461[0] 1713486033.772088 mapread 0x313000 thru 0x31c4e4 (0x94e5 bytes) 000467[0] 1713486033.781820 write 0x253000 thru 0x25b09b (0x809c bytes) 000468[0] 1713486033.787348 read 0x052000 thru 0x05934a (0x734b bytes) 000469[0] 1713486033.815587 trunc from 0x923000 to 0x9d9b0c (0xb6b0d bytes) 000470[1] 1713486033.819391 write_OD 0x4fd000 thru 0x4fffff (0x3000 bytes) 000472[0] 1713486033.841197 mapread 0x0d9000 thru 0x0e2339 (0x933a bytes) 000473[1] 1713486033.855014 trunc from 0x9d9b0d to 0x6ff6fa (0x2da412 bytes) 000479[1] 1713486033.875525 trunc from 0x6ff6fb to 0x4a4826 (0x25aed4 bytes) 000481[1] 1713486033.882982 trunc from 0x4a4827 to 0x0d8cfe (0x3cbb28 bytes) 000482[1] 1713486033.886591 write_OD 0x9a9000 thru 0x9b5fff (0xd000 bytes) 000484[1] 1713486033.903416 mapread 0x0de000 thru 0x0eaf5e (0xcf5f bytes) 000485[0] 1713486033.940836 read 0x0bd000 thru 0x0c1a0c (0x4a0d bytes) 000489[1] 1713486033.967504 mapread 0x58f000 thru 0x59ce7d (0xde7e bytes) 000490[0] 1713486033.967931 write 0x356000 thru 0x358f6b (0x2f6c bytes) 000491[1] 1713486033.980759 write_OD 0x8fb000 thru 0x905fff (0xb000 bytes) 000493[0] 1713486033.996520 read 0x870000 thru 0x87f90a (0xf90b bytes) 000494[1] 1713486034.026721 mapwrite 0x400000 thru 0x402e31 (0x2e32 bytes) 000495[0] 1713486034.056829 read 0x0d9000 thru 0x0e557a (0xc57b bytes) 000496[0] 1713486034.081196 mapread 0x823000 thru 0x826b5d (0x3b5e bytes) 000498[1] 1713486034.088031 mapwrite 0x2bc000 thru 0x2caf8e (0xef8f bytes) All operations completed A-OK! PASS 16a (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 20:20:36 (1713486036) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.145235 s, 57.8 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.126341 s, 66.4 MB/s Chance of close/open is 1 in 50 Seed set to 6037 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000010[1] 1713486037.512264 mapwrite 0x371c72 thru 0x373c0d (0x1f9c bytes) truncating to largest ever: 0x79c65d 000011[0] 1713486037.544925 trunc from 0x373c0e to 0x79c65c (0x428a4f bytes) 000012[1] 1713486037.552353 trunc from 0x79c65d to 0x1da540 (0x5c211c bytes) 000013[0] 1713486037.562787 read 0x01a4a1 thru 0x01b3ea (0xf4a bytes) 000014[0] 1713486037.568935 write 0x0838e1 thru 0x08481b (0xf3b bytes) 000023[1] 1713486037.602182 mapread 0x077025 thru 0x078793 (0x176f bytes) 000024[1] 1713486037.613227 mapread 0x0f1608 thru 0x0f1e50 (0x849 bytes) 000025[0] 1713486037.615275 read 0x132114 thru 0x1335ea (0x14d7 bytes) 000026[0] 1713486037.629488 write 0x0adbd8 thru 0x0af099 (0x14c2 bytes) 000029[0] 1713486037.643975 mapwrite 0x87024d thru 0x871618 (0x13cc bytes) 000030[1] 1713486037.686080 write 0x43af4e thru 0x43c479 (0x152c bytes) 000031[0] 1713486037.697884 mapwrite 0x18fceb thru 0x190900 (0xc16 bytes) 000039[1] 1713486037.747919 mapwrite 0x92b176 thru 0x92b819 (0x6a4 bytes) 000040[1] 1713486037.769481 write 0x65b9a2 thru 0x65cc2c (0x128b bytes) 000041[0] 1713486037.775490 write 0x0ffbe3 thru 0x101adb (0x1ef9 bytes) 000042[1] 1713486037.788572 write 0x41b2b8 thru 0x41b67a (0x3c3 bytes) 000046[0] 1713486037.801479 read 0x01a4d7 thru 0x01ba9e (0x15c8 bytes) 000048[1] 1713486037.824771 write 0x6ac897 thru 0x6ad2a2 (0xa0c bytes) 000049[0] 1713486037.831402 mapwrite 0x4f6f83 thru 0x4f7609 (0x687 bytes) 000050[1] 1713486037.867504 mapread 0x255f50 thru 0x256d7a (0xe2b bytes) 000051[0] 1713486037.879147 mapwrite 0x2e7854 thru 0x2e9282 (0x1a2f bytes) 000052[0] 1713486037.905927 read 0x5971de thru 0x598512 (0x1335 bytes) 000054[0] 1713486037.911981 trunc from 0x92b81a to 0x0eb80c (0x84000d bytes) 000057[1] 1713486037.929171 read 0x071f0b thru 0x073bef (0x1ce5 bytes) 000058[0] 1713486037.934983 trunc from 0x0eb80d to 0x320762 (0x234f56 bytes) 000059[0] 1713486037.945463 mapread 0x2ceb5e thru 0x2d070b (0x1bae bytes) 000062[1] 1713486037.972543 trunc from 0x320763 to 0x62caae (0x30c34c bytes) truncating to largest ever: 0x8100c2 000064[0] 1713486037.987260 trunc from 0x62caaf to 0x8100c1 (0x1e3613 bytes) 000065[0] 1713486037.995631 mapread 0x6ba995 thru 0x6bba7e (0x10ea bytes) 000067[0] 1713486038.018580 mapwrite 0x194c55 thru 0x195dbd (0x1169 bytes) 000069[1] 1713486038.034583 trunc from 0x8100c2 to 0x4c5f6f (0x34a152 bytes) 000070[0] 1713486038.048949 read 0x0a869d thru 0x0a88e5 (0x249 bytes) 000072[1] 1713486038.075166 mapread 0x3c55ce thru 0x3c6bfb (0x162e bytes) 000073[1] 1713486038.081274 mapwrite 0x96064b thru 0x962557 (0x1f0d bytes) 000075[0] 1713486038.109104 write 0x5852b5 thru 0x5852df (0x2b bytes) 000077[1] 1713486038.131829 write 0x3e246d thru 0x3e25b0 (0x144 bytes) 000078[1] 1713486038.136041 mapread 0x81c166 thru 0x81d95b (0x17f6 bytes) 000084[1] 1713486038.156843 mapwrite 0x39c3b5 thru 0x39d9a7 (0x15f3 bytes) 000085[1] 1713486038.178946 write 0x00ec73 thru 0x00f04f (0x3dd bytes) 000087[1] 1713486038.187095 trunc from 0x962558 to 0x5558ab (0x40ccac bytes) 000088[1] 1713486038.201213 trunc from 0x5558ac to 0x221d34 (0x333b77 bytes) 000089[0] 1713486038.208508 write 0x54d15e thru 0x54d9ac (0x84f bytes) 000090[0] 1713486038.213741 mapwrite 0x71a168 thru 0x71b96d (0x1806 bytes) 000092[1] 1713486038.243294 mapwrite 0x16d3e3 thru 0x16eac3 (0x16e1 bytes) 000094[0] 1713486038.266506 write 0x4e914c thru 0x4eaed9 (0x1d8e bytes) 000095[0] 1713486038.271733 trunc from 0x71b96e to 0x4a1ed1 (0x279a9c bytes) 000096[0] 1713486038.281741 read 0x1c65a2 thru 0x1c710c (0xb6b bytes) 000097[1] 1713486038.304108 mapread 0x205e28 thru 0x206dec (0xfc5 bytes) 000099[1] 1713486038.319545 write 0x20e5b8 thru 0x20f9b9 (0x1402 bytes) 000100[0] 1713486038.326778 trunc from 0x4a1ed2 to 0x1780b1 (0x329e20 bytes) 000101[0] 1713486038.337787 write 0x55fedd thru 0x56060a (0x72e bytes) 000102[1] 1713486038.342190 mapwrite 0x82c115 thru 0x82d3de (0x12ca bytes) 000103[1] 1713486038.373081 trunc from 0x82d3df to 0x7be5d1 (0x6ee0d bytes) 000107[0] 1713486038.386986 mapread 0x3d33ef thru 0x3d3410 (0x22 bytes) 000109[1] 1713486038.395512 mapread 0x603962 thru 0x60493d (0xfdc bytes) 000113[0] 1713486038.401810 trunc from 0x7be5d2 to 0x2b086b (0x50dd66 bytes) 000115[1] 1713486038.413603 mapwrite 0x1cf055 thru 0x1d08ec (0x1898 bytes) 000117[1] 1713486038.437383 trunc from 0x2b086c to 0x43cbd3 (0x18c368 bytes) 000119[0] 1713486038.448576 write 0x20364c thru 0x204db7 (0x176c bytes) 000120[0] 1713486038.455267 mapwrite 0x6d5e81 thru 0x6d6545 (0x6c5 bytes) 000121[1] 1713486038.489051 mapread 0x0c934a thru 0x0cafff (0x1cb6 bytes) 000123[1] 1713486038.502549 mapwrite 0x1a2828 thru 0x1a30b5 (0x88e bytes) 000134[1] 1713486038.557502 mapwrite 0x8326f7 thru 0x83330c (0xc16 bytes) 000135[0] 1713486038.583398 mapwrite 0x4fa0a5 thru 0x4fa4f1 (0x44d bytes) 000137[0] 1713486038.613918 mapread 0x65c7d3 thru 0x65dc91 (0x14bf bytes) 000139[0] 1713486038.625025 write 0x850d58 thru 0x851ac1 (0xd6a bytes) 000143[1] 1713486038.633982 read 0x27a696 thru 0x27b805 (0x1170 bytes) 000148[0] 1713486038.662403 write 0x388f26 thru 0x389d1e (0xdf9 bytes) 000149[0] 1713486038.676423 trunc from 0x851ac2 to 0x28e68e (0x5c3433 bytes) 000150[1] 1713486038.682730 read 0x1377e7 thru 0x138388 (0xba2 bytes) 000151[1] 1713486038.702530 mapwrite 0x1c59e0 thru 0x1c64a7 (0xac8 bytes) 000156[1] 1713486038.749616 read 0x051de1 thru 0x053d56 (0x1f76 bytes) 000157[0] 1713486038.766204 trunc from 0x28e68f to 0x7a04f0 (0x511e62 bytes) 000160[1] 1713486038.781006 write 0x1df55a thru 0x1df892 (0x339 bytes) 000163[1] 1713486038.802466 trunc from 0x7a04f1 to 0x003ef2 (0x79c5fe bytes) 000164[1] 1713486038.813735 write 0x32b2c4 thru 0x32bfd0 (0xd0d bytes) 000167[1] 1713486038.819773 write 0x4d2e7b thru 0x4d3d11 (0xe97 bytes) 000171[1] 1713486038.827441 read 0x40d78a thru 0x40da9c (0x313 bytes) 000172[0] 1713486038.833949 mapread 0x461e4e thru 0x462fbc (0x116f bytes) 000173[0] 1713486038.843229 mapwrite 0x1f5414 thru 0x1f6095 (0xc82 bytes) truncating to largest ever: 0x8d2d1e 000174[0] 1713486038.868673 trunc from 0x4d3d12 to 0x8d2d1d (0x3ff00c bytes) 000175[0] 1713486038.879294 write 0x2b4363 thru 0x2b4c1b (0x8b9 bytes) 000176[0] 1713486038.886000 trunc from 0x8d2d1e to 0x62a44f (0x2a88ce bytes) 000177[1] 1713486038.894361 mapwrite 0x583d3f thru 0x584c02 (0xec4 bytes) 000178[0] 1713486038.920844 mapread 0x14ab35 thru 0x14b49d (0x969 bytes) 000180[1] 1713486038.930622 mapwrite 0x972274 thru 0x9731dd (0xf6a bytes) 000183[0] 1713486038.978450 write 0x0f7cc0 thru 0x0f8229 (0x56a bytes) 000185[0] 1713486038.996840 read 0x8e72de thru 0x8e8194 (0xeb7 bytes) 000186[1] 1713486039.000636 mapwrite 0x542ee9 thru 0x5435ad (0x6c5 bytes) 000189[1] 1713486039.029885 mapwrite 0x5eb7d5 thru 0x5ebdf6 (0x622 bytes) 000190[1] 1713486039.047946 mapwrite 0x90d0f5 thru 0x90eead (0x1db9 bytes) 000191[1] 1713486039.078583 write 0x6cf6fb thru 0x6d0849 (0x114f bytes) 000193[1] 1713486039.082355 mapwrite 0x48d07d thru 0x48e5b1 (0x1535 bytes) 000194[1] 1713486039.099948 trunc from 0x9731de to 0x3d7b33 (0x59b6aa bytes) 000197[0] 1713486039.113929 mapwrite 0x70181f thru 0x702770 (0xf52 bytes) 000199[1] 1713486039.166440 trunc from 0x702771 to 0x3f57bf (0x30cfb1 bytes) 000206[1] 1713486039.193195 trunc from 0x3f57c0 to 0x36752c (0x8e293 bytes) 000210[0] 1713486039.205561 read 0x230595 thru 0x231d61 (0x17cd bytes) 000211[0] 1713486039.223300 mapwrite 0x7a94f6 thru 0x7aa1c9 (0xcd4 bytes) 000212[1] 1713486039.267717 write 0x11be26 thru 0x11bf44 (0x11f bytes) 000217[0] 1713486039.287991 mapwrite 0x358044 thru 0x359fd4 (0x1f91 bytes) 000218[1] 1713486039.308808 read 0x02d3d3 thru 0x02ed28 (0x1956 bytes) 000219[0] 1713486039.331132 read 0x3a98e9 thru 0x3aa7cd (0xee5 bytes) 000221[0] 1713486039.359854 mapwrite 0x09fb30 thru 0x0a17a0 (0x1c71 bytes) 000223[0] 1713486039.400543 mapwrite 0x97cb38 thru 0x97d8f4 (0xdbd bytes) 000224[1] 1713486039.437730 read 0x65c8a5 thru 0x65df21 (0x167d bytes) 000225[1] 1713486039.449129 write 0x857de5 thru 0x8585ba (0x7d6 bytes) 000227[1] 1713486039.485888 trunc from 0x97d8f5 to 0x74b339 (0x2325bb bytes) 000228[1] 1713486039.495773 trunc from 0x74b33a to 0x3e4103 (0x367236 bytes) 000230[1] 1713486039.502876 write 0x2ea6b7 thru 0x2ebe89 (0x17d3 bytes) 000233[0] 1713486039.503908 trunc from 0x3e4104 to 0x65e55e (0x27a45b bytes) 000234[1] 1713486039.521658 mapwrite 0x641ee9 thru 0x64211a (0x232 bytes) 000235[0] 1713486039.548033 read 0x229b80 thru 0x22a6f3 (0xb74 bytes) 000239[0] 1713486039.576888 mapwrite 0x7fc844 thru 0x7fd30a (0xac7 bytes) 000240[1] 1713486039.614807 mapread 0x528463 thru 0x529fc9 (0x1b67 bytes) 000243[1] 1713486039.634957 write 0x1dff22 thru 0x1e0154 (0x233 bytes) 000244[1] 1713486039.645037 mapwrite 0x3fcc8e thru 0x3fcf02 (0x275 bytes) 000245[0] 1713486039.661864 write 0x3df41b thru 0x3e0bca (0x17b0 bytes) 000249[0] 1713486039.672642 write 0x6940b2 thru 0x694408 (0x357 bytes) 000250[0] 1713486039.678469 read 0x6f1bc6 thru 0x6f3157 (0x1592 bytes) 000251[0] 1713486039.709903 mapread 0x0e3bd1 thru 0x0e43b9 (0x7e9 bytes) 000254[0] 1713486039.719361 write 0x56660b thru 0x5678c6 (0x12bc bytes) 000257[0] 1713486039.724188 mapread 0x3e938f thru 0x3eb0cd (0x1d3f bytes) 000259[1] 1713486039.724994 mapwrite 0x2fc67c thru 0x2fdbac (0x1531 bytes) 000260[0] 1713486039.754518 write 0x9ad039 thru 0x9adaa2 (0xa6a bytes) 000261[0] 1713486039.757054 mapread 0x4f51eb thru 0x4f6296 (0x10ac bytes) 000262[1] 1713486039.757521 read 0x27024b thru 0x270ba2 (0x958 bytes) 000263[0] 1713486039.784246 mapread 0x02a13e thru 0x02b5ff (0x14c2 bytes) 000266[1] 1713486039.790734 read 0x1724d1 thru 0x174264 (0x1d94 bytes) 000267[1] 1713486039.793852 read 0x4ab1db thru 0x4ac5b7 (0x13dd bytes) 000268[1] 1713486039.818101 mapread 0x1eb4a6 thru 0x1ecf82 (0x1add bytes) 000269[0] 1713486039.818594 mapwrite 0x9d4b69 thru 0x9d6090 (0x1528 bytes) truncating to largest ever: 0x9441c3 000270[1] 1713486039.865535 trunc from 0x9d6091 to 0x9441c2 (0x91ece bytes) 000272[0] 1713486039.873743 mapread 0x2e8381 thru 0x2e9e8a (0x1b0a bytes) 000274[0] 1713486039.880291 mapread 0x33524e thru 0x336cc5 (0x1a78 bytes) 000275[1] 1713486039.887490 write 0x027199 thru 0x028efa (0x1d62 bytes) 000278[0] 1713486039.901572 mapwrite 0x65f246 thru 0x65f448 (0x203 bytes) 000284[0] 1713486039.940379 mapwrite 0x11142f thru 0x112fd0 (0x1ba2 bytes) 000286[0] 1713486039.968825 write 0x145b70 thru 0x147b1a (0x1fab bytes) 000287[1] 1713486039.975497 mapwrite 0x5eefca thru 0x5ef89e (0x8d5 bytes) 000288[0] 1713486039.998075 write 0x38f74b thru 0x3915cf (0x1e85 bytes) 000291[1] 1713486040.012296 trunc from 0x9441c3 to 0x777621 (0x1ccba1 bytes) 000294[1] 1713486040.030609 mapread 0x3b4ef1 thru 0x3b651b (0x162b bytes) 000303[1] 1713486040.050546 mapread 0x4cd280 thru 0x4ce68c (0x140d bytes) 000304[1] 1713486040.057915 mapread 0x5f19b5 thru 0x5f1e57 (0x4a3 bytes) 000305[1] 1713486040.063565 mapread 0x040c2c thru 0x0421e1 (0x15b6 bytes) 000307[1] 1713486040.076764 read 0x1baedc thru 0x1bca0d (0x1b32 bytes) 000308[0] 1713486040.108575 trunc from 0x777622 to 0x8d2aca (0x15b4a9 bytes) 000309[0] 1713486040.123329 trunc from 0x8d2acb to 0x672cce (0x25fdfc bytes) 000310[0] 1713486040.129679 mapread 0x0b5a2b thru 0x0b6035 (0x60b bytes) 000317[1] 1713486040.140390 mapread 0x55905f thru 0x55a9d5 (0x1977 bytes) 000318[0] 1713486040.147189 mapread 0x1c4345 thru 0x1c6247 (0x1f03 bytes) 000320[0] 1713486040.150819 trunc from 0x672ccf to 0x17d8ad (0x4f5421 bytes) 000321[0] 1713486040.159759 mapread 0x046dcf thru 0x048ada (0x1d0c bytes) 000323[0] 1713486040.170729 write 0x39af21 thru 0x39b60a (0x6ea bytes) 000324[1] 1713486040.173053 trunc from 0x39b60b to 0x304b1e (0x96aec bytes) 000326[1] 1713486040.183462 mapread 0x10ea26 thru 0x10ed3b (0x316 bytes) 000328[1] 1713486040.194561 write 0x50c53f thru 0x50ce09 (0x8cb bytes) 000329[1] 1713486040.195398 mapwrite 0x2e5437 thru 0x2e5a9b (0x665 bytes) 000330[0] 1713486040.216301 write 0x137ac5 thru 0x139971 (0x1ead bytes) 000331[0] 1713486040.223805 read 0x30f018 thru 0x310a17 (0x1a00 bytes) 000333[0] 1713486040.252747 mapwrite 0x28ed36 thru 0x28f8fd (0xbc8 bytes) 000337[0] 1713486040.283883 write 0x196ed7 thru 0x198c01 (0x1d2b bytes) 000338[0] 1713486040.287377 trunc from 0x50ce0a to 0x8b7dba (0x3aafb1 bytes) 000339[1] 1713486040.297694 trunc from 0x8b7dbb to 0x62c0f9 (0x28bcc1 bytes) 000342[1] 1713486040.322448 mapread 0x032795 thru 0x03468b (0x1ef7 bytes) 000343[1] 1713486040.331123 mapwrite 0x812aba thru 0x813c43 (0x118a bytes) 000344[0] 1713486040.358250 mapwrite 0x05dc16 thru 0x05f719 (0x1b04 bytes) 000345[0] 1713486040.393953 mapread 0x1ea9bb thru 0x1eb624 (0xc6a bytes) 000346[1] 1713486040.401909 mapwrite 0x1d64cd thru 0x1d82b8 (0x1dec bytes) 000347[0] 1713486040.429989 mapread 0x5c9b9a thru 0x5cb7d2 (0x1c39 bytes) 000348[1] 1713486040.453881 write 0x191e11 thru 0x193347 (0x1537 bytes) 000349[0] 1713486040.457035 mapwrite 0x006511 thru 0x006599 (0x89 bytes) 000351[0] 1713486040.477066 read 0x734cbc thru 0x7357ec (0xb31 bytes) 000352[1] 1713486040.480621 write 0x33b906 thru 0x33d8c0 (0x1fbb bytes) 000353[1] 1713486040.488827 write 0x3bc4f2 thru 0x3bd4f7 (0x1006 bytes) 000354[0] 1713486040.494362 trunc from 0x813c44 to 0x588193 (0x28bab0 bytes) 000355[1] 1713486040.511940 mapwrite 0x16731b thru 0x168181 (0xe67 bytes) 000357[1] 1713486040.541241 mapwrite 0x3db9fc thru 0x3dd8f0 (0x1ef5 bytes) 000358[0] 1713486040.565219 read 0x560a8d thru 0x560c4f (0x1c3 bytes) 000359[1] 1713486040.570416 trunc from 0x588194 to 0x53d7b4 (0x4a9df bytes) 000361[0] 1713486040.576070 mapread 0x318662 thru 0x3189b4 (0x353 bytes) 000362[0] 1713486040.598171 mapread 0x3114ee thru 0x312309 (0xe1c bytes) 000367[1] 1713486040.614752 read 0x4c8b37 thru 0x4c9d6f (0x1239 bytes) 000369[0] 1713486040.628091 trunc from 0x53d7b5 to 0x3f8ba9 (0x144c0b bytes) 000370[1] 1713486040.641333 mapread 0x2629e2 thru 0x263759 (0xd78 bytes) 000372[0] 1713486040.647998 read 0x13e042 thru 0x13f1d6 (0x1195 bytes) 000375[0] 1713486040.663960 read 0x3f1fea thru 0x3f2e0f (0xe26 bytes) 000376[1] 1713486040.664294 read 0x06ddb2 thru 0x06f7c5 (0x1a14 bytes) 000377[0] 1713486040.686495 write 0x3d1ee9 thru 0x3d300b (0x1123 bytes) 000378[1] 1713486040.713256 read 0x250671 thru 0x2513f7 (0xd87 bytes) 000380[0] 1713486040.741029 mapwrite 0x303fea thru 0x3052b4 (0x12cb bytes) 000381[1] 1713486040.771261 trunc from 0x3f8baa to 0x1eaa44 (0x20e165 bytes) 000382[1] 1713486040.782496 mapwrite 0x3f9acf thru 0x3f9edd (0x40f bytes) 000385[0] 1713486040.811540 write 0x1aa0fe thru 0x1aa67c (0x57f bytes) 000388[0] 1713486040.827992 trunc from 0x3f9ede to 0x4eb07a (0xf119d bytes) 000389[0] 1713486040.835442 read 0x402199 thru 0x4029fb (0x863 bytes) 000392[0] 1713486040.860490 mapwrite 0x17eccf thru 0x17fdea (0x111c bytes) 000393[1] 1713486040.876800 write 0x46edaa thru 0x470077 (0x12ce bytes) 000394[1] 1713486040.885256 read 0x3dda9d thru 0x3def9a (0x14fe bytes) 000396[1] 1713486040.909798 mapread 0x23a62e thru 0x23bfc1 (0x1994 bytes) 000397[1] 1713486040.914988 mapread 0x06f3e7 thru 0x0705ba (0x11d4 bytes) 000398[0] 1713486040.918636 mapread 0x482361 thru 0x483ddd (0x1a7d bytes) 000399[1] 1713486040.928141 trunc from 0x4eb07b to 0x8d1271 (0x3e61f7 bytes) 000400[0] 1713486040.940534 read 0x08b243 thru 0x08c14e (0xf0c bytes) 000406[0] 1713486040.979116 mapread 0x57f53b thru 0x58032e (0xdf4 bytes) 000409[0] 1713486041.000062 write 0x59ecd1 thru 0x59f777 (0xaa7 bytes) 000411[1] 1713486041.014449 mapwrite 0x01cc92 thru 0x01d5da (0x949 bytes) 000412[1] 1713486041.039279 write 0x145949 thru 0x145d69 (0x421 bytes) 000414[0] 1713486041.047356 read 0x133b71 thru 0x133c37 (0xc7 bytes) 000415[0] 1713486041.070614 mapwrite 0x677b66 thru 0x67848f (0x92a bytes) 000417[1] 1713486041.107640 mapread 0x169229 thru 0x16a40a (0x11e2 bytes) 000420[0] 1713486041.118729 read 0x7ee20b thru 0x7f002a (0x1e20 bytes) 000422[1] 1713486041.122325 trunc from 0x8d1272 to 0x498e83 (0x4383ee bytes) 000423[1] 1713486041.139266 trunc from 0x498e84 to 0x7a1835 (0x3089b2 bytes) 000424[0] 1713486041.147300 write 0x888a14 thru 0x889171 (0x75e bytes) 000427[0] 1713486041.164496 trunc from 0x889172 to 0x3b9858 (0x4cf919 bytes) 000428[1] 1713486041.176903 mapread 0x1ee98a thru 0x1f0882 (0x1ef9 bytes) 000431[1] 1713486041.191037 read 0x25dea1 thru 0x25ee7d (0xfdd bytes) 000432[0] 1713486041.206198 mapwrite 0x39e024 thru 0x39e9e6 (0x9c3 bytes) 000435[0] 1713486041.255468 mapwrite 0x95fcae thru 0x960dfc (0x114f bytes) 000436[1] 1713486041.304845 mapread 0x3831b9 thru 0x383aae (0x8f6 bytes) 000440[1] 1713486041.327620 mapwrite 0x5dcf8e thru 0x5dde73 (0xee6 bytes) 000446[0] 1713486041.389631 mapread 0x0ee1cf thru 0x0eea47 (0x879 bytes) 000448[0] 1713486041.405361 trunc from 0x960dfd to 0x86aa66 (0xf6396 bytes) 000450[0] 1713486041.420905 mapwrite 0x4c0d06 thru 0x4c2572 (0x186d bytes) 000455[1] 1713486041.461706 read 0x68b811 thru 0x68c60b (0xdfb bytes) 000456[0] 1713486041.467539 trunc from 0x86aa67 to 0x193dc5 (0x6d6ca1 bytes) 000460[0] 1713486041.475594 mapread 0x0b531d thru 0x0b6bf8 (0x18dc bytes) 000462[0] 1713486041.478538 mapread 0x03701f thru 0x038002 (0xfe4 bytes) 000463[0] 1713486041.481154 mapread 0x12c7d5 thru 0x12df7f (0x17ab bytes) 000464[0] 1713486041.489430 trunc from 0x193dc6 to 0x6c209d (0x52e2d8 bytes) truncating to largest ever: 0x970537 000465[0] 1713486041.499822 trunc from 0x6c209e to 0x970536 (0x2ae499 bytes) 000469[0] 1713486041.513440 mapwrite 0x5d96ef thru 0x5da22e (0xb40 bytes) 000471[0] 1713486041.538599 trunc from 0x970537 to 0x567aa8 (0x408a8e bytes) 000472[1] 1713486041.543037 mapwrite 0x740ad7 thru 0x742235 (0x175f bytes) 000473[0] 1713486041.597653 mapwrite 0x7d8622 thru 0x7d941e (0xdfd bytes) 000474[1] 1713486041.629388 trunc from 0x7d941f to 0x533826 (0x2a5bf8 bytes) 000475[0] 1713486041.640770 mapwrite 0x82c063 thru 0x82de38 (0x1dd6 bytes) 000477[0] 1713486041.672070 mapwrite 0x2c535b thru 0x2c62af (0xf55 bytes) 000478[1] 1713486041.690530 write 0x2fcc5e thru 0x2fce0a (0x1ad bytes) 000480[1] 1713486041.701127 mapwrite 0x1d3c7d thru 0x1d414b (0x4cf bytes) 000481[0] 1713486041.723277 trunc from 0x82de39 to 0x0ad4d4 (0x780964 bytes) 000482[1] 1713486041.729334 write 0x0216a3 thru 0x02171e (0x7c bytes) 000485[0] 1713486041.742754 write 0x0fb918 thru 0x0fd422 (0x1b0b bytes) 000486[0] 1713486041.749623 mapread 0x0bd82d thru 0x0bf65d (0x1e31 bytes) 000488[1] 1713486041.758440 write 0x01954d thru 0x0198f1 (0x3a5 bytes) 000490[0] 1713486041.770149 write 0x112568 thru 0x112f67 (0xa00 bytes) 000491[0] 1713486041.773749 trunc from 0x112f68 to 0x2717c9 (0x15e862 bytes) 000492[0] 1713486041.781000 trunc from 0x2717ca to 0x41a3a5 (0x1a8bdc bytes) 000494[1] 1713486041.789783 trunc from 0x41a3a6 to 0x0edf88 (0x32c41d bytes) 000495[1] 1713486041.800087 write 0x8536c6 thru 0x8544df (0xe1a bytes) 000497[0] 1713486041.807032 trunc from 0x8544e0 to 0x7757e8 (0xdecf7 bytes) 000498[0] 1713486041.817166 write 0x3df4ef thru 0x3e0708 (0x121a bytes) 000499[0] 1713486041.819903 read 0x179fc5 thru 0x17acf4 (0xd30 bytes) 000500[1] 1713486041.845449 mapwrite 0x058334 thru 0x059ff7 (0x1cc4 bytes) All operations completed A-OK! PASS 16b (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 20:20:44 (1713486044) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 20:20:46 (1713486046) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.134449 s, 62.4 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.124067 s, 67.6 MB/s ldlm.namespaces.MGC192.168.204.126@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aaa91800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b6095000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800aaa91800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b6095000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800aaa91800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b6095000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.62935 s, 39.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.632299 s, 166 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.55293 s, 190 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.32358 s, 79.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.75954 s, 59.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.765744 s, 137 MB/s PASS 16d (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 20:20:58 (1713486058) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.059357 s, 177 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289886 s, 36.2 MB/s PASS 16e (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 20:21:01 (1713486061) TIMEOUT PASS 16f (24s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 20:21:28 (1713486088) TIMEOUT PASS 16g (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16h: mmap read after truncate file ======= 20:21:51 (1713486111) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16i: read after truncate file ============ 20:21:56 (1713486116) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16j: race dio with buffered i/o ========== 20:22:00 (1713486120) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.448531 s, 58.4 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.386292 s, 67.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.21481 s, 21.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.222295 s, 118 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.16726 s, 22.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.171563 s, 153 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.20079 s, 21.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.321023 s, 81.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.486801 s, 53.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.37106 s, 19.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.47083 s, 17.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.276142 s, 94.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.374469 s, 70.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.23457 s, 21.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.31962 s, 19.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.301729 s, 86.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.394404 s, 66.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.25395 s, 20.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.31769 s, 19.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (14s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 20:22:15 (1713486135) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 skipping zero size read truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x32cce75 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Thu Apr 18 20:22:36 EDT 2024 All operations completed A-OK! + fsx 2 done: rc=0 Thu Apr 18 20:22:37 EDT 2024 All operations completed A-OK! + fsx 3 done: rc=0 Thu Apr 18 20:22:37 EDT 2024 + fsx 4 done: rc=0 All operations completed A-OK! Thu Apr 18 20:22:37 EDT 2024 + fsx 5 done: rc=0 Thu Apr 18 20:22:37 EDT 2024 PASS 16k (27s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 17: resource creation/LVB creation race ========================================================================= 20:22:44 (1713486164) fail_loc=0x8000030a PASS 17 (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 18: mmap sanity check =========================================================================================== 20:22:49 (1713486169) mmap test1: basic mmap operation (PASS, 0.020248s) mmap test2: MAP_PRIVATE not write back (PASS, 0.013003s) mmap test3: concurrent mmap ops on two nodes (PASS, 7.9144s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0754s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.026935s) mmap test6: check mmap write/read content on two nodes (PASS, 0.043528s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0137s) PASS 18 (14s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 19: test concurrent uncached read races ========================================================================= 20:23:04 (1713486184) oleg426-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg426-client: oleg426-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 20: test extra readahead page left in cache ============================================================== 20:23:07 (1713486187) PASS 20 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 20:23:10 (1713486190) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 23: others should see updated atime while another read============================================================== 20:23:13 (1713486193) now is 1713486194 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 new atime is 1713486255 PASS 23 (63s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 20:24:18 (1713486258) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 34816 3734528 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 44032 7494656 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 34816 3734528 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 44032 7494656 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 403 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.4K 400 114.0K 1% /mnt/lustre[OST:1] filesystem_summary: 229.2K 386 228.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 403 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.4K 400 114.0K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.2K 386 228.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 9.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 34.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 43.0M 7.1G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117971 403 117568 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117168 400 116768 1% /mnt/lustre2[OST:1] filesystem_summary: 234722 386 234336 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 34816 3734528 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 44032 7494656 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 403 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.4K 400 114.0K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.2K 386 228.8K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre[MDT:0] filesystem_summary: 502242 386 501856 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117971 403 117568 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117168 400 116768 1% /mnt/lustre2[OST:1] filesystem_summary: 234722 386 234336 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 34816 3734528 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 44032 7494656 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 34816 3734528 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 44032 7494656 1% /mnt/lustre2 PASS 24a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 24b: lfs df should show both filesystems ========================================================================= 20:24:21 (1713486261) PASS 24b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 20:24:24 (1713486264) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 20:24:26 (1713486266) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 26a: allow mtime to get older ============ 20:24:29 (1713486269) utime: good mknod times 1713486268 <= 1713486269 <= 1713486269 for /mnt/lustre/f26a utime: good mknod times 1713486268 <= 1713486269 <= 1713486269 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 26b: sync mtime between ost and mds ====== 20:24:33 (1713486273) PASS 26b (3s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 30: recreate file race =================== 20:24:39 (1713486279) 0 PASS 30 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 20:24:44 (1713486284) fail_loc=0x314 PASS 31a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 20:24:48 (1713486288) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg426-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.126@tcp:/lustre 7542784 44032 7494656 1% /mnt/lustre2 PASS 31b (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31r: open-rename(replace) race =========== 20:25:00 (1713486300) -rw-r--r-- 1 root root 0 Apr 18 20:25 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (4s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_33a skipping SLOW test 33a debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 20:25:06 (1713486306) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 20:25:08 (1713486308) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33d: dependent transactions should trigger COS ========================================================== 20:25:10 (1713486310) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 20:25:13 (1713486313) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 34: no lock timeout under IO ============= 20:25:15 (1713486315) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg426-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aaa91800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b6095000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800aaa91800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b6095000.ost_server_uuid in FULL state after 0 sec oleg426-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aaa91800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b6095000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800aaa91800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800b6095000.ost_server_uuid in IDLE state after 0 sec oleg426-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aaa91800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b6095000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800aaa91800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b6095000.ost_server_uuid in FULL state after 0 sec oleg426-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aaa91800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b6095000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800aaa91800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800b6095000.ost_server_uuid in IDLE state after 0 sec oleg426-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aaa91800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b6095000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800aaa91800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b6095000.ost_server_uuid in IDLE state after 0 sec oleg426-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aaa91800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b6095000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800aaa91800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800b6095000.ost_server_uuid in IDLE state after 0 sec PASS 34 (82s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 20:26:39 (1713486399) fail_loc=0x80000317 Race attempt 0 Wait for 24351 24493 for 60 sec... total: 4000 open/close in 7.18 seconds: 556.78 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.126@tcp:/lustre 7.2G 44M 7.2G 1% /mnt/lustre 192.168.204.126@tcp:/lustre 7.2G 44M 7.2G 1% /mnt/lustre2 PASS 35 (63s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 36: handle ESTALE/open-unlink correctly == 20:27:44 (1713486464) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg426-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.261774 s, 200 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7493632) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.25605 s, 205 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7493632) after_dd(7440384) after(7491584) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.278845 s, 188 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7491584) after_dd(7440384) after(7491584) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.240527 s, 218 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7491584) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.244767 s, 214 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7493632) after_dd(7438336) after(7489536) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.25336 s, 207 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7489536) after_dd(7440384) after(7491584) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.297819 s, 176 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7491584) after_dd(7440384) after(7491584) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.315457 s, 166 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7491584) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.264265 s, 198 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7493632) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.23853 s, 220 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7493632) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.285891 s, 183 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7493632) after_dd(7442432) after(7493632) PASS 36 (227s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 20:31:33 (1713486693) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 total: 10000 create in 9.60 seconds: 1042.02 ops/second PASS 37 (12s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39a: file mtime does not change after rename ========================================================== 20:31:46 (1713486706) repeat after cancel_lru_locks PASS 39a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 20:31:50 (1713486710) repeat after cancel_lru_locks PASS 39b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39c: check truncate mtime update ================================================================================ 20:31:54 (1713486714) repeat after cancel_lru_locks PASS 39c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39d: sync write should update mtime ====== 20:31:58 (1713486718) fail_loc=0x411 fail_loc=0 PASS 39d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40a: pdirops: create vs others ======================================================================== 20:32:01 (1713486721) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40b: pdirops: open|create and others ======================================================================== 20:32:09 (1713486729) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40c: pdirops: link and others ======================================================================== 20:32:18 (1713486738) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40d: pdirops: unlink and others ======================================================================== 20:32:26 (1713486746) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40e: pdirops: rename and others ======================================================================== 20:32:35 (1713486755) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 20:32:43 (1713486763) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41b: pdirops: create vs create ======================================================================== 20:32:49 (1713486769) open: File exists Conflict PASS 41b (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41c: pdirops: create vs link ======================================================================== 20:32:54 (1713486774) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41d: pdirops: create vs unlink ======================================================================== 20:33:00 (1713486780) Conflict PASS 41d (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 20:33:06 (1713486786) Conflict PASS 41e (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 20:33:12 (1713486792) Conflict PASS 41f (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41g: pdirops: create vs getattr ======================================================================== 20:33:17 (1713486797) Conflict PASS 41g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41h: pdirops: create vs readdir ======================================================================== 20:33:23 (1713486803) Conflict PASS 41h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41i: reint_open: create vs create ======== 20:33:29 (1713486809) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (651s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 20:44:22 (1713487462) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 20:44:27 (1713487467) open: File exists Conflict PASS 42b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 20:44:33 (1713487473) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 20:44:39 (1713487479) Conflict PASS 42d (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 20:44:45 (1713487485) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 20:44:51 (1713487491) Conflict PASS 42f (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 20:44:57 (1713487497) Conflict PASS 42g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 20:45:02 (1713487502) Conflict PASS 42h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 20:45:08 (1713487508) PASS 43a (13s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43b: pdirops: unlink vs create ======================================================================== 20:45:23 (1713487523) Conflict PASS 43b (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43c: pdirops: unlink vs link ======================================================================== 20:45:29 (1713487529) Conflict PASS 43c (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 20:45:34 (1713487534) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 20:45:40 (1713487540) Conflict PASS 43e (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 20:45:46 (1713487546) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 20:45:52 (1713487552) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 20:45:57 (1713487557) Conflict PASS 43h (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 20:46:03 (1713487563) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 20:46:05 (1713487565) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (33s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43k: unlink vs create ==================== 20:46:40 (1713487600) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (359s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 20:52:41 (1713487961) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Thu Apr 18 20:52:44 EDT 2024 PASS 44a (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 20:52:46 (1713487966) open: File exists Conflict PASS 44b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 20:52:52 (1713487972) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 20:52:58 (1713487978) Conflict PASS 44d (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 20:53:04 (1713487984) Conflict PASS 44e (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 20:53:10 (1713487990) Conflict PASS 44f (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 20:53:15 (1713487995) Conflict PASS 44g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 20:53:21 (1713488001) Conflict PASS 44h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 20:53:27 (1713488007) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 20:53:30 (1713488010) PASS 45a (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45b: pdirops: rename src vs create ======================================================================== 20:53:53 (1713488033) Conflict PASS 45b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45c: pdirops: rename src vs link ======================================================================== 20:53:59 (1713488039) Conflict PASS 45c (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 20:54:05 (1713488045) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 20:54:11 (1713488051) Conflict PASS 45e (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 20:54:17 (1713488057) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 20:54:23 (1713488063) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 20:54:29 (1713488069) Conflict PASS 45h (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 20:54:34 (1713488074) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45j: read vs rename ====================== 20:54:36 (1713488076) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (393s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 21:01:11 (1713488471) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46b: pdirops: link vs create ======================================================================== 21:01:20 (1713488480) open: File exists Conflict PASS 46b (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46c: pdirops: link vs link ======================================================================== 21:01:30 (1713488490) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46d: pdirops: link vs unlink ======================================================================== 21:01:40 (1713488500) Conflict PASS 46d (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 21:01:48 (1713488508) Conflict PASS 46e (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 21:01:56 (1713488516) Conflict PASS 46f (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46g: pdirops: link vs getattr ======================================================================== 21:02:03 (1713488523) Conflict PASS 46g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46h: pdirops: link vs readdir ======================================================================== 21:02:10 (1713488530) Conflict PASS 46h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46i: pdirops: link vs remote mkdir ======= 21:02:16 (1713488536) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 21:02:19 (1713488539) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47b: pdirops: remote mkdir vs create ===== 21:02:22 (1713488542) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47c: pdirops: remote mkdir vs link ======= 21:02:25 (1713488545) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 21:02:28 (1713488548) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 21:02:31 (1713488551) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 21:02:34 (1713488554) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 21:02:37 (1713488557) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 21:02:39 (1713488559) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.0282438 s, 363 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51a: layout lock: refresh layout should work ========================================================== 21:02:47 (1713488567) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00515804 s, 30.8 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3445: kill: (11661) - No such process PASS 51a (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 21:02:53 (1713488573) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00329303 s, 311 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013372 s, 78.4 MB/s 3145728 PASS 51b (13s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 21:03:08 (1713488588) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00344437 s, 297 kB/s PASS 51c (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 21:03:15 (1713488595) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00806083 s, 130 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 21:03:19 (1713488599) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2451 0x993 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2451 0x993 0x240000400 read lease(1) has applied. PASS 51e (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 54: rename locking ======================= 21:03:24 (1713488604) llite.lustre-ffff8800aaa91800.xattr_cache=1 llite.lustre-ffff8800b6095000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505002 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-18 21:03:24.000000000 -0400 Modify: 2024-04-18 21:03:24.000000000 -0400 Change: 2024-04-18 21:03:24.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505005 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-18 21:03:30.000000000 -0400 Modify: 2024-04-18 21:03:30.000000000 -0400 Change: 2024-04-18 21:03:30.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505008 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-18 21:03:35.000000000 -0400 Modify: 2024-04-18 21:03:35.000000000 -0400 Change: 2024-04-18 21:03:35.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505007 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-18 21:03:35.000000000 -0400 Modify: 2024-04-18 21:03:35.000000000 -0400 Change: 2024-04-18 21:03:35.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505010 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-18 21:03:40.000000000 -0400 Modify: 2024-04-18 21:03:40.000000000 -0400 Change: 2024-04-18 21:03:40.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505011 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-18 21:03:40.000000000 -0400 Modify: 2024-04-18 21:03:40.000000000 -0400 Change: 2024-04-18 21:03:40.000000000 -0400 Birth: - PASS 54 (23s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55a: rename vs unlink target dir ========= 21:03:48 (1713488628) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55b: rename vs unlink source dir ========= 21:03:57 (1713488637) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55c: rename vs unlink orphan target dir == 21:04:05 (1713488645) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55d: rename file vs link ================= 21:04:18 (1713488658) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 56a: test llverdev with single large stripe ========================================================== 21:04:29 (1713488669) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1713488669 chunksize: 4096 size: 16777216000000 write offset: 3803185152kB 1.57465 MB/s read offset: 3803185152kB inf MB/s llverdev: data verified successfully PASS 56a (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 21:04:41 (1713488681) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1713488682 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 4.20946 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (33s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 60: Verify data_version behaviour ======== 21:05:15 (1713488715) PASS 60 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 70a: cd directory && rm directory ======== 21:05:18 (1713488718) PASS 70a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 70b: remove files after calling rm_entry ========================================================== 21:05:22 (1713488722) PASS 70b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71a: correct file map just after write operation is finished ========================================================== 21:05:26 (1713488726) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00234253 s, 28.0 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 21:05:28 (1713488728) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00299814 s, 21.9 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 21:05:31 (1713488731) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 21:05:34 (1713488734) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00351009 s, 18.7 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 21:05:36 (1713488736) llite.lustre-ffff8800aaa91800.xattr_cache=1 llite.lustre-ffff8800b6095000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 21:05:39 (1713488739) llite.lustre-ffff8800aaa91800.xattr_cache=1 llite.lustre-ffff8800b6095000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 74: flock deadlock: different mounts ======================================================================== 21:05:42 (1713488742) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00218948 s, 468 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00209702 s, 488 kB/s 26261: get lock2 26262: get lock1 26262: done 26262: get lock2 26261: done 26261: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 26262: cannot get lock2: Resource deadlock avoided 26262: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 26261: cannot get lock1: Resource deadlock avoided 26261: done 26261: exit rc=0 PASS 74 (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 75: osc: upcall after unuse lock============================================================================= 21:05:49 (1713488749) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0116564 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0072235 s, 145 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4031: 27008 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 76: Verify MDT open_files listing ======== 21:05:57 (1713488757) opening files: ........ closing files: ........ PASS 76 (14s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77a: check FIFO NRS policy =============== 21:06:13 (1713488773) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.114243 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913736 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105322 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286666 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189818 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198544 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369073 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348888 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366233 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400743 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303694 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385824 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374833 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332525 s, 31.5 MB/s , 0.0452545 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410957 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442839 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969657 s, 108 MB/s , 0.00954919 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193578 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105321 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103525 s, 101 MB/s , 0.0196719 s, 53.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0168823 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173508 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126052 s, 83.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128043 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130717 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014373 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014672 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148614 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145272 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162833 s, 64.4 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77b: check CRR-N NRS policy ============== 21:06:17 (1713488777) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0768753 s, 218 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00769225 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947485 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119339 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371799 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037554 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365592 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456582 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456287 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460644 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472713 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473103 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488925 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0486493 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483864 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0597771 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0613367 s, 17.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b6095000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff8800b6095000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00680872 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00554919 s, 189 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00612491 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00582299 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947494 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00956998 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00419684 s, 250 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970559 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084482 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00872 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887728 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892523 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891393 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846447 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822733 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104998 s, 99.9 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0851055 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105623 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119238 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033014 s, 31.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0380839 s, 27.5 MB/s , 0.0413761 s, 25.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0420334 s, 24.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0407462 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384888 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464985 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366059 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043988 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385998 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487272 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0481182 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0466461 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500669 s, 20.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b6095000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00709705 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00830425 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00586448 s, 179 MB/s , 0.0059919 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00718446 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00730445 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00749971 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00696359 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856943 s, 122 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00542989 s, 193 MB/s 1048576 bytes (1.0 MB) copied, 0.00830785 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00840102 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0089422 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883145 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00862227 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102187 s, 103 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77c: check ORR NRS policy ================ 21:06:21 (1713488781) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0869096 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111755 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135361 s, 77.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03241 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0443372 s, 23.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0464972 s, 22.6 MB/s 1048576 bytes (1.0 MB) copied, 0.050021 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0518735 s, 20.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430456 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541721 s, 19.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0525247 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05405 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0508297 s, 20.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0582369 s, 18.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0528984 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552744 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0563778 s, 18.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b6095000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00706726 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00784413 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895014 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00668024 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00843694 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0041875 s, 250 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100822 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906443 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957117 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101075 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953495 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108903 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979893 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104564 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104451 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125888 s, 83.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg426-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0834616 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127366 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015937 s, 65.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0308149 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326995 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238483 s, 44.0 MB/s , 0.0345495 s, 30.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431827 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358643 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046357 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0427891 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447931 s, 23.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0414851 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434775 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463065 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0479805 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0506607 s, 20.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00582166 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892069 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00819866 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00524422 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00832246 s, 126 MB/s , 0.00817682 s, 128 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00796392 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00769136 s, 136 MB/s , 0.00809086 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100257 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893766 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0107641 s, 97.4 MB/s , 0.00933376 s, 112 MB/s , 0.00891151 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935339 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115031 s, 91.2 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0826384 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00753635 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119721 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245261 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337855 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03868 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375658 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434138 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402483 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476313 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459987 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419351 s, 25.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.052168 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0515812 s, 20.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0530094 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0499584 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.054934 s, 19.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b6095000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff8800b6095000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00393446 s, 267 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00859686 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00416069 s, 252 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00872897 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779253 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00142717 s, 735 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00512679 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00427059 s, 246 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00123885 s, 846 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00124401 s, 843 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00125845 s, 833 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00169 s, 620 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892786 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123353 s, 85.0 MB/s , 0.00126947 s, 826 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00113727 s, 922 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77d: check TRR nrs policy ================ 21:06:26 (1713488786) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0875064 s, 192 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00843334 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953798 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133481 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00818989 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192776 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171815 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00640477 s, 164 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241623 s, 43.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332337 s, 31.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0342273 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154283 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353427 s, 29.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0268998 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397854 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222883 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310049 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00746717 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00735181 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010433 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00788944 s, 133 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0076034 s, 138 MB/s , 0.00844387 s, 124 MB/s , 0.00814059 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111537 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00996828 s, 105 MB/s , 0.00993623 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102427 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.010061 s, 104 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00972399 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.00962163 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107593 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106738 s, 98.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0964913 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100853 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114926 s, 91.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285095 s, 36.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0349484 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319634 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363764 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.045789 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477295 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0421291 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451141 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482017 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0560159 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0540178 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552262 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0527219 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0606841 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00730068 s, 144 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00688133 s, 152 MB/s , 0.00849966 s, 123 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00458904 s, 228 MB/s , 0.0059754 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00750197 s, 140 MB/s 1048576 bytes (1.0 MB) copied, 0.00756972 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00811602 s, 129 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00740835 s, 142 MB/s , 0.00746135 s, 141 MB/s , 0.0074597 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00776311 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00965004 s, 109 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00963776 s, 109 MB/s , 0.00962661 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983458 s, 107 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0962552 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887284 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108026 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161983 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191492 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00670645 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194871 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0066845 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149545 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00595667 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241277 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295433 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397744 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302848 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430136 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391396 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504208 s, 20.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00825879 s, 127 MB/s , 0.00743928 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107657 s, 97.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00868009 s, 121 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00924115 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.00141267 s, 742 MB/s , 0.00720144 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976924 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101462 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101792 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010469 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107593 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110191 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105355 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112034 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133115 s, 78.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg426-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77e: check TBF NID nrs policy ============ 21:06:31 (1713488791) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.26@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0828785 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105717 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0098803 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137402 s, 76.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036974 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356392 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412108 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376215 s, 27.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041107 s, 25.5 MB/s 1048576 bytes (1.0 MB) copied, 0.044996 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435954 s, 24.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0481211 s, 21.8 MB/s , 0.0463084 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416577 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0509113 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044851 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0465899 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00701177 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00723471 s, 145 MB/s , 0.00604183 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102555 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103651 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898564 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00998245 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978856 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982483 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100308 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099825 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108813 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105559 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107463 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110521 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106433 s, 98.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0829328 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167598 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016948 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0508921 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0538497 s, 19.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477429 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0486903 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0525058 s, 20.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0542577 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0613785 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0594245 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0509696 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0641068 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0631047 s, 16.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.058937 s, 17.8 MB/s , 0.0634532 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.062161 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00958132 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00688161 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00771075 s, 136 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0101658 s, 103 MB/s , 0.0126459 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103602 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095793 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114415 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114921 s, 91.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0117206 s, 89.5 MB/s , 0.011845 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125172 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140625 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129588 s, 80.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143594 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135482 s, 77.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0770326 s, 218 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125121 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164771 s, 63.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.045427 s, 23.1 MB/s , 0.0421439 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431071 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383951 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463169 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038076 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446883 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0529995 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464695 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.047469 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0502828 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454443 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0531846 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0522804 s, 20.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00762688 s, 137 MB/s , 0.00274724 s, 382 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092736 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00741669 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995053 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911017 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966347 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947887 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101007 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108129 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109813 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117106 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117165 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121498 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125369 s, 83.6 MB/s , 0.0121416 s, 86.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0856795 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106471 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111753 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472073 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459667 s, 22.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0508449 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437316 s, 24.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0443832 s, 23.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0534576 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042903 s, 24.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0546983 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474544 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0589585 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0527355 s, 19.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0519377 s, 20.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0549492 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0673507 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00755174 s, 139 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00847943 s, 124 MB/s , 0.00926407 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00743222 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826727 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00860174 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094765 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00497935 s, 211 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104356 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00917694 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00908054 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888171 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116362 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106709 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105263 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127196 s, 82.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77f: check TBF JobID nrs policy ========== 21:06:39 (1713488799) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0817495 s, 205 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00768536 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983202 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317253 s, 33.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255684 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332879 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404574 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043998 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044727 s, 23.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0380859 s, 27.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046042 s, 22.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0496846 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488539 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0512027 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450149 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464874 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483794 s, 21.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0666061 s, 15.7 MB/s , 0.067616 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0659195 s, 15.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0667471 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0663196 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0667006 s, 15.7 MB/s , 0.0670894 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0673098 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0679547 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0678575 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.067609 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0678132 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.069263 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0692376 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0684677 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0709088 s, 14.8 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.083114 s, 202 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180271 s, 58.2 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178098 s, 58.9 MB/s , 0.00959323 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138187 s, 75.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125858 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222901 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184189 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120193 s, 87.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00659588 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422937 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457443 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495515 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0578664 s, 18.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0547885 s, 19.1 MB/s , 0.0384537 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0616857 s, 17.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00532604 s, 197 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00824929 s, 127 MB/s , 0.00585095 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116941 s, 89.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00857183 s, 122 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00729179 s, 144 MB/s 1048576 bytes (1.0 MB) copied, 0.00955152 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00741827 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995868 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925124 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948381 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0096194 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105575 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114294 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00994427 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.0118556 s, 88.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0806368 s, 208 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00997552 s, 105 MB/s , 0.00943237 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383469 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041728 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459372 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05058 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0545187 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541459 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0513304 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0586537 s, 17.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.055836 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0596685 s, 17.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0601174 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0596936 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0576844 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0633598 s, 16.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00711727 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00812241 s, 129 MB/s 1048576 bytes (1.0 MB) copied, 0.0098891 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00700402 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079245 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804648 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00850868 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904029 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121149 s, 86.6 MB/s , 0.00964182 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109222 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101936 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113869 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108479 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102198 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114324 s, 91.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.095748 s, 175 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971031 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104957 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132857 s, 78.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130654 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00624554 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228753 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124358 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157952 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102591 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919613 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275344 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0542383 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386418 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411104 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0580355 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461031 s, 22.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00873477 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891477 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00777263 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108908 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108449 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925135 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880888 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00927879 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100205 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100372 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00754542 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926175 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957136 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968731 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106686 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108715 s, 96.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77g: Change TBF type directly ============ 21:06:46 (1713488806) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 21:06:51 (1713488811) oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg426-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg426-client: oleg426-server: ssh exited with exit code 2 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg426-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg426-client: oleg426-server: ssh exited with exit code 12 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg426-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg426-client: oleg426-server: ssh exited with exit code 12 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg426-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg426-client: oleg426-server: ssh exited with exit code 12 PASS 77h (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77i: Change rank of TBF rule ============= 21:06:55 (1713488815) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77j: check TBF-OPCode NRS policy ========= 21:07:02 (1713488822) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.099761 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918494 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140833 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151097 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354583 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300108 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033274 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041718 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450999 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0421003 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414878 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351692 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482838 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437154 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433344 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0458962 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398059 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126547 s, 82.9 MB/s , 0.00852712 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00839938 s, 125 MB/s , 0.00838738 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00509849 s, 206 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121127 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111507 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117428 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117544 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129971 s, 80.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.012744 s, 82.3 MB/s , 0.0143868 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146436 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136177 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142639 s, 73.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147923 s, 70.9 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg426-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93276 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4275 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (50s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 21:07:54 (1713488874) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg426-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg426-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg426-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg426-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22021 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107138 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223828 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295477 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227704 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226142 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802125 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184627 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332225 s, 31.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.174855 s, 6.0 MB/s , 0.174821 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.169868 s, 6.2 MB/s , 0.188245 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.18784 s, 5.6 MB/s , 0.184188 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.197722 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174258 s, 6.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895448 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916559 s, 114 MB/s , 0.00900236 s, 116 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0101685 s, 103 MB/s , 0.00861475 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00955017 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010488 s, 100 MB/s 1048576 bytes (1.0 MB) copied, 0.0105999 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882135 s, 119 MB/s 1048576 bytes (1.0 MB) copied, 0.00921153 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919998 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103291 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00965909 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983074 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01019 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108769 s, 96.4 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg426-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0284 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9662 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg426-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg426-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg426-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg426-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.21868 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120193 s, 87.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013594 s, 77.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138275 s, 75.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00512331 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218303 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00557339 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541277 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0550723 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172336 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173428 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.187796 s, 5.6 MB/s , 0.18501 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.197658 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.197243 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.199864 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.204897 s, 5.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00949641 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987565 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00799925 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986017 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106148 s, 98.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110102 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100355 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121026 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011373 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113654 s, 92.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128067 s, 81.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0118134 s, 88.8 MB/s , 0.0114687 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01164 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124804 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138724 s, 75.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg426-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0268 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9674 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (129s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 21:10:05 (1713489005) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg426-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg426-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg426-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg426-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.21953 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186781 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341443 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037871 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386687 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326326 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446187 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297078 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292296 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170967 s, 6.1 MB/s , 0.171034 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184536 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169863 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.177973 s, 5.9 MB/s , 0.188842 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191383 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177691 s, 5.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978334 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0097279 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.0101666 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103475 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108117 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102106 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976856 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102139 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011021 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131615 s, 79.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111172 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108559 s, 96.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116417 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115268 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112881 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134376 s, 78.0 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0311 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9768 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg426-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg426-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg426-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg426-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.23984 s, 13.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0309466 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317771 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365908 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346958 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432405 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015035 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282671 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263462 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146559 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148604 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158337 s, 6.6 MB/s 1048576 bytes (1.0 MB) copied, 0.155975 s, 6.7 MB/s , 0.140132 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172126 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.19457 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185648 s, 5.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126608 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109027 s, 96.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0131001 s, 80.0 MB/s , 0.013077 s, 80.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132714 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138071 s, 75.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0159997 s, 65.5 MB/s , 0.0138034 s, 76.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142045 s, 73.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.0167631 s, 62.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0165775 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164597 s, 63.7 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0185854 s, 56.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0169281 s, 61.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0198628 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175386 s, 59.8 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8505 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9136 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (133s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 21:12:21 (1713489141) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.301754 s, 55.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00839098 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916521 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0393118 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417224 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433039 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.04286 s, 24.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.047644 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430914 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0466463 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490009 s, 21.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0440343 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0443249 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0452145 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476483 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451489 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.049858 s, 21.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00735041 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111843 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010105 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0091566 s, 115 MB/s 1048576 bytes (1.0 MB) copied, 0.00835665 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934433 s, 112 MB/s 1048576 bytes (1.0 MB) copied, 0.00922685 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993604 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104198 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114818 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112946 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118776 s, 88.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113673 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123318 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125335 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133935 s, 78.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97116 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74539 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.26@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.26@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.252529 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161401 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113539 s, 92.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201855 s, 51.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0164651 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266858 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0901784 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0920138 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0924777 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0828253 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148481 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162456 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156932 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.249191 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.250831 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.296087 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.292983 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00740131 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00902915 s, 116 MB/s , 0.00662307 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00813884 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113915 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00836958 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869297 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888578 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00936569 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107816 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108909 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987323 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113004 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108662 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010945 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114363 s, 91.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95012 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74521 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.26@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.264547 s, 63.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189519 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188778 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472417 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0587614 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0650735 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0645753 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0687367 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0635391 s, 16.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0692478 s, 15.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0624883 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0721624 s, 14.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0699927 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0626711 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0620452 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0836408 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0709033 s, 14.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130129 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0132736 s, 79.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0189297 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152444 s, 68.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0184834 s, 56.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0205955 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144164 s, 72.7 MB/s , 0.0160206 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169245 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173018 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165941 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169111 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183662 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205512 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199866 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208248 s, 50.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95839 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87692 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.452503 s, 37.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151533 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387392 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463783 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335895 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0502409 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504928 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0513152 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459788 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420283 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0674449 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0758087 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0776934 s, 13.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.300846 s, 3.5 MB/s , 0.30286 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.35858 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.344901 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013876 s, 75.6 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153008 s, 68.5 MB/s , 0.0154963 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182026 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172546 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151249 s, 69.3 MB/s , 0.0162057 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189403 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193134 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246751 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203351 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203251 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0203245 s, 51.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0202831 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214431 s, 48.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021214 s, 49.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.84647 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74557 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96572 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74359 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.85308 s, 19.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255553 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191015 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025033 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152306 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00728886 s, 144 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0137303 s, 76.4 MB/s , 0.0241314 s, 43.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.15235 s, 6.9 MB/s , 0.149654 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192115 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185229 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1808 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179415 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.198213 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178113 s, 5.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125466 s, 83.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196253 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166212 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152643 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156569 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0162716 s, 64.4 MB/s , 0.0161722 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016814 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173668 s, 60.4 MB/s , 0.0160716 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158906 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196512 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181728 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168721 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181073 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197249 s, 53.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7032 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4438 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.256507 s, 65.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161532 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164935 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0602626 s, 17.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0491974 s, 21.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0535618 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0675195 s, 15.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.062541 s, 16.8 MB/s , 0.0540252 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0667734 s, 15.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0729878 s, 14.4 MB/s , 0.0585976 s, 17.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0696385 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0627695 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0693498 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0619514 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0657072 s, 16.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0182517 s, 57.5 MB/s , 0.0187342 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137243 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162046 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160605 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014711 s, 71.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01584 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182146 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190335 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185093 s, 56.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182312 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183326 s, 57.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018985 s, 55.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211657 s, 49.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0210735 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213984 s, 49.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95349 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7413 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.439807 s, 38.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140448 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0478394 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0478905 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394023 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0530595 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0534177 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449544 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484091 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487346 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0813621 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0761147 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0782743 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.300646 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.309991 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.353754 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.358818 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110892 s, 94.6 MB/s , 0.011033 s, 95.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111421 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139237 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124175 s, 84.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130596 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143323 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138212 s, 75.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144154 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144233 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145151 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150478 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156051 s, 67.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0143006 s, 73.3 MB/s , 0.0153077 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172828 s, 60.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.84594 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74731 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76539 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72564 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (351s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 21:18:15 (1713489495) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 21:18:20 (1713489500) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg426-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.09406 s, 339 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg426-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0161 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 21:19:12 (1713489552) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0787579 s, 213 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014108 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141013 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166286 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117036 s, 89.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114001 s, 92.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0273549 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027558 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135982 s, 77.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980465 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279275 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332182 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405856 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364698 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483904 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391316 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0560233 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011185 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916183 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100564 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111543 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095767 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00952792 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102451 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101805 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01158 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103423 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114773 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114399 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115362 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0130561 s, 80.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0122346 s, 85.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150789 s, 69.5 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg426-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95554 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97572 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.171227 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119012 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190158 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206376 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307011 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.050963 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337543 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438111 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495626 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0557625 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434236 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152482 s, 6.9 MB/s , 0.15558 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.194137 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191373 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.267684 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.259008 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114235 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116392 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134815 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147218 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126433 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128445 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132877 s, 78.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123874 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136751 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140291 s, 74.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0138395 s, 75.8 MB/s , 0.0142672 s, 73.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147289 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143052 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160505 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162139 s, 64.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg426-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg426-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94616 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9742 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (62s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77o: Changing rank should not panic ====== 21:20:16 (1713489616) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 21:20:23 (1713489623) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (35s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 21:21:00 (1713489660) mds.MDS.mdt.nrs_policies=tbf oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg426-client: oleg426-server: ssh exited with exit code 36 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 oleg426-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg426-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg426-client: oleg426-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (13s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77r: Change type of tbf policy at run time ========================================================== 21:21:15 (1713489675) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 25834 kill md_thread xargs PASS 77r (36s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 78: Enable policy and specify tunings right away ========================================================== 21:21:54 (1713489714) ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 79: xattr: intent error ================== 21:21:59 (1713489719) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506731 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-18 21:22:00.000000000 -0400 Modify: 2024-04-18 21:22:00.000000000 -0400 Change: 2024-04-18 21:22:00.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 80a: migrate directory when some children is being opened ========================================================== 21:22:14 (1713489734) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 80b: Accessing directory during migration ========================================================== 21:22:18 (1713489738) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81a: rename and stat under striped directory ========================================================== 21:22:21 (1713489741) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 21:22:24 (1713489744) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 21:22:27 (1713489747) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 21:22:30 (1713489750) total: 2500 open/close in 9.64 seconds: 259.38 ops/second snapshot_time 1713489780.734846693 secs.nsecs start_time 1713489751.236574474 secs.nsecs elapsed_time 29.498272219 secs.nsecs open 2505 samples [usecs] 96 2775 1276464 711247804 close 2504 samples [usecs] 63 2125 377494 68070672 mknod 2501 samples [usecs] 201 2724 1150188 582684214 unlink 2500 samples [usecs] 300 19597 1771838 2581269596 mkdir 2 samples [usecs] 881 1133 2014 2059850 rmdir 2 samples [usecs] 3147 4332 7479 28669833 rename 2500 samples [usecs] 243 13273 3046333 6444142571 getattr 3762 samples [usecs] 8 557 83992 3031952 parallel_rename_file 2500 samples [usecs] 243 13273 3046333 6444142571 crossdir_rename 2500 samples [usecs] 243 13273 3046333 6444142571 PASS 81d (33s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 21:23:05 (1713489785) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6897 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6897 PASS 82 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 21:23:10 (1713489790) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 84: 0-nlink race in lu_object_find() ===== 21:23:13 (1713489793) fail_loc=0x8000060b PASS 84 (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 85: Lustre API root cache race =========== 21:23:23 (1713489803) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1713489804 Finishing test test1 at 1713489805 PASS 85 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 90: open/create and unlink striped directory ========================================================== 21:23:29 (1713489809) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 91: chmod and unlink striped directory === 21:23:32 (1713489812) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 92: create remote directory under orphan directory ========================================================== 21:23:36 (1713489816) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 21:23:39 (1713489819) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6145 0x1801 0x280000400 0 6195 0x1833 0x240000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6196 0x1834 0x240000400 1 6146 0x1802 0x280000400 rc1=0 and rc2=0 PASS 93 (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 94: signal vs CP callback race =========== 21:23:50 (1713489830) osc.lustre-OST0000-osc-ffff8800aaa91800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b6095000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800aaa91800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b6095000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0653511 s, 125 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.0007 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5667: 16366 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0180635 s, 227 kB/s osc.lustre-OST0000-osc-ffff8800aaa91800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b6095000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aaa91800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b6095000.idle_timeout=debug PASS 94 (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 21:24:03 (1713489843) llite.lustre-ffff8800aaa91800.fast_read=0 llite.lustre-ffff8800b6095000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0216152 s, 97.0 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8800aaa91800.fast_read=1 llite.lustre-ffff8800b6095000.fast_read=1 PASS 95a (20s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 21:24:25 (1713489865) llite.lustre-ffff8800aaa91800.fast_read=0 llite.lustre-ffff8800b6095000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00172111 s, 7.1 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800aaa91800.fast_read=1 llite.lustre-ffff8800b6095000.fast_read=1 PASS 95b (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 21:24:36 (1713489876) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 21:24:39 (1713489879) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00185982 s, 2.2 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 21:24:44 (1713489884) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0435662 s, 48.1 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 21:24:49 (1713489889) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0280702 s, 74.7 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100e: DoM: read on open and file size ==== 21:24:53 (1713489893) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00309557 s, 3.9 MB/s PASS 100e (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101a: Discard DoM data on unlink ========= 21:24:58 (1713489898) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00089303 s, 4.6 MB/s PASS 101a (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101b: Discard DoM data on rename ========= 21:25:03 (1713489903) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000836104 s, 4.9 MB/s PASS 101b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101c: Discard DoM data on close-unlink === 21:25:08 (1713489908) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00085934 s, 4.8 MB/s PASS 101c (3s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_102 skipping ALWAYS excluded test 102 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 103: Test size correctness with lockahead ========================================================== 21:25:14 (1713489914) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1713489915 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1713489918 Finishing test test23 at 1713489918 PASS 103 (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 21:25:24 (1713489924) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 105: Glimpse and lock cancel race ======== 21:25:27 (1713489927) fail_loc=0x416 PASS 105 (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106a: Verify the btime via statx() ======= 21:25:51 (1713489951) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106b: Glimpse RPCs test for statx ======== 21:25:55 (1713489955) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106c: Verify statx attributes mask ======= 21:25:59 (1713489959) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 107a: Basic grouplock conflict =========== 21:26:03 (1713489963) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.113146 s, 92.7 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6151 0x1807 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 First grouplock blocks second one PASS 107a (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 21:26:10 (1713489970) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0604253 s, 174 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6204 0x183c 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6897 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 108a: lseek: parallel updates ============ 21:26:21 (1713489981) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00324851 s, 3.1 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.27828 s, 2.0 MB/s PASS 108a (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 109: Race with several mount instances on 1 node ========================================================== 21:26:30 (1713489990) 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 12 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 17 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 25 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 29 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 30 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 33 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 35 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/pcc_cpufreq/refcnt': 'No such device' oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 37 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 38 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 40 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 48 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 49 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 192.168.204.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg426-server' oleg426-server: oleg426-server.virtnet: executing load_modules_local oleg426-server: Loading modules from /home/green/git/lustre-release/lustre oleg426-server: detected 4 online CPUs by sysfs oleg426-server: Force libcfs to create 2 CPU partitions oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg426-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre Starting client: oleg426-client.virtnet: -o user_xattr,flock oleg426-server@tcp:/lustre /mnt/lustre2 PASS 109 (456s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 110: do not grant another lock on resend ========================================================== 21:34:09 (1713490449) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-18 21:34:09.000000000 -0400 Modify: 2024-04-18 21:34:09.000000000 -0400 Change: 2024-04-18 21:34:09.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg426-client.virtnet oleg426-client.virtnet PASS 110 (65s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 21:35:16 (1713490516) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 112: update max-inherit in default LMV === 21:35:19 (1713490519) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 113: check servers of specified fs ======= 21:35:22 (1713490522) lustre-OST0000-osc-ffff8800b368e800 active. lustre-OST0001-osc-ffff8800b368e800 active. lustre-OST0000-osc-ffff8800b368e800 active. lustre-OST0001-osc-ffff8800b368e800 active. MGC192.168.204.126@tcp active. PASS 113 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 114: implicit default LMV inherit ======== 21:35:25 (1713490525) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 21:35:28 (1713490528) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (1s) debug_raw_pointers=0 debug_raw_pointers=0 cleanup: ====================================================== == sanityn test complete, duration 4760 sec ============== 21:35:30 (1713490530) === sanityn: start cleanup 21:35:30 (1713490530) === Stopping clients: oleg426-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg426-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 21:36:26 (1713490586) ===