-----============= acceptance-small: sanityn ============----- Tue Apr 16 14:58:05 EDT 2024 excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 14:58:09 (1713293889) === Starting client oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Started clients oleg144-client.virtnet: 192.168.201.144@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg144-client.virtnet: executing check_config_client /mnt/lustre oleg144-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg144-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff88012b467000.idle_timeout=debug osc.lustre-OST0000-osc-ffff88013738e000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012b467000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88013738e000.idle_timeout=debug disable quota as required oleg144-server: oleg144-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 14:58:15 (1713293895) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109348 s, 95.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6920] debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 14:58:16 (1713293896) PASS 0 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 14:58:20 (1713293900) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 14:58:23 (1713293903) -rw-r--r-- 1 root root 0 Apr 16 14:58 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 14:58:26 (1713293906) -rw-r--r-- 1 root root 0 Apr 16 14:58 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 14:58:28 (1713293908) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 14:58:31 (1713293911) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2e: check chmod on root is propagated to others ========================================================== 14:58:34 (1713293914) total 1 -rwxrwxrwx 1 root root 0 Apr 16 14:58 f2a -rwxrwxrwx 1 root root 0 Apr 16 14:58 f2b total 1 -rwxrwxrwx 1 root root 0 Apr 16 14:58 f2a -rwxrwxrwx 1 root root 0 Apr 16 14:58 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00303527 s, 169 kB/s PASS 2e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 14:58:37 (1713293917) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2g: check blocks update on sync write ==== 14:58:39 (1713293919) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.076996 s, 27.2 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 14:58:42 (1713293922) PASS 3 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 4: fstat validation on multiple mount points ==================================================================== 14:58:44 (1713293924) PASS 4 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 14:58:48 (1713293928) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 6: remove of open file on other node ============================================================================ 14:58:51 (1713293931) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 7: remove of open directory on other node ======================================================================= 14:58:53 (1713293933) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 8: remove of open special file on other node ==================================================================== 14:58:56 (1713293936) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 14:58:59 (1713293939) PASS 9a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 9b: append to striped sparse file ======== 14:59:01 (1713293941) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00828434 s, 127 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.00901404 s, 0.3 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 14:59:04 (1713293944) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00118786 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00217004 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00186936 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00207938 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00200005 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00231498 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00236738 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00183291 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00200746 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00190638 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00167515 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00182398 s, 0.5 kB/s PASS 10a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 14:59:07 (1713293947) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00196534 s, 1.6 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00560617 s, 731 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000143665 s, 21.4 MB/s PASS 10b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 11: execution of file opened for write should return error ============================================================== 14:59:10 (1713293950) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 14:59:13 (1713293953) warning: '-runtime' deprecated, use '-t runtime' instead - link 3410 (time 1713293963.60 total 10.00 last 340.94) - link 6677 (time 1713293973.61 total 20.00 last 326.63) - link 9761 (time 1713293983.61 total 30.01 last 308.31) - link 12810 (time 1713293993.61 total 40.01 last 304.87) - link 16077 (time 1713294003.61 total 50.01 last 326.65) - link 19547 (time 1713294013.61 total 60.01 last 346.91) - link 23117 (time 1713294023.62 total 70.01 last 356.90) - link 26657 (time 1713294033.62 total 80.02 last 353.91) - link 30000 (time 1713294042.84 total 89.24 last 362.56) - link 33538 (time 1713294052.84 total 99.24 last 353.70) total: 33816 link in 100.00 seconds: 338.16 ops/second using seed 883190220 running for 100 seconds - stat 10000 (time 1713293957 ; total 3 ; last 3) - stat 20000 (time 1713293961 ; total 7 ; last 4) - stat 30000 (time 1713293964 ; total 10 ; last 3) - stat 40000 (time 1713293968 ; total 14 ; last 4) - stat 50000 (time 1713293971 ; total 17 ; last 3) - stat 60000 (time 1713293974 ; total 20 ; last 3) - stat 70000 (time 1713293978 ; total 24 ; last 4) - stat 80000 (time 1713293982 ; total 28 ; last 4) - stat 90000 (time 1713293985 ; total 31 ; last 3) - stat 100000 (time 1713293989 ; total 35 ; last 4) - stat 110000 (time 1713293992 ; total 38 ; last 3) - stat 120000 (time 1713293996 ; total 42 ; last 4) - stat 130000 (time 1713293999 ; total 45 ; last 3) - stat 140000 (time 1713294003 ; total 49 ; last 4) - stat 150000 (time 1713294009 ; total 55 ; last 6) - stat 160000 (time 1713294019 ; total 65 ; last 10) - stat 170000 (time 1713294029 ; total 75 ; last 10) - stat 180000 (time 1713294038 ; total 84 ; last 9) - stat 190000 (time 1713294047 ; total 93 ; last 9) total: 197288 stats in 100 seconds: 1972.880005 stats/second - unlinked 0 (time 1713294004 ; total 0 ; last 0) - unlinked 10000 (time 1713294047 ; total 43 ; last 43) - unlinked 20000 (time 1713294073 ; total 69 ; last 26) - unlinked 30000 (time 1713294097 ; total 93 ; last 24) unlink(/mnt/lustre2/lockdir/lockfile33816) error: No such file or directory total: 33815 unlinks in 102 seconds: 331.519623 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (23402) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (23405) - No such process PASS 12 (154s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 13: test directory page revocation ======= 15:01:49 (1713294109) f13 f13 PASS 13 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 15:01:52 (1713294112) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4697: 25009 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 15:01:55 (1713294115) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4697: 25747 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 15:01:58 (1713294118) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 26485 Terminated $DIR1/$tdir/sleep 60 PASS 14b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 15:02:00 (1713294120) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 27223 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14d: chmod of executing file is still possible ================================================================== 15:02:03 (1713294123) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 27963 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 15: test out-of-space with multiple writers ===================================================================== 15:02:07 (1713294127) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7521280 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7521280kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16a: 500 iterations of dual-mount fsx ==== 15:02:19 (1713294139) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.128649 s, 65.2 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.115194 s, 72.8 MB/s Chance of close/open is 1 in 50 Seed set to 4140 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000003[1] 1713294140.561457 mapwrite 0x97f247 thru 0x985f9e (0x6d58 bytes) 000005[0] 1713294140.602031 mapwrite 0x8b1877 thru 0x8b8900 (0x708a bytes) 000009[0] 1713294140.638422 write 0x792f07 thru 0x79686b (0x3965 bytes) 000010[1] 1713294140.646416 mapread 0x05be83 thru 0x05c80d (0x98b bytes) 000011[0] 1713294140.653651 read 0x365dd6 thru 0x369abe (0x3ce9 bytes) 000012[0] 1713294140.658860 write 0x29deee thru 0x2a61b2 (0x82c5 bytes) truncating to largest ever: 0xa3506 000013[0] 1713294140.663236 trunc from 0x985f9f to 0x0a3505 (0x8e2a99 bytes) 000015[0] 1713294140.668467 mapwrite 0x2c04be thru 0x2c4809 (0x434c bytes) 000017[1] 1713294140.691735 mapwrite 0x3c653a thru 0x3cb6fe (0x51c5 bytes) 000019[0] 1713294140.723410 mapwrite 0x5edcee thru 0x5f6d78 (0x908b bytes) 000020[1] 1713294140.749467 mapread 0x2a1d8a thru 0x2ad5f0 (0xb867 bytes) truncating to largest ever: 0x58048e 000022[1] 1713294140.759876 trunc from 0x5f6d79 to 0x58048d (0x768eb bytes) 000023[0] 1713294140.770548 write 0x56d71b thru 0x57a2ca (0xcbb0 bytes) 000024[1] 1713294140.778277 mapread 0x202313 thru 0x20cd28 (0xaa16 bytes) 000028[1] 1713294140.797043 write 0x898341 thru 0x8a446e (0xc12e bytes) 000029[0] 1713294140.801561 write 0x3f0f57 thru 0x3f7788 (0x6832 bytes) 000033[0] 1713294140.815037 mapread 0x0a533d thru 0x0ac6c4 (0x7388 bytes) truncating to largest ever: 0x85221b 000034[0] 1713294140.820162 trunc from 0x8a446f to 0x85221a (0x52254 bytes) 000035[1] 1713294140.828107 mapwrite 0x0f8852 thru 0x108363 (0xfb12 bytes) 000037[0] 1713294140.861867 write 0x2ce976 thru 0x2d0804 (0x1e8f bytes) 000039[1] 1713294140.867164 mapwrite 0x32720d thru 0x330412 (0x9206 bytes) 000040[0] 1713294140.893194 read 0x3d6b58 thru 0x3e4ed8 (0xe381 bytes) 000043[0] 1713294140.930557 trunc from 0x85221b to 0x81779f (0x3aa7b bytes) 000044[0] 1713294140.941395 write 0x57e5c2 thru 0x58daf4 (0xf533 bytes) 000047[1] 1713294140.947314 mapwrite 0x5ba919 thru 0x5bfddd (0x54c5 bytes) 000051[0] 1713294140.986274 trunc from 0x8177a0 to 0x2cc4f8 (0x54b2a7 bytes) 000054[1] 1713294140.999547 write 0x035a08 thru 0x03b9cc (0x5fc5 bytes) 000056[0] 1713294141.009150 mapwrite 0x1f3146 thru 0x1ff151 (0xc00c bytes) 000057[1] 1713294141.040998 mapwrite 0x48254d thru 0x485670 (0x3124 bytes) 000058[1] 1713294141.069232 write 0x37332e thru 0x3774d6 (0x41a9 bytes) 000060[0] 1713294141.075846 mapread 0x081e0c thru 0x08c5be (0xa7b3 bytes) 000062[0] 1713294141.094139 trunc from 0x485671 to 0x44ac10 (0x3aa60 bytes) 000066[0] 1713294141.109274 trunc from 0x44ac11 to 0x6063bd (0x1bb7ad bytes) 000069[0] 1713294141.124169 read 0x05a37e thru 0x05bfab (0x1c2e bytes) 000073[1] 1713294141.154749 trunc from 0x6063be to 0x708b73 (0x1027b6 bytes) 000074[1] 1713294141.168057 read 0x270bde thru 0x27a910 (0x9d33 bytes) 000076[0] 1713294141.174169 trunc from 0x708b74 to 0x616508 (0xf266b bytes) 000080[1] 1713294141.194151 mapread 0x06934e thru 0x071a05 (0x86b8 bytes) 000081[0] 1713294141.203768 write 0x4baf67 thru 0x4bf142 (0x41dc bytes) 000082[0] 1713294141.207161 trunc from 0x616509 to 0x187e54 (0x48e6b4 bytes) 000083[1] 1713294141.213754 mapread 0x078b84 thru 0x087940 (0xedbd bytes) truncating to largest ever: 0x988245 000084[0] 1713294141.225352 trunc from 0x187e55 to 0x988244 (0x8003f0 bytes) 000087[0] 1713294141.237558 read 0x224468 thru 0x22465f (0x1f8 bytes) 000090[0] 1713294141.259373 mapread 0x8834de thru 0x8887d3 (0x52f6 bytes) 000091[1] 1713294141.277844 mapwrite 0x2501cc thru 0x250ae6 (0x91b bytes) 000092[1] 1713294141.303368 trunc from 0x988245 to 0x86ad4a (0x11d4fa bytes) 000093[1] 1713294141.312226 mapwrite 0x3ef218 thru 0x3efa27 (0x810 bytes) 000094[0] 1713294141.329032 write 0x4e0169 thru 0x4eed05 (0xeb9d bytes) 000098[0] 1713294141.346527 trunc from 0x86ad4b to 0x095364 (0x7d59e6 bytes) 000099[0] 1713294141.355908 write 0x9b194c thru 0x9bb262 (0x9917 bytes) 000100[0] 1713294141.359247 trunc from 0x9bb263 to 0x2f8340 (0x6c2f22 bytes) 000104[0] 1713294141.370869 mapwrite 0x8d5cd2 thru 0x8daf5d (0x528c bytes) 000106[0] 1713294141.403519 write 0x14080a thru 0x149427 (0x8c1e bytes) 000109[0] 1713294141.410553 mapread 0x12b994 thru 0x138802 (0xce6f bytes) 000111[1] 1713294141.432034 mapwrite 0x09b2e8 thru 0x0aa017 (0xed30 bytes) 000112[0] 1713294141.459653 write 0x89dcfc thru 0x8ad07f (0xf384 bytes) 000113[1] 1713294141.467197 write 0x65eecd thru 0x66e9d0 (0xfb04 bytes) 000114[1] 1713294141.474510 mapread 0x06dd07 thru 0x077e8d (0xa187 bytes) 000115[0] 1713294141.487086 mapwrite 0x8afb4c thru 0x8b3812 (0x3cc7 bytes) 000117[0] 1713294141.514893 trunc from 0x8daf5e to 0x9778ce (0x9c971 bytes) 000118[1] 1713294141.523506 write 0x7bc018 thru 0x7ca359 (0xe342 bytes) 000119[1] 1713294141.531613 mapwrite 0x496cbc thru 0x497b24 (0xe69 bytes) 000126[1] 1713294141.575137 write 0x047f6e thru 0x048888 (0x91b bytes) 000127[1] 1713294141.582465 trunc from 0x9778cf to 0x8d3833 (0xa409b bytes) 000128[0] 1713294141.589675 trunc from 0x8d3834 to 0x81da52 (0xb5de1 bytes) 000130[1] 1713294141.601786 mapread 0x0f0e0c thru 0x0fd472 (0xc667 bytes) 000131[0] 1713294141.612832 write 0x5fbc74 thru 0x5fcca5 (0x1032 bytes) 000133[0] 1713294141.619287 mapwrite 0x5e033f thru 0x5ecb9e (0xc860 bytes) 000134[1] 1713294141.646949 write 0x0fce0c thru 0x0fe5c7 (0x17bc bytes) 000136[0] 1713294141.658419 write 0x5cd3db thru 0x5da921 (0xd547 bytes) 000140[0] 1713294141.674020 write 0x635fa2 thru 0x63ed2e (0x8d8d bytes) 000141[0] 1713294141.679673 read 0x31ae99 thru 0x325f33 (0xb09b bytes) 000143[0] 1713294141.708588 write 0x8d8709 thru 0x8e4895 (0xc18d bytes) 000145[1] 1713294141.718066 write 0x29bcf2 thru 0x2a8a6c (0xcd7b bytes) 000146[1] 1713294141.727544 mapread 0x35f5c9 thru 0x361238 (0x1c70 bytes) 000148[1] 1713294141.740693 read 0x748c9a thru 0x74bf05 (0x326c bytes) 000149[1] 1713294141.749554 read 0x32276e thru 0x330b03 (0xe396 bytes) 000156[0] 1713294141.753974 mapwrite 0x89969c thru 0x899de8 (0x74d bytes) 000157[1] 1713294141.776751 mapread 0x72338b thru 0x72db0e (0xa784 bytes) 000158[0] 1713294141.793340 read 0x3be4a7 thru 0x3be59d (0xf7 bytes) 000159[0] 1713294141.806591 write 0x38d02d thru 0x38d85a (0x82e bytes) 000164[0] 1713294141.820261 mapread 0x16603b thru 0x16a0f2 (0x40b8 bytes) 000166[0] 1713294141.823839 trunc from 0x8e4896 to 0x542d0f (0x3a1b86 bytes) 000167[1] 1713294141.833994 mapread 0x3adf9a thru 0x3b2bdf (0x4c46 bytes) 000169[0] 1713294141.857193 mapread 0x409764 thru 0x418bf0 (0xf48d bytes) 000171[1] 1713294141.868246 mapread 0x002a3d thru 0x00ef2f (0xc4f3 bytes) 000172[0] 1713294141.870203 mapwrite 0x806f90 thru 0x808d69 (0x1dda bytes) 000177[1] 1713294141.912134 trunc from 0x808d6a to 0x12f76c (0x6d95fd bytes) 000180[1] 1713294141.928610 mapread 0x060a08 thru 0x06147a (0xa73 bytes) 000182[1] 1713294141.934656 mapwrite 0x21d644 thru 0x2201b3 (0x2b70 bytes) 000183[1] 1713294141.960549 read 0x21be84 thru 0x2201b3 (0x4330 bytes) 000184[1] 1713294141.972462 mapread 0x1a77d6 thru 0x1ab3a4 (0x3bcf bytes) 000185[0] 1713294141.975922 write 0x682ebf thru 0x689038 (0x617a bytes) 000186[1] 1713294141.983361 mapwrite 0x49005b thru 0x491ebf (0x1e65 bytes) 000187[1] 1713294142.004934 mapwrite 0x4177d9 thru 0x4203d9 (0x8c01 bytes) 000188[0] 1713294142.031934 write 0x62c521 thru 0x630839 (0x4319 bytes) 000189[0] 1713294142.039685 trunc from 0x689039 to 0x3e4f0d (0x2a412b bytes) 000192[1] 1713294142.052114 write 0x78dd19 thru 0x79754a (0x9832 bytes) 000194[0] 1713294142.060991 mapread 0x0d8c6c thru 0x0e4cc6 (0xc05b bytes) 000196[0] 1713294142.081552 trunc from 0x79754b to 0x511175 (0x2863d5 bytes) 000198[1] 1713294142.089781 trunc from 0x511176 to 0x82320d (0x312098 bytes) 000201[1] 1713294142.107241 mapwrite 0x96d650 thru 0x97aafc (0xd4ad bytes) 000202[0] 1713294142.156570 trunc from 0x97aafd to 0x828830 (0x1522cc bytes) 000203[0] 1713294142.169535 read 0x81d512 thru 0x828830 (0xb31f bytes) 000207[1] 1713294142.201471 read 0x49dcf7 thru 0x4ad9ce (0xfcd8 bytes) 000210[0] 1713294142.232170 write 0x5e2946 thru 0x5ec236 (0x98f1 bytes) 000213[0] 1713294142.240121 mapwrite 0x3c254f thru 0x3c4ab3 (0x2565 bytes) 000214[0] 1713294142.252331 read 0x6fbe20 thru 0x6fce1a (0xffb bytes) 000217[0] 1713294142.272256 write 0x7a22a1 thru 0x7a3673 (0x13d3 bytes) 000220[1] 1713294142.276145 mapread 0x00f9a5 thru 0x015295 (0x58f1 bytes) 000221[1] 1713294142.295418 write 0x9cfe29 thru 0x9da97c (0xab54 bytes) 000223[1] 1713294142.307597 write 0x5a59f7 thru 0x5b0627 (0xac31 bytes) 000224[1] 1713294142.331767 mapwrite 0x10179f thru 0x10e4be (0xcd20 bytes) 000226[1] 1713294142.368189 read 0x51ffcc thru 0x527018 (0x704d bytes) 000227[1] 1713294142.372148 mapread 0x5c9059 thru 0x5ce59b (0x5543 bytes) 000228[1] 1713294142.372528 trunc from 0x9da97d to 0x03ef79 (0x99ba03 bytes) 000229[1] 1713294142.384384 mapread 0x0087b4 thru 0x00c11f (0x396c bytes) 000233[1] 1713294142.399512 mapwrite 0x2f9a4d thru 0x2fe2df (0x4893 bytes) 000234[1] 1713294142.432022 mapwrite 0x7b42e5 thru 0x7b486b (0x587 bytes) 000236[1] 1713294142.459091 read 0x276282 thru 0x280b21 (0xa8a0 bytes) 000237[1] 1713294142.460416 read 0x0375d2 thru 0x039860 (0x228f bytes) 000239[1] 1713294142.460828 mapread 0x57e6c9 thru 0x58dfa6 (0xf8de bytes) 000243[1] 1713294142.483078 trunc from 0x7b486c to 0x31a16f (0x49a6fc bytes) 000246[1] 1713294142.496533 mapwrite 0x66b465 thru 0x67a424 (0xefc0 bytes) 000247[0] 1713294142.534619 mapwrite 0x3f0b1b thru 0x3f4196 (0x367c bytes) 000249[0] 1713294142.566390 mapread 0x58277b thru 0x58e660 (0xbee6 bytes) 000251[0] 1713294142.589500 mapread 0x3a53c9 thru 0x3b1161 (0xbd99 bytes) 000256[1] 1713294142.618507 read 0x3fa94e thru 0x3faaac (0x15f bytes) 000257[0] 1713294142.633673 read 0x5fb0d5 thru 0x60981b (0xe747 bytes) 000259[0] 1713294142.648523 trunc from 0x67a425 to 0x4168bd (0x263b67 bytes) truncating to largest ever: 0x9dd6a8 000260[1] 1713294142.662981 trunc from 0x4168be to 0x9dd6a7 (0x5c6dea bytes) 000261[0] 1713294142.673883 mapread 0x4f6305 thru 0x4fff1b (0x9c17 bytes) 000262[1] 1713294142.690602 mapread 0x73fab6 thru 0x741014 (0x155f bytes) 000268[1] 1713294142.714291 write 0x494eef thru 0x4a078a (0xb89c bytes) 000271[1] 1713294142.726882 trunc from 0x9dd6a8 to 0x4b72c5 (0x5263e2 bytes) 000273[1] 1713294142.732488 read 0x0894b7 thru 0x08fd0e (0x6858 bytes) 000275[1] 1713294142.749700 write 0x35b365 thru 0x35db2a (0x27c6 bytes) 000277[0] 1713294142.755902 write 0x6c463b thru 0x6c9600 (0x4fc6 bytes) 000279[1] 1713294142.768211 mapread 0x1b3fda thru 0x1b8d9c (0x4dc3 bytes) 000280[1] 1713294142.773577 trunc from 0x6c9601 to 0x8e568b (0x21c08b bytes) 000281[1] 1713294142.785455 mapwrite 0x103e83 thru 0x112897 (0xea15 bytes) 000282[0] 1713294142.803629 mapread 0x43fc13 thru 0x44a4a4 (0xa892 bytes) 000284[1] 1713294142.824417 write 0x144517 thru 0x14c396 (0x7e80 bytes) 000285[1] 1713294142.825067 write 0x998a60 thru 0x9a75e7 (0xeb88 bytes) 000290[1] 1713294142.832646 trunc from 0x9a75e8 to 0x8added (0xf97fa bytes) 000291[1] 1713294142.841701 read 0x436101 thru 0x439667 (0x3567 bytes) 000293[0] 1713294142.859615 mapwrite 0x38cebc thru 0x38d7a5 (0x8ea bytes) 000294[1] 1713294142.896853 trunc from 0x8addee to 0x7f1e3e (0xbbfaf bytes) 000296[0] 1713294142.910469 write 0x54e527 thru 0x552c65 (0x473f bytes) 000297[1] 1713294142.930849 trunc from 0x7f1e3f to 0x33a52a (0x4b7914 bytes) 000298[1] 1713294142.944471 mapread 0x04bcef thru 0x0572d3 (0xb5e5 bytes) 000299[1] 1713294142.960524 trunc from 0x33a52b to 0x4c60da (0x18bbb0 bytes) 000300[0] 1713294142.967885 write 0x233097 thru 0x23c945 (0x98af bytes) 000301[1] 1713294142.975320 read 0x1c88da thru 0x1d376e (0xae95 bytes) 000302[0] 1713294142.997537 trunc from 0x4c60db to 0x7e0da8 (0x31acce bytes) 000304[1] 1713294143.015628 read 0x5508e3 thru 0x559a7d (0x919b bytes) 000306[1] 1713294143.032107 read 0x450b03 thru 0x453a25 (0x2f23 bytes) 000307[0] 1713294143.039766 read 0x07773a thru 0x07e04d (0x6914 bytes) 000310[0] 1713294143.058546 trunc from 0x7e0da9 to 0x6a14f0 (0x13f8b8 bytes) 000313[0] 1713294143.069588 trunc from 0x6a14f1 to 0x834ab5 (0x1935c5 bytes) 000315[1] 1713294143.072978 trunc from 0x834ab6 to 0x0c2311 (0x7727a4 bytes) 000318[0] 1713294143.093357 write 0x5dd39e thru 0x5e9803 (0xc466 bytes) 000320[0] 1713294143.104560 mapread 0x43edc1 thru 0x43fdc7 (0x1007 bytes) 000324[1] 1713294143.121887 trunc from 0x5e9804 to 0x626524 (0x3cd21 bytes) 000326[0] 1713294143.134277 write 0x0632de thru 0x06e9b5 (0xb6d8 bytes) 000329[0] 1713294143.148696 mapread 0x4be071 thru 0x4c38fb (0x588b bytes) 000330[0] 1713294143.161546 mapwrite 0x99bea4 thru 0x99c3d0 (0x52d bytes) 000332[1] 1713294143.195629 mapread 0x699dc5 thru 0x6a3aaa (0x9ce6 bytes) 000333[1] 1713294143.210643 mapread 0x024d22 thru 0x0279ce (0x2cad bytes) 000334[0] 1713294143.229399 trunc from 0x99c3d1 to 0x7642bc (0x238114 bytes) 000335[0] 1713294143.243742 mapread 0x3177eb thru 0x325cac (0xe4c2 bytes) 000336[1] 1713294143.261644 mapread 0x4395b7 thru 0x4429c7 (0x9411 bytes) 000337[0] 1713294143.282503 mapwrite 0x58d3d7 thru 0x58fe78 (0x2aa2 bytes) 000338[1] 1713294143.307312 trunc from 0x7642bd to 0x20ddf9 (0x5564c3 bytes) 000339[0] 1713294143.316530 trunc from 0x20ddfa to 0x85ab1d (0x64cd24 bytes) 000340[0] 1713294143.327429 write 0x6f9e03 thru 0x703f4c (0xa14a bytes) 000341[1] 1713294143.331013 mapread 0x7fe4fd thru 0x80b1d8 (0xccdc bytes) 000343[0] 1713294143.349130 mapread 0x2b35e2 thru 0x2bfd68 (0xc787 bytes) 000344[1] 1713294143.363632 trunc from 0x85ab1e to 0x6e66e3 (0x17443a bytes) 000346[0] 1713294143.373336 read 0x08805b thru 0x092166 (0xa10c bytes) 000347[0] 1713294143.389807 mapwrite 0x07b9b0 thru 0x0818eb (0x5f3c bytes) 000350[1] 1713294143.438968 write 0x79d7ca thru 0x7a4fdf (0x7816 bytes) 000353[1] 1713294143.449744 mapread 0x6e8ced thru 0x6f47ad (0xbac1 bytes) 000354[0] 1713294143.468688 trunc from 0x7a4fe0 to 0x683276 (0x121d69 bytes) 000356[0] 1713294143.481052 mapwrite 0x4c7e67 thru 0x4d6559 (0xe6f3 bytes) 000358[0] 1713294143.514013 read 0x286807 thru 0x28b19f (0x4999 bytes) 000359[1] 1713294143.531810 read 0x64e9b8 thru 0x659272 (0xa8bb bytes) 000360[1] 1713294143.547189 mapread 0x47b9f1 thru 0x48a471 (0xea81 bytes) 000366[1] 1713294143.576707 read 0x519503 thru 0x519b09 (0x607 bytes) 000367[1] 1713294143.578586 mapwrite 0x619bfd thru 0x624206 (0xa60a bytes) 000368[0] 1713294143.613064 mapread 0x2cd0d7 thru 0x2d7b92 (0xaabc bytes) 000369[1] 1713294143.617962 write 0x56e38e thru 0x5713d6 (0x3049 bytes) 000370[1] 1713294143.623583 read 0x32fcda thru 0x33a643 (0xa96a bytes) 000371[1] 1713294143.652490 write 0x1142bf thru 0x116a0e (0x2750 bytes) 000372[1] 1713294143.659619 mapwrite 0x8ca559 thru 0x8d07b6 (0x625e bytes) 000374[0] 1713294143.694409 mapwrite 0x5b1e3d thru 0x5b5331 (0x34f5 bytes) 000378[0] 1713294143.737531 write 0x17de76 thru 0x18dcfa (0xfe85 bytes) 000380[0] 1713294143.754969 read 0x5eb323 thru 0x5edb5e (0x283c bytes) 000381[1] 1713294143.774096 write 0x346e7a thru 0x34a672 (0x37f9 bytes) 000382[1] 1713294143.788498 mapread 0x6a8c59 thru 0x6a9f0d (0x12b5 bytes) 000385[0] 1713294143.803164 mapwrite 0x5634cf thru 0x56f1f7 (0xbd29 bytes) 000387[0] 1713294143.840304 trunc from 0x8d07b7 to 0x0b716e (0x819648 bytes) 000395[0] 1713294143.859363 mapwrite 0x7b2fcd thru 0x7b4bfb (0x1c2f bytes) 000396[0] 1713294143.884792 read 0x372944 thru 0x3803fe (0xdabb bytes) 000397[1] 1713294143.904127 mapwrite 0x398fdd thru 0x39928a (0x2ae bytes) 000398[1] 1713294143.930861 trunc from 0x7b4bfc to 0x4cfd21 (0x2e4eda bytes) 000399[1] 1713294143.942284 mapwrite 0x16d147 thru 0x16d440 (0x2fa bytes) 000400[0] 1713294143.959007 write 0x3180d4 thru 0x327f2b (0xfe58 bytes) 000401[0] 1713294143.968114 trunc from 0x4cfd22 to 0x611fc9 (0x1422a8 bytes) 000402[1] 1713294143.977606 trunc from 0x611fca to 0x9dd27a (0x3cb2b1 bytes) 000406[1] 1713294143.995598 mapwrite 0x3a7227 thru 0x3a98b3 (0x268d bytes) 000408[1] 1713294144.023099 mapwrite 0x8109cd thru 0x81e84e (0xde82 bytes) 000411[1] 1713294144.057237 trunc from 0x9dd27b to 0x577a31 (0x465849 bytes) 000413[0] 1713294144.068500 read 0x21328e thru 0x218436 (0x51a9 bytes) 000415[1] 1713294144.088948 mapwrite 0x09636a thru 0x09ad8c (0x4a23 bytes) 000416[0] 1713294144.113266 read 0x465522 thru 0x46e553 (0x9032 bytes) 000417[1] 1713294144.128613 trunc from 0x577a32 to 0x78e962 (0x216f31 bytes) 000421[1] 1713294144.142940 mapwrite 0x47b5ff thru 0x486562 (0xaf64 bytes) 000423[1] 1713294144.172121 read 0x539a76 thru 0x5481f5 (0xe780 bytes) 000424[1] 1713294144.193276 read 0x5b9b61 thru 0x5c445c (0xa8fc bytes) 000425[1] 1713294144.193570 read 0x2c08b9 thru 0x2c51f1 (0x4939 bytes) 000426[1] 1713294144.193877 read 0x63ee46 thru 0x64039f (0x155a bytes) 000427[0] 1713294144.194172 write 0x05f63c thru 0x0655e2 (0x5fa7 bytes) 000428[1] 1713294144.215373 mapwrite 0x1f31f6 thru 0x1f524d (0x2058 bytes) 000430[0] 1713294144.256968 write 0x90a81c thru 0x91510b (0xa8f0 bytes) 000432[1] 1713294144.269987 mapwrite 0x5ec2ac thru 0x5fb243 (0xef98 bytes) 000436[0] 1713294144.299321 read 0x1346c0 thru 0x142aa8 (0xe3e9 bytes) 000439[1] 1713294144.323616 mapread 0x47d14b thru 0x487893 (0xa749 bytes) 000440[1] 1713294144.328429 mapwrite 0x91bbf4 thru 0x91d96f (0x1d7c bytes) 000442[1] 1713294144.367483 write 0x498a6c thru 0x49cb45 (0x40da bytes) 000446[1] 1713294144.376275 write 0x74d73e thru 0x75d412 (0xfcd5 bytes) 000447[1] 1713294144.380391 read 0x59ee5d thru 0x5ae274 (0xf418 bytes) 000449[0] 1713294144.381113 trunc from 0x91d970 to 0x2b34b7 (0x66a4b8 bytes) 000450[1] 1713294144.396744 write 0x6c0e7f thru 0x6c5c46 (0x4dc8 bytes) 000451[1] 1713294144.402937 mapwrite 0x3ed842 thru 0x3f323a (0x59f9 bytes) 000453[1] 1713294144.443063 write 0x8384e3 thru 0x84564d (0xd16b bytes) 000456[1] 1713294144.447477 write 0x4f305c thru 0x4f51ba (0x215f bytes) 000457[1] 1713294144.455417 trunc from 0x84564e to 0x46bdd5 (0x3d9878 bytes) 000458[1] 1713294144.464649 read 0x3307f2 thru 0x3333e9 (0x2bf8 bytes) 000459[1] 1713294144.468160 read 0x18599c thru 0x191eef (0xc554 bytes) 000461[0] 1713294144.469589 mapread 0x2e7946 thru 0x2ebac6 (0x4181 bytes) 000463[0] 1713294144.483530 mapread 0x2613b6 thru 0x261dc9 (0xa14 bytes) 000465[1] 1713294144.489590 mapread 0x3a0ba5 thru 0x3a3047 (0x24a3 bytes) 000466[0] 1713294144.506513 trunc from 0x46bdd6 to 0x799607 (0x32d832 bytes) 000468[1] 1713294144.518857 trunc from 0x799608 to 0x51aa41 (0x27ebc6 bytes) 000469[0] 1713294144.526607 write 0x24ba44 thru 0x256e6e (0xb42b bytes) 000472[0] 1713294144.541485 write 0x160ee0 thru 0x16c820 (0xb941 bytes) 000475[0] 1713294144.553991 mapread 0x1a2704 thru 0x1ad38e (0xac8b bytes) 000476[1] 1713294144.571252 write 0x5f11c4 thru 0x5f616a (0x4fa7 bytes) 000479[0] 1713294144.581368 trunc from 0x5f616b to 0x34f8f9 (0x2a6871 bytes) 000480[0] 1713294144.591848 mapread 0x0cda88 thru 0x0d6186 (0x86ff bytes) 000483[0] 1713294144.609320 read 0x140d34 thru 0x14a494 (0x9761 bytes) 000485[0] 1713294144.621294 mapread 0x21782d thru 0x21ecd4 (0x74a8 bytes) 000486[1] 1713294144.625185 mapwrite 0x174bcf thru 0x17bea6 (0x72d8 bytes) 000489[1] 1713294144.668385 read 0x16e611 thru 0x17d680 (0xf070 bytes) 000490[0] 1713294144.683845 mapread 0x202b23 thru 0x209155 (0x6633 bytes) 000491[0] 1713294144.701043 mapread 0x1657a9 thru 0x1725b5 (0xce0d bytes) 000492[0] 1713294144.715339 read 0x01d4bb thru 0x0211c2 (0x3d08 bytes) 000493[1] 1713294144.726962 mapread 0x332174 thru 0x334f0e (0x2d9b bytes) 000494[1] 1713294144.732992 mapread 0x269fcd thru 0x279537 (0xf56b bytes) 000495[1] 1713294144.755237 mapread 0x252877 thru 0x2536f1 (0xe7b bytes) 000496[0] 1713294144.758986 mapwrite 0x8e5a5c thru 0x8f19c0 (0xbf65 bytes) 000497[1] 1713294144.803334 read 0x5aaeef thru 0x5af5a8 (0x46ba bytes) 000499[1] 1713294144.819425 trunc from 0x8f19c1 to 0x4dec71 (0x412d4f bytes) 000500[1] 1713294144.828618 mapread 0x236a0f thru 0x236af2 (0xe4 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 4144 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000003[1] 1713294145.084682 write 0x042000 thru 0x043bdb (0x1bdc bytes) 000004[1] 1713294145.087182 read 0x03b000 thru 0x042d10 (0x7d11 bytes) 000005[0] 1713294145.089313 read 0x023000 thru 0x029001 (0x6002 bytes) 000006[0] 1713294145.094923 mapwrite 0x291000 thru 0x29158a (0x58b bytes) 000008[0] 1713294145.126217 mapread 0x1d2000 thru 0x1d2a9c (0xa9d bytes) 000010[0] 1713294145.131278 read 0x169000 thru 0x178ead (0xfeae bytes) 000011[1] 1713294145.144536 write 0x998000 thru 0x9a221f (0xa220 bytes) truncating to largest ever: 0xfba9f 000013[1] 1713294145.156321 trunc from 0x9a2220 to 0x0fba9e (0x8a6781 bytes) 000015[1] 1713294145.164481 mapread 0x093000 thru 0x09ae78 (0x7e79 bytes) 000017[1] 1713294145.172308 write 0x283000 thru 0x28f415 (0xc416 bytes) 000019[1] 1713294145.176433 mapwrite 0x1b7000 thru 0x1c1c85 (0xac86 bytes) truncating to largest ever: 0x1e7366 000020[1] 1713294145.199356 trunc from 0x28f416 to 0x1e7365 (0xa80b0 bytes) 000023[0] 1713294145.210198 mapread 0x027000 thru 0x0339f3 (0xc9f4 bytes) 000024[0] 1713294145.227339 read 0x112000 thru 0x114c05 (0x2c06 bytes) 000025[0] 1713294145.237930 write 0x044000 thru 0x04446e (0x46f bytes) truncating to largest ever: 0x8d0b45 000027[1] 1713294145.248852 trunc from 0x1e7366 to 0x8d0b44 (0x6e97df bytes) 000028[0] 1713294145.260583 trunc from 0x8d0b45 to 0x83ed09 (0x91e3b bytes) 000029[1] 1713294145.265406 trunc from 0x83ed0a to 0x2ec841 (0x5524c8 bytes) 000030[1] 1713294145.270428 read 0x0ee000 thru 0x0f91a9 (0xb1aa bytes) 000032[1] 1713294145.290475 write 0x7d2000 thru 0x7d20cd (0xce bytes) 000033[1] 1713294145.294819 write 0x149000 thru 0x14eb78 (0x5b79 bytes) 000035[0] 1713294145.301569 mapread 0x14d000 thru 0x15149a (0x449b bytes) 000037[1] 1713294145.338071 trunc from 0x7d20ce to 0x335d43 (0x49c38a bytes) 000038[0] 1713294145.349381 mapread 0x28c000 thru 0x29757d (0xb57e bytes) 000043[0] 1713294145.367356 mapwrite 0x969000 thru 0x970b7a (0x7b7b bytes) 000044[0] 1713294145.399565 read 0x8fb000 thru 0x902ff8 (0x7ff9 bytes) 000048[1] 1713294145.407388 read 0x043000 thru 0x04af52 (0x7f53 bytes) 000050[1] 1713294145.447457 mapread 0x030000 thru 0x03d3bc (0xd3bd bytes) 000051[1] 1713294145.447985 mapwrite 0x60f000 thru 0x617ddf (0x8de0 bytes) 000054[0] 1713294145.501904 write 0x98f000 thru 0x99b868 (0xc869 bytes) 000055[0] 1713294145.504518 read 0x8d7000 thru 0x8db3ee (0x43ef bytes) 000056[0] 1713294145.545975 read 0x1ce000 thru 0x1d5277 (0x7278 bytes) 000061[0] 1713294145.561534 mapread 0x482000 thru 0x4861a8 (0x41a9 bytes) 000064[0] 1713294145.567886 write 0x6c1000 thru 0x6c3fef (0x2ff0 bytes) 000065[1] 1713294145.571233 mapread 0x8df000 thru 0x8ed2ec (0xe2ed bytes) 000066[1] 1713294145.602330 write 0x039000 thru 0x0420e0 (0x90e1 bytes) 000067[1] 1713294145.608951 trunc from 0x99b869 to 0x250343 (0x74b525 bytes) 000070[0] 1713294145.614009 read 0x0f9000 thru 0x0ffda1 (0x6da2 bytes) 000072[0] 1713294145.631107 mapwrite 0x1c8000 thru 0x1c91e5 (0x11e6 bytes) 000073[1] 1713294145.664065 write 0x5ac000 thru 0x5b9ce3 (0xdce4 bytes) 000078[0] 1713294145.676018 trunc from 0x5b9ce4 to 0x1634ce (0x456815 bytes) 000080[0] 1713294145.690544 read 0x078000 thru 0x08396b (0xb96c bytes) 000081[1] 1713294145.703431 mapread 0x131000 thru 0x13d031 (0xc032 bytes) 000083[1] 1713294145.721468 trunc from 0x1634cf to 0x0b583e (0xadc90 bytes) 000084[0] 1713294145.726655 mapwrite 0x009000 thru 0x015cc8 (0xccc9 bytes) 000086[1] 1713294145.765941 mapread 0x04b000 thru 0x050086 (0x5087 bytes) 000089[0] 1713294145.777779 write 0x8e7000 thru 0x8f408d (0xd08e bytes) 000090[1] 1713294145.785216 mapread 0x269000 thru 0x277bd6 (0xebd7 bytes) 000094[1] 1713294145.807519 trunc from 0x8f408e to 0x1be763 (0x73592a bytes) 000095[1] 1713294145.813604 mapwrite 0x09b000 thru 0x0a51a3 (0xa1a4 bytes) 000097[0] 1713294145.845068 mapread 0x109000 thru 0x118274 (0xf275 bytes) 000105[1] 1713294145.868966 write 0x2d9000 thru 0x2dcb8f (0x3b90 bytes) 000106[0] 1713294145.872212 mapwrite 0x4d5000 thru 0x4dca1a (0x7a1b bytes) 000107[0] 1713294145.911364 mapread 0x23c000 thru 0x24871c (0xc71d bytes) 000109[0] 1713294145.916293 trunc from 0x4dca1b to 0x5cb044 (0xee62a bytes) 000110[0] 1713294145.920545 trunc from 0x5cb045 to 0x85a38d (0x28f349 bytes) truncating to largest ever: 0x8f174d 000112[0] 1713294145.924357 trunc from 0x85a38e to 0x8f174c (0x973bf bytes) 000113[1] 1713294145.927957 read 0x2ca000 thru 0x2d94e3 (0xf4e4 bytes) 000116[0] 1713294145.967176 mapread 0x80e000 thru 0x81d1a3 (0xf1a4 bytes) 000118[1] 1713294145.985421 mapread 0x361000 thru 0x361be0 (0xbe1 bytes) 000123[0] 1713294145.986165 read 0x878000 thru 0x878ef5 (0xef6 bytes) 000129[0] 1713294145.986833 read 0x665000 thru 0x66853a (0x353b bytes) 000131[0] 1713294145.987089 mapwrite 0x44b000 thru 0x4578f4 (0xc8f5 bytes) 000132[0] 1713294146.047528 read 0x65e000 thru 0x66b371 (0xd372 bytes) 000134[1] 1713294146.055384 write 0x72e000 thru 0x733924 (0x5925 bytes) 000138[1] 1713294146.067777 mapwrite 0x508000 thru 0x510777 (0x8778 bytes) 000139[0] 1713294146.090752 mapread 0x4db000 thru 0x4df101 (0x4102 bytes) 000142[0] 1713294146.131010 mapread 0x09c000 thru 0x09ca0e (0xa0f bytes) 000143[0] 1713294146.131396 trunc from 0x8f174d to 0x5a3097 (0x34e6b5 bytes) 000144[1] 1713294146.148676 read 0x23c000 thru 0x23e356 (0x2357 bytes) 000145[1] 1713294146.172360 mapwrite 0x924000 thru 0x92bb0d (0x7b0e bytes) 000147[1] 1713294146.208246 mapwrite 0x305000 thru 0x30ba82 (0x6a83 bytes) 000150[1] 1713294146.236031 mapread 0x371000 thru 0x37804f (0x7050 bytes) 000151[1] 1713294146.245160 mapwrite 0x3bc000 thru 0x3c251c (0x651d bytes) 000152[1] 1713294146.266280 write 0x513000 thru 0x516cb6 (0x3cb7 bytes) 000153[0] 1713294146.270578 read 0x47b000 thru 0x4851d7 (0xa1d8 bytes) 000154[1] 1713294146.301562 mapwrite 0x955000 thru 0x95f590 (0xa591 bytes) 000159[1] 1713294146.354220 mapwrite 0x331000 thru 0x33682c (0x582d bytes) 000160[1] 1713294146.376665 mapwrite 0x298000 thru 0x29fc17 (0x7c18 bytes) 000161[0] 1713294146.401595 mapwrite 0x92e000 thru 0x937ff0 (0x9ff1 bytes) 000162[0] 1713294146.425245 write 0x115000 thru 0x1204ec (0xb4ed bytes) 000163[0] 1713294146.429816 trunc from 0x95f591 to 0x51859b (0x446ff5 bytes) 000164[0] 1713294146.434169 write 0x040000 thru 0x04556a (0x556b bytes) 000165[1] 1713294146.435464 write 0x97e000 thru 0x98a9de (0xc9df bytes) 000166[0] 1713294146.441685 mapwrite 0x1a6000 thru 0x1ae172 (0x8173 bytes) 000167[1] 1713294146.474930 write 0x8a6000 thru 0x8abd34 (0x5d35 bytes) 000170[0] 1713294146.489488 trunc from 0x98a9df to 0x140afa (0x849ee4 bytes) 000174[0] 1713294146.504668 mapread 0x07d000 thru 0x084b71 (0x7b72 bytes) 000175[0] 1713294146.513652 mapwrite 0x5f8000 thru 0x5fd753 (0x5754 bytes) truncating to largest ever: 0x95f6b9 000176[1] 1713294146.540440 trunc from 0x5fd754 to 0x95f6b8 (0x361f65 bytes) 000178[1] 1713294146.551106 trunc from 0x95f6b9 to 0x5af369 (0x3b034f bytes) 000180[1] 1713294146.559480 mapwrite 0x623000 thru 0x62f179 (0xc17a bytes) 000182[1] 1713294146.592961 read 0x3e0000 thru 0x3edb9b (0xdb9c bytes) 000187[1] 1713294146.619875 read 0x24c000 thru 0x25af5a (0xef5b bytes) 000188[0] 1713294146.624772 mapread 0x467000 thru 0x46e58b (0x758c bytes) 000189[0] 1713294146.641624 write 0x981000 thru 0x984df7 (0x3df8 bytes) 000190[0] 1713294146.644592 mapread 0x604000 thru 0x60a828 (0x6829 bytes) 000192[1] 1713294146.652167 trunc from 0x984df8 to 0x8136ff (0x1716f8 bytes) 000194[1] 1713294146.665662 mapwrite 0x6a0000 thru 0x6a6da5 (0x6da6 bytes) 000195[0] 1713294146.711582 read 0x73a000 thru 0x742c20 (0x8c21 bytes) 000196[0] 1713294146.751770 trunc from 0x813700 to 0x15fda0 (0x6b395f bytes) 000199[0] 1713294146.766956 read 0x01c000 thru 0x026dc3 (0xadc4 bytes) 000203[1] 1713294146.783069 read 0x112000 thru 0x11b921 (0x9922 bytes) 000205[0] 1713294146.796239 read 0x07d000 thru 0x07dd65 (0xd66 bytes) 000210[1] 1713294146.806463 mapread 0x136000 thru 0x142d80 (0xcd81 bytes) 000214[0] 1713294146.807335 mapwrite 0x49c000 thru 0x49ebc7 (0x2bc8 bytes) 000216[0] 1713294146.845615 mapwrite 0x543000 thru 0x54abf9 (0x7bfa bytes) 000217[1] 1713294146.869938 trunc from 0x54abfa to 0x5d9daf (0x8f1b6 bytes) 000218[0] 1713294146.879317 mapwrite 0x48f000 thru 0x4989ab (0x99ac bytes) truncating to largest ever: 0x9d1f8f 000222[0] 1713294146.923452 trunc from 0x5d9db0 to 0x9d1f8e (0x3f81df bytes) 000225[0] 1713294146.932661 mapwrite 0x61e000 thru 0x620776 (0x2777 bytes) 000230[1] 1713294146.969771 mapwrite 0x698000 thru 0x69e712 (0x6713 bytes) 000231[1] 1713294147.023068 mapread 0x72d000 thru 0x7370ca (0xa0cb bytes) 000234[0] 1713294147.029906 trunc from 0x9d1f8f to 0x32b2ab (0x6a6ce3 bytes) 000235[1] 1713294147.042230 write 0x558000 thru 0x55ac19 (0x2c1a bytes) 000237[1] 1713294147.049142 mapread 0x487000 thru 0x4874c6 (0x4c7 bytes) 000238[0] 1713294147.050546 write 0x852000 thru 0x856c82 (0x4c83 bytes) 000239[0] 1713294147.057568 read 0x83a000 thru 0x83a83b (0x83c bytes) 000240[1] 1713294147.093317 read 0x612000 thru 0x61f7ca (0xd7cb bytes) 000241[0] 1713294147.137504 read 0x289000 thru 0x28a4c2 (0x14c3 bytes) 000243[1] 1713294147.170852 read 0x30c000 thru 0x313697 (0x7698 bytes) 000244[1] 1713294147.171185 mapwrite 0x032000 thru 0x040141 (0xe142 bytes) 000246[0] 1713294147.238291 write 0x332000 thru 0x3387c2 (0x67c3 bytes) 000247[1] 1713294147.248125 read 0x7c4000 thru 0x7cbe62 (0x7e63 bytes) 000250[0] 1713294147.291662 mapread 0x327000 thru 0x329235 (0x2236 bytes) 000251[0] 1713294147.296267 read 0x639000 thru 0x644724 (0xb725 bytes) 000252[1] 1713294147.332808 read 0x1f5000 thru 0x20034c (0xb34d bytes) 000253[0] 1713294147.333096 write 0x129000 thru 0x1335d0 (0xa5d1 bytes) 000258[0] 1713294147.355892 mapwrite 0x84d000 thru 0x8550da (0x80db bytes) 000259[0] 1713294147.384322 mapwrite 0x967000 thru 0x96d493 (0x6494 bytes) 000262[0] 1713294147.422985 mapwrite 0x926000 thru 0x931b20 (0xbb21 bytes) 000263[0] 1713294147.439566 mapread 0x7c2000 thru 0x7caefd (0x8efe bytes) 000265[1] 1713294147.450329 mapread 0x0f4000 thru 0x1029b0 (0xe9b1 bytes) 000266[0] 1713294147.493164 write 0x839000 thru 0x839d1a (0xd1b bytes) 000268[0] 1713294147.506564 mapread 0x0ce000 thru 0x0d9640 (0xb641 bytes) 000270[0] 1713294147.519000 trunc from 0x96d494 to 0x8b9f50 (0xb3543 bytes) 000271[1] 1713294147.523389 read 0x1d0000 thru 0x1de298 (0xe299 bytes) 000274[1] 1713294147.563116 mapwrite 0x40e000 thru 0x4185c0 (0xa5c1 bytes) 000275[0] 1713294147.612369 write 0x3be000 thru 0x3c2769 (0x476a bytes) 000277[0] 1713294147.618368 mapread 0x70f000 thru 0x710dc1 (0x1dc2 bytes) 000278[0] 1713294147.620746 mapwrite 0x088000 thru 0x08ab5c (0x2b5d bytes) 000282[1] 1713294147.647348 mapread 0x766000 thru 0x772808 (0xc809 bytes) 000283[1] 1713294147.663789 mapread 0x4ad000 thru 0x4ad552 (0x553 bytes) 000284[1] 1713294147.665139 mapwrite 0x7c0000 thru 0x7ceeac (0xeead bytes) 000289[1] 1713294147.711053 trunc from 0x8b9f51 to 0x587996 (0x3325ba bytes) 000291[1] 1713294147.719617 trunc from 0x587997 to 0x28773e (0x300258 bytes) 000292[1] 1713294147.726847 read 0x029000 thru 0x02abf2 (0x1bf3 bytes) 000293[0] 1713294147.740417 trunc from 0x28773f to 0x95a785 (0x6d3047 bytes) 000297[0] 1713294147.755862 mapwrite 0x98b000 thru 0x9969e7 (0xb9e8 bytes) 000298[0] 1713294147.780603 read 0x279000 thru 0x27d636 (0x4637 bytes) 000300[0] 1713294147.821753 mapwrite 0x5f2000 thru 0x5f2be8 (0xbe9 bytes) 000302[1] 1713294147.836781 mapread 0x218000 thru 0x21b4c8 (0x34c9 bytes) 000306[0] 1713294147.880021 mapwrite 0x985000 thru 0x98e012 (0x9013 bytes) 000307[0] 1713294147.912624 read 0x7d2000 thru 0x7d9277 (0x7278 bytes) 000310[0] 1713294147.962960 write 0x68f000 thru 0x6942c3 (0x52c4 bytes) 000311[1] 1713294147.966636 trunc from 0x9969e8 to 0x36af6c (0x62ba7b bytes) 000313[0] 1713294147.981027 read 0x31e000 thru 0x32aa8a (0xca8b bytes) 000317[1] 1713294148.000372 trunc from 0x36af6d to 0x91ad71 (0x5afe05 bytes) 000318[0] 1713294148.009416 mapwrite 0x158000 thru 0x15c29f (0x42a0 bytes) 000321[0] 1713294148.059104 write 0x1d8000 thru 0x1e283f (0xa840 bytes) 000322[0] 1713294148.064009 trunc from 0x91ad72 to 0x527edb (0x3f2e96 bytes) 000324[0] 1713294148.071471 trunc from 0x527edc to 0x6dd732 (0x1b5857 bytes) 000326[0] 1713294148.079345 mapread 0x682000 thru 0x689560 (0x7561 bytes) 000329[1] 1713294148.094049 write 0x583000 thru 0x583323 (0x324 bytes) 000331[0] 1713294148.100913 write 0x7a3000 thru 0x7a7463 (0x4464 bytes) 000334[1] 1713294148.113788 mapread 0x25b000 thru 0x26165b (0x665c bytes) 000335[1] 1713294148.127025 mapread 0x5af000 thru 0x5b0ab7 (0x1ab8 bytes) 000336[1] 1713294148.129506 read 0x77f000 thru 0x785715 (0x6716 bytes) 000338[1] 1713294148.160793 read 0x60d000 thru 0x60eaa4 (0x1aa5 bytes) 000341[1] 1713294148.161763 trunc from 0x7a7464 to 0x828369 (0x80f06 bytes) 000343[1] 1713294148.179427 mapread 0x01a000 thru 0x025970 (0xb971 bytes) 000345[0] 1713294148.213781 mapwrite 0x943000 thru 0x94c185 (0x9186 bytes) 000346[1] 1713294148.248061 read 0x185000 thru 0x194550 (0xf551 bytes) 000348[1] 1713294148.285587 write 0x839000 thru 0x83d9fc (0x49fd bytes) 000353[1] 1713294148.298917 mapwrite 0x572000 thru 0x57d471 (0xb472 bytes) 000354[0] 1713294148.325610 mapread 0x4d4000 thru 0x4d54b3 (0x14b4 bytes) 000360[0] 1713294148.333456 write 0x78c000 thru 0x79b1d8 (0xf1d9 bytes) 000362[0] 1713294148.340407 trunc from 0x94c186 to 0x9bf7a8 (0x73623 bytes) 000363[1] 1713294148.347711 trunc from 0x9bf7a9 to 0x686ce8 (0x338ac0 bytes) 000366[1] 1713294148.359234 mapwrite 0x8f0000 thru 0x8f6798 (0x6799 bytes) 000368[1] 1713294148.390843 mapwrite 0x899000 thru 0x89e96c (0x596d bytes) 000370[1] 1713294148.410075 write 0x741000 thru 0x74d25a (0xc25b bytes) 000371[1] 1713294148.411639 mapread 0x859000 thru 0x863a37 (0xaa38 bytes) 000375[1] 1713294148.429663 mapwrite 0x075000 thru 0x0819e8 (0xc9e9 bytes) 000377[1] 1713294148.461096 mapread 0x12a000 thru 0x12d8c1 (0x38c2 bytes) 000378[1] 1713294148.465293 write 0x8c1000 thru 0x8cddc4 (0xcdc5 bytes) 000379[1] 1713294148.469924 trunc from 0x8f6799 to 0x704a47 (0x1f1d51 bytes) 000380[1] 1713294148.474234 write 0x21e000 thru 0x22c4db (0xe4dc bytes) 000381[1] 1713294148.479519 read 0x6b2000 thru 0x6b7e25 (0x5e26 bytes) 000384[1] 1713294148.510787 read 0x596000 thru 0x5a2706 (0xc707 bytes) 000386[0] 1713294148.511148 mapread 0x267000 thru 0x26b337 (0x4338 bytes) 000388[0] 1713294148.527110 trunc from 0x704a48 to 0x791622 (0x8cbdb bytes) 000389[0] 1713294148.532745 mapwrite 0x2e7000 thru 0x2f613c (0xf13d bytes) 000390[1] 1713294148.567483 write 0x61d000 thru 0x61db05 (0xb06 bytes) 000391[1] 1713294148.601596 write 0x7e5000 thru 0x7eab94 (0x5b95 bytes) 000393[1] 1713294148.605512 mapwrite 0x794000 thru 0x79f69d (0xb69e bytes) 000395[1] 1713294148.629537 write 0x052000 thru 0x053a5c (0x1a5d bytes) 000407[0] 1713294148.647283 mapwrite 0x8b7000 thru 0x8c60fb (0xf0fc bytes) 000410[0] 1713294148.703307 write 0x51a000 thru 0x520a45 (0x6a46 bytes) 000411[1] 1713294148.708562 read 0x627000 thru 0x62cd28 (0x5d29 bytes) 000414[0] 1713294148.740981 mapwrite 0x19f000 thru 0x1a4535 (0x5536 bytes) 000416[0] 1713294148.779054 mapwrite 0x075000 thru 0x080b1f (0xbb20 bytes) 000418[1] 1713294148.808308 mapread 0x5b7000 thru 0x5c5eb7 (0xeeb8 bytes) 000419[1] 1713294148.826125 mapread 0x394000 thru 0x39c1ee (0x81ef bytes) 000424[1] 1713294148.836821 read 0x145000 thru 0x149008 (0x4009 bytes) 000425[0] 1713294148.866177 write 0x966000 thru 0x9747f4 (0xe7f5 bytes) 000427[1] 1713294148.878905 mapread 0x32a000 thru 0x32a2ba (0x2bb bytes) 000430[1] 1713294148.892591 write 0x9d0000 thru 0x9d3233 (0x3234 bytes) 000433[1] 1713294148.899472 trunc from 0x9d3234 to 0x460b33 (0x572700 bytes) 000434[1] 1713294148.907850 write 0x996000 thru 0x9a48d7 (0xe8d8 bytes) 000435[1] 1713294148.910563 write 0x823000 thru 0x8266ec (0x36ed bytes) 000436[0] 1713294148.913203 mapwrite 0x247000 thru 0x25517c (0xe17d bytes) 000437[1] 1713294148.955335 mapread 0x58d000 thru 0x597b36 (0xab37 bytes) 000438[1] 1713294148.971997 mapwrite 0x0dd000 thru 0x0df4c2 (0x24c3 bytes) 000439[1] 1713294148.993986 write 0x7a3000 thru 0x7a7cff (0x4d00 bytes) 000440[0] 1713294148.998498 write 0x2ae000 thru 0x2bba39 (0xda3a bytes) 000441[0] 1713294149.006938 write 0x4fb000 thru 0x4fc981 (0x1982 bytes) 000442[1] 1713294149.008383 write 0x41b000 thru 0x41ba93 (0xa94 bytes) 000443[1] 1713294149.016838 read 0x462000 thru 0x46dade (0xbadf bytes) 000447[0] 1713294149.058821 write 0x8ee000 thru 0x8fb62a (0xd62b bytes) 000448[0] 1713294149.074364 mapread 0x0ed000 thru 0x0f1589 (0x458a bytes) 000453[0] 1713294149.083655 trunc from 0x9a48d8 to 0x60d42c (0x3974ab bytes) 000455[0] 1713294149.094422 mapread 0x3c2000 thru 0x3cf08d (0xd08e bytes) 000457[1] 1713294149.110607 trunc from 0x60d42d to 0x0fdcb3 (0x50f779 bytes) 000458[1] 1713294149.117426 write 0x821000 thru 0x829d67 (0x8d68 bytes) 000459[1] 1713294149.119213 mapwrite 0x2d7000 thru 0x2e5f39 (0xef3a bytes) 000460[0] 1713294149.152197 read 0x0a9000 thru 0x0b8a0b (0xfa0c bytes) 000461[1] 1713294149.186368 mapread 0x249000 thru 0x257f3e (0xef3f bytes) 000463[0] 1713294149.203582 read 0x7a6000 thru 0x7b4180 (0xe181 bytes) 000466[1] 1713294149.204049 trunc from 0x829d68 to 0x8387c0 (0xea59 bytes) 000467[1] 1713294149.216682 trunc from 0x8387c1 to 0x271c59 (0x5c6b67 bytes) 000468[1] 1713294149.223422 write 0x41b000 thru 0x421c70 (0x6c71 bytes) 000469[1] 1713294149.225367 read 0x1a1000 thru 0x1a5dec (0x4ded bytes) 000470[1] 1713294149.244002 read 0x3ab000 thru 0x3ae6c1 (0x36c2 bytes) 000471[0] 1713294149.247373 mapwrite 0x7ad000 thru 0x7b2c7b (0x5c7c bytes) 000473[1] 1713294149.314791 mapwrite 0x5e2000 thru 0x5e38ec (0x18ed bytes) 000475[0] 1713294149.370849 trunc from 0x7b2c7c to 0x2ccd9d (0x4e5ede bytes) 000478[0] 1713294149.384291 mapwrite 0x0d7000 thru 0x0e370b (0xc70c bytes) 000479[1] 1713294149.413790 mapwrite 0x070000 thru 0x07734d (0x734e bytes) 000480[1] 1713294149.442539 mapread 0x228000 thru 0x22a7ac (0x27ad bytes) 000481[1] 1713294149.446913 mapwrite 0x8f6000 thru 0x8f6262 (0x263 bytes) 000482[0] 1713294149.474624 trunc from 0x8f6263 to 0x3b477d (0x541ae5 bytes) 000484[1] 1713294149.486912 write 0x8e6000 thru 0x8f40cd (0xe0ce bytes) 000487[1] 1713294149.497343 write 0x583000 thru 0x58bbaa (0x8bab bytes) 000488[1] 1713294149.499160 trunc from 0x8f40ce to 0x162a46 (0x791687 bytes) 000492[1] 1713294149.508040 write 0x131000 thru 0x13bbb9 (0xabba bytes) 000493[1] 1713294149.509912 read 0x0a3000 thru 0x0a4639 (0x163a bytes) 000496[1] 1713294149.527868 mapwrite 0x720000 thru 0x7202dd (0x2de bytes) 000497[0] 1713294149.571711 mapread 0x3df000 thru 0x3ec2b5 (0xd2b6 bytes) truncating to largest ever: 0x9fcb95 000498[0] 1713294149.597736 trunc from 0x7202de to 0x9fcb94 (0x2dc8b7 bytes) 000499[0] 1713294149.604510 mapwrite 0x3c3000 thru 0x3cd5ee (0xa5ef bytes) 000500[1] 1713294149.642403 read 0x8bb000 thru 0x8bb0f5 (0xf6 bytes) All operations completed A-OK! PASS 16a (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 15:02:32 (1713294152) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.130549 s, 64.3 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.115423 s, 72.7 MB/s Chance of close/open is 1 in 50 Seed set to 4153 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x7ccc9a 000001[1] 1713294153.199140 trunc from 00000000 to 0x7ccc99 (0x7ccc9a bytes) 000003[1] 1713294153.208748 trunc from 0x7ccc9a to 0x2bdd50 (0x50ef49 bytes) 000005[1] 1713294153.218817 write 0x278ada thru 0x279f6f (0x1496 bytes) truncating to largest ever: 0x97c5e4 000007[0] 1713294153.227210 trunc from 0x2bdd51 to 0x97c5e3 (0x6be893 bytes) 000009[1] 1713294153.243441 trunc from 0x97c5e4 to 0x6bbeed (0x2c06f6 bytes) 000010[0] 1713294153.253377 read 0x1a1cca thru 0x1a3030 (0x1367 bytes) 000013[0] 1713294153.265046 mapread 0x3f83b7 thru 0x3f8b20 (0x76a bytes) 000014[1] 1713294153.271111 trunc from 0x6bbeee to 0x860658 (0x1a476b bytes) 000017[1] 1713294153.286444 read 0x10e7a4 thru 0x10f4e7 (0xd44 bytes) truncating to largest ever: 0x9930f4 000018[0] 1713294153.289066 trunc from 0x860659 to 0x9930f3 (0x132a9b bytes) 000019[1] 1713294153.296690 read 0x404cf4 thru 0x406c66 (0x1f73 bytes) 000022[1] 1713294153.308537 read 0x6741d0 thru 0x6743bd (0x1ee bytes) 000026[0] 1713294153.313348 mapread 0x1d7ec7 thru 0x1d93bc (0x14f6 bytes) 000029[1] 1713294153.323657 mapread 0x0ecd22 thru 0x0ede40 (0x111f bytes) 000031[1] 1713294153.345663 mapwrite 0x05336b thru 0x0543f3 (0x1089 bytes) 000032[1] 1713294153.405856 mapread 0x86fac3 thru 0x86fd8e (0x2cc bytes) 000033[1] 1713294153.406323 mapread 0x2099de thru 0x20b0a1 (0x16c4 bytes) 000035[0] 1713294153.410237 mapread 0x0f51cd thru 0x0f6e35 (0x1c69 bytes) 000039[1] 1713294153.418989 write 0x3afd60 thru 0x3b1247 (0x14e8 bytes) 000040[0] 1713294153.423250 trunc from 0x9930f4 to 0x637098 (0x35c05b bytes) 000041[0] 1713294153.439989 mapwrite 0x47cda6 thru 0x47e63a (0x1895 bytes) 000043[0] 1713294153.471788 trunc from 0x637099 to 0x4f547a (0x141c1e bytes) 000046[1] 1713294153.484827 trunc from 0x4f547b to 0x18ba42 (0x369a38 bytes) 000047[0] 1713294153.497336 mapwrite 0x930e46 thru 0x93297a (0x1b35 bytes) 000051[1] 1713294153.531387 mapwrite 0x0c1400 thru 0x0c1be4 (0x7e5 bytes) 000052[1] 1713294153.551043 write 0x8013df thru 0x80175a (0x37c bytes) 000054[0] 1713294153.558431 mapwrite 0x2fffe9 thru 0x301b72 (0x1b8a bytes) 000055[0] 1713294153.578880 mapread 0x8eb7cf thru 0x8ed17a (0x19ac bytes) 000056[1] 1713294153.586669 trunc from 0x93297b to 0x215928 (0x71d052 bytes) 000057[1] 1713294153.597986 read 0x0a3239 thru 0x0a3e49 (0xc11 bytes) 000059[1] 1713294153.614779 write 0x1e2a04 thru 0x1e2fb1 (0x5ae bytes) 000061[0] 1713294153.618559 read 0x104946 thru 0x1057d5 (0xe90 bytes) 000063[1] 1713294153.640139 trunc from 0x215929 to 0x3a8104 (0x1927dc bytes) 000065[0] 1713294153.648485 read 0x0f7585 thru 0x0f7874 (0x2f0 bytes) 000068[1] 1713294153.673809 trunc from 0x3a8105 to 0x8a8de3 (0x500cdf bytes) 000069[1] 1713294153.684538 read 0x40d0d0 thru 0x40e43d (0x136e bytes) 000070[0] 1713294153.715935 write 0x4b0301 thru 0x4b1685 (0x1385 bytes) 000071[0] 1713294153.725953 mapread 0x5b8390 thru 0x5b88b5 (0x526 bytes) 000072[1] 1713294153.732187 trunc from 0x8a8de4 to 0x757c9c (0x151147 bytes) 000075[1] 1713294153.745581 mapread 0x72c06e thru 0x72c102 (0x95 bytes) 000076[0] 1713294153.750583 mapread 0x046703 thru 0x046f40 (0x83e bytes) 000077[0] 1713294153.762675 read 0x069a0d thru 0x069c38 (0x22c bytes) 000078[1] 1713294153.779860 mapwrite 0x21e719 thru 0x220314 (0x1bfc bytes) 000080[0] 1713294153.804091 read 0x1aa4f8 thru 0x1ab088 (0xb91 bytes) 000081[1] 1713294153.828019 mapwrite 0x40af2d thru 0x40b9eb (0xabf bytes) 000082[0] 1713294153.846904 mapwrite 0x5f4cdf thru 0x5f4e1d (0x13f bytes) 000084[0] 1713294153.880968 read 0x088c23 thru 0x089171 (0x54f bytes) 000089[1] 1713294153.887666 mapwrite 0x7c9c68 thru 0x7ca3fd (0x796 bytes) 000095[1] 1713294153.924950 write 0x91353d thru 0x9137b2 (0x276 bytes) 000096[0] 1713294153.928670 write 0x4acf6e thru 0x4ad103 (0x196 bytes) 000098[0] 1713294153.940315 mapread 0x078a8e thru 0x079fc8 (0x153b bytes) 000099[1] 1713294153.970316 mapread 0x2f77fa thru 0x2f8333 (0xb3a bytes) 000104[0] 1713294153.981928 write 0x26bd14 thru 0x26d90e (0x1bfb bytes) 000105[1] 1713294153.993988 mapwrite 0x87157d thru 0x871cd7 (0x75b bytes) 000106[0] 1713294154.018605 trunc from 0x9137b3 to 0x1378b6 (0x7dbefc bytes) 000108[0] 1713294154.031777 mapread 0x00f91a thru 0x0110c8 (0x17af bytes) 000110[1] 1713294154.041324 trunc from 0x1378b7 to 0x47dfcb (0x346715 bytes) 000111[0] 1713294154.048839 mapread 0x3a65dd thru 0x3a6825 (0x249 bytes) 000112[1] 1713294154.054400 write 0x1fc7f4 thru 0x1fdd22 (0x152f bytes) 000114[1] 1713294154.062055 mapwrite 0x142e10 thru 0x142f0b (0xfc bytes) 000118[1] 1713294154.089380 write 0x50a7c3 thru 0x50b4c6 (0xd04 bytes) 000119[1] 1713294154.090042 mapread 0x21a7ed thru 0x21b28c (0xaa0 bytes) 000122[1] 1713294154.098939 mapread 0x025860 thru 0x025f6c (0x70d bytes) 000125[1] 1713294154.107132 mapwrite 0x7f7a8f thru 0x7f8890 (0xe02 bytes) 000126[1] 1713294154.126992 mapread 0x1e7fb8 thru 0x1e9213 (0x125c bytes) 000128[1] 1713294154.137854 mapwrite 0x15fbf5 thru 0x1615c2 (0x19ce bytes) 000130[1] 1713294154.157803 write 0x35d104 thru 0x35e65d (0x155a bytes) 000132[0] 1713294154.163340 mapwrite 0x089cba thru 0x08b732 (0x1a79 bytes) 000138[1] 1713294154.202864 trunc from 0x7f8891 to 0x4d13a0 (0x3274f0 bytes) 000140[1] 1713294154.208974 read 0x443197 thru 0x44438a (0x11f4 bytes) 000142[1] 1713294154.226658 mapwrite 0x3859c5 thru 0x38791f (0x1f5b bytes) 000145[0] 1713294154.246687 mapwrite 0x2e279c thru 0x2e3e03 (0x1668 bytes) 000146[1] 1713294154.271442 trunc from 0x4d13a1 to 0x5d51ab (0x103e0b bytes) 000147[0] 1713294154.280815 read 0x2c11e8 thru 0x2c21cb (0xfe4 bytes) 000148[0] 1713294154.302983 mapread 0x033be5 thru 0x03577e (0x1b9a bytes) 000150[0] 1713294154.310964 mapwrite 0x17a4da thru 0x17a749 (0x270 bytes) 000151[0] 1713294154.343485 write 0x0e9a61 thru 0x0ea88b (0xe2b bytes) 000152[1] 1713294154.348745 write 0x7f1d00 thru 0x7f380d (0x1b0e bytes) 000157[0] 1713294154.365525 trunc from 0x7f380e to 0x17dd71 (0x675a9c bytes) 000158[1] 1713294154.374084 write 0x100c76 thru 0x1019a9 (0xd34 bytes) 000159[0] 1713294154.383560 write 0x8399f3 thru 0x83a0bc (0x6ca bytes) 000160[0] 1713294154.388433 mapwrite 0x7febf2 thru 0x7ff9ba (0xdc9 bytes) 000161[0] 1713294154.411852 write 0x7f4188 thru 0x7f509d (0xf16 bytes) 000163[1] 1713294154.418752 mapread 0x1523e2 thru 0x152dc6 (0x9e5 bytes) 000164[0] 1713294154.425547 trunc from 0x83a0bd to 0x7548cd (0xe57ef bytes) 000165[0] 1713294154.432060 trunc from 0x7548ce to 0x931f1c (0x1dd64f bytes) 000166[0] 1713294154.435623 trunc from 0x931f1d to 0x854cc6 (0xdd256 bytes) 000167[1] 1713294154.438848 mapwrite 0x0f8e6e thru 0x0faadf (0x1c72 bytes) 000171[1] 1713294154.474936 read 0x73648b thru 0x737e62 (0x19d8 bytes) 000172[1] 1713294154.504378 read 0x80a486 thru 0x80af34 (0xaaf bytes) 000175[0] 1713294154.511205 mapread 0x1ff79e thru 0x20019f (0xa02 bytes) 000177[1] 1713294154.524196 trunc from 0x854cc7 to 0x0da7d6 (0x77a4f0 bytes) 000178[0] 1713294154.535407 read 0x097e5b thru 0x099176 (0x131c bytes) 000181[0] 1713294154.550368 read 0x00ca98 thru 0x00e2b0 (0x1819 bytes) 000182[1] 1713294154.553922 write 0x1a90b3 thru 0x1aa462 (0x13b0 bytes) 000183[1] 1713294154.557397 mapread 0x0eec51 thru 0x0f073a (0x1aea bytes) 000186[1] 1713294154.570675 mapwrite 0x4f1a34 thru 0x4f2657 (0xc24 bytes) 000195[1] 1713294154.621278 mapwrite 0x02a3f4 thru 0x02bb7b (0x1788 bytes) 000200[1] 1713294154.641898 mapwrite 0x660a46 thru 0x6616d1 (0xc8c bytes) 000201[1] 1713294154.672159 write 0x2e30c0 thru 0x2e3d3c (0xc7d bytes) 000202[0] 1713294154.673676 trunc from 0x6616d2 to 0x95bc90 (0x2fa5bf bytes) 000205[0] 1713294154.698955 read 0x93bcd1 thru 0x93c574 (0x8a4 bytes) 000213[0] 1713294154.736174 read 0x7a540c thru 0x7a5bb4 (0x7a9 bytes) 000215[1] 1713294154.740226 mapwrite 0x338403 thru 0x338502 (0x100 bytes) 000217[0] 1713294154.769117 read 0x830e23 thru 0x8322d3 (0x14b1 bytes) 000218[0] 1713294154.800865 write 0x37598d thru 0x375e1e (0x492 bytes) 000220[0] 1713294154.812853 read 0x79b4f6 thru 0x79ce03 (0x190e bytes) 000222[0] 1713294154.816379 mapwrite 0x856237 thru 0x8578ce (0x1698 bytes) 000223[0] 1713294154.852386 trunc from 0x95bc91 to 0x4eb244 (0x470a4c bytes) 000224[0] 1713294154.862856 read 0x1dd8f3 thru 0x1de570 (0xc7e bytes) 000226[0] 1713294154.869856 mapwrite 0x4140a8 thru 0x4147d3 (0x72c bytes) 000228[1] 1713294154.888776 mapread 0x0af692 thru 0x0af79b (0x10a bytes) 000230[1] 1713294154.901537 mapread 0x01b02d thru 0x01ca6f (0x1a43 bytes) 000232[0] 1713294154.905866 mapwrite 0x0417c3 thru 0x041e65 (0x6a3 bytes) 000233[1] 1713294154.938365 mapread 0x1b9a1b thru 0x1baca6 (0x128c bytes) 000234[1] 1713294154.950556 trunc from 0x4eb245 to 0x8a80bc (0x3bce78 bytes) 000236[1] 1713294154.966052 read 0x065159 thru 0x065490 (0x338 bytes) 000237[1] 1713294154.988749 trunc from 0x8a80bd to 0x97a2eb (0xd222f bytes) 000239[0] 1713294154.997538 trunc from 0x97a2ec to 0x6d6a4c (0x2a389f bytes) 000240[1] 1713294155.008891 write 0x861b4e thru 0x8626ab (0xb5e bytes) 000241[1] 1713294155.015426 write 0x79ecc2 thru 0x7a0519 (0x1858 bytes) 000244[1] 1713294155.034297 write 0x0fd6c2 thru 0x0fe186 (0xac5 bytes) 000246[1] 1713294155.043790 mapread 0x00918e thru 0x009f8e (0xe01 bytes) 000247[1] 1713294155.045604 trunc from 0x8626ac to 0x94993e (0xe7293 bytes) 000250[1] 1713294155.059316 trunc from 0x94993f to 0x3bc07c (0x58d8c2 bytes) 000251[0] 1713294155.067887 read 0x1b7bfc thru 0x1b90aa (0x14af bytes) 000252[1] 1713294155.093408 mapread 0x07c611 thru 0x07dc76 (0x1666 bytes) 000254[0] 1713294155.100980 mapwrite 0x2c4a68 thru 0x2c5688 (0xc21 bytes) 000258[0] 1713294155.151332 read 0x3ac4de thru 0x3ad047 (0xb6a bytes) 000260[1] 1713294155.175882 trunc from 0x3bc07d to 0x92d823 (0x5717a7 bytes) 000262[0] 1713294155.192394 mapwrite 0x427001 thru 0x42882e (0x182e bytes) 000264[1] 1713294155.220265 write 0x33138e thru 0x332b66 (0x17d9 bytes) 000265[1] 1713294155.226031 mapread 0x0f4bc7 thru 0x0f5b31 (0xf6b bytes) 000267[1] 1713294155.236873 write 0x52c2cc thru 0x52cfd9 (0xd0e bytes) 000268[0] 1713294155.243286 write 0x3e6a11 thru 0x3e7e42 (0x1432 bytes) 000269[1] 1713294155.253381 mapwrite 0x803211 thru 0x804c76 (0x1a66 bytes) 000270[1] 1713294155.280405 mapwrite 0x42f469 thru 0x4312d2 (0x1e6a bytes) 000272[1] 1713294155.301250 write 0x11874d thru 0x119d37 (0x15eb bytes) 000274[0] 1713294155.304203 mapread 0x57dec1 thru 0x57e5bd (0x6fd bytes) 000275[0] 1713294155.329303 read 0x5aa85a thru 0x5abc5f (0x1406 bytes) 000277[1] 1713294155.330691 read 0x8b9e89 thru 0x8bb086 (0x11fe bytes) 000281[1] 1713294155.334611 mapread 0x3a0375 thru 0x3a1891 (0x151d bytes) 000282[0] 1713294155.352850 trunc from 0x92d824 to 0x890c50 (0x9cbd3 bytes) 000283[0] 1713294155.371841 read 0x1c4625 thru 0x1c50d8 (0xab4 bytes) 000290[1] 1713294155.413380 read 0x7946d5 thru 0x794a36 (0x362 bytes) 000293[0] 1713294155.426307 mapread 0x1c9275 thru 0x1caf95 (0x1d21 bytes) 000295[0] 1713294155.431368 write 0x7cf0ef thru 0x7d08fa (0x180c bytes) 000297[0] 1713294155.442270 mapread 0x83cc55 thru 0x83d216 (0x5c2 bytes) 000299[1] 1713294155.446673 trunc from 0x890c51 to 0x57bbe5 (0x31506b bytes) 000302[1] 1713294155.465197 trunc from 0x57bbe6 to 0x0dac1d (0x4a0fc8 bytes) 000304[0] 1713294155.470259 mapread 0x016ed9 thru 0x01761e (0x746 bytes) 000306[0] 1713294155.481100 read 0x01fabc thru 0x0203bc (0x901 bytes) 000307[0] 1713294155.489662 read 0x050911 thru 0x051b37 (0x1227 bytes) 000309[1] 1713294155.493221 read 0x01157c thru 0x01338c (0x1e11 bytes) 000310[0] 1713294155.504248 trunc from 0x0dac1e to 0x177182 (0x9c565 bytes) 000311[0] 1713294155.514270 read 0x0e5af7 thru 0x0e650c (0xa16 bytes) 000312[0] 1713294155.524255 read 0x1304fd thru 0x1311e5 (0xce9 bytes) 000315[0] 1713294155.533083 mapwrite 0x9ac87b thru 0x9ade05 (0x158b bytes) 000317[0] 1713294155.580137 mapread 0x552819 thru 0x55465d (0x1e45 bytes) 000319[0] 1713294155.583841 mapread 0x86cd7c thru 0x86eb47 (0x1dcc bytes) 000320[0] 1713294155.598294 mapwrite 0x272b5b thru 0x2742f0 (0x1796 bytes) 000322[1] 1713294155.615079 write 0x2c8a0b thru 0x2c8d85 (0x37b bytes) 000326[0] 1713294155.650007 mapread 0x14eb80 thru 0x14ff31 (0x13b2 bytes) 000327[0] 1713294155.660133 mapwrite 0x0634c2 thru 0x0653bf (0x1efe bytes) 000328[1] 1713294155.683985 mapread 0x1d2648 thru 0x1d26f6 (0xaf bytes) 000330[1] 1713294155.690147 mapread 0x30f109 thru 0x30f67b (0x573 bytes) 000331[0] 1713294155.695882 trunc from 0x9ade06 to 0x191672 (0x81c793 bytes) 000334[1] 1713294155.710834 mapread 0x0c4c5f thru 0x0c541f (0x7c1 bytes) 000335[1] 1713294155.720781 read 0x077f6c thru 0x0782f9 (0x38e bytes) 000337[0] 1713294155.732645 read 0x15302f thru 0x153197 (0x169 bytes) 000342[0] 1713294155.742501 mapwrite 0x280fc3 thru 0x2824fc (0x153a bytes) 000344[1] 1713294155.780093 mapwrite 0x80a6bc thru 0x80aee3 (0x828 bytes) 000348[1] 1713294155.822891 mapread 0x0e8437 thru 0x0e8d2b (0x8f5 bytes) 000350[1] 1713294155.829418 write 0x8f002d thru 0x8f02f2 (0x2c6 bytes) 000352[0] 1713294155.839092 mapread 0x1f84d8 thru 0x1f8dec (0x915 bytes) 000354[1] 1713294155.863022 read 0x740e9c thru 0x74296d (0x1ad2 bytes) 000355[1] 1713294155.867038 read 0x341045 thru 0x341f08 (0xec4 bytes) 000356[1] 1713294155.891167 write 0x72eca8 thru 0x72fbf8 (0xf51 bytes) 000357[0] 1713294155.891645 mapread 0x0f0702 thru 0x0f1207 (0xb06 bytes) 000359[0] 1713294155.896130 write 0x767b78 thru 0x769631 (0x1aba bytes) 000360[0] 1713294155.921192 read 0x36ba19 thru 0x36ca85 (0x106d bytes) 000361[0] 1713294155.921436 write 0x5d2c6c thru 0x5d38e3 (0xc78 bytes) 000363[0] 1713294155.925490 read 0x0d13c5 thru 0x0d2f65 (0x1ba1 bytes) 000367[0] 1713294155.950325 read 0x7847ed thru 0x786357 (0x1b6b bytes) 000373[1] 1713294155.961412 mapread 0x6c8efa thru 0x6c9cee (0xdf5 bytes) 000375[1] 1713294155.990390 mapread 0x74ee84 thru 0x74ef8c (0x109 bytes) 000376[0] 1713294155.990740 read 0x71eff0 thru 0x720372 (0x1383 bytes) 000378[0] 1713294156.008608 mapread 0x1e6111 thru 0x1e734d (0x123d bytes) 000379[0] 1713294156.009033 mapread 0x64eed4 thru 0x65002c (0x1159 bytes) 000380[1] 1713294156.009327 mapread 0x1902af thru 0x191c16 (0x1968 bytes) 000383[1] 1713294156.009952 mapread 0x447d18 thru 0x4491f8 (0x14e1 bytes) 000384[1] 1713294156.010237 mapread 0x114350 thru 0x115706 (0x13b7 bytes) 000387[0] 1713294156.010956 write 0x0d5973 thru 0x0d6f40 (0x15ce bytes) 000389[0] 1713294156.028701 write 0x85108b thru 0x853034 (0x1faa bytes) 000391[0] 1713294156.034945 mapwrite 0x21153c thru 0x2118b0 (0x375 bytes) 000394[1] 1713294156.053226 read 0x32cc92 thru 0x32cf61 (0x2d0 bytes) 000397[0] 1713294156.073115 mapread 0x2177b8 thru 0x218903 (0x114c bytes) 000398[1] 1713294156.075987 trunc from 0x8f02f3 to 0x1a7fdb (0x748317 bytes) 000400[0] 1713294156.097529 mapwrite 0x5fd58d thru 0x5fe209 (0xc7d bytes) 000401[1] 1713294156.124325 mapwrite 0x591181 thru 0x59147c (0x2fc bytes) 000402[1] 1713294156.158783 trunc from 0x5fe20a to 0x4495e8 (0x1b4c21 bytes) 000403[1] 1713294156.170048 read 0x0e4558 thru 0x0e5d42 (0x17eb bytes) 000407[1] 1713294156.191087 trunc from 0x4495e9 to 0x08834c (0x3c129c bytes) 000412[1] 1713294156.203565 read 0x0686ed thru 0x0686f6 (0xa bytes) 000414[1] 1713294156.203990 mapread 0x03df72 thru 0x03fa60 (0x1aef bytes) 000415[0] 1713294156.207295 read 0x04ce24 thru 0x04e737 (0x1914 bytes) 000416[1] 1713294156.216926 mapread 0x020e3c thru 0x021e05 (0xfca bytes) 000417[0] 1713294156.223016 read 0x0646ec thru 0x065846 (0x115b bytes) 000418[0] 1713294156.223256 read 0x05f8cd thru 0x061269 (0x199d bytes) 000419[1] 1713294156.223453 mapread 0x02d9f4 thru 0x02e2c1 (0x8ce bytes) 000420[0] 1713294156.223808 write 0x187730 thru 0x18862e (0xeff bytes) 000421[1] 1713294156.229896 mapwrite 0x56d468 thru 0x56d5e7 (0x180 bytes) 000426[1] 1713294156.270869 trunc from 0x56d5e8 to 0x769b23 (0x1fc53c bytes) 000430[0] 1713294156.283450 write 0x1f54df thru 0x1f57b3 (0x2d5 bytes) 000433[0] 1713294156.314567 mapread 0x363eab thru 0x364645 (0x79b bytes) 000435[1] 1713294156.321991 write 0x30355f thru 0x303a3e (0x4e0 bytes) 000436[1] 1713294156.339198 write 0x7a1d9b thru 0x7a36cc (0x1932 bytes) 000439[0] 1713294156.340044 mapread 0x405a18 thru 0x405ef9 (0x4e2 bytes) 000440[0] 1713294156.362822 trunc from 0x7a36cd to 0x7263df (0x7d2ed bytes) 000441[1] 1713294156.377737 write 0x1d351d thru 0x1d5144 (0x1c28 bytes) 000446[1] 1713294156.399215 mapread 0x5056e0 thru 0x506f1e (0x183f bytes) 000449[0] 1713294156.408337 mapwrite 0x97e39c thru 0x980052 (0x1cb7 bytes) 000452[1] 1713294156.445103 trunc from 0x980053 to 0x5007d5 (0x47f87d bytes) 000454[0] 1713294156.459994 mapwrite 0x398c8d thru 0x399547 (0x8bb bytes) 000457[0] 1713294156.488026 mapread 0x4cd8e5 thru 0x4cf5be (0x1cda bytes) 000458[0] 1713294156.498048 mapread 0x26832f thru 0x2687d9 (0x4ab bytes) 000461[0] 1713294156.502833 mapread 0x37fcb9 thru 0x3805e8 (0x930 bytes) 000464[0] 1713294156.511328 mapwrite 0x726d53 thru 0x7272be (0x56c bytes) 000465[0] 1713294156.533525 write 0x31e35b thru 0x31ea87 (0x72d bytes) truncating to largest ever: 0x9b3a4a 000466[0] 1713294156.535063 trunc from 0x7272bf to 0x9b3a49 (0x28c78b bytes) 000467[0] 1713294156.541961 mapwrite 0x87ebf4 thru 0x87ecba (0xc7 bytes) 000470[1] 1713294156.566070 write 0x544b2e thru 0x545e40 (0x1313 bytes) 000471[0] 1713294156.577436 read 0x4a9766 thru 0x4aa466 (0xd01 bytes) 000472[1] 1713294156.611081 mapwrite 0x888f2b thru 0x88ac6d (0x1d43 bytes) 000473[1] 1713294156.640322 trunc from 0x9b3a4a to 0x44d071 (0x5669d8 bytes) 000474[1] 1713294156.651683 trunc from 0x44d072 to 0x362b5d (0xea514 bytes) truncating to largest ever: 0x9f9db3 000475[1] 1713294156.654913 trunc from 0x362b5e to 0x9f9db2 (0x697255 bytes) 000476[1] 1713294156.662505 write 0x642825 thru 0x6438fd (0x10d9 bytes) 000480[0] 1713294156.672688 mapwrite 0x801cdf thru 0x80307f (0x13a1 bytes) 000482[0] 1713294156.700765 mapwrite 0x62dc74 thru 0x62f219 (0x15a6 bytes) 000483[0] 1713294156.721521 read 0x7fd143 thru 0x7fde20 (0xcde bytes) 000484[0] 1713294156.744299 trunc from 0x9f9db3 to 0x02781c (0x9d2596 bytes) 000487[0] 1713294156.764651 trunc from 0x02781d to 0x11f745 (0xf7f29 bytes) 000488[1] 1713294156.771988 write 0x4b5717 thru 0x4b625f (0xb49 bytes) 000489[1] 1713294156.778886 mapread 0x29f582 thru 0x2a014d (0xbcc bytes) 000492[1] 1713294156.788473 read 0x0143dc thru 0x01461f (0x244 bytes) 000497[1] 1713294156.805436 mapwrite 0x5fc24d thru 0x5fe17f (0x1f33 bytes) 000500[1] 1713294156.849956 mapread 0x204dd7 thru 0x20559d (0x7c7 bytes) All operations completed A-OK! PASS 16b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 15:02:39 (1713294159) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 15:02:41 (1713294161) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.14048 s, 59.7 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.11737 s, 71.5 MB/s ldlm.namespaces.MGC192.168.201.144@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012b467000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88013738e000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012b467000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88013738e000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012b467000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88013738e000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.62884 s, 39.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.604911 s, 173 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.63029 s, 166 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.24893 s, 84.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.90307 s, 55.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.8472 s, 124 MB/s PASS 16d (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 15:02:54 (1713294174) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0613691 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287953 s, 36.4 MB/s PASS 16e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 15:02:57 (1713294177) TIMEOUT PASS 16f (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 15:03:20 (1713294200) TIMEOUT PASS 16g (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16h: mmap read after truncate file ======= 15:03:43 (1713294223) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16i: read after truncate file ============ 15:03:46 (1713294226) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16j: race dio with buffered i/o ========== 15:03:49 (1713294229) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.346173 s, 75.7 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.25679 s, 102 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.554592 s, 47.3 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.152712 s, 172 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.480553 s, 54.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.169456 s, 155 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.535219 s, 49.0 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.258061 s, 102 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.302152 s, 86.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.587819 s, 44.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.626052 s, 41.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.296419 s, 88.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.333669 s, 78.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.610081 s, 43.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.625206 s, 41.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.285573 s, 91.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.328667 s, 79.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.682852 s, 38.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.705027 s, 37.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 15:03:58 (1713294238) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: truncating to largest ever: 0x32cce75 test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x32cce75 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Tue Apr 16 15:04:17 EDT 2024 + fsx 2 done: rc=0 Tue Apr 16 15:04:17 EDT 2024 All operations completed A-OK! + fsx 3 done: rc=0 Tue Apr 16 15:04:17 EDT 2024 + fsx 4 done: rc=0 Tue Apr 16 15:04:17 EDT 2024 + fsx 5 done: rc=0 Tue Apr 16 15:04:17 EDT 2024 PASS 16k (20s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 17: resource creation/LVB creation race ========================================================================= 15:04:20 (1713294260) fail_loc=0x8000030a PASS 17 (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 18: mmap sanity check =========================================================================================== 15:04:25 (1713294265) mmap test1: basic mmap operation (PASS, 0.024383s) mmap test2: MAP_PRIVATE not write back (PASS, 0.015469s) mmap test3: concurrent mmap ops on two nodes (PASS, 14.68s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0772s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.030166s) mmap test6: check mmap write/read content on two nodes (PASS, 0.032767s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0125s) PASS 18 (20s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 19: test concurrent uncached read races ========================================================================= 15:04:47 (1713294287) oleg144-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg144-client: oleg144-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 20: test extra readahead page left in cache ============================================================== 15:04:50 (1713294290) PASS 20 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 15:04:53 (1713294293) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 23: others should see updated atime while another read============================================================== 15:04:56 (1713294296) now is 1713294297 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 new atime is 1713294358 PASS 23 (63s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 15:06:01 (1713294361) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 43008 7495680 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 43008 7495680 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 403 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 400 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.3K 386 228.9K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 403 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 400 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 386 228.9K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 9.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 33.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 42.0M 7.1G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117971 403 117568 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117200 400 116800 1% /mnt/lustre2[OST:1] filesystem_summary: 234754 386 234368 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 43008 7495680 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 403 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 400 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 386 228.9K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre[MDT:0] filesystem_summary: 502242 386 501856 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117971 403 117568 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117200 400 116800 1% /mnt/lustre2[OST:1] filesystem_summary: 234754 386 234368 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 43008 7495680 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 43008 7495680 1% /mnt/lustre2 PASS 24a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 24b: lfs df should show both filesystems ========================================================================= 15:06:04 (1713294364) PASS 24b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 15:06:07 (1713294367) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 15:06:10 (1713294370) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 26a: allow mtime to get older ============ 15:06:12 (1713294372) utime: good mknod times 1713294371 <= 1713294372 <= 1713294372 for /mnt/lustre/f26a utime: good mknod times 1713294371 <= 1713294372 <= 1713294372 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 26b: sync mtime between ost and mds ====== 15:06:16 (1713294376) PASS 26b (3s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 30: recreate file race =================== 15:06:22 (1713294382) 0 PASS 30 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 15:06:27 (1713294387) fail_loc=0x314 PASS 31a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 15:06:31 (1713294391) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg144-server mds-ost sync done. sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.144@tcp:/lustre 7542784 43008 7495680 1% /mnt/lustre2 PASS 31b (17s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31r: open-rename(replace) race =========== 15:06:49 (1713294409) -rw-r--r-- 1 root root 0 Apr 16 15:06 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_33a skipping SLOW test 33a debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 15:06:55 (1713294415) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 15:06:58 (1713294418) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33d: dependent transactions should trigger COS ========================================================== 15:07:00 (1713294420) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 15:07:02 (1713294422) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 34: no lock timeout under IO ============= 15:07:04 (1713294424) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg144-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012b467000.ost_server_uuid,osc.lustre-OST0000-osc-ffff88013738e000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff88012b467000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88013738e000.ost_server_uuid in FULL state after 0 sec oleg144-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012b467000.ost_server_uuid,osc.lustre-OST0001-osc-ffff88013738e000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff88012b467000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88013738e000.ost_server_uuid in IDLE state after 0 sec oleg144-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012b467000.ost_server_uuid,osc.lustre-OST0000-osc-ffff88013738e000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff88012b467000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88013738e000.ost_server_uuid in FULL state after 0 sec oleg144-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012b467000.ost_server_uuid,osc.lustre-OST0001-osc-ffff88013738e000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff88012b467000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88013738e000.ost_server_uuid in IDLE state after 0 sec oleg144-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012b467000.ost_server_uuid,osc.lustre-OST0000-osc-ffff88013738e000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff88012b467000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88013738e000.ost_server_uuid in IDLE state after 0 sec oleg144-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012b467000.ost_server_uuid,osc.lustre-OST0001-osc-ffff88013738e000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff88012b467000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88013738e000.ost_server_uuid in IDLE state after 0 sec PASS 34 (83s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 15:08:28 (1713294508) fail_loc=0x80000317 Race attempt 0 Wait for 24490 24631 for 60 sec... total: 4000 open/close in 7.13 seconds: 560.95 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.201.144@tcp:/lustre 7.2G 44M 7.2G 1% /mnt/lustre 192.168.201.144@tcp:/lustre 7.2G 44M 7.2G 1% /mnt/lustre2 PASS 35 (64s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 36: handle ESTALE/open-unlink correctly == 15:09:34 (1713294574) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg144-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.281269 s, 186 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7493632) after_dd(7440384) after(7491584) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.28483 s, 184 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7491584) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.2698 s, 194 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7493632) after_dd(7438336) after(7489536) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.329989 s, 159 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7489536) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.312208 s, 168 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7493632) after_dd(7438336) after(7489536) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.33389 s, 157 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7489536) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.259083 s, 202 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7493632) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.26795 s, 196 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7493632) after_dd(7438336) after(7489536) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.316337 s, 166 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7489536) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.281661 s, 186 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7493632) after_dd(7438336) after(7489536) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.302091 s, 174 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7489536) after_dd(7442432) after(7493632) PASS 36 (225s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 15:13:21 (1713294801) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 - create 9150 (time 1713294812.42 total 10.00 last 914.98) total: 10000 create in 11.05 seconds: 905.31 ops/second PASS 37 (14s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39a: file mtime does not change after rename ========================================================== 15:13:37 (1713294817) repeat after cancel_lru_locks PASS 39a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 15:13:41 (1713294821) repeat after cancel_lru_locks PASS 39b (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39c: check truncate mtime update ================================================================================ 15:13:45 (1713294825) repeat after cancel_lru_locks PASS 39c (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39d: sync write should update mtime ====== 15:13:49 (1713294829) fail_loc=0x411 fail_loc=0 PASS 39d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40a: pdirops: create vs others ======================================================================== 15:13:52 (1713294832) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40b: pdirops: open|create and others ======================================================================== 15:14:02 (1713294842) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40c: pdirops: link and others ======================================================================== 15:14:11 (1713294851) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40d: pdirops: unlink and others ======================================================================== 15:14:21 (1713294861) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40e: pdirops: rename and others ======================================================================== 15:14:30 (1713294870) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 15:14:40 (1713294880) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41b: pdirops: create vs create ======================================================================== 15:14:47 (1713294887) open: File exists Conflict PASS 41b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41c: pdirops: create vs link ======================================================================== 15:14:54 (1713294894) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41d: pdirops: create vs unlink ======================================================================== 15:15:01 (1713294901) Conflict PASS 41d (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 15:15:08 (1713294908) Conflict PASS 41e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 15:15:15 (1713294915) Conflict PASS 41f (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41g: pdirops: create vs getattr ======================================================================== 15:15:22 (1713294922) Conflict PASS 41g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41h: pdirops: create vs readdir ======================================================================== 15:15:30 (1713294930) Conflict PASS 41h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41i: reint_open: create vs create ======== 15:15:37 (1713294937) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (699s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 15:27:18 (1713295638) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 15:27:26 (1713295646) open: File exists Conflict PASS 42b (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 15:27:34 (1713295654) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 15:27:42 (1713295662) Conflict PASS 42d (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 15:27:50 (1713295670) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 15:27:58 (1713295678) Conflict PASS 42f (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 15:28:04 (1713295684) Conflict PASS 42g (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 15:28:12 (1713295692) Conflict PASS 42h (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 15:28:20 (1713295700) PASS 43a (20s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43b: pdirops: unlink vs create ======================================================================== 15:28:42 (1713295722) Conflict PASS 43b (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43c: pdirops: unlink vs link ======================================================================== 15:28:50 (1713295730) Conflict PASS 43c (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 15:28:58 (1713295738) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 15:29:06 (1713295746) Conflict PASS 43e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 15:29:13 (1713295753) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 15:29:19 (1713295759) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 15:29:26 (1713295766) Conflict PASS 43h (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 15:29:35 (1713295775) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 15:29:38 (1713295778) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (42s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43k: unlink vs create ==================== 15:30:22 (1713295822) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (453s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 15:37:57 (1713296277) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Tue Apr 16 15:38:01 EDT 2024 PASS 44a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 15:38:05 (1713296285) open: File exists Conflict PASS 44b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 15:38:12 (1713296292) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 15:38:19 (1713296299) Conflict PASS 44d (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 15:38:26 (1713296306) Conflict PASS 44e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 15:38:33 (1713296313) Conflict PASS 44f (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 15:38:40 (1713296320) Conflict PASS 44g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 15:38:47 (1713296327) Conflict PASS 44h (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 15:38:55 (1713296335) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 15:38:58 (1713296338) PASS 45a (31s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45b: pdirops: rename src vs create ======================================================================== 15:39:30 (1713296370) Conflict PASS 45b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45c: pdirops: rename src vs link ======================================================================== 15:39:36 (1713296376) Conflict PASS 45c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 15:39:42 (1713296382) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 15:39:48 (1713296388) Conflict PASS 45e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 15:39:55 (1713296395) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 15:40:01 (1713296401) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 15:40:07 (1713296407) Conflict PASS 45h (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 15:40:12 (1713296412) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45j: read vs rename ====================== 15:40:15 (1713296415) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (376s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 15:46:33 (1713296793) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46b: pdirops: link vs create ======================================================================== 15:46:39 (1713296799) open: File exists Conflict PASS 46b (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46c: pdirops: link vs link ======================================================================== 15:46:45 (1713296805) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46d: pdirops: link vs unlink ======================================================================== 15:46:50 (1713296810) Conflict PASS 46d (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 15:46:56 (1713296816) Conflict PASS 46e (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 15:47:02 (1713296822) Conflict PASS 46f (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46g: pdirops: link vs getattr ======================================================================== 15:47:08 (1713296828) Conflict PASS 46g (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46h: pdirops: link vs readdir ======================================================================== 15:47:14 (1713296834) Conflict PASS 46h (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46i: pdirops: link vs remote mkdir ======= 15:47:20 (1713296840) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 15:47:22 (1713296842) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47b: pdirops: remote mkdir vs create ===== 15:47:24 (1713296844) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47c: pdirops: remote mkdir vs link ======= 15:47:27 (1713296847) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 15:47:29 (1713296849) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 15:47:31 (1713296851) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 15:47:34 (1713296854) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 15:47:36 (1713296856) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 15:47:38 (1713296858) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.0272496 s, 376 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51a: layout lock: refresh layout should work ========================================================== 15:47:47 (1713296867) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00463209 s, 34.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3445: kill: (11687) - No such process PASS 51a (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 15:47:52 (1713296872) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00231952 s, 441 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116012 s, 90.4 MB/s 3145728 PASS 51b (13s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 15:48:07 (1713296887) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00387861 s, 264 kB/s PASS 51c (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 15:48:15 (1713296895) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122177 s, 85.8 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 15:48:19 (1713296899) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2449 0x991 0x280000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2449 0x991 0x280000400 read lease(1) has applied. PASS 51e (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 54: rename locking ======================= 15:48:24 (1713296904) llite.lustre-ffff88012b467000.xattr_cache=1 llite.lustre-ffff88013738e000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505002 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-16 15:48:25.000000000 -0400 Modify: 2024-04-16 15:48:25.000000000 -0400 Change: 2024-04-16 15:48:25.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505005 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-16 15:48:30.000000000 -0400 Modify: 2024-04-16 15:48:30.000000000 -0400 Change: 2024-04-16 15:48:30.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505008 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-16 15:48:35.000000000 -0400 Modify: 2024-04-16 15:48:36.000000000 -0400 Change: 2024-04-16 15:48:36.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505007 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-16 15:48:35.000000000 -0400 Modify: 2024-04-16 15:48:36.000000000 -0400 Change: 2024-04-16 15:48:36.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505010 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-16 15:48:41.000000000 -0400 Modify: 2024-04-16 15:48:41.000000000 -0400 Change: 2024-04-16 15:48:41.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505011 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-16 15:48:41.000000000 -0400 Modify: 2024-04-16 15:48:41.000000000 -0400 Change: 2024-04-16 15:48:41.000000000 -0400 Birth: - PASS 54 (24s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55a: rename vs unlink target dir ========= 15:48:49 (1713296929) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55b: rename vs unlink source dir ========= 15:48:58 (1713296938) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55c: rename vs unlink orphan target dir == 15:49:06 (1713296946) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55d: rename file vs link ================= 15:49:20 (1713296960) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 56a: test llverdev with single large stripe ========================================================== 15:49:30 (1713296970) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1713296971 chunksize: 4096 size: 16777216000000 write offset: 3609198592kB 1.68115 MB/s read offset: 3609198592kB inf MB/s llverdev: data verified successfully PASS 56a (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 15:49:42 (1713296982) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1713296984 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 4.20946 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (33s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 60: Verify data_version behaviour ======== 15:50:17 (1713297017) PASS 60 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 70a: cd directory && rm directory ======== 15:50:21 (1713297021) PASS 70a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 70b: remove files after calling rm_entry ========================================================== 15:50:24 (1713297024) PASS 70b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71a: correct file map just after write operation is finished ========================================================== 15:50:28 (1713297028) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00469726 s, 14.0 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 15:50:31 (1713297031) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00318468 s, 20.6 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 15:50:34 (1713297034) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 15:50:36 (1713297036) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00414865 s, 15.8 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 15:50:38 (1713297038) llite.lustre-ffff88012b467000.xattr_cache=1 llite.lustre-ffff88013738e000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 15:50:42 (1713297042) llite.lustre-ffff88012b467000.xattr_cache=1 llite.lustre-ffff88013738e000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 74: flock deadlock: different mounts ======================================================================== 15:50:45 (1713297045) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00264507 s, 387 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00223172 s, 459 kB/s 26292: get lock2 26293: get lock1 26292: done 26292: get lock1 26293: done 26293: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 26293: cannot get lock2: Resource deadlock avoided 26293: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 26292: cannot get lock1: Resource deadlock avoided 26292: done 26292: exit rc=0 PASS 74 (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 75: osc: upcall after unuse lock============================================================================= 15:50:52 (1713297052) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0129817 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00730493 s, 144 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4031: 27038 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 76: Verify MDT open_files listing ======== 15:51:01 (1713297061) opening files: ........ closing files: ........ PASS 76 (16s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77a: check FIFO NRS policy =============== 15:51:19 (1713297079) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0995105 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123177 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117029 s, 89.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0399341 s, 26.3 MB/s , 0.0362639 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402724 s, 26.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446763 s, 23.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0474395 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462502 s, 22.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0513415 s, 20.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0541767 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0497331 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051933 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0544774 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0551052 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0603421 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536419 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00842377 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00775699 s, 135 MB/s , 0.00876209 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104603 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101607 s, 103 MB/s , 0.0115085 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00456556 s, 230 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00468867 s, 224 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108656 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961066 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116531 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00960061 s, 109 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0101843 s, 103 MB/s , 0.0113175 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107977 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114546 s, 91.5 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77b: check CRR-N NRS policy ============== 15:51:23 (1713297083) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0882053 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817738 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00782065 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139044 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145056 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332587 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312186 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436302 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365848 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397162 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488256 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0576877 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511465 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0614562 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463489 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431155 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0496603 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00602352 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00809972 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00692771 s, 151 MB/s 1048576 bytes (1.0 MB) copied, 0.00987573 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00840632 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010291 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100314 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00955694 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897965 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894362 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893637 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920411 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966485 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112277 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110811 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128446 s, 81.6 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0861043 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00896587 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102208 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337646 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430973 s, 24.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0418583 s, 25.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0460864 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490079 s, 21.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495631 s, 21.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0483641 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430939 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472025 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0497185 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0562847 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0533252 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0581104 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0588041 s, 17.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013738e000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88013738e000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00678731 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00729866 s, 144 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00974903 s, 108 MB/s , 0.00745207 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00199891 s, 525 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100832 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885365 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934492 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00554924 s, 189 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100906 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969824 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913273 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109771 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105925 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104241 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115958 s, 90.4 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77c: check ORR NRS policy ================ 15:51:27 (1713297087) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0990202 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095041 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147945 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043635 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332041 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382865 s, 27.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358456 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501536 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414146 s, 25.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0564182 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0554594 s, 18.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552011 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0590838 s, 17.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0625456 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0581597 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.054509 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0662047 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097073 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00812726 s, 129 MB/s 1048576 bytes (1.0 MB) copied, 0.00821027 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00931288 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013355 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132317 s, 79.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117554 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123966 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121913 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124807 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125374 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0141181 s, 74.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0144379 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142707 s, 73.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148409 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145214 s, 72.2 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.10504 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140252 s, 74.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017664 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118912 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177595 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011431 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250278 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183629 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159632 s, 65.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432843 s, 24.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0382349 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428706 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171345 s, 61.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488706 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438869 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492014 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298417 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00667509 s, 157 MB/s 1048576 bytes (1.0 MB) copied, 0.00873725 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100938 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919168 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00965706 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897036 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00508121 s, 206 MB/s , 0.0101903 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0108481 s, 96.7 MB/s , 0.00992429 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980593 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970274 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0103033 s, 102 MB/s , 0.00973133 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106012 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106864 s, 98.1 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0983572 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131539 s, 79.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138716 s, 75.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139137 s, 75.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0143179 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446542 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403358 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.052124 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437589 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493615 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0576498 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0560755 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449317 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0475789 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0468952 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511585 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.068448 s, 15.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013738e000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00589405 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00683289 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00603062 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00723185 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00764717 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008606 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973927 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00906113 s, 116 MB/s 1+0 records in 1+0 records out , 0.0122928 s, 85.3 MB/s 1048576 bytes (1.0 MB) copied, 0.00836937 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00144819 s, 724 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00823682 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937138 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00971235 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.0102187 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121968 s, 86.0 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77d: check TRR nrs policy ================ 15:51:33 (1713297093) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0755661 s, 222 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00917382 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100558 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231695 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387542 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395055 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035995 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454881 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401719 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413252 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0469844 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051062 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0537202 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438386 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042632 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459451 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0498898 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00750337 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00876602 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00183416 s, 572 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0025751 s, 407 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00756755 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881351 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00900347 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895687 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922943 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107761 s, 97.3 MB/s 1048576 bytes (1.0 MB) copied, 0.00901075 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913176 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00949918 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010009 s, 105 MB/s , 0.0127626 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134435 s, 78.0 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0902909 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107267 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104391 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413327 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456579 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457055 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449868 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0545727 s, 19.2 MB/s , 0.0515082 s, 20.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0564353 s, 18.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0554701 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0545808 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0571633 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0586056 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0526176 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0553977 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0618474 s, 17.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88013738e000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00718746 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00597001 s, 176 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00716108 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106466 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103724 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00851132 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846079 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00996406 s, 105 MB/s , 0.00536821 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111993 s, 93.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0104026 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972043 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00974955 s, 108 MB/s , 0.0118232 s, 88.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0103969 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129719 s, 80.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0822245 s, 204 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102896 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00872982 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187875 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169651 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216741 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.026077 s, 40.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0116772 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272811 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135273 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.052406 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0544562 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0556314 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454088 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450922 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0497044 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.050564 s, 20.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88013738e000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00611254 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00704278 s, 149 MB/s 1048576 bytes (1.0 MB) copied, 0.00150637 s, 696 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00194399 s, 539 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00774342 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0077518 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104652 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00868953 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00141989 s, 738 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104923 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00976909 s, 107 MB/s 1048576 bytes (1.0 MB) copied, 0.00964736 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973797 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124699 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106915 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115711 s, 90.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77e: check TBF NID nrs policy ============ 15:51:38 (1713297098) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.201.44@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.128805 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103548 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144709 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262921 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034258 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339921 s, 30.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404855 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392352 s, 26.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0455971 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412501 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422372 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0466666 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0475406 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467996 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0471339 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415464 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484562 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112263 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113228 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107131 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159753 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013412 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158445 s, 66.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137484 s, 76.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0144266 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014402 s, 72.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141179 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169323 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151487 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119314 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018279 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174942 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191417 s, 54.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.152315 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103182 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115363 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364694 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405332 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357845 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432977 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0497719 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042322 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435313 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433053 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0562838 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0537339 s, 19.5 MB/s , 0.0439639 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476594 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0527534 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0563963 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0125175 s, 83.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0170225 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016307 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155914 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157982 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802662 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0165358 s, 63.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0182755 s, 57.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194091 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0226539 s, 46.3 MB/s , 0.0184183 s, 56.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0196555 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207711 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208295 s, 50.3 MB/s , 0.0198415 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211076 s, 49.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0980425 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865913 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00815148 s, 129 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288335 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282547 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314855 s, 33.3 MB/s 1048576 bytes (1.0 MB) copied, 0.030613 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0336036 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244734 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241816 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464448 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413871 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445682 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0526127 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425088 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0506336 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0516758 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972053 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00990154 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869323 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012095 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100819 s, 104 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101061 s, 104 MB/s , 0.010258 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010819 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123002 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120844 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012319 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125288 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132416 s, 79.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0129113 s, 81.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0128197 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147344 s, 71.2 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0922377 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146009 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187268 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194012 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814636 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025081 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185186 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177907 s, 58.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196778 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291253 s, 36.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0189578 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019604 s, 53.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0656545 s, 16.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0606915 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0630483 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0557816 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0550725 s, 19.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88013738e000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00758996 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00165314 s, 634 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00640606 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970279 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00907539 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00723015 s, 145 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00746385 s, 140 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0096765 s, 108 MB/s , 0.00948831 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00875122 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00870127 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918268 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950933 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905453 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898417 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116359 s, 90.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77f: check TBF JobID nrs policy ========== 15:51:47 (1713297107) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.104288 s, 161 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00731375 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893455 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128502 s, 81.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207823 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124039 s, 84.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250066 s, 41.9 MB/s 1048576 bytes (1.0 MB) copied, 0.023499 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326459 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236215 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150068 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361723 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201325 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220492 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218513 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171377 s, 61.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232274 s, 45.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00709106 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00647004 s, 162 MB/s 1048576 bytes (1.0 MB) copied, 0.0080678 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00663727 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112278 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00727714 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787281 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848131 s, 124 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0100354 s, 104 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0101279 s, 104 MB/s , 0.00965742 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104939 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112316 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125112 s, 83.8 MB/s , 0.0120183 s, 87.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130338 s, 80.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.100182 s, 167 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103438 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111777 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414626 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449845 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039586 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442759 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483791 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0528066 s, 19.9 MB/s , 0.0467189 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0554911 s, 18.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0506411 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0563458 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568502 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0538251 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0544731 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0623342 s, 16.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109262 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095736 s, 110 MB/s 1048576 bytes (1.0 MB) copied, 0.0027173 s, 386 MB/s , 0.00949933 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103536 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103373 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101968 s, 103 MB/s , 0.0102552 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0111803 s, 93.8 MB/s , 0.0122112 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128564 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119045 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122783 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124997 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122845 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136132 s, 77.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0811976 s, 207 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855106 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109108 s, 96.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180664 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0312485 s, 33.6 MB/s , 0.0326745 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307302 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345835 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296618 s, 35.4 MB/s , 0.032309 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431454 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453224 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395798 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0441075 s, 23.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0391162 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494858 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0513797 s, 20.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00701256 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00904913 s, 116 MB/s , 0.00794851 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00166582 s, 629 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885462 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00902116 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925006 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00840172 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953283 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00617616 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105278 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110585 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101305 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104705 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131234 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134232 s, 78.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0848908 s, 198 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00710264 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112684 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365429 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357694 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041495 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411764 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037827 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438486 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501983 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488947 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511071 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0521648 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0553193 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.052536 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490455 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0605908 s, 17.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88013738e000.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00681272 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00832601 s, 126 MB/s 1048576 bytes (1.0 MB) copied, 0.00702778 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00667535 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800105 s, 131 MB/s 1048576 bytes (1.0 MB) copied, 0.00818797 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104079 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897693 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008923 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00896154 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909956 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884173 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929096 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00873869 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101187 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109065 s, 96.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77g: Change TBF type directly ============ 15:51:54 (1713297114) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 15:51:58 (1713297118) oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg144-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg144-client: oleg144-server: ssh exited with exit code 2 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg144-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg144-client: oleg144-server: ssh exited with exit code 12 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg144-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg144-client: oleg144-server: ssh exited with exit code 12 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg144-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg144-client: oleg144-server: ssh exited with exit code 12 PASS 77h (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77i: Change rank of TBF rule ============= 15:52:03 (1713297123) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77j: check TBF-OPCode NRS policy ========= 15:52:11 (1713297131) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0800896 s, 209 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109735 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110719 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033717 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376737 s, 27.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0371361 s, 28.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0371788 s, 28.2 MB/s , 0.0269446 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037577 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365129 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352725 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293614 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365972 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428769 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361821 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374145 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0393953 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846473 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00841954 s, 125 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0086883 s, 121 MB/s 1048576 bytes (1.0 MB) copied, 0.00962521 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00928109 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122108 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0101118 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.0102346 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104292 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109505 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103076 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107989 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111738 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109165 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105889 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142621 s, 73.5 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93195 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4275 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (45s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 15:52:57 (1713297177) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg144-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg144-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg144-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg144-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.446997 s, 37.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113173 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129181 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015793 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198408 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220009 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00805089 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099588 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147336 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14153 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155099 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164475 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147464 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155642 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175127 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168636 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16171 s, 6.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010069 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105208 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106669 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109069 s, 96.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0110544 s, 94.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0115349 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103828 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124519 s, 84.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121205 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0120589 s, 87.0 MB/s 1048576 bytes (1.0 MB) copied, 0.012009 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123596 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131534 s, 79.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132524 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131047 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131474 s, 79.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0311 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9646 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg144-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg144-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg144-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg144-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.2187 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020797 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253312 s, 41.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0193368 s, 54.2 MB/s , 0.0327012 s, 32.1 MB/s , 0.0300378 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337355 s, 31.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0390008 s, 26.9 MB/s , 0.0186147 s, 56.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.172498 s, 6.1 MB/s , 0.172481 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.190838 s, 5.5 MB/s , 0.187759 s, 5.6 MB/s 1048576 bytes (1.0 MB) copied, 0.173379 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167377 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185652 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.198696 s, 5.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00958105 s, 109 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00738185 s, 142 MB/s , 0.00966527 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00871023 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918972 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941151 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103065 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131682 s, 79.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110195 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010535 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104086 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115281 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113512 s, 92.4 MB/s , 0.0102913 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130192 s, 80.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8351 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9575 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (134s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 15:55:13 (1713297313) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg144-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg144-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg144-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg144-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.23503 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015135 s, 69.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198791 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273924 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00927521 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149101 s, 70.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018394 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0086967 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127061 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146711 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146204 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162897 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.163045 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148413 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.171588 s, 6.1 MB/s , 0.152458 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179465 s, 5.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124615 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159409 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161991 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151346 s, 69.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191323 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194893 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199382 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019631 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167336 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165317 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0164516 s, 63.7 MB/s 1048576 bytes (1.0 MB) copied, 0.016139 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161355 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016453 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157832 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017564 s, 59.7 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8406 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9443 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg144-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg144-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg144-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg144-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.23219 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166155 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208444 s, 50.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378218 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183578 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238464 s, 44.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0405373 s, 25.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0355822 s, 29.5 MB/s , 0.0288504 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.142541 s, 7.4 MB/s 1048576 bytes (1.0 MB) copied, 0.152521 s, 6.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.151498 s, 6.9 MB/s , 0.145326 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169765 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168077 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179969 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167996 s, 6.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00946071 s, 111 MB/s , 0.00942869 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00866325 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00853078 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102662 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103805 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124353 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106084 s, 98.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136829 s, 76.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012397 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131818 s, 79.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0132033 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130485 s, 80.4 MB/s , 0.0161835 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135829 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145645 s, 72.0 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8449 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9405 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (133s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 15:57:28 (1713297448) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.269339 s, 62.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207032 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194974 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204793 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423103 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327798 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456207 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282927 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364601 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0539262 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415083 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0544874 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413491 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434059 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.048399 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411975 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429442 s, 24.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101551 s, 103 MB/s , 0.0138347 s, 75.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162402 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106336 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149688 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137343 s, 76.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013627 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137518 s, 76.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159456 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0147273 s, 71.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0156623 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165384 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0170027 s, 61.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0198852 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167884 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190944 s, 54.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95203 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74074 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.44@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.44@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.149268 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00780894 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.006592 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181124 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0307078 s, 34.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0259559 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271228 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314702 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264957 s, 39.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.027693 s, 37.9 MB/s , 0.0161194 s, 65.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.194412 s, 5.4 MB/s , 0.190836 s, 5.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.239475 s, 4.4 MB/s 1048576 bytes (1.0 MB) copied, 0.240319 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.283882 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.284752 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0149002 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0118873 s, 88.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0123901 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146799 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00601018 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143249 s, 73.2 MB/s 1048576 bytes (1.0 MB) copied, 0.014187 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148546 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161553 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016567 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164009 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182344 s, 57.5 MB/s , 0.0160222 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173866 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176182 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176485 s, 59.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9474 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74106 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.44@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.133104 s, 126 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111877 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155023 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0336688 s, 31.1 MB/s , 0.0309238 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290204 s, 36.1 MB/s , 0.0308858 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386888 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319376 s, 32.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451153 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039997 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446243 s, 23.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0519072 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535373 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535925 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454188 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457405 s, 22.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00619176 s, 169 MB/s , 0.00616827 s, 170 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00736906 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106056 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00792733 s, 132 MB/s 1048576 bytes (1.0 MB) copied, 0.00813047 s, 129 MB/s , 0.00825527 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00889716 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941801 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100444 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00975069 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993332 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102349 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104376 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983494 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115493 s, 90.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93611 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.88845 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.201676 s, 83.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144945 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197565 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216362 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307905 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416581 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411268 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305785 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435127 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436833 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388468 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0539514 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493632 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541424 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448564 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0475552 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488234 s, 21.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00918462 s, 114 MB/s , 0.00600786 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00832822 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00697203 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00747416 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00769203 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883583 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904323 s, 116 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103559 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010367 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104435 s, 100 MB/s 1048576 bytes (1.0 MB) copied, 0.0113596 s, 92.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0121945 s, 86.0 MB/s , 0.0109661 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112729 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110725 s, 94.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77979 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72985 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93261 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73607 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.150303 s, 112 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010387 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382946 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379052 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422458 s, 24.8 MB/s , 0.0387643 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373402 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.045881 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413033 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0873149 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113691 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116994 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115728 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130624 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129429 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13015 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129513 s, 8.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00962992 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977889 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100487 s, 104 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123261 s, 85.1 MB/s , 0.0110464 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116965 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119418 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157532 s, 66.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134407 s, 78.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0136485 s, 76.8 MB/s , 0.010203 s, 103 MB/s 1048576 bytes (1.0 MB) copied, 0.00467243 s, 224 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016151 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154572 s, 67.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183174 s, 57.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.679 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4468 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.175602 s, 95.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929837 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126561 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297777 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032863 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416562 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410133 s, 25.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0395218 s, 26.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0398067 s, 26.3 MB/s , 0.0372101 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0440158 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0516146 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.04027 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403734 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0514684 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460006 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0479173 s, 21.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0064821 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0069133 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986639 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00816333 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122313 s, 85.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984664 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992531 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104743 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118463 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0118081 s, 88.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0126365 s, 83.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142863 s, 73.4 MB/s , 0.0137047 s, 76.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144239 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162686 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179723 s, 58.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.98014 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7461 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.158691 s, 106 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00680326 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800305 s, 131 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0120491 s, 87.0 MB/s , 0.00756798 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015249 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154156 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184447 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00681292 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00646525 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197097 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017909 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428862 s, 24.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0681291 s, 15.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0789998 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0830027 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0719191 s, 14.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00952317 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175332 s, 59.8 MB/s 1048576 bytes (1.0 MB) copied, 0.00990191 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115753 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0144277 s, 72.7 MB/s , 0.0137829 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137723 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117902 s, 88.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201054 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169563 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174541 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144126 s, 72.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00234593 s, 447 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167117 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164002 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192233 s, 54.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.82317 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74239 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.82484 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74793 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (350s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 16:03:20 (1713297800) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 16:03:26 (1713297806) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg144-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.94732 s, 266 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg144-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0333 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 16:04:18 (1713297858) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.079322 s, 212 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898395 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869258 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168549 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214953 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027884 s, 37.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352181 s, 29.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0259315 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299269 s, 35.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354817 s, 29.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0388073 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322937 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345515 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420741 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355189 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043177 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438279 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00791282 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00740402 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00668335 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00708112 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822964 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918246 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109397 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079067 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932633 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00841951 s, 125 MB/s , 0.00850557 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112987 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101603 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01132 s, 92.6 MB/s , 0.00951306 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120589 s, 87.0 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg144-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92464 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96578 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.233363 s, 71.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122102 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00753282 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101319 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212969 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413163 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433546 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448667 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450371 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495176 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.080815 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0869684 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0831481 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.267038 s, 3.9 MB/s , 0.262434 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.317882 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.31648 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0160316 s, 65.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0170661 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181489 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153721 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212232 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199652 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172066 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173482 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191006 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193524 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195626 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211455 s, 49.6 MB/s , 0.0210978 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021878 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220135 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231528 s, 45.3 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg144-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg144-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9437 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.967 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (63s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77o: Changing rank should not panic ====== 16:05:23 (1713297923) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 16:05:27 (1713297927) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (26s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 16:05:54 (1713297954) mds.MDS.mdt.nrs_policies=tbf oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg144-client: oleg144-server: ssh exited with exit code 36 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 oleg144-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg144-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg144-client: oleg144-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77r: Change type of tbf policy at run time ========================================================== 16:06:05 (1713297965) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 25937 kill md_thread xargs PASS 77r (37s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 78: Enable policy and specify tunings right away ========================================================== 16:06:44 (1713298004) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 79: xattr: intent error ================== 16:06:48 (1713298008) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506771 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-16 16:06:48.000000000 -0400 Modify: 2024-04-16 16:06:48.000000000 -0400 Change: 2024-04-16 16:06:48.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 80a: migrate directory when some children is being opened ========================================================== 16:07:01 (1713298021) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 80b: Accessing directory during migration ========================================================== 16:07:04 (1713298024) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81a: rename and stat under striped directory ========================================================== 16:07:06 (1713298026) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 16:07:08 (1713298028) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 16:07:12 (1713298032) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 16:07:14 (1713298034) total: 2500 open/close in 6.30 seconds: 396.88 ops/second snapshot_time 1713298056.044527539 secs.nsecs start_time 1713298035.954680765 secs.nsecs elapsed_time 20.089846774 secs.nsecs open 2505 samples [usecs] 69 1572 873859 348134427 close 2504 samples [usecs] 51 303 261320 31236546 mknod 2501 samples [usecs] 179 1525 787650 283816484 unlink 2500 samples [usecs] 304 21512 1416718 3012789576 mkdir 2 samples [usecs] 695 828 1523 1168609 rmdir 2 samples [usecs] 1562 5114 6676 28592840 rename 2500 samples [usecs] 232 9650 2425688 3726207920 getattr 3760 samples [usecs] 7 918 60026 2041492 parallel_rename_file 2500 samples [usecs] 232 9650 2425688 3726207920 crossdir_rename 2500 samples [usecs] 232 9650 2425688 3726207920 PASS 81d (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 16:07:38 (1713298058) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6920 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6920 PASS 82 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 16:07:41 (1713298061) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 84: 0-nlink race in lu_object_find() ===== 16:07:44 (1713298064) fail_loc=0x8000060b PASS 84 (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 85: Lustre API root cache race =========== 16:07:52 (1713298072) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1713298073 Finishing test test1 at 1713298073 PASS 85 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 90: open/create and unlink striped directory ========================================================== 16:07:56 (1713298076) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 91: chmod and unlink striped directory === 16:07:59 (1713298079) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 92: create remote directory under orphan directory ========================================================== 16:08:02 (1713298082) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 16:08:04 (1713298084) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6170 0x181a 0x280000400 0 6169 0x1819 0x240000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6170 0x181a 0x240000400 1 6171 0x181b 0x280000400 rc1=0 and rc2=0 PASS 93 (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 94: signal vs CP callback race =========== 16:08:14 (1713298094) osc.lustre-OST0000-osc-ffff88012b467000.idle_timeout=0 osc.lustre-OST0000-osc-ffff88013738e000.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012b467000.idle_timeout=0 osc.lustre-OST0001-osc-ffff88013738e000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0564118 s, 145 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99623 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5667: 16691 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.017861 s, 229 kB/s osc.lustre-OST0000-osc-ffff88012b467000.idle_timeout=debug osc.lustre-OST0000-osc-ffff88013738e000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012b467000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88013738e000.idle_timeout=debug PASS 94 (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 16:08:26 (1713298106) llite.lustre-ffff88012b467000.fast_read=0 llite.lustre-ffff88013738e000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0234293 s, 89.5 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff88012b467000.fast_read=1 llite.lustre-ffff88013738e000.fast_read=1 PASS 95a (19s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 16:08:47 (1713298127) llite.lustre-ffff88012b467000.fast_read=0 llite.lustre-ffff88013738e000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00158074 s, 7.8 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff88012b467000.fast_read=1 llite.lustre-ffff88013738e000.fast_read=1 PASS 95b (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 16:08:56 (1713298136) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 16:08:59 (1713298139) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00162022 s, 2.5 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 16:09:02 (1713298142) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0246184 s, 85.2 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 16:09:05 (1713298145) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0230972 s, 90.8 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100e: DoM: read on open and file size ==== 16:09:08 (1713298148) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00201784 s, 5.9 MB/s PASS 100e (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101a: Discard DoM data on unlink ========= 16:09:12 (1713298152) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000572457 s, 7.2 MB/s PASS 101a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101b: Discard DoM data on rename ========= 16:09:15 (1713298155) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000618219 s, 6.6 MB/s PASS 101b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101c: Discard DoM data on close-unlink === 16:09:18 (1713298158) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000591085 s, 6.9 MB/s PASS 101c (3s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_102 skipping ALWAYS excluded test 102 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 103: Test size correctness with lockahead ========================================================== 16:09:23 (1713298163) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1713298164 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1713298166 Finishing test test23 at 1713298166 PASS 103 (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 16:09:31 (1713298171) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 105: Glimpse and lock cancel race ======== 16:09:33 (1713298173) fail_loc=0x416 PASS 105 (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106a: Verify the btime via statx() ======= 16:09:57 (1713298197) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106b: Glimpse RPCs test for statx ======== 16:10:00 (1713298200) SKIP: sanityn test_106b statx() only test SKIP 106b (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106c: Verify statx attributes mask ======= 16:10:02 (1713298202) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 107a: Basic grouplock conflict =========== 16:10:05 (1713298205) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0540951 s, 194 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6176 0x1820 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 First grouplock blocks second one PASS 107a (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 16:10:10 (1713298210) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0637476 s, 164 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6178 0x1822 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6920 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 108a: lseek: parallel updates ============ 16:10:19 (1713298219) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00175792 s, 5.7 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.17126 s, 2.0 MB/s PASS 108a (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 109: Race with several mount instances on 1 node ========================================================== 16:10:26 (1713298226) 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 7 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 8 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 11 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 12 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 13 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 16 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 19 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 27 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 30 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 31 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 34 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 35 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 38 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 42 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 44 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 45 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 49 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 50 Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 192.168.201.144@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.144@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg144-client.virtnet /mnt/lustre (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg144-server' oleg144-server: oleg144-server.virtnet: executing load_modules_local oleg144-server: Loading modules from /home/green/git/lustre-release/lustre oleg144-server: detected 4 online CPUs by sysfs oleg144-server: Force libcfs to create 2 CPU partitions oleg144-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre Starting client: oleg144-client.virtnet: -o user_xattr,flock oleg144-server@tcp:/lustre /mnt/lustre2 PASS 109 (396s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 110: do not grant another lock on resend ========================================================== 16:17:04 (1713298624) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-16 16:17:04.000000000 -0400 Modify: 2024-04-16 16:17:04.000000000 -0400 Change: 2024-04-16 16:17:04.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg144-client.virtnet oleg144-client.virtnet PASS 110 (65s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 16:18:10 (1713298690) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 112: update max-inherit in default LMV === 16:18:13 (1713298693) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 113: check servers of specified fs ======= 16:18:15 (1713298695) lustre-OST0000-osc-ffff8800a71d6000 active. lustre-OST0001-osc-ffff8800a71d6000 active. lustre-OST0000-osc-ffff8800a71d6000 active. lustre-OST0001-osc-ffff8800a71d6000 active. MGC192.168.201.144@tcp active. PASS 113 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 114: implicit default LMV inherit ======== 16:18:18 (1713298698) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 16:18:21 (1713298701) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (1s) debug_raw_pointers=0 debug_raw_pointers=0 cleanup: ====================================================== == sanityn test complete, duration 4817 sec ============== 16:18:23 (1713298703) === sanityn: start cleanup 16:18:23 (1713298703) === Stopping clients: oleg144-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg144-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 16:19:02 (1713298742) ===