-----============= acceptance-small: sanity-pcc ============----- Fri Apr 19 05:18:55 EDT 2024 === sanity-pcc: start setup 05:19:02 (1713518342) === Starting client oleg201-client.virtnet: -o user_xattr,flock oleg201-server@tcp:/lustre /mnt/lustre2 Started clients oleg201-client.virtnet: 192.168.202.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg201-client.virtnet: executing check_config_client /mnt/lustre oleg201-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg201-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b63d3000.idle_timeout=debug osc.lustre-OST0000-osc-ffff880130e90000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b63d3000.idle_timeout=debug osc.lustre-OST0001-osc-ffff880130e90000.idle_timeout=debug disable quota as required oleg201-server: oleg201-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-pcc: finish setup 05:19:17 (1713518357) === running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7672] nobody:x:99: excepting tests: Killing existing copytools on oleg201-client.virtnet pkill: pidfile not valid Try `pkill --help' for more information. Set HSM on and start Set sanity-hsm HSM policy debug_raw_pointers=Y debug_raw_pointers=Y == sanity-pcc test 21c: Verify HSM release works storing PCC-RO as HSM mirror component ========================================================== 05:19:41 (1713518381) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.0716322 s, 732 MB/s mke2fs 1.46.2.wc5 (26-Mar-2022) Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features. Discarding device blocks: 1024/51200 done Creating filesystem with 51200 1k blocks and 12824 inodes Filesystem UUID: bb6084f3-df51-42c3-af03-1c638cef9c34 Superblock backups stored on blocks: 8193, 24577, 40961 Allocating group tables: 0/7 done Writing inode tables: 0/7 done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: 0/7 done /tmp/f21c.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=bb6084f3-df51-42c3-af03-1c638cef9c34 (extents) (64bit) (large files) (huge files) Starting copytool 'agt1' on 'oleg201-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d21c.sanity-pcc/d21c.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/f21c.sanity-pcc: (0x00000009) exists archived, archive_id:2 /mnt/lustre/f21c.sanity-pcc lcm_layout_gen: 3 lcm_mirror_count: 2 lcm_entry_count: 2 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: released lmm_layout_gen: 0 lmm_stripe_offset: 0 lcme_id: 131074 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lhm_magic: 0x0BD70BD0 lhm_pattern: hsm lhm_length: 56 lhm_type: 0x00000003 (pccro) lhm_flags: exists,archived,pccro,0x109 lhm_archive_id: 0 lhm_archive_ver: 0 lhm_archive_uuid: '' /mnt/lustre/f21c.sanity-pcc: (0x0000000d) released exists archived, archive_id:2 Cleanup PCC backend on /mnt/lustre PASS 21c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanity-pcc test 30: Test lctl pcc del command ========= 05:19:49 (1713518389) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.0889414 s, 589 MB/s mke2fs 1.46.2.wc5 (26-Mar-2022) Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features. Discarding device blocks: 1024/51200 done Creating filesystem with 51200 1k blocks and 12824 inodes Filesystem UUID: 8c081ff8-b63f-4d79-95d3-e2b0c35866db Superblock backups stored on blocks: 8193, 24577, 40961 Allocating group tables: 0/7 done Writing inode tables: 0/7 done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: 0/7 done /tmp/f30.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=8c081ff8-b63f-4d79-95d3-e2b0c35866db (extents) (64bit) (large files) (huge files) Starting copytool 'agt1' on 'oleg201-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre"' pcc: - pccpath: /mnt/pcc.d30.sanity-pcc/d30.sanity-pcc hsmtool: posix_v2 rwid: 2 roid: 2 flags: 3e autocache: projid={100} check file 0:'0404' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc' check file 0:'0406' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc' check file 0:'0400' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc' check file 1:'0x240000402:0x6:0x0' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc/0404' Handle the file: 0x240000402:0x6:0x0 Detach and remove the PCC cached file: 0x240000402:0x6:0x0 check file 1:'0x240000402:0x4:0x0' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc/0406' Handle the file: 0x240000402:0x4:0x0 Remove non-cached file: /mnt/pcc.d30.sanity-pcc/d30.sanity-pcc/0406/0x240000402:0x4:0x0 flags: 1 check file 1:'0x240000402:0x2:0x0' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc/0400' Handle the file: 0x240000402:0x2:0x0 Detach and remove the PCC cached file: 0x240000402:0x2:0x0 Cleanup PCC backend on /mnt/lustre PASS 30 (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanity-pcc test 31: Test lctl pcc del command with --keep option ========================================================== 05:19:57 (1713518397) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.0700585 s, 748 MB/s mke2fs 1.46.2.wc5 (26-Mar-2022) Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features. Discarding device blocks: 1024/51200 done Creating filesystem with 51200 1k blocks and 12824 inodes Filesystem UUID: c108e1d8-909c-46a6-a91b-263f267e0e4c Superblock backups stored on blocks: 8193, 24577, 40961 Allocating group tables: 0/7 done Writing inode tables: 0/7 done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: 0/7 done /tmp/f31.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=c108e1d8-909c-46a6-a91b-263f267e0e4c (extents) (64bit) (large files) (huge files) Starting copytool 'agt1' on 'oleg201-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d31.sanity-pcc/d31.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre"' pcc: - pccpath: /mnt/pcc.d31.sanity-pcc/d31.sanity-pcc hsmtool: posix_v2 rwid: 2 roid: 2 flags: 30 autocache: projid={100} Cleanup PCC backend on /mnt/lustre PASS 31 (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanity-pcc test 32: Test for RO-PCC when PCC copy is deleted ========================================================== 05:20:05 (1713518405) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.0785765 s, 667 MB/s mke2fs 1.46.2.wc5 (26-Mar-2022) Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features. Discarding device blocks: 1024/51200 done Creating filesystem with 51200 1k blocks and 12824 inodes Filesystem UUID: c744abe6-afe9-4ee5-a98d-feb3505191b9 Superblock backups stored on blocks: 8193, 24577, 40961 Allocating group tables: 0/7 done Writing inode tables: 0/7 done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: 0/7 done /tmp/f32.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=c744abe6-afe9-4ee5-a98d-feb3505191b9 (extents) (64bit) (large files) (huge files) Starting copytool 'agt1' on 'oleg201-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d32.sanity-pcc/d32.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre"' multiop /mnt/lustre/f32.sanity-pcc vo_rc TMPPIPE=/tmp/multiop_open_wait_pipe.12810 client oleg201-client.virtnet multiop_bg started multiop_pid=12821 Stopping multiop_pid=12821 (kill 12821 on oleg201-client.virtnet) Cleanup PCC backend on /mnt/lustre PASS 32 (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanity-pcc test 36a: Stale RW-PCC copy should be deleted after remove the PCC backend ========================================================== 05:20:17 (1713518417) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.0675806 s, 776 MB/s mke2fs 1.46.2.wc5 (26-Mar-2022) Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features. Discarding device blocks: 1024/51200 done Creating filesystem with 51200 1k blocks and 12824 inodes Filesystem UUID: 4a1e5bf5-b77f-4389-9888-52e1613743c9 Superblock backups stored on blocks: 8193, 24577, 40961 Allocating group tables: 0/7 done Writing inode tables: 0/7 done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: 0/7 done /tmp/f36a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=4a1e5bf5-b77f-4389-9888-52e1613743c9 (extents) (64bit) (large files) (huge files) Starting copytool 'agt1' on 'oleg201-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d36a.sanity-pcc/d36a.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre"' check file 0:'040e' under directory '/mnt/pcc.d36a.sanity-pcc/d36a.sanity-pcc' check file 1:'0x200000402:0xc:0x0' under directory '/mnt/pcc.d36a.sanity-pcc/d36a.sanity-pcc/040e' Handle the file: 0x200000402:0xc:0x0 Remove non-cached file: /mnt/pcc.d36a.sanity-pcc/d36a.sanity-pcc/040e/0x200000402:0xc:0x0 flags: 1 Cleanup PCC backend on /mnt/lustre PASS 36a (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanity-pcc test 36b: Stale RO-PCC copy should be deleted after remove the PCC backend ========================================================== 05:20:22 (1713518422) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.0590821 s, 887 MB/s mke2fs 1.46.2.wc5 (26-Mar-2022) Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features. Discarding device blocks: 1024/51200 done Creating filesystem with 51200 1k blocks and 12824 inodes Filesystem UUID: 339ef020-77f3-4a20-b2fc-ad2d5d9dac41 Superblock backups stored on blocks: 8193, 24577, 40961 Allocating group tables: 0/7 done Writing inode tables: 0/7 done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: 0/7 done /tmp/f36b.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=339ef020-77f3-4a20-b2fc-ad2d5d9dac41 (extents) (64bit) (large files) (huge files) Starting copytool 'agt1' on 'oleg201-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d36b.sanity-pcc/d36b.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre"' check file 0:'040c' under directory '/mnt/pcc.d36b.sanity-pcc/d36b.sanity-pcc' check file 1:'0x200000402:0xe:0x0' under directory '/mnt/pcc.d36b.sanity-pcc/d36b.sanity-pcc/040c' Handle the file: 0x200000402:0xe:0x0 Remove non-cached file: /mnt/pcc.d36b.sanity-pcc/d36b.sanity-pcc/040c/0x200000402:0xe:0x0 flags: 1 Cleanup PCC backend on /mnt/lustre PASS 36b (4s) debug_raw_pointers=0 debug_raw_pointers=0 == sanity-pcc test complete, duration 90 sec ============= 05:20:26 (1713518426) === sanity-pcc: start cleanup 05:20:27 (1713518427) === Stopping clients: oleg201-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg201-client.virtnet /mnt/lustre2 opts: === sanity-pcc: finish cleanup 05:20:27 (1713518427) ===