== sanity test 115: verify dynamic thread creation============================================================================== 17:16:53 (1713388613) Stopping clients: oleg325-client.virtnet /mnt/lustre (opts:) Stopping client oleg325-client.virtnet /mnt/lustre opts: Stopping clients: oleg325-client.virtnet /mnt/lustre2 (opts:) Stopping /mnt/lustre-mds1 (opts:-f) on oleg325-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg325-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg325-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg325-server Checking servers environments Checking clients oleg325-client.virtnet environments Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg325-server' oleg325-server: oleg325-server.virtnet: executing load_modules_local oleg325-server: Loading modules from /home/green/git/lustre-release/lustre oleg325-server: detected 4 online CPUs by sysfs oleg325-server: Force libcfs to create 2 CPU partitions oleg325-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Setup mgs, mdt, osts Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg325-server: oleg325-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg325-client: oleg325-server: ssh exited with exit code 1 Started lustre-MDT0000 Starting mds2: -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg325-server: oleg325-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg325-client: oleg325-server: ssh exited with exit code 1 Started lustre-MDT0001 Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 oleg325-server: oleg325-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg325-client: oleg325-server: ssh exited with exit code 1 Started lustre-OST0000 Starting ost2: -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 oleg325-server: oleg325-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg325-client: oleg325-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg325-client.virtnet: -o user_xattr,flock oleg325-server@tcp:/lustre /mnt/lustre Starting client oleg325-client.virtnet: -o user_xattr,flock oleg325-server@tcp:/lustre /mnt/lustre Started clients oleg325-client.virtnet: 192.168.203.125@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b5d8b800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5d8b800.idle_timeout=debug disable quota as required Starting with 3 threads osc.lustre-OST0000-osc-ffff8800b5d8b800.max_rpcs_in_flight=12 osc.lustre-OST0001-osc-ffff8800b5d8b800.max_rpcs_in_flight=12 ost.OSS.ost_io.threads_max=6 striped dir -i1 -c2 -H all_char /mnt/lustre/d115.sanity Waiting for thread_started to reach thread_max .ll_ost_io thread_started 6, equal/greater than thread_max 6 /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15864 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15866 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15868 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15870 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15873 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15875 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15877 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15879 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15881 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15883 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15885 Killed ( $WTL $file $timeout ) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 12007: 15887 Killed ( $WTL $file $timeout )