+ [23-01-29 20:02:29] '[' multi == multi ']'
+ [23-01-29 20:02:29] echo 'Clients: rdma-qe-31.rdma.lab.eng.rdu2.redhat.com'
Clients: rdma-qe-31.rdma.lab.eng.rdu2.redhat.com
+ [23-01-29 20:02:29] echo 'Servers: rdma-cki-01.rdma.lab.eng.rdu2.redhat.com'
Servers: rdma-cki-01.rdma.lab.eng.rdu2.redhat.com
+ [23-01-29 20:02:29] RQA_system_info_for_debug
+ [23-01-29 20:02:29] grep -i distro /etc/motd
+ [23-01-29 20:02:29] tr -d ' '
DISTRO=CentOS-Stream-9-20230123.0
+ [23-01-29 20:02:29] cat /etc/redhat-release
CentOS Stream release 9
+ [23-01-29 20:02:29] uname -a
Linux rdma-cki-01.rdma.lab.eng.rdu2.redhat.com 5.14.0-247.rt14.248.1955_761016865.el9.x86_64 #1 SMP PREEMPT_RT Sun Jan 29 23:37:40 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
+ [23-01-29 20:02:29] cat /proc/cmdline
BOOT_IMAGE=(hd0,gpt2)/vmlinuz-5.14.0-247.rt14.248.1955_761016865.el9.x86_64 root=UUID=150a6320-c5bf-4e35-b36e-cd812e9603c7 ro efi=runtime resume=UUID=1fec4e56-5792-4a21-bb2c-f86f713c7639 console=ttyS0,115200n81 crashkernel=1G-4G:384M,4G-16G:512M,16G-64G:1G,64G-128G:2G,128G-:4G
+ [23-01-29 20:02:29] rpm -q rdma-core linux-firmware
rdma-core-41.0-3.el9.x86_64
linux-firmware-20221214-129.el9.noarch
+ [23-01-29 20:02:29] tail /sys/class/infiniband/mlx5_bond_0/fw_ver
22.99.7864
+ [23-01-29 20:02:29] lspci
+ [23-01-29 20:02:29] grep -i -e ethernet -e infiniband -e omni -e ConnectX
84:00.0 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
84:00.1 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
c5:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
c5:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
c5:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
c5:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
+ [23-01-29 20:02:29] lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   48 bits physical, 48 bits virtual
Byte Order:                      Little Endian
CPU(s):                          48
On-line CPU(s) list:             0-47
Vendor ID:                       AuthenticAMD
BIOS Vendor ID:                  Advanced Micro Devices, Inc.
Model name:                      AMD EPYC 7402P 24-Core Processor
BIOS Model name:                 AMD EPYC 7402P 24-Core Processor               
CPU family:                      23
Model:                           49
Thread(s) per core:              2
Core(s) per socket:              24
Socket(s):                       1
Stepping:                        0
Frequency boost:                 enabled
CPU max MHz:                     2800.0000
CPU min MHz:                     1500.0000
BogoMIPS:                        5589.71
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca
Virtualization:                  AMD-V
L1d cache:                       768 KiB (24 instances)
L1i cache:                       768 KiB (24 instances)
L2 cache:                        12 MiB (24 instances)
L3 cache:                        128 MiB (8 instances)
NUMA node(s):                    1
NUMA node0 CPU(s):               0-47
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Not affected
Vulnerability Retbleed:          Mitigation; untrained return thunk; SMT enabled with STIBP protection
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
+ [23-01-29 20:02:29] ibstat
CA 'mlx5_bond_0'
	CA type: MT4125
	Number of ports: 1
	Firmware version: 22.99.7864
	Hardware version: 0
	Node GUID: 0xb8599f0300f07fd0
	System image GUID: 0xb8599f0300f07fd0
	Port 1:
		State: Active
		Physical state: LinkUp
		Rate: 100
		Base lid: 0
		LMC: 0
		SM lid: 0
		Capability mask: 0x00010000
		Port GUID: 0xba599ffffef07fd0
		Link layer: Ethernet
+ [23-01-29 20:02:29] ibstatus
Infiniband device 'mlx5_bond_0' port 1 status:
	default gid:	 fe80:0000:0000:0000:ba59:9fff:fef0:7fd0
	base lid:	 0x0
	sm lid:		 0x0
	state:		 4: ACTIVE
	phys state:	 5: LinkUp
	rate:		 100 Gb/sec (4X EDR)
	link_layer:	 Ethernet

+ [23-01-29 20:02:29] /usr/bin/ibv_devinfo -v
hca_id:	mlx5_bond_0
	transport:			InfiniBand (0)
	fw_ver:				22.99.7864
	node_guid:			b859:9f03:00f0:7fd0
	sys_image_guid:			b859:9f03:00f0:7fd0
	vendor_id:			0x02c9
	vendor_part_id:			4125
	hw_ver:				0x0
	board_id:			MT_0000000359
	phys_port_cnt:			1
	max_mr_size:			0xffffffffffffffff
	page_size_cap:			0xfffffffffffff000
	max_qp:				131072
	max_qp_wr:			32768
	device_cap_flags:		0x25321c36
					BAD_PKEY_CNTR
					BAD_QKEY_CNTR
					AUTO_PATH_MIG
					CHANGE_PHY_PORT
					PORT_ACTIVE_EVENT
					SYS_IMAGE_GUID
					RC_RNR_NAK_GEN
					MEM_WINDOW
					XRC
					MEM_MGT_EXTENSIONS
					MEM_WINDOW_TYPE_2B
					RAW_IP_CSUM
					MANAGED_FLOW_STEERING
	max_sge:			30
	max_sge_rd:			30
	max_cq:				16777216
	max_cqe:			4194303
	max_mr:				16777216
	max_pd:				16777216
	max_qp_rd_atom:			16
	max_ee_rd_atom:			0
	max_res_rd_atom:		2097152
	max_qp_init_rd_atom:		16
	max_ee_init_rd_atom:		0
	atomic_cap:			ATOMIC_HCA (1)
	max_ee:				0
	max_rdd:			0
	max_mw:				16777216
	max_raw_ipv6_qp:		0
	max_raw_ethy_qp:		0
	max_mcast_grp:			2097152
	max_mcast_qp_attach:		240
	max_total_mcast_qp_attach:	503316480
	max_ah:				2147483647
	max_fmr:			0
	max_srq:			8388608
	max_srq_wr:			32767
	max_srq_sge:			31
	max_pkeys:			128
	local_ca_ack_delay:		16
	general_odp_caps:
					ODP_SUPPORT
					ODP_SUPPORT_IMPLICIT
	rc_odp_caps:
					SUPPORT_SEND
					SUPPORT_RECV
					SUPPORT_WRITE
					SUPPORT_READ
					SUPPORT_SRQ
	uc_odp_caps:
					NO SUPPORT
	ud_odp_caps:
					SUPPORT_SEND
	xrc_odp_caps:
					SUPPORT_SEND
					SUPPORT_WRITE
					SUPPORT_READ
					SUPPORT_SRQ
	completion timestamp_mask:			0x7fffffffffffffff
	hca_core_clock:			156250kHZ
	raw packet caps:
					C-VLAN stripping offload
					Scatter FCS offload
					IP csum offload
					Delay drop
	device_cap_flags_ex:		0x1425321C36
					RAW_SCATTER_FCS
					PCI_WRITE_END_PADDING
	tso_caps:
		max_tso:			262144
		supported_qp:
					SUPPORT_RAW_PACKET
	rss_caps:
		max_rwq_indirection_tables:			65536
		max_rwq_indirection_table_size:			2048
		rx_hash_function:				0x1
		rx_hash_fields_mask:				0x800000FF
		supported_qp:
					SUPPORT_RAW_PACKET
	max_wq_type_rq:			8388608
	packet_pacing_caps:
		qp_rate_limit_min:	1kbps
		qp_rate_limit_max:	100000000kbps
		supported_qp:
					SUPPORT_RAW_PACKET
	tag matching not supported

	cq moderation caps:
		max_cq_count:	65535
		max_cq_period:	4095 us

	maximum available device memory:	131072Bytes

	num_comp_vectors:		48
		port:	1
			state:			PORT_ACTIVE (4)
			max_mtu:		4096 (5)
			active_mtu:		4096 (5)
			sm_lid:			0
			port_lid:		0
			port_lmc:		0x00
			link_layer:		Ethernet
			max_msg_sz:		0x40000000
			port_cap_flags:		0x04010000
			port_cap_flags2:	0x0000
			max_vl_num:		invalid value (0)
			bad_pkey_cntr:		0x0
			qkey_viol_cntr:		0x0
			sm_sl:			0
			pkey_tbl_len:		1
			gid_tbl_len:		256
			subnet_timeout:		0
			init_type_reply:	0
			active_width:		4X (2)
			active_speed:		25.0 Gbps (32)
			phys_state:		LINK_UP (5)
			GID[  0]:		fe80:0000:0000:0000:ba59:9fff:fef0:7fd0, RoCE v1
			GID[  1]:		fe80::ba59:9fff:fef0:7fd0, RoCE v2
			GID[  2]:		fe80:0000:0000:0000:ba59:9fff:fef0:7fd0, RoCE v1
			GID[  3]:		fe80::ba59:9fff:fef0:7fd0, RoCE v2
			GID[  4]:		fe80:0000:0000:0000:ba59:9fff:fef0:7fd0, RoCE v1
			GID[  5]:		fe80::ba59:9fff:fef0:7fd0, RoCE v2
			GID[  6]:		0000:0000:0000:0000:0000:ffff:ac1f:2851, RoCE v1
			GID[  7]:		::ffff:172.31.40.81, RoCE v2
			GID[  8]:		0000:0000:0000:0000:0000:ffff:ac1f:2b51, RoCE v1
			GID[  9]:		::ffff:172.31.43.81, RoCE v2
			GID[ 10]:		0000:0000:0000:0000:0000:ffff:ac1f:2d51, RoCE v1
			GID[ 11]:		::ffff:172.31.45.81, RoCE v2

+ [23-01-29 20:02:29] iblinkinfo
ibwarn: [11676] mad_rpc_open_port: client_register for mgmt 1 failed
Failed to open (null) port 0
+ [23-01-29 20:02:29] [[ '' == *\o\p\a* ]]
+ [23-01-29 20:02:29] ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: lom_1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 48:df:37:ad:f7:84 brd ff:ff:ff:ff:ff:ff
    altname enp197s0f0
    altname eno5
    inet 10.1.217.16/24 brd 10.1.217.255 scope global dynamic noprefixroute lom_1
       valid_lft 86179sec preferred_lft 86179sec
    inet6 2620:52:0:1d9:4adf:37ff:fead:f784/64 scope global dynamic noprefixroute 
       valid_lft 2591909sec preferred_lft 604709sec
    inet6 fe80::4adf:37ff:fead:f784/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: lom_2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 48:df:37:ad:f7:85 brd ff:ff:ff:ff:ff:ff
    altname enp197s0f1
    altname eno6
4: lom_3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 48:df:37:ad:f7:86 brd ff:ff:ff:ff:ff:ff
    altname enp197s0f2
    altname eno7
5: lom_4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 48:df:37:ad:f7:87 brd ff:ff:ff:ff:ff:ff
    altname enp197s0f3
    altname eno8
6: mlx5_slave1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master mlx5_bond_roce state UP group default qlen 1000
    link/ether b8:59:9f:f0:7f:d0 brd ff:ff:ff:ff:ff:ff
    altname enp132s0f0np0
    altname ens1f0np0
7: mlx5_slave2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master mlx5_bond_roce state UP group default qlen 1000
    link/ether b8:59:9f:f0:7f:d0 brd ff:ff:ff:ff:ff:ff permaddr b8:59:9f:f0:7f:d1
    altname enp132s0f1np1
    altname ens1f1np1
8: mlx5_bond_roce: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether b8:59:9f:f0:7f:d0 brd ff:ff:ff:ff:ff:ff
    inet 172.31.40.81/24 brd 172.31.40.255 scope global dynamic noprefixroute mlx5_bond_roce
       valid_lft 3400sec preferred_lft 3400sec
    inet6 fe80::ba59:9fff:fef0:7fd0/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
9: mlx5_bond_ro.43@mlx5_bond_roce: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether b8:59:9f:f0:7f:d0 brd ff:ff:ff:ff:ff:ff
    inet 172.31.43.81/24 brd 172.31.43.255 scope global dynamic noprefixroute mlx5_bond_ro.43
       valid_lft 3403sec preferred_lft 3403sec
    inet6 fe80::ba59:9fff:fef0:7fd0/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
10: mlx5_bond_ro.45@mlx5_bond_roce: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether b8:59:9f:f0:7f:d0 brd ff:ff:ff:ff:ff:ff
    inet 172.31.45.81/24 brd 172.31.45.255 scope global dynamic noprefixroute mlx5_bond_ro.45
       valid_lft 3404sec preferred_lft 3404sec
    inet6 fe80::ba59:9fff:fef0:7fd0/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
+ [23-01-29 20:02:29] grep -i fedora /etc/redhat-release
+ [23-01-29 20:02:29] [[ '' == \i\4\0\i\w ]]
+ [23-01-29 20:02:29] hostname -A
+ [23-01-29 20:02:29] grep rdma-qe-31
+ [23-01-29 20:02:29] hostname -A
+ [23-01-29 20:02:29] grep rdma-cki-01
+ [23-01-29 20:02:29] echo '------- server is ready -----'
------- server is ready -----
+ [23-01-29 20:02:29] server
+ [23-01-29 20:02:29] source ./setup.sh
+ [23-01-29 20:02:29] server_setup
+ [23-01-29 20:02:29] is_cki
+ [23-01-29 20:02:29] typeset -l flag=true
+ [23-01-29 20:02:29] [[ true == \t\r\u\e ]]
+ [23-01-29 20:02:29] return 0
+ [23-01-29 20:02:29] run_task_env_setup
+ [23-01-29 20:02:29] echo 'Now start to run task env_setup, please be patient ...'
Now start to run task env_setup, please be patient ...
+ [23-01-29 20:02:29] cd ../../env_setup
+ [23-01-29 20:02:29] make run
make[1]: Entering directory '/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.zip/infiniband/env_setup'
test -x runtest.sh || chmod a+x runtest.sh
./runtest.sh
+ trap 'exit 1' SIGHUP SIGINT SIGQUIT SIGTERM
+ which wget
+ alias
+ eval declare -f
++ declare -f
+ /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot wget
/usr/bin/wget
+ source /usr/bin/rhts_environment.sh
++ '[' -z /mnt/testarea/tmp.d0VpM0 ']'
++ '[' -z '' ']'
+++ uname -i
++ ARCH=x86_64
++ '[' -z '' ']'
+++ cat /etc/redhat-release
+++ sed -e 's/\(.*\)release\s\([0-9]*\).*/\1\2/; s/\s//g'
++ FAMILY=CentOSStream9
++ '[' -h /mnt/testarea/current.log ']'
++ ln -sf /mnt/testarea/tmp.d0VpM0 /mnt/testarea/current.log
+ '[' -f /root/rdma-qa.sh ']'
+ source /root/rdma-qa.sh
++ PS4='$(date +"+ [%y-%m-%d %H:%M:%S]") '
++ [23-01-29 20:02:29] export NIC_PREF=
++ [23-01-29 20:02:29] NIC_PREF=
++ [23-01-29 20:02:29] export PERFTEST_FLAGS=
++ [23-01-29 20:02:29] PERFTEST_FLAGS=
+++ [23-01-29 20:02:29] hostname -s
++ [23-01-29 20:02:29] export s_hostname=rdma-cki-01
++ [23-01-29 20:02:29] s_hostname=rdma-cki-01
++ [23-01-29 20:02:29] RQA_set_pyexec
++ [23-01-29 20:02:29] PYEXEC=
++ [23-01-29 20:02:29] which python
++ [23-01-29 20:02:29] export PYEXEC=python
++ [23-01-29 20:02:29] PYEXEC=python
++ [23-01-29 20:02:29] return 0
+++ [23-01-29 20:02:29] grep -i fedora /etc/redhat-release
++ [23-01-29 20:02:29] [[ -n '' ]]
+++ [23-01-29 20:02:29] RQA_get_rhel_major
+++ [23-01-29 20:02:29] awk -F . '{print $1}'
++++ [23-01-29 20:02:29] RQA_get_rhel_release
++++ [23-01-29 20:02:29] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:02:29] echo 9
++ [23-01-29 20:02:29] [[ 9 -ge 8 ]]
++ [23-01-29 20:02:29] export 'PKGINSTALL=dnf install -y --setopt=strict=0 --nogpgcheck'
++ [23-01-29 20:02:29] PKGINSTALL='dnf install -y --setopt=strict=0 --nogpgcheck'
++ [23-01-29 20:02:29] export 'PKGREMOVE=dnf remove --noautoremove -y'
++ [23-01-29 20:02:29] PKGREMOVE='dnf remove --noautoremove -y'
++ [23-01-29 20:02:29] [[ rdma-cki-01 == \r\d\m\a\-\d\e\v\-\2\0 ]]
++ [23-01-29 20:02:29] export LATEST_MFT=http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz
++ [23-01-29 20:02:29] LATEST_MFT=http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz
++ [23-01-29 20:02:29] '[' -f /root/known-issues.sh ']'
++ [23-01-29 20:02:29] source /root/known-issues.sh
++ [23-01-29 20:02:34] RQA_load_tech_preview_modules
++ [23-01-29 20:02:34] [[ ! -e /etc/rdma/rdma.conf ]]
++ [23-01-29 20:02:34] return
++ [23-01-29 20:02:34] RQA_fix_uefi_reboot
++ [23-01-29 20:02:34] ls -l /usr/sbin/reboot
++ [23-01-29 20:02:34] grep rhts-reboot
++ [23-01-29 20:02:34] which rhts-reboot
++ [23-01-29 20:02:34] rpm -q rhts-test-env
++ [23-01-29 20:02:34] RQA_fix_ssh_ipv6_auth
+++ [23-01-29 20:02:34] RQA_get_rhel_major
+++ [23-01-29 20:02:34] awk -F . '{print $1}'
++++ [23-01-29 20:02:34] RQA_get_rhel_release
++++ [23-01-29 20:02:34] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:02:34] echo 9
++ [23-01-29 20:02:34] '[' 9 -ge 7 ']'
++ [23-01-29 20:02:34] local need_restart=0
++ [23-01-29 20:02:34] grep -E '^UseDNS yes' /etc/ssh/sshd_config
++ [23-01-29 20:02:34] grep -E '^GSSAPIStrictAcceptorCheck yes' /etc/ssh/sshd_config
++ [23-01-29 20:02:34] grep 'AddressFamily inet' /root/.ssh/config
++ [23-01-29 20:02:34] '[' 0 -eq 1 ']'
++ [23-01-29 20:02:34] RQA_fix_mlx5_ib_connected_mode
+++ [23-01-29 20:02:34] RQA_get_rhel_major
+++ [23-01-29 20:02:34] awk -F . '{print $1}'
++++ [23-01-29 20:02:34] RQA_get_rhel_release
++++ [23-01-29 20:02:34] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:02:34] echo 9
++ [23-01-29 20:02:34] [[ 9 -ge 7 ]]
+++ [23-01-29 20:02:34] RQA_get_rhel_minor
+++ [23-01-29 20:02:34] awk -F . '{print $2}'
++++ [23-01-29 20:02:34] RQA_get_rhel_release
++++ [23-01-29 20:02:34] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:02:34] echo 9
++ [23-01-29 20:02:34] [[ '' -ge 5 ]]
+++ [23-01-29 20:02:34] RQA_get_rhel_major
+++ [23-01-29 20:02:34] awk -F . '{print $1}'
++++ [23-01-29 20:02:34] RQA_get_rhel_release
++++ [23-01-29 20:02:34] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:02:34] echo 9
++ [23-01-29 20:02:34] [[ 9 -gt 7 ]]
+++ [23-01-29 20:02:34] ls /etc/sysconfig/network-scripts/
+++ [23-01-29 20:02:34] grep mlx5
+++ [23-01-29 20:02:34] grep ib
+++ [23-01-29 20:02:34] grep -v '~'
++ [23-01-29 20:02:34] RQA_nfs_rhel8_temp_fix
+++ [23-01-29 20:02:34] RQA_get_rhel_major
+++ [23-01-29 20:02:34] awk -F . '{print $1}'
++++ [23-01-29 20:02:34] RQA_get_rhel_release
++++ [23-01-29 20:02:34] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:02:34] echo 9
++ [23-01-29 20:02:34] ((  9 == 8  ))
++ [23-01-29 20:02:34] source /root/.bashrc
++ [23-01-29 20:02:35] [[ '' == \i\4\0\i\w ]]
+ [23-01-29 20:02:35] export BASHRC_FILE=/root/.bashrc
+ [23-01-29 20:02:35] BASHRC_FILE=/root/.bashrc
+ [23-01-29 20:02:35] [[ rdma-cki-01 == \r\d\m\a\-\p\e\r\f\-\0\2 ]]
+ [23-01-29 20:02:35] [[ rdma-cki-01 == \r\d\m\a\-\p\e\r\f\-\0\3 ]]
+ [23-01-29 20:02:35] export ENV_DRIVER=
+ [23-01-29 20:02:35] ENV_DRIVER=
+ [23-01-29 20:02:35] export ENV_NETWORK=roce
+ [23-01-29 20:02:35] ENV_NETWORK=roce
++ [23-01-29 20:02:35] hostname -s
+ [23-01-29 20:02:35] export host_json=./json/rdma-cki-01.json
+ [23-01-29 20:02:35] host_json=./json/rdma-cki-01.json
+ [23-01-29 20:02:35] [[ rdma-cki-01.rdma.lab.eng.rdu2.redhat.com == *\r\d\m\a\-\d\e\v\-\3\0* ]]
+ [23-01-29 20:02:35] [[ rdma-cki-01.rdma.lab.eng.rdu2.redhat.com == *\r\d\m\a\-\d\e\v\-\3\1* ]]
+ [23-01-29 20:02:35] [[ rdma-qe-31.rdma.lab.eng.rdu2.redhat.com == *\r\d\m\a\-\d\e\v\-\3\0* ]]
+ [23-01-29 20:02:35] [[ rdma-qe-31.rdma.lab.eng.rdu2.redhat.com == *\r\d\m\a\-\d\e\v\-\3\1* ]]
+ [23-01-29 20:02:35] '[' -z rdma-qe-31.rdma.lab.eng.rdu2.redhat.com -a -z rdma-cki-01.rdma.lab.eng.rdu2.redhat.com ']'
+ [23-01-29 20:02:35] RQA_set_servers_clients
++ [23-01-29 20:02:35] echo rdma-cki-01.rdma.lab.eng.rdu2.redhat.com
++ [23-01-29 20:02:35] awk -F . '{ print $1}'
+ [23-01-29 20:02:35] s_server_name=rdma-cki-01
++ [23-01-29 20:02:35] echo rdma-qe-31.rdma.lab.eng.rdu2.redhat.com
++ [23-01-29 20:02:35] awk -F . '{ print $1}'
+ [23-01-29 20:02:35] s_client_name=rdma-qe-31
+ [23-01-29 20:02:35] grep 'export SERVERS=' /root/.bashrc
+ [23-01-29 20:02:35] echo 'export SERVERS="rdma-cki-01"'
+ [23-01-29 20:02:35] grep 'export CLIENTS=' /root/.bashrc
+ [23-01-29 20:02:35] echo 'export CLIENTS="rdma-qe-31"'
+ [23-01-29 20:02:35] source /root/.bashrc
++ [23-01-29 20:02:35] '[' -f /etc/bashrc ']'
++ [23-01-29 20:02:35] . /etc/bashrc
+++ [23-01-29 20:02:35] '[' -z Y ']'
++ [23-01-29 20:02:35] '[' vt100 = vt102 ']'
++ [23-01-29 20:02:35] alias 'grep=/bin/grep --color=tty '
++ [23-01-29 20:02:35] alias vi=vim
++ [23-01-29 20:02:35] export EDITOR=vim
++ [23-01-29 20:02:35] EDITOR=vim
++ [23-01-29 20:02:35] RDMA_LOOKASIDE=http://download-node-02.eng.bos.redhat.com/qa/rhts/lookaside/rdma-testing
+++ [23-01-29 20:02:35] which rhts-reboot
+++ [23-01-29 20:02:35] alias
+++ [23-01-29 20:02:35] /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot rhts-reboot
+++ [23-01-29 20:02:35] eval declare -f
++++ [23-01-29 20:02:35] declare -f
++ [23-01-29 20:02:35] '[' '!' -x /usr/bin/rhts-reboot ']'
++ [23-01-29 20:02:35] alias shutdown=rhts-reboot
++ [23-01-29 20:02:35] export SERVERS=rdma-cki-01
++ [23-01-29 20:02:35] SERVERS=rdma-cki-01
++ [23-01-29 20:02:35] export CLIENTS=rdma-qe-31
++ [23-01-29 20:02:35] CLIENTS=rdma-qe-31
+ [23-01-29 20:02:35] RQA_rhts_or_dev_mode multi env
+ [23-01-29 20:02:35] export RUN_NUMBER=0
+ [23-01-29 20:02:35] RUN_NUMBER=0
++ [23-01-29 20:02:35] grep -i 'Job whiteboard:' /etc/motd
++ [23-01-29 20:02:35] awk -F '[' '{print $2}'
++ [23-01-29 20:02:35] sed 's/\]//g'
+ [23-01-29 20:02:35] merge_request=
+ [23-01-29 20:02:35] [[ ! -z '' ]]
+ [23-01-29 20:02:35] test -z 7478195
+ [23-01-29 20:02:35] echo 'Variable JOBID set, we'\''re running on RHTS'
Variable JOBID set, we're running on RHTS
+++ [23-01-29 20:02:35] pwd
++ [23-01-29 20:02:35] basename /mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.zip/infiniband/env_setup
+ [23-01-29 20:02:35] '[' env_setup == nfsordma_sh ']'
+ [23-01-29 20:02:35] exec
+ [23-01-29 20:02:35] exec
+ [23-01-29 20:02:35] '[' multi == multi ']'
+ [23-01-29 20:02:35] echo 'Clients: rdma-qe-31'
Clients: rdma-qe-31
+ [23-01-29 20:02:35] echo 'Servers: rdma-cki-01'
Servers: rdma-cki-01
+ [23-01-29 20:02:35] RQA_system_info_for_debug
+ [23-01-29 20:02:35] grep -i distro /etc/motd
+ [23-01-29 20:02:35] tr -d ' '
DISTRO=CentOS-Stream-9-20230123.0
+ [23-01-29 20:02:35] cat /etc/redhat-release
CentOS Stream release 9
+ [23-01-29 20:02:35] uname -a
Linux rdma-cki-01.rdma.lab.eng.rdu2.redhat.com 5.14.0-247.rt14.248.1955_761016865.el9.x86_64 #1 SMP PREEMPT_RT Sun Jan 29 23:37:40 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
+ [23-01-29 20:02:35] cat /proc/cmdline
BOOT_IMAGE=(hd0,gpt2)/vmlinuz-5.14.0-247.rt14.248.1955_761016865.el9.x86_64 root=UUID=150a6320-c5bf-4e35-b36e-cd812e9603c7 ro efi=runtime resume=UUID=1fec4e56-5792-4a21-bb2c-f86f713c7639 console=ttyS0,115200n81 crashkernel=1G-4G:384M,4G-16G:512M,16G-64G:1G,64G-128G:2G,128G-:4G
+ [23-01-29 20:02:35] rpm -q rdma-core linux-firmware
rdma-core-41.0-3.el9.x86_64
linux-firmware-20221214-129.el9.noarch
+ [23-01-29 20:02:35] tail /sys/class/infiniband/mlx5_bond_0/fw_ver
22.99.7864
+ [23-01-29 20:02:35] lspci
+ [23-01-29 20:02:35] grep -i -e ethernet -e infiniband -e omni -e ConnectX
84:00.0 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
84:00.1 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
c5:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
c5:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
c5:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
c5:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
+ [23-01-29 20:02:35] lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   48 bits physical, 48 bits virtual
Byte Order:                      Little Endian
CPU(s):                          48
On-line CPU(s) list:             0-47
Vendor ID:                       AuthenticAMD
BIOS Vendor ID:                  Advanced Micro Devices, Inc.
Model name:                      AMD EPYC 7402P 24-Core Processor
BIOS Model name:                 AMD EPYC 7402P 24-Core Processor               
CPU family:                      23
Model:                           49
Thread(s) per core:              2
Core(s) per socket:              24
Socket(s):                       1
Stepping:                        0
Frequency boost:                 enabled
CPU max MHz:                     2800.0000
CPU min MHz:                     1500.0000
BogoMIPS:                        5589.71
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca
Virtualization:                  AMD-V
L1d cache:                       768 KiB (24 instances)
L1i cache:                       768 KiB (24 instances)
L2 cache:                        12 MiB (24 instances)
L3 cache:                        128 MiB (8 instances)
NUMA node(s):                    1
NUMA node0 CPU(s):               0-47
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Not affected
Vulnerability Retbleed:          Mitigation; untrained return thunk; SMT enabled with STIBP protection
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
+ [23-01-29 20:02:35] ibstat
CA 'mlx5_bond_0'
	CA type: MT4125
	Number of ports: 1
	Firmware version: 22.99.7864
	Hardware version: 0
	Node GUID: 0xb8599f0300f07fd0
	System image GUID: 0xb8599f0300f07fd0
	Port 1:
		State: Active
		Physical state: LinkUp
		Rate: 100
		Base lid: 0
		LMC: 0
		SM lid: 0
		Capability mask: 0x00010000
		Port GUID: 0xba599ffffef07fd0
		Link layer: Ethernet
+ [23-01-29 20:02:35] ibstatus
Infiniband device 'mlx5_bond_0' port 1 status:
	default gid:	 fe80:0000:0000:0000:ba59:9fff:fef0:7fd0
	base lid:	 0x0
	sm lid:		 0x0
	state:		 4: ACTIVE
	phys state:	 5: LinkUp
	rate:		 100 Gb/sec (4X EDR)
	link_layer:	 Ethernet

+ [23-01-29 20:02:35] /usr/bin/ibv_devinfo -v
hca_id:	mlx5_bond_0
	transport:			InfiniBand (0)
	fw_ver:				22.99.7864
	node_guid:			b859:9f03:00f0:7fd0
	sys_image_guid:			b859:9f03:00f0:7fd0
	vendor_id:			0x02c9
	vendor_part_id:			4125
	hw_ver:				0x0
	board_id:			MT_0000000359
	phys_port_cnt:			1
	max_mr_size:			0xffffffffffffffff
	page_size_cap:			0xfffffffffffff000
	max_qp:				131072
	max_qp_wr:			32768
	device_cap_flags:		0x25321c36
					BAD_PKEY_CNTR
					BAD_QKEY_CNTR
					AUTO_PATH_MIG
					CHANGE_PHY_PORT
					PORT_ACTIVE_EVENT
					SYS_IMAGE_GUID
					RC_RNR_NAK_GEN
					MEM_WINDOW
					XRC
					MEM_MGT_EXTENSIONS
					MEM_WINDOW_TYPE_2B
					RAW_IP_CSUM
					MANAGED_FLOW_STEERING
	max_sge:			30
	max_sge_rd:			30
	max_cq:				16777216
	max_cqe:			4194303
	max_mr:				16777216
	max_pd:				16777216
	max_qp_rd_atom:			16
	max_ee_rd_atom:			0
	max_res_rd_atom:		2097152
	max_qp_init_rd_atom:		16
	max_ee_init_rd_atom:		0
	atomic_cap:			ATOMIC_HCA (1)
	max_ee:				0
	max_rdd:			0
	max_mw:				16777216
	max_raw_ipv6_qp:		0
	max_raw_ethy_qp:		0
	max_mcast_grp:			2097152
	max_mcast_qp_attach:		240
	max_total_mcast_qp_attach:	503316480
	max_ah:				2147483647
	max_fmr:			0
	max_srq:			8388608
	max_srq_wr:			32767
	max_srq_sge:			31
	max_pkeys:			128
	local_ca_ack_delay:		16
	general_odp_caps:
					ODP_SUPPORT
					ODP_SUPPORT_IMPLICIT
	rc_odp_caps:
					SUPPORT_SEND
					SUPPORT_RECV
					SUPPORT_WRITE
					SUPPORT_READ
					SUPPORT_SRQ
	uc_odp_caps:
					NO SUPPORT
	ud_odp_caps:
					SUPPORT_SEND
	xrc_odp_caps:
					SUPPORT_SEND
					SUPPORT_WRITE
					SUPPORT_READ
					SUPPORT_SRQ
	completion timestamp_mask:			0x7fffffffffffffff
	hca_core_clock:			156250kHZ
	raw packet caps:
					C-VLAN stripping offload
					Scatter FCS offload
					IP csum offload
					Delay drop
	device_cap_flags_ex:		0x1425321C36
					RAW_SCATTER_FCS
					PCI_WRITE_END_PADDING
	tso_caps:
		max_tso:			262144
		supported_qp:
					SUPPORT_RAW_PACKET
	rss_caps:
		max_rwq_indirection_tables:			65536
		max_rwq_indirection_table_size:			2048
		rx_hash_function:				0x1
		rx_hash_fields_mask:				0x800000FF
		supported_qp:
					SUPPORT_RAW_PACKET
	max_wq_type_rq:			8388608
	packet_pacing_caps:
		qp_rate_limit_min:	1kbps
		qp_rate_limit_max:	100000000kbps
		supported_qp:
					SUPPORT_RAW_PACKET
	tag matching not supported

	cq moderation caps:
		max_cq_count:	65535
		max_cq_period:	4095 us

	maximum available device memory:	131072Bytes

	num_comp_vectors:		48
		port:	1
			state:			PORT_ACTIVE (4)
			max_mtu:		4096 (5)
			active_mtu:		4096 (5)
			sm_lid:			0
			port_lid:		0
			port_lmc:		0x00
			link_layer:		Ethernet
			max_msg_sz:		0x40000000
			port_cap_flags:		0x04010000
			port_cap_flags2:	0x0000
			max_vl_num:		invalid value (0)
			bad_pkey_cntr:		0x0
			qkey_viol_cntr:		0x0
			sm_sl:			0
			pkey_tbl_len:		1
			gid_tbl_len:		256
			subnet_timeout:		0
			init_type_reply:	0
			active_width:		4X (2)
			active_speed:		25.0 Gbps (32)
			phys_state:		LINK_UP (5)
			GID[  0]:		fe80:0000:0000:0000:ba59:9fff:fef0:7fd0, RoCE v1
			GID[  1]:		fe80::ba59:9fff:fef0:7fd0, RoCE v2
			GID[  2]:		fe80:0000:0000:0000:ba59:9fff:fef0:7fd0, RoCE v1
			GID[  3]:		fe80::ba59:9fff:fef0:7fd0, RoCE v2
			GID[  4]:		fe80:0000:0000:0000:ba59:9fff:fef0:7fd0, RoCE v1
			GID[  5]:		fe80::ba59:9fff:fef0:7fd0, RoCE v2
			GID[  6]:		0000:0000:0000:0000:0000:ffff:ac1f:2851, RoCE v1
			GID[  7]:		::ffff:172.31.40.81, RoCE v2
			GID[  8]:		0000:0000:0000:0000:0000:ffff:ac1f:2b51, RoCE v1
			GID[  9]:		::ffff:172.31.43.81, RoCE v2
			GID[ 10]:		0000:0000:0000:0000:0000:ffff:ac1f:2d51, RoCE v1
			GID[ 11]:		::ffff:172.31.45.81, RoCE v2

+ [23-01-29 20:02:35] iblinkinfo
ibwarn: [15208] mad_rpc_open_port: client_register for mgmt 1 failed
Failed to open (null) port 0
+ [23-01-29 20:02:35] [[ '' == *\o\p\a* ]]
+ [23-01-29 20:02:35] ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: lom_1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 48:df:37:ad:f7:84 brd ff:ff:ff:ff:ff:ff
    altname enp197s0f0
    altname eno5
    inet 10.1.217.16/24 brd 10.1.217.255 scope global dynamic noprefixroute lom_1
       valid_lft 86173sec preferred_lft 86173sec
    inet6 2620:52:0:1d9:4adf:37ff:fead:f784/64 scope global dynamic noprefixroute 
       valid_lft 2591998sec preferred_lft 604798sec
    inet6 fe80::4adf:37ff:fead:f784/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: lom_2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 48:df:37:ad:f7:85 brd ff:ff:ff:ff:ff:ff
    altname enp197s0f1
    altname eno6
4: lom_3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 48:df:37:ad:f7:86 brd ff:ff:ff:ff:ff:ff
    altname enp197s0f2
    altname eno7
5: lom_4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 48:df:37:ad:f7:87 brd ff:ff:ff:ff:ff:ff
    altname enp197s0f3
    altname eno8
6: mlx5_slave1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master mlx5_bond_roce state UP group default qlen 1000
    link/ether b8:59:9f:f0:7f:d0 brd ff:ff:ff:ff:ff:ff
    altname enp132s0f0np0
    altname ens1f0np0
7: mlx5_slave2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master mlx5_bond_roce state UP group default qlen 1000
    link/ether b8:59:9f:f0:7f:d0 brd ff:ff:ff:ff:ff:ff permaddr b8:59:9f:f0:7f:d1
    altname enp132s0f1np1
    altname ens1f1np1
8: mlx5_bond_roce: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether b8:59:9f:f0:7f:d0 brd ff:ff:ff:ff:ff:ff
    inet 172.31.40.81/24 brd 172.31.40.255 scope global dynamic noprefixroute mlx5_bond_roce
       valid_lft 3394sec preferred_lft 3394sec
    inet6 fe80::ba59:9fff:fef0:7fd0/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
9: mlx5_bond_ro.43@mlx5_bond_roce: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether b8:59:9f:f0:7f:d0 brd ff:ff:ff:ff:ff:ff
    inet 172.31.43.81/24 brd 172.31.43.255 scope global dynamic noprefixroute mlx5_bond_ro.43
       valid_lft 3398sec preferred_lft 3398sec
    inet6 fe80::ba59:9fff:fef0:7fd0/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
10: mlx5_bond_ro.45@mlx5_bond_roce: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether b8:59:9f:f0:7f:d0 brd ff:ff:ff:ff:ff:ff
    inet 172.31.45.81/24 brd 172.31.45.255 scope global dynamic noprefixroute mlx5_bond_ro.45
       valid_lft 3398sec preferred_lft 3398sec
    inet6 fe80::ba59:9fff:fef0:7fd0/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
+ [23-01-29 20:02:35] PY_FILE=/root/py_parse_out.txt
+ [23-01-29 20:02:35] rm -f /root/py_parse_out.txt
+ [23-01-29 20:02:35] export result=PASS
+ [23-01-29 20:02:35] result=PASS
+ [23-01-29 20:02:35] export TEST=/kernel/infiniband/env_setup/standalone
+ [23-01-29 20:02:35] TEST=/kernel/infiniband/env_setup/standalone
+ [23-01-29 20:02:35] ip address show
+ [23-01-29 20:02:35] grep 'inet 172.31.'
+ [23-01-29 20:02:35] [[ 0 -ne 0 ]]
+ [23-01-29 20:02:35] install_ca_trust
+ [23-01-29 20:02:35] local cert_path=/etc/pki/ca-trust/source/anchors
+ [23-01-29 20:02:35] '[' -f /etc/pki/ca-trust/source/anchors/legacy.crt -a -f /etc/pki/ca-trust/source/anchors/RH-IT-Root-CA.crt ']'
+ [23-01-29 20:02:35] wget --no-check-certificate -P /etc/pki/ca-trust/source/anchors https://password.corp.redhat.com/legacy.crt
--2023-01-29 20:02:35--  https://password.corp.redhat.com/legacy.crt
Resolving password.corp.redhat.com (password.corp.redhat.com)... 10.2.111.154
Connecting to password.corp.redhat.com (password.corp.redhat.com)|10.2.111.154|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1338 (1.3K)
Saving to: ‘/etc/pki/ca-trust/source/anchors/legacy.crt’

     0K .                                                     100% 65.1M=0s

2023-01-29 20:02:35 (65.1 MB/s) - ‘/etc/pki/ca-trust/source/anchors/legacy.crt’ saved [1338/1338]

+ [23-01-29 20:02:35] wget --no-check-certificate -P /etc/pki/ca-trust/source/anchors https://password.corp.redhat.com/RH-IT-Root-CA.crt
--2023-01-29 20:02:35--  https://password.corp.redhat.com/RH-IT-Root-CA.crt
Resolving password.corp.redhat.com (password.corp.redhat.com)... 10.2.111.154
Connecting to password.corp.redhat.com (password.corp.redhat.com)|10.2.111.154|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1517 (1.5K)
Saving to: ‘/etc/pki/ca-trust/source/anchors/RH-IT-Root-CA.crt.1’

     0K .                                                     100% 62.9M=0s

2023-01-29 20:02:36 (62.9 MB/s) - ‘/etc/pki/ca-trust/source/anchors/RH-IT-Root-CA.crt.1’ saved [1517/1517]

+ [23-01-29 20:02:36] update-ca-trust enable
+ [23-01-29 20:02:37] update-ca-trust extract
+ [23-01-29 20:02:38] RQA_rhel8_brew_repo
++ [23-01-29 20:02:38] RQA_get_rhel_major
++ [23-01-29 20:02:38] awk -F . '{print $1}'
+++ [23-01-29 20:02:38] RQA_get_rhel_release
+++ [23-01-29 20:02:38] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:02:38] echo 9
+ [23-01-29 20:02:38] local major=9
+ [23-01-29 20:02:38] (( major == 8 ))
+ [23-01-29 20:02:38] pek2_lab_setup
++ [23-01-29 20:02:38] RQA_get_rdma_hostname
++ [23-01-29 20:02:38] hostname -A
++ [23-01-29 20:02:38] grep -o 'rdma.*'
++ [23-01-29 20:02:38] awk '{print $1}'
++ [23-01-29 20:02:38] awk -F . '{print $1}'
+ [23-01-29 20:02:38] [[ rdma-cki-01 == *\r\d\m\a\0* ]]
+ [23-01-29 20:02:38] common_setup
++ [23-01-29 20:02:38] RQA_get_rdma_hostname
++ [23-01-29 20:02:38] hostname -A
++ [23-01-29 20:02:38] grep -o 'rdma.*'
++ [23-01-29 20:02:38] awk '{print $1}'
++ [23-01-29 20:02:38] awk -F . '{print $1}'
+ [23-01-29 20:02:38] short_hostname=rdma-cki-01
+ [23-01-29 20:02:38] RDMA_DRIVER=
+ [23-01-29 20:02:38] RDMA_NETWORK=
+ [23-01-29 20:02:38] '[' '!' -z ']'
+ [23-01-29 20:02:38] '[' '!' -z roce ']'
+ [23-01-29 20:02:38] RDMA_NETWORK=roce
+ [23-01-29 20:02:38] inv_file=json/rdma-cki-01.json
+ [23-01-29 20:02:38] '[' '!' -f json/rdma-cki-01.json ']'
+ [23-01-29 20:02:38] RQA_check_result -r 1 -t 'No inventory file for rdma-cki-01 exists'
+ [23-01-29 20:02:38] local test_pass=0
+ [23-01-29 20:02:38] local test_skip=777
+ [23-01-29 20:02:38] test 4 -gt 0
+ [23-01-29 20:02:38] case $1 in
+ [23-01-29 20:02:38] local rc=1
+ [23-01-29 20:02:38] shift
+ [23-01-29 20:02:38] shift
+ [23-01-29 20:02:38] test 2 -gt 0
+ [23-01-29 20:02:38] case $1 in
+ [23-01-29 20:02:38] local 'msg=No inventory file for rdma-cki-01 exists'
+ [23-01-29 20:02:38] shift
+ [23-01-29 20:02:38] shift
+ [23-01-29 20:02:38] test 0 -gt 0
+ [23-01-29 20:02:38] '[' -z 1 -o -z 'No inventory file for rdma-cki-01 exists' ']'
+ [23-01-29 20:02:38] '[' -z '' ']'
+ [23-01-29 20:02:38] source /etc/environment
+ [23-01-29 20:02:38] '[' -z '' ']'
+ [23-01-29 20:02:38] __RQA_create_test_result_file
++ [23-01-29 20:02:38] rpm -q --queryformat '%{NAME}-%{VERSION}-%{RELEASE}' rdma-core
+ [23-01-29 20:02:38] local rdma_version=rdma-core-41.0-3.el9
++ [23-01-29 20:02:38] echo /kernel/infiniband/env_setup/standalone
++ [23-01-29 20:02:38] sed -r 's/\/kernel\/infiniband\///;s/\/server//;s/\/client//;s/\/standalone//'
+ [23-01-29 20:02:38] TEST_NAME=env_setup
+ [23-01-29 20:02:38] [[ ! -z rdma-cki-01 ]]
+ [23-01-29 20:02:38] [[ ! -z rdma-qe-31 ]]
+ [23-01-29 20:02:38] hosts_list=rdma-cki-01/rdma-qe-31
+ [23-01-29 20:02:38] local _bkrjb=
++ [23-01-29 20:02:38] grep JOBID= /etc/motd
++ [23-01-29 20:02:38] tr -d ' '
++ [23-01-29 20:02:38] sed s/JOBID=/J:/g
++ [23-01-29 20:02:38] head -1
+ [23-01-29 20:02:38] _bkrjb=J:7478195
+ [23-01-29 20:02:38] [[ ! -z J:7478195 ]]
+ [23-01-29 20:02:38] _bkrjb=' & Beaker job J:7478195'
++ [23-01-29 20:02:38] echo results_env_setup.txt
++ [23-01-29 20:02:38] sed 's/\//-/g'
+ [23-01-29 20:02:38] local result_filename=results_env_setup.txt
++ [23-01-29 20:02:38] mktemp -d
+ [23-01-29 20:02:38] TEST_RESULT_FILE=/tmp/tmp.ylFf7D7kXe/results_env_setup.txt
+ [23-01-29 20:02:38] echo 'env_setup test results on rdma-cki-01/rdma-qe-31 & Beaker job J:7478195:'
++ [23-01-29 20:02:38] uname -r
+ [23-01-29 20:02:38] echo '5.14.0-247.rt14.248.1955_761016865.el9.x86_64, rdma-core-41.0-3.el9, , roce,  & '
+ [23-01-29 20:02:38] echo '    Result | Status | Test'
+ [23-01-29 20:02:38] echo '  ---------+--------+------------------------------------'
+ [23-01-29 20:02:38] sed -i /TEST_RESULT_FILE/d /etc/environment
+ [23-01-29 20:02:38] echo TEST_RESULT_FILE=/tmp/tmp.ylFf7D7kXe/results_env_setup.txt
+ [23-01-29 20:02:38] source /etc/environment
++ [23-01-29 20:02:38] TEST_RESULT_FILE=/tmp/tmp.ylFf7D7kXe/results_env_setup.txt
+ [23-01-29 20:02:38] '[' 1 -eq 0 ']'
+ [23-01-29 20:02:38] '[' 1 -eq 777 ']'
+ [23-01-29 20:02:38] local test_result=FAIL
+ [23-01-29 20:02:38] export result=FAIL
+ [23-01-29 20:02:38] result=FAIL
+ [23-01-29 20:02:38] [[ ! -z '' ]]
+ [23-01-29 20:02:38] printf '%10s | %6s | %s\n' FAIL 1 'No inventory file for rdma-cki-01 exists'
+ [23-01-29 20:02:38] set +x
---
- TEST RESULT FOR env_setup
-   Test:   No inventory file for rdma-cki-01 exists
-   Result: FAIL
-   Return: 1
---
+ [23-01-29 20:02:38] return 1
+ [23-01-29 20:02:38] standalone
+ [23-01-29 20:02:38] echo -en '\n# /kernel/infiniband/env_setup variables\n'
+ [23-01-29 20:02:38] [[ -s /root/py_parse_out.txt ]]
+ [23-01-29 20:02:38] echo 'export RDMA_IPV4=""'
+ [23-01-29 20:02:38] echo 'export RDMA_IPV6=""'
+ [23-01-29 20:02:38] source /root/.bashrc
+ [23-01-29 20:02:38] RQA_check_result -r 0 -t 'Source .bashrc after updates'
+ [23-01-29 20:02:38] local test_pass=0
+ [23-01-29 20:02:38] local test_skip=777
+ [23-01-29 20:02:38] test 4 -gt 0
+ [23-01-29 20:02:38] case $1 in
+ [23-01-29 20:02:38] local rc=0
+ [23-01-29 20:02:38] shift
+ [23-01-29 20:02:38] shift
+ [23-01-29 20:02:38] test 2 -gt 0
+ [23-01-29 20:02:38] case $1 in
+ [23-01-29 20:02:38] local 'msg=Source .bashrc after updates'
+ [23-01-29 20:02:38] shift
+ [23-01-29 20:02:38] shift
+ [23-01-29 20:02:38] test 0 -gt 0
+ [23-01-29 20:02:38] '[' -z 0 -o -z 'Source .bashrc after updates' ']'
+ [23-01-29 20:02:38] '[' -z /tmp/tmp.ylFf7D7kXe/results_env_setup.txt ']'
+ [23-01-29 20:02:38] '[' -z /tmp/tmp.ylFf7D7kXe/results_env_setup.txt ']'
+ [23-01-29 20:02:38] '[' 0 -eq 0 ']'
+ [23-01-29 20:02:38] local test_result=PASS
+ [23-01-29 20:02:38] printf '%10s | %6s | %s\n' PASS 0 'Source .bashrc after updates'
+ [23-01-29 20:02:38] set +x
---
- TEST RESULT FOR env_setup
-   Test:   Source .bashrc after updates
-   Result: PASS
-   Return: 0
---
+ [23-01-29 20:02:38] RQA_overall_result
+ [23-01-29 20:02:38] source /root/known-issues.sh
+ [23-01-29 20:02:43] '[' -z /tmp/tmp.ylFf7D7kXe/results_env_setup.txt ']'
+ [23-01-29 20:02:43] '[' -z /tmp/tmp.ylFf7D7kXe/results_env_setup.txt ']'
+ [23-01-29 20:02:43] local overall_result=PASS
+ [23-01-29 20:02:43] echo 'Checking for failures and known issues:'
+ [23-01-29 20:02:43] grep FAIL /tmp/tmp.ylFf7D7kXe/results_env_setup.txt
      FAIL |      1 | No inventory file for rdma-cki-01 exists
+ [23-01-29 20:02:43] [[ ! -z '' ]]
+ [23-01-29 20:02:43] local driver=
+ [23-01-29 20:02:43] local hca=
+ [23-01-29 20:02:43] local network=roce
++ [23-01-29 20:02:43] RQA_get_rhel_major
++ [23-01-29 20:02:43] awk -F . '{print $1}'
+++ [23-01-29 20:02:43] RQA_get_rhel_release
+++ [23-01-29 20:02:43] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:02:43] echo 9
+ [23-01-29 20:02:43] local major=9
++ [23-01-29 20:02:43] RQA_get_rhel_minor
++ [23-01-29 20:02:43] awk -F . '{print $2}'
+++ [23-01-29 20:02:43] RQA_get_rhel_release
+++ [23-01-29 20:02:43] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:02:43] echo 9
+ [23-01-29 20:02:43] local minor=
++ [23-01-29 20:02:43] mktemp
+ [23-01-29 20:02:43] local test_fail_file=/tmp/tmp.l2siZdo8nF
+ [23-01-29 20:02:43] grep FAIL /tmp/tmp.ylFf7D7kXe/results_env_setup.txt
+ [23-01-29 20:02:43] read line
++ [23-01-29 20:02:43] echo FAIL '|' 1 '|' No inventory file for rdma-cki-01 exists
++ [23-01-29 20:02:43] awk -F '|' '{print $3}'
++ [23-01-29 20:02:43] xargs
+ [23-01-29 20:02:43] local 'test_key=No inventory file for rdma-cki-01 exists'
+ [23-01-29 20:02:43] local value=
+ [23-01-29 20:02:43] '[' -z '' ']'
+ [23-01-29 20:02:43] overall_result=FAIL
+ [23-01-29 20:02:43] echo '  No inventory file for rdma-cki-01 exists is NOT a known issue on any environment - consider filing a BZ'
+ [23-01-29 20:02:43] continue
+ [23-01-29 20:02:43] read line
+ [23-01-29 20:02:43] [[ FAIL == \P\A\S\S ]]
+ [23-01-29 20:02:43] echo ''
+ [23-01-29 20:02:43] report_result /kernel/infiniband/env_setup/standalone FAIL 0
+ [23-01-29 20:02:43] OUTPUTFILE=/mnt/testarea/tmp.d0VpM0
+ [23-01-29 20:02:43] rstrnt-report-result /kernel/infiniband/env_setup/standalone FAIL 0
** /kernel/infiniband/env_setup/standalone FAIL Score:0
Uploading resultoutputfile.log .done
+ [23-01-29 20:02:44] rhts-submit-log -l /tmp/tmp.ylFf7D7kXe/results_env_setup.txt
Uploading results_env_setup.txt .done
+ [23-01-29 20:02:44] rm /tmp/tmp.l2siZdo8nF
+ [23-01-29 20:02:44] __RQA_unset_test_result_file
+ [23-01-29 20:02:44] cat /tmp/tmp.ylFf7D7kXe/results_env_setup.txt
env_setup test results on rdma-cki-01/rdma-qe-31 & Beaker job J:7478195:
5.14.0-247.rt14.248.1955_761016865.el9.x86_64, rdma-core-41.0-3.el9, , roce,  & 
    Result | Status | Test
  ---------+--------+------------------------------------
      FAIL |      1 | No inventory file for rdma-cki-01 exists
      PASS |      0 | Source .bashrc after updates
Checking for failures and known issues:
  No inventory file for rdma-cki-01 exists is NOT a known issue on any environment - consider filing a BZ

+ [23-01-29 20:02:44] export TEST_RESULT_FILE=
+ [23-01-29 20:02:44] TEST_RESULT_FILE=
+ [23-01-29 20:02:44] unset TEST_RESULT_FILE
+ [23-01-29 20:02:44] sed -i /TEST_RESULT_FILE/d /etc/environment
+ [23-01-29 20:02:44] echo 'TEST_RESULT_FILE=""'
+ [23-01-29 20:02:44] source /etc/environment
++ [23-01-29 20:02:44] TEST_RESULT_FILE=
+ [23-01-29 20:02:44] return 0
+ [23-01-29 20:02:44] hostname -A
+ [23-01-29 20:02:44] grep rdma-qe-31
+ [23-01-29 20:02:44] hostname -A
+ [23-01-29 20:02:44] grep rdma-cki-01
+ [23-01-29 20:02:44] echo '------- server start test -------'
------- server start test -------
+ [23-01-29 20:02:44] export result=PASS
+ [23-01-29 20:02:44] result=PASS
+ [23-01-29 20:02:44] export TEST=/kernel/infiniband/env_setup/standalone/server
+ [23-01-29 20:02:44] TEST=/kernel/infiniband/env_setup/standalone/server
+ [23-01-29 20:02:44] server
+ [23-01-29 20:02:44] [[ '' == \s\i\w ]]
+ [23-01-29 20:02:44] rhts_sync_block -s ES_CLIENT_READY_-roce-0 rdma-qe-31
/usr/bin/rhts_sync_block -s ES_CLIENT_READY_-roce-0 rdma-qe-31 -- Blocking state(s) =  12_ES_CLIENT_READY_-roce-0
Failed to connect to remote server rdma-qe-31: Connection refused
Retrying rstrnt-sync-block for state 12_ES_CLIENT_READY_-roce-0
+ [23-01-29 20:03:44] rhts_sync_set -s ES_SERVER_READY_-roce-0
/usr/bin/rhts_sync_set -s ES_SERVER_READY_-roce-0 -- State set = 12_ES_SERVER_READY_-roce-0
++ [23-01-29 20:03:44] echo rdma-qe-31
++ [23-01-29 20:03:44] awk -F . '{print $1}'
+ [23-01-29 20:03:44] cli_ssh=rdma-qe-31
++ [23-01-29 20:03:44] ssh rdma-qe-31 echo '${RDMA_DRIVER}'
+ [23-01-29 20:03:45] CLIENT_DRIVER=bnxt
++ [23-01-29 20:03:45] ssh rdma-qe-31 echo '${RDMA_NETWORK}'
+ [23-01-29 20:03:45] CLIENT_NETWORK=roce
++ [23-01-29 20:03:45] ssh rdma-qe-31 echo '${HCA_ID}'
+ [23-01-29 20:03:46] CLIENT_HCA_ID=bnxt_re0
++ [23-01-29 20:03:46] ssh rdma-qe-31 echo '${HCA_ABRV}'
+ [23-01-29 20:03:46] CLIENT_HCA_ABRV=BCM57414
++ [23-01-29 20:03:46] ssh rdma-qe-31 echo '${DEVICE_ID}'
+ [23-01-29 20:03:46] CLIENT_DEVICE_ID=bnxt_roce
++ [23-01-29 20:03:46] ssh rdma-qe-31 echo '${DEVICE_MAC}'
+ [23-01-29 20:03:47] CLIENT_DEVICE_MAC=00:0a:f7:97:b1:62
++ [23-01-29 20:03:47] ssh rdma-qe-31 echo '${DEVICE_PORT}'
+ [23-01-29 20:03:47] CLIENT_DEVICE_PORT=1
++ [23-01-29 20:03:47] ssh rdma-qe-31 echo '${RDMA_IPV4}'
+ [23-01-29 20:03:47] CLIENT_IPV4=172.31.40.31
++ [23-01-29 20:03:47] ssh rdma-qe-31 echo '${RDMA_IPV6}'
+ [23-01-29 20:03:48] CLIENT_IPV6=fe80::20a:f7ff:fe97:b162
+ [23-01-29 20:03:48] '[' -z bnxt ']'
+ [23-01-29 20:03:48] '[' -z roce ']'
+ [23-01-29 20:03:48] '[' -z bnxt_re0 ']'
+ [23-01-29 20:03:48] '[' -z BCM57414 ']'
+ [23-01-29 20:03:48] '[' -z bnxt_roce ']'
+ [23-01-29 20:03:48] '[' -z 00:0a:f7:97:b1:62 ']'
+ [23-01-29 20:03:48] '[' -z 1 ']'
+ [23-01-29 20:03:48] '[' -z 172.31.40.31 ']'
+ [23-01-29 20:03:48] '[' -z fe80::20a:f7ff:fe97:b162 ']'
+ [23-01-29 20:03:48] echo 'export CLIENT_DRIVER="bnxt"'
+ [23-01-29 20:03:48] echo 'export CLIENT_NETWORK="roce"'
+ [23-01-29 20:03:48] echo 'export CLIENT_HCA_ID="bnxt_re0"'
+ [23-01-29 20:03:48] echo 'export CLIENT_HCA_ABRV="BCM57414"'
+ [23-01-29 20:03:48] echo 'export CLIENT_DEVICE_ID="bnxt_roce"'
+ [23-01-29 20:03:48] echo 'export CLIENT_DEVICE_MAC="00:0a:f7:97:b1:62"'
+ [23-01-29 20:03:48] echo 'export CLIENT_DEVICE_PORT="1"'
+ [23-01-29 20:03:48] echo 'export CLIENT_IPV4="172.31.40.31"'
+ [23-01-29 20:03:48] echo 'export CLIENT_IPV6="fe80::20a:f7ff:fe97:b162"'
+ [23-01-29 20:03:48] echo 'export SERVER_DRIVER=""'
+ [23-01-29 20:03:48] echo 'export SERVER_NETWORK="roce"'
+ [23-01-29 20:03:48] echo 'export SERVER_HCA_ID=""'
+ [23-01-29 20:03:48] echo 'export SERVER_HCA_ABRV=""'
+ [23-01-29 20:03:48] echo 'export SERVER_DEVICE_ID=""'
+ [23-01-29 20:03:48] echo 'export SERVER_DEVICE_MAC=""'
+ [23-01-29 20:03:48] echo 'export SERVER_DEVICE_PORT=""'
+ [23-01-29 20:03:48] echo 'export SERVER_IPV4=""'
+ [23-01-29 20:03:48] echo 'export SERVER_IPV6=""'
+ [23-01-29 20:03:48] echo 'export REMOTE_HOST="rdma-qe-31"'
+ [23-01-29 20:03:48] source /root/.bashrc
+ [23-01-29 20:03:48] RQA_check_result -r 0 -t 'source .bashrc in server function'
+ [23-01-29 20:03:48] local test_pass=0
+ [23-01-29 20:03:48] local test_skip=777
+ [23-01-29 20:03:48] test 4 -gt 0
+ [23-01-29 20:03:48] case $1 in
+ [23-01-29 20:03:48] local rc=0
+ [23-01-29 20:03:48] shift
+ [23-01-29 20:03:48] shift
+ [23-01-29 20:03:48] test 2 -gt 0
+ [23-01-29 20:03:48] case $1 in
+ [23-01-29 20:03:48] local 'msg=source .bashrc in server function'
+ [23-01-29 20:03:48] shift
+ [23-01-29 20:03:48] shift
+ [23-01-29 20:03:48] test 0 -gt 0
+ [23-01-29 20:03:48] '[' -z 0 -o -z 'source .bashrc in server function' ']'
+ [23-01-29 20:03:48] '[' -z '' ']'
+ [23-01-29 20:03:48] source /etc/environment
++ [23-01-29 20:03:48] TEST_RESULT_FILE=
+ [23-01-29 20:03:48] '[' -z '' ']'
+ [23-01-29 20:03:48] __RQA_create_test_result_file
++ [23-01-29 20:03:48] rpm -q --queryformat '%{NAME}-%{VERSION}-%{RELEASE}' rdma-core
+ [23-01-29 20:03:48] local rdma_version=rdma-core-41.0-3.el9
++ [23-01-29 20:03:48] echo /kernel/infiniband/env_setup/standalone/server
++ [23-01-29 20:03:48] sed -r 's/\/kernel\/infiniband\///;s/\/server//;s/\/client//;s/\/standalone//'
+ [23-01-29 20:03:48] TEST_NAME=env_setup
+ [23-01-29 20:03:48] [[ ! -z rdma-cki-01 ]]
+ [23-01-29 20:03:48] [[ ! -z rdma-qe-31 ]]
+ [23-01-29 20:03:48] hosts_list=rdma-cki-01/rdma-qe-31
+ [23-01-29 20:03:48] local _bkrjb=
++ [23-01-29 20:03:48] grep JOBID= /etc/motd
++ [23-01-29 20:03:48] sed s/JOBID=/J:/g
++ [23-01-29 20:03:48] tr -d ' '
++ [23-01-29 20:03:48] head -1
+ [23-01-29 20:03:48] _bkrjb=J:7478195
+ [23-01-29 20:03:48] [[ ! -z J:7478195 ]]
+ [23-01-29 20:03:48] _bkrjb=' & Beaker job J:7478195'
++ [23-01-29 20:03:48] echo results_env_setup.txt
++ [23-01-29 20:03:48] sed 's/\//-/g'
+ [23-01-29 20:03:48] local result_filename=results_env_setup.txt
++ [23-01-29 20:03:48] mktemp -d
+ [23-01-29 20:03:48] TEST_RESULT_FILE=/tmp/tmp.QfEscF2w1e/results_env_setup.txt
+ [23-01-29 20:03:48] echo 'env_setup test results on rdma-cki-01/rdma-qe-31 & Beaker job J:7478195:'
++ [23-01-29 20:03:48] uname -r
+ [23-01-29 20:03:48] echo '5.14.0-247.rt14.248.1955_761016865.el9.x86_64, rdma-core-41.0-3.el9, , roce,  & '
+ [23-01-29 20:03:48] echo '    Result | Status | Test'
+ [23-01-29 20:03:48] echo '  ---------+--------+------------------------------------'
+ [23-01-29 20:03:48] sed -i /TEST_RESULT_FILE/d /etc/environment
+ [23-01-29 20:03:48] echo TEST_RESULT_FILE=/tmp/tmp.QfEscF2w1e/results_env_setup.txt
+ [23-01-29 20:03:48] source /etc/environment
++ [23-01-29 20:03:48] TEST_RESULT_FILE=/tmp/tmp.QfEscF2w1e/results_env_setup.txt
+ [23-01-29 20:03:48] '[' 0 -eq 0 ']'
+ [23-01-29 20:03:48] local test_result=PASS
+ [23-01-29 20:03:48] printf '%10s | %6s | %s\n' PASS 0 'source .bashrc in server function'
+ [23-01-29 20:03:48] set +x
---
- TEST RESULT FOR env_setup
-   Test:   source .bashrc in server function
-   Result: PASS
-   Return: 0
---
+ [23-01-29 20:03:48] rhts_sync_set -s ES_SERVER_DONE_-roce-0
/usr/bin/rhts_sync_set -s ES_SERVER_DONE_-roce-0 -- State set = 12_ES_SERVER_DONE_-roce-0
+ [23-01-29 20:03:48] echo '--- server finishes.'
--- server finishes.
+ [23-01-29 20:03:48] rhts_sync_block -s ES_CLIENT_DONE_-roce-0 rdma-qe-31
/usr/bin/rhts_sync_block -s ES_CLIENT_DONE_-roce-0 rdma-qe-31 -- Blocking state(s) =  12_ES_CLIENT_DONE_-roce-0
Server rdma-qe-31 not reported state 12_ES_CLIENT_DONE_-roce-0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_ES_CLIENT_DONE_-roce-0
+ [23-01-29 20:04:58] RQA_overall_result
+ [23-01-29 20:04:58] source /root/known-issues.sh
+ [23-01-29 20:05:03] '[' -z /tmp/tmp.QfEscF2w1e/results_env_setup.txt ']'
+ [23-01-29 20:05:03] '[' -z /tmp/tmp.QfEscF2w1e/results_env_setup.txt ']'
+ [23-01-29 20:05:03] local overall_result=PASS
+ [23-01-29 20:05:03] echo 'Checking for failures and known issues:'
+ [23-01-29 20:05:03] grep FAIL /tmp/tmp.QfEscF2w1e/results_env_setup.txt
+ [23-01-29 20:05:03] echo '  no test failures'
+ [23-01-29 20:05:03] echo ''
+ [23-01-29 20:05:03] report_result /kernel/infiniband/env_setup/standalone/server PASS 0
+ [23-01-29 20:05:03] OUTPUTFILE=/mnt/testarea/tmp.d0VpM0
+ [23-01-29 20:05:03] rstrnt-report-result /kernel/infiniband/env_setup/standalone/server PASS 0
** /kernel/infiniband/env_setup/standalone/server PASS Score:0
Uploading resultoutputfile.log .done
+ [23-01-29 20:05:04] rhts-submit-log -l /tmp/tmp.QfEscF2w1e/results_env_setup.txt
Uploading results_env_setup.txt .done
+ [23-01-29 20:05:04] __RQA_unset_test_result_file
+ [23-01-29 20:05:04] cat /tmp/tmp.QfEscF2w1e/results_env_setup.txt
env_setup test results on rdma-cki-01/rdma-qe-31 & Beaker job J:7478195:
5.14.0-247.rt14.248.1955_761016865.el9.x86_64, rdma-core-41.0-3.el9, , roce,  & 
    Result | Status | Test
  ---------+--------+------------------------------------
      PASS |      0 | source .bashrc in server function
Checking for failures and known issues:
  no test failures

+ [23-01-29 20:05:04] export TEST_RESULT_FILE=
+ [23-01-29 20:05:04] TEST_RESULT_FILE=
+ [23-01-29 20:05:04] unset TEST_RESULT_FILE
+ [23-01-29 20:05:04] sed -i /TEST_RESULT_FILE/d /etc/environment
+ [23-01-29 20:05:04] echo 'TEST_RESULT_FILE=""'
+ [23-01-29 20:05:04] source /etc/environment
++ [23-01-29 20:05:04] TEST_RESULT_FILE=
+ [23-01-29 20:05:04] return 0
+ [23-01-29 20:05:04] [[ '' == \i\4\0\i\w ]]
+ [23-01-29 20:05:04] rm -f
+ [23-01-29 20:05:04] unset RUN_NUMBER TNAME
+ [23-01-29 20:05:04] echo ' ------ end of runtest.sh.'
 ------ end of runtest.sh.
+ [23-01-29 20:05:04] exit 0
make[1]: Leaving directory '/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.zip/infiniband/env_setup'
+ [23-01-29 20:05:04] echo 'OKAY - task env_setup is done!'
OKAY - task env_setup is done!
+ [23-01-29 20:05:04] bash -x ./tier1.sh common
+ trap 'exit 1' SIGHUP SIGINT SIGQUIT SIGTERM
+ source /root/rdma-qa.sh
++ PS4='$(date +"+ [%y-%m-%d %H:%M:%S]") '
++ [23-01-29 20:05:04] export NIC_PREF=
++ [23-01-29 20:05:04] NIC_PREF=
++ [23-01-29 20:05:04] export PERFTEST_FLAGS=
++ [23-01-29 20:05:04] PERFTEST_FLAGS=
+++ [23-01-29 20:05:04] hostname -s
++ [23-01-29 20:05:04] export s_hostname=rdma-cki-01
++ [23-01-29 20:05:04] s_hostname=rdma-cki-01
++ [23-01-29 20:05:04] RQA_set_pyexec
++ [23-01-29 20:05:04] PYEXEC=
++ [23-01-29 20:05:04] which python
++ [23-01-29 20:05:04] export PYEXEC=python
++ [23-01-29 20:05:04] PYEXEC=python
++ [23-01-29 20:05:04] return 0
+++ [23-01-29 20:05:04] grep -i fedora /etc/redhat-release
++ [23-01-29 20:05:04] [[ -n '' ]]
+++ [23-01-29 20:05:04] RQA_get_rhel_major
+++ [23-01-29 20:05:04] awk -F . '{print $1}'
++++ [23-01-29 20:05:04] RQA_get_rhel_release
++++ [23-01-29 20:05:04] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:04] echo 9
++ [23-01-29 20:05:04] [[ 9 -ge 8 ]]
++ [23-01-29 20:05:04] export 'PKGINSTALL=dnf install -y --setopt=strict=0 --nogpgcheck'
++ [23-01-29 20:05:04] PKGINSTALL='dnf install -y --setopt=strict=0 --nogpgcheck'
++ [23-01-29 20:05:04] export 'PKGREMOVE=dnf remove --noautoremove -y'
++ [23-01-29 20:05:04] PKGREMOVE='dnf remove --noautoremove -y'
++ [23-01-29 20:05:04] [[ rdma-cki-01 == \r\d\m\a\-\d\e\v\-\2\0 ]]
++ [23-01-29 20:05:04] export LATEST_MFT=http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz
++ [23-01-29 20:05:04] LATEST_MFT=http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz
++ [23-01-29 20:05:04] '[' -f /root/known-issues.sh ']'
++ [23-01-29 20:05:04] source /root/known-issues.sh
++ [23-01-29 20:05:08] RQA_load_tech_preview_modules
++ [23-01-29 20:05:08] [[ ! -e /etc/rdma/rdma.conf ]]
++ [23-01-29 20:05:08] return
++ [23-01-29 20:05:08] RQA_fix_uefi_reboot
++ [23-01-29 20:05:08] ls -l /usr/sbin/reboot
++ [23-01-29 20:05:08] grep rhts-reboot
++ [23-01-29 20:05:08] which rhts-reboot
++ [23-01-29 20:05:08] rpm -q rhts-test-env
++ [23-01-29 20:05:08] RQA_fix_ssh_ipv6_auth
+++ [23-01-29 20:05:08] RQA_get_rhel_major
+++ [23-01-29 20:05:08] awk -F . '{print $1}'
++++ [23-01-29 20:05:08] RQA_get_rhel_release
++++ [23-01-29 20:05:08] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:08] echo 9
++ [23-01-29 20:05:08] '[' 9 -ge 7 ']'
++ [23-01-29 20:05:08] local need_restart=0
++ [23-01-29 20:05:08] grep -E '^UseDNS yes' /etc/ssh/sshd_config
++ [23-01-29 20:05:08] grep -E '^GSSAPIStrictAcceptorCheck yes' /etc/ssh/sshd_config
++ [23-01-29 20:05:08] grep 'AddressFamily inet' /root/.ssh/config
++ [23-01-29 20:05:08] '[' 0 -eq 1 ']'
++ [23-01-29 20:05:08] RQA_fix_mlx5_ib_connected_mode
+++ [23-01-29 20:05:08] RQA_get_rhel_major
+++ [23-01-29 20:05:08] awk -F . '{print $1}'
++++ [23-01-29 20:05:08] RQA_get_rhel_release
++++ [23-01-29 20:05:08] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:08] echo 9
++ [23-01-29 20:05:08] [[ 9 -ge 7 ]]
+++ [23-01-29 20:05:08] RQA_get_rhel_minor
+++ [23-01-29 20:05:08] awk -F . '{print $2}'
++++ [23-01-29 20:05:08] RQA_get_rhel_release
++++ [23-01-29 20:05:08] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:08] echo 9
++ [23-01-29 20:05:08] [[ '' -ge 5 ]]
+++ [23-01-29 20:05:08] RQA_get_rhel_major
+++ [23-01-29 20:05:08] awk -F . '{print $1}'
++++ [23-01-29 20:05:08] RQA_get_rhel_release
++++ [23-01-29 20:05:08] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:08] echo 9
++ [23-01-29 20:05:08] [[ 9 -gt 7 ]]
+++ [23-01-29 20:05:08] ls /etc/sysconfig/network-scripts/
+++ [23-01-29 20:05:08] grep mlx5
+++ [23-01-29 20:05:08] grep ib
+++ [23-01-29 20:05:08] grep -v '~'
++ [23-01-29 20:05:09] RQA_nfs_rhel8_temp_fix
+++ [23-01-29 20:05:09] RQA_get_rhel_major
+++ [23-01-29 20:05:09] awk -F . '{print $1}'
++++ [23-01-29 20:05:09] RQA_get_rhel_release
++++ [23-01-29 20:05:09] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:09] echo 9
++ [23-01-29 20:05:09] ((  9 == 8  ))
++ [23-01-29 20:05:09] source /root/.bashrc
++ [23-01-29 20:05:09] [[ '' == \i\4\0\i\w ]]
+ [23-01-29 20:05:09] source /root/mpi-functions.sh
++ [23-01-29 20:05:09] export 'MPI_TIMEOUT=timeout --preserve-status --kill-after=5m 3m'
++ [23-01-29 20:05:09] MPI_TIMEOUT='timeout --preserve-status --kill-after=5m 3m'
+ [23-01-29 20:05:09] source /root/rdma-qa.sh
++ [23-01-29 20:05:09] PS4='$(date +"+ [%y-%m-%d %H:%M:%S]") '
++ [23-01-29 20:05:09] export NIC_PREF=
++ [23-01-29 20:05:09] NIC_PREF=
++ [23-01-29 20:05:09] export PERFTEST_FLAGS=
++ [23-01-29 20:05:09] PERFTEST_FLAGS=
+++ [23-01-29 20:05:09] hostname -s
++ [23-01-29 20:05:09] export s_hostname=rdma-cki-01
++ [23-01-29 20:05:09] s_hostname=rdma-cki-01
++ [23-01-29 20:05:09] RQA_set_pyexec
++ [23-01-29 20:05:09] PYEXEC=
++ [23-01-29 20:05:09] which python
++ [23-01-29 20:05:09] export PYEXEC=python
++ [23-01-29 20:05:09] PYEXEC=python
++ [23-01-29 20:05:09] return 0
+++ [23-01-29 20:05:09] grep -i fedora /etc/redhat-release
++ [23-01-29 20:05:09] [[ -n '' ]]
+++ [23-01-29 20:05:09] RQA_get_rhel_major
+++ [23-01-29 20:05:09] awk -F . '{print $1}'
++++ [23-01-29 20:05:09] RQA_get_rhel_release
++++ [23-01-29 20:05:09] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:09] echo 9
++ [23-01-29 20:05:09] [[ 9 -ge 8 ]]
++ [23-01-29 20:05:09] export 'PKGINSTALL=dnf install -y --setopt=strict=0 --nogpgcheck'
++ [23-01-29 20:05:09] PKGINSTALL='dnf install -y --setopt=strict=0 --nogpgcheck'
++ [23-01-29 20:05:09] export 'PKGREMOVE=dnf remove --noautoremove -y'
++ [23-01-29 20:05:09] PKGREMOVE='dnf remove --noautoremove -y'
++ [23-01-29 20:05:09] [[ rdma-cki-01 == \r\d\m\a\-\d\e\v\-\2\0 ]]
++ [23-01-29 20:05:09] export LATEST_MFT=http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz
++ [23-01-29 20:05:09] LATEST_MFT=http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz
++ [23-01-29 20:05:09] '[' -f /root/known-issues.sh ']'
++ [23-01-29 20:05:09] source /root/known-issues.sh
++ [23-01-29 20:05:15] RQA_load_tech_preview_modules
++ [23-01-29 20:05:15] [[ ! -e /etc/rdma/rdma.conf ]]
++ [23-01-29 20:05:15] return
++ [23-01-29 20:05:15] RQA_fix_uefi_reboot
++ [23-01-29 20:05:15] ls -l /usr/sbin/reboot
++ [23-01-29 20:05:15] grep rhts-reboot
++ [23-01-29 20:05:15] which rhts-reboot
++ [23-01-29 20:05:15] rpm -q rhts-test-env
++ [23-01-29 20:05:15] RQA_fix_ssh_ipv6_auth
+++ [23-01-29 20:05:15] RQA_get_rhel_major
+++ [23-01-29 20:05:15] awk -F . '{print $1}'
++++ [23-01-29 20:05:15] RQA_get_rhel_release
++++ [23-01-29 20:05:15] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:15] echo 9
++ [23-01-29 20:05:15] '[' 9 -ge 7 ']'
++ [23-01-29 20:05:15] local need_restart=0
++ [23-01-29 20:05:15] grep -E '^UseDNS yes' /etc/ssh/sshd_config
++ [23-01-29 20:05:15] grep -E '^GSSAPIStrictAcceptorCheck yes' /etc/ssh/sshd_config
++ [23-01-29 20:05:15] grep 'AddressFamily inet' /root/.ssh/config
++ [23-01-29 20:05:15] '[' 0 -eq 1 ']'
++ [23-01-29 20:05:15] RQA_fix_mlx5_ib_connected_mode
+++ [23-01-29 20:05:15] RQA_get_rhel_major
+++ [23-01-29 20:05:15] awk -F . '{print $1}'
++++ [23-01-29 20:05:15] RQA_get_rhel_release
++++ [23-01-29 20:05:15] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:15] echo 9
++ [23-01-29 20:05:15] [[ 9 -ge 7 ]]
+++ [23-01-29 20:05:15] RQA_get_rhel_minor
+++ [23-01-29 20:05:15] awk -F . '{print $2}'
++++ [23-01-29 20:05:15] RQA_get_rhel_release
++++ [23-01-29 20:05:15] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:15] echo 9
++ [23-01-29 20:05:15] [[ '' -ge 5 ]]
+++ [23-01-29 20:05:15] RQA_get_rhel_major
+++ [23-01-29 20:05:15] awk -F . '{print $1}'
++++ [23-01-29 20:05:15] RQA_get_rhel_release
++++ [23-01-29 20:05:15] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:15] echo 9
++ [23-01-29 20:05:15] [[ 9 -gt 7 ]]
+++ [23-01-29 20:05:15] ls /etc/sysconfig/network-scripts/
+++ [23-01-29 20:05:15] grep mlx5
+++ [23-01-29 20:05:15] grep ib
+++ [23-01-29 20:05:15] grep -v '~'
++ [23-01-29 20:05:15] RQA_nfs_rhel8_temp_fix
+++ [23-01-29 20:05:15] RQA_get_rhel_major
+++ [23-01-29 20:05:15] awk -F . '{print $1}'
++++ [23-01-29 20:05:15] RQA_get_rhel_release
++++ [23-01-29 20:05:15] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:15] echo 9
++ [23-01-29 20:05:15] ((  9 == 8  ))
++ [23-01-29 20:05:15] source /root/.bashrc
++ [23-01-29 20:05:15] [[ '' == \i\4\0\i\w ]]
+ [23-01-29 20:05:15] source /root/mpi-functions.sh
++ [23-01-29 20:05:15] export 'MPI_TIMEOUT=timeout --preserve-status --kill-after=5m 3m'
++ [23-01-29 20:05:15] MPI_TIMEOUT='timeout --preserve-status --kill-after=5m 3m'
+ [23-01-29 20:05:15] RQA_set_inline_default
+ [23-01-29 20:05:15] [[ '' != *\m\l\x\4* ]]
+ [23-01-29 20:05:15] mkdir -p /etc/rdma/rsocket
+ [23-01-29 20:05:15] echo 0
+ [23-01-29 20:05:15] RQA_set_perftest_flags
+ [23-01-29 20:05:15] PERFTEST_FLAGS='-d  -i  -F'
+ [23-01-29 20:05:15] [[ '' == \m\l\x\4 ]]
+ [23-01-29 20:05:15] PERFTEST_FLAGS='-d  -i  -F -R'
+ [23-01-29 20:05:15] export PERFTEST_FLAGS
+ [23-01-29 20:05:15] MPI_PKG=openmpi
+ [23-01-29 20:05:15] TEST_PKG=mpitests-openmpi
++ [23-01-29 20:05:15] RQA_get_rhel_major
++ [23-01-29 20:05:15] awk -F . '{print $1}'
+++ [23-01-29 20:05:15] RQA_get_rhel_release
+++ [23-01-29 20:05:15] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:05:15] echo 9
+ [23-01-29 20:05:15] [[ 9 -eq 6 ]]
++ [23-01-29 20:05:15] RQA_get_rhel_major
++ [23-01-29 20:05:15] awk -F . '{print $1}'
+++ [23-01-29 20:05:15] RQA_get_rhel_release
+++ [23-01-29 20:05:15] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:05:15] echo 9
+ [23-01-29 20:05:15] [[ 9 -eq 7 ]]
+ [23-01-29 20:05:15] MPI_setup -m openmpi -t mpitests-openmpi
+ [23-01-29 20:05:15] test 4 -gt 0
+ [23-01-29 20:05:15] case $1 in
+ [23-01-29 20:05:15] local mpi_pkg=openmpi
+ [23-01-29 20:05:15] shift
+ [23-01-29 20:05:15] shift
+ [23-01-29 20:05:15] test 2 -gt 0
+ [23-01-29 20:05:15] case $1 in
+ [23-01-29 20:05:15] local test_pkg=mpitests-openmpi
+ [23-01-29 20:05:15] shift
+ [23-01-29 20:05:15] shift
+ [23-01-29 20:05:15] test 0 -gt 0
+ [23-01-29 20:05:15] '[' -z openmpi -o -z mpitests-openmpi ']'
+ [23-01-29 20:05:15] rpm -q environment-modules
environment-modules-5.0.1-2.el9.x86_64
+ [23-01-29 20:05:15] '[' -f /etc/profile.d/modules.sh ']'
+ [23-01-29 20:05:15] source /etc/profile.d/modules.sh
++ [23-01-29 20:05:15] '[' -n /usr/bin/bash ']'
++ [23-01-29 20:05:15] '[' bash = sh ']'
++ [23-01-29 20:05:15] shell=bash
++ [23-01-29 20:05:15] '[' -f /usr/share/Modules/init/bash ']'
++ [23-01-29 20:05:15] . /usr/share/Modules/init/bash
+++ [23-01-29 20:05:15] unset _mlshdbg
+++ [23-01-29 20:05:15] '[' 0 = 1 ']'
+++ [23-01-29 20:05:15] export 'MODULES_RUN_QUARANTINE=LD_LIBRARY_PATH LD_PRELOAD'
+++ [23-01-29 20:05:15] MODULES_RUN_QUARANTINE='LD_LIBRARY_PATH LD_PRELOAD'
+++ [23-01-29 20:05:15] unset _mlre _mlIFS
+++ [23-01-29 20:05:15] '[' -n x ']'
+++ [23-01-29 20:05:15] _mlIFS=' 	
'
+++ [23-01-29 20:05:15] IFS=' '
+++ [23-01-29 20:05:15] for _mlv in ${MODULES_RUN_QUARANTINE:-}
+++ [23-01-29 20:05:15] '[' LD_LIBRARY_PATH = LD_LIBRARY_PATH -a LD_LIBRARY_PATH = LD_LIBRARY_PATH ']'
++++ [23-01-29 20:05:15] eval 'echo ${LD_LIBRARY_PATH+x}'
+++++ [23-01-29 20:05:15] echo
+++ [23-01-29 20:05:15] '[' -n '' ']'
+++ [23-01-29 20:05:15] _mlrv=MODULES_RUNENV_LD_LIBRARY_PATH
++++ [23-01-29 20:05:15] eval 'echo ${MODULES_RUNENV_LD_LIBRARY_PATH:-}'
+++++ [23-01-29 20:05:15] echo
+++ [23-01-29 20:05:15] _mlre='LD_LIBRARY_PATH='\'''\'' '
+++ [23-01-29 20:05:15] for _mlv in ${MODULES_RUN_QUARANTINE:-}
+++ [23-01-29 20:05:15] '[' LD_PRELOAD = LD_PRELOAD -a LD_PRELOAD = LD_PRELOAD ']'
++++ [23-01-29 20:05:15] eval 'echo ${LD_PRELOAD+x}'
+++++ [23-01-29 20:05:15] echo
+++ [23-01-29 20:05:15] '[' -n '' ']'
+++ [23-01-29 20:05:15] _mlrv=MODULES_RUNENV_LD_PRELOAD
++++ [23-01-29 20:05:15] eval 'echo ${MODULES_RUNENV_LD_PRELOAD:-}'
+++++ [23-01-29 20:05:15] echo
+++ [23-01-29 20:05:15] _mlre='LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' '
+++ [23-01-29 20:05:15] '[' -n 'LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' ' ']'
+++ [23-01-29 20:05:15] _mlre='eval LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' __MODULES_QUARANTINE_SET=1 '
++++ [23-01-29 20:05:15] eval 'LD_LIBRARY_PATH='\'''\''' 'LD_PRELOAD='\'''\''' __MODULES_QUARANTINE_SET=1 /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash autoinit
+++++ [23-01-29 20:05:15] LD_LIBRARY_PATH=
+++++ [23-01-29 20:05:15] LD_PRELOAD=
+++++ [23-01-29 20:05:15] __MODULES_QUARANTINE_SET=1
+++++ [23-01-29 20:05:15] /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash autoinit
+++ [23-01-29 20:05:15] _mlcode='module() {
   eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
   _mlstatus=$?;
   return $_mlstatus;
};
ml() { module ml "$@"; };
MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl; export MODULES_CMD;
unset __MODULES_AUTOINIT_INPROGRESS;
MODULESHOME=/usr/share/Modules; export MODULESHOME;
test 0;'
+++ [23-01-29 20:05:15] _mlret=0
+++ [23-01-29 20:05:15] '[' -n x ']'
+++ [23-01-29 20:05:15] IFS=' 	
'
+++ [23-01-29 20:05:15] unset _mlIFS
+++ [23-01-29 20:05:15] unset _mlre _mlv _mlrv
+++ [23-01-29 20:05:15] '[' 0 -eq 0 ']'
+++ [23-01-29 20:05:15] eval 'module() {
   eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
   _mlstatus=$?;
   return $_mlstatus;
};
ml() { module ml "$@"; };
MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl; export MODULES_CMD;
unset __MODULES_AUTOINIT_INPROGRESS;
MODULESHOME=/usr/share/Modules; export MODULESHOME;
test 0;'
++++ [23-01-29 20:05:15] MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl
++++ [23-01-29 20:05:15] export MODULES_CMD
++++ [23-01-29 20:05:15] unset __MODULES_AUTOINIT_INPROGRESS
++++ [23-01-29 20:05:15] MODULESHOME=/usr/share/Modules
++++ [23-01-29 20:05:15] export MODULESHOME
++++ [23-01-29 20:05:15] test 0
+++ [23-01-29 20:05:15] '[' -t 2 ']'
+++ [23-01-29 20:05:15] export -f module
++++ [23-01-29 20:05:15] type -t ml
+++ [23-01-29 20:05:15] '[' function = function ']'
+++ [23-01-29 20:05:15] export -f ml
+++ [23-01-29 20:05:15] '[' 5 -ge 3 ']'
+++ [23-01-29 20:05:15] [[ hxB =~ i ]]
+++ [23-01-29 20:05:15] [[ ! :/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin: =~ :/usr/share/Modules/bin: ]]
+++ [23-01-29 20:05:15] '[' '!' -n x ']'
+++ [23-01-29 20:05:15] [[ ! :/usr/share/man:: =~ :/usr/share/man: ]]
+++ [23-01-29 20:05:15] unset _mlcode _mlret
+++ [23-01-29 20:05:15] '[' -n '' ']'
+ [23-01-29 20:05:15] type module
module is a function
module () 
{ 
    eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
    _mlstatus=$?;
    return $_mlstatus
}
+ [23-01-29 20:05:15] module purge
++ [23-01-29 20:05:15] /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash purge
+ [23-01-29 20:05:15] eval
+ [23-01-29 20:05:15] _mlstatus=0
+ [23-01-29 20:05:15] return 0
+ [23-01-29 20:05:15] sed -e '/^module/d' -i /root/.bashrc
+ [23-01-29 20:05:15] dnf remove --noautoremove -y 'mpitests*' 'mpich*' 'mvapich*' 'openmpi*'
No match for argument: mpitests*
No match for argument: mpich*
Dependencies resolved.
================================================================================
 Package            Arch       Version              Repository             Size
================================================================================
Removing:
 mvapich2           x86_64     2.3.6-3.el9          @beaker-AppStream      15 M
 mvapich2-devel     x86_64     2.3.6-3.el9          @beaker-AppStream      28 M
 openmpi            x86_64     1:4.1.1-5.el9        @beaker-AppStream      10 M
 openmpi-devel      x86_64     1:4.1.1-5.el9        @beaker-AppStream     1.9 M

Transaction Summary
================================================================================
Remove  4 Packages

Freed space: 55 M
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Erasing          : mvapich2-devel-2.3.6-3.el9.x86_64                      1/4 
  Erasing          : openmpi-1:4.1.1-5.el9.x86_64                           2/4 
  Erasing          : openmpi-devel-1:4.1.1-5.el9.x86_64                     3/4 
  Erasing          : mvapich2-2.3.6-3.el9.x86_64                            4/4 
  Running scriptlet: mvapich2-2.3.6-3.el9.x86_64                            4/4 
  Verifying        : mvapich2-2.3.6-3.el9.x86_64                            1/4 
  Verifying        : mvapich2-devel-2.3.6-3.el9.x86_64                      2/4 
  Verifying        : openmpi-1:4.1.1-5.el9.x86_64                           3/4 
  Verifying        : openmpi-devel-1:4.1.1-5.el9.x86_64                     4/4 

Removed:
  mvapich2-2.3.6-3.el9.x86_64         mvapich2-devel-2.3.6-3.el9.x86_64        
  openmpi-1:4.1.1-5.el9.x86_64        openmpi-devel-1:4.1.1-5.el9.x86_64       

Complete!
+ [23-01-29 20:05:20] dnf install -y --setopt=strict=0 --nogpgcheck openmpi mpitests-openmpi environment-modules
Last metadata expiration check: 0:13:05 ago on Sun 29 Jan 2023 07:52:16 PM EST.
Package environment-modules-5.0.1-2.el9.x86_64 is already installed.
Dependencies resolved.
================================================================================
 Package               Arch        Version             Repository          Size
================================================================================
Installing:
 mpitests-openmpi      x86_64      5.8-1.el9           BUILDROOT-C9S      897 k
 openmpi               x86_64      1:4.1.1-5.el9       BUILDROOT-C9S      2.7 M
Installing dependencies:
 openmpi-devel         x86_64      1:4.1.1-5.el9       BUILDROOT-C9S      1.2 M

Transaction Summary
================================================================================
Install  3 Packages

Total download size: 4.7 M
Installed size: 17 M
Downloading Packages:
(1/3): openmpi-devel-4.1.1-5.el9.x86_64.rpm      11 MB/s | 1.2 MB     00:00    
(2/3): mpitests-openmpi-5.8-1.el9.x86_64.rpm    5.3 MB/s | 897 kB     00:00    
(3/3): openmpi-4.1.1-5.el9.x86_64.rpm           8.1 MB/s | 2.7 MB     00:00    
--------------------------------------------------------------------------------
Total                                            14 MB/s | 4.7 MB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Installing       : openmpi-devel-1:4.1.1-5.el9.x86_64                     1/3 
  Installing       : openmpi-1:4.1.1-5.el9.x86_64                           2/3 
  Installing       : mpitests-openmpi-5.8-1.el9.x86_64                      3/3 
  Running scriptlet: mpitests-openmpi-5.8-1.el9.x86_64                      3/3 
  Verifying        : mpitests-openmpi-5.8-1.el9.x86_64                      1/3 
  Verifying        : openmpi-1:4.1.1-5.el9.x86_64                           2/3 
  Verifying        : openmpi-devel-1:4.1.1-5.el9.x86_64                     3/3 

Installed:
  mpitests-openmpi-5.8-1.el9.x86_64         openmpi-1:4.1.1-5.el9.x86_64       
  openmpi-devel-1:4.1.1-5.el9.x86_64       

Complete!
+ [23-01-29 20:05:27] '[' -f /etc/profile.d/modules.sh ']'
+ [23-01-29 20:05:27] source /etc/profile.d/modules.sh
++ [23-01-29 20:05:27] '[' -n /usr/bin/bash ']'
++ [23-01-29 20:05:27] '[' bash = sh ']'
++ [23-01-29 20:05:27] shell=bash
++ [23-01-29 20:05:27] '[' -f /usr/share/Modules/init/bash ']'
++ [23-01-29 20:05:27] . /usr/share/Modules/init/bash
+++ [23-01-29 20:05:27] unset _mlshdbg
+++ [23-01-29 20:05:27] '[' 0 = 1 ']'
+++ [23-01-29 20:05:27] export 'MODULES_RUN_QUARANTINE=LD_LIBRARY_PATH LD_PRELOAD'
+++ [23-01-29 20:05:27] MODULES_RUN_QUARANTINE='LD_LIBRARY_PATH LD_PRELOAD'
+++ [23-01-29 20:05:27] unset _mlre _mlIFS
+++ [23-01-29 20:05:27] '[' -n x ']'
+++ [23-01-29 20:05:27] _mlIFS=' 	
'
+++ [23-01-29 20:05:27] IFS=' '
+++ [23-01-29 20:05:27] for _mlv in ${MODULES_RUN_QUARANTINE:-}
+++ [23-01-29 20:05:27] '[' LD_LIBRARY_PATH = LD_LIBRARY_PATH -a LD_LIBRARY_PATH = LD_LIBRARY_PATH ']'
++++ [23-01-29 20:05:27] eval 'echo ${LD_LIBRARY_PATH+x}'
+++++ [23-01-29 20:05:27] echo
+++ [23-01-29 20:05:27] '[' -n '' ']'
+++ [23-01-29 20:05:27] _mlrv=MODULES_RUNENV_LD_LIBRARY_PATH
++++ [23-01-29 20:05:27] eval 'echo ${MODULES_RUNENV_LD_LIBRARY_PATH:-}'
+++++ [23-01-29 20:05:27] echo
+++ [23-01-29 20:05:27] _mlre='LD_LIBRARY_PATH='\'''\'' '
+++ [23-01-29 20:05:27] for _mlv in ${MODULES_RUN_QUARANTINE:-}
+++ [23-01-29 20:05:27] '[' LD_PRELOAD = LD_PRELOAD -a LD_PRELOAD = LD_PRELOAD ']'
++++ [23-01-29 20:05:27] eval 'echo ${LD_PRELOAD+x}'
+++++ [23-01-29 20:05:27] echo
+++ [23-01-29 20:05:27] '[' -n '' ']'
+++ [23-01-29 20:05:27] _mlrv=MODULES_RUNENV_LD_PRELOAD
++++ [23-01-29 20:05:27] eval 'echo ${MODULES_RUNENV_LD_PRELOAD:-}'
+++++ [23-01-29 20:05:27] echo
+++ [23-01-29 20:05:27] _mlre='LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' '
+++ [23-01-29 20:05:27] '[' -n 'LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' ' ']'
+++ [23-01-29 20:05:27] _mlre='eval LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' __MODULES_QUARANTINE_SET=1 '
++++ [23-01-29 20:05:27] eval 'LD_LIBRARY_PATH='\'''\''' 'LD_PRELOAD='\'''\''' __MODULES_QUARANTINE_SET=1 /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash autoinit
+++++ [23-01-29 20:05:27] LD_LIBRARY_PATH=
+++++ [23-01-29 20:05:27] LD_PRELOAD=
+++++ [23-01-29 20:05:27] __MODULES_QUARANTINE_SET=1
+++++ [23-01-29 20:05:27] /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash autoinit
+++ [23-01-29 20:05:27] _mlcode='module() {
   eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
   _mlstatus=$?;
   return $_mlstatus;
};
ml() { module ml "$@"; };
MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl; export MODULES_CMD;
unset __MODULES_AUTOINIT_INPROGRESS;
MODULESHOME=/usr/share/Modules; export MODULESHOME;
test 0;'
+++ [23-01-29 20:05:27] _mlret=0
+++ [23-01-29 20:05:27] '[' -n x ']'
+++ [23-01-29 20:05:27] IFS=' 	
'
+++ [23-01-29 20:05:27] unset _mlIFS
+++ [23-01-29 20:05:27] unset _mlre _mlv _mlrv
+++ [23-01-29 20:05:27] '[' 0 -eq 0 ']'
+++ [23-01-29 20:05:27] eval 'module() {
   eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
   _mlstatus=$?;
   return $_mlstatus;
};
ml() { module ml "$@"; };
MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl; export MODULES_CMD;
unset __MODULES_AUTOINIT_INPROGRESS;
MODULESHOME=/usr/share/Modules; export MODULESHOME;
test 0;'
++++ [23-01-29 20:05:27] MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl
++++ [23-01-29 20:05:27] export MODULES_CMD
++++ [23-01-29 20:05:27] unset __MODULES_AUTOINIT_INPROGRESS
++++ [23-01-29 20:05:27] MODULESHOME=/usr/share/Modules
++++ [23-01-29 20:05:27] export MODULESHOME
++++ [23-01-29 20:05:27] test 0
+++ [23-01-29 20:05:27] '[' -t 2 ']'
+++ [23-01-29 20:05:27] export -f module
++++ [23-01-29 20:05:27] type -t ml
+++ [23-01-29 20:05:27] '[' function = function ']'
+++ [23-01-29 20:05:27] export -f ml
+++ [23-01-29 20:05:27] '[' 5 -ge 3 ']'
+++ [23-01-29 20:05:27] [[ hxB =~ i ]]
+++ [23-01-29 20:05:27] [[ ! :/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin: =~ :/usr/share/Modules/bin: ]]
+++ [23-01-29 20:05:27] '[' '!' -n x ']'
+++ [23-01-29 20:05:27] [[ ! :/usr/share/man:: =~ :/usr/share/man: ]]
+++ [23-01-29 20:05:27] unset _mlcode _mlret
+++ [23-01-29 20:05:27] '[' -n '' ']'
+ [23-01-29 20:05:27] [[ openmpi == *\p\s\m\2* ]]
++ [23-01-29 20:05:27] rpm -ql openmpi
++ [23-01-29 20:05:27] tail -n 1
++ [23-01-29 20:05:27] grep modulefiles
+ [23-01-29 20:05:27] local mod_file=/usr/share/modulefiles/mpi/openmpi-x86_64
+ [23-01-29 20:05:27] echo 'module purge'
+ [23-01-29 20:05:27] echo 'module load /usr/share/modulefiles/mpi/openmpi-x86_64'
+ [23-01-29 20:05:27] source /root/.bashrc
+ [23-01-29 20:05:27] module list
++ [23-01-29 20:05:27] /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash list
Currently Loaded Modulefiles:
 1) /usr/share/modulefiles/mpi/openmpi-x86_64  
+ [23-01-29 20:05:27] eval
+ [23-01-29 20:05:27] _mlstatus=0
+ [23-01-29 20:05:27] return 0
++ [23-01-29 20:05:27] echo /usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:05:27] sed -e 's|/usr/lib64/openmpi[^/]*/bin:||g'
+ [23-01-29 20:05:27] PATH=/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:05:27] echo /usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:05:27] sed -e 's|/usr/lib64/mvapich[^/]*/bin:||g'
+ [23-01-29 20:05:27] PATH=/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:05:27] echo /usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:05:27] sed -e 's|/usr/lib64/mpich[^/]*/bin:||g'
+ [23-01-29 20:05:27] PATH=/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
+ [23-01-29 20:05:27] export PATH=/usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
+ [23-01-29 20:05:27] PATH=/usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
+ [23-01-29 20:05:27] MPI_create_hostfiles
+ [23-01-29 20:05:27] '[' -z '' -o -z 172.31.40.31 ']'
+ [23-01-29 20:05:27] echo 'Please run /kernel/infiniband/env_setup before MPI_create_hostfiles'
Please run /kernel/infiniband/env_setup before MPI_create_hostfiles
+ [23-01-29 20:05:27] return 1
+ [23-01-29 20:05:27] echo 'MPI_setup: Failed to generate MPI hostfiles'
MPI_setup: Failed to generate MPI hostfiles
+ [23-01-29 20:05:27] echo 'MPI_setup: (This is expected on singlehost environment setup)'
MPI_setup: (This is expected on singlehost environment setup)
+ [23-01-29 20:05:27] '[' -x /bin/systemctl ']'
+ [23-01-29 20:05:27] systemctl stop firewalld.service
+ [23-01-29 20:05:27] systemctl disable firewalld.service
+ [23-01-29 20:05:28] systemctl status firewalld.service
○ firewalld.service - firewalld - dynamic firewall daemon
     Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; preset: enabled)
     Active: inactive (dead)
       Docs: man:firewalld(1)
+ [23-01-29 20:05:28] MPI_dual_port_fix
+ [23-01-29 20:05:28] [[ '' == *\q\i\b* ]]
++ [23-01-29 20:05:28] RQA_get_rhel_major
++ [23-01-29 20:05:28] awk -F . '{print $1}'
+++ [23-01-29 20:05:28] RQA_get_rhel_release
+++ [23-01-29 20:05:28] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:05:28] echo 9
+ [23-01-29 20:05:28] '[' 9 -ge 7 ']'
+ [23-01-29 20:05:28] RQA_pkg_install targetcli iscsi-initiator-utils lsscsi
+ [23-01-29 20:05:28] PKG_LIST=
+ [23-01-29 20:05:28] for p in "$@"
+ [23-01-29 20:05:28] rpm -q targetcli
package targetcli is not installed
+ [23-01-29 20:05:28] PKG_LIST=' targetcli'
+ [23-01-29 20:05:28] for p in "$@"
+ [23-01-29 20:05:28] rpm -q iscsi-initiator-utils
iscsi-initiator-utils-6.2.1.4-3.git2a8f9d8.el9.x86_64
+ [23-01-29 20:05:28] for p in "$@"
+ [23-01-29 20:05:28] rpm -q lsscsi
lsscsi-0.32-6.el9.x86_64
+ [23-01-29 20:05:28] '[' '!' -z ' targetcli' ']'
+ [23-01-29 20:05:28] dnf install -y --setopt=strict=0 --nogpgcheck targetcli
Last metadata expiration check: 0:13:13 ago on Sun 29 Jan 2023 07:52:16 PM EST.
Dependencies resolved.
================================================================================
 Package                 Arch       Version             Repository         Size
================================================================================
Installing:
 targetcli               noarch     2.1.53-7.el9        BUILDROOT-C9S      71 k
Installing dependencies:
 python3-configshell     noarch     1:1.1.28-7.el9      BUILDROOT-C9S      65 k
 python3-kmod            x86_64     0.9-32.el9          BUILDROOT-C9S      81 k
 python3-pyparsing       noarch     2.4.7-9.el9         BUILDROOT-C9S     149 k
 python3-rtslib          noarch     2.1.75-1.el9        BUILDROOT-C9S      90 k
 python3-urwid           x86_64     2.1.2-4.el9         BUILDROOT-C9S     768 k
 target-restore          noarch     2.1.75-1.el9        BUILDROOT-C9S      14 k

Transaction Summary
================================================================================
Install  7 Packages

Total download size: 1.2 M
Installed size: 5.0 M
Downloading Packages:
(1/7): python3-kmod-0.9-32.el9.x86_64.rpm       3.6 MB/s |  81 kB     00:00    
(2/7): python3-configshell-1.1.28-7.el9.noarch. 1.5 MB/s |  65 kB     00:00    
(3/7): python3-pyparsing-2.4.7-9.el9.noarch.rpm 2.8 MB/s | 149 kB     00:00    
(4/7): python3-rtslib-2.1.75-1.el9.noarch.rpm   2.2 MB/s |  90 kB     00:00    
(5/7): target-restore-2.1.75-1.el9.noarch.rpm   929 kB/s |  14 kB     00:00    
(6/7): targetcli-2.1.53-7.el9.noarch.rpm        5.2 MB/s |  71 kB     00:00    
(7/7): python3-urwid-2.1.2-4.el9.x86_64.rpm     9.1 MB/s | 768 kB     00:00    
--------------------------------------------------------------------------------
Total                                           8.8 MB/s | 1.2 MB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Installing       : python3-urwid-2.1.2-4.el9.x86_64                       1/7 
  Installing       : python3-pyparsing-2.4.7-9.el9.noarch                   2/7 
  Installing       : python3-configshell-1:1.1.28-7.el9.noarch              3/7 
  Installing       : python3-kmod-0.9-32.el9.x86_64                         4/7 
  Installing       : python3-rtslib-2.1.75-1.el9.noarch                     5/7 
  Installing       : target-restore-2.1.75-1.el9.noarch                     6/7 
  Running scriptlet: target-restore-2.1.75-1.el9.noarch                     6/7 
  Installing       : targetcli-2.1.53-7.el9.noarch                          7/7 
  Running scriptlet: targetcli-2.1.53-7.el9.noarch                          7/7 
  Verifying        : python3-configshell-1:1.1.28-7.el9.noarch              1/7 
  Verifying        : python3-kmod-0.9-32.el9.x86_64                         2/7 
  Verifying        : python3-pyparsing-2.4.7-9.el9.noarch                   3/7 
  Verifying        : python3-rtslib-2.1.75-1.el9.noarch                     4/7 
  Verifying        : python3-urwid-2.1.2-4.el9.x86_64                       5/7 
  Verifying        : target-restore-2.1.75-1.el9.noarch                     6/7 
  Verifying        : targetcli-2.1.53-7.el9.noarch                          7/7 

Installed:
  python3-configshell-1:1.1.28-7.el9.noarch  python3-kmod-0.9-32.el9.x86_64     
  python3-pyparsing-2.4.7-9.el9.noarch       python3-rtslib-2.1.75-1.el9.noarch 
  python3-urwid-2.1.2-4.el9.x86_64           target-restore-2.1.75-1.el9.noarch 
  targetcli-2.1.53-7.el9.noarch             

Complete!
+ [23-01-29 20:05:34] target_name=iqn.2003-01.org.linux-iscsi.rdma.x8664:test-target
++ [23-01-29 20:05:34] RQA_get_rhel_major
++ [23-01-29 20:05:34] awk -F . '{print $1}'
+++ [23-01-29 20:05:34] RQA_get_rhel_release
+++ [23-01-29 20:05:34] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:05:34] echo 9
+ [23-01-29 20:05:34] [[ 9 -ge 8 ]]
+ [23-01-29 20:05:34] RQA_pkg_install iperf3
+ [23-01-29 20:05:34] PKG_LIST=
+ [23-01-29 20:05:34] for p in "$@"
+ [23-01-29 20:05:34] rpm -q iperf3
iperf3-3.9-9.el9.x86_64
+ [23-01-29 20:05:34] '[' '!' -z '' ']'
+ [23-01-29 20:05:34] RQA_pkg_install net-tools
+ [23-01-29 20:05:34] PKG_LIST=
+ [23-01-29 20:05:34] for p in "$@"
+ [23-01-29 20:05:34] rpm -q net-tools
package net-tools is not installed
+ [23-01-29 20:05:34] PKG_LIST=' net-tools'
+ [23-01-29 20:05:34] '[' '!' -z ' net-tools' ']'
+ [23-01-29 20:05:34] dnf install -y --setopt=strict=0 --nogpgcheck net-tools
Last metadata expiration check: 0:13:19 ago on Sun 29 Jan 2023 07:52:16 PM EST.
Dependencies resolved.
================================================================================
 Package       Arch       Version                       Repository         Size
================================================================================
Installing:
 net-tools     x86_64     2.0-0.62.20160912git.el9      BUILDROOT-C9S     292 k

Transaction Summary
================================================================================
Install  1 Package

Total download size: 292 k
Installed size: 912 k
Downloading Packages:
net-tools-2.0-0.62.20160912git.el9.x86_64.rpm   9.4 MB/s | 292 kB     00:00    
--------------------------------------------------------------------------------
Total                                           8.7 MB/s | 292 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Installing       : net-tools-2.0-0.62.20160912git.el9.x86_64              1/1 
  Running scriptlet: net-tools-2.0-0.62.20160912git.el9.x86_64              1/1 
  Verifying        : net-tools-2.0-0.62.20160912git.el9.x86_64              1/1 

Installed:
  net-tools-2.0-0.62.20160912git.el9.x86_64                                     

Complete!
+ [23-01-29 20:05:38] [[ common == \c\o\m\m\o\n ]]
+ [23-01-29 20:05:38] common_tests
++ [23-01-29 20:05:38] RQA_get_driver_modules
++ [23-01-29 20:05:38] case ${RDMA_DRIVER} in
++ [23-01-29 20:05:38] echo ''
+ [23-01-29 20:05:38] driver_modules=
+ [23-01-29 20:05:38] [[ '' == *\o\p\a* ]]
+ [23-01-29 20:05:38] [[ '' == *\i\b* ]]
++ [23-01-29 20:05:38] ibstatus
++ [23-01-29 20:05:38] grep -A 7 'port '
++ [23-01-29 20:05:38] awk '/rate:/ {print $2 " " $3}'
+ [23-01-29 20:05:38] ibstatus_rate='100 Gb/sec'
+ [23-01-29 20:05:38] [[ -z '' ]]
+ [23-01-29 20:05:38] RQA_check_result -r -1 -t 'HCA_RATE not set & can'\''t compare HCA rate'
+ [23-01-29 20:05:38] local test_pass=0
+ [23-01-29 20:05:38] local test_skip=777
+ [23-01-29 20:05:38] test 4 -gt 0
+ [23-01-29 20:05:38] case $1 in
+ [23-01-29 20:05:38] local rc=-1
+ [23-01-29 20:05:38] shift
+ [23-01-29 20:05:38] shift
+ [23-01-29 20:05:38] test 2 -gt 0
+ [23-01-29 20:05:38] case $1 in
+ [23-01-29 20:05:38] local 'msg=HCA_RATE not set & can'\''t compare HCA rate'
+ [23-01-29 20:05:38] shift
+ [23-01-29 20:05:38] shift
+ [23-01-29 20:05:38] test 0 -gt 0
+ [23-01-29 20:05:38] '[' -z -1 -o -z 'HCA_RATE not set & can'\''t compare HCA rate' ']'
+ [23-01-29 20:05:38] '[' -z '' ']'
+ [23-01-29 20:05:38] source /etc/environment
++ [23-01-29 20:05:38] TEST_RESULT_FILE=
+ [23-01-29 20:05:38] '[' -z '' ']'
+ [23-01-29 20:05:38] __RQA_create_test_result_file
++ [23-01-29 20:05:38] rpm -q --queryformat '%{NAME}-%{VERSION}-%{RELEASE}' rdma-core
+ [23-01-29 20:05:39] local rdma_version=rdma-core-41.0-3.el9
++ [23-01-29 20:05:39] echo /kernel/infiniband/cki/sanity
++ [23-01-29 20:05:39] sed -r 's/\/kernel\/infiniband\///;s/\/server//;s/\/client//;s/\/standalone//'
+ [23-01-29 20:05:39] TEST_NAME=cki/sanity
+ [23-01-29 20:05:39] [[ ! -z rdma-cki-01 ]]
+ [23-01-29 20:05:39] [[ ! -z rdma-qe-31 ]]
+ [23-01-29 20:05:39] hosts_list=rdma-cki-01/rdma-qe-31
+ [23-01-29 20:05:39] local _bkrjb=
++ [23-01-29 20:05:39] grep JOBID= /etc/motd
++ [23-01-29 20:05:39] tr -d ' '
++ [23-01-29 20:05:39] sed s/JOBID=/J:/g
++ [23-01-29 20:05:39] head -1
+ [23-01-29 20:05:39] _bkrjb=J:7478195
+ [23-01-29 20:05:39] [[ ! -z J:7478195 ]]
+ [23-01-29 20:05:39] _bkrjb=' & Beaker job J:7478195'
++ [23-01-29 20:05:39] echo results_cki/sanity.txt
++ [23-01-29 20:05:39] sed 's/\//-/g'
+ [23-01-29 20:05:39] local result_filename=results_cki-sanity.txt
++ [23-01-29 20:05:39] mktemp -d
+ [23-01-29 20:05:39] TEST_RESULT_FILE=/tmp/tmp.GcKXytvlEd/results_cki-sanity.txt
+ [23-01-29 20:05:39] echo 'cki/sanity test results on rdma-cki-01/rdma-qe-31 & Beaker job J:7478195:'
++ [23-01-29 20:05:39] uname -r
+ [23-01-29 20:05:39] echo '5.14.0-247.rt14.248.1955_761016865.el9.x86_64, rdma-core-41.0-3.el9, , ,  & '
+ [23-01-29 20:05:39] echo '    Result | Status | Test'
+ [23-01-29 20:05:39] echo '  ---------+--------+------------------------------------'
+ [23-01-29 20:05:39] sed -i /TEST_RESULT_FILE/d /etc/environment
+ [23-01-29 20:05:39] echo TEST_RESULT_FILE=/tmp/tmp.GcKXytvlEd/results_cki-sanity.txt
+ [23-01-29 20:05:39] source /etc/environment
++ [23-01-29 20:05:39] TEST_RESULT_FILE=/tmp/tmp.GcKXytvlEd/results_cki-sanity.txt
+ [23-01-29 20:05:39] '[' -1 -eq 0 ']'
+ [23-01-29 20:05:39] '[' -1 -eq 777 ']'
+ [23-01-29 20:05:39] local test_result=FAIL
+ [23-01-29 20:05:39] export result=FAIL
+ [23-01-29 20:05:39] result=FAIL
+ [23-01-29 20:05:39] [[ ! -z '' ]]
+ [23-01-29 20:05:39] printf '%10s | %6s | %s\n' FAIL -1 'HCA_RATE not set & can'\''t compare HCA rate'
+ [23-01-29 20:05:39] set +x
---
- TEST RESULT FOR sanity
-   Test:   HCA_RATE not set & can't compare HCA rate
-   Result: FAIL
-   Return: -1
---
++ [23-01-29 20:05:39] RQA_get_rhel_major
++ [23-01-29 20:05:39] awk -F . '{print $1}'
+++ [23-01-29 20:05:39] RQA_get_rhel_release
+++ [23-01-29 20:05:39] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:05:39] echo 9
+ [23-01-29 20:05:39] [[ 9 -ge 8 ]]
+ [23-01-29 20:05:39] [[ '' == \i\b\0 ]]
+ [23-01-29 20:05:39] [[ '' == \i\b\1 ]]
+ [23-01-29 20:05:39] [[ '' == \o\p\a\0 ]]
+ [23-01-29 20:05:39] [[ '' == \r\o\c\e ]]
+ [23-01-29 20:05:39] [[ '' == \i\w ]]
+ [23-01-29 20:05:39] [[ '' != *\i\w* ]]
+ [23-01-29 20:05:39] /usr/sbin/ibstat
CA 'mlx5_bond_0'
	CA type: MT4125
	Number of ports: 1
	Firmware version: 22.99.7864
	Hardware version: 0
	Node GUID: 0xb8599f0300f07fd0
	System image GUID: 0xb8599f0300f07fd0
	Port 1:
		State: Active
		Physical state: LinkUp
		Rate: 100
		Base lid: 0
		LMC: 0
		SM lid: 0
		Capability mask: 0x00010000
		Port GUID: 0xba599ffffef07fd0
		Link layer: Ethernet
+ [23-01-29 20:05:39] RQA_check_result -r 0 -t /usr/sbin/ibstat
+ [23-01-29 20:05:39] local test_pass=0
+ [23-01-29 20:05:39] local test_skip=777
+ [23-01-29 20:05:39] test 4 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local rc=0
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 2 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local msg=/usr/sbin/ibstat
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 0 -gt 0
+ [23-01-29 20:05:39] '[' -z 0 -o -z /usr/sbin/ibstat ']'
+ [23-01-29 20:05:39] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:39] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:39] '[' 0 -eq 0 ']'
+ [23-01-29 20:05:39] local test_result=PASS
+ [23-01-29 20:05:39] printf '%10s | %6s | %s\n' PASS 0 /usr/sbin/ibstat
+ [23-01-29 20:05:39] set +x
---
- TEST RESULT FOR sanity
-   Test:   /usr/sbin/ibstat
-   Result: PASS
-   Return: 0
---
+ [23-01-29 20:05:39] /usr/sbin/ibstatus
Infiniband device 'mlx5_bond_0' port 1 status:
	default gid:	 fe80:0000:0000:0000:ba59:9fff:fef0:7fd0
	base lid:	 0x0
	sm lid:		 0x0
	state:		 4: ACTIVE
	phys state:	 5: LinkUp
	rate:		 100 Gb/sec (4X EDR)
	link_layer:	 Ethernet

+ [23-01-29 20:05:39] RQA_check_result -r 0 -t /usr/sbin/ibstatus
+ [23-01-29 20:05:39] local test_pass=0
+ [23-01-29 20:05:39] local test_skip=777
+ [23-01-29 20:05:39] test 4 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local rc=0
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 2 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local msg=/usr/sbin/ibstatus
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 0 -gt 0
+ [23-01-29 20:05:39] '[' -z 0 -o -z /usr/sbin/ibstatus ']'
+ [23-01-29 20:05:39] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:39] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:39] '[' 0 -eq 0 ']'
+ [23-01-29 20:05:39] local test_result=PASS
+ [23-01-29 20:05:39] printf '%10s | %6s | %s\n' PASS 0 /usr/sbin/ibstatus
+ [23-01-29 20:05:39] set +x
---
- TEST RESULT FOR sanity
-   Test:   /usr/sbin/ibstatus
-   Result: PASS
-   Return: 0
---
+ [23-01-29 20:05:39] rpm -q srptools srp_daemon
package srptools is not installed
srp_daemon-41.0-3.el9.x86_64
+ [23-01-29 20:05:39] guid_st_01=0x001175000078b784
+ [23-01-29 20:05:39] guid_st_03=0xe41d2d03001d6470
+ [23-01-29 20:05:39] echo 'a 0x001175000078b784 pkey=ffff'
+ [23-01-29 20:05:39] echo 'a 0xe41d2d03001d6470 pkey=ffff'
+ [23-01-29 20:05:39] echo d
+ [23-01-29 20:05:39] local srpd_serv=srpd.service
+ [23-01-29 20:05:39] '[' -f /usr/lib/systemd/system/srp_daemon.service ']'
+ [23-01-29 20:05:39] srpd_serv=srp_daemon.service
+ [23-01-29 20:05:39] systemctl is-active srp_daemon.service
+ [23-01-29 20:05:39] systemctl start srp_daemon.service
+ [23-01-29 20:05:39] RQA_check_result -r 0 -t 'systemctl start srp_daemon.service'
+ [23-01-29 20:05:39] local test_pass=0
+ [23-01-29 20:05:39] local test_skip=777
+ [23-01-29 20:05:39] test 4 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local rc=0
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 2 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local 'msg=systemctl start srp_daemon.service'
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 0 -gt 0
+ [23-01-29 20:05:39] '[' -z 0 -o -z 'systemctl start srp_daemon.service' ']'
+ [23-01-29 20:05:39] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:39] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:39] '[' 0 -eq 0 ']'
+ [23-01-29 20:05:39] local test_result=PASS
+ [23-01-29 20:05:39] printf '%10s | %6s | %s\n' PASS 0 'systemctl start srp_daemon.service'
+ [23-01-29 20:05:39] set +x
---
- TEST RESULT FOR sanity
-   Test:   systemctl start srp_daemon.service
-   Result: PASS
-   Return: 0
---
+ [23-01-29 20:05:39] RQA_skip_test ibsrpdm
+ [23-01-29 20:05:39] local skp=ibsrpdm
+ [23-01-29 20:05:39] local skp_rtn=0
+ [23-01-29 20:05:39] '[' '!' -z ibsrpdm ']'
+ [23-01-29 20:05:39] [[ '' == *\i\b\s\r\p\d\m* ]]
+ [23-01-29 20:05:39] return 0
+ [23-01-29 20:05:39] skip_ibsrpdm=0
+ [23-01-29 20:05:39] '[' 0 -eq 0 ']'
+ [23-01-29 20:05:39] /usr/sbin/ibsrpdm -vc
Unsupported link layer Ethernet
Resource allocation failed
Using device mlx5_bond_0 port 1
+ [23-01-29 20:05:39] RQA_check_result -r 1 -t '/usr/sbin/ibsrpdm -vc'
+ [23-01-29 20:05:39] local test_pass=0
+ [23-01-29 20:05:39] local test_skip=777
+ [23-01-29 20:05:39] test 4 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local rc=1
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 2 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local 'msg=/usr/sbin/ibsrpdm -vc'
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 0 -gt 0
+ [23-01-29 20:05:39] '[' -z 1 -o -z '/usr/sbin/ibsrpdm -vc' ']'
+ [23-01-29 20:05:39] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:39] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:39] '[' 1 -eq 0 ']'
+ [23-01-29 20:05:39] '[' 1 -eq 777 ']'
+ [23-01-29 20:05:39] local test_result=FAIL
+ [23-01-29 20:05:39] export result=FAIL
+ [23-01-29 20:05:39] result=FAIL
+ [23-01-29 20:05:39] [[ ! -z '' ]]
+ [23-01-29 20:05:39] printf '%10s | %6s | %s\n' FAIL 1 '/usr/sbin/ibsrpdm -vc'
+ [23-01-29 20:05:39] set +x
---
- TEST RESULT FOR sanity
-   Test:   /usr/sbin/ibsrpdm -vc
-   Result: FAIL
-   Return: 1
---
+ [23-01-29 20:05:39] systemctl stop srp_daemon
+ [23-01-29 20:05:39] RQA_check_result -r 0 -t 'systemctl stop srp_daemon'
+ [23-01-29 20:05:39] local test_pass=0
+ [23-01-29 20:05:39] local test_skip=777
+ [23-01-29 20:05:39] test 4 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local rc=0
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 2 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local 'msg=systemctl stop srp_daemon'
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 0 -gt 0
+ [23-01-29 20:05:39] '[' -z 0 -o -z 'systemctl stop srp_daemon' ']'
+ [23-01-29 20:05:39] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:39] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:39] '[' 0 -eq 0 ']'
+ [23-01-29 20:05:39] local test_result=PASS
+ [23-01-29 20:05:39] printf '%10s | %6s | %s\n' PASS 0 'systemctl stop srp_daemon'
+ [23-01-29 20:05:39] set +x
---
- TEST RESULT FOR sanity
-   Test:   systemctl stop srp_daemon
-   Result: PASS
-   Return: 0
---
++ [23-01-29 20:05:39] RQA_get_rhel_major
++ [23-01-29 20:05:39] awk -F . '{print $1}'
+++ [23-01-29 20:05:39] RQA_get_rhel_release
+++ [23-01-29 20:05:39] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:05:39] echo 9
+ [23-01-29 20:05:39] '[' 9 -lt 8 ']'
+ [23-01-29 20:05:39] timeout 3m ping -i 0.2 -c 100
ping: usage error: Destination address required
+ [23-01-29 20:05:39] RQA_check_result -r 1 -t 'ping self - '
+ [23-01-29 20:05:39] local test_pass=0
+ [23-01-29 20:05:39] local test_skip=777
+ [23-01-29 20:05:39] test 4 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local rc=1
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 2 -gt 0
+ [23-01-29 20:05:39] case $1 in
+ [23-01-29 20:05:39] local 'msg=ping self - '
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] shift
+ [23-01-29 20:05:39] test 0 -gt 0
+ [23-01-29 20:05:39] '[' -z 1 -o -z 'ping self - ' ']'
+ [23-01-29 20:05:40] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:40] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:40] '[' 1 -eq 0 ']'
+ [23-01-29 20:05:40] '[' 1 -eq 777 ']'
+ [23-01-29 20:05:40] local test_result=FAIL
+ [23-01-29 20:05:40] export result=FAIL
+ [23-01-29 20:05:40] result=FAIL
+ [23-01-29 20:05:40] [[ ! -z '' ]]
+ [23-01-29 20:05:40] printf '%10s | %6s | %s\n' FAIL 1 'ping self - '
+ [23-01-29 20:05:40] set +x
---
- TEST RESULT FOR sanity
-   Test:   ping self - 
-   Result: FAIL
-   Return: 1
---
+ [23-01-29 20:05:40] timeout 3m ping6 -i 0.2 -c 100 %
ping6: %: Name or service not known
+ [23-01-29 20:05:40] RQA_check_result -r 2 -t 'ping6 self - %'
+ [23-01-29 20:05:40] local test_pass=0
+ [23-01-29 20:05:40] local test_skip=777
+ [23-01-29 20:05:40] test 4 -gt 0
+ [23-01-29 20:05:40] case $1 in
+ [23-01-29 20:05:40] local rc=2
+ [23-01-29 20:05:40] shift
+ [23-01-29 20:05:40] shift
+ [23-01-29 20:05:40] test 2 -gt 0
+ [23-01-29 20:05:40] case $1 in
+ [23-01-29 20:05:40] local 'msg=ping6 self - %'
+ [23-01-29 20:05:40] shift
+ [23-01-29 20:05:40] shift
+ [23-01-29 20:05:40] test 0 -gt 0
+ [23-01-29 20:05:40] '[' -z 2 -o -z 'ping6 self - %' ']'
+ [23-01-29 20:05:40] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:40] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:05:40] '[' 2 -eq 0 ']'
+ [23-01-29 20:05:40] '[' 2 -eq 777 ']'
+ [23-01-29 20:05:40] local test_result=FAIL
+ [23-01-29 20:05:40] export result=FAIL
+ [23-01-29 20:05:40] result=FAIL
+ [23-01-29 20:05:40] [[ ! -z '' ]]
+ [23-01-29 20:05:40] printf '%10s | %6s | %s\n' FAIL 2 'ping6 self - %'
+ [23-01-29 20:05:40] set +x
---
- TEST RESULT FOR sanity
-   Test:   ping6 self - %
-   Result: FAIL
-   Return: 2
---
++ [23-01-29 20:05:40] RQA_get_rhel_major
++ [23-01-29 20:05:40] awk -F . '{print $1}'
+++ [23-01-29 20:05:40] RQA_get_rhel_release
+++ [23-01-29 20:05:40] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:05:40] echo 9
+ [23-01-29 20:05:40] [[ 9 -gt 7 ]]
++ [23-01-29 20:05:40] RQA_get_rhel_minor
++ [23-01-29 20:05:40] awk -F . '{print $2}'
+++ [23-01-29 20:05:40] RQA_get_rhel_release
+++ [23-01-29 20:05:40] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:05:40] echo 9
+ [23-01-29 20:05:40] [[ '' -gt 2 ]]
+ [23-01-29 20:05:40] echo '--- server is ready.'
--- server is ready.
+ [23-01-29 20:05:40] rhts_sync_set -s server-ready
/usr/bin/rhts_sync_set -s server-ready -- State set = 12_server-ready
+ [23-01-29 20:05:40] rhts_sync_block -s client-ready rdma-qe-31.rdma.lab.eng.rdu2.redhat.com
/usr/bin/rhts_sync_block -s client-ready rdma-qe-31.rdma.lab.eng.rdu2.redhat.com -- Blocking state(s) =  12_client-ready
+ [23-01-29 20:05:40] bash -x ./tier1.sh server
+ trap 'exit 1' SIGHUP SIGINT SIGQUIT SIGTERM
+ source /root/rdma-qa.sh
++ PS4='$(date +"+ [%y-%m-%d %H:%M:%S]") '
++ [23-01-29 20:05:40] export NIC_PREF=
++ [23-01-29 20:05:40] NIC_PREF=
++ [23-01-29 20:05:40] export PERFTEST_FLAGS=
++ [23-01-29 20:05:40] PERFTEST_FLAGS=
+++ [23-01-29 20:05:40] hostname -s
++ [23-01-29 20:05:40] export s_hostname=rdma-cki-01
++ [23-01-29 20:05:40] s_hostname=rdma-cki-01
++ [23-01-29 20:05:40] RQA_set_pyexec
++ [23-01-29 20:05:40] PYEXEC=
++ [23-01-29 20:05:40] which python
++ [23-01-29 20:05:40] export PYEXEC=python
++ [23-01-29 20:05:40] PYEXEC=python
++ [23-01-29 20:05:40] return 0
+++ [23-01-29 20:05:40] grep -i fedora /etc/redhat-release
++ [23-01-29 20:05:40] [[ -n '' ]]
+++ [23-01-29 20:05:40] RQA_get_rhel_major
+++ [23-01-29 20:05:40] awk -F . '{print $1}'
++++ [23-01-29 20:05:40] RQA_get_rhel_release
++++ [23-01-29 20:05:40] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:40] echo 9
++ [23-01-29 20:05:40] [[ 9 -ge 8 ]]
++ [23-01-29 20:05:40] export 'PKGINSTALL=dnf install -y --setopt=strict=0 --nogpgcheck'
++ [23-01-29 20:05:40] PKGINSTALL='dnf install -y --setopt=strict=0 --nogpgcheck'
++ [23-01-29 20:05:40] export 'PKGREMOVE=dnf remove --noautoremove -y'
++ [23-01-29 20:05:40] PKGREMOVE='dnf remove --noautoremove -y'
++ [23-01-29 20:05:40] [[ rdma-cki-01 == \r\d\m\a\-\d\e\v\-\2\0 ]]
++ [23-01-29 20:05:40] export LATEST_MFT=http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz
++ [23-01-29 20:05:40] LATEST_MFT=http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz
++ [23-01-29 20:05:40] '[' -f /root/known-issues.sh ']'
++ [23-01-29 20:05:40] source /root/known-issues.sh
++ [23-01-29 20:05:44] RQA_load_tech_preview_modules
++ [23-01-29 20:05:44] [[ ! -e /etc/rdma/rdma.conf ]]
++ [23-01-29 20:05:44] return
++ [23-01-29 20:05:44] RQA_fix_uefi_reboot
++ [23-01-29 20:05:44] ls -l /usr/sbin/reboot
++ [23-01-29 20:05:44] grep rhts-reboot
++ [23-01-29 20:05:44] which rhts-reboot
++ [23-01-29 20:05:44] rpm -q rhts-test-env
++ [23-01-29 20:05:44] RQA_fix_ssh_ipv6_auth
+++ [23-01-29 20:05:44] RQA_get_rhel_major
+++ [23-01-29 20:05:44] awk -F . '{print $1}'
++++ [23-01-29 20:05:44] RQA_get_rhel_release
++++ [23-01-29 20:05:44] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:44] echo 9
++ [23-01-29 20:05:44] '[' 9 -ge 7 ']'
++ [23-01-29 20:05:44] local need_restart=0
++ [23-01-29 20:05:44] grep -E '^UseDNS yes' /etc/ssh/sshd_config
++ [23-01-29 20:05:44] grep -E '^GSSAPIStrictAcceptorCheck yes' /etc/ssh/sshd_config
++ [23-01-29 20:05:44] grep 'AddressFamily inet' /root/.ssh/config
++ [23-01-29 20:05:44] '[' 0 -eq 1 ']'
++ [23-01-29 20:05:44] RQA_fix_mlx5_ib_connected_mode
+++ [23-01-29 20:05:44] RQA_get_rhel_major
+++ [23-01-29 20:05:44] awk -F . '{print $1}'
++++ [23-01-29 20:05:44] RQA_get_rhel_release
++++ [23-01-29 20:05:44] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:44] echo 9
++ [23-01-29 20:05:44] [[ 9 -ge 7 ]]
+++ [23-01-29 20:05:44] RQA_get_rhel_minor
+++ [23-01-29 20:05:44] awk -F . '{print $2}'
++++ [23-01-29 20:05:44] RQA_get_rhel_release
++++ [23-01-29 20:05:44] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:44] echo 9
++ [23-01-29 20:05:44] [[ '' -ge 5 ]]
+++ [23-01-29 20:05:44] RQA_get_rhel_major
+++ [23-01-29 20:05:44] awk -F . '{print $1}'
++++ [23-01-29 20:05:44] RQA_get_rhel_release
++++ [23-01-29 20:05:44] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:44] echo 9
++ [23-01-29 20:05:44] [[ 9 -gt 7 ]]
+++ [23-01-29 20:05:44] ls /etc/sysconfig/network-scripts/
+++ [23-01-29 20:05:44] grep mlx5
+++ [23-01-29 20:05:44] grep ib
+++ [23-01-29 20:05:44] grep -v '~'
++ [23-01-29 20:05:44] RQA_nfs_rhel8_temp_fix
+++ [23-01-29 20:05:44] RQA_get_rhel_major
+++ [23-01-29 20:05:44] awk -F . '{print $1}'
++++ [23-01-29 20:05:44] RQA_get_rhel_release
++++ [23-01-29 20:05:44] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:44] echo 9
++ [23-01-29 20:05:44] ((  9 == 8  ))
++ [23-01-29 20:05:44] source /root/.bashrc
++ [23-01-29 20:05:45] [[ '' == \i\4\0\i\w ]]
+ [23-01-29 20:05:45] source /root/mpi-functions.sh
++ [23-01-29 20:05:45] export 'MPI_TIMEOUT=timeout --preserve-status --kill-after=5m 3m'
++ [23-01-29 20:05:45] MPI_TIMEOUT='timeout --preserve-status --kill-after=5m 3m'
+ [23-01-29 20:05:45] source /root/rdma-qa.sh
++ [23-01-29 20:05:45] PS4='$(date +"+ [%y-%m-%d %H:%M:%S]") '
++ [23-01-29 20:05:45] export NIC_PREF=
++ [23-01-29 20:05:45] NIC_PREF=
++ [23-01-29 20:05:45] export PERFTEST_FLAGS=
++ [23-01-29 20:05:45] PERFTEST_FLAGS=
+++ [23-01-29 20:05:45] hostname -s
++ [23-01-29 20:05:45] export s_hostname=rdma-cki-01
++ [23-01-29 20:05:45] s_hostname=rdma-cki-01
++ [23-01-29 20:05:45] RQA_set_pyexec
++ [23-01-29 20:05:45] PYEXEC=
++ [23-01-29 20:05:45] which python
++ [23-01-29 20:05:45] export PYEXEC=python
++ [23-01-29 20:05:45] PYEXEC=python
++ [23-01-29 20:05:45] return 0
+++ [23-01-29 20:05:45] grep -i fedora /etc/redhat-release
++ [23-01-29 20:05:45] [[ -n '' ]]
+++ [23-01-29 20:05:45] RQA_get_rhel_major
+++ [23-01-29 20:05:45] awk -F . '{print $1}'
++++ [23-01-29 20:05:45] RQA_get_rhel_release
++++ [23-01-29 20:05:45] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:45] echo 9
++ [23-01-29 20:05:45] [[ 9 -ge 8 ]]
++ [23-01-29 20:05:45] export 'PKGINSTALL=dnf install -y --setopt=strict=0 --nogpgcheck'
++ [23-01-29 20:05:45] PKGINSTALL='dnf install -y --setopt=strict=0 --nogpgcheck'
++ [23-01-29 20:05:45] export 'PKGREMOVE=dnf remove --noautoremove -y'
++ [23-01-29 20:05:45] PKGREMOVE='dnf remove --noautoremove -y'
++ [23-01-29 20:05:45] [[ rdma-cki-01 == \r\d\m\a\-\d\e\v\-\2\0 ]]
++ [23-01-29 20:05:45] export LATEST_MFT=http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz
++ [23-01-29 20:05:45] LATEST_MFT=http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz
++ [23-01-29 20:05:45] '[' -f /root/known-issues.sh ']'
++ [23-01-29 20:05:45] source /root/known-issues.sh
++ [23-01-29 20:05:51] RQA_load_tech_preview_modules
++ [23-01-29 20:05:51] [[ ! -e /etc/rdma/rdma.conf ]]
++ [23-01-29 20:05:51] return
++ [23-01-29 20:05:51] RQA_fix_uefi_reboot
++ [23-01-29 20:05:51] ls -l /usr/sbin/reboot
++ [23-01-29 20:05:51] grep rhts-reboot
++ [23-01-29 20:05:51] which rhts-reboot
++ [23-01-29 20:05:51] rpm -q rhts-test-env
++ [23-01-29 20:05:51] RQA_fix_ssh_ipv6_auth
+++ [23-01-29 20:05:51] RQA_get_rhel_major
+++ [23-01-29 20:05:51] awk -F . '{print $1}'
++++ [23-01-29 20:05:51] RQA_get_rhel_release
++++ [23-01-29 20:05:51] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:51] echo 9
++ [23-01-29 20:05:51] '[' 9 -ge 7 ']'
++ [23-01-29 20:05:51] local need_restart=0
++ [23-01-29 20:05:51] grep -E '^UseDNS yes' /etc/ssh/sshd_config
++ [23-01-29 20:05:51] grep -E '^GSSAPIStrictAcceptorCheck yes' /etc/ssh/sshd_config
++ [23-01-29 20:05:51] grep 'AddressFamily inet' /root/.ssh/config
++ [23-01-29 20:05:51] '[' 0 -eq 1 ']'
++ [23-01-29 20:05:51] RQA_fix_mlx5_ib_connected_mode
+++ [23-01-29 20:05:51] RQA_get_rhel_major
+++ [23-01-29 20:05:51] awk -F . '{print $1}'
++++ [23-01-29 20:05:51] RQA_get_rhel_release
++++ [23-01-29 20:05:51] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:51] echo 9
++ [23-01-29 20:05:51] [[ 9 -ge 7 ]]
+++ [23-01-29 20:05:51] RQA_get_rhel_minor
+++ [23-01-29 20:05:51] awk -F . '{print $2}'
++++ [23-01-29 20:05:51] RQA_get_rhel_release
++++ [23-01-29 20:05:51] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:51] echo 9
++ [23-01-29 20:05:51] [[ '' -ge 5 ]]
+++ [23-01-29 20:05:51] RQA_get_rhel_major
+++ [23-01-29 20:05:51] awk -F . '{print $1}'
++++ [23-01-29 20:05:51] RQA_get_rhel_release
++++ [23-01-29 20:05:51] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:51] echo 9
++ [23-01-29 20:05:51] [[ 9 -gt 7 ]]
+++ [23-01-29 20:05:51] ls /etc/sysconfig/network-scripts/
+++ [23-01-29 20:05:51] grep mlx5
+++ [23-01-29 20:05:51] grep -v '~'
+++ [23-01-29 20:05:51] grep ib
++ [23-01-29 20:05:51] RQA_nfs_rhel8_temp_fix
+++ [23-01-29 20:05:51] RQA_get_rhel_major
+++ [23-01-29 20:05:51] awk -F . '{print $1}'
++++ [23-01-29 20:05:51] RQA_get_rhel_release
++++ [23-01-29 20:05:51] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
+++ [23-01-29 20:05:51] echo 9
++ [23-01-29 20:05:51] ((  9 == 8  ))
++ [23-01-29 20:05:51] source /root/.bashrc
++ [23-01-29 20:05:51] [[ '' == \i\4\0\i\w ]]
+ [23-01-29 20:05:51] source /root/mpi-functions.sh
++ [23-01-29 20:05:51] export 'MPI_TIMEOUT=timeout --preserve-status --kill-after=5m 3m'
++ [23-01-29 20:05:51] MPI_TIMEOUT='timeout --preserve-status --kill-after=5m 3m'
+ [23-01-29 20:05:51] RQA_set_inline_default
+ [23-01-29 20:05:51] [[ '' != *\m\l\x\4* ]]
+ [23-01-29 20:05:51] mkdir -p /etc/rdma/rsocket
+ [23-01-29 20:05:51] echo 0
+ [23-01-29 20:05:51] RQA_set_perftest_flags
+ [23-01-29 20:05:51] PERFTEST_FLAGS='-d  -i  -F'
+ [23-01-29 20:05:51] [[ '' == \m\l\x\4 ]]
+ [23-01-29 20:05:51] PERFTEST_FLAGS='-d  -i  -F -R'
+ [23-01-29 20:05:51] export PERFTEST_FLAGS
+ [23-01-29 20:05:51] MPI_PKG=openmpi
+ [23-01-29 20:05:51] TEST_PKG=mpitests-openmpi
++ [23-01-29 20:05:51] RQA_get_rhel_major
++ [23-01-29 20:05:51] awk -F . '{print $1}'
+++ [23-01-29 20:05:51] RQA_get_rhel_release
+++ [23-01-29 20:05:51] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:05:51] echo 9
+ [23-01-29 20:05:51] [[ 9 -eq 6 ]]
++ [23-01-29 20:05:51] RQA_get_rhel_major
++ [23-01-29 20:05:51] awk -F . '{print $1}'
+++ [23-01-29 20:05:51] RQA_get_rhel_release
+++ [23-01-29 20:05:51] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:05:51] echo 9
+ [23-01-29 20:05:51] [[ 9 -eq 7 ]]
+ [23-01-29 20:05:51] MPI_setup -m openmpi -t mpitests-openmpi
+ [23-01-29 20:05:51] test 4 -gt 0
+ [23-01-29 20:05:51] case $1 in
+ [23-01-29 20:05:51] local mpi_pkg=openmpi
+ [23-01-29 20:05:51] shift
+ [23-01-29 20:05:51] shift
+ [23-01-29 20:05:51] test 2 -gt 0
+ [23-01-29 20:05:51] case $1 in
+ [23-01-29 20:05:51] local test_pkg=mpitests-openmpi
+ [23-01-29 20:05:51] shift
+ [23-01-29 20:05:51] shift
+ [23-01-29 20:05:51] test 0 -gt 0
+ [23-01-29 20:05:51] '[' -z openmpi -o -z mpitests-openmpi ']'
+ [23-01-29 20:05:51] rpm -q environment-modules
environment-modules-5.0.1-2.el9.x86_64
+ [23-01-29 20:05:51] '[' -f /etc/profile.d/modules.sh ']'
+ [23-01-29 20:05:51] source /etc/profile.d/modules.sh
++ [23-01-29 20:05:51] '[' -n /usr/bin/bash ']'
++ [23-01-29 20:05:51] '[' bash = sh ']'
++ [23-01-29 20:05:51] shell=bash
++ [23-01-29 20:05:51] '[' -f /usr/share/Modules/init/bash ']'
++ [23-01-29 20:05:51] . /usr/share/Modules/init/bash
+++ [23-01-29 20:05:51] unset _mlshdbg
+++ [23-01-29 20:05:51] '[' 0 = 1 ']'
+++ [23-01-29 20:05:51] export 'MODULES_RUN_QUARANTINE=LD_LIBRARY_PATH LD_PRELOAD'
+++ [23-01-29 20:05:51] MODULES_RUN_QUARANTINE='LD_LIBRARY_PATH LD_PRELOAD'
+++ [23-01-29 20:05:51] unset _mlre _mlIFS
+++ [23-01-29 20:05:51] '[' -n x ']'
+++ [23-01-29 20:05:51] _mlIFS=' 	
'
+++ [23-01-29 20:05:51] IFS=' '
+++ [23-01-29 20:05:51] for _mlv in ${MODULES_RUN_QUARANTINE:-}
+++ [23-01-29 20:05:51] '[' LD_LIBRARY_PATH = LD_LIBRARY_PATH -a LD_LIBRARY_PATH = LD_LIBRARY_PATH ']'
++++ [23-01-29 20:05:51] eval 'echo ${LD_LIBRARY_PATH+x}'
+++++ [23-01-29 20:05:51] echo x
+++ [23-01-29 20:05:51] '[' -n x ']'
++++ [23-01-29 20:05:51] eval 'echo ${LD_LIBRARY_PATH}'
+++++ [23-01-29 20:05:51] echo /usr/lib64/openmpi/lib
+++ [23-01-29 20:05:51] _mlre='__MODULES_QUAR_LD_LIBRARY_PATH='\''/usr/lib64/openmpi/lib'\'' '
+++ [23-01-29 20:05:51] _mlrv=MODULES_RUNENV_LD_LIBRARY_PATH
++++ [23-01-29 20:05:51] eval 'echo ${MODULES_RUNENV_LD_LIBRARY_PATH:-}'
+++++ [23-01-29 20:05:51] echo
+++ [23-01-29 20:05:51] _mlre='__MODULES_QUAR_LD_LIBRARY_PATH='\''/usr/lib64/openmpi/lib'\'' LD_LIBRARY_PATH='\'''\'' '
+++ [23-01-29 20:05:51] for _mlv in ${MODULES_RUN_QUARANTINE:-}
+++ [23-01-29 20:05:51] '[' LD_PRELOAD = LD_PRELOAD -a LD_PRELOAD = LD_PRELOAD ']'
++++ [23-01-29 20:05:51] eval 'echo ${LD_PRELOAD+x}'
+++++ [23-01-29 20:05:51] echo
+++ [23-01-29 20:05:51] '[' -n '' ']'
+++ [23-01-29 20:05:51] _mlrv=MODULES_RUNENV_LD_PRELOAD
++++ [23-01-29 20:05:52] eval 'echo ${MODULES_RUNENV_LD_PRELOAD:-}'
+++++ [23-01-29 20:05:52] echo
+++ [23-01-29 20:05:52] _mlre='__MODULES_QUAR_LD_LIBRARY_PATH='\''/usr/lib64/openmpi/lib'\'' LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' '
+++ [23-01-29 20:05:52] '[' -n '__MODULES_QUAR_LD_LIBRARY_PATH='\''/usr/lib64/openmpi/lib'\'' LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' ' ']'
+++ [23-01-29 20:05:52] _mlre='eval __MODULES_QUAR_LD_LIBRARY_PATH='\''/usr/lib64/openmpi/lib'\'' LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' __MODULES_QUARANTINE_SET=1 '
++++ [23-01-29 20:05:52] eval '__MODULES_QUAR_LD_LIBRARY_PATH='\''/usr/lib64/openmpi/lib'\''' 'LD_LIBRARY_PATH='\'''\''' 'LD_PRELOAD='\'''\''' __MODULES_QUARANTINE_SET=1 /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash autoinit
+++++ [23-01-29 20:05:52] __MODULES_QUAR_LD_LIBRARY_PATH=/usr/lib64/openmpi/lib
+++++ [23-01-29 20:05:52] LD_LIBRARY_PATH=
+++++ [23-01-29 20:05:52] LD_PRELOAD=
+++++ [23-01-29 20:05:52] __MODULES_QUARANTINE_SET=1
+++++ [23-01-29 20:05:52] /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash autoinit
+++ [23-01-29 20:05:52] _mlcode='module() {
   eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
   _mlstatus=$?;
   return $_mlstatus;
};
ml() { module ml "$@"; };
MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl; export MODULES_CMD;
unset __MODULES_AUTOINIT_INPROGRESS;
MODULESHOME=/usr/share/Modules; export MODULESHOME;
test 0;'
+++ [23-01-29 20:05:52] _mlret=0
+++ [23-01-29 20:05:52] '[' -n x ']'
+++ [23-01-29 20:05:52] IFS=' 	
'
+++ [23-01-29 20:05:52] unset _mlIFS
+++ [23-01-29 20:05:52] unset _mlre _mlv _mlrv
+++ [23-01-29 20:05:52] '[' 0 -eq 0 ']'
+++ [23-01-29 20:05:52] eval 'module() {
   eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
   _mlstatus=$?;
   return $_mlstatus;
};
ml() { module ml "$@"; };
MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl; export MODULES_CMD;
unset __MODULES_AUTOINIT_INPROGRESS;
MODULESHOME=/usr/share/Modules; export MODULESHOME;
test 0;'
++++ [23-01-29 20:05:52] MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl
++++ [23-01-29 20:05:52] export MODULES_CMD
++++ [23-01-29 20:05:52] unset __MODULES_AUTOINIT_INPROGRESS
++++ [23-01-29 20:05:52] MODULESHOME=/usr/share/Modules
++++ [23-01-29 20:05:52] export MODULESHOME
++++ [23-01-29 20:05:52] test 0
+++ [23-01-29 20:05:52] '[' -t 2 ']'
+++ [23-01-29 20:05:52] export -f module
++++ [23-01-29 20:05:52] type -t ml
+++ [23-01-29 20:05:52] '[' function = function ']'
+++ [23-01-29 20:05:52] export -f ml
+++ [23-01-29 20:05:52] '[' 5 -ge 3 ']'
+++ [23-01-29 20:05:52] [[ hxB =~ i ]]
+++ [23-01-29 20:05:52] [[ ! :/usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin: =~ :/usr/share/Modules/bin: ]]
+++ [23-01-29 20:05:52] '[' '!' -n x ']'
+++ [23-01-29 20:05:52] [[ ! :/usr/share/man/openmpi-x86_64:/usr/share/man:: =~ :/usr/share/man: ]]
+++ [23-01-29 20:05:52] unset _mlcode _mlret
+++ [23-01-29 20:05:52] '[' -n '' ']'
+ [23-01-29 20:05:52] type module
module is a function
module () 
{ 
    eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
    _mlstatus=$?;
    return $_mlstatus
}
+ [23-01-29 20:05:52] module purge
++ [23-01-29 20:05:52] /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash purge
+ [23-01-29 20:05:52] eval unset 'MPI_COMPILER;' unset 'LD_LIBRARY_PATH;' 'MANPATH=/usr/share/man:;' export 'MANPATH;' unset 'MPI_INCLUDE;' unset 'MPI_MAN;' unset '_LMFILES_;' unset 'LOADEDMODULES;' unset 'PKG_CONFIG_PATH;' unset 'MPI_PYTHON3_SITEARCH;' unset 'MPI_HOME;' unset 'MPI_FORTRAN_MOD_DIR;' unset '__MODULES_LMCONFLICT;' '__MODULES_SHARE_MANPATH=:1;' export '__MODULES_SHARE_MANPATH;' unset 'MPI_SUFFIX;' unset 'MPI_SYSCONFIG;' unset 'MPI_LIB;' 'PATH=/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin;' export 'PATH;' unset 'MPI_BIN;' test '0;'
++ [23-01-29 20:05:52] unset MPI_COMPILER
++ [23-01-29 20:05:52] unset LD_LIBRARY_PATH
++ [23-01-29 20:05:52] MANPATH=/usr/share/man:
++ [23-01-29 20:05:52] export MANPATH
++ [23-01-29 20:05:52] unset MPI_INCLUDE
++ [23-01-29 20:05:52] unset MPI_MAN
++ [23-01-29 20:05:52] unset _LMFILES_
++ [23-01-29 20:05:52] unset LOADEDMODULES
++ [23-01-29 20:05:52] unset PKG_CONFIG_PATH
++ [23-01-29 20:05:52] unset MPI_PYTHON3_SITEARCH
++ [23-01-29 20:05:52] unset MPI_HOME
++ [23-01-29 20:05:52] unset MPI_FORTRAN_MOD_DIR
++ [23-01-29 20:05:52] unset __MODULES_LMCONFLICT
++ [23-01-29 20:05:52] __MODULES_SHARE_MANPATH=:1
++ [23-01-29 20:05:52] export __MODULES_SHARE_MANPATH
++ [23-01-29 20:05:52] unset MPI_SUFFIX
++ [23-01-29 20:05:52] unset MPI_SYSCONFIG
++ [23-01-29 20:05:52] unset MPI_LIB
++ [23-01-29 20:05:52] PATH=/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:05:52] export PATH
++ [23-01-29 20:05:52] unset MPI_BIN
++ [23-01-29 20:05:52] test 0
+ [23-01-29 20:05:52] _mlstatus=0
+ [23-01-29 20:05:52] return 0
+ [23-01-29 20:05:52] sed -e '/^module/d' -i /root/.bashrc
+ [23-01-29 20:05:52] dnf remove --noautoremove -y 'mpitests*' 'mpich*' 'mvapich*' 'openmpi*'
No match for argument: mpich*
No match for argument: mvapich*
Dependencies resolved.
================================================================================
 Package               Arch        Version            Repository           Size
================================================================================
Removing:
 mpitests-openmpi      x86_64      5.8-1.el9          @BUILDROOT-C9S      5.3 M
 openmpi               x86_64      1:4.1.1-5.el9      @BUILDROOT-C9S       10 M
 openmpi-devel         x86_64      1:4.1.1-5.el9      @BUILDROOT-C9S      1.9 M

Transaction Summary
================================================================================
Remove  3 Packages

Freed space: 17 M
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Erasing          : mpitests-openmpi-5.8-1.el9.x86_64                      1/3 
  Erasing          : openmpi-1:4.1.1-5.el9.x86_64                           2/3 
  Erasing          : openmpi-devel-1:4.1.1-5.el9.x86_64                     3/3 
  Running scriptlet: openmpi-devel-1:4.1.1-5.el9.x86_64                     3/3 
  Verifying        : mpitests-openmpi-5.8-1.el9.x86_64                      1/3 
  Verifying        : openmpi-1:4.1.1-5.el9.x86_64                           2/3 
  Verifying        : openmpi-devel-1:4.1.1-5.el9.x86_64                     3/3 

Removed:
  mpitests-openmpi-5.8-1.el9.x86_64         openmpi-1:4.1.1-5.el9.x86_64       
  openmpi-devel-1:4.1.1-5.el9.x86_64       

Complete!
+ [23-01-29 20:05:55] dnf install -y --setopt=strict=0 --nogpgcheck openmpi mpitests-openmpi environment-modules
Last metadata expiration check: 0:13:40 ago on Sun 29 Jan 2023 07:52:16 PM EST.
Package environment-modules-5.0.1-2.el9.x86_64 is already installed.
Dependencies resolved.
================================================================================
 Package               Arch        Version             Repository          Size
================================================================================
Installing:
 mpitests-openmpi      x86_64      5.8-1.el9           BUILDROOT-C9S      897 k
 openmpi               x86_64      1:4.1.1-5.el9       BUILDROOT-C9S      2.7 M
Installing dependencies:
 openmpi-devel         x86_64      1:4.1.1-5.el9       BUILDROOT-C9S      1.2 M

Transaction Summary
================================================================================
Install  3 Packages

Total download size: 4.7 M
Installed size: 17 M
Downloading Packages:
(1/3): openmpi-devel-4.1.1-5.el9.x86_64.rpm      10 MB/s | 1.2 MB     00:00    
(2/3): mpitests-openmpi-5.8-1.el9.x86_64.rpm    5.1 MB/s | 897 kB     00:00    
(3/3): openmpi-4.1.1-5.el9.x86_64.rpm           7.9 MB/s | 2.7 MB     00:00    
--------------------------------------------------------------------------------
Total                                            14 MB/s | 4.7 MB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Installing       : openmpi-devel-1:4.1.1-5.el9.x86_64                     1/3 
  Installing       : openmpi-1:4.1.1-5.el9.x86_64                           2/3 
  Installing       : mpitests-openmpi-5.8-1.el9.x86_64                      3/3 
  Running scriptlet: mpitests-openmpi-5.8-1.el9.x86_64                      3/3 
  Verifying        : mpitests-openmpi-5.8-1.el9.x86_64                      1/3 
  Verifying        : openmpi-1:4.1.1-5.el9.x86_64                           2/3 
  Verifying        : openmpi-devel-1:4.1.1-5.el9.x86_64                     3/3 

Installed:
  mpitests-openmpi-5.8-1.el9.x86_64         openmpi-1:4.1.1-5.el9.x86_64       
  openmpi-devel-1:4.1.1-5.el9.x86_64       

Complete!
+ [23-01-29 20:06:02] '[' -f /etc/profile.d/modules.sh ']'
+ [23-01-29 20:06:02] source /etc/profile.d/modules.sh
++ [23-01-29 20:06:02] '[' -n /usr/bin/bash ']'
++ [23-01-29 20:06:02] '[' bash = sh ']'
++ [23-01-29 20:06:02] shell=bash
++ [23-01-29 20:06:02] '[' -f /usr/share/Modules/init/bash ']'
++ [23-01-29 20:06:02] . /usr/share/Modules/init/bash
+++ [23-01-29 20:06:02] unset _mlshdbg
+++ [23-01-29 20:06:02] '[' 0 = 1 ']'
+++ [23-01-29 20:06:02] export 'MODULES_RUN_QUARANTINE=LD_LIBRARY_PATH LD_PRELOAD'
+++ [23-01-29 20:06:02] MODULES_RUN_QUARANTINE='LD_LIBRARY_PATH LD_PRELOAD'
+++ [23-01-29 20:06:02] unset _mlre _mlIFS
+++ [23-01-29 20:06:02] '[' -n x ']'
+++ [23-01-29 20:06:02] _mlIFS=' 	
'
+++ [23-01-29 20:06:02] IFS=' '
+++ [23-01-29 20:06:02] for _mlv in ${MODULES_RUN_QUARANTINE:-}
+++ [23-01-29 20:06:02] '[' LD_LIBRARY_PATH = LD_LIBRARY_PATH -a LD_LIBRARY_PATH = LD_LIBRARY_PATH ']'
++++ [23-01-29 20:06:02] eval 'echo ${LD_LIBRARY_PATH+x}'
+++++ [23-01-29 20:06:02] echo
+++ [23-01-29 20:06:02] '[' -n '' ']'
+++ [23-01-29 20:06:02] _mlrv=MODULES_RUNENV_LD_LIBRARY_PATH
++++ [23-01-29 20:06:02] eval 'echo ${MODULES_RUNENV_LD_LIBRARY_PATH:-}'
+++++ [23-01-29 20:06:02] echo
+++ [23-01-29 20:06:02] _mlre='LD_LIBRARY_PATH='\'''\'' '
+++ [23-01-29 20:06:02] for _mlv in ${MODULES_RUN_QUARANTINE:-}
+++ [23-01-29 20:06:02] '[' LD_PRELOAD = LD_PRELOAD -a LD_PRELOAD = LD_PRELOAD ']'
++++ [23-01-29 20:06:02] eval 'echo ${LD_PRELOAD+x}'
+++++ [23-01-29 20:06:02] echo
+++ [23-01-29 20:06:02] '[' -n '' ']'
+++ [23-01-29 20:06:02] _mlrv=MODULES_RUNENV_LD_PRELOAD
++++ [23-01-29 20:06:02] eval 'echo ${MODULES_RUNENV_LD_PRELOAD:-}'
+++++ [23-01-29 20:06:02] echo
+++ [23-01-29 20:06:02] _mlre='LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' '
+++ [23-01-29 20:06:02] '[' -n 'LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' ' ']'
+++ [23-01-29 20:06:02] _mlre='eval LD_LIBRARY_PATH='\'''\'' LD_PRELOAD='\'''\'' __MODULES_QUARANTINE_SET=1 '
++++ [23-01-29 20:06:02] eval 'LD_LIBRARY_PATH='\'''\''' 'LD_PRELOAD='\'''\''' __MODULES_QUARANTINE_SET=1 /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash autoinit
+++++ [23-01-29 20:06:02] LD_LIBRARY_PATH=
+++++ [23-01-29 20:06:02] LD_PRELOAD=
+++++ [23-01-29 20:06:02] __MODULES_QUARANTINE_SET=1
+++++ [23-01-29 20:06:02] /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash autoinit
+++ [23-01-29 20:06:02] _mlcode='module() {
   eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
   _mlstatus=$?;
   return $_mlstatus;
};
ml() { module ml "$@"; };
MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl; export MODULES_CMD;
unset __MODULES_AUTOINIT_INPROGRESS;
MODULESHOME=/usr/share/Modules; export MODULESHOME;
LOADEDMODULES=; export LOADEDMODULES;
test 0;'
+++ [23-01-29 20:06:02] _mlret=0
+++ [23-01-29 20:06:02] '[' -n x ']'
+++ [23-01-29 20:06:02] IFS=' 	
'
+++ [23-01-29 20:06:02] unset _mlIFS
+++ [23-01-29 20:06:02] unset _mlre _mlv _mlrv
+++ [23-01-29 20:06:02] '[' 0 -eq 0 ']'
+++ [23-01-29 20:06:02] eval 'module() {
   eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
   _mlstatus=$?;
   return $_mlstatus;
};
ml() { module ml "$@"; };
MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl; export MODULES_CMD;
unset __MODULES_AUTOINIT_INPROGRESS;
MODULESHOME=/usr/share/Modules; export MODULESHOME;
LOADEDMODULES=; export LOADEDMODULES;
test 0;'
++++ [23-01-29 20:06:02] MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl
++++ [23-01-29 20:06:02] export MODULES_CMD
++++ [23-01-29 20:06:02] unset __MODULES_AUTOINIT_INPROGRESS
++++ [23-01-29 20:06:02] MODULESHOME=/usr/share/Modules
++++ [23-01-29 20:06:02] export MODULESHOME
++++ [23-01-29 20:06:02] LOADEDMODULES=
++++ [23-01-29 20:06:02] export LOADEDMODULES
++++ [23-01-29 20:06:02] test 0
+++ [23-01-29 20:06:02] '[' -t 2 ']'
+++ [23-01-29 20:06:02] export -f module
++++ [23-01-29 20:06:02] type -t ml
+++ [23-01-29 20:06:02] '[' function = function ']'
+++ [23-01-29 20:06:02] export -f ml
+++ [23-01-29 20:06:02] '[' 5 -ge 3 ']'
+++ [23-01-29 20:06:02] [[ hxB =~ i ]]
+++ [23-01-29 20:06:02] [[ ! :/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin: =~ :/usr/share/Modules/bin: ]]
+++ [23-01-29 20:06:02] '[' '!' -n x ']'
+++ [23-01-29 20:06:02] [[ ! :/usr/share/man:: =~ :/usr/share/man: ]]
+++ [23-01-29 20:06:02] unset _mlcode _mlret
+++ [23-01-29 20:06:02] '[' -n '' ']'
+ [23-01-29 20:06:02] [[ openmpi == *\p\s\m\2* ]]
++ [23-01-29 20:06:02] rpm -ql openmpi
++ [23-01-29 20:06:02] grep modulefiles
++ [23-01-29 20:06:02] tail -n 1
+ [23-01-29 20:06:02] local mod_file=/usr/share/modulefiles/mpi/openmpi-x86_64
+ [23-01-29 20:06:02] echo 'module purge'
+ [23-01-29 20:06:02] echo 'module load /usr/share/modulefiles/mpi/openmpi-x86_64'
+ [23-01-29 20:06:02] source /root/.bashrc
+ [23-01-29 20:06:03] module list
++ [23-01-29 20:06:03] /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash list
Currently Loaded Modulefiles:
 1) /usr/share/modulefiles/mpi/openmpi-x86_64  
+ [23-01-29 20:06:03] eval
+ [23-01-29 20:06:03] _mlstatus=0
+ [23-01-29 20:06:03] return 0
++ [23-01-29 20:06:03] echo /usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:06:03] sed -e 's|/usr/lib64/openmpi[^/]*/bin:||g'
+ [23-01-29 20:06:03] PATH=/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:06:03] echo /usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:06:03] sed -e 's|/usr/lib64/mvapich[^/]*/bin:||g'
+ [23-01-29 20:06:03] PATH=/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:06:03] echo /usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
++ [23-01-29 20:06:03] sed -e 's|/usr/lib64/mpich[^/]*/bin:||g'
+ [23-01-29 20:06:03] PATH=/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
+ [23-01-29 20:06:03] export PATH=/usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
+ [23-01-29 20:06:03] PATH=/usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
+ [23-01-29 20:06:03] MPI_create_hostfiles
+ [23-01-29 20:06:03] '[' -z '' -o -z 172.31.40.31 ']'
+ [23-01-29 20:06:03] echo 'Please run /kernel/infiniband/env_setup before MPI_create_hostfiles'
Please run /kernel/infiniband/env_setup before MPI_create_hostfiles
+ [23-01-29 20:06:03] return 1
+ [23-01-29 20:06:03] echo 'MPI_setup: Failed to generate MPI hostfiles'
MPI_setup: Failed to generate MPI hostfiles
+ [23-01-29 20:06:03] echo 'MPI_setup: (This is expected on singlehost environment setup)'
MPI_setup: (This is expected on singlehost environment setup)
+ [23-01-29 20:06:03] '[' -x /bin/systemctl ']'
+ [23-01-29 20:06:03] systemctl stop firewalld.service
+ [23-01-29 20:06:03] systemctl disable firewalld.service
+ [23-01-29 20:06:03] systemctl status firewalld.service
○ firewalld.service - firewalld - dynamic firewall daemon
     Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; preset: enabled)
     Active: inactive (dead)
       Docs: man:firewalld(1)
+ [23-01-29 20:06:03] MPI_dual_port_fix
+ [23-01-29 20:06:03] [[ '' == *\q\i\b* ]]
++ [23-01-29 20:06:03] RQA_get_rhel_major
++ [23-01-29 20:06:03] awk -F . '{print $1}'
+++ [23-01-29 20:06:03] RQA_get_rhel_release
+++ [23-01-29 20:06:03] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:06:03] echo 9
+ [23-01-29 20:06:03] '[' 9 -ge 7 ']'
+ [23-01-29 20:06:03] RQA_pkg_install targetcli iscsi-initiator-utils lsscsi
+ [23-01-29 20:06:03] PKG_LIST=
+ [23-01-29 20:06:03] for p in "$@"
+ [23-01-29 20:06:03] rpm -q targetcli
targetcli-2.1.53-7.el9.noarch
+ [23-01-29 20:06:03] for p in "$@"
+ [23-01-29 20:06:03] rpm -q iscsi-initiator-utils
iscsi-initiator-utils-6.2.1.4-3.git2a8f9d8.el9.x86_64
+ [23-01-29 20:06:03] for p in "$@"
+ [23-01-29 20:06:03] rpm -q lsscsi
lsscsi-0.32-6.el9.x86_64
+ [23-01-29 20:06:03] '[' '!' -z '' ']'
+ [23-01-29 20:06:03] target_name=iqn.2003-01.org.linux-iscsi.rdma.x8664:test-target
++ [23-01-29 20:06:03] RQA_get_rhel_major
++ [23-01-29 20:06:03] awk -F . '{print $1}'
+++ [23-01-29 20:06:03] RQA_get_rhel_release
+++ [23-01-29 20:06:03] grep -o '[0-9]*\.*[0-9]*' /etc/redhat-release
++ [23-01-29 20:06:03] echo 9
+ [23-01-29 20:06:03] [[ 9 -ge 8 ]]
+ [23-01-29 20:06:03] RQA_pkg_install iperf3
+ [23-01-29 20:06:03] PKG_LIST=
+ [23-01-29 20:06:03] for p in "$@"
+ [23-01-29 20:06:03] rpm -q iperf3
iperf3-3.9-9.el9.x86_64
+ [23-01-29 20:06:04] '[' '!' -z '' ']'
+ [23-01-29 20:06:04] RQA_pkg_install net-tools
+ [23-01-29 20:06:04] PKG_LIST=
+ [23-01-29 20:06:04] for p in "$@"
+ [23-01-29 20:06:04] rpm -q net-tools
net-tools-2.0-0.62.20160912git.el9.x86_64
+ [23-01-29 20:06:04] '[' '!' -z '' ']'
+ [23-01-29 20:06:04] [[ server == \c\o\m\m\o\n ]]
+ [23-01-29 20:06:04] [[ server == \s\e\r\v\e\r ]]
+ [23-01-29 20:06:04] server_tests
+ [23-01-29 20:06:04] [[ -z 172.31.40.31 ]]
+ [23-01-29 20:06:04] timeout 3m ping -i 0.2 -c 100 172.31.40.31
PING 172.31.40.31 (172.31.40.31) 56(84) bytes of data.
64 bytes from 172.31.40.31: icmp_seq=1 ttl=64 time=0.183 ms
64 bytes from 172.31.40.31: icmp_seq=2 ttl=64 time=0.143 ms
64 bytes from 172.31.40.31: icmp_seq=3 ttl=64 time=0.130 ms
64 bytes from 172.31.40.31: icmp_seq=4 ttl=64 time=0.126 ms
64 bytes from 172.31.40.31: icmp_seq=5 ttl=64 time=0.113 ms
64 bytes from 172.31.40.31: icmp_seq=6 ttl=64 time=0.115 ms
64 bytes from 172.31.40.31: icmp_seq=7 ttl=64 time=0.131 ms
64 bytes from 172.31.40.31: icmp_seq=8 ttl=64 time=0.116 ms
64 bytes from 172.31.40.31: icmp_seq=9 ttl=64 time=0.129 ms
64 bytes from 172.31.40.31: icmp_seq=10 ttl=64 time=0.193 ms
64 bytes from 172.31.40.31: icmp_seq=11 ttl=64 time=0.116 ms
64 bytes from 172.31.40.31: icmp_seq=12 ttl=64 time=0.126 ms
64 bytes from 172.31.40.31: icmp_seq=13 ttl=64 time=0.121 ms
64 bytes from 172.31.40.31: icmp_seq=14 ttl=64 time=0.121 ms
64 bytes from 172.31.40.31: icmp_seq=15 ttl=64 time=0.134 ms
64 bytes from 172.31.40.31: icmp_seq=16 ttl=64 time=0.130 ms
64 bytes from 172.31.40.31: icmp_seq=17 ttl=64 time=0.111 ms
64 bytes from 172.31.40.31: icmp_seq=18 ttl=64 time=0.120 ms
64 bytes from 172.31.40.31: icmp_seq=19 ttl=64 time=0.101 ms
64 bytes from 172.31.40.31: icmp_seq=20 ttl=64 time=0.119 ms
64 bytes from 172.31.40.31: icmp_seq=21 ttl=64 time=0.117 ms
64 bytes from 172.31.40.31: icmp_seq=22 ttl=64 time=0.106 ms
64 bytes from 172.31.40.31: icmp_seq=23 ttl=64 time=0.121 ms
64 bytes from 172.31.40.31: icmp_seq=24 ttl=64 time=0.119 ms
64 bytes from 172.31.40.31: icmp_seq=25 ttl=64 time=0.100 ms
64 bytes from 172.31.40.31: icmp_seq=26 ttl=64 time=0.113 ms
64 bytes from 172.31.40.31: icmp_seq=27 ttl=64 time=0.138 ms
64 bytes from 172.31.40.31: icmp_seq=28 ttl=64 time=0.128 ms
64 bytes from 172.31.40.31: icmp_seq=29 ttl=64 time=0.109 ms
64 bytes from 172.31.40.31: icmp_seq=30 ttl=64 time=0.157 ms
64 bytes from 172.31.40.31: icmp_seq=31 ttl=64 time=0.130 ms
64 bytes from 172.31.40.31: icmp_seq=32 ttl=64 time=0.129 ms
64 bytes from 172.31.40.31: icmp_seq=33 ttl=64 time=0.129 ms
64 bytes from 172.31.40.31: icmp_seq=34 ttl=64 time=0.140 ms
64 bytes from 172.31.40.31: icmp_seq=35 ttl=64 time=0.131 ms
64 bytes from 172.31.40.31: icmp_seq=36 ttl=64 time=0.134 ms
64 bytes from 172.31.40.31: icmp_seq=37 ttl=64 time=0.112 ms
64 bytes from 172.31.40.31: icmp_seq=38 ttl=64 time=0.113 ms
64 bytes from 172.31.40.31: icmp_seq=39 ttl=64 time=0.128 ms
64 bytes from 172.31.40.31: icmp_seq=40 ttl=64 time=0.127 ms
64 bytes from 172.31.40.31: icmp_seq=41 ttl=64 time=0.127 ms
64 bytes from 172.31.40.31: icmp_seq=42 ttl=64 time=0.133 ms
64 bytes from 172.31.40.31: icmp_seq=43 ttl=64 time=0.113 ms
64 bytes from 172.31.40.31: icmp_seq=44 ttl=64 time=0.131 ms
64 bytes from 172.31.40.31: icmp_seq=45 ttl=64 time=0.131 ms
64 bytes from 172.31.40.31: icmp_seq=46 ttl=64 time=0.131 ms
64 bytes from 172.31.40.31: icmp_seq=47 ttl=64 time=0.124 ms
64 bytes from 172.31.40.31: icmp_seq=48 ttl=64 time=0.113 ms
64 bytes from 172.31.40.31: icmp_seq=49 ttl=64 time=0.140 ms
64 bytes from 172.31.40.31: icmp_seq=50 ttl=64 time=0.118 ms
64 bytes from 172.31.40.31: icmp_seq=51 ttl=64 time=0.125 ms
64 bytes from 172.31.40.31: icmp_seq=52 ttl=64 time=0.134 ms
64 bytes from 172.31.40.31: icmp_seq=53 ttl=64 time=0.129 ms
64 bytes from 172.31.40.31: icmp_seq=54 ttl=64 time=0.133 ms
64 bytes from 172.31.40.31: icmp_seq=55 ttl=64 time=0.128 ms
64 bytes from 172.31.40.31: icmp_seq=56 ttl=64 time=0.127 ms
64 bytes from 172.31.40.31: icmp_seq=57 ttl=64 time=0.127 ms
64 bytes from 172.31.40.31: icmp_seq=58 ttl=64 time=0.118 ms
64 bytes from 172.31.40.31: icmp_seq=59 ttl=64 time=0.122 ms
64 bytes from 172.31.40.31: icmp_seq=60 ttl=64 time=0.131 ms
64 bytes from 172.31.40.31: icmp_seq=61 ttl=64 time=0.131 ms
64 bytes from 172.31.40.31: icmp_seq=62 ttl=64 time=0.127 ms
64 bytes from 172.31.40.31: icmp_seq=63 ttl=64 time=0.130 ms
64 bytes from 172.31.40.31: icmp_seq=64 ttl=64 time=0.130 ms
64 bytes from 172.31.40.31: icmp_seq=65 ttl=64 time=0.130 ms
64 bytes from 172.31.40.31: icmp_seq=66 ttl=64 time=0.128 ms
64 bytes from 172.31.40.31: icmp_seq=67 ttl=64 time=0.112 ms
64 bytes from 172.31.40.31: icmp_seq=68 ttl=64 time=0.123 ms
64 bytes from 172.31.40.31: icmp_seq=69 ttl=64 time=0.129 ms
64 bytes from 172.31.40.31: icmp_seq=70 ttl=64 time=0.134 ms
64 bytes from 172.31.40.31: icmp_seq=71 ttl=64 time=0.127 ms
64 bytes from 172.31.40.31: icmp_seq=72 ttl=64 time=0.112 ms
64 bytes from 172.31.40.31: icmp_seq=73 ttl=64 time=0.132 ms
64 bytes from 172.31.40.31: icmp_seq=74 ttl=64 time=0.141 ms
64 bytes from 172.31.40.31: icmp_seq=75 ttl=64 time=0.133 ms
64 bytes from 172.31.40.31: icmp_seq=76 ttl=64 time=0.135 ms
64 bytes from 172.31.40.31: icmp_seq=77 ttl=64 time=0.134 ms
64 bytes from 172.31.40.31: icmp_seq=78 ttl=64 time=0.133 ms
64 bytes from 172.31.40.31: icmp_seq=79 ttl=64 time=0.130 ms
64 bytes from 172.31.40.31: icmp_seq=80 ttl=64 time=0.128 ms
64 bytes from 172.31.40.31: icmp_seq=81 ttl=64 time=0.123 ms
64 bytes from 172.31.40.31: icmp_seq=82 ttl=64 time=0.131 ms
64 bytes from 172.31.40.31: icmp_seq=83 ttl=64 time=0.113 ms
64 bytes from 172.31.40.31: icmp_seq=84 ttl=64 time=0.131 ms
64 bytes from 172.31.40.31: icmp_seq=85 ttl=64 time=0.129 ms
64 bytes from 172.31.40.31: icmp_seq=86 ttl=64 time=0.136 ms
64 bytes from 172.31.40.31: icmp_seq=87 ttl=64 time=0.138 ms
64 bytes from 172.31.40.31: icmp_seq=88 ttl=64 time=0.131 ms
64 bytes from 172.31.40.31: icmp_seq=89 ttl=64 time=0.132 ms
64 bytes from 172.31.40.31: icmp_seq=90 ttl=64 time=0.119 ms
64 bytes from 172.31.40.31: icmp_seq=91 ttl=64 time=0.121 ms
64 bytes from 172.31.40.31: icmp_seq=92 ttl=64 time=0.128 ms
64 bytes from 172.31.40.31: icmp_seq=93 ttl=64 time=0.128 ms
64 bytes from 172.31.40.31: icmp_seq=94 ttl=64 time=0.140 ms
64 bytes from 172.31.40.31: icmp_seq=95 ttl=64 time=0.128 ms
64 bytes from 172.31.40.31: icmp_seq=96 ttl=64 time=0.114 ms
64 bytes from 172.31.40.31: icmp_seq=97 ttl=64 time=0.129 ms
64 bytes from 172.31.40.31: icmp_seq=98 ttl=64 time=0.134 ms
64 bytes from 172.31.40.31: icmp_seq=99 ttl=64 time=0.129 ms
64 bytes from 172.31.40.31: icmp_seq=100 ttl=64 time=0.132 ms

--- 172.31.40.31 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 20589ms
rtt min/avg/max/mdev = 0.100/0.127/0.193/0.012 ms
+ [23-01-29 20:06:24] RQA_check_result -r 0 -t 'ping client - 172.31.40.31'
+ [23-01-29 20:06:24] local test_pass=0
+ [23-01-29 20:06:24] local test_skip=777
+ [23-01-29 20:06:24] test 4 -gt 0
+ [23-01-29 20:06:24] case $1 in
+ [23-01-29 20:06:24] local rc=0
+ [23-01-29 20:06:24] shift
+ [23-01-29 20:06:24] shift
+ [23-01-29 20:06:24] test 2 -gt 0
+ [23-01-29 20:06:24] case $1 in
+ [23-01-29 20:06:24] local 'msg=ping client - 172.31.40.31'
+ [23-01-29 20:06:24] shift
+ [23-01-29 20:06:24] shift
+ [23-01-29 20:06:24] test 0 -gt 0
+ [23-01-29 20:06:24] '[' -z 0 -o -z 'ping client - 172.31.40.31' ']'
+ [23-01-29 20:06:24] '[' -z '' ']'
+ [23-01-29 20:06:24] source /etc/environment
++ [23-01-29 20:06:24] TEST_RESULT_FILE=/tmp/tmp.GcKXytvlEd/results_cki-sanity.txt
+ [23-01-29 20:06:24] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:06:24] '[' 0 -eq 0 ']'
+ [23-01-29 20:06:24] local test_result=PASS
+ [23-01-29 20:06:24] printf '%10s | %6s | %s\n' PASS 0 'ping client - 172.31.40.31'
+ [23-01-29 20:06:24] set +x
---
- TEST RESULT FOR sanity
-   Test:   ping client - 172.31.40.31
-   Result: PASS
-   Return: 0
---
+ [23-01-29 20:06:24] timeout 3m ping6 -i 0.2 -c 100 fe80::20a:f7ff:fe97:b162%
ping6: fe80::20a:f7ff:fe97:b162%: Name or service not known
+ [23-01-29 20:06:24] RQA_check_result -r 2 -t 'ping6 client - fe80::20a:f7ff:fe97:b162%'
+ [23-01-29 20:06:24] local test_pass=0
+ [23-01-29 20:06:24] local test_skip=777
+ [23-01-29 20:06:24] test 4 -gt 0
+ [23-01-29 20:06:24] case $1 in
+ [23-01-29 20:06:24] local rc=2
+ [23-01-29 20:06:24] shift
+ [23-01-29 20:06:24] shift
+ [23-01-29 20:06:24] test 2 -gt 0
+ [23-01-29 20:06:24] case $1 in
+ [23-01-29 20:06:24] local 'msg=ping6 client - fe80::20a:f7ff:fe97:b162%'
+ [23-01-29 20:06:24] shift
+ [23-01-29 20:06:24] shift
+ [23-01-29 20:06:24] test 0 -gt 0
+ [23-01-29 20:06:24] '[' -z 2 -o -z 'ping6 client - fe80::20a:f7ff:fe97:b162%' ']'
+ [23-01-29 20:06:24] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:06:24] '[' -z /tmp/tmp.GcKXytvlEd/results_cki-sanity.txt ']'
+ [23-01-29 20:06:24] '[' 2 -eq 0 ']'
+ [23-01-29 20:06:24] '[' 2 -eq 777 ']'
+ [23-01-29 20:06:24] local test_result=FAIL
+ [23-01-29 20:06:24] export result=FAIL
+ [23-01-29 20:06:24] result=FAIL
+ [23-01-29 20:06:24] [[ ! -z '' ]]
+ [23-01-29 20:06:24] printf '%10s | %6s | %s\n' FAIL 2 'ping6 client - fe80::20a:f7ff:fe97:b162%'
+ [23-01-29 20:06:24] set +x
---
- TEST RESULT FOR sanity
-   Test:   ping6 client - fe80::20a:f7ff:fe97:b162%
-   Result: FAIL
-   Return: 2
---
+ [23-01-29 20:06:24] RQA_skip_test mpi
+ [23-01-29 20:06:24] local skp=mpi
+ [23-01-29 20:06:24] local skp_rtn=0
+ [23-01-29 20:06:24] '[' '!' -z mpi ']'
+ [23-01-29 20:06:24] [[ '' == *\m\p\i* ]]
+ [23-01-29 20:06:24] return 0
+ [23-01-29 20:06:24] skip_mpi=0
+ [23-01-29 20:06:24] '[' 0 -eq 0 ']'
+ [23-01-29 20:06:24] source /root/.bashrc
+ [23-01-29 20:06:25] rhts_sync_set -s mpi_ready_--0
/usr/bin/rhts_sync_set -s mpi_ready_--0 -- State set = 12_mpi_ready_--0
+ [23-01-29 20:06:25] rhts_sync_block -s mpi_done_--0 rdma-qe-31
/usr/bin/rhts_sync_block -s mpi_done_--0 rdma-qe-31 -- Blocking state(s) =  12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 0 bytes, ret 0 [0, 0]
Retrying rstrnt-sync-block for state 12_mpi_done_--0
Server rdma-qe-31 not reported state 12_mpi_done_--0 for Multihost Sync rcvd 5 bytes, ret 0 [1, 0]