The Problem
iscsiadm discovery times out when both the interfaces are up, but it works fine with single interface up.
# iscsiadm -m discovery -t st -p x.x.x.x iscsiadm: connect to x.x.x.x timed out iscsiadm: connect to x.x.x.x timed out
The Solution
This issue could occur when two or more Ethernet interfaces are connected to the same subnet or when iSCSI connection is configured to use any specific interface. In such cases when packets are delivered to the wrong interface then the OS will think it is sent to closed port or wrong destination and times out. In order to resolve this issue, we need to configure to accept asymmetrically routed packets on both interfaces of iSCSI client on which we are getting this error message:
1. Set sysctl’s net.ipv4.conf.all.arp_ignore to 1 to force each interface to only answer ARP requests for its own addresses. Or set net.ipv4.conf.[INTERFACE].arp_ignore to 1 for each interface that is a path to the iscsi storage.
For example, in /etc/sysctl.conf:
# vi /etc/sysctl.conf net.ipv4.conf.all.rp_filter=0 net.ipv4.conf.all.arp_ignore=1 net.ipv4.conf.all.arp_announce=2
Or, to only set it for the relevant interfaces (assuming they are eth0 and eth1):
# vi /etc/sysctl.conf net.ipv4.conf.eth0.rp_filter=0 net.ipv4.conf.eth0.arp_ignore = 1 net.ipv4.conf.eth0.arp_announce=2 net.ipv4.conf.eth1.rp_filter=0 net.ipv4.conf.eth1.arp_ignore = 1 net.ipv4.conf.eth1.arp_announce=2
2. Apply the setting using the below command:
# sysctl -p /etc/sysctl.conf
3. Check if the new settings applied successfully running:
# sysctl -a
4. Now, remove old files from /var/lib/iscsi/ifaces:
# rm -rf /var/lib/iscsi/ifaces
This should resolve iscsi discovery timeout when two interfaces are configured.
For CentOS/RHEL 7
Note that if the system is having CentOS/RHEL 7 installed, then please enter above sysctl parameters in /etc/sysctl.d/99-sysctl.conf file and then run below command to reread the settings from configuration file:
# sysctl --system
Notes
This problem can occur if an iSCSI connection is configured to use a specific network interface. If iscsi packets are delivered to the wrong interface, Linux will respond as if the data was sent to a closed port and send an RST packet. As long as the received data is coming in the wrong interface, the connection will remain broken and cannot be restored.
This can occur when multiple adapters are connected to the same subnet. By default, Linux will answer all ARP requests for any of the system’s IP addresses regardless of which interface the ARP request came from. If a different interface answers the ARP request for the iscsi connection’s IP address, traffic will be routed to the wrong interface and the connection will fail. Turning on arp_ignore forces a network interface to only answer ARP requests for its own addresses. This prevents the ARP answer from the wrong interface and keeps the iscsi connection functioning.