What’s a Link Based IPMP ?
The failure detection and repair method used by the mpathd daemon differentiates the IPMP as probe based or link based. In case of link based IPMP :
– in.mpathd daemon observes the changes to IFF_RUNNING flag on the interface to determine failure.
– No test addresses required for failure detection.
– Enabled by default (if supported by the interface).
– One of the advantage of link based IPMP is it does not depend on external sources to send ICMP reply to ensure the link status and it also saves IP addresses as it is not require any test addresses for failure detection
mpathd Configuration file
mpathd daemon is responsible to detect an interface failure . It uses a configuration file /etc/default/mpathd to set various IPMP parameters.
# cat /etc/default/mpathd # #pragma ident "@(#)mpathd.dfl 1.2 00/07/17 SMI" # # Time taken by mpathd to detect a NIC failure in ms. The minimum time # that can be specified is 100 ms. # FAILURE_DETECTION_TIME=10000 # # Failback is enabled by default. To disable failback turn off this option # FAILBACK=yes # # By default only interfaces configured as part of multipathing groups # are tracked. Turn off this option to track all network interfaces # on the system # TRACK_INTERFACES_ONLY_WITH_GROUPS=yes
The important parameters in mpathd configuration file are :
1. FAILURE_DETECTION_TIME : Time taken by mpathd to detect a NIC failure in ms (default value – 10 seconds)
2. FAILBACK : To enable or disable failback after the failed link becomes available (default vaule – yes)
3. TRACK_INTERFACES_ONLY_WITH_GROUPS – If turned on interfaces configured as part of IPMP are only monitored (default vaule – yes)
The command for in.mpathd daemon to re-read the configuration file is :
# pkill -HUP in.mpathd
Meanings of FLAGs
You would see flags such as NOFAILOVER, DEPRECATED, STANDBY etc.. in the output of “ifconfig -a” command. The meanings of these flags and parameters to enable them are:
-failover -> does not failover when the interface fails
standby -> makes the interface to be used as standby
Testing IPMP failover
We can check the failure and repair of an interface very easily using if_mpadm command. “-d” detaches the interface whereas “-r” reattaches it.
# if_mpadm -d ce0 # if_mpadm -r ce0
Most commonly used Link-Based IPMP configurations
1. Single interface Link based IPMP configuration
This is not a very useful configuration, as it does not actually provide high availability. But can be used only to get intimated when an interface is failed.
Command line :
# ifconfig e1000g0 plumb 192.168.1.2 netmask + broadcast + group IPMPgroup up
For persistent configuration across reboots edit the files :
/etc/hostname.e1000g0 192.168.1.2 netmask + broadcast + group IPMPgroup up
Before failure :
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 13
inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
groupname IPMPgroup
ether 0:c:29:f6:ef:67
After failure :
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=11000803[UP,BROADCAST,MULTICAST,IPv4,FAILED] mtu 1500 index 13
inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
groupname IPMPgroup
ether 0:c:29:f6:ef:67
2. Multiple interface Link based IPMP configuration
a. Active – Active configuration
Command line :
# ifconfig e1000g0 plumb 192.168.1.2 netmask + broadcast + group IPMPgroup up # ifconfig e1000g1 plumb group IPMPgroup up
For persistent configuration across reboots edit the files :
/etc/hostname.e1000g0 192.168.1.2 netmask + broadcast + group IPMPgroup up /etc/hostname.e1000g1 group IPMPgroup up
Before Failure :
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 14
inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
groupname IPMPgroup
ether 0:c:29:f6:ef:67
e1000g1: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 15
inet 0.0.0.0 netmask ff000000
groupname IPMPgroup
ether 0:c:29:f6:ef:71
After Failure
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=19000802[BROADCAST,MULTICAST,IPv4,NOFAILOVER,FAILED] mtu 0 index 14
inet 0.0.0.0 netmask 0
groupname IPMPgroup
ether 0:c:29:f6:ef:67
e1000g1: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 15
inet 0.0.0.0 netmask ff000000
groupname IPMPgroup
ether 0:c:29:f6:ef:71
e1000g1:1: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 15
inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
b. Active standby Configuration
Command line :
# ifconfig e1000g0 plumb 192.168.1.2 netmask + broadcast + group IPMPgroup up # ifconfig e1000g1 plumb group IPMPgroup standby up
For persistent configuration across reboots edit the files :
/etc/hostname.e1000g0 192.168.1.2 netmask + broadcast + group IPMPgroup up /etc/hostname.e1000g1 group IPMPgroup standby up
Before failure
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 20
inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
groupname IPMPgroup
ether 0:c:29:f6:ef:67
e1000g0:1: flags=1000842[BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 20
inet 0.0.0.0 netmask 0
e1000g1: flags=69000842[BROADCAST,RUNNING,MULTICAST,IPv4,NOFAILOVER,STANDBY,INACTIVE] mtu 0 index 21
inet 0.0.0.0 netmask 0
groupname IPMPgroup
ether 0:c:29:f6:ef:71
After failure
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=19000802[BROADCAST,MULTICAST,IPv4,NOFAILOVER,FAILED] mtu 0 index 20
inet 0.0.0.0 netmask 0
groupname IPMPgroup
ether 0:c:29:f6:ef:67
e1000g1: flags=21000842[BROADCAST,RUNNING,MULTICAST,IPv4,STANDBY] mtu 1500 index 21
inet 0.0.0.0 netmask 0
groupname IPMPgroup
ether 0:c:29:f6:ef:71
e1000g1:1: flags=21000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4,STANDBY] mtu 1500 index 21
inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255