The Geek Diary

HowTos | Basics | Concepts

Main navigation

How to configure Solaris 10 Link Based IPMP

By

How to configure Solaris 10 Probe based IPMP
Troubleshooting Solaris 10 IPMP

What’s a Link Based IPMP ?

The failure detection and repair method used by the mpathd daemon differentiates the IPMP as probe based or link based. In case of link based IPMP :

– The mpathd daemon uses the interface kernel driver to check the status of the interface.
– in.mpathd daemon observes the changes to IFF_RUNNING flag on the interface to determine failure.
– No test addresses required for failure detection.
– Enabled by default (if supported by the interface).
– One of the advantage of link based IPMP is it does not depend on external sources to send ICMP reply to ensure the link status and it also saves IP addresses as it is not require any test addresses for failure detection

mpathd Configuration file

mpathd daemon is responsible to detect an interface failure . It uses a configuration file /etc/default/mpathd to set various IPMP parameters.

# cat /etc/default/mpathd
#
#pragma ident   "@(#)mpathd.dfl 1.2     00/07/17 SMI"
#
# Time taken by mpathd to detect a NIC failure in ms. The minimum time
# that can be specified is 100 ms.
#
FAILURE_DETECTION_TIME=10000
#
# Failback is enabled by default. To disable failback turn off this option
#
FAILBACK=yes
#
# By default only interfaces configured as part of multipathing groups
# are tracked. Turn off this option to track all network interfaces
# on the system
#
TRACK_INTERFACES_ONLY_WITH_GROUPS=yes

The important parameters in mpathd configuration file are :
1. FAILURE_DETECTION_TIME : Time taken by mpathd to detect a NIC failure in ms (default value – 10 seconds)
2. FAILBACK : To enable or disable failback after the failed link becomes available (default vaule – yes)
3. TRACK_INTERFACES_ONLY_WITH_GROUPS – If turned on interfaces configured as part of IPMP are only monitored (default vaule – yes)

The command for in.mpathd daemon to re-read the configuration file is :

# pkill -HUP in.mpathd

Meanings of FLAGs

You would see flags such as NOFAILOVER, DEPRECATED, STANDBY etc.. in the output of “ifconfig -a” command. The meanings of these flags and parameters to enable them are:

deprecated -> can only be used as test address for IPMP and not for any actual data transfer by applications.
-failover -> does not failover when the interface fails
standby -> makes the interface to be used as standby

Testing IPMP failover
We can check the failure and repair of an interface very easily using if_mpadm command. “-d” detaches the interface whereas “-r” reattaches it.

# if_mpadm -d ce0
# if_mpadm -r ce0

Most commonly used Link-Based IPMP configurations

1. Single interface Link based IPMP configuration
This is not a very useful configuration, as it does not actually provide high availability. But can be used only to get intimated when an interface is failed.
Command line :

# ifconfig e1000g0 plumb 192.168.1.2 netmask + broadcast + group IPMPgroup up

For persistent configuration across reboots edit the files :

/etc/hostname.e1000g0
192.168.1.2 netmask + broadcast + group IPMPgroup up

Before failure :

# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 13
        inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
        groupname IPMPgroup
        ether 0:c:29:f6:ef:67

After failure :

# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=11000803[UP,BROADCAST,MULTICAST,IPv4,FAILED] mtu 1500 index 13
        inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
        groupname IPMPgroup
        ether 0:c:29:f6:ef:67

2. Multiple interface Link based IPMP configuration
a. Active – Active configuration
Command line :

# ifconfig e1000g0 plumb 192.168.1.2 netmask + broadcast + group IPMPgroup up
# ifconfig e1000g1 plumb group IPMPgroup up

For persistent configuration across reboots edit the files :

/etc/hostname.e1000g0 
192.168.1.2 netmask + broadcast + group IPMPgroup up
/etc/hostname.e1000g1 
group IPMPgroup up

Before Failure :

# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 14
        inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
        groupname IPMPgroup
        ether 0:c:29:f6:ef:67
e1000g1: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 15
        inet 0.0.0.0 netmask ff000000
        groupname IPMPgroup
        ether 0:c:29:f6:ef:71

After Failure

# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=19000802[BROADCAST,MULTICAST,IPv4,NOFAILOVER,FAILED] mtu 0 index 14
        inet 0.0.0.0 netmask 0
        groupname IPMPgroup
        ether 0:c:29:f6:ef:67
e1000g1: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 15
        inet 0.0.0.0 netmask ff000000
        groupname IPMPgroup
        ether 0:c:29:f6:ef:71
e1000g1:1: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 15
        inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255

b. Active standby Configuration
Command line :

# ifconfig e1000g0 plumb 192.168.1.2 netmask + broadcast + group IPMPgroup up
# ifconfig e1000g1 plumb group IPMPgroup standby up

For persistent configuration across reboots edit the files :

/etc/hostname.e1000g0
192.168.1.2 netmask + broadcast + group IPMPgroup up
/etc/hostname.e1000g1
group IPMPgroup standby up

Before failure

# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 20
        inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
        groupname IPMPgroup
        ether 0:c:29:f6:ef:67
e1000g0:1: flags=1000842[BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 20
        inet 0.0.0.0 netmask 0
e1000g1: flags=69000842[BROADCAST,RUNNING,MULTICAST,IPv4,NOFAILOVER,STANDBY,INACTIVE] mtu 0 index 21
        inet 0.0.0.0 netmask 0
        groupname IPMPgroup
        ether 0:c:29:f6:ef:71

After failure

# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=19000802[BROADCAST,MULTICAST,IPv4,NOFAILOVER,FAILED] mtu 0 index 20
        inet 0.0.0.0 netmask 0
        groupname IPMPgroup
        ether 0:c:29:f6:ef:67
e1000g1: flags=21000842[BROADCAST,RUNNING,MULTICAST,IPv4,STANDBY] mtu 1500 index 21
        inet 0.0.0.0 netmask 0
        groupname IPMPgroup
        ether 0:c:29:f6:ef:71
e1000g1:1: flags=21000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4,STANDBY] mtu 1500 index 21
        inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
How to configure Solaris 10 Probe based IPMP
Troubleshooting Solaris 10 IPMP