IPMP

 

Terminology note: The terms "NIC" "adapter" and "interface" are interchangeable for purposes of this discussion.

 

IPMP allows traffic addressed to an IP address associated with an interface to be moved to a backup interface if the connection through the primary interface fails. One or more interfaces are configured to be backups for another interface and when failure occurs, the IP address of the failed interface is transferred to the backup interface. When the primary interface is recovered, IPMP can automatically fail back the IP address.   IPMP can also be used to spread load over multiple interfaces attached to the same hub or switch. Although IPMP was offered in the later releases of Solaris 8, it is completely implemented in Solaris 9 and 10.  It requires two interfaces with different MAC addresses (so certain, now-rare, older interfaces won't work, and the PROM parameter local-mac-address? must be set to "true") and they must be attached to the same subnet. The MAC addresses must be different or there would be conflict as both interfaces intercepted the same traffic. 

 

To set up IPMP, all adapters/interfaces to be configured must be assigned to an IPMP group.  The group name is a Unix word that is largely the choice of the system administrator, and each group of primary and standby interfaces on a single system must belong to their own group.  A device capable of responding to a "ping" ICMP request must additionally be present on the network - this must be a device other than the system on which the IPMP group is configured and is generally the router.  Each adapter in an IPMP group will have a second logical interface configured with its own address and used for testing. This address will be deprecated, so it cannot actually accept traffic. It will also not be part of the IPMP group. The logical interface exists so that in.mpathd has an IP address associated with the physical interface that it can use to test the interface even after failover, since the main address has been failed over and can no longer be used to route traffic to the original interface. Once the interface responds to a ping request again, failback can occur. The logical interface only exists to enable failback.

 

IPMP is managed by the daemon in.mpathd, which monitors the state of interfaces using ping and by watching the flag IFF_RUNNING, which is set to 1 if the interface is "up."

If either indicator shows that the interface has failed, in.mpathd will determine that the interface is down and cause failover.  in.mpathd checks the IPMP primary and standby interfaces by sending "pings" to the default router on the network through the test interfaces.  If no reply is heard in a default period of 10 seconds, another ping is sent. After five failures, in.mpathd determines that the interface has failed, and failover occurs if the failed interface is the primary interface. in.mpathd continues to attempt to ping another host on the network through the failed interface; if communication is restored, failback to that interface can then occur.  The reply period can be configured in the file /etc/default/mpathd. The variable FAILURE_DETECTION_TIME is set to 10000 milliseconds - it can be reset to any number of milliseconds desired.  Failback is enabled in the same file with the FAILBACK=yes variable setting.

 

If the default router is down, none of the interfaces in the IPMP group will get a reply to the "ping," and no failover will occur. in.mpathd will then ping the multicast-to-all address and choose a host from those that respond. It will then again attempt to test the interfaces in the group by pinging the responding host. This prevents IPMP from performing failover because a router has been lost.

 

Configuring IPMP -

 

Requirements: 2 or more adapters (interfaces) on the same network

At least one other active host on that network

Solaris 8, release 10/00 or higher, configured in /etc/release

local-mac-address?=true

 

1. Set up two IP addresses in /etc/inet/hosts  for each interface.

One interface will probably have an IP address already. Each IP address must be associated with a unique name.

 

2. Set up the files /etc/hostname.<interface> for each of the two interfaces. In these files include a line with the hostname for the main IP address, and 2 lines stating:

<main_hostname>  netmask   +  broadcast  +  group <groupname>  up

addif  <test-hostname>  deprecated netmask  + broadcast  +  -failover up

 

The first line configures the main interface and adds it to the IPMP group <groupname>  The second line adds a virtual interface, deprecates it, and specifies that interface cannot fail over to a backup address.  Deprecating the interface prevents it from being used to send packets unless no other interface is available or the packets are specifically sent to that interface

 

Reboot. The daemon in.mpathd will start automatically

 

It is also possible to set up IPMP using ifconfig rather than setting up the hostname.<interface> files and rebooting. The following commands assume two adapters which are currently unconfigured:

 

 

ifconfig <interface>  <IP address>  netmask  +  broadcast  + group \ <groupname>  up

ifconfig <interface> addif  <IP address> deprecated  netmask  +  broadcast  + \

 -failover up

 

Repeat the process for the second adapter.

 

You may also have to start the daemon in.mpathd if you don't reboot and you've changed the /etc/default/mpathd file. (see below: if you do, and IPMP is configured you will always start in.mpathd):

/sbin/in.mpathd

 

Monitoring in.mpathd: The command if_mpadm can be used to monitor and control IPMP. Always have a console window running if you use this command, since messages are sent to that window.  It can be used to test failover setup by detaching/offlining an interface:

 

if_mpadm  -d  <interface>

 

The interface can be reattached with:

 

if_mpadm -r <interface>

 

Commands:

 

if_mpadm -d <interface> - detach an interface. The output of ifconfig -a will show that it is "OFFLINE" and traffic should be routed through the backup interface

 

if_mpadm -r <interface> - reattach an interface. The output of ifconfig -a will show that the interface is "UP": and traffic should be routed through it again.

 

ifconfig <interface>  group  <groupname> - add an active interface to a group

ifconfig <logical interface> <IP address>  deprecated  netmask  +  broadcast  + 

 -failover  up - create a test interface.

ifconfig < interface> addif  <IP address>  deprecated  netmask  +  broadcast  + 

 -failover  up - create a test interface.

 

Files:

/etc/default/mpathd -variables and default values

FAILURE_DETECTION_TIME=10000 (10k milliseconds or 10 seconds)              length of time in.mpathd will wait for a ping reply before determining that the request has failed.

FAILBACK=yes         enables failback

TRACK_INTERFACES_ONLY_WITH_GROUPS=yes - starts in.mpathd if the keyword "group" is used with the command ifconfig. Otherwise in.mpathd will only be started at boot time by /etc/rcS.d/S30network.sh or when invoked manually. A "yes" setting to this variable means that if you set up IPMP manually, the daemon in.mpathd will start, since the word "group" must be used with ifconfig in that case.

 

/etc/release - lists the version and release of the running version of Solaris

 

Definitions:

 

deprecated - in computer terms, means to use only as a last resort. A deprecated virtual interface is used only if no other is available.