1. What does HAIP stand for?
Highly Available IP (address)
2. What clusterware version was HAIP introduced in?
Oracle Grid Infrastructure 11.2.0.2
3. What is the purpose of HAIP?
Haip has two primary functions:
- HAIP allows for redundant cluster interconnect NICs on cluster nodes without requiring any OS-level bonding, teaming or aggregating of the NICs.
- HAIP provides a layer of transparency between multiple Network Interface Cards (NICs) on a given node that are used for the cluster_interconnect which is used by RDBMS and ASM instances. However HAIP is not used for the Clusterware network communication. The Clusterware utilizes a feature called “Redundant Interconnect” for communication which essentially makes use of all available cluster_interconnect paths for communication.
4. How does HAIP create a layer of transparency between the cluster interconnect NICs on a given node and the RDBMS/ASM instances?
When clusterware on a given node is started, a virtual ip address (HAIP) is dynamically allocated and linked (link-local) to each NIC configured for use as a cluster interconnect. Each NIC defined as an interconnect (using oifcfg setif) will be assigned a unique HAIP during clusterware startup. The HAIP from each interconnect NIC is then used by the the ASM and RDBMS instances and ACFS. The static ip address associated with the physical NIC is not used by the ASM and RDBMS instances. The NIC name is not used by the ASM or RDBMS instance but is used by other clusterware components.
Each NIC defined as a cluster interconnect (using oifcfg setif) on a given node will have a static ip address assigned to it and each cluster interconnect NIC on a given node must be on a unique subnet. Clusterware requires that each subnet used by the cluster interconnect have a functioning NIC on each active cluster node. If any one of the cluster interconnect NICs is down on a node, then the subnet associated with the down NIC is considered not usable by any node of the cluster.
5. How does HAIP obtain an IP address for use in the virtual link?
HAIP takes advantage of linked-local ip addresses as defined in the Internet Engineering Task Force RFC-3927. This RFC reserves the IPv4 prefix 169.254/16 specifically for linked local use.
6. How can I check if the HAIP is in use?
Use the OS command “ifconfig -a”. Look in the ASM and RDBMS instance alert logs. During instance startup the interfaces and associated HAIP will be shown.
Alert log example showing two NICs:
Starting ORACLE instance (normal) ... Private Interface '' configured from GPnP for use as a private interconnect. [name=' ', type=1, ip=<169.254.xx.182> mac= , net=169.254.0.0/xx, mask=255.255.xxx.0, use=haip:cluster_interconnect/xx] Private Interface ' ' configured from GPnP for use as a private interconnect. [name=' ', type=1, ip=169.254.xx.204, mac= , net=169.254.xxx.0/xx, mask=255.255.xxx.0, use=haip:cluster_interconnect/xx] ... Cluster communication is configured to use the following interface(s) for this instance 169.254.xx.182 169.254.xxx.204
7. Which Oracle clusterware log shows the failover or failback of a HAIP?
The ohasd orarootagent_root logs under [GRID_HOME]/log/[node]/agent/ohasd/orarootagent_root/orarootagent_root.log.
8. What happens to the HAIP if a physical NIC fails or is disabled?9. What happens to the HAIP if the failed or disabled physical NIC is brought back up?
The HAIP will failback to the physical NIC it was linked to before the failover. The HAIP will be unlinked from the redundant NIC it was temporarily on, and locally-linked to the interconnect NIC that was brought back online. As with a failover this also will happen on all nodes of the cluster to ensure the HAIP is on the same subnet on all nodes. An ARP broadcast is made to publish the changed MAC address of the NIC that each HAIP that failed back to.
10. Does HAIP require NIC bonding or teaming?
No. HAIP allows you to provide redundant NICS for the cluster interconnect without using OS level NIC bonding or teaming or aggregation. Each physical interface is presented to the clusterware and registered with the clusterware via the Oracle clusterware command “oifcfg setif”.
11. Can OS bonding or teaming still be used with HAIP?
Yes, however, Oracle’s best practice is to use unbonded, unteamed NICs, without any additional layers of virtualization.
12. Is the HAIP active even if there is only one NIC per node for the cluster interconnect?
Yes. This cannot be disabled manually (with the exception on Solaris where, when IPMP is in use, HAIP will not be enabled during installation).
13. Is there any limit to how many redundant interconnect NICs that can be used by clusterware?
Yes. Clusterware will use up to four NICs, even if more are available.
14. With HAIP, are all the interconnect NICs active concurrently?
The default is for all interconnect NICs to be active. The Oracle best practice to have all NICs active.
15. Can I manually failover or failback HAIP?
No. There is no command that allows you to directly control failover and failback of HAIP. Clusterware does this automatically.
16. Is HAIP supported with pre-11.2.0.2 databases?
The default behavior for pre-11.2.0.2 databases is to use the fixed IP address associated with NICs and not use the HAIP. The recommendation is to use OS level NIC bonding/teaming for pre 11.2.0.2 databases.
17. What is the recommendation regarding using OS level NIC virtualization with HAIP?
Oracle recommends using just HAIP, with no additional OS level NIC virtualization.
18. Can HAIP feature be disabled manually?
No, It’s NOT supported to disable HAIP while the cluster is up and running. There are certain situations where HAIP feature is disabled during installation. Please refer to Note 1210883.1 Section Miscellaneous. For IPMP implementation on Solaris case, cluster_interconnects is required to set to IPMP IP. For other platforms, OS bonding technology should be used if more than 1 private network is used.
<19. Can RDS be used with HAIP? Yes with restrictions.