The Grid Naming Service (GNS) is a new feature in the 11.2 Grid Infrastructure (GI). Its design is to simplify the management of the network configuration of the Real Application Cluster (RAC). When GNS is used, it eliminates the manual allocation of the Node and Single Client Access Name (SCAN) Virtual Internet Protocol (VIP) addresses. It also eliminates the need to configure VIP names and SCAN names in a Domain Name Server (DNS) as GNS provides the name resolution for the cluster. What this means is the GNS is configured as a delegated authority in DNS for the Cluster Nodes.
- Allocate one static IP address on a public network for the GNS VIP.
- Setup GNS as a delegate for the cluster’s sub-domain on the DNS server.
- Validate that there is a DHCP server that is addressable by the cluster nodes.
When installing GI, there are 2 options:
- Configure GNS and let it handle name resolution in the cluster subdomain.
- Choose not to configure GNS and configure each Node and SCAN name with IP addresses defined in DNS.
This post does not address option (2) as it is intended to cover how to setup the DNS servers and DHCP servers to work with GNS. This post is an example of how to configure DNS to delegate a sub-domain to GNS. It also covers how DHCP is to be configured to allocate IP address for SCAN and Node VIP addresses.
DNS Setup Example
The example below is based on bind-9.3.4-10 on Linux though the setting should be generic for bind. In this example, the GNS VIP is gns.us.eot.com with IP 120.0.0.200. The GNS subdomain is us.eot.com.
Main config /etc/named.conf:
options {
directory "/etc/named";
zone-statistics yes;
allow-transfer {"none";};
};
controls {
inet 127.0.0.1 allow { localhost; } keys { rndc; };
};
key rndc {
algorithm hmac-md5;
secret "tpyCbBuiElAdZMfkirg3RcrCtyql0kW42vT24LxXysM=";
};
zone "." {
type hint;
file "root";
};
zone "0.0.127.in-addr.arpa" {
type master;
file "127.0.0.0";
notify no;
};
zone "eot.com" {
type master;
file "eot.com";
notify no;
};
zone "0.0.120.in-addr.arpa" {
type master;
file "120.x.x.0";
notify no;
};
The main config is for a single forward domain eot.com and reverse domain for subnet 120.x.x.0.
eot.com zone config file under /etc/named as defined in named.conf file above:
$TTL 3D
@ IN SOA eot.com. root.eot.com. (
199609206 ; serial, todays date + todays serial #
8H ; refresh, seconds
2H ; retry, seconds
4W ; expire, seconds
1D ) ; minimum, seconds
NS eot.com.
MX 10 eot.com. ; Primary Mail Exchanger
TXT "EOT Domain"
localhost A 127.0.0.1
eot.com. A 120..x.x.254
dns CNAME eot.com.
$ORIGIN us.eot.com.
@ IN NS gns.us.eot.com.
gns.us.eot.com. IN A 120.x.x.200
Above setting will delegate resolution of sub-domain us.eot.com to GNS VIP 120.x.x.200.
To verify delegation
To verify delegation is working as expected, execute the following nslookup and both should resolve to same set of IP addresses once GI is up and running:
# nslookup [scan-name] [main-DNS-IP] nslookup [scan-name] [GNS-IP]
For example:
# nslookup scan.us.eot.com 120.x.x.254 # nslookup scan.us.eot.com 120.x.x.200
DHCP Server Setup Example
The example below is based on dhcp-3.0.5 on Linux:
/etc/dhcp/dhcp.conf
ddns-update-style interim;
ignore client-updates;
subnet 120.x.x.0 netmask 255.255.255.0 {
range 120.x.x.201 120.x.x.220;
default-lease-time 86400;
option routers 120.x.x.1;
option ip-forwarding off;
option broadcast-address 120.x.x.255;
option subnet-mask 255.255.255.0;
option time-offset -28800;
option ntp-servers 120.x.x.254;
option domain-name "us.eot.com";
option domain-name-servers 120.x.x.254;
}
The DHCP will provide dynamic IPs for client in the range of 120.x.x.201-120.x.x.220 with subnet mask of 255.255.255.0, the default lease time is 86400 seconds, timezone -28800 which is PST (GMT-8), with default gateway of 120.x.x.1 and default ntp and DNS server of 120.x.x.254
Without this feature, DHCP server will not allocate IP to a client which does not have a valid MAC address. As VIP and SCAN VIP are designed to failover, they will not have MAC address assigned. When Oracle request IP for these VIPs from the DHCP server, we only supply the Client Identifier, not the MAC address in the client request.
A tcpdump of the DHCP packet will show:
******************
15:59:09.165705 In 00:10:e0:3a:2c:56 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 414: vlan 2100, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 0, offset 0, flags [DF] , proto: UDP (17), length: 396) 120.x.x.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:00:00:00, length 368, xid 0x89b5e43f, Flags [Broadcast]
Client-Ethernet-Address 00:00:00:00:00:00
sname "racnode1-vip"
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Discover
MSZ Option 57, length 2: 8
Client-ID Option 61, length 12: "racnode1-vip"
******************
The 3rd party DHCP server will report error: “Oct 7 15:59:09 packet has an invalid hardware address” if it does not have support “Client Identifier” feature enabled.
Validate DHCP Server from cluvfy
To verify whether client can get IP from DHCP server, as root:
# [GRID_HOME]/bin/cluvfy comp dhcp -clustername [cluster name] Verifying DHCP Check Checking if any DHCP server exists on the network... At least one DHCP server exists on the network and is listening on port 67 Checking if DHCP server has sufficient free IP addresses for all VIPs... DHCP server was able to provide sufficient number of IP addresses The DHCP server response time is within acceptable limits Verification of DHCP Check was successful.
The verbose output looks like this:
# [GRID_HOME]/bin/cluvfy comp dhcp -clustername [cluster name] -verbose Verifying DHCP Check Checking if any DHCP server exists on the network... Checking if network CRS resource is configured and online Network CRS resource is offline or not configured. Proceeding with DHCP checks. CRS-10009: DHCP server returned server: 120.x.x.200, loan address : 120.x.x.250/255.255.255.0, lease time: 86400 At least one DHCP server exists on the network and is listening on port 67 Checking if DHCP server has sufficient free IP addresses for all VIPs... Sending DHCP "DISCOVER" packets for client ID "cluster-scan1-vip" CRS-10009: DHCP server returned server: 120.x.x.254, loan address : 120.x.x.250/255.255.255.0, lease time: 86400 Sending DHCP "REQUEST" packets for client ID "cluster-scan1-vip" CRS-10009: DHCP server returned server: 120.x.x.254, loan address : 120.x.x.250/255.255.255.0, lease time: 86400 Sending DHCP "DISCOVER" packets for client ID "cluster-scan2-vip" CRS-10009: DHCP server returned server: 120.x.x.254, loan address : 120.x.x.249/255.255.255.0, lease time: 86400 Sending DHCP "REQUEST" packets for client ID "cluster-scan2-vip" CRS-10009: DHCP server returned server: 120.x.x.254, loan address : 120.x.x.249/255.255.255.0, lease time: 86400 Sending DHCP "DISCOVER" packets for client ID "cluster-scan3-vip" CRS-10009: DHCP server returned server: 120.x.x.254, loan address : 120.x.x.248/255.255.255.0, lease time: 86400 Sending DHCP "REQUEST" packets for client ID "cluster-scan3-vip" CRS-10009: DHCP server returned server: 120.x.x.254, loan address : 120.x.x.248/255.255.255.0, lease time: 86400 Sending DHCP "DISCOVER" packets for client ID "racnode1-vip" CRS-10009: DHCP server returned server: 120.x.x.254, loan address : 120.x.x.247/255.255.255.0, lease time: 86400 Sending DHCP "REQUEST" packets for client ID "racnode1-vip" CRS-10009: DHCP server returned server: 120.x.x.254, loan address : 120.x.x.247/255.255.255.0, lease time: 86400 CRS-10012: released DHCP server lease for client ID cluster-scan1-vip on port 67 CRS-10012: released DHCP server lease for client ID cluster-scan2-vip on port 67 CRS-10012: released DHCP server lease for client ID cluster-scan3-vip on port 67 CRS-10012: released DHCP server lease for client ID racnode1-vip on port 67 DHCP server was able to provide sufficient number of IP addresses The DHCP server response time is within acceptable limits Verification of DHCP Check was successful.
If above command is not executed as root, the following error will be reported:
PRVF-0055 : User "grid" does not have sufficient authorization to run this command
If it fails to get IP from DHCP server, engage System Administrator or Network Administrator to verify DHCP server setup
Verifying DHCP Check Checking if any DHCP server exists on the network... PRVF-5801 : Failed to execute command "/ocw/grid/bin/crsctl discover dhcp -clientid cluster-scan0-vip" on all nodes PRVF-5704 : No DHCP server were discovered on the public network listening on port 67 Verification of DHCP Check was unsuccessful on all the specified nodes. crsctl discover dhcp -clientid giclu-scan0-vip CRS-10010: unable to discover DHCP server in the network listening on port 67 for client ID cluster-scan0-vip CRS-4000: Command discover failed, or completed with errors.