ASMLib/oracleasm can only be implemented on Linux OS. When using multipathing, you want to FORCE oracleasm to scan the multipathed devices first. This reduces the scan time as by default oracleasm tries to scan each and every device including the multiple paths to it.
On all nodes:
1. Shutdown all (DB and ASM):
You should stop all before restart ASMLib (eg: DB, ASM and HAS (crsctl stop crs for RAC) if you are in 11.2 – crsctl stop has):
If the diskgroups are mounted, it will fail with below error:
/etc/init.d/oracleasm restart
Unmounting ASMlib driver filesystem: umount: /dev/oracleasm: device is busy
umount: /dev/oracleasm: device is busy
Unable to unmount ASMlib driver filesystem
[FAILED]
ASMLib will not allow that.
Then, on each node, one at a time, do the following steps that are dependent on the type of multipathing software:
2. For backup, make a copy of your current /etc/sysconfig/oracleasm and /etc/sysconfig/oracleasm-_dev_oracleasm files.
a) If using DM-Multipathing or Linux OS Native Multipathing:
– Then, you want to change the following parameters in the /etc/sysconfig/oracleasm-_dev_oracleasm file to which the /etc/sysconfig/oracleasm file is really linked to:
For example:
lrwxrwxrwx 1 root root 24 Jun 13 09:58 /etc/sysconfig/oracleasm -> oracleasm-_dev_oracleasm
# ORACLEASM_SCANORDER: Matching patterns to order disk scanning ORACLEASM_SCANORDER="mpath dm" (OR ORACLEASM_SCANORDER="dm") # ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan ORACLEASM_SCANEXCLUDE="sd"
– If using EMC PowerPath Multipathing, then you want to change the following parameters in the /etc/sysconfig/oracleasm-_dev_oracleasm file to which the /etc/sysconfig/oracleasm file is really linked to:
For example:
lrwxrwxrwx 1 root root 24 Jun 13 09:58 /etc/sysconfig/oracleasm -> oracleasm-_dev_oracleasm
# ORACLEASM_SCANORDER: Matching patterns to order disk scanning ORACLEASM_SCANORDER="emcpower" # ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan ORACLEASM_SCANEXCLUDE="sd"
– If using Hitachi Dynamic Link Manager, then, you want to change the following parameters in the /etc/sysconfig/oracleasm-_dev_oracleasm file to which the /etc/sysconfig/oracleasm file is really linked to:
For example:
lrwxrwxrwx 1 root root 24 Jun 13 09:58 /etc/sysconfig/oracleasm -> oracleasm-_dev_oracleasm
# ORACLEASM_SCANORDER: Matching patterns to order disk scanning ORACLEASM_SCANORDER="sddlm" # ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan ORACLEASM_SCANEXCLUDE=""
3. As root user, save the file (oracleasm-_dev_oracleasm).
4. Restart oracleasm (as root):
# service oracleasm restart
or
# /etc/init.d/oracleasm restart
5. check /dev/oracleasm/disks/*:
$ ls -ltr /dev/oracleasm/disks/*
a) If using DM-Multipathing or Linux OS Native Multipathing, all of the Major Numbers should equal 253 or 252 or 249. For example:
# ls -l /dev/oracleasm/disks brw-rw---- 1 grid asmadmin 253, 9 Jun 4 10:35 ASM01 brw-rw---- 1 grid asmadmin 253, 4 Jun 4 10:35 ASM02 brw-rw---- 1 grid asmadmin 253, 7 Jun 4 10:35 ASM03 brw-rw---- 1 grid asmadmin 253, 11 Jun 4 10:35 ASM04 brw-rw---- 1 grid asmadmin 253, 20 Jun 4 10:35 ASM05
OR
# ls -l /dev/oracleasm/disks brw-rw---- 1 grid asmadmin 252, 9 Jun 4 10:35 ASM01 brw-rw---- 1 grid asmadmin 252, 4 Jun 4 10:35 ASM02 brw-rw---- 1 grid asmadmin 252, 7 Jun 4 10:35 ASM03 brw-rw---- 1 grid asmadmin 252, 11 Jun 4 10:35 ASM04 brw-rw---- 1 grid asmadmin 252, 20 Jun 4 10:35 ASM05
The above Major Numbers should match the multipathed device (dm*) in the file, /proc/partitions. For example:
# cat /proc/partitions major minor #blocks name ... 253 8 524281275 dm-8 253 9 524281220 dm-9 253 3 524281275 dm-3 253 4 524281220 dm-4 253 6 524281275 dm-6 253 7 524281220 dm-7 253 10 524281275 dm-10 253 11 524281220 dm-11 253 19 524281275 dm-19 253 20 524281220 dm-20 ...
b) If using EMC PowerPath Multipathing, all of the Major Numbers should equal 120. For example:
# ls -l /dev/oracleasm/disks brw-rw---- 1 grid asmadmin 120, 11 Jun 4 10:35 ASM01 brw-rw---- 1 grid asmadmin 120, 5 Jun 4 10:35 ASM02 brw-rw---- 1 grid asmadmin 120, 3 Jun 4 10:35 ASM03 brw-rw---- 1 grid asmadmin 120, 7 Jun 4 10:35 ASM04 brw-rw---- 1 grid asmadmin 120, 9 Jun 4 10:35 ASM05
2. The above Major Numbers should match the multipathed device (emcpower*) in the file, /proc/partitions.
# cat /proc/partitions major minor #blocks name ... 120 2 524281275 emcpowerb 120 3 524281220 emcpowerb1 120 4 524281275 emcpowerc 120 5 524281220 emcpowerc1 120 6 524281275 emcpowerd 120 7 524281220 emcpowerd1 120 8 524281275 emcpowere 120 9 524281220 emcpowere1 120 10 524281275 emcpowerf 120 11 524281220 emcpowerf1 ...
c) If using Hitachi Dynamic Link Manager, for example:
# ls -l /dev/oracleasm/disks brw-rw---- 1 grid asmadmin 252, 224 Jun 4 10:35 ASM01 brw-rw---- 1 grid asmadmin 252, 240 Jun 4 10:35 ASM02 brw-rw---- 1 grid asmadmin 251, 0 Jun 4 10:35 ASM03 brw-rw---- 1 grid asmadmin 251, 16 Jun 4 10:35 ASM04 brw-rw---- 1 grid asmadmin 251, 32 Jun 4 10:35 ASM05
2. The above should match the multipathed device (sddlm*) in the file, /proc/partitions.
When the /proc/partitions file has listed sddlm* devices, then that is normally an indication of Hitachi Dynamic Link Manager is being used.
# cat /proc/partitions major minor #blocks name ... 252 224 204802560 sddlmao 252 240 204802560 sddlmap 251 0 204802560 sddlmba 251 16 204802560 sddlmbb 251 32 215040000 sddlmbc ...
d) If using AIX (Please note that ASMLib/oracleasm can only be implemented on Linux OS.):
Please note that AIX can use Linux OS with different types of multipathing software, but the multipathing setup/SCAN depends on the Storage Vendor. However, most AIX Customers’ set up multipathing with mknod and there is a built in MPIO (MultiPath I/O) software in AIX 7.0.
If Customer uses HDS or EMC storage than the drivers come from the vendor.
# lsdev -Cc disk hdisk38 Available 06-09-02 MPIO IBM 2107 FC Disk hdisk39 Available 06-09-02 MPIO IBM 2107 FC Disk hdisk40 Available 06-09-02 MPIO IBM 2107 FC Disk hdisk41 Available 06-09-02 MPIO IBM 2107 FC Disk hdisk42 Available 06-09-02 MPIO IBM 2107 FC Disk hdisk43 Available 06-09-02 MPIO IBM 2107 FC Disk hdisk44 Available 06-09-02 MPIO IBM 2107 FC Disk
Beginning with AIX 5.2, the multiple path I/O function uses a unique device identifier (UDID) to identify each MPIO-capable device, regardless of the path on which it was discovered.
The UDID is saved in the device configuration database. When a device is discovered, the UDIDs in the database are checked to determine whether the device is new or whether the discovery is another path to an existing device. When multiple paths to a device are detected, the device driver or the Path Control Manager kernel extension decides which path to use for a particular request.