VxVM Tutorials
One of the best feature of VxVM or any volume manager for that matter is redundancy of data. Root disk is one of the most important disk as it contains the complete OS. So it is very important to mirror the root disk. VxVM gives us this feature to mirror the root disk. But for this we first need to take the root disk under VxVM control by encapsulating the root disk. Root disk encapsulation preserves the data. Encapsulating a boot disk is often called as rootability, which is the ability to place all root disk FS like root, swap etc under VxVM control.Let us see how we can encapsulate and mirror the root disk.
NOw first note down the root disk on which the OS is currently booted up. In our case it is c1t0d0.
# df -h / Filesystem size used avail capacity Mounted on /dev/dsk/c1t0d0s0 15G 5.9G 8.6G 41% /
Now we will use the vxdiskadm utility to encapsulate the root disk. Select option 2 : “Encapsulate one or more disks”.
# vxdiskadm ......(other options omitted for brevity)...... 2 Encapsulate one or more disks Select an operation to perform: 2 Encapsulate one or more disks Menu: VolumeManager/Disk/Encapsulate Use this operation to convert one or more disks to use the Volume Manager. This adds the disks to a disk group and replaces existing partitions with volumes. Disk encapsulation requires a reboot for the changes to take effect.
Now you can have various disk selection examples. I will be using the option “list” to list out available disks under VxVM control. The list option shows only disks that are under VxVM control i.e. either encapsulated or initialized. Now we can see our disk to be encapsulated i.e. c1t0d0
More than one disk or pattern may be entered at the prompt. Here are some disk selection examples: all: all disks c3 c4t2: all disks on both controller 3 and controller 4, target 2 c3t4d2: a single disk (in the c#t#d# naming scheme) xyz_0 : a single disk (in the enclosure based naming scheme) xyz_ : all disks on the enclosure whose name is xyz Select disk devices to encapsulate: [[pattern-list],all,list,q,?] list DEVICE DISK GROUP STATUS c1t0d0 - - online invalid c1t1d0 - - online invalid Select disk devices to encapsulate: [[pattern-list],all,list,q,?] c1t0d0 Here is the disk selected. Output format: [Device_Name] c1t0d0 Continue operation? [y,n,q,?] (default: y) y
Now select the disk group name for our rootdisk i.e. rootdg. Also specify the name of the disk as rootdisk. We will use another disk as mirror and will name it as rootmirror.
You can choose to add this disk to an existing disk group or to a new disk group. To create a new disk group, select a disk group name that does not yet exist. Which disk group [[group],list,q,?] rootdg Create a new group named rootdg? [y,n,q,?] (default: y) y Use a default disk name for the disk? [y,n,q,?] (default: y) n A new disk group will be created named rootdg and the selected disks will be encapsulated and added to this disk group with disk names that will be specified interactively. c1t0d0 Continue with operation? [y,n,q,?] (default: y) y The following disk has been selected for encapsulation. Output format: [Device_Name] c1t0d0 Continue with encapsulation? [y,n,q,?] (default: y) y Enter disk name for c1t0d0 [[name],q,?] (default: rootdg01) rootdisk A new disk group rootdg will be created and the disk device c1t0d0 will be encapsulated and added to the disk group with the disk name rootdisk. The c1t0d0 disk has been configured for encapsulation. The first stage of encapsulation has completed successfully. You should now reboot your system at the earliest possible opportunity. The encapsulation will require two or three reboots which will happen automatically after the next reboot. To reboot execute the command: shutdown -g0 -y -i6 This will update the /etc/vfstab file so that volume devices are used to mount the file systems on this disk device. You will need to update any other references such as backup scripts, databases, or manually created swap devices. Encapsulate other disks? [y,n,q,?] (default: n) n
Now exit out of the vxdiskadm utility by typing “q”. We will have to reboot the system in order to complete the root encapsulation process.
# shutdown -i6 -g0 -y
After the reboot you should notice the change in “df -h” command output:
# df -h / Filesystem size used avail capacity Mounted on /dev/vx/dsk/bootdg/rootvol 15G 6.0G 8.6G 41% /
Also notice the changes in the entries of the /etc/vfstab file. Now the file system and raw device has been changed.
# cat /etc/vfstab |grep vx /dev/vx/dsk/bootdg/swapvol - - swap - no - /dev/vx/dsk/bootdg/rootvol /dev/vx/rdsk/bootdg/rootvol / ufs no -
Check the Disk group status and check both rootvol and swapvol in vxprint command output.
# vxdg list NAME STATE ID rootdg enabled 1382248262.12.geeklab
# vxprint -htg rootdg .............. dg rootdg default default 27000 1382248262.12.geeklab dm rootdisk c1t0d0s2 auto 80321 33399135 - v rootvol - ENABLED ACTIVE 31374945 ROUND - root pl rootvol-01 rootvol ENABLED ACTIVE 31374945 CONCAT - RW sd rootdisk-02 rootvol-01 rootdisk 2024190 31374945 0 c1t0d0 ENA v swapvol - ENABLED ACTIVE 2024190 ROUND - swap pl swapvol-01 swapvol ENABLED ACTIVE 2024190 CONCAT - RW sd rootdisk-01 swapvol-01 rootdisk 0 2024190 0 c1t0d0 ENA
Now add a mirror disk to the rootdg and name it as rootmirror. We will use this diks to mirror rootvol and swapvol. Remeber to set the format and noreserve option.
# vxdisksetup -i c1t1d0 format=sliced noreserve # vxdg -g rootdg adddisk rootmirror=c1t1d0
# vxdiskadm ...... output omitted for brevity .... 6 Mirror volumes on a disk Select an operation to perform: 6
Mirror volumes on a disk Menu: VolumeManager/Disk/Mirror This operation can be used to mirror volumes on a disk. These volumes can be be mirrored onto another disk or onto any available disk space. Volumes will not be mirrored if they are already mirrored. Also, volumes that are comprised of more than one subdisk will not be mirrored. Mirroring volumes from the boot disk will produce a disk that can be used as an alternate boot disk. At the prompt below, supply the name of the disk containing the volumes to be mirrored. Enter disk name [[disk],list,q,?] list Disk group: rootdg DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE dm rootdisk c1t0d0s2 auto 80321 33399135 - dm rootmirror c1t1d0s2 auto 80321 33415200 - Enter disk name [[disk],list,q,?] rootdisk You can choose to mirror volumes from disk rootdisk onto any available disk space, or you can choose to mirror onto a specific disk. To mirror to a specific disk, select the name of that disk. To mirror to any available disk space, select "any". Enter destination disk [[disk],list,q,?] (default: any) rootmirror The requested operation is to mirror all volumes on disk rootdisk in disk group rootdg onto available disk space on disk rootmirror. VxVM NOTICE V-5-2-3650 This operation can take a long time to complete. Continue with operation? [y,n,q,?] (default: y) y VxVM vxmirror INFO V-5-2-22 Mirror volume swapvol ... VxVM vxmirror INFO V-5-2-22 Mirror volume rootvol ... VxVM INFO V-5-2-674 Mirroring of disk rootdisk is complete. Mirror volumes on another disk? [y,n,q,?] (default: n)n
Now quit the vxdiskadm utility. In case you want to check the status of the mirroring, you can open a new terminal and fire below commands.
# vxtask list TASKID PTID TYPE/STATE PCT PROGRESS 160 ATCOPY/R 25.19% 0/2024190/509952 PLXATT swapvol swapvol-02 rootdg
# vxtask monitor .......... 164 ATCOPY/R 03.52% 0/31374945/1105920 PLXATT rootvol rootvol-02 rootdg 164 ATCOPY/R 03.54% 0/31374945/1110016 PLXATT rootvol rootvol-02 rootdg 164 ATCOPY/R 03.55% 0/31374945/1114112 PLXATT rootvol rootvol-02 rootdg 164 ATCOPY/R 03.56% 0/31374945/1118208 PLXATT rootvol rootvol-02 rootdg 164 ATCOPY/R 03.58% 0/31374945/1122304 PLXATT rootvol rootvol-02 rootdg 164 ATCOPY/R 03.59% 0/31374945/1126400 PLXATT rootvol rootvol-02 rootdg 164 ATCOPY/R 03.60% 0/31374945/1130496 PLXATT rootvol rootvol-02 rootdg 164 ATCOPY/R 03.62% 0/31374945/1134592 PLXATT rootvol rootvol-02 rootdg 164 ATCOPY/R 03.63% 0/31374945/1138688 PLXATT rootvol rootvol-02 rootdg .............
Now you can check the mirrored rootvol and swapvol in the vxprint output. Check that both copies of volumes are ENABLE and ACTIVE.
# vxprint -htg rootdg ............. dg rootdg default default 27000 1382248262.12.geeklab dm rootdisk c1t0d0s2 auto 80321 33399135 - dm rootmirror c1t1d0s2 auto 80321 33415200 - v rootvol - ENABLED ACTIVE 31374945 ROUND - root pl rootvol-01 rootvol ENABLED ACTIVE 31374945 CONCAT - RW sd rootdisk-02 rootvol-01 rootdisk 2024190 31374945 0 c1t0d0 ENA pl rootvol-02 rootvol ENABLED ACTIVE 31374945 CONCAT - RW sd rootmirror-02 rootvol-02 rootmirror 2024190 31374945 0 c1t1d0 ENA v swapvol - ENABLED ACTIVE 2024190 ROUND - swap pl swapvol-01 swapvol ENABLED ACTIVE 2024190 CONCAT - RW sd rootdisk-01 swapvol-01 rootdisk 0 2024190 0 c1t0d0 ENA pl swapvol-02 swapvol ENABLED ACTIVE 2024190 CONCAT - RW sd rootmirror-01 swapvol-02 rootmirror 0 2024190 0 c1t1d0 ENA
For x86 machines
x86 machines uses BIOS instead of the OBP so to duplicate the functionality of eeprom in SPARC machine, x86 systems uses a file /boot/solaris/bootenv.rc at boot time. VxVM uses this file to locate the boot device. You should be able to see 2 variables in eeprom when you mirror the root disk
# eeprom |grep boot bootpath=/pci@0,0/pci15ad,1976@10/sd@0,0:a altbootpath=/pci@0,0/pci15ad,1976@10/sd@1,0:a
When you boot the OS you should see these 2 options on the BIOS menu. You can use up/down arrow to select the boot disk to boot from.
In case you make any changes to the bootenv.rc file you have to update the archive:
# bootadm update-archive
For SPARC machines
In case of SPARC machines we have to set use-nvramrc? parameter to true in order to enable the VxVM aliases for root disk and root mirror. You can use either OS command eeprom or set it from ok prompt.
# eeprom use-nvramrc?=true ok> setenv use-nvramrc? true
Now set the boot device order either from OS or from ok prompt.
# eeprom boot-device=rootdisk rootmirror ok> setenv boot-device rootdisk rootmirror
You can now boot the system at ok prompt by giving the command:
ok> boot rootdisk
To boot from mirror disk
ok> boot rootmirror or # eeprom nvramrc=devalias rootmirror
Test the boot disk failure
We can check the mirrored disk, by failing the boot disk from which the OS is currently booted up. To do so use vxment to disable the plex rootvol-01.
# vxmend -g rootdg off rootvol-01
check vxprint output to see the failed volume. The system should still be up and running from the other plex in the root mirror disk.
# vxprint -htg rootdg ..................... dg rootdg default default 27000 1382248262.12.geeklab dm rootdisk c1t0d0s2 auto 80321 33399135 - dm rootmirror c1t1d0s2 auto 80321 33415200 - v rootvol - ENABLED ACTIVE 31374945 ROUND - root pl rootvol-01 rootvol DISABLED OFFLINE 31374945 CONCAT - RW sd rootdisk-02 rootvol-01 rootdisk 2024190 31374945 0 c1t0d0 ENA pl rootvol-02 rootvol ENABLED ACTIVE 31374945 CONCAT - RW sd rootmirror-02 rootvol-02 rootmirror 2024190 31374945 0 c1t1d0 ENA v swapvol - ENABLED ACTIVE 2024190 ROUND - swap pl swapvol-01 swapvol ENABLED ACTIVE 2024190 CONCAT - RW sd rootdisk-01 swapvol-01 rootdisk 0 2024190 0 c1t0d0 ENA pl swapvol-02 swapvol ENABLED ACTIVE 2024190 CONCAT - RW sd rootmirror-01 swapvol-02 rootmirror 0 2024190 0 c1t1d0 ENA
To restore the plex state to original :
vxmend -g rootdg on rootvol-01
You can use the above method to simulate the root disk failure. Ensure that both plexes are synchronised before disabling the rootvol-01 plex.
Unencapsulating the root disk
Unencapsulating the boot disk is quite an easy task. We just need to run a single command and reboot the system. But before unencapsulating the root disk ensure that you have detached the mirror disk from root disk.
# vxassist -g rootdg remove mirror volume rootvol swapvol # vxunroot # shutdown -i6 -g0 -y
The vxunroot command removes the VxVM entries from the /etc/vfstab file, so that the system can boot from physical partitions.
Here a nice video from Gabriel on root disk encapsulation and mirroring of root disk under VxVM.