I wouldn’t say Solaris Volume manager is one of the best volume managers out there in the market, but its sure still used in many production setups. Lets us see how we can encapsulate and mirror a root disk under solaris volume manager. The final mirrored root disk setup should look like the one shown in figure below :
Let us first see the disk that is currently being used by the root partition.
# df -h / Filesystem size used avail capacity Mounted on /dev/dsk/c1t0d0s0 6.9G 2.9G 3.9G 43% /
To see the available disk including the root disk c1t0d0 :
# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c1t0d0
/pci@0,0/pci15ad,1976@10/sd@0,0
1. c1t1d0
/pci@0,0/pci15ad,1976@10/sd@1,0
Specify disk (enter its number): ^C
So we have one more disk c1t1d0 of same size as that of c1t0d0. I would use these two disk for root disk mirroring and encapsulation under SVM.
Copy Partition table from root to mirror disk.
The first step is to copy the exact partition table of root disk to the mirror disk. The easiest way to do this is with fmthard. What fmthard does is simply copies the partition table generated by prtvtoc to the mirror disk. Before copying the partition table make sure you make a 100 MB partition on 7th slice for having the state metabase replicas. Once you have created the 100MB partition and labeled the disk start copying the partition table to the mirror disk.
# prtvtoc /dev/rdsk/c1t0d0s2 | fmthard -s - /dev/rdsk/c1t1d0s2
Creating the state Database
The state database contains the configuration and status information of all volumes, hot spares and disk sets. To provide redundancy we create multiple copies of the state database called as state database replicas. Now in case of losing any state database replica SVM determines the valid state database replica by using majority consensus algorithm. According to the algorithm it is required to have atleast (half + 1) to be available at boot time to be able to consider any of them to be valid. Each replica takes around 4 MB of size. It is ideal to create 3 of them on each disk in case you are mirroring the root disk.
Lets now create 3 replicas on each disk.
# metadb -afc 3 c1t0d0s7 c1t1d0s7 -c -> Specifies the number of replicas to be placed on each device. Default is 1 -f -> force the creation when there is no state database replicas already present. -a -> Attach a new database device.
To see the new state database replicas we just created :
# metadb
flags first blk block count
a u 16 8192 /dev/dsk/c1t0d0s7
a u 8208 8192 /dev/dsk/c1t0d0s7
a u 16400 8192 /dev/dsk/c1t0d0s7
a u 16 8192 /dev/dsk/c1t1d0s7
a u 8208 8192 /dev/dsk/c1t1d0s7
a u 16400 8192 /dev/dsk/c1t1d0s7
Root partition
Let us now create the 2 submirrors for the root volume.
# metainit -f d11 1 1 c1t0d0s0 d11: Concat/Stripe is setup # metainit -f d12 1 1 c1t1d0s0 d12: Concat/Stripe is setup
Here 1 1 specifies the numstripes ( no of individual stripes in metadevice) and width (no of slices that make up a stripe) respectively.
Let us now create the mirror d10 which will have submirrors d11 and d12
# metainit d10 -m d11 d10: Mirror is setup
Now we have to encapsulate the root disk using the metaroot command which will update the /etc/system and /etc/vfstab files. Take backup of the files getting updated for safety.
# cp /etc/vfstab /etc/vfstab.bak # cp /etc/system /etc/system.bak
# metaroot d10
If you check the /etc/system file carefully, you’ll find an extra line :
rootdev:/pseudo/md@0:0,10,blk
Also you’ll find the entry for root filesystem replaced with :
dev/md/dsk/d10 /dev/md/rdsk/d10 / ufs 1
Now the metaroot requires a reboot before it takes effect. But before we reboot the system, we need to add one more entry in the /etc/system file to bypass the majority consensus algorithm. This enable us to boot from a single disk, which may be the requirement in many cases in production like patching the system etc. The entry is :
set md:mirrored_root_flag = 1
Now we can reboot the system.
# shutdown -i6 -g0 -y
After the reboot we can attach the other sub-mirror d12 to the mirror d10 and sync the data from d11 to d12.
# metattach d10 d12
The synching will start now. To check the status of the mirror :
# metastat d10
d10: Mirror
Submirror 0: d11
State: Okay
Submirror 1: d12
State: Resyncing
Resync in progress: 8 % done
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 14643200 blocks (7.0 GB)
d11: Submirror of d10
State: Okay
Size: 14643200 blocks (7.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s0 0 No Okay Yes
d12: Submirror of d10
State: Resyncing
Size: 14643200 blocks (7.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s0 0 No Okay Yes
Device Relocation Information:
Device Reloc Device ID
c1t0d0 Yes id1,sd@n6000c297a7eb5c765e13f7ce11a25a61
c1t1d0 Yes id1,sd@n6000c299599de680b94c732ecfef6f03
Swap partition
Now swap partition can be placed under online anytime. We need to remove the swap if it is already present.
# swap -l swapfile dev swaplo blocks free /dev/dsk/c1t0d0s1 30,1 8 1548280 1548280
# swap -d /dev/dsk/c1t0d0s1
Create the 2 submirrors :
# metainit d21 1 1 c1t0d0s1 d21: Concat/Stripe is setup # metainit d22 1 1 c1t1d0s1 d22: Concat/Stripe is setup
Create the mirror using the first submirror
# metainit d20 -m d21 d20: Mirror is setup
Now attach the remaining submirror to the mirror
# metattach d20 d22 d20: submirror d22 is attached
Check the sync status :
# metastat -c d20
d20 m 756MB d21 d22 (resync-17%)
d21 s 756MB c1t0d0s1
d22 s 756MB c1t1d0s1
We need to manually change the /etc/vfstab entry for the swap metadevice to be persistent across reboots.
/dev/dsk/c1t0d0s1 - - swap - no - to: /dev/md/dsk/d20 - - swap - no -
Also add the swap back again using the new metadevice we created for swap
# swap -a /dev/md/dsk/d20
Do not forget to set the swap as the dump device
# dumpadm -d swap
Dump content: kernel pages
Dump device: /dev/md/dsk/d20 (swap)
Savecore directory: /var/crash/geeklab
Savecore enabled: yes
Save compressed: on
Setting up the OBP variables
Now we need to update the OBP variable to specify the root and mirror boot device.First check the physical device names of the rootdisk and the mirror disk. Let us see how we can do it with the OS eeprom commands.
# ls -l /dev/dsk/c1t0d0s0 lrwxrwxrwx 1 root root 46 Nov 16 12:35 /dev/dsk/c1t0d0s0 -> ../../devices/pci@0,0/pci15ad,1976@10/sd@0,0:a # ls -l /dev/dsk/c1t1d0s0 lrwxrwxrwx 1 root root 46 Nov 16 12:35 /dev/dsk/c1t1d0s0 -> ../../devices/pci@0,0/pci15ad,1976@10/sd@1,0:a
Set the rootdisk and mirrordisk aliases in the OBP
# eeprom "nvramrc=devalias rootdisk /devices/pci@0,0/pci15ad,1976@10/sd@0,0:a mirrordisk /devices/pci@0,0/pci15ad,1976@10/sd@1,0:a"
Set the order of boot-device
# eeprom boot-device="rootdisk mirrordisk"
Set use-nvramrc to true to enable use of nvramrc variables.
# eeprom "use-nvramrc?=true"
You can also set the alias at the OK prompt if you want :
ok> nvalias rootdisk /devices/pci@0,0/pci15ad,1976@10/sd@0,0:a mirrordisk /devices/pci@0,0/pci15ad,1976@10/sd@1,0:a ok> setenv boot-device rootdisk mirrordisk
Installing boot blocks on mirror disk
For a UFS based File system :
# installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c1t1d0s0