The Geek Diary

‘Found duplicate PV’ warnings when using LVM with multipath storage in RHEL/CentOS

by

The Problem

Messages such as the following are reported when running commands such as partprobe on a CentOS/RHEL system using LVM (Logical Volume Manager) and multipathing e.g. device-mapper-multipath or EMC PowerPath, etc.:

Error: Error informing the kernel about modifications to partition /dev/sda1 -- Device or resource busy. This means Linux won't know about any changes you made to /dev/sda1 until you reboot -- so you shouldn't mount it or use it in any way before rebooting.
...

Messages such the following are reported when running the pvs command to obtain Physical Volume (PV) related information:

Found duplicate PV G9tZklKg1nrXN7wGADvwHQOQtgd4w58M: using /dev/sdb1 not /dev/sda1
Found duplicate PV G9tZklKg1nrXN7wGADvwHQOQtgd4w58M: using /dev/sda1 not /dev/emcpowera1
...

The Root Cause

The ‘Found duplicate PV’ message occurs when LVM is not configured to filter out underlying singlepath devices e.g. /dev/sd*. Multipath software e.g device-mapper-multipath, EMC PowerPath, etc. create a multipath pseudo device for each managed device. If underlying singlepath devices are not filtered out, LVM discovers, thus reports the duplicate device paths that all reference the same LVM device.

Example device mappings for two-path device-mapper-multipath configuration:

/dev/sda, /dev/sdb --> /dev/dm-0 --> /dev/mapper/mpath1

Example device mapping for two path EMC PowerPath configuration:

/dev/sda, /dev/sdb --> /dev/emcpowera

The Solution

For a new LVM setup on EMC PowerPath, please refer to “Configuring LVM2 support” in the EMC PowerPath documentation, which should have details of modifying the “/etc/lvm/lvm.conf” file to filter out the underlying paths

For a new LVM setup on device-mapper-multipath:

– Turn LVM scanning off on the underlying SCSI devices. It can be done by changing the filter parameter in “/etc/lvm/lvm.conf”.

filter = [ "a/dev/mapper/.*/", "r/dev/sd.*/" ]
Note: This setting accepts all /dev/mapper/* devices and rejects all the /dev/sd* devices while scanning the LVM disks. Refer to the LVM2 user guide for further details. If the root device is also a multipathed LVM device, then make change(s) to /etc/lvm/lvm.conf before creating a new initrd image. Where “multipath on root” is required, the lvm.conf is needed in the initrd image, which requires the initrd to be re-made.

For an existing LVM setup on EMC PowerPath or dm-multipath disks, ensure the integrity of the data on LVM volumes:

1. Change the “/etc/lvm/lvm.conf” as described above

2. In general one should configure the LVM to filter out all sd* devices and only see the /dev/emcpower* or /dev/mapper/* devices. If there are other non-mutipathed SCSI devices LVM volumes, e.g: /dev/cciss/*, then set the filter accordingly, for example:

filter = [ "r/sd.*/" "a/dev/cciss/*/" "a/dev/emcpower.*/" "a/dev/mapper/*/" ]

With this setting, the ‘lvmdiskscan’ and ‘pvscan’ commands should only find the /dev/ccisss/*, /dev/emcpower* and /dev/mapper/* devices.

3. Stop any applications/services that access LVM managed devices/filesystems.

4. Unmount all LVM-based filesystems.

5. Deactivate all Volume Groups.

# vgchange -an

6. Rescan Volume Groups.

# vgscan

7. Re-activate Volume Groups.

# vgchange -ay

8. Mount any LVM-based filesystems.

9. Start any applications/services requring access to LVM managed devices/filesystems.

Note: If LVM disks need to be referenced during bootup e.g: during kdump capture kernel booting stage or when configured as multipath-on-root, the initrd image will need to be rebuilt.

You May Also Like