The Geek Diary

After Reboot RAID1 Disk Gets Removed/Out of Sync (CentOS/RHEL)

by

The Problem

A faulty disk was replaced in CentOS/RHEL 6 machine. RAID1 was rebuilt and the server was rebooted. After reboot, disk got removed/out of sync from RAID. RAID had to be rebuilt after every reboot.

RAID rebuilt before reboot.

# mdadm --detail /dev/md1
/dev/md1:
Version : 0.90
Creation Time : Fri Feb 8 10:20:51 2013
Raid Level : raid1
Array Size : 488279488 (465.66 GiB 500.00 GB)
Used Dev Size : 488279488 (465.66 GiB 500.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Persistence : Superblock is persistent
Update Time : Thu Jul 7 09:15:04 2016
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

UUID : 1e712c78:0c3dc259:9acdafa7:d4878cfe
Events : 0.2962397

Number Major Minor RaidDevice State
0 67 2 0 active sync /dev/sdaw2
1 67 18 1 active sync /dev/sdax2

After reboot /dev/sdaw2 gets removed:

# mdadm --detail /dev/md1
/dev/md1:
version : 0.90
creation time : fri feb 8 10:20:51 2013
raid level : raid1
array size : 488279488 (465.66 gib 500.00 gb)
used dev size : 488279488 (465.66 gib 500.00 gb)
raid devices : 2
total devices : 1
preferred minor : 1
persistence : superblock is persistent

update time : fri jul 8 17:53:04 2016
state : active, degraded
active devices : 1
working devices : 1
failed devices : 0
spare devices : 0

uuid : 1e712c78:0c3dc259:9acdafa7:d4878cfe
events : 0.2976471

number major minor raiddevice state
0 0 0 0 removed
1 67 18 1 active sync /dev/sdax2
# cat /etc/mdadm.conf
array /dev/md1 level=raid1 num-devices=2 metadata=0.90 uuid=1e712c78:0c3dc259:9acdafa7:d4878cfe
devices=/dev/sdaw2,/dev/sdax2
array /dev/md0 level=raid1 num-devices=2 metadata=0.90 uuid=5e7743f6:b2efa5e2:7a28469a:06cda6cd
devices=/dev/sdaw1,/dev/sdax1

The Solution

initramfs was not picking the latest configuration from /etc/mdadm.conf.

Note: initramfs had to be recreated. Also, the procedure requires the system reboot.

1. Check “mdadm –detail /dev/mdX” (mdX will be the RAID device name) and make sure device state is “clean” and disk is in “”active sync”” mode

mdadm --detail /dev/mdX

2. Take backup of existing initramfs:

# mv /boot/initramfs-2.6.39-400.264.13.el6uek.x86_64.img /tmp/      ### example kernel, version may differ

3. Recreate initramfs with dracut:

# dracut -f /boot/initramfs-2.6.39-400.264.13.el6uek.x86_64.img 2.6.39-400.264.13.el6uek.x86_64

4. Reboot the system.

# shutdown -r now

You May Also Like