This post provides step-by-step instruction on how to recover the corrupted root filesystem. Booting up the server fails with corrupted the root filesystem as shown below.
Checking all file systems. [/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/VolGroup00/LogVol00 /dev/VolGroup00/LogVol00 contains a file system with errors, check forced. /dev/VolGroup00/LogVol00: Inodes that were part of a corrupted orphan linked list found. /dev/VolGroup00/LogVol00: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. (i.e., without -a or -p options) [FAILED] *** An error occurred during the file system check. *** Dropping you to a shell; the system will reboot *** when you leave the shell. Give root password for maintenance
The error indicates the root filesystem is corrupted. And to recover the corrupted root filesystem, please follow the steps below:
1. Boot the system from the DVD of the same major release as the system. Once the system has successfully booted from the ISO image and boot screen will appear. Type: “linux rescue” without the quotes, and hit enter at the prompt.
boot: linux rescue
2. When prompted for language, and keyboard, provide the pertinent information for the system.
3. When prompted to enable the network devices on the system, select: No
4. Select: Skip when prompted to allow the rescue environment to mount Oracle Linux installation under /mnt/sysimage directory.
5. If not using LVM, execute fsck on the device which contains the filesystem:
# e2fsck -f /dev/sda1 e2fsck 220.127.116.11 (17-May-2014) [...]
6. If using LVM, activate the volumes before running fsck.
# pvscan PV /dev/sda1 VG VolGroup00 lvm2 [96.00 MB / 0 free] Total: 1 [3.34 GB] / in use: 1 [3.34 GB] / in no VG: 0 [0 ]
# vgscan Reading all physical volumes. This may take a while... Found volume group "VolGroup00" using metadata type lvm2
# lvscan INACTIVE '/dev/VolGroup00/LogVol00' [3.34 GB] inherit
# lvdisplay --- Logical volume --- LV Name /dev/VolGroup00/LogVol00 VG Name VolGroup00 LV UUID 335qTi-Ossg-32O5-yQC7-xNON-egdI-frj4et LV Write Access read/write LV Status NOT available LV Size 3.34 GB Current LE 107 Allocation inherit Read ahead sectors 0
# vgchange -ay 1 logical volume(s) in volume group "VolGroup00" now active
# e2fsck -f -y /dev/VolGroup00/LogVol00 e2fsck 18.104.22.168 (17-May-2014) [...]
7. Exit the rescue mode, remove the DVD or boot disc media, then boot the system normally.