The Problem
On CentOS/RHEL system, an LVM volume group size has been extended and an attempt to do online resize of ext4 filesystem on LVM volume group failed with following error.
# resize2fs /dev/testvg/testlv resize2fs 1.42.9 (28-Dec-2013) Filesystem at /dev/testvg/testlv is mounted on /test; on-line resizing required old_desc_blocks = 25, new_desc_blocks = 32 resize2fs: Permission denied to resize filesystem
The offline resize by umounting the filesystem still works.
The soultion
Since the file system is in an inconsistent state, a file system check is required for this affected file system, so online resizing is not possible and the file system has to be unmounted first.
1. To verify if the filesystem is incosistent, use the dumpe2fs command output.
dumpe2fs 1.42.9 (28-Dec-2013)
Filesystem volume name: /test
Last mounted on: /test
Filesystem UUID: b0e01c5c-3b36-4621-b542-xxxxxxxxx
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean with errors
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 13107200
Block count: 52427776
Reserved block count: 2621388
Free blocks: 15377115
Free inodes: 13107015
You can also verify the same from the dmesg command output.
# dmesg .... 8.657211] EXT4-fs (dm-8): warning: mounting fs with errors, running e2fsck is recommended [ 8.658317] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: (null) [ 309.278358] EXT4-fs (dm-8): error count since last fsck: 2 [ 309.278396] EXT4-fs (dm-8): initial error at time 1457487777: ext4_journal_check_start:56 [ 309.278404] EXT4-fs (dm-8): last error at time 1457488200: ext4_journal_check_start:56 [86771.970377] EXT4-fs (dm-8): error count since last fsck: 2 [86771.970393] EXT4-fs (dm-8): initial error at time 1457487777: ext4_journal_check_start:56 [86771.970396] EXT4-fs (dm-8): last error at time 1457488200: ext4_journal_check_start:56 [123392.694192] hpet1: lost 5 rtc interrupts [173243.884657] EXT4-fs (dm-8): error count since last fsck: 2 [173243.884680] EXT4-fs (dm-8): initial error at time 1457487777: ext4_journal_check_start:56 [173243.884688] EXT4-fs (dm-8): last error at time 1457488200: ext4_journal_check_start:56
2. After you have confirmed that the filesystem has inconsistency, you can umount the filesystem.
# umount /test
3. Perform a file system check on the filesystem,
# e2fsck -y /dev/testvg/testlv
4. Try resizing the filesystem now.
# resize2fs /dev/testvg/testlv
5. Remount the file system.
# mount /dev/testvg/testlv /test
6. Check if the file system has been resized.
# df -h /test
Further Troubleshooting
If the issue still persists after the file system check , collect the strace of resize2fs. The output of the strace command can be provided to the support team to further analyze the issue.
# strace -ffxvto resize2fs.strace resize2fs /dev/testvg/testlv
Also make sure that the fsck is disabled on the device during boot, as this might render system unbootable.
# cat /etc/fstab /dev/testvg/testlv /test ext4 _netdev 0 0