Sometimes if vmware vm got shutdown inappropriately, the filesystem may crush or has error on next reboot, and / drive will become read-only and none of software usable at all.

To fix this issue, we can simply force / to be remounted and forcely repair disk.

For example, we have an Ubuntu server with disk failure, if we check its mounted disk, we’ll see / is read-only:

# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=1988484k,nr_inodes=497121,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=403992k,mode=755)
/dev/mapper/ubuntu--vg-ubuntu--lv on / type ext4 (ro,relatime,data=ordered)

and if we check its disk layout:

# df -h
Filesystem                         Size  Used Avail Use% Mounted on
udev                               1.9G     0  1.9G   0% /dev
tmpfs                              395M  6.0M  389M   2% /run
/dev/mapper/ubuntu--vg-ubuntu--lv  7.9G  3.7G  3.8G  49% /

Now forcely remount and repair it:

mount -o remount -o ro /
# fsck -fy /dev/mapper/ubuntu--vg-ubuntu--lv
fsck from util-linux 2.31.1
e2fsck 1.44.1 (24-Mar-2018)
Pass 1: Checking inodes, blocks, and sizes
Deleted inode 393287 has zero dtime.  Fix? yes

Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences:  -(83420--83430) -(383744--383884) -(387072--387784)
Fix? yes

Free blocks count wrong for group #2 (24622, counted=24633).
Fix? yes

Free blocks count wrong for group #11 (26014, counted=26868).
Fix? yes

Free blocks count wrong (1103657, counted=1104522).
Fix? yes

Inode bitmap differences:  -1508 -1511 -393287
Fix? yes

Free inodes count wrong for group #0 (2, counted=4).
Fix? yes

Free inodes count wrong for group #48 (0, counted=1).
Fix? yes

Free inodes count wrong (342841, counted=342844).
Fix? yes


/dev/mapper/ubuntu--vg-ubuntu--lv: ***** FILE SYSTEM WAS MODIFIED *****
/dev/mapper/ubuntu--vg-ubuntu--lv: ***** REBOOT SYSTEM *****
/dev/mapper/ubuntu--vg-ubuntu--lv: 181444/524288 files (0.1% non-contiguous), 992630/2097152 blocks
# reboot

After reboot, issue should be resovled.