Every few weeks, the file system of Phant’s laptop gets corrupted. Maybe at some point it won’t be fixable anymore, but these steps always worked (for now).
It’s an Thinkpad x240, and because someone else’s x240 had the same problem, it’s most likely a hardware issue with the specific model. Curiously, both problems started in January 2020. Planned obsolescence anyone? Would Lenovo do such atrocities?
NOTE: This guide mostly applies to xubuntu installations, which used full disk encryption at installation. If you have a different setup, you will have to adjust the commands. In doubt, you can ask me: https://lefherz.net/contact
Still, even if you have a different problem, this guide might help with finding your solution. It combines the knowledge I gathered during several DIY data rescue sessions.
Also, if something in this guide is not the optimal way of doing things, please let me know in the comments or per mail, and I will fix it.
Usually the problem we need to fix is the following:
- You turn the Laptop on
- It boots into GRUB
- You select Xubuntu in GRUB, or wait a few seconds
- It shows 3 lines of white warnings on black background (the errors seem non-critical) for a few seconds
- After a few seconds, it shows the blue Xubuntu decryption screen
- You enter the password, and hit enter
- After the decryption is successful, the screen turns black again, shows the warnings from before, but then starts an ISOLINUX boot rescue shell.
How to Fix the Inconsistencies
So decryption works, but booting Xubuntu from the decrypted filesystem doesn’t. In this case (and only in this case), the following steps seem to work, as long as the filesystem inconsistencies are harmless enough, and recovery is possible at all:
Get a Shell
- Boot a Xubuntu live stick (or a Tails stick, or anything with a linux shell)
- Click on “Try Xubuntu” (or login into Tails – with Tails you need to specify an Admin password)
- Open a Terminal with ctrl+alt+t (with Tails, press the Windoof key and choose “Terminal”)
Decrypt the Data
Now you have a shell. During the whole process, you can execute
lsblk to show the status of your hard drives. Is it already decrypted, is it recognized, etc.
Now decrypt your hard drive (probably /dev/sda5. If not, look at lsblk what’s the filename of the hard drive):
setxkbmap de # this changes your keyboard layout to german; necessary if your passphrase contains y,z,ä,ü,ö etc. sudo cryptsetup luksOpen /dev/sda5 sda5-decrypted # enter your passphrase to decrypt your xubuntu volume group
Now there should be the file /dev/mapper/xubuntu-vg–root, your decrypted root partition. If not, you might need to activate the xubuntu volume group with these commands:
sudo vgscan # scan for volume groups which you could mount. Maybe not necessary sudo vgchange -ay # activate all volume groups (probably only one, the xubuntu-vg)
Try Automated Fixing
At least now there should be the /dev/mapper/xubuntu-vg–root file. Now we could mount it, or check it for inconsistencies:
sudo fsck.ext4 -vn /dev/mapper/xubuntu-vg--root
This doesn’t make any changes to the partition, it just shows the errors. If it returns no output, then the error is probably somewhere else, and this guide is not helpful for your problem. If it talks about corrupt inodes, inconsistencies etc., you can try to repair it automatically. This fixes only things which are so obvious that no human interaction is necessary. It will most likely fail:
sudo fsck.ext4 -vp /dev/mapper/xubuntu-vg--root
If this runs without error, fine. Try to reboot your Laptop again and see whether it works. If not, we can try to just fix ALL the problems fsck finds. This is a bit desperate – because it might go wrong.
Backup the Partition
If it’s important data you haven’t backed up anywhere else, make a backup of the whole partition first. If you have an external hard drive or so, with enough space (several hundred GB), it will take hours. If the problems are bad, days maybe. You can backup the partition like this:
# Install the tool to copy the partition sudo apt update sudo apt install -y gddrescue # no, the g is not a typo. It's a gnome tool, that's why there is a g. # First, rescue the most important part. # Note: you have to replace the path with some path on your external hard drive. # CAREFUL: make sure you don't accidentally overwrite a file, or your entire external hard drive with that ddrescue -i0 -s50MiB /dev/mapper/xubuntu-vg--root /media/xubuntu/path/on/external/hard/drive/xubuntu-vg--root.backup mapfile ddrescue -i0 -s1MiB -d -r3 /dev/mapper/xubuntu-vg--root /media/xubuntu/path/on/external/hard/drive/xubuntu-vg--root.backup mapfile # Second, rescue some key disc areas: ddrescue -i30GiB -s10GiB /dev/mapper/xubuntu-vg--root /media/xubuntu/path/on/external/hard/drive/xubuntu-vg--root.backup mapfile ddrescue -i230 -s5GiB /dev/mapper/xubuntu-vg--root /media/xubuntu/path/on/external/hard/drive/xubuntu-vg--root.backup mapfile # Third, rescue the rest of the disk: ddrescue /dev/mapper/xubuntu-vg--root /media/xubuntu/path/on/external/hard/drive/xubuntu-vg--root.backup mapfile ddrescue -d -r3 /dev/mapper/xubuntu-vg--root /media/xubuntu/path/on/external/hard/drive/xubuntu-vg--root.backup mapfile
If you can’t install ddrescue for some reason, you can also just use dd. But in case there are hardware errors with your hard drive, it’s less safe (and slower):
sudo dd if=/dev/mapper/xubuntu-vg--root of=/media/xubuntu/path/on/external/hard/drive/xubuntu-vg--root.backup bs=4M status=progress
Try Reckless Fixing
Now that we have a recent backup of the partition, we can try the reckless fixing with fsck – it assumes that you just answer “yes” to all questions fsck asks you:
sudo fsck.ext4 -vy /dev/mapper/xubuntu-vg--root
Mount the Partition
After this, you should be able to mount the partition:
sudo mount /dev/mapper/xubuntu-vg--root /mnt
Now you can look at your files in a file browser. Your home directory will be at /mnt/home/$your-username, for example.
To see whether the boot issue was fixed, you can also reboot the laptop now:
Did it Go Wrong? Restore the Backup!
If things went south, you might need to restore the backup. This can be done with dd:
sudo dd if=/media/xubuntu/path/on/external/hard/drive/xubuntu-vg--root.backup of=/dev/mapper/xubuntu-vg--root bs=4M status=progress
If the stuff above didn’t work, and you desperately need the data, consider paying for a specialist.
If it did work – great! I hope this post was helpful.