Post-Migration Drive Issue
fsck from util-linux 2.20.1
fsck.ext3: No such file or directory while trying to open /dev/disk/by-uuid/8e0cc405-efcc-4dcb-bf88-c1d628259680
Possibly non-existent device?
mountall: fsck / [2135] terminated with status 8
mountall: Unrecoverable fsck error: /
Serious errors were found while checking the disk drive for /.
Press I to ignore, S to skip mounting, or M for manual recovery
The UUID specified in the error message points to my "root" filesystem in my fstab
> cat /etc/fstab
/proc /proc proc defaults 0 0
/sys /sys sysfs defaults 0 0
dev /dev tmpfs rw 0 0
# /dev/xvda
UUID=8e0cc405-efcc-4dcb-bf88-c1d628259680 / ext3 defaults,relatime 0 1
#/dev/xvdb
UUID=cae393fd-f525-48cb-87af-c44972692571 none swap defaults 0 0
(Oh, yea…I booted my system using the recovery image, and running fsck on the partition doesn't find any issues.)
One thing that I thought was interesting is that the symbolic link for this drive is missing from the /dev/disk/by-uuid/
However, running the "mount" command shows that it's mounted using that symbolic link
> mount -l
/dev/disk/by-uuid/8e0cc405-efcc-4dcb-bf88-c1d628259680 on / type ext3 (rw,relatime)
/proc on /proc type proc (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
/sys on /sys type sysfs (rw)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
8 Replies
You should ask Linode Support about this. If they don't have an answer for you (which they probably will) I'm sure they'd want to know about this just in case others encounter the same problem.
I don't know if this is the same problem that you're encountering, but there have been a few issues posted in this forum about 32 bit kernels running into some issues after migrating. Booting with a 64 bit kernel seemed to correct the issue.
@hoopycat:
Which distro/version are you running? It probably shouldn't be referring to filesystems by UUID, but rather by actual device (/dev/xvd*)…
Heh…forgot the important things
I'm running Ubuntu 12.04. I'm booting off of the "Latest 64-bit Kernel", but the userland is 32 bit.
This might be a bit of a newbie question, but why wouldn't I want to refer to the file-systems via their UUID? The idea of using the UUID is that even if the filesystem names change, the actual filesystem that gets mounted is always the same. Is there something different when running on a Linode (Xen)?
I'll submit a ticket with Linode (just in case it isn't something I did
I have a Ubuntu 12.04 deploy from a month or two back, and it does use device names (/dev/xvd*) instead of UUIDs… it is a 64-bit userland, tho, so it isn't the exact same image you're running.
While I think the UUID mounts can make sense in a physical machine where it helps to have a configuration that can be independent of what port a particular disk may be plugged into, I think it has less use (and can even be problematic) in a Linode where the xvd* devices are themselves logical and easily configured.
That is, using xvda (for example) for the root device delegates control over what that device is to the Linode Manager configuration, so you can easily pick and choose (or change temporarily) what disk image that maps to. I know I switch out various images for testing - sometimes just user partitions, but also sometimes the boot to test out a different distribution. Having a UUID would break in such cases, at least where you were changing non-boot devices.
– David
While booting, the kernel started printing the following messages over and over on my root partition xvda:
xvda: rw=0, want=2585907792, limit=8396800
attempt to access beyond end of device
Buffer I/O error on device xvda, logical block 1292953895
attempt to access beyond end of device
The solution after a few quick rounds with Linode support was to boot to Finnix and run fsck on my root partition. Fsck fixed a bunch of inode issues and then the machine booted cleanly. I also moved to an older 32-bit kernel, but I don't believe this made a difference.
This was the first time (in my memory) running fsck on this machine image in over 5 years with Linode. Impressive.