I think my Linux died - Please Help! [SOLVED]

By the way I did rename /lib/tls but it didn't help. I'm sill getting segfaults during bootup. Please someone help!

Thanks,

LEC

Wow, I guess this is a bigger problem than I thought. Hmmm, then I guess I have to reinstall…

Thanks,

LEC

The first set of error messages indicate that your package database is corrupt:

Fatal error, run database recovery

Your package manager (yum) should have an option to recreate the database from installed files. Find the option and run it.

The second set of error messages indicate there is something wrong with /bin/grep – perhaps some auto-install process messed up with grep or a dependency. Try reinstalling /bin/grep (package: grep) with your package manager.

HTH,

Cliff

Thank you for your response. I looked up how to rebuild the rpm database but that didn't help. And unfortunately yum doesn't work without internet access so I cannot even remove grep.

Thanks,

LEC

@lec:

I looked up how to rebuild the rpm database but that didn't help.

Like this?

http://jimmetzger.net/sediments/2006/11 … fixup.html">http://jimmetzger.net/sediments/2006/11/yumdatabasefixup.html

Yes, that's exactly what I found, on another site though. My network still doesn't start without grep and I can't run yum without a network. Catch 22…

Thanks,

LEC

This is what I get in /var/log/messages when I do a /etc/init.d/network restart:

Jan 31 20:30:37 localhost kernel: line_ioctl: tty0: ioctl TIOCLINUX called
Jan 31 20:30:37 localhost last message repeated 2 times
Jan 31 20:30:37 localhost ifdown: network-functions: line 39:  4661 Done                    basename $CONFIG
Jan 31 20:30:37 localhost ifdown:       4664 Segmentation fault      | grep -q '[^g]-'
Jan 31 20:30:37 localhost ifdown: network-functions: line 233:  4668 Done                    echo $1
Jan 31 20:30:37 localhost ifdown:       4669 Segmentation fault      | grep -q ':'
Jan 31 20:30:37 localhost ifdown: network-functions: line 233:  4670 Done                    LC_ALL=C ip -o link ls dev $1 2>/dev/null
Jan 31 20:30:37 localhost ifdown:       4671 Segmentation fault      | grep -q ",UP"
Jan 31 20:30:38 localhost ifdown: network-functions: line 233:  4677 Done                    echo $1
Jan 31 20:30:38 localhost ifdown:       4678 Segmentation fault      | grep -q ':'
Jan 31 20:30:38 localhost ifdown: network-functions: line 233:  4679 Done                    LC_ALL=C ip -o link ls dev $1 2>/dev/null
Jan 31 20:30:38 localhost ifdown:       4680 Segmentation fault      | grep -q ",UP"
Jan 31 20:30:38 localhost ifdown: network-functions: line 39:  4687 Done                    basename $CONFIG
Jan 31 20:30:38 localhost ifdown:       4689 Segmentation fault      | grep -q '[^g]-'
Jan 31 20:30:38 localhost ifdown: network-functions: line 293:  4693 Done                    LC_ALL=C ip route list match 0/0
Jan 31 20:30:38 localhost ifdown:       4694 Segmentation fault      | grep -q default
Jan 31 20:30:38 localhost ifdown: network-functions: line 293:  4695 Done                    LC_ALL=C ip route list match 0/0
Jan 31 20:30:38 localhost ifdown:       4696 Segmentation fault      | grep -q default
Jan 31 20:30:38 localhost network: Shutting down loopback interface:  succeeded
Jan 31 20:30:38 localhost kernel: line_ioctl: tty0: ioctl TIOCLINUX called
Jan 31 20:30:38 localhost sysctl: net.ipv4.ip_forward = 0
Jan 31 20:30:38 localhost sysctl: net.ipv4.conf.default.rp_filter = 1
Jan 31 20:30:38 localhost sysctl: net.ipv4.conf.default.accept_source_route = 0
Jan 31 20:30:38 localhost sysctl: kernel.sysrq = 0
Jan 31 20:30:38 localhost sysctl: kernel.core_uses_pid = 1
Jan 31 20:30:38 localhost network: Setting network parameters:  succeeded
Jan 31 20:30:38 localhost kernel: line_ioctl: tty0: ioctl TIOCLINUX called
Jan 31 20:30:38 localhost ifup: network-functions: line 39:  4733 Done                    basename $CONFIG
Jan 31 20:30:38 localhost ifup:       4736 Segmentation fault      | grep -q '[^g]-'
Jan 31 20:30:38 localhost ifup: network-functions: line 179:  4740 Done                    LC_ALL= LANG= ip -o link
Jan 31 20:30:38 localhost ifup:       4741 Segmentation fault      | grep -q $1
Jan 31 20:30:39 localhost ifup: network-functions: line 179:  4742 Done                    LC_ALL= LANG= ip -o link
Jan 31 20:30:39 localhost ifup:       4743 Segmentation fault      | grep -q $1
Jan 31 20:30:39 localhost ifup: Error adding address 127.0.0.1 for lo.
Jan 31 20:30:39 localhost ifup: ./ifup: line 240:  4744 Exit 71                 ethtool -i $DEVICE 2>/dev/null
Jan 31 20:30:39 localhost ifup:       4745 Segmentation fault      | grep -q "driver: bonding"
Jan 31 20:30:39 localhost ifup: ./ifup: line 258:  4746 Done                    iptables -L -n 2>/dev/null
Jan 31 20:30:39 localhost ifup:       4747 Segmentation fault      | LC_ALL=C grep -q RH-Lokkit-0-50-INPUT
Jan 31 20:30:39 localhost ifup: ./ifup: line 429:  4752 Done                    LC_ALL=C ip addr ls ${REALDEVICE}
Jan 31 20:30:39 localhost ifup:       4753 Segmentation fault      | LC_ALL=C grep -q "${IPADDR}/${PREFIX}"
Jan 31 20:30:39 localhost ifup: RTNETLINK answers: File exists
Jan 31 20:30:39 localhost ifup: ./ifup: line 443:  4760 Exit 71                 ethtool -i $DEVICE 2>/dev/null
Jan 31 20:30:39 localhost ifup:       4761 Segmentation fault      | grep -q "driver: bonding"
Jan 31 20:30:39 localhost ifup: network-functions: line 39:  4768 Done                    basename $CONFIG
Jan 31 20:30:39 localhost ifup:       4771 Segmentation fault      | grep -q '[^g]-'
Jan 31 20:30:39 localhost ifup: /etc/sysconfig/network-scripts/ifup-post: line 85:  4784 Done                    iptables -L -n 2>/dev/null
Jan 31 20:30:39 localhost ifup:       4785 Segmentation fault      | grep -q RH-Lokkit-0-50-INPUT
Jan 31 20:30:39 localhost network: Bringing up loopback interface:  succeeded

@lec:

Yes, that's exactly what I found, on another site though. My network still doesn't start without grep and I can't run yum without a network. Catch 22…

Ah, of course!

But there is still a way to (try to) rescue this.

Set up a Finnix profile in the LPM that includes your CentOS disk image(s) in the profile, and boot into that. Once booted, set up a chroot into your CentOS distro, enter it, and then run the yum/rpm db recovery process (above).

Then fix the rest of installation (i.e. broken grep), exit chroot and reboot into CentOS … .

Cliff

Thank you for your response. Unfortunately, I'm not too Linux savy so I only got to the point where I have Finnix setup and I can mount the disk of my CentOS. Is there any instruction on how to setup chroot into my CentOS from Finnix? I searched the wiki page but it returns 0 results for the word finnix. :(

Thanks,

LEC

@lec:

… I only got to the point where I have Finnix setup and I can mount the disk of my CentOS.

Is there any instruction on how to setup chroot into my CentOS from Finnix?

Assuming your CentOS disk is at /dev/ubdb and mounted at /mnt/ubdb (by Finnix):

mount /proc /mnt/ubdb/proc -t proc
chroot /mnt/ubdb /bin/bash

If successful, you will now have a root shell in your CentOS disk, along with network access from the Finnix boot. (Everything you do now is as if you were in a regular ssh environment in your CentOS.)

Do your repair job on the database, carry out any updates, and then make sure that your grep works properly.

echo hello > file.txt
grep hello file.txt
rm file.txt

If all is well your CentOS has been repaired and it is just a matter of issuing the reboot command via your LPM. (Make sure that CentOS, is the default profile, else an unscheduled reboot will put you back in Finnix.)

Good luck!

Cliff

Thank you Cliff. I did what you said but unfortunately it didn't work. First. :) Here's the full story.

When I chrooted to my old distro I was able to do "yum remove grep" which of course told me that there were 194 dependecies. So I googled around and found that I can remove the rpm entry and then I can install grep. So that's what I did. Unfortunately, this still didn't fix the segfaults. Since I also had a brand new install of CentOS (I figured I had to start over) I copied grep from the new distro to the old one. That fixed the segfault for grep. Cool! Reboot, same segfaults. :(

So finally, after many copy/reboot, I ended up copying the whole /bin dir to the old distro and that fixed the problem! :D

I still can't help but wonder what caused this problem. Can it be that the hard drive got damaged? Or the file system? Anybody has any ideas?