Having troubles with pv-grub and kernel

I am trying to get a custom kernel compiled and booted using pv-grub.

I am using ubuntu, and since its default kernel isn't Xen compatible I choose to download and compile a custom kernel as a test.

From a completely fresh Ubuntu 10.04 lts (32-bit) install I ran the following commands:

# Update the OS base install
aptitude update
aptitude -y safe-upgrade
aptitude -y install bash-completion
aptitude -y install kernel-package libncurses5-dev fakeroot wget bzip2

# Download new kernel
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.34.tar.gz
tar xzf linux-2.6.34.tar.gz
cd linux-2.6.34

# Take the existing working kernel config
cp /proc/config.gz .
gunzip config.gz

# update config with new parameters -- I accepted defaults on all prompts
make oldconfig

# Add the required XEN parameters to the end of the config file
cat <<eof>> config

# Required XEN options
CONFIG_PARAVIRT_GUEST=y
CONFIG_XEN=y
CONFIG_PARAVIRT=y
CONFIG_PARAVIRT_CLOCK=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_HVC_XEN=y
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y
EOF

# Make builds go faster
export CONCURRENCY_LEVEL=8

# Buid the kernel
make-kpkg clean
fakeroot make-kpkg --initrd --append-to-version=-ih1 kernel_image kernel_headers

# Install new kernel
cd ..
dpkg -i linux-headers-2.6.34-ih1_2.6.34-ih1-10.00.Custom_i386.deb
dpkg -i linux-image-2.6.34-ih1_2.6.34-ih1-10.00.Custom_i386.deb

# Install grub and configure to allow Linode pv-grub boots
aptitude -y install grub
update-grub -y

# Make changes to /boot/grub/menu.lst to reflect our environment
sed -i '/^# kopt/ c\# kopt=root=(hd0) ro' /boot/grub/menu.lst
sed -i '/^# groot/ c\# groot=(hd0)' /boot/grub/menu.lst
sed -i '/^# indomU/ c\# indomU=false' /boot/grub/menu.lst

# Update the grub files one more time
update-grub -y</eof> 

This is what my menu.lst file in /boot/grub looks like

# menu.lst - See: grub(8), info grub, update-grub(8)
#            grub-install(8), grub-floppy(8),
#            grub-md5-crypt, /usr/share/doc/grub
#            and /usr/share/doc/grub-legacy-doc/.

## default num
# Set the default entry to the entry number NUM. Numbering starts from 0, and
# the entry number 0 is the default if the command is not used.
#
# You can specify 'saved' instead of a number. In this case, the default entry
# is the entry saved with the command 'savedefault'.
# WARNING: If you are using dmraid do not use 'savedefault' or your
# array will desync and will not let you boot your system.
default         0

## timeout sec
# Set a timeout, in SEC seconds, before automatically booting the default entry
# (normally the first entry defined).
timeout         3

## hiddenmenu
# Hides the menu by default (press ESC to see the menu)
hiddenmenu

# Pretty colours
#color cyan/blue white/blue

## password ['--md5'] passwd
# If used in the first section of a menu file, disable all interactive editing
# control (menu entry editor and command-line)  and entries protected by the
# command 'lock'
# e.g. password topsecret
#      password --md5 $1$gLhU0/$aW78kHK1QfV3P2b2znUoe/
# password topsecret

#
# examples
#
# title         Windows 95/98/NT/2000
# root          (hd0,0)
# makeactive
# chainloader   +1
#
# title         Linux
# root          (hd0,1)
# kernel        /vmlinuz root=/dev/hda2 ro
#

#
# Put static boot stanzas before and/or after AUTOMAGIC KERNEL LIST

### BEGIN AUTOMAGIC KERNELS LIST
## lines between the AUTOMAGIC KERNELS LIST markers will be modified
## by the debian update-grub script except for the default options below

## DO NOT UNCOMMENT THEM, Just edit them to your needs

## ## Start Default Options ##
## default kernel options
## default kernel options for automagic boot options
## If you want special options for specific kernels use kopt_x_y_z
## where x.y.z is kernel version. Minor versions can be omitted.
## e.g. kopt=root=/dev/hda1 ro
##      kopt_2_6_8=root=/dev/hdc1 ro
##      kopt_2_6_8_2_686=root=/dev/hdc2 ro
# kopt=root=(hd0) ro

## default grub root device
## e.g. groot=(hd0,0)
# groot=(hd0)

## should update-grub create alternative automagic boot options
## e.g. alternative=true
##      alternative=false
# alternative=true

## should update-grub lock alternative automagic boot options
## e.g. lockalternative=true
##      lockalternative=false
# lockalternative=false

## additional options to use with the default boot option, but not with the
## alternatives
## e.g. defoptions=vga=791 resume=/dev/hda5
# defoptions=quiet splash

## should update-grub lock old automagic boot options
## e.g. lockold=false
##      lockold=true
# lockold=false

## Xen hypervisor options to use with the default Xen boot option
# xenhopt=

## Xen Linux kernel options to use with the default Xen boot option
# xenkopt=console=tty0

## altoption boot targets option
## multiple altoptions lines are allowed
## e.g. altoptions=(extra menu suffix) extra boot options
##      altoptions=(recovery) single
# altoptions=(recovery mode) single

## controls how many kernels should be put into the menu.lst
## only counts the first occurence of a kernel, not the
## alternative kernel options
## e.g. howmany=all
##      howmany=7
# howmany=all

## specify if running in Xen domU or have grub detect automatically
## update-grub will ignore non-xen kernels when running in domU and vice versa
## e.g. indomU=detect
##      indomU=true
##      indomU=false
# indomU=false

## should update-grub create memtest86 boot option
## e.g. memtest86=true
##      memtest86=false
# memtest86=true

## should update-grub adjust the value of the default booted system
## can be true or false
# updatedefaultentry=false

## should update-grub add savedefault to the default options
## can be true or false
# savedefault=false

## ## End Default Options ##

title           Ubuntu 10.04 LTS, kernel 2.6.34-ih1
root            (hd0)
kernel          /boot/vmlinuz-2.6.34-ih1 root=(hd0) ro quiet splash

title           Ubuntu 10.04 LTS, kernel 2.6.34-ih1 (recovery mode)
root            (hd0)
kernel          /boot/vmlinuz-2.6.34-ih1 root=(hd0) ro  single

title           Ubuntu 10.04 LTS, memtest86+
root            (hd0)
kernel          /boot/memtest86+.bin

### END DEBIAN AUTOMAGIC KERNELS LIST

When I try to boot the kernel by switching to the pv-grub-x86_32 I see this in lish:

  Booting 'Ubuntu 10.04 LTS, kernel 2.6.34-ih1'

root  (hd0)
 Filesystem type is ext2fs, using whole disk
kernel  /boot/vmlinuz-2.6.34-ih1 root=(hd0) ro quiet splash

ERROR Invalid kernel: elf_xen_note_check: ERROR: Will only load images built for the generic loader or Linux images
xc_dom_parse_image returned -1

Error 9: Unknown boot failure

Press any key to continue...

After shutting down the linode and running logview in lish I see this:

Xen Minimal OS!
start_info:   0x9e8000
  nr_pages:   131072  shared_inf: bf271000
  pt_base:    0x9eb000  mod_start:  0x0
  mod_len:    0
  flags:      0x0
  cmd_line:   (hd0)/boot/grub/menu.lst
  stack:      0x947780-0x967780
MM: Init
  _text:        0x0
  _etext:       0x60dec
  _erodata:     0x75000
  _edata:       0x7cb58
  stack start:  0x947780
  _end:         0x967d14
  start_pfn:    9f7
  max_pfn:      20000
Mapping memory range 0xc00000 - 0x20000000
setting 0x0-0x75000 readonly
skipped 0x1000
MM: Initialise page allocator for af1000(af1000)-0(20000000)
MM: done
Demand map pfns at 20001000-a0001000.
Heap resides at a0002000-e0002000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0x20001000.
Initialising scheduler
Thread "Idle": pointer: 0xa0002008, stack: 0xc00000
Initialising xenbus
Thread "xenstore": pointer: 0xa0002478, stack: 0xc10000
Dummy main: start_info=0x967880
Thread "main": pointer: 0xa00028e8, stack: 0xc20000
"main" "(hd0)/boot/grub/menu.lst"
vbd 51712 is hd0
******************* BLKFRONT for device/vbd/51712 **********

backend at /local/domain/0/backend/vbd/405/51712
Failed to read /local/domain/0/backend/vbd/405/51712/feature-flush-cache.
8388608 sectors of 0 bytes
**************************
vbd 51728 is hd1
******************* BLKFRONT for device/vbd/51728 **********

backend at /local/domain/0/backend/vbd/405/51728
Failed to read /local/domain/0/backend/vbd/405/51728/feature-flush-cache.
524288 sectors of 0 bytes
**************************
Press `ESC' to enter the menu... 0
  Booting 'Ubuntu 10.04 LTS, kernel 2.6.34-ih1'

root  (hd0)
 Filesystem type is ext2fs, using whole disk
kernel  /boot/vmlinuz-2.6.34-ih1 root=(hd0) ro quiet splash

ERROR Invalid kernel: elf_xen_note_check: ERROR: Will only load images built for the generic loader or Linux images
xc_dom_parse_image returned -1

Error 9: Unknown boot failure

Press any key to continue...

    GNU GRUB  version 0.97  (524288K lower / 0K upper memory)

 +-------------------------------------------------------------------------+
 | Ubuntu 10.04 LTS, kernel 2.6.34-ih1                                     |
 | Ubuntu 10.04 LTS, kernel 2.6.34-ih1 (recovery mode)                     |
 | Ubuntu 10.04 LTS, memtest86+                                            |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 +-------------------------------------------------------------------------+
    Use the ^ and v keys to select which entry is highlighted.
    Press enter to boot the selected OS, 'e' to edit the
    commands before booting, or 'c' for a command-line.

    GNU GRUB  version 0.97  (524288K lower / 0K upper memory)

       [ Minimal BASH-like line editing is supported.   For
         the   first   word,  TAB  lists  possible  command
         completions.  Anywhere else TAB lists the possible
         completions of a device/filename.  ESC at any time
         exits. ]

grubdom> reboot

Does anyone know what I am doing wrong? I followed the PV-GRUB wiki entry pretty closely, but I am clearly missing something.

Thanks!

5 Replies

The kernel config file is .config not config.

aye, and this

sed -i '/^# kopt/ c# kopt=root=(hd0) ro' /boot/grub/menu.lst

should be

sed -i '/^# kopt/ c# kopt=root=/dev/xvda ro' /boot/grub/menu.lst

````

uname -a

Linux li91-242 2.6.34-ih1 #1 SMP Fri Jul 2 00:54:09 UTC 2010 i686 GNU/Linux

````

It works! Thats for the help guys. I can't believe I missed the '.' on the config. Oh well.

Once I fixed that and recompiled the kernel I ended up with a filesystem error which the_guv just helped me solve.

I have done some testing this morning and the following script works perfectly on Ubuntu 10.04 x86 and amd_64. Thanks again.

KERN_VERS="2.6.34"
KERN_NAME="ih1"

# Update the OS base install 
aptitude update 
aptitude -y safe-upgrade 
aptitude -y install bash-completion kernel-package libncurses5-dev fakeroot wget

# Download new kernel 
cd /usr/src
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-${KERN_VERS}.tar.gz 
tar xzf linux-${KERN_VERS}.tar.gz 
cd linux-${KERN_VERS} 

# Take the existing working kernel config 
cp /proc/config.gz . 
gunzip config.gz 
mv config .config

# update config with new parameters -- I accepted defaults on all prompts 
yes "" | make oldconfig

# Add the required XEN parameters to the end of the config file 
cat <<eof>> .config 

# Required XEN options 
CONFIG_PARAVIRT_GUEST=y 
CONFIG_XEN=y 
CONFIG_PARAVIRT=y 
CONFIG_PARAVIRT_CLOCK=y 
CONFIG_XEN_BLKDEV_FRONTEND=y 
CONFIG_XEN_NETDEV_FRONTEND=y 
CONFIG_HVC_XEN=y 
CONFIG_XEN_BALLOON=y 
CONFIG_XEN_SCRUB_PAGES=y 
EOF

# Make builds go faster 
export CONCURRENCY_LEVEL=2

# Buid the kernel 
make-kpkg clean 
fakeroot make-kpkg --initrd --append-to-version=-${KERN_NAME} kernel_image kernel_headers 

# Install new kernel 
cd .. 
dpkg -i linux-headers-${KERN_VERS}-${KERN_NAME}_${KERN_VERS}-${KERN_NAME}-10.00.Custom_*.deb 
dpkg -i linux-image-${KERN_VERS}-${KERN_NAME}_${KERN_VERS}-${KERN_NAME}-10.00.Custom_*.deb 

# Install grub and configure to allow Linode pv-grub boots 
aptitude -y install grub 
update-grub -y 

cd /boot/grub

# Make changes to /boot/grub/menu.lst to reflect our environment 
sed -i '/^# kopt/ c\# kopt=root=/dev/xvda ro' /boot/grub/menu.lst 
sed -i '/^# groot/ c\# groot=(hd0)' /boot/grub/menu.lst 
sed -i '/^# indomU/ c\# indomU=false' /boot/grub/menu.lst 

# Update the grub files one more time 
update-grub -y 

# Apply patch to get lish console working on new kernel
cat < <eof>/etc/rc.local
# Tweaked rc.local script

# Enable console access in lish
exec /sbin/getty 38400 hvc0 &
exit 0
EOF</eof></eof> 

Change the configuration to pv-grub and reboot.

I did not find that the changes suggested in the Linode wiki were effective at getting the lish console to work again. I hacked in a getty into the rc.local script. Ugly, but effective for my needs.

cool beans invisible hardware

.. think that concurrency level can be 4 if you've got 4 processors .. some say 5 :)

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct