Printer-friendly version

Building another kernel for Ubuntu? But why?

An excellent question. Quite simply, for the purposes of future, subscriber-only lessons, we need to (reliably) build a fairly current kernel for Ubuntu 10.04 so that:

• we get the corresponding vmlinux file for that kernel for debugging purposes, and
• the new kernel can handle the root filesystem being on a logical volume, not just in a regular filesystem.

It's that second condition that recently got me into trouble since the much earlier (and much simpler) recipe for building a Ubuntu kernel failed to produce a kernel that would boot and properly mount the logical volume that is the root filesystem on my new laptop, and that's definitely not an acceptable state of affairs.

Because of that, I've perused the official Ubuntu wiki pages -- in particular, this one -- and have put together a recipe that, while considerably more complicated than what you've seen before, seems to work and, in the end, what I'm after is a set of instructions that:

• will produce a suitably bleeding edge kernel,
• works just fine for both 32- and 64-bit systems,
• generates a kernel that can mount a root filesystem that is either a regular ext* filesystem or lives inside a logical volume, and,
• creates actual packages that can be installed and uninstalled conveniently,
• gives us the corresponding raw, uncompressed vmlinux file that can be used for debugging later.

Note: I don't yet completely understand this Ubuntu/Debian build infrastructure so some of what follows I've just copied from the official instructions, but it seems to work for me so please try this out, verify that it works or leave a comment explaining where it fails.

Oh, and I'm going to advertise this post before I'm even done writing it, so follow along and check back frequently to keep up with newer developments. And now, to work ...

The necessary packages

The package installation instruction on that wiki page seems slightly out of date, so just:

$ sudo apt-get install \
 git-core \
 kernel-package \
 fakeroot \
 build-essential \
 libncurses5-dev
$

Movin' on ...

How much will you need to download and copy?

A fair bit, but let me show you what the end result will look like. Given that I placed all of the necessary content under a directory named k2 in my home directory, this is what I ended up with in that directory before I got started compiling (showing the size of each subdirectory in kilobytes so you can see how much downloading or copying you're about to do):

$ du -ks *
886460	linux-2.6
969088	ubuntu-maverick
1996	ubuntu-package
$

As you can see, those first two directories take up almost a Gig of space each, while the third directory is much smaller. So, let's get started getting all that essential content in place.

The linux-2.6/ directory

That directory is, in fact, nothing more than the "git pull" of the upstream, stock Linux development tree and if you don't currently have one, then just:

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git

which will produce the appropriately-named linux-2.6 directory at your current location.

On the other hand, if you've been following along in the lessons for this course, you probably already have that checkout so you don't need to check it out again -- just cd over to where it is (and there's no need to move it, either).

In either case, once you have that upstream Linux kernel checkout, from its top-level directory, prep it as follows:

$ cp /boot/config-$(uname -r) .config
$ yes '' | make oldconfig
$ make menuconfig   [optional]
$ make-kpkg clean

and that's it for now. Most of that should look familiar, and the only reason you should run make menuconfig is to absolutely guarantee that you've selected the kernel configuration parameters CONFIG_PROC_KCORE and CONFIG_DEBUG_INFO for later debugging (and chances are they're already selected).

That last command is a Debian-specific command for cleaning the source tree and, at that point, you've sufficiently prepped the generic Linux kernel source tree for your Ubuntu build. Now we get to the Ubuntu-specific stuff and more downloading and copying.

Downloading the Ubuntu Maverick kernel tree

For this recipe, you're also going to need to checkout the Ubuntu-specific Maverick kernel tree, and you're free to check that out wherever you want -- it doesn't need to be next to your stock tree, just run all of the following one command after another:

$ git clone git://kernel.ubuntu.com/ubuntu/ubuntu-maverick.git
$ cp -a /usr/share/kernel-package ubuntu-package
$ cp ubuntu-maverick/debian/control-scripts/{postinst,postrm,preinst,prerm} \
 ubuntu-package/pkg/image/
$ cp ubuntu-maverick/debian/control-scripts/headers-postinst \  
 ubuntu-package/pkg/headers/

Once you've done all of that, you now have your Ubuntu-specific content that you need to build your new Ubuntu kernel. At this point, you need to return to the top level of your stock Linux kernel tree for the actual build.

NOTE: All of the above should have just worked since all it involved was simply downloading and copying. If any of that failed, leave a comment describing what happened. Thanks.

Finally, building that new kernel

Once you change directory to your (prepped) stock kernel tree, you need to run the following (where you can see how you need to refer to the directory where you created that Ubuntu content -- the overlay_dir option):

$ CONCURRENCY_LEVEL=`getconf _NPROCESSORS_ONLN` \
 fakeroot make-kpkg --initrd --append-to-version=-rday \
 --overlay-dir=~/k2/ubuntu-package kernel_image kernel_headers

and just watch the build. And this is where things might go wrong.

What might go wrong?

This might go wrong, and fairly quickly:

====== making target debian/stamp/build/kernel [new prereqs: vars]======
This is kernel package version 12.032.
The changelog says we are creating 2.6.35-rc4-rday
However, I thought the version is 2.6.35-rc4-rday+
exit 1
make: *** [debian/stamp/build/kernel] Error 1

This is a known bug and involves how the kernel source scripts create the version number for the kernel about to be built. If your stock kernel tree is not currently sitting at a tagged commit, a versioning script loves to add a trailing "+" to the version string to show that, but that messes up the Ubuntu version comparison test. Apparently, a fix for that has already been pushed but, if you get this error, there's a simple fix -- just list the tags for your stock kernel tree and check out the most recent one, which is more than new enough for what we're doing:

$ git tag -l
... big snip ...
v2.6.35-rc4
$

And we'll take that last one:

$ git checkout v2.6.35-rc4

At this point, just to be safe, clean the stock kernel tree again:

$ make-kpkg clean

and rerun the build command. This time, you should be able to kick back, go for coffee and come back to a successful build.

At some point in the near future, you shouldn't need to do that last hack.

And if it worked?

If all that worked and your build terminates successfully, what you should now find in your parent directory are a couple of installable packages along the lines of:

linux-headers-2.6.35-rc4-rday_2.6.35-rc4-rday-10.00.Custom_amd64.deb
linux-image-2.6.35-rc4-rday_2.6.35-rc4-rday-10.00.Custom_amd64.deb

at which point, you would want to install them:

$ sudo dpkg -i linux*deb

and reboot to that new kernel.

Apparently, it worked

At least for me, I'm up and running under that new kernel:

$ uname -r
2.6.35-rc4-maverick
$

And, just as important, if I check the top of the stock kernel source tree that I used for the build, I have my unstripped, vmlinux file:

$ ls -l vmlinux
-rwxr-xr-x  1 rpjday rpjday 118258443 2010-07-07 07:33 vmlinux
$

and that's what I've been after this whole time, which means I can now go on to discuss how to debug a running kernel (and your loadable modules) with the standard userspace debugger gdb.

Until next lesson, which goes back to being subscriber-only.

SHAMELESS MARKETING: If you're not a subscriber to this course, it's easy -- go here. See? I told you it was easy.