Printer-friendly version

I can haz kernel programming, pleez?

Not just yet. Soon, but not just yet since there is still one more lesson worth of kernel fundamentals we need to cover before you write, build and load your first kernel module. And you'll be using what you learn in this lesson almost immediately with that first kernel module of yours, so take your time and digest all of what follows here. It's important.

And, surprisingly, there's quite a lot I want to explain here so, without further ado, away we go.

What's a loadable module, anyway?

I shouldn't need to beat this to death. Most readers know that a Linux loadable kernel module (or kernel module, or loadable module, or module, or LKM, etc, etc) is simply a compiled object file that you can load into kernel space, at which point it's now just another part of the running kernel and provides some new functionality.

The benefits of loadable modules should be obvious. Anything which is running in kernel space takes up RAM, so the larger your kernel, the more RAM it's consuming. If you need some functionality only rarely, you can build that functionality as a loadable module and load it as needed.

In addition, loadable modules give you the opportunity of writing, say, a new device driver, then loading it, testing it, identifying bugs, unloading it, fixing it, recompiling it, reloading it, and so on, all without rebooting your system. It should be obvious that this is the perfect vehicle for new device driver development. But one last point.

A lot of people use the phrases "module" and "device driver" interchangeably. Technically, that's not accurate. While loadable modules are primarily used for device drivers, they can certainly be used for other purposes -- you'll see later in this course how to write and load modules that help you debug the kernel. So while we'll certainly be referring to device drivers constantly throughout the rest of this course, try to keep the above distinction in mind.

Does your kernel support loadable modules?

Almost certainly. In fact, it would be astonishing if it didn't since all modern Linux distros come with a massive number of loadable modules ready to go. But there are exceptions.

You could, if you wanted to, configure and build a kernel that had no module support whatsoever. From the kernel configuration menus you saw in the earlier lessons, here's the section related to loadable modules:

--- Enable loadable module support
[ ]   Forced module loading
[*]   Module unloading
[ ]     Forced module unloading
[*]   Module versioning support
[*]   Source checksum for all modules          

I'm not going to spend any time explaining the above because it's highly unlikely you'll ever need to mess with it. In special cases where you're building a tiny embedded system, you might want to configure a kernel with everything built in and no module support whatsoever, but that's unusual and we won't discuss it any further, at least for now. Onward.

The two names for a kernel module

Because I like to be precise, let me explain the two different types of "names" we'll use to refer to kernel modules from now on.

First, there's what you might think of as the, say, simple name, such as vfat, as in, "Of course you can't mount that USB stick, you haven't loaded the vfat module, silly!" In this situation, the name vfat clearly identifies a particular module, and this type of name is the proper form to use in a number of module-related commands.

On the other hand, at times, you'll need to refer to the actual filename containing the loadable module code, such as vfat.ko, either as a relative pathname or possibly fully-qualified if you need to specify precisely where to find it. I mention this difference since some module commands expect to get a simple module name and they'll take it from there, whereas others need a precise filename. You'll be able to recognize the difference shortly.

And one more point. Readers who have been around since version 2.4 of the kernel might remember that loadable modules used to be stored in files with a normal object suffix of .o. Nowadays, the file prefix is .ko, short for "kernel object," otherwise known as a module.

Your current kernel's modules

For the rest of this lesson, we're going to be working with the loadable modules that come with your distribution and since I'm writing this course to match my fully-updated Ubuntu 10.04 system, if that's your distro and you've been playing with a newer kernel based on the first couple of lessons, you should reboot to the current Ubuntu version of 2.6.32-22-generic, which will guarantee that you'll see exactly the same results when you run your commands as I do. If you're running a different kernel version or a different distro, much of what follows will probably still work just fine.

First, where are all those modules that came with your OS? You might remember that they're all under /lib/modules/, in a separate subdirectory for each installed kernel. My system currently looks like this:

$ ls /lib/modules
2.6.32-21-generic  2.6.34      2.6.35-crash+
2.6.32-22-generic  2.6.34-rc7  2.6.35-kwlug+
$

and you can see which kernel version I'm running with:

$ uname -a
Linux lynx 2.6.32-22-generic #36-Ubuntu SMP Thu Jun 3 19:31:57 UTC 2010 x86_64 GNU/Linux
$

so if I wanted to recursively display all of the available modules for my current kernel, I could simply:

$ cd /lib/modules/2.6.32-22-generic/
$ find . -name "*.ko"
./kernel/arch/x86/kernel/microcode.ko
./kernel/arch/x86/kernel/msr.ko
./kernel/arch/x86/kernel/cpuid.ko
./kernel/arch/x86/kernel/cpu/mcheck/mce-xeon75xx.ko
./kernel/arch/x86/kernel/cpu/mcheck/mce-inject.ko
./kernel/arch/x86/kernel/cpu/cpufreq/p4-clockmod.ko
./kernel/arch/x86/kernel/cpu/cpufreq/speedstep-lib.ko
./kernel/arch/x86/kvm/kvm.ko
... and on and on and on ...

and shortly, we'll be picking on some of those modules to demonstrate loading and unloading, but one more point first.

You can always get the version of the running kernel with:

$ uname -r
2.6.32-22-generic
$

which is handy since you can automate generating the name of the appropriate modules directory with either of /lib/modules/`uname -r` or /lib/modules/$(uname -r). This is useful to remember since it allows you to write, say, shell scripts that manipulate modules, knowing that no matter what kernel version you're running, you'll be accessing the corresponding modules directory.

Which modules are currently loaded?

Easy -- there's the lsmod command:

$ lsmod
Module                  Size  Used by
cryptd                  8116  0 
aes_x86_64              7912  1 
aes_generic            27607  1 aes_x86_64
nls_iso8859_1           4633  1 
nls_cp437               6351  1 
vfat                   10802  1 
fat                    55350  1 vfat
binfmt_misc             7960  1 
ppdev                   6375  0 
... etc etc ...

In fact, as the man page for lsmod makes clear, "lsmod is a trivial program which nicely formats the contents of the /proc/modules [file], showing what kernel modules are currently loaded." So you could just as easily have done:

$ cat /proc/modules
cryptd 8116 0 - Live 0xffffffffa03b0000
aes_x86_64 7912 1 - Live 0xffffffffa03a9000
aes_generic 27607 1 aes_x86_64, Live 0xffffffffa039d000
nls_iso8859_1 4633 1 - Live 0xffffffffa0396000
nls_cp437 6351 1 - Live 0xffffffffa038f000
vfat 10802 1 - Live 0xffffffffa0329000
fat 55350 1 vfat, Live 0xffffffffa0313000
... etc etc ...

I think you can see the obvious aesthetic advantage of lsmod. We'll stick with that.

Loading and unloading modules

As a simple example of loading and unloading a single module, let's pick an example that already comes with our Ubuntu 10.04 system that is unlikely to have any effect on your system. From under the appropriate /lib/modules directory, I'm going to experiment on the module that provides support for the Btrfs filesystem, kernel/fs/btrfs/btrfs.ko.

In a nutshell, run the following commands that verify that that module is not loaded, then load it, then check again:

$ lsmod | grep btrfs
$ sudo modprobe btrfs
$ lsmod | grep btrfs
btrfs                 476631  0 
zlib_deflate           21834  1 btrfs
libcrc32c               1244  1 btrfs
$

And when you're done with it, simply:

$ sudo modprobe -r btrfs
$ lsmod | grep btrfs
$

Yes, it's really that simple. Now some notes about the above.

First, you need root privilege to run modprobe. Since we're running Ubuntu, we'll use the sudo utility. If you're running a different distro, figure out what it takes.

Next, you'll notice that loading that single module automatically dragged in its two dependencies that were not yet loaded. That's how modprobe works -- it examines the dependency information in the corresponding modules.dep file for the running kernel where, in our case, it would eventually find something resembling the following:

...
kernel/fs/btrfs/btrfs.ko:
  kernel/lib/zlib_deflate/zlib_deflate.ko
  kernel/lib/libcrc32c.ko
...

And, finally, there are a couple simpler (and older) utilities for loading and unloading modules -- insmod and rmmod -- but they aren't as sophisticated and don't handle dependencies, so we're going to avoid them for now. But they will suddenly become important when we start loading and unloading the modules we write. You'll see that in the next lesson.

Poking at a module with modinfo

If you just want to know about the attributes of a particular module, there's the terrific command modinfo:

$ modinfo btrfs
filename:       /lib/modules/2.6.32-22-generic/kernel/fs/btrfs/btrfs.ko
license:        GPL
srcversion:     809EFA84861CF7190ECDB22
depends:        libcrc32c,zlib_deflate
vermagic:       2.6.32-22-generic SMP mod_unload modversions
$

Note that since that's just an informational command, it doesn't require root privilege, and also note that it lists the module's dependencies.

In addition, it doesn't matter if the module is loaded or not, modinfo works equally well. Feel free to check the man page for that command to see how specific you can be in what information you're asking for.

Loadable modules and the kernel address space

Up to now, everything laying the groundwork for finally writing our first module in the next lesson has been fairly straightforward. So let's push the envelope a bit to finish this lesson off and show you something you might not have seen before.

Future lessons are going to get into working in the kernel address space, and which symbols are available in that address space and which aren't, and how loading your module affects what gets added. Well, let's look at the kernel address space symbol table right now:

$ less /proc/kallsyms
0000000000000000 D per_cpu__irq_stack_union
0000000000000000 D __per_cpu_start
0000000000004000 D per_cpu__gdt_page
0000000000005000 d per_cpu__exception_stacks
000000000000b000 d per_cpu__idt_desc
000000000000b010 d per_cpu__xen_cr0_value
000000000000b018 D per_cpu__xen_vcpu
000000000000b020 D per_cpu__xen_vcpu_info
000000000000b060 d per_cpu__mc_buffer
... etc etc ...

That listing typically goes on for tens of thousands of lines, and represents the various types of symbols and their corresponding addresses in the kernel address space. We'll be digging into that output in far more detail in a later lesson but, for now, I'm going to leave you with the following exercise.

Exercise for the student: Make sure the btrfs module is not loaded, and scan the contents of /proc/kallsyms to verify that nothing in the kernel address space seems to be related to the Btrfs. Load that module, then check for the difference. Leave your observations in the comments section. Finally, unload that module, and check again. Sound good?

Coming next -- writing and loading your very own module

Having covered everything you need to know, lesson four will finally show you the absolute basics of writing, compiling and loading your first kernel module. And I will remind you that while that next lesson is still freely available to everyone, anyone who wants to continue taking this course after lesson four will have to subscribe. But the first four lessons will remain online at no charge permanently and, yes, a full course syllabus is coming soon. I promise.

Thoughts?

UPDATE: If all goes well, Lesson Four will be up by sometime tomorrow.