Lesson 8: Module diagnostics, and that init and exit code again.

Printer-friendly versionPrinter-friendly version

What's happening today?

What's happening is that we're going to cover a couple more topics in considerable detail about how modules are loaded and unloaded, and how modules print output where you can get to it. And once we're done with that, we'll be getting into enhancing your loadable module with more and more features, including parameters, ioctl()s, /proc files and more. But all of that is going to start in Lesson 9 so, first things first.

Moving on to crash3 ...

As we've done before, let's start a whole new module directory for your crash3 module, with the standard, corresponding Makefile and the following (slightly more complicated) source in crash3.c:

/* Module source file 'crash3.c'. */

#include <linux/module.h>   // for all modules
#include <linux/init.h>     // for entry/exit macros
#include <linux/kernel.h>   // for printk() definition
#include <asm/current.h>    // process information
#include <linux/sched.h>    // for task_struct definition

static int __init hi(void)
     printk(KERN_INFO "crash3 module being loaded.\n");
     printk(KERN_INFO "User space process is '%s'\n", current->comm);
     printk(KERN_INFO "User space PID is  %i\n", current->pid);
     return 0;

static void __exit bye(void)
    printk(KERN_INFO "crash3 module being unloaded.\n");


MODULE_AUTHOR("Robert P. J. Day, http://crashcourse.ca");
MODULE_DESCRIPTION("Doing a whole lot of nothing.");

Once you create the appropriate Makefile, simply compile the module and, while tailing the log file /var/log/messages, load and unload it as normal, whereupon you should see something resembling the following in that log file:

... crash3 module being loaded.
... User space process is 'insmod'
... User space PID is  2627
... crash3 module being unloaded.

And that's it. Just verify that you can load and unload the module, and verify what you see in the /var/log/messages file as that's done. And once that's finished, the fun begins.

What's with the extra output and header files?

Note the extra header file inclusions and output lines in this version of the module. Without getting into detail, those additional header files give us the declarations of a number of kernel space data structures, while the additional printk lines print some information about the user space process that's responsible for the execution of this module (in this case, the insmod command that loaded the module).

At this point, it's not important to know exactly which header files you'll need for everything you see in future lessons; you'll learn all that in time. The only point being made here is that, as your modules get increasingly complex, you'll need to include more and more kernel space header files to pull in the corresponding declarations, that's all. And after a while, you'll have a pretty good idea which header files you need based on what your module is trying to do.

Where is your module output going, anyway?

Not surprisingly, everyone wants to print output from their modules, but you need to accept that you are no longer working in user space, where you can invoke a simple printf("Hello, world.\n"). From within your module, you have no convenient access to user space anymore, so you need to learn how you can get access to that output. And that's where the kernel space routine printk() comes in.

printk() is the kernel space equivalent of printf() but, rather than printing to the screen, printk() sends its output to the standard log file /var/log/messages, where you can view it at your leisure as you've seen already. But there's more to it than just that earlier example.

First, you can generate kernel log messages with a variety of priorities, as they're defined in the kernel header file /include/linux/kernel.h:

#define KERN_EMERG   "<0>"   /* system is unusable                */
#define KERN_ALERT   "<1>"   /* action must be taken immediately  */
#define KERN_CRIT    "<2>"   /* critical conditions               */
#define KERN_ERR     "<3>"   /* error conditions                  */
#define KERN_WARNING "<4>"   /* warning conditions                */
#define KERN_NOTICE  "<5>"   /* normal but significant condition  */
#define KERN_INFO    "<6>"   /* informational                     */
#define KERN_DEBUG   "<7>"   /* debug-level messages              */

It should be obvious from the above that you can tag your module output with whatever priority is appropriate -- we've just been using KERN_INFO until now.

Note also that there is no comma between the priority macro and the message you want to print -- that's a common mistake. But the most interesting part of this output feature is how you can configure where your output goes.

Experienced user space programmers will probably recognize those priority levels -- they are, in fact, the same ones you're used to using when you're configuring your system log messages with the syslog facility, or whatever is the equivalent on your system. On my Ubuntu system, the package is named rsyslog, and its configuration file is /etc/rsyslog.d/50-default.conf where the following excerpt should make a certain amount of sense:

auth,authpriv.*                 /var/log/auth.log
*.*;auth,authpriv.none          -/var/log/syslog
#cron.*                         /var/log/cron.log
daemon.*                        -/var/log/daemon.log
kern.*                          -/var/log/kern.log
lpr.*                           -/var/log/lpr.log
mail.*                          -/var/log/mail.log
user.*                          -/var/log/user.log
        news.none;mail.none     -/var/log/debug
        mail,news.none          -/var/log/messages

To make a long story short, the file above defines the destination of all log messages, depending on what facility produces them and what priority the message has. In our case, anything generated by a loadable module has a facility of kern, at which point the message priority determines where the message ends up. As you can see above, if you're used to wildcard patterns, it's not hard to guess that the expression *.=info means that any facility that generates a message with exactly an "info" priority will send that message to the file /var/log/messages. (Explaining more than this about the syslog package is beyond the scope of this lesson.)

Exercise for the student: Predict what will happen with a message you print from your crash3 module with a priority of KERN_DEBUG. Test it.

And about those entry and exit routines ...

There is one more feature of your module's entry and exit routines we can cover now, and it relates to those __init and __exit function qualifiers you can see up there. Technically speaking, they're not essential, they exist simply for the sake of efficiency.

The __init qualifier is the simpler of the two; it identifies any routine that can be discarded once the module has loaded, which makes perfect sense since, once a module's initialization code has executed, it has no further value and it can be thrown away, and that's why you should see that qualifier on every module entry routine you ever run across -- so that entry code doesn't just hang around in kernel memory, wasting space. And as for the __exit qualifier? Well, that one's a bit trickier.

Clearly, it can't also possibly represent code that can be discarded after loading; after all, it's the exit code so it needs to stick around until unloading. But there are two situations where the exit code can be discarded, and both of those situations represent cases where you know for a fact that that exit code will never, ever, ever be called.

The first case is when you've simply configured a kernel that doesn't support module unloading. Unusual, yes, but if you pop back into the kernel configuration menu, you do have the right to build a kernel that allows module loading but doesn't allow module unloading. Admittedly, that's a strange situation but, technically, it can be done and, if that's the case, then there's no possibility of that module ever being unloaded and, therefore, the exit code is superfluous.

The second case is if you eventually add your module code into the kernel source tree and add configuration information for it. If you then choose to add that feature to your kernel as a loadable module, then the exit code has to stay. On the other hand, if you choose to build that feature directly into the kernel, then it's not a loadable module anymore and the exit code will again never have a chance of executing.

And one more point. It's possible to have more than one routine in a source file tagged with either __init or __exit, if either your entry or exit routines call other routines for modularity. There's no reason for all your entry or exit code to exist in a single routine -- you're certainly welcome to break all that functionality over multiple functions if it makes your code more readable.

And we're almost done. Just one more obscure feature to talk about.

Does my module even need an exit routine?

An excellent question -- what if there's absolutely no cleanup required when your module is unloaded? Can you just not define an exit routine at all? Well, sort of. The problem is that, once you load that module, you can't unload it anymore.

The reasoning behind this is (apparently) that if you don't define an exit routine for your module, the kernel simply can't trust your module to simply exit and not leave a mess behind. So if you don't define an exit routine, you're not allowed to leave. It's as simple as that.

You're welcome to test this as long as you're prepared to reboot your system to get rid of that module. Just edit your module source, delete the exit routine, re-compile and reload the module. And this time, when you run lsmod, you'll see something like:

crash3                   815  0 [permanent]

Once that module is in, you're going to have a tough time getting it out:

$ sudo rmmod crash3
ERROR: Removing 'crash3': Device or resource busy

At this point, you have only two options to get rid of that module. You can either reboot the system, or you can use rmmod -f (for "force"), but only if your kernel has been configured to allow forced module unloading. And I can assure you that, if you're running the stock Ubuntu kernel, it does not come configured to allow forced module unloading:

$ sudo rmmod -f crash3
ERROR: Removing 'crash3': Device or resource busy

Exercise for the student: If you're running the stock Ubuntu kernel (and you should be), how can you verify that the kernel doesn't support forced module unloading? Besides, of course, just trying it and having it fail.

And if you're feeling ambitious, build yourself a whole new kernel that supports forced module unloading, and test it again.

Final exercise for the student: RTFS

So how did I know all of the above? It's not as if the kernel documentation is spectacularly complete but, if you're suitably ambitious, you can always Read The Fine Source.

For the keeners among you, if you want to see the details of module management, it's pretty much all in the kernel source file kernel/module.c. And, surprisingly, a lot of it is eminently readable. For instance, you don't need to be a kernel expert to take a wild guess at what this excerpt (down around line 813) does:

if (mod->init != NULL && mod->exit == NULL) {
        printed_something = 1;
        seq_printf(m, "[permanent],");

And this excerpt (around line 759) should also now make some sense:

if (mod->init && !mod->exit) {
        forced = try_force_unload(flags);
        if (!forced) {
                /* This module can't be removed */
                ret = -EBUSY;
                goto out;

And we're done here. Next lesson: module parameters.


Exercise for the student

how can you verify that the kernel doesn't support forced module unloading?

- By checking into /boot/config-2.6.32-23-generic to see if CONFIG_MODULE_FORCE_UNLOAD is set.

kernel source code

I have seen the kernel/module.c file in the linux source code I have downloaded earlier as part of the first chapter. So, the concept explained in this chapter is clear. However, I have a basic question.
I have 2.6.32-21-generic version installed as part of ubuntu10.04. In /usr/src/linux-headers-2.6.32-21-generic/kernel, I don't find the module.c file.
Am I checking in wrong path?
Do we need to install separately the src code?
Won't all the source code come with OS installation normally?

The kernel headers package is a subset of the kernel source.

In most distros, there is some kind of "kernel headers" package that contains just enough of the kernel source tree to compile modules against. That package will contain virtually *no* source since you don't need it for module compilation.

My recommendation is, if you want to keep up with kernel source developments, use "git" to check out the current kernel source tree and keep it up to date.

module code into kernel source tree...

In the paragraph "The second case is if you eventually add your module code..."
How do we add the module code to kernel source tree and the config information?
Where do we choose to make it loadable or built-in?

module code into kernel source tree...

In the paragraph "The second case is if you eventually add your module code..."
How do we add the module code to kernel source tree and the config information?
Where do we choose to make it loadable or built-in?

some error in running above program

here is my program


static int __init hi(void)
printk(KERN_INFO "A module being loaded\n");
printk(KERN_INFO "User space process is %s\n",current->comm);
printk(KERN_INFO "User space PID is %i\n",current->pid);
return 0;
static void __exit bye(void)
printk(KERN_INFO "A module being unloaded");

MODULE_DESCRIPTION("An experiment");

and following is my Makefile

KERNELDIR ?= /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
.PHONY: build clean
$(MAKE) -C $(KERNELDIR) M=$(PWD) modules
rm -rf *.o *~ core .depend .*.cmd *.ko *.mod.c
rm -rf modules.order Module.symvers
obj-m := crash3.o


I execute it and get the following error

electricity@electricity-laptop:~/LKP/pandora/temp/crashcourse/crash3$ make
make -C /lib/modules/2.6.28-11-generic/build M=/home/electricity/LKP/pandora/temp/crashcourse/crash3 modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.28-11-generic'
Building with KERNELRELEASE =2.6.28-11-generic
CC [M] /home/electricity/LKP/pandora/temp/crashcourse/crash3/crash3.o
/home/electricity/LKP/pandora/temp/crashcourse/crash3/crash3.c:3:25: error: linux/kerne.h: No such file or directory
make[2]: *** [/home/electricity/LKP/pandora/temp/crashcourse/crash3/crash3.o] Error 1
make[1]: *** [_module_/home/electricity/LKP/pandora/temp/crashcourse/crash3] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.28-11-generic'
make: *** [build] Error 2

So what mistake I did?

Re:some error in code

Ok figured out I used the header file as kerne.h and not kernel.h
also on the comments section I can not post any thing between <
in comments.

More fun

Since I'm running Ubuntu 11.04 (2.6.38-8-generic), this was instructive:

$ git tag -l # to find names of tags
$ git diff v2.6.32 -- module.c # changes since then

Is that a lot? Here's how I looked:

$ wc -l module.c # How many lines in the file?
$ git diff v2.6.32 -- module.c | wc -l # & in the diff?

Even something I'd guess was basic -- loading and unloading modules -- changes fast.

Why yes, it *is* gone!

I tried this trick, from an earlier lesson

$ grep crash3 /proc/kallsyms
00000000 t bye [crash3]
00000000 d __this_module [crash3]
00000000 t cleanup_module [crash3]

After the module's loaded, "bye()" is still around, but "hi()" has been removed by the __init directive.

I removed the __init directive from "hi()" , rebuilt the module, loaded it up, and looked again.

Try it. You'll like it.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <p> <br> <pre> <h1> <h2> <h3> <h4>
  • Lines and paragraphs break automatically.

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.