introduction and motivation
At
ProFUSION a common topic is how to optimze the system. Not just the speed, but also what and how it is done. Not just for our embedded systems, but also our desktops and laptops.
These discussions led us to be involved in projects that consider this goal, such as
Enlightenment,
ConnMan,
oFono and
systemd. With great projects, come great people and thus enlightening discussions on how to improve things. From these discussions
Lennart Poettering and
Kay Sievers put together a
A Plumber's Wish List for Linux, with one of the items being of special interest for ProFUSION as it had the potential of also helping embedded systems and speeding up or boots:
* module-init-tools: provide a proper libmodprobe.so from
module-init-tools:
Early boot tools, installers, driver install disks want to
access information about available modules to optimize
bootup handling.
One of the reasons for this is that udev will search /sys/devices for all "modalias" files and call "
/sbin/modprobe -bv $ALIAS", however many of these calls evaluate to an empty list, thus an useless execution of program that will open /lib/modules/`uname -r`/modules.*, load resources, search for something that is not there and return. This could be way cheaper if done inside udev workers.
[note: Lucas did measure and noticed 2.5x speedups, stay tuned for benchmarks]
kmod
Then our developer
Lucas De Marchi proposed to do the libmodprobe and we funded it. After some discussions with Lennart and Kay, it was decided to rename it to
kmod for shortness and good namespace. The initial goal was to achive a 1:1 replacement of module-init-tools as a library to be used by udev and systemd-modules-load and we are close to it with our
second release done today! While we miss depmod tool (planned for v3), our kmod-modprobe should be fully functional and if this proves to be true, the logic will move to libkmod to be used by udev.
we need your testing!
To ensure kmod-modprobe does what it is supposed to do, we need extensive testing, there is where we need you! Try it on as many systems as possibles and let us know. To do this we recommend:
$ git clone git://git.profusion.mobi/kmod.git
$ cd kmod
$ ./autogen.sh
$ ./configure --enable-logging \
--enable-debug --enable-tools \
--enable-zlib # if you have module.ko.gz
$ make all
$ sudo make install
$ sudo mv /sbin/modprobe /sbin/modprobe-bkp
$ sudo ln -s /usr/bin/kmod-modprobe /sbin/modprobe
$ ls -1 /sys/module/ > ~/original-modules
$ reboot
$ ls -1 /sys/module/ > ~/kmod-modules
$ diff ~/original-modules ~/kmod-modules
current users and feedback
Being a new project in a critical area of Linux system, we expected lots of criticism and rejection by people, but interestingly enough after
Lucas' announcement and
LWN article the feedback was highly positive! We even had some testers and people to help with ideas and experience.
Among the people that joined the project is the current maintainer of module-init-tools package
Jon Masters, which announced that
kmod replaces module-init-tools. He is helping a lot with his knowledge and cases from Red Hat Enterprise Linux, also providing modules from non-trivial platforms such as s390. Last but not least he provided insights to improve module handling on Linux, particularly replacing modules with better alternative as required in enterprise systems.
With Jon's blessing we'll use the same mailing list
linux-modules@vger.kernel.org and our git will move to kernel.org soon.
People from ARCH and Debian were also interested and even created packages for it! These guys were extremely helpful to test cases such as gzip modules, 64bits modules with i386 user space and so on. They are: Tom Gundersen (tomegun), Dave Reisner (falconindy) and Marco d’Itri (Md).
next steps
Based on our TODO, we have the following ideas for next steps:
- Finish libkmod-elf.c to provide information required by depmod. I'm working on this in my depmod branch;
- tools/kmod-depmod.c: create a 1:1 compatible tool to generate /lib/modules/`uname -r`/modules.*;
- libkmod should export parsed configuration to be used by kmod-modprobe.c --dumpconfig;
- create kmodmock library to be LD_PRELOAD'ed to redirect some syscalls such as init_module(), delete_module() and open() of /sys and /proc. I've started a branch for this some time ago as "unittest" branch but stopped due lack of ELF support at the time;
- create unittests and measure coverage. Given kernel modules are user-input they may be broken files and libkmod-elf.c needs to be extensively tested to avoid segmentation fault due out-of-boundaries access. This mean not trusting null terminated string in .strtab section and so on.
- create optimized modules.archive that would contain optimized search index and all modules compressed independently, but in the same file. This would save file access and could provide information we don't have today, like the uncompressed size of gzipped modules.