Category Archives: Hacking

Development and hacking in general.

kmod announcement (and how to help testing it!)

Posted on December 21, 2011 by Gustavo Barbieri| Comments Off

introduction and motivation

At ProFUSION a common topic is how to optimze the system. Not just the speed, but also what and how it is done. Not just for our embedded systems, but also our desktops and laptops.

These discussions led us to be involved in projects that consider this goal, such as Enlightenment, ConnMan, oFono and systemd. With great projects, come great people and thus enlightening discussions on how to improve things. From these discussions Lennart Poettering and Kay Sievers put together a A Plumber’s Wish List for Linux, with one of the items being of special interest for ProFUSION as it had the potential of also helping embedded systems and speeding up or boots:

* module-init-tools: provide a proper libmodprobe.so from
module-init-tools:
Early boot tools, installers, driver install disks want to
access information about available modules to optimize
bootup handling.

One of the reasons for this is that udev will search /sys/devices for all “modalias” files and call “/sbin/modprobe -bv $ALIAS“, however many of these calls evaluate to an empty list, thus an useless execution of program that will open /lib/modules/`uname -r`/modules.*, load resources, search for something that is not there and return. This could be way cheaper if done inside udev workers. [note: Lucas did measure and noticed 2.5x speedups, stay tuned for benchmarks]

kmod

Then our developer Lucas De Marchi proposed to do the libmodprobe and we funded it. After some discussions with Lennart and Kay, it was decided to rename it to kmod for shortness and good namespace. The initial goal was to achive a 1:1 replacement of module-init-tools as a library to be used by udev and systemd-modules-load and we are close to it with our second release done today! While we miss depmod tool (planned for v3), our kmod-modprobe should be fully functional and if this proves to be true, the logic will move to libkmod to be used by udev.

we need your testing!

To ensure kmod-modprobe does what it is supposed to do, we need extensive testing, there is where we need you! Try it on as many systems as possibles and let us know. To do this we recommend:

$ git clone git://git.profusion.mobi/kmod.git
$ cd kmod
$ ./autogen.sh
$ ./configure --enable-logging \
              --enable-debug --enable-tools \
              --enable-zlib # if you have module.ko.gz
$ make all
$ sudo make install
$ sudo mv /sbin/modprobe /sbin/modprobe-bkp
$ sudo ln -s /usr/bin/kmod-modprobe /sbin/modprobe
$ ls -1 /sys/module/ > ~/original-modules
$ reboot
$ ls -1 /sys/module/ > ~/kmod-modules
$ diff ~/original-modules ~/kmod-modules

current users and feedback

Being a new project in a critical area of Linux system, we expected lots of criticism and rejection by people, but interestingly enough after Lucas’ announcement and LWN article the feedback was highly positive! We even had some testers and people to help with ideas and experience.

Among the people that joined the project is the current maintainer of module-init-tools package Jon Masters, which announced that kmod replaces module-init-tools. He is helping a lot with his knowledge and cases from Red Hat Enterprise Linux, also providing modules from non-trivial platforms such as s390. Last but not least he provided insights to improve module handling on Linux, particularly replacing modules with better alternative as required in enterprise systems.

With Jon’s blessing we’ll use the same mailing list linux-modules@vger.kernel.org and our git will move to kernel.org soon.

People from ARCH and Debian were also interested and even created packages for it! These guys were extremely helpful to test cases such as gzip modules, 64bits modules with i386 user space and so on. They are: Tom Gundersen (tomegun), Dave Reisner (falconindy) and Marco d’Itri (Md).

next steps

Based on our TODO, we have the following ideas for next steps:

  • Finish libkmod-elf.c to provide information required by depmod. I’m working on this in my depmod branch;
  • tools/kmod-depmod.c: create a 1:1 compatible tool to generate /lib/modules/`uname -r`/modules.*;
  • libkmod should export parsed configuration to be used by kmod-modprobe.c –dumpconfig;
  • create kmodmock library to be LD_PRELOAD’ed to redirect some syscalls such as init_module(), delete_module() and open() of /sys and /proc. I’ve started a branch for this some time ago as “unittest” branch but stopped due lack of ELF support at the time;
  • create unittests and measure coverage. Given kernel modules are user-input they may be broken files and libkmod-elf.c needs to be extensively tested to avoid segmentation fault due out-of-boundaries access. This mean not trusting null terminated string in .strtab section and so on.
  • create optimized modules.archive that would contain optimized search index and all modules compressed independently, but in the same file. This would save file access and could provide information we don’t have today, like the uncompressed size of gzipped modules.

Comments Off

Posted in C, Free Software, Hacking, Linux, Maemo, ProFUSION

Tagged

PythonBrasil[7] = Excellent!

Posted on October 6, 2011 by Gustavo Barbieri| Comments Off

Last week from 29-Sep to 01-Oct we had the amazing 7th PythonBrasil conference, for the first time in São Paulo.

Since I’ve start to use Python in 2002 I loved the language, but after getting introduced to the PythonBrasil community in 2004 I’ve boosted my development skills, got some friends and even my first job (INdT-Recife) was a kind recommendation from Osvaldo Santana in 2005.

By 2005 we had the 1st PythonBrasil Conference, then called PyConBrasil, here in Campinas with the help of UNICAMP and our amazing non-stop contributor Rodrigo Senra. It was very cool, I even presented a talk there… and it motivated me to go to following conferences in 2006 and 2007 as well.

However if starting to work at INdT reduced my spare time since late 2006, after ProFUSION was born in 2008 I had no time to participate in the lists or even go to conferences. What a shame!

I couldn’t see how shameful it was until I did this PythonBrasil in 2011. I’ m yet to see a conference with so kind people. People still remembered me and I was ashamed when I couldn’t remind their names… although they did remember mine (NOTE TO CONFERENCE: bigger names next year!) Some would even let me know they still use Eagle-Py, something that I already forgot about. And people I had closer contact before were willing to talk as if we had met last week. Amazing.

During these talks I’ve catch up with Rodrigo Senra, Luciano, Osvaldo, Erico, Marco André, Sidnei, Fernando and many more I couldn’t remember. However one of the talks was very special: talked to Gustavo Niemeyer about Go programming language. That’s right, people were so kind and open we had a keynote about Go, and we talked a lot afterwards without problems! :-P  Gustavo showed me some nice details about the language and my mind is now burning! I must do Enlightenment Foundation Libraries (EFL) + Go = EGO, a perfect GUI tool.

But I was of use… not just a leecher! I went there to present 3 talks (slides in Portuguese!):

Last but not least, I’d like to thank everyone that did this amazing conference possible!

Comments Off

Posted in Free Software, Hacking, Life, ProFUSION, Python

Introducing EWS to help EFL on FB and PS3

Posted on October 5, 2011 by Gustavo Barbieri| Comments Off

Lots of systems do not support multi-window on their own, consider standard Linux Frame Buffer (FB) or the PlayStation3. This makes it cumbersome as you’re restricted to a single window,  likely you’ll have to rewrite your apps to behave well in this scenario.

Traditionally this problem is worked around by toolkits like Qt that provides QWS, a great help for developers. Considering QWS I decided to help our PS3 friends with something similar which I shamelessly called EWS (Ecore + Evas Single Process Windowing System).

Ecore and Evas already made 99% of this work: they abstract rendering, abstract windowing and so on. There is even support for “inlined windows” in Elementary, these are rendered to buffers that are visible inside other windows. Actually the first version of EWS was a 30min hack in Elementary called “eland” (making fun of wayland). But as not everyone would use Elementary it was better to move it lower in the stack: Ecore_Evas.

Ecore_Evas is a glue layer between Ecore and Evas that takes care to configure actual windows and setup Evas on them, to capture events from various sources and feed them to canvas, etc. If you cope with its API you’ll work with most EFL libraries and applications out-of-the-box.

Ecore_Evas_EWS was created as a new engine that builds on top of others. It will use a backing-store engine like X11, FB, DirectFB, WinCE or PS3 to create an internal Ecore_Evas. Every new window created with ecore_evas_ews_new() is rendered to an Evas using the buffer engine and its pixels end as the image source in the backing-store engine, displaying it. This is very similar to existing ecore_evas_buffer feature ecore_evas_object_image_new(), however it will handle more things automatically.

Among the new features are events to allow window management. Things like “window created”, “window moved” and “window resized” are placed in the main loop for interested peers, they may handle these and decorate the windows, offer window management (close, maximize, …). To exemplify that I’ve added support in Elementary with basic controls such as move, maximize, restore and close. Unfortunately it looks  ugly like hell and then I’m not posting any screenshot or screencast ;-) Let’s wait for people doing themes to make a great work.

The work is far from complete, bugs remain,  optimizations could be applied for OpenGL… if you consider the rules “20/80″, 80% of the code was written in 20% of the time, now we’ll take 80% to make it work for sure :-D  But it’s not bad considering it took me 2 days,  1421 lines in ecore_evas_ews.c, 543 in elu_ews_wm.c and 317 in ews.edc…

That’s why I keep saying that EFL is an amazing technology to build products. It runs fast, it’s slick and it does not get in the way. Give it a try! :-)

Comments Off

Posted in C, Free Software, Hacking, Linux, ProFUSION

Tagged ,

EFL portability shines: native PS3 support!

Posted on September 16, 2011 by Gustavo Barbieri| Comments Off

While most projects praises portability with fancy things like “gchar” (Gtk), fancy names like “LightHouse” (Qt) or completely new abstraction concepts, The Enlightenment Project always choose a simple approach to use POSIX and fill the gaps as required. This requires much less work and yields the same or better results, as we could see in the past and now being the first toolkit to be ported to PlayStation3 (native, no Linux)!

As announced by the the developer, KaKaRoTo, in our official news the port was done on top of our existing SDL backend in a couple of days, then being migrated to native PlayStation3 using unofficial SDK PSL1GHT for jailbroken systems. There is still no hardware acceleration, yet it runs fine a physics game at Full HD.

Right now we have port to MacOS and Windows, including WinCE. Given our scarce resources we do not lag behind Qt or Gtk in portability front. Basically what we have is one brave developer called Vincent Torri that is a Math professor and do it on his spare time!

Vincent could do it by keeping our approach: use POSIX and fill the gaps. He created “Evil”, a library to provide missing functions to Windows. Kakaroto then created “Escape” to do the same for PSL1GHT. Seems this approach works fine ;-)

Comments Off

Posted in C, Free Software, Hacking, Linux

Tagged , ,

ANN: LightMediaScanner 0.4.4 released

Posted on August 19, 2011 by Gustavo Barbieri| 2 Comments

I’m proud to announce LightMediaScanner 0.4.4 was released and I’d like to take some time to remind you of this awesome project ;-)

While 0.4.4 is no big leap ahead of 0.4.3 I’m doing this post more to make clear people that the project is still alive and it may be the right tool for you, particularly if you need fast media scanning on your desktop or embedded device.

Introduction (History)

LightMediaScanner (LMS) was born in 2007 during the development of Canola2 media player for Nokia N800 Maemo device. We did a previous try at media scanners for Canola1 and did learn a lot, that would be the foundation of LMS.

We had two major problems to solve:

  1. scan large amounts of media as fast as possible
  2. don’t mess with interactivity.

The first problem had more qualifiers, like the media being unreliable (broken files), be removed without prior notification (mmc removed when system was off), slow discs (sd/mmc), small amount of available RAM, low-end CPU and so on.

Traditional approaches were slow and would bring your device to unusable state for minutes, even after the scan process finished as they consumed too much memory that would swap your applications and services. They would also be naive with file scanning and would trash filesystem’s cache with useless data.

Existing Solutions Problems

At that time we had Maemo’s media scanner that was awful and Tracker was being developed to solve its problems. However Tracker was immature and had other focus, more like being fancy with new standards and trying to solve all the diseases of the world… while we need to be fast in some specific cases.

To highlight technical problems, let’s consider JPEG scanners. Everything one Media Player application would need is the  title, author/artist, date, width, height, orientation and maybe geolocation. These are few bytes, particularly because Title and Author are missing in most cases. We should open the file, get these bytes, register these and close the file. Every system at the time used libexif to do so, however it would copy all the header to memory prior to scanning. Given that most files contain thumbnails in the header, it would mean useless malloc + read of around 10-50kb, to get dozen bytes and exit. This trashes the VM and disc. Moreover, some libraries would open the file themselves and would not fadvise as POSIX_FADV_DONTNEED, thus the filesystem would keep it in cache for further access that never happen! Similar situations existed for PNG, Mp3 and others.

LightMediaScanner Solution

We decided to go with a 2 process solution to avoid crashes and hang-ups on the main process. This would be safe and we could kill the slave process if it took too long to report on a given file. Later on we introduced threaded and single process scanning that would be useful in some cases.

We also had dynamic loadable plugins (shared objects) with the media parsers to be used given matching extensions. The plugins can be selected by the user at runtime. Yes, we would not scan every file with every parser, we did trust the extension as a way to save work and it worked fine, and fast.

The main process just scans the filesystem and  pipes the files to the slave (thread or process). This slave would open the file, get the data to populate a structure to be written to database, then close the file and report back the progress. If the slave takes too long to reply or dies, it’s respawned with the next file in the queue.

The database uses SQLite3 and a configurable number of files are processed in one SQL transaction, thus we’d not hit the filesystem with writes that often. The database access is provided with helper functions that uses pre-compiled statements. However we only provide the insert/update methods, as the fetch (select) are up to the application to use (maybe using their own ORM framework).

Some metadata formats did not specify encoding (ASCII was assumed) and fallbacks can be applied with the charsets. It will use iconv in a priority list to figure out what works.

Just the Indexer!

LightMediaScanner is not a daemon-service, it does not talk DBus neither requires you to speak  RDF or SPARQL. If you want, you can create one with it, but it does not provide this stuff out of the box. Basically you call it to check for media and then access the database yourself, using the SQLite3 directly or using some ORM, doing caches or not, it’s up to you.

The database tables are quite simple and try to provide common useful relations. It’s not designed to cover all weird cases in the world (like multiple artists per song or which lens did you use with your camera).

Using it!

Get the code from our GIT or use the release tarball, see http://people.profusion.mobi/cgit.cgi/lightmediascanner.git/. It uses standard autoconf/automake to build, so should be familiar to most developers.

It hard-depends only on SQLite3 and iconv (most libC ships with it). Optional dependencies are libvorbis, libmp4v2 and libflac.

You can call the scanner from C, C++ or Python using our bindings. Comprehensive example can be found at src/bin/test.c

References: previous post about lightmediascanner 0.2.0 release

→ 2 Comments

Posted in C, Free Software, Hacking, Linux, Maemo, ProFUSION, Python

Tagged ,

ESC Brazil – RealTime Linux with RT_PREEMPT

Posted on June 2, 2011 by Gustavo Barbieri| Comments Off

The first Embedded Systems Conference Brazil was held at São Paulo on 24 to May 25 2011 and ProFUSION was there to do a technical talk.

Our contributor Lucas De Marchi proposed the talk “Usando Linux como Sistema de Tempo Real” (Using Linux as an Real Time Operating System). He did his master thesis on that topic for 2 years at Politecnico di Milano, so he knows a lot about the topic.

Unfortunately Lucas is one of those geeks that try to live a real-world life and plays football (soccer for americans) and in the week before he broke his foot… Ouch!

So yours truly was elected to present the talk for him. I already did play with realtime Linux in the past and I knew the concepts from past experiences, thus I accepted the challenge and did the talk. According to the attendees it was a good talk!

People got amazed with Linux’s Real Time capabilities and the ability to mix it with regular (“fair”) tasks, being able to change a process from FAIR to RT during runtime, no recompile was needed! This was demonstrated in a Freescale’s i.MX31 running music player running with regular priority and ping flood and a heavy CPU task, the music skipped and it was clear the deadlines were not being met. Running chrt and raising the priority above Kernel Threads that handle interruptions immediately fixed the skips. We also ran a software oscilloscope at desktop that plotted the achieved deadlines and it confirmed what we were listening.

If you’re not aware, RT_PREEMPT is a patchset (from pre-git days, not a git tree called -rt) that Ingo Molnar and Thomas Gleixner run to improve Real-Time behavior for the Linux kernel. As concepts are tested and patches mature they are moved into mainline tree, benefiting everyone using it, even your desktop running PulseAudio now! The major trick of this work was to handle interruptions in Kernel Threads instead of immediately as it used to be. These threads are RT themselves at priority 50 and if you want you can have a regular user process thread at a higher priority and it will run before the kernel. This was the case with the pingflood, with a regular kernel the music could skip due kernel stopping the playback process to handle the network interruption.

Last but not least, if you’re not familiar with RT have something clear in your mind. Real Time doesn’t mean Real Fast. It just mean you have guarantees to execute things when they should and this is mainly done by means of a preemptive kernel. In some cases RT will turn things slower, as you may switch contexts more often due preemption, but you’ll know you’ll be on time (as opposed to batch processing). Linux being a General Purpose Operating System (GPOS) was not designed with such thing in mind, but it was successfully converted into one!

Comments Off

Posted in Free Software, Hacking, Linux, ProFUSION

Tagged

DesktopSummit 2011

Posted on June 2, 2011 by Gustavo Barbieri| Comments Off

Hi all,

After I started on twitter and facebook the blog posts reduced a lot… anyways I’d like to mention I’ll attend DesktopSummit 2011 in Berlin to represent the Enlightenment (aka E17) team and thus an alternative view to the traditional GNOME x KDE.

Fortunately I have couple of friends in both GNOME and KDE so I can express our E17 voice without being ignored (at least immediately ;-P). Dunno if it’s me or E17 being the underdog, but people don’t feel afraid to listen to our ideas and they even try to incorporate those that make sense, like Enlightenment’s Edje that resulted in QEdje that in turn resulted into QML that boosts Qt development these days.

My focus will be:

  • performance matters: even if you have a fast hardware today and an even faster hardware tomorrow, you should care about performance. I’m not talking about neat picking premature optimizations, but to design platforms and standards considering performance. It’s often not harder, just need to be considered from the beginning. What bugs us most is the stupid freedesktop.org thumbnail standard, that forces use of PNG even if most of our pictures are in JPEG, causing slower thumbnailing time, bigger disk footprint and slower load times. Second in the list is the traditional abuse of XML, praised years ago and suddenly GNOME realized it was a bad idea (given GSetting/DConf), but it’s deeply inside our desktops with fontconfig and others (want faster application load times? Don’t use XML.);
  • Broader Standards: A standard must be thought outside the “I’m implementing it for myself” scope. Similar to above, we should consider broader standards and not get a piece of GNOME or KDE and say it’s a standard. Freedesktop.org fails to get standards because of this stupid approach. See the last flamewars around simple topics as the systray replacement. Actually the systray standard is a well known bad example, people wrote it like if GNOME was the only way to do this. Enlightenment officially ignored this standard due stupid bugs with XEmbed specification and approach (as EFL uses the “windowless widgets” for a while, the XEmbed made no sense and caused user experience problems);
  • Embedded is the future. Okay, it will be a DesktopSummit, but as Rasterman predicted  in 2002 the desktop game is unlike to be changed from Microsoft dominance and we better look for new fields where we can compete equally. Given Android, WebOS, Maemo, MeeGo and others, it’s clear that Linux does dominated the mobile world. With appliances (from TVs to DVR to Refrigerators) being dominated for a while (after the embedded cpus got more powerful, most embedded OS got replaced with Linux). Thus together with performance we must not restrict ourselves to the keyboard-mouse-X11-big-screens approach that we used before. This is not just about scalable (as in multiplication) GUI as people believed SVG would help, but we need different GUI for different environments and our toolkits and infrastructure have to help with that… something that Apple is currently doing quite well with their iOS UI toolkit providing universal applications that run optimized on both iPad and iPhone/iPod.

If you’re also attending this summit, let’s meet there and have some beers.

Comments Off

Posted in Free Software, Hacking, Linux, Maemo, ProFUSION

Tagged ,

ANN: liblogger

Posted on March 9, 2011 by Gustavo Barbieri| Comments Off

“There is no better tool to debug than printf()”
– Latin proverb from an old Chinese man.

That’s what every programmer knows. Despite interesting tools like profilers, debuggers and other tracing tool, for a reason or another we often resort to good old printf(). The logic behind this is quite simple: it’s available everywhere and have no impact on performace (mostly, on both cases — don’t be picky!), yet it allows you to check out if your program flow is as you expected and if not, what’s happening. The bad part, of course, is that it’s invasive and cumbersome to type, often introduces errors by itself (come on, raise your hand if you never ever wasted hours debugging to notice that the bug was in the debug message!)

GDB and some tools like ltrace or GLibC’s latrace are not always available, particularly on embededd systems they may be available but almost unusable. I hit that in the past weeks during some debugging on an old MIPS where I had to investigate why my supposedly correct program did not work, but a proprietary one did. Documentation is almost inexistent and makes you more clueless than not, thus I had to compare what we were doing differently.

At first I started doing an old trick to create a shared library to be $LD_PRELOADed on my own that would log what was being called, then dlopen()/dlsym() the actual symbol, call it, then log again, then return. I did it for a handful functions, some I even printed out structure members and although it worked, soon I got bored and hit by the laziness that most programmers have: I must make this automatic. Liblogger was born.

Liblogger is Python program that will parse a C header file and generate a source file that does exactly what I described above: log entrance, call the actual function and keep its return value, then log exit, then return the value. That’s the basics this single-file without external dependencies do. This source file can then be compiled into a shared object or into the final application.

Of course I wanted it to be as helpful as possible, so this first version already includes some nifty features:

  • source file can be compile-time configured using CPP defines to produce colored, thread-safe, timestamped (gettimeofday, clock_gettime), indented (nesting level) output. This output can go to stderr or some given file;
  • generates makefile for you, creating most of aforementioned variations by default. Just pick the one you want to use and LD_PRELOAD it;
  • generate types file with all parsed enums, structs, unions and typedefs;
  • generate formatters file with custom functions to print out known enums, structs and unions. No more copy + sed on the original structs to produce printf() statements manually!
  • will keep errno intact so functions that rely on it will work as it will not be reset by fprintf() calls;
  • can be compiled in a thread-safe way, doing locks around dlopen(), dlsym() and the log FILE* to keep messages atomic. It will also keep the indentation level per-thread and print out thread ids (if not main thread);
  • Behavior is completely controlled with a INI-style configuration file. It can (check for more at README.txt):
    • tell the parser tokens to ignore;
    • filter (positively/negatively) functions to log;
    • specify type formatters you want to use. Couple are provided for basic C types, and you can request it to generate custom formatters for parsed types as mentioned above;
    • specify return value checkers you want to use. Couple are provided, including one that checks for errno and logs the value as a string;
    • if types of parameters and return are safe to be used by formatters, this can be used at various levels to provide black-white lists for these and keep it useful and safe;
    • special handling of return (output) parameters, their values can be dereferenced and printed out after the actual function is called;

Usage should be pretty straightforward but I distribute two examples, one for dbus and another for jpeglib. Part of the dbus example is reproduced below:

  • dbus.cfg
    • [global]
headers = dbus/dbus.h
overrides = formatters-%(header_name)s.c
# regular expression for all defines in dbus-macros.h
ignore-tokens-regexp = (DBUS_MACROS_H|DBUS_BEGIN_DECLS|DBUS_END_DECLS|DBUS_BEGIN_DECLS|DBUS_END_DECLS|DBUS_DEPRECATED|DBUS_DEPRECATED|DBUS_DEPRECATED|DBUS_EXPORT|DBUS_EXPORT|DBUS_EXPORT|DBUS_EXPORT)

[type-formatters]
DBusHandleMessageFunction = %(prefix)s_log_fmt_pointer
DBusFreeFunction = %(prefix)s_log_fmt_pointer
DBusHandleMessageFunction = %(prefix)s_log_fmt_pointer

[safe-formatters]
DBusHandleMessageFunction = true
DBusFreeFunction = true
DBusHandleMessageFunction = true

# be pedantic about safety, let's specify all strings independently

[func-dbus_message_new_method_call]
parameter-bus_name-safe = true
parameter-path-safe = true
parameter-interface-safe = true
parameter-method-safe = true
  • call: 
    liblogger.py \
    --config dbus.cfg \
    --makefile Makefile.dbus-connection \
    --makefile-cflags "\`pkg-config --cflags dbus-1\` -I." \
    --custom-formatters formatters-connection.c \
    /usr/include/dbus-1.0/dbus/dbus-connection.h \
    libdbus-1.so \
    log-dbus-connection.c
make -f Makefile.dbus-connection
LD_PRELOAD=./log-dbus-connection-color-timestamp.so d-feet

Comments Off

Posted in C, Free Software, Hacking, Linux, Maemo, ProFUSION

Tagged

MeeGo, Nokia… light at the end?

Posted on February 14, 2011 by Gustavo Barbieri| Comments Off

Hello people, as most of open source lovers I’m still trying to digest Nokia’s move to WP7. But although many are worried about the future of Meego or the Linux on mobiles, I’m quite confident.

First, as said before MeeGo is not just about Nokia or phones, it’s also being pushed as In-Vehicle Infotainment (IVI) and is already adopted by BMW, which you can see daily contributions on projects such as ConnMan. So let’s not assume it is the end of the world given Nokia’s action.

Second, although Android and WebOS are indeed Linux since they use this kernel, we often want more traditional user-space stack, like MeeGo would be. For those,  don’t loose hope! Today (Feb14, 2011) we got a major announcement at MWC by LiMo: LiMo Foundation Unveils LiMo 4. It’s based on X11, WebKit, GNOME and… EFL!

Wait? Where do you see EFL in that announcement? You’ll have to check What is the LiMo 4 Platform? block diagram and see EFL is now a first class component at it, together with GNOME that was there since first release. Yeah, we surely could use better publicity at EFL side ;-)

The release is still just text and no code, which should be available by July2011, however you can get code straight from EFL SVN.

ProFUSION team is happy due our contributions to make this possible, after all we help with EFL and WebKit-EFL developments!

Comments Off

Posted in Free Software, Hacking, Linux, Maemo, ProFUSION

Tagged , ,

systemd!

Posted on October 29, 2010 by Gustavo Barbieri| Comments Off

Yeah, I did not write for a while, but that does not mean I’m dead or changed interests, just that running an always growing company is taking lots of time, with the spare time I get being invested in something interesting and relevant, usually that does not translate to blog posts :-P  However I felt like doing it today.

While most people know me due my graphical user interfaces work, likely related to Enlightenment, I’m one of those guys that like to consider the system as a whole and not just a subset of it. My main interest since 1998 is graphics, from good old days of tcl/tk and gtk1 to pygame/sdl to qt3/4 and later enlightenment. But I always kept an eye on other bits like boot, networking, sound and so on.

After getting into ConnMan to have a good (and fast) network infrastructure, I started to consider similar bloat-reducing options to other parts of the system. Talking to friends like Holtmann, Lennart and ProFUSION co-workers about the stupid stuff we do in our distributions, like abusing shell scripts for no good… even tried to resurrect Gentoo’s bug #69579 (Bootup Time Reduction/General Speedup).

My first idea was simple, hackish and selfish: as I’m Gentoo user, I could write a shell script compiler that generates threaded-C code with the exact state machine I’d have using /etc/init.d with OpenRC. It was simple (or not so complex) as most of the scripts are start-stop-daemon wrappers with dead-simple usage of OpenRC’s dependency information (need, use, after, before…). However it was hackish because couple of them are not simple and would require hacks, either special cases or rewrite them natively, to make it work. Last but not least, it was selfish as likely it would just fix my problem, or with some luck it could benefit Gentoo as a whole. All of that after I loosing faith in Upstart of couple of tries and talks with related developers.

However sometimes it is better to talk to smart(er?) people before you go do your own stuff. I don’t feel like I suffer from NIH-syndrome. So talking here and there, João Paulo pointed that Lennart had a still secret project to do exactly what I was aiming, but knowing Lenny, it ought to be good. So I immediately /q -freenode mezcalero and started chatting, with he demonstrating a great cleverness and even proving me wrong in some assumptions, he already thought of much more like merging inted features and how to isolate services in their own control groups… yeah, he got me in systemd’s bandwagon.

As I still have tons of other stuff to do, I put the idea on hold and went back to it after several posts and announcements that Fedora 14 would use it. At the first free time slot I checked out the code and Gentoo’s “port”. Well, as expected with young projects it did not cover some special cases, like systems without IPv6 (mine) and Gentoo’s /etc/init.d scripts were too smart due OpenRC’s dependency resolver that would make their usage more harm than good. So I had work to do, but it was done and contributed back, with awesome results.

The project and community, also composed of other nice and helpful hackers such as Kay Sievers and Michael Biebl and I felt like it was an excellent project ProFUSION could collaborate, then we invested 2 developers to help it: Fabiano Fidencio and Lucas de Marchi, that got it running on Arch (OpenEmbedded on the way) and added some missing features, like native poweroff/halt/reboot, compile time toggle to disable SysV backward compatibility and so on, runtime fallbacks if no IPv6 and so on.

Right now systemd is very powerful, it can almost fulfill all my needs and likely can manage everything an embedded system would use on its own. From its lightness that reduced my boot in 10s and 2000 process, to nice helpers like its syslog-kmsg bridge. In my opinion, the biggest missing feature is cron-like features to reduce yet-another daemon from boot and centralize it in the “system daemon — systemd”.

If you don’t know it yet, give it a try. It’s young however very mature, handling complex cases, providing nice utilities and trying to unify Linux boot process. A short list of its awesomeness would be:

  • each service is isolated in its own control group, being able to apply limits and avoid children processes to exit their parent’s control (ie: php/perl from cgi-bin exit from their apache tree). Aside from safety, this allows monitoring when a forking service died as all process in its cgroup are gone.
  • socket activation, like inted/xinetd, allows seldom used servers to not add load to boot process yet they still work as expected, like sshd and cupsd for desktop systems.
  • lazy mount of filesystems by means of kernel automount. Similarly to socket activation, this lazy mount avoids adding load to boot process and postpone the actual mount when the system actually need files from such mountpoint.
  • easy to read/write configuration files (units) with ini-like syntax, matching other freedesktop.org projects such as dbus.
  • native support, properly written in C, to various boot tasks such as loading modules, load/save random seed, clean temporary files, setup console fonts and keys, enforcing quota checks, creating required directories with correct permissions and so on.
  • useful helpers such as systemd-kmsg-syslogd that directs /dev/log to /dev/kmsg, that avoids the need of a proper system logger with logrotate on embedded (or desktops that doesn’t need persistent logging).
  • awesome readahead code that uses fanotify (temporarily disabled in 2.6.36) to passively collect read files, even while replaying already collected from previous boot — without stepping on its own toe or major performance hit.
  • proper integration with DBus to provide race-free activation of system services.
  • various agent interfaces to request user intervention, like passwords.
  • proper remote control interface (uses DBus)
  • proper (optional) SELinux and auditd labeling and interaction.

All in all I really like Systemd and I foresee a brilliant future for it, given major distros adopt it officially (Fedora and OpenSuse at least?)

Comments Off

Posted in C, Free Software, Hacking, Linux, ProFUSION

Tagged