Petter Reinholdtsen

What is the most supported MIME type in Debian?
16th January 2013

The DEP-11 proposal to add AppStream information to the Debian archive, is a proposal to make it possible for a Desktop application to propose to the user some package to install to gain support for a given MIME type, font, library etc. that is currently missing. With such mechanism in place, it would be possible for the desktop to automatically propose and install leocad if some LDraw file is downloaded by the browser.

To get some idea about the current content of the archive, I decided to write a simple program to extract all .desktop files from the Debian archive and look up the claimed MIME support there. The result can be found on the Skolelinux FTP site. Using the collected information, it become possible to answer the question in the title. Here are the 20 most supported MIME types in Debian stable (Squeeze), testing (Wheezy) and unstable (Sid). The complete list is available from the link above.

Debian Stable:

  count MIME type
  ----- -----------------------
     25 audio/mpeg
     24 audio/x-mp3
     24 application/ogg
     23 text/plain
     21 image/tiff
     20 image/jpeg
     20 audio/x-flac
     19 image/png
     19 image/gif
     19 audio/x-wav
     19 audio/x-mpegurl
     18 image/bmp
     18 audio/x-scpls
     18 audio/x-mpeg
     16 audio/x-musepack
     16 application/x-ogg
     14 image/x-portable-pixmap
     14 image/x-portable-bitmap
     13 video/x-ms-asf
     13 video/mpeg

Debian Testing:

  count MIME type
  ----- -----------------------
     21 text/plain
     21 audio/x-mp3
     21 audio/mpeg
     20 application/ogg
     19 audio/x-wav
     18 image/tiff
     18 image/jpeg
     18 audio/x-flac
     17 image/png
     17 image/gif
     17 audio/x-mpegurl
     16 audio/x-scpls
     15 image/bmp
     15 audio/x-mpeg
     14 application/x-ogg
     13 video/x-ms-asf
     13 audio/x-musepack
     12 video/x-ms-wmv
     12 video/x-msvideo
     12 video/quicktime

Debian Unstable:

  count MIME type
  ----- -----------------------
     23 audio/mpeg
     22 text/plain
     21 audio/x-mp3
     21 application/ogg
     20 audio/x-wav
     19 image/tiff
     19 audio/x-flac
     18 image/jpeg
     17 image/png
     17 image/gif
     17 audio/x-mpegurl
     16 image/bmp
     16 audio/x-scpls
     16 audio/x-mpeg
     14 audio/x-musepack
     14 application/x-ogg
     13 video/x-ms-asf
     13 video/mpeg
     13 audio/mp4
     12 video/x-ms-wmv

I am told that PackageKit can provide an API to access the kind of information mentioned in DEP-11. I have not yet had time to look at it, but hope the PackageKit people in Debian are on top of these issues.

Tags: debian, english.
Using modalias info to find packages handling my hardware
15th January 2013

Yesterday, I wrote about the modalias values provided by the Linux kernel following my hope for better dongle support in Debian. Using this knowledge, I have tested how modalias values attached to package names can be used to map packages to hardware. This allow the system to look up and suggest relevant packages when I plug in some new hardware into my machine, and replace discover and discover-data as the database used to map hardware to packages.

I create a modaliases file with entries like the following, containing package name, kernel module name (if relevant, otherwise the package name) and globs matching the relevant hardware modalias.

Package: package-name
Modaliases: module(modaliasglob, modaliasglob, modaliasglob)

It is fairly trivial to write code to find the relevant packages for a given modalias value using this file.

An entry like this would suggest the video and picture application cheese for many USB web cameras (interface bus class 0E01):

Package: cheese
Modaliases: cheese(usb:v*p*d*dc*dsc*dp*ic0Eisc01ip*)

An entry like this would suggest the pcmciautils package when a CardBus bridge (bus class 0607) PCI device is present:

Package: pcmciautils
Modaliases: pcmciautils(pci:v*d*sv*sd*bc06sc07i*)

An entry like this would suggest the package colorhug-client when plugging in a ColorHug with USB IDs 04D8:F8DA:

Package: colorhug-client
Modaliases: colorhug-client(usb:v04D8pF8DAd*)

I believe the format is compatible with the format of the Packages file in the Debian archive. Ubuntu already uses their Packages file to store their mappings from packages to hardware.

By adding a XB-Modaliases: header in debian/control, any .deb can announce the hardware it support in a way my prototype understand. This allow those publishing packages in an APT source outside the Debian archive as well as those backporting packages to make sure the hardware mapping are included in the package meta information. I've tested such header in the pymissile package, and its modalias mapping is working as it should with my prototype. It even made it to Ubuntu Raring.

To test if it was possible to look up supported hardware using only the shell tools available in the Debian installer, I wrote a shell implementation of the lookup code. The idea is to create files for each modalias and let the shell do the matching. Please check out and try the hw-support-lookup shell script. It run without any extra dependencies and fetch the hardware mappings from the Debian archive and the subversion repository where I currently work on my prototype.

When I use it on a machine with a yubikey inserted, it suggest to install yubikey-personalization:

% ./hw-support-lookup
yubikey-personalization
%

When I run it on my Thinkpad X40 with a PCMCIA/CardBus slot, it propose to install the pcmciautils package:

% ./hw-support-lookup
pcmciautils
%

If you know of any hardware-package mapping that should be added to my database, please tell me about it.

It could be possible to generate several of the mappings between packages and hardware. One source would be to look at packages with kernel modules, ie packages with *.ko files in /lib/modules/, and extract their modalias information. Another would be to look at packages with udev rules, ie packages with files in /lib/udev/rules.d/, and extract their vendor/model information to generate a modalias matching rule. I have not tested any of these to see if it work.

If you want to help implementing a system to let us propose what packages to install when new hardware is plugged into a Debian machine, please send me an email or talk to me on #debian-devel.

Tags: debian, english.
Modalias strings - a practical way to map "stuff" to hardware
14th January 2013

While looking into how to look up Debian packages based on hardware information, to find the packages that support a given piece of hardware, I refreshed my memory regarding modalias values, and decided to document the details. Here are my findings so far, also available in the Debian Edu subversion repository:

Modalias decoded

This document try to explain what the different types of modalias values stands for. It is in part based on information from <URL: https://wiki.archlinux.org/index.php/Modalias >, <URL: http://unix.stackexchange.com/questions/26132/how-to-assign-usb-driver-to-device >, <URL: http://code.metager.de/source/history/linux/stable/scripts/mod/file2alias.c > and <URL: http://cvs.savannah.gnu.org/viewvc/dmidecode/dmidecode.c?root=dmidecode&view=markup >.

The modalias entries for a given Linux machine can be found using this shell script:

find /sys -name modalias -print0 | xargs -0 cat | sort -u

The supported modalias globs for a given kernel module can be found using modinfo:

% /sbin/modinfo psmouse | grep alias:
alias:          serio:ty05pr*id*ex*
alias:          serio:ty01pr*id*ex*
%

PCI subtype

A typical PCI entry can look like this. This is an Intel Host Bridge memory controller:

pci:v00008086d00002770sv00001028sd000001ADbc06sc00i00

This represent these values:

 v   00008086  (vendor)
 d   00002770  (device)
 sv  00001028  (subvendor)
 sd  000001AD  (subdevice)
 bc  06        (bus class)
 sc  00        (bus subclass)
 i   00        (interface)

The vendor/device values are the same values outputted from 'lspci -n' as 8086:2770. The bus class/subclass is also shown by lspci as 0600. The 0600 class is a host bridge. Other useful bus values are 0300 (VGA compatible card) and 0200 (Ethernet controller).

Not sure how to figure out the interface value, nor what it means.

USB subtype

Some typical USB entries can look like this. This is an internal USB hub in a laptop:

usb:v1D6Bp0001d0206dc09dsc00dp00ic09isc00ip00

Here is the values included in this alias:

 v    1D6B  (device vendor)
 p    0001  (device product)
 d    0206  (bcddevice)
 dc     09  (device class)
 dsc    00  (device subclass)
 dp     00  (device protocol)
 ic     09  (interface class)
 isc    00  (interface subclass)
 ip     00  (interface protocol)

The 0900 device class/subclass means hub. Some times the relevant class is in the interface class section. For a simple USB web camera, these alias entries show up:

usb:v0AC8p3420d5000dcEFdsc02dp01ic01isc01ip00
usb:v0AC8p3420d5000dcEFdsc02dp01ic01isc02ip00
usb:v0AC8p3420d5000dcEFdsc02dp01ic0Eisc01ip00
usb:v0AC8p3420d5000dcEFdsc02dp01ic0Eisc02ip00

Interface class 0E01 is video control, 0E02 is video streaming (aka camera), 0101 is audio control device and 0102 is audio streaming (aka microphone). Thus this is a camera with microphone included.

ACPI subtype

The ACPI type is used for several non-PCI/USB stuff. This is an IR receiver in a Thinkpad X40:

acpi:IBM0071:PNP0511:

The values between the colons are IDs.

DMI subtype

The DMI table contain lots of information about the computer case and model. This is an entry for a IBM Thinkpad X40, fetched from /sys/devices/virtual/dmi/id/modalias:

dmi:bvnIBM:bvr1UETB6WW(1.66):bd06/15/2005:svnIBM:pn2371H4G:pvrThinkPadX40:rvnIBM:rn2371H4G:rvrNotAvailable:cvnIBM:ct10:cvrNotAvailable:

The values present are

 bvn  IBM            (BIOS vendor)
 bvr  1UETB6WW(1.66) (BIOS version)
 bd   06/15/2005     (BIOS date)
 svn  IBM            (system vendor)
 pn   2371H4G        (product name)
 pvr  ThinkPadX40    (product version)
 rvn  IBM            (board vendor)
 rn   2371H4G        (board name)
 rvr  NotAvailable   (board version)
 cvn  IBM            (chassis vendor)
 ct   10             (chassis type)
 cvr  NotAvailable   (chassis version)

The chassis type 10 is Notebook. Other interesting values can be found in the dmidecode source:

  3 Desktop
  4 Low Profile Desktop
  5 Pizza Box
  6 Mini Tower
  7 Tower
  8 Portable
  9 Laptop
 10 Notebook
 11 Hand Held
 12 Docking Station
 13 All In One
 14 Sub Notebook
 15 Space-saving
 16 Lunch Box
 17 Main Server Chassis
 18 Expansion Chassis
 19 Sub Chassis
 20 Bus Expansion Chassis
 21 Peripheral Chassis
 22 RAID Chassis
 23 Rack Mount Chassis
 24 Sealed-case PC
 25 Multi-system
 26 CompactPCI
 27 AdvancedTCA
 28 Blade
 29 Blade Enclosing

The chassis type values are not always accurately set in the DMI table. For example my home server is a tower, but the DMI modalias claim it is a desktop.

SerIO subtype

This type is used for PS/2 mouse plugs. One example is from my test machine:

serio:ty01pr00id00ex00

The values present are

  ty  01  (type)
  pr  00  (prototype)
  id  00  (id)
  ex  00  (extra)

This type is supported by the psmouse driver. I am not sure what the valid values are.

Other subtypes

There are heaps of other modalias subtypes according to file2alias.c. There is the rest of the list from that source: amba, ap, bcma, ccw, css, eisa, hid, i2c, ieee1394, input, ipack, isapnp, mdio, of, parisc, pcmcia, platform, scsi, sdio, spi, ssb, vio, virtio, vmbus, x86cpu and zorro. I did not spend time documenting all of these, as they do not seem relevant for my intended use with mapping hardware to packages when new stuff is inserted during run time.

Looking up kernel modules using modalias values

To check which kernel modules provide support for a given modalias, one can use the following shell script:

  for id in $(find /sys -name modalias -print0 | xargs -0 cat | sort -u); do \
    echo "$id" ; \
    /sbin/modprobe --show-depends "$id"|sed 's/^/  /' ; \
  done

The output can look like this (only the first few entries as the list is very long on my test machine):

  acpi:ACPI0003:
    insmod /lib/modules/2.6.32-5-686/kernel/drivers/acpi/ac.ko 
  acpi:device:
  FATAL: Module acpi:device: not found.
  acpi:IBM0068:
    insmod /lib/modules/2.6.32-5-686/kernel/drivers/char/nvram.ko 
    insmod /lib/modules/2.6.32-5-686/kernel/drivers/leds/led-class.ko 
    insmod /lib/modules/2.6.32-5-686/kernel/net/rfkill/rfkill.ko 
    insmod /lib/modules/2.6.32-5-686/kernel/drivers/platform/x86/thinkpad_acpi.ko 
  acpi:IBM0071:PNP0511:
    insmod /lib/modules/2.6.32-5-686/kernel/lib/crc-ccitt.ko 
    insmod /lib/modules/2.6.32-5-686/kernel/net/irda/irda.ko 
    insmod /lib/modules/2.6.32-5-686/kernel/drivers/net/irda/nsc-ircc.ko 
  [...]

If you want to help implementing a system to let us propose what packages to install when new hardware is plugged into a Debian machine, please send me an email or talk to me on #debian-devel.

Update 2013-01-15: Rewrite "cat $(find ...)" to "find ... -print0 | xargs -0 cat" to make sure it handle directories in /sys/ with space in them.

Tags: debian, english.
Moved the pymissile Debian packaging to collab-maint
10th January 2013

As part of my investigation on how to improve the support in Debian for hardware dongles, I dug up my old Mark and Spencer USB Rocket Launcher and updated the Debian package pymissile to make sure udev will fix the device permissions when it is plugged in. I also added a "Modaliases" header to test it in the Debian archive and hopefully make the package be proposed by jockey in Ubuntu when a user plug in his rocket launcher. In the process I moved the source to a git repository under collab-maint, to make it easier for any DD to contribute. Upstream is not very active, but the software still work for me even after five years of relative silence. The new git repository is not listed in the uploaded package yet, because I want to test the other changes a bit more before I upload the new version. If you want to check out the new version with a .desktop file included, visit the gitweb view or use "git clone git://anonscm.debian.org/collab-maint/pymissile.git".

Tags: debian, english, robot.
Lets make hardware dongles easier to use in Debian
9th January 2013

One thing that annoys me with Debian and Linux distributions in general, is that there is a great package management system with the ability to automatically install software packages by downloading them from the distribution mirrors, but no way to get it to automatically install the packages I need to use the hardware I plug into my machine. Even if the package to use it is easily available from the Linux distribution. When I plug in a LEGO Mindstorms NXT, it could suggest to automatically install the python-nxt, nbc and t2n packages I need to talk to it. When I plug in a Yubikey, it could propose the yubikey-personalization package. The information required to do this is available, but no-one have pulled all the pieces together.

Some years ago, I proposed to use the discover subsystem to implement this. The idea is fairly simple:

I am not sure what the best way to implement this is, but my initial idea was to use dbus events to discover new hardware, the discover database to find packages and PackageKit to install packages.

Yesterday, I found time to try to implement this idea, and the draft package is now checked into the Debian Edu subversion repository. In the process, I updated the discover-data package to map the USB ids of LEGO Mindstorms and Yubikey devices to the relevant packages in Debian, and uploaded a new version 2.2013.01.09 to unstable. I also discovered that the current discover package in Debian no longer discovered any USB devices, because /proc/bus/usb/devices is no longer present. I ported it to use libusb as a fall back option to get it working. The fixed package version 2.1.2-6 is now in experimental (didn't upload it to unstable because of the freeze).

With this prototype in place, I can insert my Yubikey, and get this desktop notification to show up (only once, the first time it is inserted):

For this prototype to be really useful, some way to automatically install the proposed packages by pressing the "Please install program(s)" button should to be implemented.

If this idea seem useful to you, and you want to help make it happen, please help me update the discover-data database with mappings from hardware to Debian packages. Check if 'discover-pkginstall -l' list the package you would like to have installed when a given hardware device is inserted into your computer, and report bugs using reportbug if it isn't. Or, if you know of a better way to provide such mapping, please let me know.

This prototype need more work, and there are several questions that should be considered before it is ready for production use. Is dbus the correct way to detect new hardware? At the moment I look for HAL dbus events on the system bus, because that is the events I could see on my Debian Squeeze KDE desktop. Are there better events to use? How should the user be notified? Is the desktop notification mechanism the best option, or should the background daemon raise a popup instead? How should packages be installed? When should they not be installed?

If you want to help getting such feature implemented in Debian, please send me an email. :)

Tags: debian, english.
New IRC channel for LEGO designers using Debian
2nd January 2013

During Christmas, I have worked a bit on the Debian support for LEGO Mindstorm NXT. My son and I have played a bit with my NXT set, and I discovered I had to build all the tools myself because none were already in Debian Squeeze. If Debian support for LEGO is something you care about, please join me on the IRC channel #debian-lego (server irc.debian.org). There is a lot that could be done to improve the Debian support for LEGO designers. For example both CAD software and Mindstorm compilers are missing. :)

Update 2012-01-03: A project page including links to Lego related packages is now available.

Tags: debian, english, robot.
Lenker for 2013-01-01
1st January 2013

Her er noen lenker til tekster jeg har satt pris på å lese den siste måneden.

Og et godt nytt år til dere alle!

Tags: bankid, lenker, norsk, opphavsrett, personvern.
A Christmas present for Skolelinux / Debian Edu
28th December 2012

I was happy to discover a few days ago that the Skolelinux / Debian Edu project also this year received a Christmas present from Another Agency in Trondheim. NOK 1000,- showed up on our donation account December 24th. I want to express our thanks for this very welcome present. As the Debian Edu / Skolelinux project is very short on funding these days, and thus lack the money to do regular developer gatherings, this donation was most welcome. One developer gathering cost around NOK 15 000,-, so we need quite a lot more to keep the development pace we want. Thus, I hope their example this year is followed by many others. :)

The public list of donors can be found on the donation page for the project, which also contain instructions if you want to donate to the project.

Tags: debian edu, english.
How to backport bitcoin-qt version 0.7.2-2 to Debian Squeeze
25th December 2012

Let me start by wishing you all marry Christmas and a happy new year! I hope next year will prove to be a good year.

Bitcoin, the digital decentralised "currency" that allow people to transfer bitcoins between each other with minimal overhead, is a very interesting experiment. And as I wrote a few days ago, the bitcoin situation in Debian is about to improve a bit. The new debian source package (version 0.7.2-2) was uploaded yesterday, and is waiting in the NEW queue for one of the ftpmasters to approve the new bitcoin-qt package name.

And thanks to the great work of Jonas and the rest of the bitcoin team in Debian, you can easily test the package in Debian Squeeze using the following steps to get a set of working packages:

git clone git://git.debian.org/git/collab-maint/bitcoin
cd bitcoin
DEB_MAINTAINER_MODE=1 DEB_BUILD_OPTIONS=noupnp fakeroot debian/rules clean
DEB_BUILD_OPTIONS=noupnp git-buildpackage --git-ignore-new

You might have to install some build dependencies as well. The list of commands should give you two packages, bitcoind and bitcoin-qt, ready for use in a Squeeze environment. Note that the client will download the complete set of bitcoin "blocks", which need around 5.6 GiB of data on my machine at the moment. Make sure your ~/.bitcoin/ directory have lots of spare room if you want to download all the blocks. The client will warn if the disk is getting full, so there is not really a problem if you got too little room, but you will not be able to get all the features out of the client.

As usual, if you use bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Tags: bitcoin, debian, english.
A word on bitcoin support in Debian
21st December 2012

It has been a while since I wrote about bitcoin, the decentralised peer-to-peer based crypto-currency, and the reason is simply that I have been busy elsewhere. But two days ago, I started looking at the state of bitcoin in Debian again to try to recover my old bitcoin wallet. The package is now maintained by a team of people, and the grunt work had already been done by this team. We owe a huge thank you to all these team members. :) But I was sad to discover that the bitcoin client is missing in Wheezy. It is only available in Sid (and an outdated client from backports). The client had several RC bugs registered in BTS blocking it from entering testing. To try to help the team and improve the situation, I spent some time providing patches and triaging the bug reports. I also had a look at the bitcoin package available from Matt Corallo in a PPA for Ubuntu, and moved the useful pieces from that version into the Debian package.

After checking with the main package maintainer Jonas Smedegaard on IRC, I pushed several patches into the collab-maint git repository to improve the package. It now contains fixes for the RC issues (not from me, but fixed by Scott Howard), build rules for a Qt GUI client package, konqueror support for the bitcoin: URI and bash completion setup. As I work on Debian Squeeze, I also created a patch to backport the latest version. Jonas is going to look at it and try to integrate it into the git repository before uploading a new version to unstable.

I would very much like bitcoin to succeed, to get rid of the centralized control currently exercised in the monetary system. I find it completely unacceptable that the USA government is collecting transaction data for almost all international money transfers (most are done in USD and transaction logs shipped to the spooks), and that the major credit card companies can block legal money transactions to Wikileaks. But for bitcoin to succeed, more people need to use bitcoins, and more people need to accept bitcoins when they sell products and services. Improving the bitcoin support in Debian is a small step in the right direction, but not enough. Unfortunately the user experience when browsing the web and wanting to pay with bitcoin is still not very good. The bitcoin: URI is a step in the right direction, but need to work in most or every browser in use. Also the bitcoin-qt client is too heavy to fire up to do a quick transaction. I believe there are other clients available, but have not tested them.

My experiment with bitcoins showed that at least some of my readers use bitcoin. I received 20.15 BTC so far on the address I provided in my blog two years ago, as can be seen on the blockexplorer service. Thank you everyone for your donation. The blockexplorer service demonstrates quite well that bitcoin is not quite anonymous and untracked. :) I wonder if the number of users have gone up since then. If you use bitcoin and want to show your support of my activity, please send Bitcoin donations to the same address as last time, 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Tags: bitcoin, debian, english.

RSS feed

Created by Chronicle v4.4