X-Git-Url: https://pere.pagekite.me/gitweb/homepage.git/blobdiff_plain/c7856f0988d8b4ea27e2e63f439c6a83e534698a..341853e378bfec1ccec2c11e5622dbd7eb7dfee3:/blog/tags/isenkram/index.html diff --git a/blog/tags/isenkram/index.html b/blog/tags/isenkram/index.html index fa87f6d3da..47d28cc74f 100644 --- a/blog/tags/isenkram/index.html +++ b/blog/tags/isenkram/index.html @@ -20,6 +20,556 @@

Entries tagged "isenkram".

+
+
+ What is the most supported MIME type in Debian in 2018? +
+
+ 9th July 2018 +
+
+

Five years ago, +I +measured what the most supported MIME type in Debian was, by +analysing the desktop files in all packages in the archive. Since +then, the DEP-11 AppStream system has been put into production, making +the task a lot easier. This made me want to repeat the measurement, +to see how much things changed. Here are the new numbers, for +unstable only this time: + +

Debian Unstable:

+ +
+  count MIME type
+  ----- -----------------------
+     56 image/jpeg
+     55 image/png
+     49 image/tiff
+     48 image/gif
+     39 image/bmp
+     38 text/plain
+     37 audio/mpeg
+     34 application/ogg
+     33 audio/x-flac
+     32 audio/x-mp3
+     30 audio/x-wav
+     30 audio/x-vorbis+ogg
+     29 image/x-portable-pixmap
+     27 inode/directory
+     27 image/x-portable-bitmap
+     27 audio/x-mpeg
+     26 application/x-ogg
+     25 audio/x-mpegurl
+     25 audio/ogg
+     24 text/html
+
+ +

The list was created like this using a sid chroot: "cat +/var/lib/apt/lists/*sid*_dep11_Components-amd64.yml.gz| zcat | awk '/^ +- \S+\/\S+$/ {print $2 }' | sort | uniq -c | sort -nr | head -20"

+ +

It is interesting to see how image formats have passed text/plain +as the most announced supported MIME type. These days, thanks to the +AppStream system, if you run into a file format you do not know, and +want to figure out which packages support the format, you can find the +MIME type of the file using "file --mime <filename>", and then +look up all packages announcing support for this format in their +AppStream metadata (XML or .desktop file) using "appstreamcli +what-provides mimetype <mime-type>. For example if you, like +me, want to know which packages support inode/directory, you can get a +list like this:

+ +

+% appstreamcli what-provides mimetype inode/directory | grep Package: | sort
+Package: anjuta
+Package: audacious
+Package: baobab
+Package: cervisia
+Package: chirp
+Package: dolphin
+Package: doublecmd-common
+Package: easytag
+Package: enlightenment
+Package: ephoto
+Package: filelight
+Package: gwenview
+Package: k4dirstat
+Package: kaffeine
+Package: kdesvn
+Package: kid3
+Package: kid3-qt
+Package: nautilus
+Package: nemo
+Package: pcmanfm
+Package: pcmanfm-qt
+Package: qweborf
+Package: ranger
+Package: sirikali
+Package: spacefm
+Package: spacefm
+Package: vifm
+%
+

+ +

Using the same method, I can quickly discover that the Sketchup file +format is not yet supported by any package in Debian:

+ +

+% appstreamcli what-provides mimetype  application/vnd.sketchup.skp
+Could not find component providing 'mimetype::application/vnd.sketchup.skp'.
+%
+

+ +

Yesterday I used it to figure out which packages support the STL 3D +format:

+ +

+% appstreamcli what-provides mimetype  application/sla|grep Package
+Package: cura
+Package: meshlab
+Package: printrun
+%
+

+ +

PS: A new version of Cura was uploaded to Debian yesterday.

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+ +
+
+ + + Tags: debian, english, isenkram. + + +
+
+
+ +
+
+ Appstream just learned how to map hardware to packages too! +
+
+ 23rd December 2016 +
+
+

I received a very nice Christmas present today. As my regular +readers probably know, I have been working on the +the Isenkram +system for many years. The goal of the Isenkram system is to make +it easier for users to figure out what to install to get a given piece +of hardware to work in Debian, and a key part of this system is a way +to map hardware to packages. Isenkram have its own mapping database, +and also uses data provided by each package using the AppStream +metadata format. And today, +AppStream in +Debian learned to look up hardware the same way Isenkram is doing it, +ie using fnmatch():

+ +

+% appstreamcli what-provides modalias \
+  usb:v1130p0202d0100dc00dsc00dp00ic03isc00ip00in00
+Identifier: pymissile [generic]
+Name: pymissile
+Summary: Control original Striker USB Missile Launcher
+Package: pymissile
+% appstreamcli what-provides modalias usb:v0694p0002d0000
+Identifier: libnxt [generic]
+Name: libnxt
+Summary: utility library for talking to the LEGO Mindstorms NXT brick
+Package: libnxt
+---
+Identifier: t2n [generic]
+Name: t2n
+Summary: Simple command-line tool for Lego NXT
+Package: t2n
+---
+Identifier: python-nxt [generic]
+Name: python-nxt
+Summary: Python driver/interface/wrapper for the Lego Mindstorms NXT robot
+Package: python-nxt
+---
+Identifier: nbc [generic]
+Name: nbc
+Summary: C compiler for LEGO Mindstorms NXT bricks
+Package: nbc
+%
+

+ +

A similar query can be done using the combined AppStream and +Isenkram databases using the isenkram-lookup tool:

+ +

+% isenkram-lookup usb:v1130p0202d0100dc00dsc00dp00ic03isc00ip00in00
+pymissile
+% isenkram-lookup usb:v0694p0002d0000
+libnxt
+nbc
+python-nxt
+t2n
+%
+

+ +

You can find modalias values relevant for your machine using +cat $(find /sys/devices/ -name modalias). + +

If you want to make this system a success and help Debian users +make the most of the hardware they have, please +helpadd +AppStream metadata for your package following the guidelines +documented in the wiki. So far only 11 packages provide such +information, among the several hundred hardware specific packages in +Debian. The Isenkram database on the other hand contain 101 packages, +mostly related to USB dongles. Most of the packages with hardware +mapping in AppStream are LEGO Mindstorms related, because I have, as +part of my involvement in +the Debian LEGO +team given priority to making sure LEGO users get proposed the +complete set of packages in Debian for that particular hardware. The +team also got a nice Christmas present today. The +nxt-firmware +package made it into Debian. With this package in place, it is +now possible to use the LEGO Mindstorms NXT unit with only free +software, as the nxt-firmware package contain the source and firmware +binaries for the NXT brick.

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+ +
+
+ + + Tags: debian, english, isenkram. + + +
+
+
+ +
+
+ Isenkram updated with a lot more hardware-package mappings +
+
+ 20th December 2016 +
+
+

The Isenkram +system I wrote two years ago to make it easier in Debian to find +and install packages to get your hardware dongles to work, is still +going strong. It is a system to look up the hardware present on or +connected to the current system, and map the hardware to Debian +packages. It can either be done using the tools in isenkram-cli or +using the user space daemon in the isenkram package. The latter will +notify you, when inserting new hardware, about what packages to +install to get the dongle working. It will even provide a button to +click on to ask packagekit to install the packages.

+ +

Here is an command line example from my Thinkpad laptop:

+ +

+% isenkram-lookup  
+bluez
+cheese
+ethtool
+fprintd
+fprintd-demo
+gkrellm-thinkbat
+hdapsd
+libpam-fprintd
+pidgin-blinklight
+thinkfan
+tlp
+tp-smapi-dkms
+tp-smapi-source
+tpb
+%
+

+ +

It can also list the firware package providing firmware requested +by the load kernel modules, which in my case is an empty list because +I have all the firmware my machine need: + +

+% /usr/sbin/isenkram-autoinstall-firmware -l
+info: did not find any firmware files requested by loaded kernel modules.  exiting
+%
+

+ +

The last few days I had a look at several of the around 250 +packages in Debian with udev rules. These seem like good candidates +to install when a given hardware dongle is inserted, and I found +several that should be proposed by isenkram. I have not had time to +check all of them, but am happy to report that now there are 97 +packages packages mapped to hardware by Isenkram. 11 of these +packages provide hardware mapping using AppStream, while the rest are +listed in the modaliases file provided in isenkram.

+ +

These are the packages with hardware mappings at the moment. The +marked packages are also announcing their hardware +support using AppStream, for everyone to use:

+ +

air-quality-sensor, alsa-firmware-loaders, argyll, +array-info, avarice, avrdude, b43-fwcutter, +bit-babbler, bluez, bluez-firmware, brltty, +broadcom-sta-dkms, calibre, cgminer, cheese, colord, +colorhug-client, dahdi-firmware-nonfree, dahdi-linux, +dfu-util, dolphin-emu, ekeyd, ethtool, firmware-ipw2x00, fprintd, +fprintd-demo, galileo, gkrellm-thinkbat, gphoto2, +gpsbabel, gpsbabel-gui, gpsman, gpstrans, gqrx-sdr, gr-fcdproplus, +gr-osmosdr, gtkpod, hackrf, hdapsd, hdmi2usb-udev, hpijs-ppds, hplip, +ipw3945-source, ipw3945d, kde-config-tablet, kinect-audio-setup, +libnxt, libpam-fprintd, lomoco, +madwimax, minidisc-utils, mkgmap, msi-keyboard, mtkbabel, +nbc, nqc, nut-hal-drivers, ola, +open-vm-toolbox, open-vm-tools, openambit, pcgminer, pcmciautils, +pcscd, pidgin-blinklight, printer-driver-splix, +pymissile, python-nxt, qlandkartegt, +qlandkartegt-garmin, rosegarden, rt2x00-source, sispmctl, +soapysdr-module-hackrf, solaar, squeak-plugins-scratch, sunxi-tools, +t2n, thinkfan, thinkfinger-tools, tlp, tp-smapi-dkms, +tp-smapi-source, tpb, tucnak, uhd-host, usbmuxd, viking, +virtualbox-ose-guest-x11, w1retap, xawtv, xserver-xorg-input-vmmouse, +xserver-xorg-input-wacom, xserver-xorg-video-qxl, +xserver-xorg-video-vmware, yubikey-personalization and +zd1211-firmware

+ +

If you know of other packages, please let me know with a wishlist +bug report against the isenkram-cli package, and ask the package +maintainer to +add AppStream +metadata according to the guidelines to provide the information +for everyone. In time, I hope to get rid of the isenkram specific +hardware mapping and depend exclusively on AppStream.

+ +

Note, the AppStream metadata for broadcom-sta-dkms is matching too +much hardware, and suggest that the package with with any ethernet +card. See bug #838735 for +the details. I hope the maintainer find time to address it soon. In +the mean time I provide an override in isenkram.

+ +
+
+ + + Tags: debian, english, isenkram. + + +
+
+
+ +
+
+ Isenkram, Appstream and udev make life as a LEGO builder easier +
+
+ 7th October 2016 +
+
+

The Isenkram +system provide a practical and easy way to figure out which +packages support the hardware in a given machine. The command line +tool isenkram-lookup and the tasksel options provide a +convenient way to list and install packages relevant for the current +hardware during system installation, both user space packages and +firmware packages. The GUI background daemon on the other hand provide +a pop-up proposing to install packages when a new dongle is inserted +while using the computer. For example, if you plug in a smart card +reader, the system will ask if you want to install pcscd if +that package isn't already installed, and if you plug in a USB video +camera the system will ask if you want to install cheese if +cheese is currently missing. This already work just fine.

+ +

But Isenkram depend on a database mapping from hardware IDs to +package names. When I started no such database existed in Debian, so +I made my own data set and included it with the isenkram package and +made isenkram fetch the latest version of this database from git using +http. This way the isenkram users would get updated package proposals +as soon as I learned more about hardware related packages.

+ +

The hardware is identified using modalias strings. The modalias +design is from the Linux kernel where most hardware descriptors are +made available as a strings that can be matched using filename style +globbing. It handle USB, PCI, DMI and a lot of other hardware related +identifiers.

+ +

The downside to the Isenkram specific database is that there is no +information about relevant distribution / Debian version, making +isenkram propose obsolete packages too. But along came AppStream, a +cross distribution mechanism to store and collect metadata about +software packages. When I heard about the proposal, I contacted the +people involved and suggested to add a hardware matching rule using +modalias strings in the specification, to be able to use AppStream for +mapping hardware to packages. This idea was accepted and AppStream is +now a great way for a package to announce the hardware it support in a +distribution neutral way. I wrote +a +recipe on how to add such meta-information in a blog post last +December. If you have a hardware related package in Debian, please +announce the relevant hardware IDs using AppStream.

+ +

In Debian, almost all packages that can talk to a LEGO Mindestorms +RCX or NXT unit, announce this support using AppStream. The effect is +that when you insert such LEGO robot controller into your Debian +machine, Isenkram will propose to install the packages needed to get +it working. The intention is that this should allow the local user to +start programming his robot controller right away without having to +guess what packages to use or which permissions to fix.

+ +

But when I sat down with my son the other day to program our NXT +unit using his Debian Stretch computer, I discovered something +annoying. The local console user (ie my son) did not get access to +the USB device for programming the unit. This used to work, but no +longer in Jessie and Stretch. After some investigation and asking +around on #debian-devel, I discovered that this was because udev had +changed the mechanism used to grant access to local devices. The +ConsoleKit mechanism from /lib/udev/rules.d/70-udev-acl.rules +no longer applied, because LDAP users no longer was added to the +plugdev group during login. Michael Biebl told me that this method +was obsolete and the new method used ACLs instead. This was good +news, as the plugdev mechanism is a mess when using a remote user +directory like LDAP. Using ACLs would make sure a user lost device +access when she logged out, even if the user left behind a background +process which would retain the plugdev membership with the ConsoleKit +setup. Armed with this knowledge I moved on to fix the access problem +for the LEGO Mindstorms related packages.

+ +

The new system uses a udev tag, 'uaccess'. It can either be +applied directly for a device, or is applied in +/lib/udev/rules.d/70-uaccess.rules for classes of devices. As the +LEGO Mindstorms udev rules did not have a class, I decided to add the +tag directly in the udev rules files included in the packages. Here +is one example. For the nqc C compiler for the RCX, the +/lib/udev/rules.d/60-nqc.rules file now look like this: + +

+SUBSYSTEM=="usb", ACTION=="add", ATTR{idVendor}=="0694", ATTR{idProduct}=="0001", \
+    SYMLINK+="rcx-%k", TAG+="uaccess"
+

+ +

The key part is the 'TAG+="uaccess"' at the end. I suspect all +packages using plugdev in their /lib/udev/rules.d/ files should be +changed to use this tag (either directly or indirectly via +70-uaccess.rules). Perhaps a lintian check should be created +to detect this?

+ +

I've been unable to find good documentation on the uaccess feature. +It is unclear to me if the uaccess tag is an internal implementation +detail like the udev-acl tag used by +/lib/udev/rules.d/70-udev-acl.rules. If it is, I guess the +indirect method is the preferred way. Michael +asked for more +documentation from the systemd project and I hope it will make +this clearer. For now I use the generic classes when they exist and +is already handled by 70-uaccess.rules, and add the tag +directly if no such class exist.

+ +

To learn more about the isenkram system, please check out +my +blog posts tagged isenkram.

+ +

To help out making life for LEGO constructors in Debian easier, +please join us on our IRC channel +#debian-lego and join +the Debian +LEGO team in the Alioth project we created yesterday. A mailing +list is not yet created, but we are working on it. :)

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+ +
+
+ + + Tags: debian, english, isenkram, lego. + + +
+
+
+ +
+
+ Isenkram with PackageKit support - new version 0.23 available in Debian unstable +
+
+ 25th May 2016 +
+
+

The isenkram +system is a user-focused solution in Debian for handling hardware +related packages. The idea is to have a database of mappings between +hardware and packages, and pop up a dialog suggesting for the user to +install the packages to use a given hardware dongle. Some use cases +are when you insert a Yubikey, it proposes to install the software +needed to control it; when you insert a braille reader list it +proposes to install the packages needed to send text to the reader; +and when you insert a ColorHug screen calibrator it suggests to +install the driver for it. The system work well, and even have a few +command line tools to install firmware packages and packages for the +hardware already in the machine (as opposed to hotpluggable hardware).

+ +

The system was initially written using aptdaemon, because I found +good documentation and example code on how to use it. But aptdaemon +is going away and is generally being replaced by +PackageKit, +so Isenkram needed a rewrite. And today, thanks to the great patch +from my college Sunil Mohan Adapa in the FreedomBox project, the +rewrite finally took place. I've just uploaded a new version of +Isenkram into Debian Unstable with the patch included, and the default +for the background daemon is now to use PackageKit. To check it out, +install the isenkram package and insert some hardware dongle +and see if it is recognised.

+ +

If you want to know what kind of packages isenkram would propose for +the machine it is running on, you can check out the isenkram-lookup +program. This is what it look like on a Thinkpad X230:

+ +

+% isenkram-lookup 
+bluez
+cheese
+fprintd
+fprintd-demo
+gkrellm-thinkbat
+hdapsd
+libpam-fprintd
+pidgin-blinklight
+thinkfan
+tleds
+tp-smapi-dkms
+tp-smapi-source
+tpb
+%p
+

+ +

The hardware mappings come from several places. The preferred way +is for packages to announce their hardware support using +the +cross distribution appstream system. +See +previous +blog posts about isenkram to learn how to do that.

+ +
+
+ + + Tags: debian, english, isenkram. + + +
+
+
+
Using appstream with isenkram to install hardware related packages in Debian @@ -1163,6 +1713,58 @@ please send me an email. :)

Archive