The new "best" multimedia player in Debian?

6th June 2016

When I set out a few weeks ago to figure out -which -multimedia player in Debian claimed to support most file formats / -MIME types, I was a bit surprised how varied the sets of MIME types -the various players claimed support for. The range was from 55 to 130 -MIME types. I suspect most media formats are supported by all -players, but this is not really reflected in the MimeTypes values in -their desktop files. There are probably also some bogus MIME types -listed, but it is hard to identify which one this is.

- -

Anyway, in the mean time I got in touch with upstream for some of -the players suggesting to add more MIME types to their desktop files, -and decided to spend some time myself improving the situation for my -favorite media player VLC. The fixes for VLC entered Debian unstable -yesterday. The complete list of MIME types can be seen on the -Multimedia -player MIME type support status Debian wiki page.

- -

The new "best" multimedia player in Debian? It is VLC, followed by -totem, parole, kplayer, gnome-mpv, mpv, smplayer, mplayer-gui and -kmplayer. I am sure some of the other players desktop files support -several of the formats currently listed as working only with vlc, -toten and parole.

- -

A sad observation is that only 14 MIME types are listed as -supported by all the tested multimedia players in Debian in their -desktop files: audio/mpeg, audio/vnd.rn-realaudio, audio/x-mpegurl, -audio/x-ms-wma, audio/x-scpls, audio/x-wav, video/mp4, video/mpeg, -video/quicktime, video/vnd.rn-realvideo, video/x-matroska, -video/x-ms-asf, video/x-ms-wmv and video/x-msvideo. Personally I find -it sad that video/ogg and video/webm is not supported by all the media -players in Debian. As far as I can tell, all of them can handle both -formats.

Release 0.1.1 of free software archive system Nikita announced

10th June 2017

I am very happy to report that the +Nikita Noark 5 +core project tagged its second release today. The free software +solution is an implementation of the Norwegian archive standard Noark +5 used by government offices in Norway. These were the changes in +version 0.1.1 since version 0.1.0 (from NEWS.md): + +

Continued work on the angularjs GUI, including document upload.
Implemented correspondencepartPerson, correspondencepartUnit and + correspondencepartInternal
Applied for coverity coverage and started submitting code on + regualr basis.
Started fixing bugs reported by coverity
Corrected and completed HATEOAS links to make sure entire API is + available via URLs in _links.
Corrected all relation URLs to use trailing slash.
Add initial support for storing data in ElasticSearch.
Now able to receive and store uploaded files in the archive.
Changed JSON output for object lists to have relations in _links.
Improve JSON output for empty object lists.
Now uses correct MIME type application/vnd.noark5-v4+json.
Added support for docker container images.
Added simple API browser implemented in JavaScript/Angular.
Started on archive client implemented in JavaScript/Angular.
Started on prototype to show the public mail journal.
Improved performance by disabling Sprint FileWatcher.
Added support for 'arkivskaper', 'saksmappe' and 'journalpost'.
Added support for some metadata codelists.
Added support for Cross-origin resource sharing (CORS).
Changed login method from Basic Auth to JSON Web Token (RFC 7519) + style.
Added support for GET-ing ny-* URLs.
Added support for modifying entities using PUT and eTag.
Added support for returning XML output on request.
Removed support for English field and class names, limiting ourself + to the official names.
...

+ +

If this sound interesting to you, please contact us on IRC (#nikita +on irc.freenode.net) or email +(nikita-noark +mailing list).

- Tags: debian, debian edu, english, multimedia, video. + Tags: english, nuug, offentlig innsyn, standard.

@@ -67,118 +80,99 @@ formats.

A program should be able to open its own files on Linux

5th June 2016

Many years ago, when koffice was fresh and with few users, I -decided to test its presentation tool when making the slides for a -talk I was giving for NUUG on Japhar, a free Java virtual machine. I -wrote the first draft of the slides, saved the result and went to bed -the day before I would give the talk. The next day I took a plane to -the location where the meeting should take place, and on the plane I -started up koffice again to polish the talk a bit, only to discover -that kpresenter refused to load its own data file. I cursed a bit and -started making the slides again from memory, to have something to -present when I arrived. I tested that the saved files could be -loaded, and the day seemed to be rescued. I continued to polish the -slides until I suddenly discovered that the saved file could no longer -be loaded into kpresenter. In the end I had to rewrite the slides -three times, condensing the content until the talk became shorter and -shorter. After the talk I was able to pinpoint the problem – -kpresenter wrote inline images in a way itself could not understand. -Eventually that bug was fixed and kpresenter ended up being a great -program to make slides. The point I'm trying to make is that we -expect a program to be able to load its own data files, and it is -embarrassing to its developers if it can't.

- -

Did you ever experience a program failing to load its own data -files from the desktop file browser? It is not a uncommon problem. A -while back I discovered that the screencast recorder -gtk-recordmydesktop would save an Ogg Theora video file the KDE file -browser would refuse to open. No video player claimed to understand -such file. I tracked down the cause being file --mime-type -returning the application/ogg MIME type, which no video player I had -installed listed as a MIME type they would understand. I asked for -file to change its -behavour and use the MIME type video/ogg instead. I also asked -several video players to add video/ogg to their desktop files, to give -the file browser an idea what to do about Ogg Theora files. After a -while, the desktop file browsers in Debian started to handle the -output from gtk-recordmydesktop properly.

- -

But history repeats itself. A few days ago I tested the music -system Rosegarden again, and I discovered that the KDE and xfce file -browsers did not know what to do with the Rosegarden project files -(*.rg). I've reported the -rosegarden problem to BTS and a fix is commited to git and will be -included in the next upload. To increase the chance of me remembering -how to fix the problem next time some program fail to load its files -from the file browser, here are some notes on how to fix it.

- -

The file browsers in Debian in general operates on MIME types. -There are two sources for the MIME type of a given file. The output from -file --mime-type mentioned above, and the content of the -shared MIME type registry (under /usr/share/mime/). The file MIME -type is mapped to programs supporting the MIME type, and this -information is collected from -the -desktop files available in /usr/share/applications/. If there is -one desktop file claiming support for the MIME type of the file, it is -activated when asking to open a given file. If there are more, one -can normally select which one to use by right-clicking on the file and -selecting the wanted one using 'Open with' or similar. In general -this work well. But it depend on each program picking a good MIME -type (preferably -a -MIME type registered with IANA), file and/or the shared MIME -registry recognizing the file and the desktop file to list the MIME -type in its list of supported MIME types.

- -

The /usr/share/mime/packages/rosegarden.xml entry for -the -Shared MIME database look like this:

Idea for storing trusted timestamps in a Noark 5 archive

7th June 2017

This is a copy of +an +email I posted to the nikita-noark mailing list. Please follow up +there if you would like to discuss this topic. The background is that +we are making a free software archive system based on the Norwegian +Noark +5 standard for government archives.

+ +

I've been wondering a bit lately how trusted timestamps could be +stored in Noark 5. +Trusted +timestamps can be used to verify that some information +(document/file/checksum/metadata) have not been changed since a +specific time in the past. This is useful to verify the integrity of +the documents in the archive.

+ +

Then it occured to me, perhaps the trusted timestamps could be +stored as dokument variants (ie dokumentobjekt referered to from +dokumentbeskrivelse) with the filename set to the hash it is +stamping?

+ +

Given a "dokumentbeskrivelse" with an associated "dokumentobjekt", +a new dokumentobjekt is associated with "dokumentbeskrivelse" with the +same attributes as the stamped dokumentobjekt except these +attributes:

+ +

format -> "RFC3161" +
mimeType -> "application/timestamp-reply" +
formatDetaljer -> "<source URL for timestamp service>" +
filenavn -> "<sjekksum>.tsr" + +

+ +

This assume a service following +IETF RFC 3161 is +used, which specifiy the given MIME type for replies and the .tsr file +ending for the content of such trusted timestamp. As far as I can +tell from the Noark 5 specifications, it is OK to have several +variants/renderings of a dokument attached to a given +dokumentbeskrivelse objekt. It might be stretching it a bit to make +some of these variants represent crypto-signatures useful for +verifying the document integrity instead of representing the dokument +itself.

+ +

Using the source of the service in formatDetaljer allow several +timestamping services to be used. This is useful to spread the risk +of key compromise over several organisations. It would only be a +problem to trust the timestamps if all of the organisations are +compromised.

+ +

The following oneliner on Linux can be used to generate the tsr +file. $input is the path to the file to checksum, and $sha256 is the +SHA-256 checksum of the file (ie the ".tsr" value mentioned +above).

-<?xml version="1.0" encoding="UTF-8"?>
-<mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info">
-  <mime-type type="audio/x-rosegarden">
-    <sub-class-of type="application/x-gzip"/>
-    <comment>Rosegarden project file</comment>
-    <glob pattern="*.rg"/>
-  </mime-type>
-</mime-info>
+openssl ts -query -data "$inputfile" -cert -sha256 -no_nonce \
+  | curl -s -H "Content-Type: application/timestamp-query" \
+      --data-binary "@-" http://zeitstempel.dfn.de > $sha256.tsr

This states that audio/x-rosegarden is a kind of application/x-gzip -(it is a gzipped XML file). Note, it is much better to use an -official MIME type registered with IANA than it is to make up ones own -unofficial ones like the x-rosegarden type used by rosegarden.

- -

The desktop file of the rosegarden program failed to list -audio/x-rosegarden in its list of supported MIME types, causing the -file browsers to have no idea what to do with *.rg files:

To verify the timestamp, you first need to download the public key +of the trusted timestamp service, for example using this command:

-% grep Mime /usr/share/applications/rosegarden.desktop
-MimeType=audio/x-rosegarden-composition;audio/x-rosegarden-device;audio/x-rosegarden-project;audio/x-rosegarden-template;audio/midi;
-X-KDE-NativeMimeType=audio/x-rosegarden-composition
-%
+wget -O ca-cert.txt \
+  https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt

The fix was to add "audio/x-rosegarden;" at the end of the -MimeType= line.

Note, the public key should be stored alongside the timestamps in +the archive to make sure it is also available 100 years from now. It +is probably a good idea to standardise how and were to store such +public keys, to make it easier to find for those trying to verify +documents 100 or 1000 years from now. :)

If you run into a file which fail to open the correct program when -selected from the file browser, please check out the output from -file --mime-type for the file, ensure the file ending and -MIME type is registered somewhere under /usr/share/mime/ and check -that some desktop file under /usr/share/applications/ is claiming -support for this MIME type. If not, please report a bug to have it -fixed. :)

The verification itself is a simple openssl command:

+ +

+openssl ts -verify -data $inputfile -in $sha256.tsr \
+  -CAfile ca-cert.txt -text
+

+ +

Is there any reason this approach would not work? Is it somehow against +the Noark 5 specification?

- Tags: debian, english. + Tags: english, offentlig innsyn, standard.

@@ -186,36 +180,61 @@ fixed. :)

Tor - from its creators mouth 11 years ago

28th May 2016

A little more than 11 years ago, one of the creators of Tor, and -the current President of the Tor -project, Roger Dingledine, gave a talk for the members of the -Norwegian Unix User group (NUUG). A -video of the talk was recorded, and today, thanks to the great help -from David Noble, I finally was able to publish the video of the talk -on Frikanalen, the Norwegian open channel TV station where NUUG -currently publishes its talks. You can -watch the live stream using a web -browser with WebM support, or check out the recording on the video -on demand page for the talk -"Tor: Anonymous -communication for the US Department of Defence...and you.".

- -

Here is the video included for those of you using browsers with -HTML video and Ogg Theora support:

- -

I guess the gist of the talk can be summarised quite simply: If you -want to help the military in USA (and everyone else), use Tor. :)

NÃ¥r nynorskoversettelsen svikter til eksamen...

3rd June 2017

Aftenposten +melder i dag om feil i eksamensoppgavene for eksamen i politikk og +menneskerettigheter, der teksten i bokmÃ¥ls og nynorskutgaven ikke var +like. Oppgaveteksten er gjengitt i artikkelen, og jeg ble nysgjerring +pÃ¥ om den fri oversetterlÃ¸sningen +Apertium ville gjort en bedre +jobb enn Utdanningsdirektoratet. Det kan se slik ut.

+ +

Her er bokmÃ¥lsoppgaven fra eksamenen:

+ +

+
DrÃ¸ft utfordringene knyttet til nasjonalstatenes og andre aktÃ¸rers +rolle og muligheter til Ã¥ hÃ¥ndtere internasjonale utfordringer, som +for eksempel flykningekrisen.
+ +
Vedlegge er eksempler pÃ¥ tekster som kan gi relevante perspektiver +pÃ¥ temaet:
+
+
Flykningeregnskapet 2016, UNHCR og IDMC +
Â«GrenselÃ¸st Europa for fallÂ» A-Magasinet, 26. november 2015 +
+ +

+ +

Dette oversetter Apertium slik:

+ +

+
DrÃ¸ft utfordringane knytte til nasjonalstatane sine og rolla til +andre aktÃ¸rar og hÃ¸ve til Ã¥ handtera internasjonale utfordringar, som +til dÃ¸mes *flykningekrisen.
+ +
Vedleggja er dÃ¸me pÃ¥ tekster som kan gje relevante perspektiv pÃ¥ +temaet:
+ +
+
*Flykningeregnskapet 2016, *UNHCR og *IDMC
+
Â«*GrenselÃ¸st Europa for fallÂ» A-Magasinet, 26. november 2015
+
+ +

+ +

Ord som ikke ble forstÃ¥tt er markert med stjerne (*), og trenger +ekstra sprÃ¥ksjekk. Men ingen ord er forsvunnet, slik det var i +oppgaven elevene fikk presentert pÃ¥ eksamen. Jeg mistenker dog at +"andre aktÃ¸rers rolle og muligheter til ..." burde vÃ¦rt oversatt til +"rolla til andre aktÃ¸rar og deira hÃ¸ve til ..." eller noe slikt, men +det er kanskje flisespikking. Det understreker vel bare at det alltid +trengs korrekturlesning etter automatisk oversettelse.

- Tags: english, frikanalen, nuug, video. + Tags: debian, norsk, stavekontroll.

@@ -223,67 +242,67 @@ want to help the military in USA (and everyone else), use Tor. :)

Isenkram with PackageKit support - new version 0.23 available in Debian unstable

25th May 2016

The isenkram -system is a user-focused solution in Debian for handling hardware -related packages. The idea is to have a database of mappings between -hardware and packages, and pop up a dialog suggesting for the user to -install the packages to use a given hardware dongle. Some use cases -are when you insert a Yubikey, it proposes to install the software -needed to control it; when you insert a braille reader list it -proposes to install the packages needed to send text to the reader; -and when you insert a ColorHug screen calibrator it suggests to -install the driver for it. The system work well, and even have a few -command line tools to install firmware packages and packages for the -hardware already in the machine (as opposed to hotpluggable hardware).

- -

The system was initially written using aptdaemon, because I found -good documentation and example code on how to use it. But aptdaemon -is going away and is generally being replaced by -PackageKit, -so Isenkram needed a rewrite. And today, thanks to the great patch -from my college Sunil Mohan Adapa in the FreedomBox project, the -rewrite finally took place. I've just uploaded a new version of -Isenkram into Debian Unstable with the patch included, and the default -for the background daemon is now to use PackageKit. To check it out, -install the isenkram package and insert some hardware dongle -and see if it is recognised.

- -

If you want to know what kind of packages isenkram would propose for -the machine it is running on, you can check out the isenkram-lookup -program. This is what it look like on a Thinkpad X230:

- -

-% isenkram-lookup 
-bluez
-cheese
-fprintd
-fprintd-demo
-gkrellm-thinkbat
-hdapsd
-libpam-fprintd
-pidgin-blinklight
-thinkfan
-tleds
-tp-smapi-dkms
-tp-smapi-source
-tpb
-%p
-

- -

The hardware mappings come from several places. The preferred way -is for packages to announce their hardware support using -the -cross distribution appstream system. -See -previous -blog posts about isenkram to learn how to do that.

Epost inn som arkivformat i Riksarkivarens forskrift?

27th April 2017

I disse dager, med frist 1. mai, har Riksarkivaren ute en hÃ¸ring pÃ¥ +sin forskrift. Som en kan se er det ikke mye tid igjen fÃ¸r fristen +som gÃ¥r ut pÃ¥ sÃ¸ndag. Denne forskriften er det som lister opp hvilke +formater det er greit Ã¥ arkivere i +Noark +5-lÃ¸sninger i Norge.

+ +

Jeg fant hÃ¸ringsdokumentene hos +Norsk +ArkivrÃ¥d etter Ã¥ ha blitt tipset pÃ¥ epostlisten til +fri +programvareprosjektet Nikita Noark5-Core, som lager et Noark 5 +Tjenestegresesnitt. Jeg er involvert i Nikita-prosjektet og takket +vÃ¦re min interesse for tjenestegrensesnittsprosjektet har jeg lest en +god del Noark 5-relaterte dokumenter, og til min overraskelse oppdaget +at standard epost ikke er pÃ¥ listen over godkjente formater som kan +arkiveres. HÃ¸ringen med frist sÃ¸ndag er en glimrende mulighet til Ã¥ +forsÃ¸ke Ã¥ gjÃ¸re noe med det. Jeg holder pÃ¥ med +egen +hÃ¸ringsuttalelse, og lurer pÃ¥ om andre er interessert i Ã¥ stÃ¸tte +forslaget om Ã¥ tillate arkivering av epost som epost i arkivet.

+ +

Er du igang med Ã¥ skrive egen hÃ¸ringsuttalelse allerede? I sÃ¥ fall +kan du jo vurdere Ã¥ ta med en formulering om epost-lagring. Jeg tror +ikke det trengs sÃ¥ mye. Her et kort forslag til tekst:

+ +

+ +
Viser til hÃ¸ring sendt ut 2017-02-17 (Riksarkivarens referanse + 2016/9840 HELHJO), og tillater oss Ã¥ sende inn noen innspill om + revisjon av Forskrift om utfyllende tekniske og arkivfaglige + bestemmelser om behandling av offentlige arkiver (Riksarkivarens + forskrift).
+ +
SvÃ¦rt mye av vÃ¥r kommuikasjon foregÃ¥r i dag pÃ¥ e-post.Â Vi + foreslÃ¥r derfor at Internett-e-post, slik det er beskrevet i IETF + RFC 5322, + https://tools.ietf.org/html/rfc5322. bÃ¸r + inn som godkjent dokumentformat.Â Vi foreslÃ¥r at forskriftens + oversikt over godkjente dokumentformater ved innlevering i Â§ 5-16 + endres til Ã¥ ta med Internett-e-post.
+ +

+ +

Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan +epost kan lagres i en Noark 5-struktur, og holder pÃ¥ Ã¥ skrive et +forslag om hvordan dette kan gjÃ¸res som vil bli sendt over til +arkivverket sÃ¥ snart det er ferdig. De som er interesserte kan +fÃ¸lge +fremdriften pÃ¥ web.

+ +

Oppdatering 2017-04-28: I dag ble hÃ¸ringuttalelsen jeg skrev + sendt + inn av foreningen NUUG.

- Tags: debian, english, isenkram. + Tags: norsk, offentlig innsyn, standard.

@@ -291,62 +310,52 @@ blog posts about isenkram to learn how to do that.

Discharge rate estimate in new battery statistics collector for Debian

23rd May 2016

Yesterday I updated the -battery-stats -package in Debian with a few patches sent to me by skilled and -enterprising users. There were some nice user and visible changes. -First of all, both desktop menu entries now work. A design flaw in -one of the script made the history graph fail to show up (its PNG was -dumped in ~/.xsession-errors) if no controlling TTY was available. -The script worked when called from the command line, but not when -called from the desktop menu. I changed this to look for a DISPLAY -variable or a TTY before deciding where to draw the graph, and now the -graph window pop up as expected.

- -

The next new feature is a discharge rate estimator in one of the -graphs (the one showing the last few hours). New is also the user of -colours showing charging in blue and discharge in red. The percentages -of this graph is relative to last full charge, not battery design -capacity.

- -

The other graph show the entire history of the collected battery -statistics, comparing it to the design capacity of the battery to -visualise how the battery life time get shorter over time. The red -line in this graph is what the previous graph considers 100 percent: - -

- -

In this graph you can see that I only charge the battery to 80 -percent of last full capacity, and how the capacity of the battery is -shrinking. :(

- -

The last new feature is in the collector, which now will handle -more hardware models. On some hardware, Linux power supply -information is stored in /sys/class/power_supply/ACAD/, while the -collector previously only looked in /sys/class/power_supply/AC/. Now -both are checked to figure if there is power connected to the -machine.

- -

If you are interested in how your laptop battery is doing, please -check out the -battery-stats -in Debian unstable, or rebuild it on Jessie to get it working on -Debian stable. :) The upstream source is available from github. -Patches are very welcome.

- -

As usual, if you use Bitcoin and want to show your support of my -activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Offentlig elektronisk postjournal blokkerer tilgang for utvalgte webklienter

20th April 2017

Jeg oppdaget i dag at nettstedet som +publiserer offentlige postjournaler fra statlige etater, OEP, har +begynt Ã¥ blokkerer enkelte typer webklienter fra Ã¥ fÃ¥ tilgang. Vet +ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl +og curl. For Ã¥ teste selv, kjÃ¸r fÃ¸lgende:

+ +

+% curl -v -s https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
+< HTTP/1.1 404 Not Found
+% curl -v -s --header 'User-Agent:Opera/12.0' https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
+< HTTP/1.1 200 OK
+%
+

+ +

Her kan en se at tjenesten gir Â«404 Not FoundÂ» for curl i +standardoppsettet, mens den gir Â«200 OKÂ» hvis curl hevder Ã¥ vÃ¦re Opera +versjon 12.0. Offentlig elektronisk postjournal startet blokkeringen +2017-03-02.

+ +

Blokkeringen vil gjÃ¸re det litt vanskeligere Ã¥ maskinelt hente +informasjon fra oep.no. Kan blokkeringen vÃ¦re gjort for Ã¥ hindre +automatisert innsamling av informasjon fra OEP, slik Pressens +Offentlighetsutvalg gjorde for Ã¥ dokumentere hvordan departementene +hindrer innsyn i +rapporten +Â«Slik hindrer departementer innsynÂ» som ble publiserte i januar +2017. Det virker usannsynlig, da det jo er trivielt Ã¥ bytte +User-Agent til noe nytt.

+ +

Finnes det juridisk grunnlag for det offentlige Ã¥ diskriminere +webklienter slik det gjÃ¸res her? Der tilgang gis eller ikke alt etter +hva klienten sier at den heter? Da OEP eies av DIFI og driftes av +Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to +aktÃ¸rene man kan be om innsyn i for Ã¥ forstÃ¥ hva som har skjedd. Men +postjournalen +til DIFI viser kun to dokumenter det siste Ã¥ret mellom DIFI og +Basefarm. +Mimes brÃ¸nn neste, +tenker jeg.

- Tags: debian, english. + Tags: norsk, offentlig innsyn.

@@ -354,34 +363,101 @@ activities, please send Bitcoin donations to my address

French edition of Lawrence Lessigs book Cultura Libre on Amazon and Barnes & Noble

21st May 2016

A few weeks ago the French paperback edition of Lawrence Lessigs -2004 book Cultura Libre was published. Today I noticed that the book -is now available from book stores. You can now buy it from -Amazon -($19.99), -Barnes -& Noble ($?) and as always from -Lulu.com -($19.99). The revenue is donated to the Creative Commons project. If -you buy from Lulu.com, they currently get $10.59, while if you buy -from one of the book stores most of the revenue go to the book store -and the Creative Commons project get much (not sure how much -less).

- -

I was a bit surprised to discover that there is a kindle edition -sold by Amazon Digital Services LLC on Amazon. Not quite sure how -that edition was created, but if you want to download a electronic -edition (PDF, EPUB, Mobi) generated from the same files used to create -the paperback edition, they are -available -from github.

Free software archive system Nikita now able to store documents

19th March 2017

The Nikita +Noark 5 core project is implementing the Norwegian standard for +keeping an electronic archive of government documents. +The +Noark 5 standard document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +announced +it supported the project. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches.

+ +

If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(#nikita on +irc.freenode.net) and +the +project mailing list.

+ +

When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +archive-pdf to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +fonds, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester:

+ +

+~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
+using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
+using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
+
+ 0 - Title of the test case file created 2017-03-18T23:49:32.103446
+ 1 - Title of the test file created 2017-03-18T23:49:32.103446
+Select which mappe you want (or search term): 0
+Uploading mangelmelding/mangler.pdf
+  PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
+  File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
+~/src//noark5-tester$
+

+ +

You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The archive-pdf +tool can be found in the git repository for the API tester.

+ +

In the project, I have been mostly working on +the API +tester so far, while getting to know the code base. The API +tester currently use +the HATEOAS links +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification.

+ +

The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +started +writing down the questions we get from implementing it. We use a +format inspired by how The +Austin Group collect defect reports for the POSIX standard with +their +instructions for the MANTIS defect tracker system, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a request for a procedure for submitting defect reports :). + +

The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python.

- Tags: docbook, english, freeculture. + Tags: english, nuug, offentlig innsyn, standard.

@@ -389,67 +465,114 @@ from github.

I want the courts to be involved before the police can hijack a news site DNS domain (#domstolkontroll)

19th May 2016

I just donated to the -NUUG defence -"fond" to fund the effort in Norway to get the seizure of the news -site popcorn-time.no tested in court. I hope everyone that agree with -me will do the same.

- -

Would you be worried if you knew the police in your country could -hijack DNS domains of news sites covering free software system without -talking to a judge first? I am. What if the free software system -combined search engine lookups, bittorrent downloads and video playout -and was called Popcorn Time? Would that affect your view? It still -make me worried.

- -

In March 2016, the Norwegian police seized (as in forced NORID to -change the IP address pointed to by it to one controlled by the -police) the DNS domain popcorn-time.no, without any supervision from -the courts. I did not know about the web site back then, and assumed -the courts had been involved, and was very surprised when I discovered -that the police had hijacked the DNS domain without asking a judge for -permission first. I was even more surprised when I had a look at -the web -site content on the Internet Archive, and only found news coverage -about Popcorn Time, not any material published without the right -holders permissions.

- -

The seizure was widely covered in the Norwegian press (see for -example Hegnar Online and -ITavisen -and -NRK), -at first due to the press release sent out by Ãkokrim, but then based -on -protests -from the law professor Olav Torvund and -lawyer -Jon Wessel-Aas. It even got some -coverage -on TorrentFreak.

- -

I - -wrote about the case a month ago, when the -Norwegian Unix User Group (NUUG), -where I am an active member, decided to ask the courts to test this seizure. -The request was denied, but NUUG and its co-requestor EFN have not -given up, and now they are rallying for support to get the seizure -legally challenged. They accept both bank and Bitcoin transfer for -those that want to support the request.

- -

If you as me believe news sites about free software should not be -censored, even if the free software have both legal and illegal -applications, and that DNS hijacking should be tested by the courts, I -suggest you show -your support by donating to NUUG. +

Detecting NFS hangs on Linux without hanging yourself...

9th March 2017

Over the years, administrating thousand of NFS mounting linux +computers at the time, I often needed a way to detect if the machine +was experiencing NFS hang. If you try to use df or look at a +file or directory affected by the hang, the process (and possibly the +shell) will hang too. So you want to be able to detect this without +risking the detection process getting stuck too. It has not been +obvious how to do this. When the hang has lasted a while, it is +possible to find messages like these in dmesg:

+ +

+nfs: server nfsserver not responding, still trying +
nfs: server nfsserver OK +

+ +

It is hard to know if the hang is still going on, and it is hard to +be sure looking in dmesg is going to work. If there are lots of other +messages in dmesg the lines might have rotated out of site before they +are noticed.

+ +

While reading through the nfs client implementation in linux kernel +code, I came across some statistics that seem to give a way to detect +it. The om_timeouts sunrpc value in the kernel will increase every +time the above log entry is inserted into dmesg. And after digging a +bit further, I discovered that this value show up in +/proc/self/mountstats on Linux.

+ +

The mountstats content seem to be shared between files using the +same file system context, so it is enough to check one of the +mountstats files to get the state of the mount point for the machine. +I assume this will not show lazy umounted NFS points, nor NFS mount +points in a different process context (ie with a different filesystem +view), but that does not worry me.

+ +

The content for a NFS mount point look similar to this:

+ +

+[...]
+device /dev/mapper/Debian-var mounted on /var with fstype ext3
+device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1
+        opts:   rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all
+        age:    7863311
+        caps:   caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255
+        sec:    flavor=1,pseudoflavor=1
+        events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0 
+        bytes:  166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809 
+        RPC iostats version: 1.0  p/v: 100003/3 (nfs)
+        xprt:   tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820
+        per-op statistics
+                NULL: 0 0 0 0 0 0 0 0
+             GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132
+             SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943
+              LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511
+              ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140
+            READLINK: 125 125 0 20472 18620 0 1112 1118
+                READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693
+               WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771
+              CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398
+               MKDIR: 3680 3680 0 773980 993920 26 23990 24245
+             SYMLINK: 903 903 0 233428 245488 6 5865 5917
+               MKNOD: 80 80 0 20148 21760 0 299 304
+              REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636
+               RMDIR: 3367 3367 0 645112 484848 22 5782 6002
+              RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288
+                LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579
+             READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917
+         READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937
+              FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022
+              FSINFO: 2 2 0 232 328 0 1 1
+            PATHCONF: 1 1 0 116 140 0 0 0
+              COMMIT: 0 0 0 0 0 0 0 0
+
+device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
+[...]
+

+ +

The key number to look at is the third number in the per-op list. +It is the number of NFS timeouts experiences per file system +operation. Here 22 write timeouts and 5 access timeouts. If these +numbers are increasing, I believe the machine is experiencing NFS +hang. Unfortunately the timeout value do not start to increase right +away. The NFS operations need to time out first, and this can take a +while. The exact timeout value depend on the setup. For example the +defaults for TCP and UDP mount points are quite different, and the +timeout value is affected by the soft, hard, timeo and retrans NFS +mount options.

+ +

The only way I have been able to get working on Debian and RedHat +Enterprise Linux for getting the timeout count is to peek in /proc/. +But according to +Solaris +10 System Administration Guide: Network Services, the 'nfsstat -c' +command can be used to get these timeout values. But this do not work +on Linux, as far as I can tell. I +asked Debian about this, +but have not seen any replies yet.

+ +

Is there a better way to figure out if a Linux NFS client is +experiencing NFS hangs? Is there a way to detect which processes are +affected? Is there a way to get the NFS mount going quickly once the +network problem causing the NFS hang has been cleared? I would very +much welcome some clues, as we regularly run into NFS hangs.

- Tags: english, nuug, offentlig innsyn, opphavsrett. + Tags: debian, english, sysadmin.

@@ -457,25 +580,44 @@ your support by donating to NUUG.

Debian now with ZFS on Linux included

12th May 2016

Today, after many years of hard work from many people, -ZFS for Linux finally entered -Debian. The package status can be seen on -the package tracker -for zfs-linux. and -the -team status page. If you want to help out, please join us. -The -source code is available via git on Alioth. It would also be -great if you could help out with -the dkms package, as -it is an important piece of the puzzle to get ZFS working.

How does it feel to be wiretapped, when you should be doing the wiretapping...

8th March 2017

So the new president in the United States of America claim to be +surprised to discover that he was wiretapped during the election +before he was elected president. He even claim this must be illegal. +Well, doh, if it is one thing the confirmations from Snowden +documented, it is that the entire population in USA is wiretapped, one +way or another. Of course the president candidates were wiretapped, +alongside the senators, judges and the rest of the people in USA.

+ +

Next, the Federal Bureau of Investigation ask the Department of +Justice to go public rejecting the claims that Donald Trump was +wiretapped illegally. I fail to see the relevance, given that I am +sure the surveillance industry in USA believe they have all the legal +backing they need to conduct mass surveillance on the entire +world.

+ +

There is even the director of the FBI stating that he never saw an +order requesting wiretapping of Donald Trump. That is not very +surprising, given how the FISA court work, with all its activity being +secret. Perhaps he only heard about it?

+ +

What I find most sad in this story is how Norwegian journalists +present it. In a news reports the other day in the radio from the +Norwegian National broadcasting Company (NRK), I heard the journalist +claim that 'the FBI denies any wiretapping', while the reality is that +'the FBI denies any illegal wiretapping'. There is a fundamental and +important difference, and it make me sad that the journalists are +unable to grasp it.

+ +

Update 2017-03-13: Look like +The +Intercept report that US Senator Rand Paul confirm what I state above.

- Tags: debian, english. + Tags: english, surveillance.

@@ -483,49 +625,33 @@ it is an important piece of the puzzle to get ZFS working.

What is the best multimedia player in Debian?

8th May 2016

Where I set out to figure out which multimedia player in -Debian claim support for most file formats.

- -

A few years ago, I had a look at the media support for Browser -plugins in Debian, to get an idea which plugins to include in Debian -Edu. I created a script to extract the set of supported MIME types -for each plugin, and used this to find out which multimedia browser -plugin supported most file formats / media types. -The -result can still be seen on the Debian wiki, even though it have -not been updated for a while. But browser plugins are less relevant -these days, so I thought it was time to look at standalone -players.

- -

A few days ago I was tired of VLC not being listed as a viable -player when I wanted to play videos from the Norwegian National -Broadcasting Company, and decided to investigate why. The cause is a -missing MIME type in the VLC -desktop file. In the process I wrote a script to compare the set -of MIME types announced in the desktop file and the browser plugin, -only to discover that there is quite a large difference between the -two for VLC. This discovery made me dig up the script I used to -compare browser plugins, and adjust it to compare desktop files -instead, to try to figure out which multimedia player in Debian -support most file formats.

- -

The result can be seen on the Debian Wiki, as -a -table listing all MIME types supported by one of the packages included -in the table, with the package supporting most MIME types being -listed first in the table.

- -

The best multimedia player in Debian? It is totem, followed by -parole, kplayer, mpv, vlc, smplayer mplayer-gui gnome-mpv and -kmplayer. Time for the other players to update their announced MIME -support?

Norwegian BokmÃ¥l translation of The Debian Administrator's Handbook complete, proofreading in progress

3rd March 2017

For almost a year now, we have been working on making a Norwegian +BokmÃ¥l edition of The Debian +Administrator's Handbook. Now, thanks to the tireless effort of +Ole-Erik, Ingrid and Andreas, the initial translation is complete, and +we are working on the proof reading to ensure consistent language and +use of correct computer science terms. The plan is to make the book +available on paper, as well as in electronic form. For that to +happen, the proof reading must be completed and all the figures need +to be translated. If you want to help out, get in touch.

+ +

A + +fresh PDF edition in A4 format (the final book will have smaller +pages) of the book created every morning is available for +proofreading. If you find any errors, please +visit +Weblate and correct the error. The +state +of the translation including figures is a useful source for those +provide Norwegian bokmÃ¥l screen shots and figures.

- Tags: debian, debian edu, english, multimedia, video. + Tags: debian, debian-handbook, english.

@@ -533,24 +659,72 @@ support?

The Pyra - handheld computer with Debian preinstalled

4th May 2016

A friend of mine made me aware of -The Pyra, a -handheld computer which will be delivered with Debian preinstalled. I -would love to get one of those for my birthday. :)

- -

The machine is a complete ARM-based PC with micro HDMI, SATA, USB -plugs and many others connectors, and include a full keyboard and a 5" -LCD touch screen. The 6000mAh battery is claimed to provide a whole -day of battery life time, but I have not seen any independent tests -confirming this. The vendor is still collecting preorders, and the -last I heard last night was that 22 more orders were needed before -production started.

- -

As far as I know, this is the first handheld preinstalled with -Debian. Please let me know if you know of any others. Is it the -first computer being sold with Debian preinstalled?

Unlimited randomness with the ChaosKey?

1st March 2017

A few days ago I ordered a small batch of +the ChaosKey, a small +USB dongle for generating entropy created by Bdale Garbee and Keith +Packard. Yesterday it arrived, and I am very happy to report that it +work great! According to its designers, to get it to work out of the +box, you need the Linux kernel version 4.1 or later. I tested on a +Debian Stretch machine (kernel version 4.9), and there it worked just +fine, increasing the available entropy very quickly. I wrote a small +test oneliner to test. It first print the current entropy level, +drain /dev/random, and then print the entropy level for five seconds. +Here is the situation without the ChaosKey inserted:

+ +

+% cat /proc/sys/kernel/random/entropy_avail; \
+  dd bs=1M if=/dev/random of=/dev/null count=1; \
+  for n in $(seq 1 5); do \
+     cat /proc/sys/kernel/random/entropy_avail; \
+     sleep 1; \
+  done
+300
+0+1 oppfÃ¸ringer inn
+0+1 oppfÃ¸ringer ut
+28 byte kopiert, 0,000264565 s, 106 kB/s
+4
+8
+12
+17
+21
+%
+

+ +

The entropy level increases by 3-4 every second. In such case any +application requiring random bits (like a HTTPS enabled web server) +will halt and wait for more entrpy. And here is the situation with +the ChaosKey inserted:

+ +

+% cat /proc/sys/kernel/random/entropy_avail; \
+  dd bs=1M if=/dev/random of=/dev/null count=1; \
+  for n in $(seq 1 5); do \
+     cat /proc/sys/kernel/random/entropy_avail; \
+     sleep 1; \
+  done
+1079
+0+1 oppfÃ¸ringer inn
+0+1 oppfÃ¸ringer ut
+104 byte kopiert, 0,000487647 s, 213 kB/s
+433
+1028
+1031
+1035
+1038
+%
+

+ +

Quite the difference. :) I bought a few more than I need, in case +someone want to buy one here in Norway. :)

+ +

Update: The dongle was presented at Debconf last year. You might +find the talk +recording illuminating. It explains exactly what the source of +randomness is, if you are unable to spot it from the schema drawing +available from the ChaosKey web site linked at the start of this blog +post.

@@ -570,6 +744,21 @@ first computer being sold with Debian preinstalled?