Epost inn som arkivformat i Riksarkivarens forskrift?

27th April 2017

I disse dager, med frist 1. mai, har Riksarkivaren ute en hÃ¸ring pÃ¥ -sin forskrift. Som en kan se er det ikke mye tid igjen fÃ¸r fristen -som gÃ¥r ut pÃ¥ sÃ¸ndag. Denne forskriften er det som lister opp hvilke -formater det er greit Ã¥ arkivere i -Noark -5-lÃ¸sninger i Norge.

- -

Jeg fant hÃ¸ringsdokumentene hos -Norsk -ArkivrÃ¥d etter Ã¥ ha blitt tipset pÃ¥ epostlisten til -fri -programvareprosjektet Nikita Noark5-Core, som lager et Noark 5 -Tjenestegresesnitt. Jeg er involvert i Nikita-prosjektet og takket -vÃ¦re min interesse for tjenestegrensesnittsprosjektet har jeg lest en -god del Noark 5-relaterte dokumenter, og til min overraskelse oppdaget -at standard epost ikke er pÃ¥ listen over godkjente formater som kan -arkiveres. HÃ¸ringen med frist sÃ¸ndag er en glimrende mulighet til Ã¥ -forsÃ¸ke Ã¥ gjÃ¸re noe med det. Jeg holder pÃ¥ med -egen -hÃ¸ringsuttalelse, og lurer pÃ¥ om andre er interessert i Ã¥ stÃ¸tte -forslaget om Ã¥ tillate arkivering av epost som epost i arkivet.

- -

Er du igang med Ã¥ skrive egen hÃ¸ringsuttalelse allerede? I sÃ¥ fall -kan du jo vurdere Ã¥ ta med en formulering om epost-lagring. Jeg tror -ikke det trengs sÃ¥ mye. Her et kort forslag til tekst:

- -

- -
Viser til hÃ¸ring sendt ut 2017-02-17 (Riksarkivarens referanse - 2016/9840 HELHJO), og tillater oss Ã¥ sende inn noen innspill om - revisjon av Forskrift om utfyllende tekniske og arkivfaglige - bestemmelser om behandling av offentlige arkiver (Riksarkivarens - forskrift).
- -
SvÃ¦rt mye av vÃ¥r kommuikasjon foregÃ¥r i dag pÃ¥ e-post.Â Vi - foreslÃ¥r derfor at Internett-e-post, slik det er beskrevet i IETF - RFC 5322, - https://tools.ietf.org/html/rfc5322. bÃ¸r - inn som godkjent dokumentformat.Â Vi foreslÃ¥r at forskriftens - oversikt over godkjente dokumentformater ved innlevering i Â§ 5-16 - endres til Ã¥ ta med Internett-e-post.
- -

- -

Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan -epost kan lagres i en Noark 5-struktur, og holder pÃ¥ Ã¥ skrive et -forslag om hvordan dette kan gjÃ¸res som vil bli sendt over til -arkivverket sÃ¥ snart det er ferdig. De som er interesserte kan -fÃ¸lge -fremdriften pÃ¥ web.

Software created using taxpayersâ money should be Free Software

30th August 2018

It might seem obvious that software created using tax money should +be available for everyone to use and improve. Free Software +Foundation Europe recentlystarted a campaign to help get more people +to understand this, and I just signed the petition on +Public Money, Public Code to help +them. I hope you too will do the same.

- Tags: norsk, offentlig innsyn. + Tags: english, opphavsrett.

@@ -84,52 +40,71 @@ fremdriften pÃ¥ web.

Offentlig elektronisk postjournal blokkerer tilgang for utvalgte webklienter

20th April 2017

Jeg oppdaget i dag at nettstedet som -publiserer offentlige postjournaler fra statlige etater, OEP, har -begynt Ã¥ blokkerer enkelte typer webklienter fra Ã¥ fÃ¥ tilgang. Vet -ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl -og curl. For Ã¥ teste selv, kjÃ¸r fÃ¸lgende:

- -

-% curl -v -s https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
-< HTTP/1.1 404 Not Found
-% curl -v -s --header 'User-Agent:Opera/12.0' https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
-< HTTP/1.1 200 OK
-%
-

- -

Her kan en se at tjenesten gir Â«404 Not FoundÂ» for curl i -standardoppsettet, mens den gir Â«200 OKÂ» hvis curl hevder Ã¥ vÃ¦re Opera -versjon 12.0. Offentlig elektronisk postjournal startet blokkeringen -2017-03-02.

- -

Blokkeringen vil gjÃ¸re det litt vanskeligere Ã¥ maskinelt hente -informasjon fra oep.no. Kan blokkeringen vÃ¦re gjort for Ã¥ hindre -automatisert innsamling av informasjon fra OEP, slik Pressens -Offentlighetsutvalg gjorde for Ã¥ dokumentere hvordan departementene -hindrer innsyn i -rapporten -Â«Slik hindrer departementer innsynÂ» som ble publiserte i januar -2017. Det virker usannsynlig, da det jo er trivielt Ã¥ bytte -User-Agent til noe nytt.

- -

Finnes det juridisk grunnlag for det offentlige Ã¥ diskriminere -webklienter slik det gjÃ¸res her? Der tilgang gis eller ikke alt etter -hva klienten sier at den heter? Da OEP eies av DIFI og driftes av -Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to -aktÃ¸rene man kan be om innsyn i for Ã¥ forstÃ¥ hva som har skjedd. Men -postjournalen -til DIFI viser kun to dokumenter det siste Ã¥ret mellom DIFI og -Basefarm. -Mimes brÃ¸nn neste, -tenker jeg.

A bit more on privacy respecting health monitor / fitness tracker

13th August 2018

A few days ago, I wondered if there are any privacy respecting +health monitors and/or fitness trackers available for sale these days. +I would like to buy one, but do not want to share my personal data +with strangers, nor be forced to have a mobile phone to get data out +of the unit. I've received some ideas, and would like to share them +with you. + +One interesting data point was a pointer to a Free Software app for +Android named +Gadgetbridge. +It provide cloudless collection and storing of data from a variety of +trackers. Its +list +of supported devices is a good indicator for units where the +protocol is fairly open, as it is obviously being handled by Free +Software. Other units are reportedly encrypting the collected +information with their own public key, making sure only the vendor +cloud service is able to extract data from the unit. The people +contacting me about Gadgetbirde said they were using +Amazfit +Bip and +Xiaomi +Band 3.

+ +

I also got a suggestion to look at some of the units from Garmin. +I was told their GPS watches can be connected via USB and show up as a +USB storage device with +Garmin +FIT files containing the collected measurements. While +proprietary, FIT files apparently can be read at least by +GPSBabel and the +GpxPod Nextcloud +app. It is unclear to me if they can read step count and heart rate +data. The person I talked to was using a +Garmin Forerunner +935, which is a fairly expensive unit. I doubt it is worth it for +a unit where the vendor clearly is trying its best to move from open +to closed systems. I still remember when Garmin dropped NMEA support +in its GPSes.

+ +

A final idea was to build ones own unit, perhaps by basing it on a +wearable hardware platforms like +the Flora Geo +Watch. Sound like fun, but I had more money than time to spend on +the topic, so I suspect it will have to wait for another time.

+ +

While I was working on tracking down links, I came across an +inspiring TED talk by Dave Debronkart about +being a +e-patient, and discovered the web site +Participatory +Medicine. If you too want to track your own health and fitness +without having information about your private life floating around on +computers owned by others, I recommend checking it out.

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: norsk, offentlig innsyn. + Tags: english.

@@ -137,101 +112,36 @@ tenker jeg.

Free software archive system Nikita now able to store documents

19th March 2017

The Nikita -Noark 5 core project is implementing the Norwegian standard for -keeping an electronic archive of government documents. -The -Noark 5 standard document the requirement for data systems used by -the archives in the Norwegian government, and the Noark 5 web interface -specification document a REST web service for storing, searching and -retrieving documents and metadata in such archive. I've been involved -in the project since a few weeks before Christmas, when the Norwegian -Unix User Group -announced -it supported the project. I believe this is an important project, -and hope it can make it possible for the government archives in the -future to use free software to keep the archives we citizens depend -on. But as I do not hold such archive myself, personally my first use -case is to store and analyse public mail journal metadata published -from the government. I find it useful to have a clear use case in -mind when developing, to make sure the system scratches one of my -itches.

- -

If you would like to help make sure there is a free software -alternatives for the archives, please join our IRC channel -(#nikita on -irc.freenode.net) and -the -project mailing list.

- -

When I got involved, the web service could store metadata about -documents. But a few weeks ago, a new milestone was reached when it -became possible to store full text documents too. Yesterday, I -completed an implementation of a command line tool -archive-pdf to upload a PDF file to the archive using this -API. The tool is very simple at the moment, and find existing -fonds, series and -files while asking the user to select which one to use if more than -one exist. Once a file is identified, the PDF is associated with the -file and uploaded, using the title extracted from the PDF itself. The -process is fairly similar to visiting the archive, opening a cabinet, -locating a file and storing a piece of paper in the archive. Here is -a test run directly after populating the database with test data using -our API tester:

- -

-~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
-using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
-using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
-
- 0 - Title of the test case file created 2017-03-18T23:49:32.103446
- 1 - Title of the test file created 2017-03-18T23:49:32.103446
-Select which mappe you want (or search term): 0
-Uploading mangelmelding/mangler.pdf
-  PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
-  File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
-~/src//noark5-tester$
-

- -

You can see here how the fonds (arkiv) and serie (arkivdel) only had -one option, while the user need to choose which file (mappe) to use -among the two created by the API tester. The archive-pdf -tool can be found in the git repository for the API tester.

- -

In the project, I have been mostly working on -the API -tester so far, while getting to know the code base. The API -tester currently use -the HATEOAS links -to traverse the entire exposed service API and verify that the exposed -operations and objects match the specification, as well as trying to -create objects holding metadata and uploading a simple XML file to -store. The tester has proved very useful for finding flaws in our -implementation, as well as flaws in the reference site and the -specification.

- -

The test document I uploaded is a summary of all the specification -defects we have collected so far while implementing the web service. -There are several unclear and conflicting parts of the specification, -and we have -started -writing down the questions we get from implementing it. We use a -format inspired by how The -Austin Group collect defect reports for the POSIX standard with -their -instructions for the MANTIS defect tracker system, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a request for a procedure for submitting defect reports :). - -

The Nikita project is implemented using Java and Spring, and is -fairly easy to get up and running using Docker containers for those -that want to test the current code base. The API tester is -implemented in Python.

Privacy respecting health monitor / fitness tracker?

7th August 2018

Dear lazyweb,

+ +

I wonder, is there a fitness tracker / health monitor available for +sale today that respect the users privacy? With this I mean a +watch/bracelet capable of measuring pulse rate and other +fitness/health related values (and by all means, also the correct time +and location if possible), which is only provided for +me to extract/read from the unit with computer without a radio beacon +and Internet connection. In other words, it do not depend on a cell +phone app, and do make the measurements available via other peoples +computer (aka "the cloud"). The collected data should be available +using only free software. I'm not interested in depending on some +non-free software that will leave me high and dry some time in the +future. I've been unable to find any such unit. I would like to buy +it. The ones I have seen for sale here in Norway are proud to report +that they share my health data with strangers (aka "cloud enabled"). +Is there an alternative? I'm not interested in giving money to people +requiring me to accept "privacy terms" to allow myself to measure my +own health.

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: english, nuug, offentlig innsyn, standard. + Tags: english.

@@ -239,114 +149,75 @@ implemented in Python.

Detecting NFS hangs on Linux without hanging yourself...

9th March 2017

Over the years, administrating thousand of NFS mounting linux -computers at the time, I often needed a way to detect if the machine -was experiencing NFS hang. If you try to use df or look at a -file or directory affected by the hang, the process (and possibly the -shell) will hang too. So you want to be able to detect this without -risking the detection process getting stuck too. It has not been -obvious how to do this. When the hang has lasted a while, it is -possible to find messages like these in dmesg:

- -

-nfs: server nfsserver not responding, still trying -
nfs: server nfsserver OK -

- -

It is hard to know if the hang is still going on, and it is hard to -be sure looking in dmesg is going to work. If there are lots of other -messages in dmesg the lines might have rotated out of site before they -are noticed.

- -

While reading through the nfs client implementation in linux kernel -code, I came across some statistics that seem to give a way to detect -it. The om_timeouts sunrpc value in the kernel will increase every -time the above log entry is inserted into dmesg. And after digging a -bit further, I discovered that this value show up in -/proc/self/mountstats on Linux.

- -

The mountstats content seem to be shared between files using the -same file system context, so it is enough to check one of the -mountstats files to get the state of the mount point for the machine. -I assume this will not show lazy umounted NFS points, nor NFS mount -points in a different process context (ie with a different filesystem -view), but that does not worry me.

- -

The content for a NFS mount point look similar to this:

Sharing images with friends and family using RSS and EXIF/XMP metadata

31st July 2018

For a while now, I have looked for a sensible way to share images +with my family using a self hosted solution, as it is unacceptable to +place images from my personal life under the control of strangers +working for data hoarders like Google or Dropbox. The last few days I +have drafted an approach that might work out, and I would like to +share it with you. I would like to publish images on a server under +my control, and point some Internet connected display units using some +free and open standard to the images I published. As my primary +language is not limited to ASCII, I need to store metadata using +UTF-8. Many years ago, I hoped to find a digital photo frame capable +of reading a RSS feed with image references (aka using the +<enclosure> RSS tag), but was unable to find a current supplier +of such frames. In the end I gave up that approach.

+ +

Some months ago, I discovered that +XScreensaver is able to +read images from a RSS feed, and used it to set up a screen saver on +my home info screen, showing images from the Daily images feed from +NASA. This proved to work well. More recently I discovered that +Kodi (both using +OpenELEC and +LibreELEC) provide the +Feedreader +screen saver capable of reading a RSS feed with images and news. For +fun, I used it this summer to test Kodi on my parents TV by hooking up +a Raspberry PI unit with LibreELEC, and wanted to provide them with a +screen saver showing selected pictures from my selection.

+ +

Armed with motivation and a test photo frame, I set out to generate +a RSS feed for the Kodi instance. I adjusted my Freedombox instance, created +/var/www/html/privatepictures/, wrote a small Perl script to extract +title and description metadata from the photo files and generate the +RSS file. I ended up using Perl instead of python, as the +libimage-exiftool-perl Debian package seemed to handle the EXIF/XMP +tags I ended up using, while python3-exif did not. The relevant EXIF +tags only support ASCII, so I had to find better alternatives. XMP +seem to have the support I need.

+ +

I am a bit unsure which EXIF/XMP tags to use, as I would like to +use tags that can be easily added/updated using normal free software +photo managing software. I ended up using the tags set using this +exiftool command, as these tags can also be set using digiKam:

-[...]
-device /dev/mapper/Debian-var mounted on /var with fstype ext3
-device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1
-        opts:   rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all
-        age:    7863311
-        caps:   caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255
-        sec:    flavor=1,pseudoflavor=1
-        events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0 
-        bytes:  166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809 
-        RPC iostats version: 1.0  p/v: 100003/3 (nfs)
-        xprt:   tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820
-        per-op statistics
-                NULL: 0 0 0 0 0 0 0 0
-             GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132
-             SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943
-              LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511
-              ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140
-            READLINK: 125 125 0 20472 18620 0 1112 1118
-                READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693
-               WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771
-              CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398
-               MKDIR: 3680 3680 0 773980 993920 26 23990 24245
-             SYMLINK: 903 903 0 233428 245488 6 5865 5917
-               MKNOD: 80 80 0 20148 21760 0 299 304
-              REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636
-               RMDIR: 3367 3367 0 645112 484848 22 5782 6002
-              RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288
-                LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579
-             READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917
-         READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937
-              FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022
-              FSINFO: 2 2 0 232 328 0 1 1
-            PATHCONF: 1 1 0 116 140 0 0 0
-              COMMIT: 0 0 0 0 0 0 0 0
-
-device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
-[...]
-

+exiftool -headline='The RSS image title' \
+  -description='The RSS image description.' \
+  -subject+=for-family photo.jpeg
+

+ +

I initially tried the "-title" and "keyword" tags, but they were +invisible in digiKam, so I changed to "-headline" and "-subject". I +use the keyword/subject 'for-family' to flag that the photo should be +shared with my family. Images with this keyword set are located and +copied into my Freedombox for the RSS generating script to find.

+ +

Are there better ways to do this? Get in touch if you have better +suggestions.

The key number to look at is the third number in the per-op list. -It is the number of NFS timeouts experiences per file system -operation. Here 22 write timeouts and 5 access timeouts. If these -numbers are increasing, I believe the machine is experiencing NFS -hang. Unfortunately the timeout value do not start to increase right -away. The NFS operations need to time out first, and this can take a -while. The exact timeout value depend on the setup. For example the -defaults for TCP and UDP mount points are quite different, and the -timeout value is affected by the soft, hard, timeo and retrans NFS -mount options.

- -

The only way I have been able to get working on Debian and RedHat -Enterprise Linux for getting the timeout count is to peek in /proc/. -But according to -Solaris -10 System Administration Guide: Network Services, the 'nfsstat -c' -command can be used to get these timeout values. But this do not work -on Linux, as far as I can tell. I -asked Debian about this, -but have not seen any replies yet.

- -

Is there a better way to figure out if a Linux NFS client is -experiencing NFS hangs? Is there a way to detect which processes are -affected? Is there a way to get the NFS mount going quickly once the -network problem causing the NFS hang has been cleared? I would very -much welcome some clues, as we regularly run into NFS hangs.

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: debian, english, sysadmin. + Tags: debian, english.

@@ -354,44 +225,105 @@ much welcome some clues, as we regularly run into NFS hangs.

How does it feel to be wiretapped, when you should be doing the wiretapping...

8th March 2017

So the new president in the United States of America claim to be -surprised to discover that he was wiretapped during the election -before he was elected president. He even claim this must be illegal. -Well, doh, if it is one thing the confirmations from Snowden -documented, it is that the entire population in USA is wiretapped, one -way or another. Of course the president candidates were wiretapped, -alongside the senators, judges and the rest of the people in USA.

- -

Next, the Federal Bureau of Investigation ask the Department of -Justice to go public rejecting the claims that Donald Trump was -wiretapped illegally. I fail to see the relevance, given that I am -sure the surveillance industry in USA believe they have all the legal -backing they need to conduct mass surveillance on the entire -world.

- -

There is even the director of the FBI stating that he never saw an -order requesting wiretapping of Donald Trump. That is not very -surprising, given how the FISA court work, with all its activity being -secret. Perhaps he only heard about it?

- -

What I find most sad in this story is how Norwegian journalists -present it. In a news reports the other day in the radio from the -Norwegian National broadcasting Company (NRK), I heard the journalist -claim that 'the FBI denies any wiretapping', while the reality is that -'the FBI denies any illegal wiretapping'. There is a fundamental and -important difference, and it make me sad that the journalists are -unable to grasp it.

- -

Update 2017-03-13: Look like -The -Intercept report that US Senator Rand Paul confirm what I state above.

Simple streaming the Linux desktop to Kodi using GStreamer and RTP

12th July 2018

Last night, I wrote +a +recipe to stream a Linux desktop using VLC to a instance of Kodi. +During the day I received valuable feedback, and thanks to the +suggestions I have been able to rewrite the recipe into a much simpler +approach requiring no setup at all. It is a single script that take +care of it all.

+ +

This new script uses GStreamer instead of VLC to capture the +desktop and stream it to Kodi. This fixed the video quality issue I +saw initially. It further removes the need to add a m3u file on the +Kodi machine, as it instead connects to +the JSON-RPC API in +Kodi and simply ask Kodi to play from the stream created using +GStreamer. Streaming the desktop to Kodi now become trivial. Copy +the script below, run it with the DNS name or IP address of the kodi +server to stream to as the only argument, and watch your screen show +up on the Kodi screen. Note, it depend on multicast on the local +network, so if you need to stream outside the local network, the +script must be modified. Also note, I have no idea if audio work, as +I only care about the picture part.

+ +

+#!/bin/sh
+#
+# Stream the Linux desktop view to Kodi.  See
+# http://people.skolelinux.org/pere/blog/Streaming_the_Linux_desktop_to_Kodi_using_VLC_and_RTSP.html
+# for backgorund information.
+
+# Make sure the stream is stopped in Kodi and the gstreamer process is
+# killed if something go wrong (for example if curl is unable to find the
+# kodi server).  Do the same when interrupting this script.
+kodicmd() {
+    host="$1"
+    cmd="$2"
+    params="$3"
+    curl --silent --header 'Content-Type: application/json' \
+	 --data-binary "{ \"id\": 1, \"jsonrpc\": \"2.0\", \"method\": \"$cmd\", \"params\": $params }" \
+	 "http://$host/jsonrpc"
+}
+cleanup() {
+    if [ -n "$kodihost" ] ; then
+	# Stop the playing when we end
+	playerid=$(kodicmd "$kodihost" Player.GetActivePlayers "{}" |
+			    jq .result[].playerid)
+	kodicmd "$kodihost" Player.Stop "{ \"playerid\" : $playerid }" > /dev/null
+    fi
+    if [ "$gstpid" ] && kill -0 "$gstpid" >/dev/null 2>&1; then
+	kill "$gstpid"
+    fi
+}
+trap cleanup EXIT INT
+
+if [ -n "$1" ]; then
+    kodihost=$1
+    shift
+else
+    kodihost=kodi.local
+fi
+
+mcast=239.255.0.1
+mcastport=1234
+mcastttl=1
+
+pasrc=$(pactl list | grep -A2 'Source #' | grep 'Name: .*\.monitor$' | \
+  cut -d" " -f2|head -1)
+gst-launch-1.0 ximagesrc use-damage=0 ! video/x-raw,framerate=30/1 ! \
+  videoconvert ! queue2 ! \
+  x264enc bitrate=8000 speed-preset=superfast tune=zerolatency qp-min=30 \
+  key-int-max=15 bframes=2 ! video/x-h264,profile=high ! queue2 ! \
+  mpegtsmux alignment=7 name=mux ! rndbuffersize max=1316 min=1316 ! \
+  udpsink host=$mcast port=$mcastport ttl-mc=$mcastttl auto-multicast=1 sync=0 \
+  pulsesrc device=$pasrc ! audioconvert ! queue2 ! avenc_aac ! queue2 ! mux. \
+  > /dev/null 2>&1 &
+gstpid=$!
+
+# Give stream a second to get going
+sleep 1
+
+# Ask kodi to start streaming using its JSON-RPC API
+kodicmd "$kodihost" Player.Open \
+	"{\"item\": { \"file\": \"udp://@$mcast:$mcastport\" } }" > /dev/null
+
+# wait for gst to end
+wait "$gstpid"
+

+ +

I hope you find the approach useful. I know I do.

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: english, surveillance. + Tags: debian, english, video.

@@ -399,33 +331,152 @@ Intercept report that US Senator Rand Paul confirm what I state above.

Norwegian BokmÃ¥l translation of The Debian Administrator's Handbook complete, proofreading in progress

3rd March 2017

For almost a year now, we have been working on making a Norwegian -BokmÃ¥l edition of The Debian -Administrator's Handbook. Now, thanks to the tireless effort of -Ole-Erik, Ingrid and Andreas, the initial translation is complete, and -we are working on the proof reading to ensure consistent language and -use of correct computer science terms. The plan is to make the book -available on paper, as well as in electronic form. For that to -happen, the proof reading must be completed and all the figures need -to be translated. If you want to help out, get in touch.

- -

A - -fresh PDF edition in A4 format (the final book will have smaller -pages) of the book created every morning is available for -proofreading. If you find any errors, please -visit -Weblate and correct the error. The -state -of the translation including figures is a useful source for those -provide Norwegian bokmÃ¥l screen shots and figures.

Streaming the Linux desktop to Kodi using VLC and RTSP

12th July 2018

PS: See +the +followup post for a even better approach.

+ +

A while back, I was asked by a friend how to stream the desktop to +my projector connected to Kodi. I sadly had to admit that I had no +idea, as it was a task I never had tried. Since then, I have been +looking for a way to do so, preferable without much extra software to +install on either side. Today I found a way that seem to kind of +work. Not great, but it is a start.

+ +

I had a look at several approaches, for example +using uPnP +DLNA as described in 2011, but it required a uPnP server, fuse and +local storage enough to store the stream locally. This is not going +to work well for me, lacking enough free space, and it would +impossible for my friend to get working.

+ +

Next, it occurred to me that perhaps I could use VLC to create a +video stream that Kodi could play. Preferably using +broadcast/multicast, to avoid having to change any setup on the Kodi +side when starting such stream. Unfortunately, the only recipe I +could find using multicast used the rtp protocol, and this protocol +seem to not be supported by Kodi.

+ +

On the other hand, the rtsp protocol is working! Unfortunately I +have to specify the IP address of the streaming machine in both the +sending command and the file on the Kodi server. But it is showing my +desktop, and thus allow us to have a shared look on the big screen at +the programs I work on.

+ +

I did not spend much time investigating codeces. I combined the +rtp and rtsp recipes from +the +VLC Streaming HowTo/Command Line Examples, and was able to get +this working on the desktop/streaming end.

+ +

+vlc screen:// --sout \
+  '#transcode{vcodec=mp4v,acodec=mpga,vb=800,ab=128}:rtp{dst=projector.local,port=1234,sdp=rtsp://192.168.11.4:8080/test.sdp}'
+

+ +

I ssh-ed into my Kodi box and created a file like this with the +same IP address:

+ +

+echo rtsp://192.168.11.4:8080/test.sdp \
+  > /storage/videos/screenstream.m3u
+

+ +

Note the 192.168.11.4 IP address is my desktops IP address. As far +as I can tell the IP must be hardcoded for this to work. In other +words, if someone elses machine is going to do the steaming, you have +to update screenstream.m3u on the Kodi machine and adjust the vlc +recipe. To get started, locate the file in Kodi and select the m3u +file while the VLC stream is running. The desktop then show up in my +big screen. :)

+ +

When using the same technique to stream a video file with audio, +the audio quality is really bad. No idea if the problem is package +loss or bad parameters for the transcode. I do not know VLC nor Kodi +enough to tell.

+ +

Update 2018-07-12: Johannes Schauer send me a few +succestions and reminded me about an important step. The "screen:" +input source is only available once the vlc-plugin-access-extra +package is installed on Debian. Without it, you will see this error +message: "VLC is unable to open the MRL 'screen://'. Check the log +for details." He further found that it is possible to drop some parts +of the VLC command line to reduce the amount of hardcoded information. +It is also useful to consider using cvlc to avoid having the VLC +window in the desktop view. In sum, this give us this command line on +the source end + +

+cvlc screen:// --sout \
+  '#transcode{vcodec=mp4v,acodec=mpga,vb=800,ab=128}:rtp{sdp=rtsp://:8080/}'
+

+ +

and this on the Kodi end

+ +

+echo rtsp://192.168.11.4:8080/ \
+  > /storage/videos/screenstream.m3u
+

+ +

Still bad image quality, though. But I did discover that streaming +a DVD using dvdsimple:///dev/dvd as the source had excellent video and +audio quality, so I guess the issue is in the input or transcoding +parts, not the rtsp part. I've tried to change the vb and ab +parameters to use more bandwidth, but it did not make a +difference.

+ +

I further received a suggestion from Einar Haraldseid to try using +gstreamer instead of VLC, and this proved to work great! He also +provided me with the trick to get Kodi to use a multicast stream as +its source. By using this monstrous oneliner, I can stream my desktop +with good video quality in reasonable framerate to the 239.255.0.1 +multicast address on port 1234: + +

+gst-launch-1.0 ximagesrc use-damage=0 ! video/x-raw,framerate=30/1 ! \
+  videoconvert ! queue2 ! \
+  x264enc bitrate=8000 speed-preset=superfast tune=zerolatency qp-min=30 \
+  key-int-max=15 bframes=2 ! video/x-h264,profile=high ! queue2 ! \
+  mpegtsmux alignment=7 name=mux ! rndbuffersize max=1316 min=1316 ! \
+  udpsink host=239.255.0.1 port=1234 ttl-mc=1 auto-multicast=1 sync=0 \
+  pulsesrc device=$(pactl list | grep -A2 'Source #' | \
+    grep 'Name: .*\.monitor$' |  cut -d" " -f2|head -1) ! \
+  audioconvert ! queue2 ! avenc_aac ! queue2 ! mux.
+

+ +

and this on the Kodi end

+ +

+echo udp://@239.255.0.1:1234 \
+  > /storage/videos/screenstream.m3u
+

+ +

Note the trick to pick a valid pulseaudio source. It might not +pick the one you need. This approach will of course lead to trouble +if more than one source uses the same multicast port and address. +Note the ttl-mc=1 setting, which limit the multicast packages to the +local network. If the value is increased, your screen will be +broadcasted further, one network "hop" for each increase (read up on +multicast to learn more. :)!

+ +

Having cracked how to get Kodi to receive multicast streams, I +could use this VLC command to stream to the same multicast address. +The image quality is way better than the rtsp approach, but gstreamer +seem to be doing a better job.

+ +

+cvlc screen:// --sout '#transcode{vcodec=mp4v,acodec=mpga,vb=800,ab=128}:rtp{mux=ts,dst=239.255.0.1,port=1234,sdp=sap}'
+

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: debian, debian-handbook, english. + Tags: debian, english, video.

@@ -433,77 +484,121 @@ provide Norwegian bokmÃ¥l screen shots and figures.

Unlimited randomness with the ChaosKey?

1st March 2017

A few days ago I ordered a small batch of -the ChaosKey, a small -USB dongle for generating entropy created by Bdale Garbee and Keith -Packard. Yesterday it arrived, and I am very happy to report that it -work great! According to its designers, to get it to work out of the -box, you need the Linux kernel version 4.1 or later. I tested on a -Debian Stretch machine (kernel version 4.9), and there it worked just -fine, increasing the available entropy very quickly. I wrote a small -test oneliner to test. It first print the current entropy level, -drain /dev/random, and then print the entropy level for five seconds. -Here is the situation without the ChaosKey inserted:

What is the most supported MIME type in Debian in 2018?

9th July 2018

Five years ago, +I +measured what the most supported MIME type in Debian was, by +analysing the desktop files in all packages in the archive. Since +then, the DEP-11 AppStream system has been put into production, making +the task a lot easier. This made me want to repeat the measurement, +to see how much things changed. Here are the new numbers, for +unstable only this time: + +

Debian Unstable:

+ +

+  count MIME type
+  ----- -----------------------
+     56 image/jpeg
+     55 image/png
+     49 image/tiff
+     48 image/gif
+     39 image/bmp
+     38 text/plain
+     37 audio/mpeg
+     34 application/ogg
+     33 audio/x-flac
+     32 audio/x-mp3
+     30 audio/x-wav
+     30 audio/x-vorbis+ogg
+     29 image/x-portable-pixmap
+     27 inode/directory
+     27 image/x-portable-bitmap
+     27 audio/x-mpeg
+     26 application/x-ogg
+     25 audio/x-mpegurl
+     25 audio/ogg
+     24 text/html
+

+ +

The list was created like this using a sid chroot: "cat +/var/lib/apt/lists/*sid*_dep11_Components-amd64.yml.gz| zcat | awk '/^ +- \S+\/\S+$/ {print $2 }' | sort | uniq -c | sort -nr | head -20"

+ +

It is interesting to see how image formats have passed text/plain +as the most announced supported MIME type. These days, thanks to the +AppStream system, if you run into a file format you do not know, and +want to figure out which packages support the format, you can find the +MIME type of the file using "file --mime <filename>", and then +look up all packages announcing support for this format in their +AppStream metadata (XML or .desktop file) using "appstreamcli +what-provides mimetype <mime-type>. For example if you, like +me, want to know which packages support inode/directory, you can get a +list like this:

-% cat /proc/sys/kernel/random/entropy_avail; \ - dd bs=1M if=/dev/random of=/dev/null count=1; \ - for n in $(seq 1 5); do \ - cat /proc/sys/kernel/random/entropy_avail; \ - sleep 1; \ - done -300 -0+1 oppfÃ¸ringer inn -0+1 oppfÃ¸ringer ut -28 byte kopiert, 0,000264565 s, 106 kB/s -4 -8 -12 -17 -21 +

+% appstreamcli what-provides mimetype inode/directory | grep Package: | sort
+Package: anjuta
+Package: audacious
+Package: baobab
+Package: cervisia
+Package: chirp
+Package: dolphin
+Package: doublecmd-common
+Package: easytag
+Package: enlightenment
+Package: ephoto
+Package: filelight
+Package: gwenview
+Package: k4dirstat
+Package: kaffeine
+Package: kdesvn
+Package: kid3
+Package: kid3-qt
+Package: nautilus
+Package: nemo
+Package: pcmanfm
+Package: pcmanfm-qt
+Package: qweborf
+Package: ranger
+Package: sirikali
+Package: spacefm
+Package: spacefm
+Package: vifm
 %
-

The entropy level increases by 3-4 every second. In such case any -application requiring random bits (like a HTTPS enabled web server) -will halt and wait for more entrpy. And here is the situation with -the ChaosKey inserted:

Using the same method, I can quickly discover that the Sketchup file +format is not yet supported by any package in Debian:

-% cat /proc/sys/kernel/random/entropy_avail; \ - dd bs=1M if=/dev/random of=/dev/null count=1; \ - for n in $(seq 1 5); do \ - cat /proc/sys/kernel/random/entropy_avail; \ - sleep 1; \ - done -1079 -0+1 oppfÃ¸ringer inn -0+1 oppfÃ¸ringer ut -104 byte kopiert, 0,000487647 s, 213 kB/s -433 -1028 -1031 -1035 -1038 +

+% appstreamcli what-provides mimetype  application/vnd.sketchup.skp
+Could not find component providing 'mimetype::application/vnd.sketchup.skp'.
 %
-

+ +

Yesterday I used it to figure out which packages support the STL 3D +format:

Quite the difference. :) I bought a few more than I need, in case -someone want to buy one here in Norway. :)

+% appstreamcli what-provides mimetype  application/sla|grep Package
+Package: cura
+Package: meshlab
+Package: printrun
+%
+

Update: The dongle was presented at Debconf last year. You might -find the talk -recording illuminating. It explains exactly what the source of -randomness is, if you are unable to spot it from the schema drawing -available from the ChaosKey web site linked at the start of this blog -post.

PS: A new version of Cura was uploaded to Debian yesterday.

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: debian, english. + Tags: debian, english, isenkram.

@@ -511,34 +606,83 @@ post.

Detect OOXML files with undefined behaviour?

21st February 2017

I just noticed -the -new Norwegian proposal for archiving rules in the goverment list -ECMA-376 -/ ISO/IEC 29500 (aka OOXML) as valid formats to put in long term -storage. Luckily such files will only be accepted based on -pre-approval from the National Archive. Allowing OOXML files to be -used for long term storage might seem like a good idea as long as we -forget that there are plenty of ways for a "valid" OOXML document to -have content with no defined interpretation in the standard, which -lead to a question and an idea.

- -

Is there any tool to detect if a OOXML document depend on such -undefined behaviour? It would be useful for the National Archive (and -anyone else interested in verifying that a document is well defined) -to have such tool available when considering to approve the use of -OOXML. I'm aware of the -officeotron OOXML -validator, but do not know how complete it is nor if it will -report use of undefined behaviour. Are there other similar tools -available? Please send me an email if you know of any such tool.

Debian APT upgrade without enough free space on the disk...

8th July 2018

Quite regularly, I let my Debian Sid/Unstable chroot stay untouch +for a while, and when I need to update it there is not enough free +space on the disk for apt to do a normal 'apt upgrade'. I normally +would resolve the issue by doing 'apt install <somepackages>' to +upgrade only some of the packages in one batch, until the amount of +packages to download fall below the amount of free space available. +Today, I had about 500 packages to upgrade, and after a while I got +tired of trying to install chunks of packages manually. I concluded +that I did not have the spare hours required to complete the task, and +decided to see if I could automate it. I came up with this small +script which I call 'apt-in-chunks':

+ +

+#!/bin/sh
+#
+# Upgrade packages when the disk is too full to upgrade every
+# upgradable package in one lump.  Fetching packages to upgrade using
+# apt, and then installing using dpkg, to avoid changing the package
+# flag for manual/automatic.
+
+set -e
+
+ignore() {
+    if [ "$1" ]; then
+	grep -v "$1"
+    else
+	cat
+    fi
+}
+
+for p in $(apt list --upgradable | ignore "$@" |cut -d/ -f1 | grep -v '^Listing...'); do
+    echo "Upgrading $p"
+    apt clean
+    apt install --download-only -y $p
+    for f in /var/cache/apt/archives/*.deb; do
+	if [ -e "$f" ]; then
+	    dpkg -i /var/cache/apt/archives/*.deb
+	    break
+	fi
+    done
+done
+

+ +

The script will extract the list of packages to upgrade, try to +download the packages needed to upgrade one package, install the +downloaded packages using dpkg. The idea is to upgrade packages +without changing the APT mark for the package (ie the one recording of +the package was manually requested or pulled in as a dependency). To +use it, simply run it as root from the command line. If it fail, try +'apt install -f' to clean up the mess and run the script again. This +might happen if the new packages conflict with one of the old +packages. dpkg is unable to remove, while apt can do this.

+ +

It take one option, a package to ignore in the list of packages to +upgrade. The option to ignore a package is there to be able to skip +the packages that are simply too large to unpack. Today this was +'ghc', but I have run into other large packages causing similar +problems earlier (like TeX).

+ +

Update 2018-07-08: Thanks to Paul Wise, I am aware of two +alternative ways to handle this. The "unattended-upgrades +--minimal-upgrade-steps" option will try to calculate upgrade sets for +each package to upgrade, and then upgrade them in order, smallest set +first. It might be a better option than my above mentioned script. +Also, "aptutude upgrade" can upgrade single packages, thus avoiding +the need for using "dpkg -i" in the script above.

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: english, nuug, standard. + Tags: debian, english.

@@ -546,33 +690,32 @@ available? Please send me an email if you know of any such tool.

Ruling ignored our objections to the seizure of popcorn-time.no (#domstolkontroll)

13th February 2017

A few days ago, we received the ruling from -my -day in court. The case in question is a challenge of the seizure -of the DNS domain popcorn-time.no. The ruling simply did not mention -most of our arguments, and seemed to take everything ÃKOKRIM said at -face value, ignoring our demonstration and explanations. But it is -hard to tell for sure, as we still have not seen most of the documents -in the case and thus were unprepared and unable to contradict several -of the claims made in court by the opposition. We are considering an -appeal, but it is partly a question of funding, as it is costing us -quite a bit to pay for our lawyer. If you want to help, please -donate to the -NUUG defense fund.

- -

The details of the case, as far as we know it, is available in -Norwegian from -the NUUG -blog. This also include -the -ruling itself.

The worlds only stone power plant?

30th June 2018

So far, at least hydro-electric power, coal power, wind power, +solar power, and wood power are well known. Until a few days ago, I +had never heard of stone power. Then I learn about a quarry in a +mountain in +Bremanger i +Norway, where +the +Bremanger Quarry company is extracting stone and dumping the stone +into a shaft leading to its shipping harbour. This downward movement +in this shaft is used to produce electricity. In short, it is using +falling rocks instead of falling water to produce electricity, and +according to its own statements it is producing more power than it is +using, and selling the surplus electricity to the Norwegian power +grid. I find the concept truly amazing. Is this the worlds only +stone power plant?

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: english, nuug, offentlig innsyn, opphavsrett. + Tags: english.

@@ -580,86 +723,66 @@ ruling itself.

A day in court challenging seizure of popcorn-time.no for #domstolkontroll

3rd February 2017

- -

On Wednesday, I spent the entire day in court in Follo Tingrett -representing the member association -NUUG, alongside the member -association EFN and the DNS registrar -IMC, challenging the seizure of the DNS name popcorn-time.no. It -was interesting to sit in a court of law for the first time in my -life. Our team can be seen in the picture above: attorney Ola -TellesbÃ¸, EFN board member Tom Fredrik Blenning, IMC CEO Morten Emil -Eriksen and NUUG board member Petter Reinholdtsen.

- -

The -case at hand is that the Norwegian National Authority for -Investigation and Prosecution of Economic and Environmental Crime (aka -Ãkokrim) decided on their own, to seize a DNS domain early last -year, without following -the -official policy of the Norwegian DNS authority which require a -court decision. The web site in question was a site covering Popcorn -Time. And Popcorn Time is the name of a technology with both legal -and illegal applications. Popcorn Time is a client combining -searching a Bittorrent directory available on the Internet with -downloading/distribute content via Bittorrent and playing the -downloaded content on screen. It can be used illegally if it is used -to distribute content against the will of the right holder, but it can -also be used legally to play a lot of content, for example the -millions of movies -available from the -Internet Archive or the collection -available from Vodo. We created -a -video demonstrating legally use of Popcorn Time and played it in -Court. It can of course be downloaded using Bittorrent.

- -

I did not quite know what to expect from a day in court. The -government held on to their version of the story and we held on to -ours, and I hope the judge is able to make sense of it all. We will -know in two weeks time. Unfortunately I do not have high hopes, as -the Government have the upper hand here with more knowledge about the -case, better training in handling criminal law and in general higher -standing in the courts than fairly unknown DNS registrar and member -associations. It is expensive to be right also in Norway. So far the -case have cost more than NOK 70 000,-. To help fund the case, NUUG -and EFN have asked for donations, and managed to collect around NOK 25 -000,- so far. Given the presentation from the Government, I expect -the government to appeal if the case go our way. And if the case do -not go our way, I hope we have enough funding to appeal.

- -

From the other side came two people from Ãkokrim. On the benches, -appearing to be part of the group from the government were two people -from the Simonsen Vogt Wiik lawyer office, and three others I am not -quite sure who was. Ãkokrim had proposed to present two witnesses -from The Motion Picture Association, but this was rejected because -they did not speak Norwegian and it was a bit late to bring in a -translator, but perhaps the two from MPA were present anyway. All -seven appeared to know each other. Good to see the case is take -seriously.

- -

If you, like me, believe the courts should be involved before a DNS -domain is hijacked by the government, or you believe the Popcorn Time -technology have a lot of useful and legal applications, I suggest you -too donate to -the NUUG defense fund. Both Bitcoin and bank transfer are -available. If NUUG get more than we need for the legal action (very -unlikely), the rest will be spend promoting free software, open -standards and unix-like operating systems in Norway, so no matter what -happens the money will be put to good use.

- -

If you want to lean more about the case, I recommend you check out -the blog -posts from NUUG covering the case. They cover the legal arguments -on both sides.

Add-on to control the projector from within Kodi

26th June 2018

My movie playing setup involve Kodi, +OpenELEC (probably soon to be +replaced with LibreELEC) and an +Infocus IN76 video projector. My projector can be controlled via both +a infrared remote controller, and a RS-232 serial line. The vendor of +my projector, InFocus, had been +sensible enough to document the serial protocol in its user manual, so +it is easily available, and I used it some years ago to write +a +small script to control the projector. For a while now, I longed +for a setup where the projector was controlled by Kodi, for example in +such a way that when the screen saver went on, the projector was +turned off, and when the screen saver exited, the projector was turned +on again.

+ +

A few days ago, with very good help from parts of my family, I +managed to find a Kodi Add-on for controlling a Epson projector, and +got in touch with its author to see if we could join forces and make a +Add-on with support for several projectors. To my pleasure, he was +positive to the idea, and we set out to add InFocus support to his +add-on, and make the add-on suitable for the official Kodi add-on +repository.

+ +

The Add-on is now working (for me, at least), with a few minor +adjustments. The most important change I do relative to the master +branch in the github repository is embedding the +pyserial module in +the add-on. The long term solution is to make a "script" type +pyserial module for Kodi, that can be pulled in as a dependency in +Kodi. But until that in place, I embed it.

+ +

The add-on can be configured to turn on the projector when Kodi +starts, off when Kodi stops as well as turn the projector off when the +screensaver start and on when the screesaver stops. It can also be +told to set the projector source when turning on the projector. + +

If this sound interesting to you, check out +the +project github repository. Perhaps you can send patches to +support your projector too? As soon as we find time to wrap up the +latest changes, it should be available for easy installation using any +Kodi instance.

+ +

For future improvements, I would like to add projector model +detection and the ability to adjust the brightness level of the +projector from within Kodi. We also need to figure out how to handle +the cooling period of the projector. My projector refuses to turn on +for 60 seconds after it was turned off. This is not handled well by +the add-on at the moment.

+ +

As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: english, nuug, offentlig innsyn, opphavsrett. + Tags: english, multimedia, video.

@@ -674,6 +797,25 @@ on both sides.