X-Git-Url: http://pere.pagekite.me/gitweb/homepage.git/blobdiff_plain/a58ae4e89ee7e24a8e6f800b8e722560f2dcb3c1..b279793fc3681f16b43cc7dba38f0d6f4e22d247:/blog/index.html diff --git a/blog/index.html b/blog/index.html index 6f7e4f87ad..133f41dd38 100644 --- a/blog/index.html +++ b/blog/index.html @@ -20,37 +20,59 @@
-
Tor - from its creators mouth 11 years ago
-
28th May 2016
-

A little more than 11 years ago, one of the creators of Tor, and -the current President of the Tor project, Roger Dingledine, gave a -talk for the members of the Norwegian Unix User group (NUUG). A video -of the talk was recorded, and today I finally was able to publish the -video of the talk on Frikanalen, the Norwegian open channel TV station -where NUUG currently publishes its talks. You can -watch the live stream using a web -browser with WebM support, or check out the recording on the video -on demand page for the talk -"Tor: Anonymous -communication for the US Department of Defense...and you.".

- -

Here is the video included for those of you using browsers with -HTML video and Ogg Theora support:

- -

Browser does not support HTML5 video

- Please use Chrome, Opera, Firefox or any other browser which - supports Ogg Theora. -

- -

I guess the gist of the talk can be summarized quite simply: If you -want to help the military in USA (and everyone else), use Tor. :)

+ +
10th June 2017
+

I am very happy to report that the +Nikita Noark 5 +core project tagged its second release today. The free software +solution is an implementation of the Norwegian archive standard Noark +5 used by government offices in Norway. These were the changes in +version 0.1.1 since version 0.1.0 (from NEWS.md): + +

    + +
  • Continued work on the angularjs GUI, including document upload.
  • +
  • Implemented correspondencepartPerson, correspondencepartUnit and + correspondencepartInternal
  • +
  • Applied for coverity coverage and started submitting code on + regualr basis.
  • +
  • Started fixing bugs reported by coverity
  • +
  • Corrected and completed HATEOAS links to make sure entire API is + available via URLs in _links.
  • +
  • Corrected all relation URLs to use trailing slash.
  • +
  • Add initial support for storing data in ElasticSearch.
  • +
  • Now able to receive and store uploaded files in the archive.
  • +
  • Changed JSON output for object lists to have relations in _links.
  • +
  • Improve JSON output for empty object lists.
  • +
  • Now uses correct MIME type application/vnd.noark5-v4+json.
  • +
  • Added support for docker container images.
  • +
  • Added simple API browser implemented in JavaScript/Angular.
  • +
  • Started on archive client implemented in JavaScript/Angular.
  • +
  • Started on prototype to show the public mail journal.
  • +
  • Improved performance by disabling Sprint FileWatcher.
  • +
  • Added support for 'arkivskaper', 'saksmappe' and 'journalpost'.
  • +
  • Added support for some metadata codelists.
  • +
  • Added support for Cross-origin resource sharing (CORS).
  • +
  • Changed login method from Basic Auth to JSON Web Token (RFC 7519) + style.
  • +
  • Added support for GET-ing ny-* URLs.
  • +
  • Added support for modifying entities using PUT and eTag.
  • +
  • Added support for returning XML output on request.
  • +
  • Removed support for English field and class names, limiting ourself + to the official names.
  • +
  • ...
  • + +
+ +

If this sound interesting to you, please contact us on IRC (#nikita +on irc.freenode.net) or email +(nikita-noark +mailing list).

@@ -58,67 +80,99 @@ want to help the military in USA (and everyone else), use Tor. :)

- -
25th May 2016
-

The isenkram -system is a user-focused solution in Debian for handling hardware -related packages. The idea is to have a database of mappings between -hardware and packages, and pop up a dialog suggesting for the user to -install the packages to use a given hardware dongle. Some use cases -are when you insert a Yubikey, it proposes to install the software -needed to control it; when you insert a braille reader list it -proposes to install the packages needed to send text to the reader; -and when you insert a ColorHug screen calibrator it suggests to -install the driver for it. The system work well, and even have a few -command line tools to install firmware packages and packages for the -hardware already in the machine (as opposed to hotpluggable hardware).

- -

The system was initially written using aptdaemon, because I found -good documentation and example code on how to use it. But aptdaemon -is going away and is generally being replaced by -PackageKit, -so Isenkram needed a rewrite. And today, thanks to the great patch -from my college Sunil Mohan Adapa in the FreedomBox project, the -rewrite finally took place. I've just uploaded a new version of -Isenkram into Debian Unstable with the patch included, and the default -for the background daemon is now to use PackageKit. To check it out, -install the isenkram package and insert some hardware dongle -and see if it is recognised.

- -

If you want to know what kind of packages isenkram would propose for -the machine it is running on, you can check out the isenkram-lookup -program. This is what it look like on a Thinkpad X230:

+ +
7th June 2017
+

This is a copy of +an +email I posted to the nikita-noark mailing list. Please follow up +there if you would like to discuss this topic. The background is that +we are making a free software archive system based on the Norwegian +Noark +5 standard for government archives.

+ +

I've been wondering a bit lately how trusted timestamps could be +stored in Noark 5. +Trusted +timestamps can be used to verify that some information +(document/file/checksum/metadata) have not been changed since a +specific time in the past. This is useful to verify the integrity of +the documents in the archive.

+ +

Then it occured to me, perhaps the trusted timestamps could be +stored as dokument variants (ie dokumentobjekt referered to from +dokumentbeskrivelse) with the filename set to the hash it is +stamping?

+ +

Given a "dokumentbeskrivelse" with an associated "dokumentobjekt", +a new dokumentobjekt is associated with "dokumentbeskrivelse" with the +same attributes as the stamped dokumentobjekt except these +attributes:

+ +
    + +
  • format -> "RFC3161" +
  • mimeType -> "application/timestamp-reply" +
  • formatDetaljer -> "<source URL for timestamp service>" +
  • filenavn -> "<sjekksum>.tsr" + +
+ +

This assume a service following +IETF RFC 3161 is +used, which specifiy the given MIME type for replies and the .tsr file +ending for the content of such trusted timestamp. As far as I can +tell from the Noark 5 specifications, it is OK to have several +variants/renderings of a dokument attached to a given +dokumentbeskrivelse objekt. It might be stretching it a bit to make +some of these variants represent crypto-signatures useful for +verifying the document integrity instead of representing the dokument +itself.

+ +

Using the source of the service in formatDetaljer allow several +timestamping services to be used. This is useful to spread the risk +of key compromise over several organisations. It would only be a +problem to trust the timestamps if all of the organisations are +compromised.

+ +

The following oneliner on Linux can be used to generate the tsr +file. $input is the path to the file to checksum, and $sha256 is the +SHA-256 checksum of the file (ie the ".tsr" value mentioned +above).

+ +

+openssl ts -query -data "$inputfile" -cert -sha256 -no_nonce \
+  | curl -s -H "Content-Type: application/timestamp-query" \
+      --data-binary "@-" http://zeitstempel.dfn.de > $sha256.tsr
+

+ +

To verify the timestamp, you first need to download the public key +of the trusted timestamp service, for example using this command:

+ +

+wget -O ca-cert.txt \
+  https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt
+

+ +

Note, the public key should be stored alongside the timestamps in +the archive to make sure it is also available 100 years from now. It +is probably a good idea to standardise how and were to store such +public keys, to make it easier to find for those trying to verify +documents 100 or 1000 years from now. :)

+ +

The verification itself is a simple openssl command:

-% isenkram-lookup 
-bluez
-cheese
-fprintd
-fprintd-demo
-gkrellm-thinkbat
-hdapsd
-libpam-fprintd
-pidgin-blinklight
-thinkfan
-tleds
-tp-smapi-dkms
-tp-smapi-source
-tpb
-%p
+openssl ts -verify -data $inputfile -in $sha256.tsr \
+  -CAfile ca-cert.txt -text
 

-

The hardware mappings come from several places. The preferred way -is for packages to announce their hardware support using -the -cross distribution appstream system. -See -previous -blog posts about isenkram to learn how to do that.

+

Is there any reason this approach would not work? Is it somehow against +the Noark 5 specification?

@@ -126,62 +180,61 @@ blog posts about isenkram to learn how to do that.

- -
23rd May 2016
-

Yesterday I updated the -battery-stats -package in Debian with a few patches sent to me by skilled and -enterprising users. There were some nice user and visible changes. -First of all, both desktop menu entries now work. A design flaw in -one of the script made the history graph fail to show up (its PNG was -dumped in ~/.xsession-errors) if no controlling TTY was available. -The script worked when called from the command line, but not when -called from the desktop menu. I changed this to look for a DISPLAY -variable or a TTY before deciding where to draw the graph, and now the -graph window pop up as expected.

- -

The next new feature is a discharge rate estimator in one of the -graphs (the one showing the last few hours). New is also the user of -colours showing charging in blue and discharge in red. The percentages -of this graph is relative to last full charge, not battery design -capacity.

- -

- -

The other graph show the entire history of the collected battery -statistics, comparing it to the design capacity of the battery to -visualise how the battery life time get shorter over time. The red -line in this graph is what the previous graph considers 100 percent: - -

- -

In this graph you can see that I only charge the battery to 80 -percent of last full capacity, and how the capacity of the battery is -shrinking. :(

- -

The last new feature is in the collector, which now will handle -more hardware models. On some hardware, Linux power supply -information is stored in /sys/class/power_supply/ACAD/, while the -collector previously only looked in /sys/class/power_supply/AC/. Now -both are checked to figure if there is power connected to the -machine.

- -

If you are interested in how your laptop battery is doing, please -check out the -battery-stats -in Debian unstable, or rebuild it on Jessie to get it working on -Debian stable. :) The upstream source is available from github. -Patches are very welcome.

- -

As usual, if you use Bitcoin and want to show your support of my -activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+ +
3rd June 2017
+

Aftenposten +melder i dag om feil i eksamensoppgavene for eksamen i politikk og +menneskerettigheter, der teksten i bokmåls og nynorskutgaven ikke var +like. Oppgaveteksten er gjengitt i artikkelen, og jeg ble nysgjerring +på om den fri oversetterløsningen +Apertium ville gjort en bedre +jobb enn Utdanningsdirektoratet. Det kan se slik ut.

+ +

Her er bokmålsoppgaven fra eksamenen:

+ +
+

Drøft utfordringene knyttet til nasjonalstatenes og andre aktørers +rolle og muligheter til å håndtere internasjonale utfordringer, som +for eksempel flykningekrisen.

+ +

Vedlegge er eksempler på tekster som kan gi relevante perspektiver +på temaet:

+
    +
  1. Flykningeregnskapet 2016, UNHCR og IDMC +
  2. «Grenseløst Europa for fall» A-Magasinet, 26. november 2015 +
+ +
+ +

Dette oversetter Apertium slik:

+ +
+

Drøft utfordringane knytte til nasjonalstatane sine og rolla til +andre aktørar og høve til å handtera internasjonale utfordringar, som +til dømes *flykningekrisen.

+ +

Vedleggja er døme på tekster som kan gje relevante perspektiv på +temaet:

+ +
    +
  1. *Flykningeregnskapet 2016, *UNHCR og *IDMC
  2. +
  3. «*Grenseløst Europa for fall» A-Magasinet, 26. november 2015
  4. +
+ +
+ +

Ord som ikke ble forstått er markert med stjerne (*), og trenger +ekstra språksjekk. Men ingen ord er forsvunnet, slik det var i +oppgaven elevene fikk presentert på eksamen. Jeg mistenker dog at +"andre aktørers rolle og muligheter til ..." burde vært oversatt til +"rolla til andre aktørar og deira høve til ..." eller noe slikt, men +det er kanskje flisespikking. Det understreker vel bare at det alltid +trengs korrekturlesning etter automatisk oversettelse.

- Tags: debian, english. + Tags: debian, norsk, stavekontroll.
@@ -189,34 +242,67 @@ activities, please send Bitcoin donations to my address
- -
21st May 2016
-

A few weeks ago the French paperback edition of Lawrence Lessigs -2004 book Cultura Libre was published. Today I noticed that the book -is now available from book stores. You can now buy it from -Amazon -($19.99), -Barnes -& Noble ($?) and as always from -Lulu.com -($19.99). The revenue is donated to the Creative Commons project. If -you buy from Lulu.com, they currently get $10.59, while if you buy -from one of the book stores most of the revenue go to the book store -and the Creative Commons project get much (not sure how much -less).

- -

I was a bit surprised to discover that there is a kindle edition -sold by Amazon Digital Services LLC on Amazon. Not quite sure how -that edition was created, but if you want to download a electronic -edition (PDF, EPUB, Mobi) generated from the same files used to create -the paperback edition, they are -available -from github.

+ +
27th April 2017
+

I disse dager, med frist 1. mai, har Riksarkivaren ute en høring på +sin forskrift. Som en kan se er det ikke mye tid igjen før fristen +som går ut på søndag. Denne forskriften er det som lister opp hvilke +formater det er greit å arkivere i +Noark +5-løsninger i Norge.

+ +

Jeg fant høringsdokumentene hos +Norsk +Arkivråd etter å ha blitt tipset på epostlisten til +fri +programvareprosjektet Nikita Noark5-Core, som lager et Noark 5 +Tjenestegresesnitt. Jeg er involvert i Nikita-prosjektet og takket +være min interesse for tjenestegrensesnittsprosjektet har jeg lest en +god del Noark 5-relaterte dokumenter, og til min overraskelse oppdaget +at standard epost ikke er på listen over godkjente formater som kan +arkiveres. Høringen med frist søndag er en glimrende mulighet til å +forsøke å gjøre noe med det. Jeg holder på med +egen +høringsuttalelse, og lurer på om andre er interessert i å støtte +forslaget om å tillate arkivering av epost som epost i arkivet.

+ +

Er du igang med å skrive egen høringsuttalelse allerede? I så fall +kan du jo vurdere å ta med en formulering om epost-lagring. Jeg tror +ikke det trengs så mye. Her et kort forslag til tekst:

+ +

+ +

Viser til høring sendt ut 2017-02-17 (Riksarkivarens referanse + 2016/9840 HELHJO), og tillater oss å sende inn noen innspill om + revisjon av Forskrift om utfyllende tekniske og arkivfaglige + bestemmelser om behandling av offentlige arkiver (Riksarkivarens + forskrift).

+ +

Svært mye av vår kommuikasjon foregår i dag på e-post.  Vi + foreslår derfor at Internett-e-post, slik det er beskrevet i IETF + RFC 5322, + https://tools.ietf.org/html/rfc5322. bør + inn som godkjent dokumentformat.  Vi foreslår at forskriftens + oversikt over godkjente dokumentformater ved innlevering i § 5-16 + endres til å ta med Internett-e-post.

+ +

+ +

Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan +epost kan lagres i en Noark 5-struktur, og holder på å skrive et +forslag om hvordan dette kan gjøres som vil bli sendt over til +arkivverket så snart det er ferdig. De som er interesserte kan +følge +fremdriften på web.

+ +

Oppdatering 2017-04-28: I dag ble høringuttalelsen jeg skrev + sendt + inn av foreningen NUUG.

@@ -224,67 +310,52 @@ from github.

- -
19th May 2016
-

I just donated to the -NUUG defence -"fond" to fund the effort in Norway to get the seizure of the news -site popcorn-time.no tested in court. I hope everyone that agree with -me will do the same.

- -

Would you be worried if you knew the police in your country could -hijack DNS domains of news sites covering free software system without -talking to a judge first? I am. What if the free software system -combined search engine lookups, bittorrent downloads and video playout -and was called Popcorn Time? Would that affect your view? It still -make me worried.

- -

In March 2016, the Norwegian police seized (as in forced NORID to -change the IP address pointed to by it to one controlled by the -police) the DNS domain popcorn-time.no, without any supervision from -the courts. I did not know about the web site back then, and assumed -the courts had been involved, and was very surprised when I discovered -that the police had hijacked the DNS domain without asking a judge for -permission first. I was even more surprised when I had a look at -the web -site content on the Internet Archive, and only found news coverage -about Popcorn Time, not any material published without the right -holders permissions.

- -

The seizure was widely covered in the Norwegian press (see for -example Hegnar Online and -ITavisen -and -NRK), -at first due to the press release sent out by Økokrim, but then based -on -protests -from the law professor Olav Torvund and -lawyer -Jon Wessel-Aas. It even got some -coverage -on TorrentFreak.

- -

I - -wrote about the case a month ago, when the -Norwegian Unix User Group (NUUG), -where I am an active member, decided to ask the courts to test this seizure. -The request was denied, but NUUG and its co-requestor EFN have not -given up, and now they are rallying for support to get the seizure -legally challenged. They accept both bank and Bitcoin transfer for -those that want to support the request.

- -

If you as me believe news sites about free software should not be -censored, even if the free software have both legal and illegal -applications, and that DNS hijacking should be tested by the courts, I -suggest you show -your support by donating to NUUG. +

+
20th April 2017
+

Jeg oppdaget i dag at nettstedet som +publiserer offentlige postjournaler fra statlige etater, OEP, har +begynt å blokkerer enkelte typer webklienter fra å få tilgang. Vet +ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl +og curl. For å teste selv, kjør følgende:

+ +
+% curl -v -s https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
+< HTTP/1.1 404 Not Found
+% curl -v -s --header 'User-Agent:Opera/12.0' https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
+< HTTP/1.1 200 OK
+%
+
+ +

Her kan en se at tjenesten gir «404 Not Found» for curl i +standardoppsettet, mens den gir «200 OK» hvis curl hevder å være Opera +versjon 12.0. Offentlig elektronisk postjournal startet blokkeringen +2017-03-02.

+ +

Blokkeringen vil gjøre det litt vanskeligere å maskinelt hente +informasjon fra oep.no. Kan blokkeringen være gjort for å hindre +automatisert innsamling av informasjon fra OEP, slik Pressens +Offentlighetsutvalg gjorde for å dokumentere hvordan departementene +hindrer innsyn i +rapporten +«Slik hindrer departementer innsyn» som ble publiserte i januar +2017. Det virker usannsynlig, da det jo er trivielt å bytte +User-Agent til noe nytt.

+ +

Finnes det juridisk grunnlag for det offentlige å diskriminere +webklienter slik det gjøres her? Der tilgang gis eller ikke alt etter +hva klienten sier at den heter? Da OEP eies av DIFI og driftes av +Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to +aktørene man kan be om innsyn i for å forstå hva som har skjedd. Men +postjournalen +til DIFI viser kun to dokumenter det siste året mellom DIFI og +Basefarm. +Mimes brønn neste, +tenker jeg.

@@ -292,25 +363,101 @@ your support by donating to NUUG.
- -
12th May 2016
-

Today, after many years of hard work from many people, -ZFS for Linux finally entered -Debian. The package status can be seen on -the package tracker -for zfs-linux. and -the -team status page. If you want to help out, please join us. -The -source code is available via git on Alioth. It would also be -great if you could help out with -the dkms package, as -it is an important piece of the puzzle to get ZFS working.

+ +
19th March 2017
+

The Nikita +Noark 5 core project is implementing the Norwegian standard for +keeping an electronic archive of government documents. +The +Noark 5 standard document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +announced +it supported the project. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches.

+ +

If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(#nikita on +irc.freenode.net) and +the +project mailing list.

+ +

When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +archive-pdf to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +fonds, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester:

+ +

+~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
+using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
+using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
+
+ 0 - Title of the test case file created 2017-03-18T23:49:32.103446
+ 1 - Title of the test file created 2017-03-18T23:49:32.103446
+Select which mappe you want (or search term): 0
+Uploading mangelmelding/mangler.pdf
+  PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
+  File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
+~/src//noark5-tester$
+

+ +

You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The archive-pdf +tool can be found in the git repository for the API tester.

+ +

In the project, I have been mostly working on +the API +tester so far, while getting to know the code base. The API +tester currently use +the HATEOAS links +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification.

+ +

The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +started +writing down the questions we get from implementing it. We use a +format inspired by how The +Austin Group collect defect reports for the POSIX standard with +their +instructions for the MANTIS defect tracker system, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a request for a procedure for submitting defect reports :). + +

The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python.

@@ -318,49 +465,114 @@ it is an important piece of the puzzle to get ZFS working.

- -
8th May 2016
-

Where I set out to figure out which multimedia player in -Debian claim support for most file formats.

- -

A few years ago, I had a look at the media support for Browser -plugins in Debian, to get an idea which plugins to include in Debian -Edu. I created a script to extract the set of supported MIME types -for each plugin, and used this to find out which multimedia browser -plugin supported most file formats / media types. -The -result can still be seen on the Debian wiki, even though it have -not been updated for a while. But browser plugins are less relevant -these days, so I thought it was time to look at standalone -players.

- -

A few days ago I was tired of VLC not being listed as a viable -player when I wanted to play videos from the Norwegian National -Broadcasting Company, and decided to investigate why. The cause is a -missing MIME type in the VLC -desktop file. In the process I wrote a script to compare the set -of MIME types announced in the desktop file and the browser plugin, -only to discover that there is quite a large difference between the -two for VLC. This discovery made me dig up the script I used to -compare browser plugins, and adjust it to compare desktop files -instead, to try to figure out which multimedia player in Debian -support most file formats.

- -

The result can be seen on the Debian Wiki, as -a -table listing all MIME types supported by one of the packages included -in the table, with the package supporting most MIME types being -listed first in the table.

- -

The best multimedia player in Debian? It is totem, followed by -parole, kplayer, mpv, vlc, smplayer mplayer-gui gnome-mpv and -kmplayer. Time for the other players to update their announced MIME -support?

+ +
9th March 2017
+

Over the years, administrating thousand of NFS mounting linux +computers at the time, I often needed a way to detect if the machine +was experiencing NFS hang. If you try to use df or look at a +file or directory affected by the hang, the process (and possibly the +shell) will hang too. So you want to be able to detect this without +risking the detection process getting stuck too. It has not been +obvious how to do this. When the hang has lasted a while, it is +possible to find messages like these in dmesg:

+ +

+nfs: server nfsserver not responding, still trying +
nfs: server nfsserver OK +

+ +

It is hard to know if the hang is still going on, and it is hard to +be sure looking in dmesg is going to work. If there are lots of other +messages in dmesg the lines might have rotated out of site before they +are noticed.

+ +

While reading through the nfs client implementation in linux kernel +code, I came across some statistics that seem to give a way to detect +it. The om_timeouts sunrpc value in the kernel will increase every +time the above log entry is inserted into dmesg. And after digging a +bit further, I discovered that this value show up in +/proc/self/mountstats on Linux.

+ +

The mountstats content seem to be shared between files using the +same file system context, so it is enough to check one of the +mountstats files to get the state of the mount point for the machine. +I assume this will not show lazy umounted NFS points, nor NFS mount +points in a different process context (ie with a different filesystem +view), but that does not worry me.

+ +

The content for a NFS mount point look similar to this:

+ +

+[...]
+device /dev/mapper/Debian-var mounted on /var with fstype ext3
+device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1
+        opts:   rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all
+        age:    7863311
+        caps:   caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255
+        sec:    flavor=1,pseudoflavor=1
+        events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0 
+        bytes:  166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809 
+        RPC iostats version: 1.0  p/v: 100003/3 (nfs)
+        xprt:   tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820
+        per-op statistics
+                NULL: 0 0 0 0 0 0 0 0
+             GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132
+             SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943
+              LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511
+              ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140
+            READLINK: 125 125 0 20472 18620 0 1112 1118
+                READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693
+               WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771
+              CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398
+               MKDIR: 3680 3680 0 773980 993920 26 23990 24245
+             SYMLINK: 903 903 0 233428 245488 6 5865 5917
+               MKNOD: 80 80 0 20148 21760 0 299 304
+              REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636
+               RMDIR: 3367 3367 0 645112 484848 22 5782 6002
+              RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288
+                LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579
+             READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917
+         READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937
+              FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022
+              FSINFO: 2 2 0 232 328 0 1 1
+            PATHCONF: 1 1 0 116 140 0 0 0
+              COMMIT: 0 0 0 0 0 0 0 0
+
+device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
+[...]
+

+ +

The key number to look at is the third number in the per-op list. +It is the number of NFS timeouts experiences per file system +operation. Here 22 write timeouts and 5 access timeouts. If these +numbers are increasing, I believe the machine is experiencing NFS +hang. Unfortunately the timeout value do not start to increase right +away. The NFS operations need to time out first, and this can take a +while. The exact timeout value depend on the setup. For example the +defaults for TCP and UDP mount points are quite different, and the +timeout value is affected by the soft, hard, timeo and retrans NFS +mount options.

+ +

The only way I have been able to get working on Debian and RedHat +Enterprise Linux for getting the timeout count is to peek in /proc/. +But according to +Solaris +10 System Administration Guide: Network Services, the 'nfsstat -c' +command can be used to get these timeout values. But this do not work +on Linux, as far as I can tell. I +asked Debian about this, +but have not seen any replies yet.

+ +

Is there a better way to figure out if a Linux NFS client is +experiencing NFS hangs? Is there a way to detect which processes are +affected? Is there a way to get the NFS mount going quickly once the +network problem causing the NFS hang has been cleared? I would very +much welcome some clues, as we regularly run into NFS hangs.

@@ -368,29 +580,44 @@ support?

- -
4th May 2016
-
A friend of mine made me aware of -The Pyra, a -handheld computer which will be delivered with Debian preinstalled. I -would love to get one of those for my birthday. :)

- -

The machine is a complete ARM-based PC with micro HDMI, SATA, USB -plugs and many others connectors, and include a full keyboard and a 5" -LCD touch screen. The 6000mAh battery is claimed to provide a whole -day of battery life time, but I have not seen any independent tests -confirming this. The vendor is still collecting preorders, and the -last I heard last night was that 22 more orders were needed before -production started.

- -

As far as I know, this is the first handheld preinstalled with -Debian. Please let me know if you know of any others. Is it the -first computer being sold with Debian preinstalled?

+ +
8th March 2017
+

So the new president in the United States of America claim to be +surprised to discover that he was wiretapped during the election +before he was elected president. He even claim this must be illegal. +Well, doh, if it is one thing the confirmations from Snowden +documented, it is that the entire population in USA is wiretapped, one +way or another. Of course the president candidates were wiretapped, +alongside the senators, judges and the rest of the people in USA.

+ +

Next, the Federal Bureau of Investigation ask the Department of +Justice to go public rejecting the claims that Donald Trump was +wiretapped illegally. I fail to see the relevance, given that I am +sure the surveillance industry in USA believe they have all the legal +backing they need to conduct mass surveillance on the entire +world.

+ +

There is even the director of the FBI stating that he never saw an +order requesting wiretapping of Donald Trump. That is not very +surprising, given how the FISA court work, with all its activity being +secret. Perhaps he only heard about it?

+ +

What I find most sad in this story is how Norwegian journalists +present it. In a news reports the other day in the radio from the +Norwegian National broadcasting Company (NRK), I heard the journalist +claim that 'the FBI denies any wiretapping', while the reality is that +'the FBI denies any illegal wiretapping'. There is a fundamental and +important difference, and it make me sad that the journalists are +unable to grasp it.

+ +

Update 2017-03-13: Look like +The +Intercept report that US Senator Rand Paul confirm what I state above.

- Tags: debian, english. + Tags: english, surveillance.
@@ -398,28 +625,33 @@ first computer being sold with Debian preinstalled?

- -
18th April 2016
-

It is days like today I am really happy to be a member of -the Norwegian Unix User group, a -member association for those of us believing in free software, open -standards and unix-like operating systems. NUUG announced today it -will -try -to bring the seizure of the DNS domain popcorn-time.no as -unlawful, to stand up for the principle that writing about a -controversial topic is not infringing copyrights, and censuring web -pages by hijacking DNS domain should be decided by the courts, not the -police. The DNS domain was seized by the Norwegian National Authority -for Investigation and Prosecution of Economic and Environmental Crime -a month ago. I hope this bring more paying members to NUUG to give -the association the financial muscle needed to bring this case as far -as it must go to stop this kind of DNS hijacking.

+ +
3rd March 2017
+

For almost a year now, we have been working on making a Norwegian +Bokmål edition of The Debian +Administrator's Handbook. Now, thanks to the tireless effort of +Ole-Erik, Ingrid and Andreas, the initial translation is complete, and +we are working on the proof reading to ensure consistent language and +use of correct computer science terms. The plan is to make the book +available on paper, as well as in electronic form. For that to +happen, the proof reading must be completed and all the figures need +to be translated. If you want to help out, get in touch.

+ +

A + +fresh PDF edition in A4 format (the final book will have smaller +pages) of the book created every morning is available for +proofreading. If you find any errors, please +visit +Weblate and correct the error. The +state +of the translation including figures is a useful source for those +provide Norwegian bokmål screen shots and figures.

@@ -427,50 +659,77 @@ as it must go to stop this kind of DNS hijacking.

- -
15th April 2016
-

I dag tok jeg mot til meg og pakket sammen en ny versjon av den -frie norske stavekontrollen, ca. tre og et halvt år etter forrige -gang. Resultatet kan lastes ned fra -no.speling.org-prosjeksiden, både -som kildekodepakke og som "pack"-fil som kanskje fortsatt kan brukes -av OpenOffice.org/LibreOffice. Byggesystemet trenger oppussing, men i -denne omgang hadde jeg bare tid til å fikse byggefeil forårsaket av -endringer i GNU grep. De øvrige endringene var gjort tidligere i -påvente av en ny utgave.

- -

Her er det som er nytt (fra NEWS-fila i -kildekodepakken):

- -

Release 2.2 (2016-04-15)

- -
    - -
  • Rewrite how scripts/speling2words handle tripple consonants, to - avoid importing duplicate words from no.speling.org, and getting - rid of the existing duplicates in norsk.words.
  • -
  • Remove duplicate entries with tripple consonants from norsk.words.
  • -
  • Update frequency for entries in norsk.words based on - (ran 'make - freq-update').
  • -
  • Correct nn ispell build, avoid crash in munchlist causing lots of - words to fall out of the database.
  • -
  • Use grep -a to convince grep it is working on text files, to work - with newer grep versions.
  • - -
  • Remove some words disputed in the no.speling.org review process: -
      -
    • apparent (nb)
    • -
    • likke (nb)
    • -
    • ugjest, ugjesten, ugjestens (nb)
    • -
  • - -
+ +
1st March 2017
+

A few days ago I ordered a small batch of +the ChaosKey, a small +USB dongle for generating entropy created by Bdale Garbee and Keith +Packard. Yesterday it arrived, and I am very happy to report that it +work great! According to its designers, to get it to work out of the +box, you need the Linux kernel version 4.1 or later. I tested on a +Debian Stretch machine (kernel version 4.9), and there it worked just +fine, increasing the available entropy very quickly. I wrote a small +test oneliner to test. It first print the current entropy level, +drain /dev/random, and then print the entropy level for five seconds. +Here is the situation without the ChaosKey inserted:

+ +
+% cat /proc/sys/kernel/random/entropy_avail; \
+  dd bs=1M if=/dev/random of=/dev/null count=1; \
+  for n in $(seq 1 5); do \
+     cat /proc/sys/kernel/random/entropy_avail; \
+     sleep 1; \
+  done
+300
+0+1 oppføringer inn
+0+1 oppføringer ut
+28 byte kopiert, 0,000264565 s, 106 kB/s
+4
+8
+12
+17
+21
+%
+
+ +

The entropy level increases by 3-4 every second. In such case any +application requiring random bits (like a HTTPS enabled web server) +will halt and wait for more entrpy. And here is the situation with +the ChaosKey inserted:

+ +
+% cat /proc/sys/kernel/random/entropy_avail; \
+  dd bs=1M if=/dev/random of=/dev/null count=1; \
+  for n in $(seq 1 5); do \
+     cat /proc/sys/kernel/random/entropy_avail; \
+     sleep 1; \
+  done
+1079
+0+1 oppføringer inn
+0+1 oppføringer ut
+104 byte kopiert, 0,000487647 s, 213 kB/s
+433
+1028
+1031
+1035
+1038
+%
+
+ +

Quite the difference. :) I bought a few more than I need, in case +someone want to buy one here in Norway. :)

+ +

Update: The dongle was presented at Debconf last year. You might +find the talk +recording illuminating. It explains exactly what the source of +randomness is, if you are unable to spot it from the schema drawing +available from the ChaosKey web site linked at the start of this blog +post.

@@ -485,6 +744,21 @@ kildekodepakken):

Archive