New blog post on sales numbers.

[homepage.git] / blog / index.rss

<?xml version="1.0" encoding="utf-8"?>
<rss version='2.0' xmlns:lj='http://www.livejournal.org/rss/lj/1.0/' xmlns:atom="http://www.w3.org/2005/Atom">
        <channel>
                <title>Petter Reinholdtsen</title>
                <description></description>
                <link>http://people.skolelinux.org/pere/blog/</link>
                <atom:link href="http://people.skolelinux.org/pere/blog/index.rss" rel="self" type="application/rss+xml" />
        
        <item>
                <title>Release 0.1.1 of free software archive system Nikita announced</title>
                <link>http://people.skolelinux.org/pere/blog/Release_0_1_1_of_free_software_archive_system_Nikita_announced.html</link>
                <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Release_0_1_1_of_free_software_archive_system_Nikita_announced.html</guid>
                <pubDate>Sat, 10 Jun 2017 00:40:00 +0200</pubDate>
                <description>&lt;p&gt;I am very happy to report that the
&lt;a href=&quot;https://github.com/hiOA-ABI/nikita-noark5-core&quot;&gt;Nikita Noark 5
core project&lt;/a&gt; tagged its second release today.  The free software
solution is an implementation of the Norwegian archive standard Noark
5 used by government offices in Norway.  These were the changes in
version 0.1.1 since version 0.1.0 (from NEWS.md):

&lt;ul&gt;

 &lt;li&gt;Continued work on the angularjs GUI, including document upload.&lt;/li&gt;
 &lt;li&gt;Implemented correspondencepartPerson, correspondencepartUnit and
   correspondencepartInternal&lt;/li&gt;
 &lt;li&gt;Applied for coverity coverage and started submitting code on
   regualr basis.&lt;/li&gt;
 &lt;li&gt;Started fixing bugs reported by coverity&lt;/li&gt;
 &lt;li&gt;Corrected and completed HATEOAS links to make sure entire API is
   available via URLs in _links.&lt;/li&gt;
 &lt;li&gt;Corrected all relation URLs to use trailing slash.&lt;/li&gt;
 &lt;li&gt;Add initial support for storing data in ElasticSearch.&lt;/li&gt;
 &lt;li&gt;Now able to receive and store uploaded files in the archive.&lt;/li&gt;
 &lt;li&gt;Changed JSON output for object lists to have relations in _links.&lt;/li&gt;
 &lt;li&gt;Improve JSON output for empty object lists.&lt;/li&gt;
 &lt;li&gt;Now uses correct MIME type application/vnd.noark5-v4+json.&lt;/li&gt;
 &lt;li&gt;Added support for docker container images.&lt;/li&gt;
 &lt;li&gt;Added simple API browser implemented in JavaScript/Angular.&lt;/li&gt;
 &lt;li&gt;Started on archive client implemented in JavaScript/Angular.&lt;/li&gt;
 &lt;li&gt;Started on prototype to show the public mail journal.&lt;/li&gt;
 &lt;li&gt;Improved performance by disabling Sprint FileWatcher.&lt;/li&gt;
 &lt;li&gt;Added support for &#39;arkivskaper&#39;, &#39;saksmappe&#39; and &#39;journalpost&#39;.&lt;/li&gt;
 &lt;li&gt;Added support for some metadata codelists.&lt;/li&gt;
 &lt;li&gt;Added support for Cross-origin resource sharing (CORS).&lt;/li&gt;
 &lt;li&gt;Changed login method from Basic Auth to JSON Web Token (RFC 7519)
   style.&lt;/li&gt;
 &lt;li&gt;Added support for GET-ing ny-* URLs.&lt;/li&gt;
 &lt;li&gt;Added support for modifying entities using PUT and eTag.&lt;/li&gt;
 &lt;li&gt;Added support for returning XML output on request.&lt;/li&gt;
 &lt;li&gt;Removed support for English field and class names, limiting ourself
   to the official names.&lt;/li&gt;
 &lt;li&gt;...&lt;/li&gt;
 
&lt;/ul&gt;

&lt;p&gt;If this sound interesting to you, please contact us on IRC (#nikita
on irc.freenode.net) or email
(&lt;a href=&quot;https://lists.nuug.no/mailman/listinfo/nikita-noark&quot;&gt;nikita-noark
mailing list).&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>Idea for storing trusted timestamps in a Noark 5 archive</title>
                <link>http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html</link>
                <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html</guid>
                <pubDate>Wed, 7 Jun 2017 21:40:00 +0200</pubDate>
                <description>&lt;p&gt;&lt;em&gt;This is a copy of
&lt;a href=&quot;https://lists.nuug.no/pipermail/nikita-noark/2017-June/000297.html&quot;&gt;an
email I posted to the nikita-noark mailing list&lt;/a&gt;.  Please follow up
there if you would like to discuss this topic.  The background is that
we are making a free software archive system based on the Norwegian
&lt;a href=&quot;https://www.arkivverket.no/forvaltning-og-utvikling/regelverk-og-standarder/noark-standarden&quot;&gt;Noark
5 standard&lt;/a&gt; for government archives.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;I&#39;ve been wondering a bit lately how trusted timestamps could be
stored in Noark 5.
&lt;a href=&quot;https://en.wikipedia.org/wiki/Trusted_timestamping&quot;&gt;Trusted
timestamps&lt;/a&gt; can be used to verify that some information
(document/file/checksum/metadata) have not been changed since a
specific time in the past.  This is useful to verify the integrity of
the documents in the archive.&lt;/p&gt;

&lt;p&gt;Then it occured to me, perhaps the trusted timestamps could be
stored as dokument variants (ie dokumentobjekt referered to from
dokumentbeskrivelse) with the filename set to the hash it is
stamping?&lt;/p&gt;

&lt;p&gt;Given a &quot;dokumentbeskrivelse&quot; with an associated &quot;dokumentobjekt&quot;,
a new dokumentobjekt is associated with &quot;dokumentbeskrivelse&quot; with the
same attributes as the stamped dokumentobjekt except these
attributes:&lt;/p&gt;

&lt;ul&gt;

&lt;li&gt;format -&gt; &quot;RFC3161&quot;
&lt;li&gt;mimeType -&gt; &quot;application/timestamp-reply&quot;
&lt;li&gt;formatDetaljer -&gt; &quot;&amp;lt;source URL for timestamp service&amp;gt;&quot;
&lt;li&gt;filenavn -&gt; &quot;&amp;lt;sjekksum&amp;gt;.tsr&quot;

&lt;/ul&gt;

&lt;p&gt;This assume a service following
&lt;a href=&quot;https://tools.ietf.org/html/rfc3161&quot;&gt;IETF RFC 3161&lt;/a&gt; is
used, which specifiy the given MIME type for replies and the .tsr file
ending for the content of such trusted timestamp.  As far as I can
tell from the Noark 5 specifications, it is OK to have several
variants/renderings of a dokument attached to a given
dokumentbeskrivelse objekt.  It might be stretching it a bit to make
some of these variants represent crypto-signatures useful for
verifying the document integrity instead of representing the dokument
itself.&lt;/p&gt;

&lt;p&gt;Using the source of the service in formatDetaljer allow several
timestamping services to be used.  This is useful to spread the risk
of key compromise over several organisations.  It would only be a
problem to trust the timestamps if all of the organisations are
compromised.&lt;/p&gt;

&lt;p&gt;The following oneliner on Linux can be used to generate the tsr
file.  $input is the path to the file to checksum, and $sha256 is the
SHA-256 checksum of the file (ie the &quot;&lt;sjekksum&gt;.tsr&quot; value mentioned
above).&lt;/p&gt;

&lt;p&gt;&lt;blockquote&gt;&lt;pre&gt;
openssl ts -query -data &quot;$inputfile&quot; -cert -sha256 -no_nonce \
  | curl -s -H &quot;Content-Type: application/timestamp-query&quot; \
      --data-binary &quot;@-&quot; http://zeitstempel.dfn.de &gt; $sha256.tsr
&lt;/pre&gt;&lt;/blockquote&gt;&lt;/p&gt;

&lt;p&gt;To verify the timestamp, you first need to download the public key
of the trusted timestamp service, for example using this command:&lt;/p&gt;

&lt;p&gt;&lt;blockquote&gt;&lt;pre&gt;
wget -O ca-cert.txt \
  https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt
&lt;/pre&gt;&lt;/blockquote&gt;&lt;/p&gt;

&lt;p&gt;Note, the public key should be stored alongside the timestamps in
the archive to make sure it is also available 100 years from now.  It
is probably a good idea to standardise how and were to store such
public keys, to make it easier to find for those trying to verify
documents 100 or 1000 years from now. :)&lt;/p&gt;

&lt;p&gt;The verification itself is a simple openssl command:&lt;/p&gt;

&lt;p&gt;&lt;blockquote&gt;&lt;pre&gt;
openssl ts -verify -data $inputfile -in $sha256.tsr \
  -CAfile ca-cert.txt -text
&lt;/pre&gt;&lt;/blockquote&gt;&lt;/p&gt;

&lt;p&gt;Is there any reason this approach would not work?  Is it somehow against
the Noark 5 specification?&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>Når nynorskoversettelsen svikter til eksamen...</title>
                <link>http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html</link>
                <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html</guid>
                <pubDate>Sat, 3 Jun 2017 08:20:00 +0200</pubDate>
                <description>&lt;p&gt;&lt;a href=&quot;http://www.aftenposten.no/norge/Krever-at-elever-ma-fa-annullert-eksamen-etter-rot-med-oppgavetekster-622459b.html&quot;&gt;Aftenposten
melder i dag&lt;/a&gt; om feil i eksamensoppgavene for eksamen i politikk og
menneskerettigheter, der teksten i bokmåls og nynorskutgaven ikke var
like.  Oppgaveteksten er gjengitt i artikkelen, og jeg ble nysgjerring
på om den fri oversetterløsningen
&lt;a href=&quot;https://www.apertium.org/&quot;&gt;Apertium&lt;/a&gt; ville gjort en bedre
jobb enn Utdanningsdirektoratet.  Det kan se slik ut.&lt;/p&gt;

&lt;p&gt;Her er bokmålsoppgaven fra eksamenen:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Drøft utfordringene knyttet til nasjonalstatenes og andre aktørers
rolle og muligheter til å håndtere internasjonale utfordringer, som
for eksempel flykningekrisen.&lt;/p&gt;

&lt;p&gt;Vedlegge er eksempler på tekster som kan gi relevante perspektiver
på temaet:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Flykningeregnskapet 2016, UNHCR og IDMC
&lt;li&gt;«Grenseløst Europa for fall» A-Magasinet, 26. november 2015
&lt;/ol&gt;

&lt;/blockquote&gt;

&lt;p&gt;Dette oversetter Apertium slik:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Drøft utfordringane knytte til nasjonalstatane sine og rolla til
andre aktørar og høve til å handtera internasjonale utfordringar, som
til dømes *flykningekrisen.&lt;/p&gt;

&lt;p&gt;Vedleggja er døme på tekster som kan gje relevante perspektiv på
temaet:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;*Flykningeregnskapet 2016, *UNHCR og *IDMC&lt;/li&gt;
&lt;li&gt;«*Grenseløst Europa for fall» A-Magasinet, 26. november 2015&lt;/li&gt;
&lt;/ol&gt;

&lt;/blockquote&gt;

&lt;p&gt;Ord som ikke ble forstått er markert med stjerne (*), og trenger
ekstra språksjekk.  Men ingen ord er forsvunnet, slik det var i
oppgaven elevene fikk presentert på eksamen.  Jeg mistenker dog at
&quot;andre aktørers rolle og muligheter til ...&quot; burde vært oversatt til
&quot;rolla til andre aktørar og deira høve til ...&quot; eller noe slikt, men
det er kanskje flisespikking.  Det understreker vel bare at det alltid
trengs korrekturlesning etter automatisk oversettelse.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>Epost inn som arkivformat i Riksarkivarens forskrift?</title>
                <link>http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html</link>
                <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html</guid>
                <pubDate>Thu, 27 Apr 2017 11:30:00 +0200</pubDate>
                <description>&lt;p&gt;I disse dager, med frist 1. mai, har Riksarkivaren ute en høring på
sin forskrift.  Som en kan se er det ikke mye tid igjen før fristen
som går ut på søndag.  Denne forskriften er det som lister opp hvilke
formater det er greit å arkivere i
&lt;a href=&quot;http://www.arkivverket.no/arkivverket/Offentleg-forvalting/Noark/Noark-5&quot;&gt;Noark
5-løsninger&lt;/a&gt; i Norge.&lt;/p&gt;

&lt;p&gt;Jeg fant høringsdokumentene hos
&lt;a href=&quot;https://www.arkivrad.no/aktuelt/riksarkivarens-forskrift-pa-horing&quot;&gt;Norsk
Arkivråd&lt;/a&gt; etter å ha blitt tipset på epostlisten til
&lt;a href=&quot;https://github.com/hiOA-ABI/nikita-noark5-core&quot;&gt;fri
programvareprosjektet Nikita Noark5-Core&lt;/a&gt;, som lager et Noark 5
Tjenestegresesnitt.  Jeg er involvert i Nikita-prosjektet og takket
være min interesse for tjenestegrensesnittsprosjektet har jeg lest en
god del Noark 5-relaterte dokumenter, og til min overraskelse oppdaget
at standard epost ikke er på listen over godkjente formater som kan
arkiveres.  Høringen med frist søndag er en glimrende mulighet til å
forsøke å gjøre noe med det.  Jeg holder på med
&lt;a href=&quot;https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/hoering-arkivforskrift.tex&quot;&gt;egen
høringsuttalelse&lt;/a&gt;, og lurer på om andre er interessert i å støtte
forslaget om å tillate arkivering av epost som epost i arkivet.&lt;/p&gt;

&lt;p&gt;Er du igang med å skrive egen høringsuttalelse allerede?  I så fall
kan du jo vurdere å ta med en formulering om epost-lagring. Jeg tror
ikke det trengs så mye.  Her et kort forslag til tekst:&lt;/p&gt;

&lt;p&gt;&lt;blockquote&gt;

  &lt;p&gt;Viser til høring sendt ut 2017-02-17 (Riksarkivarens referanse
  2016/9840 HELHJO), og tillater oss å sende inn noen innspill om
  revisjon av Forskrift om utfyllende tekniske og arkivfaglige
  bestemmelser om behandling av offentlige arkiver (Riksarkivarens
  forskrift).&lt;/p&gt;

  &lt;p&gt;Svært mye av vår kommuikasjon foregår i dag på e-post.  Vi
  foreslår derfor at Internett-e-post, slik det er beskrevet i IETF
  RFC 5322,
  &lt;a href=&quot;https://tools.ietf.org/html/rfc5322&quot;&gt;https://tools.ietf.org/html/rfc5322&lt;/a&gt;. bør
  inn som godkjent dokumentformat.  Vi foreslår at forskriftens
  oversikt over godkjente dokumentformater ved innlevering i § 5-16
  endres til å ta med Internett-e-post.&lt;/p&gt;

&lt;/blockquote&gt;&lt;/p&gt;

&lt;p&gt;Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan
epost kan lagres i en Noark 5-struktur, og holder på å skrive et
forslag om hvordan dette kan gjøres som vil bli sendt over til
arkivverket så snart det er ferdig.  De som er interesserte kan
&lt;a href=&quot;https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/epostlagring.md&quot;&gt;følge
fremdriften på web&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Oppdatering 2017-04-28: I dag ble høringuttalelsen jeg skrev
  &lt;a href=&quot;https://www.nuug.no/news/NUUGs_h_ringuttalelse_til_Riksarkivarens_forskrift.shtml&quot;&gt;sendt
  inn av foreningen NUUG&lt;/a&gt;.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>Offentlig elektronisk postjournal blokkerer tilgang for utvalgte webklienter</title>
                <link>http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html</link>
                <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html</guid>
                <pubDate>Thu, 20 Apr 2017 13:00:00 +0200</pubDate>
                <description>&lt;p&gt;Jeg oppdaget i dag at &lt;a href=&quot;https://www.oep.no/&quot;&gt;nettstedet som
publiserer offentlige postjournaler fra statlige etater&lt;/a&gt;, OEP, har
begynt å blokkerer enkelte typer webklienter fra å få tilgang.  Vet
ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl
og curl.  For å teste selv, kjør følgende:&lt;/p&gt;

&lt;blockquote&gt;&lt;pre&gt;
% curl -v -s https://www.oep.no/pub/report.xhtml?reportId=3 2&gt;&amp;1 |grep &#39;&lt; HTTP&#39;
&lt; HTTP/1.1 404 Not Found
% curl -v -s --header &#39;User-Agent:Opera/12.0&#39; https://www.oep.no/pub/report.xhtml?reportId=3 2&gt;&amp;1 |grep &#39;&lt; HTTP&#39;
&lt; HTTP/1.1 200 OK
%
&lt;/pre&gt;&lt;/blockquote&gt;

&lt;p&gt;Her kan en se at tjenesten gir «404 Not Found» for curl i
standardoppsettet, mens den gir «200 OK» hvis curl hevder å være Opera
versjon 12.0.  Offentlig elektronisk postjournal startet blokkeringen
2017-03-02.&lt;/p&gt;

&lt;p&gt;Blokkeringen vil gjøre det litt vanskeligere å maskinelt hente
informasjon fra oep.no.  Kan blokkeringen være gjort for å hindre
automatisert innsamling av informasjon fra OEP, slik Pressens
Offentlighetsutvalg gjorde for å dokumentere hvordan departementene
hindrer innsyn i
&lt;a href=&quot;http://presse.no/dette-mener-np/undergraver-offentlighetsloven/&quot;&gt;rapporten
«Slik hindrer departementer innsyn» som ble publiserte i januar
2017&lt;/a&gt;.  Det virker usannsynlig, da det jo er trivielt å bytte
User-Agent til noe nytt.&lt;/p&gt;

&lt;p&gt;Finnes det juridisk grunnlag for det offentlige å diskriminere
webklienter slik det gjøres her?  Der tilgang gis eller ikke alt etter
hva klienten sier at den heter?  Da OEP eies av DIFI og driftes av
Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to
aktørene man kan be om innsyn i for å forstå hva som har skjedd.  Men
&lt;a href=&quot;https://www.oep.no/search/result.html?period=dateRange&amp;fromDate=01.01.2016&amp;toDate=01.04.2017&amp;dateType=documentDate&amp;caseDescription=&amp;descType=both&amp;caseNumber=&amp;documentNumber=&amp;sender=basefarm&amp;senderType=both&amp;documentType=all&amp;legalAuthority=&amp;archiveCode=&amp;list2=196&amp;searchType=advanced&amp;Search=Search+in+records&quot;&gt;postjournalen
til DIFI viser kun to dokumenter&lt;/a&gt; det siste året mellom DIFI og
Basefarm.
&lt;a href=&quot;https://www.mimesbronn.no/request/blokkering_av_tilgang_til_oep_fo&quot;&gt;Mimes brønn neste&lt;/a&gt;,
tenker jeg.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>Free software archive system Nikita now able to store documents</title>
                <link>http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html</link>
                <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html</guid>
                <pubDate>Sun, 19 Mar 2017 08:00:00 +0100</pubDate>
                <description>&lt;p&gt;The &lt;a href=&quot;https://github.com/hiOA-ABI/nikita-noark5-core&quot;&gt;Nikita
Noark 5 core project&lt;/a&gt; is implementing the Norwegian standard for
keeping an electronic archive of government documents.
&lt;a href=&quot;http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version&quot;&gt;The
Noark 5 standard&lt;/a&gt; document the requirement for data systems used by
the archives in the Norwegian government, and the Noark 5 web interface
specification document a REST web service for storing, searching and
retrieving documents and metadata in such archive.  I&#39;ve been involved
in the project since a few weeks before Christmas, when the Norwegian
Unix User Group
&lt;a href=&quot;https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml&quot;&gt;announced
it supported the project&lt;/a&gt;.  I believe this is an important project,
and hope it can make it possible for the government archives in the
future to use free software to keep the archives we citizens depend
on.  But as I do not hold such archive myself, personally my first use
case is to store and analyse public mail journal metadata published
from the government.  I find it useful to have a clear use case in
mind when developing, to make sure the system scratches one of my
itches.&lt;/p&gt;

&lt;p&gt;If you would like to help make sure there is a free software
alternatives for the archives, please join our IRC channel
(&lt;a href=&quot;irc://irc.freenode.net/%23nikita&quot;&quot;&gt;#nikita on
irc.freenode.net&lt;/a&gt;) and
&lt;a href=&quot;https://lists.nuug.no/mailman/listinfo/nikita-noark&quot;&gt;the
project mailing list&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;When I got involved, the web service could store metadata about
documents.  But a few weeks ago, a new milestone was reached when it
became possible to store full text documents too.  Yesterday, I
completed an implementation of a command line tool
&lt;tt&gt;archive-pdf&lt;/tt&gt; to upload a PDF file to the archive using this
API.  The tool is very simple at the moment, and find existing
&lt;a href=&quot;https://en.wikipedia.org/wiki/Fonds&quot;&gt;fonds&lt;/a&gt;, series and
files while asking the user to select which one to use if more than
one exist.  Once a file is identified, the PDF is associated with the
file and uploaded, using the title extracted from the PDF itself.  The
process is fairly similar to visiting the archive, opening a cabinet,
locating a file and storing a piece of paper in the archive.  Here is
a test run directly after populating the database with test data using
our API tester:&lt;/p&gt;

&lt;p&gt;&lt;blockquote&gt;&lt;pre&gt;
~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446

 0 - Title of the test case file created 2017-03-18T23:49:32.103446
 1 - Title of the test file created 2017-03-18T23:49:32.103446
Select which mappe you want (or search term): 0
Uploading mangelmelding/mangler.pdf
  PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
  File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
~/src//noark5-tester$
&lt;/pre&gt;&lt;/blockquote&gt;&lt;/p&gt;

&lt;p&gt;You can see here how the fonds (arkiv) and serie (arkivdel) only had
one option, while the user need to choose which file (mappe) to use
among the two created by the API tester.  The &lt;tt&gt;archive-pdf&lt;/tt&gt;
tool can be found in the git repository for the API tester.&lt;/p&gt;

&lt;p&gt;In the project, I have been mostly working on
&lt;a href=&quot;https://github.com/petterreinholdtsen/noark5-tester&quot;&gt;the API
tester&lt;/a&gt; so far, while getting to know the code base.  The API
tester currently use
&lt;a href=&quot;https://en.wikipedia.org/wiki/HATEOAS&quot;&gt;the HATEOAS links&lt;/a&gt;
to traverse the entire exposed service API and verify that the exposed
operations and objects match the specification, as well as trying to
create objects holding metadata and uploading a simple XML file to
store.  The tester has proved very useful for finding flaws in our
implementation, as well as flaws in the reference site and the
specification.&lt;/p&gt;

&lt;p&gt;The test document I uploaded is a summary of all the specification
defects we have collected so far while implementing the web service.
There are several unclear and conflicting parts of the specification,
and we have
&lt;a href=&quot;https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding&quot;&gt;started
writing down&lt;/a&gt; the questions we get from implementing it.  We use a
format inspired by how &lt;a href=&quot;http://www.opengroup.org/austin/&quot;&gt;The
Austin Group&lt;/a&gt; collect defect reports for the POSIX standard with
&lt;a href=&quot;http://www.opengroup.org/austin/mantis.html&quot;&gt;their
instructions for the MANTIS defect tracker system&lt;/a&gt;, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a &lt;a href=&quot;https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md&quot;&gt;request for a procedure for submitting defect reports&lt;/a&gt; :).

&lt;p&gt;The Nikita project is implemented using Java and Spring, and is
fairly easy to get up and running using Docker containers for those
that want to test the current code base.  The API tester is
implemented in Python.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>Detecting NFS hangs on Linux without hanging yourself...</title>
                <link>http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html</link>
                <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html</guid>
                <pubDate>Thu, 9 Mar 2017 15:20:00 +0100</pubDate>
                <description>&lt;p&gt;Over the years, administrating thousand of NFS mounting linux
computers at the time, I often needed a way to detect if the machine
was experiencing NFS hang.  If you try to use &lt;tt&gt;df&lt;/tt&gt; or look at a
file or directory affected by the hang, the process (and possibly the
shell) will hang too.  So you want to be able to detect this without
risking the detection process getting stuck too.  It has not been
obvious how to do this.  When the hang has lasted a while, it is
possible to find messages like these in dmesg:&lt;/p&gt;

&lt;p&gt;&lt;blockquote&gt;
nfs: server nfsserver not responding, still trying
&lt;br&gt;nfs: server nfsserver OK
&lt;/blockquote&gt;&lt;/p&gt;
  
&lt;p&gt;It is hard to know if the hang is still going on, and it is hard to
be sure looking in dmesg is going to work.  If there are lots of other
messages in dmesg the lines might have rotated out of site before they
are noticed.&lt;/p&gt;

&lt;p&gt;While reading through the nfs client implementation in linux kernel
code, I came across some statistics that seem to give a way to detect
it.  The om_timeouts sunrpc value in the kernel will increase every
time the above log entry is inserted into dmesg.  And after digging a
bit further, I discovered that this value show up in
/proc/self/mountstats on Linux.&lt;/p&gt;

&lt;p&gt;The mountstats content seem to be shared between files using the
same file system context, so it is enough to check one of the
mountstats files to get the state of the mount point for the machine.
I assume this will not show lazy umounted NFS points, nor NFS mount
points in a different process context (ie with a different filesystem
view), but that does not worry me.&lt;/p&gt;

&lt;p&gt;The content for a NFS mount point look similar to this:&lt;/p&gt;

&lt;p&gt;&lt;blockquote&gt;&lt;pre&gt;
[...]
device /dev/mapper/Debian-var mounted on /var with fstype ext3
device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1
        opts:   rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all
        age:    7863311
        caps:   caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255
        sec:    flavor=1,pseudoflavor=1
        events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0 
        bytes:  166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809 
        RPC iostats version: 1.0  p/v: 100003/3 (nfs)
        xprt:   tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820
        per-op statistics
                NULL: 0 0 0 0 0 0 0 0
             GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132
             SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943
              LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511
              ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140
            READLINK: 125 125 0 20472 18620 0 1112 1118
                READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693
               WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771
              CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398
               MKDIR: 3680 3680 0 773980 993920 26 23990 24245
             SYMLINK: 903 903 0 233428 245488 6 5865 5917
               MKNOD: 80 80 0 20148 21760 0 299 304
              REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636
               RMDIR: 3367 3367 0 645112 484848 22 5782 6002
              RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288
                LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579
             READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917
         READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937
              FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022
              FSINFO: 2 2 0 232 328 0 1 1
            PATHCONF: 1 1 0 116 140 0 0 0
              COMMIT: 0 0 0 0 0 0 0 0

device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
[...]
&lt;/pre&gt;&lt;/blockquote&gt;&lt;/p&gt;

&lt;p&gt;The key number to look at is the third number in the per-op list.
It is the number of NFS timeouts experiences per file system
operation.  Here 22 write timeouts and 5 access timeouts.  If these
numbers are increasing, I believe the machine is experiencing NFS
hang.  Unfortunately the timeout value do not start to increase right
away.  The NFS operations need to time out first, and this can take a
while.  The exact timeout value depend on the setup.  For example the
defaults for TCP and UDP mount points are quite different, and the
timeout value is affected by the soft, hard, timeo and retrans NFS
mount options.&lt;/p&gt;

&lt;p&gt;The only way I have been able to get working on Debian and RedHat
Enterprise Linux for getting the timeout count is to peek in /proc/.
But according to
&lt;ahref=&quot;http://docs.oracle.com/cd/E19253-01/816-4555/netmonitor-12/index.html&quot;&gt;Solaris
10 System Administration Guide: Network Services&lt;/a&gt;, the &#39;nfsstat -c&#39;
command can be used to get these timeout values.  But this do not work
on Linux, as far as I can tell.  I
&lt;ahref=&quot;http://bugs.debian.org/857043&quot;&gt;asked Debian about this&lt;/a&gt;,
but have not seen any replies yet.&lt;/p&gt;

&lt;p&gt;Is there a better way to figure out if a Linux NFS client is
experiencing NFS hangs?  Is there a way to detect which processes are
affected?  Is there a way to get the NFS mount going quickly once the
network problem causing the NFS hang has been cleared?  I would very
much welcome some clues, as we regularly run into NFS hangs.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>How does it feel to be wiretapped, when you should be doing the wiretapping...</title>
                <link>http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html</link>
                <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html</guid>
                <pubDate>Wed, 8 Mar 2017 11:50:00 +0100</pubDate>
                <description>&lt;p&gt;So the new president in the United States of America claim to be
surprised to discover that he was wiretapped during the election
before he was elected president.  He even claim this must be illegal.
Well, doh, if it is one thing the confirmations from Snowden
documented, it is that the entire population in USA is wiretapped, one
way or another.  Of course the president candidates were wiretapped,
alongside the senators, judges and the rest of the people in USA.&lt;/p&gt;

&lt;p&gt;Next, the Federal Bureau of Investigation ask the Department of
Justice to go public rejecting the claims that Donald Trump was
wiretapped illegally.  I fail to see the relevance, given that I am
sure the surveillance industry in USA believe they have all the legal
backing they need to conduct mass surveillance on the entire
world.&lt;/p&gt;

&lt;p&gt;There is even the director of the FBI stating that he never saw an
order requesting wiretapping of Donald Trump.  That is not very
surprising, given how the FISA court work, with all its activity being
secret.  Perhaps he only heard about it?&lt;/p&gt;

&lt;p&gt;What I find most sad in this story is how Norwegian journalists
present it.  In a news reports the other day in the radio from the
Norwegian National broadcasting Company (NRK), I heard the journalist
claim that &#39;the FBI denies any wiretapping&#39;, while the reality is that
&#39;the FBI denies any illegal wiretapping&#39;.  There is a fundamental and
important difference, and it make me sad that the journalists are
unable to grasp it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Update 2017-03-13:&lt;/strong&gt; Look like
&lt;a href=&quot;https://theintercept.com/2017/03/13/rand-paul-is-right-nsa-routinely-monitors-americans-communications-without-warrants/&quot;&gt;The
Intercept report that US Senator Rand Paul confirm what I state above&lt;/a&gt;.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>Norwegian Bokmål translation of The Debian Administrator&#39;s Handbook complete, proofreading in progress</title>
                <link>http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html</link>
                <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html</guid>
                <pubDate>Fri, 3 Mar 2017 14:50:00 +0100</pubDate>
                <description>&lt;p&gt;For almost a year now, we have been working on making a Norwegian
Bokmål edition of &lt;a href=&quot;https://debian-handbook.info/&quot;&gt;The Debian
Administrator&#39;s Handbook&lt;/a&gt;.  Now, thanks to the tireless effort of
Ole-Erik, Ingrid and Andreas, the initial translation is complete, and
we are working on the proof reading to ensure consistent language and
use of correct computer science terms.  The plan is to make the book
available on paper, as well as in electronic form.  For that to
happen, the proof reading must be completed and all the figures need
to be translated.  If you want to help out, get in touch.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://people.skolelinux.org/pere/debian-handbook/debian-handbook-nb-NO.pdf&quot;&gt;A

fresh PDF edition&lt;/a&gt; in A4 format (the final book will have smaller
pages) of the book created every morning is available for
proofreading.  If you find any errors, please
&lt;a href=&quot;https://hosted.weblate.org/projects/debian-handbook/&quot;&gt;visit
Weblate and correct the error&lt;/a&gt;.  The
&lt;a href=&quot;http://l.github.io/debian-handbook/stat/nb-NO/index.html&quot;&gt;state
of the translation including figures&lt;/a&gt; is a useful source for those
provide Norwegian bokmål screen shots and figures.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>Unlimited randomness with the ChaosKey?</title>
                <link>http://people.skolelinux.org/pere/blog/Unlimited_randomness_with_the_ChaosKey_.html</link>
                <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Unlimited_randomness_with_the_ChaosKey_.html</guid>
                <pubDate>Wed, 1 Mar 2017 20:50:00 +0100</pubDate>
                <description>&lt;p&gt;A few days ago I ordered a small batch of
&lt;a href=&quot;http://altusmetrum.org/ChaosKey/&quot;&gt;the ChaosKey&lt;/a&gt;, a small
USB dongle for generating entropy created by Bdale Garbee and Keith
Packard.  Yesterday it arrived, and I am very happy to report that it
work great!  According to its designers, to get it to work out of the
box, you need the Linux kernel version 4.1 or later.  I tested on a
Debian Stretch machine (kernel version 4.9), and there it worked just
fine, increasing the available entropy very quickly.  I wrote a small
test oneliner to test.  It first print the current entropy level,
drain /dev/random, and then print the entropy level for five seconds.
Here is the situation without the ChaosKey inserted:&lt;/p&gt;

&lt;blockquote&gt;&lt;pre&gt;
% cat /proc/sys/kernel/random/entropy_avail; \
  dd bs=1M if=/dev/random of=/dev/null count=1; \
  for n in $(seq 1 5); do \
     cat /proc/sys/kernel/random/entropy_avail; \
     sleep 1; \
  done
300
0+1 oppføringer inn
0+1 oppføringer ut
28 byte kopiert, 0,000264565 s, 106 kB/s
4
8
12
17
21
%
&lt;/pre&gt;&lt;/blockquote&gt;

&lt;p&gt;The entropy level increases by 3-4 every second.  In such case any
application requiring random bits (like a HTTPS enabled web server)
will halt and wait for more entrpy.  And here is the situation with
the ChaosKey inserted:&lt;/p&gt;

&lt;blockquote&gt;&lt;pre&gt;
% cat /proc/sys/kernel/random/entropy_avail; \
  dd bs=1M if=/dev/random of=/dev/null count=1; \
  for n in $(seq 1 5); do \
     cat /proc/sys/kernel/random/entropy_avail; \
     sleep 1; \
  done
1079
0+1 oppføringer inn
0+1 oppføringer ut
104 byte kopiert, 0,000487647 s, 213 kB/s
433
1028
1031
1035
1038
%
&lt;/pre&gt;&lt;/blockquote&gt;

&lt;p&gt;Quite the difference. :) I bought a few more than I need, in case
someone want to buy one here in Norway. :)&lt;/p&gt;

&lt;p&gt;Update: The dongle was presented at Debconf last year.  You might
find &lt;a href=&quot;https://debconf16.debconf.org/talks/94/&quot;&gt;the talk
recording illuminating&lt;/a&gt;.  It explains exactly what the source of
randomness is, if you are unable to spot it from the schema drawing
available from the ChaosKey web site linked at the start of this blog
post.&lt;/p&gt;
</description>
        </item>
        
        </channel>
</rss>