1 <?xml version=
"1.0" encoding=
"utf-8"?>
2 <rss version='
2.0' xmlns:lj='http://www.livejournal.org/rss/lj/
1.0/' xmlns:
atom=
"http://www.w3.org/2005/Atom">
4 <title>Petter Reinholdtsen
</title>
5 <description></description>
6 <link>http://people.skolelinux.org/pere/blog/
</link>
7 <atom:link href=
"http://people.skolelinux.org/pere/blog/index.rss" rel=
"self" type=
"application/rss+xml" />
10 <title>Updated sales number for my Free Culture paper editions
</title>
11 <link>http://people.skolelinux.org/pere/blog/Updated_sales_number_for_my_Free_Culture_paper_editions.html
</link>
12 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Updated_sales_number_for_my_Free_Culture_paper_editions.html
</guid>
13 <pubDate>Mon,
12 Jun
2017 11:
40:
00 +
0200</pubDate>
14 <description><p
>It is pleasing to see that the work we put down in publishing new
15 editions of the classic
<a href=
"http://www.free-culture.cc/
">Free
16 Culture book
</a
> by the founder of the Creative Commons movement,
17 Lawrence Lessig, is still being appreciated. I had a look at the
18 latest sales numbers for the paper edition today. Not too impressive,
19 but some buyers still exist. All the revenue from the books are sent
20 to the
<a href=
"https://creativecommons.org/
">Creative Commons
21 Corporation
</a
>, and they receive the largest cut if you buy directly
22 from Lulu. Most books are sold via Amazon, with Ingram second and
23 only a small fraction directly from Lulu. The ebook edition is
24 available for free from
25 <a href=
"https://github.com/petterreinholdtsen/free-culture-lessig
">Github
</a
>.
</p
>
27 <table border=
"0">
28 <tr
><th rowspan=
"2" valign=
"bottom
">Title / language
</th
><th colspan=
"3">Quantity
</th
></tr
>
29 <tr
><th
>2016 jan-jun
</th
><th
>2016 jul-dec
</th
><th
>2017 jan-may
</th
></tr
>
32 <td
><a href=
"http://www.lulu.com/shop/lawrence-lessig/culture-libre/paperback/product-
22645082.html
">Culture Libre / French
</a
></td
>
33 <td align=
"right
">3</td
>
34 <td align=
"right
">6</td
>
35 <td align=
"right
">15</td
>
39 <td
><a href=
"http://www.lulu.com/shop/lawrence-lessig/fri-kultur/paperback/product-
22441576.html
">Fri kultur / Norwegian
</a
></td
>
40 <td align=
"right
">7</td
>
41 <td align=
"right
">1</td
>
42 <td align=
"right
">0</td
>
46 <td
><a href=
"http://www.lulu.com/shop/lawrence-lessig/free-culture/paperback/product-
22440520.html
">Free Culture / English
</a
></td
>
47 <td align=
"right
">14</td
>
48 <td align=
"right
">27</td
>
49 <td align=
"right
">16</td
>
53 <td
>Total
</td
>
54 <td align=
"right
">24</td
>
55 <td align=
"right
">34</td
>
56 <td align=
"right
">31</td
>
61 <p
>A bit sad to see the low sales number on the Norwegian edition, and
62 a bit surprising the English edition still selling so well.
</p
>
64 <p
>If you would like to translate and publish the book in your native
65 language, I would be happy to help make it happen. Please get in
71 <title>Release
0.1.1 of free software archive system Nikita announced
</title>
72 <link>http://people.skolelinux.org/pere/blog/Release_0_1_1_of_free_software_archive_system_Nikita_announced.html
</link>
73 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Release_0_1_1_of_free_software_archive_system_Nikita_announced.html
</guid>
74 <pubDate>Sat,
10 Jun
2017 00:
40:
00 +
0200</pubDate>
75 <description><p
>I am very happy to report that the
76 <a href=
"https://github.com/hiOA-ABI/nikita-noark5-core
">Nikita Noark
5
77 core project
</a
> tagged its second release today. The free software
78 solution is an implementation of the Norwegian archive standard Noark
79 5 used by government offices in Norway. These were the changes in
80 version
0.1.1 since version
0.1.0 (from NEWS.md):
84 <li
>Continued work on the angularjs GUI, including document upload.
</li
>
85 <li
>Implemented correspondencepartPerson, correspondencepartUnit and
86 correspondencepartInternal
</li
>
87 <li
>Applied for coverity coverage and started submitting code on
88 regualr basis.
</li
>
89 <li
>Started fixing bugs reported by coverity
</li
>
90 <li
>Corrected and completed HATEOAS links to make sure entire API is
91 available via URLs in _links.
</li
>
92 <li
>Corrected all relation URLs to use trailing slash.
</li
>
93 <li
>Add initial support for storing data in ElasticSearch.
</li
>
94 <li
>Now able to receive and store uploaded files in the archive.
</li
>
95 <li
>Changed JSON output for object lists to have relations in _links.
</li
>
96 <li
>Improve JSON output for empty object lists.
</li
>
97 <li
>Now uses correct MIME type application/vnd.noark5-v4+json.
</li
>
98 <li
>Added support for docker container images.
</li
>
99 <li
>Added simple API browser implemented in JavaScript/Angular.
</li
>
100 <li
>Started on archive client implemented in JavaScript/Angular.
</li
>
101 <li
>Started on prototype to show the public mail journal.
</li
>
102 <li
>Improved performance by disabling Sprint FileWatcher.
</li
>
103 <li
>Added support for
'arkivskaper
',
'saksmappe
' and
'journalpost
'.
</li
>
104 <li
>Added support for some metadata codelists.
</li
>
105 <li
>Added support for Cross-origin resource sharing (CORS).
</li
>
106 <li
>Changed login method from Basic Auth to JSON Web Token (RFC
7519)
108 <li
>Added support for GET-ing ny-* URLs.
</li
>
109 <li
>Added support for modifying entities using PUT and eTag.
</li
>
110 <li
>Added support for returning XML output on request.
</li
>
111 <li
>Removed support for English field and class names, limiting ourself
112 to the official names.
</li
>
113 <li
>...
</li
>
117 <p
>If this sound interesting to you, please contact us on IRC (#nikita
118 on irc.freenode.net) or email
119 (
<a href=
"https://lists.nuug.no/mailman/listinfo/nikita-noark
">nikita-noark
120 mailing list).
</p
>
125 <title>Idea for storing trusted timestamps in a Noark
5 archive
</title>
126 <link>http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html
</link>
127 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html
</guid>
128 <pubDate>Wed,
7 Jun
2017 21:
40:
00 +
0200</pubDate>
129 <description><p
><em
>This is a copy of
130 <a href=
"https://lists.nuug.no/pipermail/nikita-noark/
2017-June/
000297.html
">an
131 email I posted to the nikita-noark mailing list
</a
>. Please follow up
132 there if you would like to discuss this topic. The background is that
133 we are making a free software archive system based on the Norwegian
134 <a href=
"https://www.arkivverket.no/forvaltning-og-utvikling/regelverk-og-standarder/noark-standarden
">Noark
135 5 standard
</a
> for government archives.
</em
></p
>
137 <p
>I
've been wondering a bit lately how trusted timestamps could be
139 <a href=
"https://en.wikipedia.org/wiki/Trusted_timestamping
">Trusted
140 timestamps
</a
> can be used to verify that some information
141 (document/file/checksum/metadata) have not been changed since a
142 specific time in the past. This is useful to verify the integrity of
143 the documents in the archive.
</p
>
145 <p
>Then it occured to me, perhaps the trusted timestamps could be
146 stored as dokument variants (ie dokumentobjekt referered to from
147 dokumentbeskrivelse) with the filename set to the hash it is
150 <p
>Given a
"dokumentbeskrivelse
" with an associated
"dokumentobjekt
",
151 a new dokumentobjekt is associated with
"dokumentbeskrivelse
" with the
152 same attributes as the stamped dokumentobjekt except these
153 attributes:
</p
>
157 <li
>format -
> "RFC3161
"
158 <li
>mimeType -
> "application/timestamp-reply
"
159 <li
>formatDetaljer -
> "&lt;source URL for timestamp service
&gt;
"
160 <li
>filenavn -
> "&lt;sjekksum
&gt;.tsr
"
164 <p
>This assume a service following
165 <a href=
"https://tools.ietf.org/html/rfc3161
">IETF RFC
3161</a
> is
166 used, which specifiy the given MIME type for replies and the .tsr file
167 ending for the content of such trusted timestamp. As far as I can
168 tell from the Noark
5 specifications, it is OK to have several
169 variants/renderings of a dokument attached to a given
170 dokumentbeskrivelse objekt. It might be stretching it a bit to make
171 some of these variants represent crypto-signatures useful for
172 verifying the document integrity instead of representing the dokument
175 <p
>Using the source of the service in formatDetaljer allow several
176 timestamping services to be used. This is useful to spread the risk
177 of key compromise over several organisations. It would only be a
178 problem to trust the timestamps if all of the organisations are
179 compromised.
</p
>
181 <p
>The following oneliner on Linux can be used to generate the tsr
182 file. $input is the path to the file to checksum, and $sha256 is the
183 SHA-
256 checksum of the file (ie the
"<sjekksum
>.tsr
" value mentioned
186 <p
><blockquote
><pre
>
187 openssl ts -query -data
"$inputfile
" -cert -sha256 -no_nonce \
188 | curl -s -H
"Content-Type: application/timestamp-query
" \
189 --data-binary
"@-
" http://zeitstempel.dfn.de
> $sha256.tsr
190 </pre
></blockquote
></p
>
192 <p
>To verify the timestamp, you first need to download the public key
193 of the trusted timestamp service, for example using this command:
</p
>
195 <p
><blockquote
><pre
>
196 wget -O ca-cert.txt \
197 https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt
198 </pre
></blockquote
></p
>
200 <p
>Note, the public key should be stored alongside the timestamps in
201 the archive to make sure it is also available
100 years from now. It
202 is probably a good idea to standardise how and were to store such
203 public keys, to make it easier to find for those trying to verify
204 documents
100 or
1000 years from now. :)
</p
>
206 <p
>The verification itself is a simple openssl command:
</p
>
208 <p
><blockquote
><pre
>
209 openssl ts -verify -data $inputfile -in $sha256.tsr \
210 -CAfile ca-cert.txt -text
211 </pre
></blockquote
></p
>
213 <p
>Is there any reason this approach would not work? Is it somehow against
214 the Noark
5 specification?
</p
>
219 <title>Når nynorskoversettelsen svikter til eksamen...
</title>
220 <link>http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html
</link>
221 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html
</guid>
222 <pubDate>Sat,
3 Jun
2017 08:
20:
00 +
0200</pubDate>
223 <description><p
><a href=
"http://www.aftenposten.no/norge/Krever-at-elever-ma-fa-annullert-eksamen-etter-rot-med-oppgavetekster-
622459b.html
">Aftenposten
224 melder i dag
</a
> om feil i eksamensoppgavene for eksamen i politikk og
225 menneskerettigheter, der teksten i bokmåls og nynorskutgaven ikke var
226 like. Oppgaveteksten er gjengitt i artikkelen, og jeg ble nysgjerring
227 på om den fri oversetterløsningen
228 <a href=
"https://www.apertium.org/
">Apertium
</a
> ville gjort en bedre
229 jobb enn Utdanningsdirektoratet. Det kan se slik ut.
</p
>
231 <p
>Her er bokmålsoppgaven fra eksamenen:
</p
>
234 <p
>Drøft utfordringene knyttet til nasjonalstatenes og andre aktørers
235 rolle og muligheter til å håndtere internasjonale utfordringer, som
236 for eksempel flykningekrisen.
</p
>
238 <p
>Vedlegge er eksempler på tekster som kan gi relevante perspektiver
241 <li
>Flykningeregnskapet
2016, UNHCR og IDMC
242 <li
>«Grenseløst Europa for fall» A-Magasinet,
26. november
2015
247 <p
>Dette oversetter Apertium slik:
</p
>
250 <p
>Drøft utfordringane knytte til nasjonalstatane sine og rolla til
251 andre aktørar og høve til å handtera internasjonale utfordringar, som
252 til dømes *flykningekrisen.
</p
>
254 <p
>Vedleggja er døme på tekster som kan gje relevante perspektiv på
258 <li
>*Flykningeregnskapet
2016, *UNHCR og *IDMC
</li
>
259 <li
>«*Grenseløst Europa for fall» A-Magasinet,
26. november
2015</li
>
264 <p
>Ord som ikke ble forstått er markert med stjerne (*), og trenger
265 ekstra språksjekk. Men ingen ord er forsvunnet, slik det var i
266 oppgaven elevene fikk presentert på eksamen. Jeg mistenker dog at
267 "andre aktørers rolle og muligheter til ...
" burde vært oversatt til
268 "rolla til andre aktørar og deira høve til ...
" eller noe slikt, men
269 det er kanskje flisespikking. Det understreker vel bare at det alltid
270 trengs korrekturlesning etter automatisk oversettelse.
</p
>
275 <title>Epost inn som arkivformat i Riksarkivarens forskrift?
</title>
276 <link>http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html
</link>
277 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html
</guid>
278 <pubDate>Thu,
27 Apr
2017 11:
30:
00 +
0200</pubDate>
279 <description><p
>I disse dager, med frist
1. mai, har Riksarkivaren ute en høring på
280 sin forskrift. Som en kan se er det ikke mye tid igjen før fristen
281 som går ut på søndag. Denne forskriften er det som lister opp hvilke
282 formater det er greit å arkivere i
283 <a href=
"http://www.arkivverket.no/arkivverket/Offentleg-forvalting/Noark/Noark-
5">Noark
284 5-løsninger
</a
> i Norge.
</p
>
286 <p
>Jeg fant høringsdokumentene hos
287 <a href=
"https://www.arkivrad.no/aktuelt/riksarkivarens-forskrift-pa-horing
">Norsk
288 Arkivråd
</a
> etter å ha blitt tipset på epostlisten til
289 <a href=
"https://github.com/hiOA-ABI/nikita-noark5-core
">fri
290 programvareprosjektet Nikita Noark5-Core
</a
>, som lager et Noark
5
291 Tjenestegresesnitt. Jeg er involvert i Nikita-prosjektet og takket
292 være min interesse for tjenestegrensesnittsprosjektet har jeg lest en
293 god del Noark
5-relaterte dokumenter, og til min overraskelse oppdaget
294 at standard epost ikke er på listen over godkjente formater som kan
295 arkiveres. Høringen med frist søndag er en glimrende mulighet til å
296 forsøke å gjøre noe med det. Jeg holder på med
297 <a href=
"https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/hoering-arkivforskrift.tex
">egen
298 høringsuttalelse
</a
>, og lurer på om andre er interessert i å støtte
299 forslaget om å tillate arkivering av epost som epost i arkivet.
</p
>
301 <p
>Er du igang med å skrive egen høringsuttalelse allerede? I så fall
302 kan du jo vurdere å ta med en formulering om epost-lagring. Jeg tror
303 ikke det trengs så mye. Her et kort forslag til tekst:
</p
>
305 <p
><blockquote
>
307 <p
>Viser til høring sendt ut
2017-
02-
17 (Riksarkivarens referanse
308 2016/
9840 HELHJO), og tillater oss å sende inn noen innspill om
309 revisjon av Forskrift om utfyllende tekniske og arkivfaglige
310 bestemmelser om behandling av offentlige arkiver (Riksarkivarens
311 forskrift).
</p
>
313 <p
>Svært mye av vår kommuikasjon foregår i dag på e-post. Vi
314 foreslår derfor at Internett-e-post, slik det er beskrevet i IETF
316 <a href=
"https://tools.ietf.org/html/rfc5322
">https://tools.ietf.org/html/rfc5322
</a
>. bør
317 inn som godkjent dokumentformat. Vi foreslår at forskriftens
318 oversikt over godkjente dokumentformater ved innlevering i §
5-
16
319 endres til å ta med Internett-e-post.
</p
>
321 </blockquote
></p
>
323 <p
>Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan
324 epost kan lagres i en Noark
5-struktur, og holder på å skrive et
325 forslag om hvordan dette kan gjøres som vil bli sendt over til
326 arkivverket så snart det er ferdig. De som er interesserte kan
327 <a href=
"https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/epostlagring.md
">følge
328 fremdriften på web
</a
>.
</p
>
330 <p
>Oppdatering
2017-
04-
28: I dag ble høringuttalelsen jeg skrev
331 <a href=
"https://www.nuug.no/news/NUUGs_h_ringuttalelse_til_Riksarkivarens_forskrift.shtml
">sendt
332 inn av foreningen NUUG
</a
>.
</p
>
337 <title>Offentlig elektronisk postjournal blokkerer tilgang for utvalgte webklienter
</title>
338 <link>http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html
</link>
339 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html
</guid>
340 <pubDate>Thu,
20 Apr
2017 13:
00:
00 +
0200</pubDate>
341 <description><p
>Jeg oppdaget i dag at
<a href=
"https://www.oep.no/
">nettstedet som
342 publiserer offentlige postjournaler fra statlige etater
</a
>, OEP, har
343 begynt å blokkerer enkelte typer webklienter fra å få tilgang. Vet
344 ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl
345 og curl. For å teste selv, kjør følgende:
</p
>
347 <blockquote
><pre
>
348 % curl -v -s https://www.oep.no/pub/report.xhtml?reportId=
3 2>&1 |grep
'< HTTP
'
349 < HTTP/
1.1 404 Not Found
350 % curl -v -s --header
'User-Agent:Opera/
12.0' https://www.oep.no/pub/report.xhtml?reportId=
3 2>&1 |grep
'< HTTP
'
353 </pre
></blockquote
>
355 <p
>Her kan en se at tjenesten gir «
404 Not Found» for curl i
356 standardoppsettet, mens den gir «
200 OK» hvis curl hevder å være Opera
357 versjon
12.0. Offentlig elektronisk postjournal startet blokkeringen
358 2017-
03-
02.
</p
>
360 <p
>Blokkeringen vil gjøre det litt vanskeligere å maskinelt hente
361 informasjon fra oep.no. Kan blokkeringen være gjort for å hindre
362 automatisert innsamling av informasjon fra OEP, slik Pressens
363 Offentlighetsutvalg gjorde for å dokumentere hvordan departementene
365 <a href=
"http://presse.no/dette-mener-np/undergraver-offentlighetsloven/
">rapporten
366 «Slik hindrer departementer innsyn» som ble publiserte i januar
367 2017</a
>. Det virker usannsynlig, da det jo er trivielt å bytte
368 User-Agent til noe nytt.
</p
>
370 <p
>Finnes det juridisk grunnlag for det offentlige å diskriminere
371 webklienter slik det gjøres her? Der tilgang gis eller ikke alt etter
372 hva klienten sier at den heter? Da OEP eies av DIFI og driftes av
373 Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to
374 aktørene man kan be om innsyn i for å forstå hva som har skjedd. Men
375 <a href=
"https://www.oep.no/search/result.html?period=dateRange
&fromDate=
01.01.2016&toDate=
01.04.2017&dateType=documentDate
&caseDescription=
&descType=both
&caseNumber=
&documentNumber=
&sender=basefarm
&senderType=both
&documentType=all
&legalAuthority=
&archiveCode=
&list2=
196&searchType=advanced
&Search=Search+in+records
">postjournalen
376 til DIFI viser kun to dokumenter
</a
> det siste året mellom DIFI og
378 <a href=
"https://www.mimesbronn.no/request/blokkering_av_tilgang_til_oep_fo
">Mimes brønn neste
</a
>,
379 tenker jeg.
</p
>
384 <title>Free software archive system Nikita now able to store documents
</title>
385 <link>http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html
</link>
386 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html
</guid>
387 <pubDate>Sun,
19 Mar
2017 08:
00:
00 +
0100</pubDate>
388 <description><p
>The
<a href=
"https://github.com/hiOA-ABI/nikita-noark5-core
">Nikita
389 Noark
5 core project
</a
> is implementing the Norwegian standard for
390 keeping an electronic archive of government documents.
391 <a href=
"http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-
5/English-version
">The
392 Noark
5 standard
</a
> document the requirement for data systems used by
393 the archives in the Norwegian government, and the Noark
5 web interface
394 specification document a REST web service for storing, searching and
395 retrieving documents and metadata in such archive. I
've been involved
396 in the project since a few weeks before Christmas, when the Norwegian
398 <a href=
"https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml
">announced
399 it supported the project
</a
>. I believe this is an important project,
400 and hope it can make it possible for the government archives in the
401 future to use free software to keep the archives we citizens depend
402 on. But as I do not hold such archive myself, personally my first use
403 case is to store and analyse public mail journal metadata published
404 from the government. I find it useful to have a clear use case in
405 mind when developing, to make sure the system scratches one of my
408 <p
>If you would like to help make sure there is a free software
409 alternatives for the archives, please join our IRC channel
410 (
<a href=
"irc://irc.freenode.net/%
23nikita
"">#nikita on
411 irc.freenode.net
</a
>) and
412 <a href=
"https://lists.nuug.no/mailman/listinfo/nikita-noark
">the
413 project mailing list
</a
>.
</p
>
415 <p
>When I got involved, the web service could store metadata about
416 documents. But a few weeks ago, a new milestone was reached when it
417 became possible to store full text documents too. Yesterday, I
418 completed an implementation of a command line tool
419 <tt
>archive-pdf
</tt
> to upload a PDF file to the archive using this
420 API. The tool is very simple at the moment, and find existing
421 <a href=
"https://en.wikipedia.org/wiki/Fonds
">fonds
</a
>, series and
422 files while asking the user to select which one to use if more than
423 one exist. Once a file is identified, the PDF is associated with the
424 file and uploaded, using the title extracted from the PDF itself. The
425 process is fairly similar to visiting the archive, opening a cabinet,
426 locating a file and storing a piece of paper in the archive. Here is
427 a test run directly after populating the database with test data using
428 our API tester:
</p
>
430 <p
><blockquote
><pre
>
431 ~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
432 using arkiv: Title of the test fonds created
2017-
03-
18T23:
49:
32.103446
433 using arkivdel: Title of the test series created
2017-
03-
18T23:
49:
32.103446
435 0 - Title of the test case file created
2017-
03-
18T23:
49:
32.103446
436 1 - Title of the test file created
2017-
03-
18T23:
49:
32.103446
437 Select which mappe you want (or search term):
0
438 Uploading mangelmelding/mangler.pdf
439 PDF title: Mangler i spesifikasjonsdokumentet for NOARK
5 Tjenestegrensesnitt
440 File
2017/
1: Title of the test case file created
2017-
03-
18T23:
49:
32.103446
441 ~/src//noark5-tester$
442 </pre
></blockquote
></p
>
444 <p
>You can see here how the fonds (arkiv) and serie (arkivdel) only had
445 one option, while the user need to choose which file (mappe) to use
446 among the two created by the API tester. The
<tt
>archive-pdf
</tt
>
447 tool can be found in the git repository for the API tester.
</p
>
449 <p
>In the project, I have been mostly working on
450 <a href=
"https://github.com/petterreinholdtsen/noark5-tester
">the API
451 tester
</a
> so far, while getting to know the code base. The API
453 <a href=
"https://en.wikipedia.org/wiki/HATEOAS
">the HATEOAS links
</a
>
454 to traverse the entire exposed service API and verify that the exposed
455 operations and objects match the specification, as well as trying to
456 create objects holding metadata and uploading a simple XML file to
457 store. The tester has proved very useful for finding flaws in our
458 implementation, as well as flaws in the reference site and the
459 specification.
</p
>
461 <p
>The test document I uploaded is a summary of all the specification
462 defects we have collected so far while implementing the web service.
463 There are several unclear and conflicting parts of the specification,
465 <a href=
"https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding
">started
466 writing down
</a
> the questions we get from implementing it. We use a
467 format inspired by how
<a href=
"http://www.opengroup.org/austin/
">The
468 Austin Group
</a
> collect defect reports for the POSIX standard with
469 <a href=
"http://www.opengroup.org/austin/mantis.html
">their
470 instructions for the MANTIS defect tracker system
</a
>, in lack of an official way to structure defect reports for Noark
5 (our first submitted defect report was a
<a href=
"https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/
2017-
03-
15-mangel-prosess.md
">request for a procedure for submitting defect reports
</a
> :).
472 <p
>The Nikita project is implemented using Java and Spring, and is
473 fairly easy to get up and running using Docker containers for those
474 that want to test the current code base. The API tester is
475 implemented in Python.
</p
>
480 <title>Detecting NFS hangs on Linux without hanging yourself...
</title>
481 <link>http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html
</link>
482 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html
</guid>
483 <pubDate>Thu,
9 Mar
2017 15:
20:
00 +
0100</pubDate>
484 <description><p
>Over the years, administrating thousand of NFS mounting linux
485 computers at the time, I often needed a way to detect if the machine
486 was experiencing NFS hang. If you try to use
<tt
>df
</tt
> or look at a
487 file or directory affected by the hang, the process (and possibly the
488 shell) will hang too. So you want to be able to detect this without
489 risking the detection process getting stuck too. It has not been
490 obvious how to do this. When the hang has lasted a while, it is
491 possible to find messages like these in dmesg:
</p
>
493 <p
><blockquote
>
494 nfs: server nfsserver not responding, still trying
495 <br
>nfs: server nfsserver OK
496 </blockquote
></p
>
498 <p
>It is hard to know if the hang is still going on, and it is hard to
499 be sure looking in dmesg is going to work. If there are lots of other
500 messages in dmesg the lines might have rotated out of site before they
501 are noticed.
</p
>
503 <p
>While reading through the nfs client implementation in linux kernel
504 code, I came across some statistics that seem to give a way to detect
505 it. The om_timeouts sunrpc value in the kernel will increase every
506 time the above log entry is inserted into dmesg. And after digging a
507 bit further, I discovered that this value show up in
508 /proc/self/mountstats on Linux.
</p
>
510 <p
>The mountstats content seem to be shared between files using the
511 same file system context, so it is enough to check one of the
512 mountstats files to get the state of the mount point for the machine.
513 I assume this will not show lazy umounted NFS points, nor NFS mount
514 points in a different process context (ie with a different filesystem
515 view), but that does not worry me.
</p
>
517 <p
>The content for a NFS mount point look similar to this:
</p
>
519 <p
><blockquote
><pre
>
521 device /dev/mapper/Debian-var mounted on /var with fstype ext3
522 device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=
1.1
523 opts: rw,vers=
3,rsize=
65536,wsize=
65536,namlen=
255,acregmin=
3,acregmax=
60,acdirmin=
30,acdirmax=
60,soft,nolock,proto=tcp,timeo=
600,retrans=
2,sec=sys,mountaddr=
129.240.3.145,mountvers=
3,mountport=
4048,mountproto=udp,local_lock=all
525 caps: caps=
0x3fe7,wtmult=
4096,dtsize=
8192,bsize=
0,namlen=
255
526 sec: flavor=
1,pseudoflavor=
1
527 events:
61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0
528 bytes:
166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809
529 RPC iostats version:
1.0 p/v:
100003/
3 (nfs)
530 xprt: tcp
925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820
532 NULL:
0 0 0 0 0 0 0 0
533 GETATTR:
61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132
534 SETATTR:
463469 463470 0 92005440 66739536 63787 603235 687943
535 LOOKUP:
17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511
536 ACCESS:
14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140
537 READLINK:
125 125 0 20472 18620 0 1112 1118
538 READ:
4214236 4214237 0 715608524 41328653212 89884 22622768 22806693
539 WRITE:
8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771
540 CREATE:
171708 171708 0 38084748 46702272 873 1041833 1050398
541 MKDIR:
3680 3680 0 773980 993920 26 23990 24245
542 SYMLINK:
903 903 0 233428 245488 6 5865 5917
543 MKNOD:
80 80 0 20148 21760 0 299 304
544 REMOVE:
429921 429921 0 79796004 61908192 3313 2710416 2741636
545 RMDIR:
3367 3367 0 645112 484848 22 5782 6002
546 RENAME:
466201 466201 0 130026184 121212260 7075 5935207 5961288
547 LINK:
289155 289155 0 72775556 67083960 2199 2565060 2585579
548 READDIR:
2933237 2933237 0 516506204 13973833412 10385 3190199 3297917
549 READDIRPLUS:
1652839 1652839 0 298640972 6895997744 84735 14307895 14448937
550 FSSTAT:
6144 6144 0 1010516 1032192 51 9654 10022
551 FSINFO:
2 2 0 232 328 0 1 1
552 PATHCONF:
1 1 0 116 140 0 0 0
553 COMMIT:
0 0 0 0 0 0 0 0
555 device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
557 </pre
></blockquote
></p
>
559 <p
>The key number to look at is the third number in the per-op list.
560 It is the number of NFS timeouts experiences per file system
561 operation. Here
22 write timeouts and
5 access timeouts. If these
562 numbers are increasing, I believe the machine is experiencing NFS
563 hang. Unfortunately the timeout value do not start to increase right
564 away. The NFS operations need to time out first, and this can take a
565 while. The exact timeout value depend on the setup. For example the
566 defaults for TCP and UDP mount points are quite different, and the
567 timeout value is affected by the soft, hard, timeo and retrans NFS
568 mount options.
</p
>
570 <p
>The only way I have been able to get working on Debian and RedHat
571 Enterprise Linux for getting the timeout count is to peek in /proc/.
573 <ahref=
"http://docs.oracle.com/cd/E19253-
01/
816-
4555/netmonitor-
12/index.html
">Solaris
574 10 System Administration Guide: Network Services
</a
>, the
'nfsstat -c
'
575 command can be used to get these timeout values. But this do not work
576 on Linux, as far as I can tell. I
577 <ahref=
"http://bugs.debian.org/
857043">asked Debian about this
</a
>,
578 but have not seen any replies yet.
</p
>
580 <p
>Is there a better way to figure out if a Linux NFS client is
581 experiencing NFS hangs? Is there a way to detect which processes are
582 affected? Is there a way to get the NFS mount going quickly once the
583 network problem causing the NFS hang has been cleared? I would very
584 much welcome some clues, as we regularly run into NFS hangs.
</p
>
589 <title>How does it feel to be wiretapped, when you should be doing the wiretapping...
</title>
590 <link>http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html
</link>
591 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html
</guid>
592 <pubDate>Wed,
8 Mar
2017 11:
50:
00 +
0100</pubDate>
593 <description><p
>So the new president in the United States of America claim to be
594 surprised to discover that he was wiretapped during the election
595 before he was elected president. He even claim this must be illegal.
596 Well, doh, if it is one thing the confirmations from Snowden
597 documented, it is that the entire population in USA is wiretapped, one
598 way or another. Of course the president candidates were wiretapped,
599 alongside the senators, judges and the rest of the people in USA.
</p
>
601 <p
>Next, the Federal Bureau of Investigation ask the Department of
602 Justice to go public rejecting the claims that Donald Trump was
603 wiretapped illegally. I fail to see the relevance, given that I am
604 sure the surveillance industry in USA believe they have all the legal
605 backing they need to conduct mass surveillance on the entire
608 <p
>There is even the director of the FBI stating that he never saw an
609 order requesting wiretapping of Donald Trump. That is not very
610 surprising, given how the FISA court work, with all its activity being
611 secret. Perhaps he only heard about it?
</p
>
613 <p
>What I find most sad in this story is how Norwegian journalists
614 present it. In a news reports the other day in the radio from the
615 Norwegian National broadcasting Company (NRK), I heard the journalist
616 claim that
'the FBI denies any wiretapping
', while the reality is that
617 'the FBI denies any illegal wiretapping
'. There is a fundamental and
618 important difference, and it make me sad that the journalists are
619 unable to grasp it.
</p
>
621 <p
><strong
>Update
2017-
03-
13:
</strong
> Look like
622 <a href=
"https://theintercept.com/
2017/
03/
13/rand-paul-is-right-nsa-routinely-monitors-americans-communications-without-warrants/
">The
623 Intercept report that US Senator Rand Paul confirm what I state above
</a
>.
</p
>
628 <title>Norwegian Bokmål translation of The Debian Administrator
's Handbook complete, proofreading in progress
</title>
629 <link>http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html
</link>
630 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html
</guid>
631 <pubDate>Fri,
3 Mar
2017 14:
50:
00 +
0100</pubDate>
632 <description><p
>For almost a year now, we have been working on making a Norwegian
633 Bokmål edition of
<a href=
"https://debian-handbook.info/
">The Debian
634 Administrator
's Handbook
</a
>. Now, thanks to the tireless effort of
635 Ole-Erik, Ingrid and Andreas, the initial translation is complete, and
636 we are working on the proof reading to ensure consistent language and
637 use of correct computer science terms. The plan is to make the book
638 available on paper, as well as in electronic form. For that to
639 happen, the proof reading must be completed and all the figures need
640 to be translated. If you want to help out, get in touch.
</p
>
642 <p
><a href=
"http://people.skolelinux.org/pere/debian-handbook/debian-handbook-nb-NO.pdf
">A
644 fresh PDF edition
</a
> in A4 format (the final book will have smaller
645 pages) of the book created every morning is available for
646 proofreading. If you find any errors, please
647 <a href=
"https://hosted.weblate.org/projects/debian-handbook/
">visit
648 Weblate and correct the error
</a
>. The
649 <a href=
"http://l.github.io/debian-handbook/stat/nb-NO/index.html
">state
650 of the translation including figures
</a
> is a useful source for those
651 provide Norwegian bokmål screen shots and figures.
</p
>