X-Git-Url: http://pere.pagekite.me/gitweb/homepage.git/blobdiff_plain/55372de68ea018b89d8bc929ee1a9b90e9a7cf73..aa4069048e4b39f85304deb4c9abc90695b8441c:/blog/index.rss diff --git a/blog/index.rss b/blog/index.rss index 9f17a118f1..e42e39614e 100644 --- a/blog/index.rss +++ b/blog/index.rss @@ -7,1237 +7,634 @@ - MakerCon Nordic videos now available on Frikanalen - http://people.skolelinux.org/pere/blog/MakerCon_Nordic_videos_now_available_on_Frikanalen.html - http://people.skolelinux.org/pere/blog/MakerCon_Nordic_videos_now_available_on_Frikanalen.html - Thu, 2 Jul 2015 14:10:00 +0200 - <p>Last oktober I was involved on behalf of -<a href="http://www.nuug.no/">NUUG</a> with recording the talks at -<a href="http://www.makercon.no/">MakerCon Nordic</a>, a conference for -the Maker movement. Since then it has been the plan to publish the -recordings on <a href="http://www.frikanalen.no/">Frikanalen</a>, which -finally happened the last few days. A few talks are missing because -the speakers asked the organizers to not publish them, but most of the -talks are available. The talks are being broadcasted on RiksTV -channel 50 and using multicast on Uninett, as well as being available -from the Frikanalen web site. The unedited recordings are -<a href="https://www.youtube.com/user/MakerConNordic/">available on -Youtube too</a>.</p> - -<p>This is the list of talks available at the moment. Visit the -<a href="http://beta.frikanalen.no/video/?q=makercon">Frikanalen video -pages</a> to view them.</p> + Idea for storing trusted timestamps in a Noark 5 archive + http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html + http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html + Wed, 7 Jun 2017 21:40:00 +0200 + <p><em>This is a copy of +<a href="https://lists.nuug.no/pipermail/nikita-noark/2017-June/000297.html">an +email I posted to the nikita-noark mailing list</a>. Please follow up +there if you would like to discuss this topic. The background is that +we are making a free software archive system based on the Norwegian +<a href="https://www.arkivverket.no/forvaltning-og-utvikling/regelverk-og-standarder/noark-standarden">Noark +5 standard</a> for government archives.</em></p> + +<p>I've been wondering a bit lately how trusted timestamps could be +stored in Noark 5. +<a href="https://en.wikipedia.org/wiki/Trusted_timestamping">Trusted +timestamps</a> can be used to verify that some information +(document/file/checksum/metadata) have not been changed since a +specific time in the past. This is useful to verify the integrity of +the documents in the archive.</p> + +<p>Then it occured to me, perhaps the trusted timestamps could be +stored as dokument variants (ie dokumentobjekt referered to from +dokumentbeskrivelse) with the filename set to the hash it is +stamping?</p> + +<p>Given a "dokumentbeskrivelse" with an associated "dokumentobjekt", +a new dokumentobjekt is associated with "dokumentbeskrivelse" with the +same attributes as the stamped dokumentobjekt except these +attributes:</p> <ul> - <li>Evolutionary algorithms as a design tool - from art - to robotics (Kyrre Glette)</li> - - <li>Make and break (Hans Gerhard Meier)</li> - - <li>Making a one year school course for young makers - (Olav Helland)</li> - - <li>Innovation Inspiration - IPR Databases as a Source of - Inspiration (Hege Langlo)</li> - - <li>Making a toy for makers (Erik Torstensson)</li> - - <li>How to make 3D printer electronics (Elias Bakken)</li> - - <li>Hovering Clouds: Looking at online tool offerings for Product - Design and 3D Printing (William Kempton)</li> - - <li>Travelling maker stories (Øyvind Nydal Dahl)</li> - - <li>Making the first Maker Faire in Sweden (Nils Olander)</li> - - <li>Breaking the mold: Printing 1000’s of parts (Espen Sivertsen)</li> - - <li>Ultimaker — and open source 3D printing (Erik de Bruijn)</li> - - <li>Autodesk’s 3D Printing Platform: Sparking innovation (Hilde - Sevens)</li> - - <li>How Making is Changing the World – and How You Can Too! - (Jennifer Turliuk)</li> - - <li>Open-Source Adventuring: OpenROV, OpenExplorer and the Future of - Connected Exploration (David Lang)</li> - - <li>Making in Norway (Haakon Karlsen Jr., Graham Hayward and Jens - Dyvik)</li> - - <li>The Impact of the Maker Movement (Mike Senese)</li> +<li>format -> "RFC3161" +<li>mimeType -> "application/timestamp-reply" +<li>formatDetaljer -> "&lt;source URL for timestamp service&gt;" +<li>filenavn -> "&lt;sjekksum&gt;.tsr" </ul> -<p>Part of the reason this took so long was that the scripts NUUG had -to prepare a recording for publication were five years old and no -longer worked with the current video processing tools (command line -argument changes). In addition, we needed better audio normalization, -which sent me on a detour to -<a href="http://people.skolelinux.org/pere/blog/Measuring_and_adjusting_the_loudness_of_a_TV_channel_using_bs1770gain.html">package -bs1770gain for Debian</a>. Now this is in place and it became a lot -easier to publish NUUG videos on Frikanalen.</p> +<p>This assume a service following +<a href="https://tools.ietf.org/html/rfc3161">IETF RFC 3161</a> is +used, which specifiy the given MIME type for replies and the .tsr file +ending for the content of such trusted timestamp. As far as I can +tell from the Noark 5 specifications, it is OK to have several +variants/renderings of a dokument attached to a given +dokumentbeskrivelse objekt. It might be stretching it a bit to make +some of these variants represent crypto-signatures useful for +verifying the document integrity instead of representing the dokument +itself.</p> + +<p>Using the source of the service in formatDetaljer allow several +timestamping services to be used. This is useful to spread the risk +of key compromise over several organisations. It would only be a +problem to trust the timestamps if all of the organisations are +compromised.</p> + +<p>The following oneliner on Linux can be used to generate the tsr +file. $input is the path to the file to checksum, and $sha256 is the +SHA-256 checksum of the file (ie the "<sjekksum>.tsr" value mentioned +above).</p> + +<p><blockquote><pre> +openssl ts -query -data "$inputfile" -cert -sha256 -no_nonce \ + | curl -s -H "Content-Type: application/timestamp-query" \ + --data-binary "@-" http://zeitstempel.dfn.de > $sha256.tsr +</pre></blockquote></p> + +<p>To verify the timestamp, you first need to download the public key +of the trusted timestamp service, for example using this command:</p> + +<p><blockquote><pre> +wget -O ca-cert.txt \ + https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt +</pre></blockquote></p> + +<p>Note, the public key should be stored alongside the timestamps in +the archive to make sure it is also available 100 years from now. It +is probably a good idea to standardise how and were to store such +public keys, to make it easier to find for those trying to verify +documents 100 or 1000 years from now. :)</p> + +<p>The verification itself is a simple openssl command:</p> + +<p><blockquote><pre> +openssl ts -verify -data $inputfile -in $sha256.tsr \ + -CAfile ca-cert.txt -text +</pre></blockquote></p> + +<p>Is there any reason this approach would not work? Is it somehow against +the Noark 5 specification?</p> - Hvem fører Medietilsynet tilsyn med? - http://people.skolelinux.org/pere/blog/Hvem_f_rer_Medietilsynet_tilsyn_med_.html - http://people.skolelinux.org/pere/blog/Hvem_f_rer_Medietilsynet_tilsyn_med_.html - Sun, 28 Jun 2015 23:20:00 +0200 - <p>I en global verden med eierskap på tvers, trengs det informasjon om -hvem som har kontrollen i selskaper og bedrifter. Og for å få tilgang -til slik informasjon for alle som ønsker å analysere eierskap, holder -med ikke med nasjonale databaser over eierskap, det må globale -samledatabaser med åpne data til. Heldigvis finnes det en -internasjonal bevegelse for å gjøre selskapsinformasjon for alle land -offentlig tilgjengelig. En slik database heter -<a href="http://opencorporates.com">OpenCorporates</a>, der -informasjonen er gratis tilgjengelig med en "del på samme -vilkår"-lisens. De samler inn selskapsinformasjon, eierskap, -konsesjonstildelinger og lignende. De manglet ganske mye for Norge, -da bruksvilkårene til Norsk offentlig informasjon i stor grad -blokkerer OpenCorporates fra å samle den inn.</p> - -<p>Men jeg er jo involvert i -<a href="http://www.frikanalen.no/">Frikanalen</a>, som har -TV-konsesjon, og tenkte det kunne være fint om informasjon om alle -mediakonsesjoner var tilgjengelig i OpenCorporates, så jeg sendte -avgårde følgende spørsmål til Medietilsynet 2015-06-22:</p> + Når nynorskoversettelsen svikter til eksamen... + http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html + http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html + Sat, 3 Jun 2017 08:20:00 +0200 + <p><a href="http://www.aftenposten.no/norge/Krever-at-elever-ma-fa-annullert-eksamen-etter-rot-med-oppgavetekster-622459b.html">Aftenposten +melder i dag</a> om feil i eksamensoppgavene for eksamen i politikk og +menneskerettigheter, der teksten i bokmåls og nynorskutgaven ikke var +like. Oppgaveteksten er gjengitt i artikkelen, og jeg ble nysgjerring +på om den fri oversetterløsningen +<a href="https://www.apertium.org/">Apertium</a> ville gjort en bedre +jobb enn Utdanningsdirektoratet. Det kan se slik ut.</p> + +<p>Her er bokmålsoppgaven fra eksamenen:</p> <blockquote> -<p>Hei. Finnes det en oversikt over enhetene som Mediatilsynet fører -tilsyn med som åpne data? Jeg lette etter den på -&lt;URL:<a href="http://data.norge.no/">http://data.norge.no/</a>&gt; og -&lt;URL:<a href="http://hotell.difi.no/">http://hotell.difi.no/</a>&gt; uten å -finne noe der, og fant heller ikke noe under -&lt;URL:<a href="http://www.medietilsynet.no/">http://www.medietilsynet.no/</a>&gt;.</p> - -<p>Jeg tenker på alle som har fått kringkastingskonsesjon og alle som -er omtalt under -&lt;URL:<a href="http://www.medietilsynet.no/mediebildet/">http://www.medietilsynet.no/mediebildet/</a>&gt;.</p> - -<p>Jeg skulle gjerne hatt dette maskinlesbart, og inkludert -organisasjonsnummer og hva slags forhold mediatilsynet har til -organisasjonene. Tanken er å importere det i -&lt;URL:<a href="https://opencorporates.com/">https://opencorporates.com/</a>&gt; for analyse, så det bør ikke ha -bruksbegresninger som gjør dette umulig.</p> +<p>Drøft utfordringene knyttet til nasjonalstatenes og andre aktørers +rolle og muligheter til å håndtere internasjonale utfordringer, som +for eksempel flykningekrisen.</p> + +<p>Vedlegge er eksempler på tekster som kan gi relevante perspektiver +på temaet:</p> +<ol> +<li>Flykningeregnskapet 2016, UNHCR og IDMC +<li>«Grenseløst Europa for fall» A-Magasinet, 26. november 2015 +</ol> + </blockquote> -<p>To dager senere fikk jeg svar, med de datasettene de hadde -tilgjengelig. Svaret fra Hanne Sekkelsten hos Medietilsynet var -informativt og imøtekommende.</p> +<p>Dette oversetter Apertium slik:</p> <blockquote> -<p>Vi viser til din e-post av 22. juni, der du ber om å få tilsendt oversikter -over aktører Medietilsynet fører tilsyn med.</p> - -<p>Medietilsynet fører tilsyn med kringkastere og audiovisuelle -bestillingstjenester som omfattes av kringkastingsloven, og med -eierskap i aviser, fjernsyn radio og elektroniske medier etter -medieeierskapsloven. I tillegg vil Medietilsynet etter at -beskyttelsesloven trer i kraft fra 1. juli ha tilsyn med en rekke nye -aktører. Nærmere informasjon om den nye loven finnes på Medietilsynets -nettsted, her: -<a href="http://www.medietilsynet.no/mediebransjen/bildeprogramloven/">http://www.medietilsynet.no/mediebransjen/bildeprogramloven/</a> -. Vi har ikke utarbeidet oversikter over alle aktørene, men vi sender -deg her de listene vi har utarbeidet, hentet fra våre databaser. Vi -har dessverre ikke ferdige rapporter som inneholder -organisasjonsnummer. Dersom du ønsker flere opplysninger ber vi om at -du tar kontakt med oss slik at vi kan finne ut av hvilke opplysninger -du trenger, og hvilke vi kan fremskaffe.</p> - -<p>Vedlagt følger: +<p>Drøft utfordringane knytte til nasjonalstatane sine og rolla til +andre aktørar og høve til å handtera internasjonale utfordringar, som +til dømes *flykningekrisen.</p> -<ul> +<p>Vedleggja er døme på tekster som kan gje relevante perspektiv på +temaet:</p> - <li>Konsesjoner - lokalradio FM pr. 31.12.2014 [<a href="http://people.skolelinux.org/pere/blog/images/2015-06-28-medietilsynsdata/Konsesjoner_lokalradio_fm_felles_31.12.2014.pdf">PDF (original)</a>]</li> +<ol> +<li>*Flykningeregnskapet 2016, *UNHCR og *IDMC</li> +<li>«*Grenseløst Europa for fall» A-Magasinet, 26. november 2015</li> +</ol> - <li>Konsesjoner - lokalfjernsyn i det digitale bakkenettet for fjernsyn, pr. mars 2015 [<a href="http://people.skolelinux.org/pere/blog/images/2015-06-28-medietilsynsdata/Lokalfjernsyn_DTT_konsesjoner_mars_2015.docx">DOCX (original)</a>, -<a href="http://people.skolelinux.org/pere/blog/images/2015-06-28-medietilsynsdata/Lokalfjernsyn_DTT_konsesjoner_mars_2015.pdf">PDF</a>]</li> +</blockquote> - <li>Konsesjoner - DAB-radio, status mars 2015 [<a href="http://people.skolelinux.org/pere/blog/images/2015-06-28-medietilsynsdata/Konsesjonaernummer_DAB.xps">XPS (original)</a>, -<a href="http://people.skolelinux.org/pere/blog/images/2015-06-28-medietilsynsdata/Konsesjonaernummer_DAB.pdf">PDF</a>]</li> +<p>Ord som ikke ble forstått er markert med stjerne (*), og trenger +ekstra språksjekk. Men ingen ord er forsvunnet, slik det var i +oppgaven elevene fikk presentert på eksamen. Jeg mistenker dog at +"andre aktørers rolle og muligheter til ..." burde vært oversatt til +"rolla til andre aktørar og deira høve til ..." eller noe slikt, men +det er kanskje flisespikking. Det understreker vel bare at det alltid +trengs korrekturlesning etter automatisk oversettelse.</p> + + + + + Epost inn som arkivformat i Riksarkivarens forskrift? + http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html + http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html + Thu, 27 Apr 2017 11:30:00 +0200 + <p>I disse dager, med frist 1. mai, har Riksarkivaren ute en høring på +sin forskrift. Som en kan se er det ikke mye tid igjen før fristen +som går ut på søndag. Denne forskriften er det som lister opp hvilke +formater det er greit å arkivere i +<a href="http://www.arkivverket.no/arkivverket/Offentleg-forvalting/Noark/Noark-5">Noark +5-løsninger</a> i Norge.</p> + +<p>Jeg fant høringsdokumentene hos +<a href="https://www.arkivrad.no/aktuelt/riksarkivarens-forskrift-pa-horing">Norsk +Arkivråd</a> etter å ha blitt tipset på epostlisten til +<a href="https://github.com/hiOA-ABI/nikita-noark5-core">fri +programvareprosjektet Nikita Noark5-Core</a>, som lager et Noark 5 +Tjenestegresesnitt. Jeg er involvert i Nikita-prosjektet og takket +være min interesse for tjenestegrensesnittsprosjektet har jeg lest en +god del Noark 5-relaterte dokumenter, og til min overraskelse oppdaget +at standard epost ikke er på listen over godkjente formater som kan +arkiveres. Høringen med frist søndag er en glimrende mulighet til å +forsøke å gjøre noe med det. Jeg holder på med +<a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/hoering-arkivforskrift.tex">egen +høringsuttalelse</a>, og lurer på om andre er interessert i å støtte +forslaget om å tillate arkivering av epost som epost i arkivet.</p> + +<p>Er du igang med å skrive egen høringsuttalelse allerede? I så fall +kan du jo vurdere å ta med en formulering om epost-lagring. Jeg tror +ikke det trengs så mye. Her et kort forslag til tekst:</p> - <li>Registreringspliktige kringkastere - status mars 2015: [<a href="http://people.skolelinux.org/pere/blog/images/2015-06-28-medietilsynsdata/Registreringspliktige_kringkastere_register_-_aktive.xps">XPS (original)</a>, <a href="http://people.skolelinux.org/pere/blog/images/2015-06-28-medietilsynsdata/Registreringspliktige_kringkastere_register_-_aktive.pdf">PDF</a>] +<p><blockquote> - <ul> - <li>Kabelsendt fjernsyn - <li>Satellittsendt fjernsyn - <li>Nett-tv - <li>Kabelsendt radio - <li>Satellittsendt radio - <li>Nett-radio - </ul></li> -</ul> + <p>Viser til høring sendt ut 2017-02-17 (Riksarkivarens referanse + 2016/9840 HELHJO), og tillater oss å sende inn noen innspill om + revisjon av Forskrift om utfyllende tekniske og arkivfaglige + bestemmelser om behandling av offentlige arkiver (Riksarkivarens + forskrift).</p> -<p>Vi må ta forbehold om at det kan være enkelte feil i oversiktene -siden disse ikke er oppdaterte pr. dags dato. Vi vil foreta nye -oppdateringer i august.</p> + <p>Svært mye av vår kommuikasjon foregår i dag på e-post.  Vi + foreslår derfor at Internett-e-post, slik det er beskrevet i IETF + RFC 5322, + <a href="https://tools.ietf.org/html/rfc5322">https://tools.ietf.org/html/rfc5322</a>. bør + inn som godkjent dokumentformat.  Vi foreslår at forskriftens + oversikt over godkjente dokumentformater ved innlevering i § 5-16 + endres til å ta med Internett-e-post.</p> -<p>Med hilsen</p> +</blockquote></p> -<p>Hanne Nistad Sekkelsten -<br>Seniorrådgiver | Senior Legal Adviser -<br>Medietilsynet | Norwegian Media Authority -<br>A: Nygata 4, NO-1607 Fredrikstad -<br>T: [telefonnummer fjernet] -<br>E: [adresse fjernet] | W: www.medietilsynet.no -</p> -</blockquote> +<p>Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan +epost kan lagres i en Noark 5-struktur, og holder på å skrive et +forslag om hvordan dette kan gjøres som vil bli sendt over til +arkivverket så snart det er ferdig. De som er interesserte kan +<a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/epostlagring.md">følge +fremdriften på web</a>.</p> -<p>Desverre er formatene for ustrukturerte til maskinell behandling og -mangler endel informasjon, men det er gode oversikter over hvem -Medietilsynet fører tilsyn med. Filene er på formatene PDF, XPS -(XML-basert PDF-lignende format fra Microsoft) og DOCX, så det vil -være en stor jobb å strukturere informasjonen på en måte som kan -importeres i OpenCorporates. Svaret er ikke i tråd med -<a href="http://lovdata.no/dokument/SF/forskrift/2013-04-05-959">Forskrift -om IT-standarder i offentlig forvaltning</a> som sier epostvedlegg -skal sendes som PDF, så jeg har gjorde PDF-utgaver av XPS og -DOCX-utgavene tilgjengelig for å gjøre det enklere for alle å se -innholdet.</p> +<p>Oppdatering 2017-04-28: I dag ble høringuttalelsen jeg skrev + <a href="https://www.nuug.no/news/NUUGs_h_ringuttalelse_til_Riksarkivarens_forskrift.shtml">sendt + inn av foreningen NUUG</a>.</p> - Graphing the Norwegian company ownership structure - http://people.skolelinux.org/pere/blog/Graphing_the_Norwegian_company_ownership_structure.html - http://people.skolelinux.org/pere/blog/Graphing_the_Norwegian_company_ownership_structure.html - Mon, 15 Jun 2015 14:00:00 +0200 - <p>It is a bit work to figure out the ownership structure of companies -in Norway. The information is publicly available, but one need to -recursively look up ownership for all owners to figure out the complete -ownership graph of a given set of companies. To save me the work in -the future, I wrote a script to do this automatically, outputting the -ownership structure using the Graphviz/dotty format. The data source -is web scraping from <a href="http://www.proff.no/">Proff</a>, because -I failed to find a useful source directly from the official keepers of -the ownership data, <a href="http://www.brreg.no/">Brønnøysundsregistrene</a>.</p> - -<p>To get an ownership graph for a set of companies, fetch -<a href="https://github.com/petterreinholdtsen/brreg-norway-ownership-graph">the code from git</a> and run it using the organisation number. I'm -using the Norwegian newspaper Dagbladet as an example here, as its -ownership structure is very simple:</p> - -<pre> -% time ./bin/eierskap-dotty 958033540 > dagbladet.dot - -real 0m2.841s -user 0m0.184s -sys 0m0.036s + Offentlig elektronisk postjournal blokkerer tilgang for utvalgte webklienter + http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html + http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html + Thu, 20 Apr 2017 13:00:00 +0200 + <p>Jeg oppdaget i dag at <a href="https://www.oep.no/">nettstedet som +publiserer offentlige postjournaler fra statlige etater</a>, OEP, har +begynt å blokkerer enkelte typer webklienter fra å få tilgang. Vet +ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl +og curl. For å teste selv, kjør følgende:</p> + +<blockquote><pre> +% curl -v -s https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP' +< HTTP/1.1 404 Not Found +% curl -v -s --header 'User-Agent:Opera/12.0' https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP' +< HTTP/1.1 200 OK % -</pre> - -<p>The script accept several organisation numbers on the command line, -allowing a cluster of companies to be graphed in the same image. The -resulting dot file for the example above look like this. The edges -are labeled with the ownership percentage, and the nodes uses the -organisation number as their name and the name as the label:</p> - -<pre> -digraph ownership { -rankdir = LR; -"Aller Holding A/s" -> "910119877" [label="100%"] -"910119877" -> "998689015" [label="100%"] -"998689015" -> "958033540" [label="99%"] -"974530600" -> "958033540" [label="1%"] -"958033540" [label="AS DAGBLADET"] -"998689015" [label="Berner Media Holding AS"] -"974530600" [label="Dagbladets Stiftelse"] -"910119877" [label="Aller Media AS"] -} -</pre> - -<p>To view the ownership graph, run "<tt>dotty dagbladet.dot</tt>" or -convert it to a PNG using "<tt>dot -T png dagbladet.dot > -dagbladet.png</tt>". The result can be seen below:</p> - -<img src="http://people.skolelinux.org/pere/blog/images/2015-06-15-ownership-graphs-norway-dagbladet.png" width="80%"> - -<p>Note that I suspect the "Aller Holding A/S" entry to be incorrect -data in the official ownership register, as that name is not -registered in the official company register for Norway. The ownership -register is sensitive to typos and there seem to be no strict checking -of the ownership links.</p> - -<p>Let me know if you improve the script or find better data sources. -The code is licensed according to GPL 2 or newer.</p> - -<p>Update 2015-06-15: Since the initial post I've been told that -"<a href="http://www.proff.dk/firma/carl-allers-etablissement-aktieselskab/københavn-v/hovedkontorer/13624518-3/">Aller -Holding A/S</a>" is a Danish company, which explain why it did not -have a Norwegian organisation number. I've also been told that there -is a <a href="http://www.brreg.no/automatiske/webservices/">web -services API available</a> from Brønnøysundsregistrene, for those -willing to accept the terms or pay the price.</p> - - - - - Measuring and adjusting the loudness of a TV channel using bs1770gain - http://people.skolelinux.org/pere/blog/Measuring_and_adjusting_the_loudness_of_a_TV_channel_using_bs1770gain.html - http://people.skolelinux.org/pere/blog/Measuring_and_adjusting_the_loudness_of_a_TV_channel_using_bs1770gain.html - Thu, 11 Jun 2015 13:40:00 +0200 - <p>Television loudness is the source of frustration for viewers -everywhere. Some channels are very load, others are less loud, and -ads tend to shout very high to get the attention of the viewers, and -the viewers do not like this. This fact is well known to the TV -channels. See for example the BBC white paper -"<a href="http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP202.pdf">Terminology -for loudness and level dBTP, LU, and all that</a>" from 2011 for a -summary of the problem domain. To better address the need for even -loadness, the TV channels got together several years ago to agree on a -new way to measure loudness in digital files as one step in -standardizing loudness. From this came the ITU-R standard BS.1770, -"<a href="http://www.itu.int/rec/R-REC-BS.1770/en">Algorithms to -measure audio programme loudness and true-peak audio level</a>".</p> - -<p>The ITU-R BS.1770 specification describe an algorithm to measure -loadness in LUFS (Loudness Units, referenced to Full Scale). But -having a way to measure is not enough. To get the same loudness -across TV channels, one also need to decide which value to standardize -on. For European TV channels, this was done in the EBU Recommondaton -R128, "<a href="https://tech.ebu.ch/docs/r/r128.pdf">Loudness -normalisation and permitted maximum level of audio signals</a>", which -specifies a recommended level of -23 LUFS. In Norway, I have been -told that NRK, TV2, MTG and SBS have decided among themselves to -follow the R128 recommondation for playout from 2016-03-01.</p> - -<p>There are free software available to measure and adjust the loudness -level using the LUFS. In Debian, I am aware of a library named -<a href="https://tracker.debian.org/pkg/libebur128">libebur128</a> -able to measure the loudness and since yesterday morning a new binary -named <a href="http://bs1770gain.sourceforge.net">bs1770gain</a> -capable of both measuring and adjusting was uploaded and is waiting -for NEW processing. I plan to maintain the latter in Debian under the -<a href="https://qa.debian.org/developer.php?email=pkg-multimedia-maintainers%40lists.alioth.debian.org">Debian -multimedia</a> umbrella.</p> - -<p>The free software based TV channel I am involved in, -<a href="http://www.frikanalen.no/">Frikanalen</a>, plan to follow the -R128 recommondation ourself as soon as we can adjust the software to -do so, and the bs1770gain tool seem like a good fit for that part of -the puzzle to measure loudness on new video uploaded to Frikanalen. -Personally, I plan to use bs1770gain to adjust the loudness of videos -I upload to Frikanalen on behalf of <a href="http://www.nuug.no/">the -NUUG member organisation</a>. The program seem to be able to measure -the LUFS value of any media file handled by ffmpeg, but I've only -successfully adjusted the LUFS value of WAV files. I suspect it -should be able to adjust it for all the formats handled by ffmpeg.</p> +</pre></blockquote> + +<p>Her kan en se at tjenesten gir «404 Not Found» for curl i +standardoppsettet, mens den gir «200 OK» hvis curl hevder å være Opera +versjon 12.0. Offentlig elektronisk postjournal startet blokkeringen +2017-03-02.</p> + +<p>Blokkeringen vil gjøre det litt vanskeligere å maskinelt hente +informasjon fra oep.no. Kan blokkeringen være gjort for å hindre +automatisert innsamling av informasjon fra OEP, slik Pressens +Offentlighetsutvalg gjorde for å dokumentere hvordan departementene +hindrer innsyn i +<a href="http://presse.no/dette-mener-np/undergraver-offentlighetsloven/">rapporten +«Slik hindrer departementer innsyn» som ble publiserte i januar +2017</a>. Det virker usannsynlig, da det jo er trivielt å bytte +User-Agent til noe nytt.</p> + +<p>Finnes det juridisk grunnlag for det offentlige å diskriminere +webklienter slik det gjøres her? Der tilgang gis eller ikke alt etter +hva klienten sier at den heter? Da OEP eies av DIFI og driftes av +Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to +aktørene man kan be om innsyn i for å forstå hva som har skjedd. Men +<a href="https://www.oep.no/search/result.html?period=dateRange&fromDate=01.01.2016&toDate=01.04.2017&dateType=documentDate&caseDescription=&descType=both&caseNumber=&documentNumber=&sender=basefarm&senderType=both&documentType=all&legalAuthority=&archiveCode=&list2=196&searchType=advanced&Search=Search+in+records">postjournalen +til DIFI viser kun to dokumenter</a> det siste året mellom DIFI og +Basefarm. +<a href="https://www.mimesbronn.no/request/blokkering_av_tilgang_til_oep_fo">Mimes brønn neste</a>, +tenker jeg.</p> - Hva gjør at NRK kan distribuere H.264-video uten patentavtale med MPEG LA? - http://people.skolelinux.org/pere/blog/Hva_gj_r_at_NRK_kan_distribuere_H_264_video_uten_patentavtale_med_MPEG_LA_.html - http://people.skolelinux.org/pere/blog/Hva_gj_r_at_NRK_kan_distribuere_H_264_video_uten_patentavtale_med_MPEG_LA_.html - Wed, 10 Jun 2015 15:20:00 +0200 - <p>Helt siden jeg i 2012 fikk beskjed fra MPEG LA om at -<a href="http://people.skolelinux.org/pere/blog/MPEG_LA_mener_NRK_m__ha_avtale_med_dem_for___kringkaste_og_publisere_H_264_video.html">NRK -trengte patentavtale med dem</a> hvis de distribuerte H.264-video til -sluttbrukere, har jeg lurt på hva som gjør at NRK ikke har slik -avtale. For noen dager siden fikk jeg endelig gjort noe med min -undring, og sendte 2015-05-28 følgende epost til info (at) nrk.no med -tittel "Hva gjør at NRK kan distribuere H.264-video uten patentavtale -med MPEG LA?":</p> - -<p><blockquote> -<p>Jeg lurer på en ting rundt NRKs bruk av H.264-video på sine -websider samt distribusjon via RiksTV og kabel-TV. Har NRK vurdert om -det er behov for en patentavtale med -<a href="http://www.mpegla.com/">MPEG LA</a> slik det står i -programvarelisensene til blant annet Apple Final Cut Studio, Adobe -Premiere Pro, Avid og Apples Final Cut Pro X?</p> - -<p>Hvis dere har vurdert dette, hva var utfallet av en slik vurdering?</p> - -<p>Hvis dere ikke har vurdert dette, har NRK planer om å vurdere behovet -for patentavtale?</p> - -<p>I følge en artikkel på -<a href="https://nrkbeta.no/2012/02/01/siste-kutt-for-final-cut/">NRK -Beta i 2012</a> har NRK brukt eller testet både Apple Final Cut -Studio, Adobe Premiere Pro, Avid og Apples Final Cut Pro X til bruk -for å redigere video før sending. Alle disse har bruksvilkår -understøttet av opphavsretten som sier at de kun kan brukes til å lage -filmer til personlig og ikke-kommersiell bruk - med mindre en har en -lisensavtale med MPEG LA om bruk av patenter utstedt i USA for H.264. -Se f.eks. <a href="http://www.avid.com/static/resources/common/documents/corporate/LICENSE.pdf">bruksvilkårene for Avid</a>, <a href="http://news.cnet.com/8301-30685_3-20000101-264.html">Adobe Premiere</a> og <a href="http://images.apple.com/legal/sla/docs/finalcutstudio2.pdf">Apple Final -Cut Studio</a> og søk etter "MPEG LA".</p> - -<p>Dette får meg til å lure på om det er brudd på opphavsretten å bruke -disse verktøyene i strid med bruksvilkårene uten patentavtale med MPEG -LA. Men NRK bruker jo tilsynelatende disse verktøyene uten patentavtale -med MPEG LA.</p> - -<p>I følge forfatteren av Open Broadcast Encoder finnes det to typer -H.264-relaterte avtaler en kan få med MPEG LA. Det er én for å lage -programvare og utstyr som produserer H.264-video, og en annen for å -kringkaste video som bruker H.264. Dette forteller meg at selv om -produsentene av utstyr og programvare som NRK bruker har en slik avtale -med MPEG LA, så trenges det en egen avtale for å kringkaste video på det -formatet.</p> - -<p>I følge Ryan Rodriguez hos MPEG LA, da jeg spurte ham på epost i -juni 2012, har NRK ikke en slik avtale med MPEG LA. Han sa videre at -NRK trenger en slik avtale hvis NRK tilbyr H.264-kodet video til -sluttbrukere. Jeg sjekket listen med -<a href="http://www.mpegla.com/main/programs/AVC/Pages/Licensees.aspx">organisasjoner -med avtale med MPEG LA</a> og NRK står fortsatt ikke der.</p> - -<p>Jeg lurer dermed på hva som gjør at NRK kan bruke de overnevnte -videoredigeringsverktøyene, som tilsynelatende har krav om avtale med -MPEG LA for å kunne brukes slik NRK bruker dem, til å lage videofiler -for distribusjon uten å ha en avtale med MPEG LA om distribusjon av -H.264-video? Dette er spesielt interessant å vite for oss andre som -også vurderer å spre H.264-video etter å ha redigert dem med disse mye -brukte videoredigeringsverktøyene.</p> -</blockquote></p> - -<p>Samme dag fikk jeg automatisk svar om at min henvendelse hadde fått -saksid 1294699. Jeg fikk deretter følgende respons fra NRK -2015-06-09:</p> - -<p><blockquote> -<p>Hei, beklager lang svartid, men det tok litt tid å finne ut hvem som kunne -svare på dette.</p> - -<p>For selskaper som leverer h.264 til sluttbrukere på nett (f.eks -NRKs nett- tv utgaver som bruker h.264) - og som leverer slike -tjenester uten betaling fra forbrukere – er det heller ikke påkrevd -noen patentavtale.</p> - -<p><a href="http://www.businesswire.com/news/home/20100825006629/en/MPEG-LA%E2%80%99s-AVC-License-Charge-Royalties-Internet#.VWb2ws_774Y">http://www.businesswire.com/news/home/20100825006629/en/MPEG-LA%E2%80%99s-AVC-License-Charge-Royalties-Internet#.VWb2ws_774Y</a></p> - -<p>Med vennlig hilsen -<br>Gunn Helen Berg -<br>Informasjonskonsulent, Publikumsservice</p> - -<p>NRK -<br>Strategidivisjonen -<Br>Sentralbord: +47 23 04 70 00 -<br>Post: NRK Publikumsservice, 8608 Mo i Rana -<br>nrk.no / info (at) nrk.no</p> -</blockquote></p> - -Da dette ikke helt var svar på det jeg lurte på, sendte jeg samme dag -oppfølgerepost tilbake: - -<p><blockquote> -<p>[Gunn Helen Berg] -<br>> Hei, beklager lang svartid, men det tok litt tid å finne ut hvem som -<br>> kunne svare på dette.</p> - -<p>Takk for svar. Men det besvarte ikke helt det jeg spurte om.</p> - -<p>> For selskaper som leverer h.264 til sluttbrukere på nett (f.eks NRKs -<br>> nett- tv utgaver som bruker h.264) - og som leverer slike tjenester -<br>> uten betaling fra forbrukere – er det heller ikke påkrevd noen -<br>> patentavtale. -<br>> -<br>> http://www.businesswire.com/news/home/20100825006629/en/MPEG-LA%E2%80%99s-AVC-License-Charge-Royalties-Internet#.VWb2ws_774Y</p> - -<p>Spørsmålet er ikke kun om MPEG LA krever patentavtale eller ikke -(hvilket ikke helt besvares av pressemeldingen omtalt over, gitt at -pressemeldingen kom i 2010, to år før MPEG LA ansvarlige for -internasjonal lisensiering egen Ryan Rodriguez fortalte meg på epost -at NRK trenger en lisens.</p> - -<p>Det er uklart fra pressemeldingen hva "Internet Broadcast AVC -Video" konkret betyr, men i følge en -<a href="http://www.mpegla.com/main/programs/avc/Documents/avcweb.pdf">presentasjon -fra MPEG LA med tema "AVC PAtent Portfoli License Briefing" datert -2015-05-15</a> gjelder "Internet Broadcast AVC Video" kun kringkasting -på Internet som ikke tilbyr valg av enkeltinnslag ("not -title-by-title"), hvilket jo NRK gjør på sine nettsider. I tillegg -kringkaster jo NRK H.264-video også utenom Internet (RiksTV, kabel, -satelitt), hvilket helt klart ikke er dekket av vilkårene omtalt i -pressemeldingen.</p> - -<p>Spørsmålet mitt er hvordan NRK kan bruke verktøy med bruksvilkår -som krever avtale med MPEG LA for det NRK bruker dem til, når NRK ikke -har avtale med MPEG LA. Hvis jeg forsto spørsmålet riktig, så mener -NRK at dere ikke trenger avtale med MPEG LA, men uten slik avtale kan -dere vel ikke bruke hverken Apple Final Cut Studio, Adobe Premiere -Pro, Avid eller Apples Final Cut Pro X for å redigere video før -sending?</p> - -<p>Mine konkrete spørsmål var altså:</p> - -<ul> - -<li>Hvis NRK har vurdert om det er behov for en patentavtale med MPEG LA - slik det er krav om i programvarelisensene til blant annet Apple - Final Cut Studio, Adobe Premiere Pro, Avid og Apples Final Cut Pro X, - hva var utfallet av en slik vurdering? Kan jeg få kopi av vurderingen - hvis den er gjort skriftlig?</li> - -<li>Hvis NRK ikke har vurdert dette, har NRK planer om å vurdere behovet - for patentavtale?</li> - -<li>Hva slags saksnummer fikk min henvendelse i NRKs offentlige - postjournal? Jeg ser at postjournalen ikke er publisert for den - aktuelle perioden ennå, så jeg fikk ikke sjekket selv.</li> - -</ul> -</blockquote></p> - -<p>Det hjelper å ha funnet rette vedkommende i NRK, for denne gangen -fikk jeg svar tilbake dagen etter (2015-06-10), fra Geir Børdalen i -NRK:</p> - -<p><blockquote> -<p>Hei Petter Reinholdtsen</p> - -<p>Jeg har sjekket saken med distribusjonssjef for tv, Arild Hellgren -(som var teknologidirektør da bakkenettet ble satt opp). NRK v/ -Hellgren hadde møte med MPEG LA sammen med den europeiske -kringkastingsunionen EBU før bakkenettet for TV ble satt opp -(igangsatt høsten 2007). I dette møtet ble det avklart at NRK/EBU ikke -trengte noen patentavtale for h.264 i forbindelse med oppsett av -bakkenettet eller bruk av MPEG4 h.264 som kompresjonsalgoritme fordi -tjenesten «in full»(nor: helt) var betalt av utsendelseselskapene og -ikke av forbrukerne.</p> - -<p><a href="http://www.nrk.no/oppdrag/digitalt-bakkenett-1.3214555">http://www.nrk.no/oppdrag/digitalt-bakkenett-1.3214555</a></p> - -<p>Det er også klart slått fast at selskaper som leverer video basert -på MPEG4 h.264 til sluttbrukere på nett, heller ikke påkrevd noen -patentavtale – så lenge de leverer slike tjenester uten betaling fra -sluttbrukere.</p> - -<a href="http://www.businesswire.com/news/home/20100825006629/en/MPEG-LA%E2%80%99s-AVC-License-Charge-Royalties-Internet#.VWb2ws_774Y">http://www.businesswire.com/news/home/20100825006629/en/MPEG-LA%E2%80%99s-AVC-License-Charge-Royalties-Internet#.VWb2ws_774Y</a> - -<p>“MPEG LA announced today that its AVC Patent Portfolio License will -continue not to charge royalties for Internet Video that is free to -end users (known as “Internet Broadcast AVC Video”) during the entire -life of this License. MPEG LA previously announced it would not charge -royalties for such video through December 31, 2015 (see -<a href="http://www.mpegla.com/Lists/MPEG%20LA%20News%20List/Attachments/226/n-10-02-02.pdf">http://www.mpegla.com/Lists/MPEG%20LA%20News%20List/Attachments/226/n-10-02-02.pdf</a>), -and today’s announcement makes clear that royalties will continue not -to be charged for such video beyond that time. Products and services -other than Internet Broadcast AVC Video continue to be -royalty-bearing.”</p> - -<p>Vi har derfor ikke noe behov for å vurdere noen patentavtale med -MPEG LA.</p> - -<p>Understreker for øvrig at NRK ikke er låst til MPEG4 – h.264 som -utsendelsesformat – og at vi har brukt og bruker flere andre -alternativer i våre tjenester. Ulike «devicer» har ofte behov for -forskjellige løsninger – og NRK har forsøkt å levere med best mulig -kvalitet /økonomi /stabilitet avhengig av -plattform. Produksjonsformater i NRK spenner for øvrig over en rekke -forskjellige formater – hvor MPEG4 bare er en av disse. Når NRK kjøper -teknisk utstyr er betaling for kodekstøtte ofte en del av -anskaffelsesprisen for denne maskinvaren (enten dette er spesialiserte -enkodere eller forskjellige typer produksjonsutstyr).</p> - -<p>Vennlig hilsen -<br>Geir Børdalen</p> - -<p>________________________________________ -<br>Geir Børdalen -<br>Investeringsansvarlig NRK / Hovedprosjektleder - Origo -<br>Avdeling for utvikling, innovasjon, investering og eiendom -<br>NRK medietjenester -<br>Sentralbord: +47 23 04 70 00 -<br>Post: NRK, AUTV (RBM5), Pb. 8500 Majorstuen, 0340 Oslo -<br>nrk.no -</blockquote></p> - -<p>Et godt og grundig svar, som var informativt om hvordan NRK tenker -rundt patentavtale med MPEG LA, men heller ikke helt besvarte det jeg -lurte på, så jeg sendte epostoppfølging samme dag.</p> - -<p><blockquote> -<p>[Geir Børdalen] -<br>> Hei Petter Reinholdtsen</p> - -<p>Hei, og takk for raskt svar. Er min henvendelse journalført slik -at den dukker opp i NRKs postjournal?</p> - -<p>Svaret ditt var meget nyttig, og jeg forstår ut fra det du skriver -at avklaringen med MPEG LA rundt H.264-distribusjon via bakkenettet -gjelder alle TV-kanaler i Norge. Hvilke saksnummer fikk dokumenter -som ble opprettet i forbindelse med det omtalte møtet NRK v/Hellgren -og EBU hadde med MPEG LA (dvs. referater, avtaler, etc), -f.eks. dokumentet der formuleringen "in full" som du omtaler -finnes?<p> - -<p>Men det er et par ting jeg fortsatt ikke forstår. Det ene er -hvorfor NRKs forståelse av hva "Internet Broadcast AVC Video" dekker -ser ut til å avvike fra det som presenteres i -<a href="http://www.mpegla.com/main/programs/avc/Documents/avcweb.pdf">lysark -fra MPEG LA</a> i mai, der MPEG LA på lysark med overskriften -"AVC/H.264 License Terms Participation Fees" og undertittel "Where -remuneration is from other sources" skriver "Internet Broadcast AVC -Video (not title-by-title, not subscription) – no royalty for life of -the AVC Patent Portfolio License".</p> - -<p>Her leser jeg MPEG LA dithen at det kun er kringkasting uten -abonnement via Internet som er dekket at vilkårne omtalt i -pressemeldingen, mens jeg forstår deg dithen at NRK mener NRKs -nettsider som også har enkeltfilmer og innslag (som jeg forstår dekket -av formuleringen "title-by-title") dekkes av "Internet Broadcast AVC -Video" fra MPEG LA. Hva baserer dere denne tolkningen på? Jeg har -ikke sett noe skriftlig fra MPEG LA som støtter NRKs tolkning, og -lurer på om dere har andre kilder enn den pressemeldingen fra 5 år -tilbake, der NRKS forståelse av hva "Internet Broadcast AVC Video" -dekker er beskrevet?</p> - -<p>Det andre er at eposten din ikke nevnte spørsmålet mitt om -bruksvilkårene til videoredigeringsverktøyene som NRK bruker. Disse -har som tidligere nevnt krav om at de kun skal brukes til private og -ikke-kommersielle formål med mindre en har avtale med MPEG LA, og uten -avtale med MPEG LA kan det jo virke som om NRK bruker verktøyene i -strid med bruksvilkårene. Hva gjør at disse bruksvilkårene ikke -gjelder for NRK?</p> -</blockquote></p> - -<p>Noen minutter senere får jeg foreløpig siste svar i -føljetongen:</p> - -<p><blockquote> -<p>Hei igjen</p> - -<p>Vårt dokumentarkiv har fått en kopi (journalføringsnr kan jeg -dessverre ikke gi deg).<p> - -<p>> Svaret ditt var meget nyttig, og jeg forstår ut fra det du -<br>> skriver at avklaringen med MPEG LA rundt H.264-distribusjon via -<br>> bakkenettet gjelder alle TV-kanaler i Norge.</p> - -<p>Svar: Kan ikke svare for andre enn for NRK/EBU - og for bakkenettet -i Norge er det kun NRK som er et lisensbasert selskap. Kan ikke gi noe -svar på saksnr på dokumenter eller ytterligere informasjon da jeg selv -ikke var del i dette.</p> - -<p>> Men det er et par ting jeg fortsatt ikke forstår. ...</p> - -<p>Svar: Kan ikke gå ytterligere inn i dette fra min side og mitt -fagfelt som er produksjon/publisering og systemstrukturene bak -disse. For øvrig ligger det etter vår formening ingen begrensninger -for NRK i mulighetene til publisering mht til kodek i -produksjonssystemer. Som tidligere skrevet mener vi at NRK ikke -trenger noen avtale med MPEG LA og støtter oss til det vi allerede har -kommunisert i forrige epost.</p> - -<p>Mvh -<br>Geir Børdalen</p> -</blockquote></p> - -<p>Det syntes vanskelig å komme videre når NRK ikke ønsker å gå inn i -problemstillingen rundt bruksvilkårene til videoredigeringsverktøyene -NRK bruker, så jeg sendte takk for svarene og avsluttet utvekslingen -så langt:</p> - -<p><blockquote> -<p>Tusen takk for rask respons, og oppklarende forklaring om hvordan -NRK tenker rundt MPEG LA.</p> - -<p>Jeg vil høre med NRK-arkivet for å se om de kan spore opp de -omtalte dokumentene. Jeg setter pris på om du kan dele titler, dato -eller annen informasjon som kan gjøre det enklere for arkivet å finne -dem.</p> - -<p>Når det gjelder hvordan bruksvilkårene til -videoredigeringsverktøyene skal tolkes, så skal jeg høre med MPEG LA -og produsentene av verktøyene for å forsøke å få klarhet i hva de -mener er rikgig rettstilstand.</p> -</blockquote></p> - -<p>Jeg ble litt klokere, men fortsatt er det uklart for meg hva som er -grunnlaget til NRK for å se bort fra bruksvilkår i -videoredigeringsprogramvare som krever MPEG LA-avtale til alt annet -enn privat og ikke-kommersiell bruk.</p> + Free software archive system Nikita now able to store documents + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + Sun, 19 Mar 2017 08:00:00 +0100 + <p>The <a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita +Noark 5 core project</a> is implementing the Norwegian standard for +keeping an electronic archive of government documents. +<a href="http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version">The +Noark 5 standard</a> document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +<a href="https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml">announced +it supported the project</a>. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches.</p> + +<p>If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(<a href="irc://irc.freenode.net/%23nikita"">#nikita on +irc.freenode.net</a>) and +<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">the +project mailing list</a>.</p> + +<p>When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +<tt>archive-pdf</tt> to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +<a href="https://en.wikipedia.org/wiki/Fonds">fonds</a>, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester:</p> + +<p><blockquote><pre> +~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf +using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446 +using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446 + + 0 - Title of the test case file created 2017-03-18T23:49:32.103446 + 1 - Title of the test file created 2017-03-18T23:49:32.103446 +Select which mappe you want (or search term): 0 +Uploading mangelmelding/mangler.pdf + PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt + File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446 +~/src//noark5-tester$ +</pre></blockquote></p> + +<p>You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The <tt>archive-pdf</tt> +tool can be found in the git repository for the API tester.</p> + +<p>In the project, I have been mostly working on +<a href="https://github.com/petterreinholdtsen/noark5-tester">the API +tester</a> so far, while getting to know the code base. The API +tester currently use +<a href="https://en.wikipedia.org/wiki/HATEOAS">the HATEOAS links</a> +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification.</p> + +<p>The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +<a href="https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding">started +writing down</a> the questions we get from implementing it. We use a +format inspired by how <a href="http://www.opengroup.org/austin/">The +Austin Group</a> collect defect reports for the POSIX standard with +<a href="http://www.opengroup.org/austin/mantis.html">their +instructions for the MANTIS defect tracker system</a>, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a <a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md">request for a procedure for submitting defect reports</a> :). + +<p>The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python.</p> - Blir det virkelig krav om fingeravtrykk i nasjonale ID-kort? - http://people.skolelinux.org/pere/blog/Blir_det_virkelig_krav_om_fingeravtrykk_i_nasjonale_ID_kort_.html - http://people.skolelinux.org/pere/blog/Blir_det_virkelig_krav_om_fingeravtrykk_i_nasjonale_ID_kort_.html - Tue, 12 May 2015 11:30:00 +0200 - <p>Noen finner det vanskelig å tro at Stortinget faktisk har vedtatt å -kreve at alle norske borgerne må avgi fingeravtrykk til politiet for å -fungere i samfunnet. Jeg er blitt spurt hva som er grunnlaget for -<a href="http://people.skolelinux.org/pere/blog/Norwegian_citizens_now_required_by_law_to_give_their_fingerprint_to_the_police.html">min -påstand i forrige bloggpost</a> om at det nå blir krav om å avgi -fingeravtrykk til politiet for å fungere som borger i Norge. De som -spør klarer ikke lese det ut fra det som er vedtatt. Her er en liten -oppsummering om hva jeg baserer det på. Det sies ikke direkte i -hverken proposisjon, innstilling eller vedtak, men fremgår når en ser -på indirekte formuleringer.</p> - -<p>I -<a href="https://www.regjeringen.no/nb/dokumenter/prop.-66-l-2014-2015/id2399703/">stortingsproposisjon -66</a>, avsnitt 6.3.5 (Avgivelse av biometriske personopplysninger) -står det<p> + Detecting NFS hangs on Linux without hanging yourself... + http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html + http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html + Thu, 9 Mar 2017 15:20:00 +0100 + <p>Over the years, administrating thousand of NFS mounting linux +computers at the time, I often needed a way to detect if the machine +was experiencing NFS hang. If you try to use <tt>df</tt> or look at a +file or directory affected by the hang, the process (and possibly the +shell) will hang too. So you want to be able to detect this without +risking the detection process getting stuck too. It has not been +obvious how to do this. When the hang has lasted a while, it is +possible to find messages like these in dmesg:</p> <p><blockquote> - - <p>Departementet foreslår at både ansiktsfoto og fingeravtrykk skal - kunne opptas og lagres som identifikasjonsdata i de nasjonale - ID-kortene, på samme måte som i passene. Lovforslaget er derfor - utformet i tråd med passloven § 6 annet ledd, som fastslår at det - til bruk for senere verifisering eller kontroll av passinnehaverens - identitet kan innhentes og lagres i passet biometrisk - personinformasjon i form av ansiktsfoto og fingeravtrykk (to - fingre). Dagens ordning med lagring av ansiktsfoto og fingeravtrykk - i et kontaktløst smartkort i passet er basert på internasjonale - standarder. Fingeravtrykkene i nasjonalt ID-kort vil bli beskyttet - på samme måte som fingeravtrykkene i passene.</p> - - <p>[...]</p> - - <p>For norske forhold understreker departementet at innføring av - nasjonale ID-kort sammen med innføring av nye systemer for sikrere - utstedelse og kontroll av pass og relaterte dokumenter gir mulighet - til å utforme ordningen slik at den best mulig møter utfordringene - forbundet med identitetskriminalitet. Det tilsier at fingeravtrykk - opptas og lagres i alle nasjonale ID-kort.</p> +nfs: server nfsserver not responding, still trying +<br>nfs: server nfsserver OK </blockquote></p> - -<p>Departementet sier altså at sin anbefaling er at fingeravtrykk skal -opptas og lagres i alle nasjonale ID-kort. Det skrives som om det -blir valgfritt, på samme måten som det skrives passloven, der det i -loven sier at det kan -«<a href="https://lovdata.no/dokument/NL/lov/1997-06-19-82#§6">innhentes -og lagres i passet biometrisk personinformasjon i form av ansiktsfoto -og fingeravtrykk (to fingre)</a>». Men på tross av bruken av «kan» i -passloven er det innført krav om å avgi fingeravtrykk for å få et pass -i Norge. Proposisjonen sier i tillegg i del 1 (Proposisjonens -hovedinnhold) at ID-kortene skal være like pålitelig som pass og ha -samme sikkerhetsnivå som pass. Departementet foreslår altså at -ID-kortene skal gis etter samme regler som for pass.</p> - -<p>Formuleringene fra hovedinnholdet i proposisjonen er videreført i -<a href="https://www.stortinget.no/no/Saker-og-publikasjoner/Publikasjoner/Innstillinger/Stortinget/2014-2015/inns-201415-243/?lvl=0">innstillingen -fra stortingskomiteen</a>, der det konkret står «De foreslåtte reglene -vil gi befolkningen tilbud om et offentlig utstedt identitetsbevis som -vil være like pålitelig som passet, og mer praktisk å bruke som -legitimasjon» og «Det nasjonale ID-kortet skal også holde samme -sikkerhetsnivå som passet». Komiteen har altså ingen kommentarer -eller innsigelser til dette forslaget, og gjorde i debatten da saken -ble vedtatt det klart at dette var en god sak og at en enstemmig -komité var glad for resultatet. Stortinget har dermed stilt seg helt -og fullt bak departementets forslag.</p> - -<p>For meg er det åpenbart når en leser proposisjonen at «like -pålitelig» og «samme sikkerhetsnivå» vil bli tolket av departementet -som «med samme biometrisk informasjon som i passene», og departementet -forklarer i tillegg i proposisjonen at de har tenkt at -fingeravtrykkene «vil bli beskyttet på samme måte som fingeravtrykkene -i passene». Jeg ser det dermed som åpenbart at den samme -tvangsinnhentingen av fingeravtrykk som gjelder for pass vil bli -viderført til de nasjonale ID-kortene.</p> - -<p>Det eneste som kan endre dette er massive protester fra -befolkningen på at folk som ikke er mistenkt for noe kriminelt skal -tvinges til å gi fingeravtrykket til politiet for å f.eks. kunne få -bankkonto eller stemme ved valg. Det kunne få departementet til å -snu. Det tror jeg ikke vil skje.</p> + +<p>It is hard to know if the hang is still going on, and it is hard to +be sure looking in dmesg is going to work. If there are lots of other +messages in dmesg the lines might have rotated out of site before they +are noticed.</p> + +<p>While reading through the nfs client implementation in linux kernel +code, I came across some statistics that seem to give a way to detect +it. The om_timeouts sunrpc value in the kernel will increase every +time the above log entry is inserted into dmesg. And after digging a +bit further, I discovered that this value show up in +/proc/self/mountstats on Linux.</p> + +<p>The mountstats content seem to be shared between files using the +same file system context, so it is enough to check one of the +mountstats files to get the state of the mount point for the machine. +I assume this will not show lazy umounted NFS points, nor NFS mount +points in a different process context (ie with a different filesystem +view), but that does not worry me.</p> + +<p>The content for a NFS mount point look similar to this:</p> + +<p><blockquote><pre> +[...] +device /dev/mapper/Debian-var mounted on /var with fstype ext3 +device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1 + opts: rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all + age: 7863311 + caps: caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255 + sec: flavor=1,pseudoflavor=1 + events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0 + bytes: 166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809 + RPC iostats version: 1.0 p/v: 100003/3 (nfs) + xprt: tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820 + per-op statistics + NULL: 0 0 0 0 0 0 0 0 + GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132 + SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943 + LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511 + ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140 + READLINK: 125 125 0 20472 18620 0 1112 1118 + READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693 + WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771 + CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398 + MKDIR: 3680 3680 0 773980 993920 26 23990 24245 + SYMLINK: 903 903 0 233428 245488 6 5865 5917 + MKNOD: 80 80 0 20148 21760 0 299 304 + REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636 + RMDIR: 3367 3367 0 645112 484848 22 5782 6002 + RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288 + LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579 + READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917 + READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937 + FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022 + FSINFO: 2 2 0 232 328 0 1 1 + PATHCONF: 1 1 0 116 140 0 0 0 + COMMIT: 0 0 0 0 0 0 0 0 + +device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc +[...] +</pre></blockquote></p> + +<p>The key number to look at is the third number in the per-op list. +It is the number of NFS timeouts experiences per file system +operation. Here 22 write timeouts and 5 access timeouts. If these +numbers are increasing, I believe the machine is experiencing NFS +hang. Unfortunately the timeout value do not start to increase right +away. The NFS operations need to time out first, and this can take a +while. The exact timeout value depend on the setup. For example the +defaults for TCP and UDP mount points are quite different, and the +timeout value is affected by the soft, hard, timeo and retrans NFS +mount options.</p> + +<p>The only way I have been able to get working on Debian and RedHat +Enterprise Linux for getting the timeout count is to peek in /proc/. +But according to +<ahref="http://docs.oracle.com/cd/E19253-01/816-4555/netmonitor-12/index.html">Solaris +10 System Administration Guide: Network Services</a>, the 'nfsstat -c' +command can be used to get these timeout values. But this do not work +on Linux, as far as I can tell. I +<ahref="http://bugs.debian.org/857043">asked Debian about this</a>, +but have not seen any replies yet.</p> + +<p>Is there a better way to figure out if a Linux NFS client is +experiencing NFS hangs? Is there a way to detect which processes are +affected? Is there a way to get the NFS mount going quickly once the +network problem causing the NFS hang has been cleared? I would very +much welcome some clues, as we regularly run into NFS hangs.</p> - Norwegian citizens now required by law to give their fingerprint to the police - http://people.skolelinux.org/pere/blog/Norwegian_citizens_now_required_by_law_to_give_their_fingerprint_to_the_police.html - http://people.skolelinux.org/pere/blog/Norwegian_citizens_now_required_by_law_to_give_their_fingerprint_to_the_police.html - Sun, 10 May 2015 16:00:00 +0200 - <p>5 days ago, the Norwegian Parliament decided, unanimously, that all -citizens of Norway, no matter if they are suspected of something -criminal or not, are -<a href="https://www.holderdeord.no/votes/1430838871e">required to -give fingerprints to the police</a> (vote details from Holder de -ord). The law make it sound like it will be optional, but in a few -years there will be no option any more. The ID will be required to -vote, to get a bank account, a bank card, to change address on the -post office, to receive an electronic ID or to get a drivers license -and many other tasks required to function in Norway. The banks plan -to stop providing their own ID on the bank cards when this new -national ID is introduced, and the national road authorities plan to -change the drivers license to no longer be usable as identity cards. -In effect, to function as a citizen in Norway a national ID card will -be required, and to get it one need to provide the fingerprints to -the police.</p> - -<p>In addition to handing the fingerprint to the police (which -promised to not make a copy of the fingerprint image at that point in -time, but say nothing about doing it later), a picture of the -fingerprint will be stored on the RFID chip, along with a picture of -the face and other information about the person. Some of the -information will be encrypted, but the encryption will be the same -system as currently used in the passports. The codes to decrypt will -be available to a lot of government offices and their suppliers around -the globe, but for those that do not know anyone in those circles it -is good to know that -<a href="http://www.theguardian.com/technology/2006/nov/17/news.homeaffairs">the -encryption is already broken</a>. And they -<a href="http://www.networkworld.com/article/2215057/wireless/bad-guys-could-read-rfid-passports-at-217-feet--maybe-a-lot-more.html">can -be read from 70 meters away</a>. This can be mitigated a bit by -keeping it in a Faraday cage (metal box or metal wire container), but -one will be required to take it out of there often enough to expose -ones private and personal information to a lot of people that have no -business getting access to that information.</p> - -<p>The new Norwegian national IDs are a vehicle for identity theft, -and I feel sorry for us all having politicians accepting such invasion -of privacy without any objections. So are the Norwegian passports, -but it has been possible to function in Norway without those so far. -That option is going away with the passing of the new law. In this, I -envy the Germans, because for them it is optional how much biometric -information is stored in their national ID.</p> - -<p>And if forced collection of fingerprints was not bad enough, the -information collected in the national ID card register can be handed -over to foreign intelligence services and police authorities, "when -extradition is not considered disproportionate".</p> - -<p>Update 2015-05-12: For those unable to believe that the Parliament -really could make such decision, I wrote -<a href="http://people.skolelinux.org/pere/blog/Blir_det_virkelig_krav_om_fingeravtrykk_i_nasjonale_ID_kort_.html">a -summary of the sources I have</a> for concluding the way I do -(Norwegian Only, as the sources are all in Norwegian).</p> + How does it feel to be wiretapped, when you should be doing the wiretapping... + http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html + http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html + Wed, 8 Mar 2017 11:50:00 +0100 + <p>So the new president in the United States of America claim to be +surprised to discover that he was wiretapped during the election +before he was elected president. He even claim this must be illegal. +Well, doh, if it is one thing the confirmations from Snowden +documented, it is that the entire population in USA is wiretapped, one +way or another. Of course the president candidates were wiretapped, +alongside the senators, judges and the rest of the people in USA.</p> + +<p>Next, the Federal Bureau of Investigation ask the Department of +Justice to go public rejecting the claims that Donald Trump was +wiretapped illegally. I fail to see the relevance, given that I am +sure the surveillance industry in USA believe they have all the legal +backing they need to conduct mass surveillance on the entire +world.</p> + +<p>There is even the director of the FBI stating that he never saw an +order requesting wiretapping of Donald Trump. That is not very +surprising, given how the FISA court work, with all its activity being +secret. Perhaps he only heard about it?</p> + +<p>What I find most sad in this story is how Norwegian journalists +present it. In a news reports the other day in the radio from the +Norwegian National broadcasting Company (NRK), I heard the journalist +claim that 'the FBI denies any wiretapping', while the reality is that +'the FBI denies any illegal wiretapping'. There is a fundamental and +important difference, and it make me sad that the journalists are +unable to grasp it.</p> + +<p><strong>Update 2017-03-13:</strong> Look like +<a href="https://theintercept.com/2017/03/13/rand-paul-is-right-nsa-routinely-monitors-americans-communications-without-warrants/">The +Intercept report that US Senator Rand Paul confirm what I state above</a>.</p> - What would it cost to store all phone calls in Norway? - http://people.skolelinux.org/pere/blog/What_would_it_cost_to_store_all_phone_calls_in_Norway_.html - http://people.skolelinux.org/pere/blog/What_would_it_cost_to_store_all_phone_calls_in_Norway_.html - Fri, 1 May 2015 19:30:00 +0200 - <p>Many years ago, a friend of mine calculated how much it would cost -to store the sound of all phone calls in Norway, and came up with the -cost of around 20 million NOK (2.4 mill EUR) for all the calls in a -year. I got curious and wondered what the same calculation would look -like today. To do so one need an idea of how much data storage is -needed for each minute of sound, how many minutes all the calls in -Norway sums up to, and the cost of data storage.</p> - -<p>The 2005 numbers are from -<a href="http://www.digi.no/analyser/2005/10/04/vi-prater-stadig-mindre-i-roret">digi.no</a>, -the 2012 numbers are from -<a href="http://www.nkom.no/aktuelt/nyheter/fortsatt-vekst-i-det-norske-ekommarkedet">a -NKOM report</a>, and I got the 2013 numbers after asking NKOM via -email. I was told the numbers for 2014 will be presented May 20th, -and decided not to wait for those, as I doubt they will be very -different from the numbers from 2013.</p> - -<p>The amount of data storage per minute sound depend on the wanted -quality, and for phone calls it is generally believed that 8 Kbit/s is -enough. See for example a -<a href="http://www.cisco.com/c/en/us/support/docs/voice/voice-quality/7934-bwidth-consume.html#topic1">summary -on voice quality from Cisco</a> for some alternatives. 8 Kbit/s is 60 -Kbytes/min, and this can be multiplied with the number of call minutes -to get the storage requirements.</p> - -<p>Storage prices varies a lot, depending on speed, backup strategies, -availability requirements etc. But a simple way to calculate can be -to use the price of a TiB-disk (around 1000 NOK / 120 EUR) and double -it to take space, power and redundancy into account. It could be much -higher with high speed and good redundancy requirements.</p> - -<p>But back to the question, What would it cost to store all phone -calls in Norway? Not much. Here is a small table showing the -estimated cost, which is within the budget constraint of most medium -and large organisations:</p> - -<table border="1"> -<tr><th>Year</th><th>Call minutes</th><th>Size</th><th>Price in NOK / EUR</th></tr> -<tr><td>2005</td><td align="right">24 000 000 000</td><td align="right">1.3 PiB</td><td align="right">3 mill / 358 000</td></tr> -<tr><td>2012</td><td align="right">18 000 000 000</td><td align="right">1.0 PiB</td><td align="right">2.2 mill / 262 000</td></tr> -<tr><td>2013</td><td align="right">17 000 000 000</td><td align="right">950 TiB</td><td align="right">2.1 mill / 250 000</td></tr> -</table> - -<p>This is the cost of buying the storage. Maintenance need to be -taken into account too, but calculating that is left as an exercise -for the reader. But it is obvious to me from those numbers that -recording the sound of all phone calls in Norway is not going to be -stopped because it is too expensive. I wonder if someone already is -collecting the data?</p> + Norwegian Bokmål translation of The Debian Administrator's Handbook complete, proofreading in progress + http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html + http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html + Fri, 3 Mar 2017 14:50:00 +0100 + <p>For almost a year now, we have been working on making a Norwegian +Bokmål edition of <a href="https://debian-handbook.info/">The Debian +Administrator's Handbook</a>. Now, thanks to the tireless effort of +Ole-Erik, Ingrid and Andreas, the initial translation is complete, and +we are working on the proof reading to ensure consistent language and +use of correct computer science terms. The plan is to make the book +available on paper, as well as in electronic form. For that to +happen, the proof reading must be completed and all the figures need +to be translated. If you want to help out, get in touch.</p> + +<p><a href="http://people.skolelinux.org/pere/debian-handbook/debian-handbook-nb-NO.pdf">A + +fresh PDF edition</a> in A4 format (the final book will have smaller +pages) of the book created every morning is available for +proofreading. If you find any errors, please +<a href="https://hosted.weblate.org/projects/debian-handbook/">visit +Weblate and correct the error</a>. The +<a href="http://l.github.io/debian-handbook/stat/nb-NO/index.html">state +of the translation including figures</a> is a useful source for those +provide Norwegian bokmål screen shots and figures.</p> - First Jessie based Debian Edu beta release - http://people.skolelinux.org/pere/blog/First_Jessie_based_Debian_Edu_beta_release.html - http://people.skolelinux.org/pere/blog/First_Jessie_based_Debian_Edu_beta_release.html - Sun, 26 Apr 2015 14:10:00 +0200 - <p>I am happy to report that the Debian Edu team sent out -<a href="https://lists.debian.org/debian-edu-announce/2015/04/msg00000.html">this -announcement today</a>:</p> - -<pre> -the Debian Edu / Skolelinux project is pleased to announce the first -*beta* release of Debian Edu "Jessie" 8.0+edu0~b1, which for the first -time is composed entirely of packages from the current Debian stable -release, Debian 8 "Jessie". - -(As most reading this will know, Debian "Jessie" hasn't actually been -released by now. The release is still in progress but should finish -later today ;) - -We expect to make a final release of Debian Edu "Jessie" in the coming -weeks, timed with the first point release of Debian Jessie. Upgrades -from this beta release of Debian Edu Jessie to the final release will -be possible and encouraged! - -Please report feedback to debian-edu@lists.debian.org and/or submit -bugs: http://wiki.debian.org/DebianEdu/HowTo/ReportBugs - -Debian Edu - sometimes also known as "Skolelinux" - is a complete -operating system for schools, universities and other -organisations. Through its pre- prepared installation profiles -administrators can install servers, workstations and laptops which -will work in harmony on the school network. With Debian Edu, the -teachers themselves or their technical support staff can roll out a -complete multi-user, multi-machine study environment within hours or -days. - -Debian Edu is already in use at several hundred schools all over the -world, particularly in Germany, Spain and Norway. Installations come -with hundreds of applications pre-installed, plus the whole Debian -archive of thousands of compatible packages within easy reach. - -For those who want to give Debian Edu Jessie a try, download and -installation instructions are available, including detailed -instructions in the manual explaining the first steps, such as setting -up a network or adding users. Please note that the password for the -user your prompted for during installation must have a length of at -least 5 characters! - -== Where to download == - -A multi-architecture CD / usbstick image (649 MiB) for network booting -can be downloaded at the following locations: - - http://ftp.skolelinux.org/skolelinux-cd/debian-edu-8.0+edu0~b1-CD.iso - rsync -avzP ftp.skolelinux.org::skolelinux-cd/debian-edu-8.0+edu0~b1-CD.iso . - -The SHA1SUM of this image is: 54a524d16246cddd8d2cfd6ea52f2dd78c47ee0a - -Alternatively an extended DVD / usbstick image (4.9 GiB) is also -available, with more software included (saving additional download -time): - - http://ftp.skolelinux.org/skolelinux-cd/debian-edu-8.0+edu0~b1-USB.iso - rsync -avzP ftp.skolelinux.org::skolelinux-cd/debian-edu-8.0+edu0~b1-USB.iso - -The SHA1SUM of this image is: fb1f1504a490c077a48653898f9d6a461cb3c636 - -Sources are available from the Debian archive, see -http://ftp.debian.org/debian-cd/8.0.0/source/ for some download -options. - -== Debian Edu Jessie manual in seven languages == - -Please see https://wiki.debian.org/DebianEdu/Documentation/Jessie/ for -the English version of the Debian Edu jessie manual. - -This manual has been fully translated to German, French, Italian, -Danish, Dutch and Norwegian Bokmål. A partly translated version exists -for Spanish. See http://maintainer.skolelinux.org/debian-edu-doc/ for -online version of the translated manual. - -More information about Debian 8 "Jessie" itself is provided in the -release notes and the installation manual: -- http://www.debian.org/releases/jessie/releasenotes -- http://www.debian.org/releases/jessie/installmanual - - -== Errata / known problems == - - It takes up to 15 minutes for a changed hostname to be updated via - DHCP (#780461). - - The hostname script fails to update LTSP server hostname (#783087). - -Workaround: run update-hostname-from-ip on the client to update the -hostname immediately. - -Check https://wiki.debian.org/DebianEdu/Status/Jessie for a possibly -more current and complete list. - -== Some more details about Debian Edu 8.0+edu0~b1 Codename Jessie released 2015-04-25 == - -=== Software updates === - -Everything which is new in Debian 8 Jessie, e.g.: - - * Linux kernel 3.16.7-ctk9; for the i386 architecture, support for - i486 processors has been dropped; oldest supported ones: i586 (like - Intel Pentium and AMD K5). - - * Desktop environments KDE Plasma Workspaces 4.11.13, GNOME 3.14, - Xfce 4.12, LXDE 0.5.6 - * new optional desktop environment: MATE 1.8 - * KDE Plasma Workspaces is installed by default; to choose one of - the others see the manual. - * the browsers Iceweasel 31 ESR and Chromium 41 - * LibreOffice 4.3.3 - * GOsa 2.7.4 - * LTSP 5.5.4 - * CUPS print system 1.7.5 - * new boot framework: systemd - * Educational toolbox GCompris 14.12 - * Music creator Rosegarden 14.02 - * Image editor Gimp 2.8.14 - * Virtual stargazer Stellarium 0.13.1 - * golearn 0.9 - * tuxpaint 0.9.22 - * New version of debian-installer from Debian Jessie. - * Debian Jessie includes about 43000 packages available for installation. - * More information about Debian 8 Jessie is provided in its release - notes and the installation manual, see the link above. - -=== Installation changes === - - Installations done via PXE now also install firmware automatically - for the hardware present. - -=== Fixed bugs === - -A number of bugs have been fixed in this release; the most noticeable -from a user perspective: - - * Inserting incorrect DNS information in Gosa will no longer break - DNS completely, but instead stop DNS updates until the incorrect - information is corrected (710362) - - * shutdown-at-night now shuts the system down if gdm3 is used (775608). - -=== Sugar desktop removed === - -As the Sugar desktop was removed from Debian Jessie, it is also not -available in Debian Edu jessie. - - -== About Debian Edu / Skolelinux == - -Debian Edu, also known as Skolelinux, is a Linux distribution based on -Debian providing an out-of-the box environment of a completely -configured school network. Directly after installation a school server -running all services needed for a school network is set up just -waiting for users and machines being added via GOsa², a comfortable -Web-UI. A netbooting environment is prepared using PXE, so after -initial installation of the main server from CD or USB stick all other -machines can be installed via the network. The provided school server -provides LDAP database and Kerberos authentication service, -centralized home directories, DHCP server, web proxy and many other -services. The desktop contains more than 60 educational software -packages and more are available from the Debian archive, and schools -can choose between KDE, GNOME, LXDE, Xfce and MATE desktop -environment. - -== About Debian == - -The Debian Project was founded in 1993 by Ian Murdock to be a truly -free community project. Since then the project has grown to be one of -the largest and most influential open source projects. Thousands of -volunteers from all over the world work together to create and -maintain Debian software. Available in 70 languages, and supporting a -huge range of computer types, Debian calls itself the universal -operating system. + Unlimited randomness with the ChaosKey? + http://people.skolelinux.org/pere/blog/Unlimited_randomness_with_the_ChaosKey_.html + http://people.skolelinux.org/pere/blog/Unlimited_randomness_with_the_ChaosKey_.html + Wed, 1 Mar 2017 20:50:00 +0100 + <p>A few days ago I ordered a small batch of +<a href="http://altusmetrum.org/ChaosKey/">the ChaosKey</a>, a small +USB dongle for generating entropy created by Bdale Garbee and Keith +Packard. Yesterday it arrived, and I am very happy to report that it +work great! According to its designers, to get it to work out of the +box, you need the Linux kernel version 4.1 or later. I tested on a +Debian Stretch machine (kernel version 4.9), and there it worked just +fine, increasing the available entropy very quickly. I wrote a small +test oneliner to test. It first print the current entropy level, +drain /dev/random, and then print the entropy level for five seconds. +Here is the situation without the ChaosKey inserted:</p> + +<blockquote><pre> +% cat /proc/sys/kernel/random/entropy_avail; \ + dd bs=1M if=/dev/random of=/dev/null count=1; \ + for n in $(seq 1 5); do \ + cat /proc/sys/kernel/random/entropy_avail; \ + sleep 1; \ + done +300 +0+1 oppføringer inn +0+1 oppføringer ut +28 byte kopiert, 0,000264565 s, 106 kB/s +4 +8 +12 +17 +21 +% +</pre></blockquote> + +<p>The entropy level increases by 3-4 every second. In such case any +application requiring random bits (like a HTTPS enabled web server) +will halt and wait for more entrpy. And here is the situation with +the ChaosKey inserted:</p> + +<blockquote><pre> +% cat /proc/sys/kernel/random/entropy_avail; \ + dd bs=1M if=/dev/random of=/dev/null count=1; \ + for n in $(seq 1 5); do \ + cat /proc/sys/kernel/random/entropy_avail; \ + sleep 1; \ + done +1079 +0+1 oppføringer inn +0+1 oppføringer ut +104 byte kopiert, 0,000487647 s, 213 kB/s +433 +1028 +1031 +1035 +1038 +% +</pre></blockquote> -== Thanks == +<p>Quite the difference. :) I bought a few more than I need, in case +someone want to buy one here in Norway. :)</p> -Thanks to everyone making Debian and Debian Edu / Skolelinux happen! -You rock. -</pre> +<p>Update: The dongle was presented at Debconf last year. You might +find <a href="https://debconf16.debconf.org/talks/94/">the talk +recording illuminating</a>. It explains exactly what the source of +randomness is, if you are unable to spot it from the schema drawing +available from the ChaosKey web site linked at the start of this blog +post.</p> - Debian Edu interview: Shirish Agarwal - http://people.skolelinux.org/pere/blog/Debian_Edu_interview__Shirish_Agarwal.html - http://people.skolelinux.org/pere/blog/Debian_Edu_interview__Shirish_Agarwal.html - Wed, 15 Apr 2015 09:20:00 +0200 - <p>It was a surprise to me to learn that project to create a complete -computer system for schools I've involved in, -<a href="http://www.skolelinux.org/">Debian Edu / Skolelinux</a>, was -being used in India. But apparently it is, and I managed to get an -interview with one of the friends of the project there, Shirish -Agarwal.</p> - -<p><strong>Who are you, and how do you spend your days?</strong></p> - -<p>My name is Shirish Agarwal. Based out of the educational and -historical city of Pune, from the western state of Maharashtra, India. -My bread comes from giving training, giving policy tips, -installations on free software to mom and pop shops in different -fields from Desktop publishing to retail shops as well as work with -few software start-ups as well.</p> - -<p><strong>How did you get in contact with the Skolelinux / Debian Edu -project?</strong></p> - -<p>It started innocently enough. I have been using Debian for a few -years and in one local minidebconf / debutsav I was asked if there was -anything for schools or education. I had worked / played with free -educational softwares such as Gcompris and Stellarium for my many -nieces and nephews so researched and found Debian Edu or Skolelinux as -it was known then. Since then I have started using the various -education meta-packages provided by the project.</p> - -<p><strong>What do you see as the advantages of Skolelinux / Debian -Edu?</strong></p> - -<p>It's closest I have seen where a package full of educational -software are packed, which are free and open (both literally and -figuratively). Even if I take the simplest software which is -gcompris, the number of activities therein are amazing. Another one of -the softwares that I have liked for a long time is stellarium. Even -pysycache is cool except for couple of issues I encountered -<a href="https://bugs.debian.org/781841">#781841</a> and -<a href="https://bugs.debian.org/781842">#781842</a>.</p> - -<p>I prefer software installed on the system over web based solutions, -as a web site can disappear any time but the software on disk has the -possibility of a larger life span. Of course with both it's more a -question if it has enough users who make it fun or sustainable or both -for the developer per-se.</p> - -<p><strong>What do you see as the disadvantages of Skolelinux / Debian -Edu?</strong></p> - -<p>I do see that the Debian Edu team seems to be short-handed and I -think more efforts should be made to make it popular and ask and take -help from people and the larger community wherever possible.</p> - -<p>I don't see any disadvantage to use Skolelinux apart from the fact -that most apps. are generic which is good or bad how you see it. -However, saying that I do acknowledge the fact that the canvas is -pretty big and there are lot of interesting ideas that could be done -but for reasons not known not done or if done I don't know about them. -Let me share some of the ideas (these are more upstream based but -still) I have had for a long time :</p> - -<p>1. Classical maths question of two trains in opposing directions -each running @x kmph/mph at y distance, when they will meet and how -far would each travel and similar questions like these. - -<p>The computer is a fantastic system where questions like these can -be drawn, animated and the methodology and answers teased out in -interactive manner. While sites such as the -<a href="http://mathforum.org/dr.math/faq/faq.two.trains.html">Ask -Dr. Math FAQ on The Two Trains problem</a> (as an example or point of -inspiration) can be used there is lot more that can be done. I dunno -if there is a free software which does something like this. The idea -being a blend of objects + animation + interaction which does -this. The whole interaction could be gamified with points or sounds or -colourful celebration whenever the user gets even part of the question -or/and methodology right. That would help reinforce good behaviour. -This understanding could be used to share/showcase everything from how -the first wheel came to be, to evolution to how astronomy started, -psychics and everything in-between.</p> - -<p>One specific idea in the train part was having the Linux mascot on -one train and the BSD or GNU mascot on the other train and they -meeting somewhere in-between. Characters from blender movies could -also be used.</p> - -<p>2. Loads of crossword-puzzles with reference to subjects: We have -enormous data sets in Wikipedia and Wikitionary. I don't think it -should be a big job to design crossword puzzles. Using categories and -sub-categories it should be doable to have Q&A single word answers -from the existing data-sets. What would make it easy or hard could be -the length of the word + existence of many or few vowels depending on -the user's input.</p> - -<p>3. Jigsaw puzzles - We already have a great software called -palapeli with number of slicers making it pretty interesting. What -needs to be done is to download large number of public domain and -copyleft images, tease and use IPTC tags to categorise them into -nature, history etc. and let it loose. This could turn to be really -huge collection of images. One source could be taken from -commons.wikimedia.org, others could be huge collection of royalty-free -stock photos. Potential is immense.</p> - -<p>Apart from this, free software suffers in two directions, we lag -both in development (of using new features per-se) and maintenance a -lot. This is more so in educational software as these applications -need to be timely and the opportunity cost of missing deadlines is -immense. If we are able to solve issues of funding for development and -maintenance of such software I don't see any big difficulties. I know -of few start-ups in and around India who would love to develop and -maintain such software if funding issues could be solved.</p> - -<p><strong>Which free software do you use daily?</strong></p> - -<p>That would be huge list. Some of the softwares are obviously apt, -aptitude, debdelta, leafpad, the shell of course (zsh nowadays), -quassel for IRC. In games I use shisen-sho while card-games are evenly -between kpat and Aiselriot. In desktops it's a tie between -gnome-flashback and mate.</p> - -<p><strong>Which strategy do you believe is the right one to use to -get schools to use free software?</strong></p> - -<p>I think it should first start with using specific FOSS apps. in -whatever environment they are. If it's MS-Windows or Mac so be it. -Once they are habitual with the apps. and there is buy-in from the -school management then it could be installed anywhere. Most of the -people now understand the concept of a repository because of the -various online stores so it isn't hard to convince on that front.</p> - -<p>What is harder is having enough people with technical skills and -passion to service them. If you get buy-in from one or two teachers -then ideas like above could also be asked to be done as a project as -well.</p> - -<p>I think where we fall short more than anything is in marketing. For -instance, Debian has this whole range of fonts in its archive but -there isn't even a page where all those different fonts in the La -Ipsum format could be tried out for newcomers.</p> - -<p>One of the issues faced constantly in installations is with updates -and upgrades. People have this myth that each update and upgrade -means the user interface will / has to change. I have seen this -innumerable times. That perhaps is one of the reasons which browsers -like Iceweasel / Firefox change user interfaces so much, not because -it might be needed or be functional but because people believe that -changed user interfaces are better. This, can easily be pointed with -the user interfaces changed with almost every MS-Windows and Mac OS -releases.</p> - -<p>The problems with Debian Edu for deployment are many. The biggest -is the huge gap between what is taught in schools and what Debian Edu -is aimed at. - -<p>Me and my friends did teach on week-ends in a government school for -around 2 years, and -<a href="https://flossexperiences.wordpress.com/2012/10/08/sharings/">gathered -some experience</a> there. Some of the things we learnt/discovered -there was :</p> - -<ol> - - <li>Most of the teachers are very territorial about their subjects - and they do not want you to teach anything out of the - portion/syllabus given.</li> - - <li>They want any activity on the system in accordance to whatever - is in the syllabus.</li> - - <li>There are huge barriers both with the English language and at - times with objects or whatever. An example, let's say in gcompris - you have objects falling down and you have to name them and let's - say the falling object is a hat or a fedora hat, this would not be - as recognizable as say a - <a href="https://en.wikipedia.org/wiki/Puneri_Pagadi">Puneri - Pagdi</a> so there is need to inject local objects, words wherever - possible. Especially for word-games there are so many hindi words - which have become part of english vocabulary (for instance in - parley), those could be made into a hinglish collection or - something but that is something for upstream to do.</li> - -</ol> + Detect OOXML files with undefined behaviour? + http://people.skolelinux.org/pere/blog/Detect_OOXML_files_with_undefined_behaviour_.html + http://people.skolelinux.org/pere/blog/Detect_OOXML_files_with_undefined_behaviour_.html + Tue, 21 Feb 2017 00:20:00 +0100 + <p>I just noticed +<a href="http://www.arkivrad.no/aktuelt/riksarkivarens-forskrift-pa-horing">the +new Norwegian proposal for archiving rules in the goverment</a> list +<a href="http://www.ecma-international.org/publications/standards/Ecma-376.htm">ECMA-376</a> +/ ISO/IEC 29500 (aka OOXML) as valid formats to put in long term +storage. Luckily such files will only be accepted based on +pre-approval from the National Archive. Allowing OOXML files to be +used for long term storage might seem like a good idea as long as we +forget that there are plenty of ways for a "valid" OOXML document to +have content with no defined interpretation in the standard, which +lead to a question and an idea.</p> + +<p>Is there any tool to detect if a OOXML document depend on such +undefined behaviour? It would be useful for the National Archive (and +anyone else interested in verifying that a document is well defined) +to have such tool available when considering to approve the use of +OOXML. I'm aware of the +<a href="https://github.com/arlm/officeotron/">officeotron OOXML +validator</a>, but do not know how complete it is nor if it will +report use of undefined behaviour. Are there other similar tools +available? Please send me an email if you know of any such tool.</p>