X-Git-Url: http://pere.pagekite.me/gitweb/homepage.git/blobdiff_plain/31e78b216f079bfa709d1aa5fa4a5d91da9f46bd..aa4069048e4b39f85304deb4c9abc90695b8441c:/blog/index.rss diff --git a/blog/index.rss b/blog/index.rss index 2f02129b6d..e42e39614e 100644 --- a/blog/index.rss +++ b/blog/index.rss @@ -7,736 +7,634 @@ - Full battery stats collector is now available in Debian - http://people.skolelinux.org/pere/blog/Full_battery_stats_collector_is_now_available_in_Debian.html - http://people.skolelinux.org/pere/blog/Full_battery_stats_collector_is_now_available_in_Debian.html - Wed, 23 Mar 2016 22:10:00 +0100 - <p>Since this morning, the battery-stats package in Debian include an -extended collector that will collect the complete battery history for -later processing and graphing. The original collector store the -battery level as percentage of last full level, while the new -collector also record battery vendor, model, serial number, design -full level, last full level and current battery level. This make it -possible to predict the lifetime of the battery as well as visualise -the energy flow when the battery is charging or discharging.</p> - -<p>The new tools are available in <tt>/usr/share/battery-stats/</tt> -in the version 0.5.1 package in unstable. Get the new battery level graph -and lifetime prediction by running: - -<p><pre> -/usr/share/battery-stats/battery-stats-graph /var/log/battery-stats.csv -</pre></p> - -<p>Or select the 'Battery Level Graph' from your application menu.</p> - -<p>The flow in/out of the battery can be seen by running (no menu -entry yet):</p> - -<p><pre> -/usr/share/battery-stats/battery-stats-graph-flow -</pre></p> - -<p>I'm not quite happy with the way the data is visualised, at least -when there are few data points. The graphs look a bit better with a -few years of data.</p> - -<p>A while back one important feature I use in the battery stats -collector broke in Debian. The scripts in -<tt>/usr/lib/pm-utils/power.d/</tt> were no longer executed. I -suspect it happened when Jessie started using systemd, but I do not -know. The issue is reported as bug #818649 against pm-utils. I -managed to work around it by adding an udev rule to call the collector -script every time the power connector is connected and disconnected. -With this fix in place it was finally time to make a new release of -the package, and get it into Debian.</p> - -<p>If you are interested in how your laptop battery is doing, please -check out the -<a href="https://tracker.debian.org/pkg/battery-stats">battery-stats</a> -in Debian unstable, or rebuild it on Jessie to get it working on -Debian stable. :) The upstream source is available from -<a href="https://github.com/petterreinholdtsen/battery-stats">github</a>. -As always, patches are very welcome.</p> + Idea for storing trusted timestamps in a Noark 5 archive + http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html + http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html + Wed, 7 Jun 2017 21:40:00 +0200 + <p><em>This is a copy of +<a href="https://lists.nuug.no/pipermail/nikita-noark/2017-June/000297.html">an +email I posted to the nikita-noark mailing list</a>. Please follow up +there if you would like to discuss this topic. The background is that +we are making a free software archive system based on the Norwegian +<a href="https://www.arkivverket.no/forvaltning-og-utvikling/regelverk-og-standarder/noark-standarden">Noark +5 standard</a> for government archives.</em></p> + +<p>I've been wondering a bit lately how trusted timestamps could be +stored in Noark 5. +<a href="https://en.wikipedia.org/wiki/Trusted_timestamping">Trusted +timestamps</a> can be used to verify that some information +(document/file/checksum/metadata) have not been changed since a +specific time in the past. This is useful to verify the integrity of +the documents in the archive.</p> + +<p>Then it occured to me, perhaps the trusted timestamps could be +stored as dokument variants (ie dokumentobjekt referered to from +dokumentbeskrivelse) with the filename set to the hash it is +stamping?</p> + +<p>Given a "dokumentbeskrivelse" with an associated "dokumentobjekt", +a new dokumentobjekt is associated with "dokumentbeskrivelse" with the +same attributes as the stamped dokumentobjekt except these +attributes:</p> + +<ul> + +<li>format -> "RFC3161" +<li>mimeType -> "application/timestamp-reply" +<li>formatDetaljer -> "&lt;source URL for timestamp service&gt;" +<li>filenavn -> "&lt;sjekksum&gt;.tsr" + +</ul> + +<p>This assume a service following +<a href="https://tools.ietf.org/html/rfc3161">IETF RFC 3161</a> is +used, which specifiy the given MIME type for replies and the .tsr file +ending for the content of such trusted timestamp. As far as I can +tell from the Noark 5 specifications, it is OK to have several +variants/renderings of a dokument attached to a given +dokumentbeskrivelse objekt. It might be stretching it a bit to make +some of these variants represent crypto-signatures useful for +verifying the document integrity instead of representing the dokument +itself.</p> + +<p>Using the source of the service in formatDetaljer allow several +timestamping services to be used. This is useful to spread the risk +of key compromise over several organisations. It would only be a +problem to trust the timestamps if all of the organisations are +compromised.</p> + +<p>The following oneliner on Linux can be used to generate the tsr +file. $input is the path to the file to checksum, and $sha256 is the +SHA-256 checksum of the file (ie the "<sjekksum>.tsr" value mentioned +above).</p> + +<p><blockquote><pre> +openssl ts -query -data "$inputfile" -cert -sha256 -no_nonce \ + | curl -s -H "Content-Type: application/timestamp-query" \ + --data-binary "@-" http://zeitstempel.dfn.de > $sha256.tsr +</pre></blockquote></p> + +<p>To verify the timestamp, you first need to download the public key +of the trusted timestamp service, for example using this command:</p> + +<p><blockquote><pre> +wget -O ca-cert.txt \ + https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt +</pre></blockquote></p> + +<p>Note, the public key should be stored alongside the timestamps in +the archive to make sure it is also available 100 years from now. It +is probably a good idea to standardise how and were to store such +public keys, to make it easier to find for those trying to verify +documents 100 or 1000 years from now. :)</p> + +<p>The verification itself is a simple openssl command:</p> + +<p><blockquote><pre> +openssl ts -verify -data $inputfile -in $sha256.tsr \ + -CAfile ca-cert.txt -text +</pre></blockquote></p> + +<p>Is there any reason this approach would not work? Is it somehow against +the Noark 5 specification?</p> - UsingQR - "Electronic" paper invoices using JSON and QR codes - http://people.skolelinux.org/pere/blog/UsingQR____Electronic__paper_invoices_using_JSON_and_QR_codes.html - http://people.skolelinux.org/pere/blog/UsingQR____Electronic__paper_invoices_using_JSON_and_QR_codes.html - Sat, 19 Mar 2016 09:40:00 +0100 - <p>Back in 2013 I proposed -<a href="http://people.skolelinux.org/pere/blog/_Electronic__paper_invoices___using_vCard_in_a_QR_code.html">a -way to make paper and PDF invoices easier to process electronically by -adding a QR code with the key information about the invoice</a>. I -suggested using vCard field definition, to get some standard format -for name and address, but any format would work. I did not do -anything about the proposal, but hoped someone one day would make -something like it. It would make it possible to efficiently send -machine readable invoices directly between seller and buyer.</p> - -<p>This was the background when I came across a proposal and -specification from the web based accounting and invoicing supplier -<a href="http://www.visma.com/">Visma</a> in Sweden called -<a href="http://usingqr.com/">UsingQR</a>. Their PDF invoices contain -a QR code with the key information of the invoice in JSON format. -This is the typical content of a QR code following the UsingQR -specification (based on a real world example, some numbers replaced to -get a more bogus entry). I've reformatted the JSON to make it easier -to read. Normally this is all on one long line:</p> - -<p><img src="http://people.skolelinux.org/pere/blog/images/2016-03-19-qr-invoice.png" align="right"><pre> -{ - "vh":500.00, - "vm":0, - "vl":0, - "uqr":1, - "tp":1, - "nme":"Din Leverandør", - "cc":"NO", - "cid":"997912345 MVA", - "iref":"12300001", - "idt":"20151022", - "ddt":"20151105", - "due":2500.0000, - "cur":"NOK", - "pt":"BBAN", - "acc":"17202612345", - "bc":"BIENNOK1", - "adr":"0313 OSLO" -} -</pre></p> - -</p>The interpretation of the fields can be found in the -<a href="http://usingqr.com/wp-content/uploads/2014/06/UsingQR_specification1.pdf">format -specification</a> (revision 2 from june 2014). The format seem to -have most of the information needed to handle accounting and payment -of invoices, at least the fields I have needed so far here in -Norway.</p> - -<p>Unfortunately, the site and document do not mention anything about -the patent, trademark and copyright status of the format and the -specification. Because of this, I asked the people behind it back in -November to clarify. Ann-Christine Savlid (ann-christine.savlid (at) -visma.com) replied that Visma had not applied for patent or trademark -protection for this format, and that there were no copyright based -usage limitations for the format. I urged her to make sure this was -explicitly written on the web pages and in the specification, but -unfortunately this has not happened yet. So I guess if there is -submarine patents, hidden trademarks or a will to sue for copyright -infringements, those starting to use the UsingQR format might be at -risk, but if this happen there is some legal defense in the fact that -the people behind the format claimed it was safe to do so. At least -with patents, there is always -<a href="http://www.paperspecs.com/paper-news/beware-the-qr-code-patent-trap/">a -chance of getting sued...</a></p> - -<p>I also asked if they planned to maintain the format in an -independent standard organization to give others more confidence that -they would participate in the standardization process on equal terms -with Visma, but they had no immediate plans for this. Their plan was -to work with banks to try to get more users of the format, and -evaluate the way forward if the format proved to be popular. I hope -they conclude that using an open standard organisation like -<a href="http://www.ietf.org/">IETF</a> is the correct place to -maintain such specification.</p> - -<p><strong>Update 2016-03-20</strong>: Via Twitter I became aware of -<a href="https://news.ycombinator.com/item?id=11319492">some comments -about this blog post</a> that had several useful links and references to -similar systems. In the Czech republic, the Czech Banking Association -standard #26, with short name SPAYD, uses QR codes with payment -information. More information is available from the Wikipedia page on -<a href="https://en.wikipedia.org/wiki/Short_Payment_Descriptor">Short -Payment Descriptor</a>. And in Germany, there is a system named -<a href="http://www.bezahlcode.de/">BezahlCode</a>, -(<a href="http://www.bezahlcode.de/wp-content/uploads/BezahlCode_TechDok.pdf">specification -v1.8 2013-12-05 available as PDF</a>), which uses QR codes with -URL-like formatting using "bank:" as the URI schema/protocol to -provide the payment information. There is also the -<a href="http://www.ferd-net.de/front_content.php?idcat=231">ZUGFeRD</a> -file format that perhaps could be transfered using QR codes, but I am -not sure if it is done already. Last, in Bolivia there are reports -that tax information since november 2014 need to be printed in QR -format on invoices. I have not been able to track down a -specification for this format, because of my limited language skill -sets.</p> + Når nynorskoversettelsen svikter til eksamen... + http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html + http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html + Sat, 3 Jun 2017 08:20:00 +0200 + <p><a href="http://www.aftenposten.no/norge/Krever-at-elever-ma-fa-annullert-eksamen-etter-rot-med-oppgavetekster-622459b.html">Aftenposten +melder i dag</a> om feil i eksamensoppgavene for eksamen i politikk og +menneskerettigheter, der teksten i bokmåls og nynorskutgaven ikke var +like. Oppgaveteksten er gjengitt i artikkelen, og jeg ble nysgjerring +på om den fri oversetterløsningen +<a href="https://www.apertium.org/">Apertium</a> ville gjort en bedre +jobb enn Utdanningsdirektoratet. Det kan se slik ut.</p> + +<p>Her er bokmålsoppgaven fra eksamenen:</p> + +<blockquote> +<p>Drøft utfordringene knyttet til nasjonalstatenes og andre aktørers +rolle og muligheter til å håndtere internasjonale utfordringer, som +for eksempel flykningekrisen.</p> + +<p>Vedlegge er eksempler på tekster som kan gi relevante perspektiver +på temaet:</p> +<ol> +<li>Flykningeregnskapet 2016, UNHCR og IDMC +<li>«Grenseløst Europa for fall» A-Magasinet, 26. november 2015 +</ol> + +</blockquote> + +<p>Dette oversetter Apertium slik:</p> + +<blockquote> +<p>Drøft utfordringane knytte til nasjonalstatane sine og rolla til +andre aktørar og høve til å handtera internasjonale utfordringar, som +til dømes *flykningekrisen.</p> + +<p>Vedleggja er døme på tekster som kan gje relevante perspektiv på +temaet:</p> + +<ol> +<li>*Flykningeregnskapet 2016, *UNHCR og *IDMC</li> +<li>«*Grenseløst Europa for fall» A-Magasinet, 26. november 2015</li> +</ol> + +</blockquote> + +<p>Ord som ikke ble forstått er markert med stjerne (*), og trenger +ekstra språksjekk. Men ingen ord er forsvunnet, slik det var i +oppgaven elevene fikk presentert på eksamen. Jeg mistenker dog at +"andre aktørers rolle og muligheter til ..." burde vært oversatt til +"rolla til andre aktørar og deira høve til ..." eller noe slikt, men +det er kanskje flisespikking. Det understreker vel bare at det alltid +trengs korrekturlesning etter automatisk oversettelse.</p> - Making battery measurements a little easier in Debian - http://people.skolelinux.org/pere/blog/Making_battery_measurements_a_little_easier_in_Debian.html - http://people.skolelinux.org/pere/blog/Making_battery_measurements_a_little_easier_in_Debian.html - Tue, 15 Mar 2016 15:00:00 +0100 - <p>Back in September, I blogged about -<a href="http://people.skolelinux.org/pere/blog/The_life_and_death_of_a_laptop_battery.html">the -system I wrote to collect statistics about my laptop battery</a>, and -how it showed the decay and death of this battery (now replaced). I -created a simple deb package to handle the collection and graphing, -but did not want to upload it to Debian as there were already -<a href="https://tracker.debian.org/pkg/battery-stats">a battery-stats -package in Debian</a> that should do the same thing, and I did not see -a point of uploading a competing package when battery-stats could be -fixed instead. I reported a few bugs about its non-function, and -hoped someone would step in and fix it. But no-one did.</p> - -<p>I got tired of waiting a few days ago, and took matters in my own -hands. The end result is that I am now the new upstream developer of -battery stats (<a href="https://github.com/petterreinholdtsen/battery-stats">available from github</a>) and part of the team maintaining -battery-stats in Debian, and the package in Debian unstable is finally -able to collect battery status using the <tt>/sys/class/power_supply/</tt> -information provided by the Linux kernel. If you install the -battery-stats package from unstable now, you will be able to get a -graph of the current battery fill level, to get some idea about the -status of the battery. The source package build and work just fine in -Debian testing and stable (and probably oldstable too, but I have not -tested). The default graph you get for that system look like this:</p> - -<p align="center"><img src="http://people.skolelinux.org/pere/blog/images/2016-03-15-battery-stats-graph-example.png" width="70%" align="center"></p> - -<p>My plans for the future is to merge my old scripts into the -battery-stats package, as my old scripts collected a lot more details -about the battery. The scripts are merged into the upstream -battery-stats git repository already, but I am not convinced they work -yet, as I changed a lot of paths along the way. Will have to test a -bit more before I make a new release.</p> - -<p>I will also consider changing the file format slightly, as I -suspect the way I combine several values into one field might make it -impossible to know the type of the value when using it for processing -and graphing.</p> - -<p>If you would like I would like to keep an close eye on your laptop -battery, check out the battery-stats package in -<a href="https://tracker.debian.org/pkg/battery-stats">Debian</a> and -on -<a href="https://github.com/petterreinholdtsen/battery-stats">github</a>. -I would love some help to improve the system further.</p> + Epost inn som arkivformat i Riksarkivarens forskrift? + http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html + http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html + Thu, 27 Apr 2017 11:30:00 +0200 + <p>I disse dager, med frist 1. mai, har Riksarkivaren ute en høring på +sin forskrift. Som en kan se er det ikke mye tid igjen før fristen +som går ut på søndag. Denne forskriften er det som lister opp hvilke +formater det er greit å arkivere i +<a href="http://www.arkivverket.no/arkivverket/Offentleg-forvalting/Noark/Noark-5">Noark +5-løsninger</a> i Norge.</p> + +<p>Jeg fant høringsdokumentene hos +<a href="https://www.arkivrad.no/aktuelt/riksarkivarens-forskrift-pa-horing">Norsk +Arkivråd</a> etter å ha blitt tipset på epostlisten til +<a href="https://github.com/hiOA-ABI/nikita-noark5-core">fri +programvareprosjektet Nikita Noark5-Core</a>, som lager et Noark 5 +Tjenestegresesnitt. Jeg er involvert i Nikita-prosjektet og takket +være min interesse for tjenestegrensesnittsprosjektet har jeg lest en +god del Noark 5-relaterte dokumenter, og til min overraskelse oppdaget +at standard epost ikke er på listen over godkjente formater som kan +arkiveres. Høringen med frist søndag er en glimrende mulighet til å +forsøke å gjøre noe med det. Jeg holder på med +<a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/hoering-arkivforskrift.tex">egen +høringsuttalelse</a>, og lurer på om andre er interessert i å støtte +forslaget om å tillate arkivering av epost som epost i arkivet.</p> + +<p>Er du igang med å skrive egen høringsuttalelse allerede? I så fall +kan du jo vurdere å ta med en formulering om epost-lagring. Jeg tror +ikke det trengs så mye. Her et kort forslag til tekst:</p> + +<p><blockquote> + + <p>Viser til høring sendt ut 2017-02-17 (Riksarkivarens referanse + 2016/9840 HELHJO), og tillater oss å sende inn noen innspill om + revisjon av Forskrift om utfyllende tekniske og arkivfaglige + bestemmelser om behandling av offentlige arkiver (Riksarkivarens + forskrift).</p> + + <p>Svært mye av vår kommuikasjon foregår i dag på e-post.  Vi + foreslår derfor at Internett-e-post, slik det er beskrevet i IETF + RFC 5322, + <a href="https://tools.ietf.org/html/rfc5322">https://tools.ietf.org/html/rfc5322</a>. bør + inn som godkjent dokumentformat.  Vi foreslår at forskriftens + oversikt over godkjente dokumentformater ved innlevering i § 5-16 + endres til å ta med Internett-e-post.</p> + +</blockquote></p> + +<p>Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan +epost kan lagres i en Noark 5-struktur, og holder på å skrive et +forslag om hvordan dette kan gjøres som vil bli sendt over til +arkivverket så snart det er ferdig. De som er interesserte kan +<a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/epostlagring.md">følge +fremdriften på web</a>.</p> + +<p>Oppdatering 2017-04-28: I dag ble høringuttalelsen jeg skrev + <a href="https://www.nuug.no/news/NUUGs_h_ringuttalelse_til_Riksarkivarens_forskrift.shtml">sendt + inn av foreningen NUUG</a>.</p> - Creating, updating and checking debian/copyright semi-automatically - http://people.skolelinux.org/pere/blog/Creating__updating_and_checking_debian_copyright_semi_automatically.html - http://people.skolelinux.org/pere/blog/Creating__updating_and_checking_debian_copyright_semi_automatically.html - Fri, 19 Feb 2016 15:00:00 +0100 - <p>Making packages for Debian requires quite a lot of attention to -details. And one of the details is the content of the -debian/copyright file, which should list all relevant licenses used by -the code in the package in question, preferably in -<a href="https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/">machine -readable DEP5 format</a>.</p> - -<p>For large packages with lots of contributors it is hard to write -and update this file manually, and if you get some detail wrong, the -package is normally rejected by the ftpmasters. So getting it right -the first time around get the package into Debian faster, and save -both you and the ftpmasters some work.. Today, while trying to figure -out what was wrong with -<a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=686447">the -zfsonlinux copyright file</a>, I decided to spend some time on -figuring out the options for doing this job automatically, or at least -semi-automatically.</p> - -<p>Lucikly, there are at least two tools available for generating the -file based on the code in the source package, -<tt><a href="https://tracker.debian.org/pkg/debmake">debmake</a></tt> -and <tt><a href="https://tracker.debian.org/pkg/cme">cme</a></tt>. I'm -not sure which one of them came first, but both seem to be able to -create a sensible draft file. As far as I can tell, none of them can -be trusted to get the result just right, so the content need to be -polished a bit before the file is OK to upload. I found the debmake -option in -<a href="http://goofying-with-debian.blogspot.com/2014/07/debmake-checking-source-against-dep-5.html">a -blog posts from 2014</a>. - -<p>To generate using debmake, use the -cc option: - -<p><pre> -debmake -cc > debian/copyright -</pre></p> - -<p>Note there are some problems with python and non-ASCII names, so -this might not be the best option.</p> - -<p>The cme option is based on a config parsing library, and I found -this approach in -<a href="https://ddumont.wordpress.com/2015/04/05/improving-creation-of-debian-copyright-file/">a -blog post from 2015</a>. To generate using cme, use the 'update -dpkg-copyright' option: - -<p><pre> -cme update dpkg-copyright -</pre></p> - -<p>This will create or update debian/copyright. The cme tool seem to -handle UTF-8 names better than debmake.</p> - -<p>When the copyright file is created, I would also like some help to -check if the file is correct. For this I found two good options, -<tt>debmake -k</tt> and <tt>license-reconcile</tt>. The former seem -to focus on license types and file matching, and is able to detect -ineffective blocks in the copyright file. The latter reports missing -copyright holders and years, but was confused by inconsistent license -names (like CDDL vs. CDDL-1.0). I suspect it is good to use both and -fix all issues reported by them before uploading. But I do not know -if the tools and the ftpmasters agree on what is important to fix in a -copyright file, so the package might still be rejected.</p> - -<p>The devscripts tool <tt>licensecheck</tt> deserve mentioning. It -will read through the source and try to find all copyright statements. -It is not comparing the result to the content of debian/copyright, but -can be useful when verifying the content of the copyright file.</p> - -<p>Are you aware of better tools in Debian to create and update -debian/copyright file. Please let me know, or blog about it on -planet.debian.org.</p> - -<p>As usual, if you use Bitcoin and want to show your support of my -activities, please send Bitcoin donations to my address -<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p> - -<p><strong>Update 2016-02-20</strong>: I got a tip from Mike Gabriel -on how to use licensecheck and cdbs to create a draft copyright file - -<p><pre> -licensecheck --copyright -r `find * -type f` | \ - /usr/lib/cdbs/licensecheck2dep5 > debian/copyright.auto -</pre></p> - -<p>He mentioned that he normally check the generated file into the -version control system to make it easier to discover license and -copyright changes in the upstream source. I will try to do the same -with my packages in the future.</p> - -<p><strong>Update 2016-02-21</strong>: The cme author recommended -against using -quiet for new users, so I removed it from the proposed -command line.</p> - - - - - Using appstream in Debian to locate packages with firmware and mime type support - http://people.skolelinux.org/pere/blog/Using_appstream_in_Debian_to_locate_packages_with_firmware_and_mime_type_support.html - http://people.skolelinux.org/pere/blog/Using_appstream_in_Debian_to_locate_packages_with_firmware_and_mime_type_support.html - Thu, 4 Feb 2016 16:40:00 +0100 - <p>The <a href="https://wiki.debian.org/DEP-11">appstream system</a> -is taking shape in Debian, and one provided feature is a very -convenient way to tell you which package to install to make a given -firmware file available when the kernel is looking for it. This can -be done using apt-file too, but that is for someone else to blog -about. :)</p> - -<p>Here is a small recipe to find the package with a given firmware -file, in this example I am looking for ctfw-3.2.3.0.bin, randomly -picked from the set of firmware announced using appstream in Debian -unstable. In general you would be looking for the firmware requested -by the kernel during kernel module loading. To find the package -providing the example file, do like this:</p> - -<blockquote><pre> -% apt install appstream -[...] -% apt update -[...] -% appstreamcli what-provides firmware:runtime ctfw-3.2.3.0.bin | \ - awk '/Package:/ {print $2}' -firmware-qlogic -% -</pre></blockquote> - -<p>See <a href="https://wiki.debian.org/AppStream/Guidelines">the -appstream wiki</a> page to learn how to embed the package metadata in -a way appstream can use.</p> - -<p>This same approach can be used to find any package supporting a -given MIME type. This is very useful when you get a file you do not -know how to handle. First find the mime type using <tt>file ---mime-type</tt>, and next look up the package providing support for -it. Lets say you got an SVG file. Its MIME type is image/svg+xml, -and you can find all packages handling this type like this:</p> + Offentlig elektronisk postjournal blokkerer tilgang for utvalgte webklienter + http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html + http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html + Thu, 20 Apr 2017 13:00:00 +0200 + <p>Jeg oppdaget i dag at <a href="https://www.oep.no/">nettstedet som +publiserer offentlige postjournaler fra statlige etater</a>, OEP, har +begynt å blokkerer enkelte typer webklienter fra å få tilgang. Vet +ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl +og curl. For å teste selv, kjør følgende:</p> <blockquote><pre> -% apt install appstream -[...] -% apt update -[...] -% appstreamcli what-provides mimetype image/svg+xml | \ - awk '/Package:/ {print $2}' -bkchem -phototonic -inkscape -shutter -tetzle -geeqie -xia -pinta -gthumb -karbon -comix -mirage -viewnior -postr -ristretto -kolourpaint4 -eog -eom -gimagereader -midori +% curl -v -s https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP' +< HTTP/1.1 404 Not Found +% curl -v -s --header 'User-Agent:Opera/12.0' https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP' +< HTTP/1.1 200 OK % </pre></blockquote> -<p>I believe the MIME types are fetched from the desktop file for -packages providing appstream metadata.</p> +<p>Her kan en se at tjenesten gir «404 Not Found» for curl i +standardoppsettet, mens den gir «200 OK» hvis curl hevder å være Opera +versjon 12.0. Offentlig elektronisk postjournal startet blokkeringen +2017-03-02.</p> + +<p>Blokkeringen vil gjøre det litt vanskeligere å maskinelt hente +informasjon fra oep.no. Kan blokkeringen være gjort for å hindre +automatisert innsamling av informasjon fra OEP, slik Pressens +Offentlighetsutvalg gjorde for å dokumentere hvordan departementene +hindrer innsyn i +<a href="http://presse.no/dette-mener-np/undergraver-offentlighetsloven/">rapporten +«Slik hindrer departementer innsyn» som ble publiserte i januar +2017</a>. Det virker usannsynlig, da det jo er trivielt å bytte +User-Agent til noe nytt.</p> + +<p>Finnes det juridisk grunnlag for det offentlige å diskriminere +webklienter slik det gjøres her? Der tilgang gis eller ikke alt etter +hva klienten sier at den heter? Da OEP eies av DIFI og driftes av +Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to +aktørene man kan be om innsyn i for å forstå hva som har skjedd. Men +<a href="https://www.oep.no/search/result.html?period=dateRange&fromDate=01.01.2016&toDate=01.04.2017&dateType=documentDate&caseDescription=&descType=both&caseNumber=&documentNumber=&sender=basefarm&senderType=both&documentType=all&legalAuthority=&archiveCode=&list2=196&searchType=advanced&Search=Search+in+records">postjournalen +til DIFI viser kun to dokumenter</a> det siste året mellom DIFI og +Basefarm. +<a href="https://www.mimesbronn.no/request/blokkering_av_tilgang_til_oep_fo">Mimes brønn neste</a>, +tenker jeg.</p> - Creepy, visualise geotagged social media information - nice free software - http://people.skolelinux.org/pere/blog/Creepy__visualise_geotagged_social_media_information___nice_free_software.html - http://people.skolelinux.org/pere/blog/Creepy__visualise_geotagged_social_media_information___nice_free_software.html - Sun, 24 Jan 2016 10:50:00 +0100 - <p>Most people seem not to realise that every time they walk around -with the computerised radio beacon known as a mobile phone their -position is tracked by the phone company and often stored for a long -time (like every time a SMS is received or sent). And if their -computerised radio beacon is capable of running programs (often called -mobile apps) downloaded from the Internet, these programs are often -also capable of tracking their location (if the app requested access -during installation). And when these programs send out information to -central collection points, the location is often included, unless -extra care is taken to not send the location. The provided -information is used by several entities, for good and bad (what is -good and bad, depend on your point of view). What is certain, is that -the private sphere and the right to free movement is challenged and -perhaps even eradicated for those announcing their location this way, -when they share their whereabouts with private and public -entities.</p> - -<p align="center"><img width="70%" src="http://people.skolelinux.org/pere/blog/images/2016-01-24-nice-creepy-desktop-window.png"></p> - -<p>The phone company logs provide a register of locations to check out -when one want to figure out what the tracked person was doing. It is -unavailable for most of us, but provided to selected government -officials, company staff, those illegally buying information from -unfaithful servants and crackers stealing the information. But the -public information can be collected and analysed, and a free software -tool to do so is called -<a href="http://www.geocreepy.com/">Creepy or Cree.py</a>. I -discovered it when I read -<a href="http://www.aftenposten.no/kultur/Slik-kan-du-bli-overvaket-pa-Twitter-og-Instagram-uten-a-ane-det-7787884.html">an -article about Creepy</a> in the Norwegian newspaper Aftenposten i -November 2014, and decided to check if it was available in Debian. -The python program was in Debian, but -<a href="https://tracker.debian.org/pkg/creepy">the version in -Debian</a> was completely broken and practically unmaintained. I -uploaded a new version which did not work quite right, but did not -have time to fix it then. This Christmas I decided to finally try to -get Creepy operational in Debian. Now a fixed version is available in -Debian unstable and testing, and almost all Debian specific patches -are now included -<a href="https://github.com/jkakavas/creepy">upstream</a>.</p> - -<p>The Creepy program visualises geolocation information fetched from -Twitter, Instagram, Flickr and Google+, and allow one to get a -complete picture of every social media message posted recently in a -given area, or track the movement of a given individual across all -these services. Earlier it was possible to use the search API of at -least some of these services without identifying oneself, but these -days it is impossible. This mean that to use Creepy, you need to -configure it to log in as yourself on these services, and provide -information to them about your search interests. This should be taken -into account when using Creepy, as it will also share information -about yourself with the services.</p> - -<p>The picture above show the twitter messages sent from (or at least -geotagged with a position from) the city centre of Oslo, the capital -of Norway. One useful way to use Creepy is to first look at -information tagged with an area of interest, and next look at all the -information provided by one or more individuals who was in the area. -I tested it by checking out which celebrity provide their location in -twitter messages by checkout out who sent twitter messages near a -Norwegian TV station, and next could track their position over time, -making it possible to locate their home and work place, among other -things. A similar technique have been -<a href="http://www.buzzfeed.com/maxseddon/does-this-soldiers-instagram-account-prove-russia-is-covertl">used -to locate Russian soldiers in Ukraine</a>, and it is both a powerful -tool to discover lying governments, and a useful tool to help people -understand the value of the private information they provide to the -public.</p> - -<p>The package is not trivial to backport to Debian Stable/Jessie, as -it depend on several python modules currently missing in Jessie (at -least python-instagram, python-flickrapi and -python-requests-toolbelt).</p> - -<p>(I have uploaded -<a href="https://screenshots.debian.net/package/creepy">the image to -screenshots.debian.net</a> and licensed it under the same terms as the -Creepy program in Debian.)</p> + Free software archive system Nikita now able to store documents + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + Sun, 19 Mar 2017 08:00:00 +0100 + <p>The <a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita +Noark 5 core project</a> is implementing the Norwegian standard for +keeping an electronic archive of government documents. +<a href="http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version">The +Noark 5 standard</a> document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +<a href="https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml">announced +it supported the project</a>. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches.</p> + +<p>If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(<a href="irc://irc.freenode.net/%23nikita"">#nikita on +irc.freenode.net</a>) and +<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">the +project mailing list</a>.</p> + +<p>When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +<tt>archive-pdf</tt> to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +<a href="https://en.wikipedia.org/wiki/Fonds">fonds</a>, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester:</p> + +<p><blockquote><pre> +~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf +using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446 +using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446 + + 0 - Title of the test case file created 2017-03-18T23:49:32.103446 + 1 - Title of the test file created 2017-03-18T23:49:32.103446 +Select which mappe you want (or search term): 0 +Uploading mangelmelding/mangler.pdf + PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt + File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446 +~/src//noark5-tester$ +</pre></blockquote></p> + +<p>You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The <tt>archive-pdf</tt> +tool can be found in the git repository for the API tester.</p> + +<p>In the project, I have been mostly working on +<a href="https://github.com/petterreinholdtsen/noark5-tester">the API +tester</a> so far, while getting to know the code base. The API +tester currently use +<a href="https://en.wikipedia.org/wiki/HATEOAS">the HATEOAS links</a> +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification.</p> + +<p>The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +<a href="https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding">started +writing down</a> the questions we get from implementing it. We use a +format inspired by how <a href="http://www.opengroup.org/austin/">The +Austin Group</a> collect defect reports for the POSIX standard with +<a href="http://www.opengroup.org/austin/mantis.html">their +instructions for the MANTIS defect tracker system</a>, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a <a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md">request for a procedure for submitting defect reports</a> :). + +<p>The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python.</p> - Always download Debian packages using Tor - the simple recipe - http://people.skolelinux.org/pere/blog/Always_download_Debian_packages_using_Tor___the_simple_recipe.html - http://people.skolelinux.org/pere/blog/Always_download_Debian_packages_using_Tor___the_simple_recipe.html - Fri, 15 Jan 2016 00:30:00 +0100 - <p>During his DebConf15 keynote, Jacob Appelbaum -<a href="https://summit.debconf.org/debconf15/meeting/331/what-is-to-be-done/">observed -that those listening on the Internet lines would have good reason to -believe a computer have a given security hole</a> if it download a -security fix from a Debian mirror. This is a good reason to always -use encrypted connections to the Debian mirror, to make sure those -listening do not know which IP address to attack. In August, Richard -Hartmann observed that encryption was not enough, when it was possible -to interfere download size to security patches or the fact that -download took place shortly after a security fix was released, and -<a href="http://richardhartmann.de/blog/posts/2015/08/24-Tor-enabled_Debian_mirror/">proposed -to always use Tor to download packages from the Debian mirror</a>. He -was not the first to propose this, as the -<tt><a href="https://tracker.debian.org/pkg/apt-transport-tor">apt-transport-tor</a></tt> -package by Tim Retout already existed to make it easy to convince apt -to use <a href="https://www.torproject.org/">Tor</a>, but I was not -aware of that package when I read the blog post from Richard.</p> - -<p>Richard discussed the idea with Peter Palfrader, one of the Debian -sysadmins, and he set up a Tor hidden service on one of the central -Debian mirrors using the address vwakviie2ienjx6t.onion, thus making -it possible to download packages directly between two tor nodes, -making sure the network traffic always were encrypted.</p> - -<p>Here is a short recipe for enabling this on your machine, by -installing <tt>apt-transport-tor</tt> and replacing http and https -urls with tor+http and tor+https, and using the hidden service instead -of the official Debian mirror site. I recommend installing -<tt>etckeeper</tt> before you start to have a history of the changes -done in /etc/.</p> - -<blockquote><pre> -apt install apt-transport-tor -sed -i 's% http://ftp.debian.org/% tor+http://vwakviie2ienjx6t.onion/%' /etc/apt/sources.list -sed -i 's% http% tor+http%' /etc/apt/sources.list -</pre></blockquote> - -<p>If you have more sources listed in /etc/apt/sources.list.d/, run -the sed commands for these too. The sed command is assuming your are -using the ftp.debian.org Debian mirror. Adjust the command (or just -edit the file manually) to match your mirror.</p> - -<p>This work in Debian Jessie and later. Note that tools like -<tt>apt-file</tt> only recently started using the apt transport -system, and do not work with these tor+http URLs. For -<tt>apt-file</tt> you need the version currently in experimental, -which need a recent apt version currently only in unstable. So if you -need a working <tt>apt-file</tt>, this is not for you.</p> - -<p>Another advantage from this change is that your machine will start -using Tor regularly and at fairly random intervals (every time you -update the package lists or upgrade or install a new package), thus -masking other Tor traffic done from the same machine. Using Tor will -become normal for the machine in question.</p> - -<p>On <a href="https://wiki.debian.org/FreedomBox">Freedombox</a>, APT -is set up by default to use <tt>apt-transport-tor</tt> when Tor is -enabled. It would be great if it was the default on any Debian -system.</p> + Detecting NFS hangs on Linux without hanging yourself... + http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html + http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html + Thu, 9 Mar 2017 15:20:00 +0100 + <p>Over the years, administrating thousand of NFS mounting linux +computers at the time, I often needed a way to detect if the machine +was experiencing NFS hang. If you try to use <tt>df</tt> or look at a +file or directory affected by the hang, the process (and possibly the +shell) will hang too. So you want to be able to detect this without +risking the detection process getting stuck too. It has not been +obvious how to do this. When the hang has lasted a while, it is +possible to find messages like these in dmesg:</p> + +<p><blockquote> +nfs: server nfsserver not responding, still trying +<br>nfs: server nfsserver OK +</blockquote></p> + +<p>It is hard to know if the hang is still going on, and it is hard to +be sure looking in dmesg is going to work. If there are lots of other +messages in dmesg the lines might have rotated out of site before they +are noticed.</p> + +<p>While reading through the nfs client implementation in linux kernel +code, I came across some statistics that seem to give a way to detect +it. The om_timeouts sunrpc value in the kernel will increase every +time the above log entry is inserted into dmesg. And after digging a +bit further, I discovered that this value show up in +/proc/self/mountstats on Linux.</p> + +<p>The mountstats content seem to be shared between files using the +same file system context, so it is enough to check one of the +mountstats files to get the state of the mount point for the machine. +I assume this will not show lazy umounted NFS points, nor NFS mount +points in a different process context (ie with a different filesystem +view), but that does not worry me.</p> + +<p>The content for a NFS mount point look similar to this:</p> + +<p><blockquote><pre> +[...] +device /dev/mapper/Debian-var mounted on /var with fstype ext3 +device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1 + opts: rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all + age: 7863311 + caps: caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255 + sec: flavor=1,pseudoflavor=1 + events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0 + bytes: 166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809 + RPC iostats version: 1.0 p/v: 100003/3 (nfs) + xprt: tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820 + per-op statistics + NULL: 0 0 0 0 0 0 0 0 + GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132 + SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943 + LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511 + ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140 + READLINK: 125 125 0 20472 18620 0 1112 1118 + READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693 + WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771 + CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398 + MKDIR: 3680 3680 0 773980 993920 26 23990 24245 + SYMLINK: 903 903 0 233428 245488 6 5865 5917 + MKNOD: 80 80 0 20148 21760 0 299 304 + REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636 + RMDIR: 3367 3367 0 645112 484848 22 5782 6002 + RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288 + LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579 + READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917 + READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937 + FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022 + FSINFO: 2 2 0 232 328 0 1 1 + PATHCONF: 1 1 0 116 140 0 0 0 + COMMIT: 0 0 0 0 0 0 0 0 + +device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc +[...] +</pre></blockquote></p> + +<p>The key number to look at is the third number in the per-op list. +It is the number of NFS timeouts experiences per file system +operation. Here 22 write timeouts and 5 access timeouts. If these +numbers are increasing, I believe the machine is experiencing NFS +hang. Unfortunately the timeout value do not start to increase right +away. The NFS operations need to time out first, and this can take a +while. The exact timeout value depend on the setup. For example the +defaults for TCP and UDP mount points are quite different, and the +timeout value is affected by the soft, hard, timeo and retrans NFS +mount options.</p> + +<p>The only way I have been able to get working on Debian and RedHat +Enterprise Linux for getting the timeout count is to peek in /proc/. +But according to +<ahref="http://docs.oracle.com/cd/E19253-01/816-4555/netmonitor-12/index.html">Solaris +10 System Administration Guide: Network Services</a>, the 'nfsstat -c' +command can be used to get these timeout values. But this do not work +on Linux, as far as I can tell. I +<ahref="http://bugs.debian.org/857043">asked Debian about this</a>, +but have not seen any replies yet.</p> + +<p>Is there a better way to figure out if a Linux NFS client is +experiencing NFS hangs? Is there a way to detect which processes are +affected? Is there a way to get the NFS mount going quickly once the +network problem causing the NFS hang has been cleared? I would very +much welcome some clues, as we regularly run into NFS hangs.</p> - Nedlasting fra NRK, som Matroska med undertekster - http://people.skolelinux.org/pere/blog/Nedlasting_fra_NRK__som_Matroska_med_undertekster.html - http://people.skolelinux.org/pere/blog/Nedlasting_fra_NRK__som_Matroska_med_undertekster.html - Sat, 2 Jan 2016 13:50:00 +0100 - <p>Det kommer stadig nye løsninger for å ta lagre unna innslag fra NRK -for å se på det senere. For en stund tilbake kom jeg over et script -nrkopptak laget av Ingvar Hagelund. Han fjernet riktignok sitt script -etter forespørsel fra Erik Bolstad i NRK, men noen tok heldigvis og -gjorde det <a href="https://github.com/liangqi/nrkopptak">tilgjengelig -via github</a>.</p> - -<p>Scriptet kan lagre som MPEG4 eller Matroska, og bake inn -undertekster i fila på et vis som blant annet VLC forstår. For å -bruke scriptet, kopier ned git-arkivet og kjør</p> - -<p><pre> -nrkopptak/bin/nrk-opptak k <ahref="https://tv.nrk.no/serie/bmi-turne/MUHH45000115/sesong-1/episode-1">https://tv.nrk.no/serie/bmi-turne/MUHH45000115/sesong-1/episode-1</a> -</pre></p> - -<p>URL-eksemplet er dagens toppsak på tv.nrk.no. Argument 'k' ber -scriptet laste ned og lagre som Matroska. Det finnes en rekke andre -muligheter for valg av kvalitet og format.</p> - -<p>Jeg foretrekker dette scriptet fremfor youtube-dl, som -<a href="http://people.skolelinux.org/pere/blog/Hvordan_enkelt_laste_ned_filmer_fra_NRK_med_den__nye__l_sningen.html"> -nevnt i 2014 støtter NRK</a> og en rekke andre videokilder, på grunn -av at nrkopptak samler undertekster og video i en enkelt fil, hvilket -gjør håndtering enklere på disk.</p> + How does it feel to be wiretapped, when you should be doing the wiretapping... + http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html + http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html + Wed, 8 Mar 2017 11:50:00 +0100 + <p>So the new president in the United States of America claim to be +surprised to discover that he was wiretapped during the election +before he was elected president. He even claim this must be illegal. +Well, doh, if it is one thing the confirmations from Snowden +documented, it is that the entire population in USA is wiretapped, one +way or another. Of course the president candidates were wiretapped, +alongside the senators, judges and the rest of the people in USA.</p> + +<p>Next, the Federal Bureau of Investigation ask the Department of +Justice to go public rejecting the claims that Donald Trump was +wiretapped illegally. I fail to see the relevance, given that I am +sure the surveillance industry in USA believe they have all the legal +backing they need to conduct mass surveillance on the entire +world.</p> + +<p>There is even the director of the FBI stating that he never saw an +order requesting wiretapping of Donald Trump. That is not very +surprising, given how the FISA court work, with all its activity being +secret. Perhaps he only heard about it?</p> + +<p>What I find most sad in this story is how Norwegian journalists +present it. In a news reports the other day in the radio from the +Norwegian National broadcasting Company (NRK), I heard the journalist +claim that 'the FBI denies any wiretapping', while the reality is that +'the FBI denies any illegal wiretapping'. There is a fundamental and +important difference, and it make me sad that the journalists are +unable to grasp it.</p> + +<p><strong>Update 2017-03-13:</strong> Look like +<a href="https://theintercept.com/2017/03/13/rand-paul-is-right-nsa-routinely-monitors-americans-communications-without-warrants/">The +Intercept report that US Senator Rand Paul confirm what I state above</a>.</p> - OpenALPR, find car license plates in video streams - nice free software - http://people.skolelinux.org/pere/blog/OpenALPR__find_car_license_plates_in_video_streams___nice_free_software.html - http://people.skolelinux.org/pere/blog/OpenALPR__find_car_license_plates_in_video_streams___nice_free_software.html - Wed, 23 Dec 2015 01:00:00 +0100 - <p>When I was a kid, we used to collect "car numbers", as we used to -call the car license plate numbers in those days. I would write the -numbers down in my little book and compare notes with the other kids -to see how many region codes we had seen and if we had seen some -exotic or special region codes and numbers. It was a fun game to pass -time, as we kids have plenty of it.</p> - -<p>A few days I came across -<a href="https://github.com/openalpr/openalpr">the OpenALPR -project</a>, a free software project to automatically discover and -report license plates in images and video streams, and provide the -"car numbers" in a machine readable format. I've been looking for -such system for a while now, because I believe it is a bad idea that the -<a href="https://en.wikipedia.org/wiki/Automatic_number_plate_recognition">automatic -number plate recognition</a> tool only is available in the hands of -the powerful, and want it to be available also for the powerless to -even the score when it comes to surveillance and sousveillance. I -discovered the developer -<a href="https://bugs.debian.org/747509">wanted to get the tool into -Debian</a>, and as I too wanted it to be in Debian, I volunteered to -help him get it into shape to get the package uploaded into the Debian -archive.</p> - -<p>Today we finally managed to get the package into shape and uploaded -it into Debian, where it currently -<a href="https://ftp-master.debian.org//new/openalpr_2.2.1-1.html">waits -in the NEW queue</a> for review by the Debian ftpmasters.</p> - -<p>I guess you are wondering why on earth such tool would be useful -for the common folks, ie those not running a large government -surveillance system? Well, I plan to put it in a computer on my bike -and in my car, tracking the cars nearby and allowing me to be notified -when number plates on my watch list are discovered. Another use case -was suggested by a friend of mine, who wanted to set it up at his home -to open the car port automatically when it discovered the plate on his -car. When I mentioned it perhaps was a bit foolhardy to allow anyone -capable of placing his license plate number of a piece of cardboard to -open his car port, men replied that it was always unlocked anyway. I -guess for such use case it make sense. I am sure there are other use -cases too, for those with imagination and a vision.</p> - -<p>If you want to build your own version of the Debian package, check -out the upstream git source and symlink ./distros/debian to ./debian/ -before running "debuild" to build the source. Or wait a bit until the -package show up in unstable.</p> + Norwegian Bokmål translation of The Debian Administrator's Handbook complete, proofreading in progress + http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html + http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html + Fri, 3 Mar 2017 14:50:00 +0100 + <p>For almost a year now, we have been working on making a Norwegian +Bokmål edition of <a href="https://debian-handbook.info/">The Debian +Administrator's Handbook</a>. Now, thanks to the tireless effort of +Ole-Erik, Ingrid and Andreas, the initial translation is complete, and +we are working on the proof reading to ensure consistent language and +use of correct computer science terms. The plan is to make the book +available on paper, as well as in electronic form. For that to +happen, the proof reading must be completed and all the figures need +to be translated. If you want to help out, get in touch.</p> + +<p><a href="http://people.skolelinux.org/pere/debian-handbook/debian-handbook-nb-NO.pdf">A + +fresh PDF edition</a> in A4 format (the final book will have smaller +pages) of the book created every morning is available for +proofreading. If you find any errors, please +<a href="https://hosted.weblate.org/projects/debian-handbook/">visit +Weblate and correct the error</a>. The +<a href="http://l.github.io/debian-handbook/stat/nb-NO/index.html">state +of the translation including figures</a> is a useful source for those +provide Norwegian bokmål screen shots and figures.</p> - Using appstream with isenkram to install hardware related packages in Debian - http://people.skolelinux.org/pere/blog/Using_appstream_with_isenkram_to_install_hardware_related_packages_in_Debian.html - http://people.skolelinux.org/pere/blog/Using_appstream_with_isenkram_to_install_hardware_related_packages_in_Debian.html - Sun, 20 Dec 2015 12:20:00 +0100 - <p>Around three years ago, I created -<a href="http://packages.qa.debian.org/isenkram">the isenkram -system</a> to get a more practical solution in Debian for handing -hardware related packages. A GUI system in the isenkram package will -present a pop-up dialog when some hardware dongle supported by -relevant packages in Debian is inserted into the machine. The same -lookup mechanism to detect packages is available as command line -tools in the isenkram-cli package. In addition to mapping hardware, -it will also map kernel firmware files to packages and make it easy to -install needed firmware packages automatically. The key for this -system to work is a good way to map hardware to packages, in other -words, allow packages to announce what hardware they will work -with.</p> - -<p>I started by providing data files in the isenkram source, and -adding code to download the latest version of these data files at run -time, to ensure every user had the most up to date mapping available. -I also added support for storing the mapping in the Packages file in -the apt repositories, but did not push this approach because while I -was trying to figure out how to best store hardware/package mappings, -<a href="http://www.freedesktop.org/software/appstream/docs/">the -appstream system</a> was announced. I got in touch and suggested to -add the hardware mapping into that data set to be able to use -appstream as a data source, and this was accepted at least for the -Debian version of appstream.</p> - -<p>A few days ago using appstream in Debian for this became possible, -and today I uploaded a new version 0.20 of isenkram adding support for -appstream as a data source for mapping hardware to packages. The only -package so far using appstream to announce its hardware support is my -pymissile package. I got help from Matthias Klumpp with figuring out -how do add the required -<a href="https://appstream.debian.org/html/sid/main/metainfo/pymissile.html">metadata -in pymissile</a>. I added a file debian/pymissile.metainfo.xml with -this content:</p> + Unlimited randomness with the ChaosKey? + http://people.skolelinux.org/pere/blog/Unlimited_randomness_with_the_ChaosKey_.html + http://people.skolelinux.org/pere/blog/Unlimited_randomness_with_the_ChaosKey_.html + Wed, 1 Mar 2017 20:50:00 +0100 + <p>A few days ago I ordered a small batch of +<a href="http://altusmetrum.org/ChaosKey/">the ChaosKey</a>, a small +USB dongle for generating entropy created by Bdale Garbee and Keith +Packard. Yesterday it arrived, and I am very happy to report that it +work great! According to its designers, to get it to work out of the +box, you need the Linux kernel version 4.1 or later. I tested on a +Debian Stretch machine (kernel version 4.9), and there it worked just +fine, increasing the available entropy very quickly. I wrote a small +test oneliner to test. It first print the current entropy level, +drain /dev/random, and then print the entropy level for five seconds. +Here is the situation without the ChaosKey inserted:</p> <blockquote><pre> -&lt;?xml version="1.0" encoding="UTF-8"?&gt; -&lt;component&gt; - &lt;id&gt;pymissile&lt;/id&gt; - &lt;metadata_license&gt;MIT&lt;/metadata_license&gt; - &lt;name&gt;pymissile&lt;/name&gt; - &lt;summary&gt;Control original Striker USB Missile Launcher&lt;/summary&gt; - &lt;description&gt; - &lt;p&gt; - Pymissile provides a curses interface to control an original - Marks and Spencer / Striker USB Missile Launcher, as well as a - motion control script to allow a webcamera to control the - launcher. - &lt;/p&gt; - &lt;/description&gt; - &lt;provides&gt; - &lt;modalias&gt;usb:v1130p0202d*&lt;/modalias&gt; - &lt;/provides&gt; -&lt;/component&gt; +% cat /proc/sys/kernel/random/entropy_avail; \ + dd bs=1M if=/dev/random of=/dev/null count=1; \ + for n in $(seq 1 5); do \ + cat /proc/sys/kernel/random/entropy_avail; \ + sleep 1; \ + done +300 +0+1 oppføringer inn +0+1 oppføringer ut +28 byte kopiert, 0,000264565 s, 106 kB/s +4 +8 +12 +17 +21 +% </pre></blockquote> -<p>The key for isenkram is the component/provides/modalias value, -which is a glob style match rule for hardware specific strings -(modalias strings) provided by the Linux kernel. In this case, it -will map to all USB devices with vendor code 1130 and product code -0202.</p> - -<p>Note, it is important that the license of all the metadata files -are compatible to have permissions to aggregate them into archive wide -appstream files. Matthias suggested to use MIT or BSD licenses for -these files. A challenge is figuring out a good id for the data, as -it is supposed to be globally unique and shared across distributions -(in other words, best to coordinate with upstream what to use). But -it can be changed later or, so we went with the package name as -upstream for this project is dormant.</p> - -<p>To get the metadata file installed in the correct location for the -mirror update scripts to pick it up and include its content the -appstream data source, the file must be installed in the binary -package under /usr/share/appdata/. I did this by adding the following -line to debian/pymissile.install:</p> +<p>The entropy level increases by 3-4 every second. In such case any +application requiring random bits (like a HTTPS enabled web server) +will halt and wait for more entrpy. And here is the situation with +the ChaosKey inserted:</p> <blockquote><pre> -debian/pymissile.metainfo.xml usr/share/appdata +% cat /proc/sys/kernel/random/entropy_avail; \ + dd bs=1M if=/dev/random of=/dev/null count=1; \ + for n in $(seq 1 5); do \ + cat /proc/sys/kernel/random/entropy_avail; \ + sleep 1; \ + done +1079 +0+1 oppføringer inn +0+1 oppføringer ut +104 byte kopiert, 0,000487647 s, 213 kB/s +433 +1028 +1031 +1035 +1038 +% </pre></blockquote> -<p>With that in place, the command line tool isenkram-lookup will list -all packages useful on the current computer automatically, and the GUI -pop-up handler will propose to install the package not already -installed if a hardware dongle is inserted into the machine in -question.</p> - -<p>Details of the modalias field in appstream is available from the -<a href="https://wiki.debian.org/DEP-11">DEP-11</a> proposal.</p> - -<p>To locate the modalias values of all hardware present in a machine, -try running this command on the command line:</p> +<p>Quite the difference. :) I bought a few more than I need, in case +someone want to buy one here in Norway. :)</p> -<blockquote><pre> -cat $(find /sys/devices/|grep modalias) -</pre></blockquote> - -<p>To learn more about the isenkram system, please check out -<a href="http://people.skolelinux.org/pere/blog/tags/isenkram/">my -blog posts tagged isenkram</a>.</p> +<p>Update: The dongle was presented at Debconf last year. You might +find <a href="https://debconf16.debconf.org/talks/94/">the talk +recording illuminating</a>. It explains exactly what the source of +randomness is, if you are unable to spot it from the schema drawing +available from the ChaosKey web site linked at the start of this blog +post.</p> + + + + + Detect OOXML files with undefined behaviour? + http://people.skolelinux.org/pere/blog/Detect_OOXML_files_with_undefined_behaviour_.html + http://people.skolelinux.org/pere/blog/Detect_OOXML_files_with_undefined_behaviour_.html + Tue, 21 Feb 2017 00:20:00 +0100 + <p>I just noticed +<a href="http://www.arkivrad.no/aktuelt/riksarkivarens-forskrift-pa-horing">the +new Norwegian proposal for archiving rules in the goverment</a> list +<a href="http://www.ecma-international.org/publications/standards/Ecma-376.htm">ECMA-376</a> +/ ISO/IEC 29500 (aka OOXML) as valid formats to put in long term +storage. Luckily such files will only be accepted based on +pre-approval from the National Archive. Allowing OOXML files to be +used for long term storage might seem like a good idea as long as we +forget that there are plenty of ways for a "valid" OOXML document to +have content with no defined interpretation in the standard, which +lead to a question and an idea.</p> + +<p>Is there any tool to detect if a OOXML document depend on such +undefined behaviour? It would be useful for the National Archive (and +anyone else interested in verifying that a document is well defined) +to have such tool available when considering to approve the use of +OOXML. I'm aware of the +<a href="https://github.com/arlm/officeotron/">officeotron OOXML +validator</a>, but do not know how complete it is nor if it will +report use of undefined behaviour. Are there other similar tools +available? Please send me an email if you know of any such tool.</p>