X-Git-Url: http://pere.pagekite.me/gitweb/homepage.git/blobdiff_plain/328c89afd117fbfefa611662267b9bb0d7582137..53798080ccfe7c0d9ad356a83ded5b219b186c9b:/blog/index.rss diff --git a/blog/index.rss b/blog/index.rss index 7ed2e67de6..0a657f587a 100644 --- a/blog/index.rss +++ b/blog/index.rss @@ -7,737 +7,664 @@ - Full battery stats collector is now available in Debian - http://people.skolelinux.org/pere/blog/Full_battery_stats_collector_is_now_available_in_Debian.html - http://people.skolelinux.org/pere/blog/Full_battery_stats_collector_is_now_available_in_Debian.html - Wed, 23 Mar 2016 22:10:00 +0100 - <p>Since this morning, the battery-stats package in Debian include an -extended collector that will collect the complete battery history for -later processing and graphing. The original collector store the -battery level as percentage of last full level, while the new -collector also record battery vendor, model, serial number, design -full level, last full level and current battery level. This make it -possible to predict the lifetime of the battery as well as visualise -the energy flow when the battery is charging or discharging.</p> - -<p>The new tools are available in <tt>/usr/share/battery-stats/</tt> -in the version 0.5.1 package in unstable. Get the new battery level graph -and lifetime prediction by running: - -<p><pre> -/usr/share/battery-stats/battery-stats-graph /var/log/battery-stats.csv -</pre></p> - -<p>Or select the 'Battery Level Graph' from your application menu.</p> - -<p>The flow in/out of the battery can be seen by running (no menu -entry yet):</p> - -<p><pre> -/usr/share/battery-stats/battery-stats-graph-flow -</pre></p> - -<p>I'm not quite happy with the way the data is visualised, at least -when there are few data points. The graphs look a bit better with a -few years of data.</p> - -<p>A while back one important feature I use in the battery stats -collector broke in Debian. The scripts in -<tt>/usr/lib/pm-utils/power.d/</tt> were no longer executed. I -suspect it happened when Jessie started using systemd, but I do not -know. The issue is reported as -<a href="https://bugs.debian.org/818649">bug #818649</a> against -pm-utils. I managed to work around it by adding an udev rule to call -the collector script every time the power connector is connected and -disconnected. With this fix in place it was finally time to make a -new release of the package, and get it into Debian.</p> - -<p>If you are interested in how your laptop battery is doing, please -check out the -<a href="https://tracker.debian.org/pkg/battery-stats">battery-stats</a> -in Debian unstable, or rebuild it on Jessie to get it working on -Debian stable. :) The upstream source is available from -<a href="https://github.com/petterreinholdtsen/battery-stats">github</a>. -As always, patches are very welcome.</p> - - - - - UsingQR - "Electronic" paper invoices using JSON and QR codes - http://people.skolelinux.org/pere/blog/UsingQR____Electronic__paper_invoices_using_JSON_and_QR_codes.html - http://people.skolelinux.org/pere/blog/UsingQR____Electronic__paper_invoices_using_JSON_and_QR_codes.html - Sat, 19 Mar 2016 09:40:00 +0100 - <p>Back in 2013 I proposed -<a href="http://people.skolelinux.org/pere/blog/_Electronic__paper_invoices___using_vCard_in_a_QR_code.html">a -way to make paper and PDF invoices easier to process electronically by -adding a QR code with the key information about the invoice</a>. I -suggested using vCard field definition, to get some standard format -for name and address, but any format would work. I did not do -anything about the proposal, but hoped someone one day would make -something like it. It would make it possible to efficiently send -machine readable invoices directly between seller and buyer.</p> - -<p>This was the background when I came across a proposal and -specification from the web based accounting and invoicing supplier -<a href="http://www.visma.com/">Visma</a> in Sweden called -<a href="http://usingqr.com/">UsingQR</a>. Their PDF invoices contain -a QR code with the key information of the invoice in JSON format. -This is the typical content of a QR code following the UsingQR -specification (based on a real world example, some numbers replaced to -get a more bogus entry). I've reformatted the JSON to make it easier -to read. Normally this is all on one long line:</p> - -<p><img src="http://people.skolelinux.org/pere/blog/images/2016-03-19-qr-invoice.png" align="right"><pre> -{ - "vh":500.00, - "vm":0, - "vl":0, - "uqr":1, - "tp":1, - "nme":"Din Leverandør", - "cc":"NO", - "cid":"997912345 MVA", - "iref":"12300001", - "idt":"20151022", - "ddt":"20151105", - "due":2500.0000, - "cur":"NOK", - "pt":"BBAN", - "acc":"17202612345", - "bc":"BIENNOK1", - "adr":"0313 OSLO" -} -</pre></p> - -</p>The interpretation of the fields can be found in the -<a href="http://usingqr.com/wp-content/uploads/2014/06/UsingQR_specification1.pdf">format -specification</a> (revision 2 from june 2014). The format seem to -have most of the information needed to handle accounting and payment -of invoices, at least the fields I have needed so far here in -Norway.</p> - -<p>Unfortunately, the site and document do not mention anything about -the patent, trademark and copyright status of the format and the -specification. Because of this, I asked the people behind it back in -November to clarify. Ann-Christine Savlid (ann-christine.savlid (at) -visma.com) replied that Visma had not applied for patent or trademark -protection for this format, and that there were no copyright based -usage limitations for the format. I urged her to make sure this was -explicitly written on the web pages and in the specification, but -unfortunately this has not happened yet. So I guess if there is -submarine patents, hidden trademarks or a will to sue for copyright -infringements, those starting to use the UsingQR format might be at -risk, but if this happen there is some legal defense in the fact that -the people behind the format claimed it was safe to do so. At least -with patents, there is always -<a href="http://www.paperspecs.com/paper-news/beware-the-qr-code-patent-trap/">a -chance of getting sued...</a></p> - -<p>I also asked if they planned to maintain the format in an -independent standard organization to give others more confidence that -they would participate in the standardization process on equal terms -with Visma, but they had no immediate plans for this. Their plan was -to work with banks to try to get more users of the format, and -evaluate the way forward if the format proved to be popular. I hope -they conclude that using an open standard organisation like -<a href="http://www.ietf.org/">IETF</a> is the correct place to -maintain such specification.</p> - -<p><strong>Update 2016-03-20</strong>: Via Twitter I became aware of -<a href="https://news.ycombinator.com/item?id=11319492">some comments -about this blog post</a> that had several useful links and references to -similar systems. In the Czech republic, the Czech Banking Association -standard #26, with short name SPAYD, uses QR codes with payment -information. More information is available from the Wikipedia page on -<a href="https://en.wikipedia.org/wiki/Short_Payment_Descriptor">Short -Payment Descriptor</a>. And in Germany, there is a system named -<a href="http://www.bezahlcode.de/">BezahlCode</a>, -(<a href="http://www.bezahlcode.de/wp-content/uploads/BezahlCode_TechDok.pdf">specification -v1.8 2013-12-05 available as PDF</a>), which uses QR codes with -URL-like formatting using "bank:" as the URI schema/protocol to -provide the payment information. There is also the -<a href="http://www.ferd-net.de/front_content.php?idcat=231">ZUGFeRD</a> -file format that perhaps could be transfered using QR codes, but I am -not sure if it is done already. Last, in Bolivia there are reports -that tax information since november 2014 need to be printed in QR -format on invoices. I have not been able to track down a -specification for this format, because of my limited language skill -sets.</p> + Detecting NFS hangs on Linux without hanging yourself... + http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html + http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html + Thu, 9 Mar 2017 15:20:00 +0100 + <p>Over the years, administrating thousand of NFS mounting linux +computers at the time, I often needed a way to detect if the machine +was experiencing NFS hang. If you try to use <tt>df</tt> or look at a +file or directory affected by the hang, the process (and possibly the +shell) will hang too. So you want to be able to detect this without +risking the detection process getting stuck too. It has not been +obvious how to do this. When the hang has lasted a while, it is +possible to find messages like these in dmesg:</p> + +<p><blockquote> +nfs: server nfsserver not responding, still trying +<br>nfs: server nfsserver OK +</blockquote></p> + +<p>It is hard to know if the hang is still going on, and it is hard to +be sure looking in dmesg is going to work. If there are lots of other +messages in dmesg the lines might have rotated out of site before they +are noticed.</p> + +<p>While reading through the nfs client implementation in linux kernel +code, I came across some statistics that seem to give a way to detect +it. The om_timeouts sunrpc value in the kernel will increase every +time the above log entry is inserted into dmesg. And after digging a +bit further, I discovered that this value show up in +/proc/self/mountstats on Linux.</p> + +<p>The mountstats content seem to be shared between files using the +same file system context, so it is enough to check one of the +mountstats files to get the state of the mount point for the machine. +I assume this will not show lazy umounted NFS points, nor NFS mount +points in a different process context (ie with a different filesystem +view), but that does not worry me.</p> + +<p>The content for a NFS mount point look similar to this:</p> + +<p><blockquote><pre> +[...] +device /dev/mapper/Debian-var mounted on /var with fstype ext3 +device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1 + opts: rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all + age: 7863311 + caps: caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255 + sec: flavor=1,pseudoflavor=1 + events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0 + bytes: 166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809 + RPC iostats version: 1.0 p/v: 100003/3 (nfs) + xprt: tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820 + per-op statistics + NULL: 0 0 0 0 0 0 0 0 + GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132 + SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943 + LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511 + ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140 + READLINK: 125 125 0 20472 18620 0 1112 1118 + READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693 + WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771 + CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398 + MKDIR: 3680 3680 0 773980 993920 26 23990 24245 + SYMLINK: 903 903 0 233428 245488 6 5865 5917 + MKNOD: 80 80 0 20148 21760 0 299 304 + REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636 + RMDIR: 3367 3367 0 645112 484848 22 5782 6002 + RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288 + LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579 + READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917 + READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937 + FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022 + FSINFO: 2 2 0 232 328 0 1 1 + PATHCONF: 1 1 0 116 140 0 0 0 + COMMIT: 0 0 0 0 0 0 0 0 + +device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc +[...] +</pre></blockquote></p> + +<p>The key number to look at is the third number in the per-op list. +It is the number of NFS timeouts experiences per file system +operation. Here 22 write timeouts and 5 access timeouts. If these +numbers are increasing, I believe the machine is experiencing NFS +hang. Unfortunately the timeout value do not start to increase right +away. The NFS operations need to time out first, and this can take a +while. The exact timeout value depend on the setup. For example the +defaults for TCP and UDP mount points are quite different, and the +timeout value is affected by the soft, hard, timeo and retrans NFS +mount options.</p> + +<p>The only way I have been able to get working on Debian and RedHat +Enterprise Linux for getting the timeout count is to peek in /proc/. +But according to +<ahref="http://docs.oracle.com/cd/E19253-01/816-4555/netmonitor-12/index.html">Solaris +10 System Administration Guide: Network Services</a>, the 'nfsstat -c' +command can be used to get these timeout values. But this do not work +on Linux, as far as I can tell. I +<ahref="http://bugs.debian.org/857043">asked Debian about this</a>, +but have not seen any replies yet.</p> + +<p>Is there a better way to figure out if a Linux NFS client is +experiencing NFS hangs? Is there a way to detect which processes are +affected? Is there a way to get the NFS mount going quickly once the +network problem causing the NFS hang has been cleared? I would very +much welcome some clues, as we regularly run into NFS hangs.</p> - Making battery measurements a little easier in Debian - http://people.skolelinux.org/pere/blog/Making_battery_measurements_a_little_easier_in_Debian.html - http://people.skolelinux.org/pere/blog/Making_battery_measurements_a_little_easier_in_Debian.html - Tue, 15 Mar 2016 15:00:00 +0100 - <p>Back in September, I blogged about -<a href="http://people.skolelinux.org/pere/blog/The_life_and_death_of_a_laptop_battery.html">the -system I wrote to collect statistics about my laptop battery</a>, and -how it showed the decay and death of this battery (now replaced). I -created a simple deb package to handle the collection and graphing, -but did not want to upload it to Debian as there were already -<a href="https://tracker.debian.org/pkg/battery-stats">a battery-stats -package in Debian</a> that should do the same thing, and I did not see -a point of uploading a competing package when battery-stats could be -fixed instead. I reported a few bugs about its non-function, and -hoped someone would step in and fix it. But no-one did.</p> - -<p>I got tired of waiting a few days ago, and took matters in my own -hands. The end result is that I am now the new upstream developer of -battery stats (<a href="https://github.com/petterreinholdtsen/battery-stats">available from github</a>) and part of the team maintaining -battery-stats in Debian, and the package in Debian unstable is finally -able to collect battery status using the <tt>/sys/class/power_supply/</tt> -information provided by the Linux kernel. If you install the -battery-stats package from unstable now, you will be able to get a -graph of the current battery fill level, to get some idea about the -status of the battery. The source package build and work just fine in -Debian testing and stable (and probably oldstable too, but I have not -tested). The default graph you get for that system look like this:</p> - -<p align="center"><img src="http://people.skolelinux.org/pere/blog/images/2016-03-15-battery-stats-graph-example.png" width="70%" align="center"></p> - -<p>My plans for the future is to merge my old scripts into the -battery-stats package, as my old scripts collected a lot more details -about the battery. The scripts are merged into the upstream -battery-stats git repository already, but I am not convinced they work -yet, as I changed a lot of paths along the way. Will have to test a -bit more before I make a new release.</p> - -<p>I will also consider changing the file format slightly, as I -suspect the way I combine several values into one field might make it -impossible to know the type of the value when using it for processing -and graphing.</p> - -<p>If you would like I would like to keep an close eye on your laptop -battery, check out the battery-stats package in -<a href="https://tracker.debian.org/pkg/battery-stats">Debian</a> and -on -<a href="https://github.com/petterreinholdtsen/battery-stats">github</a>. -I would love some help to improve the system further.</p> + How does it feel to be wiretapped, when you should be doing the wiretapping... + http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html + http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html + Wed, 8 Mar 2017 11:50:00 +0100 + <p>So the new president in the United States of America claim to be +surprised to discover that he was wiretapped during the election +before he was elected president. He even claim this must be illegal. +Well, doh, if it is one thing the confirmations from Snowden +documented, it is that the entire population in USA is wiretapped, one +way or another. Of course the president candidates were wiretapped, +alongside the senators, judges and the rest of the people in USA.</p> + +<p>Next, the Federal Bureau of Investigation ask the Department of +Justice to go public rejecting the claims that Donald Trump was +wiretapped illegally. I fail to see the relevance, given that I am +sure the surveillance industry in USA believe they have all the legal +backing they need to conduct mass surveillance on the entire +world.</p> + +<p>There is even the director of the FBI stating that he never saw an +order requesting wiretapping of Donald Trump. That is not very +surprising, given how the FISA court work, with all its activity being +secret. Perhaps he only heard about it?</p> + +<p>What I find most sad in this story is how Norwegian journalists +present it. In a news reports the other day in the radio from the +Norwegian National broadcasting Company (NRK), I heard the journalist +claim that 'the FBI denies any wiretapping', while the reality is that +'the FBI denies any illegal wiretapping'. There is a fundamental and +important difference, and it make me sad that the journalists are +unable to grasp it.</p> + +<p><strong>Update 2017-03-13:</strong> Look like +<a href="https://theintercept.com/2017/03/13/rand-paul-is-right-nsa-routinely-monitors-americans-communications-without-warrants/">The +Intercept report that US Senator Rand Paul confirm what I state above</a>.</p> - Creating, updating and checking debian/copyright semi-automatically - http://people.skolelinux.org/pere/blog/Creating__updating_and_checking_debian_copyright_semi_automatically.html - http://people.skolelinux.org/pere/blog/Creating__updating_and_checking_debian_copyright_semi_automatically.html - Fri, 19 Feb 2016 15:00:00 +0100 - <p>Making packages for Debian requires quite a lot of attention to -details. And one of the details is the content of the -debian/copyright file, which should list all relevant licenses used by -the code in the package in question, preferably in -<a href="https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/">machine -readable DEP5 format</a>.</p> - -<p>For large packages with lots of contributors it is hard to write -and update this file manually, and if you get some detail wrong, the -package is normally rejected by the ftpmasters. So getting it right -the first time around get the package into Debian faster, and save -both you and the ftpmasters some work.. Today, while trying to figure -out what was wrong with -<a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=686447">the -zfsonlinux copyright file</a>, I decided to spend some time on -figuring out the options for doing this job automatically, or at least -semi-automatically.</p> - -<p>Lucikly, there are at least two tools available for generating the -file based on the code in the source package, -<tt><a href="https://tracker.debian.org/pkg/debmake">debmake</a></tt> -and <tt><a href="https://tracker.debian.org/pkg/cme">cme</a></tt>. I'm -not sure which one of them came first, but both seem to be able to -create a sensible draft file. As far as I can tell, none of them can -be trusted to get the result just right, so the content need to be -polished a bit before the file is OK to upload. I found the debmake -option in -<a href="http://goofying-with-debian.blogspot.com/2014/07/debmake-checking-source-against-dep-5.html">a -blog posts from 2014</a>. - -<p>To generate using debmake, use the -cc option: - -<p><pre> -debmake -cc > debian/copyright -</pre></p> - -<p>Note there are some problems with python and non-ASCII names, so -this might not be the best option.</p> - -<p>The cme option is based on a config parsing library, and I found -this approach in -<a href="https://ddumont.wordpress.com/2015/04/05/improving-creation-of-debian-copyright-file/">a -blog post from 2015</a>. To generate using cme, use the 'update -dpkg-copyright' option: - -<p><pre> -cme update dpkg-copyright -</pre></p> - -<p>This will create or update debian/copyright. The cme tool seem to -handle UTF-8 names better than debmake.</p> - -<p>When the copyright file is created, I would also like some help to -check if the file is correct. For this I found two good options, -<tt>debmake -k</tt> and <tt>license-reconcile</tt>. The former seem -to focus on license types and file matching, and is able to detect -ineffective blocks in the copyright file. The latter reports missing -copyright holders and years, but was confused by inconsistent license -names (like CDDL vs. CDDL-1.0). I suspect it is good to use both and -fix all issues reported by them before uploading. But I do not know -if the tools and the ftpmasters agree on what is important to fix in a -copyright file, so the package might still be rejected.</p> - -<p>The devscripts tool <tt>licensecheck</tt> deserve mentioning. It -will read through the source and try to find all copyright statements. -It is not comparing the result to the content of debian/copyright, but -can be useful when verifying the content of the copyright file.</p> - -<p>Are you aware of better tools in Debian to create and update -debian/copyright file. Please let me know, or blog about it on -planet.debian.org.</p> - -<p>As usual, if you use Bitcoin and want to show your support of my -activities, please send Bitcoin donations to my address -<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p> - -<p><strong>Update 2016-02-20</strong>: I got a tip from Mike Gabriel -on how to use licensecheck and cdbs to create a draft copyright file - -<p><pre> -licensecheck --copyright -r `find * -type f` | \ - /usr/lib/cdbs/licensecheck2dep5 > debian/copyright.auto -</pre></p> - -<p>He mentioned that he normally check the generated file into the -version control system to make it easier to discover license and -copyright changes in the upstream source. I will try to do the same -with my packages in the future.</p> - -<p><strong>Update 2016-02-21</strong>: The cme author recommended -against using -quiet for new users, so I removed it from the proposed -command line.</p> + Norwegian Bokmål translation of The Debian Administrator's Handbook complete, proofreading in progress + http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html + http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html + Fri, 3 Mar 2017 14:50:00 +0100 + <p>For almost a year now, we have been working on making a Norwegian +Bokmål edition of <a href="https://debian-handbook.info/">The Debian +Administrator's Handbook</a>. Now, thanks to the tireless effort of +Ole-Erik, Ingrid and Andreas, the initial translation is complete, and +we are working on the proof reading to ensure consistent language and +use of correct computer science terms. The plan is to make the book +available on paper, as well as in electronic form. For that to +happen, the proof reading must be completed and all the figures need +to be translated. If you want to help out, get in touch.</p> + +<p><a href="http://people.skolelinux.org/pere/debian-handbook/debian-handbook-nb-NO.pdf">A + +fresh PDF edition</a> in A4 format (the final book will have smaller +pages) of the book created every morning is available for +proofreading. If you find any errors, please +<a href="https://hosted.weblate.org/projects/debian-handbook/">visit +Weblate and correct the error</a>. The +<a href="http://l.github.io/debian-handbook/stat/nb-NO/index.html">state +of the translation including figures</a> is a useful source for those +provide Norwegian bokmål screen shots and figures.</p> - Using appstream in Debian to locate packages with firmware and mime type support - http://people.skolelinux.org/pere/blog/Using_appstream_in_Debian_to_locate_packages_with_firmware_and_mime_type_support.html - http://people.skolelinux.org/pere/blog/Using_appstream_in_Debian_to_locate_packages_with_firmware_and_mime_type_support.html - Thu, 4 Feb 2016 16:40:00 +0100 - <p>The <a href="https://wiki.debian.org/DEP-11">appstream system</a> -is taking shape in Debian, and one provided feature is a very -convenient way to tell you which package to install to make a given -firmware file available when the kernel is looking for it. This can -be done using apt-file too, but that is for someone else to blog -about. :)</p> - -<p>Here is a small recipe to find the package with a given firmware -file, in this example I am looking for ctfw-3.2.3.0.bin, randomly -picked from the set of firmware announced using appstream in Debian -unstable. In general you would be looking for the firmware requested -by the kernel during kernel module loading. To find the package -providing the example file, do like this:</p> + Unlimited randomness with the ChaosKey? + http://people.skolelinux.org/pere/blog/Unlimited_randomness_with_the_ChaosKey_.html + http://people.skolelinux.org/pere/blog/Unlimited_randomness_with_the_ChaosKey_.html + Wed, 1 Mar 2017 20:50:00 +0100 + <p>A few days ago I ordered a small batch of +<a href="http://altusmetrum.org/ChaosKey/">the ChaosKey</a>, a small +USB dongle for generating entropy created by Bdale Garbee and Keith +Packard. Yesterday it arrived, and I am very happy to report that it +work great! According to its designers, to get it to work out of the +box, you need the Linux kernel version 4.1 or later. I tested on a +Debian Stretch machine (kernel version 4.9), and there it worked just +fine, increasing the available entropy very quickly. I wrote a small +test oneliner to test. It first print the current entropy level, +drain /dev/random, and then print the entropy level for five seconds. +Here is the situation without the ChaosKey inserted:</p> <blockquote><pre> -% apt install appstream -[...] -% apt update -[...] -% appstreamcli what-provides firmware:runtime ctfw-3.2.3.0.bin | \ - awk '/Package:/ {print $2}' -firmware-qlogic +% cat /proc/sys/kernel/random/entropy_avail; \ + dd bs=1M if=/dev/random of=/dev/null count=1; \ + for n in $(seq 1 5); do \ + cat /proc/sys/kernel/random/entropy_avail; \ + sleep 1; \ + done +300 +0+1 oppføringer inn +0+1 oppføringer ut +28 byte kopiert, 0,000264565 s, 106 kB/s +4 +8 +12 +17 +21 % </pre></blockquote> -<p>See <a href="https://wiki.debian.org/AppStream/Guidelines">the -appstream wiki</a> page to learn how to embed the package metadata in -a way appstream can use.</p> - -<p>This same approach can be used to find any package supporting a -given MIME type. This is very useful when you get a file you do not -know how to handle. First find the mime type using <tt>file ---mime-type</tt>, and next look up the package providing support for -it. Lets say you got an SVG file. Its MIME type is image/svg+xml, -and you can find all packages handling this type like this:</p> +<p>The entropy level increases by 3-4 every second. In such case any +application requiring random bits (like a HTTPS enabled web server) +will halt and wait for more entrpy. And here is the situation with +the ChaosKey inserted:</p> <blockquote><pre> -% apt install appstream -[...] -% apt update -[...] -% appstreamcli what-provides mimetype image/svg+xml | \ - awk '/Package:/ {print $2}' -bkchem -phototonic -inkscape -shutter -tetzle -geeqie -xia -pinta -gthumb -karbon -comix -mirage -viewnior -postr -ristretto -kolourpaint4 -eog -eom -gimagereader -midori +% cat /proc/sys/kernel/random/entropy_avail; \ + dd bs=1M if=/dev/random of=/dev/null count=1; \ + for n in $(seq 1 5); do \ + cat /proc/sys/kernel/random/entropy_avail; \ + sleep 1; \ + done +1079 +0+1 oppføringer inn +0+1 oppføringer ut +104 byte kopiert, 0,000487647 s, 213 kB/s +433 +1028 +1031 +1035 +1038 % </pre></blockquote> -<p>I believe the MIME types are fetched from the desktop file for -packages providing appstream metadata.</p> +<p>Quite the difference. :) I bought a few more than I need, in case +someone want to buy one here in Norway. :)</p> + +<p>Update: The dongle was presented at Debconf last year. You might +find <a href="https://debconf16.debconf.org/talks/94/">the talk +recording illuminating</a>. It explains exactly what the source of +randomness is, if you are unable to spot it from the schema drawing +available from the ChaosKey web site linked at the start of this blog +post.</p> - Creepy, visualise geotagged social media information - nice free software - http://people.skolelinux.org/pere/blog/Creepy__visualise_geotagged_social_media_information___nice_free_software.html - http://people.skolelinux.org/pere/blog/Creepy__visualise_geotagged_social_media_information___nice_free_software.html - Sun, 24 Jan 2016 10:50:00 +0100 - <p>Most people seem not to realise that every time they walk around -with the computerised radio beacon known as a mobile phone their -position is tracked by the phone company and often stored for a long -time (like every time a SMS is received or sent). And if their -computerised radio beacon is capable of running programs (often called -mobile apps) downloaded from the Internet, these programs are often -also capable of tracking their location (if the app requested access -during installation). And when these programs send out information to -central collection points, the location is often included, unless -extra care is taken to not send the location. The provided -information is used by several entities, for good and bad (what is -good and bad, depend on your point of view). What is certain, is that -the private sphere and the right to free movement is challenged and -perhaps even eradicated for those announcing their location this way, -when they share their whereabouts with private and public -entities.</p> - -<p align="center"><img width="70%" src="http://people.skolelinux.org/pere/blog/images/2016-01-24-nice-creepy-desktop-window.png"></p> - -<p>The phone company logs provide a register of locations to check out -when one want to figure out what the tracked person was doing. It is -unavailable for most of us, but provided to selected government -officials, company staff, those illegally buying information from -unfaithful servants and crackers stealing the information. But the -public information can be collected and analysed, and a free software -tool to do so is called -<a href="http://www.geocreepy.com/">Creepy or Cree.py</a>. I -discovered it when I read -<a href="http://www.aftenposten.no/kultur/Slik-kan-du-bli-overvaket-pa-Twitter-og-Instagram-uten-a-ane-det-7787884.html">an -article about Creepy</a> in the Norwegian newspaper Aftenposten i -November 2014, and decided to check if it was available in Debian. -The python program was in Debian, but -<a href="https://tracker.debian.org/pkg/creepy">the version in -Debian</a> was completely broken and practically unmaintained. I -uploaded a new version which did not work quite right, but did not -have time to fix it then. This Christmas I decided to finally try to -get Creepy operational in Debian. Now a fixed version is available in -Debian unstable and testing, and almost all Debian specific patches -are now included -<a href="https://github.com/jkakavas/creepy">upstream</a>.</p> - -<p>The Creepy program visualises geolocation information fetched from -Twitter, Instagram, Flickr and Google+, and allow one to get a -complete picture of every social media message posted recently in a -given area, or track the movement of a given individual across all -these services. Earlier it was possible to use the search API of at -least some of these services without identifying oneself, but these -days it is impossible. This mean that to use Creepy, you need to -configure it to log in as yourself on these services, and provide -information to them about your search interests. This should be taken -into account when using Creepy, as it will also share information -about yourself with the services.</p> - -<p>The picture above show the twitter messages sent from (or at least -geotagged with a position from) the city centre of Oslo, the capital -of Norway. One useful way to use Creepy is to first look at -information tagged with an area of interest, and next look at all the -information provided by one or more individuals who was in the area. -I tested it by checking out which celebrity provide their location in -twitter messages by checkout out who sent twitter messages near a -Norwegian TV station, and next could track their position over time, -making it possible to locate their home and work place, among other -things. A similar technique have been -<a href="http://www.buzzfeed.com/maxseddon/does-this-soldiers-instagram-account-prove-russia-is-covertl">used -to locate Russian soldiers in Ukraine</a>, and it is both a powerful -tool to discover lying governments, and a useful tool to help people -understand the value of the private information they provide to the -public.</p> - -<p>The package is not trivial to backport to Debian Stable/Jessie, as -it depend on several python modules currently missing in Jessie (at -least python-instagram, python-flickrapi and -python-requests-toolbelt).</p> - -<p>(I have uploaded -<a href="https://screenshots.debian.net/package/creepy">the image to -screenshots.debian.net</a> and licensed it under the same terms as the -Creepy program in Debian.)</p> + Detect OOXML files with undefined behaviour? + http://people.skolelinux.org/pere/blog/Detect_OOXML_files_with_undefined_behaviour_.html + http://people.skolelinux.org/pere/blog/Detect_OOXML_files_with_undefined_behaviour_.html + Tue, 21 Feb 2017 00:20:00 +0100 + <p>I just noticed +<a href="http://www.arkivrad.no/aktuelt/riksarkivarens-forskrift-pa-horing">the +new Norwegian proposal for archiving rules in the goverment</a> list +<a href="http://www.ecma-international.org/publications/standards/Ecma-376.htm">ECMA-376</a> +/ ISO/IEC 29500 (aka OOXML) as valid formats to put in long term +storage. Luckily such files will only be accepted based on +pre-approval from the National Archive. Allowing OOXML files to be +used for long term storage might seem like a good idea as long as we +forget that there are plenty of ways for a "valid" OOXML document to +have content with no defined interpretation in the standard, which +lead to a question and an idea.</p> + +<p>Is there any tool to detect if a OOXML document depend on such +undefined behaviour? It would be useful for the National Archive (and +anyone else interested in verifying that a document is well defined) +to have such tool available when considering to approve the use of +OOXML. I'm aware of the +<a href="https://github.com/arlm/officeotron/">officeotron OOXML +validator</a>, but do not know how complete it is nor if it will +report use of undefined behaviour. Are there other similar tools +available? Please send me an email if you know of any such tool.</p> - Always download Debian packages using Tor - the simple recipe - http://people.skolelinux.org/pere/blog/Always_download_Debian_packages_using_Tor___the_simple_recipe.html - http://people.skolelinux.org/pere/blog/Always_download_Debian_packages_using_Tor___the_simple_recipe.html - Fri, 15 Jan 2016 00:30:00 +0100 - <p>During his DebConf15 keynote, Jacob Appelbaum -<a href="https://summit.debconf.org/debconf15/meeting/331/what-is-to-be-done/">observed -that those listening on the Internet lines would have good reason to -believe a computer have a given security hole</a> if it download a -security fix from a Debian mirror. This is a good reason to always -use encrypted connections to the Debian mirror, to make sure those -listening do not know which IP address to attack. In August, Richard -Hartmann observed that encryption was not enough, when it was possible -to interfere download size to security patches or the fact that -download took place shortly after a security fix was released, and -<a href="http://richardhartmann.de/blog/posts/2015/08/24-Tor-enabled_Debian_mirror/">proposed -to always use Tor to download packages from the Debian mirror</a>. He -was not the first to propose this, as the -<tt><a href="https://tracker.debian.org/pkg/apt-transport-tor">apt-transport-tor</a></tt> -package by Tim Retout already existed to make it easy to convince apt -to use <a href="https://www.torproject.org/">Tor</a>, but I was not -aware of that package when I read the blog post from Richard.</p> - -<p>Richard discussed the idea with Peter Palfrader, one of the Debian -sysadmins, and he set up a Tor hidden service on one of the central -Debian mirrors using the address vwakviie2ienjx6t.onion, thus making -it possible to download packages directly between two tor nodes, -making sure the network traffic always were encrypted.</p> - -<p>Here is a short recipe for enabling this on your machine, by -installing <tt>apt-transport-tor</tt> and replacing http and https -urls with tor+http and tor+https, and using the hidden service instead -of the official Debian mirror site. I recommend installing -<tt>etckeeper</tt> before you start to have a history of the changes -done in /etc/.</p> - -<blockquote><pre> -apt install apt-transport-tor -sed -i 's% http://ftp.debian.org/% tor+http://vwakviie2ienjx6t.onion/%' /etc/apt/sources.list -sed -i 's% http% tor+http%' /etc/apt/sources.list -</pre></blockquote> - -<p>If you have more sources listed in /etc/apt/sources.list.d/, run -the sed commands for these too. The sed command is assuming your are -using the ftp.debian.org Debian mirror. Adjust the command (or just -edit the file manually) to match your mirror.</p> - -<p>This work in Debian Jessie and later. Note that tools like -<tt>apt-file</tt> only recently started using the apt transport -system, and do not work with these tor+http URLs. For -<tt>apt-file</tt> you need the version currently in experimental, -which need a recent apt version currently only in unstable. So if you -need a working <tt>apt-file</tt>, this is not for you.</p> - -<p>Another advantage from this change is that your machine will start -using Tor regularly and at fairly random intervals (every time you -update the package lists or upgrade or install a new package), thus -masking other Tor traffic done from the same machine. Using Tor will -become normal for the machine in question.</p> - -<p>On <a href="https://wiki.debian.org/FreedomBox">Freedombox</a>, APT -is set up by default to use <tt>apt-transport-tor</tt> when Tor is -enabled. It would be great if it was the default on any Debian -system.</p> + Ruling ignored our objections to the seizure of popcorn-time.no (#domstolkontroll) + http://people.skolelinux.org/pere/blog/Ruling_ignored_our_objections_to_the_seizure_of_popcorn_time_no___domstolkontroll_.html + http://people.skolelinux.org/pere/blog/Ruling_ignored_our_objections_to_the_seizure_of_popcorn_time_no___domstolkontroll_.html + Mon, 13 Feb 2017 21:30:00 +0100 + <p>A few days ago, we received the ruling from +<a href="http://people.skolelinux.org/pere/blog/A_day_in_court_challenging_seizure_of_popcorn_time_no_for__domstolkontroll.html">my +day in court</a>. The case in question is a challenge of the seizure +of the DNS domain popcorn-time.no. The ruling simply did not mention +most of our arguments, and seemed to take everything ØKOKRIM said at +face value, ignoring our demonstration and explanations. But it is +hard to tell for sure, as we still have not seen most of the documents +in the case and thus were unprepared and unable to contradict several +of the claims made in court by the opposition. We are considering an +appeal, but it is partly a question of funding, as it is costing us +quite a bit to pay for our lawyer. If you want to help, please +<a href="http://www.nuug.no/dns-beslag-donasjon.shtml">donate to the +NUUG defense fund</a>.</p> + +<p>The details of the case, as far as we know it, is available in +Norwegian from +<a href="https://www.nuug.no/news/tags/dns-domenebeslag/">the NUUG +blog</a>. This also include +<a href="https://www.nuug.no/news/Avslag_etter_rettslig_h_ring_om_DNS_beslaget___vurderer_veien_videre.shtml">the +ruling itself</a>.</p> - Nedlasting fra NRK, som Matroska med undertekster - http://people.skolelinux.org/pere/blog/Nedlasting_fra_NRK__som_Matroska_med_undertekster.html - http://people.skolelinux.org/pere/blog/Nedlasting_fra_NRK__som_Matroska_med_undertekster.html - Sat, 2 Jan 2016 13:50:00 +0100 - <p>Det kommer stadig nye løsninger for å ta lagre unna innslag fra NRK -for å se på det senere. For en stund tilbake kom jeg over et script -nrkopptak laget av Ingvar Hagelund. Han fjernet riktignok sitt script -etter forespørsel fra Erik Bolstad i NRK, men noen tok heldigvis og -gjorde det <a href="https://github.com/liangqi/nrkopptak">tilgjengelig -via github</a>.</p> - -<p>Scriptet kan lagre som MPEG4 eller Matroska, og bake inn -undertekster i fila på et vis som blant annet VLC forstår. For å -bruke scriptet, kopier ned git-arkivet og kjør</p> - -<p><pre> -nrkopptak/bin/nrk-opptak k <ahref="https://tv.nrk.no/serie/bmi-turne/MUHH45000115/sesong-1/episode-1">https://tv.nrk.no/serie/bmi-turne/MUHH45000115/sesong-1/episode-1</a> -</pre></p> - -<p>URL-eksemplet er dagens toppsak på tv.nrk.no. Argument 'k' ber -scriptet laste ned og lagre som Matroska. Det finnes en rekke andre -muligheter for valg av kvalitet og format.</p> - -<p>Jeg foretrekker dette scriptet fremfor youtube-dl, som -<a href="http://people.skolelinux.org/pere/blog/Hvordan_enkelt_laste_ned_filmer_fra_NRK_med_den__nye__l_sningen.html"> -nevnt i 2014 støtter NRK</a> og en rekke andre videokilder, på grunn -av at nrkopptak samler undertekster og video i en enkelt fil, hvilket -gjør håndtering enklere på disk.</p> + A day in court challenging seizure of popcorn-time.no for #domstolkontroll + http://people.skolelinux.org/pere/blog/A_day_in_court_challenging_seizure_of_popcorn_time_no_for__domstolkontroll.html + http://people.skolelinux.org/pere/blog/A_day_in_court_challenging_seizure_of_popcorn_time_no_for__domstolkontroll.html + Fri, 3 Feb 2017 11:10:00 +0100 + <p align="center"><img width="70%" src="http://people.skolelinux.org/pere/blog/images/2017-02-01-popcorn-time-in-court.jpeg"></p> + +<p>On Wednesday, I spent the entire day in court in Follo Tingrett +representing <a href="https://www.nuug.no/">the member association +NUUG</a>, alongside <a href="https://www.efn.no/">the member +association EFN</a> and <a href="http://www.imc.no">the DNS registrar +IMC</a>, challenging the seizure of the DNS name popcorn-time.no. It +was interesting to sit in a court of law for the first time in my +life. Our team can be seen in the picture above: attorney Ola +Tellesbø, EFN board member Tom Fredrik Blenning, IMC CEO Morten Emil +Eriksen and NUUG board member Petter Reinholdtsen.</p> + +<p><a href="http://www.domstol.no/no/Enkelt-domstol/follo-tingrett/Nar-gar-rettssaken/Beramming/?cid=AAAA1701301512081262234UJFBVEZZZZZEJBAvtale">The +case at hand</a> is that the Norwegian National Authority for +Investigation and Prosecution of Economic and Environmental Crime (aka +Økokrim) decided on their own, to seize a DNS domain early last +year, without following +<a href="https://www.norid.no/no/regelverk/navnepolitikk/#link12">the +official policy of the Norwegian DNS authority</a> which require a +court decision. The web site in question was a site covering Popcorn +Time. And Popcorn Time is the name of a technology with both legal +and illegal applications. Popcorn Time is a client combining +searching a Bittorrent directory available on the Internet with +downloading/distribute content via Bittorrent and playing the +downloaded content on screen. It can be used illegally if it is used +to distribute content against the will of the right holder, but it can +also be used legally to play a lot of content, for example the +millions of movies +<a href="https://archive.org/details/movies">available from the +Internet Archive</a> or the collection +<a href="http://vodo.net/films/">available from Vodo</a>. We created +<a href="magnet:?xt=urn:btih:86c1802af5a667ca56d3918aecb7d3c0f7173084&dn=PresentasjonFolloTingrett.mov&tr=udp%3A%2F%2Fpublic.popcorn-tracker.org%3A6969%2Fannounce">a +video demonstrating legally use of Popcorn Time</a> and played it in +Court. It can of course be downloaded using Bittorrent.</p> + +<p>I did not quite know what to expect from a day in court. The +government held on to their version of the story and we held on to +ours, and I hope the judge is able to make sense of it all. We will +know in two weeks time. Unfortunately I do not have high hopes, as +the Government have the upper hand here with more knowledge about the +case, better training in handling criminal law and in general higher +standing in the courts than fairly unknown DNS registrar and member +associations. It is expensive to be right also in Norway. So far the +case have cost more than NOK 70 000,-. To help fund the case, NUUG +and EFN have asked for donations, and managed to collect around NOK 25 +000,- so far. Given the presentation from the Government, I expect +the government to appeal if the case go our way. And if the case do +not go our way, I hope we have enough funding to appeal.</p> + +<p>From the other side came two people from Økokrim. On the benches, +appearing to be part of the group from the government were two people +from the Simonsen Vogt Wiik lawyer office, and three others I am not +quite sure who was. Økokrim had proposed to present two witnesses +from The Motion Picture Association, but this was rejected because +they did not speak Norwegian and it was a bit late to bring in a +translator, but perhaps the two from MPA were present anyway. All +seven appeared to know each other. Good to see the case is take +seriously.</p> + +<p>If you, like me, believe the courts should be involved before a DNS +domain is hijacked by the government, or you believe the Popcorn Time +technology have a lot of useful and legal applications, I suggest you +too <a href="http://www.nuug.no/dns-beslag-donasjon.shtml">donate to +the NUUG defense fund</a>. Both Bitcoin and bank transfer are +available. If NUUG get more than we need for the legal action (very +unlikely), the rest will be spend promoting free software, open +standards and unix-like operating systems in Norway, so no matter what +happens the money will be put to good use.</p> + +<p>If you want to lean more about the case, I recommend you check out +<a href="https://www.nuug.no/news/tags/dns-domenebeslag/">the blog +posts from NUUG covering the case</a>. They cover the legal arguments +on both sides.</p> - OpenALPR, find car license plates in video streams - nice free software - http://people.skolelinux.org/pere/blog/OpenALPR__find_car_license_plates_in_video_streams___nice_free_software.html - http://people.skolelinux.org/pere/blog/OpenALPR__find_car_license_plates_in_video_streams___nice_free_software.html - Wed, 23 Dec 2015 01:00:00 +0100 - <p>When I was a kid, we used to collect "car numbers", as we used to -call the car license plate numbers in those days. I would write the -numbers down in my little book and compare notes with the other kids -to see how many region codes we had seen and if we had seen some -exotic or special region codes and numbers. It was a fun game to pass -time, as we kids have plenty of it.</p> - -<p>A few days I came across -<a href="https://github.com/openalpr/openalpr">the OpenALPR -project</a>, a free software project to automatically discover and -report license plates in images and video streams, and provide the -"car numbers" in a machine readable format. I've been looking for -such system for a while now, because I believe it is a bad idea that the -<a href="https://en.wikipedia.org/wiki/Automatic_number_plate_recognition">automatic -number plate recognition</a> tool only is available in the hands of -the powerful, and want it to be available also for the powerless to -even the score when it comes to surveillance and sousveillance. I -discovered the developer -<a href="https://bugs.debian.org/747509">wanted to get the tool into -Debian</a>, and as I too wanted it to be in Debian, I volunteered to -help him get it into shape to get the package uploaded into the Debian -archive.</p> - -<p>Today we finally managed to get the package into shape and uploaded -it into Debian, where it currently -<a href="https://ftp-master.debian.org//new/openalpr_2.2.1-1.html">waits -in the NEW queue</a> for review by the Debian ftpmasters.</p> - -<p>I guess you are wondering why on earth such tool would be useful -for the common folks, ie those not running a large government -surveillance system? Well, I plan to put it in a computer on my bike -and in my car, tracking the cars nearby and allowing me to be notified -when number plates on my watch list are discovered. Another use case -was suggested by a friend of mine, who wanted to set it up at his home -to open the car port automatically when it discovered the plate on his -car. When I mentioned it perhaps was a bit foolhardy to allow anyone -capable of placing his license plate number of a piece of cardboard to -open his car port, men replied that it was always unlocked anyway. I -guess for such use case it make sense. I am sure there are other use -cases too, for those with imagination and a vision.</p> - -<p>If you want to build your own version of the Debian package, check -out the upstream git source and symlink ./distros/debian to ./debian/ -before running "debuild" to build the source. Or wait a bit until the -package show up in unstable.</p> + Nasjonalbiblioteket avslutter sin ulovlige bruk av Google Skjemaer + http://people.skolelinux.org/pere/blog/Nasjonalbiblioteket_avslutter_sin_ulovlige_bruk_av_Google_Skjemaer.html + http://people.skolelinux.org/pere/blog/Nasjonalbiblioteket_avslutter_sin_ulovlige_bruk_av_Google_Skjemaer.html + Thu, 12 Jan 2017 09:40:00 +0100 + <p>I dag fikk jeg en skikkelig gladmelding. Bakgrunnen er at før jul +arrangerte Nasjonalbiblioteket +<a href="http://www.nb.no/Bibliotekutvikling/Kunnskapsorganisering/Nasjonalt-verksregister/Seminar-om-verksregister">et +seminar om sitt knakende gode tiltak «verksregister»</a>. Eneste +måten å melde seg på dette seminaret var å sende personopplysninger +til Google via Google Skjemaer. Dette syntes jeg var tvilsom praksis, +da det bør være mulig å delta på seminarer arrangert av det offentlige +uten å måtte dele sine interesser, posisjon og andre +personopplysninger med Google. Jeg ba derfor om innsyn via +<a href="https://www.mimesbronn.no/">Mimes brønn</a> i +<a href="https://www.mimesbronn.no/request/personopplysninger_til_google_sk">avtaler +og vurderinger Nasjonalbiblioteket hadde rundt dette</a>. +Personopplysningsloven legger klare rammer for hva som må være på +plass før en kan be tredjeparter, spesielt i utlandet, behandle +personopplysninger på sine vegne, så det burde eksistere grundig +dokumentasjon før noe slikt kan bli lovlig. To jurister hos +Nasjonalbiblioteket mente først dette var helt i orden, og at Googles +standardavtale kunne brukes som databehandlingsavtale. Det syntes jeg +var merkelig, men har ikke hatt kapasitet til å følge opp saken før +for to dager siden.</p> + +<p>Gladnyheten i dag, som kom etter at jeg tipset Nasjonalbiblioteket +om at Datatilsynet underkjente Googles standardavtaler som +databehandleravtaler i 2011, er at Nasjonalbiblioteket har bestemt seg +for å avslutte bruken av Googles Skjemaer/Apps og gå i dialog med DIFI +for å finne bedre måter å håndtere påmeldinger i tråd med +personopplysningsloven. Det er fantastisk å se at av og til hjelper +det å spørre hva i alle dager det offentlige holder på med.</p> - Using appstream with isenkram to install hardware related packages in Debian - http://people.skolelinux.org/pere/blog/Using_appstream_with_isenkram_to_install_hardware_related_packages_in_Debian.html - http://people.skolelinux.org/pere/blog/Using_appstream_with_isenkram_to_install_hardware_related_packages_in_Debian.html - Sun, 20 Dec 2015 12:20:00 +0100 - <p>Around three years ago, I created -<a href="http://packages.qa.debian.org/isenkram">the isenkram -system</a> to get a more practical solution in Debian for handing -hardware related packages. A GUI system in the isenkram package will -present a pop-up dialog when some hardware dongle supported by -relevant packages in Debian is inserted into the machine. The same -lookup mechanism to detect packages is available as command line -tools in the isenkram-cli package. In addition to mapping hardware, -it will also map kernel firmware files to packages and make it easy to -install needed firmware packages automatically. The key for this -system to work is a good way to map hardware to packages, in other -words, allow packages to announce what hardware they will work -with.</p> - -<p>I started by providing data files in the isenkram source, and -adding code to download the latest version of these data files at run -time, to ensure every user had the most up to date mapping available. -I also added support for storing the mapping in the Packages file in -the apt repositories, but did not push this approach because while I -was trying to figure out how to best store hardware/package mappings, -<a href="http://www.freedesktop.org/software/appstream/docs/">the -appstream system</a> was announced. I got in touch and suggested to -add the hardware mapping into that data set to be able to use -appstream as a data source, and this was accepted at least for the -Debian version of appstream.</p> - -<p>A few days ago using appstream in Debian for this became possible, -and today I uploaded a new version 0.20 of isenkram adding support for -appstream as a data source for mapping hardware to packages. The only -package so far using appstream to announce its hardware support is my -pymissile package. I got help from Matthias Klumpp with figuring out -how do add the required -<a href="https://appstream.debian.org/html/sid/main/metainfo/pymissile.html">metadata -in pymissile</a>. I added a file debian/pymissile.metainfo.xml with -this content:</p> - -<blockquote><pre> -&lt;?xml version="1.0" encoding="UTF-8"?&gt; -&lt;component&gt; - &lt;id&gt;pymissile&lt;/id&gt; - &lt;metadata_license&gt;MIT&lt;/metadata_license&gt; - &lt;name&gt;pymissile&lt;/name&gt; - &lt;summary&gt;Control original Striker USB Missile Launcher&lt;/summary&gt; - &lt;description&gt; - &lt;p&gt; - Pymissile provides a curses interface to control an original - Marks and Spencer / Striker USB Missile Launcher, as well as a - motion control script to allow a webcamera to control the - launcher. - &lt;/p&gt; - &lt;/description&gt; - &lt;provides&gt; - &lt;modalias&gt;usb:v1130p0202d*&lt;/modalias&gt; - &lt;/provides&gt; -&lt;/component&gt; -</pre></blockquote> - -<p>The key for isenkram is the component/provides/modalias value, -which is a glob style match rule for hardware specific strings -(modalias strings) provided by the Linux kernel. In this case, it -will map to all USB devices with vendor code 1130 and product code -0202.</p> - -<p>Note, it is important that the license of all the metadata files -are compatible to have permissions to aggregate them into archive wide -appstream files. Matthias suggested to use MIT or BSD licenses for -these files. A challenge is figuring out a good id for the data, as -it is supposed to be globally unique and shared across distributions -(in other words, best to coordinate with upstream what to use). But -it can be changed later or, so we went with the package name as -upstream for this project is dormant.</p> - -<p>To get the metadata file installed in the correct location for the -mirror update scripts to pick it up and include its content the -appstream data source, the file must be installed in the binary -package under /usr/share/appdata/. I did this by adding the following -line to debian/pymissile.install:</p> - -<blockquote><pre> -debian/pymissile.metainfo.xml usr/share/appdata -</pre></blockquote> - -<p>With that in place, the command line tool isenkram-lookup will list -all packages useful on the current computer automatically, and the GUI -pop-up handler will propose to install the package not already -installed if a hardware dongle is inserted into the machine in -question.</p> - -<p>Details of the modalias field in appstream is available from the -<a href="https://wiki.debian.org/DEP-11">DEP-11</a> proposal.</p> + Bryter NAV sin egen personvernerklæring? + http://people.skolelinux.org/pere/blog/Bryter_NAV_sin_egen_personvernerkl_ring_.html + http://people.skolelinux.org/pere/blog/Bryter_NAV_sin_egen_personvernerkl_ring_.html + Wed, 11 Jan 2017 06:50:00 +0100 + <p>Jeg leste med interesse en nyhetssak hos +<a href="http://www.digi.no/artikler/nav-avslorer-trygdemisbruk-ved-a-spore-ip-adresser/367394">digi.no</a> +og +<a href="https://www.nrk.no/buskerud/trygdesvindlere-avslores-av-utenlandske-ip-adresser-1.13313461">NRK</a> +om at det ikke bare er meg, men at også NAV bedriver geolokalisering +av IP-adresser, og at det gjøres analyse av IP-adressene til de som +sendes inn meldekort for å se om meldekortet sendes inn fra +utenlandske IP-adresser. Politiadvokat i Drammen, Hans Lyder Haare, +er sitert i NRK på at «De to er jo blant annet avslørt av +IP-adresser. At man ser at meldekortet kommer fra utlandet.»</p> + +<p>Jeg synes det er fint at det blir bedre kjent at IP-adresser +knyttes til enkeltpersoner og at innsamlet informasjon brukes til å +stedsbestemme personer også av aktører her i Norge. Jeg ser det som +nok et argument for å bruke +<a href="https://www.torproject.org/">Tor</a> så mye som mulig for å +gjøre gjøre IP-lokalisering vanskeligere, slik at en kan beskytte sin +privatsfære og unngå å dele sin fysiske plassering med +uvedkommede.</p> + +<P>Men det er en ting som bekymrer meg rundt denne nyheten. Jeg ble +tipset (takk #nuug) om +<a href="https://www.nav.no/no/NAV+og+samfunn/Kontakt+NAV/Teknisk+brukerstotte/Snarveier/personvernerkl%C3%A6ring-for-arbeids-og-velferdsetaten">NAVs +personvernerklæring</a>, som under punktet «Personvern og statistikk» +lyder:</p> + +<p><blockquote> + +<p>«Når du besøker nav.no, etterlater du deg elektroniske spor. Sporene +dannes fordi din nettleser automatisk sender en rekke opplysninger til +NAVs tjener (server-maskin) hver gang du ber om å få vist en side. Det +er eksempelvis opplysninger om hvilken nettleser og -versjon du +bruker, og din internettadresse (ip-adresse). For hver side som vises, +lagres følgende opplysninger:</p> + +<ul> +<li>hvilken side du ser på</li> +<li>dato og tid</li> +<li>hvilken nettleser du bruker</li> +<li>din ip-adresse</li> +</ul> + +<p>Ingen av opplysningene vil bli brukt til å identifisere +enkeltpersoner. NAV bruker disse opplysningene til å generere en +samlet statistikk som blant annet viser hvilke sider som er mest +populære. Statistikken er et redskap til å forbedre våre +tjenester.»</p> + +</blockquote></p> + +<p>Jeg klarer ikke helt å se hvordan analyse av de besøkendes +IP-adresser for å se hvem som sender inn meldekort via web fra en +IP-adresse i utlandet kan gjøres uten å komme i strid med påstanden om +at «ingen av opplysningene vil bli brukt til å identifisere +enkeltpersoner». Det virker dermed for meg som at NAV bryter sine +egen personvernerklæring, hvilket +<a href="http://people.skolelinux.org/pere/blog/Er_lover_brutt_n_r_personvernpolicy_ikke_stemmer_med_praksis_.html">Datatilsynet +fortalte meg i starten av desember antagelig er brudd på +personopplysningsloven</a>. + +<p>I tillegg er personvernerklæringen ganske misvisende i og med at +NAVs nettsider ikke bare forsyner NAV med personopplysninger, men i +tillegg ber brukernes nettleser kontakte fem andre nettjenere +(script.hotjar.com, static.hotjar.com, vars.hotjar.com, +www.google-analytics.com og www.googletagmanager.com), slik at +personopplysninger blir gjort tilgjengelig for selskapene Hotjar og +Google , og alle som kan lytte på trafikken på veien (som FRA, GCHQ og +NSA). Jeg klarer heller ikke se hvordan slikt spredning av +personopplysninger kan være i tråd med kravene i +personopplysningloven, eller i tråd med NAVs personvernerklæring.</p> + +<p>Kanskje NAV bør ta en nøye titt på sin personvernerklæring? Eller +kanskje Datatilsynet bør gjøre det?</p> + + + + + Where did that package go? &mdash; geolocated IP traceroute + http://people.skolelinux.org/pere/blog/Where_did_that_package_go___mdash__geolocated_IP_traceroute.html + http://people.skolelinux.org/pere/blog/Where_did_that_package_go___mdash__geolocated_IP_traceroute.html + Mon, 9 Jan 2017 12:20:00 +0100 + <p>Did you ever wonder where the web trafic really flow to reach the +web servers, and who own the network equipment it is flowing through? +It is possible to get a glimpse of this from using traceroute, but it +is hard to find all the details. Many years ago, I wrote a system to +map the Norwegian Internet (trying to figure out if our plans for a +network game service would get low enough latency, and who we needed +to talk to about setting up game servers close to the users. Back +then I used traceroute output from many locations (I asked my friends +to run a script and send me their traceroute output) to create the +graph and the map. The output from traceroute typically look like +this: -<p>To locate the modalias values of all hardware present in a machine, -try running this command on the command line:</p> +<p><pre> +traceroute to www.stortinget.no (85.88.67.10), 30 hops max, 60 byte packets + 1 uio-gw10.uio.no (129.240.202.1) 0.447 ms 0.486 ms 0.621 ms + 2 uio-gw8.uio.no (129.240.24.229) 0.467 ms 0.578 ms 0.675 ms + 3 oslo-gw1.uninett.no (128.39.65.17) 0.385 ms 0.373 ms 0.358 ms + 4 te3-1-2.br1.fn3.as2116.net (193.156.90.3) 1.174 ms 1.172 ms 1.153 ms + 5 he16-1-1.cr1.san110.as2116.net (195.0.244.234) 2.627 ms he16-1-1.cr2.oslosda310.as2116.net (195.0.244.48) 3.172 ms he16-1-1.cr1.san110.as2116.net (195.0.244.234) 2.857 ms + 6 ae1.ar8.oslosda310.as2116.net (195.0.242.39) 0.662 ms 0.637 ms ae0.ar8.oslosda310.as2116.net (195.0.242.23) 0.622 ms + 7 89.191.10.146 (89.191.10.146) 0.931 ms 0.917 ms 0.955 ms + 8 * * * + 9 * * * +[...] +</pre></p> -<blockquote><pre> -cat $(find /sys/devices/|grep modalias) -</pre></blockquote> +<p>This show the DNS names and IP addresses of (at least some of the) +network equipment involved in getting the data traffic from me to the +www.stortinget.no server, and how long it took in milliseconds for a +package to reach the equipment and return to me. Three packages are +sent, and some times the packages do not follow the same path. This +is shown for hop 5, where three different IP addresses replied to the +traceroute request.</p> + +<p>There are many ways to measure trace routes. Other good traceroute +implementations I use are traceroute (using ICMP packages) mtr (can do +both ICMP, UDP and TCP) and scapy (python library with ICMP, UDP, TCP +traceroute and a lot of other capabilities). All of them are easily +available in <a href="https://www.debian.org/">Debian</a>.</p> + +<p>This time around, I wanted to know the geographic location of +different route points, to visualize how visiting a web page spread +information about the visit to a lot of servers around the globe. The +background is that a web site today often will ask the browser to get +from many servers the parts (for example HTML, JSON, fonts, +JavaScript, CSS, video) required to display the content. This will +leak information about the visit to those controlling these servers +and anyone able to peek at the data traffic passing by (like your ISP, +the ISPs backbone provider, FRA, GCHQ, NSA and others).</p> + +<p>Lets pick an example, the Norwegian parliament web site +www.stortinget.no. It is read daily by all members of parliament and +their staff, as well as political journalists, activits and many other +citizens of Norway. A visit to the www.stortinget.no web site will +ask your browser to contact 8 other servers: ajax.googleapis.com, +insights.hotjar.com, script.hotjar.com, static.hotjar.com, +stats.g.doubleclick.net, www.google-analytics.com, +www.googletagmanager.com and www.netigate.se. I extracted this by +asking <a href="http://phantomjs.org/">PhantomJS</a> to visit the +Stortinget web page and tell me all the URLs PhantomJS downloaded to +render the page (in HAR format using +<a href="https://github.com/ariya/phantomjs/blob/master/examples/netsniff.js">their +netsniff example</a>. I am very grateful to Gorm for showing me how +to do this). My goal is to visualize network traces to all IP +addresses behind these DNS names, do show where visitors personal +information is spread when visiting the page.</p> + +<p align="center"><a href="www.stortinget.no-geoip.kml"><img +src="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-geoip-small.png" alt="map of combined traces for URLs used by www.stortinget.no using GeoIP"/></a></p> + +<p>When I had a look around for options, I could not find any good +free software tools to do this, and decided I needed my own traceroute +wrapper outputting KML based on locations looked up using GeoIP. KML +is easy to work with and easy to generate, and understood by several +of the GIS tools I have available. I got good help from by NUUG +colleague Anders Einar with this, and the result can be seen in +<a href="https://github.com/petterreinholdtsen/kmltraceroute">my +kmltraceroute git repository</a>. Unfortunately, the quality of the +free GeoIP databases I could find (and the for-pay databases my +friends had access to) is not up to the task. The IP addresses of +central Internet infrastructure would typically be placed near the +controlling companies main office, and not where the router is really +located, as you can see from <a href="www.stortinget.no-geoip.kml">the +KML file I created</a> using the GeoLite City dataset from MaxMind. + +<p align="center"><a href="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-scapy.svg"><img +src="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-scapy-small.png" alt="scapy traceroute graph for URLs used by www.stortinget.no"/></a></p> + +<p>I also had a look at the visual traceroute graph created by +<a href="http://www.secdev.org/projects/scapy/">the scrapy project</a>, +showing IP network ownership (aka AS owner) for the IP address in +question. +<a href="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-scapy.svg">The +graph display a lot of useful information about the traceroute in SVG +format</a>, and give a good indication on who control the network +equipment involved, but it do not include geolocation. This graph +make it possible to see the information is made available at least for +UNINETT, Catchcom, Stortinget, Nordunet, Google, Amazon, Telia, Level +3 Communications and NetDNA.</p> + +<p align="center"><a href="https://geotraceroute.com/index.php?node=4&host=www.stortinget.no"><img +src="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-geotraceroute-small.png" alt="example geotraceroute view for www.stortinget.no"/></a></p> + +<p>In the process, I came across the +<a href="https://geotraceroute.com/">web service GeoTraceroute</a> by +Salim Gasmi. Its methology of combining guesses based on DNS names, +various location databases and finally use latecy times to rule out +candidate locations seemed to do a very good job of guessing correct +geolocation. But it could only do one trace at the time, did not have +a sensor in Norway and did not make the geolocations easily available +for postprocessing. So I contacted the developer and asked if he +would be willing to share the code (he refused until he had time to +clean it up), but he was interested in providing the geolocations in a +machine readable format, and willing to set up a sensor in Norway. So +since yesterday, it is possible to run traces from Norway in this +service thanks to a sensor node set up by +<a href="https://www.nuug.no/">the NUUG assosiation</a>, and get the +trace in KML format for further processing.</p> + +<p align="center"><a href="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-geotraceroute-kml-join.kml"><img +src="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-geotraceroute-kml-join.png" alt="map of combined traces for URLs used by www.stortinget.no using geotraceroute"/></a></p> + +<p>Here we can see a lot of trafic passes Sweden on its way to +Denmark, Germany, Holland and Ireland. Plenty of places where the +Snowden confirmations verified the traffic is read by various actors +without your best interest as their top priority.</p> + +<p>Combining KML files is trivial using a text editor, so I could loop +over all the hosts behind the urls imported by www.stortinget.no and +ask for the KML file from GeoTraceroute, and create a combined KML +file with all the traces (unfortunately only one of the IP addresses +behind the DNS name is traced this time. To get them all, one would +have to request traces using IP number instead of DNS names from +GeoTraceroute). That might be the next step in this project.</p> + +<p>Armed with these tools, I find it a lot easier to figure out where +the IP traffic moves and who control the boxes involved in moving it. +And every time the link crosses for example the Swedish border, we can +be sure Swedish Signal Intelligence (FRA) is listening, as GCHQ do in +Britain and NSA in USA and cables around the globe. (Hm, what should +we tell them? :) Keep that in mind if you ever send anything +unencrypted over the Internet.</p> + +<p>PS: KML files are drawn using +<a href="http://ivanrublev.me/kml/">the KML viewer from Ivan +Rublev<a/>, as it was less cluttered than the local Linux application +Marble. There are heaps of other options too.</p> -<p>To learn more about the isenkram system, please check out -<a href="http://people.skolelinux.org/pere/blog/tags/isenkram/">my -blog posts tagged isenkram</a>.</p> +<p>As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&label=PetterReinholdtsenBlog">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>