Detecting NFS hangs on Linux without hanging yourself...

9th March 2017

Over the years, administrating thousand of NFS mounting linux -computers at the time, I often needed a way to detect if the machine -was experiencing NFS hang. If you try to use df or look at a -file or directory affected by the hang, the process (and possibly the -shell) will hang too. So you want to be able to detect this without -risking the detection process getting stuck too. It has not been -obvious how to do this. When the hang has lasted a while, it is -possible to find messages like these in dmesg:

- -

-nfs: server nfsserver not responding, still trying -
nfs: server nfsserver OK -

- -

It is hard to know if the hang is still going on, and it is hard to -be sure looking in dmesg is going to work. If there are lots of other -messages in dmesg the lines might have rotated out of site before they -are noticed.

- -

While reading through the nfs client implementation in linux kernel -code, I came across some statistics that seem to give a way to detect -it. The om_timeouts sunrpc value in the kernel will increase every -time the above log entry is inserted into dmesg. And after digging a -bit further, I discovered that this value show up in -/proc/self/mountstats on Linux.

- -

The mountstats content seem to be shared between files using the -same file system context, so it is enough to check one of the -mountstats files to get the state of the mount point for the machine. -I assume this will not show lazy umounted NFS points, nor NFS mount -points in a different process context (ie with a different filesystem -view), but that does not worry me.

- -

The content for a NFS mount point look similar to this:

- -

-[...]
-device /dev/mapper/Debian-var mounted on /var with fstype ext3
-device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1
-        opts:   rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all
-        age:    7863311
-        caps:   caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255
-        sec:    flavor=1,pseudoflavor=1
-        events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0 
-        bytes:  166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809 
-        RPC iostats version: 1.0  p/v: 100003/3 (nfs)
-        xprt:   tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820
-        per-op statistics
-                NULL: 0 0 0 0 0 0 0 0
-             GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132
-             SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943
-              LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511
-              ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140
-            READLINK: 125 125 0 20472 18620 0 1112 1118
-                READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693
-               WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771
-              CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398
-               MKDIR: 3680 3680 0 773980 993920 26 23990 24245
-             SYMLINK: 903 903 0 233428 245488 6 5865 5917
-               MKNOD: 80 80 0 20148 21760 0 299 304
-              REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636
-               RMDIR: 3367 3367 0 645112 484848 22 5782 6002
-              RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288
-                LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579
-             READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917
-         READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937
-              FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022
-              FSINFO: 2 2 0 232 328 0 1 1
-            PATHCONF: 1 1 0 116 140 0 0 0
-              COMMIT: 0 0 0 0 0 0 0 0
-
-device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
-[...]
-

- -

The key number to look at is the third number in the per-op list. -It is the number of NFS timeouts experiences per file system -operation. Here 22 write timeouts and 5 access timeouts. If these -numbers are increasing, I believe the machine is experiencing NFS -hang. Unfortunately the timeout value do not start to increase right -away. The NFS operations need to time out first, and this can take a -while. The exact timeout value depend on the setup. For example the -defaults for TCP and UDP mount points are quite different, and the -timeout value is affected by the soft, hard, timeo and retrans NFS -mount options.

- -

The only way I have been able to get working on Debian and RedHat -Enterprise Linux for getting the timeout count is to peek in /proc/. -But according to -Solaris -10 System Administration Guide: Network Services, the 'nfsstat -c' -command can be used to get these timeout values. But this do not work -on Linux, as far as I can tell. I -asked Debian about this, -but have not seen any replies yet.

- -

Is there a better way to figure out if a Linux NFS client is -experiencing NFS hangs? Is there a way to detect which processes are -affected? Is there a way to get the NFS mount going quickly once the -network problem causing the NFS hang has been cleared? I would very -much welcome some clues, as we regularly run into NFS hangs.

Datalagringsdirektivet kaster skygger over HÃ¸yre og Arbeiderpartiet

7th September 2017

For noen dager siden publiserte Jon Wessel-Aas en bloggpost om +Â«Konklusjonen om datalagring som +EU-kommisjonen ikke ville at vi skulle fÃ¥ seÂ». Det er en +interessant gjennomgang av EU-domstolens syn pÃ¥ snurpenotovervÃ¥kning +av befolkningen, som er klar pÃ¥ at det er i strid med +EU-lovgivingen.

+ +

Valgkampen gÃ¥r for fullt i Norge, og om noen fÃ¥ dager er siste +frist for Ã¥ avgi stemme. En ting er sikkert, HÃ¸yre og Arbeiderpartiet +fÃ¥r ikke min stemme +denne +gangen heller. Jeg har ikke glemt at de tvang igjennom loven som +skulle pÃ¥legge alle data- og teletjenesteleverandÃ¸rer Ã¥ overvÃ¥ke alle +sine kunder. En lov som er vedtatt, og aldri opphevet igjen.

+ +

Det er tydelig fra diskusjonen rundt grenselÃ¸s digital overvÃ¥kning +(eller "Digital Grenseforsvar" som det kalles i Orvellisk nytale) at +hverken HÃ¸yre og Arbeiderpartiet har noen prinsipielle sperrer mot Ã¥ +overvÃ¥ke hele befolkningen, og diskusjonen sÃ¥ langt tyder pÃ¥ at flere +av de andre partiene heller ikke har det. Mange av +de som stemte +for Datalagringsdirektivet i Stortinget (64 fra Arbeiderpartiet, +25 fra HÃ¸yre) er fortsatt aktive og argumenterer fortsatt for Ã¥ radere +vekk mer av innbyggernes privatsfÃ¦re.

+ +

NÃ¥r myndighetene demonstrerer sin mistillit til folket, tror jeg +folket selv bÃ¸r legge litt innsats i Ã¥ verne sitt privatliv, ved Ã¥ ta +i bruk ende-til-ende-kryptert kommunikasjon med sine kjente og kjÃ¦re, +og begrense hvor mye privat informasjon som deles med uvedkommende. +Det er jo ingenting som tyder pÃ¥ at myndighetene kommer til Ã¥ vÃ¦re vÃ¥r +privatsfÃ¦re. +Det +er mange muligheter. Selv har jeg litt sans for +Ring, som er basert pÃ¥ p2p-teknologi +uten sentral kontroll, er fri programvare, og stÃ¸tter meldinger, tale +og video. Systemet er tilgjengelig ut av boksen fra +Debian og +Ubuntu, og det finnes pakker for Android, MacOSX +og Windows. ForelÃ¸pig er det fÃ¥ brukere med Ring, slik at jeg ogsÃ¥ +bruker Signal som +nettleserutvidelse.

- Tags: debian, english, sysadmin. + Tags: dld, norsk, personvern, stortinget, surveillance, valg.

@@ -135,44 +75,96 @@ much welcome some clues, as we regularly run into NFS hangs.

How does it feel to be wiretapped, when you should be doing the wiretapping...

8th March 2017

So the new president in the United States of America claim to be -surprised to discover that he was wiretapped during the election -before he was elected president. He even claim this must be illegal. -Well, doh, if it is one thing the confirmations from Snowden -documented, it is that the entire population in USA is wiretapped, one -way or another. Of course the president candidates were wiretapped, -alongside the senators, judges and the rest of the people in USA.

- -

Next, the Federal Bureau of Investigation ask the Department of -Justice to go public rejecting the claims that Donald Trump was -wiretapped illegally. I fail to see the relevance, given that I am -sure the surveillance industry in USA believe they have all the legal -backing they need to conduct mass surveillance on the entire -world.

- -

There is even the director of the FBI stating that he never saw an -order requesting wiretapping of Donald Trump. That is not very -surprising, given how the FISA court work, with all its activity being -secret. Perhaps he only heard about it?

- -

What I find most sad in this story is how Norwegian journalists -present it. In a news reports the other day in the radio from the -Norwegian National broadcasting Company (NRK), I heard the journalist -claim that 'the FBI denies any wiretapping', while the reality is that -'the FBI denies any illegal wiretapping'. There is a fundamental and -important difference, and it make me sad that the journalists are -unable to grasp it.

- -

Update 2017-03-13: Look like -The -Intercept report that US Senator Rand Paul confirm what I state above.

Simpler recipe on how to make a simple $7 IMSI Catcher using Debian

9th August 2017

On friday, I came across an interesting article in the Norwegian +web based ICT news magazine digi.no on +how +to collect the IMSI numbers of nearby cell phones using the cheap +DVB-T software defined radios. The article refered to instructions +and a recipe by +Keld Norman on Youtube on how to make a simple $7 IMSI Catcher, and I decided to test them out.

+ +

The instructions said to use Ubuntu, install pip using apt (to +bypass apt), use pip to install pybombs (to bypass both apt and pip), +and the ask pybombs to fetch and build everything you need from +scratch. I wanted to see if I could do the same on the most recent +Debian packages, but this did not work because pybombs tried to build +stuff that no longer build with the most recent openssl library or +some other version skew problem. While trying to get this recipe +working, I learned that the apt->pip->pybombs route was a long detour, +and the only piece of software dependency missing in Debian was the +gr-gsm package. I also found out that the lead upstream developer of +gr-gsm (the name stand for GNU Radio GSM) project already had a set of +Debian packages provided in an Ubuntu PPA repository. All I needed to +do was to dget the Debian source package and built it.

+ +

The IMSI collector is a python script listening for packages on the +loopback network device and printing to the terminal some specific GSM +packages with IMSI numbers in them. The code is fairly short and easy +to understand. The reason this work is because gr-gsm include a tool +to read GSM data from a software defined radio like a DVB-T USB stick +and other software defined radios, decode them and inject them into a +network device on your Linux machine (using the loopback device by +default). This proved to work just fine, and I've been testing the +collector for a few days now.

+ +

The updated and simpler recipe is thus to

+ +

start with a Debian machine running Stretch or newer,
build and install the gr-gsm package available from +http://ppa.launchpad.net/ptrkrysik/gr-gsm/ubuntu/pool/main/g/gr-gsm/,
clone the git repostory from https://github.com/Oros42/IMSI-catcher,
run grgsm_livemon and adjust the frequency until the terminal +where it was started is filled with a stream of text (meaning you +found a GSM station).
go into the IMSI-catcher directory and run 'sudo python simple_IMSI-catcher.py' to extract the IMSI numbers.

+ +

To make it even easier in the future to get this sniffer up and +running, I decided to package +the gr-gsm project +for Debian (WNPP +#871055), and the package was uploaded into the NEW queue today. +Luckily the gnuradio maintainer has promised to help me, as I do not +know much about gnuradio stuff yet.

+ +

I doubt this "IMSI cacher" is anywhere near as powerfull as +commercial tools like +The +Spy Phone Portable IMSI / IMEI Catcher or the +Harris +Stingray, but I hope the existance of cheap alternatives can make +more people realise how their whereabouts when carrying a cell phone +is easily tracked. Seeing the data flow on the screen, realizing that +I live close to a police station and knowing that the police is also +wearing cell phones, I wonder how hard it would be for criminals to +track the position of the police officers to discover when there are +police near by, or for foreign military forces to track the location +of the Norwegian military forces, or for anyone to track the location +of government officials...

+ +

It is worth noting that the data reported by the IMSI-catcher +script mentioned above is only a fraction of the data broadcasted on +the GSM network. It will only collect one frequency at the time, +while a typical phone will be using several frequencies, and not all +phones will be using the frequencies tracked by the grgsm_livemod +program. Also, there is a lot of radio chatter being ignored by the +simple_IMSI-catcher script, which would be collected by extending the +parser code. I wonder if gr-gsm can be set up to listen to more than +one frequency?

- Tags: english, surveillance. + Tags: debian, english, personvern, surveillance.

@@ -180,28 +172,32 @@ Intercept report that US Senator Rand Paul confirm what I state above.

Norwegian BokmÃ¥l translation of The Debian Administrator's Handbook complete, proofreading in progress

3rd March 2017

For almost a year now, we have been working on making a Norwegian -BokmÃ¥l edition of The Debian -Administrator's Handbook. Now, thanks to the tireless effort of -Ole-Erik, Ingrid and Andreas, the initial translation is complete, and -we are working on the proof reading to ensure consistent language and -use of correct computer science terms. The plan is to make the book -available on paper, as well as in electronic form. For that to -happen, the proof reading must be completed and all the figures need -to be translated. If you want to help out, get in touch.

- -

A - -fresh PDF edition in A4 format (the final book will have smaller -pages) of the book created every morning is available for -proofreading. If you find any errors, please -visit -Weblate and correct the error. The -state -of the translation including figures is a useful source for those -provide Norwegian bokmÃ¥l screen shots and figures.

Norwegian BokmÃ¥l edition of Debian Administrator's Handbook is now available

25th July 2017

+ +

I finally received a copy of the Norwegian BokmÃ¥l edition of +"The Debian Administrator's +Handbook". This test copy arrived in the mail a few days ago, and +I am very happy to hold the result in my hand. We spent around one and a half year translating it. This paperbook edition +is available +from lulu.com. If you buy it quickly, you save 25% on the list +price. The book is also available for download in electronic form as +PDF, EPUB and Mobipocket, as can be +read online +as a web page.

+ +

This is the second book I publish (the first was the book +"Free Culture" by Lawrence Lessig +in +English, +French +and +Norwegian +BokmÃ¥l), and I am very excited to finally wrap up this +project. I hope +"HÃ¥ndbok +for Debian-administratoren" will be well received.

@@ -214,77 +210,50 @@ provide Norwegian bokmÃ¥l screen shots and figures.

Unlimited randomness with the ChaosKey?

1st March 2017

A few days ago I ordered a small batch of -the ChaosKey, a small -USB dongle for generating entropy created by Bdale Garbee and Keith -Packard. Yesterday it arrived, and I am very happy to report that it -work great! According to its designers, to get it to work out of the -box, you need the Linux kernel version 4.1 or later. I tested on a -Debian Stretch machine (kernel version 4.9), and there it worked just -fine, increasing the available entropy very quickly. I wrote a small -test oneliner to test. It first print the current entropy level, -drain /dev/random, and then print the entropy level for five seconds. -Here is the situation without the ChaosKey inserted:

Â«Rapporten ser ikke pÃ¥ informasjonssikkerhet knyttet til personlig integritetÂ»

27th June 2017

Jeg kom over teksten +Â«Killing +car privacy by federal mandateÂ» av Leonid Reyzin pÃ¥ Freedom to +Tinker i dag, og det gleder meg Ã¥ se en god gjennomgang om hvorfor det +er et urimelig inngrep i privatsfÃ¦ren Ã¥ la alle biler kringkaste sin +posisjon og bevegelse via radio. Det omtalte forslaget basert pÃ¥ +Dedicated Short Range Communication (DSRC) kalles Basic Safety Message +(BSM) i USA og Cooperative Awareness Message (CAM) i Europa, og det +norske Vegvesenet er en av de som ser ut til Ã¥ kunne tenke seg Ã¥ +pÃ¥legge alle biler Ã¥ fjerne nok en bit av innbyggernes privatsfÃ¦re. +Anbefaler alle Ã¥ lese det som stÃ¥r der. + +

Mens jeg tittet litt pÃ¥ DSRC pÃ¥ biler i Norge kom jeg over et sitat +jeg synes er illustrativt for hvordan det offentlige Norge hÃ¥ndterer +problemstillinger rundt innbyggernes privatsfÃ¦re i SINTEF-rapporten +Â«Informasjonssikkerhet +i AutoPASS-brikkerÂ» av Trond Foss:

-% cat /proc/sys/kernel/random/entropy_avail; \
-  dd bs=1M if=/dev/random of=/dev/null count=1; \
-  for n in $(seq 1 5); do \
-     cat /proc/sys/kernel/random/entropy_avail; \
-     sleep 1; \
-  done
-300
-0+1 oppfÃ¸ringer inn
-0+1 oppfÃ¸ringer ut
-28 byte kopiert, 0,000264565 s, 106 kB/s
-4
-8
-12
-17
-21
-%
-

- -

The entropy level increases by 3-4 every second. In such case any -application requiring random bits (like a HTTPS enabled web server) -will halt and wait for more entrpy. And here is the situation with -the ChaosKey inserted:

- -

-% cat /proc/sys/kernel/random/entropy_avail; \
-  dd bs=1M if=/dev/random of=/dev/null count=1; \
-  for n in $(seq 1 5); do \
-     cat /proc/sys/kernel/random/entropy_avail; \
-     sleep 1; \
-  done
-1079
-0+1 oppfÃ¸ringer inn
-0+1 oppfÃ¸ringer ut
-104 byte kopiert, 0,000487647 s, 213 kB/s
-433
-1028
-1031
-1035
-1038
-%
-

- -

Quite the difference. :) I bought a few more than I need, in case -someone want to buy one here in Norway. :)

+Â«Rapporten ser ikke pÃ¥ informasjonssikkerhet knyttet til personlig + integritet.Â» +

Update: The dongle was presented at Debconf last year. You might -find the talk -recording illuminating. It explains exactly what the source of -randomness is, if you are unable to spot it from the schema drawing -available from the ChaosKey web site linked at the start of this blog -post.

SÃ¥ enkelt kan det tydeligvis gjÃ¸res nÃ¥r en vurderer +informasjonssikkerheten. Det holder vel at folkene pÃ¥ toppen kan si +at Â«Personvernet er ivaretattÂ», som jo er den populÃ¦re intetsigende +frasen som gjÃ¸r at mange tror enkeltindividers integritet tas vare pÃ¥. +Sitatet fikk meg til Ã¥ undres pÃ¥ hvor ofte samme tilnÃ¦rming, Ã¥ bare se +bort fra behovet for personlig itegritet, blir valgt nÃ¥r en velger Ã¥ +legge til rette for nok et inngrep i privatsfÃ¦ren til personer i +Norge. Det er jo sjelden det fÃ¥r reaksjoner. Historien om +reaksjonene pÃ¥ Helse SÃ¸r-Ãsts tjenesteutsetting er jo sÃ¸rgelig nok et +unntak og toppen av isfjellet, desverre. Tror jeg fortsatt takker nei +til bÃ¥de AutoPASS og holder meg sÃ¥ langt unna det norske helsevesenet +som jeg kan, inntil de har demonstrert og dokumentert at de verdsetter +individets privatsfÃ¦re og personlige integritet hÃ¸yere enn kortsiktig +gevist og samfunnsnytte.

- Tags: debian, english. + Tags: norsk, personvern, sikkerhet.

@@ -292,34 +261,66 @@ post.

Detect OOXML files with undefined behaviour?

21st February 2017

I just noticed -the -new Norwegian proposal for archiving rules in the goverment list -ECMA-376 -/ ISO/IEC 29500 (aka OOXML) as valid formats to put in long term -storage. Luckily such files will only be accepted based on -pre-approval from the National Archive. Allowing OOXML files to be -used for long term storage might seem like a good idea as long as we -forget that there are plenty of ways for a "valid" OOXML document to -have content with no defined interpretation in the standard, which -lead to a question and an idea.

- -

Is there any tool to detect if a OOXML document depend on such -undefined behaviour? It would be useful for the National Archive (and -anyone else interested in verifying that a document is well defined) -to have such tool available when considering to approve the use of -OOXML. I'm aware of the -officeotron OOXML -validator, but do not know how complete it is nor if it will -report use of undefined behaviour. Are there other similar tools -available? Please send me an email if you know of any such tool.

Updated sales number for my Free Culture paper editions

12th June 2017

It is pleasing to see that the work we put down in publishing new +editions of the classic Free +Culture book by the founder of the Creative Commons movement, +Lawrence Lessig, is still being appreciated. I had a look at the +latest sales numbers for the paper edition today. Not too impressive, +but happy to see some buyers still exist. All the revenue from the +books is sent to the Creative +Commons Corporation, and they receive the largest cut if you buy +directly from Lulu. Most books are sold via Amazon, with Ingram +second and only a small fraction directly from Lulu. The ebook +edition is available for free from +Github.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Title / language	Quantity
Title / language	2016 jan-jun	2016 jul-dec	2017 jan-may
Culture Libre / French	3	6	15
Fri kultur / Norwegian	7	1	0
Free Culture / English	14	27	16
Total	24	34	31

+ +

A bit sad to see the low sales number on the Norwegian edition, and +a bit surprising the English edition still selling so well.

+ +

If you would like to translate and publish the book in your native +language, I would be happy to help make it happen. Please get in +touch.

- Tags: english, nuug, standard. + Tags: docbook, english, freeculture.

@@ -327,33 +328,59 @@ available? Please send me an email if you know of any such tool.

Ruling ignored our objections to the seizure of popcorn-time.no (#domstolkontroll)

13th February 2017

A few days ago, we received the ruling from -my -day in court. The case in question is a challenge of the seizure -of the DNS domain popcorn-time.no. The ruling simply did not mention -most of our arguments, and seemed to take everything ÃKOKRIM said at -face value, ignoring our demonstration and explanations. But it is -hard to tell for sure, as we still have not seen most of the documents -in the case and thus were unprepared and unable to contradict several -of the claims made in court by the opposition. We are considering an -appeal, but it is partly a question of funding, as it is costing us -quite a bit to pay for our lawyer. If you want to help, please -donate to the -NUUG defense fund.

- -

The details of the case, as far as we know it, is available in -Norwegian from -the NUUG -blog. This also include -the -ruling itself.

Release 0.1.1 of free software archive system Nikita announced

10th June 2017

I am very happy to report that the +Nikita Noark 5 +core project tagged its second release today. The free software +solution is an implementation of the Norwegian archive standard Noark +5 used by government offices in Norway. These were the changes in +version 0.1.1 since version 0.1.0 (from NEWS.md): + +

Continued work on the angularjs GUI, including document upload.
Implemented correspondencepartPerson, correspondencepartUnit and + correspondencepartInternal
Applied for coverity coverage and started submitting code on + regualr basis.
Started fixing bugs reported by coverity
Corrected and completed HATEOAS links to make sure entire API is + available via URLs in _links.
Corrected all relation URLs to use trailing slash.
Add initial support for storing data in ElasticSearch.
Now able to receive and store uploaded files in the archive.
Changed JSON output for object lists to have relations in _links.
Improve JSON output for empty object lists.
Now uses correct MIME type application/vnd.noark5-v4+json.
Added support for docker container images.
Added simple API browser implemented in JavaScript/Angular.
Started on archive client implemented in JavaScript/Angular.
Started on prototype to show the public mail journal.
Improved performance by disabling Sprint FileWatcher.
Added support for 'arkivskaper', 'saksmappe' and 'journalpost'.
Added support for some metadata codelists.
Added support for Cross-origin resource sharing (CORS).
Changed login method from Basic Auth to JSON Web Token (RFC 7519) + style.
Added support for GET-ing ny-* URLs.
Added support for modifying entities using PUT and eTag.
Added support for returning XML output on request.
Removed support for English field and class names, limiting ourself + to the official names.
...

+ +

If this sound interesting to you, please contact us on IRC (#nikita +on irc.freenode.net) or email +(nikita-noark +mailing list).

- Tags: english, nuug, offentlig innsyn, opphavsrett. + Tags: english, nuug, offentlig innsyn, standard.

@@ -361,86 +388,99 @@ ruling itself.

A day in court challenging seizure of popcorn-time.no for #domstolkontroll

3rd February 2017

- -

On Wednesday, I spent the entire day in court in Follo Tingrett -representing the member association -NUUG, alongside the member -association EFN and the DNS registrar -IMC, challenging the seizure of the DNS name popcorn-time.no. It -was interesting to sit in a court of law for the first time in my -life. Our team can be seen in the picture above: attorney Ola -TellesbÃ¸, EFN board member Tom Fredrik Blenning, IMC CEO Morten Emil -Eriksen and NUUG board member Petter Reinholdtsen.

- -

The -case at hand is that the Norwegian National Authority for -Investigation and Prosecution of Economic and Environmental Crime (aka -Ãkokrim) decided on their own, to seize a DNS domain early last -year, without following -the -official policy of the Norwegian DNS authority which require a -court decision. The web site in question was a site covering Popcorn -Time. And Popcorn Time is the name of a technology with both legal -and illegal applications. Popcorn Time is a client combining -searching a Bittorrent directory available on the Internet with -downloading/distribute content via Bittorrent and playing the -downloaded content on screen. It can be used illegally if it is used -to distribute content against the will of the right holder, but it can -also be used legally to play a lot of content, for example the -millions of movies -available from the -Internet Archive or the collection -available from Vodo. We created -a -video demonstrating legally use of Popcorn Time and played it in -Court. It can of course be downloaded using Bittorrent.

- -

I did not quite know what to expect from a day in court. The -government held on to their version of the story and we held on to -ours, and I hope the judge is able to make sense of it all. We will -know in two weeks time. Unfortunately I do not have high hopes, as -the Government have the upper hand here with more knowledge about the -case, better training in handling criminal law and in general higher -standing in the courts than fairly unknown DNS registrar and member -associations. It is expensive to be right also in Norway. So far the -case have cost more than NOK 70 000,-. To help fund the case, NUUG -and EFN have asked for donations, and managed to collect around NOK 25 -000,- so far. Given the presentation from the Government, I expect -the government to appeal if the case go our way. And if the case do -not go our way, I hope we have enough funding to appeal.

- -

From the other side came two people from Ãkokrim. On the benches, -appearing to be part of the group from the government were two people -from the Simonsen Vogt Wiik lawyer office, and three others I am not -quite sure who was. Ãkokrim had proposed to present two witnesses -from The Motion Picture Association, but this was rejected because -they did not speak Norwegian and it was a bit late to bring in a -translator, but perhaps the two from MPA were present anyway. All -seven appeared to know each other. Good to see the case is take -seriously.

- -

If you, like me, believe the courts should be involved before a DNS -domain is hijacked by the government, or you believe the Popcorn Time -technology have a lot of useful and legal applications, I suggest you -too donate to -the NUUG defense fund. Both Bitcoin and bank transfer are -available. If NUUG get more than we need for the legal action (very -unlikely), the rest will be spend promoting free software, open -standards and unix-like operating systems in Norway, so no matter what -happens the money will be put to good use.

- -

If you want to lean more about the case, I recommend you check out -the blog -posts from NUUG covering the case. They cover the legal arguments -on both sides.

Idea for storing trusted timestamps in a Noark 5 archive

7th June 2017

This is a copy of +an +email I posted to the nikita-noark mailing list. Please follow up +there if you would like to discuss this topic. The background is that +we are making a free software archive system based on the Norwegian +Noark +5 standard for government archives.

+ +

I've been wondering a bit lately how trusted timestamps could be +stored in Noark 5. +Trusted +timestamps can be used to verify that some information +(document/file/checksum/metadata) have not been changed since a +specific time in the past. This is useful to verify the integrity of +the documents in the archive.

+ +

Then it occured to me, perhaps the trusted timestamps could be +stored as dokument variants (ie dokumentobjekt referered to from +dokumentbeskrivelse) with the filename set to the hash it is +stamping?

+ +

Given a "dokumentbeskrivelse" with an associated "dokumentobjekt", +a new dokumentobjekt is associated with "dokumentbeskrivelse" with the +same attributes as the stamped dokumentobjekt except these +attributes:

+ +

format -> "RFC3161" +
mimeType -> "application/timestamp-reply" +
formatDetaljer -> "<source URL for timestamp service>" +
filenavn -> "<sjekksum>.tsr" + +

+ +

This assume a service following +IETF RFC 3161 is +used, which specifiy the given MIME type for replies and the .tsr file +ending for the content of such trusted timestamp. As far as I can +tell from the Noark 5 specifications, it is OK to have several +variants/renderings of a dokument attached to a given +dokumentbeskrivelse objekt. It might be stretching it a bit to make +some of these variants represent crypto-signatures useful for +verifying the document integrity instead of representing the dokument +itself.

+ +

Using the source of the service in formatDetaljer allow several +timestamping services to be used. This is useful to spread the risk +of key compromise over several organisations. It would only be a +problem to trust the timestamps if all of the organisations are +compromised.

+ +

The following oneliner on Linux can be used to generate the tsr +file. $input is the path to the file to checksum, and $sha256 is the +SHA-256 checksum of the file (ie the ".tsr" value mentioned +above).

+ +

+openssl ts -query -data "$inputfile" -cert -sha256 -no_nonce \
+  | curl -s -H "Content-Type: application/timestamp-query" \
+      --data-binary "@-" http://zeitstempel.dfn.de > $sha256.tsr
+

+ +

To verify the timestamp, you first need to download the public key +of the trusted timestamp service, for example using this command:

+ +

+wget -O ca-cert.txt \
+  https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt
+

+ +

Note, the public key should be stored alongside the timestamps in +the archive to make sure it is also available 100 years from now. It +is probably a good idea to standardise how and were to store such +public keys, to make it easier to find for those trying to verify +documents 100 or 1000 years from now. :)

+ +

The verification itself is a simple openssl command:

+ +

+openssl ts -verify -data $inputfile -in $sha256.tsr \
+  -CAfile ca-cert.txt -text
+

+ +

Is there any reason this approach would not work? Is it somehow against +the Noark 5 specification?

- Tags: english, nuug, offentlig innsyn, opphavsrett. + Tags: english, offentlig innsyn, standard.

@@ -448,41 +488,61 @@ on both sides.

Nasjonalbiblioteket avslutter sin ulovlige bruk av Google Skjemaer

12th January 2017

I dag fikk jeg en skikkelig gladmelding. Bakgrunnen er at fÃ¸r jul -arrangerte Nasjonalbiblioteket -et -seminar om sitt knakende gode tiltak Â«verksregisterÂ». Eneste -mÃ¥ten Ã¥ melde seg pÃ¥ dette seminaret var Ã¥ sende personopplysninger -til Google via Google Skjemaer. Dette syntes jeg var tvilsom praksis, -da det bÃ¸r vÃ¦re mulig Ã¥ delta pÃ¥ seminarer arrangert av det offentlige -uten Ã¥ mÃ¥tte dele sine interesser, posisjon og andre -personopplysninger med Google. Jeg ba derfor om innsyn via -Mimes brÃ¸nn i -avtaler -og vurderinger Nasjonalbiblioteket hadde rundt dette. -Personopplysningsloven legger klare rammer for hva som mÃ¥ vÃ¦re pÃ¥ -plass fÃ¸r en kan be tredjeparter, spesielt i utlandet, behandle -personopplysninger pÃ¥ sine vegne, sÃ¥ det burde eksistere grundig -dokumentasjon fÃ¸r noe slikt kan bli lovlig. To jurister hos -Nasjonalbiblioteket mente fÃ¸rst dette var helt i orden, og at Googles -standardavtale kunne brukes som databehandlingsavtale. Det syntes jeg -var merkelig, men har ikke hatt kapasitet til Ã¥ fÃ¸lge opp saken fÃ¸r -for to dager siden.

- -

Gladnyheten i dag, som kom etter at jeg tipset Nasjonalbiblioteket -om at Datatilsynet underkjente Googles standardavtaler som -databehandleravtaler i 2011, er at Nasjonalbiblioteket har bestemt seg -for Ã¥ avslutte bruken av Googles Skjemaer/Apps og gÃ¥ i dialog med DIFI -for Ã¥ finne bedre mÃ¥ter Ã¥ hÃ¥ndtere pÃ¥meldinger i trÃ¥d med -personopplysningsloven. Det er fantastisk Ã¥ se at av og til hjelper -det Ã¥ spÃ¸rre hva i alle dager det offentlige holder pÃ¥ med.

NÃ¥r nynorskoversettelsen svikter til eksamen...

3rd June 2017

Aftenposten +melder i dag om feil i eksamensoppgavene for eksamen i politikk og +menneskerettigheter, der teksten i bokmÃ¥ls og nynorskutgaven ikke var +like. Oppgaveteksten er gjengitt i artikkelen, og jeg ble nysgjerring +pÃ¥ om den fri oversetterlÃ¸sningen +Apertium ville gjort en bedre +jobb enn Utdanningsdirektoratet. Det kan se slik ut.

+ +

Her er bokmÃ¥lsoppgaven fra eksamenen:

+ +

+
DrÃ¸ft utfordringene knyttet til nasjonalstatenes og andre aktÃ¸rers +rolle og muligheter til Ã¥ hÃ¥ndtere internasjonale utfordringer, som +for eksempel flykningekrisen.
+ +
Vedlegge er eksempler pÃ¥ tekster som kan gi relevante perspektiver +pÃ¥ temaet:
+
+
Flykningeregnskapet 2016, UNHCR og IDMC +
Â«GrenselÃ¸st Europa for fallÂ» A-Magasinet, 26. november 2015 +
+ +

+ +

Dette oversetter Apertium slik:

+ +

+
DrÃ¸ft utfordringane knytte til nasjonalstatane sine og rolla til +andre aktÃ¸rar og hÃ¸ve til Ã¥ handtera internasjonale utfordringar, som +til dÃ¸mes *flykningekrisen.
+ +
Vedleggja er dÃ¸me pÃ¥ tekster som kan gje relevante perspektiv pÃ¥ +temaet:
+ +
+
*Flykningeregnskapet 2016, *UNHCR og *IDMC
+
Â«*GrenselÃ¸st Europa for fallÂ» A-Magasinet, 26. november 2015
+
+ +

+ +

Ord som ikke ble forstÃ¥tt er markert med stjerne (*), og trenger +ekstra sprÃ¥ksjekk. Men ingen ord er forsvunnet, slik det var i +oppgaven elevene fikk presentert pÃ¥ eksamen. Jeg mistenker dog at +"andre aktÃ¸rers rolle og muligheter til ..." burde vÃ¦rt oversatt til +"rolla til andre aktÃ¸rar og deira hÃ¸ve til ..." eller noe slikt, men +det er kanskje flisespikking. Det understreker vel bare at det alltid +trengs korrekturlesning etter automatisk oversettelse.

- Tags: norsk, personvern, surveillance, web. + Tags: debian, norsk, stavekontroll.

@@ -490,86 +550,67 @@ det Ã¥ spÃ¸rre hva i alle dager det offentlige holder pÃ¥ med.

Bryter NAV sin egen personvernerklÃ¦ring?

11th January 2017

Jeg leste med interesse en nyhetssak hos -digi.no -og -NRK -om at det ikke bare er meg, men at ogsÃ¥ NAV bedriver geolokalisering -av IP-adresser, og at det gjÃ¸res analyse av IP-adressene til de som -sendes inn meldekort for Ã¥ se om meldekortet sendes inn fra -utenlandske IP-adresser. Politiadvokat i Drammen, Hans Lyder Haare, -er sitert i NRK pÃ¥ at Â«De to er jo blant annet avslÃ¸rt av -IP-adresser. At man ser at meldekortet kommer fra utlandet.Â»

- -

Jeg synes det er fint at det blir bedre kjent at IP-adresser -knyttes til enkeltpersoner og at innsamlet informasjon brukes til Ã¥ -stedsbestemme personer ogsÃ¥ av aktÃ¸rer her i Norge. Jeg ser det som -nok et argument for Ã¥ bruke -Tor sÃ¥ mye som mulig for Ã¥ -gjÃ¸re gjÃ¸re IP-lokalisering vanskeligere, slik at en kan beskytte sin -privatsfÃ¦re og unngÃ¥ Ã¥ dele sin fysiske plassering med -uvedkommede.

- -

Men det er en ting som bekymrer meg rundt denne nyheten. Jeg ble -tipset (takk #nuug) om -NAVs -personvernerklÃ¦ring, som under punktet Â«Personvern og statistikkÂ» -lyder:

Epost inn som arkivformat i Riksarkivarens forskrift?

27th April 2017

I disse dager, med frist 1. mai, har Riksarkivaren ute en hÃ¸ring pÃ¥ +sin forskrift. Som en kan se er det ikke mye tid igjen fÃ¸r fristen +som gÃ¥r ut pÃ¥ sÃ¸ndag. Denne forskriften er det som lister opp hvilke +formater det er greit Ã¥ arkivere i +Noark +5-lÃ¸sninger i Norge.

+ +

Jeg fant hÃ¸ringsdokumentene hos +Norsk +ArkivrÃ¥d etter Ã¥ ha blitt tipset pÃ¥ epostlisten til +fri +programvareprosjektet Nikita Noark5-Core, som lager et Noark 5 +Tjenestegresesnitt. Jeg er involvert i Nikita-prosjektet og takket +vÃ¦re min interesse for tjenestegrensesnittsprosjektet har jeg lest en +god del Noark 5-relaterte dokumenter, og til min overraskelse oppdaget +at standard epost ikke er pÃ¥ listen over godkjente formater som kan +arkiveres. HÃ¸ringen med frist sÃ¸ndag er en glimrende mulighet til Ã¥ +forsÃ¸ke Ã¥ gjÃ¸re noe med det. Jeg holder pÃ¥ med +egen +hÃ¸ringsuttalelse, og lurer pÃ¥ om andre er interessert i Ã¥ stÃ¸tte +forslaget om Ã¥ tillate arkivering av epost som epost i arkivet.

+ +

Er du igang med Ã¥ skrive egen hÃ¸ringsuttalelse allerede? I sÃ¥ fall +kan du jo vurdere Ã¥ ta med en formulering om epost-lagring. Jeg tror +ikke det trengs sÃ¥ mye. Her et kort forslag til tekst:

-
Â«NÃ¥r du besÃ¸ker nav.no, etterlater du deg elektroniske spor. Sporene -dannes fordi din nettleser automatisk sender en rekke opplysninger til -NAVs tjener (server-maskin) hver gang du ber om Ã¥ fÃ¥ vist en side. Det -er eksempelvis opplysninger om hvilken nettleser og -versjon du -bruker, og din internettadresse (ip-adresse). For hver side som vises, -lagres fÃ¸lgende opplysninger:
+
Viser til hÃ¸ring sendt ut 2017-02-17 (Riksarkivarens referanse + 2016/9840 HELHJO), og tillater oss Ã¥ sende inn noen innspill om + revisjon av Forskrift om utfyllende tekniske og arkivfaglige + bestemmelser om behandling av offentlige arkiver (Riksarkivarens + forskrift).
-
-
hvilken side du ser pÃ¥
-
dato og tid
-
hvilken nettleser du bruker
-
din ip-adresse
-
- -
Ingen av opplysningene vil bli brukt til Ã¥ identifisere -enkeltpersoner. NAV bruker disse opplysningene til Ã¥ generere en -samlet statistikk som blant annet viser hvilke sider som er mest -populÃ¦re. Statistikken er et redskap til Ã¥ forbedre vÃ¥re -tjenester.Â»
+
SvÃ¦rt mye av vÃ¥r kommuikasjon foregÃ¥r i dag pÃ¥ e-post.Â Vi + foreslÃ¥r derfor at Internett-e-post, slik det er beskrevet i IETF + RFC 5322, + https://tools.ietf.org/html/rfc5322. bÃ¸r + inn som godkjent dokumentformat.Â Vi foreslÃ¥r at forskriftens + oversikt over godkjente dokumentformater ved innlevering i Â§ 5-16 + endres til Ã¥ ta med Internett-e-post.

Jeg klarer ikke helt Ã¥ se hvordan analyse av de besÃ¸kendes -IP-adresser for Ã¥ se hvem som sender inn meldekort via web fra en -IP-adresse i utlandet kan gjÃ¸res uten Ã¥ komme i strid med pÃ¥standen om -at Â«ingen av opplysningene vil bli brukt til Ã¥ identifisere -enkeltpersonerÂ». Det virker dermed for meg som at NAV bryter sine -egen personvernerklÃ¦ring, hvilket -Datatilsynet -fortalte meg i starten av desember antagelig er brudd pÃ¥ -personopplysningsloven. - -

I tillegg er personvernerklÃ¦ringen ganske misvisende i og med at -NAVs nettsider ikke bare forsyner NAV med personopplysninger, men i -tillegg ber brukernes nettleser kontakte fem andre nettjenere -(script.hotjar.com, static.hotjar.com, vars.hotjar.com, -www.google-analytics.com og www.googletagmanager.com), slik at -personopplysninger blir gjort tilgjengelig for selskapene Hotjar og -Google , og alle som kan lytte pÃ¥ trafikken pÃ¥ veien (som FRA, GCHQ og -NSA). Jeg klarer heller ikke se hvordan slikt spredning av -personopplysninger kan vÃ¦re i trÃ¥d med kravene i -personopplysningloven, eller i trÃ¥d med NAVs personvernerklÃ¦ring.

- -

Kanskje NAV bÃ¸r ta en nÃ¸ye titt pÃ¥ sin personvernerklÃ¦ring? Eller -kanskje Datatilsynet bÃ¸r gjÃ¸re det?

Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan +epost kan lagres i en Noark 5-struktur, og holder pÃ¥ Ã¥ skrive et +forslag om hvordan dette kan gjÃ¸res som vil bli sendt over til +arkivverket sÃ¥ snart det er ferdig. De som er interesserte kan +fÃ¸lge +fremdriften pÃ¥ web.

+ +

Oppdatering 2017-04-28: I dag ble hÃ¸ringuttalelsen jeg skrev + sendt + inn av foreningen NUUG.

- Tags: norsk, nuug, personvern, surveillance. + Tags: norsk, offentlig innsyn, standard.

@@ -577,164 +618,52 @@ kanskje Datatilsynet bÃ¸r gjÃ¸re det?

Where did that package go? — geolocated IP traceroute

9th January 2017

Did you ever wonder where the web trafic really flow to reach the -web servers, and who own the network equipment it is flowing through? -It is possible to get a glimpse of this from using traceroute, but it -is hard to find all the details. Many years ago, I wrote a system to -map the Norwegian Internet (trying to figure out if our plans for a -network game service would get low enough latency, and who we needed -to talk to about setting up game servers close to the users. Back -then I used traceroute output from many locations (I asked my friends -to run a script and send me their traceroute output) to create the -graph and the map. The output from traceroute typically look like -this: - -

-traceroute to www.stortinget.no (85.88.67.10), 30 hops max, 60 byte packets
- 1  uio-gw10.uio.no (129.240.202.1)  0.447 ms  0.486 ms  0.621 ms
- 2  uio-gw8.uio.no (129.240.24.229)  0.467 ms  0.578 ms  0.675 ms
- 3  oslo-gw1.uninett.no (128.39.65.17)  0.385 ms  0.373 ms  0.358 ms
- 4  te3-1-2.br1.fn3.as2116.net (193.156.90.3)  1.174 ms  1.172 ms  1.153 ms
- 5  he16-1-1.cr1.san110.as2116.net (195.0.244.234)  2.627 ms he16-1-1.cr2.oslosda310.as2116.net (195.0.244.48)  3.172 ms he16-1-1.cr1.san110.as2116.net (195.0.244.234)  2.857 ms
- 6  ae1.ar8.oslosda310.as2116.net (195.0.242.39)  0.662 ms  0.637 ms ae0.ar8.oslosda310.as2116.net (195.0.242.23)  0.622 ms
- 7  89.191.10.146 (89.191.10.146)  0.931 ms  0.917 ms  0.955 ms
- 8  * * *
- 9  * * *
-[...]
-

- -

This show the DNS names and IP addresses of (at least some of the) -network equipment involved in getting the data traffic from me to the -www.stortinget.no server, and how long it took in milliseconds for a -package to reach the equipment and return to me. Three packages are -sent, and some times the packages do not follow the same path. This -is shown for hop 5, where three different IP addresses replied to the -traceroute request.

- -

There are many ways to measure trace routes. Other good traceroute -implementations I use are traceroute (using ICMP packages) mtr (can do -both ICMP, UDP and TCP) and scapy (python library with ICMP, UDP, TCP -traceroute and a lot of other capabilities). All of them are easily -available in Debian.

- -

This time around, I wanted to know the geographic location of -different route points, to visualize how visiting a web page spread -information about the visit to a lot of servers around the globe. The -background is that a web site today often will ask the browser to get -from many servers the parts (for example HTML, JSON, fonts, -JavaScript, CSS, video) required to display the content. This will -leak information about the visit to those controlling these servers -and anyone able to peek at the data traffic passing by (like your ISP, -the ISPs backbone provider, FRA, GCHQ, NSA and others).

- -

Lets pick an example, the Norwegian parliament web site -www.stortinget.no. It is read daily by all members of parliament and -their staff, as well as political journalists, activits and many other -citizens of Norway. A visit to the www.stortinget.no web site will -ask your browser to contact 8 other servers: ajax.googleapis.com, -insights.hotjar.com, script.hotjar.com, static.hotjar.com, -stats.g.doubleclick.net, www.google-analytics.com, -www.googletagmanager.com and www.netigate.se. I extracted this by -asking PhantomJS to visit the -Stortinget web page and tell me all the URLs PhantomJS downloaded to -render the page (in HAR format using -their -netsniff example. I am very grateful to Gorm for showing me how -to do this). My goal is to visualize network traces to all IP -addresses behind these DNS names, do show where visitors personal -information is spread when visiting the page.

- -

When I had a look around for options, I could not find any good -free software tools to do this, and decided I needed my own traceroute -wrapper outputting KML based on locations looked up using GeoIP. KML -is easy to work with and easy to generate, and understood by several -of the GIS tools I have available. I got good help from by NUUG -colleague Anders Einar with this, and the result can be seen in -my -kmltraceroute git repository. Unfortunately, the quality of the -free GeoIP databases I could find (and the for-pay databases my -friends had access to) is not up to the task. The IP addresses of -central Internet infrastructure would typically be placed near the -controlling companies main office, and not where the router is really -located, as you can see from the -KML file I created using the GeoLite City dataset from MaxMind. - -

- -

I also had a look at the visual traceroute graph created by -the scrapy project, -showing IP network ownership (aka AS owner) for the IP address in -question. -The -graph display a lot of useful information about the traceroute in SVG -format, and give a good indication on who control the network -equipment involved, but it do not include geolocation. This graph -make it possible to see the information is made available at least for -UNINETT, Catchcom, Stortinget, Nordunet, Google, Amazon, Telia, Level -3 Communications and NetDNA.

- -

In the process, I came across the -web service GeoTraceroute by -Salim Gasmi. Its methology of combining guesses based on DNS names, -various location databases and finally use latecy times to rule out -candidate locations seemed to do a very good job of guessing correct -geolocation. But it could only do one trace at the time, did not have -a sensor in Norway and did not make the geolocations easily available -for postprocessing. So I contacted the developer and asked if he -would be willing to share the code (he refused until he had time to -clean it up), but he was interested in providing the geolocations in a -machine readable format, and willing to set up a sensor in Norway. So -since yesterday, it is possible to run traces from Norway in this -service thanks to a sensor node set up by -the NUUG assosiation, and get the -trace in KML format for further processing.

- -

Here we can see a lot of trafic passes Sweden on its way to -Denmark, Germany, Holland and Ireland. Plenty of places where the -Snowden confirmations verified the traffic is read by various actors -without your best interest as their top priority.

- -

Combining KML files is trivial using a text editor, so I could loop -over all the hosts behind the urls imported by www.stortinget.no and -ask for the KML file from GeoTraceroute, and create a combined KML -file with all the traces (unfortunately only one of the IP addresses -behind the DNS name is traced this time. To get them all, one would -have to request traces using IP number instead of DNS names from -GeoTraceroute). That might be the next step in this project.

- -

Armed with these tools, I find it a lot easier to figure out where -the IP traffic moves and who control the boxes involved in moving it. -And every time the link crosses for example the Swedish border, we can -be sure Swedish Signal Intelligence (FRA) is listening, as GCHQ do in -Britain and NSA in USA and cables around the globe. (Hm, what should -we tell them? :) Keep that in mind if you ever send anything -unencrypted over the Internet.

- -

PS: KML files are drawn using -the KML viewer from Ivan -Rublev , as it was less cluttered than the local Linux application -Marble. There are heaps of other options too.

- -

As usual, if you use Bitcoin and want to show your support of my -activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Offentlig elektronisk postjournal blokkerer tilgang for utvalgte webklienter

20th April 2017

Jeg oppdaget i dag at nettstedet som +publiserer offentlige postjournaler fra statlige etater, OEP, har +begynt Ã¥ blokkerer enkelte typer webklienter fra Ã¥ fÃ¥ tilgang. Vet +ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl +og curl. For Ã¥ teste selv, kjÃ¸r fÃ¸lgende:

+ +

+% curl -v -s https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
+< HTTP/1.1 404 Not Found
+% curl -v -s --header 'User-Agent:Opera/12.0' https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
+< HTTP/1.1 200 OK
+%
+

+ +

Her kan en se at tjenesten gir Â«404 Not FoundÂ» for curl i +standardoppsettet, mens den gir Â«200 OKÂ» hvis curl hevder Ã¥ vÃ¦re Opera +versjon 12.0. Offentlig elektronisk postjournal startet blokkeringen +2017-03-02.

+ +

Blokkeringen vil gjÃ¸re det litt vanskeligere Ã¥ maskinelt hente +informasjon fra oep.no. Kan blokkeringen vÃ¦re gjort for Ã¥ hindre +automatisert innsamling av informasjon fra OEP, slik Pressens +Offentlighetsutvalg gjorde for Ã¥ dokumentere hvordan departementene +hindrer innsyn i +rapporten +Â«Slik hindrer departementer innsynÂ» som ble publiserte i januar +2017. Det virker usannsynlig, da det jo er trivielt Ã¥ bytte +User-Agent til noe nytt.

+ +

Finnes det juridisk grunnlag for det offentlige Ã¥ diskriminere +webklienter slik det gjÃ¸res her? Der tilgang gis eller ikke alt etter +hva klienten sier at den heter? Da OEP eies av DIFI og driftes av +Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to +aktÃ¸rene man kan be om innsyn i for Ã¥ forstÃ¥ hva som har skjedd. Men +postjournalen +til DIFI viser kun to dokumenter det siste Ã¥ret mellom DIFI og +Basefarm. +Mimes brÃ¸nn neste, +tenker jeg.