syslog-trusted-timestamp - chain of trusted timestamps for your syslog

2nd April 2016

Two years ago, I had -a -look at trusted timestamping options available, and among -other things noted a still open -bug in the tsget script -included in openssl that made it harder than necessary to use openssl -as a trusted timestamping client. A few days ago I was told -the Norwegian government office DIFI is -close to releasing their own trusted timestamp service, and in the -process I was happy to learn about a replacement for the tsget script -using only curl:

- -

-openssl ts -query -data "/etc/shells" -cert -sha256 -no_nonce \
-  | curl -s -H "Content-Type: application/timestamp-query" \
-         --data-binary "@-" http://zeitstempel.dfn.de > etc-shells.tsr
-openssl ts -reply -text -in etc-shells.tsr
-

- -

This produces a binary timestamp file (etc-shells.tsr) which can be -used to verify that the content of the file /etc/shell with the -calculated sha256 hash existed at the point in time when the request -was made. The last command extract the content of the etc-shells.tsr -in human readable form. The idea behind such timestamp is to be able -to prove using cryptography that the content of a file have not -changed since the file was stamped.

- -

To verify that the file on disk match the public key signature in -the timestamp file, run the following commands. It make sure you have -the required certificate for the trusted timestamp service available -and use it to compare the file content with the timestamp. In -production, one should of course use a better method to verify the -service certificate.

- -

-wget -O ca-cert.txt https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt
-openssl ts -verify -data /etc/shells -in etc-shells.tsr -CAfile ca-cert.txt -text
-

- -

Wikipedia have a lot more information about -trusted -Timestamping and -linked -timestamping, and there are several trusted timestamping services -around, both as commercial services and as free and public services. -Among the latter is -the -zeitstempel.dfn.de service mentioned above and -freetsa.org service linked to from the -wikipedia web site. I believe the DIFI service should show up on -https://tsa.difi.no, but it is not available to the public at the -moment. I hope this will change when it is into production. The -RFC 3161 trusted -timestamping protocol standard is even implemented in LibreOffice, -Microsoft Office and Adobe Acrobat, making it possible to verify when -a document was created.

- -

I would find it useful to be able to use such trusted timestamp -service to make it possible to verify that my stored syslog files have -not been tampered with. This is not a new idea. I found one example -implemented on the Endian network appliances where -the -configuration of such feature was described in 2012.

- -

But I could not find any free implementation of such feature when I -searched, so I decided to try to -build -a prototype named syslog-trusted-timestamp. My idea is to -generate a timestamp of the old log files after they are rotated, and -store the timestamp in the new log file just after rotation. This -will form a chain that would make it possible to see if any old log -files are tampered with. But syslog is bad at handling kilobytes of -binary data, so I decided to base64 encode the timestamp and add an ID -and line sequence numbers to the base64 data to make it possible to -reassemble the timestamp file again. To use it, simply run it like -this: - -

-syslog-trusted-timestamp /path/to/list-of-log-files
-

- -

This will send a timestamp from one or more timestamp services (not -yet decided nor implemented) for each listed file to the syslog using -logger(1). To verify the timestamp, the same program is used with the ---verify option:

- -

-syslog-trusted-timestamp --verify /path/to/log-file /path/to/log-with-timestamp
-

- -

The verification step is not yet well designed. The current -implementation depend on the file path being unique and unchanging, -and this is not a solid assumption. It also uses process number as -timestamp ID, and this is bound to create ID collisions. I hope to -have time to come up with a better way to handle timestamp IDs and -verification later.

- -

Please check out -the -prototype for syslog-trusted-timestamp on github and send -suggestions and improvement, or let me know if there already exist a -similar system for timestamping logs already to allow me to join -forces with others with the same interest.

- -

As usual, if you use Bitcoin and want to show your support of my -activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Simpler recipe on how to make a simple $7 IMSI Catcher using Debian

9th August 2017

On friday, I came across an interesting article in the Norwegian +web based ICT news magazine digi.no on +how +to collect the IMSI numbers of nearby cell phones using the cheap +DVB-T software defined radios. The article refered to instructions +and a recipe by +Keld Norman on Youtube on how to make a simple $7 IMSI Catcher, and I decided to test them out.

+ +

The instructions said to use Ubuntu, install pip using apt (to +bypass apt), use pip to install pybombs (to bypass both apt and pip), +and the ask pybombs to fetch and build everything you need from +scratch. I wanted to see if I could do the same on the most recent +Debian packages, but this did not work because pybombs tried to build +stuff that no longer build with the most recent openssl library or +some other version skew problem. While trying to get this recipe +working, I learned that the apt->pip->pybombs route was a long detour, +and the only piece of software dependency missing in Debian was the +gr-gsm package. I also found out that the lead upstream developer of +gr-gsm (the name stand for GNU Radio GSM) project already had a set of +Debian packages provided in an Ubuntu PPA repository. All I needed to +do was to dget the Debian source package and built it.

+ +

The IMSI collector is a python script listening for packages on the +loopback network device and printing to the terminal some specific GSM +packages with IMSI numbers in them. The code is fairly short and easy +to understand. The reason this work is because gr-gsm include a tool +to read GSM data from a software defined radio like a DVB-T USB stick +and other software defined radios, decode them and inject them into a +network device on your Linux machine (using the loopback device by +default). This proved to work just fine, and I've been testing the +collector for a few days now.

+ +

The updated and simpler recipe is thus to

+ +

start with a Debian machine running Stretch or newer,
build and install the gr-gsm package available from +http://ppa.launchpad.net/ptrkrysik/gr-gsm/ubuntu/pool/main/g/gr-gsm/,
clone the git repostory from https://github.com/Oros42/IMSI-catcher,
run grgsm_livemon and adjust the frequency until the terminal +where it was started is filled with a stream of text (meaning you +found a GSM station).
go into the IMSI-catcher directory and run 'sudo python simple_IMSI-catcher.py' to extract the IMSI numbers.

+ +

To make it even easier in the future to get this sniffer up and +running, I decided to package +the gr-gsm project +for Debian (WNPP +#871055), and the package was uploaded into the NEW queue today. +Luckily the gnuradio maintainer has promised to help me, as I do not +know much about gnuradio stuff yet.

+ +

I doubt this "IMSI cacher" is anywhere near as powerfull as +commercial tools like +The +Spy Phone Portable IMSI / IMEI Catcher or the +Harris +Stingray, but I hope the existance of cheap alternatives can make +more people realise how their whereabouts when carrying a cell phone +is easily tracked. Seeing the data flow on the screen, realizing that +I live close to a police station and knowing that the police is also +wearing cell phones, I wonder how hard it would be for criminals to +track the position of the police officers to discover when there are +police near by, or for foreign military forces to track the location +of the Norwegian military forces, or for anyone to track the location +of government officials...

+ +

It is worth noting that the data reported by the IMSI-catcher +script mentioned above is only a fraction of the data broadcasted on +the GSM network. It will only collect one frequency at the time, +while a typical phone will be using several frequencies, and not all +phones will be using the frequencies tracked by the grgsm_livemod +program. Also, there is a lot of radio chatter being ignored by the +simple_IMSI-catcher script, which would be collected by extending the +parser code. I wonder if gr-gsm can be set up to listen to more than +one frequency?

- Tags: english, sikkerhet. + Tags: debian, english, personvern, surveillance.

@@ -141,61 +117,37 @@ activities, please send Bitcoin donations to my address

Full battery stats collector is now available in Debian

23rd March 2016

Since this morning, the battery-stats package in Debian include an -extended collector that will collect the complete battery history for -later processing and graphing. The original collector store the -battery level as percentage of last full level, while the new -collector also record battery vendor, model, serial number, design -full level, last full level and current battery level. This make it -possible to predict the lifetime of the battery as well as visualise -the energy flow when the battery is charging or discharging.

- -

The new tools are available in /usr/share/battery-stats/ -in the version 0.5.1 package in unstable. Get the new battery level graph -and lifetime prediction by running: - -

-/usr/share/battery-stats/battery-stats-graph /var/log/battery-stats.csv
-

- -

Or select the 'Battery Level Graph' from your application menu.

- -

The flow in/out of the battery can be seen by running (no menu -entry yet):

- -

-/usr/share/battery-stats/battery-stats-graph-flow
-

- -

I'm not quite happy with the way the data is visualised, at least -when there are few data points. The graphs look a bit better with a -few years of data.

- -

A while back one important feature I use in the battery stats -collector broke in Debian. The scripts in -/usr/lib/pm-utils/power.d/ were no longer executed. I -suspect it happened when Jessie started using systemd, but I do not -know. The issue is reported as -bug #818649 against -pm-utils. I managed to work around it by adding an udev rule to call -the collector script every time the power connector is connected and -disconnected. With this fix in place it was finally time to make a -new release of the package, and get it into Debian.

- -

If you are interested in how your laptop battery is doing, please -check out the -battery-stats -in Debian unstable, or rebuild it on Jessie to get it working on -Debian stable. :) The upstream source is available from -github. -As always, patches are very welcome.

Norwegian BokmÃ¥l edition of Debian Administrator's Handbook is now available

25th July 2017

+ +

I finally received a copy of the Norwegian BokmÃ¥l edition of +"The Debian Administrator's +Handbook". This test copy arrived in the mail a few days ago, and +I am very happy to hold the result in my hand. We spent around one and a half year translating it. This paperbook edition +is available +from lulu.com. If you buy it quickly, you save 25% on the list +price. The book is also available for download in electronic form as +PDF, EPUB and Mobipocket, as can be +read online +as a web page.

+ +

This is the second book I publish (the first was the book +"Free Culture" by Lawrence Lessig +in +English, +French +and +Norwegian +BokmÃ¥l), and I am very excited to finally wrap up this +project. I hope +"HÃ¥ndbok +for Debian-administratoren" will be well received.

- Tags: debian, english. + Tags: debian, debian-handbook, english.

@@ -203,109 +155,50 @@ As always, patches are very welcome.

UsingQR - "Electronic" paper invoices using JSON and QR codes

19th March 2016

Back in 2013 I proposed -a -way to make paper and PDF invoices easier to process electronically by -adding a QR code with the key information about the invoice. I -suggested using vCard field definition, to get some standard format -for name and address, but any format would work. I did not do -anything about the proposal, but hoped someone one day would make -something like it. It would make it possible to efficiently send -machine readable invoices directly between seller and buyer.

- -

This was the background when I came across a proposal and -specification from the web based accounting and invoicing supplier -Visma in Sweden called -UsingQR. Their PDF invoices contain -a QR code with the key information of the invoice in JSON format. -This is the typical content of a QR code following the UsingQR -specification (based on a real world example, some numbers replaced to -get a more bogus entry). I've reformatted the JSON to make it easier -to read. Normally this is all on one long line:

- -

-{
- "vh":500.00,
- "vm":0,
- "vl":0,
- "uqr":1,
- "tp":1,
- "nme":"Din LeverandÃ¸r",
- "cc":"NO",
- "cid":"997912345 MVA",
- "iref":"12300001",
- "idt":"20151022",
- "ddt":"20151105",
- "due":2500.0000,
- "cur":"NOK",
- "pt":"BBAN",
- "acc":"17202612345",
- "bc":"BIENNOK1",
- "adr":"0313 OSLO"
-}
-

- -

The interpretation of the fields can be found in the -format -specification (revision 2 from june 2014). The format seem to -have most of the information needed to handle accounting and payment -of invoices, at least the fields I have needed so far here in -Norway.

- -

Unfortunately, the site and document do not mention anything about -the patent, trademark and copyright status of the format and the -specification. Because of this, I asked the people behind it back in -November to clarify. Ann-Christine Savlid (ann-christine.savlid (at) -visma.com) replied that Visma had not applied for patent or trademark -protection for this format, and that there were no copyright based -usage limitations for the format. I urged her to make sure this was -explicitly written on the web pages and in the specification, but -unfortunately this has not happened yet. So I guess if there is -submarine patents, hidden trademarks or a will to sue for copyright -infringements, those starting to use the UsingQR format might be at -risk, but if this happen there is some legal defense in the fact that -the people behind the format claimed it was safe to do so. At least -with patents, there is always -a -chance of getting sued...

- -

I also asked if they planned to maintain the format in an -independent standard organization to give others more confidence that -they would participate in the standardization process on equal terms -with Visma, but they had no immediate plans for this. Their plan was -to work with banks to try to get more users of the format, and -evaluate the way forward if the format proved to be popular. I hope -they conclude that using an open standard organisation like -IETF is the correct place to -maintain such specification.

- -

Update 2016-03-20: Via Twitter I became aware of -some comments -about this blog post that had several useful links and references to -similar systems. In the Czech republic, the Czech Banking Association -standard #26, with short name SPAYD, uses QR codes with payment -information. More information is available from the Wikipedia page on -Short -Payment Descriptor. And in Germany, there is a system named -BezahlCode, -(specification -v1.8 2013-12-05 available as PDF), which uses QR codes with -URL-like formatting using "bank:" as the URI schema/protocol to -provide the payment information. There is also the -ZUGFeRD -file format that perhaps could be transfered using QR codes, but I am -not sure if it is done already. Last, in Bolivia there are reports -that tax information since november 2014 need to be printed in QR -format on invoices. I have not been able to track down a -specification for this format, because of my limited language skill -sets.

Â«Rapporten ser ikke pÃ¥ informasjonssikkerhet knyttet til personlig integritetÂ»

27th June 2017

Jeg kom over teksten +Â«Killing +car privacy by federal mandateÂ» av Leonid Reyzin pÃ¥ Freedom to +Tinker i dag, og det gleder meg Ã¥ se en god gjennomgang om hvorfor det +er et urimelig inngrep i privatsfÃ¦ren Ã¥ la alle biler kringkaste sin +posisjon og bevegelse via radio. Det omtalte forslaget basert pÃ¥ +Dedicated Short Range Communication (DSRC) kalles Basic Safety Message +(BSM) i USA og Cooperative Awareness Message (CAM) i Europa, og det +norske Vegvesenet er en av de som ser ut til Ã¥ kunne tenke seg Ã¥ +pÃ¥legge alle biler Ã¥ fjerne nok en bit av innbyggernes privatsfÃ¦re. +Anbefaler alle Ã¥ lese det som stÃ¥r der. + +

Mens jeg tittet litt pÃ¥ DSRC pÃ¥ biler i Norge kom jeg over et sitat +jeg synes er illustrativt for hvordan det offentlige Norge hÃ¥ndterer +problemstillinger rundt innbyggernes privatsfÃ¦re i SINTEF-rapporten +Â«Informasjonssikkerhet +i AutoPASS-brikkerÂ» av Trond Foss:

+ +

+Â«Rapporten ser ikke pÃ¥ informasjonssikkerhet knyttet til personlig + integritet.Â» +

+ +

SÃ¥ enkelt kan det tydeligvis gjÃ¸res nÃ¥r en vurderer +informasjonssikkerheten. Det holder vel at folkene pÃ¥ toppen kan si +at Â«Personvernet er ivaretattÂ», som jo er den populÃ¦re intetsigende +frasen som gjÃ¸r at mange tror enkeltindividers integritet tas vare pÃ¥. +Sitatet fikk meg til Ã¥ undres pÃ¥ hvor ofte samme tilnÃ¦rming, Ã¥ bare se +bort fra behovet for personlig itegritet, blir valgt nÃ¥r en velger Ã¥ +legge til rette for nok et inngrep i privatsfÃ¦ren til personer i +Norge. Det er jo sjelden det fÃ¥r reaksjoner. Historien om +reaksjonene pÃ¥ Helse SÃ¸r-Ãsts tjenesteutsetting er jo sÃ¸rgelig nok et +unntak og toppen av isfjellet, desverre. Tror jeg fortsatt takker nei +til bÃ¥de AutoPASS og holder meg sÃ¥ langt unna det norske helsevesenet +som jeg kan, inntil de har demonstrert og dokumentert at de verdsetter +individets privatsfÃ¦re og personlige integritet hÃ¸yere enn kortsiktig +gevist og samfunnsnytte.

- Tags: english, standard. + Tags: norsk, personvern, sikkerhet.

@@ -313,57 +206,66 @@ sets.

Making battery measurements a little easier in Debian

15th March 2016

Back in September, I blogged about -the -system I wrote to collect statistics about my laptop battery, and -how it showed the decay and death of this battery (now replaced). I -created a simple deb package to handle the collection and graphing, -but did not want to upload it to Debian as there were already -a battery-stats -package in Debian that should do the same thing, and I did not see -a point of uploading a competing package when battery-stats could be -fixed instead. I reported a few bugs about its non-function, and -hoped someone would step in and fix it. But no-one did.

- -

I got tired of waiting a few days ago, and took matters in my own -hands. The end result is that I am now the new upstream developer of -battery stats (available from github) and part of the team maintaining -battery-stats in Debian, and the package in Debian unstable is finally -able to collect battery status using the /sys/class/power_supply/ -information provided by the Linux kernel. If you install the -battery-stats package from unstable now, you will be able to get a -graph of the current battery fill level, to get some idea about the -status of the battery. The source package build and work just fine in -Debian testing and stable (and probably oldstable too, but I have not -tested). The default graph you get for that system look like this:

- -

My plans for the future is to merge my old scripts into the -battery-stats package, as my old scripts collected a lot more details -about the battery. The scripts are merged into the upstream -battery-stats git repository already, but I am not convinced they work -yet, as I changed a lot of paths along the way. Will have to test a -bit more before I make a new release.

- -

I will also consider changing the file format slightly, as I -suspect the way I combine several values into one field might make it -impossible to know the type of the value when using it for processing -and graphing.

- -

If you would like I would like to keep an close eye on your laptop -battery, check out the battery-stats package in -Debian and -on -github. -I would love some help to improve the system further.

Updated sales number for my Free Culture paper editions

12th June 2017

It is pleasing to see that the work we put down in publishing new +editions of the classic Free +Culture book by the founder of the Creative Commons movement, +Lawrence Lessig, is still being appreciated. I had a look at the +latest sales numbers for the paper edition today. Not too impressive, +but happy to see some buyers still exist. All the revenue from the +books is sent to the Creative +Commons Corporation, and they receive the largest cut if you buy +directly from Lulu. Most books are sold via Amazon, with Ingram +second and only a small fraction directly from Lulu. The ebook +edition is available for free from +Github.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Title / language	Quantity
Title / language	2016 jan-jun	2016 jul-dec	2017 jan-may
Culture Libre / French	3	6	15
Fri kultur / Norwegian	7	1	0
Free Culture / English	14	27	16
Total	24	34	31

+ +

A bit sad to see the low sales number on the Norwegian edition, and +a bit surprising the English edition still selling so well.

+ +

If you would like to translate and publish the book in your native +language, I would be happy to help make it happen. Please get in +touch.

- Tags: debian, english. + Tags: docbook, english, freeculture.

@@ -371,105 +273,59 @@ I would love some help to improve the system further.

Creating, updating and checking debian/copyright semi-automatically

19th February 2016

Making packages for Debian requires quite a lot of attention to -details. And one of the details is the content of the -debian/copyright file, which should list all relevant licenses used by -the code in the package in question, preferably in -machine -readable DEP5 format.

- -

For large packages with lots of contributors it is hard to write -and update this file manually, and if you get some detail wrong, the -package is normally rejected by the ftpmasters. So getting it right -the first time around get the package into Debian faster, and save -both you and the ftpmasters some work.. Today, while trying to figure -out what was wrong with -the -zfsonlinux copyright file, I decided to spend some time on -figuring out the options for doing this job automatically, or at least -semi-automatically.

- -

Lucikly, there are at least two tools available for generating the -file based on the code in the source package, -debmake -and cme. I'm -not sure which one of them came first, but both seem to be able to -create a sensible draft file. As far as I can tell, none of them can -be trusted to get the result just right, so the content need to be -polished a bit before the file is OK to upload. I found the debmake -option in -a -blog posts from 2014. - -

To generate using debmake, use the -cc option: - -

-debmake -cc > debian/copyright
-

- -

Note there are some problems with python and non-ASCII names, so -this might not be the best option.

- -

The cme option is based on a config parsing library, and I found -this approach in -a -blog post from 2015. To generate using cme, use the 'update -dpkg-copyright' option: - -

-cme update dpkg-copyright
-

- -

This will create or update debian/copyright. The cme tool seem to -handle UTF-8 names better than debmake.

- -

When the copyright file is created, I would also like some help to -check if the file is correct. For this I found two good options, -debmake -k and license-reconcile. The former seem -to focus on license types and file matching, and is able to detect -ineffective blocks in the copyright file. The latter reports missing -copyright holders and years, but was confused by inconsistent license -names (like CDDL vs. CDDL-1.0). I suspect it is good to use both and -fix all issues reported by them before uploading. But I do not know -if the tools and the ftpmasters agree on what is important to fix in a -copyright file, so the package might still be rejected.

- -

The devscripts tool licensecheck deserve mentioning. It -will read through the source and try to find all copyright statements. -It is not comparing the result to the content of debian/copyright, but -can be useful when verifying the content of the copyright file.

- -

Are you aware of better tools in Debian to create and update -debian/copyright file. Please let me know, or blog about it on -planet.debian.org.

- -

As usual, if you use Bitcoin and want to show your support of my -activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- -

Update 2016-02-20: I got a tip from Mike Gabriel -on how to use licensecheck and cdbs to create a draft copyright file - -

-licensecheck --copyright -r `find * -type f` | \
-  /usr/lib/cdbs/licensecheck2dep5 > debian/copyright.auto
-

- -

He mentioned that he normally check the generated file into the -version control system to make it easier to discover license and -copyright changes in the upstream source. I will try to do the same -with my packages in the future.

- -

Update 2016-02-21: The cme author recommended -against using -quiet for new users, so I removed it from the proposed -command line.

Release 0.1.1 of free software archive system Nikita announced

10th June 2017

I am very happy to report that the +Nikita Noark 5 +core project tagged its second release today. The free software +solution is an implementation of the Norwegian archive standard Noark +5 used by government offices in Norway. These were the changes in +version 0.1.1 since version 0.1.0 (from NEWS.md): + +

Continued work on the angularjs GUI, including document upload.
Implemented correspondencepartPerson, correspondencepartUnit and + correspondencepartInternal
Applied for coverity coverage and started submitting code on + regualr basis.
Started fixing bugs reported by coverity
Corrected and completed HATEOAS links to make sure entire API is + available via URLs in _links.
Corrected all relation URLs to use trailing slash.
Add initial support for storing data in ElasticSearch.
Now able to receive and store uploaded files in the archive.
Changed JSON output for object lists to have relations in _links.
Improve JSON output for empty object lists.
Now uses correct MIME type application/vnd.noark5-v4+json.
Added support for docker container images.
Added simple API browser implemented in JavaScript/Angular.
Started on archive client implemented in JavaScript/Angular.
Started on prototype to show the public mail journal.
Improved performance by disabling Sprint FileWatcher.
Added support for 'arkivskaper', 'saksmappe' and 'journalpost'.
Added support for some metadata codelists.
Added support for Cross-origin resource sharing (CORS).
Changed login method from Basic Auth to JSON Web Token (RFC 7519) + style.
Added support for GET-ing ny-* URLs.
Added support for modifying entities using PUT and eTag.
Added support for returning XML output on request.
Removed support for English field and class names, limiting ourself + to the official names.
...

+ +

If this sound interesting to you, please contact us on IRC (#nikita +on irc.freenode.net) or email +(nikita-noark +mailing list).

- Tags: debian, english. + Tags: english, nuug, offentlig innsyn, standard.

@@ -477,81 +333,99 @@ command line.

Using appstream in Debian to locate packages with firmware and mime type support

4th February 2016

The appstream system -is taking shape in Debian, and one provided feature is a very -convenient way to tell you which package to install to make a given -firmware file available when the kernel is looking for it. This can -be done using apt-file too, but that is for someone else to blog -about. :)

- -

Here is a small recipe to find the package with a given firmware -file, in this example I am looking for ctfw-3.2.3.0.bin, randomly -picked from the set of firmware announced using appstream in Debian -unstable. In general you would be looking for the firmware requested -by the kernel during kernel module loading. To find the package -providing the example file, do like this:

Idea for storing trusted timestamps in a Noark 5 archive

7th June 2017

This is a copy of +an +email I posted to the nikita-noark mailing list. Please follow up +there if you would like to discuss this topic. The background is that +we are making a free software archive system based on the Norwegian +Noark +5 standard for government archives.

+ +

I've been wondering a bit lately how trusted timestamps could be +stored in Noark 5. +Trusted +timestamps can be used to verify that some information +(document/file/checksum/metadata) have not been changed since a +specific time in the past. This is useful to verify the integrity of +the documents in the archive.

+ +

Then it occured to me, perhaps the trusted timestamps could be +stored as dokument variants (ie dokumentobjekt referered to from +dokumentbeskrivelse) with the filename set to the hash it is +stamping?

+ +

Given a "dokumentbeskrivelse" with an associated "dokumentobjekt", +a new dokumentobjekt is associated with "dokumentbeskrivelse" with the +same attributes as the stamped dokumentobjekt except these +attributes:

-% apt install appstream
-[...]
-% apt update
-[...]
-% appstreamcli what-provides firmware:runtime ctfw-3.2.3.0.bin | \
-  awk '/Package:/ {print $2}'
-firmware-qlogic
-%
-

See the -appstream wiki page to learn how to embed the package metadata in -a way appstream can use.

format -> "RFC3161" +
mimeType -> "application/timestamp-reply" +
formatDetaljer -> "<source URL for timestamp service>" +
filenavn -> "<sjekksum>.tsr" -
This same approach can be used to find any package supporting a -given MIME type. This is very useful when you get a file you do not -know how to handle. First find the mime type using file ---mime-type, and next look up the package providing support for -it. Lets say you got an SVG file. Its MIME type is image/svg+xml, -and you can find all packages handling this type like this:
+

-% apt install appstream
-[...]
-% apt update
-[...]
-% appstreamcli what-provides mimetype image/svg+xml | \
-  awk '/Package:/ {print $2}'
-bkchem
-phototonic
-inkscape
-shutter
-tetzle
-geeqie
-xia
-pinta
-gthumb
-karbon
-comix
-mirage
-viewnior
-postr
-ristretto
-kolourpaint4
-eog
-eom
-gimagereader
-midori
-%
-

This assume a service following +IETF RFC 3161 is +used, which specifiy the given MIME type for replies and the .tsr file +ending for the content of such trusted timestamp. As far as I can +tell from the Noark 5 specifications, it is OK to have several +variants/renderings of a dokument attached to a given +dokumentbeskrivelse objekt. It might be stretching it a bit to make +some of these variants represent crypto-signatures useful for +verifying the document integrity instead of representing the dokument +itself.

+ +

Using the source of the service in formatDetaljer allow several +timestamping services to be used. This is useful to spread the risk +of key compromise over several organisations. It would only be a +problem to trust the timestamps if all of the organisations are +compromised.

+ +

The following oneliner on Linux can be used to generate the tsr +file. $input is the path to the file to checksum, and $sha256 is the +SHA-256 checksum of the file (ie the ".tsr" value mentioned +above).

+ +

+openssl ts -query -data "$inputfile" -cert -sha256 -no_nonce \
+  | curl -s -H "Content-Type: application/timestamp-query" \
+      --data-binary "@-" http://zeitstempel.dfn.de > $sha256.tsr
+

+ +

To verify the timestamp, you first need to download the public key +of the trusted timestamp service, for example using this command:

+ +

+wget -O ca-cert.txt \
+  https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt
+

+ +

Note, the public key should be stored alongside the timestamps in +the archive to make sure it is also available 100 years from now. It +is probably a good idea to standardise how and were to store such +public keys, to make it easier to find for those trying to verify +documents 100 or 1000 years from now. :)

I believe the MIME types are fetched from the desktop file for -packages providing appstream metadata.

The verification itself is a simple openssl command:

+ +

+openssl ts -verify -data $inputfile -in $sha256.tsr \
+  -CAfile ca-cert.txt -text
+

+ +

Is there any reason this approach would not work? Is it somehow against +the Noark 5 specification?

- Tags: debian, english. + Tags: english, offentlig innsyn, standard.

@@ -559,91 +433,61 @@ packages providing appstream metadata.

Creepy, visualise geotagged social media information - nice free software

24th January 2016

Most people seem not to realise that every time they walk around -with the computerised radio beacon known as a mobile phone their -position is tracked by the phone company and often stored for a long -time (like every time a SMS is received or sent). And if their -computerised radio beacon is capable of running programs (often called -mobile apps) downloaded from the Internet, these programs are often -also capable of tracking their location (if the app requested access -during installation). And when these programs send out information to -central collection points, the location is often included, unless -extra care is taken to not send the location. The provided -information is used by several entities, for good and bad (what is -good and bad, depend on your point of view). What is certain, is that -the private sphere and the right to free movement is challenged and -perhaps even eradicated for those announcing their location this way, -when they share their whereabouts with private and public -entities.

- -

The phone company logs provide a register of locations to check out -when one want to figure out what the tracked person was doing. It is -unavailable for most of us, but provided to selected government -officials, company staff, those illegally buying information from -unfaithful servants and crackers stealing the information. But the -public information can be collected and analysed, and a free software -tool to do so is called -Creepy or Cree.py. I -discovered it when I read -an -article about Creepy in the Norwegian newspaper Aftenposten i -November 2014, and decided to check if it was available in Debian. -The python program was in Debian, but -the version in -Debian was completely broken and practically unmaintained. I -uploaded a new version which did not work quite right, but did not -have time to fix it then. This Christmas I decided to finally try to -get Creepy operational in Debian. Now a fixed version is available in -Debian unstable and testing, and almost all Debian specific patches -are now included -upstream.

- -

The Creepy program visualises geolocation information fetched from -Twitter, Instagram, Flickr and Google+, and allow one to get a -complete picture of every social media message posted recently in a -given area, or track the movement of a given individual across all -these services. Earlier it was possible to use the search API of at -least some of these services without identifying oneself, but these -days it is impossible. This mean that to use Creepy, you need to -configure it to log in as yourself on these services, and provide -information to them about your search interests. This should be taken -into account when using Creepy, as it will also share information -about yourself with the services.

- -

The picture above show the twitter messages sent from (or at least -geotagged with a position from) the city centre of Oslo, the capital -of Norway. One useful way to use Creepy is to first look at -information tagged with an area of interest, and next look at all the -information provided by one or more individuals who was in the area. -I tested it by checking out which celebrity provide their location in -twitter messages by checkout out who sent twitter messages near a -Norwegian TV station, and next could track their position over time, -making it possible to locate their home and work place, among other -things. A similar technique have been -used -to locate Russian soldiers in Ukraine, and it is both a powerful -tool to discover lying governments, and a useful tool to help people -understand the value of the private information they provide to the -public.

- -

The package is not trivial to backport to Debian Stable/Jessie, as -it depend on several python modules currently missing in Jessie (at -least python-instagram, python-flickrapi and -python-requests-toolbelt).

- -

(I have uploaded -the image to -screenshots.debian.net and licensed it under the same terms as the -Creepy program in Debian.)

NÃ¥r nynorskoversettelsen svikter til eksamen...

3rd June 2017

Aftenposten +melder i dag om feil i eksamensoppgavene for eksamen i politikk og +menneskerettigheter, der teksten i bokmÃ¥ls og nynorskutgaven ikke var +like. Oppgaveteksten er gjengitt i artikkelen, og jeg ble nysgjerring +pÃ¥ om den fri oversetterlÃ¸sningen +Apertium ville gjort en bedre +jobb enn Utdanningsdirektoratet. Det kan se slik ut.

+ +

Her er bokmÃ¥lsoppgaven fra eksamenen:

+ +

+
DrÃ¸ft utfordringene knyttet til nasjonalstatenes og andre aktÃ¸rers +rolle og muligheter til Ã¥ hÃ¥ndtere internasjonale utfordringer, som +for eksempel flykningekrisen.
+ +
Vedlegge er eksempler pÃ¥ tekster som kan gi relevante perspektiver +pÃ¥ temaet:
+
+
Flykningeregnskapet 2016, UNHCR og IDMC +
Â«GrenselÃ¸st Europa for fallÂ» A-Magasinet, 26. november 2015 +
+ +

+ +

Dette oversetter Apertium slik:

+ +

+
DrÃ¸ft utfordringane knytte til nasjonalstatane sine og rolla til +andre aktÃ¸rar og hÃ¸ve til Ã¥ handtera internasjonale utfordringar, som +til dÃ¸mes *flykningekrisen.
+ +
Vedleggja er dÃ¸me pÃ¥ tekster som kan gje relevante perspektiv pÃ¥ +temaet:
+ +
+
*Flykningeregnskapet 2016, *UNHCR og *IDMC
+
Â«*GrenselÃ¸st Europa for fallÂ» A-Magasinet, 26. november 2015
+
+ +

+ +

Ord som ikke ble forstÃ¥tt er markert med stjerne (*), og trenger +ekstra sprÃ¥ksjekk. Men ingen ord er forsvunnet, slik det var i +oppgaven elevene fikk presentert pÃ¥ eksamen. Jeg mistenker dog at +"andre aktÃ¸rers rolle og muligheter til ..." burde vÃ¦rt oversatt til +"rolla til andre aktÃ¸rar og deira hÃ¸ve til ..." eller noe slikt, men +det er kanskje flisespikking. Det understreker vel bare at det alltid +trengs korrekturlesning etter automatisk oversettelse.

- Tags: debian, english, nice free software. + Tags: debian, norsk, stavekontroll.

@@ -651,72 +495,67 @@ Creepy program in Debian.)

Always download Debian packages using Tor - the simple recipe

15th January 2016

During his DebConf15 keynote, Jacob Appelbaum -observed -that those listening on the Internet lines would have good reason to -believe a computer have a given security hole if it download a -security fix from a Debian mirror. This is a good reason to always -use encrypted connections to the Debian mirror, to make sure those -listening do not know which IP address to attack. In August, Richard -Hartmann observed that encryption was not enough, when it was possible -to interfere download size to security patches or the fact that -download took place shortly after a security fix was released, and -proposed -to always use Tor to download packages from the Debian mirror. He -was not the first to propose this, as the -apt-transport-tor -package by Tim Retout already existed to make it easy to convince apt -to use Tor, but I was not -aware of that package when I read the blog post from Richard.

- -

Richard discussed the idea with Peter Palfrader, one of the Debian -sysadmins, and he set up a Tor hidden service on one of the central -Debian mirrors using the address vwakviie2ienjx6t.onion, thus making -it possible to download packages directly between two tor nodes, -making sure the network traffic always were encrypted.

- -

Here is a short recipe for enabling this on your machine, by -installing apt-transport-tor and replacing http and https -urls with tor+http and tor+https, and using the hidden service instead -of the official Debian mirror site. I recommend installing -etckeeper before you start to have a history of the changes -done in /etc/.

- -

-apt install apt-transport-tor
-sed -i 's% http://ftp.debian.org/% tor+http://vwakviie2ienjx6t.onion/%' /etc/apt/sources.list
-sed -i 's% http% tor+http%' /etc/apt/sources.list
-

- -

If you have more sources listed in /etc/apt/sources.list.d/, run -the sed commands for these too. The sed command is assuming your are -using the ftp.debian.org Debian mirror. Adjust the command (or just -edit the file manually) to match your mirror.

- -

This work in Debian Jessie and later. Note that tools like -apt-file only recently started using the apt transport -system, and do not work with these tor+http URLs. For -apt-file you need the version currently in experimental, -which need a recent apt version currently only in unstable. So if you -need a working apt-file, this is not for you.

- -

Another advantage from this change is that your machine will start -using Tor regularly and at fairly random intervals (every time you -update the package lists or upgrade or install a new package), thus -masking other Tor traffic done from the same machine. Using Tor will -become normal for the machine in question.

- -

On Freedombox, APT -is set up by default to use apt-transport-tor when Tor is -enabled. It would be great if it was the default on any Debian -system.

Epost inn som arkivformat i Riksarkivarens forskrift?

27th April 2017

I disse dager, med frist 1. mai, har Riksarkivaren ute en hÃ¸ring pÃ¥ +sin forskrift. Som en kan se er det ikke mye tid igjen fÃ¸r fristen +som gÃ¥r ut pÃ¥ sÃ¸ndag. Denne forskriften er det som lister opp hvilke +formater det er greit Ã¥ arkivere i +Noark +5-lÃ¸sninger i Norge.

+ +

Jeg fant hÃ¸ringsdokumentene hos +Norsk +ArkivrÃ¥d etter Ã¥ ha blitt tipset pÃ¥ epostlisten til +fri +programvareprosjektet Nikita Noark5-Core, som lager et Noark 5 +Tjenestegresesnitt. Jeg er involvert i Nikita-prosjektet og takket +vÃ¦re min interesse for tjenestegrensesnittsprosjektet har jeg lest en +god del Noark 5-relaterte dokumenter, og til min overraskelse oppdaget +at standard epost ikke er pÃ¥ listen over godkjente formater som kan +arkiveres. HÃ¸ringen med frist sÃ¸ndag er en glimrende mulighet til Ã¥ +forsÃ¸ke Ã¥ gjÃ¸re noe med det. Jeg holder pÃ¥ med +egen +hÃ¸ringsuttalelse, og lurer pÃ¥ om andre er interessert i Ã¥ stÃ¸tte +forslaget om Ã¥ tillate arkivering av epost som epost i arkivet.

+ +

Er du igang med Ã¥ skrive egen hÃ¸ringsuttalelse allerede? I sÃ¥ fall +kan du jo vurdere Ã¥ ta med en formulering om epost-lagring. Jeg tror +ikke det trengs sÃ¥ mye. Her et kort forslag til tekst:

+ +

+ +
Viser til hÃ¸ring sendt ut 2017-02-17 (Riksarkivarens referanse + 2016/9840 HELHJO), og tillater oss Ã¥ sende inn noen innspill om + revisjon av Forskrift om utfyllende tekniske og arkivfaglige + bestemmelser om behandling av offentlige arkiver (Riksarkivarens + forskrift).
+ +
SvÃ¦rt mye av vÃ¥r kommuikasjon foregÃ¥r i dag pÃ¥ e-post.Â Vi + foreslÃ¥r derfor at Internett-e-post, slik det er beskrevet i IETF + RFC 5322, + https://tools.ietf.org/html/rfc5322. bÃ¸r + inn som godkjent dokumentformat.Â Vi foreslÃ¥r at forskriftens + oversikt over godkjente dokumentformater ved innlevering i Â§ 5-16 + endres til Ã¥ ta med Internett-e-post.
+ +

+ +

Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan +epost kan lagres i en Noark 5-struktur, og holder pÃ¥ Ã¥ skrive et +forslag om hvordan dette kan gjÃ¸res som vil bli sendt over til +arkivverket sÃ¥ snart det er ferdig. De som er interesserte kan +fÃ¸lge +fremdriften pÃ¥ web.

+ +

Oppdatering 2017-04-28: I dag ble hÃ¸ringuttalelsen jeg skrev + sendt + inn av foreningen NUUG.

- Tags: debian, english, sikkerhet. + Tags: norsk, offentlig innsyn, standard.

@@ -724,37 +563,52 @@ system.

Nedlasting fra NRK, som Matroska med undertekster

2nd January 2016

Det kommer stadig nye lÃ¸sninger for Ã¥ ta lagre unna innslag fra NRK -for Ã¥ se pÃ¥ det senere. For en stund tilbake kom jeg over et script -nrkopptak laget av Ingvar Hagelund. Han fjernet riktignok sitt script -etter forespÃ¸rsel fra Erik Bolstad i NRK, men noen tok heldigvis og -gjorde det tilgjengelig -via github.

- -

Scriptet kan lagre som MPEG4 eller Matroska, og bake inn -undertekster i fila pÃ¥ et vis som blant annet VLC forstÃ¥r. For Ã¥ -bruke scriptet, kopier ned git-arkivet og kjÃ¸r

- -

-nrkopptak/bin/nrk-opptak k https://tv.nrk.no/serie/bmi-turne/MUHH45000115/sesong-1/episode-1
-

- -

URL-eksemplet er dagens toppsak pÃ¥ tv.nrk.no. Argument 'k' ber -scriptet laste ned og lagre som Matroska. Det finnes en rekke andre -muligheter for valg av kvalitet og format.

- -

Jeg foretrekker dette scriptet fremfor youtube-dl, som - -nevnt i 2014 stÃ¸tter NRK og en rekke andre videokilder, pÃ¥ grunn -av at nrkopptak samler undertekster og video i en enkelt fil, hvilket -gjÃ¸r hÃ¥ndtering enklere pÃ¥ disk.

Offentlig elektronisk postjournal blokkerer tilgang for utvalgte webklienter

20th April 2017

Jeg oppdaget i dag at nettstedet som +publiserer offentlige postjournaler fra statlige etater, OEP, har +begynt Ã¥ blokkerer enkelte typer webklienter fra Ã¥ fÃ¥ tilgang. Vet +ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl +og curl. For Ã¥ teste selv, kjÃ¸r fÃ¸lgende:

+ +

+% curl -v -s https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
+< HTTP/1.1 404 Not Found
+% curl -v -s --header 'User-Agent:Opera/12.0' https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
+< HTTP/1.1 200 OK
+%
+

+ +

Her kan en se at tjenesten gir Â«404 Not FoundÂ» for curl i +standardoppsettet, mens den gir Â«200 OKÂ» hvis curl hevder Ã¥ vÃ¦re Opera +versjon 12.0. Offentlig elektronisk postjournal startet blokkeringen +2017-03-02.

+ +

Blokkeringen vil gjÃ¸re det litt vanskeligere Ã¥ maskinelt hente +informasjon fra oep.no. Kan blokkeringen vÃ¦re gjort for Ã¥ hindre +automatisert innsamling av informasjon fra OEP, slik Pressens +Offentlighetsutvalg gjorde for Ã¥ dokumentere hvordan departementene +hindrer innsyn i +rapporten +Â«Slik hindrer departementer innsynÂ» som ble publiserte i januar +2017. Det virker usannsynlig, da det jo er trivielt Ã¥ bytte +User-Agent til noe nytt.

+ +

Finnes det juridisk grunnlag for det offentlige Ã¥ diskriminere +webklienter slik det gjÃ¸res her? Der tilgang gis eller ikke alt etter +hva klienten sier at den heter? Da OEP eies av DIFI og driftes av +Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to +aktÃ¸rene man kan be om innsyn i for Ã¥ forstÃ¥ hva som har skjedd. Men +postjournalen +til DIFI viser kun to dokumenter det siste Ã¥ret mellom DIFI og +Basefarm. +Mimes brÃ¸nn neste, +tenker jeg.

- Tags: multimedia, norsk, video, web. + Tags: norsk, offentlig innsyn.

@@ -762,58 +616,101 @@ gjÃ¸r hÃ¥ndtering enklere pÃ¥ disk.

OpenALPR, find car license plates in video streams - nice free software

23rd December 2015

When I was a kid, we used to collect "car numbers", as we used to -call the car license plate numbers in those days. I would write the -numbers down in my little book and compare notes with the other kids -to see how many region codes we had seen and if we had seen some -exotic or special region codes and numbers. It was a fun game to pass -time, as we kids have plenty of it.

- -

A few days I came across -the OpenALPR -project, a free software project to automatically discover and -report license plates in images and video streams, and provide the -"car numbers" in a machine readable format. I've been looking for -such system for a while now, because I believe it is a bad idea that the -automatic -number plate recognition tool only is available in the hands of -the powerful, and want it to be available also for the powerless to -even the score when it comes to surveillance and sousveillance. I -discovered the developer -wanted to get the tool into -Debian, and as I too wanted it to be in Debian, I volunteered to -help him get it into shape to get the package uploaded into the Debian -archive.

- -

Today we finally managed to get the package into shape and uploaded -it into Debian, where it currently -waits -in the NEW queue for review by the Debian ftpmasters.

- -

I guess you are wondering why on earth such tool would be useful -for the common folks, ie those not running a large government -surveillance system? Well, I plan to put it in a computer on my bike -and in my car, tracking the cars nearby and allowing me to be notified -when number plates on my watch list are discovered. Another use case -was suggested by a friend of mine, who wanted to set it up at his home -to open the car port automatically when it discovered the plate on his -car. When I mentioned it perhaps was a bit foolhardy to allow anyone -capable of placing his license plate number of a piece of cardboard to -open his car port, men replied that it was always unlocked anyway. I -guess for such use case it make sense. I am sure there are other use -cases too, for those with imagination and a vision.

- -

If you want to build your own version of the Debian package, check -out the upstream git source and symlink ./distros/debian to ./debian/ -before running "debuild" to build the source. Or wait a bit until the -package show up in unstable.

Free software archive system Nikita now able to store documents

19th March 2017

The Nikita +Noark 5 core project is implementing the Norwegian standard for +keeping an electronic archive of government documents. +The +Noark 5 standard document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +announced +it supported the project. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches.

+ +

If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(#nikita on +irc.freenode.net) and +the +project mailing list.

+ +

When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +archive-pdf to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +fonds, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester:

+ +

+~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
+using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
+using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
+
+ 0 - Title of the test case file created 2017-03-18T23:49:32.103446
+ 1 - Title of the test file created 2017-03-18T23:49:32.103446
+Select which mappe you want (or search term): 0
+Uploading mangelmelding/mangler.pdf
+  PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
+  File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
+~/src//noark5-tester$
+

+ +

You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The archive-pdf +tool can be found in the git repository for the API tester.

+ +

In the project, I have been mostly working on +the API +tester so far, while getting to know the code base. The API +tester currently use +the HATEOAS links +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification.

+ +

The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +started +writing down the questions we get from implementing it. We use a +format inspired by how The +Austin Group collect defect reports for the POSIX standard with +their +instructions for the MANTIS defect tracker system, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a request for a procedure for submitting defect reports :). + +

The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python.

- Tags: debian, english, nice free software. + Tags: english, nuug, offentlig innsyn, standard.

@@ -828,6 +725,25 @@ package show up in unstable.

Archive