+ Petter Reinholdtsen + +

@@ -614,7 +722,7 @@ post.

@@ -650,9 +758,9 @@ post.

Free software archive system Nikita now able to store documents

@@ -682,7 +790,7 @@ post.

diff --git a/blog/draft/2013-links.txt b/blog/draft/2013-links.txt index 75ba61771c..2e900309af 100644 --- a/blog/draft/2013-links.txt +++ b/blog/draft/2013-links.txt @@ -94,3 +94,5 @@ http://marxisme.no/cory-doctorow/ https://wikileaks.org/google-is-not-what-it-seems/ http://www.dinside.no/bolig/registrerer-om-du-er-hjemme-og-om-du-har-gjester/66919524 + +https://www.youtube.com/watch?v=tw4DdF8dSjE diff --git a/blog/index.html b/blog/index.html index 1a839e55ab..c894e4cefb 100644 --- a/blog/index.html +++ b/blog/index.html @@ -19,6 +19,108 @@ +

19th March 2017

The Nikita +Noark 5 core project is implementing the Norwegian standard for +keeping an electronic archive of government documents. +The +Noark 5 standard document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +announced +it supported the project. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches.

+ +

If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(#nikita on +irc.freenode.net) and +the +project mailing list.

+ +

When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +archive-pdf to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +fonds, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester:

+ +

+~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
+using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
+using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
+
+ 0 - Title of the test case file created 2017-03-18T23:49:32.103446
+ 1 - Title of the test file created 2017-03-18T23:49:32.103446
+Select which mappe you want (or search term): 0
+Uploading mangelmelding/mangler.pdf
+  PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
+  File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
+~/src//noark5-tester$
+

+ +

You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The archive-pdf +tool can be found in the git repository for the API tester.

+ +

In the project, I have been mostly working on +the API +tester so far, while getting to know the code base. The API +tester currently use +the HATEOAS links +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification.

+ +

The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +started +writing down the questions we get from implementing it. We use a +format inspired by how The +Austin Group collect defect reports for the POSIX standard with +their +instructions for the MANTIS defect tracker system, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a request for a procedure for submitting defect reports :). + +

The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python.

+ + + Tags: english, nuug, offentlig innsyn, standard. + + +

Detecting NFS hangs on Linux without hanging yourself...

9th March 2017

@@ -576,171 +678,6 @@ kanskje Datatilsynet bÃ¸r gjÃ¸re det?

Where did that package go? — geolocated IP traceroute

9th January 2017

Did you ever wonder where the web trafic really flow to reach the -web servers, and who own the network equipment it is flowing through? -It is possible to get a glimpse of this from using traceroute, but it -is hard to find all the details. Many years ago, I wrote a system to -map the Norwegian Internet (trying to figure out if our plans for a -network game service would get low enough latency, and who we needed -to talk to about setting up game servers close to the users. Back -then I used traceroute output from many locations (I asked my friends -to run a script and send me their traceroute output) to create the -graph and the map. The output from traceroute typically look like -this: - -

-traceroute to www.stortinget.no (85.88.67.10), 30 hops max, 60 byte packets
- 1  uio-gw10.uio.no (129.240.202.1)  0.447 ms  0.486 ms  0.621 ms
- 2  uio-gw8.uio.no (129.240.24.229)  0.467 ms  0.578 ms  0.675 ms
- 3  oslo-gw1.uninett.no (128.39.65.17)  0.385 ms  0.373 ms  0.358 ms
- 4  te3-1-2.br1.fn3.as2116.net (193.156.90.3)  1.174 ms  1.172 ms  1.153 ms
- 5  he16-1-1.cr1.san110.as2116.net (195.0.244.234)  2.627 ms he16-1-1.cr2.oslosda310.as2116.net (195.0.244.48)  3.172 ms he16-1-1.cr1.san110.as2116.net (195.0.244.234)  2.857 ms
- 6  ae1.ar8.oslosda310.as2116.net (195.0.242.39)  0.662 ms  0.637 ms ae0.ar8.oslosda310.as2116.net (195.0.242.23)  0.622 ms
- 7  89.191.10.146 (89.191.10.146)  0.931 ms  0.917 ms  0.955 ms
- 8  * * *
- 9  * * *
-[...]
-

- -

This show the DNS names and IP addresses of (at least some of the) -network equipment involved in getting the data traffic from me to the -www.stortinget.no server, and how long it took in milliseconds for a -package to reach the equipment and return to me. Three packages are -sent, and some times the packages do not follow the same path. This -is shown for hop 5, where three different IP addresses replied to the -traceroute request.

- -

There are many ways to measure trace routes. Other good traceroute -implementations I use are traceroute (using ICMP packages) mtr (can do -both ICMP, UDP and TCP) and scapy (python library with ICMP, UDP, TCP -traceroute and a lot of other capabilities). All of them are easily -available in Debian.

- -

This time around, I wanted to know the geographic location of -different route points, to visualize how visiting a web page spread -information about the visit to a lot of servers around the globe. The -background is that a web site today often will ask the browser to get -from many servers the parts (for example HTML, JSON, fonts, -JavaScript, CSS, video) required to display the content. This will -leak information about the visit to those controlling these servers -and anyone able to peek at the data traffic passing by (like your ISP, -the ISPs backbone provider, FRA, GCHQ, NSA and others).

- -

Lets pick an example, the Norwegian parliament web site -www.stortinget.no. It is read daily by all members of parliament and -their staff, as well as political journalists, activits and many other -citizens of Norway. A visit to the www.stortinget.no web site will -ask your browser to contact 8 other servers: ajax.googleapis.com, -insights.hotjar.com, script.hotjar.com, static.hotjar.com, -stats.g.doubleclick.net, www.google-analytics.com, -www.googletagmanager.com and www.netigate.se. I extracted this by -asking PhantomJS to visit the -Stortinget web page and tell me all the URLs PhantomJS downloaded to -render the page (in HAR format using -their -netsniff example. I am very grateful to Gorm for showing me how -to do this). My goal is to visualize network traces to all IP -addresses behind these DNS names, do show where visitors personal -information is spread when visiting the page.

- -

When I had a look around for options, I could not find any good -free software tools to do this, and decided I needed my own traceroute -wrapper outputting KML based on locations looked up using GeoIP. KML -is easy to work with and easy to generate, and understood by several -of the GIS tools I have available. I got good help from by NUUG -colleague Anders Einar with this, and the result can be seen in -my -kmltraceroute git repository. Unfortunately, the quality of the -free GeoIP databases I could find (and the for-pay databases my -friends had access to) is not up to the task. The IP addresses of -central Internet infrastructure would typically be placed near the -controlling companies main office, and not where the router is really -located, as you can see from the -KML file I created using the GeoLite City dataset from MaxMind. - -

- -

I also had a look at the visual traceroute graph created by -the scrapy project, -showing IP network ownership (aka AS owner) for the IP address in -question. -The -graph display a lot of useful information about the traceroute in SVG -format, and give a good indication on who control the network -equipment involved, but it do not include geolocation. This graph -make it possible to see the information is made available at least for -UNINETT, Catchcom, Stortinget, Nordunet, Google, Amazon, Telia, Level -3 Communications and NetDNA.

- -

In the process, I came across the -web service GeoTraceroute by -Salim Gasmi. Its methology of combining guesses based on DNS names, -various location databases and finally use latecy times to rule out -candidate locations seemed to do a very good job of guessing correct -geolocation. But it could only do one trace at the time, did not have -a sensor in Norway and did not make the geolocations easily available -for postprocessing. So I contacted the developer and asked if he -would be willing to share the code (he refused until he had time to -clean it up), but he was interested in providing the geolocations in a -machine readable format, and willing to set up a sensor in Norway. So -since yesterday, it is possible to run traces from Norway in this -service thanks to a sensor node set up by -the NUUG assosiation, and get the -trace in KML format for further processing.

- -

Here we can see a lot of trafic passes Sweden on its way to -Denmark, Germany, Holland and Ireland. Plenty of places where the -Snowden confirmations verified the traffic is read by various actors -without your best interest as their top priority.

- -

Combining KML files is trivial using a text editor, so I could loop -over all the hosts behind the urls imported by www.stortinget.no and -ask for the KML file from GeoTraceroute, and create a combined KML -file with all the traces (unfortunately only one of the IP addresses -behind the DNS name is traced this time. To get them all, one would -have to request traces using IP number instead of DNS names from -GeoTraceroute). That might be the next step in this project.

- -

Armed with these tools, I find it a lot easier to figure out where -the IP traffic moves and who control the boxes involved in moving it. -And every time the link crosses for example the Swedish border, we can -be sure Swedish Signal Intelligence (FRA) is listening, as GCHQ do in -Britain and NSA in USA and cables around the globe. (Hm, what should -we tell them? :) Keep that in mind if you ever send anything -unencrypted over the Internet.

- -

PS: KML files are drawn using -the KML viewer from Ivan -Rublev , as it was less cluttered than the local Linux application -Marble. There are heaps of other options too.

- -

As usual, if you use Bitcoin and want to show your support of my -activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- - - Tags: debian, english, kart, nuug, personvern, stortinget, surveillance, web. - - -

@@ -756,7 +693,7 @@ activities, please send Bitcoin donations to my address

@@ -1038,7 +975,7 @@ activities, please send Bitcoin donations to my address

@@ -1074,9 +1011,9 @@ activities, please send Bitcoin donations to my address

@@ -1106,7 +1043,7 @@ activities, please send Bitcoin donations to my address

diff --git a/blog/index.rss b/blog/index.rss index 0a657f587a..8780170dce 100644 --- a/blog/index.rss +++ b/blog/index.rss @@ -6,6 +6,102 @@ http://people.skolelinux.org/pere/blog/ + + Free software archive system Nikita now able to store documents + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + Sun, 19 Mar 2017 08:00:00 +0100 + The <a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita +Noark 5 core project</a> is implementing the Norwegian standard for +keeping an electronic archive of government documents. +<a href="http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version">The +Noark 5 standard</a> document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +<a href="https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml">announced +it supported the project</a>. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches. + +If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(<a href="irc://irc.freenode.net/%23nikita"">#nikita on +irc.freenode.net</a>) and +<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">the +project mailing list</a>. + +When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +<tt>archive-pdf</tt> to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +<a href="https://en.wikipedia.org/wiki/Fonds">fonds</a>, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester: + +<blockquote><pre> +~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf +using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446 +using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446 + + 0 - Title of the test case file created 2017-03-18T23:49:32.103446 + 1 - Title of the test file created 2017-03-18T23:49:32.103446 +Select which mappe you want (or search term): 0 +Uploading mangelmelding/mangler.pdf + PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt + File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446 +~/src//noark5-tester$ +</pre></blockquote> + +You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The <tt>archive-pdf</tt> +tool can be found in the git repository for the API tester. + +In the project, I have been mostly working on +<a href="https://github.com/petterreinholdtsen/noark5-tester">the API +tester</a> so far, while getting to know the code base. The API +tester currently use +<a href="https://en.wikipedia.org/wiki/HATEOAS">the HATEOAS links</a> +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification. + +The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +<a href="https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding">started +writing down</a> the questions we get from implementing it. We use a +format inspired by how <a href="http://www.opengroup.org/austin/">The +Austin Group</a> collect defect reports for the POSIX standard with +<a href="http://www.opengroup.org/austin/mantis.html">their +instructions for the MANTIS defect tracker system</a>, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a <a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md">request for a procedure for submitting defect reports</a> :). + +The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python. + + + Detecting NFS hangs on Linux without hanging yourself... http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html @@ -509,164 +605,5 @@ kanskje Datatilsynet bÃ¸r gjÃ¸re det? - - Where did that package go? — geolocated IP traceroute - http://people.skolelinux.org/pere/blog/Where_did_that_package_go___mdash__geolocated_IP_traceroute.html - http://people.skolelinux.org/pere/blog/Where_did_that_package_go___mdash__geolocated_IP_traceroute.html - Mon, 9 Jan 2017 12:20:00 +0100 - Did you ever wonder where the web trafic really flow to reach the -web servers, and who own the network equipment it is flowing through? -It is possible to get a glimpse of this from using traceroute, but it -is hard to find all the details. Many years ago, I wrote a system to -map the Norwegian Internet (trying to figure out if our plans for a -network game service would get low enough latency, and who we needed -to talk to about setting up game servers close to the users. Back -then I used traceroute output from many locations (I asked my friends -to run a script and send me their traceroute output) to create the -graph and the map. The output from traceroute typically look like -this: - -<pre> -traceroute to www.stortinget.no (85.88.67.10), 30 hops max, 60 byte packets - 1 uio-gw10.uio.no (129.240.202.1) 0.447 ms 0.486 ms 0.621 ms - 2 uio-gw8.uio.no (129.240.24.229) 0.467 ms 0.578 ms 0.675 ms - 3 oslo-gw1.uninett.no (128.39.65.17) 0.385 ms 0.373 ms 0.358 ms - 4 te3-1-2.br1.fn3.as2116.net (193.156.90.3) 1.174 ms 1.172 ms 1.153 ms - 5 he16-1-1.cr1.san110.as2116.net (195.0.244.234) 2.627 ms he16-1-1.cr2.oslosda310.as2116.net (195.0.244.48) 3.172 ms he16-1-1.cr1.san110.as2116.net (195.0.244.234) 2.857 ms - 6 ae1.ar8.oslosda310.as2116.net (195.0.242.39) 0.662 ms 0.637 ms ae0.ar8.oslosda310.as2116.net (195.0.242.23) 0.622 ms - 7 89.191.10.146 (89.191.10.146) 0.931 ms 0.917 ms 0.955 ms - 8 * * * - 9 * * * -[...] -</pre> - -This show the DNS names and IP addresses of (at least some of the) -network equipment involved in getting the data traffic from me to the -www.stortinget.no server, and how long it took in milliseconds for a -package to reach the equipment and return to me. Three packages are -sent, and some times the packages do not follow the same path. This -is shown for hop 5, where three different IP addresses replied to the -traceroute request. - -There are many ways to measure trace routes. Other good traceroute -implementations I use are traceroute (using ICMP packages) mtr (can do -both ICMP, UDP and TCP) and scapy (python library with ICMP, UDP, TCP -traceroute and a lot of other capabilities). All of them are easily -available in <a href="https://www.debian.org/">Debian</a>. - -This time around, I wanted to know the geographic location of -different route points, to visualize how visiting a web page spread -information about the visit to a lot of servers around the globe. The -background is that a web site today often will ask the browser to get -from many servers the parts (for example HTML, JSON, fonts, -JavaScript, CSS, video) required to display the content. This will -leak information about the visit to those controlling these servers -and anyone able to peek at the data traffic passing by (like your ISP, -the ISPs backbone provider, FRA, GCHQ, NSA and others). - -Lets pick an example, the Norwegian parliament web site -www.stortinget.no. It is read daily by all members of parliament and -their staff, as well as political journalists, activits and many other -citizens of Norway. A visit to the www.stortinget.no web site will -ask your browser to contact 8 other servers: ajax.googleapis.com, -insights.hotjar.com, script.hotjar.com, static.hotjar.com, -stats.g.doubleclick.net, www.google-analytics.com, -www.googletagmanager.com and www.netigate.se. I extracted this by -asking <a href="http://phantomjs.org/">PhantomJS</a> to visit the -Stortinget web page and tell me all the URLs PhantomJS downloaded to -render the page (in HAR format using -<a href="https://github.com/ariya/phantomjs/blob/master/examples/netsniff.js">their -netsniff example</a>. I am very grateful to Gorm for showing me how -to do this). My goal is to visualize network traces to all IP -addresses behind these DNS names, do show where visitors personal -information is spread when visiting the page. - -<a href="www.stortinget.no-geoip.kml"><img -src="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-geoip-small.png" alt="map of combined traces for URLs used by www.stortinget.no using GeoIP"/></a> - -When I had a look around for options, I could not find any good -free software tools to do this, and decided I needed my own traceroute -wrapper outputting KML based on locations looked up using GeoIP. KML -is easy to work with and easy to generate, and understood by several -of the GIS tools I have available. I got good help from by NUUG -colleague Anders Einar with this, and the result can be seen in -<a href="https://github.com/petterreinholdtsen/kmltraceroute">my -kmltraceroute git repository</a>. Unfortunately, the quality of the -free GeoIP databases I could find (and the for-pay databases my -friends had access to) is not up to the task. The IP addresses of -central Internet infrastructure would typically be placed near the -controlling companies main office, and not where the router is really -located, as you can see from <a href="www.stortinget.no-geoip.kml">the -KML file I created</a> using the GeoLite City dataset from MaxMind. - -<a href="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-scapy.svg"><img -src="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-scapy-small.png" alt="scapy traceroute graph for URLs used by www.stortinget.no"/></a> - -I also had a look at the visual traceroute graph created by -<a href="http://www.secdev.org/projects/scapy/">the scrapy project</a>, -showing IP network ownership (aka AS owner) for the IP address in -question. -<a href="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-scapy.svg">The -graph display a lot of useful information about the traceroute in SVG -format</a>, and give a good indication on who control the network -equipment involved, but it do not include geolocation. This graph -make it possible to see the information is made available at least for -UNINETT, Catchcom, Stortinget, Nordunet, Google, Amazon, Telia, Level -3 Communications and NetDNA. - -<a href="https://geotraceroute.com/index.php?node=4&host=www.stortinget.no"><img -src="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-geotraceroute-small.png" alt="example geotraceroute view for www.stortinget.no"/></a> - -In the process, I came across the -<a href="https://geotraceroute.com/">web service GeoTraceroute</a> by -Salim Gasmi. Its methology of combining guesses based on DNS names, -various location databases and finally use latecy times to rule out -candidate locations seemed to do a very good job of guessing correct -geolocation. But it could only do one trace at the time, did not have -a sensor in Norway and did not make the geolocations easily available -for postprocessing. So I contacted the developer and asked if he -would be willing to share the code (he refused until he had time to -clean it up), but he was interested in providing the geolocations in a -machine readable format, and willing to set up a sensor in Norway. So -since yesterday, it is possible to run traces from Norway in this -service thanks to a sensor node set up by -<a href="https://www.nuug.no/">the NUUG assosiation</a>, and get the -trace in KML format for further processing. - -<a href="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-geotraceroute-kml-join.kml"><img -src="http://people.skolelinux.org/pere/blog/images/2017-01-09-www.stortinget.no-geotraceroute-kml-join.png" alt="map of combined traces for URLs used by www.stortinget.no using geotraceroute"/></a> - -Here we can see a lot of trafic passes Sweden on its way to -Denmark, Germany, Holland and Ireland. Plenty of places where the -Snowden confirmations verified the traffic is read by various actors -without your best interest as their top priority. - -Combining KML files is trivial using a text editor, so I could loop -over all the hosts behind the urls imported by www.stortinget.no and -ask for the KML file from GeoTraceroute, and create a combined KML -file with all the traces (unfortunately only one of the IP addresses -behind the DNS name is traced this time. To get them all, one would -have to request traces using IP number instead of DNS names from -GeoTraceroute). That might be the next step in this project. - -Armed with these tools, I find it a lot easier to figure out where -the IP traffic moves and who control the boxes involved in moving it. -And every time the link crosses for example the Swedish border, we can -be sure Swedish Signal Intelligence (FRA) is listening, as GCHQ do in -Britain and NSA in USA and cables around the globe. (Hm, what should -we tell them? :) Keep that in mind if you ever send anything -unencrypted over the Internet. - -PS: KML files are drawn using -<a href="http://ivanrublev.me/kml/">the KML viewer from Ivan -Rublev<a/>, as it was less cluttered than the local Linux application -Marble. There are heaps of other options too. - -As usual, if you use Bitcoin and want to show your support of my -activities, please send Bitcoin donations to my address -<a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&label=PetterReinholdtsenBlog">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a>. - - - diff --git a/blog/jXplorer__a_very_nice_LDAP_GUI.html b/blog/jXplorer__a_very_nice_LDAP_GUI.html index b0fb349a4e..dcb35c634c 100644 --- a/blog/jXplorer__a_very_nice_LDAP_GUI.html +++ b/blog/jXplorer__a_very_nice_LDAP_GUI.html @@ -60,7 +60,7 @@ and remove the failing query. Nothing big, but very annoying.

@@ -342,7 +342,7 @@ and remove the failing query. Nothing big, but very annoying.

@@ -378,9 +378,9 @@ and remove the failing query. Nothing big, but very annoying.

@@ -410,7 +410,7 @@ and remove the failing query. Nothing big, but very annoying.

diff --git a/blog/listadmin__the_quick_way_to_moderate_mailman_lists___nice_free_software.html b/blog/listadmin__the_quick_way_to_moderate_mailman_lists___nice_free_software.html index 7197bc29b4..4d29d7f897 100644 --- a/blog/listadmin__the_quick_way_to_moderate_mailman_lists___nice_free_software.html +++ b/blog/listadmin__the_quick_way_to_moderate_mailman_lists___nice_free_software.html @@ -129,7 +129,7 @@ sure why.

@@ -411,7 +411,7 @@ sure why.

@@ -447,9 +447,9 @@ sure why.

@@ -479,7 +479,7 @@ sure why.

diff --git a/blog/sitemap.xml b/blog/sitemap.xml index 9f6de80315..29b0a991eb 100644 --- a/blog/sitemap.xml +++ b/blog/sitemap.xml @@ -1015,6 +1015,11 @@ 0.50 weekly + + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + 0.50 + weekly + http://people.skolelinux.org/pere/blog/Free_software_car_computer_solution_.html 0.50 diff --git a/blog/syslog_trusted_timestamp___chain_of_trusted_timestamps_for_your_syslog.html b/blog/syslog_trusted_timestamp___chain_of_trusted_timestamps_for_your_syslog.html index ceba6fac58..f8ea03894a 100644 --- a/blog/syslog_trusted_timestamp___chain_of_trusted_timestamps_for_your_syslog.html +++ b/blog/syslog_trusted_timestamp___chain_of_trusted_timestamps_for_your_syslog.html @@ -153,7 +153,7 @@ activities, please send Bitcoin donations to my address

@@ -435,7 +435,7 @@ activities, please send Bitcoin donations to my address

@@ -471,9 +471,9 @@ activities, please send Bitcoin donations to my address

@@ -503,7 +503,7 @@ activities, please send Bitcoin donations to my address

diff --git a/blog/systemd__an_interesting_alternative_to_upstart.html b/blog/systemd__an_interesting_alternative_to_upstart.html index 32eeb6b330..3c1fc4382c 100644 --- a/blog/systemd__an_interesting_alternative_to_upstart.html +++ b/blog/systemd__an_interesting_alternative_to_upstart.html @@ -78,7 +78,7 @@ with parallel booting enabled by default.

@@ -360,7 +360,7 @@ with parallel booting enabled by default.

@@ -396,9 +396,9 @@ with parallel booting enabled by default.

@@ -428,7 +428,7 @@ with parallel booting enabled by default.

diff --git a/blog/tags/3d-printer/index.html b/blog/tags/3d-printer/index.html index cc4338d5e1..3847e0fa6d 100644 --- a/blog/tags/3d-printer/index.html +++ b/blog/tags/3d-printer/index.html @@ -610,7 +610,7 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

@@ -892,7 +892,7 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

@@ -928,9 +928,9 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

@@ -960,7 +960,7 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

diff --git a/blog/tags/amiga/index.html b/blog/tags/amiga/index.html index 642447e460..4674bf575c 100644 --- a/blog/tags/amiga/index.html +++ b/blog/tags/amiga/index.html @@ -64,7 +64,7 @@ pakke. Kanskje Aros kunne vÃ¦rt interessant for et NUUG-foredrag?

@@ -346,7 +346,7 @@ pakke. Kanskje Aros kunne vÃ¦rt interessant for et NUUG-foredrag?

@@ -382,9 +382,9 @@ pakke. Kanskje Aros kunne vÃ¦rt interessant for et NUUG-foredrag?

@@ -414,7 +414,7 @@ pakke. Kanskje Aros kunne vÃ¦rt interessant for et NUUG-foredrag?

diff --git a/blog/tags/aros/index.html b/blog/tags/aros/index.html index d9516cd010..dcfe5f486d 100644 --- a/blog/tags/aros/index.html +++ b/blog/tags/aros/index.html @@ -64,7 +64,7 @@ pakke. Kanskje Aros kunne vÃ¦rt interessant for et NUUG-foredrag?

@@ -346,7 +346,7 @@ pakke. Kanskje Aros kunne vÃ¦rt interessant for et NUUG-foredrag?

@@ -382,9 +382,9 @@ pakke. Kanskje Aros kunne vÃ¦rt interessant for et NUUG-foredrag?

@@ -414,7 +414,7 @@ pakke. Kanskje Aros kunne vÃ¦rt interessant for et NUUG-foredrag?

diff --git a/blog/tags/bankid/index.html b/blog/tags/bankid/index.html index c6ce5ee913..11fbcf8aeb 100644 --- a/blog/tags/bankid/index.html +++ b/blog/tags/bankid/index.html @@ -476,7 +476,7 @@ omtalt over. Anbefalt lesning.

@@ -758,7 +758,7 @@ omtalt over. Anbefalt lesning.

@@ -794,9 +794,9 @@ omtalt over. Anbefalt lesning.

@@ -826,7 +826,7 @@ omtalt over. Anbefalt lesning.

diff --git a/blog/tags/bitcoin/index.html b/blog/tags/bitcoin/index.html index 341816c4e7..b6673ba6fb 100644 --- a/blog/tags/bitcoin/index.html +++ b/blog/tags/bitcoin/index.html @@ -633,7 +633,7 @@ donations to the address

@@ -915,7 +915,7 @@ donations to the address

@@ -951,9 +951,9 @@ donations to the address

@@ -983,7 +983,7 @@ donations to the address

diff --git a/blog/tags/bootsystem/index.html b/blog/tags/bootsystem/index.html index 7aacea315a..d51b237576 100644 --- a/blog/tags/bootsystem/index.html +++ b/blog/tags/bootsystem/index.html @@ -1276,7 +1276,7 @@ insserv'. Will need to test if that work. :)

@@ -1558,7 +1558,7 @@ insserv'. Will need to test if that work. :)

@@ -1594,9 +1594,9 @@ insserv'. Will need to test if that work. :)

@@ -1626,7 +1626,7 @@ insserv'. Will need to test if that work. :)

diff --git a/blog/tags/bsa/index.html b/blog/tags/bsa/index.html index 7aa9727e67..853349e1ec 100644 --- a/blog/tags/bsa/index.html +++ b/blog/tags/bsa/index.html @@ -152,7 +152,7 @@ pÃ¥ Slashdot.

@@ -434,7 +434,7 @@ pÃ¥ Slashdot.

@@ -470,9 +470,9 @@ pÃ¥ Slashdot.

@@ -502,7 +502,7 @@ pÃ¥ Slashdot.

diff --git a/blog/tags/chrpath/index.html b/blog/tags/chrpath/index.html index bc80993363..a367d4485e 100644 --- a/blog/tags/chrpath/index.html +++ b/blog/tags/chrpath/index.html @@ -159,7 +159,7 @@ include a testsuite check.

@@ -441,7 +441,7 @@ include a testsuite check.

@@ -477,9 +477,9 @@ include a testsuite check.

@@ -509,7 +509,7 @@ include a testsuite check.

diff --git a/blog/tags/debian edu/index.html b/blog/tags/debian edu/index.html index b79638141e..d186dfc077 100644 --- a/blog/tags/debian edu/index.html +++ b/blog/tags/debian edu/index.html @@ -14952,7 +14952,7 @@ be the only one fitting our needs. :/

@@ -15234,7 +15234,7 @@ be the only one fitting our needs. :/

@@ -15270,9 +15270,9 @@ be the only one fitting our needs. :/

@@ -15302,7 +15302,7 @@ be the only one fitting our needs. :/

diff --git a/blog/tags/debian-handbook/index.html b/blog/tags/debian-handbook/index.html index 569cc77dab..c424696148 100644 --- a/blog/tags/debian-handbook/index.html +++ b/blog/tags/debian-handbook/index.html @@ -155,7 +155,7 @@ available for many more languages.

@@ -437,7 +437,7 @@ available for many more languages.

@@ -473,9 +473,9 @@ available for many more languages.

@@ -505,7 +505,7 @@ available for many more languages.

diff --git a/blog/tags/debian/index.html b/blog/tags/debian/index.html index ab726510c0..eb909633c6 100644 --- a/blog/tags/debian/index.html +++ b/blog/tags/debian/index.html @@ -12323,7 +12323,7 @@ be the only one fitting our needs. :/

@@ -12605,7 +12605,7 @@ be the only one fitting our needs. :/

@@ -12641,9 +12641,9 @@ be the only one fitting our needs. :/

@@ -12673,7 +12673,7 @@ be the only one fitting our needs. :/

diff --git a/blog/tags/digistan/index.html b/blog/tags/digistan/index.html index cf83e351f0..69d335dc30 100644 --- a/blog/tags/digistan/index.html +++ b/blog/tags/digistan/index.html @@ -1251,7 +1251,7 @@ produkter basert pÃ¥ standarden.

@@ -1533,7 +1533,7 @@ produkter basert pÃ¥ standarden.

@@ -1569,9 +1569,9 @@ produkter basert pÃ¥ standarden.

@@ -1601,7 +1601,7 @@ produkter basert pÃ¥ standarden.

diff --git a/blog/tags/dld/index.html b/blog/tags/dld/index.html index b27a17ce47..03671bba94 100644 --- a/blog/tags/dld/index.html +++ b/blog/tags/dld/index.html @@ -1528,7 +1528,7 @@ fÃ¥r snart se om det stemmer.

@@ -1810,7 +1810,7 @@ fÃ¥r snart se om det stemmer.

@@ -1846,9 +1846,9 @@ fÃ¥r snart se om det stemmer.

@@ -1878,7 +1878,7 @@ fÃ¥r snart se om det stemmer.

diff --git a/blog/tags/docbook/index.html b/blog/tags/docbook/index.html index c82112d185..b68e70a2c6 100644 --- a/blog/tags/docbook/index.html +++ b/blog/tags/docbook/index.html @@ -1582,7 +1582,7 @@ slik at du kan oppdatere direkte.

@@ -1864,7 +1864,7 @@ slik at du kan oppdatere direkte.

@@ -1900,9 +1900,9 @@ slik at du kan oppdatere direkte.

@@ -1932,7 +1932,7 @@ slik at du kan oppdatere direkte.

diff --git a/blog/tags/drivstoffpriser/index.html b/blog/tags/drivstoffpriser/index.html index bc583e2780..65af306b35 100644 --- a/blog/tags/drivstoffpriser/index.html +++ b/blog/tags/drivstoffpriser/index.html @@ -493,7 +493,7 @@ en liste med stasjoner pÃ¥ samme format som PriserVedStasjoner.

@@ -775,7 +775,7 @@ en liste med stasjoner pÃ¥ samme format som PriserVedStasjoner.

@@ -811,9 +811,9 @@ en liste med stasjoner pÃ¥ samme format som PriserVedStasjoner.

@@ -843,7 +843,7 @@ en liste med stasjoner pÃ¥ samme format som PriserVedStasjoner.

diff --git a/blog/tags/english/english.rss b/blog/tags/english/english.rss index 0042eaee1a..b40e232eb4 100644 --- a/blog/tags/english/english.rss +++ b/blog/tags/english/english.rss @@ -6,6 +6,102 @@ http://people.skolelinux.org/pere/blog/ + + Free software archive system Nikita now able to store documents + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + Sun, 19 Mar 2017 08:00:00 +0100 + The <a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita +Noark 5 core project</a> is implementing the Norwegian standard for +keeping an electronic archive of government documents. +<a href="http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version">The +Noark 5 standard</a> document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +<a href="https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml">announced +it supported the project</a>. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches. + +If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(<a href="irc://irc.freenode.net/%23nikita"">#nikita on +irc.freenode.net</a>) and +<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">the +project mailing list</a>. + +When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +<tt>archive-pdf</tt> to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +<a href="https://en.wikipedia.org/wiki/Fonds">fonds</a>, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester: + +<blockquote><pre> +~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf +using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446 +using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446 + + 0 - Title of the test case file created 2017-03-18T23:49:32.103446 + 1 - Title of the test file created 2017-03-18T23:49:32.103446 +Select which mappe you want (or search term): 0 +Uploading mangelmelding/mangler.pdf + PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt + File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446 +~/src//noark5-tester$ +</pre></blockquote> + +You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The <tt>archive-pdf</tt> +tool can be found in the git repository for the API tester. + +In the project, I have been mostly working on +<a href="https://github.com/petterreinholdtsen/noark5-tester">the API +tester</a> so far, while getting to know the code base. The API +tester currently use +<a href="https://en.wikipedia.org/wiki/HATEOAS">the HATEOAS links</a> +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification. + +The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +<a href="https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding">started +writing down</a> the questions we get from implementing it. We use a +format inspired by how <a href="http://www.opengroup.org/austin/">The +Austin Group</a> collect defect reports for the POSIX standard with +<a href="http://www.opengroup.org/austin/mantis.html">their +instructions for the MANTIS defect tracker system</a>, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a <a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md">request for a procedure for submitting defect reports</a> :). + +The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python. + + + Detecting NFS hangs on Linux without hanging yourself... http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html diff --git a/blog/tags/english/index.html b/blog/tags/english/index.html index fa5e20b173..0ddfc3dd34 100644 --- a/blog/tags/english/index.html +++ b/blog/tags/english/index.html @@ -20,6 +20,114 @@

Entries tagged "english".

+ Free software archive system Nikita now able to store documents +

+ 19th March 2017 +

+ +

If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(#nikita on +irc.freenode.net) and +the +project mailing list.

+ +

+~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
+using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
+using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
+
+ 0 - Title of the test case file created 2017-03-18T23:49:32.103446
+ 1 - Title of the test file created 2017-03-18T23:49:32.103446
+Select which mappe you want (or search term): 0
+Uploading mangelmelding/mangler.pdf
+  PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
+  File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
+~/src//noark5-tester$
+

+ +

+ + + Tags: english, nuug, offentlig innsyn, standard. + + +

Detecting NFS hangs on Linux without hanging yourself... @@ -28048,7 +28156,7 @@ be the only one fitting our needs. :/

@@ -28330,7 +28438,7 @@ be the only one fitting our needs. :/

@@ -28366,9 +28474,9 @@ be the only one fitting our needs. :/

@@ -28398,7 +28506,7 @@ be the only one fitting our needs. :/

diff --git a/blog/tags/fiksgatami/index.html b/blog/tags/fiksgatami/index.html index 1a6b6b58f8..ab50d3ed5d 100644 --- a/blog/tags/fiksgatami/index.html +++ b/blog/tags/fiksgatami/index.html @@ -1344,7 +1344,7 @@ med dem. Dette blir bra.

@@ -1626,7 +1626,7 @@ med dem. Dette blir bra.

@@ -1662,9 +1662,9 @@ med dem. Dette blir bra.

@@ -1694,7 +1694,7 @@ med dem. Dette blir bra.

diff --git a/blog/tags/fildeling/index.html b/blog/tags/fildeling/index.html index 6193fa67f3..3fbb454f50 100644 --- a/blog/tags/fildeling/index.html +++ b/blog/tags/fildeling/index.html @@ -637,7 +637,7 @@ og fildeling av slike filer er fullt ut lovlig.

@@ -919,7 +919,7 @@ og fildeling av slike filer er fullt ut lovlig.

@@ -955,9 +955,9 @@ og fildeling av slike filer er fullt ut lovlig.

@@ -987,7 +987,7 @@ og fildeling av slike filer er fullt ut lovlig.

diff --git a/blog/tags/freeculture/index.html b/blog/tags/freeculture/index.html index 54a7494884..0636ddeee0 100644 --- a/blog/tags/freeculture/index.html +++ b/blog/tags/freeculture/index.html @@ -2117,7 +2117,7 @@ slik at du kan oppdatere direkte.

@@ -2399,7 +2399,7 @@ slik at du kan oppdatere direkte.

@@ -2435,9 +2435,9 @@ slik at du kan oppdatere direkte.

@@ -2467,7 +2467,7 @@ slik at du kan oppdatere direkte.

diff --git a/blog/tags/freedombox/index.html b/blog/tags/freedombox/index.html index 4580fd20d8..97c9ec63eb 100644 --- a/blog/tags/freedombox/index.html +++ b/blog/tags/freedombox/index.html @@ -938,7 +938,7 @@ default password is 'secret'.

@@ -1220,7 +1220,7 @@ default password is 'secret'.

@@ -1256,9 +1256,9 @@ default password is 'secret'.

@@ -1288,7 +1288,7 @@ default password is 'secret'.

diff --git a/blog/tags/frikanalen/index.html b/blog/tags/frikanalen/index.html index 4755713e4e..a79ff4d6f1 100644 --- a/blog/tags/frikanalen/index.html +++ b/blog/tags/frikanalen/index.html @@ -1424,7 +1424,7 @@ NUUG lykkes med Ã¥ fÃ¥ ut sine opptak med like stor suksess.

@@ -1706,7 +1706,7 @@ NUUG lykkes med Ã¥ fÃ¥ ut sine opptak med like stor suksess.

@@ -1742,9 +1742,9 @@ NUUG lykkes med Ã¥ fÃ¥ ut sine opptak med like stor suksess.

@@ -1774,7 +1774,7 @@ NUUG lykkes med Ã¥ fÃ¥ ut sine opptak med like stor suksess.

diff --git a/blog/tags/h264/index.html b/blog/tags/h264/index.html index be046e3946..a2474178b0 100644 --- a/blog/tags/h264/index.html +++ b/blog/tags/h264/index.html @@ -2510,7 +2510,7 @@ sure hope it was using the announced Ogg Theora support. :)

@@ -2792,7 +2792,7 @@ sure hope it was using the announced Ogg Theora support. :)

@@ -2828,9 +2828,9 @@ sure hope it was using the announced Ogg Theora support. :)

@@ -2860,7 +2860,7 @@ sure hope it was using the announced Ogg Theora support. :)

diff --git a/blog/tags/intervju/index.html b/blog/tags/intervju/index.html index a8d60b4d95..ba52d8531b 100644 --- a/blog/tags/intervju/index.html +++ b/blog/tags/intervju/index.html @@ -5132,7 +5132,7 @@ veldig bra utvalg av gratis spill som er av hÃ¸y kvalitet. Veldig lett

@@ -5414,7 +5414,7 @@ veldig bra utvalg av gratis spill som er av hÃ¸y kvalitet. Veldig lett

@@ -5450,9 +5450,9 @@ veldig bra utvalg av gratis spill som er av hÃ¸y kvalitet. Veldig lett

@@ -5482,7 +5482,7 @@ veldig bra utvalg av gratis spill som er av hÃ¸y kvalitet. Veldig lett

diff --git a/blog/tags/isenkram/index.html b/blog/tags/isenkram/index.html index 92e3358ebb..b74b2cc3fb 100644 --- a/blog/tags/isenkram/index.html +++ b/blog/tags/isenkram/index.html @@ -1592,7 +1592,7 @@ please send me an email. :)

@@ -1874,7 +1874,7 @@ please send me an email. :)

@@ -1910,9 +1910,9 @@ please send me an email. :)

@@ -1942,7 +1942,7 @@ please send me an email. :)

diff --git a/blog/tags/kart/index.html b/blog/tags/kart/index.html index 14c0f193b3..20d2aa7ece 100644 --- a/blog/tags/kart/index.html +++ b/blog/tags/kart/index.html @@ -1585,7 +1585,7 @@ det viser at behovet for fribruks-sjÃ¸kart er til stedet.

@@ -1867,7 +1867,7 @@ det viser at behovet for fribruks-sjÃ¸kart er til stedet.

@@ -1903,9 +1903,9 @@ det viser at behovet for fribruks-sjÃ¸kart er til stedet.

@@ -1935,7 +1935,7 @@ det viser at behovet for fribruks-sjÃ¸kart er til stedet.

diff --git a/blog/tags/ldap/index.html b/blog/tags/ldap/index.html index 6d9e09ad0d..16fac24ccc 100644 --- a/blog/tags/ldap/index.html +++ b/blog/tags/ldap/index.html @@ -1054,7 +1054,7 @@ new IETF work group?

@@ -1336,7 +1336,7 @@ new IETF work group?

@@ -1372,9 +1372,9 @@ new IETF work group?

@@ -1404,7 +1404,7 @@ new IETF work group?

diff --git a/blog/tags/lenker/index.html b/blog/tags/lenker/index.html index 96424debe7..004fac9950 100644 --- a/blog/tags/lenker/index.html +++ b/blog/tags/lenker/index.html @@ -649,7 +649,7 @@ Word 2007 hÃ¥ndterer ODF dÃ¥rlig

@@ -931,7 +931,7 @@ Word 2007 hÃ¥ndterer ODF dÃ¥rlig

@@ -967,9 +967,9 @@ Word 2007 hÃ¥ndterer ODF dÃ¥rlig

@@ -999,7 +999,7 @@ Word 2007 hÃ¥ndterer ODF dÃ¥rlig

diff --git a/blog/tags/lsdvd/index.html b/blog/tags/lsdvd/index.html index dc388c5f6b..794c0b83fd 100644 --- a/blog/tags/lsdvd/index.html +++ b/blog/tags/lsdvd/index.html @@ -135,7 +135,7 @@ list. :)

@@ -417,7 +417,7 @@ list. :)

@@ -453,9 +453,9 @@ list. :)

@@ -485,7 +485,7 @@ list. :)

diff --git a/blog/tags/ltsp/index.html b/blog/tags/ltsp/index.html index 2b174b6e15..4a8038afd0 100644 --- a/blog/tags/ltsp/index.html +++ b/blog/tags/ltsp/index.html @@ -69,7 +69,7 @@ of these cards.

@@ -351,7 +351,7 @@ of these cards.

@@ -387,9 +387,9 @@ of these cards.

@@ -419,7 +419,7 @@ of these cards.

diff --git a/blog/tags/mesh network/index.html b/blog/tags/mesh network/index.html index 008eb0a79c..171b62569f 100644 --- a/blog/tags/mesh network/index.html +++ b/blog/tags/mesh network/index.html @@ -719,7 +719,7 @@ mesh system.

@@ -1001,7 +1001,7 @@ mesh system.

@@ -1037,9 +1037,9 @@ mesh system.

@@ -1069,7 +1069,7 @@ mesh system.

diff --git a/blog/tags/multimedia/index.html b/blog/tags/multimedia/index.html index af10e1f31e..31834263e0 100644 --- a/blog/tags/multimedia/index.html +++ b/blog/tags/multimedia/index.html @@ -3298,7 +3298,7 @@ be the only one fitting our needs. :/

@@ -3580,7 +3580,7 @@ be the only one fitting our needs. :/

@@ -3616,9 +3616,9 @@ be the only one fitting our needs. :/

@@ -3648,7 +3648,7 @@ be the only one fitting our needs. :/

diff --git a/blog/tags/nice free software/index.html b/blog/tags/nice free software/index.html index 69a9f55846..8ddcc3d4b3 100644 --- a/blog/tags/nice free software/index.html +++ b/blog/tags/nice free software/index.html @@ -1057,7 +1057,7 @@ section.

@@ -1339,7 +1339,7 @@ section.

@@ -1375,9 +1375,9 @@ section.

@@ -1407,7 +1407,7 @@ section.

diff --git a/blog/tags/norsk/index.html b/blog/tags/norsk/index.html index 78a2ce4748..caa737a447 100644 --- a/blog/tags/norsk/index.html +++ b/blog/tags/norsk/index.html @@ -25817,7 +25817,7 @@ forsÃ¸k.

@@ -26099,7 +26099,7 @@ forsÃ¸k.

@@ -26135,9 +26135,9 @@ forsÃ¸k.

@@ -26167,7 +26167,7 @@ forsÃ¸k.

diff --git a/blog/tags/nuug/index.html b/blog/tags/nuug/index.html index 4037278794..d932d98917 100644 --- a/blog/tags/nuug/index.html +++ b/blog/tags/nuug/index.html @@ -20,6 +20,114 @@

Entries tagged "nuug".

+ Free software archive system Nikita now able to store documents +

+ 19th March 2017 +

+ +

If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(#nikita on +irc.freenode.net) and +the +project mailing list.

+ +

+~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
+using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
+using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
+
+ 0 - Title of the test case file created 2017-03-18T23:49:32.103446
+ 1 - Title of the test file created 2017-03-18T23:49:32.103446
+Select which mappe you want (or search term): 0
+Uploading mangelmelding/mangler.pdf
+  PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
+  File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
+~/src//noark5-tester$
+

+ +

+ + + Tags: english, nuug, offentlig innsyn, standard. + + +

Detect OOXML files with undefined behaviour? @@ -13304,7 +13412,7 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

@@ -13586,7 +13694,7 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

@@ -13622,9 +13730,9 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

@@ -13654,7 +13762,7 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

diff --git a/blog/tags/nuug/nuug.rss b/blog/tags/nuug/nuug.rss index 4631eaaa05..0c0b8add95 100644 --- a/blog/tags/nuug/nuug.rss +++ b/blog/tags/nuug/nuug.rss @@ -6,6 +6,102 @@ http://people.skolelinux.org/pere/blog/ + + Free software archive system Nikita now able to store documents + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + Sun, 19 Mar 2017 08:00:00 +0100 + The <a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita +Noark 5 core project</a> is implementing the Norwegian standard for +keeping an electronic archive of government documents. +<a href="http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version">The +Noark 5 standard</a> document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +<a href="https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml">announced +it supported the project</a>. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches. + +If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(<a href="irc://irc.freenode.net/%23nikita"">#nikita on +irc.freenode.net</a>) and +<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">the +project mailing list</a>. + +When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +<tt>archive-pdf</tt> to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +<a href="https://en.wikipedia.org/wiki/Fonds">fonds</a>, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester: + +<blockquote><pre> +~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf +using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446 +using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446 + + 0 - Title of the test case file created 2017-03-18T23:49:32.103446 + 1 - Title of the test file created 2017-03-18T23:49:32.103446 +Select which mappe you want (or search term): 0 +Uploading mangelmelding/mangler.pdf + PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt + File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446 +~/src//noark5-tester$ +</pre></blockquote> + +You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The <tt>archive-pdf</tt> +tool can be found in the git repository for the API tester. + +In the project, I have been mostly working on +<a href="https://github.com/petterreinholdtsen/noark5-tester">the API +tester</a> so far, while getting to know the code base. The API +tester currently use +<a href="https://en.wikipedia.org/wiki/HATEOAS">the HATEOAS links</a> +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification. + +The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +<a href="https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding">started +writing down</a> the questions we get from implementing it. We use a +format inspired by how <a href="http://www.opengroup.org/austin/">The +Austin Group</a> collect defect reports for the POSIX standard with +<a href="http://www.opengroup.org/austin/mantis.html">their +instructions for the MANTIS defect tracker system</a>, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a <a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md">request for a procedure for submitting defect reports</a> :). + +The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python. + + + Detect OOXML files with undefined behaviour? http://people.skolelinux.org/pere/blog/Detect_OOXML_files_with_undefined_behaviour_.html diff --git a/blog/tags/offentlig innsyn/index.html b/blog/tags/offentlig innsyn/index.html index 82a07e9e34..1e9714f9ce 100644 --- a/blog/tags/offentlig innsyn/index.html +++ b/blog/tags/offentlig innsyn/index.html @@ -20,6 +20,114 @@

Entries tagged "offentlig innsyn".

+ Free software archive system Nikita now able to store documents +

+ 19th March 2017 +

+ +

If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(#nikita on +irc.freenode.net) and +the +project mailing list.

+ +

+~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
+using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
+using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
+
+ 0 - Title of the test case file created 2017-03-18T23:49:32.103446
+ 1 - Title of the test file created 2017-03-18T23:49:32.103446
+Select which mappe you want (or search term): 0
+Uploading mangelmelding/mangler.pdf
+  PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
+  File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
+~/src//noark5-tester$
+

+ +

+ + + Tags: english, nuug, offentlig innsyn, standard. + + +

Ruling ignored our objections to the seizure of popcorn-time.no (#domstolkontroll) @@ -3517,7 +3625,7 @@ til Ã¥ levere hver uke. Har ikke undersÃ¸kt noen av de andre.

@@ -3799,7 +3907,7 @@ til Ã¥ levere hver uke. Har ikke undersÃ¸kt noen av de andre.

@@ -3835,9 +3943,9 @@ til Ã¥ levere hver uke. Har ikke undersÃ¸kt noen av de andre.

@@ -3867,7 +3975,7 @@ til Ã¥ levere hver uke. Har ikke undersÃ¸kt noen av de andre.

diff --git a/blog/tags/offentlig innsyn/offentlig innsyn.rss b/blog/tags/offentlig innsyn/offentlig innsyn.rss index cfad7f4009..670885bcea 100644 --- a/blog/tags/offentlig innsyn/offentlig innsyn.rss +++ b/blog/tags/offentlig innsyn/offentlig innsyn.rss @@ -6,6 +6,102 @@ http://people.skolelinux.org/pere/blog/ + + Free software archive system Nikita now able to store documents + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + Sun, 19 Mar 2017 08:00:00 +0100 + The <a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita +Noark 5 core project</a> is implementing the Norwegian standard for +keeping an electronic archive of government documents. +<a href="http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version">The +Noark 5 standard</a> document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +<a href="https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml">announced +it supported the project</a>. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches. + +If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(<a href="irc://irc.freenode.net/%23nikita"">#nikita on +irc.freenode.net</a>) and +<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">the +project mailing list</a>. + +When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +<tt>archive-pdf</tt> to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +<a href="https://en.wikipedia.org/wiki/Fonds">fonds</a>, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester: + +<blockquote><pre> +~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf +using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446 +using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446 + + 0 - Title of the test case file created 2017-03-18T23:49:32.103446 + 1 - Title of the test file created 2017-03-18T23:49:32.103446 +Select which mappe you want (or search term): 0 +Uploading mangelmelding/mangler.pdf + PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt + File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446 +~/src//noark5-tester$ +</pre></blockquote> + +You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The <tt>archive-pdf</tt> +tool can be found in the git repository for the API tester. + +In the project, I have been mostly working on +<a href="https://github.com/petterreinholdtsen/noark5-tester">the API +tester</a> so far, while getting to know the code base. The API +tester currently use +<a href="https://en.wikipedia.org/wiki/HATEOAS">the HATEOAS links</a> +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification. + +The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +<a href="https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding">started +writing down</a> the questions we get from implementing it. We use a +format inspired by how <a href="http://www.opengroup.org/austin/">The +Austin Group</a> collect defect reports for the POSIX standard with +<a href="http://www.opengroup.org/austin/mantis.html">their +instructions for the MANTIS defect tracker system</a>, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a <a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md">request for a procedure for submitting defect reports</a> :). + +The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python. + + + Ruling ignored our objections to the seizure of popcorn-time.no (#domstolkontroll) http://people.skolelinux.org/pere/blog/Ruling_ignored_our_objections_to_the_seizure_of_popcorn_time_no___domstolkontroll_.html diff --git a/blog/tags/open311/index.html b/blog/tags/open311/index.html index 0dcc7222d9..3dd887190a 100644 --- a/blog/tags/open311/index.html +++ b/blog/tags/open311/index.html @@ -164,7 +164,7 @@ work like the free software project communities I am used to.

@@ -446,7 +446,7 @@ work like the free software project communities I am used to.

@@ -482,9 +482,9 @@ work like the free software project communities I am used to.

@@ -514,7 +514,7 @@ work like the free software project communities I am used to.

diff --git a/blog/tags/opphavsrett/index.html b/blog/tags/opphavsrett/index.html index 9d43756214..5485048f2c 100644 --- a/blog/tags/opphavsrett/index.html +++ b/blog/tags/opphavsrett/index.html @@ -7384,7 +7384,7 @@ database.

@@ -7666,7 +7666,7 @@ database.

@@ -7702,9 +7702,9 @@ database.

@@ -7734,7 +7734,7 @@ database.

diff --git a/blog/tags/personvern/index.html b/blog/tags/personvern/index.html index d80497cb85..7e7ec234b0 100644 --- a/blog/tags/personvern/index.html +++ b/blog/tags/personvern/index.html @@ -9511,7 +9511,7 @@ kontanter for noen dager siden.

@@ -9793,7 +9793,7 @@ kontanter for noen dager siden.

@@ -9829,9 +9829,9 @@ kontanter for noen dager siden.

@@ -9861,7 +9861,7 @@ kontanter for noen dager siden.

diff --git a/blog/tags/raid/index.html b/blog/tags/raid/index.html index c4fb2a800d..76be98f6df 100644 --- a/blog/tags/raid/index.html +++ b/blog/tags/raid/index.html @@ -105,7 +105,7 @@ disk(s) is failing when the RAID is running short on disks.

@@ -387,7 +387,7 @@ disk(s) is failing when the RAID is running short on disks.

@@ -423,9 +423,9 @@ disk(s) is failing when the RAID is running short on disks.

@@ -455,7 +455,7 @@ disk(s) is failing when the RAID is running short on disks.

diff --git a/blog/tags/reactos/index.html b/blog/tags/reactos/index.html index 086dd64deb..0f62b835f1 100644 --- a/blog/tags/reactos/index.html +++ b/blog/tags/reactos/index.html @@ -104,7 +104,7 @@ image.

@@ -386,7 +386,7 @@ image.

@@ -422,9 +422,9 @@ image.

@@ -454,7 +454,7 @@ image.

diff --git a/blog/tags/reprap/index.html b/blog/tags/reprap/index.html index 8e6ac50878..d600dd058e 100644 --- a/blog/tags/reprap/index.html +++ b/blog/tags/reprap/index.html @@ -531,7 +531,7 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

@@ -813,7 +813,7 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

@@ -849,9 +849,9 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

@@ -881,7 +881,7 @@ hÃ¥per det ikke gÃ¥r tapt pÃ¥ samme vis.

diff --git a/blog/tags/rfid/index.html b/blog/tags/rfid/index.html index c8024e52c1..17fde89fbd 100644 --- a/blog/tags/rfid/index.html +++ b/blog/tags/rfid/index.html @@ -435,7 +435,7 @@ mer om de nye biometriske passene.

@@ -717,7 +717,7 @@ mer om de nye biometriske passene.

@@ -753,9 +753,9 @@ mer om de nye biometriske passene.

@@ -785,7 +785,7 @@ mer om de nye biometriske passene.

diff --git a/blog/tags/robot/index.html b/blog/tags/robot/index.html index d85c3bf2cf..713c2060f2 100644 --- a/blog/tags/robot/index.html +++ b/blog/tags/robot/index.html @@ -541,7 +541,7 @@ firmwaren. :)

@@ -823,7 +823,7 @@ firmwaren. :)

@@ -859,9 +859,9 @@ firmwaren. :)

@@ -891,7 +891,7 @@ firmwaren. :)

diff --git a/blog/tags/rss/index.html b/blog/tags/rss/index.html index 34ab5ec4c8..21b0289652 100644 --- a/blog/tags/rss/index.html +++ b/blog/tags/rss/index.html @@ -58,7 +58,7 @@ forsÃ¸k.

@@ -340,7 +340,7 @@ forsÃ¸k.

@@ -376,9 +376,9 @@ forsÃ¸k.

@@ -408,7 +408,7 @@ forsÃ¸k.

diff --git a/blog/tags/ruter/index.html b/blog/tags/ruter/index.html index f2810b616d..6d65388052 100644 --- a/blog/tags/ruter/index.html +++ b/blog/tags/ruter/index.html @@ -277,7 +277,7 @@ OsloomrÃ¥det.

@@ -559,7 +559,7 @@ OsloomrÃ¥det.

@@ -595,9 +595,9 @@ OsloomrÃ¥det.

@@ -627,7 +627,7 @@ OsloomrÃ¥det.

diff --git a/blog/tags/scraperwiki/index.html b/blog/tags/scraperwiki/index.html index bbf234c70e..4c4358941f 100644 --- a/blog/tags/scraperwiki/index.html +++ b/blog/tags/scraperwiki/index.html @@ -147,7 +147,7 @@ av Ã¥ dele informasjon med andre uten bruksbegresninger.

@@ -429,7 +429,7 @@ av Ã¥ dele informasjon med andre uten bruksbegresninger.

@@ -465,9 +465,9 @@ av Ã¥ dele informasjon med andre uten bruksbegresninger.

@@ -497,7 +497,7 @@ av Ã¥ dele informasjon med andre uten bruksbegresninger.

diff --git a/blog/tags/sikkerhet/index.html b/blog/tags/sikkerhet/index.html index c8bcadaf88..f982387bba 100644 --- a/blog/tags/sikkerhet/index.html +++ b/blog/tags/sikkerhet/index.html @@ -4260,7 +4260,7 @@ betydelige.

@@ -4542,7 +4542,7 @@ betydelige.

@@ -4578,9 +4578,9 @@ betydelige.

@@ -4610,7 +4610,7 @@ betydelige.

diff --git a/blog/tags/sitesummary/index.html b/blog/tags/sitesummary/index.html index b6192f27aa..925f0c2b77 100644 --- a/blog/tags/sitesummary/index.html +++ b/blog/tags/sitesummary/index.html @@ -263,7 +263,7 @@ everything is taken care of.

@@ -545,7 +545,7 @@ everything is taken care of.

@@ -581,9 +581,9 @@ everything is taken care of.

@@ -613,7 +613,7 @@ everything is taken care of.

diff --git a/blog/tags/skepsis/index.html b/blog/tags/skepsis/index.html index f504280b70..9863fd0baf 100644 --- a/blog/tags/skepsis/index.html +++ b/blog/tags/skepsis/index.html @@ -443,7 +443,7 @@ skyskrapere. Takke meg til en tur til mÃ¥nen.

@@ -725,7 +725,7 @@ skyskrapere. Takke meg til en tur til mÃ¥nen.

@@ -761,9 +761,9 @@ skyskrapere. Takke meg til en tur til mÃ¥nen.

@@ -793,7 +793,7 @@ skyskrapere. Takke meg til en tur til mÃ¥nen.

diff --git a/blog/tags/standard/index.html b/blog/tags/standard/index.html index 22c9bc9a33..c679015f55 100644 --- a/blog/tags/standard/index.html +++ b/blog/tags/standard/index.html @@ -20,6 +20,114 @@

Entries tagged "standard".

+ Free software archive system Nikita now able to store documents +

+ 19th March 2017 +

+ +

If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(#nikita on +irc.freenode.net) and +the +project mailing list.

+ +

+~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
+using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
+using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
+
+ 0 - Title of the test case file created 2017-03-18T23:49:32.103446
+ 1 - Title of the test file created 2017-03-18T23:49:32.103446
+Select which mappe you want (or search term): 0
+Uploading mangelmelding/mangler.pdf
+  PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
+  File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
+~/src//noark5-tester$
+

+ +

+ + + Tags: english, nuug, offentlig innsyn, standard. + + +

Detect OOXML files with undefined behaviour? @@ -4697,7 +4805,7 @@ Kjenner kun til ufullstendige lÃ¸sninger for slikt.

@@ -4979,7 +5087,7 @@ Kjenner kun til ufullstendige lÃ¸sninger for slikt.

@@ -5015,9 +5123,9 @@ Kjenner kun til ufullstendige lÃ¸sninger for slikt.

@@ -5047,7 +5155,7 @@ Kjenner kun til ufullstendige lÃ¸sninger for slikt.

diff --git a/blog/tags/standard/standard.rss b/blog/tags/standard/standard.rss index 3706a4a17e..28456fb141 100644 --- a/blog/tags/standard/standard.rss +++ b/blog/tags/standard/standard.rss @@ -6,6 +6,102 @@ http://people.skolelinux.org/pere/blog/ + + Free software archive system Nikita now able to store documents + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + Sun, 19 Mar 2017 08:00:00 +0100 + The <a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita +Noark 5 core project</a> is implementing the Norwegian standard for +keeping an electronic archive of government documents. +<a href="http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version">The +Noark 5 standard</a> document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +<a href="https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml">announced +it supported the project</a>. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches. + +If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(<a href="irc://irc.freenode.net/%23nikita"">#nikita on +irc.freenode.net</a>) and +<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">the +project mailing list</a>. + +When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +<tt>archive-pdf</tt> to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +<a href="https://en.wikipedia.org/wiki/Fonds">fonds</a>, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester: + +<blockquote><pre> +~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf +using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446 +using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446 + + 0 - Title of the test case file created 2017-03-18T23:49:32.103446 + 1 - Title of the test file created 2017-03-18T23:49:32.103446 +Select which mappe you want (or search term): 0 +Uploading mangelmelding/mangler.pdf + PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt + File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446 +~/src//noark5-tester$ +</pre></blockquote> + +You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The <tt>archive-pdf</tt> +tool can be found in the git repository for the API tester. + +In the project, I have been mostly working on +<a href="https://github.com/petterreinholdtsen/noark5-tester">the API +tester</a> so far, while getting to know the code base. The API +tester currently use +<a href="https://en.wikipedia.org/wiki/HATEOAS">the HATEOAS links</a> +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification. + +The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +<a href="https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding">started +writing down</a> the questions we get from implementing it. We use a +format inspired by how <a href="http://www.opengroup.org/austin/">The +Austin Group</a> collect defect reports for the POSIX standard with +<a href="http://www.opengroup.org/austin/mantis.html">their +instructions for the MANTIS defect tracker system</a>, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a <a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md">request for a procedure for submitting defect reports</a> :). + +The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python. + + + Detect OOXML files with undefined behaviour? http://people.skolelinux.org/pere/blog/Detect_OOXML_files_with_undefined_behaviour_.html diff --git a/blog/tags/stavekontroll/index.html b/blog/tags/stavekontroll/index.html index d2605f67b9..5d00cd8325 100644 --- a/blog/tags/stavekontroll/index.html +++ b/blog/tags/stavekontroll/index.html @@ -380,7 +380,7 @@ stavekontrollen.

@@ -662,7 +662,7 @@ stavekontrollen.

@@ -698,9 +698,9 @@ stavekontrollen.

@@ -730,7 +730,7 @@ stavekontrollen.

diff --git a/blog/tags/stortinget/index.html b/blog/tags/stortinget/index.html index 3cf8e22c16..cbd0dda560 100644 --- a/blog/tags/stortinget/index.html +++ b/blog/tags/stortinget/index.html @@ -1085,7 +1085,7 @@ at vi i NUUG har fÃ¥tt operativ en norsk utgave av

@@ -1367,7 +1367,7 @@ at vi i NUUG har fÃ¥tt operativ en norsk utgave av

@@ -1403,9 +1403,9 @@ at vi i NUUG har fÃ¥tt operativ en norsk utgave av

@@ -1435,7 +1435,7 @@ at vi i NUUG har fÃ¥tt operativ en norsk utgave av

diff --git a/blog/tags/surveillance/index.html b/blog/tags/surveillance/index.html index 9bd2854dc8..5476e259b7 100644 --- a/blog/tags/surveillance/index.html +++ b/blog/tags/surveillance/index.html @@ -4443,7 +4443,7 @@ automatisk over i spesialkartet.

@@ -4725,7 +4725,7 @@ automatisk over i spesialkartet.

@@ -4761,9 +4761,9 @@ automatisk over i spesialkartet.

@@ -4793,7 +4793,7 @@ automatisk over i spesialkartet.

diff --git a/blog/tags/sysadmin/index.html b/blog/tags/sysadmin/index.html index 3299672847..fa0b2ad7c1 100644 --- a/blog/tags/sysadmin/index.html +++ b/blog/tags/sysadmin/index.html @@ -337,7 +337,7 @@ and DHCP updates I wanted to do.

@@ -619,7 +619,7 @@ and DHCP updates I wanted to do.

@@ -655,9 +655,9 @@ and DHCP updates I wanted to do.

@@ -687,7 +687,7 @@ and DHCP updates I wanted to do.

diff --git a/blog/tags/usenix/index.html b/blog/tags/usenix/index.html index b940890f51..7c91b6673e 100644 --- a/blog/tags/usenix/index.html +++ b/blog/tags/usenix/index.html @@ -233,7 +233,7 @@ for other sites to consider too?

@@ -515,7 +515,7 @@ for other sites to consider too?

@@ -551,9 +551,9 @@ for other sites to consider too?

@@ -583,7 +583,7 @@ for other sites to consider too?

diff --git a/blog/tags/valg/index.html b/blog/tags/valg/index.html index 9d3693311b..0ef1054863 100644 --- a/blog/tags/valg/index.html +++ b/blog/tags/valg/index.html @@ -945,7 +945,7 @@ inneholdt i Iran hvis de ikke hadde hemmelige valg?

@@ -1227,7 +1227,7 @@ inneholdt i Iran hvis de ikke hadde hemmelige valg?

@@ -1263,9 +1263,9 @@ inneholdt i Iran hvis de ikke hadde hemmelige valg?

@@ -1295,7 +1295,7 @@ inneholdt i Iran hvis de ikke hadde hemmelige valg?

diff --git a/blog/tags/video/index.html b/blog/tags/video/index.html index 9a789a408b..ed1e4c2d77 100644 --- a/blog/tags/video/index.html +++ b/blog/tags/video/index.html @@ -4918,7 +4918,7 @@ larger stick as well.

@@ -5200,7 +5200,7 @@ larger stick as well.

@@ -5236,9 +5236,9 @@ larger stick as well.

@@ -5268,7 +5268,7 @@ larger stick as well.

diff --git a/blog/tags/vitenskap/index.html b/blog/tags/vitenskap/index.html index 3f7a83befd..f545482ef8 100644 --- a/blog/tags/vitenskap/index.html +++ b/blog/tags/vitenskap/index.html @@ -399,7 +399,7 @@ skyskrapere. Takke meg til en tur til mÃ¥nen.

@@ -681,7 +681,7 @@ skyskrapere. Takke meg til en tur til mÃ¥nen.

@@ -717,9 +717,9 @@ skyskrapere. Takke meg til en tur til mÃ¥nen.

@@ -749,7 +749,7 @@ skyskrapere. Takke meg til en tur til mÃ¥nen.

diff --git a/blog/tags/web/index.html b/blog/tags/web/index.html index 2908140a70..060ff2d72c 100644 --- a/blog/tags/web/index.html +++ b/blog/tags/web/index.html @@ -3701,7 +3701,7 @@ be the only one fitting our needs. :/

@@ -3983,7 +3983,7 @@ be the only one fitting our needs. :/

@@ -4019,9 +4019,9 @@ be the only one fitting our needs. :/