X-Git-Url: https://pere.pagekite.me/gitweb/homepage.git/blobdiff_plain/fc20a6cd50c2669321f60fc80703c58917dc5081..9b90db63be4ef8ae3cc2db3385af1738048fe6f4:/blog/archive/2017/03/03.rss diff --git a/blog/archive/2017/03/03.rss b/blog/archive/2017/03/03.rss index ca8c34500d..dc0aec379b 100644 --- a/blog/archive/2017/03/03.rss +++ b/blog/archive/2017/03/03.rss @@ -6,6 +6,102 @@ http://people.skolelinux.org/pere/blog/ + + Free software archive system Nikita now able to store documents + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html + Sun, 19 Mar 2017 08:00:00 +0100 + <p>The <a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita +Noark 5 core project</a> is implementing the Norwegian standard for +keeping an electronic archive of government documents. +<a href="http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version">The +Noark 5 standard</a> document the requirement for data systems used by +the archives in the Norwegian government, and the Noark 5 web interface +specification document a REST web service for storing, searching and +retrieving documents and metadata in such archive. I've been involved +in the project since a few weeks before Christmas, when the Norwegian +Unix User Group +<a href="https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml">announced +it supported the project</a>. I believe this is an important project, +and hope it can make it possible for the government archives in the +future to use free software to keep the archives we citizens depend +on. But as I do not hold such archive myself, personally my first use +case is to store and analyse public mail journal metadata published +from the government. I find it useful to have a clear use case in +mind when developing, to make sure the system scratches one of my +itches.</p> + +<p>If you would like to help make sure there is a free software +alternatives for the archives, please join our IRC channel +(<a href="irc://irc.freenode.net/%23nikita"">#nikita on +irc.freenode.net</a>) and +<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">the +project mailing list</a>.</p> + +<p>When I got involved, the web service could store metadata about +documents. But a few weeks ago, a new milestone was reached when it +became possible to store full text documents too. Yesterday, I +completed an implementation of a command line tool +<tt>archive-pdf</tt> to upload a PDF file to the archive using this +API. The tool is very simple at the moment, and find existing +<a href="https://en.wikipedia.org/wiki/Fonds">fonds</a>, series and +files while asking the user to select which one to use if more than +one exist. Once a file is identified, the PDF is associated with the +file and uploaded, using the title extracted from the PDF itself. The +process is fairly similar to visiting the archive, opening a cabinet, +locating a file and storing a piece of paper in the archive. Here is +a test run directly after populating the database with test data using +our API tester:</p> + +<p><blockquote><pre> +~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf +using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446 +using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446 + + 0 - Title of the test case file created 2017-03-18T23:49:32.103446 + 1 - Title of the test file created 2017-03-18T23:49:32.103446 +Select which mappe you want (or search term): 0 +Uploading mangelmelding/mangler.pdf + PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt + File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446 +~/src//noark5-tester$ +</pre></blockquote></p> + +<p>You can see here how the fonds (arkiv) and serie (arkivdel) only had +one option, while the user need to choose which file (mappe) to use +among the two created by the API tester. The <tt>archive-pdf</tt> +tool can be found in the git repository for the API tester.</p> + +<p>In the project, I have been mostly working on +<a href="https://github.com/petterreinholdtsen/noark5-tester">the API +tester</a> so far, while getting to know the code base. The API +tester currently use +<a href="https://en.wikipedia.org/wiki/HATEOAS">the HATEOAS links</a> +to traverse the entire exposed service API and verify that the exposed +operations and objects match the specification, as well as trying to +create objects holding metadata and uploading a simple XML file to +store. The tester has proved very useful for finding flaws in our +implementation, as well as flaws in the reference site and the +specification.</p> + +<p>The test document I uploaded is a summary of all the specification +defects we have collected so far while implementing the web service. +There are several unclear and conflicting parts of the specification, +and we have +<a href="https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding">started +writing down</a> the questions we get from implementing it. We use a +format inspired by how <a href="http://www.opengroup.org/austin/">The +Austin Group</a> collect defect reports for the POSIX standard with +<a href="http://www.opengroup.org/austin/mantis.html">their +instructions for the MANTIS defect tracker system</a>, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a <a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md">request for a procedure for submitting defect reports</a> :). + +<p>The Nikita project is implemented using Java and Spring, and is +fairly easy to get up and running using Docker containers for those +that want to test the current code base. The API tester is +implemented in Python.</p> + + + Detecting NFS hangs on Linux without hanging yourself... http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html