blog/archive/2018/10/10.rss

   1 <?xml version="1.0" encoding="ISO-8859-1"?>
   2 <rss version='2.0' xmlns:lj='http://www.livejournal.org/rss/lj/1.0/'>
   3         <channel>
   4                 <title>Petter Reinholdtsen - Entries from October 2018</title>
   5                 <description>Entries from October 2018</description>
   6                 <link>http://people.skolelinux.org/pere/blog/</link>
   7
   8
   9         <item>
  10                 <title>Release 0.2 of free software archive system Nikita announced</title>
  11                 <link>http://people.skolelinux.org/pere/blog/Release_0_2_of_free_software_archive_system_Nikita_announced.html</link>
  12                 <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Release_0_2_of_free_software_archive_system_Nikita_announced.html</guid>
  13                 <pubDate>Thu, 18 Oct 2018 14:40:00 +0200</pubDate>
  14                 <description>&lt;p&gt;This morning, the new release of the
  15 &lt;a href=&quot;https://gitlab.com/OsloMet-ABI/nikita-noark5-core/&quot;&gt;Nikita
  16 Noark 5 core project&lt;/a&gt; was
  17 &lt;a href=&quot;https://lists.nuug.no/pipermail/nikita-noark/2018-October/000406.html&quot;&gt;announced
  18 on the project mailing list&lt;/a&gt;.  The free software solution is an
  19 implementation of the Norwegian archive standard Noark 5 used by
  20 government offices in Norway.  These were the changes in version 0.2
  21 since version 0.1.1 (from NEWS.md):
  22
  23 &lt;ul&gt;
  24   &lt;li&gt;Fix typos in REL names&lt;/li&gt;
  25   &lt;li&gt;Tidy up error message reporting&lt;/li&gt;
  26   &lt;li&gt;Fix issue where we used Integer.valueOf(), not Integer.getInteger()&lt;/li&gt;
  27   &lt;li&gt;Change some String handling to StringBuffer&lt;/li&gt;
  28   &lt;li&gt;Fix error reporting&lt;/li&gt;
  29   &lt;li&gt;Code tidy-up&lt;/li&gt;
  30   &lt;li&gt;Fix issue using static non-synchronized SimpleDateFormat to avoid
  31       race conditions&lt;/li&gt;
  32   &lt;li&gt;Fix problem where deserialisers were treating integers as strings&lt;/li&gt;
  33   &lt;li&gt;Update methods to make them null-safe&lt;/li&gt;
  34   &lt;li&gt;Fix many issues reported by coverity&lt;/li&gt;
  35   &lt;li&gt;Improve equals(), compareTo() and hash() in domain model&lt;/li&gt;
  36   &lt;li&gt;Improvements to the domain model for metadata classes&lt;/li&gt;
  37   &lt;li&gt;Fix CORS issues when downloading document&lt;/li&gt;
  38   &lt;li&gt;Implementation of case-handling with registryEntry and document upload&lt;/li&gt;
  39   &lt;li&gt;Better support in Javascript for OPTIONS&lt;/li&gt;
  40   &lt;li&gt;Adding concept description of mail integration&lt;/li&gt;
  41   &lt;li&gt;Improve setting of default values for GET on ny-journalpost&lt;/li&gt;
  42   &lt;li&gt;Better handling of required values during deserialisation &lt;/li&gt;
  43   &lt;li&gt;Changed tilknyttetDato (M620) from date to dateTime&lt;/li&gt;
  44   &lt;li&gt;Corrected some opprettetDato (M600) (de)serialisation errors.&lt;/li&gt;
  45   &lt;li&gt;Improve parse error reporting.&lt;/li&gt;
  46   &lt;li&gt;Started on OData search and filtering.&lt;/li&gt;
  47   &lt;li&gt;Added Contributor Covenant Code of Conduct to project.&lt;/li&gt;
  48   &lt;li&gt;Moved repository and project from Github to Gitlab.&lt;/li&gt;
  49   &lt;li&gt;Restructured repository, moved code into src/ and web/.&lt;/li&gt;
  50   &lt;li&gt;Updated code to use Spring Boot version 2.&lt;/li&gt;
  51   &lt;li&gt;Added support for OAuth2 authentication.&lt;/li&gt;
  52   &lt;li&gt;Fixed several bugs discovered by Coverity.&lt;/li&gt;
  53   &lt;li&gt;Corrected handling of date/datetime fields.&lt;/li&gt;
  54   &lt;li&gt;Improved error reporting when rejecting during deserializatoin.&lt;/li&gt;
  55   &lt;li&gt;Adjusted default values provided for ny-arkivdel, ny-mappe,
  56    ny-saksmappe, ny-journalpost and ny-dokumentbeskrivelse.&lt;/li&gt;
  57   &lt;li&gt;Several fixes for korrespondansepart*.&lt;/li&gt;
  58   &lt;li&gt;Updated web GUI:
  59      &lt;ul&gt;
  60        &lt;li&gt;Now handle both file upload and download.&lt;/li&gt;
  61        &lt;li&gt;Uses new OAuth2 authentication for login.&lt;/li&gt;
  62        &lt;li&gt;Forms now fetches default values from API using GET.&lt;/li&gt;
  63        &lt;li&gt;Added RFC 822 (email), TIFF and JPEG to list of possible file formats.&lt;/li&gt;
  64      &lt;/ul&gt;&lt;/li&gt;
  65 &lt;/ul&gt;
  66
  67 &lt;p&gt;The changes and improvements are extensive.  Running diffstat on
  68 the changes between git tab 0.1.1 and 0.2 show 1098 files changed,
  69 108666 insertions(+), 54066 deletions(-).&lt;/p&gt;
  70
  71 &lt;p&gt;If free and open standardized archiving API sound interesting to
  72 you, please contact us on IRC
  73 (&lt;a href=&quot;irc://irc.freenode.net/%23nikita&quot;&gt;#nikita on
  74 irc.freenode.net&lt;/a&gt;) or email
  75 (&lt;a href=&quot;https://lists.nuug.no/mailman/listinfo/nikita-noark&quot;&gt;nikita-noark
  76 mailing list&lt;/a&gt;).&lt;/p&gt;
  77
  78 &lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
  79 activities, please send Bitcoin donations to my address
  80 &lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;
  81 </description>
  82         </item>
  83
  84         <item>
  85                 <title>Fetching trusted timestamps using the rfc3161ng python module</title>
  86                 <link>http://people.skolelinux.org/pere/blog/Fetching_trusted_timestamps_using_the_rfc3161ng_python_module.html</link>
  87                 <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Fetching_trusted_timestamps_using_the_rfc3161ng_python_module.html</guid>
  88                 <pubDate>Mon, 8 Oct 2018 12:30:00 +0200</pubDate>
  89                 <description>&lt;p&gt;I have  earlier covered the basics of trusted timestamping using the
  90 &#39;openssl ts&#39; client.  See blog post for
  91 &lt;a href=&quot;http://people.skolelinux.org/pere/blog/Public_Trusted_Timestamping_services_for_everyone.html&quot;&gt;2014&lt;/a&gt;,
  92 &lt;a href=&quot;http://people.skolelinux.org/pere/blog/syslog_trusted_timestamp___chain_of_trusted_timestamps_for_your_syslog.html&quot;&gt;2016&lt;/a&gt;
  93 and
  94 &lt;a href=&quot;http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html&quot;&gt;2017&lt;/a&gt;
  95 for those stories.  But some times I want to integrate the timestamping
  96 in other code, and recently I needed to integrate it into Python.
  97 After searching a bit, I found
  98 &lt;a href=&quot;https://dev.entrouvert.org/projects/python-rfc3161&quot;&gt;the
  99 rfc3161 library&lt;/a&gt; which seemed like a good fit, but I soon
 100 discovered it only worked for python version 2, and I needed something
 101 that work with python version 3.  Luckily I next came across
 102 &lt;a href=&quot;https://github.com/trbs/rfc3161ng/&quot;&gt;the rfc3161ng library&lt;/a&gt;,
 103 a fork of the original rfc3161 library.  Not only is it working with
 104 python 3, it have fixed a few of the bugs in the original library, and
 105 it has an active maintainer.  I decided to wrap it up and make it
 106 &lt;a href=&quot;https://tracker.debian.org/pkg/python-rfc3161ng&quot;&gt;available in
 107 Debian&lt;/a&gt;, and a few days ago it entered Debian unstable and testing.&lt;/p&gt;
 108
 109 &lt;p&gt;Using the library is fairly straight forward.  The only slightly
 110 problematic step is to fetch the required certificates to verify the
 111 timestamp.  For some services it is straight forward, while for others
 112 I have not yet figured out how to do it.  Here is a small standalone
 113 code example based on of the integration tests in the library code:&lt;/p&gt;
 114
 115 &lt;pre&gt;
 116 #!/usr/bin/python3
 117
 118 &quot;&quot;&quot;
 119
 120 Python 3 script demonstrating how to use the rfc3161ng module to
 121 get trusted timestamps.
 122
 123 The license of this code is the same as the license of the rfc3161ng
 124 library, ie MIT/BSD.
 125
 126 &quot;&quot;&quot;
 127
 128 import os
 129 import pyasn1.codec.der
 130 import rfc3161ng
 131 import subprocess
 132 import tempfile
 133 import urllib.request
 134
 135 def store(f, data):
 136     f.write(data)
 137     f.flush()
 138     f.seek(0)
 139
 140 def fetch(url, f=None):
 141     response = urllib.request.urlopen(url)
 142     data = response.read()
 143     if f:
 144         store(f, data)
 145     return data
 146
 147 def main():
 148     with tempfile.NamedTemporaryFile() as cert_f,\
 149          tempfile.NamedTemporaryFile() as ca_f,\
 150          tempfile.NamedTemporaryFile() as msg_f,\
 151          tempfile.NamedTemporaryFile() as tsr_f:
 152
 153         # First fetch certificates used by service
 154         certificate_data = fetch(&#39;https://freetsa.org/files/tsa.crt&#39;, cert_f)
 155         ca_data_data = fetch(&#39;https://freetsa.org/files/cacert.pem&#39;, ca_f)
 156
 157         # Then timestamp the message
 158         timestamper = \
 159             rfc3161ng.RemoteTimestamper(&#39;http://freetsa.org/tsr&#39;,
 160                                         certificate=certificate_data)
 161         data = b&quot;Python forever!\n&quot;
 162         tsr = timestamper(data=data, return_tsr=True)
 163
 164         # Finally, convert message and response to something &#39;openssl ts&#39; can verify
 165         store(msg_f, data)
 166         store(tsr_f, pyasn1.codec.der.encoder.encode(tsr))
 167         args = [&quot;openssl&quot;, &quot;ts&quot;, &quot;-verify&quot;,
 168                 &quot;-data&quot;, msg_f.name,
 169                 &quot;-in&quot;, tsr_f.name,
 170                 &quot;-CAfile&quot;, ca_f.name,
 171                 &quot;-untrusted&quot;, cert_f.name]
 172         subprocess.check_call(args)
 173
 174 if &#39;__main__&#39; == __name__:
 175    main()
 176 &lt;/pre&gt;
 177
 178 &lt;p&gt;The code fetches the required certificates, store them as temporary
 179 files, timestamp a simple message, store the message and timestamp to
 180 disk and ask &#39;openssl ts&#39; to verify the timestamp.  A timestamp is
 181 around 1.5 kiB in size, and should be fairly easy to store for future
 182 use.&lt;/p&gt;
 183
 184 &lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
 185 activities, please send Bitcoin donations to my address
 186 &lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;
 187 </description>
 188         </item>
 189
 190         <item>
 191                 <title>Automatic Google Drive sync using grive in Debian</title>
 192                 <link>http://people.skolelinux.org/pere/blog/Automatic_Google_Drive_sync_using_grive_in_Debian.html</link>
 193                 <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Automatic_Google_Drive_sync_using_grive_in_Debian.html</guid>
 194                 <pubDate>Thu, 4 Oct 2018 15:20:00 +0200</pubDate>
 195                 <description>&lt;p&gt;A few days, I rescued a Windows victim over to Debian.  To try to
 196 rescue the remains, I helped set up automatic sync with Google Drive.
 197 I did not find any sensible Debian package handling this
 198 automatically, so I rebuild the grive2 source from
 199 &lt;a href=&quot;http://www.webupd8.org/&quot;&gt;the Ubuntu UPD8 PPA&lt;/a&gt; to do the
 200 task and added a autostart desktop entry and a small shell script to
 201 run in the background while the user is logged in to do the sync.
 202 Here is a sketch of the setup for future reference.&lt;/p&gt;
 203
 204 &lt;p&gt;I first created &lt;tt&gt;~/googledrive&lt;/tt&gt;, entered the directory and
 205 ran &#39;&lt;tt&gt;grive -a&lt;/tt&gt;&#39; to authenticate the machine/user.  Next, I
 206 created a autostart hook in &lt;tt&gt;~/.config/autostart/grive.desktop&lt;/tt&gt;
 207 to start the sync when the user log in:&lt;/p&gt;
 208
 209 &lt;p&gt;&lt;blockquote&gt;&lt;pre&gt;
 210 [Desktop Entry]
 211 Name=Google drive autosync
 212 Type=Application
 213 Exec=/home/user/bin/grive-sync
 214 &lt;/pre&gt;&lt;/blockquote&gt;&lt;/p&gt;
 215
 216 &lt;p&gt;Finally, I wrote the &lt;tt&gt;~/bin/grive-sync&lt;/tt&gt; script to sync
 217 ~/googledrive/ with the files in Google Drive.&lt;/p&gt;
 218
 219 &lt;p&gt;&lt;blockquote&gt;&lt;pre&gt;
 220 #!/bin/sh
 221 set -e
 222 cd ~/
 223 cleanup() {
 224     if [ &quot;$syncpid&quot; ] ; then
 225         kill $syncpid
 226     fi
 227 }
 228 trap cleanup EXIT INT QUIT
 229 /usr/lib/grive/grive-sync.sh listen googledrive 2&gt;&amp;1 | sed &quot;s%^%$0:%&quot; &amp;
 230 syncpdi=$!
 231 while true; do
 232     if ! xhost &gt;/dev/null 2&gt;&amp;1 ; then
 233         echo &quot;no DISPLAY, exiting as the user probably logged out&quot;
 234         exit 1
 235     fi
 236     if [ ! -e /run/user/1000/grive-sync.sh_googledrive ] ; then
 237         /usr/lib/grive/grive-sync.sh sync googledrive
 238     fi
 239     sleep 300
 240 done 2&gt;&amp;1 | sed &quot;s%^%$0:%&quot;
 241 &lt;/pre&gt;&lt;/blockquote&gt;&lt;/p&gt;
 242
 243 &lt;p&gt;Feel free to use the setup if you want.  It can be assumed to be
 244 GNU GPL v2 licensed (or any later version, at your leisure), but I
 245 doubt this code is possible to claim copyright on.&lt;/p&gt;
 246
 247 &lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
 248 activities, please send Bitcoin donations to my address
 249 &lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;
 250 </description>
 251         </item>
 252
 253         </channel>
 254 </rss>