<atom:link href="http://people.skolelinux.org/pere/blog/index.rss" rel="self" type="application/rss+xml" />
<item>
- <title>Release 0.1.1 of free software archive system Nikita announced</title>
- <link>http://people.skolelinux.org/pere/blog/Release_0_1_1_of_free_software_archive_system_Nikita_announced.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Release_0_1_1_of_free_software_archive_system_Nikita_announced.html</guid>
- <pubDate>Sat, 10 Jun 2017 00:40:00 +0200</pubDate>
- <description><p>I am very happy to report that the
-<a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita Noark 5
-core project</a> tagged its second release today. The free software
-solution is an implementation of the Norwegian archive standard Noark
-5 used by government offices in Norway. These were the changes in
-version 0.1.1 since version 0.1.0 (from NEWS.md):
-
-<ul>
-
- <li>Continued work on the angularjs GUI, including document upload.</li>
- <li>Implemented correspondencepartPerson, correspondencepartUnit and
- correspondencepartInternal</li>
- <li>Applied for coverity coverage and started submitting code on
- regualr basis.</li>
- <li>Started fixing bugs reported by coverity</li>
- <li>Corrected and completed HATEOAS links to make sure entire API is
- available via URLs in _links.</li>
- <li>Corrected all relation URLs to use trailing slash.</li>
- <li>Add initial support for storing data in ElasticSearch.</li>
- <li>Now able to receive and store uploaded files in the archive.</li>
- <li>Changed JSON output for object lists to have relations in _links.</li>
- <li>Improve JSON output for empty object lists.</li>
- <li>Now uses correct MIME type application/vnd.noark5-v4+json.</li>
- <li>Added support for docker container images.</li>
- <li>Added simple API browser implemented in JavaScript/Angular.</li>
- <li>Started on archive client implemented in JavaScript/Angular.</li>
- <li>Started on prototype to show the public mail journal.</li>
- <li>Improved performance by disabling Sprint FileWatcher.</li>
- <li>Added support for 'arkivskaper', 'saksmappe' and 'journalpost'.</li>
- <li>Added support for some metadata codelists.</li>
- <li>Added support for Cross-origin resource sharing (CORS).</li>
- <li>Changed login method from Basic Auth to JSON Web Token (RFC 7519)
- style.</li>
- <li>Added support for GET-ing ny-* URLs.</li>
- <li>Added support for modifying entities using PUT and eTag.</li>
- <li>Added support for returning XML output on request.</li>
- <li>Removed support for English field and class names, limiting ourself
- to the official names.</li>
- <li>...</li>
-
-</ul>
-
-<p>If this sound interesting to you, please contact us on IRC (#nikita
-on irc.freenode.net) or email
-(<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">nikita-noark
-mailing list).</p>
+ <title>Time for an official MIME type for patches?</title>
+ <link>http://people.skolelinux.org/pere/blog/Time_for_an_official_MIME_type_for_patches_.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Time_for_an_official_MIME_type_for_patches_.html</guid>
+ <pubDate>Thu, 1 Nov 2018 08:15:00 +0100</pubDate>
+ <description><p>As part of my involvement in
+<a href="https://gitlab.com/OsloMet-ABI/nikita-noark5-core">the Nikita
+archive API project</a>, I've been importing a fairly large lump of
+emails into a test instance of the archive to see how well this would
+go. I picked a subset of <a href="https://notmuchmail.org/">my
+notmuch email database</a>, all public emails sent to me via
+@lists.debian.org, giving me a set of around 216 000 emails to import.
+In the process, I had a look at the various attachments included in
+these emails, to figure out what to do with attachments, and noticed
+that one of the most common attachment formats do not have
+<a href="https://www.iana.org/assignments/media-types/media-types.xhtml">an
+official MIME type</a> registered with IANA/IETF. The output from
+diff, ie the input for patch, is on the top 10 list of formats
+included in these emails. At the moment people seem to use either
+text/x-patch or text/x-diff, but neither is officially registered. It
+would be better if one official MIME type were registered and used
+everywhere.</p>
+
+<p>To try to get one official MIME type for these files, I've brought
+up the topic on
+<a href="https://www.ietf.org/mailman/listinfo/media-types">the
+media-types mailing list</a>. If you are interested in discussion
+which MIME type to use as the official for patch files, or involved in
+making software using a MIME type for patches, perhaps you would like
+to join the discussion?</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</description>
</item>
<item>
- <title>Idea for storing trusted timestamps in a Noark 5 archive</title>
- <link>http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html</guid>
- <pubDate>Wed, 7 Jun 2017 21:40:00 +0200</pubDate>
- <description><p><em>This is a copy of
-<a href="https://lists.nuug.no/pipermail/nikita-noark/2017-June/000297.html">an
-email I posted to the nikita-noark mailing list</a>. Please follow up
-there if you would like to discuss this topic. The background is that
-we are making a free software archive system based on the Norwegian
-<a href="https://www.arkivverket.no/forvaltning-og-utvikling/regelverk-og-standarder/noark-standarden">Noark
-5 standard</a> for government archives.</em></p>
-
-<p>I've been wondering a bit lately how trusted timestamps could be
-stored in Noark 5.
-<a href="https://en.wikipedia.org/wiki/Trusted_timestamping">Trusted
-timestamps</a> can be used to verify that some information
-(document/file/checksum/metadata) have not been changed since a
-specific time in the past. This is useful to verify the integrity of
-the documents in the archive.</p>
-
-<p>Then it occured to me, perhaps the trusted timestamps could be
-stored as dokument variants (ie dokumentobjekt referered to from
-dokumentbeskrivelse) with the filename set to the hash it is
-stamping?</p>
-
-<p>Given a "dokumentbeskrivelse" with an associated "dokumentobjekt",
-a new dokumentobjekt is associated with "dokumentbeskrivelse" with the
-same attributes as the stamped dokumentobjekt except these
-attributes:</p>
-
-<ul>
-
-<li>format -> "RFC3161"
-<li>mimeType -> "application/timestamp-reply"
-<li>formatDetaljer -> "&lt;source URL for timestamp service&gt;"
-<li>filenavn -> "&lt;sjekksum&gt;.tsr"
-
-</ul>
-
-<p>This assume a service following
-<a href="https://tools.ietf.org/html/rfc3161">IETF RFC 3161</a> is
-used, which specifiy the given MIME type for replies and the .tsr file
-ending for the content of such trusted timestamp. As far as I can
-tell from the Noark 5 specifications, it is OK to have several
-variants/renderings of a dokument attached to a given
-dokumentbeskrivelse objekt. It might be stretching it a bit to make
-some of these variants represent crypto-signatures useful for
-verifying the document integrity instead of representing the dokument
-itself.</p>
-
-<p>Using the source of the service in formatDetaljer allow several
-timestamping services to be used. This is useful to spread the risk
-of key compromise over several organisations. It would only be a
-problem to trust the timestamps if all of the organisations are
-compromised.</p>
-
-<p>The following oneliner on Linux can be used to generate the tsr
-file. $input is the path to the file to checksum, and $sha256 is the
-SHA-256 checksum of the file (ie the "<sjekksum>.tsr" value mentioned
-above).</p>
-
-<p><blockquote><pre>
-openssl ts -query -data "$inputfile" -cert -sha256 -no_nonce \
- | curl -s -H "Content-Type: application/timestamp-query" \
- --data-binary "@-" http://zeitstempel.dfn.de > $sha256.tsr
-</pre></blockquote></p>
-
-<p>To verify the timestamp, you first need to download the public key
-of the trusted timestamp service, for example using this command:</p>
-
-<p><blockquote><pre>
-wget -O ca-cert.txt \
- https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt
-</pre></blockquote></p>
-
-<p>Note, the public key should be stored alongside the timestamps in
-the archive to make sure it is also available 100 years from now. It
-is probably a good idea to standardise how and were to store such
-public keys, to make it easier to find for those trying to verify
-documents 100 or 1000 years from now. :)</p>
-
-<p>The verification itself is a simple openssl command:</p>
-
-<p><blockquote><pre>
-openssl ts -verify -data $inputfile -in $sha256.tsr \
- -CAfile ca-cert.txt -text
-</pre></blockquote></p>
-
-<p>Is there any reason this approach would not work? Is it somehow against
-the Noark 5 specification?</p>
+ <title>Measuring the speaker frequency response using the AUDMES free software GUI - nice free software</title>
+ <link>http://people.skolelinux.org/pere/blog/Measuring_the_speaker_frequency_response_using_the_AUDMES_free_software_GUI___nice_free_software.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Measuring_the_speaker_frequency_response_using_the_AUDMES_free_software_GUI___nice_free_software.html</guid>
+ <pubDate>Mon, 22 Oct 2018 08:40:00 +0200</pubDate>
+ <description><p><img src="http://people.skolelinux.org/pere/blog/images/2018-10-22-audmes-measure-speakers.png" align="right" width="40%"/></p>
+
+<p>My current home stereo is a patchwork of various pieces I got on
+flee markeds over the years. It is amazing what kind of equipment
+show up there. I've been wondering for a while if it was possible to
+measure how well this equipment is working together, and decided to
+see how far I could get using free software. After trawling the web I
+came across an article from DIY Audio and Video on
+<a href="https://www.diyaudioandvideo.com/Tutorial/SpeakerResponseTesting/">Speaker
+Testing and Analysis</a> describing how to test speakers, and it listing
+several software options, among them
+<a href="https://sourceforge.net/projects/audmes/">AUDio MEasurement
+System (AUDMES)</a>. It is the only free software system I could find
+focusing on measuring speakers and audio frequency response. In the
+process I also found an interesting article from NOVO on
+<a href="http://novo.press/understanding-speaker-specifications-and-frequency-response/">Understanding
+Speaker Specifications and Frequency Response</a> and an article from
+ecoustics on
+<a href="https://www.ecoustics.com/articles/understanding-speaker-frequency-response/">Understanding
+Speaker Frequency Response</a>, with a lot of information on what to
+look for and how to interpret the graphs. Armed with this knowledge,
+I set out to measure the state of my speakers.</p>
+
+<p>The first hurdle was that AUDMES hadn't seen a commit for 10 years
+and did not build with current compilers and libraries. I got in
+touch with its author, who no longer was spending time on the program
+but gave me write access to the subversion repository on Sourceforge.
+The end result is that now the code build on Linux and is capable of
+saving and loading the collected frequency response data in CSV
+format. The application is quite nice and flexible, and I was able to
+select the input and output audio interfaces independently. This made
+it possible to use a USB mixer as the input source, while sending
+output via my laptop headphone connection. I lacked the hardware and
+cabling to figure out a different way to get independent cabling to
+speakers and microphone.</p>
+
+<p>Using this setup I could see how a large range of high frequencies
+apparently were not making it out of my speakers. The picture show
+the frequency response measurement of one of the speakers. Note the
+frequency lines seem to be slightly misaligned, compared to the CSV
+output from the program. I can not hear several of these are high
+frequencies, according to measurement from
+<a href="http://freehearingtestsoftware.com">Free Hearing Test
+Software</a>, an freeware system to measure your hearing (still
+looking for a free software alternative), so I do not know if they are
+coming out out the speakers. I thus do not quite know how to figure
+out if the missing frequencies is a problem with the microphone, the
+amplifier or the speakers, but I managed to rule out the audio card in my
+PC by measuring my Bose noise canceling headset using its own
+microphone. This setup was able to see the high frequency tones, so
+the problem with my stereo had to be in the amplifier or speakers.</p>
+
+<p>Anyway, to try to role out one factor I ended up picking up a new
+set of speakers at a flee marked, and these work a lot better than the
+old speakers, so I guess the microphone and amplifier is OK. If you
+need to measure your own speakers, check out AUDMES. If more people
+get involved, perhaps the project could become good enough to
+<a href="https://bugs.debian.org/910876">include in Debian</a>? And if
+you know of some other free software to measure speakers and amplifier
+performance, please let me know. I am aware of the freeware option
+<a href="https://www.roomeqwizard.com/">REW</a>, but I want something
+that can be developed also when the vendor looses interest.</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</description>
</item>
<item>
- <title>Når nynorskoversettelsen svikter til eksamen...</title>
- <link>http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html</guid>
- <pubDate>Sat, 3 Jun 2017 08:20:00 +0200</pubDate>
- <description><p><a href="http://www.aftenposten.no/norge/Krever-at-elever-ma-fa-annullert-eksamen-etter-rot-med-oppgavetekster-622459b.html">Aftenposten
-melder i dag</a> om feil i eksamensoppgavene for eksamen i politikk og
-menneskerettigheter, der teksten i bokmåls og nynorskutgaven ikke var
-like. Oppgaveteksten er gjengitt i artikkelen, og jeg ble nysgjerring
-på om den fri oversetterløsningen
-<a href="https://www.apertium.org/">Apertium</a> ville gjort en bedre
-jobb enn Utdanningsdirektoratet. Det kan se slik ut.</p>
-
-<p>Her er bokmålsoppgaven fra eksamenen:</p>
-
-<blockquote>
-<p>Drøft utfordringene knyttet til nasjonalstatenes og andre aktørers
-rolle og muligheter til å håndtere internasjonale utfordringer, som
-for eksempel flykningekrisen.</p>
-
-<p>Vedlegge er eksempler på tekster som kan gi relevante perspektiver
-på temaet:</p>
-<ol>
-<li>Flykningeregnskapet 2016, UNHCR og IDMC
-<li>«Grenseløst Europa for fall» A-Magasinet, 26. november 2015
-</ol>
-
-</blockquote>
-
-<p>Dette oversetter Apertium slik:</p>
-
-<blockquote>
-<p>Drøft utfordringane knytte til nasjonalstatane sine og rolla til
-andre aktørar og høve til å handtera internasjonale utfordringar, som
-til dømes *flykningekrisen.</p>
-
-<p>Vedleggja er døme på tekster som kan gje relevante perspektiv på
-temaet:</p>
-
-<ol>
-<li>*Flykningeregnskapet 2016, *UNHCR og *IDMC</li>
-<li>«*Grenseløst Europa for fall» A-Magasinet, 26. november 2015</li>
-</ol>
-
-</blockquote>
-
-<p>Ord som ikke ble forstått er markert med stjerne (*), og trenger
-ekstra språksjekk. Men ingen ord er forsvunnet, slik det var i
-oppgaven elevene fikk presentert på eksamen. Jeg mistenker dog at
-"andre aktørers rolle og muligheter til ..." burde vært oversatt til
-"rolla til andre aktørar og deira høve til ..." eller noe slikt, men
-det er kanskje flisespikking. Det understreker vel bare at det alltid
-trengs korrekturlesning etter automatisk oversettelse.</p>
+ <title>Web browser integration of VLC with Bittorrent support</title>
+ <link>http://people.skolelinux.org/pere/blog/Web_browser_integration_of_VLC_with_Bittorrent_support.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Web_browser_integration_of_VLC_with_Bittorrent_support.html</guid>
+ <pubDate>Sun, 21 Oct 2018 09:50:00 +0200</pubDate>
+ <description><p>Bittorrent is as far as I know, currently the most efficient way to
+distribute content on the Internet. It is used all by all sorts of
+content providers, from national TV stations like
+<a href="https://www.nrk.no/">NRK</a>, Linux distributors like
+<a href="https://www.debian.org/">Debian</a> and
+<a href="https://www.ubuntu.com/">Ubuntu</a>, and of course the
+<a href="https://archive.org/">Internet archive</A>.
+
+<p>Almost a month ago
+<a href="https://tracker.debian.org/pkg/vlc-plugin-bittorrent">a new
+package adding Bittorrent support to VLC</a> became available in
+Debian testing and unstable. To test it, simply install it like
+this:</p>
+
+<p><pre>
+apt install vlc-plugin-bittorrent
+</pre></p>
+
+<p>Since the plugin was made available for the first time in Debian,
+several improvements have been made to it. In version 2.2-4, now
+available in both testing and unstable, a desktop file is provided to
+teach browsers to start VLC when the user click on torrent files or
+magnet links. The last part is thanks to me finally understanding
+what the strange x-scheme-handler style MIME types in desktop files
+are used for. By adding x-scheme-handler/magnet to the MimeType entry
+in the desktop file, at least the browsers Firefox and Chromium will
+suggest to start VLC when selecting a magnet URI on a web page. The
+end result is that now, with the plugin installed in Buster and Sid,
+one can visit any
+<a href="https://archive.org/details/CopyingIsNotTheft1080p">Internet
+Archive page with movies</a> using a web browser and click on the
+torrent link to start streaming the movie.</p>
+
+<p>Note, there is still some misfeatures in the plugin. One is the
+fact that it will hang and
+<a href="https://github.com/johang/vlc-bittorrent/issues/13">block VLC
+from exiting until the torrent streaming starts</a>. Another is the
+fact that it
+<a href="https://github.com/johang/vlc-bittorrent/issues/9">will pick
+and play a random file in a multi file torrent</a>. This is not
+always the video file you want. Combined with the first it can be a
+bit hard to get the video streaming going. But when it work, it seem
+to do a good job.</p>
+
+<p>For the Debian packaging, I would love to find a good way to test
+if the plugin work with VLC using autopkgtest. I tried, but do not
+know enough of the inner workings of VLC to get it working. For now
+the autopkgtest script is only checking if the .so file was
+successfully loaded by VLC. If you have any suggestions, please
+submit a patch to the Debian bug tracking system.</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</description>
</item>
<item>
- <title>Epost inn som arkivformat i Riksarkivarens forskrift?</title>
- <link>http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html</guid>
- <pubDate>Thu, 27 Apr 2017 11:30:00 +0200</pubDate>
- <description><p>I disse dager, med frist 1. mai, har Riksarkivaren ute en høring på
-sin forskrift. Som en kan se er det ikke mye tid igjen før fristen
-som går ut på søndag. Denne forskriften er det som lister opp hvilke
-formater det er greit å arkivere i
-<a href="http://www.arkivverket.no/arkivverket/Offentleg-forvalting/Noark/Noark-5">Noark
-5-løsninger</a> i Norge.</p>
-
-<p>Jeg fant høringsdokumentene hos
-<a href="https://www.arkivrad.no/aktuelt/riksarkivarens-forskrift-pa-horing">Norsk
-Arkivråd</a> etter å ha blitt tipset på epostlisten til
-<a href="https://github.com/hiOA-ABI/nikita-noark5-core">fri
-programvareprosjektet Nikita Noark5-Core</a>, som lager et Noark 5
-Tjenestegresesnitt. Jeg er involvert i Nikita-prosjektet og takket
-være min interesse for tjenestegrensesnittsprosjektet har jeg lest en
-god del Noark 5-relaterte dokumenter, og til min overraskelse oppdaget
-at standard epost ikke er på listen over godkjente formater som kan
-arkiveres. Høringen med frist søndag er en glimrende mulighet til å
-forsøke å gjøre noe med det. Jeg holder på med
-<a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/hoering-arkivforskrift.tex">egen
-høringsuttalelse</a>, og lurer på om andre er interessert i å støtte
-forslaget om å tillate arkivering av epost som epost i arkivet.</p>
-
-<p>Er du igang med å skrive egen høringsuttalelse allerede? I så fall
-kan du jo vurdere å ta med en formulering om epost-lagring. Jeg tror
-ikke det trengs så mye. Her et kort forslag til tekst:</p>
-
-<p><blockquote>
-
- <p>Viser til høring sendt ut 2017-02-17 (Riksarkivarens referanse
- 2016/9840 HELHJO), og tillater oss å sende inn noen innspill om
- revisjon av Forskrift om utfyllende tekniske og arkivfaglige
- bestemmelser om behandling av offentlige arkiver (Riksarkivarens
- forskrift).</p>
-
- <p>Svært mye av vår kommuikasjon foregår i dag på e-post. Vi
- foreslår derfor at Internett-e-post, slik det er beskrevet i IETF
- RFC 5322,
- <a href="https://tools.ietf.org/html/rfc5322">https://tools.ietf.org/html/rfc5322</a>. bør
- inn som godkjent dokumentformat. Vi foreslår at forskriftens
- oversikt over godkjente dokumentformater ved innlevering i § 5-16
- endres til å ta med Internett-e-post.</p>
-
-</blockquote></p>
-
-<p>Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan
-epost kan lagres i en Noark 5-struktur, og holder på å skrive et
-forslag om hvordan dette kan gjøres som vil bli sendt over til
-arkivverket så snart det er ferdig. De som er interesserte kan
-<a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/epostlagring.md">følge
-fremdriften på web</a>.</p>
-
-<p>Oppdatering 2017-04-28: I dag ble høringuttalelsen jeg skrev
- <a href="https://www.nuug.no/news/NUUGs_h_ringuttalelse_til_Riksarkivarens_forskrift.shtml">sendt
- inn av foreningen NUUG</a>.</p>
+ <title>Release 0.2 of free software archive system Nikita announced</title>
+ <link>http://people.skolelinux.org/pere/blog/Release_0_2_of_free_software_archive_system_Nikita_announced.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Release_0_2_of_free_software_archive_system_Nikita_announced.html</guid>
+ <pubDate>Thu, 18 Oct 2018 14:40:00 +0200</pubDate>
+ <description><p>This morning, the new release of the
+<a href="https://gitlab.com/OsloMet-ABI/nikita-noark5-core/">Nikita
+Noark 5 core project</a> was
+<a href="https://lists.nuug.no/pipermail/nikita-noark/2018-October/000406.html">announced
+on the project mailing list</a>. The free software solution is an
+implementation of the Norwegian archive standard Noark 5 used by
+government offices in Norway. These were the changes in version 0.2
+since version 0.1.1 (from NEWS.md):
+
+<ul>
+ <li>Fix typos in REL names</li>
+ <li>Tidy up error message reporting</li>
+ <li>Fix issue where we used Integer.valueOf(), not Integer.getInteger()</li>
+ <li>Change some String handling to StringBuffer</li>
+ <li>Fix error reporting</li>
+ <li>Code tidy-up</li>
+ <li>Fix issue using static non-synchronized SimpleDateFormat to avoid
+ race conditions</li>
+ <li>Fix problem where deserialisers were treating integers as strings</li>
+ <li>Update methods to make them null-safe</li>
+ <li>Fix many issues reported by coverity</li>
+ <li>Improve equals(), compareTo() and hash() in domain model</li>
+ <li>Improvements to the domain model for metadata classes</li>
+ <li>Fix CORS issues when downloading document</li>
+ <li>Implementation of case-handling with registryEntry and document upload</li>
+ <li>Better support in Javascript for OPTIONS</li>
+ <li>Adding concept description of mail integration</li>
+ <li>Improve setting of default values for GET on ny-journalpost</li>
+ <li>Better handling of required values during deserialisation </li>
+ <li>Changed tilknyttetDato (M620) from date to dateTime</li>
+ <li>Corrected some opprettetDato (M600) (de)serialisation errors.</li>
+ <li>Improve parse error reporting.</li>
+ <li>Started on OData search and filtering.</li>
+ <li>Added Contributor Covenant Code of Conduct to project.</li>
+ <li>Moved repository and project from Github to Gitlab.</li>
+ <li>Restructured repository, moved code into src/ and web/.</li>
+ <li>Updated code to use Spring Boot version 2.</li>
+ <li>Added support for OAuth2 authentication.</li>
+ <li>Fixed several bugs discovered by Coverity.</li>
+ <li>Corrected handling of date/datetime fields.</li>
+ <li>Improved error reporting when rejecting during deserializatoin.</li>
+ <li>Adjusted default values provided for ny-arkivdel, ny-mappe,
+ ny-saksmappe, ny-journalpost and ny-dokumentbeskrivelse.</li>
+ <li>Several fixes for korrespondansepart*.</li>
+ <li>Updated web GUI:
+ <ul>
+ <li>Now handle both file upload and download.</li>
+ <li>Uses new OAuth2 authentication for login.</li>
+ <li>Forms now fetches default values from API using GET.</li>
+ <li>Added RFC 822 (email), TIFF and JPEG to list of possible file formats.</li>
+ </ul></li>
+</ul>
+
+<p>The changes and improvements are extensive. Running diffstat on
+the changes between git tab 0.1.1 and 0.2 show 1098 files changed,
+108666 insertions(+), 54066 deletions(-).</p>
+
+<p>If free and open standardized archiving API sound interesting to
+you, please contact us on IRC
+(<a href="irc://irc.freenode.net/%23nikita">#nikita on
+irc.freenode.net</a>) or email
+(<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">nikita-noark
+mailing list</a>).</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</description>
</item>
<item>
- <title>Offentlig elektronisk postjournal blokkerer tilgang for utvalgte webklienter</title>
- <link>http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html</guid>
- <pubDate>Thu, 20 Apr 2017 13:00:00 +0200</pubDate>
- <description><p>Jeg oppdaget i dag at <a href="https://www.oep.no/">nettstedet som
-publiserer offentlige postjournaler fra statlige etater</a>, OEP, har
-begynt å blokkerer enkelte typer webklienter fra å få tilgang. Vet
-ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl
-og curl. For å teste selv, kjør følgende:</p>
-
-<blockquote><pre>
-% curl -v -s https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
-< HTTP/1.1 404 Not Found
-% curl -v -s --header 'User-Agent:Opera/12.0' https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
-< HTTP/1.1 200 OK
-%
-</pre></blockquote>
-
-<p>Her kan en se at tjenesten gir «404 Not Found» for curl i
-standardoppsettet, mens den gir «200 OK» hvis curl hevder å være Opera
-versjon 12.0. Offentlig elektronisk postjournal startet blokkeringen
-2017-03-02.</p>
-
-<p>Blokkeringen vil gjøre det litt vanskeligere å maskinelt hente
-informasjon fra oep.no. Kan blokkeringen være gjort for å hindre
-automatisert innsamling av informasjon fra OEP, slik Pressens
-Offentlighetsutvalg gjorde for å dokumentere hvordan departementene
-hindrer innsyn i
-<a href="http://presse.no/dette-mener-np/undergraver-offentlighetsloven/">rapporten
-«Slik hindrer departementer innsyn» som ble publiserte i januar
-2017</a>. Det virker usannsynlig, da det jo er trivielt å bytte
-User-Agent til noe nytt.</p>
-
-<p>Finnes det juridisk grunnlag for det offentlige å diskriminere
-webklienter slik det gjøres her? Der tilgang gis eller ikke alt etter
-hva klienten sier at den heter? Da OEP eies av DIFI og driftes av
-Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to
-aktørene man kan be om innsyn i for å forstå hva som har skjedd. Men
-<a href="https://www.oep.no/search/result.html?period=dateRange&fromDate=01.01.2016&toDate=01.04.2017&dateType=documentDate&caseDescription=&descType=both&caseNumber=&documentNumber=&sender=basefarm&senderType=both&documentType=all&legalAuthority=&archiveCode=&list2=196&searchType=advanced&Search=Search+in+records">postjournalen
-til DIFI viser kun to dokumenter</a> det siste året mellom DIFI og
-Basefarm.
-<a href="https://www.mimesbronn.no/request/blokkering_av_tilgang_til_oep_fo">Mimes brønn neste</a>,
-tenker jeg.</p>
+ <title>Fetching trusted timestamps using the rfc3161ng python module</title>
+ <link>http://people.skolelinux.org/pere/blog/Fetching_trusted_timestamps_using_the_rfc3161ng_python_module.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Fetching_trusted_timestamps_using_the_rfc3161ng_python_module.html</guid>
+ <pubDate>Mon, 8 Oct 2018 12:30:00 +0200</pubDate>
+ <description><p>I have earlier covered the basics of trusted timestamping using the
+'openssl ts' client. See blog post for
+<a href="http://people.skolelinux.org/pere/blog/Public_Trusted_Timestamping_services_for_everyone.html">2014</a>,
+<a href="http://people.skolelinux.org/pere/blog/syslog_trusted_timestamp___chain_of_trusted_timestamps_for_your_syslog.html">2016</a>
+and
+<a href="http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html">2017</a>
+for those stories. But some times I want to integrate the timestamping
+in other code, and recently I needed to integrate it into Python.
+After searching a bit, I found
+<a href="https://dev.entrouvert.org/projects/python-rfc3161">the
+rfc3161 library</a> which seemed like a good fit, but I soon
+discovered it only worked for python version 2, and I needed something
+that work with python version 3. Luckily I next came across
+<a href="https://github.com/trbs/rfc3161ng/">the rfc3161ng library</a>,
+a fork of the original rfc3161 library. Not only is it working with
+python 3, it have fixed a few of the bugs in the original library, and
+it has an active maintainer. I decided to wrap it up and make it
+<a href="https://tracker.debian.org/pkg/python-rfc3161ng">available in
+Debian</a>, and a few days ago it entered Debian unstable and testing.</p>
+
+<p>Using the library is fairly straight forward. The only slightly
+problematic step is to fetch the required certificates to verify the
+timestamp. For some services it is straight forward, while for others
+I have not yet figured out how to do it. Here is a small standalone
+code example based on of the integration tests in the library code:</p>
+
+<pre>
+#!/usr/bin/python3
+
+"""
+
+Python 3 script demonstrating how to use the rfc3161ng module to
+get trusted timestamps.
+
+The license of this code is the same as the license of the rfc3161ng
+library, ie MIT/BSD.
+
+"""
+
+import os
+import pyasn1.codec.der
+import rfc3161ng
+import subprocess
+import tempfile
+import urllib.request
+
+def store(f, data):
+ f.write(data)
+ f.flush()
+ f.seek(0)
+
+def fetch(url, f=None):
+ response = urllib.request.urlopen(url)
+ data = response.read()
+ if f:
+ store(f, data)
+ return data
+
+def main():
+ with tempfile.NamedTemporaryFile() as cert_f,\
+ tempfile.NamedTemporaryFile() as ca_f,\
+ tempfile.NamedTemporaryFile() as msg_f,\
+ tempfile.NamedTemporaryFile() as tsr_f:
+
+ # First fetch certificates used by service
+ certificate_data = fetch('https://freetsa.org/files/tsa.crt', cert_f)
+ ca_data_data = fetch('https://freetsa.org/files/cacert.pem', ca_f)
+
+ # Then timestamp the message
+ timestamper = \
+ rfc3161ng.RemoteTimestamper('http://freetsa.org/tsr',
+ certificate=certificate_data)
+ data = b"Python forever!\n"
+ tsr = timestamper(data=data, return_tsr=True)
+
+ # Finally, convert message and response to something 'openssl ts' can verify
+ store(msg_f, data)
+ store(tsr_f, pyasn1.codec.der.encoder.encode(tsr))
+ args = ["openssl", "ts", "-verify",
+ "-data", msg_f.name,
+ "-in", tsr_f.name,
+ "-CAfile", ca_f.name,
+ "-untrusted", cert_f.name]
+ subprocess.check_call(args)
+
+if '__main__' == __name__:
+ main()
+</pre>
+
+<p>The code fetches the required certificates, store them as temporary
+files, timestamp a simple message, store the message and timestamp to
+disk and ask 'openssl ts' to verify the timestamp. A timestamp is
+around 1.5 kiB in size, and should be fairly easy to store for future
+use.</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</description>
</item>
<item>
- <title>Free software archive system Nikita now able to store documents</title>
- <link>http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html</guid>
- <pubDate>Sun, 19 Mar 2017 08:00:00 +0100</pubDate>
- <description><p>The <a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita
-Noark 5 core project</a> is implementing the Norwegian standard for
-keeping an electronic archive of government documents.
-<a href="http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version">The
-Noark 5 standard</a> document the requirement for data systems used by
-the archives in the Norwegian government, and the Noark 5 web interface
-specification document a REST web service for storing, searching and
-retrieving documents and metadata in such archive. I've been involved
-in the project since a few weeks before Christmas, when the Norwegian
-Unix User Group
-<a href="https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml">announced
-it supported the project</a>. I believe this is an important project,
-and hope it can make it possible for the government archives in the
-future to use free software to keep the archives we citizens depend
-on. But as I do not hold such archive myself, personally my first use
-case is to store and analyse public mail journal metadata published
-from the government. I find it useful to have a clear use case in
-mind when developing, to make sure the system scratches one of my
-itches.</p>
-
-<p>If you would like to help make sure there is a free software
-alternatives for the archives, please join our IRC channel
-(<a href="irc://irc.freenode.net/%23nikita"">#nikita on
-irc.freenode.net</a>) and
-<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">the
-project mailing list</a>.</p>
-
-<p>When I got involved, the web service could store metadata about
-documents. But a few weeks ago, a new milestone was reached when it
-became possible to store full text documents too. Yesterday, I
-completed an implementation of a command line tool
-<tt>archive-pdf</tt> to upload a PDF file to the archive using this
-API. The tool is very simple at the moment, and find existing
-<a href="https://en.wikipedia.org/wiki/Fonds">fonds</a>, series and
-files while asking the user to select which one to use if more than
-one exist. Once a file is identified, the PDF is associated with the
-file and uploaded, using the title extracted from the PDF itself. The
-process is fairly similar to visiting the archive, opening a cabinet,
-locating a file and storing a piece of paper in the archive. Here is
-a test run directly after populating the database with test data using
-our API tester:</p>
+ <title>Automatic Google Drive sync using grive in Debian</title>
+ <link>http://people.skolelinux.org/pere/blog/Automatic_Google_Drive_sync_using_grive_in_Debian.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Automatic_Google_Drive_sync_using_grive_in_Debian.html</guid>
+ <pubDate>Thu, 4 Oct 2018 15:20:00 +0200</pubDate>
+ <description><p>A few days, I rescued a Windows victim over to Debian. To try to
+rescue the remains, I helped set up automatic sync with Google Drive.
+I did not find any sensible Debian package handling this
+automatically, so I rebuild the grive2 source from
+<a href="http://www.webupd8.org/">the Ubuntu UPD8 PPA</a> to do the
+task and added a autostart desktop entry and a small shell script to
+run in the background while the user is logged in to do the sync.
+Here is a sketch of the setup for future reference.</p>
+
+<p>I first created <tt>~/googledrive</tt>, entered the directory and
+ran '<tt>grive -a</tt>' to authenticate the machine/user. Next, I
+created a autostart hook in <tt>~/.config/autostart/grive.desktop</tt>
+to start the sync when the user log in:</p>
+
+<p><blockquote><pre>
+[Desktop Entry]
+Name=Google drive autosync
+Type=Application
+Exec=/home/user/bin/grive-sync
+</pre></blockquote></p>
+
+<p>Finally, I wrote the <tt>~/bin/grive-sync</tt> script to sync
+~/googledrive/ with the files in Google Drive.</p>
<p><blockquote><pre>
-~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
-using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
-using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
-
- 0 - Title of the test case file created 2017-03-18T23:49:32.103446
- 1 - Title of the test file created 2017-03-18T23:49:32.103446
-Select which mappe you want (or search term): 0
-Uploading mangelmelding/mangler.pdf
- PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
- File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
-~/src//noark5-tester$
+#!/bin/sh
+set -e
+cd ~/
+cleanup() {
+ if [ "$syncpid" ] ; then
+ kill $syncpid
+ fi
+}
+trap cleanup EXIT INT QUIT
+/usr/lib/grive/grive-sync.sh listen googledrive 2>&1 | sed "s%^%$0:%" &
+syncpdi=$!
+while true; do
+ if ! xhost >/dev/null 2>&1 ; then
+ echo "no DISPLAY, exiting as the user probably logged out"
+ exit 1
+ fi
+ if [ ! -e /run/user/1000/grive-sync.sh_googledrive ] ; then
+ /usr/lib/grive/grive-sync.sh sync googledrive
+ fi
+ sleep 300
+done 2>&1 | sed "s%^%$0:%"
</pre></blockquote></p>
-<p>You can see here how the fonds (arkiv) and serie (arkivdel) only had
-one option, while the user need to choose which file (mappe) to use
-among the two created by the API tester. The <tt>archive-pdf</tt>
-tool can be found in the git repository for the API tester.</p>
-
-<p>In the project, I have been mostly working on
-<a href="https://github.com/petterreinholdtsen/noark5-tester">the API
-tester</a> so far, while getting to know the code base. The API
-tester currently use
-<a href="https://en.wikipedia.org/wiki/HATEOAS">the HATEOAS links</a>
-to traverse the entire exposed service API and verify that the exposed
-operations and objects match the specification, as well as trying to
-create objects holding metadata and uploading a simple XML file to
-store. The tester has proved very useful for finding flaws in our
-implementation, as well as flaws in the reference site and the
-specification.</p>
-
-<p>The test document I uploaded is a summary of all the specification
-defects we have collected so far while implementing the web service.
-There are several unclear and conflicting parts of the specification,
-and we have
-<a href="https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding">started
-writing down</a> the questions we get from implementing it. We use a
-format inspired by how <a href="http://www.opengroup.org/austin/">The
-Austin Group</a> collect defect reports for the POSIX standard with
-<a href="http://www.opengroup.org/austin/mantis.html">their
-instructions for the MANTIS defect tracker system</a>, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a <a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md">request for a procedure for submitting defect reports</a> :).
-
-<p>The Nikita project is implemented using Java and Spring, and is
-fairly easy to get up and running using Docker containers for those
-that want to test the current code base. The API tester is
-implemented in Python.</p>
+<p>Feel free to use the setup if you want. It can be assumed to be
+GNU GPL v2 licensed (or any later version, at your leisure), but I
+doubt this code is possible to claim copyright on.</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</description>
</item>
<item>
- <title>Detecting NFS hangs on Linux without hanging yourself...</title>
- <link>http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html</guid>
- <pubDate>Thu, 9 Mar 2017 15:20:00 +0100</pubDate>
- <description><p>Over the years, administrating thousand of NFS mounting linux
-computers at the time, I often needed a way to detect if the machine
-was experiencing NFS hang. If you try to use <tt>df</tt> or look at a
-file or directory affected by the hang, the process (and possibly the
-shell) will hang too. So you want to be able to detect this without
-risking the detection process getting stuck too. It has not been
-obvious how to do this. When the hang has lasted a while, it is
-possible to find messages like these in dmesg:</p>
-
-<p><blockquote>
-nfs: server nfsserver not responding, still trying
-<br>nfs: server nfsserver OK
-</blockquote></p>
-
-<p>It is hard to know if the hang is still going on, and it is hard to
-be sure looking in dmesg is going to work. If there are lots of other
-messages in dmesg the lines might have rotated out of site before they
-are noticed.</p>
-
-<p>While reading through the nfs client implementation in linux kernel
-code, I came across some statistics that seem to give a way to detect
-it. The om_timeouts sunrpc value in the kernel will increase every
-time the above log entry is inserted into dmesg. And after digging a
-bit further, I discovered that this value show up in
-/proc/self/mountstats on Linux.</p>
-
-<p>The mountstats content seem to be shared between files using the
-same file system context, so it is enough to check one of the
-mountstats files to get the state of the mount point for the machine.
-I assume this will not show lazy umounted NFS points, nor NFS mount
-points in a different process context (ie with a different filesystem
-view), but that does not worry me.</p>
-
-<p>The content for a NFS mount point look similar to this:</p>
+ <title>Valutakrambod - A python and bitcoin love story</title>
+ <link>http://people.skolelinux.org/pere/blog/Valutakrambod___A_python_and_bitcoin_love_story.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Valutakrambod___A_python_and_bitcoin_love_story.html</guid>
+ <pubDate>Sat, 29 Sep 2018 22:20:00 +0200</pubDate>
+ <description><p>It would come as no surprise to anyone that I am interested in
+bitcoins and virtual currencies. I've been keeping an eye on virtual
+currencies for many years, and it is part of the reason a few months
+ago, I started writing a python library for collecting currency
+exchange rates and trade on virtual currency exchanges. I decided to
+name the end result valutakrambod, which perhaps can be translated to
+small currency shop.</p>
+
+<p>The library uses the tornado python library to handle HTTP and
+websocket connections, and provide a asynchronous system for
+connecting to and tracking several services. The code is available
+from
+<a href="http://github.com/petterreinholdtsen/valutakrambod">github</a>.</p>
+
+</p>There are two example clients of the library. One is very simple and
+list every updated buy/sell price received from the various services.
+This code is started by running bin/btc-rates and call the client code
+in valutakrambod/client.py. The simple client look like this:</p>
<p><blockquote><pre>
-[...]
-device /dev/mapper/Debian-var mounted on /var with fstype ext3
-device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1
- opts: rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all
- age: 7863311
- caps: caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255
- sec: flavor=1,pseudoflavor=1
- events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0
- bytes: 166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809
- RPC iostats version: 1.0 p/v: 100003/3 (nfs)
- xprt: tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820
- per-op statistics
- NULL: 0 0 0 0 0 0 0 0
- GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132
- SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943
- LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511
- ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140
- READLINK: 125 125 0 20472 18620 0 1112 1118
- READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693
- WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771
- CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398
- MKDIR: 3680 3680 0 773980 993920 26 23990 24245
- SYMLINK: 903 903 0 233428 245488 6 5865 5917
- MKNOD: 80 80 0 20148 21760 0 299 304
- REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636
- RMDIR: 3367 3367 0 645112 484848 22 5782 6002
- RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288
- LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579
- READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917
- READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937
- FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022
- FSINFO: 2 2 0 232 328 0 1 1
- PATHCONF: 1 1 0 116 140 0 0 0
- COMMIT: 0 0 0 0 0 0 0 0
-
-device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
-[...]
+import functools
+import tornado.ioloop
+import valutakrambod
+class SimpleClient(object):
+ def __init__(self):
+ self.services = []
+ self.streams = []
+ pass
+ def newdata(self, service, pair, changed):
+ print("%-15s %s-%s: %8.3f %8.3f" % (
+ service.servicename(),
+ pair[0],
+ pair[1],
+ service.rates[pair]['ask'],
+ service.rates[pair]['bid'])
+ )
+ async def refresh(self, service):
+ await service.fetchRates(service.wantedpairs)
+ def run(self):
+ self.ioloop = tornado.ioloop.IOLoop.current()
+ self.services = valutakrambod.service.knownServices()
+ for e in self.services:
+ service = e()
+ service.subscribe(self.newdata)
+ stream = service.websocket()
+ if stream:
+ self.streams.append(stream)
+ else:
+ # Fetch information from non-streaming services immediately
+ self.ioloop.call_later(len(self.services),
+ functools.partial(self.refresh, service))
+ # as well as regularly
+ service.periodicUpdate(60)
+ for stream in self.streams:
+ stream.connect()
+ try:
+ self.ioloop.start()
+ except KeyboardInterrupt:
+ print("Interrupted by keyboard, closing all connections.")
+ pass
+ for stream in self.streams:
+ stream.close()
</pre></blockquote></p>
-<p>The key number to look at is the third number in the per-op list.
-It is the number of NFS timeouts experiences per file system
-operation. Here 22 write timeouts and 5 access timeouts. If these
-numbers are increasing, I believe the machine is experiencing NFS
-hang. Unfortunately the timeout value do not start to increase right
-away. The NFS operations need to time out first, and this can take a
-while. The exact timeout value depend on the setup. For example the
-defaults for TCP and UDP mount points are quite different, and the
-timeout value is affected by the soft, hard, timeo and retrans NFS
-mount options.</p>
-
-<p>The only way I have been able to get working on Debian and RedHat
-Enterprise Linux for getting the timeout count is to peek in /proc/.
-But according to
-<ahref="http://docs.oracle.com/cd/E19253-01/816-4555/netmonitor-12/index.html">Solaris
-10 System Administration Guide: Network Services</a>, the 'nfsstat -c'
-command can be used to get these timeout values. But this do not work
-on Linux, as far as I can tell. I
-<ahref="http://bugs.debian.org/857043">asked Debian about this</a>,
-but have not seen any replies yet.</p>
-
-<p>Is there a better way to figure out if a Linux NFS client is
-experiencing NFS hangs? Is there a way to detect which processes are
-affected? Is there a way to get the NFS mount going quickly once the
-network problem causing the NFS hang has been cleared? I would very
-much welcome some clues, as we regularly run into NFS hangs.</p>
+<p>The library client loops over all known "public" services,
+initialises it, subscribes to any updates from the service, checks and
+activates websocket streaming if the service provide it, and if no
+streaming is supported, fetches information from the service and sets
+up a periodic update every 60 seconds. The output from this client
+can look like this:</p>
+
+<p><blockquote><pre>
+Bl3p BTC-EUR: 5687.110 5653.690
+Bl3p BTC-EUR: 5687.110 5653.690
+Bl3p BTC-EUR: 5687.110 5653.690
+Hitbtc BTC-USD: 6594.560 6593.690
+Hitbtc BTC-USD: 6594.560 6593.690
+Bl3p BTC-EUR: 5687.110 5653.690
+Hitbtc BTC-USD: 6594.570 6593.690
+Bitstamp EUR-USD: 1.159 1.154
+Hitbtc BTC-USD: 6594.570 6593.690
+Hitbtc BTC-USD: 6594.580 6593.690
+Hitbtc BTC-USD: 6594.580 6593.690
+Hitbtc BTC-USD: 6594.580 6593.690
+Bl3p BTC-EUR: 5687.110 5653.690
+Paymium BTC-EUR: 5680.000 5620.240
+</pre></blockquote></p>
+
+<p>The exchange order book is tracked in addition to the best buy/sell
+price, for those that need to know the details.</p>
+
+<p>The other example client is focusing on providing a curses view
+with updated buy/sell prices as soon as they are received from the
+services. This code is located in bin/btc-rates-curses and activated
+by using the '-c' argument. Without the argument the "curses" output
+is printed without using curses, which is useful for debugging. The
+curses view look like this:</p>
+
+<p><blockquote><pre>
+ Name Pair Bid Ask Spr Ftcd Age
+ BitcoinsNorway BTCEUR 5591.8400 5711.0800 2.1% 16 nan 60
+ Bitfinex BTCEUR 5671.0000 5671.2000 0.0% 16 22 59
+ Bitmynt BTCEUR 5580.8000 5807.5200 3.9% 16 41 60
+ Bitpay BTCEUR 5663.2700 nan nan% 15 nan 60
+ Bitstamp BTCEUR 5664.8400 5676.5300 0.2% 0 1 1
+ Bl3p BTCEUR 5653.6900 5684.9400 0.5% 0 nan 19
+ Coinbase BTCEUR 5600.8200 5714.9000 2.0% 15 nan nan
+ Kraken BTCEUR 5670.1000 5670.2000 0.0% 14 17 60
+ Paymium BTCEUR 5620.0600 5680.0000 1.1% 1 7515 nan
+ BitcoinsNorway BTCNOK 52898.9700 54034.6100 2.1% 16 nan 60
+ Bitmynt BTCNOK 52960.3200 54031.1900 2.0% 16 41 60
+ Bitpay BTCNOK 53477.7833 nan nan% 16 nan 60
+ Coinbase BTCNOK 52990.3500 54063.0600 2.0% 15 nan nan
+ MiraiEx BTCNOK 52856.5300 54100.6000 2.3% 16 nan nan
+ BitcoinsNorway BTCUSD 6495.5300 6631.5400 2.1% 16 nan 60
+ Bitfinex BTCUSD 6590.6000 6590.7000 0.0% 16 23 57
+ Bitpay BTCUSD 6564.1300 nan nan% 15 nan 60
+ Bitstamp BTCUSD 6561.1400 6565.6200 0.1% 0 2 1
+ Coinbase BTCUSD 6504.0600 6635.9700 2.0% 14 nan 117
+ Gemini BTCUSD 6567.1300 6573.0700 0.1% 16 89 nan
+ Hitbtc+BTCUSD 6592.6200 6594.2100 0.0% 0 0 0
+ Kraken BTCUSD 6565.2000 6570.9000 0.1% 15 17 58
+ Exchangerates EURNOK 9.4665 9.4665 0.0% 16 107789 nan
+ Norgesbank EURNOK 9.4665 9.4665 0.0% 16 107789 nan
+ Bitstamp EURUSD 1.1537 1.1593 0.5% 4 5 1
+ Exchangerates EURUSD 1.1576 1.1576 0.0% 16 107789 nan
+ BitcoinsNorway LTCEUR 1.0000 49.0000 98.0% 16 nan nan
+ BitcoinsNorway LTCNOK 492.4800 503.7500 2.2% 16 nan 60
+ BitcoinsNorway LTCUSD 1.0221 49.0000 97.9% 15 nan nan
+ Norgesbank USDNOK 8.1777 8.1777 0.0% 16 107789 nan
+</pre></blockquote></p>
+
+<p>The code for this client is too complex for a simple blog post, so
+you will have to check out the git repository to figure out how it
+work. What I can tell is how the three last numbers on each line
+should be interpreted. The first is how many seconds ago information
+was received from the service. The second is how long ago, according
+to the service, the provided information was updated. The last is an
+estimate on how often the buy/sell values change.</p>
+
+<p>If you find this library useful, or would like to improve it, I
+would love to hear from you. Note that for some of the services I've
+implemented a trading API. It might be the topic of a future blog
+post.</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</description>
</item>
<item>
- <title>How does it feel to be wiretapped, when you should be doing the wiretapping...</title>
- <link>http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html</guid>
- <pubDate>Wed, 8 Mar 2017 11:50:00 +0100</pubDate>
- <description><p>So the new president in the United States of America claim to be
-surprised to discover that he was wiretapped during the election
-before he was elected president. He even claim this must be illegal.
-Well, doh, if it is one thing the confirmations from Snowden
-documented, it is that the entire population in USA is wiretapped, one
-way or another. Of course the president candidates were wiretapped,
-alongside the senators, judges and the rest of the people in USA.</p>
-
-<p>Next, the Federal Bureau of Investigation ask the Department of
-Justice to go public rejecting the claims that Donald Trump was
-wiretapped illegally. I fail to see the relevance, given that I am
-sure the surveillance industry in USA believe they have all the legal
-backing they need to conduct mass surveillance on the entire
-world.</p>
-
-<p>There is even the director of the FBI stating that he never saw an
-order requesting wiretapping of Donald Trump. That is not very
-surprising, given how the FISA court work, with all its activity being
-secret. Perhaps he only heard about it?</p>
-
-<p>What I find most sad in this story is how Norwegian journalists
-present it. In a news reports the other day in the radio from the
-Norwegian National broadcasting Company (NRK), I heard the journalist
-claim that 'the FBI denies any wiretapping', while the reality is that
-'the FBI denies any illegal wiretapping'. There is a fundamental and
-important difference, and it make me sad that the journalists are
-unable to grasp it.</p>
-
-<p><strong>Update 2017-03-13:</strong> Look like
-<a href="https://theintercept.com/2017/03/13/rand-paul-is-right-nsa-routinely-monitors-americans-communications-without-warrants/">The
-Intercept report that US Senator Rand Paul confirm what I state above</a>.</p>
+ <title>VLC in Debian now can do bittorrent streaming</title>
+ <link>http://people.skolelinux.org/pere/blog/VLC_in_Debian_now_can_do_bittorrent_streaming.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/VLC_in_Debian_now_can_do_bittorrent_streaming.html</guid>
+ <pubDate>Mon, 24 Sep 2018 21:20:00 +0200</pubDate>
+ <description><p>Back in February, I got curious to see
+<a href="http://people.skolelinux.org/pere/blog/Using_VLC_to_stream_bittorrent_sources.html">if
+VLC now supported Bittorrent streaming</a>. It did not, despite the
+fact that the idea and code to handle such streaming had been floating
+around for years. I did however find
+<a href="https://github.com/johang/vlc-bittorrent">a standalone plugin
+for VLC</a> to do it, and half a year later I decided to wrap up the
+plugin and get it into Debian. I uploaded it to NEW a few days ago,
+and am very happy to report that it
+<a href="https://tracker.debian.org/pkg/vlc-plugin-bittorrent">entered
+Debian</a> a few hours ago, and should be available in Debian/Unstable
+tomorrow, and Debian/Testing in a few days.</p>
+
+<p>With the vlc-plugin-bittorrent package installed you should be able
+to stream videos using a simple call to</p>
+
+<p><blockquote><pre>
+vlc https://archive.org/download/TheGoat/TheGoat_archive.torrent
+</pre></blockquote></p>
+
+</p>It can handle magnet links too. Now if only native vlc had
+bittorrent support. Then a lot more would be helping each other to
+share public domain and creative commons movies. The plugin need some
+stability work with seeking and picking the right file in a torrent
+with many files, but is already usable. Please note that the plugin
+is not removing downloaded files when vlc is stopped, so it can fill
+up your disk if you are not careful. Have fun. :)</p>
+
+<p>I would love to get help maintaining this package. Get in touch if
+you are interested.</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</description>
</item>
<item>
- <title>Norwegian Bokmål translation of The Debian Administrator's Handbook complete, proofreading in progress</title>
- <link>http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Norwegian_Bokm_l_translation_of_The_Debian_Administrator_s_Handbook_complete__proofreading_in_progress.html</guid>
- <pubDate>Fri, 3 Mar 2017 14:50:00 +0100</pubDate>
- <description><p>For almost a year now, we have been working on making a Norwegian
-Bokmål edition of <a href="https://debian-handbook.info/">The Debian
-Administrator's Handbook</a>. Now, thanks to the tireless effort of
-Ole-Erik, Ingrid and Andreas, the initial translation is complete, and
-we are working on the proof reading to ensure consistent language and
-use of correct computer science terms. The plan is to make the book
-available on paper, as well as in electronic form. For that to
-happen, the proof reading must be completed and all the figures need
-to be translated. If you want to help out, get in touch.</p>
-
-<p><a href="http://people.skolelinux.org/pere/debian-handbook/debian-handbook-nb-NO.pdf">A
-
-fresh PDF edition</a> in A4 format (the final book will have smaller
-pages) of the book created every morning is available for
-proofreading. If you find any errors, please
-<a href="https://hosted.weblate.org/projects/debian-handbook/">visit
-Weblate and correct the error</a>. The
-<a href="http://l.github.io/debian-handbook/stat/nb-NO/index.html">state
-of the translation including figures</a> is a useful source for those
-provide Norwegian bokmål screen shots and figures.</p>
+ <title>Using the Kodi API to play Youtube videos</title>
+ <link>http://people.skolelinux.org/pere/blog/Using_the_Kodi_API_to_play_Youtube_videos.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Using_the_Kodi_API_to_play_Youtube_videos.html</guid>
+ <pubDate>Sun, 2 Sep 2018 23:40:00 +0200</pubDate>
+ <description><p>I continue to explore my Kodi installation, and today I wanted to
+tell it to play a youtube URL I received in a chat, without having to
+insert search terms using the on-screen keyboard. After searching the
+web for API access to the Youtube plugin and testing a bit, I managed
+to find a recipe that worked. If you got a kodi instance with its API
+available from http://kodihost/jsonrpc, you can try the following to
+have check out a nice cover band.</p>
+
+<p><blockquote><pre>curl --silent --header 'Content-Type: application/json' \
+ --data-binary '{ "id": 1, "jsonrpc": "2.0", "method": "Player.Open",
+ "params": {"item": { "file":
+ "plugin://plugin.video.youtube/play/?video_id=LuRGVM9O0qg" } } }' \
+ http://projector.local/jsonrpc</pre></blockquote></p>
+
+<p>I've extended kodi-stream program to take a video source as its
+first argument. It can now handle direct video links, youtube links
+and 'desktop' to stream my desktop to Kodi. It is almost like a
+Chromecast. :)</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</description>
</item>
<item>
- <title>Unlimited randomness with the ChaosKey?</title>
- <link>http://people.skolelinux.org/pere/blog/Unlimited_randomness_with_the_ChaosKey_.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Unlimited_randomness_with_the_ChaosKey_.html</guid>
- <pubDate>Wed, 1 Mar 2017 20:50:00 +0100</pubDate>
- <description><p>A few days ago I ordered a small batch of
-<a href="http://altusmetrum.org/ChaosKey/">the ChaosKey</a>, a small
-USB dongle for generating entropy created by Bdale Garbee and Keith
-Packard. Yesterday it arrived, and I am very happy to report that it
-work great! According to its designers, to get it to work out of the
-box, you need the Linux kernel version 4.1 or later. I tested on a
-Debian Stretch machine (kernel version 4.9), and there it worked just
-fine, increasing the available entropy very quickly. I wrote a small
-test oneliner to test. It first print the current entropy level,
-drain /dev/random, and then print the entropy level for five seconds.
-Here is the situation without the ChaosKey inserted:</p>
-
-<blockquote><pre>
-% cat /proc/sys/kernel/random/entropy_avail; \
- dd bs=1M if=/dev/random of=/dev/null count=1; \
- for n in $(seq 1 5); do \
- cat /proc/sys/kernel/random/entropy_avail; \
- sleep 1; \
- done
-300
-0+1 oppføringer inn
-0+1 oppføringer ut
-28 byte kopiert, 0,000264565 s, 106 kB/s
-4
-8
-12
-17
-21
-%
-</pre></blockquote>
-
-<p>The entropy level increases by 3-4 every second. In such case any
-application requiring random bits (like a HTTPS enabled web server)
-will halt and wait for more entrpy. And here is the situation with
-the ChaosKey inserted:</p>
-
-<blockquote><pre>
-% cat /proc/sys/kernel/random/entropy_avail; \
- dd bs=1M if=/dev/random of=/dev/null count=1; \
- for n in $(seq 1 5); do \
- cat /proc/sys/kernel/random/entropy_avail; \
- sleep 1; \
- done
-1079
-0+1 oppføringer inn
-0+1 oppføringer ut
-104 byte kopiert, 0,000487647 s, 213 kB/s
-433
-1028
-1031
-1035
-1038
-%
-</pre></blockquote>
-
-<p>Quite the difference. :) I bought a few more than I need, in case
-someone want to buy one here in Norway. :)</p>
-
-<p>Update: The dongle was presented at Debconf last year. You might
-find <a href="https://debconf16.debconf.org/talks/94/">the talk
-recording illuminating</a>. It explains exactly what the source of
-randomness is, if you are unable to spot it from the schema drawing
-available from the ChaosKey web site linked at the start of this blog
-post.</p>
+ <title>Software created using taxpayers’ money should be Free Software</title>
+ <link>http://people.skolelinux.org/pere/blog/Software_created_using_taxpayers__money_should_be_Free_Software.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Software_created_using_taxpayers__money_should_be_Free_Software.html</guid>
+ <pubDate>Thu, 30 Aug 2018 13:50:00 +0200</pubDate>
+ <description><p>It might seem obvious that software created using tax money should
+be available for everyone to use and improve. Free Software
+Foundation Europe recentlystarted a campaign to help get more people
+to understand this, and I just signed the petition on
+<a href="https://publiccode.eu/">Public Money, Public Code</a> to help
+them. I hope you too will do the same.</p>
</description>
</item>
projects / homepage.git / blobdiff