<div class="entry">
- <div class="title"><a href="http://people.skolelinux.org/pere/blog/_Rapporten_ser_ikke_p__informasjonssikkerhet_knyttet_til_personlig_integritet_.html">«Rapporten ser ikke på informasjonssikkerhet knyttet til personlig integritet»</a></div>
- <div class="date">27th June 2017</div>
- <div class="body"><p>Jeg kom over teksten
-«<a href="https://freedom-to-tinker.com/2017/06/21/killing-car-privacy-by-federal-mandate/">Killing
-car privacy by federal mandate</a>» av Leonid Reyzin på Freedom to
-Tinker i dag, og det gleder meg å se en god gjennomgang om hvorfor det
-er et urimelig inngrep i privatsfæren å la alle biler kringkaste sin
-posisjon og bevegelse via radio. Det omtalte forslaget basert på
-Dedicated Short Range Communication (DSRC) kalles Basic Safety Message
-(BSM) i USA og Cooperative Awareness Message (CAM) i Europa, og det
-norske Vegvesenet er en av de som ser ut til å kunne tenke seg å
-pålegge alle biler å fjerne nok en bit av innbyggernes privatsfære.
-Anbefaler alle å lese det som står der.
-
-<p>Mens jeg tittet litt på DSRC på biler i Norge kom jeg over et sitat
-jeg synes er illustrativt for hvordan det offentlige Norge håndterer
-problemstillinger rundt innbyggernes privatsfære i SINTEF-rapporten
-«<a href="https://www.sintef.no/publikasjoner/publikasjon/Download/?pubid=SINTEF+A23933">Informasjonssikkerhet
-i AutoPASS-brikker</a>» av Trond Foss:</p>
+ <div class="title"><a href="http://people.skolelinux.org/pere/blog/Idea_for_finding_all_public_domain_movies_in_the_USA.html">Idea for finding all public domain movies in the USA</a></div>
+ <div class="date">13th December 2017</div>
+ <div class="body"><p>While looking at
+<a href="http://onlinebooks.library.upenn.edu/cce/">the scanned copies
+for the copyright renewal entries for movies published in the USA</a>,
+an idea occurred to me. The number of renewals are so few per year, it
+should be fairly quick to transcribe them all and add references to
+the corresponding IMDB title ID. This would give the (presumably)
+complete list of movies published 28 years earlier that did _not_
+enter the public domain for the transcribed year. By fetching the
+list of USA movies published 28 years earlier and subtract the movies
+with renewals, we should be left with movies registered in IMDB that
+are now in the public domain. For the year 1955 (which is the one I
+have looked at the most), the total number of pages to transcribe is
+21. For the 28 years from 1950 to 1978, it should be in the range
+500-600 pages. It is just a few days of work, and spread among a
+small group of people it should be doable in a few weeks of spare
+time.</p>
+
+<p>A typical copyright renewal entry look like this (the first one
+listed for 1955):</p>
<p><blockquote>
-«Rapporten ser ikke på informasjonssikkerhet knyttet til personlig
- integritet.»
+ ADAM AND EVIL, a photoplay in seven reels by Metro-Goldwyn-Mayer
+ Distribution Corp. (c) 17Aug27; L24293. Loew's Incorporated (PWH);
+ 10Jun55; R151558.
</blockquote></p>
-<p>Så enkelt kan det tydeligvis gjøres når en vurderer
-informasjonssikkerheten. Det holder vel at folkene på toppen kan si
-at «Personvernet er ivaretatt», som jo er den populære intetsigende
-frasen som gjør at mange tror enkeltindividers integritet tas vare på.
-Sitatet fikk meg til å undres på hvor ofte samme tilnærming, å bare se
-bort fra behovet for personlig itegritet, blir valgt når en velger å
-legge til rette for nok et inngrep i privatsfæren til personer i
-Norge. Det er jo sjelden det får reaksjoner. Historien om
-reaksjonene på Helse Sør-Østs tjenesteutsetting er jo sørgelig nok et
-unntak og toppen av isfjellet, desverre. Tror jeg fortsatt takker nei
-til både AutoPASS og holder meg så langt unna det norske helsevesenet
-som jeg kan, inntil de har demonstrert og dokumentert at de verdsetter
-individets privatsfære og personlige integritet høyere enn kortsiktig
-gevist og samfunnsnytte.</p>
+<p>The movie title as well as registration and renewal dates are easy
+enough to locate by a program (split on first comma and look for
+DDmmmYY). The rest of the text is not required to find the movie in
+IMDB, but is useful to confirm the correct movie is found. I am not
+quite sure what the L and R numbers mean, but suspect they are
+reference numbers into the archive of the US Copyright Office.</p>
+
+<p>Tracking down the equivalent IMDB title ID is probably going to be
+a manual task, but given the year it is fairly easy to search for the
+movie title using for example
+<a href="http://www.imdb.com/find?q=adam+and+evil+1927&s=all">http://www.imdb.com/find?q=adam+and+evil+1927&s=all</a>.
+Using this search, I find that the equivalent IMDB title ID for the
+first renewal entry from 1955 is
+<a href="http://www.imdb.com/title/tt0017588/">http://www.imdb.com/title/tt0017588/</a>.</p>
+
+<p>I suspect the best way to do this would be to make a specialised
+web service to make it easy for contributors to transcribe and track
+down IMDB title IDs. In the web service, once a entry is transcribed,
+the title and year could be extracted from the text, a search in IMDB
+conducted for the user to pick the equivalent IMDB title ID right
+away. By spreading out the work among volunteers, it would also be
+possible to make at least two persons transcribe the same entries to
+be able to discover any typos introduced. But I will need help to
+make this happen, as I lack the spare time to do all of this on my
+own. If you would like to help, please get in touch. Perhaps you can
+draft a web service for crowd sourcing the task?</p>
+
+<p>Note, Project Gutenberg already have some
+<a href="http://www.gutenberg.org/ebooks/search/?query=copyright+office+renewals">transcribed
+copies of the US Copyright Office renewal protocols</a>, but I have
+not been able to find any film renewals there, so I suspect they only
+have copies of renewal for written works. I have not been able to find
+any transcribed versions of movie renewals so far. Perhaps they exist
+somewhere?</p>
+
+<p>I would love to figure out methods for finding all the public
+domain works in other countries too, but it is a lot harder. At least
+for Norway and Great Britain, such work involve tracking down the
+people involved in making the movie and figuring out when they died.
+It is hard enough to figure out who was part of making a movie, but I
+do not know how to automate such procedure without a registry of every
+person involved in making movies and their death year.</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</div>
<div class="tags">
- Tags: <a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk</a>, <a href="http://people.skolelinux.org/pere/blog/tags/personvern">personvern</a>, <a href="http://people.skolelinux.org/pere/blog/tags/sikkerhet">sikkerhet</a>.
+ Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett</a>, <a href="http://people.skolelinux.org/pere/blog/tags/verkidetfri">verkidetfri</a>.
</div>
<div class="padding"></div>
<div class="entry">
- <div class="title"><a href="http://people.skolelinux.org/pere/blog/Updated_sales_number_for_my_Free_Culture_paper_editions.html">Updated sales number for my Free Culture paper editions</a></div>
- <div class="date">12th June 2017</div>
- <div class="body"><p>It is pleasing to see that the work we put down in publishing new
-editions of the classic <a href="http://www.free-culture.cc/">Free
-Culture book</a> by the founder of the Creative Commons movement,
-Lawrence Lessig, is still being appreciated. I had a look at the
-latest sales numbers for the paper edition today. Not too impressive,
-but happy to see some buyers still exist. All the revenue from the
-books is sent to the <a href="https://creativecommons.org/">Creative
-Commons Corporation</a>, and they receive the largest cut if you buy
-directly from Lulu. Most books are sold via Amazon, with Ingram
-second and only a small fraction directly from Lulu. The ebook
-edition is available for free from
-<a href="https://github.com/petterreinholdtsen/free-culture-lessig">Github</a>.</p>
-
-<table border="0">
-<tr><th rowspan="2" valign="bottom">Title / language</th><th colspan="3">Quantity</th></tr>
-<tr><th>2016 jan-jun</th><th>2016 jul-dec</th><th>2017 jan-may</th></tr>
-
-<tr>
- <td><a href="http://www.lulu.com/shop/lawrence-lessig/culture-libre/paperback/product-22645082.html">Culture Libre / French</a></td>
- <td align="right">3</td>
- <td align="right">6</td>
- <td align="right">15</td>
-</tr>
-
-<tr>
- <td><a href="http://www.lulu.com/shop/lawrence-lessig/fri-kultur/paperback/product-22441576.html">Fri kultur / Norwegian</a></td>
- <td align="right">7</td>
- <td align="right">1</td>
- <td align="right">0</td>
-</tr>
-
-<tr>
- <td><a href="http://www.lulu.com/shop/lawrence-lessig/free-culture/paperback/product-22440520.html">Free Culture / English</a></td>
- <td align="right">14</td>
- <td align="right">27</td>
- <td align="right">16</td>
-</tr>
-
-<tr>
- <td>Total</td>
- <td align="right">24</td>
- <td align="right">34</td>
- <td align="right">31</td>
-</tr>
-
-</table>
-
-<p>A bit sad to see the low sales number on the Norwegian edition, and
-a bit surprising the English edition still selling so well.</p>
-
-<p>If you would like to translate and publish the book in your native
-language, I would be happy to help make it happen. Please get in
-touch.</p>
+ <div class="title"><a href="http://people.skolelinux.org/pere/blog/Is_the_short_movie__Empty_Socks__from_1927_in_the_public_domain_or_not_.html">Is the short movie «Empty Socks» from 1927 in the public domain or not?</a></div>
+ <div class="date"> 5th December 2017</div>
+ <div class="body"><p>Three years ago, a presumed lost animation film,
+<a href="https://en.wikipedia.org/wiki/Empty_Socks">Empty Socks from
+1927</a>, was discovered in the Norwegian National Library. At the
+time it was discovered, it was generally assumed to be copyrighted by
+The Walt Disney Company, and I blogged about
+<a href="http://people.skolelinux.org/pere/blog/Opphavsretts_status_for__Empty_Socks__fra_1927_.html">my
+reasoning to conclude</a> that it would would enter the Norwegian
+equivalent of the public domain in 2053, based on my understanding of
+Norwegian Copyright Law. But a few days ago, I came across
+<a href="http://www.toonzone.net/forums/threads/exposed-disneys-repurchase-of-oswald-the-rabbit-a-sham.4792291/">a
+blog post claiming the movie was already in the public domain</a>, at
+least in USA. The reasoning is as follows: The film was released in
+November or Desember 1927 (sources disagree), and presumably
+registered its copyright that year. At that time, right holders of
+movies registered by the copyright office received government
+protection for there work for 28 years. After 28 years, the copyright
+had to be renewed if the wanted the government to protect it further.
+The blog post I found claim such renewal did not happen for this
+movie, and thus it entered the public domain in 1956. Yet someone
+claim the copyright was renewed and the movie is still copyright
+protected. Can anyone help me to figure out which claim is correct?
+I have not been able to find Empty Socks in Catalog of copyright
+entries. Ser.3 pt.12-13 v.9-12 1955-1958 Motion Pictures
+<a href="http://onlinebooks.library.upenn.edu/cce/1955r.html#film">available
+from the University of Pennsylvania</a>, neither in
+<a href="https://babel.hathitrust.org/cgi/pt?id=mdp.39015084451130;page=root;view=image;size=100;seq=83;num=45">page
+45 for the first half of 1955</a>, nor in
+<a href="https://babel.hathitrust.org/cgi/pt?id=mdp.39015084451130;page=root;view=image;size=100;seq=175;num=119">page
+119 for the second half of 1955</a>. It is of course possible that
+the renewal entry was left out of the printed catalog by mistake. Is
+there some way to rule out this possibility? Please help, and update
+the wikipedia page with your findings.
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</div>
<div class="tags">
- Tags: <a href="http://people.skolelinux.org/pere/blog/tags/docbook">docbook</a>, <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/freeculture">freeculture</a>.
+ Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/freeculture">freeculture</a>, <a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett</a>, <a href="http://people.skolelinux.org/pere/blog/tags/verkidetfri">verkidetfri</a>, <a href="http://people.skolelinux.org/pere/blog/tags/video">video</a>.
</div>
<div class="padding"></div>
<div class="entry">
- <div class="title"><a href="http://people.skolelinux.org/pere/blog/Release_0_1_1_of_free_software_archive_system_Nikita_announced.html">Release 0.1.1 of free software archive system Nikita announced</a></div>
- <div class="date">10th June 2017</div>
- <div class="body"><p>I am very happy to report that the
-<a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita Noark 5
-core project</a> tagged its second release today. The free software
-solution is an implementation of the Norwegian archive standard Noark
-5 used by government offices in Norway. These were the changes in
-version 0.1.1 since version 0.1.0 (from NEWS.md):
-
-<ul>
-
- <li>Continued work on the angularjs GUI, including document upload.</li>
- <li>Implemented correspondencepartPerson, correspondencepartUnit and
- correspondencepartInternal</li>
- <li>Applied for coverity coverage and started submitting code on
- regualr basis.</li>
- <li>Started fixing bugs reported by coverity</li>
- <li>Corrected and completed HATEOAS links to make sure entire API is
- available via URLs in _links.</li>
- <li>Corrected all relation URLs to use trailing slash.</li>
- <li>Add initial support for storing data in ElasticSearch.</li>
- <li>Now able to receive and store uploaded files in the archive.</li>
- <li>Changed JSON output for object lists to have relations in _links.</li>
- <li>Improve JSON output for empty object lists.</li>
- <li>Now uses correct MIME type application/vnd.noark5-v4+json.</li>
- <li>Added support for docker container images.</li>
- <li>Added simple API browser implemented in JavaScript/Angular.</li>
- <li>Started on archive client implemented in JavaScript/Angular.</li>
- <li>Started on prototype to show the public mail journal.</li>
- <li>Improved performance by disabling Sprint FileWatcher.</li>
- <li>Added support for 'arkivskaper', 'saksmappe' and 'journalpost'.</li>
- <li>Added support for some metadata codelists.</li>
- <li>Added support for Cross-origin resource sharing (CORS).</li>
- <li>Changed login method from Basic Auth to JSON Web Token (RFC 7519)
- style.</li>
- <li>Added support for GET-ing ny-* URLs.</li>
- <li>Added support for modifying entities using PUT and eTag.</li>
- <li>Added support for returning XML output on request.</li>
- <li>Removed support for English field and class names, limiting ourself
- to the official names.</li>
- <li>...</li>
-
-</ul>
-
-<p>If this sound interesting to you, please contact us on IRC (#nikita
-on irc.freenode.net) or email
-(<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">nikita-noark
-mailing list).</p>
+ <div class="title"><a href="http://people.skolelinux.org/pere/blog/Metadata_proposal_for_movies_on_the_Internet_Archive.html">Metadata proposal for movies on the Internet Archive</a></div>
+ <div class="date">28th November 2017</div>
+ <div class="body"><p>It would be easier to locate the movie you want to watch in
+<a href="https://www.archive.org/">the Internet Archive</a>, if the
+metadata about each movie was more complete and accurate. In the
+archiving community, a well known saying state that good metadata is a
+love letter to the future. The metadata in the Internet Archive could
+use a face lift for the future to love us back. Here is a proposal
+for a small improvement that would make the metadata more useful
+today. I've been unable to find any document describing the various
+standard fields available when uploading videos to the archive, so
+this proposal is based on my best quess and searching through several
+of the existing movies.</p>
+
+<p>I have a few use cases in mind. First of all, I would like to be
+able to count the number of distinct movies in the Internet Archive,
+without duplicates. I would further like to identify the IMDB title
+ID of the movies in the Internet Archive, to be able to look up a IMDB
+title ID and know if I can fetch the video from there and share it
+with my friends.</p>
+
+<p>Second, I would like the Butter data provider for The Internet
+archive
+(<a href="https://github.com/butterproviders/butter-provider-archive">available
+from github</a>), to list as many of the good movies as possible. The
+plugin currently do a search in the archive with the following
+parameters:</p>
+
+<p><pre>
+collection:moviesandfilms
+AND NOT collection:movie_trailers
+AND -mediatype:collection
+AND format:"Archive BitTorrent"
+AND year
+</pre></p>
+
+<p>Most of the cool movies that fail to show up in Butter do so
+because the 'year' field is missing. The 'year' field is populated by
+the year part from the 'date' field, and should be when the movie was
+released (date or year). Two such examples are
+<a href="https://archive.org/details/SidneyOlcottsBen-hur1905">Ben Hur
+from 1905</a> and
+<a href="https://archive.org/details/Caminandes2GranDillama">Caminandes
+2: Gran Dillama from 2013</a>, where the year metadata field is
+missing.</p>
+
+So, my proposal is simply, for every movie in The Internet Archive
+where an IMDB title ID exist, please fill in these metadata fields
+(note, they can be updated also long after the video was uploaded, but
+as far as I can tell, only by the uploader):
+
+<dl>
+
+<dt>mediatype</dt>
+<dd>Should be 'movie' for movies.</dd>
+
+<dt>collection</dt>
+<dd>Should contain 'moviesandfilms'.</dd>
+
+<dt>title</dt>
+<dd>The title of the movie, without the publication year.</dd>
+
+<dt>date</dt>
+<dd>The data or year the movie was released. This make the movie show
+up in Butter, as well as make it possible to know the age of the
+movie and is useful to figure out copyright status.</dd>
+
+<dt>director</dt>
+<dd>The director of the movie. This make it easier to know if the
+correct movie is found in movie databases.</dd>
+
+<dt>publisher</dt>
+<dd>The production company making the movie. Also useful for
+identifying the correct movie.</dd>
+
+<dt>links</dt>
+
+<dd>Add a link to the IMDB title page, for example like this: <a
+href="http://www.imdb.com/title/tt0028496/">Movie in
+IMDB</a>. This make it easier to find duplicates and allow for
+counting of number of unique movies in the Archive. Other external
+references, like to TMDB, could be added like this too.</dd>
+
+</dl>
+
+<p>I did consider proposing a Custom field for the IMDB title ID (for
+example 'imdb_title_url', 'imdb_code' or simply 'imdb', but suspect it
+will be easier to simply place it in the links free text field.</p>
+
+<p>I created
+<a href="https://github.com/petterreinholdtsen/public-domain-free-imdb">a
+list of IMDB title IDs for several thousand movies in the Internet
+Archive</a>, but I also got a list of several thousand movies without
+such IMDB title ID (and quite a few duplicates). It would be great if
+this data set could be integrated into the Internet Archive metadata
+to be available for everyone in the future, but with the current
+policy of leaving metadata editing to the uploaders, it will take a
+while before this happen. If you have uploaded movies into the
+Internet Archive, you can help. Please consider following my proposal
+above for your movies, to ensure that movie is properly
+counted. :)</p>
+
+<p>The list is mostly generated using wikidata, which based on
+Wikipedia articles make it possible to link between IMDB and movies in
+the Internet Archive. But there are lots of movies without a
+Wikipedia article, and some movies where only a collection page exist
+(like for <a href="https://en.wikipedia.org/wiki/Caminandes">the
+Caminandes example above</a>, where there are three movies but only
+one Wikidata entry).</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</div>
<div class="tags">
- Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/nuug">nuug</a>, <a href="http://people.skolelinux.org/pere/blog/tags/offentlig innsyn">offentlig innsyn</a>, <a href="http://people.skolelinux.org/pere/blog/tags/standard">standard</a>.
+ Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett</a>, <a href="http://people.skolelinux.org/pere/blog/tags/verkidetfri">verkidetfri</a>.
</div>
<div class="padding"></div>
<div class="entry">
- <div class="title"><a href="http://people.skolelinux.org/pere/blog/Idea_for_storing_trusted_timestamps_in_a_Noark_5_archive.html">Idea for storing trusted timestamps in a Noark 5 archive</a></div>
- <div class="date"> 7th June 2017</div>
- <div class="body"><p><em>This is a copy of
-<a href="https://lists.nuug.no/pipermail/nikita-noark/2017-June/000297.html">an
-email I posted to the nikita-noark mailing list</a>. Please follow up
-there if you would like to discuss this topic. The background is that
-we are making a free software archive system based on the Norwegian
-<a href="https://www.arkivverket.no/forvaltning-og-utvikling/regelverk-og-standarder/noark-standarden">Noark
-5 standard</a> for government archives.</em></p>
-
-<p>I've been wondering a bit lately how trusted timestamps could be
-stored in Noark 5.
-<a href="https://en.wikipedia.org/wiki/Trusted_timestamping">Trusted
-timestamps</a> can be used to verify that some information
-(document/file/checksum/metadata) have not been changed since a
-specific time in the past. This is useful to verify the integrity of
-the documents in the archive.</p>
-
-<p>Then it occured to me, perhaps the trusted timestamps could be
-stored as dokument variants (ie dokumentobjekt referered to from
-dokumentbeskrivelse) with the filename set to the hash it is
-stamping?</p>
-
-<p>Given a "dokumentbeskrivelse" with an associated "dokumentobjekt",
-a new dokumentobjekt is associated with "dokumentbeskrivelse" with the
-same attributes as the stamped dokumentobjekt except these
-attributes:</p>
-
-<ul>
-
-<li>format -> "RFC3161"
-<li>mimeType -> "application/timestamp-reply"
-<li>formatDetaljer -> "<source URL for timestamp service>"
-<li>filenavn -> "<sjekksum>.tsr"
-
-</ul>
-
-<p>This assume a service following
-<a href="https://tools.ietf.org/html/rfc3161">IETF RFC 3161</a> is
-used, which specifiy the given MIME type for replies and the .tsr file
-ending for the content of such trusted timestamp. As far as I can
-tell from the Noark 5 specifications, it is OK to have several
-variants/renderings of a dokument attached to a given
-dokumentbeskrivelse objekt. It might be stretching it a bit to make
-some of these variants represent crypto-signatures useful for
-verifying the document integrity instead of representing the dokument
-itself.</p>
-
-<p>Using the source of the service in formatDetaljer allow several
-timestamping services to be used. This is useful to spread the risk
-of key compromise over several organisations. It would only be a
-problem to trust the timestamps if all of the organisations are
-compromised.</p>
-
-<p>The following oneliner on Linux can be used to generate the tsr
-file. $input is the path to the file to checksum, and $sha256 is the
-SHA-256 checksum of the file (ie the "<sjekksum>.tsr" value mentioned
-above).</p>
-
-<p><blockquote><pre>
-openssl ts -query -data "$inputfile" -cert -sha256 -no_nonce \
- | curl -s -H "Content-Type: application/timestamp-query" \
- --data-binary "@-" http://zeitstempel.dfn.de > $sha256.tsr
-</pre></blockquote></p>
-
-<p>To verify the timestamp, you first need to download the public key
-of the trusted timestamp service, for example using this command:</p>
-
-<p><blockquote><pre>
-wget -O ca-cert.txt \
- https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt
-</pre></blockquote></p>
-
-<p>Note, the public key should be stored alongside the timestamps in
-the archive to make sure it is also available 100 years from now. It
-is probably a good idea to standardise how and were to store such
-public keys, to make it easier to find for those trying to verify
-documents 100 or 1000 years from now. :)</p>
-
-<p>The verification itself is a simple openssl command:</p>
-
-<p><blockquote><pre>
-openssl ts -verify -data $inputfile -in $sha256.tsr \
- -CAfile ca-cert.txt -text
-</pre></blockquote></p>
-
-<p>Is there any reason this approach would not work? Is it somehow against
-the Noark 5 specification?</p>
+ <div class="title"><a href="http://people.skolelinux.org/pere/blog/Legal_to_share_more_than_3000_movies_listed_on_IMDB_.html">Legal to share more than 3000 movies listed on IMDB?</a></div>
+ <div class="date">18th November 2017</div>
+ <div class="body"><p>A month ago, I blogged about my work to
+<a href="http://people.skolelinux.org/pere/blog/Locating_IMDB_IDs_of_movies_in_the_Internet_Archive_using_Wikidata.html">automatically
+check the copyright status of IMDB entries</a>, and try to count the
+number of movies listed in IMDB that is legal to distribute on the
+Internet. I have continued to look for good data sources, and
+identified a few more. The code used to extract information from
+various data sources is available in
+<a href="https://github.com/petterreinholdtsen/public-domain-free-imdb">a
+git repository</a>, currently available from github.</p>
+
+<p>So far I have identified 3186 unique IMDB title IDs. To gain
+better understanding of the structure of the data set, I created a
+histogram of the year associated with each movie (typically release
+year). It is interesting to notice where the peaks and dips in the
+graph are located. I wonder why they are placed there. I suspect
+World War II caused the dip around 1940, but what caused the peak
+around 2010?</p>
+
+<p align="center"><img src="http://people.skolelinux.org/pere/blog/images/2017-11-18-verk-i-det-fri-filmer.png" /></p>
+
+<p>I've so far identified ten sources for IMDB title IDs for movies in
+the public domain or with a free license. This is the statistics
+reported when running 'make stats' in the git repository:</p>
+
+<pre>
+ 249 entries ( 6 unique) with and 288 without IMDB title ID in free-movies-archive-org-butter.json
+ 2301 entries ( 540 unique) with and 0 without IMDB title ID in free-movies-archive-org-wikidata.json
+ 830 entries ( 29 unique) with and 0 without IMDB title ID in free-movies-icheckmovies-archive-mochard.json
+ 2109 entries ( 377 unique) with and 0 without IMDB title ID in free-movies-imdb-pd.json
+ 291 entries ( 122 unique) with and 0 without IMDB title ID in free-movies-letterboxd-pd.json
+ 144 entries ( 135 unique) with and 0 without IMDB title ID in free-movies-manual.json
+ 350 entries ( 1 unique) with and 801 without IMDB title ID in free-movies-publicdomainmovies.json
+ 4 entries ( 0 unique) with and 124 without IMDB title ID in free-movies-publicdomainreview.json
+ 698 entries ( 119 unique) with and 118 without IMDB title ID in free-movies-publicdomaintorrents.json
+ 8 entries ( 8 unique) with and 196 without IMDB title ID in free-movies-vodo.json
+ 3186 unique IMDB title IDs in total
+</pre>
+
+<p>The entries without IMDB title ID are candidates to increase the
+data set, but might equally well be duplicates of entries already
+listed with IMDB title ID in one of the other sources, or represent
+movies that lack a IMDB title ID. I've seen examples of all these
+situations when peeking at the entries without IMDB title ID. Based
+on these data sources, the lower bound for movies listed in IMDB that
+are legal to distribute on the Internet is between 3186 and 4713.
+
+<p>It would be great for improving the accuracy of this measurement,
+if the various sources added IMDB title ID to their metadata. I have
+tried to reach the people behind the various sources to ask if they
+are interested in doing this, without any replies so far. Perhaps you
+can help me get in touch with the people behind VODO, Public Domain
+Torrents, Public Domain Movies and Public Domain Review to try to
+convince them to add more metadata to their movie entries?</p>
+
+<p>Another way you could help is by adding pages to Wikipedia about
+movies that are legal to distribute on the Internet. If such page
+exist and include a link to both IMDB and The Internet Archive, the
+script used to generate free-movies-archive-org-wikidata.json should
+pick up the mapping as soon as wikidata is updates.</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</div>
<div class="tags">
- Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/offentlig innsyn">offentlig innsyn</a>, <a href="http://people.skolelinux.org/pere/blog/tags/standard">standard</a>.
+ Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett</a>, <a href="http://people.skolelinux.org/pere/blog/tags/verkidetfri">verkidetfri</a>.
</div>
<div class="padding"></div>
<div class="entry">
- <div class="title"><a href="http://people.skolelinux.org/pere/blog/N_r_nynorskoversettelsen_svikter_til_eksamen___.html">Når nynorskoversettelsen svikter til eksamen...</a></div>
- <div class="date"> 3rd June 2017</div>
- <div class="body"><p><a href="http://www.aftenposten.no/norge/Krever-at-elever-ma-fa-annullert-eksamen-etter-rot-med-oppgavetekster-622459b.html">Aftenposten
-melder i dag</a> om feil i eksamensoppgavene for eksamen i politikk og
-menneskerettigheter, der teksten i bokmåls og nynorskutgaven ikke var
-like. Oppgaveteksten er gjengitt i artikkelen, og jeg ble nysgjerring
-på om den fri oversetterløsningen
-<a href="https://www.apertium.org/">Apertium</a> ville gjort en bedre
-jobb enn Utdanningsdirektoratet. Det kan se slik ut.</p>
-
-<p>Her er bokmålsoppgaven fra eksamenen:</p>
-
-<blockquote>
-<p>Drøft utfordringene knyttet til nasjonalstatenes og andre aktørers
-rolle og muligheter til å håndtere internasjonale utfordringer, som
-for eksempel flykningekrisen.</p>
-
-<p>Vedlegge er eksempler på tekster som kan gi relevante perspektiver
-på temaet:</p>
-<ol>
-<li>Flykningeregnskapet 2016, UNHCR og IDMC
-<li>«Grenseløst Europa for fall» A-Magasinet, 26. november 2015
-</ol>
-
-</blockquote>
-
-<p>Dette oversetter Apertium slik:</p>
-
-<blockquote>
-<p>Drøft utfordringane knytte til nasjonalstatane sine og rolla til
-andre aktørar og høve til å handtera internasjonale utfordringar, som
-til dømes *flykningekrisen.</p>
-
-<p>Vedleggja er døme på tekster som kan gje relevante perspektiv på
-temaet:</p>
-
-<ol>
-<li>*Flykningeregnskapet 2016, *UNHCR og *IDMC</li>
-<li>«*Grenseløst Europa for fall» A-Magasinet, 26. november 2015</li>
-</ol>
-
-</blockquote>
-
-<p>Ord som ikke ble forstått er markert med stjerne (*), og trenger
-ekstra språksjekk. Men ingen ord er forsvunnet, slik det var i
-oppgaven elevene fikk presentert på eksamen. Jeg mistenker dog at
-"andre aktørers rolle og muligheter til ..." burde vært oversatt til
-"rolla til andre aktørar og deira høve til ..." eller noe slikt, men
-det er kanskje flisespikking. Det understreker vel bare at det alltid
-trengs korrekturlesning etter automatisk oversettelse.</p>
+ <div class="title"><a href="http://people.skolelinux.org/pere/blog/Some_notes_on_fault_tolerant_storage_systems.html">Some notes on fault tolerant storage systems</a></div>
+ <div class="date"> 1st November 2017</div>
+ <div class="body"><p>If you care about how fault tolerant your storage is, you might
+find these articles and papers interesting. They have formed how I
+think of when designing a storage system.</p>
+
+<ul>
+
+<li>USENIX :login; <a
+href="https://www.usenix.org/publications/login/summer2017/ganesan">Redundancy
+Does Not Imply Fault Tolerance. Analysis of Distributed Storage
+Reactions to Single Errors and Corruptions</a> by Aishwarya Ganesan,
+Ramnatthan Alagappan, Andrea C. Arpaci-Dusseau, and Remzi
+H. Arpaci-Dusseau</li>
+
+<li>ZDNet
+<a href="http://www.zdnet.com/article/why-raid-5-stops-working-in-2009/">Why
+RAID 5 stops working in 2009</a> by Robin Harris</li>
+
+<li>ZDNet
+<a href="http://www.zdnet.com/article/why-raid-6-stops-working-in-2019/">Why
+RAID 6 stops working in 2019</a> by Robin Harris</li>
+
+<li>USENIX FAST'07
+<a href="http://research.google.com/archive/disk_failures.pdf">Failure
+Trends in a Large Disk Drive Population</a> by Eduardo Pinheiro,
+Wolf-Dietrich Weber and Luiz André Barroso</li>
+
+<li>USENIX ;login: <a
+href="https://www.usenix.org/system/files/login/articles/hughes12-04.pdf">Data
+Integrity. Finding Truth in a World of Guesses and Lies</a> by Doug
+Hughes</li>
+
+<li>USENIX FAST'08
+<a href="https://www.usenix.org/events/fast08/tech/full_papers/bairavasundaram/bairavasundaram_html/">An
+Analysis of Data Corruption in the Storage Stack</a> by
+L. N. Bairavasundaram, G. R. Goodson, B. Schroeder, A. C.
+Arpaci-Dusseau, and R. H. Arpaci-Dusseau</li>
+
+<li>USENIX FAST'07 <a
+href="https://www.usenix.org/legacy/events/fast07/tech/schroeder/schroeder_html/">Disk
+failures in the real world: what does an MTTF of 1,000,000 hours mean
+to you?</a> by B. Schroeder and G. A. Gibson.</li>
+
+<li>USENIX ;login: <a
+href="https://www.usenix.org/events/fast08/tech/full_papers/jiang/jiang_html/">Are
+Disks the Dominant Contributor for Storage Failures? A Comprehensive
+Study of Storage Subsystem Failure Characteristics</a> by Weihang
+Jiang, Chongfeng Hu, Yuanyuan Zhou, and Arkady Kanevsky</li>
+
+<li>SIGMETRICS 2007
+<a href="http://research.cs.wisc.edu/adsl/Publications/latent-sigmetrics07.pdf">An
+analysis of latent sector errors in disk drives</a> by
+L. N. Bairavasundaram, G. R. Goodson, S. Pasupathy, and J. Schindler</li>
+
+</ul>
+
+<p>Several of these research papers are based on data collected from
+hundred thousands or millions of disk, and their findings are eye
+opening. The short story is simply do not implicitly trust RAID or
+redundant storage systems. Details matter. And unfortunately there
+are few options on Linux addressing all the identified issues. Both
+ZFS and Btrfs are doing a fairly good job, but have legal and
+practical issues on their own. I wonder how cluster file systems like
+Ceph do in this regard. After all, there is an old saying, you know
+you have a distributed system when the crash of a computer you have
+never heard of stops you from getting any work done. The same holds
+true if fault tolerance do not work.</p>
+
+<p>Just remember, in the end, it do not matter how redundant, or how
+fault tolerant your storage is, if you do not continuously monitor its
+status to detect and replace failed disks.</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</div>
<div class="tags">
- Tags: <a href="http://people.skolelinux.org/pere/blog/tags/debian">debian</a>, <a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk</a>, <a href="http://people.skolelinux.org/pere/blog/tags/stavekontroll">stavekontroll</a>.
+ Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/raid">raid</a>, <a href="http://people.skolelinux.org/pere/blog/tags/sysadmin">sysadmin</a>.
</div>
<div class="padding"></div>
<div class="entry">
- <div class="title"><a href="http://people.skolelinux.org/pere/blog/Epost_inn_som_arkivformat_i_Riksarkivarens_forskrift_.html">Epost inn som arkivformat i Riksarkivarens forskrift?</a></div>
- <div class="date">27th April 2017</div>
- <div class="body"><p>I disse dager, med frist 1. mai, har Riksarkivaren ute en høring på
-sin forskrift. Som en kan se er det ikke mye tid igjen før fristen
-som går ut på søndag. Denne forskriften er det som lister opp hvilke
-formater det er greit å arkivere i
-<a href="http://www.arkivverket.no/arkivverket/Offentleg-forvalting/Noark/Noark-5">Noark
-5-løsninger</a> i Norge.</p>
-
-<p>Jeg fant høringsdokumentene hos
-<a href="https://www.arkivrad.no/aktuelt/riksarkivarens-forskrift-pa-horing">Norsk
-Arkivråd</a> etter å ha blitt tipset på epostlisten til
-<a href="https://github.com/hiOA-ABI/nikita-noark5-core">fri
-programvareprosjektet Nikita Noark5-Core</a>, som lager et Noark 5
-Tjenestegresesnitt. Jeg er involvert i Nikita-prosjektet og takket
-være min interesse for tjenestegrensesnittsprosjektet har jeg lest en
-god del Noark 5-relaterte dokumenter, og til min overraskelse oppdaget
-at standard epost ikke er på listen over godkjente formater som kan
-arkiveres. Høringen med frist søndag er en glimrende mulighet til å
-forsøke å gjøre noe med det. Jeg holder på med
-<a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/hoering-arkivforskrift.tex">egen
-høringsuttalelse</a>, og lurer på om andre er interessert i å støtte
-forslaget om å tillate arkivering av epost som epost i arkivet.</p>
-
-<p>Er du igang med å skrive egen høringsuttalelse allerede? I så fall
-kan du jo vurdere å ta med en formulering om epost-lagring. Jeg tror
-ikke det trengs så mye. Her et kort forslag til tekst:</p>
-
-<p><blockquote>
-
- <p>Viser til høring sendt ut 2017-02-17 (Riksarkivarens referanse
- 2016/9840 HELHJO), og tillater oss å sende inn noen innspill om
- revisjon av Forskrift om utfyllende tekniske og arkivfaglige
- bestemmelser om behandling av offentlige arkiver (Riksarkivarens
- forskrift).</p>
-
- <p>Svært mye av vår kommuikasjon foregår i dag på e-post. Vi
- foreslår derfor at Internett-e-post, slik det er beskrevet i IETF
- RFC 5322,
- <a href="https://tools.ietf.org/html/rfc5322">https://tools.ietf.org/html/rfc5322</a>. bør
- inn som godkjent dokumentformat. Vi foreslår at forskriftens
- oversikt over godkjente dokumentformater ved innlevering i § 5-16
- endres til å ta med Internett-e-post.</p>
-
-</blockquote></p>
-
-<p>Som del av arbeidet med tjenestegrensesnitt har vi testet hvordan
-epost kan lagres i en Noark 5-struktur, og holder på å skrive et
-forslag om hvordan dette kan gjøres som vil bli sendt over til
-arkivverket så snart det er ferdig. De som er interesserte kan
-<a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/docs/epostlagring.md">følge
-fremdriften på web</a>.</p>
-
-<p>Oppdatering 2017-04-28: I dag ble høringuttalelsen jeg skrev
- <a href="https://www.nuug.no/news/NUUGs_h_ringuttalelse_til_Riksarkivarens_forskrift.shtml">sendt
- inn av foreningen NUUG</a>.</p>
+ <div class="title"><a href="http://people.skolelinux.org/pere/blog/Web_services_for_writing_academic_LaTeX_papers_as_a_team.html">Web services for writing academic LaTeX papers as a team</a></div>
+ <div class="date">31st October 2017</div>
+ <div class="body"><p>I was surprised today to learn that a friend in academia did not
+know there are easily available web services available for writing
+LaTeX documents as a team. I thought it was common knowledge, but to
+make sure at least my readers are aware of it, I would like to mention
+these useful services for writing LaTeX documents. Some of them even
+provide a WYSIWYG editor to ease writing even further.</p>
+
+<p>There are two commercial services available,
+<a href="https://sharelatex.com">ShareLaTeX</a> and
+<a href="https://overleaf.com">Overleaf</a>. They are very easy to
+use. Just start a new document, select which publisher to write for
+(ie which LaTeX style to use), and start writing. Note, these two
+have announced their intention to join forces, so soon it will only be
+one joint service. I've used both for different documents, and they
+work just fine. While
+<a href="https://github.com/sharelatex/sharelatex">ShareLaTeX is free
+software</a>, while the latter is not. According to <a
+href="https://www.overleaf.com/help/17-is-overleaf-open-source">a
+announcement from Overleaf</a>, they plan to keep the ShareLaTeX code
+base maintained as free software.</p>
+
+But these two are not the only alternatives.
+<a href="https://app.fiduswriter.org/">Fidus Writer</a> is another free
+software solution with <a href="https://github.com/fiduswriter">the
+source available on github</a>. I have not used it myself. Several
+others can be found on the nice
+<a href="https://alternativeto.net/software/sharelatex/">alterntiveTo
+web service</a>.
+
+<p>If you like Google Docs or Etherpad, but would like to write
+documents in LaTeX, you should check out these services. You can even
+host your own, if you want to. :)</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</div>
<div class="tags">
- Tags: <a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk</a>, <a href="http://people.skolelinux.org/pere/blog/tags/offentlig innsyn">offentlig innsyn</a>, <a href="http://people.skolelinux.org/pere/blog/tags/standard">standard</a>.
+ Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>.
</div>
<div class="padding"></div>
<div class="entry">
- <div class="title"><a href="http://people.skolelinux.org/pere/blog/Offentlig_elektronisk_postjournal_blokkerer_tilgang_for_utvalgte_webklienter.html">Offentlig elektronisk postjournal blokkerer tilgang for utvalgte webklienter</a></div>
- <div class="date">20th April 2017</div>
- <div class="body"><p>Jeg oppdaget i dag at <a href="https://www.oep.no/">nettstedet som
-publiserer offentlige postjournaler fra statlige etater</a>, OEP, har
-begynt å blokkerer enkelte typer webklienter fra å få tilgang. Vet
-ikke hvor mange det gjelder, men det gjelder i hvert fall libwww-perl
-og curl. For å teste selv, kjør følgende:</p>
-
-<blockquote><pre>
-% curl -v -s https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
-< HTTP/1.1 404 Not Found
-% curl -v -s --header 'User-Agent:Opera/12.0' https://www.oep.no/pub/report.xhtml?reportId=3 2>&1 |grep '< HTTP'
-< HTTP/1.1 200 OK
-%
-</pre></blockquote>
-
-<p>Her kan en se at tjenesten gir «404 Not Found» for curl i
-standardoppsettet, mens den gir «200 OK» hvis curl hevder å være Opera
-versjon 12.0. Offentlig elektronisk postjournal startet blokkeringen
-2017-03-02.</p>
-
-<p>Blokkeringen vil gjøre det litt vanskeligere å maskinelt hente
-informasjon fra oep.no. Kan blokkeringen være gjort for å hindre
-automatisert innsamling av informasjon fra OEP, slik Pressens
-Offentlighetsutvalg gjorde for å dokumentere hvordan departementene
-hindrer innsyn i
-<a href="http://presse.no/dette-mener-np/undergraver-offentlighetsloven/">rapporten
-«Slik hindrer departementer innsyn» som ble publiserte i januar
-2017</a>. Det virker usannsynlig, da det jo er trivielt å bytte
-User-Agent til noe nytt.</p>
-
-<p>Finnes det juridisk grunnlag for det offentlige å diskriminere
-webklienter slik det gjøres her? Der tilgang gis eller ikke alt etter
-hva klienten sier at den heter? Da OEP eies av DIFI og driftes av
-Basefarm, finnes det kanskje noen dokumenter sendt mellom disse to
-aktørene man kan be om innsyn i for å forstå hva som har skjedd. Men
-<a href="https://www.oep.no/search/result.html?period=dateRange&fromDate=01.01.2016&toDate=01.04.2017&dateType=documentDate&caseDescription=&descType=both&caseNumber=&documentNumber=&sender=basefarm&senderType=both&documentType=all&legalAuthority=&archiveCode=&list2=196&searchType=advanced&Search=Search+in+records">postjournalen
-til DIFI viser kun to dokumenter</a> det siste året mellom DIFI og
-Basefarm.
-<a href="https://www.mimesbronn.no/request/blokkering_av_tilgang_til_oep_fo">Mimes brønn neste</a>,
-tenker jeg.</p>
+ <div class="title"><a href="http://people.skolelinux.org/pere/blog/Locating_IMDB_IDs_of_movies_in_the_Internet_Archive_using_Wikidata.html">Locating IMDB IDs of movies in the Internet Archive using Wikidata</a></div>
+ <div class="date">25th October 2017</div>
+ <div class="body"><p>Recently, I needed to automatically check the copyright status of a
+set of <a href="http://www.imdb.com/">The Internet Movie database
+(IMDB)</a> entries, to figure out which one of the movies they refer
+to can be freely distributed on the Internet. This proved to be
+harder than it sounds. IMDB for sure list movies without any
+copyright protection, where the copyright protection has expired or
+where the movie is lisenced using a permissive license like one from
+Creative Commons. These are mixed with copyright protected movies,
+and there seem to be no way to separate these classes of movies using
+the information in IMDB.</p>
+
+<p>First I tried to look up entries manually in IMDB,
+<a href="https://www.wikipedia.org/">Wikipedia</a> and
+<a href="https://www.archive.org/">The Internet Archive</a>, to get a
+feel how to do this. It is hard to know for sure using these sources,
+but it should be possible to be reasonable confident a movie is "out
+of copyright" with a few hours work per movie. As I needed to check
+almost 20,000 entries, this approach was not sustainable. I simply
+can not work around the clock for about 6 years to check this data
+set.</p>
+
+<p>I asked the people behind The Internet Archive if they could
+introduce a new metadata field in their metadata XML for IMDB ID, but
+was told that they leave it completely to the uploaders to update the
+metadata. Some of the metadata entries had IMDB links in the
+description, but I found no way to download all metadata files in bulk
+to locate those ones and put that approach aside.</p>
+
+<p>In the process I noticed several Wikipedia articles about movies
+had links to both IMDB and The Internet Archive, and it occured to me
+that I could use the Wikipedia RDF data set to locate entries with
+both, to at least get a lower bound on the number of movies on The
+Internet Archive with a IMDB ID. This is useful based on the
+assumption that movies distributed by The Internet Archive can be
+legally distributed on the Internet. With some help from the RDF
+community (thank you DanC), I was able to come up with this query to
+pass to <a href="https://query.wikidata.org/">the SPARQL interface on
+Wikidata</a>:
+
+<p><pre>
+SELECT ?work ?imdb ?ia ?when ?label
+WHERE
+{
+ ?work wdt:P31/wdt:P279* wd:Q11424.
+ ?work wdt:P345 ?imdb.
+ ?work wdt:P724 ?ia.
+ OPTIONAL {
+ ?work wdt:P577 ?when.
+ ?work rdfs:label ?label.
+ FILTER(LANG(?label) = "en").
+ }
+}
+</pre></p>
+
+<p>If I understand the query right, for every film entry anywhere in
+Wikpedia, it will return the IMDB ID and The Internet Archive ID, and
+when the movie was released and its English title, if either or both
+of the latter two are available. At the moment the result set contain
+2338 entries. Of course, it depend on volunteers including both
+correct IMDB and The Internet Archive IDs in the wikipedia articles
+for the movie. It should be noted that the result will include
+duplicates if the movie have entries in several languages. There are
+some bogus entries, either because The Internet Archive ID contain a
+typo or because the movie is not available from The Internet Archive.
+I did not verify the IMDB IDs, as I am unsure how to do that
+automatically.</p>
+
+<p>I wrote a small python script to extract the data set from Wikidata
+and check if the XML metadata for the movie is available from The
+Internet Archive, and after around 1.5 hour it produced a list of 2097
+free movies and their IMDB ID. In total, 171 entries in Wikidata lack
+the refered Internet Archive entry. I assume the 70 "disappearing"
+entries (ie 2338-2097-171) are duplicate entries.</p>
+
+<p>This is not too bad, given that The Internet Archive report to
+contain <a href="https://archive.org/details/feature_films">5331
+feature films</a> at the moment, but it also mean more than 3000
+movies are missing on Wikipedia or are missing the pair of references
+on Wikipedia.</p>
+
+<p>I was curious about the distribution by release year, and made a
+little graph to show how the amount of free movies is spread over the
+years:<p>
+
+<p><img src="http://people.skolelinux.org/pere/blog/images/2017-10-25-verk-i-det-fri-filmer.png"></p>
+
+<p>I expect the relative distribution of the remaining 3000 movies to
+be similar.</p>
+
+<p>If you want to help, and want to ensure Wikipedia can be used to
+cross reference The Internet Archive and The Internet Movie Database,
+please make sure entries like this are listed under the "External
+links" heading on the Wikipedia article for the movie:</p>
+
+<p><pre>
+* {{Internet Archive film|id=FightingLady}}
+* {{IMDb title|id=0036823|title=The Fighting Lady}}
+</pre></p>
+
+<p>Please verify the links on the final page, to make sure you did not
+introduce a typo.</p>
+
+<p>Here is the complete list, if you want to correct the 171
+identified Wikipedia entries with broken links to The Internet
+Archive: <a href="http://www.wikidata.org/entity/Q1140317">Q1140317</a>,
+<a href="http://www.wikidata.org/entity/Q458656">Q458656</a>,
+<a href="http://www.wikidata.org/entity/Q458656">Q458656</a>,
+<a href="http://www.wikidata.org/entity/Q470560">Q470560</a>,
+<a href="http://www.wikidata.org/entity/Q743340">Q743340</a>,
+<a href="http://www.wikidata.org/entity/Q822580">Q822580</a>,
+<a href="http://www.wikidata.org/entity/Q480696">Q480696</a>,
+<a href="http://www.wikidata.org/entity/Q128761">Q128761</a>,
+<a href="http://www.wikidata.org/entity/Q1307059">Q1307059</a>,
+<a href="http://www.wikidata.org/entity/Q1335091">Q1335091</a>,
+<a href="http://www.wikidata.org/entity/Q1537166">Q1537166</a>,
+<a href="http://www.wikidata.org/entity/Q1438334">Q1438334</a>,
+<a href="http://www.wikidata.org/entity/Q1479751">Q1479751</a>,
+<a href="http://www.wikidata.org/entity/Q1497200">Q1497200</a>,
+<a href="http://www.wikidata.org/entity/Q1498122">Q1498122</a>,
+<a href="http://www.wikidata.org/entity/Q865973">Q865973</a>,
+<a href="http://www.wikidata.org/entity/Q834269">Q834269</a>,
+<a href="http://www.wikidata.org/entity/Q841781">Q841781</a>,
+<a href="http://www.wikidata.org/entity/Q841781">Q841781</a>,
+<a href="http://www.wikidata.org/entity/Q1548193">Q1548193</a>,
+<a href="http://www.wikidata.org/entity/Q499031">Q499031</a>,
+<a href="http://www.wikidata.org/entity/Q1564769">Q1564769</a>,
+<a href="http://www.wikidata.org/entity/Q1585239">Q1585239</a>,
+<a href="http://www.wikidata.org/entity/Q1585569">Q1585569</a>,
+<a href="http://www.wikidata.org/entity/Q1624236">Q1624236</a>,
+<a href="http://www.wikidata.org/entity/Q4796595">Q4796595</a>,
+<a href="http://www.wikidata.org/entity/Q4853469">Q4853469</a>,
+<a href="http://www.wikidata.org/entity/Q4873046">Q4873046</a>,
+<a href="http://www.wikidata.org/entity/Q915016">Q915016</a>,
+<a href="http://www.wikidata.org/entity/Q4660396">Q4660396</a>,
+<a href="http://www.wikidata.org/entity/Q4677708">Q4677708</a>,
+<a href="http://www.wikidata.org/entity/Q4738449">Q4738449</a>,
+<a href="http://www.wikidata.org/entity/Q4756096">Q4756096</a>,
+<a href="http://www.wikidata.org/entity/Q4766785">Q4766785</a>,
+<a href="http://www.wikidata.org/entity/Q880357">Q880357</a>,
+<a href="http://www.wikidata.org/entity/Q882066">Q882066</a>,
+<a href="http://www.wikidata.org/entity/Q882066">Q882066</a>,
+<a href="http://www.wikidata.org/entity/Q204191">Q204191</a>,
+<a href="http://www.wikidata.org/entity/Q204191">Q204191</a>,
+<a href="http://www.wikidata.org/entity/Q1194170">Q1194170</a>,
+<a href="http://www.wikidata.org/entity/Q940014">Q940014</a>,
+<a href="http://www.wikidata.org/entity/Q946863">Q946863</a>,
+<a href="http://www.wikidata.org/entity/Q172837">Q172837</a>,
+<a href="http://www.wikidata.org/entity/Q573077">Q573077</a>,
+<a href="http://www.wikidata.org/entity/Q1219005">Q1219005</a>,
+<a href="http://www.wikidata.org/entity/Q1219599">Q1219599</a>,
+<a href="http://www.wikidata.org/entity/Q1643798">Q1643798</a>,
+<a href="http://www.wikidata.org/entity/Q1656352">Q1656352</a>,
+<a href="http://www.wikidata.org/entity/Q1659549">Q1659549</a>,
+<a href="http://www.wikidata.org/entity/Q1660007">Q1660007</a>,
+<a href="http://www.wikidata.org/entity/Q1698154">Q1698154</a>,
+<a href="http://www.wikidata.org/entity/Q1737980">Q1737980</a>,
+<a href="http://www.wikidata.org/entity/Q1877284">Q1877284</a>,
+<a href="http://www.wikidata.org/entity/Q1199354">Q1199354</a>,
+<a href="http://www.wikidata.org/entity/Q1199354">Q1199354</a>,
+<a href="http://www.wikidata.org/entity/Q1199451">Q1199451</a>,
+<a href="http://www.wikidata.org/entity/Q1211871">Q1211871</a>,
+<a href="http://www.wikidata.org/entity/Q1212179">Q1212179</a>,
+<a href="http://www.wikidata.org/entity/Q1238382">Q1238382</a>,
+<a href="http://www.wikidata.org/entity/Q4906454">Q4906454</a>,
+<a href="http://www.wikidata.org/entity/Q320219">Q320219</a>,
+<a href="http://www.wikidata.org/entity/Q1148649">Q1148649</a>,
+<a href="http://www.wikidata.org/entity/Q645094">Q645094</a>,
+<a href="http://www.wikidata.org/entity/Q5050350">Q5050350</a>,
+<a href="http://www.wikidata.org/entity/Q5166548">Q5166548</a>,
+<a href="http://www.wikidata.org/entity/Q2677926">Q2677926</a>,
+<a href="http://www.wikidata.org/entity/Q2698139">Q2698139</a>,
+<a href="http://www.wikidata.org/entity/Q2707305">Q2707305</a>,
+<a href="http://www.wikidata.org/entity/Q2740725">Q2740725</a>,
+<a href="http://www.wikidata.org/entity/Q2024780">Q2024780</a>,
+<a href="http://www.wikidata.org/entity/Q2117418">Q2117418</a>,
+<a href="http://www.wikidata.org/entity/Q2138984">Q2138984</a>,
+<a href="http://www.wikidata.org/entity/Q1127992">Q1127992</a>,
+<a href="http://www.wikidata.org/entity/Q1058087">Q1058087</a>,
+<a href="http://www.wikidata.org/entity/Q1070484">Q1070484</a>,
+<a href="http://www.wikidata.org/entity/Q1080080">Q1080080</a>,
+<a href="http://www.wikidata.org/entity/Q1090813">Q1090813</a>,
+<a href="http://www.wikidata.org/entity/Q1251918">Q1251918</a>,
+<a href="http://www.wikidata.org/entity/Q1254110">Q1254110</a>,
+<a href="http://www.wikidata.org/entity/Q1257070">Q1257070</a>,
+<a href="http://www.wikidata.org/entity/Q1257079">Q1257079</a>,
+<a href="http://www.wikidata.org/entity/Q1197410">Q1197410</a>,
+<a href="http://www.wikidata.org/entity/Q1198423">Q1198423</a>,
+<a href="http://www.wikidata.org/entity/Q706951">Q706951</a>,
+<a href="http://www.wikidata.org/entity/Q723239">Q723239</a>,
+<a href="http://www.wikidata.org/entity/Q2079261">Q2079261</a>,
+<a href="http://www.wikidata.org/entity/Q1171364">Q1171364</a>,
+<a href="http://www.wikidata.org/entity/Q617858">Q617858</a>,
+<a href="http://www.wikidata.org/entity/Q5166611">Q5166611</a>,
+<a href="http://www.wikidata.org/entity/Q5166611">Q5166611</a>,
+<a href="http://www.wikidata.org/entity/Q324513">Q324513</a>,
+<a href="http://www.wikidata.org/entity/Q374172">Q374172</a>,
+<a href="http://www.wikidata.org/entity/Q7533269">Q7533269</a>,
+<a href="http://www.wikidata.org/entity/Q970386">Q970386</a>,
+<a href="http://www.wikidata.org/entity/Q976849">Q976849</a>,
+<a href="http://www.wikidata.org/entity/Q7458614">Q7458614</a>,
+<a href="http://www.wikidata.org/entity/Q5347416">Q5347416</a>,
+<a href="http://www.wikidata.org/entity/Q5460005">Q5460005</a>,
+<a href="http://www.wikidata.org/entity/Q5463392">Q5463392</a>,
+<a href="http://www.wikidata.org/entity/Q3038555">Q3038555</a>,
+<a href="http://www.wikidata.org/entity/Q5288458">Q5288458</a>,
+<a href="http://www.wikidata.org/entity/Q2346516">Q2346516</a>,
+<a href="http://www.wikidata.org/entity/Q5183645">Q5183645</a>,
+<a href="http://www.wikidata.org/entity/Q5185497">Q5185497</a>,
+<a href="http://www.wikidata.org/entity/Q5216127">Q5216127</a>,
+<a href="http://www.wikidata.org/entity/Q5223127">Q5223127</a>,
+<a href="http://www.wikidata.org/entity/Q5261159">Q5261159</a>,
+<a href="http://www.wikidata.org/entity/Q1300759">Q1300759</a>,
+<a href="http://www.wikidata.org/entity/Q5521241">Q5521241</a>,
+<a href="http://www.wikidata.org/entity/Q7733434">Q7733434</a>,
+<a href="http://www.wikidata.org/entity/Q7736264">Q7736264</a>,
+<a href="http://www.wikidata.org/entity/Q7737032">Q7737032</a>,
+<a href="http://www.wikidata.org/entity/Q7882671">Q7882671</a>,
+<a href="http://www.wikidata.org/entity/Q7719427">Q7719427</a>,
+<a href="http://www.wikidata.org/entity/Q7719444">Q7719444</a>,
+<a href="http://www.wikidata.org/entity/Q7722575">Q7722575</a>,
+<a href="http://www.wikidata.org/entity/Q2629763">Q2629763</a>,
+<a href="http://www.wikidata.org/entity/Q2640346">Q2640346</a>,
+<a href="http://www.wikidata.org/entity/Q2649671">Q2649671</a>,
+<a href="http://www.wikidata.org/entity/Q7703851">Q7703851</a>,
+<a href="http://www.wikidata.org/entity/Q7747041">Q7747041</a>,
+<a href="http://www.wikidata.org/entity/Q6544949">Q6544949</a>,
+<a href="http://www.wikidata.org/entity/Q6672759">Q6672759</a>,
+<a href="http://www.wikidata.org/entity/Q2445896">Q2445896</a>,
+<a href="http://www.wikidata.org/entity/Q12124891">Q12124891</a>,
+<a href="http://www.wikidata.org/entity/Q3127044">Q3127044</a>,
+<a href="http://www.wikidata.org/entity/Q2511262">Q2511262</a>,
+<a href="http://www.wikidata.org/entity/Q2517672">Q2517672</a>,
+<a href="http://www.wikidata.org/entity/Q2543165">Q2543165</a>,
+<a href="http://www.wikidata.org/entity/Q426628">Q426628</a>,
+<a href="http://www.wikidata.org/entity/Q426628">Q426628</a>,
+<a href="http://www.wikidata.org/entity/Q12126890">Q12126890</a>,
+<a href="http://www.wikidata.org/entity/Q13359969">Q13359969</a>,
+<a href="http://www.wikidata.org/entity/Q13359969">Q13359969</a>,
+<a href="http://www.wikidata.org/entity/Q2294295">Q2294295</a>,
+<a href="http://www.wikidata.org/entity/Q2294295">Q2294295</a>,
+<a href="http://www.wikidata.org/entity/Q2559509">Q2559509</a>,
+<a href="http://www.wikidata.org/entity/Q2559912">Q2559912</a>,
+<a href="http://www.wikidata.org/entity/Q7760469">Q7760469</a>,
+<a href="http://www.wikidata.org/entity/Q6703974">Q6703974</a>,
+<a href="http://www.wikidata.org/entity/Q4744">Q4744</a>,
+<a href="http://www.wikidata.org/entity/Q7766962">Q7766962</a>,
+<a href="http://www.wikidata.org/entity/Q7768516">Q7768516</a>,
+<a href="http://www.wikidata.org/entity/Q7769205">Q7769205</a>,
+<a href="http://www.wikidata.org/entity/Q7769988">Q7769988</a>,
+<a href="http://www.wikidata.org/entity/Q2946945">Q2946945</a>,
+<a href="http://www.wikidata.org/entity/Q3212086">Q3212086</a>,
+<a href="http://www.wikidata.org/entity/Q3212086">Q3212086</a>,
+<a href="http://www.wikidata.org/entity/Q18218448">Q18218448</a>,
+<a href="http://www.wikidata.org/entity/Q18218448">Q18218448</a>,
+<a href="http://www.wikidata.org/entity/Q18218448">Q18218448</a>,
+<a href="http://www.wikidata.org/entity/Q6909175">Q6909175</a>,
+<a href="http://www.wikidata.org/entity/Q7405709">Q7405709</a>,
+<a href="http://www.wikidata.org/entity/Q7416149">Q7416149</a>,
+<a href="http://www.wikidata.org/entity/Q7239952">Q7239952</a>,
+<a href="http://www.wikidata.org/entity/Q7317332">Q7317332</a>,
+<a href="http://www.wikidata.org/entity/Q7783674">Q7783674</a>,
+<a href="http://www.wikidata.org/entity/Q7783704">Q7783704</a>,
+<a href="http://www.wikidata.org/entity/Q7857590">Q7857590</a>,
+<a href="http://www.wikidata.org/entity/Q3372526">Q3372526</a>,
+<a href="http://www.wikidata.org/entity/Q3372642">Q3372642</a>,
+<a href="http://www.wikidata.org/entity/Q3372816">Q3372816</a>,
+<a href="http://www.wikidata.org/entity/Q3372909">Q3372909</a>,
+<a href="http://www.wikidata.org/entity/Q7959649">Q7959649</a>,
+<a href="http://www.wikidata.org/entity/Q7977485">Q7977485</a>,
+<a href="http://www.wikidata.org/entity/Q7992684">Q7992684</a>,
+<a href="http://www.wikidata.org/entity/Q3817966">Q3817966</a>,
+<a href="http://www.wikidata.org/entity/Q3821852">Q3821852</a>,
+<a href="http://www.wikidata.org/entity/Q3420907">Q3420907</a>,
+<a href="http://www.wikidata.org/entity/Q3429733">Q3429733</a>,
+<a href="http://www.wikidata.org/entity/Q774474">Q774474</a></p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</div>
<div class="tags">
- Tags: <a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk</a>, <a href="http://people.skolelinux.org/pere/blog/tags/offentlig innsyn">offentlig innsyn</a>.
+ Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett</a>, <a href="http://people.skolelinux.org/pere/blog/tags/verkidetfri">verkidetfri</a>.
</div>
<div class="padding"></div>
<div class="entry">
- <div class="title"><a href="http://people.skolelinux.org/pere/blog/Free_software_archive_system_Nikita_now_able_to_store_documents.html">Free software archive system Nikita now able to store documents</a></div>
- <div class="date">19th March 2017</div>
- <div class="body"><p>The <a href="https://github.com/hiOA-ABI/nikita-noark5-core">Nikita
-Noark 5 core project</a> is implementing the Norwegian standard for
-keeping an electronic archive of government documents.
-<a href="http://www.arkivverket.no/arkivverket/Offentlig-forvaltning/Noark/Noark-5/English-version">The
-Noark 5 standard</a> document the requirement for data systems used by
-the archives in the Norwegian government, and the Noark 5 web interface
-specification document a REST web service for storing, searching and
-retrieving documents and metadata in such archive. I've been involved
-in the project since a few weeks before Christmas, when the Norwegian
-Unix User Group
-<a href="https://www.nuug.no/news/NOARK5_kjerne_som_fri_programvare_f_r_epostliste_hos_NUUG.shtml">announced
-it supported the project</a>. I believe this is an important project,
-and hope it can make it possible for the government archives in the
-future to use free software to keep the archives we citizens depend
-on. But as I do not hold such archive myself, personally my first use
-case is to store and analyse public mail journal metadata published
-from the government. I find it useful to have a clear use case in
-mind when developing, to make sure the system scratches one of my
-itches.</p>
-
-<p>If you would like to help make sure there is a free software
-alternatives for the archives, please join our IRC channel
-(<a href="irc://irc.freenode.net/%23nikita"">#nikita on
-irc.freenode.net</a>) and
-<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">the
-project mailing list</a>.</p>
-
-<p>When I got involved, the web service could store metadata about
-documents. But a few weeks ago, a new milestone was reached when it
-became possible to store full text documents too. Yesterday, I
-completed an implementation of a command line tool
-<tt>archive-pdf</tt> to upload a PDF file to the archive using this
-API. The tool is very simple at the moment, and find existing
-<a href="https://en.wikipedia.org/wiki/Fonds">fonds</a>, series and
-files while asking the user to select which one to use if more than
-one exist. Once a file is identified, the PDF is associated with the
-file and uploaded, using the title extracted from the PDF itself. The
-process is fairly similar to visiting the archive, opening a cabinet,
-locating a file and storing a piece of paper in the archive. Here is
-a test run directly after populating the database with test data using
-our API tester:</p>
-
-<p><blockquote><pre>
-~/src//noark5-tester$ ./archive-pdf mangelmelding/mangler.pdf
-using arkiv: Title of the test fonds created 2017-03-18T23:49:32.103446
-using arkivdel: Title of the test series created 2017-03-18T23:49:32.103446
-
- 0 - Title of the test case file created 2017-03-18T23:49:32.103446
- 1 - Title of the test file created 2017-03-18T23:49:32.103446
-Select which mappe you want (or search term): 0
-Uploading mangelmelding/mangler.pdf
- PDF title: Mangler i spesifikasjonsdokumentet for NOARK 5 Tjenestegrensesnitt
- File 2017/1: Title of the test case file created 2017-03-18T23:49:32.103446
-~/src//noark5-tester$
-</pre></blockquote></p>
-
-<p>You can see here how the fonds (arkiv) and serie (arkivdel) only had
-one option, while the user need to choose which file (mappe) to use
-among the two created by the API tester. The <tt>archive-pdf</tt>
-tool can be found in the git repository for the API tester.</p>
-
-<p>In the project, I have been mostly working on
-<a href="https://github.com/petterreinholdtsen/noark5-tester">the API
-tester</a> so far, while getting to know the code base. The API
-tester currently use
-<a href="https://en.wikipedia.org/wiki/HATEOAS">the HATEOAS links</a>
-to traverse the entire exposed service API and verify that the exposed
-operations and objects match the specification, as well as trying to
-create objects holding metadata and uploading a simple XML file to
-store. The tester has proved very useful for finding flaws in our
-implementation, as well as flaws in the reference site and the
-specification.</p>
-
-<p>The test document I uploaded is a summary of all the specification
-defects we have collected so far while implementing the web service.
-There are several unclear and conflicting parts of the specification,
-and we have
-<a href="https://github.com/petterreinholdtsen/noark5-tester/tree/master/mangelmelding">started
-writing down</a> the questions we get from implementing it. We use a
-format inspired by how <a href="http://www.opengroup.org/austin/">The
-Austin Group</a> collect defect reports for the POSIX standard with
-<a href="http://www.opengroup.org/austin/mantis.html">their
-instructions for the MANTIS defect tracker system</a>, in lack of an official way to structure defect reports for Noark 5 (our first submitted defect report was a <a href="https://github.com/petterreinholdtsen/noark5-tester/blob/master/mangelmelding/sendt/2017-03-15-mangel-prosess.md">request for a procedure for submitting defect reports</a> :).
-
-<p>The Nikita project is implemented using Java and Spring, and is
-fairly easy to get up and running using Docker containers for those
-that want to test the current code base. The API tester is
-implemented in Python.</p>
+ <div class="title"><a href="http://people.skolelinux.org/pere/blog/A_one_way_wall_on_the_border_.html">A one-way wall on the border?</a></div>
+ <div class="date">14th October 2017</div>
+ <div class="body"><p>I find it fascinating how many of the people being locked inside
+the proposed border wall between USA and Mexico support the idea. The
+proposal to keep Mexicans out reminds me of
+<a href="http://www.history.com/news/10-things-you-may-not-know-about-the-berlin-wall">the
+propaganda twist from the East Germany government</a> calling the wall
+the “Antifascist Bulwark” after erecting the Berlin Wall, claiming
+that the wall was erected to keep enemies from creeping into East
+Germany, while it was obvious to the people locked inside it that it
+was erected to keep the people from escaping.</p>
+
+<p>Do the people in USA supporting this wall really believe it is a
+one way wall, only keeping people on the outside from getting in,
+while not keeping people in the inside from getting out?</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</div>
<div class="tags">
- Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/nuug">nuug</a>, <a href="http://people.skolelinux.org/pere/blog/tags/offentlig innsyn">offentlig innsyn</a>, <a href="http://people.skolelinux.org/pere/blog/tags/standard">standard</a>.
+ Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>.
</div>
<div class="padding"></div>
<div class="entry">
- <div class="title"><a href="http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html">Detecting NFS hangs on Linux without hanging yourself...</a></div>
- <div class="date"> 9th March 2017</div>
- <div class="body"><p>Over the years, administrating thousand of NFS mounting linux
-computers at the time, I often needed a way to detect if the machine
-was experiencing NFS hang. If you try to use <tt>df</tt> or look at a
-file or directory affected by the hang, the process (and possibly the
-shell) will hang too. So you want to be able to detect this without
-risking the detection process getting stuck too. It has not been
-obvious how to do this. When the hang has lasted a while, it is
-possible to find messages like these in dmesg:</p>
-
-<p><blockquote>
-nfs: server nfsserver not responding, still trying
-<br>nfs: server nfsserver OK
-</blockquote></p>
-
-<p>It is hard to know if the hang is still going on, and it is hard to
-be sure looking in dmesg is going to work. If there are lots of other
-messages in dmesg the lines might have rotated out of site before they
-are noticed.</p>
-
-<p>While reading through the nfs client implementation in linux kernel
-code, I came across some statistics that seem to give a way to detect
-it. The om_timeouts sunrpc value in the kernel will increase every
-time the above log entry is inserted into dmesg. And after digging a
-bit further, I discovered that this value show up in
-/proc/self/mountstats on Linux.</p>
-
-<p>The mountstats content seem to be shared between files using the
-same file system context, so it is enough to check one of the
-mountstats files to get the state of the mount point for the machine.
-I assume this will not show lazy umounted NFS points, nor NFS mount
-points in a different process context (ie with a different filesystem
-view), but that does not worry me.</p>
-
-<p>The content for a NFS mount point look similar to this:</p>
-
-<p><blockquote><pre>
-[...]
-device /dev/mapper/Debian-var mounted on /var with fstype ext3
-device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1
- opts: rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all
- age: 7863311
- caps: caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255
- sec: flavor=1,pseudoflavor=1
- events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0
- bytes: 166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809
- RPC iostats version: 1.0 p/v: 100003/3 (nfs)
- xprt: tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820
- per-op statistics
- NULL: 0 0 0 0 0 0 0 0
- GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132
- SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943
- LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511
- ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140
- READLINK: 125 125 0 20472 18620 0 1112 1118
- READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693
- WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771
- CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398
- MKDIR: 3680 3680 0 773980 993920 26 23990 24245
- SYMLINK: 903 903 0 233428 245488 6 5865 5917
- MKNOD: 80 80 0 20148 21760 0 299 304
- REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636
- RMDIR: 3367 3367 0 645112 484848 22 5782 6002
- RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288
- LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579
- READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917
- READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937
- FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022
- FSINFO: 2 2 0 232 328 0 1 1
- PATHCONF: 1 1 0 116 140 0 0 0
- COMMIT: 0 0 0 0 0 0 0 0
-
-device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
-[...]
-</pre></blockquote></p>
-
-<p>The key number to look at is the third number in the per-op list.
-It is the number of NFS timeouts experiences per file system
-operation. Here 22 write timeouts and 5 access timeouts. If these
-numbers are increasing, I believe the machine is experiencing NFS
-hang. Unfortunately the timeout value do not start to increase right
-away. The NFS operations need to time out first, and this can take a
-while. The exact timeout value depend on the setup. For example the
-defaults for TCP and UDP mount points are quite different, and the
-timeout value is affected by the soft, hard, timeo and retrans NFS
-mount options.</p>
-
-<p>The only way I have been able to get working on Debian and RedHat
-Enterprise Linux for getting the timeout count is to peek in /proc/.
-But according to
-<ahref="http://docs.oracle.com/cd/E19253-01/816-4555/netmonitor-12/index.html">Solaris
-10 System Administration Guide: Network Services</a>, the 'nfsstat -c'
-command can be used to get these timeout values. But this do not work
-on Linux, as far as I can tell. I
-<ahref="http://bugs.debian.org/857043">asked Debian about this</a>,
-but have not seen any replies yet.</p>
-
-<p>Is there a better way to figure out if a Linux NFS client is
-experiencing NFS hangs? Is there a way to detect which processes are
-affected? Is there a way to get the NFS mount going quickly once the
-network problem causing the NFS hang has been cleared? I would very
-much welcome some clues, as we regularly run into NFS hangs.</p>
+ <div class="title"><a href="http://people.skolelinux.org/pere/blog/Generating_3D_prints_in_Debian_using_Cura_and_Slic3r__prusa_.html">Generating 3D prints in Debian using Cura and Slic3r(-prusa)</a></div>
+ <div class="date"> 9th October 2017</div>
+ <div class="body"><p>At my nearby maker space,
+<a href="http://sonen.ifi.uio.no/">Sonen</a>, I heard the story that it
+was easier to generate gcode files for theyr 3D printers (Ultimake 2+)
+on Windows and MacOS X than Linux, because the software involved had
+to be manually compiled and set up on Linux while premade packages
+worked out of the box on Windows and MacOS X. I found this annoying,
+as the software involved,
+<a href="https://github.com/Ultimaker/Cura">Cura</a>, is free software
+and should be trivial to get up and running on Linux if someone took
+the time to package it for the relevant distributions. I even found
+<a href="https://bugs.debian.org/706656">a request for adding into
+Debian</a> from 2013, which had seem some activity over the years but
+never resulted in the software showing up in Debian. So a few days
+ago I offered my help to try to improve the situation.</p>
+
+<p>Now I am very happy to see that all the packages required by a
+working Cura in Debian are uploaded into Debian and waiting in the NEW
+queue for the ftpmasters to have a look. You can track the progress
+on
+<a href="https://qa.debian.org/developer.php?email=3dprinter-general%40lists.alioth.debian.org">the
+status page for the 3D printer team</a>.</p>
+
+<p>The uploaded packages are a bit behind upstream, and was uploaded
+now to get slots in <a href="https://ftp-master.debian.org/new.html">the NEW
+queue</a> while we work up updating the packages to the latest
+upstream version.</p>
+
+<p>On a related note, two competitors for Cura, which I found harder
+to use and was unable to configure correctly for Ultimaker 2+ in the
+short time I spent on it, are already in Debian. If you are looking
+for 3D printer "slicers" and want something already available in
+Debian, check out
+<a href="https://tracker.debian.org/pkg/slic3r">slic3r</a> and
+<a href="https://tracker.debian.org/pkg/slic3r-prusa">slic3r-prusa</a>.
+The latter is a fork of the former.</p>
+
+<p>As usual, if you use Bitcoin and want to show your support of my
+activities, please send Bitcoin donations to my address
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</div>
<div class="tags">
- Tags: <a href="http://people.skolelinux.org/pere/blog/tags/debian">debian</a>, <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/sysadmin">sysadmin</a>.
+ Tags: <a href="http://people.skolelinux.org/pere/blog/tags/3d-printer">3d-printer</a>, <a href="http://people.skolelinux.org/pere/blog/tags/debian">debian</a>, <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>.
</div>
<div class="padding"></div>
<div class="entry">
- <div class="title"><a href="http://people.skolelinux.org/pere/blog/How_does_it_feel_to_be_wiretapped__when_you_should_be_doing_the_wiretapping___.html">How does it feel to be wiretapped, when you should be doing the wiretapping...</a></div>
- <div class="date"> 8th March 2017</div>
- <div class="body"><p>So the new president in the United States of America claim to be
-surprised to discover that he was wiretapped during the election
-before he was elected president. He even claim this must be illegal.
-Well, doh, if it is one thing the confirmations from Snowden
-documented, it is that the entire population in USA is wiretapped, one
-way or another. Of course the president candidates were wiretapped,
-alongside the senators, judges and the rest of the people in USA.</p>
-
-<p>Next, the Federal Bureau of Investigation ask the Department of
-Justice to go public rejecting the claims that Donald Trump was
-wiretapped illegally. I fail to see the relevance, given that I am
-sure the surveillance industry in USA believe they have all the legal
-backing they need to conduct mass surveillance on the entire
-world.</p>
-
-<p>There is even the director of the FBI stating that he never saw an
-order requesting wiretapping of Donald Trump. That is not very
-surprising, given how the FISA court work, with all its activity being
-secret. Perhaps he only heard about it?</p>
-
-<p>What I find most sad in this story is how Norwegian journalists
-present it. In a news reports the other day in the radio from the
-Norwegian National broadcasting Company (NRK), I heard the journalist
-claim that 'the FBI denies any wiretapping', while the reality is that
-'the FBI denies any illegal wiretapping'. There is a fundamental and
-important difference, and it make me sad that the journalists are
-unable to grasp it.</p>
-
-<p><strong>Update 2017-03-13:</strong> Look like
-<a href="https://theintercept.com/2017/03/13/rand-paul-is-right-nsa-routinely-monitors-americans-communications-without-warrants/">The
-Intercept report that US Senator Rand Paul confirm what I state above</a>.</p>
+ <div class="title"><a href="http://people.skolelinux.org/pere/blog/Mangler_du_en_skrue__eller_har_du_en_skrue_l_s_.html">Mangler du en skrue, eller har du en skrue løs?</a></div>
+ <div class="date"> 4th October 2017</div>
+ <div class="body">Når jeg holder på med ulike prosjekter, så trenger jeg stadig ulike
+skruer. Det siste prosjektet jeg holder på med er å lage
+<a href="https://www.thingiverse.com/thing:676916">en boks til en
+HDMI-touch-skjerm</a> som skal brukes med Raspberry Pi. Boksen settes
+sammen med skruer og bolter, og jeg har vært i tvil om hvor jeg kan
+få tak i de riktige skruene. Clas Ohlson og Jernia i nærheten har
+sjelden hatt det jeg trenger. Men her om dagen fikk jeg et fantastisk
+tips for oss som bor i Oslo.
+<a href="http://www.zachskruer.no/">Zachariassen Jernvare AS</a> i
+<a href="http://www.openstreetmap.org/?mlat=59.93421&mlon=10.76795#map=19/59.93421/10.76795">Hegermannsgate
+23A på Torshov</a> har et fantastisk utvalg, og åpent mellom 09:00 og
+17:00. De selger skruer, muttere, bolter, skiver etc i løs vekt, og
+så langt har jeg fått alt jeg har lett etter. De har i tillegg det
+meste av annen jernvare, som verktøy, lamper, ledninger, etc. Jeg
+håper de har nok kunder til å holde det gående lenge, da dette er en
+butikk jeg kommer til å besøke ofte. Butikken er et funn å ha i
+nabolaget for oss som liker å bygge litt selv. :)</p>
+
+<p>Som vanlig, hvis du bruker Bitcoin og ønsker å vise din støtte til
+det jeg driver med, setter jeg pris på om du sender Bitcoin-donasjoner
+til min adresse
+<b><a href="bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b</a></b>.</p>
</div>
<div class="tags">
- Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/surveillance">surveillance</a>.
+ Tags: <a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk</a>.
</div>
<li><a href="http://people.skolelinux.org/pere/blog/archive/2017/06/">June (5)</a></li>
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2017/07/">July (1)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2017/08/">August (1)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2017/09/">September (3)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2017/10/">October (5)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2017/11/">November (3)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2017/12/">December (2)</a></li>
+
</ul></li>
<li>2016
<h2>Tags</h2>
<ul>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/3d-printer">3d-printer (13)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/3d-printer">3d-printer (14)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/amiga">amiga (1)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/chrpath">chrpath (2)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/debian">debian (149)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/debian">debian (154)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/debian edu">debian edu (158)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/debian-handbook">debian-handbook (3)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/debian-handbook">debian-handbook (4)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/digistan">digistan (10)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/dld">dld (16)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/dld">dld (17)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/docbook">docbook (24)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/drivstoffpriser">drivstoffpriser (4)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/english">english (349)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/english">english (362)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/fiksgatami">fiksgatami (23)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/fildeling">fildeling (12)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/freeculture">freeculture (30)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/freeculture">freeculture (31)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/freedombox">freedombox (9)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/ldap">ldap (9)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/lego">lego (4)</a></li>
+
<li><a href="http://people.skolelinux.org/pere/blog/tags/lenker">lenker (8)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/lsdvd">lsdvd (2)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/nice free software">nice free software (9)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk (291)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk (293)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/nuug">nuug (189)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/open311">open311 (2)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett (64)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett (69)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/personvern">personvern (100)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/personvern">personvern (104)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/raid">raid (1)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/raid">raid (2)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/reactos">reactos (1)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/stavekontroll">stavekontroll (6)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/stortinget">stortinget (11)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/stortinget">stortinget (12)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/surveillance">surveillance (48)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/surveillance">surveillance (52)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/sysadmin">sysadmin (3)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/sysadmin">sysadmin (4)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/usenix">usenix (2)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/valg">valg (8)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/valg">valg (9)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/verkidetfri">verkidetfri (8)</a></li>
- <li><a href="http://people.skolelinux.org/pere/blog/tags/video">video (59)</a></li>
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/video">video (60)</a></li>
<li><a href="http://people.skolelinux.org/pere/blog/tags/vitenskap">vitenskap (4)</a></li>
projects / homepage.git / blobdiff