1 <?xml version=
"1.0" encoding=
"utf-8"?>
2 <rss version='
2.0' xmlns:lj='http://www.livejournal.org/rss/lj/
1.0/'
>
4 <title>Petter Reinholdtsen - Entries tagged nice free software
</title>
5 <description>Entries tagged nice free software
</description>
6 <link>http://people.skolelinux.org/pere/blog/
</link>
10 <title>youtube-dl for nedlasting fra NRK med undertekster - nice free software
</title>
11 <link>http://people.skolelinux.org/pere/blog/youtube_dl_for_nedlasting_fra_NRK_med_undertekster___nice_free_software.html
</link>
12 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/youtube_dl_for_nedlasting_fra_NRK_med_undertekster___nice_free_software.html
</guid>
13 <pubDate>Sat,
28 Apr
2018 10:
30:
00 +
0200</pubDate>
14 <description><p
>I
<a href=
"https://no.wikipedia.org/wiki/VHS
">VHS-kassettenes
</a
>
15 tid var det rett frem å ta vare på et TV-program en ønsket å kunne se
16 senere, uten å være avhengig av at programmet ble sendt på nytt.
17 Kanskje ønsket en å se programmet på hytten der det ikke var
18 TV-signal, eller av andre grunner ha det tilgjengelig for fremtidig
19 fornøyelse. Dette er blitt vanskeligere med introduksjon av
20 digital-TV og webstreaming, der opptak til harddisk er utenfor de
21 flestes kontroll hvis de bruker ufri programvare og bokser kontrollert
22 av andre. Men for NRK her i Norge, finnes det heldigvis flere fri
23 programvare-alternativer, som jeg har
24 <a href=
"http://people.skolelinux.org/pere/blog/Hvordan_enkelt_laste_ned_filmer_fra_NRK.html
">skrevet
</a
>
25 <a href=
"http://people.skolelinux.org/pere/blog/Hvordan_enkelt_laste_ned_filmer_fra_NRK_med_den__nye__l_sningen.html
">om
</a
>
26 <a href=
"http://people.skolelinux.org/pere/blog/Nedlasting_fra_NRK__som_Matroska_med_undertekster.html
">før
</a
>.
27 Så lenge kilden for nedlastingen er lovlig lagt ut på nett (hvilket
28 jeg antar NRK gjør), så er slik lagring til privat bruk også lovlig i
31 <p
>Sist jeg så på saken, i
2016, nevnte jeg at
32 <a href=
"https://rg3.github.com/youtube-dl/
">youtube-dl
</a
> ikke kunne
33 bake undertekster fra NRK inn i videofilene, og at jeg derfor
34 foretrakk andre alternativer. Nylig oppdaget jeg at dette har endret
35 seg. Fordelen med youtube-dl er at den er tilgjengelig direkte fra
36 Linux-distribusjoner som
<a href=
"https://www.debian.org/
">Debian
</a
>
37 og
<a href=
"https://www.ubuntu.com/
">Ubuntu
</a
>, slik at en slipper å
38 finne ut selv hvordan en skal få dem til å virke.
</p
>
40 <p
>For å laste ned et NRK-innslag med undertekster, og få den norske
41 underteksten pakket inn i videofilen, så kan følgende kommando
45 youtube-dl --write-sub --sub-format ttml \
46 --convert-subtitles srt --embed-subs \
47 https://tv.nrk.no/serie/ramm-ferdig-gaa/MUHU11000316/
27-
04-
2018
48 </pre
></p
>
50 <p
>URL-eksemplet er dagens toppsak på tv.nrk.no. Resultatet er en
51 MP4-fil med filmen og undertekster som kan spilles av med VLC. Merk
52 at VLC ikke viser frem undertekster før du aktiverer dem. For å gjøre
53 det, høyreklikk med musa i fremviservinduet, velg menyvalget for
54 undertekst og så norsk språk. Jeg testet også
'--write-auto-sub
',
55 men det kommandolinjeargumentet ser ikke ut til å fungere, så jeg
56 endte opp med settet med argumentlisten over, som jeg fant i en
57 feilrapport i youtube-dl-prosjektets samling over feilrapporter.
</p
>
59 <p
>Denne støtten i youtube-dl gjør det svært enkelt å lagre
60 NRK-innslag, det være seg nyheter, filmer, serier eller dokumentater,
61 for å ha dem tilgjengelig for fremtidig referanse og bruk, uavhengig
62 av hvor lenge innslagene ligger tilgjengelig hos NRK. Så får det ikke
63 hjelpe at NRKs jurister mener at det er
64 <a href=
"http://people.skolelinux.org/pere/blog/Best___ikke_fortelle_noen_at_streaming_er_nedlasting___.html
">vesensforskjellig
65 å legge tilgjengelig for nedlasting og for streaming
</a
>, når det rent
66 teknisk er samme sak.
</p
>
68 <p
>Programmet youtube-dl støtter også en rekke andre nettsteder, se
69 prosjektoversikten for
70 <a href=
"http://rg3.github.io/youtube-dl/supportedsites.html
">en
71 komplett liste
</a
>.
</p
>
76 <title>Oolite, a life in space as vagabond and mercenary - nice free software
</title>
77 <link>http://people.skolelinux.org/pere/blog/Oolite__a_life_in_space_as_vagabond_and_mercenary___nice_free_software.html
</link>
78 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Oolite__a_life_in_space_as_vagabond_and_mercenary___nice_free_software.html
</guid>
79 <pubDate>Sun,
11 Dec
2016 11:
40:
00 +
0100</pubDate>
80 <description><p align=
"center
"><img width=
"70%
" src=
"http://people.skolelinux.org/pere/blog/images/
2016-
12-
11-nice-oolite.png
"/
></p
>
82 <p
>In my early years, I played
83 <a href=
"http://wiki.alioth.net/index.php/Classic_Elite
">the epic game
84 Elite
</a
> on my PC. I spent many months trading and fighting in
85 space, and reached the
'elite
' fighting status before I moved on. The
86 original Elite game was available on Commodore
64 and the IBM PC
87 edition I played had a
64 KB executable. I am still impressed today
88 that the authors managed to squeeze both a
3D engine and details about
89 more than
2000 planet systems across
7 galaxies into a binary so
92 <p
>I have known about
<a href=
"http://www.oolite.org/
">the free
93 software game Oolite inspired by Elite
</a
> for a while, but did not
94 really have time to test it properly until a few days ago. It was
95 great to discover that my old knowledge about trading routes were
96 still valid. But my fighting and flying abilities were gone, so I had
97 to retrain to be able to dock on a space station. And I am still not
98 able to make much resistance when I am attacked by pirates, so I
99 bougth and mounted the most powerful laser in the rear to be able to
100 put up at least some resistance while fleeing for my life. :)
</p
>
102 <p
>When playing Elite in the late eighties, I had to discover
103 everything on my own, and I had long lists of prices seen on different
104 planets to be able to decide where to trade what. This time I had the
106 <a href=
"http://wiki.alioth.net/index.php/Main_Page
">Elite wiki
</a
>,
107 where information about each planet is easily available with common
108 price ranges and suggested trading routes. This improved my ability
109 to earn money and I have been able to earn enough to buy a lot of
110 useful equipent in a few days. I believe I originally played for
111 months before I could get a docking computer, while now I could get it
112 after less then a week.
</p
>
114 <p
>If you like science fiction and dreamed of a life as a vagabond in
115 space, you should try out Oolite. It is available for Linux, MacOSX
116 and Windows, and is included in Debian and derivatives since
2011.
</p
>
118 <p
>As usual, if you use Bitcoin and want to show your support of my
119 activities, please send Bitcoin donations to my address
120 <b
><a href=
"bitcoin:
15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b
">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b
</a
></b
>.
</p
>
125 <title>Coz can help you find bottlenecks in multi-threaded software - nice free software
</title>
126 <link>http://people.skolelinux.org/pere/blog/Coz_can_help_you_find_bottlenecks_in_multi_threaded_software___nice_free_software.html
</link>
127 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Coz_can_help_you_find_bottlenecks_in_multi_threaded_software___nice_free_software.html
</guid>
128 <pubDate>Thu,
11 Aug
2016 12:
00:
00 +
0200</pubDate>
129 <description><p
>This summer, I read a great article
130 "<a href=
"https://www.usenix.org/publications/login/summer2016/curtsinger
">coz:
131 This Is the Profiler You
're Looking For
</a
>" in USENIX ;login: about
132 how to profile multi-threaded programs. It presented a system for
133 profiling software by running experiences in the running program,
134 testing how run time performance is affected by
"speeding up
" parts of
135 the code to various degrees compared to a normal run. It does this by
136 slowing down parallel threads while the
"faster up
" code is running
137 and measure how this affect processing time. The processing time is
138 measured using probes inserted into the code, either using progress
139 counters (COZ_PROGRESS) or as latency meters (COZ_BEGIN/COZ_END). It
140 can also measure unmodified code by measuring complete the program
141 runtime and running the program several times instead.
</p
>
143 <p
>The project and presentation was so inspiring that I would like to
144 get the system into Debian. I
145 <a href=
"https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=
830708">created
146 a WNPP request for it
</a
> and contacted upstream to try to make the
147 system ready for Debian by sending patches. The build process need to
148 be changed a bit to avoid running
'git clone
' to get dependencies, and
149 to include the JavaScript web page used to visualize the collected
150 profiling information included in the source package.
151 But I expect that should work out fairly soon.
</p
>
153 <p
>The way the system work is fairly simple. To run an coz experiment
154 on a binary with debug symbols available, start the program like this:
156 <p
><blockquote
><pre
>
157 coz run --- program-to-run
158 </pre
></blockquote
></p
>
160 <p
>This will create a text file profile.coz with the instrumentation
161 information. To show what part of the code affect the performance
162 most, use a web browser and either point it to
163 <a href=
"http://plasma-umass.github.io/coz/
">http://plasma-umass.github.io/coz/
</a
>
164 or use the copy from git (in the gh-pages branch). Check out this web
165 site to have a look at several example profiling runs and get an idea what the end result from the profile runs look like. To make the
166 profiling more useful you include
&lt;coz.h
&gt; and insert the
167 COZ_PROGRESS or COZ_BEGIN and COZ_END at appropriate places in the
168 code, rebuild and run the profiler. This allow coz to do more
169 targeted experiments.
</p
>
171 <p
>A video published by ACM
172 <a href=
"https://www.youtube.com/watch?v=jE0V-p1odPg
">presenting the
173 Coz profiler
</a
> is available from Youtube. There is also a paper
174 from the
25th Symposium on Operating Systems Principles available
176 <a href=
"https://www.usenix.org/conference/atc16/technical-sessions/presentation/curtsinger
">Coz:
177 finding code that counts with causal profiling
</a
>.
</p
>
179 <p
><a href=
"https://github.com/plasma-umass/coz
">The source code
</a
>
180 for Coz is available from github. It will only build with clang
182 <a href=
"https://gcc.gnu.org/bugzilla/show_bug.cgi?id=
55606">C++
183 feature missing in GCC
</a
>, but I
've submitted
184 <a href=
"https://github.com/plasma-umass/coz/pull/
67">a patch to solve
185 it
</a
> and hope it will be included in the upstream source soon.
</p
>
187 <p
>Please get in touch if you, like me, would like to see this piece
188 of software in Debian. I would very much like some help with the
189 packaging effort, as I lack the in depth knowledge on how to package
190 C++ libraries.
</p
>
195 <title>Creepy, visualise geotagged social media information - nice free software
</title>
196 <link>http://people.skolelinux.org/pere/blog/Creepy__visualise_geotagged_social_media_information___nice_free_software.html
</link>
197 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Creepy__visualise_geotagged_social_media_information___nice_free_software.html
</guid>
198 <pubDate>Sun,
24 Jan
2016 10:
50:
00 +
0100</pubDate>
199 <description><p
>Most people seem not to realise that every time they walk around
200 with the computerised radio beacon known as a mobile phone their
201 position is tracked by the phone company and often stored for a long
202 time (like every time a SMS is received or sent). And if their
203 computerised radio beacon is capable of running programs (often called
204 mobile apps) downloaded from the Internet, these programs are often
205 also capable of tracking their location (if the app requested access
206 during installation). And when these programs send out information to
207 central collection points, the location is often included, unless
208 extra care is taken to not send the location. The provided
209 information is used by several entities, for good and bad (what is
210 good and bad, depend on your point of view). What is certain, is that
211 the private sphere and the right to free movement is challenged and
212 perhaps even eradicated for those announcing their location this way,
213 when they share their whereabouts with private and public
216 <p align=
"center
"><img width=
"70%
" src=
"http://people.skolelinux.org/pere/blog/images/
2016-
01-
24-nice-creepy-desktop-window.png
"></p
>
218 <p
>The phone company logs provide a register of locations to check out
219 when one want to figure out what the tracked person was doing. It is
220 unavailable for most of us, but provided to selected government
221 officials, company staff, those illegally buying information from
222 unfaithful servants and crackers stealing the information. But the
223 public information can be collected and analysed, and a free software
224 tool to do so is called
225 <a href=
"http://www.geocreepy.com/
">Creepy or Cree.py
</a
>. I
226 discovered it when I read
227 <a href=
"http://www.aftenposten.no/kultur/Slik-kan-du-bli-overvaket-pa-Twitter-og-Instagram-uten-a-ane-det-
7787884.html
">an
228 article about Creepy
</a
> in the Norwegian newspaper Aftenposten i
229 November
2014, and decided to check if it was available in Debian.
230 The python program was in Debian, but
231 <a href=
"https://tracker.debian.org/pkg/creepy
">the version in
232 Debian
</a
> was completely broken and practically unmaintained. I
233 uploaded a new version which did not work quite right, but did not
234 have time to fix it then. This Christmas I decided to finally try to
235 get Creepy operational in Debian. Now a fixed version is available in
236 Debian unstable and testing, and almost all Debian specific patches
238 <a href=
"https://github.com/jkakavas/creepy
">upstream
</a
>.
</p
>
240 <p
>The Creepy program visualises geolocation information fetched from
241 Twitter, Instagram, Flickr and Google+, and allow one to get a
242 complete picture of every social media message posted recently in a
243 given area, or track the movement of a given individual across all
244 these services. Earlier it was possible to use the search API of at
245 least some of these services without identifying oneself, but these
246 days it is impossible. This mean that to use Creepy, you need to
247 configure it to log in as yourself on these services, and provide
248 information to them about your search interests. This should be taken
249 into account when using Creepy, as it will also share information
250 about yourself with the services.
</p
>
252 <p
>The picture above show the twitter messages sent from (or at least
253 geotagged with a position from) the city centre of Oslo, the capital
254 of Norway. One useful way to use Creepy is to first look at
255 information tagged with an area of interest, and next look at all the
256 information provided by one or more individuals who was in the area.
257 I tested it by checking out which celebrity provide their location in
258 twitter messages by checkout out who sent twitter messages near a
259 Norwegian TV station, and next could track their position over time,
260 making it possible to locate their home and work place, among other
261 things. A similar technique have been
262 <a href=
"http://www.buzzfeed.com/maxseddon/does-this-soldiers-instagram-account-prove-russia-is-covertl
">used
263 to locate Russian soldiers in Ukraine
</a
>, and it is both a powerful
264 tool to discover lying governments, and a useful tool to help people
265 understand the value of the private information they provide to the
268 <p
>The package is not trivial to backport to Debian Stable/Jessie, as
269 it depend on several python modules currently missing in Jessie (at
270 least python-instagram, python-flickrapi and
271 python-requests-toolbelt).
</p
>
273 <p
>(I have uploaded
274 <a href=
"https://screenshots.debian.net/package/creepy
">the image to
275 screenshots.debian.net
</a
> and licensed it under the same terms as the
276 Creepy program in Debian.)
</p
>
281 <title>OpenALPR, find car license plates in video streams - nice free software
</title>
282 <link>http://people.skolelinux.org/pere/blog/OpenALPR__find_car_license_plates_in_video_streams___nice_free_software.html
</link>
283 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/OpenALPR__find_car_license_plates_in_video_streams___nice_free_software.html
</guid>
284 <pubDate>Wed,
23 Dec
2015 01:
00:
00 +
0100</pubDate>
285 <description><p
>When I was a kid, we used to collect
"car numbers
", as we used to
286 call the car license plate numbers in those days. I would write the
287 numbers down in my little book and compare notes with the other kids
288 to see how many region codes we had seen and if we had seen some
289 exotic or special region codes and numbers. It was a fun game to pass
290 time, as we kids have plenty of it.
</p
>
292 <p
>A few days I came across
293 <a href=
"https://github.com/openalpr/openalpr
">the OpenALPR
294 project
</a
>, a free software project to automatically discover and
295 report license plates in images and video streams, and provide the
296 "car numbers
" in a machine readable format. I
've been looking for
297 such system for a while now, because I believe it is a bad idea that the
298 <a href=
"https://en.wikipedia.org/wiki/Automatic_number_plate_recognition
">automatic
299 number plate recognition
</a
> tool only is available in the hands of
300 the powerful, and want it to be available also for the powerless to
301 even the score when it comes to surveillance and sousveillance. I
302 discovered the developer
303 <a href=
"https://bugs.debian.org/
747509">wanted to get the tool into
304 Debian
</a
>, and as I too wanted it to be in Debian, I volunteered to
305 help him get it into shape to get the package uploaded into the Debian
308 <p
>Today we finally managed to get the package into shape and uploaded
309 it into Debian, where it currently
310 <a href=
"https://ftp-master.debian.org//new/openalpr_2.2
.1-
1.html
">waits
311 in the NEW queue
</a
> for review by the Debian ftpmasters.
</p
>
313 <p
>I guess you are wondering why on earth such tool would be useful
314 for the common folks, ie those not running a large government
315 surveillance system? Well, I plan to put it in a computer on my bike
316 and in my car, tracking the cars nearby and allowing me to be notified
317 when number plates on my watch list are discovered. Another use case
318 was suggested by a friend of mine, who wanted to set it up at his home
319 to open the car port automatically when it discovered the plate on his
320 car. When I mentioned it perhaps was a bit foolhardy to allow anyone
321 capable of placing his license plate number of a piece of cardboard to
322 open his car port, men replied that it was always unlocked anyway. I
323 guess for such use case it make sense. I am sure there are other use
324 cases too, for those with imagination and a vision.
</p
>
326 <p
>If you want to build your own version of the Debian package, check
327 out the upstream git source and symlink ./distros/debian to ./debian/
328 before running
"debuild
" to build the source. Or wait a bit until the
329 package show up in unstable.
</p
>
334 <title>listadmin, the quick way to moderate mailman lists - nice free software
</title>
335 <link>http://people.skolelinux.org/pere/blog/listadmin__the_quick_way_to_moderate_mailman_lists___nice_free_software.html
</link>
336 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/listadmin__the_quick_way_to_moderate_mailman_lists___nice_free_software.html
</guid>
337 <pubDate>Wed,
22 Oct
2014 20:
00:
00 +
0200</pubDate>
338 <description><p
>If you ever had to moderate a mailman list, like the ones on
339 alioth.debian.org, you know the web interface is fairly slow to
340 operate. First you visit one web page, enter the moderation password
341 and get a new page shown with a list of all the messages to moderate
342 and various options for each email address. This take a while for
343 every list you moderate, and you need to do it regularly to do a good
344 job as a list moderator. But there is a quick alternative,
345 <a href=
"http://heim.ifi.uio.no/kjetilho/hacks/#listadmin
">the
346 listadmin program
</a
>. It allow you to check lists for new messages
347 to moderate in a fraction of a second. Here is a test run on two
348 lists I recently took over:
</p
>
350 <p
><blockquote
><pre
>
351 % time listadmin xiph
352 fetching data for pkg-xiph-commits@lists.alioth.debian.org ... nothing in queue
353 fetching data for pkg-xiph-maint@lists.alioth.debian.org ... nothing in queue
359 </pre
></blockquote
></p
>
361 <p
>In
1.7 seconds I had checked two mailing lists and confirmed that
362 there are no message in the moderation queue. Every morning I
363 currently moderate
68 mailman lists, and it normally take around two
364 minutes. When I took over the two pkg-xiph lists above a few days
365 ago, there were
400 emails waiting in the moderator queue. It took me
366 less than
15 minutes to process them all using the listadmin
369 <p
>If you install
370 <a href=
"https://tracker.debian.org/pkg/listadmin
">the listadmin
371 package
</a
> from Debian and create a file
<tt
>~/.listadmin.ini
</tt
>
372 with content like this, the moderation task is a breeze:
</p
>
374 <p
><blockquote
><pre
>
375 username username@example.org
378 discard_if_reason
"Posting restricted to members only. Remove us from your mail list.
"
381 adminurl https://{domain}/mailman/admindb/{list}
382 mailman-list@lists.example.com
385 other-list@otherserver.example.org
386 </pre
></blockquote
></p
>
388 <p
>There are other options to set as well. Check the manual page to
389 learn the details.
</p
>
391 <p
>If you are forced to moderate lists on a mailman installation where
392 the SSL certificate is self signed or not properly signed by a
393 generally accepted signing authority, you can set a environment
394 variable when calling listadmin to disable SSL verification:
</p
>
396 <p
><blockquote
><pre
>
397 PERL_LWP_SSL_VERIFY_HOSTNAME=
0 listadmin
398 </pre
></blockquote
></p
>
400 <p
>If you want to moderate a subset of the lists you take care of, you
401 can provide an argument to the listadmin script like I do in the
402 initial screen dump (the xiph argument). Using an argument, only
403 lists matching the argument string will be processed. This make it
404 quick to accept messages if you notice the moderation request in your
407 <p
>Without the listadmin program, I would never be the moderator of
68
408 mailing lists, as I simply do not have time to spend on that if the
409 process was any slower. The listadmin program have saved me hours of
410 time I could spend elsewhere over the years. It truly is nice free
413 <p
>As usual, if you use Bitcoin and want to show your support of my
414 activities, please send Bitcoin donations to my address
415 <b
><a href=
"bitcoin:
15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b
">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b
</a
></b
>.
</p
>
417 <p
>Update
2014-
10-
27: Added missing
'username
' statement in
418 configuration example. Also, I
've been told that the
419 PERL_LWP_SSL_VERIFY_HOSTNAME=
0 setting do not work for everyone. Not
425 <title>S3QL, a locally mounted cloud file system - nice free software
</title>
426 <link>http://people.skolelinux.org/pere/blog/S3QL__a_locally_mounted_cloud_file_system___nice_free_software.html
</link>
427 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/S3QL__a_locally_mounted_cloud_file_system___nice_free_software.html
</guid>
428 <pubDate>Wed,
9 Apr
2014 11:
30:
00 +
0200</pubDate>
429 <description><p
>For a while now, I have been looking for a sensible offsite backup
430 solution for use at home. My requirements are simple, it must be
431 cheap and locally encrypted (in other words, I keep the encryption
432 keys, the storage provider do not have access to my private files).
433 One idea me and my friends had many years ago, before the cloud
434 storage providers showed up, was to use Google mail as storage,
435 writing a Linux block device storing blocks as emails in the mail
436 service provided by Google, and thus get heaps of free space. On top
437 of this one can add encryption, RAID and volume management to have
438 lots of (fairly slow, I admit that) cheap and encrypted storage. But
439 I never found time to implement such system. But the last few weeks I
440 have looked at a system called
441 <a href=
"https://bitbucket.org/nikratio/s3ql/
">S3QL
</a
>, a locally
442 mounted network backed file system with the features I need.
</p
>
444 <p
>S3QL is a fuse file system with a local cache and cloud storage,
445 handling several different storage providers, any with Amazon S3,
446 Google Drive or OpenStack API. There are heaps of such storage
447 providers. S3QL can also use a local directory as storage, which
448 combined with sshfs allow for file storage on any ssh server. S3QL
449 include support for encryption, compression, de-duplication, snapshots
450 and immutable file systems, allowing me to mount the remote storage as
451 a local mount point, look at and use the files as if they were local,
452 while the content is stored in the cloud as well. This allow me to
453 have a backup that should survive fire. The file system can not be
454 shared between several machines at the same time, as only one can
455 mount it at the time, but any machine with the encryption key and
456 access to the storage service can mount it if it is unmounted.
</p
>
458 <p
>It is simple to use. I
'm using it on Debian Wheezy, where the
459 package is included already. So to get started, run
<tt
>apt-get
460 install s3ql
</tt
>. Next, pick a storage provider. I ended up picking
461 Greenqloud, after reading their nice recipe on
462 <a href=
"https://greenqloud.zendesk.com/entries/
44611757-How-To-Use-S3QL-to-mount-a-StorageQloud-bucket-on-Debian-Wheezy
">how
463 to use S3QL with their Amazon S3 service
</a
>, because I trust the laws
464 in Iceland more than those in USA when it come to keeping my personal
465 data safe and private, and thus would rather spend money on a company
466 in Iceland. Another nice recipe is available from the article
467 <a href=
"http://www.admin-magazine.com/HPC/Articles/HPC-Cloud-Storage
">S3QL
468 Filesystem for HPC Storage
</a
> by Jeff Layton in the HPC section of
469 Admin magazine. When the provider is picked, figure out how to get
470 the API key needed to connect to the storage API. With Greencloud,
471 the key did not show up until I had added payment details to my
474 <p
>Armed with the API access details, it is time to create the file
475 system. First, create a new bucket in the cloud. This bucket is the
476 file system storage area. I picked a bucket name reflecting the
477 machine that was going to store data there, but any name will do.
478 I
'll refer to it as
<tt
>bucket-name
</tt
> below. In addition, one need
479 the API login and password, and a locally created password. Store it
480 all in ~root/.s3ql/authinfo2 like this:
482 <p
><blockquote
><pre
>
484 storage-url: s3c://s.greenqloud.com:
443/bucket-name
485 backend-login: API-login
486 backend-password: API-password
487 fs-passphrase: local-password
488 </pre
></blockquote
></p
>
490 <p
>I create my local passphrase using
<tt
>pwget
50</tt
> or similar,
491 but any sensible way to create a fairly random password should do it.
492 Armed with these details, it is now time to run mkfs, entering the API
493 details and password to create it:
</p
>
495 <p
><blockquote
><pre
>
496 # mkdir -m
700 /var/lib/s3ql-cache
497 # mkfs.s3ql --cachedir /var/lib/s3ql-cache --authfile /root/.s3ql/authinfo2 \
498 --ssl s3c://s.greenqloud.com:
443/bucket-name
500 Enter backend password:
501 Before using S3QL, make sure to read the user
's guide, especially
502 the
'Important Rules to Avoid Loosing Data
' section.
503 Enter encryption password:
504 Confirm encryption password:
505 Generating random encryption key...
506 Creating metadata tables...
516 Compressing and uploading metadata...
517 Wrote
0.00 MB of compressed metadata.
518 #
</pre
></blockquote
></p
>
520 <p
>The next step is mounting the file system to make the storage available.
522 <p
><blockquote
><pre
>
523 # mount.s3ql --cachedir /var/lib/s3ql-cache --authfile /root/.s3ql/authinfo2 \
524 --ssl --allow-root s3c://s.greenqloud.com:
443/bucket-name /s3ql
525 Using
4 upload threads.
526 Downloading and decompressing metadata...
536 Mounting filesystem...
538 Filesystem Size Used Avail Use% Mounted on
539 s3c://s.greenqloud.com:
443/bucket-name
1.0T
0 1.0T
0% /s3ql
541 </pre
></blockquote
></p
>
543 <p
>The file system is now ready for use. I use rsync to store my
544 backups in it, and as the metadata used by rsync is downloaded at
545 mount time, no network traffic (and storage cost) is triggered by
546 running rsync. To unmount, one should not use the normal umount
547 command, as this will not flush the cache to the cloud storage, but
548 instead running the umount.s3ql command like this:
550 <p
><blockquote
><pre
>
553 </pre
></blockquote
></p
>
555 <p
>There is a fsck command available to check the file system and
556 correct any problems detected. This can be used if the local server
557 crashes while the file system is mounted, to reset the
"already
558 mounted
" flag. This is what it look like when processing a working
559 file system:
</p
>
561 <p
><blockquote
><pre
>
562 # fsck.s3ql --force --ssl s3c://s.greenqloud.com:
443/bucket-name
563 Using cached metadata.
564 File system seems clean, checking anyway.
565 Checking DB integrity...
566 Creating temporary extra indices...
567 Checking lost+found...
568 Checking cached objects...
569 Checking names (refcounts)...
570 Checking contents (names)...
571 Checking contents (inodes)...
572 Checking contents (parent inodes)...
573 Checking objects (reference counts)...
574 Checking objects (backend)...
575 ..processed
5000 objects so far..
576 ..processed
10000 objects so far..
577 ..processed
15000 objects so far..
578 Checking objects (sizes)...
579 Checking blocks (referenced objects)...
580 Checking blocks (refcounts)...
581 Checking inode-block mapping (blocks)...
582 Checking inode-block mapping (inodes)...
583 Checking inodes (refcounts)...
584 Checking inodes (sizes)...
585 Checking extended attributes (names)...
586 Checking extended attributes (inodes)...
587 Checking symlinks (inodes)...
588 Checking directory reachability...
589 Checking unix conventions...
590 Checking referential integrity...
591 Dropping temporary indices...
592 Backing up old metadata...
602 Compressing and uploading metadata...
603 Wrote
0.89 MB of compressed metadata.
605 </pre
></blockquote
></p
>
607 <p
>Thanks to the cache, working on files that fit in the cache is very
608 quick, about the same speed as local file access. Uploading large
609 amount of data is to me limited by the bandwidth out of and into my
610 house. Uploading
685 MiB with a
100 MiB cache gave me
305 kiB/s,
611 which is very close to my upload speed, and downloading the same
612 Debian installation ISO gave me
610 kiB/s, close to my download speed.
613 Both were measured using
<tt
>dd
</tt
>. So for me, the bottleneck is my
614 network, not the file system code. I do not know what a good cache
615 size would be, but suspect that the cache should e larger than your
616 working set.
</p
>
618 <p
>I mentioned that only one machine can mount the file system at the
619 time. If another machine try, it is told that the file system is
622 <p
><blockquote
><pre
>
623 # mount.s3ql --cachedir /var/lib/s3ql-cache --authfile /root/.s3ql/authinfo2 \
624 --ssl --allow-root s3c://s.greenqloud.com:
443/bucket-name /s3ql
625 Using
8 upload threads.
626 Backend reports that fs is still mounted elsewhere, aborting.
628 </pre
></blockquote
></p
>
630 <p
>The file content is uploaded when the cache is full, while the
631 metadata is uploaded once every
24 hour by default. To ensure the
632 file system content is flushed to the cloud, one can either umount the
633 file system, or ask S3QL to flush the cache and metadata using
636 <p
><blockquote
><pre
>
637 # s3qlctrl upload-meta /s3ql
638 # s3qlctrl flushcache /s3ql
640 </pre
></blockquote
></p
>
642 <p
>If you are curious about how much space your data uses in the
643 cloud, and how much compression and deduplication cut down on the
644 storage usage, you can use s3qlstat on the mounted file system to get
647 <p
><blockquote
><pre
>
649 Directory entries:
9141
652 Total data size:
22049.38 MB
653 After de-duplication:
21955.46 MB (
99.57% of total)
654 After compression:
21877.28 MB (
99.22% of total,
99.64% of de-duplicated)
655 Database size:
2.39 MB (uncompressed)
656 (some values do not take into account not-yet-uploaded dirty blocks in cache)
658 </pre
></blockquote
></p
>
660 <p
>I mentioned earlier that there are several possible suppliers of
661 storage. I did not try to locate them all, but am aware of at least
662 <a href=
"https://www.greenqloud.com/
">Greenqloud
</a
>,
663 <a href=
"http://drive.google.com/
">Google Drive
</a
>,
664 <a href=
"http://aws.amazon.com/s3/
">Amazon S3 web serivces
</a
>,
665 <a href=
"http://www.rackspace.com/
">Rackspace
</a
> and
666 <a href=
"http://crowncloud.net/
">Crowncloud
</A
>. The latter even
667 accept payment in Bitcoin. Pick one that suit your need. Some of
668 them provide several GiB of free storage, but the prize models are
669 quite different and you will have to figure out what suits you
672 <p
>While researching this blog post, I had a look at research papers
673 and posters discussing the S3QL file system. There are several, which
674 told me that the file system is getting a critical check by the
675 science community and increased my confidence in using it. One nice
677 "<a href=
"http://www.lanl.gov/orgs/adtsc/publications/science_highlights_2013/docs/pg68_69.pdf
">An
678 Innovative Parallel Cloud Storage System using OpenStack’s SwiftObject
679 Store and Transformative Parallel I/O Approach
</a
>" by Hsing-Bung
680 Chen, Benjamin McClelland, David Sherrill, Alfred Torrez, Parks Fields
681 and Pamela Smith. Please have a look.
</p
>
683 <p
>Given my problems with different file systems earlier, I decided to
684 check out the mounted S3QL file system to see if it would be usable as
685 a home directory (in other word, that it provided POSIX semantics when
686 it come to locking and umask handling etc). Running
687 <a href=
"http://people.skolelinux.org/pere/blog/Testing_if_a_file_system_can_be_used_for_home_directories___.html
">my
688 test code to check file system semantics
</a
>, I was happy to discover that
689 no error was found. So the file system can be used for home
690 directories, if one chooses to do so.
</p
>
692 <p
>If you do not want a locally file system, and want something that
693 work without the Linux fuse file system, I would like to mention the
694 <a href=
"http://www.tarsnap.com/
">Tarsnap service
</a
>, which also
695 provide locally encrypted backup using a command line client. It have
696 a nicer access control system, where one can split out read and write
697 access, allowing some systems to write to the backup and others to
698 only read from it.
</p
>
700 <p
>As usual, if you use Bitcoin and want to show your support of my
701 activities, please send Bitcoin donations to my address
702 <b
><a href=
"bitcoin:
15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b
">15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b
</a
></b
>.
</p
>
707 <title>ReactOS Windows clone - nice free software
</title>
708 <link>http://people.skolelinux.org/pere/blog/ReactOS_Windows_clone___nice_free_software.html
</link>
709 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/ReactOS_Windows_clone___nice_free_software.html
</guid>
710 <pubDate>Tue,
1 Apr
2014 12:
10:
00 +
0200</pubDate>
711 <description><p
>Microsoft have announced that Windows XP reaches its end of life
712 2014-
04-
08, in
7 days. But there are heaps of machines still running
713 Windows XP, and depending on Windows XP to run their applications, and
714 upgrading will be expensive, both when it comes to money and when it
715 comes to the amount of effort needed to migrate from Windows XP to a
716 new operating system. Some obvious options (buy new a Windows
717 machine, buy a MacOSX machine, install Linux on the existing machine)
718 are already well known and covered elsewhere. Most of them involve
719 leaving the user applications installed on Windows XP behind and
720 trying out replacements or updated versions. In this blog post I want
721 to mention one strange bird that allow people to keep the hardware and
722 the existing Windows XP applications and run them on a free software
723 operating system that is Windows XP compatible.
</p
>
725 <p
><a href=
"http://www.reactos.org/
">ReactOS
</a
> is a free software
726 operating system (GNU GPL licensed) working on providing a operating
727 system that is binary compatible with Windows, able to run windows
728 programs directly and to use Windows drivers for hardware directly.
729 The project goal is for Windows user to keep their existing machines,
730 drivers and software, and gain the advantages from user a operating
731 system without usage limitations caused by non-free licensing. It is
732 a Windows clone running directly on the hardware, so quite different
733 from the approach taken by
<a href=
"http://www.winehq.org/
">the Wine
734 project
</a
>, which make it possible to run Windows binaries on
737 <p
>The ReactOS project share code with the Wine project, so most
738 shared libraries available on Windows are already implemented already.
739 There is also a software manager like the one we are used to on Linux,
740 allowing the user to install free software applications with a simple
741 click directly from the Internet. Check out the
742 <a href=
"http://www.reactos.org/screenshots
">screen shots on the
743 project web site
</a
> for an idea what it look like (it looks just like
744 Windows before metro).
</p
>
746 <p
>I do not use ReactOS myself, preferring Linux and Unix like
747 operating systems. I
've tested it, and it work fine in a virt-manager
748 virtual machine. The browser, minesweeper, notepad etc is working
749 fine as far as I can tell. Unfortunately, my main test application
750 is the software included on a CD with the Lego Mindstorms NXT, which
751 seem to install just fine from CD but fail to leave any binaries on
752 the disk after the installation. So no luck with that test software.
753 No idea why, but hope someone else figure out and fix the problem.
754 I
've tried the ReactOS Live ISO on a physical machine, and it seemed
755 to work just fine. If you like Windows and want to keep running your
756 old Windows binaries, check it out by
757 <a href=
"http://www.reactos.org/download
">downloading
</a
> the
758 installation CD, the live CD or the preinstalled virtual machine
764 <title>Video DVD reader library / python-dvdvideo - nice free software
</title>
765 <link>http://people.skolelinux.org/pere/blog/Video_DVD_reader_library___python_dvdvideo___nice_free_software.html
</link>
766 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Video_DVD_reader_library___python_dvdvideo___nice_free_software.html
</guid>
767 <pubDate>Fri,
21 Mar
2014 15:
25:
00 +
0100</pubDate>
768 <description><p
>Keeping your DVD collection safe from scratches and curious
769 children fingers while still having it available when you want to see a
770 movie is not straight forward. My preferred method at the moment is
771 to store a full copy of the ISO on a hard drive, and use VLC, Popcorn
772 Hour or other useful players to view the resulting file. This way the
773 subtitles and bonus material are still available and using the ISO is
774 just like inserting the original DVD record in the DVD player.
</p
>
776 <p
>Earlier I used dd for taking security copies, but it do not handle
777 DVDs giving read errors (which are quite a few of them). I
've also
779 <a href=
"http://people.skolelinux.org/pere/blog/Ripping_problematic_DVDs_using_dvdbackup_and_genisoimage.html
">dvdbackup
780 and genisoimage
</a
>, but these days I use the marvellous python library
782 <a href=
"http://bblank.thinkmo.de/blog/new-software-python-dvdvideo
">python-dvdvideo
</a
>
783 written by Bastian Blank. It is
784 <a href=
"http://packages.qa.debian.org/p/python-dvdvideo.html
">in Debian
785 already
</a
> and the binary package name is python3-dvdvideo. Instead
786 of trying to read every block from the DVD, it parses the file
787 structure and figure out which block on the DVD is actually in used,
788 and only read those blocks from the DVD. This work surprisingly well,
789 and I have been able to almost backup my entire DVD collection using
790 this method.
</p
>
792 <p
>So far, python-dvdvideo have failed on between
10 and
793 20 DVDs, which is a small fraction of my collection. The most common
795 <a href=
"https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=
720831">DVDs
796 using UTF-
16 instead of UTF-
8 characters
</a
>, which according to
797 Bastian is against the DVD specification (and seem to cause some
798 players to fail too). A rarer problem is what seem to be inconsistent
799 DVD structures, as the python library
800 <a href=
"https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=
723079">claim
801 there is a overlap between objects
</a
>. An equally rare problem claim
802 <a href=
"https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=
741878">some
803 value is out of range
</a
>. No idea what is going on there. I wish I
804 knew enough about the DVD format to fix these, to ensure my movie
805 collection will stay with me in the future.
</p
>
807 <p
>So, if you need to keep your DVDs safe, back them up using
808 python-dvdvideo. :)
</p
>
813 <title>Free Timetabling Software - nice free software
</title>
814 <link>http://people.skolelinux.org/pere/blog/Free_Timetabling_Software___nice_free_software.html
</link>
815 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Free_Timetabling_Software___nice_free_software.html
</guid>
816 <pubDate>Sat,
7 Jul
2012 09:
50:
00 +
0200</pubDate>
817 <description><p
>Included in
<a href=
"http://www.skolelinux.org/
">Debian Edu /
818 Skolelinux
</a
> is a large collection of end user and school specific
819 software. It is one of the packages not installed by default but
820 provided in the Debian archive for schools to install if they want to,
821 is a system to automatically plan the school time table using
822 information about available teachers, classes and rooms, combined with
823 the list of required courses and how many hours each topic should
824 receive. The software is
826 <a href=
"http://lalescu.ro/liviu/fet/
">named FET
</a
>, and it provide a
827 graphical user interface to input the required information, save the
828 result in a fairly simple XML format, and generate time tables for
829 both teachers and students. It is available both for
830 <a href=
"http://lalescu.ro/liviu/fet/download.html
">Linux, MacOSX and
831 Windows
</a
>.
</p
>
833 <p
>This is
<a href=
"http://lalescu.ro/liviu/fet/features.html
">the
834 feature list
</a
>, liftet from the project web site:
</p
>
838 <li
>FET is free software, licensed under the GNU GPL v2 or later.
839 You can freely use, copy, modify and redistribute it
</li
>
841 <li
>Localized to en_US (US English, default), ar (Arabic), ca
842 (Catalan), da (Danish), de (German), el (Greek), es (Spanish), fa
843 (Persian), fr (French), gl (Galician), he (Hebrew), hu
844 (Hungarian), id (Indonesian), it (Italian), lt (Lithuanian), mk
845 (Macedonian), ms (Malay), nl (Dutch), pl (Polish), pt_BR
846 (Brazilian Portuguese), ro (Romanian), ru (Russian), si (Sinhala),
847 sk (Slovak), sr (Serbian), tr (Turkish), uk (Ukrainian), uz
848 (Uzbek) and vi (Vietnamese) (incompletely for some languages)
851 <li
>Fully automatic generation algorithm, allowing also
852 semi-automatic or manual allocation
</li
>
854 <li
>Platform independent implementation, allowing running on
855 GNU/Linux, Windows, Mac and any system that Qt supports
</li
>
857 <li
>Flexible modular XML format for the input file, allowing editing
858 with an XML editor or by hand (besides FET interface)
</li
>
860 <li
>Import/export from CSV format
</li
>
862 <li
>The resulted timetables are exported into HTML, XML and CSV
865 <li
>Flexible students structure, organized into sets: years, groups
866 and subgroups. FET allows overlapping years and groups and
867 non-overlapping subgroups. You can even define individual students
868 (as separate sets)
</li
>
870 <li
>Each constraint has a weight percentage, from
0.0% to
100.0%
871 (but some special constraints are allowed to have only
100% weight
872 percentage)
</li
>
874 <li
>Limits for the algorithm (all these limits can be increased on
875 demand, as a custom version, because this would require a bit more
878 <li
>Maximum total number of hours (periods) per day:
60</li
>
879 <li
>Maximum number of working days per week:
35</li
>
880 <li
>Maximum total number of teachers:
6000</li
>
881 <li
>Maximum total number of sets of students:
30000</li
>
882 <li
>Maximum total number of subjects:
6000</li
>
883 <li
>Virtually unlimited number of activity tags
</li
>
884 <li
>Maximum number of activities:
30000</li
>
885 <li
>Maximum number of rooms:
6000</li
>
886 <li
>Maximum number of buildings:
6000</li
>
887 <li
>Possibility of adding multiple teachers and
888 students sets for each activity. (it is possible
889 also to have no teachers or no students sets for an
891 <li
>Virtually unlimited number of time constraints
</li
>
892 <li
>Virtually unlimited number of space constraints
</li
>
893 </ul
></li
>
895 <li
>A large and flexible palette of time constraints:
897 <li
>Break periods
</li
>
898 <li
>For teacher(s):
900 <li
>Not available periods
</li
>
901 <li
>Max/min days per week
</li
>
902 <li
>Max gaps per day/week
</li
>
903 <li
>Max hours daily/continuously
</li
>
904 <li
>Min hours daily
</li
>
905 <li
>Max hours daily/continuously with an activity tag
</li
>
907 <li
>Respect working in an hourly interval a max number of
908 days per week
</li
>
909 </ul
></li
>
910 <li
>For students (sets):
912 <li
>Not available periods
</li
>
913 <li
>Begins early (specify max allowed beginnings at second hour)
</li
>
914 <li
>Max gaps per day/week
</li
>
915 <li
>Max hours daily/continuously
</li
>
916 <li
>Min hours daily
</li
>
917 <li
>Max hours daily/continuously with an activity tag
</li
>
919 <li
>Respect working in an hourly interval a max number of
920 days per week
</li
>
921 </ul
></li
>
922 <li
>For an activity or a set of activities/subactivities:
924 <li
>A single preferred starting time
</li
>
925 <li
>A set of preferred starting times
</li
>
926 <li
>A set of preferred time slots
</li
>
927 <li
>Min/max days between them
</li
>
928 <li
>End(s) students day
</li
>
929 <li
>Same starting time/day/hour
</li
>
930 <li
>Occupy max time slots from selection (a complex and
931 flexible constraint, useful in many situations)
</li
>
932 <li
>Consecutive, ordered, grouped (for
2 or
3 (sub)activities)
</li
>
933 <li
>Not overlapping
</li
>
934 <li
>Max simultaneous in selected time slots
</li
>
935 <li
>Min gaps between a set of (sub)activities
</li
>
936 </ul
></li
>
937 </ul
></li
>
939 <li
>A large and flexible palette of space constraints:
941 <li
>Room not available periods
</li
>
942 <li
>For teacher(s):
944 <li
>Home room(s)
</li
>
945 <li
>Max building changes per day/week
</li
>
946 <li
>Min gaps between building changes
</li
>
950 <li
>For students (sets):
952 <li
>Home room(s)
</li
>
953 <li
>Max building changes per day/week
</li
>
954 <li
>Min gaps between building changes
</li
>
957 <li
>Preferred room(s):
959 <li
>For a subject
</li
>
960 <li
>For an activity tag
</li
>
961 <li
>For a subject and an activity tag
</li
>
962 <li
>Individually for a (sub)activity
</li
>
966 <li
>For a set of activities:
968 <li
>Occupy a maximum number of different rooms
</li
>
973 </ul
></p
>
975 <p
>I have not used it myself, as I am not involved in time table
976 planning at a school, but it seem to work fine when I test it. If you
977 need to set up your schools time table, and is tired of doing it
978 manually, check it out.
980 A quick summary on how to use it can be found in
981 <a href=
"http://marvelsoft.co.in/wp/
2012/
03/generate-timetable-for-state-cbse-icse-igcse-schools-free/
">a
982 blog post from MarvelSoft
</a
>. If you find FET useful, please provide
983 a recipe for the Debian Edu project in the
984 <a href=
"http://wiki.debian.org/DebianEdu#Howtos
">Debian Edu HowTo
985 section
</a
>.
</p
>