X-Git-Url: http://pere.pagekite.me/gitweb/homepage.git/blobdiff_plain/1d3434513bcbe7dac2e9cff1345baa398760b003..bb38026f1c7da4cc23680c02a66d2912d2925a58:/blog/index.html diff --git a/blog/index.html b/blog/index.html index e96aeb360e..2bdeeec807 100644 --- a/blog/index.html +++ b/blog/index.html @@ -20,76 +20,148 @@
-
Legal to share more than 3000 movies listed on IMDB?
-
18th November 2017
-

A month ago, I blogged about my work to -automatically -check the copyright status of IMDB entries, and try to count the -number of movies listed in IMDB that is legal to distribute on the -Internet. I have continued to look for good data sources, and -identified a few more. The code used to extract information from -various data sources is available in -a -git repository, currently available from github.

- -

So far I have identified 3186 unique IMDB title IDs. To gain -better understanding of the structure of the data set, I created a -histogram of the year associated with each movie (typically release -year). It is interesting to notice where the peaks and dips in the -graph are located. I wonder why they are placed there. I suspect -World War II caused the dip around 1940, but what caused the peak -around 2010?

- -

- -

I've so far identified ten sources for IMDB title IDs for movies in -the public domain or with a free license. This is the statistics -reported when running 'make stats' in the git repository:

- -
-  249 entries (    6 unique) with and   288 without IMDB title ID in free-movies-archive-org-butter.json
- 2301 entries (  540 unique) with and     0 without IMDB title ID in free-movies-archive-org-wikidata.json
-  830 entries (   29 unique) with and     0 without IMDB title ID in free-movies-icheckmovies-archive-mochard.json
- 2109 entries (  377 unique) with and     0 without IMDB title ID in free-movies-imdb-pd.json
-  291 entries (  122 unique) with and     0 without IMDB title ID in free-movies-letterboxd-pd.json
-  144 entries (  135 unique) with and     0 without IMDB title ID in free-movies-manual.json
-  350 entries (    1 unique) with and   801 without IMDB title ID in free-movies-publicdomainmovies.json
-    4 entries (    0 unique) with and   124 without IMDB title ID in free-movies-publicdomainreview.json
-  698 entries (  119 unique) with and   118 without IMDB title ID in free-movies-publicdomaintorrents.json
-    8 entries (    8 unique) with and   196 without IMDB title ID in free-movies-vodo.json
- 3186 unique IMDB title IDs in total
-
- -

The entries without IMDB title ID are candidates to increase the -data set, but might equally well be duplicates of entries already -listed with IMDB title ID in one of the other sources, or represent -movies that lack a IMDB title ID. I've seen examples of all these -situations when peeking at the entries without IMDB title ID. Based -on these data sources, the lower bound for movies listed in IMDB that -are legal to distribute on the Internet is between 3186 and 4713. - -

It would be great for improving the accuracy of this measurement, -if the various sources added IMDB title ID to their metadata. I have -tried to reach the people behind the various sources to ask if they -are interested in doing this, without any replies so far. Perhaps you -can help me get in touch with the people behind VODO, Public Domain -Torrents, Public Domain Movies and Public Domain Review to try to -convince them to add more metadata to their movie entries?

- -

Another way you could help is by adding pages to Wikipedia about -movies that are legal to distribute on the Internet. If such page -exist and include a link to both IMDB and The Internet Archive, the -script used to generate free-movies-archive-org-wikidata.json should -pick up the mapping as soon as wikidata is updates.

+
Streaming the Linux desktop to Kodi using VLC and RTSP
+
12th July 2018
+

A while back, I was asked by a friend how to stream the desktop to +my projector connected to Kodi. I sadly had to admit that I had no +idea, as it was a task I never had tried. Since then, I have been +looking for a way to do so, preferable without much extra software to +install on either side. Today I found a way that seem to kind of +work. Not great, but it is a start.

+ +

I had a look at several approaches, for example +using uPnP +DLNA as described in 2011, but it required a uPnP server, fuse and +local storage enough to store the stream locally. This is not going +to work well for me, lacking enough free space, and it would +impossible for my friend to get working.

+ +

Next, it occurred to me that perhaps I could use VLC to create a +video stream that Kodi could play. Preferably using +broadcast/multicast, to avoid having to change any setup on the Kodi +side when starting such stream. Unfortunately, the only recipe I +could find using multicast used the rtp protocol, and this protocol +seem to not be supported by Kodi.

+ +

On the other hand, the rtsp protocol is working! Unfortunately I +have to specify the IP address of the streaming machine in both the +sending command and the file on the Kodi server. But it is showing my +desktop, and thus allow us to have a shared look on the big screen at +the programs I work on.

+ +

I did not spend much time investigating codeces. I combined the +rtp and rtsp recipes from +the +VLC Streaming HowTo/Command Line Examples, and was able to get +this working on the desktop/streaming end.

+ +
+vlc screen:// --sout \
+  '#transcode{vcodec=mp4v,acodec=mpga,vb=800,ab=128}:rtp{dst=projector.local,port=1234,sdp=rtsp://192.168.11.4:8080/test.sdp}'
+
+ +

I ssh-ed into my Kodi box and created a file like this with the +same IP address:

+ +
+echo rtsp://192.168.11.4:8080/test.sdp \
+  > /storage/videos/screenstream.m3u
+
+ +

Note the 192.168.11.4 IP address is my desktops IP address. As far +as I can tell the IP must be hardcoded for this to work. In other +words, if someone elses machine is going to do the steaming, you have +to update screenstream.m3u on the Kodi machine and adjust the vlc +recipe. To get started, locate the file in Kodi and select the m3u +file while the VLC stream is running. The desktop then show up in my +big screen. :)

+ +

When using the same technique to stream a video file with audio, +the audio quality is really bad. No idea if the problem is package +loss or bad parameters for the transcode. I do not know VLC nor Kodi +enough to tell.

+ +

Update 2018-07-12: Johannes Schauer send me a few +succestions and reminded me about an important step. The "screen:" +input source is only available once the vlc-plugin-access-extra +package is installed on Debian. Without it, you will see this error +message: "VLC is unable to open the MRL 'screen://'. Check the log +for details." He further found that it is possible to drop some parts +of the VLC command line to reduce the amount of hardcoded information. +It is also useful to consider using cvlc to avoid having the VLC +window in the desktop view. In sum, this give us this command line on +the source end + +

+cvlc screen:// --sout \
+  '#transcode{vcodec=mp4v,acodec=mpga,vb=800,ab=128}:rtp{sdp=rtsp://:8080/}'
+
+ +

and this on the Kodi end

+ +

+echo rtsp://192.168.11.4:8080/ \
+  > /storage/videos/screenstream.m3u
+
+ +

Still bad image quality, though. But I did discover that streaming +a DVD using dvdsimple:///dev/dvd as the source had excellent video and +audio quality, so I guess the issue is in the input or transcoding +parts, not the rtsp part. I've tried to change the vb and ab +parameters to use more bandwidth, but it did not make a +difference.

+ +

I further received a suggestion from Einar Haraldseid to try using +gstreamer instead of VLC, and this proved to work great! He also +provided me with the trick to get Kodi to use a multicast stream as +its source. By using this monstrous oneliner, I can stream my desktop +with good video quality in reasonable framerate to the 239.255.0.1 +multicast address on port 1234: + +

+gst-launch-1.0 ximagesrc use-damage=0 ! video/x-raw,framerate=30/1 ! \
+  videoconvert ! queue2 ! \
+  x264enc bitrate=8000 speed-preset=superfast tune=zerolatency qp-min=30 \
+  key-int-max=15 bframes=2 ! video/x-h264,profile=high ! queue2 ! \
+  mpegtsmux alignment=7 name=mux ! rndbuffersize max=1316 min=1316 ! \
+  udpsink host=239.255.0.1 port=1234 ttl-mc=1 auto-multicast=1 sync=0 \
+  pulsesrc device=$(pactl list | grep -A2 'Source #' | \
+    grep 'Name: .*\.monitor$' |  cut -d" " -f2|head -1) ! \
+  audioconvert ! queue2 ! avenc_aac ! queue2 ! mux.
+
+ +

and this on the Kodi end

+ +

+echo udp://@239.255.0.1:1234 \
+  > /storage/videos/screenstream.m3u
+
+ +

Note the trick to pick a valid pulseaudio source. It might not +pick the one you need. This approach will of course lead to trouble +if more than one source uses the same multicast port and address. +Note the ttl-mc=1 setting, which limit the multicast packages to the +local network. If the value is increased, your screen will be +broadcasted further, one network "hop" for each increase (read up on +multicast to learn more. :)!

+ +

Having cracked how to get Kodi to receive multicast streams, I +could use this VLC command to stream to the same multicast address. +The image quality is way better than the rtsp approach, but gstreamer +seem to be doing a better job.

+ +
+cvlc screen:// --sout '#transcode{vcodec=mp4v,acodec=mpga,vb=800,ab=128}:rtp{mux=ts,dst=239.255.0.1,port=1234,sdp=sap}'
+

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: english, opphavsrett. + Tags: debian, english, video.
@@ -97,87 +169,121 @@ activities, please send Bitcoin donations to my address
-
Some notes on fault tolerant storage systems
-
1st November 2017
-

If you care about how fault tolerant your storage is, you might -find these articles and papers interesting. They have formed how I -think of when designing a storage system.

- - +
+  count MIME type
+  ----- -----------------------
+     56 image/jpeg
+     55 image/png
+     49 image/tiff
+     48 image/gif
+     39 image/bmp
+     38 text/plain
+     37 audio/mpeg
+     34 application/ogg
+     33 audio/x-flac
+     32 audio/x-mp3
+     30 audio/x-wav
+     30 audio/x-vorbis+ogg
+     29 image/x-portable-pixmap
+     27 inode/directory
+     27 image/x-portable-bitmap
+     27 audio/x-mpeg
+     26 application/x-ogg
+     25 audio/x-mpegurl
+     25 audio/ogg
+     24 text/html
+
-

Several of these research papers are based on data collected from -hundred thousands or millions of disk, and their findings are eye -opening. The short story is simply do not implicitly trust RAID or -redundant storage systems. Details matter. And unfortunately there -are few options on Linux addressing all the identified issues. Both -ZFS and Btrfs are doing a fairly good job, but have legal and -practical issues on their own. I wonder how cluster file systems like -Ceph do in this regard. After all, there is an old saying, you know -you have a distributed system when the crash of a computer you have -never heard of stops you from getting any work done. The same holds -true if fault tolerance do not work.

- -

Just remember, in the end, it do not matter how redundant, or how -fault tolerant your storage is, if you do not continuously monitor its -status to detect and replace failed disks.

+

The list was created like this using a sid chroot: "cat +/var/lib/apt/lists/*sid*_dep11_Components-amd64.yml.gz| zcat | awk '/^ +- \S+\/\S+$/ {print $2 }' | sort | uniq -c | sort -nr | head -20"

+ +

It is interesting to see how image formats have passed text/plain +as the most announced supported MIME type. These days, thanks to the +AppStream system, if you run into a file format you do not know, and +want to figure out which packages support the format, you can find the +MIME type of the file using "file --mime <filename>", and then +look up all packages announcing support for this format in their +AppStream metadata (XML or .desktop file) using "appstreamcli +what-provides mimetype <mime-type>. For example if you, like +me, want to know which packages support inode/directory, you can get a +list like this:

+ +

+% appstreamcli what-provides mimetype inode/directory | grep Package: | sort
+Package: anjuta
+Package: audacious
+Package: baobab
+Package: cervisia
+Package: chirp
+Package: dolphin
+Package: doublecmd-common
+Package: easytag
+Package: enlightenment
+Package: ephoto
+Package: filelight
+Package: gwenview
+Package: k4dirstat
+Package: kaffeine
+Package: kdesvn
+Package: kid3
+Package: kid3-qt
+Package: nautilus
+Package: nemo
+Package: pcmanfm
+Package: pcmanfm-qt
+Package: qweborf
+Package: ranger
+Package: sirikali
+Package: spacefm
+Package: spacefm
+Package: vifm
+%
+

+ +

Using the same method, I can quickly discover that the Sketchup file +format is not yet supported by any package in Debian:

+ +

+% appstreamcli what-provides mimetype  application/vnd.sketchup.skp
+Could not find component providing 'mimetype::application/vnd.sketchup.skp'.
+%
+

+ +

Yesterday I used it to figure out which packages support the STL 3D +format:

+ +

+% appstreamcli what-provides mimetype  application/sla|grep Package
+Package: cura
+Package: meshlab
+Package: printrun
+%
+

+ +

PS: A new version of Cura was uploaded to Debian yesterday.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: english, raid, sysadmin. + Tags: debian, english, isenkram.
@@ -185,49 +291,83 @@ activities, please send Bitcoin donations to my address
-
Web services for writing academic LaTeX papers as a team
-
31st October 2017
-

I was surprised today to learn that a friend in academia did not -know there are easily available web services available for writing -LaTeX documents as a team. I thought it was common knowledge, but to -make sure at least my readers are aware of it, I would like to mention -these useful services for writing LaTeX documents. Some of them even -provide a WYSIWYG editor to ease writing even further.

- -

There are two commercial services available, -ShareLaTeX and -Overleaf. They are very easy to -use. Just start a new document, select which publisher to write for -(ie which LaTeX style to use), and start writing. Note, these two -have announced their intention to join forces, so soon it will only be -one joint service. I've used both for different documents, and they -work just fine. While -ShareLaTeX is free -software, while the latter is not. According to a -announcement from Overleaf, they plan to keep the ShareLaTeX code -base maintained as free software.

- -But these two are not the only alternatives. -Fidus Writer is another free -software solution with the -source available on github. I have not used it myself. Several -others can be found on the nice -alterntiveTo -web service. - -

If you like Google Docs or Etherpad, but would like to write -documents in LaTeX, you should check out these services. You can even -host your own, if you want to. :)

+
Debian APT upgrade without enough free space on the disk...
+
8th July 2018
+

Quite regularly, I let my Debian Sid/Unstable chroot stay untouch +for a while, and when I need to update it there is not enough free +space on the disk for apt to do a normal 'apt upgrade'. I normally +would resolve the issue by doing 'apt install <somepackages>' to +upgrade only some of the packages in one batch, until the amount of +packages to download fall below the amount of free space available. +Today, I had about 500 packages to upgrade, and after a while I got +tired of trying to install chunks of packages manually. I concluded +that I did not have the spare hours required to complete the task, and +decided to see if I could automate it. I came up with this small +script which I call 'apt-in-chunks':

+ +

+#!/bin/sh
+#
+# Upgrade packages when the disk is too full to upgrade every
+# upgradable package in one lump.  Fetching packages to upgrade using
+# apt, and then installing using dpkg, to avoid changing the package
+# flag for manual/automatic.
+
+set -e
+
+ignore() {
+    if [ "$1" ]; then
+	grep -v "$1"
+    else
+	cat
+    fi
+}
+
+for p in $(apt list --upgradable | ignore "$@" |cut -d/ -f1 | grep -v '^Listing...'); do
+    echo "Upgrading $p"
+    apt clean
+    apt install --download-only -y $p
+    for f in /var/cache/apt/archives/*.deb; do
+	if [ -e "$f" ]; then
+	    dpkg -i /var/cache/apt/archives/*.deb
+	    break
+	fi
+    done
+done
+

+ +

The script will extract the list of packages to upgrade, try to +download the packages needed to upgrade one package, install the +downloaded packages using dpkg. The idea is to upgrade packages +without changing the APT mark for the package (ie the one recording of +the package was manually requested or pulled in as a dependency). To +use it, simply run it as root from the command line. If it fail, try +'apt install -f' to clean up the mess and run the script again. This +might happen if the new packages conflict with one of the old +packages. dpkg is unable to remove, while apt can do this.

+ +

It take one option, a package to ignore in the list of packages to +upgrade. The option to ignore a package is there to be able to skip +the packages that are simply too large to unpack. Today this was +'ghc', but I have run into other large packages causing similar +problems earlier (like TeX).

+ +

Update 2018-07-08: Thanks to Paul Wise, I am aware of two +alternative ways to handle this. The "unattended-upgrades +--minimal-upgrade-steps" option will try to calculate upgrade sets for +each package to upgrade, and then upgrade them in order, smallest set +first. It might be a better option than my above mentioned script. +Also, "aptutude upgrade" can upgrade single packages, thus avoiding +the need for using "dpkg -i" in the script above.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: english. + Tags: debian, english.
@@ -235,292 +375,32 @@ activities, please send Bitcoin donations to my address
-
Locating IMDB IDs of movies in the Internet Archive using Wikidata
-
25th October 2017
-

Recently, I needed to automatically check the copyright status of a -set of The Internet Movie database -(IMDB) entries, to figure out which one of the movies they refer -to can be freely distributed on the Internet. This proved to be -harder than it sounds. IMDB for sure list movies without any -copyright protection, where the copyright protection has expired or -where the movie is lisenced using a permissive license like one from -Creative Commons. These are mixed with copyright protected movies, -and there seem to be no way to separate these classes of movies using -the information in IMDB.

- -

First I tried to look up entries manually in IMDB, -Wikipedia and -The Internet Archive, to get a -feel how to do this. It is hard to know for sure using these sources, -but it should be possible to be reasonable confident a movie is "out -of copyright" with a few hours work per movie. As I needed to check -almost 20,000 entries, this approach was not sustainable. I simply -can not work around the clock for about 6 years to check this data -set.

- -

I asked the people behind The Internet Archive if they could -introduce a new metadata field in their metadata XML for IMDB ID, but -was told that they leave it completely to the uploaders to update the -metadata. Some of the metadata entries had IMDB links in the -description, but I found no way to download all metadata files in bulk -to locate those ones and put that approach aside.

- -

In the process I noticed several Wikipedia articles about movies -had links to both IMDB and The Internet Archive, and it occured to me -that I could use the Wikipedia RDF data set to locate entries with -both, to at least get a lower bound on the number of movies on The -Internet Archive with a IMDB ID. This is useful based on the -assumption that movies distributed by The Internet Archive can be -legally distributed on the Internet. With some help from the RDF -community (thank you DanC), I was able to come up with this query to -pass to the SPARQL interface on -Wikidata: - -

-SELECT ?work ?imdb ?ia ?when ?label
-WHERE
-{
-  ?work wdt:P31/wdt:P279* wd:Q11424.
-  ?work wdt:P345 ?imdb.
-  ?work wdt:P724 ?ia.
-  OPTIONAL {
-        ?work wdt:P577 ?when.
-        ?work rdfs:label ?label.
-        FILTER(LANG(?label) = "en").
-  }
-}
-

- -

If I understand the query right, for every film entry anywhere in -Wikpedia, it will return the IMDB ID and The Internet Archive ID, and -when the movie was released and its English title, if either or both -of the latter two are available. At the moment the result set contain -2338 entries. Of course, it depend on volunteers including both -correct IMDB and The Internet Archive IDs in the wikipedia articles -for the movie. It should be noted that the result will include -duplicates if the movie have entries in several languages. There are -some bogus entries, either because The Internet Archive ID contain a -typo or because the movie is not available from The Internet Archive. -I did not verify the IMDB IDs, as I am unsure how to do that -automatically.

- -

I wrote a small python script to extract the data set from Wikidata -and check if the XML metadata for the movie is available from The -Internet Archive, and after around 1.5 hour it produced a list of 2097 -free movies and their IMDB ID. In total, 171 entries in Wikidata lack -the refered Internet Archive entry. I assume the 70 "disappearing" -entries (ie 2338-2097-171) are duplicate entries.

- -

This is not too bad, given that The Internet Archive report to -contain 5331 -feature films at the moment, but it also mean more than 3000 -movies are missing on Wikipedia or are missing the pair of references -on Wikipedia.

- -

I was curious about the distribution by release year, and made a -little graph to show how the amount of free movies is spread over the -years:

- -

- -

I expect the relative distribution of the remaining 3000 movies to -be similar.

- -

If you want to help, and want to ensure Wikipedia can be used to -cross reference The Internet Archive and The Internet Movie Database, -please make sure entries like this are listed under the "External -links" heading on the Wikipedia article for the movie:

- -

-* {{Internet Archive film|id=FightingLady}}
-* {{IMDb title|id=0036823|title=The Fighting Lady}}
-

- -

Please verify the links on the final page, to make sure you did not -introduce a typo.

- -

Here is the complete list, if you want to correct the 171 -identified Wikipedia entries with broken links to The Internet -Archive: Q1140317, -Q458656, -Q458656, -Q470560, -Q743340, -Q822580, -Q480696, -Q128761, -Q1307059, -Q1335091, -Q1537166, -Q1438334, -Q1479751, -Q1497200, -Q1498122, -Q865973, -Q834269, -Q841781, -Q841781, -Q1548193, -Q499031, -Q1564769, -Q1585239, -Q1585569, -Q1624236, -Q4796595, -Q4853469, -Q4873046, -Q915016, -Q4660396, -Q4677708, -Q4738449, -Q4756096, -Q4766785, -Q880357, -Q882066, -Q882066, -Q204191, -Q204191, -Q1194170, -Q940014, -Q946863, -Q172837, -Q573077, -Q1219005, -Q1219599, -Q1643798, -Q1656352, -Q1659549, -Q1660007, -Q1698154, -Q1737980, -Q1877284, -Q1199354, -Q1199354, -Q1199451, -Q1211871, -Q1212179, -Q1238382, -Q4906454, -Q320219, -Q1148649, -Q645094, -Q5050350, -Q5166548, -Q2677926, -Q2698139, -Q2707305, -Q2740725, -Q2024780, -Q2117418, -Q2138984, -Q1127992, -Q1058087, -Q1070484, -Q1080080, -Q1090813, -Q1251918, -Q1254110, -Q1257070, -Q1257079, -Q1197410, -Q1198423, -Q706951, -Q723239, -Q2079261, -Q1171364, -Q617858, -Q5166611, -Q5166611, -Q324513, -Q374172, -Q7533269, -Q970386, -Q976849, -Q7458614, -Q5347416, -Q5460005, -Q5463392, -Q3038555, -Q5288458, -Q2346516, -Q5183645, -Q5185497, -Q5216127, -Q5223127, -Q5261159, -Q1300759, -Q5521241, -Q7733434, -Q7736264, -Q7737032, -Q7882671, -Q7719427, -Q7719444, -Q7722575, -Q2629763, -Q2640346, -Q2649671, -Q7703851, -Q7747041, -Q6544949, -Q6672759, -Q2445896, -Q12124891, -Q3127044, -Q2511262, -Q2517672, -Q2543165, -Q426628, -Q426628, -Q12126890, -Q13359969, -Q13359969, -Q2294295, -Q2294295, -Q2559509, -Q2559912, -Q7760469, -Q6703974, -Q4744, -Q7766962, -Q7768516, -Q7769205, -Q7769988, -Q2946945, -Q3212086, -Q3212086, -Q18218448, -Q18218448, -Q18218448, -Q6909175, -Q7405709, -Q7416149, -Q7239952, -Q7317332, -Q7783674, -Q7783704, -Q7857590, -Q3372526, -Q3372642, -Q3372816, -Q3372909, -Q7959649, -Q7977485, -Q7992684, -Q3817966, -Q3821852, -Q3420907, -Q3429733, -Q774474

+
The worlds only stone power plant?
+
30th June 2018
+

So far, at least hydro-electric power, coal power, wind power, +solar power, and wood power are well known. Until a few days ago, I +had never heard of stone power. Then I learn about a quarry in a +mountain in +Bremanger i +Norway, where +the +Bremanger Quarry company is extracting stone and dumping the stone +into a shaft leading to its shipping harbour. This downward movement +in this shaft is used to produce electricity. In short, it is using +falling rocks instead of falling water to produce electricity, and +according to its own statements it is producing more power than it is +using, and selling the surplus electricity to the Norwegian power +grid. I find the concept truly amazing. Is this the worlds only +stone power plant?

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: english, opphavsrett. + Tags: english.
@@ -528,30 +408,66 @@ activities, please send Bitcoin donations to my address
-
A one-way wall on the border?
-
14th October 2017
-

I find it fascinating how many of the people being locked inside -the proposed border wall between USA and Mexico support the idea. The -proposal to keep Mexicans out reminds me of -the -propaganda twist from the East Germany government calling the wall -the “Antifascist Bulwark” after erecting the Berlin Wall, claiming -that the wall was erected to keep enemies from creeping into East -Germany, while it was obvious to the people locked inside it that it -was erected to keep the people from escaping.

- -

Do the people in USA supporting this wall really believe it is a -one way wall, only keeping people on the outside from getting in, -while not keeping people in the inside from getting out?

+
Add-on to control the projector from within Kodi
+
26th June 2018
+

My movie playing setup involve Kodi, +OpenELEC (probably soon to be +replaced with LibreELEC) and an +Infocus IN76 video projector. My projector can be controlled via both +a infrared remote controller, and a RS-232 serial line. The vendor of +my projector, InFocus, had been +sensible enough to document the serial protocol in its user manual, so +it is easily available, and I used it some years ago to write +a +small script to control the projector. For a while now, I longed +for a setup where the projector was controlled by Kodi, for example in +such a way that when the screen saver went on, the projector was +turned off, and when the screen saver exited, the projector was turned +on again.

+ +

A few days ago, with very good help from parts of my family, I +managed to find a Kodi Add-on for controlling a Epson projector, and +got in touch with its author to see if we could join forces and make a +Add-on with support for several projectors. To my pleasure, he was +positive to the idea, and we set out to add InFocus support to his +add-on, and make the add-on suitable for the official Kodi add-on +repository.

+ +

The Add-on is now working (for me, at least), with a few minor +adjustments. The most important change I do relative to the master +branch in the github repository is embedding the +pyserial module in +the add-on. The long term solution is to make a "script" type +pyserial module for Kodi, that can be pulled in as a dependency in +Kodi. But until that in place, I embed it.

+ +

The add-on can be configured to turn on the projector when Kodi +starts, off when Kodi stops as well as turn the projector off when the +screensaver start and on when the screesaver stops. It can also be +told to set the projector source when turning on the projector. + +

If this sound interesting to you, check out +the +project github repository. Perhaps you can send patches to +support your projector too? As soon as we find time to wrap up the +latest changes, it should be available for easy installation using any +Kodi instance.

+ +

For future improvements, I would like to add projector model +detection and the ability to adjust the brightness level of the +projector from within Kodi. We also need to figure out how to handle +the cooling period of the projector. My projector refuses to turn on +for 60 seconds after it was turned off. This is not handled well by +the add-on at the moment.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

- Tags: english. + Tags: english, multimedia, video.
@@ -559,52 +475,71 @@ activities, please send Bitcoin donations to my address
-
Generating 3D prints in Debian using Cura and Slic3r(-prusa)
-
9th October 2017
-

At my nearby maker space, -Sonen, I heard the story that it -was easier to generate gcode files for theyr 3D printers (Ultimake 2+) -on Windows and MacOS X than Linux, because the software involved had -to be manually compiled and set up on Linux while premade packages -worked out of the box on Windows and MacOS X. I found this annoying, -as the software involved, -Cura, is free software -and should be trivial to get up and running on Linux if someone took -the time to package it for the relevant distributions. I even found -a request for adding into -Debian from 2013, which had seem some activity over the years but -never resulted in the software showing up in Debian. So a few days -ago I offered my help to try to improve the situation.

- -

Now I am very happy to see that all the packages required by a -working Cura in Debian are uploaded into Debian and waiting in the NEW -queue for the ftpmasters to have a look. You can track the progress -on -the -status page for the 3D printer team.

- -

The uploaded packages are a bit behind upstream, and was uploaded -now to get slots in the NEW -queue while we work up updating the packages to the latest -upstream version.

- -

On a related note, two competitors for Cura, which I found harder -to use and was unable to configure correctly for Ultimaker 2+ in the -short time I spent on it, are already in Debian. If you are looking -for 3D printer "slicers" and want something already available in -Debian, check out -slic3r and -slic3r-prusa. -The latter is a fork of the former.

+
youtube-dl for nedlasting fra NRK med undertekster - nice free software
+
28th April 2018
+

I VHS-kassettenes +tid var det rett frem å ta vare på et TV-program en ønsket å kunne se +senere, uten å være avhengig av at programmet ble sendt på nytt. +Kanskje ønsket en å se programmet på hytten der det ikke var +TV-signal, eller av andre grunner ha det tilgjengelig for fremtidig +fornøyelse. Dette er blitt vanskeligere med introduksjon av +digital-TV og webstreaming, der opptak til harddisk er utenfor de +flestes kontroll hvis de bruker ufri programvare og bokser kontrollert +av andre. Men for NRK her i Norge, finnes det heldigvis flere fri +programvare-alternativer, som jeg har +skrevet +om +før. +Så lenge kilden for nedlastingen er lovlig lagt ut på nett (hvilket +jeg antar NRK gjør), så er slik lagring til privat bruk også lovlig i +Norge.

+ +

Sist jeg så på saken, i 2016, nevnte jeg at +youtube-dl ikke kunne +bake undertekster fra NRK inn i videofilene, og at jeg derfor +foretrakk andre alternativer. Nylig oppdaget jeg at dette har endret +seg. Fordelen med youtube-dl er at den er tilgjengelig direkte fra +Linux-distribusjoner som Debian +og Ubuntu, slik at en slipper å +finne ut selv hvordan en skal få dem til å virke.

+ +

For å laste ned et NRK-innslag med undertekster, og få den norske +underteksten pakket inn i videofilen, så kan følgende kommando +brukes:

-

As usual, if you use Bitcoin and want to show your support of my -activities, please send Bitcoin donations to my address -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+

+youtube-dl --write-sub --sub-format ttml \
+  --convert-subtitles srt --embed-subs \
+  https://tv.nrk.no/serie/ramm-ferdig-gaa/MUHU11000316/27-04-2018
+

+ +

URL-eksemplet er dagens toppsak på tv.nrk.no. Resultatet er en +MP4-fil med filmen og undertekster som kan spilles av med VLC. Merk +at VLC ikke viser frem undertekster før du aktiverer dem. For å gjøre +det, høyreklikk med musa i fremviservinduet, velg menyvalget for +undertekst og så norsk språk. Jeg testet også '--write-auto-sub', +men det kommandolinjeargumentet ser ikke ut til å fungere, så jeg +endte opp med settet med argumentlisten over, som jeg fant i en +feilrapport i youtube-dl-prosjektets samling over feilrapporter.

+ +

Denne støtten i youtube-dl gjør det svært enkelt å lagre +NRK-innslag, det være seg nyheter, filmer, serier eller dokumentater, +for å ha dem tilgjengelig for fremtidig referanse og bruk, uavhengig +av hvor lenge innslagene ligger tilgjengelig hos NRK. Så får det ikke +hjelpe at NRKs jurister mener at det er +vesensforskjellig +å legge tilgjengelig for nedlasting og for streaming, når det rent +teknisk er samme sak.

+ +

Programmet youtube-dl støtter også en rekke andre nettsteder, se +prosjektoversikten for +en +komplett liste.

- Tags: 3d-printer, debian, english. + Tags: multimedia, nice free software, norsk, video, web.
@@ -612,35 +547,27 @@ activities, please send Bitcoin donations to my address
-
Mangler du en skrue, eller har du en skrue løs?
-
4th October 2017
-
Når jeg holder på med ulike prosjekter, så trenger jeg stadig ulike -skruer. Det siste prosjektet jeg holder på med er å lage -en boks til en -HDMI-touch-skjerm som skal brukes med Raspberry Pi. Boksen settes -sammen med skruer og bolter, og jeg har vært i tvil om hvor jeg kan -få tak i de riktige skruene. Clas Ohlson og Jernia i nærheten har -sjelden hatt det jeg trenger. Men her om dagen fikk jeg et fantastisk -tips for oss som bor i Oslo. -Zachariassen Jernvare AS i -Hegermannsgate -23A på Torshov har et fantastisk utvalg, og åpent mellom 09:00 og -17:00. De selger skruer, muttere, bolter, skiver etc i løs vekt, og -så langt har jeg fått alt jeg har lett etter. De har i tillegg det -meste av annen jernvare, som verktøy, lamper, ledninger, etc. Jeg -håper de har nok kunder til å holde det gående lenge, da dette er en -butikk jeg kommer til å besøke ofte. Butikken er et funn å ha i -nabolaget for oss som liker å bygge litt selv. :)

- -

Som vanlig, hvis du bruker Bitcoin og ønsker å vise din støtte til -det jeg driver med, setter jeg pris på om du sender Bitcoin-donasjoner -til min adresse -15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

+
Stortingsflertallet går inn for ny IP-basert sensurinfrastruktur i Norge
+
24th April 2018
+

VG, +Dagbladet +og +NRK +melder i dag at flertallet i Familie- og kulturkomiteen på Stortinget +har bestemt seg for å introdusere en ny sensurinfrastruktur i Norge. +Fra før har Norge en «frivillig» sensurinfrastruktur basert på +DNS-navn, der de største ISP-ene basert på en liste med DNS-navn +forgifter DNS-svar og omdirigerer til et annet IP-nummer enn det som +ligger i DNS. Nå kommer altså IP-basert omdirigering i tillegg. Når +infrastrukturen er på plass, er sensur av IP-adresser redusert et +spørsmål om hvilke IP-nummer som skal blokkeres. Listen over +IP-adresser vil naturligvis endre seg etter hvert som myndighetene +endrer seg. Det er ingen betryggende tanke.

- Tags: norsk. + Tags: norsk, sikkerhet.
@@ -648,64 +575,40 @@ til min adresse
-
Visualizing GSM radio chatter using gr-gsm and Hopglass
-
29th September 2017
-

Every mobile phone announce its existence over radio to the nearby -mobile cell towers. And this radio chatter is available for anyone -with a radio receiver capable of receiving them. Details about the -mobile phones with very good accuracy is of course collected by the -phone companies, but this is not the topic of this blog post. The -mobile phone radio chatter make it possible to figure out when a cell -phone is nearby, as it include the SIM card ID (IMSI). By paying -attention over time, one can see when a phone arrive and when it leave -an area. I believe it would be nice to make this information more -available to the general public, to make more people aware of how -their phones are announcing their whereabouts to anyone that care to -listen.

- -

I am very happy to report that we managed to get something -visualizing this information up and running for -Oslo Skaperfestival 2017 -(Oslo Makers Festival) taking place today and tomorrow at Deichmanske -library. The solution is based on the -simple -recipe for listening to GSM chatter I posted a few days ago, and -will show up at the stand of Åpen -Sone from the Computer Science department of the University of -Oslo. The presentation will show the nearby mobile phones (aka -IMSIs) as dots in a web browser graph, with lines to the dot -representing mobile base station it is talking to. It was working in -the lab yesterday, and was moved into place this morning.

- -

We set up a fairly powerful desktop machine using Debian -Buster/Testing with several (five, I believe) RTL2838 DVB-T receivers -connected and visualize the visible cell phone towers using an -English version of -Hopglass. A fairly powerfull machine is needed as the -grgsm_livemon_headless processes from -gr-gsm converting -the radio signal to data packages is quite CPU intensive.

- -

The frequencies to listen to, are identified using a slightly -patched scan-and-livemon (to set the --args values for each receiver), -and the Hopglass data is generated using the -patches -in my meshviewer-output branch. For some reason we could not get -more than four SDRs working. There is also a geographical map trying -to show the location of the base stations, but I believe their -coordinates are hardcoded to some random location in Germany, I -believe. The code should be replaced with code to look up location in -a text file, a sqlite database or one of the online databases -mentioned in -the github -issue for the topic. - -

If this sound interesting, visit the stand at the festival!

+
En grunn til å takke nei til usikker digital post
+
2nd April 2018
+

Brevpost er beskyttet av straffelovens bestemmelse som gjør det +kriminelt å åpne andres brev. Dette følger av (ny) straffelovs +§ 205 +(Krenkelse av retten til privat kommunikasjon), som sier at «Med +bot eller fengsel inntil 2 år straffes den som uberettiget ... c) +åpner brev eller annen lukket skriftlig meddelelse som er adressert +til en annen, eller på annen måte skaffer seg uberettiget tilgang til +innholdet.» Dette gjelder såvel postbud som alle andre som har +befatning med brevet etter at avsender har befatning med et lukket +brev. Tilsvarende står også tidligere utgaver av den norske +straffeloven.

+ +

Når en registrerer seg på usikre digitale postkasseløsningene, som +f.eks. Digipost og e-Boks, og slik tar disse i bruk, så gir en de som +står bak løsningene tillatelse til å åpne sine brev. Dette er +nødvendig for at innholdet i digital post skal kunne vises frem til +mottaker via tjenestens websider. Dermed gjelder ikke straffelovens +paragraf om forbud mot å åpne brev, da tilgangen ikke lenger er +uberettiget. En gir altså fremmede tilgang til å lese sin +korrespondanse. I tillegg vil bruk av slike usikre digitale +postbokser føre til at det blir registrert når du leser brevene, hvor +du befinner deg (vha. tilkoblingens IP-adresse), hvilket utstyr du +bruker og en rekke annen personlig informasjon som ikke er +tilgjengelig når papirpost brukes. Jeg foretrekker at det er +lovmessig beskyttelse av min korrespondanse, som jo inneholder privat +og personlig informasjon. Det bidrar til litt bedre vern av personlig +integritet i dagens norske samfunn.

- Tags: debian, english, personvern, surveillance. + Tags: norsk, personvern, surveillance.
@@ -713,83 +616,34 @@ issue for the topic.
-
Easier recipe to observe the cell phones around you
-
24th September 2017
-

A little more than a month ago I wrote -how -to observe the SIM card ID (aka IMSI number) of mobile phones talking -to nearby mobile phone base stations using Debian GNU/Linux and a -cheap USB software defined radio, and thus being able to pinpoint -the location of people and equipment (like cars and trains) with an -accuracy of a few kilometer. Since then we have worked to make the -procedure even simpler, and it is now possible to do this without any -manual frequency tuning and without building your own packages.

- -

The gr-gsm -package is now included in Debian testing and unstable, and the -IMSI-catcher code no longer require root access to fetch and decode -the GSM data collected using gr-gsm.

- -

Here is an updated recipe, using packages built by Debian and a git -clone of two python scripts:

- -
    - -
  1. Start with a Debian machine running the Buster version (aka - testing).
    2. - -
  2. Run 'apt install gr-gsm python-numpy python-scipy - python-scapy' as root to install required packages.
    3. - -
  3. Fetch the code decoding GSM packages using 'git clone - github.com/Oros42/IMSI-catcher.git'.
    4. - -
  4. Insert USB software defined radio supported by GNU Radio.
    5. - -
  5. Enter the IMSI-catcher directory and run 'python - scan-and-livemon' to locate the frequency of nearby base - stations and start listening for GSM packages on one of them.
    6. - -
  6. Enter the IMSI-catcher directory and run 'python - simple_IMSI-catcher.py' to display the collected information.
    7. - -
- -

Note, due to a bug somewhere the scan-and-livemon program (actually -its underlying -program grgsm_scanner) do not work with the HackRF radio. It does -work with RTL 8232 and other similar USB radio receivers you can get -very cheaply -(for example -from ebay), so for now the solution is to scan using the RTL radio -and only use HackRF for fetching GSM data.

- -

As far as I can tell, a cell phone only show up on one of the -frequencies at the time, so if you are going to track and count every -cell phone around you, you need to listen to all the frequencies used. -To listen to several frequencies, use the --numrecv argument to -scan-and-livemon to use several receivers. Further, I am not sure if -phones using 3G or 4G will show as talking GSM to base stations, so -this approach might not see all phones around you. I typically see -0-400 IMSI numbers an hour when looking around where I live.

- -

I've tried to run the scanner on a -Raspberry Pi 2 and 3 -running Debian Buster, but the grgsm_livemon_headless process seem -to be too CPU intensive to keep up. When GNU Radio print 'O' to -stdout, I am told there it is caused by a buffer overflow between the -radio and GNU Radio, caused by the program being unable to read the -GSM data fast enough. If you see a stream of 'O's from the terminal -where you started scan-and-livemon, you need a give the process more -CPU power. Perhaps someone are able to optimize the code to a point -where it become possible to set up RPi3 based GSM sniffers? I tried -using Raspbian instead of Debian, but there seem to be something wrong -with GNU Radio on raspbian, causing glibc to abort().

+
Self-appointed leaders of the Free World
+
22nd March 2018
+

The leaders of the worlds have started to congratulate the +re-elected Russian head of state, and this causes some criticism. I +am though a little fascinated by a comment from USA senator John McCain, +sited +by The Hill and others: + +

+

"An American president does not lead the Free World by +congratulating dictators on winning sham elections."

+

+ +

While I totally agree with the senator here, the way the quote is +phrased make me suspect that he is unaware of the simple fact that USA +have not lead the Free World since at least before its government +kidnapped a +completely innocent Canadian citizen in transit on his way home to +Canada via John F. Kennedy International Airport in September 2002 and +sent him to be tortured in Syria for a year.

+ +

USA might be running ahead, but the path they are taking is not the +one taken by any Free World.

- Tags: debian, english, personvern, surveillance. + Tags: english.
@@ -797,54 +651,63 @@ with GNU Radio on raspbian, causing glibc to abort().

-
Datalagringsdirektivet kaster skygger over Høyre og Arbeiderpartiet
-
7th September 2017
-

For noen dager siden publiserte Jon Wessel-Aas en bloggpost om -«Konklusjonen om datalagring som -EU-kommisjonen ikke ville at vi skulle få se». Det er en -interessant gjennomgang av EU-domstolens syn på snurpenotovervåkning -av befolkningen, som er klar på at det er i strid med -EU-lovgivingen.

- -

Valgkampen går for fullt i Norge, og om noen få dager er siste -frist for å avgi stemme. En ting er sikkert, Høyre og Arbeiderpartiet -får ikke min stemme -denne -gangen heller. Jeg har ikke glemt at de tvang igjennom loven som -skulle pålegge alle data- og teletjenesteleverandører å overvåke alle -sine kunder. En lov som er vedtatt, og aldri opphevet igjen.

- -

Det er tydelig fra diskusjonen rundt grenseløs digital overvåkning -(eller "Digital Grenseforsvar" som det kalles i Orvellisk nytale) at -hverken Høyre og Arbeiderpartiet har noen prinsipielle sperrer mot å -overvåke hele befolkningen, og diskusjonen så langt tyder på at flere -av de andre partiene heller ikke har det. Mange av -de som stemte -for Datalagringsdirektivet i Stortinget (64 fra Arbeiderpartiet, -25 fra Høyre) er fortsatt aktive og argumenterer fortsatt for å radere -vekk mer av innbyggernes privatsfære.

- -

Når myndighetene demonstrerer sin mistillit til folket, tror jeg -folket selv bør legge litt innsats i å verne sitt privatliv, ved å ta -i bruk ende-til-ende-kryptert kommunikasjon med sine kjente og kjære, -og begrense hvor mye privat informasjon som deles med uvedkommende. -Det er jo ingenting som tyder på at myndighetene kommer til å være vår -privatsfære. -Det -er mange muligheter. Selv har jeg litt sans for -Ring, som er basert på p2p-teknologi -uten sentral kontroll, er fri programvare, og støtter meldinger, tale -og video. Systemet er tilgjengelig ut av boksen fra -Debian og -Ubuntu, og det -finnes pakker for Android, MacOSX og Windows. Foreløpig er det få -brukere med Ring, slik at jeg også bruker -Signal som nettleserutvidelse.

+
Facebooks ability to sell your personal information is the real Cambridge Analytica scandal
+
21st March 2018
+

So, Cambridge Analytica is getting some well deserved criticism for +(mis)using information it got from Facebook about 50 million people, +mostly in the USA. What I find a bit surprising, is how little +criticism Facebook is getting for handing the information over to +Cambridge Analytica and others in the first place. And what about the +people handing their private and personal information to Facebook? +And last, but not least, what about the government offices who are +handing information about the visitors of their web pages to Facebook? +No-one who looked at the terms of use of Facebook should be surprised +that information about peoples interests, political views, personal +lifes and whereabouts would be sold by Facebook.

+ +

What I find to be the real scandal is the fact that Facebook is +selling your personal information, not that one of the buyers used it +in a way Facebook did not approve when exposed. It is well known that +Facebook is selling out their users privacy, but a scandal +nevertheless. Of course the information provided to them by Facebook +would be misused by one of the parties given access to personal +information about the millions of Facebook users. Collected +information will be misused sooner or later. The only way to avoid +such misuse, is to not collect the information in the first place. If +you do not want Facebook to hand out information about yourself for +the use and misuse of its customers, do not give Facebook the +information.

+ +

Personally, I would recommend to completely remove your Facebook +account, and take back some control of your personal information. +According +to The Guardian, it is a bit hard to find out how to request +account removal (and not just 'disabling'). You need to +visit +a specific Facebook page and click on 'let us know' on that page +to get to the +real account deletion screen. Perhaps something to consider? I +would not trust the information to really be deleted (who knows, +perhaps NSA, GCHQ and FRA already got a copy), but it might reduce the +exposure a bit.

+ +

If you want to learn more about the capabilities of Cambridge +Analytica, I recommend to see the video recording of the one hour talk +Paul-Olivier Dehaye gave to NUUG last april about + +Data collection, psychometric profiling and their impact on +politics.

+ +

And if you want to communicate with your friends and loved ones, +use some end-to-end encrypted method like +Signal or +Ring, and stop sharing your private +messages with strangers like Facebook and Google.

- Tags: dld, norsk, personvern, stortinget, surveillance, valg. + Tags: english, personvern.
@@ -859,6 +722,23 @@ brukere med Ring, slik at jeg også bruker

Archive