Petter Reinholdtsen

After many years break from the package and a vain hope that development would be continued by someone else, I finally pulled my acts together this morning and wrapped up a new release of chrpath, the command line tool to modify the rpath and runpath of already compiled ELF programs. The update was triggered by the persistence of Isha Vishnoi at IBM, which needed a new config.guess file to get support for the ppc64le architecture (powerpc 64-bit Little Endian) he is working on. I checked the Debian, Ubuntu and Fedora packages for interesting patches (failed to find the source from OpenSUSE and Mandriva packages), and found quite a few nice fixes. These are the release notes:

New in 0.15 released 2013-11-24:

• Updated config.sub and config.guess from the GNU project to work with newer architectures. Thanks to isha vishnoi for the heads up.
• Updated README with current URLs.
• Added byteswap fix found in Ubuntu, credited Jeremy Kerr and Matthias Klose.
• Added missing help for -k|--keepgoing option, using patch by Petr Machata found in Fedora.
• Rewrite removal of RPATH/RUNPATH to make sure the entry in .dynamic is a NULL terminated string. Based on patch found in Fedora credited Axel Thimm and Christian Krause.

You can download the new version 0.15 from alioth. Please let us know via the Alioth project if something is wrong with the new release. The test suite did not discover any old errors, so if you find a new one, please also include a testsuite check.

I fjor sommer lagde jeg en offentlig tilgjengelig SQL-database over offentlig anbud basert på skraping av HTML-data fra Doffin. Den har stått og gått siden da, og har nå ca. 28000 oppføringer. Jeg oppdaget da jeg tittet innom at noen oppføringer var ikke blitt med, antagelig på grunn av at de fikk tildelt sekvensnummer i Doffin en godt stund før de ble publisert, slik at min nettsideskraper som fortsatte skrapingen der den slapp sist ikke fikk dem med seg. Jeg har fikset litt slik at skraperen nå ser litt tilbake i tid for å se om den har gått glipp av noen oppføringer, og har skrapet på nytt fra midten av september 2013 og fremover. Det bør dermed bli en mer komplett database for kommende måneder. Hvis jeg får tid skal jeg forsøke å skrape "glemte" data fra før midten av september 2013, men tør ikke garantere at det blir prioritert med det første.

Men målet med denne bloggposten er å vise hvordan denne Doffin-databasen kan brukes og integreres med en RSS-leser, slik at en kan la datamaskinen holde et øye med Doffin-annonseringer etter nøkkelord. En kan lage sitt eget søk ved å besøke API-et hos Scraperwiki, velge format rss2 og så legge inn noe ala dette i "query in SQL":

select title, scrapedurl as link, abstract as description,
       publishdate as pubDate from 'swdata'
   where abstract like '%linux%' or title like '%linux%'
   order by seq desc limit 20

Dette vil søke opp alle anbud med ordet linux i oppsummering eller tittel. En kan lage mer avanserte søk hvis en ønsker det. URL-en som dukker opp nederst på siden kan en så gi til sin RSS-leser (jeg bruker akregator selv), og så automatisk få beskjed hvis det dukker opp anbud med det aktuelle nøkkelordet i teksten. Merk at kapasiteten og ytelsen hos Scraperwiki er begrenset, så ikke be RSS-leseren hente ned oftere enn en gang hver dag.

Du lurer kanskje på hva slags informasjon en kan få ut fra denne databasen. Her er to RSS-kilder, med søkeordet "linux", søkeordet "fri programvare" og søkeordet "odf". Det er bare å søke på det en er interessert i. Kopier gjerne datasettet og sett opp din egen tjeneste hvis du vil gjøre mer avanserte søk. SQLite-filen med Doffin-oppføringer kan lastes med fra Scraperwiki for de som vil grave dypere.

Drones, flying robots, are getting more and more popular. The most know ones are the killer drones used by some government to murder people they do not like without giving them the chance of a fair trial, but the technology have many good uses too, from mapping and forest maintenance to photography and search and rescue. I am sure it is just a question of time before "bad drones" are in the hands of private enterprises and not only state criminals but petty criminals too. The drone technology is very useful and very dangerous. To have some control over the use of drones, I agree with Daniel Suarez in his TED talk "The kill decision shouldn't belong to a robot", where he suggested this little gem to keep the good while limiting the bad use of drones:

Each robot and drone should have a cryptographically signed I.D. burned in at the factory that can be used to track its movement through public spaces. We have license plates on cars, tail numbers on aircraft. This is no different. And every citizen should be able to download an app that shows the population of drones and autonomous vehicles moving through public spaces around them, both right now and historically. And civic leaders should deploy sensors and civic drones to detect rogue drones, and instead of sending killer drones of their own up to shoot them down, they should notify humans to their presence. And in certain very high-security areas, perhaps civic drones would snare them and drag them off to a bomb disposal facility.

But notice, this is more an immune system than a weapons system. It would allow us to avail ourselves of the use of autonomous vehicles and drones while still preserving our open, civil society.

The key is that every citizen should be able to read the radio beacons sent from the drones in the area, to be able to check both the government and others use of drones. For such control to be effective, everyone must be able to do it. What should such beacon contain? At least formal owner, purpose, contact information and GPS location. Probably also the origin and target position of the current flight. And perhaps some registration number to be able to look up the drone in a central database tracking their movement. Robots should not have privacy. It is people who need privacy.

Today NUUG and Hackeriet announced our plans to join forces and create a wireless community network in Oslo. The workshop to help people get started will take place Thursday 2013-11-28, but we already are collecting the geolocation of people joining forces to make this happen. We have 9 locations plotted on the map, but we will need more before we have a connected mesh spread across Oslo. If this sound interesting to you, please join us at the workshop. If you are too impatient to wait 15 days, please join us on the IRC channel #nuug on irc.freenode.net right away. :)

Continuing my research into mesh networking, I was recommended to use TP-Link 3040 and 3600 access points as mesh nodes, and the pair I bought arrived on Friday. Here are my notes on how to set up the MR3040 as a mesh node using OpenWrt.

I started by following the instructions on the OpenWRT wiki for TL-MR3040, and downloaded the recommended firmware image (openwrt-ar71xx-generic-tl-mr3040-v2-squashfs-factory.bin) and uploaded it into the original web interface. The flashing went fine, and the machine was available via telnet on the ethernet port. After logging in and setting the root password, ssh was available and I could start to set it up as a batman-adv mesh node.

I started off by reading the instructions from Wireless Africa, which had quite a lot of useful information, but eventually I followed the recipe from the Open Mesh wiki for using batman-adv on OpenWrt. A small snag was the fact that the opkg install kmod-batman-adv command did not work as it should. The batman-adv kernel module would fail to load because its dependency crc16 was not already loaded. I reported the bug to the openwrt project and hope it will be fixed soon. But the problem only seem to affect initial testing of batman-adv, as configuration seem to work when booting from scratch.

The setup is done using files in /etc/config/. I did not bridge the Ethernet and mesh interfaces this time, to be able to hook up the box on my local network and log into it for configuration updates. The following files were changed and look like this after modifying them:

/etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdbf:4c12:3fed::/48'

config interface 'lan'
        option ifname 'eth0'
        option type 'bridge'
        option proto 'dhcp'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option hostname 'tl-mr3040'
        option ip6assign '60'

config interface 'mesh'
        option ifname 'adhoc0'
        option mtu '1528'
        option proto 'batadv'
        option mesh 'bat0'

/etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option channel '11'
        option hwmode '11ng'
        option path 'platform/ar933x_wmac'
        option htmode 'HT20'
        list ht_capab 'SHORT-GI-20'
        list ht_capab 'SHORT-GI-40'
        list ht_capab 'RX-STBC1'
        list ht_capab 'DSSS_CCK-40'
        option disabled '0'

config wifi-iface 'wmesh'
        option device 'radio0'
        option ifname 'adhoc0'
        option network 'mesh'
        option encryption 'none'
        option mode 'adhoc'
        option bssid '02:BA:00:00:00:01'
        option ssid 'meshfx@hackeriet'

/etc/config/batman-adv

config 'mesh' 'bat0'
        option interfaces 'adhoc0'
        option 'aggregated_ogms'
        option 'ap_isolation'
        option 'bonding'
        option 'fragmentation'
        option 'gw_bandwidth'
        option 'gw_mode'
        option 'gw_sel_class'
        option 'log_level'
        option 'orig_interval'
        option 'vis_mode'
        option 'bridge_loop_avoidance'
        option 'distributed_arp_table'
        option 'network_coding'
        option 'hop_penalty'

# yet another batX instance
# config 'mesh' 'bat5'
#       option 'interfaces' 'second_mesh'

The mesh node is now operational. I have yet to test its range, but I hope it is good. I have not yet tested the TP-Link 3600 box still wrapped up in plastic.

If one of the points of switching to a new init system in Debian is to get rid of huge init.d scripts, I doubt we need to switch away from sysvinit and init.d scripts at all. Here is an example init.d script, ie a rewrite of /etc/init.d/rsyslog:

#!/lib/init/init-d-script
### BEGIN INIT INFO
# Provides:          rsyslog
# Required-Start:    $remote_fs $time
# Required-Stop:     umountnfs $time
# X-Stop-After:      sendsigs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: enhanced syslogd
# Description:       Rsyslog is an enhanced multi-threaded syslogd.
#                    It is quite compatible to stock sysklogd and can be 
#                    used as a drop-in replacement.
### END INIT INFO
DESC="enhanced syslogd"
DAEMON=/usr/sbin/rsyslogd

Pretty minimalistic to me... For the record, the original sysv-rc script was 137 lines, and the above is just 15 lines, most of it meta info/comments.

How to do this, you ask? Well, one create a new script /lib/init/init-d-script looking something like this:

#!/bin/sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service

#
do_start()
{
	# Return
	#   0 if daemon has been started
	#   1 if daemon was already running
	#   2 if daemon could not be started
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
		|| return 1
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
		$DAEMON_ARGS \
		|| return 2
	# Add code here, if necessary, that waits for the process to be ready
	# to handle requests from services started subsequently which depend
	# on this one.  As a last resort, sleep for some time.
}

#
# Function that stops the daemon/service
#
do_stop()
{
	# Return
	#   0 if daemon has been stopped
	#   1 if daemon was already stopped
	#   2 if daemon could not be stopped
	#   other if a failure occurred
	start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
	RETVAL="$?"
	[ "$RETVAL" = 2 ] && return 2
	# Wait for children to finish too if this is a daemon that forks
	# and if the daemon is only ever run from this initscript.
	# If the above conditions are not satisfied then add some other code
	# that waits for the process to drop all resources that could be
	# needed by services started subsequently.  A last resort is to
	# sleep for some time.
	start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
	[ "$?" = 2 ] && return 2
	# Many daemons don't delete their pidfiles when they exit.
	rm -f $PIDFILE
	return "$RETVAL"
}

#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
	#
	# If the daemon can reload its configuration without
	# restarting (for example, when it is sent a SIGHUP),
	# then implement that here.
	#
	start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
	return 0
}

SCRIPTNAME=$1
scriptbasename="$(basename $1)"
echo "SN: $scriptbasename"
if [ "$scriptbasename" != "init-d-library" ] ; then
    script="$1"
    shift
    . $script
else
    exit 0
fi

NAME=$(basename $DAEMON)
PIDFILE=/var/run/$NAME.pid

# Exit if the package is not installed
#[ -x "$DAEMON" ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

case "$1" in
  start)
	[ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
	do_start
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  stop)
	[ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
	do_stop
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  status)
	status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $?
	;;
  #reload|force-reload)
	#
	# If do_reload() is not implemented then leave this commented out
	# and leave 'force-reload' as an alias for 'restart'.
	#
	#log_daemon_msg "Reloading $DESC" "$NAME"
	#do_reload
	#log_end_msg $?
	#;;
  restart|force-reload)
	#
	# If the "reload" option is implemented then remove the
	# 'force-reload' alias
	#
	log_daemon_msg "Restarting $DESC" "$NAME"
	do_stop
	case "$?" in
	  0|1)
		do_start
		case "$?" in
			0) log_end_msg 0 ;;
			1) log_end_msg 1 ;; # Old process is still running
			*) log_end_msg 1 ;; # Failed to start
		esac
		;;
	  *)
		# Failed to stop
		log_end_msg 1
		;;
	esac
	;;
  *)
	echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
	exit 3
	;;
esac

:

It is based on /etc/init.d/skeleton, and could be improved quite a lot. I did not really polish the approach, so it might not always work out of the box, but you get the idea. I did not try very hard to optimize it nor make it more robust either.

A better argument for switching init system in Debian than reducing the size of init scripts (which is a good thing to do anyway), is to get boot system that is able to handle the kernel events sensibly and robustly, and do not depend on the boot to run sequentially. The boot and the kernel have not behaved sequentially in years.

The SPICE protocol for remote display access is the preferred solution with oVirt and RedHat Enterprise Virtualization, and I was sad to discover the other day that the browser plugin needed to use these systems seamlessly was missing in Debian. The request for a package was from 2012-04-10 with no progress since 2013-04-01, so I decided to wrap up a package based on the great work from Cajus Pollmeier and put it in a collab-maint maintained git repository to get a package I could use. I would very much like others to help me maintain the package (or just take over, I do not mind), but as no-one had volunteered so far, I just uploaded it to NEW. I hope it will be available in Debian in a few days.

The source is now available from http://anonscm.debian.org/gitweb/?p=collab-maint/spice-xpi.git;a=summary.

The vmdebootstrap program is a a very nice system to create virtual machine images. It create a image file, add a partition table, mount it and run debootstrap in the mounted directory to create a Debian system on a stick. Yesterday, I decided to try to teach it how to make images for Raspberry Pi, as part of a plan to simplify the build system for the FreedomBox project. The FreedomBox project already uses vmdebootstrap for the virtualbox images, but its current build system made multistrap based system for Dreamplug images, and it is lacking support for Raspberry Pi.

Armed with the knowledge on how to build "foreign" (aka non-native architecture) chroots for Raspberry Pi, I dived into the vmdebootstrap code and adjusted it to be able to build armel images on my amd64 Debian laptop. I ended up giving vmdebootstrap five new options, allowing me to replicate the image creation process I use to make Debian Jessie based mesh node images for the Raspberry Pi. First, the --foreign /path/to/binfm_handler option tell vmdebootstrap to call debootstrap with --foreign and to copy the handler into the generated chroot before running the second stage. This allow vmdebootstrap to create armel images on an amd64 host. Next I added two new options --bootsize size and --boottype fstype to teach it to create a separate /boot/ partition with the given file system type, allowing me to create an image with a vfat partition for the /boot/ stuff. I also added a --variant variant option to allow me to create smaller images without the Debian base system packages installed. Finally, I added an option --no-extlinux to tell vmdebootstrap to not install extlinux as a boot loader. It is not needed on the Raspberry Pi and probably most other non-x86 architectures. The changes were accepted by the upstream author of vmdebootstrap yesterday and today, and is now available from the upstream project page.

To use it to build a Raspberry Pi image using Debian Jessie, first create a small script (the customize script) to add the non-free binary blob needed to boot the Raspberry Pi and the APT source list:

#!/bin/sh
set -e # Exit on first error
rootdir="$1"
cd "$rootdir"
cat <<EOF > etc/apt/sources.list
deb http://http.debian.net/debian/ jessie main contrib non-free
EOF
# Install non-free binary blob needed to boot Raspberry Pi.  This
# install a kernel somewhere too.
wget https://raw.github.com/Hexxeh/rpi-update/master/rpi-update \
    -O $rootdir/usr/bin/rpi-update
chmod a+x $rootdir/usr/bin/rpi-update
mkdir -p $rootdir/lib/modules
touch $rootdir/boot/start.elf
chroot $rootdir rpi-update

Next, fetch the latest vmdebootstrap script and call it like this to build the image:

sudo ./vmdebootstrap \
    --variant minbase \
    --arch armel \
    --distribution jessie \
    --mirror http://http.debian.net/debian \
    --image test.img \
    --size 600M \
    --bootsize 64M \
    --boottype vfat \
    --log-level debug \
    --verbose \
    --no-kernel \
    --no-extlinux \
    --root-password raspberry \
    --hostname raspberrypi \
    --foreign /usr/bin/qemu-arm-static \
    --customize `pwd`/customize \
    --package netbase \
    --package git-core \
    --package binutils \
    --package ca-certificates \
    --package wget \
    --package kmod

The list of packages being installed are the ones needed by rpi-update to make the image bootable on the Raspberry Pi, with the exception of netbase, which is needed by debootstrap to find /etc/hosts with the minbase variant. I really wish there was a way to set up an Raspberry Pi using only packages in the Debian archive, but that is not possible as far as I know, because it boots from the GPU using a non-free binary blob.

The build host need debootstrap, kpartx and qemu-user-static and probably a few others installed. I have not checked the complete build dependency list.

The resulting image will not use the hardware floating point unit on the Raspberry PI, because the armel architecture in Debian is not optimized for that use. So the images created will be a bit slower than Raspbian based images.

De siste måneders eksponering av den totale overvåkningen som foregår i den vestlige verden dokumenterer hvor sårbare vi er. Men det slår meg at de som er mest sårbare for dette, myndighetspersoner på alle nivåer, neppe har innsett at de selv er de mest interessante personene å lage profiler på, for å kunne påvirke dem.

For å ta et lite eksempel: Stortingets nettsted, www.stortinget.no (og forsåvidt også data.stortinget.no), inneholder informasjon om det som foregår på Stortinget, og jeg antar de største brukerne av informasjonen der er representanter og rådgivere på Stortinget. Intet overraskende med det. Det som derimot er mer skjult er at Stortingets nettsted bruker Google Analytics, hvilket gjør at enhver som besøker nettsidene der også rapporterer om besøket via Internett-linjer som passerer Sverige, England og videre til USA. Det betyr at informasjon om ethvert besøk på stortingets nettsider kan snappes opp av svensk, britisk og USAs etterretningsvesen. De kan dermed holde et øye med hvilke Stortingssaker stortingsrepresentantene synes er interessante å sjekke ut, og hvilke sider rådgivere og andre på stortinget synes er interessant å besøke, når de gjør det og hvilke andre representanter som sjekker de samme sidene omtrent samtidig. Stortingets bruk av Google Analytics gjør det dermed enkelt for utenlands etteretning å spore representantenes aktivitet og interesse. Hvis noen av representantene bruker Google Mail eller noen andre tjenestene som krever innlogging, så vil det være enda enklere å finne ut nøyaktig hvilke personer som bruker hvilke nettlesere og dermed knytte informasjonen opp til enkeltpersoner på Stortinget.

Og jo flere nettsteder som bruker Google Analytics, jo bedre oversikt over stortingsrepresentantenes lesevaner og interesse blir tilgjengelig for svensk, britisk og USAs etterretning. Hva de kan bruke den informasjonen til overlater jeg til leseren å undres over.

The last few days I have been experimenting with the batman-adv mesh technology. I want to gain some experience to see if it will fit the Freedombox project, and together with my neighbors try to build a mesh network around the park where I live. Batman-adv is a layer 2 mesh system ("ethernet" in other words), where the mesh network appear as if all the mesh clients are connected to the same switch.

My hardware of choice was the Linksys WRT54GL routers I had lying around, but I've been unable to get them working with batman-adv. So instead, I started playing with a Raspberry Pi, and tried to get it working as a mesh node. My idea is to use it to create a mesh node which function as a switch port, where everything connected to the Raspberry Pi ethernet plug is connected (bridged) to the mesh network. This allow me to hook a wifi base station like the Linksys WRT54GL to the mesh by plugging it into a Raspberry Pi, and allow non-mesh clients to hook up to the mesh. This in turn is useful for Android phones using the Serval Project voip client, allowing every one around the playground to phone and message each other for free. The reason is that Android phones do not see ad-hoc wifi networks (they are filtered away from the GUI view), and can not join the mesh without being rooted. But if they are connected using a normal wifi base station, they can talk to every client on the local network.

To get this working, I've created a debian package meshfx-node and a script build-rpi-mesh-node to create the Raspberry Pi boot image. I'm using Debian Jessie (and not Raspbian), to get more control over the packages available. Unfortunately a huge binary blob need to be inserted into the boot image to get it booting, but I'll ignore that for now. Also, as Debian lack support for the CPU features available in the Raspberry Pi, the system do not use the hardware floating point unit. I hope the routing performance isn't affected by the lack of hardware FPU support.

To create an image, run the following with a sudo enabled user after inserting the target SD card into the build machine:

% wget -O build-rpi-mesh-node \
    https://raw.github.com/petterreinholdtsen/meshfx-node/master/build-rpi-mesh-node
% sudo bash -x ./build-rpi-mesh-node > build.log 2>&1
% dd if=/root/rpi/rpi_basic_jessie_$(date +%Y%m%d).img of=/dev/mmcblk0 bs=1M
%

Booting with the resulting SD card on a Raspberry PI with a USB wifi card inserted should give you a mesh node. At least it does for me with a the wifi card I am using. The default mesh settings are the ones used by the Oslo mesh project at Hackeriet, as I mentioned in an earlier blog post about this mesh testing.

The mesh node was not horribly expensive either. I bought everything over the counter in shops nearby. If I had ordered online from the lowest bidder, the price should be significantly lower:

Now my mesh network at home consist of one laptop in the basement connected to my production network, one Raspberry Pi node on the 1th floor that can be seen by my neighbor across the park, and one play-node I use to develop the image building script. And some times I hook up my work horse laptop to the mesh to test it. I look forward to figuring out what kind of latency the batman-adv setup will give, and how much packet loss we will experience around the park. :)