Petter Reinholdtsen

Automatic upgrade testing from Lenny to Squeeze
2010-06-11 22:50

The last few days I have done some upgrade testing in Debian, to see if the upgrade from Lenny to Squeeze will go smoothly. A few bugs have been discovered and reported in the process (#585410 in nagios3-cgi, #584879 already fixed in enscript and #584861 in kdebase-workspace-data), and to get a more regular testing going on, I am working on a script to automate the test.

The idea is to create a Lenny chroot and use tasksel to install a Gnome or KDE desktop installation inside the chroot before upgrading it. To ensure no services are started in the chroot, a policy-rc.d script is inserted. To make sure tasksel believe it is to install a desktop on a laptop, the tasksel tests are replaced in the chroot (only acceptable because this is a throw-away chroot).

A naive upgrade from Lenny to Squeeze using aptitude dist-upgrade currently always fail because udev refuses to upgrade with the kernel in Lenny, so to avoid that problem the file /etc/udev/kernel-upgrade is created. The bug report #566000 make me suspect this problem do not trigger in a chroot, but I touch the file anyway to make sure the upgrade go well. Testing on virtual and real hardware have failed me because of udev so far, and creating this file do the trick in such settings anyway. This is a known issue and the current udev behaviour is intended by the udev maintainer because he lack the resources to rewrite udev to keep working with old kernels or something like that. I really wish the udev upstream would keep udev backwards compatible, to avoid such upgrade problem, but given that they fail to do so, I guess documenting the way out of this mess is the best option we got for Debian Squeeze.

Anyway, back to the task at hand, testing upgrades. This test script, which I call upgrade-test for now, is doing the trick:

#!/bin/sh
set -ex

if [ "$1" ] ; then
    desktop=$1
else
    desktop=gnome
fi

from=lenny
to=squeeze

exec < /dev/null
unset LANG
mirror=http://ftp.skolelinux.org/debian
tmpdir=chroot-$from-upgrade-$to-$desktop
fuser -mv .
debootstrap $from $tmpdir $mirror
chroot $tmpdir aptitude update
cat > $tmpdir/usr/sbin/policy-rc.d <<EOF
#!/bin/sh
exit 101
EOF
chmod a+rx $tmpdir/usr/sbin/policy-rc.d
exit_cleanup() {
    umount $tmpdir/proc
}
mount -t proc proc $tmpdir/proc
# Make sure proc is unmounted also on failure
trap exit_cleanup EXIT INT

chroot $tmpdir aptitude -y install debconf-utils

# Make sure tasksel autoselection trigger.  It need the test scripts
# to return the correct answers.
echo tasksel tasksel/desktop multiselect $desktop | \
    chroot $tmpdir debconf-set-selections

# Include the desktop and laptop task
for test in desktop laptop ; do
    echo > $tmpdir/usr/lib/tasksel/tests/$test <<EOF
#!/bin/sh
exit 2
EOF
    chmod a+rx $tmpdir/usr/lib/tasksel/tests/$test
done

DEBIAN_FRONTEND=noninteractive
DEBIAN_PRIORITY=critical
export DEBIAN_FRONTEND DEBIAN_PRIORITY
chroot $tmpdir tasksel --new-install

echo deb $mirror $to main > $tmpdir/etc/apt/sources.list
chroot $tmpdir aptitude update
touch $tmpdir/etc/udev/kernel-upgrade
chroot $tmpdir aptitude -y dist-upgrade
fuser -mv

I suspect it would be useful to test upgrades with both apt-get and with aptitude, but I have not had time to look at how they behave differently so far. I hope to get a cron job running to do the test regularly and post the result on the web. The Gnome upgrade currently work, while the KDE upgrade fail because of the bug in kdebase-workspace-data

I am not quite sure what kind of extract from the huge upgrade logs (KDE 167 KiB, Gnome 516 KiB) it make sense to include in this blog post, so I will refrain from trying. I can report that for Gnome, aptitude report 760 packages upgraded, 448 newly installed, 129 to remove and 1 not upgraded and 1024MB need to be downloaded while for KDE the same numbers are 702 packages upgraded, 507 newly installed, 193 to remove and 0 not upgraded and 1117MB need to be downloaded

I am very happy to notice that the Gnome desktop + laptop upgrade is able to migrate to dependency based boot sequencing and parallel booting without a hitch. Was unsure if there were still bugs with packages failing to clean up their obsolete init.d script during upgrades, and no such problem seem to affect the Gnome desktop+laptop packages.

Tags: bootsystem, debian, debian edu, english.
Skolelinux er laget for sentraldrifting, naturligvis
2010-06-09 12:30

Det er merkelig hvordan myter om Skolelinux overlever. En slik myte er at Skolelinux ikke kan sentraldriftes og ha sentralt plasserte tjenermaskiner. I siste Computerworld Norge er IT-sjef Viggo Billdal i Steinkjer intervjuet, og forteller uten blygsel:

Vi hadde Skolelinux, men det har vi sluttet med. Vi testet om det lønte seg med Microsoft eller en åpen plattform. Vi fant ut at Microsoft egentlig var totalt sett bedre egnet. Det var store driftskostnader med Skolelinux, blant annet på grunn av desentraliserte servere. Det var komplisert, så vi gikk vekk fra det og bruker nå bare Windows.

En rask sjekk mot den norske brukerlista i Skolelinuxprosjektet forteller at Steinkjers forsøk foregikk fram til 2004/2005, og at Røysing skole i Steinkjer skal ha vært svært fornøyd med Skolelinux men at kommunen overkjørte skolen og krevde at de gikk over til Windows. Et søk på nettet sendte meg til Dagens IT nr. 18 2005 hvor en kan lese på side 18:

Inge Tømmerås ved Røysing skole i Steinkjer kjører ennå Microsoft, men forteller at kompetanseutfordringen med Skolelinux ikke var så stor. ­ Jeg syntes Skolelinux var utrolig lett å drifte uten forkunnskaper. Men man må jo selvsagt ha tilgang på ekstern kompetanse til installasjoner og maskinvarefeil, sier Tømmerås.

Som systemarkitekten bak Skolelinux, kan jeg bare riste på hodet over påstanden om at Skolelinux krever desentraliserte tjenere. Skolelinux-arkitekturen er laget for sentralisert drift og plassering av tjenerne lokalt eller sentralt alt etter behov og nettkapasitet. Den er modellert på nettverks- og tjenerløsningen som brukes på Universitetet i Tromsø og Oslo, der jeg jobber med utvikling av driftstjenester. Dette er det heldigvis noen som har fått med seg, og jeg er glad for å kunne sitere fra en kommentar på den overnevnte artikkelen. Min venn og gamle kollega Sturle Sunde forteller der:

I Flora kommune køyrer vi Skulelinux på skular med alt frå 15 til meir enn 500 elevar. Dei store skulane har eigen tenar, for det er mest praktisk. Eg, som er driftsansvarleg for heile nettet, ser sjeldan dei tenarane fysisk, men at dei står der gjer skulane mindre avhengige av eksterne linjer som er trege eller dyre. Dei minste skulane har ikkje eigen tenar. Å bruke sentral tenar er heller ikkje noko problem. Småskulane klarar seg fint med 1 mbit-linje til ein sentral tenar eller tenaren på ein større skule.

Det beste med Skulelinux er halvtjukke klientar. Dei treng ikkje harddisk og brukar minimalt med ressursar på tenaren fordi dei køyrer programma lokalt. Eit klasserom med 30 sju-åtte år gamle maskiner har mykje meir CPU og RAM totalt enn nokon moderne tenar til under millionen. Det trengst to kommandoar på den sentrale tenaren for å oppdatere alle klientane, både tynne og halvtjukke. Vi har ingen problem med diskar som ryk heller, som var eit problem før fordi elevane sat og sparka i maskinene. Og dei krev lite bandbreidde i nettet, so det er fullt mogleg å køyre slike på småskular med trege linjer mot tenaren på ein større skule.

Flora kommune har nesten 800 Linux-maskiner i sitt skulenett, og ein person som tek seg av drift av heile nettet, inkludert tenarar, klientar, operativsystem, programvare, heimekontorløysing og administrasjon av brukarar.

No skal det seiast at vi ikkje køyrer rein Skulelinux ut av boksen. Vi har gjort ein del tilpassingar mot noko Novell-greier som var der frå før, og som har komplisert installasjonen vår. Etter at oppsettet var gjort har løysinga vore stabil og kravd minimalt med arbeid.

Jeg vet at Narvik, Harstad og Oslo er kommuner der Skolelinux sentraldriftes med sentrale tjenere. Det forteller meg at Steinkjers IT-sjef neppe bør skylde på Skolelinux-løsningen for sine 5 år gamle minner.

Tags: debian edu, norsk, nuug.
Upstart or sysvinit - as init.d scripts see it
2010-06-06 23:55

If Debian is to migrate to upstart on Linux, I expect some init.d scripts to migrate (some of) their operations to upstart job while keeping the init.d for hurd and kfreebsd. The packages with such needs will need a way to get their init.d scripts to behave differently when used with sysvinit and with upstart. Because of this, I had a look at the environment variables set when a init.d script is running under upstart, and when it is not.

With upstart, I notice these environment variables are set when a script is started from rcS.d/ (ignoring some irrelevant ones like COLUMNS):

DEFAULT_RUNLEVEL=2
previous=N
PREVLEVEL=
RUNLEVEL=
runlevel=S
UPSTART_EVENTS=startup
UPSTART_INSTANCE=
UPSTART_JOB=rc-sysinit

With sysvinit, these environment variables are set for the same script.

INIT_VERSION=sysvinit-2.88
previous=N
PREVLEVEL=N
RUNLEVEL=S
runlevel=S

The RUNLEVEL and PREVLEVEL environment variables passed on from sysvinit are not set by upstart. Not sure if it is intentional or not to not be compatible with sysvinit in this regard.

For scripts needing to behave differently when upstart is used, looking for the UPSTART_JOB environment variable seem to be a good choice.

Tags: bootsystem, debian, english.
A manual for standards wars...
2010-06-06 14:15

Via the blog of Rob Weir I came across the very interesting essay named The Art of Standards Wars (PDF 25 pages). I recommend it for everyone following the standards wars of today.

Tags: debian, debian edu, english, standard.
Sitesummary tip: Listing computer hardware models used at site
2010-06-03 12:05

When using sitesummary at a site to track machines, it is possible to get a list of the machine types in use thanks to the DMI information extracted from each machine. The script to do so is included in the sitesummary package, and here is example output from the Skolelinux build servers:

maintainer:~# /usr/lib/sitesummary/hardware-model-summary
  vendor                    count
  Dell Computer Corporation     1
    PowerEdge 1750              1
  IBM                           1
    eserver xSeries 345 -[8670M1X]-     1
  Intel                         2
  [no-dmi-info]                 3
maintainer:~#

The quality of the report depend on the quality of the DMI tables provided in each machine. Here there are Intel machines without model information listed with Intel as vendor and mo model, and virtual Xen machines listed as [no-dmi-info]. One can add -l as a command line option to list the individual machines.

A larger list is available from the the city of Narvik, which uses Skolelinux on all their shools and also provide the basic sitesummary report publicly. In their report there are ~1400 machines. I know they use both Ubuntu and Skolelinux on their machines, and as sitesummary is available in both distributions, it is trivial to get all of them to report to the same central collector.

Tags: debian, debian edu, english, sitesummary.
Togsatsing på norsk, mot sykkel
2010-06-02 23:45

Det står dårlig til med toget når en finner på å la det kappkjøre med sykkel... Jeg tror det trengs strukturendringer for å få fikset på togproblemene i Norge.

Mon tro hva toglinje mellom Narvik og Tromsø ville hatt slags effekt på området der?

Tags: norsk.
KDM fail at boot with NVidia cards - and no one try to fix it?
2010-06-01 17:05

It is strange to watch how a bug in Debian causing KDM to fail to start at boot when an NVidia video card is used is handled. The problem seem to be that the nvidia X.org driver uses a long time to initialize, and this duration is longer than kdm is configured to wait.

I came across two bugs related to this issue, #583312 initially filed against initscripts and passed on to nvidia-glx when it became obvious that the nvidia drivers were involved, and #524751 initially filed against kdm and passed on to src:nvidia-graphics-drivers for unknown reasons.

To me, it seem that no-one is interested in actually solving the problem nvidia video card owners experience and make sure the Debian distribution work out of the box for these users. The nvidia driver maintainers expect kdm to be set up to wait longer, while kdm expect the nvidia driver maintainers to fix the driver to start faster, and while they wait for each other I guess the users end up switching to a distribution that work for them. I have no idea what the solution is, but I am pretty sure that waiting for each other is not it.

I wonder why we end up handling bugs this way.

Tags: bootsystem, debian, debian edu, english.
Parallellized boot seem to hold up well in Debian/testing
2010-05-27 23:55

A few days ago, parallel booting was enabled in Debian/testing. The feature seem to hold up pretty well, but three fairly serious issues are known and should be solved:

All in all not many surprising issues, and all of them seem solvable before Squeeze is released. In addition to these there are some packages with bugs in their dependencies and run level settings, which I expect will be fixed in a reasonable time span.

If you report any problems with dependencies in init.d scripts to the BTS, please usertag the report to get it to show up at the list of usertagged bugs related to this.

Update: Correct bug number to file-rc issue.

Tags: bootsystem, debian, debian edu, english.
More flexible firmware handling in debian-installer
2010-05-22 21:30

After a long break from debian-installer development, I finally found time today to return to the project. Having to spend less time working dependency based boot in debian, as it is almost complete now, definitely helped freeing some time.

A while back, I ran into a problem while working on Debian Edu. We include some firmware packages on the Debian Edu CDs, those needed to get disk and network controllers working. Without having these firmware packages available during installation, it is impossible to install Debian Edu on the given machine, and because our target group are non-technical people, asking them to provide firmware packages on an external medium is a support pain. Initially, I expected it to be enough to include the firmware packages on the CD to get debian-installer to find and use them. This proved to be wrong. Next, I hoped it was enough to symlink the relevant firmware packages to some useful location on the CD (tried /cdrom/ and /cdrom/firmware/). This also proved to not work, and at this point I found time to look at the debian-installer code to figure out what was going to work.

The firmware loading code is in the hw-detect package, and a closer look revealed that it would only look for firmware packages outside the installation media, so the CD was never checked for firmware packages. It would only check USB sticks, floppies and other "external" media devices. Today I changed it to also look in the /cdrom/firmware/ directory on the mounted CD or DVD, which should solve the problem I ran into with Debian edu. I also changed it to look in /firmware/, to make sure the installer also find firmware provided in the initrd when booting the installer via PXE, to allow us to provide the same feature in the PXE setup included in Debian Edu.

To make sure firmware deb packages with a license questions are not activated without asking if the license is accepted, I extended hw-detect to look for preinst scripts in the firmware packages, and run these before activating the firmware during installation. The license question is asked using debconf in the preinst, so this should solve the issue for the firmware packages I have looked at so far.

If you want to discuss the details of these features, please contact us on debian-boot@lists.debian.org.

Tags: debian, debian edu, english.
Magnetstripeinnhold i billetter fra Flytoget og Hurtigruten
2010-05-21 16:00

For en stund tilbake kjøpte jeg en magnetkortleser for å kunne titte på hva som er skrevet inn på magnetstripene til ulike kort. Har ikke hatt tid til å analysere mange kort så langt, men tenkte jeg skulle dele innholdet på to kort med mine lesere.

For noen dager siden tok jeg flyet til Harstad og Hurtigruten til Bergen. Flytoget fra Oslo S til flyplassen ga meg en billett med magnetstripe. Påtrykket finner jeg følgende informasjon:

Flytoget Airport Express Train

Fra - Til        : Oslo Sentralstasjon
Kategori         : Voksen
Pris             : Nok 170,00
Herav mva. 8,00% : NOK 12,59
Betaling         : Kontant
Til - Fra        : Oslo Lufthavn
Utstedt:         : 08.05.10
Gyldig Fra-Til   : 08.05.10-07.11.10
Billetttype      : Enkeltbillett

102-1015-100508-48382-01-08

På selve magnetstripen er innholdet ;E?+900120011=23250996541068112619257138248441708433322932704083389389062603279671261502492655?. Aner ikke hva innholdet representerer, og det er lite overlapp mellom det jeg ser trykket på billetten og det jeg ser av tegn i magnetstripen. Håper det betyr at de bruker kryptografiske metoder for å gjøre det vanskelig å forfalske billetter.

Den andre billetten er fra Hurtigruten, der jeg mistenker at strekkoden på fronten er mer brukt enn magnetstripen (det var i hvert fall den biten vi stakk inn i dørlåsen).

Påtrykket forsiden er følgende:

Romnummer 727
Hurtigruten
Midnatsol
Reinholdtsen
Petter
Bookingno: SAX69   0742193
Harstad-Bergen
Dep: 09.05.2010 Arr: 12.05.2010
Lugar fra Risøyhamn
Kost: FRO=4

På selve magnetstripen er innholdet ;1316010007421930=00000000000000000000?+E?. Heller ikke her ser jeg mye korrespondanse mellom påtrykk og magnetstripe.

Tags: norsk, nuug, sikkerhet.

RSS feed

Created by Chronicle v3.7