Petter Reinholdtsen

Entries tagged "english".

The sorry state of multimedia browser plugins in Debian
2008-11-25 00:10

Recently I have spent some time evaluating the multimedia browser plugins available in Debian Lenny, to see which one we should use by default in Debian Edu. We need an embedded video playing plugin with control buttons to pause or stop the video, and capable of streaming all the multimedia content available on the web. The test results and notes are available on the Debian wiki. I was surprised how few of the plugins are able to fill this need. My personal video player favorite, VLC, has a really bad plugin which fail on a lot of the test pages. A lot of the MIME types I would expect to work with any free software player (like video/ogg), just do not work. And simple formats like the audio/x-mplegurl format (m3u playlists), just isn't supported by the totem and vlc plugins. I hope the situation will improve soon. No wonder sites use the proprietary Adobe flash to play video.

For Lenny, we seem to end up with the mplayer plugin. It seem to be the only one fitting our needs. :/

Tags: debian, debian edu, english, multimedia, web.
Devcamp brought us closer to the Lenny based Debian Edu release
2008-12-07 12:00

This weekend we had a small developer gathering for Debian Edu in Oslo. Most of Saturday was used for the general assemly for the member organization, but the rest of the weekend I used to tune the LTSP installation. LTSP now work out of the box on the 10-network. Acer Aspire One proved to be a very nice thin client, with both screen, mouse and keybard in a small box. Was working on getting the diskless workstation setup configured out of the box, but did not finish it before the weekend was up.

Did not find time to look at the 4 VGA cards in one box we got from the Brazilian group, so that will have to wait for the next development gathering. Would love to have the Debian Edu installer automatically detect and configure a multiseat setup when it find one of these cards.

Tags: debian, debian edu, english, ltsp.
Software video mixer on a USB stick
2008-12-28 15:40

The Norwegian Unix User Group is recording our montly presentation on video, and recently we have worked on improving the quality of the recordings by mixing the slides directly with the video stream. For this, we use the dvswitch package from the Debian video team. As this require quite one computer per video source, and NUUG do not have enough laptops available, we need to borrow laptops. And to avoid having to install extra software on these borrwed laptops, I have wrapped up all the programs needed on a bootable USB stick. The software required is dvswitch with assosiated source, sink and mixer applications and dvgrab. To allow this setup to work without any configuration, I've patched dvswitch to use avahi to connect the various parts together. And to allow us to use laptops without firewire plugs, I upgraded dvgrab to the one from Debian/unstable to get one that work with USB sources. We have not yet tested this setup in a production setup, but I hope it will work properly, and allow us to set up a video mixer in a very short time frame. We will need it for Go Open 2009.

The USB image is for a 1 GB memory stick, but can be used on any larger stick as well.

Tags: english, nuug, video.
When web browser developers make a video player...
2009-01-17 18:50

As part of the work we do in NUUG to publish video recordings of our monthly presentations, we provide a page with embedded video for easy access to the recording. Putting a good set of HTML tags together to get working embedded video in all browsers and across all operating systems is not easy. I hope this will become easier when the <video> tag is implemented in all browsers, but I am not sure. We provide the recordings in several formats, MPEG1, Ogg Theora, H.264 and Quicktime, and want the browser/media plugin to pick one it support and use it to play the recording, using whatever embed mechanism the browser understand. There is at least four different tags to use for this, the new HTML5 <video> tag, the <object> tag, the <embed> tag and the <applet> tag. All of these take a lot of options, and finding the best options is a major challenge.

I just tested the experimental Opera browser available from labs.opera.com, to see how it handled a <video> tag with a few video sources and no extra attributes. I was not very impressed. The browser start by fetching a picture from the video stream. Not sure if it is the first frame, but it is definitely very early in the recording. So far, so good. Next, instead of streaming the 76 MiB video file, it start to download all of it, but do not start to play the video. This mean I have to wait for several minutes for the downloading to finish. When the download is done, the playing of the video do not start! Waiting for the download, but I do not get to see the video? Some testing later, I discover that I have to add the controls="true" attribute to be able to get a play button to pres to start the video. Adding autoplay="true" did not help. I sure hope this is a misfeature of the test version of Opera, and that future implementations of the <video> tag will stream recordings by default, or at least start playing when the download is done.

The test page I used (since changed to add more attributes) is available from the nuug site. Will have to test it with the new Firefox too.

In the test process, I discovered a missing feature. I was unable to find a way to get the URL of the playing video out of Opera, so I am not quite sure it picked the Ogg Theora version of the video. I sure hope it was using the announced Ogg Theora support. :)

Tags: english, multimedia, nuug, video, web.
Using bar codes at a computing center
2009-02-20 08:50

At work with the University of Oslo, we have several hundred computers in our computing center. This give us a challenge in tracking the location and cabling of the computers, when they are added, moved and removed. Some times the location register is not updated when a computer is inserted or moved and we then have to search the room for the "missing" computer.

In the last issue of Linux Journal, I came across a project libdmtx to write and read bar code blocks as defined in the The Data Matrix Standard. This is bar codes that can be read with a normal digital camera, for example that on a cell phone, and several such bar codes can be read by libdmtx from one picture. The bar code standard allow up to 2 KiB to be written in the tag. There is another project with a bar code writer written in postscript capable of creating such bar codes, but this was the first time I found a tool to read these bar codes.

It occurred to me that this could be used to tag and track the machines in our computing center. If both racks and computers are tagged this way, we can use a picture of the rack and all its computers to detect the rack location of any computer in that rack. If we do this regularly for the entire room, we will find all locations, and can detect movements and removals.

I decided to test if this would work in practice, and picked a random rack and tagged all the machines with their names. Next, I took pictures with my digital camera, and gave the dmtxread program these JPEG pictures to see how many tags it could read. This worked fairly well. If the pictures was well focused and not taken from the side, all tags in the image could be read. Because of limited space between the racks, I was unable to get a good picture of the entire rack, but could without problem read all tags from a picture covering about half the rack. I had to limit the search time used by dmtxread to 60000 ms to make sure it terminated in a reasonable time frame.

My conclusion is that this could work, and we should probably look at adjusting our computer tagging procedures to use bar codes for easier automatic tracking of computers.

Tags: english, nuug.
Checking server hardware support status for Dell, HP and IBM servers
2009-02-28 23:50

At work, we have a few hundred Linux servers, and with that amount of hardware it is important to keep track of when the hardware support contract expire for each server. We have a machine (and service) register, which until recently did not contain much useful besides the machine room location and contact information for the system owner for each machine. To make it easier for us to track support contract status, I've recently spent time on extending the machine register to include information about when the support contract expire, and to tag machines with expired contracts to make it easy to get a list of such machines. I extended a perl script already being used to import information about machines into the register, to also do some screen scraping off the sites of Dell, HP and IBM (our majority of machines are from these vendors), and automatically check the support status for the relevant machines. This make the support status information easily available and I hope it will make it easier for the computer owner to know when to get new hardware or renew the support contract. The result of this work documented that 27% of the machines in the registry is without a support contract, and made it very easy to find them. 27% might seem like a lot, but I see it more as the case of us using machines a bit longer than the 3 years a normal support contract last, to have test machines and a platform for less important services. After all, the machines without a contract are working fine at the moment and the lack of contract is only a problem if any of them break down. When that happen, we can either fix it using spare parts from other machines or move the service to another old machine.

I believe the code for screen scraping the Dell site was originally written by Trond Hasle Amundsen, and later adjusted by me and Morten Werner Forsbring. The HP scraping was written by me after reading a nice article in ;login: about how to use WWW::Mechanize, and the IBM scraping was written by me based on the Dell code. I know the HTML parsing could be done using nice libraries, but did not want to introduce more dependencies. This is the current incarnation:

use LWP::Simple;
use POSIX;
use WWW::Mechanize;
use Date::Parse;
[...]
sub get_support_info {
    my ($machine, $model, $serial, $productnumber) = @_;
    my $str;

    if ( $model =~ m/^Dell / ) {
        # fetch website from Dell support
        my $url = "http://support.euro.dell.com/support/topics/topic.aspx/emea/shared/support/my_systems_info/no/details?c=no&cs=nodhs1&l=no&s=dhs&ServiceTag=$serial";
        my $webpage = get($url);
        return undef unless ($webpage);

        my $daysleft = -1;
        my @lines = split(/\n/, $webpage);
        foreach my $line (@lines) {
            next unless ($line =~ m/Beskrivelse/);
            $line =~ s/<[^>]+?>/;/gm;
            $line =~ s/^.+?;(Beskrivelse;)/$1/;

            my @f = split(/\;/, $line);
            @f = @f[13 .. $#f];
            my $lastend = "";
            while ($f[3] eq "DELL") {
                my ($type, $startstr, $endstr, $days) = @f[0, 5, 7, 10];

                my $start = POSIX::strftime("%Y-%m-%d",
                                            localtime(str2time($startstr)));
                my $end = POSIX::strftime("%Y-%m-%d",
                                          localtime(str2time($endstr)));
                $str .= "$type $start -> $end ";
                @f = @f[14 .. $#f];
                $lastend = $end if ($end gt $lastend);
            }
            my $today = POSIX::strftime("%Y-%m-%d", localtime(time));
            tag_machine_unsupported($machine)
                if ($lastend lt $today);
        }
    } elsif ( $model =~ m/^HP / ) {
        my $mech = WWW::Mechanize->new();
        my $url =
            'http://www1.itrc.hp.com/service/ewarranty/warrantyInput.do';
        $mech->get($url);
        my $fields = {
            'BODServiceID' => 'NA',
            'RegisteredPurchaseDate' => '',
            'country' => 'NO',
            'productNumber' => $productnumber,
            'serialNumber1' => $serial,
        };
        $mech->submit_form( form_number => 2,
                            fields      => $fields );
        # Next step is screen scraping
        my $content = $mech->content();

        $content =~ s/<[^>]+?>/;/gm;
        $content =~ s/\s+/ /gm;
        $content =~ s/;\s*;/;;/gm;
        $content =~ s/;[\s;]+/;/gm;

        my $today = POSIX::strftime("%Y-%m-%d", localtime(time));

        while ($content =~ m/;Warranty Type;/) {
            my ($type, $status, $startstr, $stopstr) = $content =~
                m/;Warranty Type;([^;]+);.+?;Status;(\w+);Start Date;([^;]+);End Date;([^;]+);/;
            $content =~ s/^.+?;Warranty Type;//;
            my $start = POSIX::strftime("%Y-%m-%d",
                                        localtime(str2time($startstr)));
            my $end = POSIX::strftime("%Y-%m-%d",
                                      localtime(str2time($stopstr)));

            $str .= "$type ($status) $start -> $end ";

            tag_machine_unsupported($machine)
                if ($end lt $today);
        }
    } elsif ( $model =~ m/^IBM / ) {
        # This code ignore extended support contracts.
        my ($producttype) = $model =~ m/.*-\[(.{4}).+\]-/;
        if ($producttype && $serial) {
            my $content =
                get("http://www-947.ibm.com/systems/support/supportsite.wss/warranty?action=warranty&brandind=5000008&Submit=Submit&type=$producttype&serial=$serial");
            if ($content) {
                $content =~ s/<[^>]+?>/;/gm;
                $content =~ s/\s+/ /gm;
                $content =~ s/;\s*;/;;/gm;
                $content =~ s/;[\s;]+/;/gm;

                $content =~ s/^.+?;Warranty status;//;
                my ($status, $end) = $content =~ m/;Warranty status;([^;]+)\s*;Expiration date;(\S+) ;/;

                $str .= "($status) -> $end ";

                my $today = POSIX::strftime("%Y-%m-%d", localtime(time));
                tag_machine_unsupported($machine)
                    if ($end lt $today);
            }
        }
    }
    return $str;
}

Here are some examples on how to use the function, using fake serial numbers. The information passed in as arguments are fetched from dmidecode.

print get_support_info("hp.host", "HP ProLiant BL460c G1", "1234567890"
                       "447707-B21");
print get_support_info("dell.host", "Dell Inc. PowerEdge 2950", "1234567");
print get_support_info("ibm.host", "IBM eserver xSeries 345 -[867061X]-",
                       "1234567");

I would recommend this approach for tracking support contracts for everyone with more than a few computers to administer. :)

Update 2009-03-06: The IBM page do not include extended support contracts, so it is useless in that case. The original Dell code do not handle extended support contracts either, but has been updated to do so.

Tags: english, nuug.
Time for new LDAP schemas replacing RFC 2307?
2009-03-29 20:30

The state of standardized LDAP schemas on Linux is far from optimal. There is RFC 2307 documenting one way to store NIS maps in LDAP, and a modified version of this normally called RFC 2307bis, with some modifications to be compatible with Active Directory. The RFC specification handle the content of a lot of system databases, but do not handle DNS zones and DHCP configuration.

In Debian Edu/Skolelinux, we would like to store information about users, SMB clients/hosts, filegroups, netgroups (users and hosts), DHCP and DNS configuration, and LTSP configuration in LDAP. These objects have a lot in common, but with the current LDAP schemas it is not possible to have one object per entity. For example, one need to have at least three LDAP objects for a given computer, one with the SMB related stuff, one with DNS information and another with DHCP information. The schemas provided for DNS and DHCP are impossible to combine into one LDAP object. In addition, it is impossible to implement quick queries for netgroup membership, because of the way NIS triples are implemented. It just do not scale. I believe it is time for a few RFC specifications to cleam up this mess.

I would like to have one LDAP object representing each computer in the network, and this object can then keep the SMB (ie host key), DHCP (mac address/name) and DNS (name/IP address) settings in one place. It need to be efficently stored to make sure it scale well.

I would also like to have a quick way to map from a user or computer and to the net group this user or computer is a member.

Active Directory have done a better job than unix heads like myself in this regard, and the unix side need to catch up. Time to start a new IETF work group?

Tags: debian, debian edu, english, nuug.
Returning from Skolelinux developer gathering
2009-03-29 21:00

I'm sitting on the train going home from this weekends Debian Edu/Skolelinux development gathering. I got a bit done tuning the desktop, and looked into the dynamic service location protocol implementation avahi. It look like it could be useful for us. Almost 30 people participated, and I believe it was a great environment to get to know the Skolelinux system. Walter Bender, involved in the development of the Sugar educational platform, presented his stuff and also helped me improve my OLPC installation. He also showed me that his Turtle Art application can be used in standalone mode, and we agreed that I would help getting it packaged for Debian. As a standalone application it would be great for Debian Edu. We also tried to get the video conferencing working with two OLPCs, but that proved to be too hard for us. The application seem to need more work before it is ready for me. I look forward to getting home and relax now. :)

Tags: debian, debian edu, english, nuug.
Standardize on protocols and formats, not vendors and applications
2009-03-30 11:50

Where I work at the University of Oslo, one decision stand out as a very good one to form a long lived computer infrastructure. It is the simple one, lost by many in todays computer industry: Standardize on open network protocols and open exchange/storage formats, not applications. Applications come and go, while protocols and files tend to stay, and thus one want to make it easy to change application and vendor, while avoiding conversion costs and locking users to a specific platform or application.

This approach make it possible to replace the client applications independently of the server applications. One can even allow users to use several different applications as long as they handle the selected protocol and format. In the normal case, only one client application is recommended and users only get help if they choose to use this application, but those that want to deviate from the easy path are not blocked from doing so.

It also allow us to replace the server side without forcing the users to replace their applications, and thus allow us to select the best server implementation at any moment, when scale and resouce requirements change.

I strongly recommend standardizing - on open network protocols and open formats, but I would never recommend standardizing on a single application that do not use open network protocol or open formats.

Tags: debian, english, nuug, standard.
Recording video from cron using VLC
2009-04-05 10:00

One think I have wanted to figure out for a along time is how to run vlc from cron to do recording of video streams on the net. The task is trivial with mplayer, but I do not really trust the security of mplayer (it crashes too often on strange input), and thus prefer vlc. I finally found a way to do it today. I spent an hour or so searching the web for recipes and reading the documentation. The hardest part was to get rid of the GUI window, but after finding the dummy interface, the command line finally presented itself:

URL=http://www.ping.uio.no/video/rms-oslo_2009.ogg
SAVEFILE=rms.ogg
DISPLAY= vlc -q $URL \
  --sout="#duplicate{dst=std{access=file,url='$SAVEFILE'},dst=nodisplay}" \
  --intf=dummy

The command stream the URL and store it in the SAVEFILE by duplicating the output stream to "nodisplay" and the file, using the dummy interface. The dummy interface and the nodisplay output make sure no X interface is needed.

The cron job then need to start this job with the appropriate URL and file name to save, sleep for the duration wanted, and then kill the vlc process with SIGTERM. Here is a complete script vlc-record to use from at or cron:

#!/bin/sh
set -e
URL="$1"
SAVEFILE="$2"
DURATION="$3"
DISPLAY= vlc -q "$URL" \
  --sout="#duplicate{dst=std{access=file,url='$SAVEFILE'},dst=nodisplay}" \
  --intf=dummy < /dev/null > /dev/null 2>&1 &
pid=$!
sleep $DURATION
kill $pid
wait $pid
Tags: english, nuug, video.
No patch is not better than a useless patch
2009-04-28 09:30

Julien Blache claim that no patch is better than a useless patch. I completely disagree, as a patch allow one to discuss a concrete and proposed solution, and also prove that the issue at hand is important enough for someone to spent time on fixing it. No patch do not provide any of these positive properties.

Tags: debian, english, nuug.
Two projects that have improved the quality of free software a lot
2009-05-02 15:00

There are two software projects that have had huge influence on the quality of free software, and I wanted to mention both in case someone do not yet know them.

The first one is valgrind, a tool to detect and expose errors in the memory handling of programs. It is easy to use, all one need to do is to run 'valgrind program', and it will report any problems on stdout. It is even better if the program include debug information. With debug information, it is able to report the source file name and line number where the problem occurs. It can report things like 'reading past memory block in file X line N, the memory block was allocated in file Y, line M', and 'using uninitialised value in control logic'. This tool has made it trivial to investigate reproducible crash bugs in programs, and have reduced the number of this kind of bugs in free software a lot.

The second one is Coverity which is a source code checker. It is able to process the source of a program and find problems in the logic without running the program. It started out as the Stanford Checker and became well known when it was used to find bugs in the Linux kernel. It is now a commercial tool and the company behind it is running a community service for the free software community, where a lot of free software projects get their source checked for free. Several thousand defects have been found and fixed so far. It can find errors like 'lock L taken in file X line N is never released if exiting in line M', or 'the code in file Y lines O to P can never be executed'. The projects included in the community service project have managed to get rid of a lot of reliability problems thanks to Coverity.

I believe tools like this, that are able to automatically find errors in the source, are vital to improve the quality of software and make sure we can get rid of the crashing and failing software we are surrounded by today.

Tags: debian, english.
Debian boots quicker and quicker
2009-06-24 21:40

I spent Monday and tuesday this week in London with a lot of the people involved in the boot system on Debian and Ubuntu, to see if we could find more ways to speed up the boot system. This was an Ubuntu funded developer gathering. It was quite productive. We also discussed the future of boot systems, and ways to handle the increasing number of boot issues introduced by the Linux kernel becoming more and more asynchronous and event base. The Ubuntu approach using udev and upstart might be a good way forward. Time will show.

Anyway, there are a few ways at the moment to speed up the boot process in Debian. All of these should be applied to get a quick boot:

These points are based on the Google summer of code work done by Carlos Villegas.

Support for makefile-style concurrency during boot was uploaded to unstable yesterday. When we tested it, we were able to cut 6 seconds from the boot sequence. It depend on very correct dependency declaration in all init.d scripts, so I expect us to find edge cases where the dependences in some scripts are slightly wrong when we start using this.

On our IRC channel for this effort, #pkg-sysvinit, a new idea was introduced by Raphael Geissert today, one that could affect the startup speed as well. Instead of starting some scripts concurrently from rcS.d/ and another set of scripts from rc2.d/, it would be possible to run a of them in the same process. A quick way to test this would be to enable insserv and run 'mv /etc/rc2.d/S* /etc/rcS.d/; insserv'. Will need to test if that work. :)

Tags: debian, english.
Taking over sysvinit development
2009-07-22 23:00

After several years of frustration with the lack of activity from the existing sysvinit upstream developer, I decided a few weeks ago to take over the package and become the new upstream. The number of patches to track for the Debian package was becoming a burden, and the lack of synchronization between the distribution made it hard to keep the package up to date.

On the new sysvinit team is the SuSe maintainer Dr. Werner Fink, and my Debian co-maintainer Kel Modderman. About 10 days ago, I made a new upstream tarball with version number 2.87dsf (for Debian, SuSe and Fedora), based on the patches currently in use in these distributions. We Debian maintainers plan to move to this tarball as the new upstream as soon as we find time to do the merge. Since the new tarball was created, we agreed with Werner at SuSe to make a new upstream project at Savannah, and continue development there. The project is registered and currently waiting for approval by the Savannah administrators, and as soon as it is approved, we will import the old versions from svn and continue working on the future release.

It is a bit ironic that this is done now, when some of the involved distributions are moving to upstart as a syvinit replacement.

Tags: debian, english, nuug.
Debian has switched to dependency based boot sequencing
2009-07-27 23:50

Since this evening, with the upload of sysvinit version 2.87dsf-2, and the upload of insserv version 1.12.0-10 yesterday, Debian unstable have been migrated to using dependency based boot sequencing. This conclude work me and others have been doing for the last three days. It feels great to see this finally part of the default Debian installation. Now we just need to weed out the last few problems that are bound to show up, to get everything ready for Squeeze.

The next step is migrating /sbin/init from sysvinit to upstart, and fixing the more fundamental problem of handing the event based non-predictable kernel in the early boot.

Tags: debian, english, nuug.
ISO still hope to fix OOXML
2009-08-08 14:00

According to a blog post from Torsten Werner, the current defect report for ISO 29500 (ISO OOXML) is 809 pages. His interesting point is that the defect report is 71 pages more than the full ODF 1.1 specification. Personally I find it more interesting that ISO still believe ISO OOXML can be fixed in ISO. Personally, I believe it is broken beyon repair, and I completely lack any trust in ISO for being able to get anywhere close to solving the problems. I was part of the Norwegian committee involved in the OOXML fast track process, and was not impressed with Standard Norway and ISO in how they handled it.

These days I focus on ODF instead, which seem like a specification with the future ahead of it. We are working in NUUG to organise a ODF seminar this autumn.

Tags: english, nuug, standard.
Relative popularity of document formats (MS Office vs. ODF)
2009-08-12 15:50

Just for fun, I did a search right now on Google for a few file ODF and MS Office based formats (not to be mistaken for ISO or ECMA OOXML), to get an idea of their relative usage. I searched using 'filetype:odt' and equvalent terms, and got these results:

TypeODFMS Office
Tekst odt:282000 docx:308000
Presentasjon odp:75600 pptx:183000
Regneark ods:26500 xlsx:145000

Next, I added a 'site:no' limit to get the numbers for Norway, and got these numbers:

TypeODFMS Office
Tekst odt:2480 docx:4460
Presentasjon odp:299 pptx:741
Regneark ods:187 xlsx:372

I wonder how these numbers change over time.

I am aware of Google returning different results and numbers based on where the search is done, so I guess these numbers will differ if they are conduced in another country. Because of this, I did the same search from a machine in California, USA, a few minutes after the search done from a machine here in Norway.

TypeODFMS Office
Tekst odt:129000 docx:308000
Presentasjon odp:44200 pptx:93900
Regneark ods:26500 xlsx:82400

And with 'site:no':

TypeODFMS Office
Tekst odt:2480 docx:3410
Presentasjon odp:175 pptx:604
Regneark ods:186 xlsx:296

Interesting difference, not sure what to conclude from these numbers.

Tags: english, nuug, standard, web.
Automatic Munin and Nagios configuration
2010-01-27 15:15

One of the new features in the next Debian/Lenny based release of Debian Edu/Skolelinux, which is scheduled for release in the next few days, is automatic configuration of the service monitoring system Nagios. The previous release had automatic configuration of trend analysis using Munin, and this Lenny based release take that a step further.

When installing a Debian Edu Main-server, it is automatically configured as a Munin and Nagios server. In addition, it is configured to be a server for the SiteSummary system I have written for use in Debian Edu. The SiteSummary system is inspired by a system used by the University of Oslo where I work. In short, the system provide a centralised collector of information about the computers on the network, and a client on each computer submitting information to this collector. This allow for automatic information on which packages are installed on each machine, which kernel the machines are using, what kind of configuration the packages got etc. This also allow us to automatically generate Munin and Nagios configuration.

All computers reporting to the sitesummary collector with the munin-node package installed is automatically enabled as a Munin client and graphs from the statistics collected from that machine show up automatically on http://www/munin/ on the Main-server.

All non-laptop computers reporting to the sitesummary collector are automatically monitored for network presence (ping and any network services detected). In addition, all computers (also laptops) with the nagios-nrpe-server package installed and configured the way sitesummary would configure it, are monitored for full disks, software raid status, swap free and other checks that need to run locally on the machine.

The result is that the administrator on a school using Debian Edu based on Lenny will be able to check the health of his installation with one look at the Nagios settings, without having to spend any time keeping the Nagios configuration up-to-date.

The only configuration one need to do to get Nagios up and running is to set the password used to get access via HTTP. The system administrator need to run "htpasswd /etc/nagios3/htpasswd.users nagiosadmin" to create a nagiosadmin user and set a password for it to be able to log into the Nagios web pages. After that, everything is taken care of.

Tags: debian edu, english, nuug, sitesummary.
Debian Edu / Skolelinux based on Lenny released, work continues
2010-02-11 17:15

On Tuesday, the Debian/Lenny based version of Skolelinux was finally shipped. This was a major leap forward for the project, and I am very pleased that we finally got the release wrapped up. Work on the first point release starts imediately, as we plan to get that one out a month after the major release, to include all fixes for bugs we found and fixed too late in the release process to include last Tuesday.

Perhaps it even is time for some partying?

After this first point release, my plan is to focus again on the next major release, based on Squeeze. We will try to get as many of the fixes we need into the official Debian packages before the freeze, and have just a few weeks or months to make it happen.

Tags: debian edu, english, nuug.
After 6 years of waiting, the Xreset.d feature is implemented
2010-03-06 18:15

6 years ago, as part of the Debian Edu development I am involved in, I asked for a hook in the kdm and gdm setup to run scripts as root when the user log out. A bug was submitted against the xfree86-common package in 2004 (#230422), and revisited every time Debian Edu was working on a new release. Today, this finally paid off.

The framework for this feature was today commited to the git repositry for the xorg package, and the git repository for xdm has been updated to use this framework. Next on my agenda is to make sure kdm and gdm also add code to use this framework.

In Debian Edu, we want to ability to run commands as root when the user log out, to get rid of runaway processes and do general cleanup after a user. With this framework in place, we finally can do that in a generic way that work with all display managers using this framework. My goal is to get all display managers in Debian use it, similar to how they use the Xsession.d framework today.

Tags: debian edu, english, nuug.
Kerberos for Debian Edu/Squeeze?
2010-04-14 17:20

Yesterdays NUUG presentation about Kerberos was inspiring, and reminded me about the need to start using Kerberos in Skolelinux. Setting up a Kerberos server seem to be straight forward, and if we get this in place a long time before the Squeeze version of Debian freezes, we have a chance to migrate Skolelinux away from NFSv3 for the home directories, and over to an architecture where the infrastructure do not have to trust IP addresses and machines, and instead can trust users and cryptographic keys instead.

A challenge will be integration and administration. Is there a Kerberos implementation for Debian where one can control the administration access in Kerberos using LDAP groups? With it, the school administration will have to maintain access control using flat files on the main server, which give a huge potential for errors.

A related question I would like to know is how well Kerberos and pam-ccreds (offline password check) work together. Anyone know?

Next step will be to use Kerberos for access control in Lwat and Nagios. I have no idea how much work that will be to implement. We would also need to document how to integrate with Windows AD, as such shared network will require two Kerberos realms that need to cooperate to work properly.

I believe a good start would be to start using Kerberos on the skolelinux.no machines, and this way get ourselves experience with configuration and integration. A natural starting point would be setting up ldap.skolelinux.no as the Kerberos server, and migrate the rest of the machines from PAM via LDAP to PAM via Kerberos one at the time.

If you would like to contribute to get this working in Skolelinux, I recommend you to see the video recording from yesterdays NUUG presentation, and start using Kerberos at home. The video show show up in a few days.

Tags: debian edu, english, nuug.
Great book: "Content: Selected Essays on Technology, Creativity, Copyright, and the Future of the Future"
2010-04-19 17:10

The last few weeks i have had the pleasure of reading a thought-provoking collection of essays by Cory Doctorow, on topics touching copyright, virtual worlds, the future of man when the conscience mind can be duplicated into a computer and many more. The book titled "Content: Selected Essays on Technology, Creativity, Copyright, and the Future of the Future" is available with few restrictions on the web, for example from his own site. I read the epub-version from feedbooks using fbreader and my N810. I strongly recommend this book.

Tags: english, fildeling, nuug, opphavsrett, personvern, sikkerhet, web.
Thoughts on roaming laptop setup for Debian Edu
2010-04-28 20:40

For some years now, I have wondered how we should handle laptops in Debian Edu. The Debian Edu infrastructure is mostly designed to handle stationary computers, and less suited for computers that come and go.

Now I finally believe I have an sensible idea on how to adjust Debian Edu for laptops, by introducing a new profile for them, for example called Roaming Workstations. Here are my thought on this. The setup would consist of the following:

I believe all the pieces to implement this are in Debian/testing at the moment. If we work quickly, we should be able to get this ready in time for the Squeeze release to freeze. Some of the pieces need tweaking, like libpam-ccreds should get support for pam-auth-update (#566718) and nslcd (or perhaps debian-edu-config) should get some integration code to stop its daemon when the LDAP server is unavailable to avoid long timeouts when disconnected from the net. If we get Kerberos enabled, we need to make sure we avoid long timeouts there too.

If you want to help out with implementing this for Debian Edu, please contact us on debian-edu@lists.debian.org.

Tags: debian edu, english, nuug.
Forcing new users to change their password on first login
2010-05-02 13:47

One interesting feature in Active Directory, is the ability to create a new user with an expired password, and thus force the user to change the password on the first login attempt.

I'm not quite sure how to do that with the LDAP setup in Debian Edu, but did some initial testing with a local account. The account and password aging information is available in /etc/shadow, but unfortunately, it is not possible to specify an expiration time for passwords, only a maximum age for passwords.

A freshly created account (using adduser test) will have these settings in /etc/shadow:

root@tjener:~# chage -l test
Last password change                                    : May 02, 2010
Password expires                                        : never
Password inactive                                       : never
Account expires                                         : never
Minimum number of days between password change          : 0
Maximum number of days between password change          : 99999
Number of days of warning before password expires       : 7
root@tjener:~#

The only way I could come up with to create a user with an expired account, is to change the date of the last password change to the lowest value possible (January 1th 1970), and the maximum password age to the difference in days between that date and today. To make it simple, I went for 30 years (30 * 365 = 10950) and January 2th (to avoid testing if 0 is a valid value).

After using these commands to set it up, it seem to work as intended:

root@tjener:~# chage -d 1 test; chage -M 10950 test
root@tjener:~# chage -l test
Last password change                                    : Jan 02, 1970
Password expires                                        : never
Password inactive                                       : never
Account expires                                         : never
Minimum number of days between password change          : 0
Maximum number of days between password change          : 10950
Number of days of warning before password expires       : 7
root@tjener:~#  

So far I have tested this with ssh and console, and kdm (in Squeeze) login, and all ask for a new password before login in the user (with ssh, I was thrown out and had to log in again).

Perhaps we should set up something similar for Debian Edu, to make sure only the user itself have the account password?

If you want to comment on or help out with implementing this for Debian Edu, please contact us on debian-edu@lists.debian.org.

Update 2010-05-02 17:20: Paul Tötterman tells me on IRC that the shadow(8) page in Debian/testing now state that setting the date of last password change to zero (0) will force the password to be changed on the first login. This was not mentioned in the manual in Lenny, so I did not notice this in my initial testing. I have tested it on Squeeze, and 'chage -d 0 username' do work there. I have not tested it on Lenny yet.

Update 2010-05-02-19:05: Jim Paris tells me via email that an equivalent command to expire a password is 'passwd -e username', which insert zero into the date of the last password change.

Tags: debian edu, english, nuug, sikkerhet.
Parallellizing the boot in Debian Squeeze - ready for wider testing
2010-05-06 23:25

These days, the init.d script dependencies in Squeeze are quite complete, so complete that it is actually possible to run all the init.d scripts in parallell based on these dependencies. If you want to test your Squeeze system, make sure dependency based boot sequencing is enabled, and add this line to /etc/default/rcS:

CONCURRENCY=makefile

That is it. It will cause sysv-rc to use the startpar tool to run scripts in parallel using the dependency information stored in /etc/init.d/.depend.boot, /etc/init.d/.depend.start and /etc/init.d/.depend.stop to order the scripts. Startpar is configured to try to start the kdm and gdm scripts as early as possible, and will start the facilities required by kdm or gdm as early as possible to make this happen.

Give it a try, and see if you like the result. If some services fail to start properly, it is most likely because they have incomplete init.d script dependencies in their startup script (or some of their dependent scripts have incomplete dependencies). Report bugs and get the package maintainers to fix it. :)

Running scripts in parallel could be the default in Debian when we manage to get the init.d script dependencies complete and correct. I expect we will get there in Squeeze+1, if we get manage to test and fix the remaining issues.

If you report any problems with dependencies in init.d scripts to the BTS, please usertag the report to get it to show up at the list of usertagged bugs related to this.

Tags: debian, english.
systemd, an interesting alternative to upstart
2010-05-13 22:20

The last few days a new boot system called systemd has been introduced to the free software world. I have not yet had time to play around with it, but it seem to be a very interesting alternative to upstart, and might prove to be a good alternative for Debian when we are able to switch to an event based boot system. Tollef is in the process of getting systemd into Debian, and I look forward to seeing how well it work. I like the fact that systemd handles init.d scripts with dependency information natively, allowing them to run in parallel where upstart at the moment do not.

Unfortunately do systemd have the same problem as upstart regarding platform support. It only work on recent Linux kernels, and also need some new kernel features enabled to function properly. This means kFreeBSD and Hurd ports of Debian will need a port or a different boot system. Not sure how that will be handled if systemd proves to be the way forward.

In the mean time, based on the input on debian-devel@ regarding parallel booting in Debian, I have decided to enable full parallel booting as the default in Debian as soon as possible (probably this weekend or early next week), to see if there are any remaining serious bugs in the init.d dependencies. A new version of the sysvinit package implementing this change is already in experimental. If all go well, Squeeze will be released with parallel booting enabled by default.

Tags: debian, english, nuug.
Sitesummary tip: Listing MAC address of all clients
2010-05-14 21:10

In the recent Debian Edu versions, the sitesummary system is used to keep track of the machines in the school network. Each machine will automatically report its status to the central server after boot and once per night. The network setup is also reported, and using this information it is possible to get the MAC address of all network interfaces in the machines. This is useful to update the DHCP configuration.

To give some idea how to use sitesummary, here is a one-liner to ist all MAC addresses of all machines reporting to sitesummary. Run this on the collector host:

perl -MSiteSummary -e 'for_all_hosts(sub { print join(" ", get_macaddresses(shift)), "\n"; });'

This will list all MAC addresses assosiated with all machine, one line per machine and with space between the MAC addresses.

To allow system administrators easier job at adding static DHCP addresses for hosts, it would be possible to extend this to fetch machine information from sitesummary and update the DHCP and DNS tables in LDAP using this information. Such tool is unfortunately not written yet.

Tags: debian, debian edu, english, sitesummary.
Parallellized boot is now the default in Debian/unstable
2010-05-14 22:40

Since this evening, parallel booting is the default in Debian/unstable for machines using dependency based boot sequencing. Apparently the testing of concurrent booting has been wider than expected, if I am to believe the input on debian-devel@, and I concluded a few days ago to move forward with the feature this weekend, to give us some time to detect any remaining problems before Squeeze is frozen. If serious problems are detected, it is simple to change the default back to sequential boot. The upload of the new sysvinit package also activate a new upstream version.

More information about dependency based boot sequencing is available from the Debian wiki. It is currently possible to disable parallel booting when one run into problems caused by it, by adding this line to /etc/default/rcS:

CONCURRENCY=none

If you report any problems with dependencies in init.d scripts to the BTS, please usertag the report to get it to show up at the list of usertagged bugs related to this.

Tags: debian, debian edu, english.
Pieces of the roaming laptop puzzle in Debian
2010-05-19 19:00

Today, the last piece of the puzzle for roaming laptops in Debian Edu finally entered the Debian archive. Today, the new libpam-mklocaluser package was accepted. Two days ago, two other pieces was accepted into unstable. The pam-python package needed by libpam-mklocaluser, and the sssd package passed NEW on Monday. In addition, the libpam-ccreds package we need is in experimental (version 10-4) since Saturday, and hopefully will be moved to unstable soon.

This collection of packages allow for two different setups for roaming laptops. The traditional setup would be using libpam-ccreds, nscd and libpam-mklocaluser with LDAP or Kerberos authentication, which should work out of the box if the configuration changes proposed for nscd in BTS report #485282 is implemented. The alternative setup is to use sssd with libpam-mklocaluser to connect to LDAP or Kerberos and let sssd take care of the caching of passwords and group information.

I have so far been unable to get sssd to work with the LDAP server at the University, but suspect the issue is some SSL/GnuTLS related problem with the server certificate. I plan to update the Debian package to version 1.2, which is scheduled for next week, and hope to find time to make sure the next release will include both the Debian/Ubuntu specific patches. Upstream is friendly and responsive, and I am sure we will find a good solution.

The idea is to set up the roaming laptops to authenticate using LDAP or Kerberos and create a local user with home directory in /home/ when a usre in LDAP logs in via KDM or GDM for the first time, and cache the password for offline checking, as well as caching group memberhips and other relevant LDAP information. The libpam-mklocaluser package was created to make sure the local home directory is in /home/, instead of /site/server/directory/ which would be the home directory if pam_mkhomedir was used. To avoid confusion with support requests and configuration, we do not want local laptops to have users in a path that is used for the same users home directory on the home directory servers.

One annoying problem with gdm is that it do not show the PAM message passed to the user from libpam-mklocaluser when the local user is created. Instead gdm simply reject the login with some generic message. The message is shown in kdm, ssh and login, so I guess it is a bug in gdm. Have not investigated if there is some other message type that can be used instead to get gdm to also show the message.

If you want to help out with implementing this for Debian Edu, please contact us on debian-edu@lists.debian.org.

Tags: debian edu, english, nuug.
More flexible firmware handling in debian-installer
2010-05-22 21:30

After a long break from debian-installer development, I finally found time today to return to the project. Having to spend less time working dependency based boot in debian, as it is almost complete now, definitely helped freeing some time.

A while back, I ran into a problem while working on Debian Edu. We include some firmware packages on the Debian Edu CDs, those needed to get disk and network controllers working. Without having these firmware packages available during installation, it is impossible to install Debian Edu on the given machine, and because our target group are non-technical people, asking them to provide firmware packages on an external medium is a support pain. Initially, I expected it to be enough to include the firmware packages on the CD to get debian-installer to find and use them. This proved to be wrong. Next, I hoped it was enough to symlink the relevant firmware packages to some useful location on the CD (tried /cdrom/ and /cdrom/firmware/). This also proved to not work, and at this point I found time to look at the debian-installer code to figure out what was going to work.

The firmware loading code is in the hw-detect package, and a closer look revealed that it would only look for firmware packages outside the installation media, so the CD was never checked for firmware packages. It would only check USB sticks, floppies and other "external" media devices. Today I changed it to also look in the /cdrom/firmware/ directory on the mounted CD or DVD, which should solve the problem I ran into with Debian edu. I also changed it to look in /firmware/, to make sure the installer also find firmware provided in the initrd when booting the installer via PXE, to allow us to provide the same feature in the PXE setup included in Debian Edu.

To make sure firmware deb packages with a license questions are not activated without asking if the license is accepted, I extended hw-detect to look for preinst scripts in the firmware packages, and run these before activating the firmware during installation. The license question is asked using debconf in the preinst, so this should solve the issue for the firmware packages I have looked at so far.

If you want to discuss the details of these features, please contact us on debian-boot@lists.debian.org.

Tags: debian, debian edu, english.
Parallellized boot seem to hold up well in Debian/testing
2010-05-27 23:55

A few days ago, parallel booting was enabled in Debian/testing. The feature seem to hold up pretty well, but three fairly serious issues are known and should be solved:

All in all not many surprising issues, and all of them seem solvable before Squeeze is released. In addition to these there are some packages with bugs in their dependencies and run level settings, which I expect will be fixed in a reasonable time span.

If you report any problems with dependencies in init.d scripts to the BTS, please usertag the report to get it to show up at the list of usertagged bugs related to this.

Update: Correct bug number to file-rc issue.

Tags: debian, debian edu, english.
KDM fail at boot with NVidia cards - and no one try to fix it?
2010-06-01 17:05

It is strange to watch how a bug in Debian causing KDM to fail to start at boot when an NVidia video card is used is handled. The problem seem to be that the nvidia X.org driver uses a long time to initialize, and this duration is longer than kdm is configured to wait.

I came across two bugs related to this issue, #583312 initially filed against initscripts and passed on to nvidia-glx when it became obvious that the nvidia drivers were involved, and #524751 initially filed against kdm and passed on to src:nvidia-graphics-drivers for unknown reasons.

To me, it seem that no-one is interested in actually solving the problem nvidia video card owners experience and make sure the Debian distribution work out of the box for these users. The nvidia driver maintainers expect kdm to be set up to wait longer, while kdm expect the nvidia driver maintainers to fix the driver to start faster, and while they wait for each other I guess the users end up switching to a distribution that work for them. I have no idea what the solution is, but I am pretty sure that waiting for each other is not it.

I wonder why we end up handling bugs this way.

Tags: debian, debian edu, english.

RSS Feed