Petter Reinholdtsen

Doffin er det offentlige Norges portal for annonsering av anbudsutlysninger. Nettstedet er mye brukt, men rådataene er ikke enkelt tilgjengelig for almenheten. For å bøte på det har jeg som et NUUG-prosjekt- laget en Scraperwiki-skraper som lager en SQL-database med nøkkelinformasjonen fra Doffin, slik at det er mulig å søke og analysere Doffin-oppføringene. Det publiseres mellom 900-1500 anbud hver måned via Doffin. Jeg har ingen konkrete planer for analyse, men håper enklere tilgjengelighet gjør at flere griper sjansen. På sikt håper jeg å bruke denne databasen som grunnlag for å lage en database over anbudsdokumenter, som i dag ikke er tilgjengelig fra doffin, men må bestilles fra hver enkelt utlyser.

Jeg var litt usikker på om det var rettighetsproblemer knyttet til skraping av informasjon fra Doffin, men ser i vilkårene til Doffin følgende:

Opphavsrett mv

Ved innleggig av kunngjøringer på Doffin aksepterer Oppdragsgiver at opphavsrett og andre rettigheter til materialet overføres til Fornyings- og administrasjonsdepartementet, eller den departementet utnevner. Oppdragsgiver har imidlertid bruksrett til materialet, og kan benytte det til de formål de måtte ønske.

Hvis informasjonen er vernet av opphavsrettsloven (hvilket jeg tviler på, gitt lovens begrensninger om informasjon fra det offentlige), så er det FAD som har de immaterielle rettighetene. FAD er en del av staten, som opererer på vegne av folket, og enhver borger skulle dermed ha rett til å videreformidle informasjonen. Ingen av offentlighetslovens unntak kommer til anvendelse, i og med at informasjonen allerede er publisert.

During my work on Debian Edu based on Squeeze, I came across some issues that should be addressed in the Wheezy release. I finally found time to wrap up my notes and provide quick summary of what I found, with a bit explanation.

• We need to rewrite our package installation framework, as tasksel changed from using tasksel tasks to using meta packages (aka packages with dependencies like our education-* packages), and our installation system depend on tasksel tasks in /usr/share/tasksel/debian-edu-tasks.desc for package installation.
• Enable Kerberos login for more services. Now with the Kerberos foundation in place, we should use it to get single sign on with more services, and avoiding unneeded password / login questions. We should at least try to enable it for these services:
  • CUPS for admins to add/configure printers and users when using quotas.
  • Nagios for admins checking the system status.
  • GOsa for admins updating LDAP and users changing their passwords.
  • LDAP for admins updating LDAP.
  • Squid for users when exam mode / filtering is active.
  • ssh for admins and users to save a password prompt.
• When we move GOsa to use Kerberos instead of LDAP bind to authenticate users, we should try to block or at least limit access to use LDAP bind for authentication, to ensure Kerberos is used when it is intended, and nothing fall back to using the less safe LDAP bind
• Merge debian-edu-config and debian-edu-install. The split made sense when d-e-install did a lot more, but these days it is just an inconvenience when we update the debconf preseeding values.
• Fix partman-auto to allow us to abort the installation before touching the disk if the disk is too small. This is BTS report #653305 and the d-i developers are fine with the patch and someone just need to apply it and upload. After this is done we need to adjust debian-edu-install to use this new hook.
• Adjust to new LTSP framework (boot time config instead of install time config). LTSP changed its design, and our hooks to install packages and update the configuration is most likely not going to work in Wheezy.
• Consider switching to NBD instead of NFS for LTSP root, to allow the Kernel to cache files in its normal file cache, possibly speeding up KDE login on slow networks.
• Make it possible to create expired user passwords that need to change on first login. This is useful when handing out password on paper, to make sure only the user know the password. This require fixes to the PAM handling of kdm and gdm.
• Make GUI for adding new machines automatically from sitesummary. The current command line script is not very friendly to people most familiar with GUIs. This should probably be integrated into GOsa to have it available where the admin will be looking for it..
• We should find way for Nagios to check that the DHCP service actually is working (as in handling out IP addresses). None of the Nagios checks I have found so far have been working for me.
• We should switch from libpam-nss-ldapd to sssd for all profiles using LDAP, and not only on for roaming workstations, to have less packages to configure and consistent setup across all profiles.
• We should configure Kerberos to update LDAP and Samba password when changing password using the Kerberos protocol. The hook was requested in BTS report #588968 and is now available in Wheezy. We might need to write a MIT Kerberos plugin in C to get this.
• We should clean up the set of applications installed by default.
  • reduce the number of chemistry visualisers
  • consider dropping xpaint
  • and probably more?
• Some hardware need external firmware to work properly. This is mostly the case for WiFi network cards, but there are some other examples too. For popular laptops to work out of the box, such firmware need to be installed from non-free, and we should provide some GUI to do this. Ubuntu already have this implemented, and we could consider using their packages. At the moment we have some command line script to do this (one for the running system, another for the LTSP chroot).
• In Squeeze, we provide KDE, Gnome and LXDE as desktop options. We should extend the list to Xfce and Sugar, and preferably find a way to install several and allow the admin or the user to select which one to use.
• The golearn tool from the goplay package make it easy to check out interesting educational packages. We should work on the package tagging in Debian to ensure it represent all the useful educational packages, and extend the tool to allow it to use packagekit to install new applications with a simple mouse click.
• The Squeeze version got half a exam solution already in place, with the introduction of iptable based network blocking, but for it to be a complete exam solution the Squid proxy need to enable filtering/blocking as well when the exam mode is enabled. We should implement a way to easily enable this for the schools that want it, instead of the "it is documented" method of today.
• A feature used in several schools is the ability for a teacher to "take over" the desktop of individual or all computers in the room. There are at least three implementations, italc, controlaula og epoptes and we should pick one of them and make it trivial to set it up in a school. The challenges is how to distribute crypto keys and how to group computers in one room and how to set up which machine/user can control the machines in a given room.
• Tablets and surf boards are getting more and more popular, and we should look into providing a good solution for integrating these into the Debian Edu network. Not quite sure how. Perhaps we should provide a installation profile with better touch screen support for them, or add some sync services to allow them to exchange configuration and data with the central server. This should be investigated.

I guess we will discover more as we continue to work on the Wheezy version.

De siste månedene har jeg sammen med andre i NUUG jobbet med å gjøre det enklere å få innsyn i offentlige dokumenter, og dette inkluderer å gjøre postjournaler enklere tilgjengelig for maskinell analyse og søk. I den forbindelse tok jeg i dag å tittet litt på Offentlig Elektronisk Postjournal (OEP), FAD/DIFIs fellesløsning for departementer og statlige etater.

Her er antall oppføringer pr. måned det siste året, summert opp ved hjelp av en Scraperwiki-database. Merk at denne Scraperwikien har ytelsesproblemer pga. mengden data.

Det er også interessant å se hvor ofte ulike etater sender inn sine journaloppføringer til OEP. OEP inneholder en liste med sist innrapporteringsdato for alle som sender til OEP, og ved å se når og hvor ofte det blir sendt inn fra etatene kan frekvensen beregnes. Her er bunnlista, dvs. de som leverer sjeldnere enn hver 10. dag beregnet fra en Scraperwiki-database:

En kan beregne lignende frekvenser ved å sammenligne journalføringsdato med publiseringsdato for hver enkelt oppføring i OEP, men den lenken legger jeg ikke ved her for å unngå at søkemotorroboter begynner å overbelaste Scraperwiki-databasen.

Jeg har spurt Norges geologiske undersøkelse om hvorfor de leverer så sjelden, og det kommer av at de har problemer etter oppgradering av arkivsystemet sitt og jobber med å fikse det slik at de kan gå tilbake til å levere hver uke. Har ikke undersøkt noen av de andre.

Slashdot got a story about Intel planning a TV with face recognition to recognise the viewer, and it occurred to me that it would be more interesting to turn it around, and do face recognition on the TV image itself. It could let the viewer know who is present on the screen, and perhaps look up their credibility, company affiliation, previous appearances etc for the viewer to better evaluate what is being said and done. That would be a feature I would be willing to pay for.

I would not be willing to pay for a TV that point a camera on my household, like the big brother feature apparently proposed by Intel. It is the telescreen idea fetched straight out of the book 1984 by George Orwell.

A few days ago I reported how to get the support status out of Dell using an unofficial and undocumented SOAP API, which I since have found out was discovered by Daniel De Marco in february. Combined with my web scraping code for HP, Dell and IBM from 2009, I got inspired and wrote a web service based on Scraperwiki to make it easy to look up the support status and get a machine readable result back.

This is what it look like at the moment when asking for the JSON output:

% GET https://views.scraperwiki.com/run/computer-hardware-support-status/?format=json&vendor=Dell&servicetag=2v1xwn1
supportstatus({"servicetag": "2v1xwn1", "warrantyend": "2013-11-24", "shipped": "2010-11-24", "scrapestamputc": "2012-06-06T20:26:56.965847", "scrapedurl": "http://143.166.84.118/services/assetservice.asmx?WSDL", "vendor": "Dell", "productid": ""})
%

It currently support Dell and HP, and I am hoping for help to add support for other vendors. The python source is available on Scraperwiki and I welcome help with adding more features.

Jeg oppdaget nylig en artikkel i Adresseavisa i Trondheim som nevner FiksGataMi, og der Trondheim kommune ser ut til å fortelle at de ikke følger forvaltningslovens krav ved mottak av meldinger sendt inn via FiksGataMi. La derfor nettopp inn denne kommentaren til artikkelen:

Her er en liten faktaoppdatering om FiksGataMi-tjenesten, da noen ser ut til å ha misforstått hvordan den fungerer.

FiksGataMi er et privat initiativ opprettet og drevet av medlemsforeningen NUUG. FiksGataMi tar imot meldinger om feil i offentlig infrastruktur, og sender meldingen skriftlig videre på vegne av innmelder til aktuell aktør i det offentlige, det være seg kommune, fylke eller vegvesenregion. Offentlig etat blir valgt ut fra geografisk plassering og kategori valgt av innsender. Offentlige etater er i følge forvaltningsloven pliktig å følge opp og besvare skriftlige henvendelser, og hvis noen av mottakerne ikke gjør dette kan en klage på lovbrudd i det offentlige. FiksGataMi fungerer dermed som en slags epostklient for innbyggerne der kopi av innsendte meldinger gjøres tilgjengelig og knyttes til kartplassering for enkel gjenfinning. Å sende inn nye problemrapporter via FiksGataMi er dermed ikke avhengig av at kommunen aktivt må følge med på meldinger hos FiksGataMi, da de får dem tilsendt på sine offisielle epostmottakspunkter. Hvorvidt noe blir fikset og om innbyggerne er opp til mottaker av meldingene. For Trondheim kommune er mottaksadressen bydrift.vegdrift@trondheim.kommune.no, en adresse jeg inntil jeg leste denne artikkelen trodde ble håndtert i henhold til forvaltningslovens krav.

Kan ellers fortelle at 57 kommuner lenker til FiksGataMi fra sine nettsider, og at 37 % (4182 av 11266 ) av problemrapportene sendt inn via FiksGataMi er markert som løst i løsningen. Trondheim kommune har fått tilbud om å få rapportene levert på datamaskinlesbart format i stedet for epost, men har ikke takket ja så langt.

Vennlig hilsen Petter Reinholdtsen, en av NUUG-folkene bak FiksGataMi.no

Det høres for meg ut som om innbyggerne i Trondheim burde klage på kommunens potensielle lovbrudd.

Back in 2010, Mike Gabriel showed up on the Debian Edu and Skolelinux mailing list. He quickly proved to be a valuable developer, and thanks to his tireless effort we now have Kerberos integrated into the Debian Edu Squeeze version.

Who are you, and how do you spend your days?

My name is Mike Gabriel, I am 38 years old and live near Kiel, Schleswig-Holstein, Germany. I live together with a wonderful partner (Angela Fuß) and two own children and two bonus children (contributed by Angela).

During the day I am part-time employed as a system administrator and part-time working as an IT consultant. The consultancy work touches free software topics wherever and whenever possible. During the nights I am a free software developer. In the gaps I also train in becoming an osteopath.

Starting in 2010 we (Andreas Buchholz, Angela Fuß, Mike Gabriel) have set up a free software project in the area of Kiel that aims at introducing free software into schools. The project's name is "IT-Zukunft Schule" (IT future for schools). The project links IT skills with communication skills.

How did you get in contact with the Skolelinux/Debian Edu project?

While preparing our own customised Linux distribution for "IT-Zukunft Schule" we were repeatedly asked if we really wanted to reinvent the wheel. What schools really need is already available, people said. From this impulse we started evaluating other Linux distributions that target being used for school networks.

At the end we short-listed two approaches and compared them: a commercial Linux distribution developed by a company in Bremen, Germany, and Skolelinux / Debian Edu. Between 12/2010 and 03/2011 we went to several events and met people being responsible for marketing and development of either of the distributions. Skolelinux / Debian Edu was by far much more convincing compared to the other product that got short-listed beforehand--across the full spectrum. What was most attractive for me personally: the perspective of collaboration within the developmental branch of the Debian Edu project itself.

In parallel with this, we talked to many local and not-so-local people. People teaching at schools, headmasters, politicians, data protection experts, other IT professionals.

We came to two conclusions:

First, a technical conclusion: What schools need is available in bits and pieces here and there, and none of the solutions really fit by 100%. Any school we have seen has a very individual IT setup whereas most of each school's requirements could mapped by a standard IT solution. The requirement to this IT solution is flexibility and customisability, so that individual adaptations here and there are possible. In terms of re-distributing and rolling out such a standardised IT system for schools (a system that is still to some degree customisable) there is still a lot of work to do here locally. Debian Edu / Skolelinux has been our choice as the starting point.

Second, a holistic conclusion: What schools need does not exist at all (or we missed it so far). There are several technical solutions for handling IT at schools that tend to make a good impression. What has been missing completely here in Germany, though, is the enrolment of people into using IT and teaching with IT. "IT-Zukunft Schule" tries to provide an approach for this.

Only some schools have some sort of a media concept which explains, defines and gives guidance on how to use IT in class. Most schools in Northern Germany do not have an IT service provider, the school's IT equipment is managed by one or (if the school is lucky) two (admin) teachers, most of the workload these admin teachers get done in there spare time.

We were surprised that only a very few admin teachers were networked with colleagues from other schools. Basically, every school here around has its individual approach of providing IT equipment to teachers and students and the exchange of ideas has been quasi non-existent until 2010/2011.

Quite some (non-admin) teachers try to avoid using IT technology in class as a learning medium completely. Several reasons for this avoidance do exist.

We discovered that no-one has ever taken a closer look at this social part of IT management in schools, so far. On our quest journey for a technical IT solution for schools, we discussed this issue with several teachers, headmasters, politicians, other IT professionals and they all confirmed: a holistic approach of considering IT management at schools, an approach that includes the people in place, will be new and probably a gain for all.

What do you see as the advantages of Skolelinux/Debian Edu?

There is a list of advantages: international context, openness to any kind of contributions, do-ocracy policy, the closeness to Debian, the different installation scenarios possible (from stand-alone workstation to complex multi-server sites), the transparency within project communication, honest communication within the group of developers, etc.

What do you see as the disadvantages of Skolelinux/Debian Edu?

Every coin has two sides:

Technically: BTS issue #311188, tricky upgradability of a Debian Edu main server, network client installations on top of a plain vanilla Debian installation should become possible sometime in the near future, one could think about splitting the very complex package debian-edu-config into several portions (to make it easier for new developers to contribute).

Another issue I see is that we (as Debian Edu developers) should find out more about the network of people who do the marketing for Debian Edu / Skolelinux. There is a very active group in Germany promoting Skolelinux on the bigger Linux Days within Germany. Are there other groups like that in other countries? How can we bring these marketing people together (marketing group A with group B and all of them with the group of Debian Edu developers)? During the last meeting of the German Skolelinux group, I got the impression of people there being rather disconnected from the development department of Debian Edu / Skolelinux.

Which free software do you use daily?

For my daily business, I do not use commercial software at all.

For normal stuff I use Iceweasel/Firefox, Libreoffice.org. For serious text writing I prefer LaTeX. I use gimp, inkscape, scribus for more artistic tasks. I run virtual machines in KVM and Virtualbox.

I am one of the upstream developers of X2Go. In 2010 I started the development of a Python based X2Go Client, called PyHoca-GUI. PyHoca-GUI has brought forth a Python X2Go Client API that currently is being integrated in Ubuntu's software center.

For communications I have my own Kolab server running using Horde as web-based groupware client. For IRC I love to use irssi, for Jabber I have several clients that I use, mostly pidgin, though. I am also the Debian maintainer of Coccinella, a Jabber-based interactive whiteboard.

My favourite terminal emulator is KDE's Yakuake.

Which strategy do you believe is the right one to use to get schools to use free software?

Communicate, communicate, communicate. Enrol people, enrol people, enrol people.

A few years ago I wrote how to extract support status for your Dell and HP servers. Recently I have learned from colleges here at the University of Oslo that Dell have made this even easier, by providing a SOAP based web service. Given the service tag, one can now query the Dell servers and get machine readable information about the support status. This perl code demonstrate how to do it:

use strict;
use warnings;
use SOAP::Lite;
use Data::Dumper;
my $GUID = '11111111-1111-1111-1111-111111111111';
my $App = 'test';
my $servicetag = $ARGV[0] or die "Please supply a servicetag. $!\n";
my ($deal, $latest, @dates);
my $s = SOAP::Lite
    -> uri('http://support.dell.com/WebServices/')
    -> on_action( sub { join '', @_ } )
    -> proxy('http://xserv.dell.com/services/assetservice.asmx')
    ;
my $a = $s->GetAssetInformation(
    SOAP::Data->name('guid')->value($GUID)->type(''),
    SOAP::Data->name('applicationName')->value($App)->type(''),
    SOAP::Data->name('serviceTags')->value($servicetag)->type(''),
);
print Dumper($a -> result) ;

The output can look like this:

$VAR1 = {
          'Asset' => {
                     'Entitlements' => {
                                       'EntitlementData' => [
                                                            {
                                                              'EntitlementType' => 'Expired',
                                                              'EndDate' => '2009-07-29T00:00:00',
                                                              'Provider' => '',
                                                              'StartDate' => '2006-07-29T00:00:00',
                                                              'DaysLeft' => '0'
                                                            },
                                                            {
                                                              'EntitlementType' => 'Expired',
                                                              'EndDate' => '2009-07-29T00:00:00',
                                                              'Provider' => '',
                                                              'StartDate' => '2006-07-29T00:00:00',
                                                              'DaysLeft' => '0'
                                                            },
                                                            {
                                                              'EntitlementType' => 'Expired',
                                                              'EndDate' => '2007-07-29T00:00:00',
                                                              'Provider' => '',
                                                              'StartDate' => '2006-07-29T00:00:00',
                                                              'DaysLeft' => '0'
                                                            }
                                                          ]
                                     },
                     'AssetHeaderData' => {
                                          'SystemModel' => 'GX620',
                                          'ServiceTag' => '8DSGD2J',
                                          'SystemShipDate' => '2006-07-29T19:00:00-05:00',
                                          'Buid' => '2323',
                                          'Region' => 'Europe',
                                          'SystemID' => 'PLX_GX620',
                                          'SystemType' => 'OptiPlex'
                                        }
                   }
        };

I have not been able to find any documentation from Dell about this service outside the inline documentation, and according to one comment it can have stability issues, but it is a lot better than scraping HTML pages. :)

Wonder if HP and other server vendors have a similar service. If you know of one, drop me an email. :)

På onsdag rakk jeg såvidt innom Oslo Open Data Forums møte, og fikk lagt hendene mine på DIFIs helt nye veileder "Åpne data. Del og skap verdier. Veileder i tilgjengeliggjøring av offentlig data" (også tilgjengelig som PDF fra DIFI).

Veilederen er veldig bra, og nevner viktige problemstillinger og skisserer f.eks. både verdiskapningspotensialet og formatmulighetene som en bør ha i bakhodet når en publiserer offentlig informasjon på maskinlesbart format. Kildekoden til veilederen er tilgjengelig via github, og en kan rapportere tilbakemeldinger og forslag til forbedringer der (eller via epost og twitter for de som ønsker det).

Det eneste jeg virkelig savnet i veilederen var omtale av w3cs stjerneklassifisering av åpne datakilder, som jeg tror ville være nyttige mentale knagger for de som vurderer å publisere sin informasjon på som åpne data. Jeg har rapportert en github-bug om dette, så får vi se hvordan den blir behandlet.

Det slo meg at det var veldig lite konkret i veilederen om valg av bruksvilkår ved publisering (aka lisens), men jeg er ikke sikker på om det hører hjemme der, da det er et vanskelig tema som kanskje heller hører hjemme i sin egen veileder. Uansett, anbefaler alle å ta en titt på veilederen og sikre at alle offentlige etater en har kontakt med får en kopi.

A few days ago my color calibration gadget ColorHug arrived in the mail, and I've had a few days to test it. As all my machines are running Debian Squeeze, where the calibration software is missing (it is present in Wheezy and Sid), I ran the calibration using the Fedora based live CD. This worked just fine. So far I have only done the quick calibration. It was slow enough for me, so I will leave the more extensive calibration for another day.

After calibration, I get a ICC color profile file that can be passed to programs understanding such tools. KDE do not seem to understand it out of the box, so I searched for command line tools to use to load the color profile into X. xcalib was the first one I found, and it seem to work fine for single monitor setups. But for my video player, a laptop with a flat screen attached, it was unable to load the color profile for the correct monitor. After searching a bit, I discovered that the dispwin tool from the argyll package would do what I wanted, and a simple

dispwin -d 1 profile.icc

later I had the color profile loaded for the correct monitor. The result was a bit more pink than I expected. I guess I picked the wrong monitor type for the "led" monitor I got, but the result is good enough for now.