Petter Reinholdtsen

Just for fun I had a look at the popcon number of ZFS related packages in Debian, and was quite surprised with what I found. I use ZFS myself at home, but did not really expect many others to do so. But I might be wrong.

According to the popcon results for spl-linux, there are 1019 Debian installations, or 0.53% of the population, with the package installed. As far as I know the only use of the spl-linux package is as a support library for ZFS on Linux, so I use it here as proxy for measuring the number of ZFS installation on Linux in Debian. In the kFreeBSD variant of Debian the ZFS feature is already available, and there the popcon results for zfsutils show 1625 Debian installations or 0.84% of the population. So I guess I am not alone in using ZFS on Debian.

But even though the Debian project leader Lucas Nussbaum announced in April 2015 that the legal obstacles blocking ZFS on Debian were cleared, the package is still not in Debian. The package is again in the NEW queue. Several uploads have been rejected so far because the debian/copyright file was incomplete or wrong, but there is no reason to give up. The current status can be seen on the team status page, and the source code is available on Alioth.

As I want ZFS to be included in next version of Debian to make sure my home server can function in the future using only official Debian packages, and the current blocker is to get he debian/copyright file accepted by the FTP masters in Debian, I decided a while back to try to help out the team. This was the background for my blog post about creating, updating and checking debian/copyright semi-automatically, and I used the techniques I explored there to try to find any errors in the copyright file. It is not very easy to check every one of the around 2000 files in the source package, but I hope we this time got it right. If you want to help out, check out the git source and try to find missing entries in the debian/copyright file.

For litt mer enn et år siden laget jeg litt statistikk for 2013 og 2014 basert på Offentlig elektronisk postjournal (OEP). Tallene finnes i tre bloggposter, en over hvem som er mest hemmelighetsfulle, en annen med hvor mye som mangler og en om hvor stor andel dokumenter som journalføres mer enn 180 dager etter dokumentdato. Jeg synes det er interessant å se hvordan det utvikler seg over tid, så her er tallene for 2015 og 2014. 2014-tallene bør være de samme som sist, men kan ha litt avvik på grunn av endringer/korreksjoner gjort i OEP. Denne gangen har jeg samlet alle tallene i samme bloggpost.

Først kommer statistikken over antall dokumenter som er registrert unntatt offentligheten i OEP. Merk at klassifiseringen i OEP er foreløpig, slik at en kan få innsyn i dokumenter merket unntatt offentlighet når en spør, etter nærmere vurdering. Det er dermed ikke gitt at prosentsatsen representerer hvor mange dokumenter som ville blitt holdt tilbake på forespørsel.

2015			2014			Instans
Prosent	Antall u.off.	Totalt	Prosent	Antall u.off.	Totalt
0.0	0	2	0.0	0	0	Valgdirektoratet
0.0	0	30	0.0	0	0	Innstillingsrådet for dommere
0.3	11	3303	0.2	8	3589	Medietilsynet
0.6	56	8013	0.9	60	6146	Utlendingsnemnda
0.9	2	206	0.0	0	0	Norsk Romsenter
1.0	182	17238	3.7	764	20353	Norsk Filminstitutt
1.6	483	29920	2.7	720	26364	Direktoratet for byggkvalitet
1.7	30	1674	2.5	42	1649	Fredskorpset
2.4	2815	116570	2.5	2098	82445	Landbruksdirektoratet Oslo
2.5	621	24150	2.5	689	26855	Direktoratet for utviklingssamarbeid
2.6	62	2340	13.3	288	2164	Kunst i offentlige rom
3.2	272	8391	4.9	388	7838	Statens strålevern
3.7	372	9970	4.4	480	10808	Petroleumstilsynet
4.4	456	10318	3.7	408	10782	Forbrukerombudet
5.0	14	278	11.0	22	200	Kompetansesenter for distriktsutvikling
6.2	467	7486	4.8	385	7973	Sysselmannen på Svalbard
6.3	294	4651	7.4	304	4078	Språkrådet
6.5	1241	18817	6.5	1352	20776	Riksantikvaren - Direktoratet for kulturminneforvaltning
6.8	5381	79048	7.9	5981	75393	Fiskeridirektoratet
7.2	4201	57778	2.8	1542	53911	Norges vassdrags- og energidirektorat
7.4	2028	27385	10.8	2342	21539	Kulturdepartementet
7.7	13994	179777	7.9	13743	171924	Arbeidstilsynet
8.6	3256	37553	8.5	3203	37675	Nærings- og fiskeridepartementet
9.8	7256	73440	8.8	6541	74189	Statens vegvesen Region midt
10.7	3802	35467	7.5	2780	36664	Forsvarsbygg
11.0	1157	10448	9.6	1148	11935	Landbruks- og matdepartementet
11.1	14089	126652	11.7	12439	105491	Statens vegvesen Region vest
11.1	1756	15726	6.9	1242	17980	Brønnøysundregistrene
11.2	3417	30277	9.3	2653	28291	Lotteri- og stiftelsestilsynet
12.2	2012	16490	13.0	2183	16736	Fylkesmannen i Hedmark
12.6	9751	77145	15.3	11188	72933	Sjøfartsdirektoratet
12.8	11031	85996	10.5	7895	74653	Arkivverket
12.8	21126	164906	12.6	19230	151541	Statens vegvesen Region øst
12.9	941	7247	22.8	1687	7393	Domstoladministrasjonen
13.0	14170	108284	14.0	15650	111147	Statens vegvesen Region sør
13.4	1394	10364	17.5	1484	8440	Direktoratet for mineralforvaltning med Bergmesteren for Svalbard
13.6	5297	38707	11.5	4366	37961	Statens vegvesen Vegdirektoratet
14.4	3001	20820	13.0	2943	22557	Samferdselsdepartementet
14.4	9453	65642	12.7	7841	61425	Statens vegvesen Region nord
14.6	6998	47670	11.2	5179	46025	Statsbygg
15.4	3984	25769	14.6	3663	25010	Kunnskapsdepartementet
15.6	6769	43139	8.5	3697	43364	Justervesenet
15.8	178	1126	16.1	184	1140	Landinfo
16.2	516	3183	24.2	919	3790	Landbruksdirektoratet Alta
16.2	1859	11424	12.0	1100	9120	Havforskningsinstituttet
16.3	142	869	0.0	0	0	Fylkesmannen i Aust-Agder (frem til 31.12.15)
16.6	2081	12522	21.1	2722	12850	Statens jernbanetilsyn
17.2	10669	62023	20.9	11602	55388	Utdanningsdirektoratet
18.4	11189	60577	12.9	6743	51881	Miljødirektoratet
18.5	27193	146803	49.9	2549	5108	Norsk kulturråd
18.8	2499	13281	20.8	2888	13853	Olje- og energidepartementet
18.8	7804	41362	15.0	5824	38769	Luftfartstilsynet
19.1	479	2505	21.1	563	2660	Norges geologiske undersøkelse
19.2	13777	71409	20.4	13318	65121	Utenriksdepartementet
19.8	3605	18194	22.1	4177	18833	Klima- og miljødepartementet
20.6	1736	8412	23.1	2018	8701	Statens lånekasse for utdanning
20.6	2930	14220	18.7	2459	13138	Barne-, ungdoms- og familiedirektoratet
21.0	3786	17975	18.4	3213	17452	Fylkesmannen i Telemark
21.4	4930	22971	22.5	4565	20202	Fylkesmannen i Møre og Romsdal
22.0	8244	37385	17.5	6420	36587	Statens kartverk
23.0	2212	9593	23.0	1797	7799	Norsk Akkreditering
23.0	6622	28777	21.4	6142	28622	Nasjonal kommunikasjonsmyndighet
23.4	2672	11405	25.7	3039	11812	Oljedirektoratet
23.6	5430	22918	20.2	4847	23968	Fylkesmannen i Oppland
23.8	3657	15334	22.6	3277	14499	Fylkesmannen i Aust-Agder
23.8	8182	34325	23.7	7754	32598	Kommunal- og moderniseringsdepartementet
24.0	5806	24103	27.4	5998	21864	Barne-, likestillings- og inkluderingsdepartementet
24.2	197	814	31.6	196	620	Statens institutt for rusmiddelforskning
24.4	3266	13338	21.6	2600	11993	Senter for internasjonalisering av utdanning
24.6	4728	19208	24.4	4584	18715	Fylkesmannen i Vest-Agder
24.8	3711	14933	6.4	795	12300	Direktoratet for forvaltning og IKT
24.9	5002	20033	21.6	3997	18424	Fylkesmannen i Nord-Trøndelag
25.1	1246	4954	27.2	1305	4781	Konkurransetilsynet
25.6	5674	22128	20.4	4936	24189	Helse- og omsorgsdepartementet
26.9	22947	85293	24.3	21337	87572	Helsedirektoratet
27.2	3648	13401	18.5	1898	10234	Vox, nasjonalt fagorgan for kompetansepolitikk
27.6	23161	83795	29.7	22587	75973	Statens legemiddelverk
27.7	6373	22941	28.3	6449	22738	Fylkesmannen i Sør-Trøndelag
27.9	784	2807	0.0	0	0	Jernbaneverket
29.6	1290	4352	47.0	1401	2976	Nasjonalt organ for kvalitet i utdanningen (NOKUT)
29.8	6990	23388	30.2	7263	23997	Fylkesmannen i Buskerud
29.9	2932	9778	32.3	3439	10643	Statsministerens kontor
30.3	147	485	0.0	0	0	Fylkesmannen i Vest-Agder (frem til 31.12.115)
31.5	9472	30047	28.9	9498	32839	Fylkesmannen i Rogaland
31.8	1627	5112	34.2	2082	6072	Datatilsynet
31.9	1128	3528	32.9	1302	3950	Statens pensjonskasse
31.9	13718	42918	32.2	12887	39921	Fylkesmannen i Hordaland
32.0	7510	23464	33.0	7544	22837	Fylkesmannen i Vestfold
32.5	1116	3427	32.4	1261	3883	Nasjonalbiblioteket
32.8	1662	5053	48.2	2689	5578	Direktoratet for økonomistyring
33.1	6811	20567	28.7	5041	17513	Fylkesmannen i Sogn og Fjordane
33.7	6975	20666	29.1	6110	20994	Fylkesmannen i Finnmark
33.8	7254	21402	33.5	7138	21304	Finansdepartementet
34.1	387	1133	0.0	0	0	Tilsynsutvalget for dommere
34.5	10466	30261	42.7	7416	17342	Integrerings- og mangfoldsdirektoratet
36.6	16005	43619	40.7	17996	44140	Finanstilsynet
36.7	7170	19525	36.6	7481	20434	Fylkesmannen i Østfold
40.2	16584	41211	32.7	12187	37237	Kystverket
40.9	4942	12080	46.3	5934	12802	Forsvarsdepartementet
41.4	13295	32050	41.3	11805	28568	Fylkesmannen i Troms
43.0	9595	22275	43.9	10288	23422	Arbeids- og sosialdepartementet
44.6	9809	21967	48.0	9627	20021	Politidirektoratet
45.2	25158	55611	43.8	22379	51043	Justis- og beredskapsdepartementet
49.5	18312	36976	49.6	13752	27719	Fylkesmannen i Nordland
49.8	27453	55028	50.0	24782	49473	Fylkesmannen i Oslo og Akershus
50.3	32997	65513	39.5	24644	62276	Direktoratet for samfunnssikkerhet og beredskap
54.4	11213	20606	49.4	7519	15204	Utlendingsdirektoratet
54.5	10338	18948	39.7	5986	15043	Statistisk sentralbyrå
54.8	5075	9256	56.0	4781	8528	Direktoratet for nødkommunikasjon
55.4	7765	13998	46.7	6278	13421	Nasjonalt folkehelseinstitutt
57.6	1690	2931	70.2	3401	4840	Statens innkrevingssentral
58.0	7019	12101	39.2	3671	9352	Departementenes sikkerhets- og serviceorganisasjon
66.2	17293	26097	68.1	18027	26463	Skattedirektoratet
69.5	3949	5676	60.1	2576	4284	Kriminalomsorgsdirektoratet
70.8	16713	23581	70.6	12556	17777	Statens helsetilsyn
71.1	1146	1610	76.5	1160	1516	Statens arbeidsmiljøinstitutt
79.2	9343	11791	74.7	8328	11144	Toll- og avgiftsdirektoratet
83.7	122317	145994	82.2	112645	137034	Husbanken
85.6	84427	98626	87.0	82682	94950	Norges forskningsråd
91.1	609	668	0.0	0	0	Direktoratet for e-helse
91.4	10838	11854	87.1	10278	11798	Garantiinstituttet for eksportkreditt
98.9	17373	17563	0.0	0	0	Forbrukerrådet

Interessant å se hvordan Utenriksdepartementet tilsynelatende er blitt ekstremt mer hemmelighetsfull på tre år. I 2015 var 99,8% av 71409 dokumenter merket som unntatt offentlighet, mens i 2014 var det 44,3% av 65121 og og i 2013 var det bare 12,1% av 67828. Lurer på hva som har skjedd. (Oppdatering 13:40: Dette er en teknisk feil i databasen, da UD har begynt å legge inn '-' der det ikke er unntak, mens det tidligere var et tomt felt. Skal lage ny statistikk, men det tar noen timer å lage ny tabell. Oppdatering 15:30: Ny tabell på plass, mer forklaring nederst i teksten.)

Dernest kommer statistikken over hvor mange saksnummer og dokumenter som ikke finnes i OEP (dvs. hull i saksnummer- og dokumentnummerserien). Se tidligere omtalte bloggpost for å forstå hvordan slike hull kan oppstå uten at det er noe galt eller mistenkelig med det.

Saksnummer						Dokumentnummer			Instans
2015			2014			2015
%	Upubl. saksnr.	Totalt	%	Upubl. saksnr	Totalt	%	Upubl. dok.nr.	Totalt
0.0	0	1	0.0	0	0	0.0	0	2	Valgdirektoratet
0.0	0	85	5.2	6	116	0.0	0	450	Statens institutt for rusmiddelforskning
0.5	100	21258	42.4	9849	23215	0.6	475	85333	Norsk kulturråd
0.6	80	13924	1.0	154	15663	0.2	75	41603	Justervesenet
1.2	20	1622	0.4	7	1568	0.6	90	15028	Statistisk sentralbyrå
1.4	67	4623	1.5	74	5066	1.5	237	15762	Arbeids- og sosialdepartementet
1.4	166	11932	0.9	84	9863	1.7	510	29452	Direktoratet for byggkvalitet
1.4	451	32677	1.0	276	28515	0.4	298	78470	Arkivverket
1.9	11	589	3.0	27	891	0.6	14	2211	Statens innkrevingssentral
1.9	41	2162	2.2	44	1973	1.1	84	7583	Direktoratet for mineralforvaltning med Bergmesteren for Svalbard
2.5	68	2728	4.0	99	2488	2.1	197	9393	Forbrukerombudet
2.7	47	1758	0.6	8	1282	5.1	445	8359	Vox, nasjonalt fagorgan for kompetansepolitikk
2.9	157	5351	3.0	129	4326	0.7	95	12763	Brønnøysundregistrene
3.2	293	9121	2.0	132	6571	6.4	1653	24287	Statens kartverk
3.5	40	1151	3.8	51	1326	6.0	275	4271	Sysselmannen på Svalbard
3.6	53	1458	2.0	30	1470	1.6	155	9331	Norsk Filminstitutt
3.7	313	8449	3.2	227	7092	1.3	413	31241	Lotteri- og stiftelsestilsynet
3.8	645	17136	5.5	942	17213	2.9	1688	57442	Statens legemiddelverk
4.7	353	7482	12.4	1001	8069	2.6	662	25153	Norges vassdrags- og energidirektorat
4.9	940	19148	5.0	896	17781	1.6	1031	63978	Fiskeridirektoratet
5.1	61	1206	4.4	47	1060	3.0	134	4317	Statens strålevern
5.5	95	1721	6.9	126	1813	0.1	8	8231	Statsministerens kontor
5.5	720	12985	14.2	2803	19734	1.4	641	46830	Direktoratet for samfunnssikkerhet og beredskap
5.7	252	4425	4.9	228	4638	1.8	298	16197	Barne-, likestillings- og inkluderingsdepartementet
5.9	207	3534	5.2	207	3997	2.3	230	9930	Klima- og miljødepartementet
5.9	731	12360	6.6	976	14712	2.9	1174	39309	Miljødirektoratet
6.2	472	7629	7.5	496	6649	2.2	1011	44077	Utdanningsdirektoratet
6.5	434	6680	3.2	202	6250	1.5	364	23587	Nasjonal kommunikasjonsmyndighet
6.6	19	287	6.9	28	408	6.5	122	1742	Landbruksdirektoratet Alta
6.8	400	5867	9.0	707	7835	3.0	761	24952	Kommunal- og moderniseringsdepartementet
7.5	496	6587	13.5	1324	9785	1.8	509	27790	Nærings- og fiskeridepartementet
7.6	938	12297	6.6	741	11218	5.6	3412	57624	Norges forskningsråd
8.2	122	1484	10.2	169	1664	6.8	680	9291	Direktoratet for utviklingssamarbeid
8.2	705	8648	9.3	843	9040	3.1	1269	40179	Justis- og beredskapsdepartementet
8.6	97	1133	5.8	78	1336	6.7	179	2511	Medietilsynet
8.6	425	4941	13.6	510	3758	11.4	2721	21076	Integrerings- og mangfoldsdirektoratet
8.7	491	5662	6.8	309	4514	7.1	1712	22269	Luftfartstilsynet
9.0	193	2134	7.6	125	1650	9.4	1442	13937	Statens helsetilsyn
9.3	435	4690	7.6	395	5216	1.3	188	13881	Helse- og omsorgsdepartementet
9.6	142	1477	5.5	81	1486	2.0	155	7737	Petroleumstilsynet
9.6	169	1760	6.9	151	2181	1.2	67	5418	Landbruks- og matdepartementet
9.7	473	4890	11.0	594	5398	2.1	432	19704	Kulturdepartementet
10.0	2	20	17.4	4	23	11.2	14	111	Kompetansesenter for distriktsutvikling
10.1	49	485	6.1	30	492	13.2	486	3187	Konkurransetilsynet
10.2	133	1300	11.5	201	1745	5.3	168	3012	Språkrådet
10.4	1471	14109	9.8	1165	11893	3.4	1976	55929	Helsedirektoratet
10.5	267	2535	7.3	203	2787	4.1	344	8104	Riksantikvaren - Direktoratet for kulturminneforvaltning
10.6	100	941	9.8	95	973	5.7	358	5875	Utlendingsnemnda
10.7	58	541	9.4	51	542	1.6	22	1370	Statens arbeidsmiljøinstitutt
11.0	144	1314	6.9	102	1478	9.5	466	4435	Statens jernbanetilsyn
11.4	487	4281	7.0	189	2700	1.1	253	22570	Samferdselsdepartementet
12.5	622	4957	10.1	497	4902	7.9	1407	16479	Politidirektoratet
12.7	73	573	21.1	107	507	14.8	327	1877	Nasjonalbiblioteket
12.7	738	5829	8.7	519	5949	2.6	414	15481	Finansdepartementet
13.0	176	1352	6.7	81	1204	1.6	120	7347	Oljedirektoratet
13.9	856	6145	10.6	679	6383	5.7	1112	18326	Kunnskapsdepartementet
14.1	715	5073	10.3	609	5933	39.5	15199	23238	Kystverket
14.3	240	1676	2.2	33	1478	1.9	68	3459	Datatilsynet
14.3	319	2223	13.0	301	2309	1.1	65	5903	Olje- og energidepartementet
14.5	24	166	0.0	0	0	17.4	59	281	Norsk Romsenter
14.5	1945	13404	8.7	1096	12632	5.0	1962	37143	Finanstilsynet
15.1	87	578	21.6	94	435	2.8	66	2292	Norges geologiske undersøkelse
15.2	156	1028	12.5	137	1097	2.7	312	11389	Direktoratet for forvaltning og IKT
15.9	501	3157	12.7	379	2996	6.5	550	7966	Nasjonalt folkehelseinstitutt
16.1	58	360	13.8	48	348	8.2	653	7294	Senter for internasjonalisering av utdanning
17.1	692	4040	9.2	285	3085	6.1	484	7439	Forsvarsdepartementet
17.8	161	904	24.9	255	1023	9.7	616	5752	Direktoratet for nødkommunikasjon
17.8	1278	7162	18.7	1359	7269	2.0	458	21904	Fylkesmannen i Troms
17.9	68	379	7.6	27	355	1.9	79	3981	Norsk Akkreditering
19.3	878	4557	19.6	930	4737	2.9	378	12737	Fylkesmannen i Sogn og Fjordane
19.6	1645	8394	21.0	1821	8659	6.4	1621	23898	Fylkesmannen i Nordland
21.0	72	343	0.0	0	0	1.0	10	950	Direktoratet for e-helse
21.0	1102	5237	39.6	2409	6079	30.4	8669	19865	Forsvarsbygg
23.0	384	1671	27.4	544	1986	3.6	251	6633	Statens lånekasse for utdanning
23.1	216	937	23.5	195	829	2.6	110	4116	Direktoratet for økonomistyring
23.7	274	1154	26.0	379	1456	2.5	206	7995	Departementenes sikkerhets- og serviceorganisasjon
24.1	32	133	19.8	24	121	4.4	53	1149	Kunst i offentlige rom
26.0	14552	56067	63.6	67940	106802	7.6	12846	155665	Arbeidstilsynet
28.1	460	1635	27.2	401	1475	8.3	481	5348	Domstoladministrasjonen
31.2	815	2608	34.9	1267	3627	4.1	598	14039	Statsbygg
32.1	88	274	21.6	50	232	5.3	55	974	Fredskorpset
32.9	267	812	33.0	232	702	8.1	239	2717	Nasjonalt organ for kvalitet i utdanningen (NOKUT)
33.0	1665	5046	29.0	1520	5249	5.9	968	15372	Fylkesmannen i Finnmark
33.3	506	1521	27.4	421	1538	5.7	406	6687	Havforskningsinstituttet
34.0	24084	70775	53.8	39567	73495	0.5	515	109638	Landbruksdirektoratet Oslo
37.9	6306	16638	38.4	6123	15947	1.3	443	34527	Fylkesmannen i Hordaland
38.0	3267	8600	39.8	3060	7698	4.3	738	16551	Fylkesmannen i Nord-Trøndelag
40.1	3359	8370	35.9	3135	8744	9.5	1603	15300	Fylkesmannen i Oppland
41.8	3052	7298	43.0	3285	7645	0.8	142	16847	Fylkesmannen i Vestfold
42.0	2260	5377	39.4	2115	5365	2.3	296	12662	Fylkesmannen i Telemark
42.0	2677	6373	42.2	2826	6700	3.5	425	11560	Fylkesmannen i Vest-Agder
42.2	2302	5460	42.7	2283	5350	2.1	217	10030	Fylkesmannen i Aust-Agder
42.3	3249	7688	42.5	3359	7911	3.6	602	15959	Fylkesmannen i Møre og Romsdal
43.1	4750	11013	29.0	1435	4944	4.7	925	18789	Utlendingsdirektoratet
45.3	511	1129	34.1	415	1217	14.9	784	4468	Garantiinstituttet for eksportkreditt
48.4	4663	9627	50.5	5064	10024	4.9	869	16950	Fylkesmannen i Sør-Trøndelag
49.1	4160	8480	48.7	4261	8747	1.0	170	16593	Fylkesmannen i Buskerud
49.9	8296	16617	26.0	3844	14785	1.7	696	41263	Utenriksdepartementet
51.4	7375	14353	43.0	5492	12773	6.0	1457	22780	Fylkesmannen i Rogaland
51.5	4432	8610	51.4	4472	8703	3.8	494	12494	Fylkesmannen i Hedmark
54.3	4718	8689	52.5	4648	8852	3.9	567	13903	Fylkesmannen i Østfold
60.2	6147	10216	99.8	8133	8153	4.0	1010	24464	Forbrukerrådet
60.9	45659	75035	3.5	926	26466	5.7	5706	94908	Husbanken
62.3	15943	25595	59.4	14766	24867	6.6	2978	42345	Fylkesmannen i Oslo og Akershus
76.5	6354	8309	87.2	13391	15363	12.3	1011	7202	Barne-, ungdoms- og familiedirektoratet
78.8	204887	259963	69.6	109962	157962	2.9	3765	128230	Statens vegvesen Region øst
82.5	78652	95340	78.5	54063	68912	2.4	1358	55810	Sjøfartsdirektoratet
83.1	8088	9733	94.6	7535	7968	11.3	953	7468	Jernbaneverket
85.1	221265	259972	80.9	127825	157956	2.2	2345	102032	Statens vegvesen Region vest
86.6	225041	259956	78.4	123819	157956	3.2	2699	81896	Statens vegvesen Region sør
89.9	1466	1630	96.3	1381	1434	19.6	267	1093	Tilsynsutvalget for dommere
89.9	2498	2780	95.4	4656	4883	1.8	18	996	Landinfo
91.2	237188	259944	87.1	137656	157962	2.6	1480	55993	Statens vegvesen Region midt
91.7	5198	5671	99.1	5201	5247	3.3	25	744	Fylkesmannen i Aust-Agder (frem til 31.12.15)
93.4	242735	259930	90.7	143324	157956	3.8	1949	48772	Statens vegvesen Region nord
96.0	6190	6446	99.4	6117	6157	1.9	7	354	Fylkesmannen i Vest-Agder (frem til 31.12.115)
97.2	23095	23766	96.9	23911	24666	42.0	1746	2409	Kriminalomsorgsdirektoratet
97.3	38602	39679	98.2	60831	61922	3.1	83	2626	Statens pensjonskasse
97.4	253080	259768	96.0	151565	157870	9.0	2294	23257	Statens vegvesen Vegdirektoratet
97.8	84989	86865	97.2	101452	104332	12.2	983	7089	Toll- og avgiftsdirektoratet
98.0	1542	1573	99.1	889	897	5.7	4	66	Innstillingsrådet for dommere
99.4	3934	3957	99.6	944	948	52.9	37	33	Barne- og likestillingsdepartementet
99.6	1199296	1204394	99.4	990362	995873	3.5	624	17298	Skattedirektoratet

Til sist er statistikken over hvor stor andel av utgående, inngående, og andre dokumenter som blir journalført mer enn 180 dager etter dokumentets dato, dvs. hvor stor andel som blir journalført ekstremt sent for utgående og interne dokumenter, mens det for innkommende like gjerne kan være at det tok lang tid før de ble mottatt.

2015									Instans
Utgående			Innkommende			Annet
%	# > 180d liggetid	Totalt	%	# > 180d liggetid	Totalt	%	# > 180d liggetid	Totalt
0.0	0	1	0.0	0	1	0.0	0	0	Valgdirektoratet
0.0	0	17	0.0	0	13	0.0	0	0	Innstillingsrådet for dommere
0.0	0	20	0.7	2	258	0.0	0	0	Kompetansesenter for distriktsutvikling
0.0	0	50	0.0	0	156	0.0	0	0	Norsk Romsenter
0.0	0	14410	0.0	0	20625	0.0	0	1941	Fylkesmannen i Nordland
0.0	0	19475	0.0	0	33158	0.0	0	2395	Fylkesmannen i Oslo og Akershus
0.0	2	6513	0.0	2	8366	0.0	1	2684	Forbrukerrådet
0.0	2	14848	0.0	12	13720	0.1	2	1709	Lotteri- og stiftelsestilsynet
0.0	3	3242	0.1	10	6728	0.0	0	0	Petroleumstilsynet
0.0	35	39417	1.8	55	3023	4.1	29	699	Justervesenet
0.1	1	563	0.1	1	568	0.0	0	2	Tilsynsutvalget for dommere
0.1	2	1412	0.1	2	1889	0.0	0	2	Medietilsynet
0.1	3	1922	0.2	7	3032	0.0	0	0	Konkurransetilsynet
0.1	3	2861	0.2	11	4625	0.0	0	0	Sysselmannen på Svalbard
0.1	4	2019	0.3	10	3086	28.5	2	7	Datatilsynet
0.1	10	6724	0.0	5	11621	0.0	0	1180	Fylkesmannen i Østfold
0.1	15	7701	0.2	19	7959	3.0	2	66	Brønnøysundregistrene
0.1	23	13317	0.1	33	16601	0.0	0	2	Direktoratet for byggkvalitet
0.1	207	106179	0.3	285	73027	1.0	6	571	Arbeidstilsynet
0.2	4	1958	1.9	46	2394	0.0	0	0	Nasjonalt organ for kvalitet i utdanningen (NOKUT)
0.2	6	2924	0.1	10	6854	0.0	0	0	Statsministerens kontor
0.2	42	14193	0.1	46	26329	7.4	179	2396	Fylkesmannen i Hordaland
0.3	11	3042	0.1	8	6898	0.2	1	378	Forbrukerombudet
0.3	12	3792	0.0	5	6364	12.0	25	208	Direktoratet for mineralforvaltning med Bergmesteren for Svalbard
0.3	13	3468	0.9	62	6883	1.4	25	1729	Forsvarsdepartementet
0.3	15	4463	0.5	40	7903	1.2	2	156	Statens jernbanetilsyn
0.3	19	5470	0.7	104	13525	6.5	68	1038	Fylkesmannen i Nord-Trøndelag
0.3	130	36848	0.3	148	46603	7.1	183	2545	Arkivverket
0.3	130	38432	1.6	691	42125	1.5	442	27727	Statens vegvesen Region sør
0.4	26	6320	0.1	21	12884	0.0	0	4	Fylkesmannen i Vest-Agder
0.4	29	6530	2.4	362	14872	0.0	0	0	Finansdepartementet
0.4	35	7248	0.4	58	12097	2.2	30	1321	Fylkesmannen i Finnmark
0.4	64	14367	0.4	128	26712	7.0	20	283	Luftfartstilsynet
0.5	6	1090	1.7	25	1415	0.0	0	0	Norges geologiske undersøkelse
0.5	30	5810	1.0	71	6812	1.7	14	779	Vox, nasjonalt fagorgan for kompetansepolitikk
0.5	38	7595	0.6	93	15083	17.4	51	293	Fylkesmannen i Møre og Romsdal
0.5	321	63780	0.2	135	51738	2.0	22	1052	Landbruksdirektoratet Oslo
0.6	21	3049	0.8	71	8356	0.0	0	0	Oljedirektoratet
0.6	58	8800	0.6	123	20412	1.9	163	8341	Nærings- og fiskeridepartementet
0.7	56	7339	1.1	90	8125	25.8	458	1774	Norsk Filminstitutt
0.7	191	26033	0.7	307	42834	2.1	215	10181	Fiskeridirektoratet
0.7	357	44844	0.5	301	52202	1.9	590	29606	Statens vegvesen Region vest
0.8	2	239	0.0	0	246	0.0	0	0	Fylkesmannen i Vest-Agder (frem til 31.12.115)
0.8	119	14631	0.3	39	12465	12.9	217	1681	Nasjonal kommunikasjonsmyndighet
0.9	36	3886	2.0	94	4505	0.0	0	0	Statens strålevern
0.9	41	4128	1.5	141	9101	7.6	4	52	Olje- og energidepartementet
0.9	76	7829	0.6	95	14804	4.2	35	831	Fylkesmannen i Vestfold
0.9	385	41153	0.5	330	62989	0.9	404	42661	Norsk kulturråd
1.0	37	3538	0.2	19	6906	0.0	0	4	Landbruks- og matdepartementet
1.1	753	64926	0.6	433	67975	2.3	747	32005	Statens vegvesen Region øst
1.2	33	2558	2.0	97	4643	0.0	0	46	Domstoladministrasjonen
1.2	49	3849	0.4	20	4078	6.3	31	485	Statens lånekasse for utdanning
1.2	91	7120	0.6	83	13700	0.0	0	0	Samferdselsdepartementet
1.2	179	14775	0.5	167	28844	0.0	0	0	Finanstilsynet
1.3	86	6175	0.3	39	11738	1.8	82	4362	Arbeids- og sosialdepartementet
1.3	101	7741	0.8	115	14226	0.0	0	0	Politidirektoratet
1.3	229	16626	0.8	264	31782	3.9	283	7203	Justis- og beredskapsdepartementet
1.4	13	873	1.5	23	1467	0.0	0	0	Kunst i offentlige rom
1.4	28	2000	1.0	26	2446	7.8	16	205	Språkrådet
1.4	146	10001	0.6	132	19008	2.9	31	1038	Fylkesmannen i Rogaland
1.4	176	12255	0.3	64	18762	9.1	95	1033	Fylkesmannen i Troms
1.5	137	8801	2.2	190	8578	10.7	169	1569	Statistisk sentralbyrå
1.5	298	19717	2.5	833	32313	11.0	637	5748	Norges vassdrags- og energidirektorat
1.5	376	24323	0.4	119	24404	3.5	606	16915	Statens vegvesen Region nord
1.5	424	26628	0.9	268	29189	3.5	623	17623	Statens vegvesen Region midt
1.7	79	4582	8.5	431	5070	6.6	142	2139	Toll- og avgiftsdirektoratet
1.7	469	26133	3.0	923	30195	20.5	1891	9185	Direktoratet for samfunnssikkerhet og beredskap
1.8	23	1272	2.9	64	2155	0.0	0	0	Nasjonalbiblioteket
1.9	150	7763	1.3	202	14603	4.8	50	1022	Fylkesmannen i Buskerud
2.0	25	1217	1.3	21	1556	3.1	13	410	Landbruksdirektoratet Alta
2.0	136	6656	1.7	97	5480	9.3	261	2797	Direktoratet for forvaltning og IKT
2.0	263	12584	0.8	113	13273	3.7	484	12850	Statens vegvesen Vegdirektoratet
2.1	184	8700	0.3	68	17957	1.5	117	7668	Kommunal- og moderniseringsdepartementet
2.3	65	2777	1.6	71	4405	6.9	58	831	Utlendingsnemnda
2.3	118	4986	0.2	30	10584	1.1	11	920	Fylkesmannen i Hedmark
2.4	26	1068	1.2	21	1640	1.7	4	223	Statens innkrevingssentral
2.5	101	3979	6.7	354	5219	8.6	251	2903	Departementenes sikkerhets- og serviceorganisasjon
2.5	119	4698	3.0	134	4434	26.6	123	461	Norsk Akkreditering
2.6	45	1705	0.6	15	2438	0.4	7	1533	Kriminalomsorgsdirektoratet
2.6	140	5260	1.4	165	11499	14.7	212	1435	Klima- og miljødepartementet
2.6	172	6552	0.6	88	13413	4.0	168	4138	Barne-, likestillings- og inkluderingsdepartementet
2.7	63	2292	9.3	259	2761	0.0	0	0	Direktoratet for økonomistyring
2.9	22	741	1.4	13	869	0.0	0	0	Statens arbeidsmiljøinstitutt
2.9	1766	59502	3.7	2457	66026	3.5	729	20466	Husbanken
3.0	198	6525	0.3	36	10925	9.1	48	525	Fylkesmannen i Telemark
3.2	92	2824	1.6	126	7663	8.3	114	1367	Garantiinstituttet for eksportkreditt
3.2	113	3508	1.2	98	7916	0.0	0	0	Havforskningsinstituttet
3.2	178	5477	0.5	48	9264	13.3	79	593	Fylkesmannen i Aust-Agder
3.2	211	6542	2.3	178	7678	0.0	0	0	Barne-, ungdoms- og familiedirektoratet
3.2	499	15144	2.2	562	24560	2.4	767	31705	Utenriksdepartementet
3.3	266	7831	0.3	44	11717	12.5	128	1019	Fylkesmannen i Sogn og Fjordane
3.3	275	8151	2.7	323	11629	16.6	1052	6317	Skattedirektoratet
3.4	887	25797	1.6	569	34780	0.0	0	0	Miljødirektoratet
3.4	1378	39669	1.6	847	49862	2.9	267	9095	Norges forskningsråd
3.6	277	7638	0.3	26	8473	11.8	888	7470	Statens helsetilsyn
3.6	291	8056	1.1	110	9677	13.7	394	2873	Utlendingsdirektoratet
3.7	305	8158	2.9	366	12300	12.8	475	3692	Direktoratet for utviklingssamarbeid
3.7	730	19384	1.1	242	20898	5.5	52	929	Kystverket
4.2	271	6327	1.3	174	12490	0.0	0	0	Riksantikvaren - Direktoratet for kulturminneforvaltning
4.3	19	432	0.0	0	386	7.8	4	51	Fylkesmannen i Aust-Agder (frem til 31.12.15)
4.4	458	10325	3.4	623	18141	14.3	1003	7001	Forsvarsbygg
4.6	22	476	11.4	198	1722	15.4	94	609	Jernbaneverket
4.6	774	16824	8.4	1329	15723	15.6	756	4838	Statens kartverk
5.1	85	1635	1.4	27	1892	0.0	0	1	Statens pensjonskasse
5.1	339	6602	5.1	381	7396	0.0	0	0	Nasjonalt folkehelseinstitutt
5.2	377	7225	0.7	112	15036	9.8	67	680	Fylkesmannen i Sør-Trøndelag
5.7	376	6526	0.3	59	15602	0.0	0	0	Helse- og omsorgsdepartementet
6.1	1645	26666	0.7	220	31017	15.3	665	4340	Utdanningsdirektoratet
6.2	470	7517	0.2	51	19868	0.0	0	0	Kulturdepartementet
6.7	48	706	7.6	74	968	0.0	0	0	Fredskorpset
6.8	19	279	4.6	20	429	4.7	5	106	Statens institutt for rusmiddelforskning
6.9	2173	31077	3.1	1342	43025	32.9	3691	11191	Helsedirektoratet
7.4	1053	14189	6.0	764	12638	32.5	1119	3434	Integrerings- og mangfoldsdirektoratet
8.1	322	3961	7.3	271	3675	8.7	141	1620	Direktoratet for nødkommunikasjon
8.6	51	590	6.8	12	174	23.7	86	362	Landinfo
10.8	1034	9557	1.4	238	16191	9.5	2	21	Kunnskapsdepartementet
13.0	764	5873	5.5	417	7451	42.8	6	14	Senter for internasjonalisering av utdanning
14.6	4088	27956	0.3	171	43364	6.9	402	5825	Sjøfartsdirektoratet
20.2	4464	22016	6.6	3076	46377	2.3	366	15402	Statens legemiddelverk
24.3	5085	20862	9.5	1486	15545	38.7	4368	11263	Statsbygg
28.0	1859	6637	3.2	488	15219	21.6	230	1062	Fylkesmannen i Oppland
32.5	41	126	41.7	172	412	60.0	78	130	Direktoratet for e-helse

Jeg lurer virkelig på hvordan så mange i det offentlige klarer å ikke journalføre utgående dokumenter med en gang de gjøres ferdig. Er det manglende opplæring, vanskelige systemer, vond vilje eller noe helt annet som er årsaken? Det hadde vært interessant å finne ut om det var noen fellestrekk for dokumenter som ikke blir journalført før det er gått et halvt år.

Oppdatering 2016-04-07: En nærmere undersøkelse av OEP-oppføringene til Utenriksdepartementet viste at det var en teknisk feil måten jeg laget statistikken, der alle oppføringer med '-' i unntakgsgrunnlagsfeltet ble talt opp som unntatt offentlighet. Det var UD ikke alene om. Det viste seg å gjelde Barne-, likestillings- og inkluderingsdepartementet (4), Helse- og omsorgsdepartementet (3525), Landbruks- og matdepartementet (1753), Nærings- og fiskeridepartementet (11780), Nærings- og handelsdepartementet (7), Nærings- og handelsdepartementet (frem til 31.12.13) (10417) og Utenriksdepartementet (78798) også. Antall påvirkede oppføringer står i parentes. Jeg har fikset problemet og oppdatert tabellen for hemmelighold over. Oppføringene for Nærings- og handelsdepartementet, Landbruks- og matdepartementet, Utenriksdepartementet og Helse- og omsorgsdepartementet fikk ny plassering i lista.

Two years ago, I had a look at trusted timestamping options available, and among other things noted a still open bug in the tsget script included in openssl that made it harder than necessary to use openssl as a trusted timestamping client. A few days ago I was told the Norwegian government office DIFI is close to releasing their own trusted timestamp service, and in the process I was happy to learn about a replacement for the tsget script using only curl:

openssl ts -query -data "/etc/shells" -cert -sha256 -no_nonce \
  | curl -s -H "Content-Type: application/timestamp-query" \
         --data-binary "@-" http://zeitstempel.dfn.de > etc-shells.tsr
openssl ts -reply -text -in etc-shells.tsr

This produces a binary timestamp file (etc-shells.tsr) which can be used to verify that the content of the file /etc/shell with the calculated sha256 hash existed at the point in time when the request was made. The last command extract the content of the etc-shells.tsr in human readable form. The idea behind such timestamp is to be able to prove using cryptography that the content of a file have not changed since the file was stamped.

To verify that the file on disk match the public key signature in the timestamp file, run the following commands. It make sure you have the required certificate for the trusted timestamp service available and use it to compare the file content with the timestamp. In production, one should of course use a better method to verify the service certificate.

wget -O ca-cert.txt https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt
openssl ts -verify -data /etc/shells -in etc-shells.tsr -CAfile ca-cert.txt -text

Wikipedia have a lot more information about trusted Timestamping and linked timestamping, and there are several trusted timestamping services around, both as commercial services and as free and public services. Among the latter is the zeitstempel.dfn.de service mentioned above and freetsa.org service linked to from the wikipedia web site. I believe the DIFI service should show up on https://tsa.difi.no, but it is not available to the public at the moment. I hope this will change when it is into production. The RFC 3161 trusted timestamping protocol standard is even implemented in LibreOffice, Microsoft Office and Adobe Acrobat, making it possible to verify when a document was created.

I would find it useful to be able to use such trusted timestamp service to make it possible to verify that my stored syslog files have not been tampered with. This is not a new idea. I found one example implemented on the Endian network appliances where the configuration of such feature was described in 2012.

But I could not find any free implementation of such feature when I searched, so I decided to try to build a prototype named syslog-trusted-timestamp. My idea is to generate a timestamp of the old log files after they are rotated, and store the timestamp in the new log file just after rotation. This will form a chain that would make it possible to see if any old log files are tampered with. But syslog is bad at handling kilobytes of binary data, so I decided to base64 encode the timestamp and add an ID and line sequence numbers to the base64 data to make it possible to reassemble the timestamp file again. To use it, simply run it like this:

syslog-trusted-timestamp /path/to/list-of-log-files

This will send a timestamp from one or more timestamp services (not yet decided nor implemented) for each listed file to the syslog using logger(1). To verify the timestamp, the same program is used with the --verify option:

syslog-trusted-timestamp --verify /path/to/log-file /path/to/log-with-timestamp

The verification step is not yet well designed. The current implementation depend on the file path being unique and unchanging, and this is not a solid assumption. It also uses process number as timestamp ID, and this is bound to create ID collisions. I hope to have time to come up with a better way to handle timestamp IDs and verification later.

Please check out the prototype for syslog-trusted-timestamp on github and send suggestions and improvement, or let me know if there already exist a similar system for timestamping logs already to allow me to join forces with others with the same interest.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Since this morning, the battery-stats package in Debian include an extended collector that will collect the complete battery history for later processing and graphing. The original collector store the battery level as percentage of last full level, while the new collector also record battery vendor, model, serial number, design full level, last full level and current battery level. This make it possible to predict the lifetime of the battery as well as visualise the energy flow when the battery is charging or discharging.

The new tools are available in /usr/share/battery-stats/ in the version 0.5.1 package in unstable. Get the new battery level graph and lifetime prediction by running:

/usr/share/battery-stats/battery-stats-graph /var/log/battery-stats.csv

Or select the 'Battery Level Graph' from your application menu.

The flow in/out of the battery can be seen by running (no menu entry yet):

/usr/share/battery-stats/battery-stats-graph-flow

I'm not quite happy with the way the data is visualised, at least when there are few data points. The graphs look a bit better with a few years of data.

A while back one important feature I use in the battery stats collector broke in Debian. The scripts in /usr/lib/pm-utils/power.d/ were no longer executed. I suspect it happened when Jessie started using systemd, but I do not know. The issue is reported as bug #818649 against pm-utils. I managed to work around it by adding an udev rule to call the collector script every time the power connector is connected and disconnected. With this fix in place it was finally time to make a new release of the package, and get it into Debian.

If you are interested in how your laptop battery is doing, please check out the battery-stats in Debian unstable, or rebuild it on Jessie to get it working on Debian stable. :) The upstream source is available from github. As always, patches are very welcome.

Back in 2013 I proposed a way to make paper and PDF invoices easier to process electronically by adding a QR code with the key information about the invoice. I suggested using vCard field definition, to get some standard format for name and address, but any format would work. I did not do anything about the proposal, but hoped someone one day would make something like it. It would make it possible to efficiently send machine readable invoices directly between seller and buyer.

This was the background when I came across a proposal and specification from the web based accounting and invoicing supplier Visma in Sweden called UsingQR. Their PDF invoices contain a QR code with the key information of the invoice in JSON format. This is the typical content of a QR code following the UsingQR specification (based on a real world example, some numbers replaced to get a more bogus entry). I've reformatted the JSON to make it easier to read. Normally this is all on one long line:

{
 "vh":500.00,
 "vm":0,
 "vl":0,
 "uqr":1,
 "tp":1,
 "nme":"Din Leverandør",
 "cc":"NO",
 "cid":"997912345 MVA",
 "iref":"12300001",
 "idt":"20151022",
 "ddt":"20151105",
 "due":2500.0000,
 "cur":"NOK",
 "pt":"BBAN",
 "acc":"17202612345",
 "bc":"BIENNOK1",
 "adr":"0313 OSLO"
}

The interpretation of the fields can be found in the format specification (revision 2 from june 2014). The format seem to have most of the information needed to handle accounting and payment of invoices, at least the fields I have needed so far here in Norway.

Unfortunately, the site and document do not mention anything about the patent, trademark and copyright status of the format and the specification. Because of this, I asked the people behind it back in November to clarify. Ann-Christine Savlid (ann-christine.savlid (at) visma.com) replied that Visma had not applied for patent or trademark protection for this format, and that there were no copyright based usage limitations for the format. I urged her to make sure this was explicitly written on the web pages and in the specification, but unfortunately this has not happened yet. So I guess if there is submarine patents, hidden trademarks or a will to sue for copyright infringements, those starting to use the UsingQR format might be at risk, but if this happen there is some legal defense in the fact that the people behind the format claimed it was safe to do so. At least with patents, there is always a chance of getting sued...

I also asked if they planned to maintain the format in an independent standard organization to give others more confidence that they would participate in the standardization process on equal terms with Visma, but they had no immediate plans for this. Their plan was to work with banks to try to get more users of the format, and evaluate the way forward if the format proved to be popular. I hope they conclude that using an open standard organisation like IETF is the correct place to maintain such specification.

Update 2016-03-20: Via Twitter I became aware of some comments about this blog post that had several useful links and references to similar systems. In the Czech republic, the Czech Banking Association standard #26, with short name SPAYD, uses QR codes with payment information. More information is available from the Wikipedia page on Short Payment Descriptor. And in Germany, there is a system named BezahlCode, (specification v1.8 2013-12-05 available as PDF), which uses QR codes with URL-like formatting using "bank:" as the URI schema/protocol to provide the payment information. There is also the ZUGFeRD file format that perhaps could be transfered using QR codes, but I am not sure if it is done already. Last, in Bolivia there are reports that tax information since november 2014 need to be printed in QR format on invoices. I have not been able to track down a specification for this format, because of my limited language skill sets.

Back in September, I blogged about the system I wrote to collect statistics about my laptop battery, and how it showed the decay and death of this battery (now replaced). I created a simple deb package to handle the collection and graphing, but did not want to upload it to Debian as there were already a battery-stats package in Debian that should do the same thing, and I did not see a point of uploading a competing package when battery-stats could be fixed instead. I reported a few bugs about its non-function, and hoped someone would step in and fix it. But no-one did.

I got tired of waiting a few days ago, and took matters in my own hands. The end result is that I am now the new upstream developer of battery stats (available from github) and part of the team maintaining battery-stats in Debian, and the package in Debian unstable is finally able to collect battery status using the /sys/class/power_supply/ information provided by the Linux kernel. If you install the battery-stats package from unstable now, you will be able to get a graph of the current battery fill level, to get some idea about the status of the battery. The source package build and work just fine in Debian testing and stable (and probably oldstable too, but I have not tested). The default graph you get for that system look like this:

My plans for the future is to merge my old scripts into the battery-stats package, as my old scripts collected a lot more details about the battery. The scripts are merged into the upstream battery-stats git repository already, but I am not convinced they work yet, as I changed a lot of paths along the way. Will have to test a bit more before I make a new release.

I will also consider changing the file format slightly, as I suspect the way I combine several values into one field might make it impossible to know the type of the value when using it for processing and graphing.

If you would like I would like to keep an close eye on your laptop battery, check out the battery-stats package in Debian and on github. I would love some help to improve the system further.

Making packages for Debian requires quite a lot of attention to details. And one of the details is the content of the debian/copyright file, which should list all relevant licenses used by the code in the package in question, preferably in machine readable DEP5 format.

For large packages with lots of contributors it is hard to write and update this file manually, and if you get some detail wrong, the package is normally rejected by the ftpmasters. So getting it right the first time around get the package into Debian faster, and save both you and the ftpmasters some work.. Today, while trying to figure out what was wrong with the zfsonlinux copyright file, I decided to spend some time on figuring out the options for doing this job automatically, or at least semi-automatically.

Lucikly, there are at least two tools available for generating the file based on the code in the source package, debmake and cme. I'm not sure which one of them came first, but both seem to be able to create a sensible draft file. As far as I can tell, none of them can be trusted to get the result just right, so the content need to be polished a bit before the file is OK to upload. I found the debmake option in a blog posts from 2014.

To generate using debmake, use the -cc option:

debmake -cc > debian/copyright

Note there are some problems with python and non-ASCII names, so this might not be the best option.

The cme option is based on a config parsing library, and I found this approach in a blog post from 2015. To generate using cme, use the 'update dpkg-copyright' option:

cme update dpkg-copyright

This will create or update debian/copyright. The cme tool seem to handle UTF-8 names better than debmake.

When the copyright file is created, I would also like some help to check if the file is correct. For this I found two good options, debmake -k and license-reconcile. The former seem to focus on license types and file matching, and is able to detect ineffective blocks in the copyright file. The latter reports missing copyright holders and years, but was confused by inconsistent license names (like CDDL vs. CDDL-1.0). I suspect it is good to use both and fix all issues reported by them before uploading. But I do not know if the tools and the ftpmasters agree on what is important to fix in a copyright file, so the package might still be rejected.

The devscripts tool licensecheck deserve mentioning. It will read through the source and try to find all copyright statements. It is not comparing the result to the content of debian/copyright, but can be useful when verifying the content of the copyright file.

Are you aware of better tools in Debian to create and update debian/copyright file. Please let me know, or blog about it on planet.debian.org.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Update 2016-02-20: I got a tip from Mike Gabriel on how to use licensecheck and cdbs to create a draft copyright file

licensecheck --copyright -r `find * -type f` | \
  /usr/lib/cdbs/licensecheck2dep5 > debian/copyright.auto

He mentioned that he normally check the generated file into the version control system to make it easier to discover license and copyright changes in the upstream source. I will try to do the same with my packages in the future.

Update 2016-02-21: The cme author recommended against using -quiet for new users, so I removed it from the proposed command line.

The appstream system is taking shape in Debian, and one provided feature is a very convenient way to tell you which package to install to make a given firmware file available when the kernel is looking for it. This can be done using apt-file too, but that is for someone else to blog about. :)

Here is a small recipe to find the package with a given firmware file, in this example I am looking for ctfw-3.2.3.0.bin, randomly picked from the set of firmware announced using appstream in Debian unstable. In general you would be looking for the firmware requested by the kernel during kernel module loading. To find the package providing the example file, do like this:

% apt install appstream
[...]
% apt update
[...]
% appstreamcli what-provides firmware:runtime ctfw-3.2.3.0.bin | \
  awk '/Package:/ {print $2}'
firmware-qlogic
%

See the appstream wiki page to learn how to embed the package metadata in a way appstream can use.

This same approach can be used to find any package supporting a given MIME type. This is very useful when you get a file you do not know how to handle. First find the mime type using file --mime-type, and next look up the package providing support for it. Lets say you got an SVG file. Its MIME type is image/svg+xml, and you can find all packages handling this type like this:

% apt install appstream
[...]
% apt update
[...]
% appstreamcli what-provides mimetype image/svg+xml | \
  awk '/Package:/ {print $2}'
bkchem
phototonic
inkscape
shutter
tetzle
geeqie
xia
pinta
gthumb
karbon
comix
mirage
viewnior
postr
ristretto
kolourpaint4
eog
eom
gimagereader
midori
%

I believe the MIME types are fetched from the desktop file for packages providing appstream metadata.

Most people seem not to realise that every time they walk around with the computerised radio beacon known as a mobile phone their position is tracked by the phone company and often stored for a long time (like every time a SMS is received or sent). And if their computerised radio beacon is capable of running programs (often called mobile apps) downloaded from the Internet, these programs are often also capable of tracking their location (if the app requested access during installation). And when these programs send out information to central collection points, the location is often included, unless extra care is taken to not send the location. The provided information is used by several entities, for good and bad (what is good and bad, depend on your point of view). What is certain, is that the private sphere and the right to free movement is challenged and perhaps even eradicated for those announcing their location this way, when they share their whereabouts with private and public entities.

The phone company logs provide a register of locations to check out when one want to figure out what the tracked person was doing. It is unavailable for most of us, but provided to selected government officials, company staff, those illegally buying information from unfaithful servants and crackers stealing the information. But the public information can be collected and analysed, and a free software tool to do so is called Creepy or Cree.py. I discovered it when I read an article about Creepy in the Norwegian newspaper Aftenposten i November 2014, and decided to check if it was available in Debian. The python program was in Debian, but the version in Debian was completely broken and practically unmaintained. I uploaded a new version which did not work quite right, but did not have time to fix it then. This Christmas I decided to finally try to get Creepy operational in Debian. Now a fixed version is available in Debian unstable and testing, and almost all Debian specific patches are now included upstream.

The Creepy program visualises geolocation information fetched from Twitter, Instagram, Flickr and Google+, and allow one to get a complete picture of every social media message posted recently in a given area, or track the movement of a given individual across all these services. Earlier it was possible to use the search API of at least some of these services without identifying oneself, but these days it is impossible. This mean that to use Creepy, you need to configure it to log in as yourself on these services, and provide information to them about your search interests. This should be taken into account when using Creepy, as it will also share information about yourself with the services.

The picture above show the twitter messages sent from (or at least geotagged with a position from) the city centre of Oslo, the capital of Norway. One useful way to use Creepy is to first look at information tagged with an area of interest, and next look at all the information provided by one or more individuals who was in the area. I tested it by checking out which celebrity provide their location in twitter messages by checkout out who sent twitter messages near a Norwegian TV station, and next could track their position over time, making it possible to locate their home and work place, among other things. A similar technique have been used to locate Russian soldiers in Ukraine, and it is both a powerful tool to discover lying governments, and a useful tool to help people understand the value of the private information they provide to the public.

The package is not trivial to backport to Debian Stable/Jessie, as it depend on several python modules currently missing in Jessie (at least python-instagram, python-flickrapi and python-requests-toolbelt).

(I have uploaded the image to screenshots.debian.net and licensed it under the same terms as the Creepy program in Debian.)

During his DebConf15 keynote, Jacob Appelbaum observed that those listening on the Internet lines would have good reason to believe a computer have a given security hole if it download a security fix from a Debian mirror. This is a good reason to always use encrypted connections to the Debian mirror, to make sure those listening do not know which IP address to attack. In August, Richard Hartmann observed that encryption was not enough, when it was possible to interfere download size to security patches or the fact that download took place shortly after a security fix was released, and proposed to always use Tor to download packages from the Debian mirror. He was not the first to propose this, as the apt-transport-tor package by Tim Retout already existed to make it easy to convince apt to use Tor, but I was not aware of that package when I read the blog post from Richard.

Richard discussed the idea with Peter Palfrader, one of the Debian sysadmins, and he set up a Tor hidden service on one of the central Debian mirrors using the address vwakviie2ienjx6t.onion, thus making it possible to download packages directly between two tor nodes, making sure the network traffic always were encrypted.

Here is a short recipe for enabling this on your machine, by installing apt-transport-tor and replacing http and https urls with tor+http and tor+https, and using the hidden service instead of the official Debian mirror site. I recommend installing etckeeper before you start to have a history of the changes done in /etc/.

apt install apt-transport-tor
sed -i 's% http://ftp.debian.org/% tor+http://vwakviie2ienjx6t.onion/%' /etc/apt/sources.list
sed -i 's% http% tor+http%' /etc/apt/sources.list

If you have more sources listed in /etc/apt/sources.list.d/, run the sed commands for these too. The sed command is assuming your are using the ftp.debian.org Debian mirror. Adjust the command (or just edit the file manually) to match your mirror.

This work in Debian Jessie and later. Note that tools like apt-file only recently started using the apt transport system, and do not work with these tor+http URLs. For apt-file you need the version currently in experimental, which need a recent apt version currently only in unstable. So if you need a working apt-file, this is not for you.

Another advantage from this change is that your machine will start using Tor regularly and at fairly random intervals (every time you update the package lists or upgrade or install a new package), thus masking other Tor traffic done from the same machine. Using Tor will become normal for the machine in question.

On Freedombox, APT is set up by default to use apt-transport-tor when Tor is enabled. It would be great if it was the default on any Debian system.