Generated.

[homepage.git] / blog / index.rss

<?xml version="1.0" encoding="utf-8"?>
<rss version='2.0' xmlns:lj='http://www.livejournal.org/rss/lj/1.0/' xmlns:atom="http://www.w3.org/2005/Atom">
        <channel>
                <title>Petter Reinholdtsen</title>
                <description></description>
                <link>https://people.skolelinux.org/pere/blog/</link>
                <atom:link href="https://people.skolelinux.org/pere/blog/index.rss" rel="self" type="application/rss+xml" />
        
        <item>
                <title>45 orphaned Debian packages moved to git, 391 to go</title>
                <link>https://people.skolelinux.org/pere/blog/45_orphaned_Debian_packages_moved_to_git__391_to_go.html</link>
                <guid isPermaLink="true">https://people.skolelinux.org/pere/blog/45_orphaned_Debian_packages_moved_to_git__391_to_go.html</guid>
                <pubDate>Thu, 25 Apr 2024 22:00:00 +0200</pubDate>
                <description>&lt;p&gt;Nine days ago, I started migrating orphaned Debian packages with no
version control system listed in debian/control of the source to git.
At the time there were 438 such packages.  Now there are 391,
according to the UDD.  In reality it is slightly less, as there is a
delay between uploads and UDD updates.  In the nine days since, I have
thus been able to work my way through ten percent of the packages.  I
am starting to run out of steam, and hope someone else will also help
brushing some dust of these packages.  Here is a recipe how to do it.

I start by picking a random package by querying the UDD for a list of
10 random packages from the set of remaining packages:

&lt;blockquote&gt;&lt;pre&gt;
PGPASSWORD=&quot;udd-mirror&quot; psql --port=5432 --host=udd-mirror.debian.net \
  --username=udd-mirror udd -c &quot;select source from sources \
   where release = &#39;sid&#39; and (vcs_url ilike &#39;%anonscm.debian.org%&#39; \
   OR vcs_browser ilike &#39;%anonscm.debian.org%&#39; or vcs_url IS NULL \
   OR vcs_browser IS NULL) AND maintainer ilike &#39;%packages@qa.debian.org%&#39; \
   order by random() limit 10;&quot;
&lt;/pre&gt;&lt;/blockquote&gt;

&lt;p&gt;Next, I visit http://salsa.debian.org/debian and search for the
package name, to ensure no git repository already exist.  If it does,
I clone it and try to get it to an uploadable state, and add the Vcs-*
entries in d/control to make the repository more widely known.  These
packages are a minority, so I will not cover that use case here.&lt;/p&gt;

&lt;p&gt;For packages without an existing git repository, I run the
following script &lt;tt&gt;debian-snap-to-salsa&lt;/tt&gt; to prepare a git
repository with the existing packaging.&lt;/p&gt;

&lt;blockquote&gt;&lt;pre&gt;
#!/bin/sh
#
# See also https://bugs.debian.org/804722#31

set -e

# Move to this Standards-Version.
SV_LATEST=4.7.0

PKG=&quot;$1&quot;

if [ -z &quot;$PKG&quot; ]; then
    echo &quot;usage: $0 &lt;pkgname&gt;&quot;
    exit 1
fi

if [ -e &quot;${PKG}-salsa&quot; ]; then
    echo &quot;error: ${PKG}-salsa already exist, aborting.&quot;
    exit 1
fi

if [ -z &quot;ALLOWFAILURE&quot; ] ; then
    ALLOWFAILURE=false
fi

# Fetch every snapshotted source package.  Manually loop until all
# transfers succeed, as &#39;gbp import-dscs --debsnap&#39; do not fail on
# download failures.
until debsnap --force -v $PKG || $ALLOWFAILURE ; do sleep 1; done
mkdir ${PKG}-salsa; cd ${PKG}-salsa
git init

# Specify branches to override any debian/gbp.conf file present in the
# source package.
gbp import-dscs  --debian-branch=master --upstream-branch=upstream \
    --pristine-tar ../source-$PKG/*.dsc

# Add Vcs pointing to Salsa Debian project (must be manually created
# and pushed to).
if ! grep -q ^Vcs- debian/control ; then
    awk &quot;BEGIN { s=1 } /^\$/ { if (s==1) { print \&quot;Vcs-Browser: https://salsa.debian.org/debian/$PKG\&quot;; print \&quot;Vcs-Git: https://salsa.debian.org/debian/$PKG.git\&quot; }; s=0 } { print }&quot; &lt; debian/control &gt; debian/control.new &amp;&amp; mv debian/control.new debian/control
    git commit -m &quot;Updated vcs in d/control to Salsa.&quot; debian/control
fi

# Tell gbp to enforce the use of pristine-tar.
inifile +inifile debian/gbp.conf +create +section DEFAULT +key pristine-tar +value True
git add debian/gbp.conf
git commit -m &quot;Added d/gbp.conf to enforce the use of pristine-tar.&quot; debian/gbp.conf

# Update to latest Standards-Version.
SV=&quot;$(grep ^Standards-Version: debian/control|awk &#39;{print $2}&#39;)&quot;
if [ $SV_LATEST != $SV ]; then
    sed -i &quot;s/\(Standards-Version: \)\(.*\)/\1$SV_LATEST/&quot; debian/control
    git commit -m &quot;Updated Standards-Version from $SV to $SV_LATEST.&quot; debian/control
fi

if grep -q pkg-config debian/control; then
    sed -i s/pkg-config/pkgconf/ debian/control
    git commit -m &quot;Replaced obsolete pkg-config build dependency with pkgconf.&quot; debian/control
fi

if grep -q libncurses5-dev debian/control; then
    sed -i s/libncurses5-dev/libncurses-dev/ debian/control
    git commit -m &quot;Replaced obsolete libncurses5-dev build dependency with libncurses-dev.&quot; debian/control
fi
&lt;/pre&gt;&lt;/blockquote&gt;

Some times the debsnap script fail to download some of the versions.
In those cases I investigate, and if I decide the failing versions
will not be missed, I call it using ALLOWFAILURE=true to ignore the
problem and create the git repository anyway.&lt;/p&gt;

&lt;p&gt;With the git repository in place, I do a test build (gbp
buildpackage) to ensure the build is actually working.  If it does not
I pick a different package, or if the build failure is trivial to fix,
I fix it before continuing.  At this stage I revisit
http://salsa.debian.org/debian and create the project under this group
for the package.  I then follow the instructions to publish the local
git repository.  Here is from a recent example:&lt;/p&gt;

&lt;blockquote&gt;&lt;pre&gt;
git remote add origin git@salsa.debian.org:debian/perl-byacc.git
git push --set-upstream origin master upstream pristine-tar
git push --tags
&lt;/pre&gt;&lt;/blockquote&gt;

&lt;p&gt;With a working build, I have a look at the build rules if I want to
remove some more dust.  I normally try to move to debhelper compat
level 13, which involves removing debian/compat and modifying
debian/control to build depend on debhelper-compat (=13).  I also test
with &#39;Rules-Requires-Root: no&#39; in debian/control and verify in
debian/rules that hardening is enabled, and include all of these if
the package still build.  If it fail to build with level 13, I try
with 12, 11, 10 and so on until I find a level where it build, as I do
not want to spend a lot of time fixing build issues.&lt;/p&gt;

&lt;p&gt;Some times, when I feel inspired, I make sure debian/copyright is
converted to the machine readable format, often by starting with
&#39;debhelper -cc&#39; and then cleaning up the autogenerated content until
it matches realities.  If I feel like it, I might also clean up
non-dh-based debian/rules files to use the short style dh build
rules.&lt;/p&gt;

&lt;p&gt;Once I have removed all the dust I care to process for the package,
I run &#39;gbp dch&#39; to generate a debian/changelog entry based on the
commits done so far, run &#39;dch -r&#39; to switch from &#39;UNRELEASED&#39; to
&#39;unstable&#39; and get an editor to make sure the &#39;QA upload&#39; marker is in
place and that all long commit descriptions are wrapped into sensible
lengths, run &#39;debcommit --release -a&#39; to commit and tag the new
debian/changelog entry, run &#39;debuild -S&#39; to build a source only
package, and &#39;dput ../perl-byacc_2.0-10_source.changes&#39; to do the
upload.  During the entire process, and many times per step, I run
&#39;debuild&#39; to verify the changes done still work.  I also some times
verify the set of built files using &#39;find debian&#39; to see if I can spot
any problems (like no file in usr/bin any more or empty package).  I
also try to fix all lintian issues reported at the end of each
&#39;debuild&#39; run.&lt;/p&gt;

&lt;p&gt;If I find Debian specific patches, I try to ensure their metadata
is fairly up to date and some times I even try to reach out to
upstream, to make the upstream project aware of the patches.  Most of
my emails bounce, so the success rate is low.  For projects with no
Homepage entry in debian/control I try to track down one, and for
packages with no debian/watch file I try to create one.  But at least
for some of the packages I have been unable to find a functioning
upstream, and must skip both of these.&lt;/p&gt;

&lt;p&gt;If I could handle ten percent in nine days, twenty people could
complete the rest in less then five days.  I use approximately twenty
minutes per package, when I have twenty minutes spare time to spend.
Perhaps you got twenty minutes to spare too?&lt;/p&gt;

&lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
&lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>RAID status from LSI Megaraid controllers in Debian</title>
                <link>https://people.skolelinux.org/pere/blog/RAID_status_from_LSI_Megaraid_controllers_in_Debian.html</link>
                <guid isPermaLink="true">https://people.skolelinux.org/pere/blog/RAID_status_from_LSI_Megaraid_controllers_in_Debian.html</guid>
                <pubDate>Wed, 17 Apr 2024 17:00:00 +0200</pubDate>
                <description>&lt;p&gt;I am happy to report that
&lt;a href=&quot;https://github.com/namiltd/megactl&quot;&gt;the megactl package&lt;/a&gt;,
useful to fetch RAID status when using the LSI Megaraid controller,
now is available in Debian.  It passed NEW a few days ago, and is now
&lt;a href=&quot;https://tracker.debian.org/pkg/megactl&quot;&gt;available in
unstable&lt;/a&gt;, and probably showing up in testing in a weeks time.  The
new version should provide Appstream hardware mapping and should
integrate nicely with isenkram.&lt;/p&gt;

&lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
&lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;

</description>
        </item>
        
        <item>
                <title>Time to move orphaned Debian packages to git</title>
                <link>https://people.skolelinux.org/pere/blog/Time_to_move_orphaned_Debian_packages_to_git.html</link>
                <guid isPermaLink="true">https://people.skolelinux.org/pere/blog/Time_to_move_orphaned_Debian_packages_to_git.html</guid>
                <pubDate>Sun, 14 Apr 2024 09:30:00 +0200</pubDate>
                <description>&lt;p&gt;There are several packages in Debian without a associated git
repository with the packaging history.  This is unfortunate and it
would be nice if more of these would do so.  Quote a lot of these are
without a maintainer, ie listed as maintained by the
&#39;&lt;a href=&quot;https://qa.debian.org/developer.php?email=packages%40qa.debian.org&quot;&gt;Debian
QA Group&lt;/a&gt;&#39; place holder.  In fact, 438 packages have this property
according to UDD (&lt;tt&gt;SELECT source FROM sources WHERE release = &#39;sid&#39;
AND (vcs_url ilike &#39;%anonscm.debian.org%&#39; OR vcs_browser ilike
&#39;%anonscm.debian.org%&#39; or vcs_url IS NULL OR vcs_browser IS NULL) AND
maintainer ilike &#39;%packages@qa.debian.org%&#39;;&lt;/tt&gt;).  Such packages can
be updated without much coordination by any Debian developer, as they
are considered orphaned.&lt;/p&gt;

&lt;p&gt;To try to improve the situation and reduce the number of packages
without associated git repository, I started a few days ago to search
out candiates and provide them with a git repository under the
&#39;debian&#39; collaborative Salsa project.  I started with the packages
pointing to obsolete Alioth git repositories, and am now working my
way across the ones completely without git references.  In addition to
updating the Vcs-* debian/control fields, I try to update
Standards-Version, debhelper compat level, simplify d/rules, switch to
Rules-Requires-Root: no and fix lintian issues reported.  I only
implement those that are trivial to fix, to avoid spending too much
time on each orphaned package.  So far my experience is that it take
aproximately 20 minutes to convert a package without any git
references, and a lot more for packages with existing git repositories
incompatible with git-buildpackages.&lt;/p&gt;

&lt;p&gt;So far I have converted 10 packages, and I will keep going until I
run out of steam.  As should be clear from the numbers, there is
enough packages remaining for more people to do the same without
stepping on each others toes.  I find it useful to start by searching
for a git repo already on salsa, as I find that some times a git repo
has already been created, but no new version is uploaded to Debian
yet.  In those cases I start with the existing git repository.  I
convert to the git-buildpackage+pristine-tar workflow, and ensure a
debian/gbp.conf file with &quot;pristine-tar=True&quot; is added early, to avoid
uploading a orig.tar.gz with the wrong checksum by mistake.  Did that
three times in the begin before I remembered my mistake.&lt;/p&gt;

&lt;p&gt;So, if you are a Debian Developer and got some spare time, perhaps
considering migrating some orphaned packages to git?&lt;/p&gt;

&lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
&lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>Plain text accounting file from your bitcoin transactions</title>
                <link>https://people.skolelinux.org/pere/blog/Plain_text_accounting_file_from_your_bitcoin_transactions.html</link>
                <guid isPermaLink="true">https://people.skolelinux.org/pere/blog/Plain_text_accounting_file_from_your_bitcoin_transactions.html</guid>
                <pubDate>Thu, 7 Mar 2024 18:00:00 +0100</pubDate>
                <description>&lt;p&gt;A while back I wrote a small script to extract the Bitcoin
transactions in a wallet in the
&lt;ahref=&quot;https://plaintextaccounting.org/&quot;&gt;ledger plain text accounting
format&lt;/a&gt;.  The last few days I spent some time to get it working
better with more special cases.  In case it can be useful for others,
here is a copy:&lt;/p&gt;

&lt;p&gt;&lt;blockquote&gt;&lt;pre&gt;
#!/usr/bin/python3
#  -*- coding: utf-8 -*-
#  Copyright (c) 2023-2024 Petter Reinholdtsen

from decimal import Decimal
import json
import subprocess
import time

import numpy

def format_float(num):
    return numpy.format_float_positional(num, trim=&#39;-&#39;)

accounts = {
    u&#39;amount&#39; : &#39;Assets:BTC:main&#39;,
}

addresses = {
    &#39;&lt;some address&gt;&#39; : &#39;Assets:bankkonto&#39;,
    &#39;&lt;some address&gt;&#39; : &#39;Assets:bankkonto&#39;,
}

def exec_json(cmd):
    proc = subprocess.Popen(cmd,stdout=subprocess.PIPE)
    j = json.loads(proc.communicate()[0], parse_float=Decimal)
    return j

def list_txs():
    # get all transactions for all accounts / addresses
    c = 0
    txs = []
    txidfee = {}
    limit=100000
    cmd = [&#39;bitcoin-cli&#39;, &#39;listtransactions&#39;, &#39;*&#39;, str(limit)]
    if True:
        txs.extend(exec_json(cmd))
    else:
        # Useful for debugging
        with open(&#39;transactions.json&#39;) as f:
            txs.extend(json.load(f, parse_float=Decimal))
    #print txs
    for tx in sorted(txs, key=lambda a: a[&#39;time&#39;]):
#        print tx[&#39;category&#39;]
        if &#39;abandoned&#39; in tx and tx[&#39;abandoned&#39;]:
            continue
        if &#39;confirmations&#39; in tx and 0 &gt;= tx[&#39;confirmations&#39;]:
            continue
        when = time.strftime(&#39;%Y-%m-%d %H:%M&#39;, time.localtime(tx[&#39;time&#39;]))
        if &#39;message&#39; in tx:
            desc = tx[&#39;message&#39;]
        elif &#39;comment&#39; in tx:
            desc = tx[&#39;comment&#39;]
        elif &#39;label&#39; in tx:
            desc = tx[&#39;label&#39;]
        else:
            desc = &#39;n/a&#39;
        print(&quot;%s %s&quot; % (when, desc))
        if &#39;address&#39; in tx:
            print(&quot;  ; to bitcoin address %s&quot; % tx[&#39;address&#39;])
        else:
            print(&quot;  ; missing address in transaction, txid=%s&quot; % tx[&#39;txid&#39;])
        print(f&quot;  ; amount={tx[&#39;amount&#39;]}&quot;)
        if &#39;fee&#39;in tx:
            print(f&quot;  ; fee={tx[&#39;fee&#39;]}&quot;)
        for f in accounts.keys():
            if f in tx and Decimal(0) != tx[f]:
                amount = tx[f]
                print(&quot;  %-20s   %s BTC&quot; % (accounts[f], format_float(amount)))
        if &#39;fee&#39; in tx and Decimal(0) != tx[&#39;fee&#39;]:
            # Make sure to list fee used in several transactions only once.
            if &#39;fee&#39; in tx and tx[&#39;txid&#39;] in txidfee \
               and tx[&#39;fee&#39;] == txidfee[tx[&#39;txid&#39;]]:
                True
            else:
                fee = tx[&#39;fee&#39;]
                print(&quot;  %-20s   %s BTC&quot; % (accounts[&#39;amount&#39;], format_float(fee)))
                print(&quot;  %-20s   %s BTC&quot; % (&#39;Expences:BTC-fee&#39;, format_float(-fee)))
                txidfee[tx[&#39;txid&#39;]] = tx[&#39;fee&#39;]

        if &#39;address&#39; in tx and tx[&#39;address&#39;] in addresses:
            print(&quot;  %s&quot; % addresses[tx[&#39;address&#39;]])
        else:
            if &#39;generate&#39; == tx[&#39;category&#39;]:
                print(&quot;  Income:BTC-mining&quot;)
            else:
                if amount &lt; Decimal(0):
                    print(f&quot;  Assets:unknown:sent:update-script-addr-{tx[&#39;address&#39;]}&quot;)
                else:
                    print(f&quot;  Assets:unknown:received:update-script-addr-{tx[&#39;address&#39;]}&quot;)

        print()
        c = c + 1
    print(&quot;# Found %d transactions&quot; % c)
    if limit == c:
        print(f&quot;# Warning: Limit {limit} reached, consider increasing limit.&quot;)

def main():
    list_txs()

main()
&lt;/pre&gt;&lt;/blockquote&gt;&lt;/p&gt;

&lt;p&gt;It is more of a proof of concept, and I do not expect it to handle
all edge cases, but it worked for me, and perhaps you can find it
useful too.&lt;/p&gt;

&lt;p&gt;To get a more interesting result, it is useful to map accounts sent
to or received from to accounting accounts, using the
&lt;tt&gt;addresses&lt;/tt&gt; hash.  As these will be very context dependent, I
leave out my list to allow each user to fill out their own list of
accounts.  Out of the box, &#39;ledger reg BTC:main&#39; should be able to
show the amount of BTCs present in the wallet at any given time in the
past.  For other and more valuable analysis, a account plan need to be
set up in the &lt;tt&gt;addresses&lt;/tt&gt; hash.  Here is an example
transaction:&lt;/p&gt;

&lt;p&gt;&lt;blockquote&gt;&lt;pre&gt;
2024-03-07 17:00 Donated to good cause
    Assets:BTC:main                           -0.1 BTC
    Assets:BTC:main                       -0.00001 BTC
    Expences:BTC-fee                       0.00001 BTC
    Expences:donations                         0.1 BTC
&lt;/pre&gt;&lt;/blockquote&gt;&lt;/p&gt;

&lt;p&gt;It need a running Bitcoin Core daemon running, as it connect to it
using &lt;tt&gt;bitcoin-cli listtransactions * 100000&lt;/tt&gt; to extract the
transactions listed in the Wallet.&lt;/p&gt;

&lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
&lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>RAID status from LSI Megaraid controllers using free software</title>
                <link>https://people.skolelinux.org/pere/blog/RAID_status_from_LSI_Megaraid_controllers_using_free_software.html</link>
                <guid isPermaLink="true">https://people.skolelinux.org/pere/blog/RAID_status_from_LSI_Megaraid_controllers_using_free_software.html</guid>
                <pubDate>Sun, 3 Mar 2024 22:40:00 +0100</pubDate>
                <description>&lt;p&gt;The last few days I have revisited RAID setup using the LSI
Megaraid controller.  These are a family of controllers called PERC by
Dell, and is present in several old PowerEdge servers, and I recently
got my hands on one of these.  I had forgotten how to handle this RAID
controller in Debian, so I had to take a peek in the
&lt;a href=&quot;https://wiki.debian.org/LinuxRaidForAdmins&quot;&gt;Debian wiki page
&quot;Linux and Hardware RAID: an administrator&#39;s summary&quot;&lt;/a&gt; to remember
what kind of software is available to configure and monitor the disks
and controller.  I prefer Free Software alternatives to proprietary
tools, as the later tend to fall into disarray once the manufacturer
loose interest, and often do not work with newer Linux Distributions.
Sadly there is no free software tool to configure the RAID setup, only
to monitor it. RAID can provide improved reliability and resilience in
a storage solution, but only if it is being regularly checked and any
broken disks are being replaced in time.  I thus want to ensure some
automatic monitoring is available.&lt;/p&gt;

&lt;p&gt;In the discovery process, I came across a old free software tool to
monitor PERC2, PERC3, PERC4 and PERC5 controllers, which to my
surprise is not present in debian.  To help change that I created a
&lt;a href=&quot;https://bugs.debian.org/1065322&quot;&gt;request for packaging of the
megactl package&lt;/a&gt;, and tried to track down a usable version.
&lt;a href=&quot;https://sourceforge.net/p/megactl/&quot;&gt;The original project
site&lt;/a&gt; is on Sourceforge, but as far as I can tell that project has
been dead for more than 15 years.  I managed to find a
&lt;a href=&quot;https://github.com/hmage/megactl&quot;&gt;more recent fork on
github&lt;/a&gt; from user hmage, but it is unclear to me if this is still
being maintained.  It has not seen much improvements since 2016.  A
&lt;a href=&quot;https://github.com/namiltd/megactl&quot;&gt;more up to date
edition&lt;/a&gt; is a git fork from the original github fork by user
namiltd, and this newer fork seem a lot more promising.  The owner of
this github repository has replied to change proposals within hours,
and had already added some improvements and support for more hardware.
Sadly he is reluctant to commit to maintaining the tool and stated in
&lt;a href=&quot;https://github.com/namiltd/megactl/pull/1&quot;&gt;my first pull
request&lt;/A&gt; that he think a new release should be made based on the
git repository owned by hmage.  I perfectly understand this
reluctance, as I feel the same about maintaining yet another package
in Debian when I barely have time to take care of the ones I already
maintain, but do not really have high hopes that hmage will have time
to spend on it and hope namiltd will change his mind.&lt;/p&gt;

&lt;p&gt;In any case, I created
&lt;a href=&quot;https://salsa.debian.org/debian/megactl&quot;&gt;a draft package&lt;/a&gt;
based on the namiltd edition and put it under the debian group on
salsa.debian.org.  If you own a Dell PowerEdge server with one of the
PERC controllers, or any other RAID controller using the megaraid or
megaraid_sas Linux kernel modules, you might want to check it out.  If
enough people are interested, perhaps the package will make it into
the Debian archive.&lt;/p&gt;

&lt;p&gt;There are two tools provided, megactl for the megaraid Linux kernel
module, and megasasctl for the megaraid_sas Linux kernel module. The
simple output from the command on one of my machines look like this
(yes, I know some of the disks have problems. :).&lt;/p&gt;

&lt;pre&gt;
# megasasctl 
a0       PERC H730 Mini           encl:1 ldrv:2  batt:good
a0d0       558GiB RAID 1   1x2  optimal
a0d1      3067GiB RAID 0   1x11 optimal
a0e32s0     558GiB  a0d0  online   errs: media:0  other:19
a0e32s1     279GiB  a0d1  online  
a0e32s2     279GiB  a0d1  online  
a0e32s3     279GiB  a0d1  online  
a0e32s4     279GiB  a0d1  online  
a0e32s5     279GiB  a0d1  online  
a0e32s6     279GiB  a0d1  online  
a0e32s8     558GiB  a0d0  online   errs: media:0  other:17
a0e32s9     279GiB  a0d1  online  
a0e32s10    279GiB  a0d1  online  
a0e32s11    279GiB  a0d1  online  
a0e32s12    279GiB  a0d1  online  
a0e32s13    279GiB  a0d1  online  

#
&lt;/pre&gt;

&lt;p&gt;In addition to displaying a simple status report, it can also test
individual drives and print the various event logs.  Perhaps you too
find it useful?&lt;/p&gt;

&lt;p&gt;In the packaging process I provided some patches upstream to
improve installation and ensure
&lt;ahref=&quot;https://github.com/namiltd/megactl/pull/2&quot;&gt;a Appstream
metainfo file is provided&lt;/a&gt; to list all supported HW, to allow
&lt;a href=&quot;https://tracker.debian.org/isenkram&quot;&gt;isenkram&lt;/a&gt; to propose
the package on all servers with a relevant PCI card.&lt;/p&gt;

&lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
&lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;

</description>
        </item>
        
        <item>
                <title>Frokostseminar om Noark 5 i Oslo tirsdag 2024-03-12</title>
                <link>https://people.skolelinux.org/pere/blog/Frokostseminar_om_Noark_5_i_Oslo_tirsdag_2024_03_12.html</link>
                <guid isPermaLink="true">https://people.skolelinux.org/pere/blog/Frokostseminar_om_Noark_5_i_Oslo_tirsdag_2024_03_12.html</guid>
                <pubDate>Tue, 27 Feb 2024 15:15:00 +0100</pubDate>
                <description>&lt;p&gt;Nikita-prosjektet, der jeg er involvert, inviterer i samarbeid med
Oslo Byarkiv, forskningsgruppen METAINFO og foreningen NUUG, til et
frokostseminar om Noark 5 og Noark 5 Tjenestegrensesnitt tirsdag
2024-03-12.  Seminaret finner sted ved Oslo byarkiv.  Vi håper å få
til videostrømming via Internett av presentasjoner og paneldiskusjon.
Oppdatert program og lenker til påmeldingsskjema er
&lt;a href=&quot;https://noark.codeberg.page/noark5-seminars/2023-03-12-noark-workshop.html&quot;&gt;tilgjengelig
fra Nikita-prosjektet&lt;/a&gt;.  Arrangementet er gratis.

&lt;p&gt;Som vanlig, hvis du bruker Bitcoin og ønsker å vise din støtte til
det jeg driver med, setter jeg pris på om du sender Bitcoin-donasjoner
til min adresse
&lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;. Merk,
betaling med bitcoin er ikke anonymt. :)&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>Welcome out of prison, Mickey, hope you find some freedom!</title>
                <link>https://people.skolelinux.org/pere/blog/Welcome_out_of_prison__Mickey__hope_you_find_some_freedom_.html</link>
                <guid isPermaLink="true">https://people.skolelinux.org/pere/blog/Welcome_out_of_prison__Mickey__hope_you_find_some_freedom_.html</guid>
                <pubDate>Mon, 1 Jan 2024 21:00:00 +0100</pubDate>
                <description>&lt;p align=&quot;center&quot;&gt;&lt;img src=&quot;https://people.skolelinux.org/pere/blog/images/2024-01-01-mikke-verk-i-det-fri.jpeg&quot;/&gt;&lt;/p&gt;

&lt;p&gt;Today, the animation figure Mickey Mouse finally was released from
the corporate copyright prison, as the 1928 movie
&lt;a href=&quot;https://en.wikipedia.org/wiki/Steamboat_Willie&quot;&gt;Steamboat
Willie&lt;/a&gt; entered the public domain in USA.  This movie was the first
public appearance of Mickey Mouse.  Sadly the figure is still on
probation, thanks to trademark laws and a the Disney corporations
powerful pack of lawyers, as described in the 2017 article
in &lt;a href=&quot;https://priceonomics.com/how-mickey-mouse-evades-the-public-domain/&quot;&gt;&quot;How
Mickey Mouse Evades the Public Domain&quot;&lt;/a&gt; from Priceonomics.  On the
positive side, the primary driver for repeated extentions of the
duration of copyright has been Disney thanks to Mickey Mouse and the
2028 movie, and as it now in the public domain I hope it will cause
less urge to extend the already unreasonable long copyright
duration.&lt;/p&gt;

&lt;p&gt;The first book I published, the 2004 book &lt;a
href=&quot;https://free-culture.cc/&quot;&gt;&quot;Free Culture&quot; by Lawrence Lessig&lt;/a&gt;,
published 2015 in
&lt;a href=&quot;https://people.skolelinux.org/pere/publisher/#frikultur&quot;&gt;English,
French and Norwegian Bokmål&lt;/a&gt;, touch on the story of Disney pushed
for extending the copyright duration in USA.  It is a great book
explaining problems with the current copyright regime and why we need
Creative Commons movement, and I strongly recommend everyone to read
it.&lt;/p&gt;

&lt;p&gt;This movie (with
&lt;a href=&quot;https://www.imdb.com/title/tt0019422/&quot;&gt;IMDB ID tt0019422&lt;/a&gt;)
is now available from the Internet Archive.  Two copies have been
uploaded so far, one uploaded
&lt;a href=&quot;https://archive.org/details/SteamboatWillie&quot;&gt;2015-11-04&lt;/a&gt;
(&lt;a href=&quot;https://archive.org/download/SteamboatWillie/SteamboatWillie_archive.torrent&quot;&gt;torrent&lt;/a&gt;)
and the other
&lt;a href=&quot;https://archive.org/details/steamboat-willie-mickey&quot;&gt;2023-01-01&lt;/a&gt;
(&lt;a href=&quot;https://archive.org/download/steamboat-willie-mickey/steamboat-willie-mickey_archive.torrent&quot;&gt;torrent&lt;/a&gt;) - see
&lt;a href=&quot;https://people.skolelinux.org/pere/blog/VLC_bittorrent_plugin_still_going_strong__new_upload_2_14_4.html&quot;&gt;VLC
bittorrent plugin&lt;/a&gt; for streaming the video using the torrent link.
I am very happy to see
&lt;a href=&quot;https://people.skolelinux.org/pere/blog/Legal_to_share_more_than_16_000_movies_listed_on_IMDB_.html&quot;&gt;the
number of public domain movies&lt;/a&gt; increasing.  I look forward to
when those are the majority.  Perhaps it will reduce the urge of the
copyright industry to control its customers.&lt;/p&gt;

&lt;p&gt;A more
&lt;a href=&quot;https://publicdomainreview.org/features/entering-the-public-domain/2024/&quot;&gt;comprehensive
list of works entering the public domain in 2024&lt;/a&gt; is available from
the Public Domain Review.&lt;/p&gt;

&lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
&lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>VLC bittorrent plugin still going strong, new upload 2.14-4</title>
                <link>https://people.skolelinux.org/pere/blog/VLC_bittorrent_plugin_still_going_strong__new_upload_2_14_4.html</link>
                <guid isPermaLink="true">https://people.skolelinux.org/pere/blog/VLC_bittorrent_plugin_still_going_strong__new_upload_2_14_4.html</guid>
                <pubDate>Sun, 31 Dec 2023 10:45:00 +0100</pubDate>
                <description>&lt;p&gt;The other day I uploaded a new version of
&lt;a href=&quot;https://tracker.debian.org/pkg/vlc-plugin-bittorrent&quot;&gt;the VLC
bittorrent plugin&lt;/a&gt; to Debian, version 2.14-4, to fix a few
packaging issues.  This plugin extend VLC allowing it to stream videos
directly from a bittorrent source using both torrent files and magnet
links, as easy as using a HTTP or local file source.  I believe such
protocol support is a vital feature in VLC, allowing efficient
streaming from sources such at the 11 million movies in
&lt;a href=&quot;https://archive.org/&quot;&gt;the Internet Archive&lt;/a&gt;.  Bittorrent is
one of the most efficient content distribution protocols on the
Internet, without centralised control, and should be used more.&lt;/p&gt;

&lt;p&gt;The new version is now both in Debian Unstable and Testing, as well
as Ubuntu.  While looking after the package, I decided to ask the VLC
upstream community if there was any hope to get Bittorrent support
into the official VLC program, and was very happy to learn that
someone is already working on it.  I hope we can see some fruits of
that labour next year, but do not hold my breath.  In the mean time we
can use the plugin, which is already
&lt;a href=&quot;https://qa.debian.org/popcon.php?package=vlc-plugin-bittorrent&quot;&gt;installed
by 0.23 percent of the Debian population&lt;/a&gt; according to
popularity-contest.  It could use a new upstream release, and I hope
the upstream developer soon find time to polish it even more.&lt;/p&gt;

&lt;p&gt;It is worth noting that the plugin store the downloaded files in
&lt;tt&gt;~/Downloads/vlc-bittorrent/&lt;/tt&gt;, which can quickly fill up the
user home directory during use.  Users of the plugin should keep an
eye with disk usage when streaming a bittorrent source.&lt;/p&gt;
 
&lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
&lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>«Når «på» blir «pÃ¥»: Et reservoar av tegn sett fra depotet» i tidsskriftet Aksess</title>
                <link>https://people.skolelinux.org/pere/blog/_N_r__p___blir__p_____Et_reservoar_av_tegn_sett_fra_depotet__i_tidsskriftet_Aksess.html</link>
                <guid isPermaLink="true">https://people.skolelinux.org/pere/blog/_N_r__p___blir__p_____Et_reservoar_av_tegn_sett_fra_depotet__i_tidsskriftet_Aksess.html</guid>
                <pubDate>Wed, 15 Nov 2023 09:20:00 +0100</pubDate>
                <description>&lt;p&gt;For noen uker siden skrev en kamerat og meg
&lt;a href=&quot;https://www.aksess-tidsskrift.no/fordypning/175530&quot;&gt;en
artikkel om tegnsett&lt;/a&gt; i
&lt;a href=&quot;https://www.aksess-tidsskrift.no/&quot;&gt;arkivtidsskriftet
Aksess&lt;/a&gt; både på web og i papirutgave nr. 3 2023.  Her er det som
nettopp ble publisert.&lt;/p&gt;

&lt;blockquote&gt;

&lt;p&gt;&lt;strong&gt;Når «på» blir «pÃ¥»: Et reservoar av tegn sett fra
depotet&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;av Thomas Sødring og Petter Reinholdtsen&lt;/p&gt;

&lt;p&gt;De færreste av oss tenker over hva som skjer dypere i datamaskinen
mens vi sitter der og skriver noe på tastaturet. Når du trykker på
tasten «Å», så vises bokstaven Å. Men noen ganger blir det
feil. Hvorfor det – og hva er viktig å være klar over i
arkivsammenheng?&lt;/p&gt;

&lt;p&gt;Dersom bokstaver tolkes forskjellig mellom systemer, blir det fort
rot, dette kalles mojibake blant kjennere, etter det japanske
uttrykket for tegnomforming. Det er en lang historie her som tidvis
har vært preget av rot.  Noen husker kanskje tilbake til en tid der
bokstavene æ, ø og å ofte var ødelagt i e-poster – et klassisk
eksempel på tegnsettproblemstilling.&lt;/p&gt;

&lt;p id=&quot;tegnsett_access_nå_og_før&quot;&gt;&lt;strong&gt;«Nå» og «før»&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Tid er et skjult problem for depot fordi vi danner dokumentasjon i
en kontekst som er preget av å være «nå».  Vår forståelse av verden og
bruken av teknologi er utgangspunktet for denne konteksten. Tenk selv
hvordan verden har utviklet seg de siste 20 årene, hva samfunnet er
opptatt av, og hvordan vi bruker teknologi i hverdagen. Tid er et
skjult problem fordi når vi trekker dokumentasjon ut av systemer og
deponerer for langtidsbevaring, er konteksten til materialet «nå», men
verden går videre. Ettersom teknologien og måten vi bruker den på,
utvikler seg, blir «nå» til «før», og dokumentasjonen befinner seg
snart i en «før»-kontekst.&lt;/p&gt;

&lt;p&gt;Dette med «før» og «nå» i forhold til dokumentasjonens kontekst er
noe vi er veldig lite bevisste på, men det er en problemstilling
depotarkivene eier og forvalter.  En av disse utfordringene er hvorfor
«Ø» ikke nødvendigvis er det samme som «Ø», og hvorfor det i det hele
tatt gir mening å si noe sånt. Vi snakker her om noe som heter
tegnsett, som er en avtalt måte å representere bokstaver, tall og
andre symboler på slik at vi på en feilfri måte kan utveksle tekst
mellom datasystemer.&lt;/p&gt;

&lt;p&gt;Tegnsettproblemstillingen er satt sammen av fire fasetter;
repertoar, representasjon, koding og uttegning.&lt;/p&gt;

&lt;p id=&quot;tegnsett_access_repertoarer&quot;&gt;&lt;strong&gt;Repertoarer&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Repertoar er en samling med tegn og symboler som kan
representeres. Tenk norsk alfabet eller japanske piktogrammer, men
også matematiske og elektroniske symboler. Bokstaven «stor a» kan være
en oppføring i et slikt repertoar. For å kunne brukes i en datamaskin
trenger hver oppføring i et slikt repertoar en representasjon, hvilket
i datamaskinsammenheng betyr at det tilordnes et tall. Tallet kan
lagres på ulike vis i en eller flere kodingsformater. For eksempel kan
en skrive tallet ti som både 10, X og A, i henholdsvis
titallssystemet, romertallssystemet og sekstentallssystemet.&lt;/p&gt;

&lt;p&gt;Hvis en skal kunne lese inn filer og vite hvilket tall og hvilken
representasjon og instans i et repertoar det er snakk om, så må en
vite hvordan tallet er kodet. Sist, men ikke minst, for å kunne bruke
symbolet til noe må det kunne være kjent hvordan det skal se ut eller
tegnes på ark. Det finnes utallige skrifttyper med norske bokstaver,
alle litt forskjellige, og skal en kunne tegne en stor A på skjermen,
så må datamaskinen vite hva den skal tegne. Skrifttyper inneholder
informasjon om hvordan ulike tall skal tegnes. De inneholder ikke
alltid alle symbolene som er brukt i en tekst, hvilket gjør at ikke
alle forståtte tegn vil kunne vises på skjerm eller ark.&lt;/p&gt;

&lt;p&gt;Hver av disse fasettene må være avklart for å kunne ta vare på og vise
frem tekst med en datamaskin. Kombinasjon av repertoar, representasjon
og koding er det en kaller et tegnsett. Kombinasjonen av
representasjon og uttegning kalles en skrifttype. De fleste
skrifttyper har også informasjon om repertoar, men det finnes
skrifttyper som kun kobler mellom tallkode og uttegning, uten å
fortelle noe om hvordan tallkodene egentlig skal tolkes.&lt;/p&gt;

&lt;p id=&quot;tegnsett_access_fra_ascii_til_iso_8859&quot;&gt;&lt;strong&gt;Fra ASCII til ISO-8859&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Vi begynner historien med ASCII (American Standard Code for
Information Interchange) som har en historie som spores tilbake til
1963. Utgangspunktet til ASCII var at det kunne kode opp til 128
forskjellige symboler i vanlig bruk i USA. De visuelle symbolene i
ASCII er de små og store bokstavene (a til z og A til Z), tall (0 til
9) og tegnsettingssymboler (for eksempel semikolon, komma og
punktum). ASCII har også noen usynlige symboler som ble brukt for
bl.a. kommunikasjon. Før ASCII var det for eksempel teleks-tegnsett
med plass til bare 32 tegn og EBCDIC med plass til 256 tegn, alle med
en helt annen rekkefølge på symbolene enn ASCII, men de har vært lite
brukt de siste femti årene. Et eksempel på noen utvalgte symboler i
repertoaret til ASCII vises i tabell 1.&lt;/p&gt;

&lt;table align=&quot;center&quot; width=&quot;50%&quot;&gt;

&lt;caption&gt;Tabell 1. Eksempel på utvalgte symboler hentet fra
ASCII-tegnsettet. Kolonnen «Binær» viser symbolets verdi i
totallssystemet (1 og 0 tall), mens kolonnen «Desimal» viser symbolets
verdi i titallssystemet.&lt;/caption&gt;

&lt;tbody&gt;
&lt;tr&gt;
&lt;th&gt;Grafisk&lt;/th&gt;
&lt;th&gt;Binær&lt;/th&gt;
&lt;th&gt;Desimal&lt;/th&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A&lt;/td&gt;
&lt;td&gt;1000001&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;65&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;M&lt;/td&gt;
&lt;td&gt;1001101&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;77&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Z&lt;/td&gt;
&lt;td&gt;1011010&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;90&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;a&lt;/td&gt;
&lt;td&gt;1100001&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;97&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;m&lt;/td&gt;
&lt;td&gt;1101101&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;109&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;z&lt;/td&gt;
&lt;td&gt;1111010&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;122&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0110000&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;48&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;0111001&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;58&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;;&lt;/td&gt;
&lt;td&gt;0111011&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;59&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;Det opprinnelige ASCII-tegnsettet ble også omtalt som ASCII-7 og
brukte 7 bits (0 og 1) for å representere symboler. Datamaskiner er
ofte konfigurert til å jobbe med enheter der bits er gruppert som 4
eller 8 bits . Det lå en mulighet i å ta i bruk bit åtte. En slik
endring ville gjøre det mulig for datamaskiner å øke antall symboler
de kunne representere, noe som ga en økning fra 128 forskjellige
symboler til 256 forskjellige symboler. Det ble åpnet for å innlemme
de nordiske bokstavene sammen med ASCII, og dette ble etter hvert
standardisert som ISO-8859-1. Tabell 2 viser deler av ISO-8859-1 som
støtter de norske bokstavene.&lt;/p&gt;

&lt;p&gt;Det sier seg selv at muligheten til å representere inntil 256 symboler
ikke holder når vi snakker om en global verden, og det ble gjort et
standardiseringsløp som tok utgangspunkt i ASCII-7 med en utvidelse
til å bruke den åttende biten for ulike språkgrupper. Denne standarden
heter ISO-8859 og er inndelt i opptil 16 varianter, altså fra
ISO-8859-1 til ISO-8859-16.&lt;/p&gt;

&lt;table align=&quot;center&quot; width=&quot;50%&quot;&gt;

&lt;caption&gt;Tabell 2. Koding av de norske symbolene slik de er definert i
ISO-8859-1 tegnsettet.&lt;/caption&gt;

&lt;tbody&gt;
&lt;tr&gt;
&lt;th&gt;Grafisk&lt;/th&gt;
&lt;th&gt;Binær&lt;/th&gt;
&lt;th&gt;Desimal&lt;/th&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Æ&lt;/td&gt;
&lt;td&gt;11000110&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;198&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Ø&lt;/td&gt;
&lt;td&gt;11011000&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;216&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Å&lt;/td&gt;
&lt;td&gt;11000101&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;197&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;æ&lt;/td&gt;
&lt;td&gt;11100110&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;230&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ø&lt;/td&gt;
&lt;td&gt;11111000&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;248&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;å&lt;/td&gt;
&lt;td&gt;11100101&lt;/td&gt;
&lt;td align=&quot;right&quot;&gt;229&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;Norske tegn er definert i ISO-8859-1, som også omtales som Latin 1, de
fleste samiske tegn er definert i ISO-8859-4 (Latin 4) mens tilgang
til €-symbolet kom med ISO-8859-15 (Latin 9). ISO-8859-15 er en
revisjon av ISO-8859-1 som fjerner noen lite brukte symboler og
erstatter bokstaver som er mer brukt, og introduserer €-symbolet.  Det
er viktig å merke at alle ISO-8859-variantene har overlapp med
ASCII-7, noe som ga samvirke med de engelskspråklige landene som ikke
trengte å gjøre noe. Det innebærer også at de første 128 verdiene i
ISO-8859-variantene representerer de samme symbolene. Det er først når
du kommer til tolkningen av de resterende 128 verdiene med nummer 128
til 255, at det oppsto tolkningsutfordringer mellom
ISO-8859-variantene.&lt;/p&gt;

&lt;p&gt;ISO-8859-verdenen fungerte godt så lenge tegnsettet som ble brukt når
innhold ble skapt, også ble brukt når innhold ble gjengitt og du ikke
trengte å kombinere innhold fra forskjellige tegnsett i samme
dokument.  Utfordringen med bruken av ISO-8859-variantene ble raskt
tydelig i en mer globalisert verden med utveksling av tekst på tvers
av landegrenser der tekstlig innhold i dokumenter, e-poster og
websider kunne bli skrevet med ett tegnsett og gjengitt med et annet
tegnsett.&lt;/p&gt;

&lt;table align=&quot;center&quot; width=&quot;60%&quot;&gt;

&lt;caption&gt;Tabell 3. Viser tolkning av verdiene som er tilegnet de
norske symbolene i ISO-8859-1 i de andre ISO 8859-variatene. Merk
ISO-8859-12 ikke finnes da arbeidet ble avsluttet.&lt;sup&gt;[&lt;a id=&quot;tegnsett_access_footnoteref_1&quot; href=&quot;#tegnsett_access_footnotedef_1&quot; title=&quot;View footnote.&quot;&gt;1&lt;/a&gt;]&lt;/sup&gt;&lt;/caption&gt;

&lt;tbody&gt;
&lt;tr&gt;
&lt;th&gt;Binærverdi&lt;/th&gt;
&lt;th&gt;1&lt;/th&gt;
&lt;th&gt;2&lt;/th&gt;
&lt;th&gt;3&lt;/th&gt;
&lt;th&gt;4&lt;/th&gt;
&lt;th&gt;5&lt;/th&gt;
&lt;th&gt;6&lt;/th&gt;
&lt;th&gt;7&lt;/th&gt;
&lt;th&gt;8&lt;/th&gt;
&lt;th&gt;9&lt;/th&gt;
&lt;th&gt;10&lt;/th&gt;
&lt;th&gt;11&lt;/th&gt;
&lt;th&gt;13&lt;/th&gt;
&lt;th&gt;14&lt;/th&gt;
&lt;th&gt;15&lt;/th&gt;
&lt;th&gt;16&lt;/th&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11000110&lt;/td&gt;
&lt;td&gt;Æ&lt;/td&gt;
&lt;td&gt;Ć&lt;/td&gt;
&lt;td&gt;Ĉ&lt;/td&gt;
&lt;td&gt;Æ&lt;/td&gt;
&lt;td&gt;Ц&lt;/td&gt;
&lt;td&gt;ئ&lt;/td&gt;
&lt;td&gt;Ζ&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;Æ&lt;/td&gt;
&lt;td&gt;Æ&lt;/td&gt;
&lt;td&gt;ฦ&lt;/td&gt;
&lt;td&gt;Ę&lt;/td&gt;
&lt;td&gt;Æ&lt;/td&gt;
&lt;td&gt;Æ&lt;/td&gt;
&lt;td&gt;Æ&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11011000&lt;/td&gt;
&lt;td&gt;Ø&lt;/td&gt;
&lt;td&gt;Ř&lt;/td&gt;
&lt;td&gt;Ĝ&lt;/td&gt;
&lt;td&gt;Ø&lt;/td&gt;
&lt;td&gt;и&lt;/td&gt;
&lt;td&gt;ظ&lt;/td&gt;
&lt;td&gt;Ψ&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;Ø&lt;/td&gt;
&lt;td&gt;Ø&lt;/td&gt;
&lt;td&gt;ุ&lt;/td&gt;
&lt;td&gt;Ų&lt;/td&gt;
&lt;td&gt;Ø&lt;/td&gt;
&lt;td&gt;Ø&lt;/td&gt;
&lt;td&gt;Ű&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11000101&lt;/td&gt;
&lt;td&gt;Å&lt;/td&gt;
&lt;td&gt;Ĺ&lt;/td&gt;
&lt;td&gt;Ċ&lt;/td&gt;
&lt;td&gt;Å&lt;/td&gt;
&lt;td&gt;Х&lt;/td&gt;
&lt;td&gt;إ&lt;/td&gt;
&lt;td&gt;Ε&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;Å&lt;/td&gt;
&lt;td&gt;Å&lt;/td&gt;
&lt;td&gt;ล&lt;/td&gt;
&lt;td&gt;Å&lt;/td&gt;
&lt;td&gt;Å&lt;/td&gt;
&lt;td&gt;Å&lt;/td&gt;
&lt;td&gt;Ć&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11100110&lt;/td&gt;
&lt;td&gt;æ&lt;/td&gt;
&lt;td&gt;ć&lt;/td&gt;
&lt;td&gt;ĉ&lt;/td&gt;
&lt;td&gt;æ&lt;/td&gt;
&lt;td&gt;ц&lt;/td&gt;
&lt;td&gt;ن&lt;/td&gt;
&lt;td&gt;ζ&lt;/td&gt;
&lt;td&gt;ז&lt;/td&gt;
&lt;td&gt;æ&lt;/td&gt;
&lt;td&gt;æ&lt;/td&gt;
&lt;td&gt;ๆ&lt;/td&gt;
&lt;td&gt;ę&lt;/td&gt;
&lt;td&gt;æ&lt;/td&gt;
&lt;td&gt;æ&lt;/td&gt;
&lt;td&gt;v&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11111000&lt;/td&gt;
&lt;td&gt;ø&lt;/td&gt;
&lt;td&gt;ř&lt;/td&gt;
&lt;td&gt;ĝ&lt;/td&gt;
&lt;td&gt;ø&lt;/td&gt;
&lt;td&gt;ј&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;ψ&lt;/td&gt;
&lt;td&gt;ר&lt;/td&gt;
&lt;td&gt;ø&lt;/td&gt;
&lt;td&gt;ø&lt;/td&gt;
&lt;td&gt;๘&lt;/td&gt;
&lt;td&gt;ų&lt;/td&gt;
&lt;td&gt;ø&lt;/td&gt;
&lt;td&gt;ø&lt;/td&gt;
&lt;td&gt;ű&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11100101&lt;/td&gt;
&lt;td&gt;å&lt;/td&gt;
&lt;td&gt;ĺ&lt;/td&gt;
&lt;td&gt;ċ&lt;/td&gt;
&lt;td&gt;å&lt;/td&gt;
&lt;td&gt;х&lt;/td&gt;
&lt;td&gt;م&lt;/td&gt;
&lt;td&gt;ε&lt;/td&gt;
&lt;td&gt;ו&lt;/td&gt;
&lt;td&gt;å&lt;/td&gt;
&lt;td&gt;å&lt;/td&gt;
&lt;td&gt;ๅ&lt;/td&gt;
&lt;td&gt;å&lt;/td&gt;
&lt;td&gt;å&lt;/td&gt;
&lt;td&gt;å&lt;/td&gt;
&lt;td&gt;ć&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;Denne problemstillingen er illustrert i tabell 3, der vi ser verdiene
tilegnet de norske symbolene i ISO-8859-1 i kolonne «1». I de øvrige
kolonnene ser vi hvilket symbol verdien får i de andre
ISO-8859-variantene. Tar vi utgangspunkt i tabell 3, kan vi se at
ordet lærlingspørsmål gjengitt med ISO-8859-2 (kolonne 2) blir
lćrlingspřrsmĺl, mens det blir lζrlingspψrsmεl med ISO- 8859-7
(kolonne 7). Med ISO-8859-2 blir «æ» til «ć», «ø» til «ř» og «å» til
 «ĺ». I ISO-8859-7 blir «æ» til «ζ», «ø» til «ψ», mens «å» blir «ε».&lt;/p&gt;

&lt;p&gt;Det er egentlig ingen utfordring med dette så lenge du vet hvilket
tegnsett innholdet ditt er representert med, og det ikke har skjedd
omforminger som du ikke er klar over. Det er det siste som er
problematisk, spesielt de datasystemene som har vært i bruk de siste
20 årene, som ikke har noe innebygd funksjonalitet for å forvalte
tegnsettproblematikken. Et godt eksempel på dette er
Microsoft-tegnsettet Windows-1252, som ble forvekslet som 100 %
kompatibel med ISO-8859-1, men hadde byttet ut plassene fra 127 til
159. Historisk vil det finnes en del variasjon i hvilket tegnsett som
har vært i bruk, og hvor vellykket konvertering mellom tegnsett har
vært.&lt;/p&gt;

&lt;p id=&quot;tegnsett_access_unicode_som_løsning&quot;&gt;&lt;strong&gt;Unicode som løsning&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Tegnsettforvirring ble etter hvert et irritasjonsmoment og
samvirkeproblem. Ofte fikk man en e-post der æøå var erstattet av rare
symboler fordi e-posten hadde vært innom et eller annet datasystem som
ikke brukte samme tegnsett.&lt;/p&gt;

&lt;p&gt;For å løse dette samvirkeproblemet for tegnsett ble det startet et
arbeid og en ny standard så dagens lys etter hvert. Denne standarden
fikk navnet Unicode (ISO/ IEC 10646) og skulle resultere i et tegnsett
som alle skulle være enige om. Unicode er et repertoar og en
representasjon, dvs. navngivning og tilordning av tallverdi til alle
symboler i bruk i verden i dag.  Oppføringer i Unicode skrives gjerne
U+XXXX der XXXX er tallkoden i sekstentallssystemet som oppføringen
har i Unicode-katalogen. Her finner vi tegn brukt av både levende og
døde språk, konstruerte språk, tekniske symboler, morsomme tegninger
(såkalte emojier) og tegn ingen vet hva betyr eller skal brukes
til. Et morsomt eksempel er i nettartikkelen: U+237C ⍼ RIGHT ANGLE
WITH DOWNWARDS ZIGZAG ARROW, av Jonathan Chan.&lt;sup&gt;[&lt;a id=&quot;tegnsett_access_footnoteref_2&quot; href=&quot;#tegnsett_access_footnotedef_2&quot; title=&quot;View footnote.&quot;&gt;2&lt;/a&gt;]&lt;/sup&gt;&lt;/p&gt;

&lt;p&gt;Sammen med Unicode kom det tre måter å kode disse tallene på; UTF-8,
UTF-16 og UTF-32. Av datatekniske årsaker er UTF-8 mye brukt, spesielt
når det gjelder utveksling av tekst over Internett, mens UTF-16 er
brukt en del til tekstfiler lagret på Windows. En utfordring med
Unicode og UTF-variantene er at disse gir flere måter å kode samme
symbol på med en kombinasjonsmekanisme. Dette kan gi utfordringer ved
søk, hvis en skal søke etter et ord som har ett eller flere symboler
som kan skrives på ulikt vis, så er det ikke sikkert at søkesystemet
vil finne alle forekomster. For eksempel kan bokstaven U+00F8 «Latin
Small Letter O with Stroke» kodes som den tradisjonelle norske tegnet
ø, men også som o kombinert med skråstrek U+0338. Begge deler er
gyldig bruk av Unicode, selv om det er tradisjon for å foretrekke å
«normalisere» kombinasjoner som enkelttegn der det er mulig, nettopp
for å forenkle søk.&lt;/p&gt;

&lt;p id=&quot;tegnsett_access_bare_unicode_fremover&quot;&gt;&lt;strong&gt;Bare Unicode fremover&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Forvaltningens bruk av tegnsett er regulert i Forskrift om
IT-standarder i offentlig forvaltning&lt;sup&gt;[&lt;a id=&quot;tegnsett_access_footnoteref_3&quot; href=&quot;#tegnsett_access_footnotedef_3&quot; title=&quot;View footnote.&quot;&gt;3&lt;/a&gt;]&lt;/sup&gt;. Her står det: «Ved all
utveksling av informasjon mellom forvaltningsorganer og fra
forvaltningsorgan til innbyggere og næringsliv skal tegnsettstandarden
ISO/IEC 10646 representert ved UTF8 benyttes.»  Det er forskjellige
bruksområder til UTF-8, UTF-16 og UTF-32, men UTF-8 er kodingen vi
kjenner mest til.  Det er flere grunner at UTF-8 «vant» konkurransen
til å bli den utvalgte. Den kanskje viktigste er at UTF-8 er fullt
samvirkende med ASCII-7, slik at den engelskspråklige delen av verden
kunne rulle ut UTF-8 uten å merke noe forskjell. En tekstfil med kun
ASCII-tekst vil være identisk på disken hvis den lagres som UTF-8 og
ASCII. UTF-16 og UTF-32 byr på noen optimaliseringer som gjør dem
relevant for spesifikke problemområder, men for det meste vil vi aldri
oppleve disse standardene på nært hold i hverdagen. Det er uansett kun
bruken av UTF-8 som er lovregulert i Norge.&lt;/p&gt;

&lt;p&gt;Det er ikke slik at hele verden bruker ISO/IEC 10646 og UTF-8. Kina
har egne standarder for tegnsett, mye brukt er GB 18030, som er
Unicode med en annen koding enn UTF-8, mens Taiwan og andre asiatiske
land gjerne bruker Big5 eller andre tegnsett.&lt;/p&gt;

&lt;p&gt;UTF-8 er dominerende i Norge, men det er tidsperioder der forskjellige
datasystemer utvekslet data i henhold til ISO-8859-1, ISO-8859-15,
Windows-1252, Codepage 865 og ISO-646-60 / Codepage 1016 mens
overgangen til UTF-8 pågikk. Det er ikke slik at et datasystem enkelt
kan tvinges til å bruke et tegnsett, da det er flere lag i et
datasystem som må settes opp til å bruke riktig tegnsett, og
tegnsettproblemet fort oppstår når det er et eller annet i
datasystemet som bruker feil tegnsett.&lt;/p&gt;

&lt;p&gt;Et klassisk eksempel på problemet er en utveksling av tekst mellom to
systemer der teksten i utgangspunktet er kodet i UTF-8, men går
gjennom noe som er ISO-8859-1 underveis. Dette kan vises med at ordet
«på» i et slik scenario ender opp som «pÃ¥». Det er mulig å spore
dette tilbake til verdiene symbolene er tilordnet i tegnsettene. «på»
blir til «pÃ¥» fordi «å» i UTF-8 er representert med U+C3AF, og dersom
vi ser på hva disse verdiene representerer, ser vi at
sekstentallssystemverdien C3 er 1100 0011 i totallssystemet og
symbolet med dette tallet i ISO-8859-1 er Ã.&lt;/p&gt;

&lt;p&gt;Vi ser det samme med sekstentallssystemverdien A5, som er 1010 0101 i
totallssystemet, og tilsvarende symbol i ISO-8859-1 er ¥. Slik
mojibake kan lett skje hvis «på» i utgangspunktet var representert med
UTF-8, men ble behandlet med et system som bruker ISO-8859-1. Det er
ingen automatikk i å fange opp slike ødeleggelser mens tekstlig
innhold utveksles mellom datasystemer.&lt;/p&gt;

&lt;p&gt;En utfordring for depotarkivene er at bruken av tegnsett ikke alltid
har vært regulert, og at det kan finnes flere dokumentasjonssamlinger
som er opprettet med varierende tegnsett før gjeldende forskrift
inntraff – uten at det er mulig å avlede fra filene hvilket tegnsett
som ble brukt. Et eksempel på dette er €-symbolet, som kom først etter
at ISO-8859-1 var tatt i bruk. Det kan bli en utfordring for et
depotarkiv, men så lenge det er kjent hvilket tegnsett var i bruk, så
bør det gå bra.  Riksarkivarens
forskrift&lt;sup&gt;[&lt;a id=&quot;tegnsett_access_footnoteref_4&quot; href=&quot;#tegnsett_access_footnotedef_4&quot; title=&quot;View footnote.&quot;&gt;4&lt;/a&gt;]&lt;/sup&gt;
formaliserer dette ved å kreve følgende:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;§ 5-11. Tegnsett i arkivuttrekk&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Arkivuttrekk og medfølgende struktur- og innholdsbeskrivelser skal
      overføres som ren tekst i ukryptert form, og benytte godkjent
      tegnsett.&lt;/li&gt;

  &lt;li&gt;Godkjente tegnsett er:
    &lt;ol&gt;
      &lt;li&gt;Unicode UTF-8&lt;br&gt;
          (ISO/IEC 10646-1:2000 Annex D)&lt;/li&gt;
      &lt;li&gt;ISO 8859-1:1998, Latin 1&lt;/li&gt;
      &lt;li&gt;ISO 8859-4:1998, Latin 4 for samiske tegn.&lt;/li&gt;
    &lt;/ol&gt;&lt;/li&gt;

  &lt;li&gt;Andre tegnsett aksepteres bare etter avtale med Arkivverket.&lt;/li&gt;
&lt;/ol&gt;
&lt;/blockquote&gt;

&lt;p id=&quot;tegnsett_access_ditt_ansvar&quot;&gt;&lt;strong&gt;Ditt ansvar&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;På mange måter burde ikke tegnsett være et problem i 2023, men sånn er
det nok ikke. Land som har oppgradert til UTF-8 som primærtegnsett for
utveksling av tekstlig innhold, begrenser problematikken betraktelig,
men globalt sett så er tegnsettutfordringen ikke løst fordi ikke alle
er enige om å bruke samme tegnsett.  Det kan være geopolitiske eller
kulturelle hensyn som ligger til grunn for dette.&lt;/p&gt;

&lt;p&gt;Det er uansett verdt å merke at selv om bruken av UTF-8 skulle bli
100% utbredt, så er det et historisk perspektiv (ASCII-7,
ISO-8859-variantene, UTF-8) her som gjør tegnsett til et problemområde
arkivarene må forstå og håndtere.  Som danningsarkivar har du et
ansvar for å vite hvilket tegnsett systemene og databasene dere
forvalter, er i samsvar med. Det er noe IT-avdelingen din eller
programvareleverandørene enkelt skal kunne svare på, og svaret skal
være UTF-8 for alle nye systemer.&lt;/p&gt;

&lt;hr&gt;

&lt;p id=&quot;tegnsett_access_footnotedef_1&quot;&gt;&lt;a href=&quot;#tegnsett_access_footnoteref_1&quot;&gt;1&lt;/a&gt;. Tegnsettkilde &lt;a href=&quot;https://en.wikipedia.org/wiki/ISO/IEC_8859&quot;&gt;https://en.wikipedia.org/wiki/ISO/IEC_8859&lt;/a&gt;&lt;/p&gt;

&lt;p id=&quot;tegnsett_access_footnotedef_2&quot;&gt;&lt;a href=&quot;#tegnsett_access_footnoteref_2&quot;&gt;2&lt;/a&gt;. &lt;a href=&quot;https://ionathan.ch/2022/04/09/angzarr.html&quot;&gt;https://ionathan.ch/2022/04/09/angzarr.html&lt;/a&gt;&lt;/p&gt;

&lt;p id=&quot;tegnsett_access_footnotedef_3&quot;&gt;&lt;a href=&quot;#tegnsett_access_footnoteref_3&quot;&gt;3&lt;/a&gt;. &lt;a href=&quot;https://lovdata.no/dokument/SF/forskrift/2013-04-05-959/%C2%A78#%C2%A78&quot;&gt;https://lovdata.no/dokument/SF/forskrift/2013-04-05-959/%C2%A78#%C2%A78&lt;/a&gt;&lt;/p&gt;

&lt;p id=&quot;tegnsett_access_footnotedef_4&quot;&gt;&lt;a href=&quot;#tegnsett_access_footnoteref_4&quot;&gt;4&lt;/a&gt;. &lt;a href=&quot;https://lovdata.no/forskrift/2017-12-19-2286/§5-11&quot;&gt;https://lovdata.no/forskrift/2017-12-19-2286/§5-11&lt;/a&gt;&lt;/p&gt;

&lt;/blockquote&gt;

&lt;p&gt;For øvrig burde varsleren Edward Snowden få politisk asyl i Norge.&lt;/p&gt;

&lt;p&gt;Som vanlig, hvis du bruker Bitcoin og ønsker å vise din støtte til
det jeg driver med, setter jeg pris på om du sender Bitcoin-donasjoner
til min adresse
&lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;. Merk,
betaling med bitcoin er ikke anonymt. :)&lt;/p&gt;
</description>
        </item>
        
        <item>
                <title>New and improved sqlcipher in Debian for accessing Signal database</title>
                <link>https://people.skolelinux.org/pere/blog/New_and_improved_sqlcipher_in_Debian_for_accessing_Signal_database.html</link>
                <guid isPermaLink="true">https://people.skolelinux.org/pere/blog/New_and_improved_sqlcipher_in_Debian_for_accessing_Signal_database.html</guid>
                <pubDate>Sun, 12 Nov 2023 12:00:00 +0100</pubDate>
                <description>&lt;p&gt;For a while now I wanted to have direct access to the
&lt;a href=&quot;https://signal.org/&quot;&gt;Signal&lt;/a&gt; database of messages and
channels of my Desktop edition of Signal.  I prefer the enforced end
to end encryption of Signal these days for my communication with
friends and family, to increase the level of safety and privacy as
well as raising the cost of the mass surveillance government and
non-government entities practice these days.  In August I came across
a nice
&lt;a href=&quot;https://www.yoranbrondsema.com/post/the-guide-to-extracting-statistics-from-your-signal-conversations/&quot;&gt;recipe
on how to use sqlcipher to extract statistics from the Signal
database&lt;/a&gt; explaining how to do this.  Unfortunately this did not
work with the version of sqlcipher in Debian.  The
&lt;a href=&quot;http://tracker.debian.org/sqlcipher/&quot;&gt;sqlcipher&lt;/a&gt;
package is a &quot;fork&quot; of the sqlite package with added support for
encrypted databases.  Sadly the current Debian maintainer
&lt;a href=&quot;https://bugs.debian.org/961598&quot;&gt;announced more than three
years ago that he did not have time to maintain sqlcipher&lt;/a&gt;, so it
seemed unlikely to be upgraded by the maintainer.  I was reluctant to
take on the job myself, as I have very limited experience maintaining
shared libraries in Debian.  After waiting and hoping for a few
months, I gave up the last  week, and set out to update the package.  In
the process I orphaned it to make it more obvious for the next person
looking at it that the package need proper maintenance.&lt;/p&gt;

&lt;p&gt;The version in Debian was around five years old, and quite a lot of
changes had taken place upstream into the Debian maintenance git
repository.  After spending a few days importing the new upstream
versions, realising that upstream did not care much for SONAME
versioning as I saw library symbols being both added and removed with
minor version number changes to the project, I concluded that I had to
do a SONAME bump of the library package to avoid surprising the
reverse dependencies.  I even added a simple
autopkgtest script to ensure the package work as intended.  Dug deep
into the hole of learning shared library maintenance, I set out a few
days ago to upload the new version to Debian experimental to see what
the quality assurance framework in Debian had to say about the result.
The feedback told me the pacakge was not too shabby, and yesterday I
uploaded the latest version to Debian unstable.  It should enter
testing today or tomorrow, perhaps delayed by
&lt;a href=&quot;https://bugs.debian.org/1055812&quot;&gt;a small library
transition&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Armed with a new version of sqlcipher, I can now have a look at the
SQL database in ~/.config/Signal/sql/db.sqlite.  First, one need to
fetch the encryption key from the Signal configuration using this
simple JSON extraction command:&lt;/p&gt;

&lt;pre&gt;/usr/bin/jq -r &#39;.&quot;key&quot;&#39; ~/.config/Signal/config.json&lt;/pre&gt;

&lt;p&gt;Assuming the result from that command is &#39;secretkey&#39;, which is a
hexadecimal number representing the key used to encrypt the database.
Next, one can now connect to the database and inject the encryption
key for access via SQL to fetch information from the database.  Here
is an example dumping the database structure:&lt;/p&gt;

&lt;pre&gt;
% sqlcipher ~/.config/Signal/sql/db.sqlite
sqlite&gt; PRAGMA key = &quot;x&#39;secretkey&#39;&quot;;
sqlite&gt; .schema
CREATE TABLE sqlite_stat1(tbl,idx,stat);
CREATE TABLE conversations(
      id STRING PRIMARY KEY ASC,
      json TEXT,

      active_at INTEGER,
      type STRING,
      members TEXT,
      name TEXT,
      profileName TEXT
    , profileFamilyName TEXT, profileFullName TEXT, e164 TEXT, serviceId TEXT, groupId TEXT, profileLastFetchedAt INTEGER);
CREATE TABLE identityKeys(
      id STRING PRIMARY KEY ASC,
      json TEXT
    );
CREATE TABLE items(
      id STRING PRIMARY KEY ASC,
      json TEXT
    );
CREATE TABLE sessions(
      id TEXT PRIMARY KEY,
      conversationId TEXT,
      json TEXT
    , ourServiceId STRING, serviceId STRING);
CREATE TABLE attachment_downloads(
    id STRING primary key,
    timestamp INTEGER,
    pending INTEGER,
    json TEXT
  );
CREATE TABLE sticker_packs(
    id TEXT PRIMARY KEY,
    key TEXT NOT NULL,

    author STRING,
    coverStickerId INTEGER,
    createdAt INTEGER,
    downloadAttempts INTEGER,
    installedAt INTEGER,
    lastUsed INTEGER,
    status STRING,
    stickerCount INTEGER,
    title STRING
  , attemptedStatus STRING, position INTEGER DEFAULT 0 NOT NULL, storageID STRING, storageVersion INTEGER, storageUnknownFields BLOB, storageNeedsSync
      INTEGER DEFAULT 0 NOT NULL);
CREATE TABLE stickers(
    id INTEGER NOT NULL,
    packId TEXT NOT NULL,

    emoji STRING,
    height INTEGER,
    isCoverOnly INTEGER,
    lastUsed INTEGER,
    path STRING,
    width INTEGER,

    PRIMARY KEY (id, packId),
    CONSTRAINT stickers_fk
      FOREIGN KEY (packId)
      REFERENCES sticker_packs(id)
      ON DELETE CASCADE
  );
CREATE TABLE sticker_references(
    messageId STRING,
    packId TEXT,
    CONSTRAINT sticker_references_fk
      FOREIGN KEY(packId)
      REFERENCES sticker_packs(id)
      ON DELETE CASCADE
  );
CREATE TABLE emojis(
    shortName TEXT PRIMARY KEY,
    lastUsage INTEGER
  );
CREATE TABLE messages(
        rowid INTEGER PRIMARY KEY ASC,
        id STRING UNIQUE,
        json TEXT,
        readStatus INTEGER,
        expires_at INTEGER,
        sent_at INTEGER,
        schemaVersion INTEGER,
        conversationId STRING,
        received_at INTEGER,
        source STRING,
        hasAttachments INTEGER,
        hasFileAttachments INTEGER,
        hasVisualMediaAttachments INTEGER,
        expireTimer INTEGER,
        expirationStartTimestamp INTEGER,
        type STRING,
        body TEXT,
        messageTimer INTEGER,
        messageTimerStart INTEGER,
        messageTimerExpiresAt INTEGER,
        isErased INTEGER,
        isViewOnce INTEGER,
        sourceServiceId TEXT, serverGuid STRING NULL, sourceDevice INTEGER, storyId STRING, isStory INTEGER
        GENERATED ALWAYS AS (type IS &#39;story&#39;), isChangeCreatedByUs INTEGER NOT NULL DEFAULT 0, isTimerChangeFromSync INTEGER
        GENERATED ALWAYS AS (
          json_extract(json, &#39;$.expirationTimerUpdate.fromSync&#39;) IS 1
        ), seenStatus NUMBER default 0, storyDistributionListId STRING, expiresAt INT
        GENERATED ALWAYS
        AS (ifnull(
          expirationStartTimestamp + (expireTimer * 1000),
          9007199254740991
        )), shouldAffectActivity INTEGER
        GENERATED ALWAYS AS (
          type IS NULL
          OR
          type NOT IN (
            &#39;change-number-notification&#39;,
            &#39;contact-removed-notification&#39;,
            &#39;conversation-merge&#39;,
            &#39;group-v1-migration&#39;,
            &#39;keychange&#39;,
            &#39;message-history-unsynced&#39;,
            &#39;profile-change&#39;,
            &#39;story&#39;,
            &#39;universal-timer-notification&#39;,
            &#39;verified-change&#39;
          )
        ), shouldAffectPreview INTEGER
        GENERATED ALWAYS AS (
          type IS NULL
          OR
          type NOT IN (
            &#39;change-number-notification&#39;,
            &#39;contact-removed-notification&#39;,
            &#39;conversation-merge&#39;,
            &#39;group-v1-migration&#39;,
            &#39;keychange&#39;,
            &#39;message-history-unsynced&#39;,
            &#39;profile-change&#39;,
            &#39;story&#39;,
            &#39;universal-timer-notification&#39;,
            &#39;verified-change&#39;
          )
        ), isUserInitiatedMessage INTEGER
        GENERATED ALWAYS AS (
          type IS NULL
          OR
          type NOT IN (
            &#39;change-number-notification&#39;,
            &#39;contact-removed-notification&#39;,
            &#39;conversation-merge&#39;,
            &#39;group-v1-migration&#39;,
            &#39;group-v2-change&#39;,
            &#39;keychange&#39;,
            &#39;message-history-unsynced&#39;,
            &#39;profile-change&#39;,
            &#39;story&#39;,
            &#39;universal-timer-notification&#39;,
            &#39;verified-change&#39;
          )
        ), mentionsMe INTEGER NOT NULL DEFAULT 0, isGroupLeaveEvent INTEGER
        GENERATED ALWAYS AS (
          type IS &#39;group-v2-change&#39; AND
          json_array_length(json_extract(json, &#39;$.groupV2Change.details&#39;)) IS 1 AND
          json_extract(json, &#39;$.groupV2Change.details[0].type&#39;) IS &#39;member-remove&#39; AND
          json_extract(json, &#39;$.groupV2Change.from&#39;) IS NOT NULL AND
          json_extract(json, &#39;$.groupV2Change.from&#39;) IS json_extract(json, &#39;$.groupV2Change.details[0].aci&#39;)
        ), isGroupLeaveEventFromOther INTEGER
        GENERATED ALWAYS AS (
          isGroupLeaveEvent IS 1
          AND
          isChangeCreatedByUs IS 0
        ), callId TEXT
        GENERATED ALWAYS AS (
          json_extract(json, &#39;$.callId&#39;)
        ));
CREATE TABLE sqlite_stat4(tbl,idx,neq,nlt,ndlt,sample);
CREATE TABLE jobs(
        id TEXT PRIMARY KEY,
        queueType TEXT STRING NOT NULL,
        timestamp INTEGER NOT NULL,
        data STRING TEXT
      );
CREATE TABLE reactions(
        conversationId STRING,
        emoji STRING,
        fromId STRING,
        messageReceivedAt INTEGER,
        targetAuthorAci STRING,
        targetTimestamp INTEGER,
        unread INTEGER
      , messageId STRING);
CREATE TABLE senderKeys(
        id TEXT PRIMARY KEY NOT NULL,
        senderId TEXT NOT NULL,
        distributionId TEXT NOT NULL,
        data BLOB NOT NULL,
        lastUpdatedDate NUMBER NOT NULL
      );
CREATE TABLE unprocessed(
        id STRING PRIMARY KEY ASC,
        timestamp INTEGER,
        version INTEGER,
        attempts INTEGER,
        envelope TEXT,
        decrypted TEXT,
        source TEXT,
        serverTimestamp INTEGER,
        sourceServiceId STRING
      , serverGuid STRING NULL, sourceDevice INTEGER, receivedAtCounter INTEGER, urgent INTEGER, story INTEGER);
CREATE TABLE sendLogPayloads(
        id INTEGER PRIMARY KEY ASC,

        timestamp INTEGER NOT NULL,
        contentHint INTEGER NOT NULL,
        proto BLOB NOT NULL
      , urgent INTEGER, hasPniSignatureMessage INTEGER DEFAULT 0 NOT NULL);
CREATE TABLE sendLogRecipients(
        payloadId INTEGER NOT NULL,

        recipientServiceId STRING NOT NULL,
        deviceId INTEGER NOT NULL,

        PRIMARY KEY (payloadId, recipientServiceId, deviceId),

        CONSTRAINT sendLogRecipientsForeignKey
          FOREIGN KEY (payloadId)
          REFERENCES sendLogPayloads(id)
          ON DELETE CASCADE
      );
CREATE TABLE sendLogMessageIds(
        payloadId INTEGER NOT NULL,

        messageId STRING NOT NULL,

        PRIMARY KEY (payloadId, messageId),

        CONSTRAINT sendLogMessageIdsForeignKey
          FOREIGN KEY (payloadId)
          REFERENCES sendLogPayloads(id)
          ON DELETE CASCADE
      );
CREATE TABLE preKeys(
        id STRING PRIMARY KEY ASC,
        json TEXT
      , ourServiceId NUMBER
        GENERATED ALWAYS AS (json_extract(json, &#39;$.ourServiceId&#39;)));
CREATE TABLE signedPreKeys(
        id STRING PRIMARY KEY ASC,
        json TEXT
      , ourServiceId NUMBER
        GENERATED ALWAYS AS (json_extract(json, &#39;$.ourServiceId&#39;)));
CREATE TABLE badges(
        id TEXT PRIMARY KEY,
        category TEXT NOT NULL,
        name TEXT NOT NULL,
        descriptionTemplate TEXT NOT NULL
      );
CREATE TABLE badgeImageFiles(
        badgeId TEXT REFERENCES badges(id)
          ON DELETE CASCADE
          ON UPDATE CASCADE,
        &#39;order&#39; INTEGER NOT NULL,
        url TEXT NOT NULL,
        localPath TEXT,
        theme TEXT NOT NULL
      );
CREATE TABLE storyReads (
        authorId STRING NOT NULL,
        conversationId STRING NOT NULL,
        storyId STRING NOT NULL,
        storyReadDate NUMBER NOT NULL,

        PRIMARY KEY (authorId, storyId)
      );
CREATE TABLE storyDistributions(
        id STRING PRIMARY KEY NOT NULL,
        name TEXT,

        senderKeyInfoJson STRING
      , deletedAtTimestamp INTEGER, allowsReplies INTEGER, isBlockList INTEGER, storageID STRING, storageVersion INTEGER, storageUnknownFields BLOB, storageNeedsSync INTEGER);
CREATE TABLE storyDistributionMembers(
        listId STRING NOT NULL REFERENCES storyDistributions(id)
          ON DELETE CASCADE
          ON UPDATE CASCADE,
        serviceId STRING NOT NULL,

        PRIMARY KEY (listId, serviceId)
      );
CREATE TABLE uninstalled_sticker_packs (
        id STRING NOT NULL PRIMARY KEY,
        uninstalledAt NUMBER NOT NULL,
        storageID STRING,
        storageVersion NUMBER,
        storageUnknownFields BLOB,
        storageNeedsSync INTEGER NOT NULL
      );
CREATE TABLE groupCallRingCancellations(
        ringId INTEGER PRIMARY KEY,
        createdAt INTEGER NOT NULL
      );
CREATE TABLE IF NOT EXISTS &#39;messages_fts_data&#39;(id INTEGER PRIMARY KEY, block BLOB);
CREATE TABLE IF NOT EXISTS &#39;messages_fts_idx&#39;(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS &#39;messages_fts_content&#39;(id INTEGER PRIMARY KEY, c0);
CREATE TABLE IF NOT EXISTS &#39;messages_fts_docsize&#39;(id INTEGER PRIMARY KEY, sz BLOB);
CREATE TABLE IF NOT EXISTS &#39;messages_fts_config&#39;(k PRIMARY KEY, v) WITHOUT ROWID;
CREATE TABLE edited_messages(
        messageId STRING REFERENCES messages(id)
          ON DELETE CASCADE,
        sentAt INTEGER,
        readStatus INTEGER
      , conversationId STRING);
CREATE TABLE mentions (
        messageId REFERENCES messages(id) ON DELETE CASCADE,
        mentionAci STRING,
        start INTEGER,
        length INTEGER
      );
CREATE TABLE kyberPreKeys(
        id STRING PRIMARY KEY NOT NULL,
        json TEXT NOT NULL, ourServiceId NUMBER
        GENERATED ALWAYS AS (json_extract(json, &#39;$.ourServiceId&#39;)));
CREATE TABLE callsHistory (
        callId TEXT PRIMARY KEY,
        peerId TEXT NOT NULL, -- conversation id (legacy) | uuid | groupId | roomId
        ringerId TEXT DEFAULT NULL, -- ringer uuid
        mode TEXT NOT NULL, -- enum &quot;Direct&quot; | &quot;Group&quot;
        type TEXT NOT NULL, -- enum &quot;Audio&quot; | &quot;Video&quot; | &quot;Group&quot;
        direction TEXT NOT NULL, -- enum &quot;Incoming&quot; | &quot;Outgoing
        -- Direct: enum &quot;Pending&quot; | &quot;Missed&quot; | &quot;Accepted&quot; | &quot;Deleted&quot;
        -- Group: enum &quot;GenericGroupCall&quot; | &quot;OutgoingRing&quot; | &quot;Ringing&quot; | &quot;Joined&quot; | &quot;Missed&quot; | &quot;Declined&quot; | &quot;Accepted&quot; | &quot;Deleted&quot;
        status TEXT NOT NULL,
        timestamp INTEGER NOT NULL,
        UNIQUE (callId, peerId) ON CONFLICT FAIL
      );
[ dropped all indexes to save space in this blog post ]
CREATE TRIGGER messages_on_view_once_update AFTER UPDATE ON messages
      WHEN
        new.body IS NOT NULL AND new.isViewOnce = 1
      BEGIN
        DELETE FROM messages_fts WHERE rowid = old.rowid;
      END;
CREATE TRIGGER messages_on_insert AFTER INSERT ON messages
      WHEN new.isViewOnce IS NOT 1 AND new.storyId IS NULL
      BEGIN
        INSERT INTO messages_fts
          (rowid, body)
        VALUES
          (new.rowid, new.body);
      END;
CREATE TRIGGER messages_on_delete AFTER DELETE ON messages BEGIN
        DELETE FROM messages_fts WHERE rowid = old.rowid;
        DELETE FROM sendLogPayloads WHERE id IN (
          SELECT payloadId FROM sendLogMessageIds
          WHERE messageId = old.id
        );
        DELETE FROM reactions WHERE rowid IN (
          SELECT rowid FROM reactions
          WHERE messageId = old.id
        );
        DELETE FROM storyReads WHERE storyId = old.storyId;
      END;
CREATE VIRTUAL TABLE messages_fts USING fts5(
        body,
        tokenize = &#39;signal_tokenizer&#39;
      );
CREATE TRIGGER messages_on_update AFTER UPDATE ON messages
      WHEN
        (new.body IS NULL OR old.body IS NOT new.body) AND
         new.isViewOnce IS NOT 1 AND new.storyId IS NULL
      BEGIN
        DELETE FROM messages_fts WHERE rowid = old.rowid;
        INSERT INTO messages_fts
          (rowid, body)
        VALUES
          (new.rowid, new.body);
      END;
CREATE TRIGGER messages_on_insert_insert_mentions AFTER INSERT ON messages
      BEGIN
        INSERT INTO mentions (messageId, mentionAci, start, length)
        
    SELECT messages.id, bodyRanges.value -&gt;&gt; &#39;mentionAci&#39; as mentionAci,
      bodyRanges.value -&gt;&gt; &#39;start&#39; as start,
      bodyRanges.value -&gt;&gt; &#39;length&#39; as length
    FROM messages, json_each(messages.json -&gt;&gt; &#39;bodyRanges&#39;) as bodyRanges
    WHERE bodyRanges.value -&gt;&gt; &#39;mentionAci&#39; IS NOT NULL
  
        AND messages.id = new.id;
      END;
CREATE TRIGGER messages_on_update_update_mentions AFTER UPDATE ON messages
      BEGIN
        DELETE FROM mentions WHERE messageId = new.id;
        INSERT INTO mentions (messageId, mentionAci, start, length)
        
    SELECT messages.id, bodyRanges.value -&gt;&gt; &#39;mentionAci&#39; as mentionAci,
      bodyRanges.value -&gt;&gt; &#39;start&#39; as start,
      bodyRanges.value -&gt;&gt; &#39;length&#39; as length
    FROM messages, json_each(messages.json -&gt;&gt; &#39;bodyRanges&#39;) as bodyRanges
    WHERE bodyRanges.value -&gt;&gt; &#39;mentionAci&#39; IS NOT NULL
  
        AND messages.id = new.id;
      END;
sqlite&gt;
&lt;/pre&gt;

&lt;p&gt;Finally I have the tool needed to inspect and process Signal
messages that I need, without using the vendor provided client.  Now
on to transforming it to a more useful format.&lt;/p&gt;

&lt;p&gt;As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
&lt;b&gt;&lt;a href=&quot;bitcoin:15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&quot;&gt;15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b&lt;/a&gt;&lt;/b&gt;.&lt;/p&gt;
</description>
        </item>
        
        </channel>
</rss>