Nine days ago, I started migrating orphaned Debian packages with no +version control system listed in debian/control of the source to git. +At the time there were 438 such packages. Now there are 391, +according to the UDD. In reality it is slightly less, as there is a +delay between uploads and UDD updates. In the nine days since, I have +thus been able to work my way through ten percent of the packages. I +am starting to run out of steam, and hope someone else will also help +brushing some dust of these packages. Here is a recipe how to do it. + +I start by picking a random package by querying the UDD for a list of +10 random packages from the set of remaining packages: + +
+ ++PGPASSWORD="udd-mirror" psql --port=5432 --host=udd-mirror.debian.net \ + --username=udd-mirror udd -c "select source from sources \ + where release = 'sid' and (vcs_url ilike '%anonscm.debian.org%' \ + OR vcs_browser ilike '%anonscm.debian.org%' or vcs_url IS NULL \ + OR vcs_browser IS NULL) AND maintainer ilike '%packages@qa.debian.org%' \ + order by random() limit 10;" +
Next, I visit http://salsa.debian.org/debian and search for the +package name, to ensure no git repository already exist. If it does, +I clone it and try to get it to an uploadable state, and add the Vcs-* +entries in d/control to make the repository more widely known. These +packages are a minority, so I will not cover that use case here.
+ +For packages without an existing git repository, I run the +following script debian-snap-to-salsa to prepare a git +repository with the existing packaging.
+ ++ +Some times the debsnap script fail to download some of the versions. +In those cases I investigate, and if I decide the failing versions +will not be missed, I call it using ALLOWFAILURE=true to ignore the +problem and create the git repository anyway. + ++#!/bin/sh +# +# See also https://bugs.debian.org/804722#31 + +set -e + +# Move to this Standards-Version. +SV_LATEST=4.7.0 + +PKG="$1" + +if [ -z "$PKG" ]; then + echo "usage: $0" + exit 1 +fi + +if [ -e "${PKG}-salsa" ]; then + echo "error: ${PKG}-salsa already exist, aborting." + exit 1 +fi + +if [ -z "ALLOWFAILURE" ] ; then + ALLOWFAILURE=false +fi + +# Fetch every snapshotted source package. Manually loop until all +# transfers succeed, as 'gbp import-dscs --debsnap' do not fail on +# download failures. +until debsnap --force -v $PKG || $ALLOWFAILURE ; do sleep 1; done +mkdir ${PKG}-salsa; cd ${PKG}-salsa +git init + +# Specify branches to override any debian/gbp.conf file present in the +# source package. +gbp import-dscs --debian-branch=master --upstream-branch=upstream \ + --pristine-tar ../source-$PKG/*.dsc + +# Add Vcs pointing to Salsa Debian project (must be manually created +# and pushed to). +if ! grep -q ^Vcs- debian/control ; then + awk "BEGIN { s=1 } /^\$/ { if (s==1) { print \"Vcs-Browser: https://salsa.debian.org/debian/$PKG\"; print \"Vcs-Git: https://salsa.debian.org/debian/$PKG.git\" }; s=0 } { print }" < debian/control > debian/control.new && mv debian/control.new debian/control + git commit -m "Updated vcs in d/control to Salsa." debian/control +fi + +# Tell gbp to enforce the use of pristine-tar. +inifile +inifile debian/gbp.conf +create +section DEFAULT +key pristine-tar +value True +git add debian/gbp.conf +git commit -m "Added d/gbp.conf to enforce the use of pristine-tar." debian/gbp.conf + +# Update to latest Standards-Version. +SV="$(grep ^Standards-Version: debian/control|awk '{print $2}')" +if [ $SV_LATEST != $SV ]; then + sed -i "s/\(Standards-Version: \)\(.*\)/\1$SV_LATEST/" debian/control + git commit -m "Updated Standards-Version from $SV to $SV_LATEST." debian/control +fi + +if grep -q pkg-config debian/control; then + sed -i s/pkg-config/pkgconf/ debian/control + git commit -m "Replaced obsolete pkg-config build dependency with pkgconf." debian/control +fi + +if grep -q libncurses5-dev debian/control; then + sed -i s/libncurses5-dev/libncurses-dev/ debian/control + git commit -m "Replaced obsolete libncurses5-dev build dependency with libncurses-dev." debian/control +fi +
With the git repository in place, I do a test build (gbp +buildpackage) to ensure the build is actually working. If it does not +I pick a different package, or if the build failure is trivial to fix, +I fix it before continuing. At this stage I revisit +http://salsa.debian.org/debian and create the project under this group +for the package. I then follow the instructions to publish the local +git repository. Here is from a recent example:
+ ++ ++git remote add origin git@salsa.debian.org:debian/perl-byacc.git +git push --set-upstream origin master upstream pristine-tar +git push --tags +
With a working build, I have a look at the build rules if I want to +remove some more dust. I normally try to move to debhelper compat +level 13, which involves removing debian/compat and modifying +debian/control to build depend on debhelper-compat (=13). I also test +with 'Rules-Requires-Root: no' in debian/control and verify in +debian/rules that hardening is enabled, and include all of these if +the package still build. If it fail to build with level 13, I try +with 12, 11, 10 and so on until I find a level where it build, as I do +not want to spend a lot of time fixing build issues.
+ +Some times, when I feel inspired, I make sure debian/copyright is +converted to the machine readable format, often by starting with +'debhelper -cc' and then cleaning up the autogenerated content until +it matches realities. If I feel like it, I might also clean up +non-dh-based debian/rules files to use the short style dh build +rules.
+ +Once I have removed all the dust I care to process for the package, +I run 'gbp dch' to generate a debian/changelog entry based on the +commits done so far, run 'dch -r' to switch from 'UNRELEASED' to +'unstable' and get an editor to make sure the 'QA upload' marker is in +place and that all long commit descriptions are wrapped into sensible +lengths, run 'debcommit --release -a' to commit and tag the new +debian/changelog entry, run 'debuild -S' to build a source only +package, and 'dput ../perl-byacc_2.0-10_source.changes' to do the +upload. During the entire process, and many times per step, I run +'debuild' to verify the changes done still work. I also some times +verify the set of built files using 'find debian' to see if I can spot +any problems (like no file in usr/bin any more or empty package). I +also try to fix all lintian issues reported at the end of each +'debuild' run.
+ +If I find Debian specific patches, I try to ensure their metadata +is fairly up to date and some times I even try to reach out to +upstream, to make the upstream project aware of the patches. Most of +my emails bounce, so the success rate is low. For projects with no +Homepage entry in debian/control I try to track down one, and for +packages with no debian/watch file I try to create one. But at least +for some of the packages I have been unable to find a functioning +upstream, and must skip both of these.
+ +If I could handle ten percent in nine days, twenty people could +complete the rest in less then five days. I use approximately twenty +minutes per package, when I have twenty minutes spare time to spend. +Perhaps you got twenty minutes to spare too?
+ +As usual, if you use Bitcoin and want to show your support of my +activities, please send Bitcoin donations to my address +15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.
+