Posts in 'debian'

Migrating packaging from Bazaar to Git

A while ago I migrated most of my packages from Bazaar to Git. The rest of the world has decided to use Git for version control, and I don't have enough reason to stubbornly stick with Bazaar and make it harder for myself to collaborate with others.

So I'm moving away from a workflow I know and have polished over the last few years - including the various bzr plugins and other tools involved. Trying to do the same thing using git is frustrating and time-consuming, but I'm sure that will improve with time. In particular, I haven't found a good way to merge in a new upstream release (from a tarball) while referencing the relevant upstream commits, like bzr merge-upstream can. Is there a good way to do this? What helper tools can you recommend for maintaining a Debian package in git?

Having been upstream for bzr-git earlier, I used its git-remote-bzr implementation to do the conversions of the commits and tags:

% git clone bzr::/path/to/bzr/foo.bzr /path/to/git/foo.git

One of my last contributions to bzr-git was a bzr git-push-pristine-tar-deltas subcommand, which will export all bzr-builddeb-style pristine-tar metadata to a pristine-tar branch in a Git repository that can be used by pristine-tar directly or through something like git-buildpackage.

Once you have created a git clone of your bzr branch, it should be a matter of running bzr git-push-pristine-tar-deltas with the target git repository and the Debian package name:

% cd /path/to/bzr/foo.bzr
% bzr git-push-pristine-tar-deltas /path/to/git/foo.git foo
% cd /path/to/git/foo.git foo
% git branch
*  master
   pristine-tar

comments.

Samba 4 and OpenChange daily Ubuntu packages

Daily builds

As of a month ago there are Ubuntu archives with fresh packages of Samba 4 and OpenChange, built on a daily basis day from the latest upstream revision.

This means that it is now possible to run a version of Samba 4 that is less than 24 hours old, without having to know how to extract source code from the version control system that upstream is using, without having to know how to build and install an application from source, but perhaps most importantly: without having to go through the tedious process of manually updating the source code and rebuilding.

OpenChange is tightly coupled to Samba 4, so installing a new version of OpenChange usually involves installing a new version of Samba 4 as well. To make matters more confusing, the two projects use different version control systems (Samba 4 is in Git, while OpenChange is in Subversion) and different build systems (Samba 4 uses waf, OpenChange uses autoconf and make).

I have been involved in Samba 4 and OpenChange as an upstream developer and more recently also as a packager for both Debian and Ubuntu.

As an upstream developer for both these projects it is important for me that users can easily run the development versions. It makes it possible for interested users to confirm the fixes for issues they have reported and to test new features. The more users run the development version, the more confident I can be as a developer that doing a release will not cause any unexpected surprises.

As a packager it is useful to know when there are upstream changes that are going to break my package with the next release.

Recipes

The daily builds work using so-called recipes which describe how to build a Debian source package from a set of Bazaar branches. For example, the Samba 4 recipe looks like this:

# bzr-builder format 0.2 deb-version 4.0.0~alpha14~bzr{revno}~ppa{revno:packaging}+{revno:debian}
lp:samba
merge debian lp:~samba-team/samba/unstable
merge packaging lp:~samba-team/samba/4.0-ppa-maverick

This dictates that a source package should be built by taking the upstream Samba branch and merging the Debian packaging and some recipe-specific tweaking. The last bit on the first line indicates the version string to be used when generating a changelog entry for the daily build.

Every night Launchpad (through bzr-builder) merges these branches and attempts to build the resulting source package, e-mailing me in case of build problems. Generally I fix issues that come up by committing directly to upstream VCS or to the Debian packaging branch. There is no overhead in maintaining the daily build after I've set it up.

For more information on creating source package recipes, see getting started.

Toolchain

The entire toolchain that does the daily package builds for Ubuntu is Free Software, and I have contributed to various bits of that toolchain over the years. It's exciting to see everything come together.

Soyuz

Launchpad consists of multiple pillars - one of those pillars is Soyuz, which I hack on as part of my day job at Canonical. Soyuz is responsible for the archive management and package building. Debian source packages (a combination of upstream source code and packaging metadata) get uploaded by users and then built for various architectures on our buildfarm and published to the Ubuntu archive or to users personal package archives.

Launchpad-code

Another pillar of Launchpad is Launchpad-code, which is responsible for the hosting and management of version control branches. Launchpad users can either host their branches on Launchpad directly or mirror branches (either native Bazaar branches or branches in a foreign format such as Subversion, Git or Mercurial). The mirrorring of native and foreign branches happens using standard Bazaar API's. In the case of Samba and OpenChange we import the branches of the upstream projects (Samba is in Git, OpenChange is in Subversion) and the packaging for both projects is in Bazaar.

Launchad-code calls out to Bazaar to do the actual mirrorring. Over the last few years I have done a lot of work to improve Bazaars support for foreign branches, in particular on supporting Subversion, Git and Mercurial. As the code mirrorring in Launchpad is one of the biggest users of bzr-svn and bzr-git it has helped find some of the more obscure bugs in those plugins over the last few years, to the point where there are only a handful of issues with Git imports and Subversion imports left.

bzr-git and dulwich

bzr-git provides transparent access to Git repositories from within Bazaar and is built on top of Dulwich. Dulwich is a Python library that provides access to the Git file formats and protocols that is completely independent of Bazaar. James Westby originally started it and I adopted it for bzr-git and further extended it. There are now several other projects that use it as well, including hg-git, and rabbitvcs. Apart from James and myself, almost two dozen other people have contributed it so far.

bzr-svn and subvertpy

bzr-svn provides transparant access to Subversion repositories in Bazaar. When I grew frustrated with the existing Subversion Python bindings for various reasons, I decided to create independent Python bindings for Subversion from scratch. These bindings have since been split out into a separate project - subvertpy - and other projects have since also started using them, e.g. hgsubversion and basie.

Using the daily builds

To use the Samba 4 and OpenChange daily builds (Ubuntu Maverick only for now), run:

$ apt-add-repository ppa:samba-team/ppa
$ apt-add-repository ppa:openchange/daily-builds

Currently Playing: Karnivool - Themata

comments.

DebCamp / DebConf9

So far I'm very much enjoying my first DebCamp / DebConf. It's nice to finally meet a lot of people in person that I have worked together with or talked to on IRC in the last few years. Cáceres is a relatively small town with a nice old city center.

I arrived early for DebCamp and spent the first few days here working on fixing bugs in the Bazaar and Samba packages as well as discussing the integration between Samba 4 and Kerberos with Sam (both in general and on Debian specifically). In trying to set up a Samba 4 domain we found a number of bugs in the provisioning script, most of which seem to be fixed now.

In the last few days I've mostly worked on getting Samba 4 and OpenChange ready to go into Sid (they're in experimental only at the moment) and have discussed bzr-builddeb and related Bazaar issues with James.

Currently Playing: Pixies - Velouria

comments.

DebConf

I'm looking forward to going to my first DebCamp/DebConf. I won't be giving a talk, but I hope to work together with others on integrating Samba 3 and 4 better with the rest of the system and VCS integration.

comments.

Finally a DD

After spending a little bit more than three years in the Debian New Maintainers queue, I have finally become a Debian Developer.

Many thanks to the various people who have helped me through NM and have sponsored my uploads over the past few years.

comments.

Squashing Debian RC Bugs

I spent most of my weekend at a-eskwadraat, for the Debian Bugsquashing Party and the UKP (qualifying round for the national championships), right after I got back from New York. Unfortunately my jetlag caught up with me on the sunday, so I overslept and woke up around four. :-(

My contribution to the BSP was small; I didn't fix any release-critical bugs though I did work on getting the next CtrlProxy out, which should fix at least one RC bug.

Currently playing: The Black Crowes - Struttin' Blues

Update(15 October): Oops, looks like I still had this entry marked as "Draft" instead of "Publish"

comments.

NUnit packages and SVN integration in mono

My new nunit packages just got accepted by ftp-master. The CLI Policy has been updated.

In the mean while, I've continued work on the svn plugin for bzr. The refactored bzr API is working great for foreign branch implementation. The only two things that are odd so far are the fact that I have to implement a phony 'Transport' subclass and I think the name of 'BzrDir' should be changed, though I'm not sure to what.

comments.

Samba4 Debian packages in experimental

In the past week, we have migrated the Debian packaging for Samba 4 to the SVN repository of the Debian Samba packaging team at alioth.

Mostly thanks to the hard work of Steinar Gunderson in the past week, the packages are now ready for, and have been uploaded to Debian experimental. Most importantly, there is now integration for upgrading from Samba 3 to Samba 4 using the upgrade scripts.

I will no longer upload Samba4 Debian packages to jelmer.vernstok.nl.

Currently Playing: Porcupine Tree - Radioactive Toy

comments.

NUnit in unstable

My first package (nunit) is now in the Debian archive. It was approved by ftp-master two days ago.

Thanks go to Andrew "ajmitch" Mitchell for sponsoring and to Mirco "meebey" Bauer and David "Dajobe" Beckett for comments and feedback. See the pkg-mono pages for details.

Meebey has uploaded another package (gnunit) that provides a Gtk# interface to NUnit. Screenshot

comments.

Samba4 debian packages

Now that the Samba4 Debian packages appear to be somewhat working, I've set up a daily cronjob that generates packages for the latest SVN revision. Add the following line to your "/etc/apt/sources.list" to try them out:

deb http://jelmer.vernstok.nl/debian/autobuild .

Please note that these packages are still very much alpha, as is the Samba4 code. Comments are welcome

comments.

NUnit packages

I'm currently fixing the last few quirks in my NUnit Debian packages, after getting a some good feedback from the folks at #debian-mono. Just a few more hours before I'll put into pkg-mono's SVN. Until then, snapshots are available at:

deb ftp://autobuild.vernstok.nl/pub/releases/nunit/debian ./
deb-src ftp://autobuild.vernstok.nl/pub/releases/nunit/debian ./

I wonder what I'll work on next. Perhaps nantcontrib or ndoc?

comments.