Posts – Page 3

Build from branch

At the moment I am returning home after three very productive and awesome weeks in Wellington, Sydney and Strasbourg.

I spent the first week in the West Plaza in Wellington, working together with fellow Launchpad developers on getting the basics of building from branches working. We eventually managed to get something working at the end of Friday afternoon. We split the work up at the beginning of the week and then worked on it in pairs for a couple of days before integrating all work on Friday. At the end of the week William managed to get a basic source package build from recipe through the queue.

Pair-programming with Jono and Michael was very educational, I suspect I'll be a fair bit quicker when I get back to hacking on Launchpad by myself. It's scary to see how some people can make the changes that would take me a full day in a mere hour.

Tim picked up my initial work on support for Mercurial imports and completed and landed it during the sprint. Since the rollout on Wednesday it is possible to request Mercurial imports on Launchpad. Most imports (e.g. mutt, dovecot, hg) seem to work fine, with the main exception being the really large Mercurial repositories such as OpenOffice.org and OpenJDK. This is because of (known) scaling issues that will be fixed in one of the next releases of bzr-hg.

This was the first time I was back in Wellington since 2006, and the weather this year was exactly as I remembered it; showers and wind, with the occasional day of sunshine. For a capital the city centre is quite small, but it has its charm and the view from the various hills around the bay is amazing.

On the weekend I met up with Andrew and Kirsty and we did some hiking around Wellington (where the weather allowed it).

comments.

My first week as a Launchpad developer: impressions

Roughly a week ago I joined Julian, Muharem and Michael, working on the Soyuz component of Launchpad. For now I've been working on easy Soyuz bugs, as a way of becoming more familiar with the internals. I'm working from home but I had the chance to hang out with some of the other Launchpad developers, including the full Soyuz team, at UDS Lucid in Dallas.

Launchpad is different from most other FOSS projects I have worked on so far. Some things I noticed during my first week:

The codebase is big and well tied together. I don't think I've ever used grep and ctags as often as I have in the last week. Fortunately, the directory structure makes it relatively easy to predict where to look for things.

Reviews are really quick - no long round-trips between author and reviewer trying to get a branch landed. This is a really really great thing.

It's easy to find somebody familiar with a particular piece of code and it doesn't take long to get an answer when you ask questions. I'm still getting used to this - I tend to ask questions sporadically because I have gotten used to having to wait a couple of days for an answer that's actually useful.

Setting up the development environment takes some time. Or perhaps I'm spoiled by Bazaar where "bzr branch lp:bzr bzr && ./bzr/bzr selftest" is all you need to start hacking. And it seems like karmic is the only platform on which things work - I tried with Debian Sid and Lucid as well, but things broke in strange and unusual ways.

The test suite is heavy and takes long to start up, something that makes proper TDD too hard. I also managed to run into some unexplainable problems where the librarian wouldn't shut down on one of my systems. Since there is only one instance of the database it is not really possible to run multiple instances of the testsuite at the same time unless you use chroots or something like that - this makes it hard to work on multiple branches at the same time, something which would especially be nice since the testsuite is slow (so you can run the testsuite in one branch, hack in another and alterate).

Doctests, while fast, a bit of a nuisance. Because of the setup/teardown overhead that is paid for every single test, doc tests are a lot faster than unit tests. On the other hand, pdb doesn't play well with doc tests - it doesn't show any context. Conceptually I also prefer small unit tests over doc tests, since they're quicker to read, easier to understand and there's less side-effects from previous instructions in the test that could affect the code that's being tested.

And for those that know me well; yes, getting used to somewhat regular working hours was indeed a challenge, but I seem to have managed.

comments.

US: Observations

These past few days in the US were a bit of a rollercoaster. Some random observations:

  • The mentor summit was very nice and well organized (or rather: well disorganized). Lots of awesome people around from a wide variety of projects and nationalities.
  • "Next Generation VCS" seems to be an alias for git these days in the minds of most people.
  • I didn't write a single line of code in almost a week, something that is very rare.
  • Driving an automatic gives you two spare limbs to use for other things. What those other things are, I have yet to figure out.
  • Is the fact that your kid was student of the month or the fact that you own two cats and a dog really something that belongs on a bumper sticker?
  • Gas is cheap (compared to Europe). I drove 300 miles on a $30 tank.
  • The malls in the Bay Area are some of the biggest I've ever seen, but strangely enough they seem to lack both book- and cd-stores.
  • It is legal to turn right on a red traffic sign in California unless otherwise indicated. It took me a while to realize this until people repeatedly started honking behind me...
  • The waiver I had to sign to be able to skydive in California was scary. I can live with my operating system coming without implied warranty or fitness for a particular purpose, but my parachute?
  • I stopped pretending to have any regularity in my sleeping habits. 6 AM flights? It seemed like a good idea at the time.

comments.

CtrlProxy: Looking for a new maintainer

After over 7 years of working on it off and on, I'm looking for somebody to help maintain (and eventually take over) CtrlProxy. I started working on CtrlProxy somewhere in 2002, only a short while after Wilmer started hacking on BitlBee. If I remember correctly I started working on it because I didn't want to run a separate dircproxy (the only real competitor at the time) instance (with configuration) for each IRC network that I connected to. It was also just a good excuse to play with the IRC protocol a bit.

Over the years, CtrlProxy has served as a playground for me to try out new and interesting things. It's been rewritten or severely refactored several times in its early history, the latest time being the 3.0 release (from 2005). I've tried different build systems, I've tried different implementation languages, I've tried different configuration file formats, I've tried different support libraries, I've tried different version control systems, I've tried different documentation formats. So while it's definitely been a very educational project for me personally, I haven't really had the time or the interest to dedicate to the project that it deserved during the last few years. This was mostly because there were other more interesting FOSS projects I spent my spare cycles on.

These days there are plenty of other good IRC proxies out there, such as BIP, so I doubt CtrlProxy will be missed if it were to disappear. Despite that, if anybody is interested in taking over, please send me an email (jelmer@samba.org) or contact me on IRC (jelmer on the OFTC and Freenode networks).

Currently Playing: Anathema - Shroud of False

comments.

Summer of Code 2009

For this years (the fifth?) Summer of Code, I participated once again as a mentor for the Samba and OpenChange projects.

Samba was assigned four slots this year: one was a CIFSFS project mentored by Steve French and the other three were Python projects related to Samba 4, co-mentored by Andrew and me. Our students did very well this year, although we unfortunately had to drop one after the mid-term evaluations due to lack of effort. Nonetheless, we're very happy with the results of the other two projects:

Calin Crisan (France) converted the rest of the applications in SambaGtk to Python, and worked on a GTK+ user manager for Samba and Windows. With his improvements, it is now possible to edit registries, manage users, inspect the endpoint mapper, plan tasks and manage services on a remote Windows machine using a GTK+ application on a Linux workstation.

Ricardo Velhote (Portugal) designed and implemented a new version of SWAT - the Samba Web Administration Tool. Unlike the old SWAT, his implementation is more than just a simple web-based editor for smb.conf. As we were expecting at the start of the Summer of Code, not all of the functionality could be implemented properly in a couple of months, not while getting the design and infrastructure right. With a basic version working, we now hope the remaining subsystems can be contributed with help from the community.

I'm planning to merge Calin's improvements to Samba-Gtk into the mainline in the next month or so. SWAT is a standalone application and will continue to live as a separate project, while being a part of the Samba ecosystem. Congratulations, Calin and Ricardo!

comments.

DebCamp / DebConf9

So far I'm very much enjoying my first DebCamp / DebConf. It's nice to finally meet a lot of people in person that I have worked together with or talked to on IRC in the last few years. Cáceres is a relatively small town with a nice old city center.

I arrived early for DebCamp and spent the first few days here working on fixing bugs in the Bazaar and Samba packages as well as discussing the integration between Samba 4 and Kerberos with Sam (both in general and on Debian specifically). In trying to set up a Samba 4 domain we found a number of bugs in the provisioning script, most of which seem to be fixed now.

In the last few days I've mostly worked on getting Samba 4 and OpenChange ready to go into Sid (they're in experimental only at the moment) and have discussed bzr-builddeb and related Bazaar issues with James.

Currently Playing: Pixies - Velouria

comments.

DebConf

I'm looking forward to going to my first DebCamp/DebConf. I won't be giving a talk, but I hope to work together with others on integrating Samba 3 and 4 better with the rest of the system and VCS integration.

comments.

"Franky" Talk at SambaXP

I'll be giving a talk at the next NLLGG meeting about the Franky project.

Update: Slides

comments.

SambaXP 2009

Last week most of the Samba team met again for our annual conference in Göttingen. It was nice seeing everybody again, specially the folks I hadn't seen since the last one.

Together with Andrew and his wife Kirsty I took the train from Amsterdam into Germany a couple of days early and we did some sightseeing together with Anatoli and Nadezhda during the weekend. There's still plenty of things to discover in Göttingen for me, even though I've already been there about two dozen times. We did a tour of the city walls, visited some of the churches and climbed the tower.

Julien's talk about OpenChange was interesting and humorous as always. Volkers' tutorial on asynchronous programming in C. Even though I've spent quite some time working with and looking at these API's it was nice going through them step by step once again. It's a strange thing to wrap your head around.

Andrew and I also gave our yearly "State of Samba 4" talk again. As I've mentioned in other places, I'm really excited about the social effects of the Franky project. Once again I was reminded that giving a talk the morning after the conference party (this year in the "Oriental Lounge") is a bad idea.

Several of my fellow Debian Samba maintainers made it to SambaXP, it was nice to see Christian, Luk, Michael and Noël there. We made some decisions about the direction of the Samba packages, and a plan to allow the Samba 3 and Samba 4 packages to be installed on the same system. Unfortunately I had to miss Christian's talk because it was in the same timeslot as Jeff's talk about the CIFS kernel module.

comments.

Finally a DD

After spending a little bit more than three years in the Debian New Maintainers queue, I have finally become a Debian Developer.

Many thanks to the various people who have helped me through NM and have sponsored my uploads over the past few years.

comments.

bzr-builddeb FTW

% bzr branch deb:line6-usb-source debian
Retrieving Vcs locating from line6-usb-source Debian version 0.7.4-1
Branched 354 revision(s).
% bzr merge-upstream https://line6linux.svn.sourceforge.net/svnroot/line6linux/driver/trunk
All changes applied successfully.
Using version string 0.7.4+svn511 for upstream branch.
The new upstream version has been imported. You should now update the changelog (try dch -v 0.7.4+svn511-1 "New upstream snapshot."), resolve any conflicts, and then commit.
% dch -v 0.7.4+svn511-1 "New upstream snapshot.
% bzr builddeb
Building using working tree
Preparing the build area: ../build-area
Purging the build dir: ../build-area/line6-usb-0.7.4+svn511
[...]
Placing result in /home/jelmer/bzr/line6-usb/result
% ls ../result
line6-usb_0.7.4+svn511-1_amd64.changes  line6-usb_0.7.4+svn511-1.dsc
line6-usb-source_0.7.4+svn511-1_all.deb
line6-usb_0.7.4+svn511-1.diff.gz        line6-usb_0.7.4+svn511.orig.tar.gz

Currently Playing: Phideaux - Microdeath Softstar

comments.

Reconciling the Samba 3 and Samba 4 source code trees

While a few of us have been working very hard on Samba 4 to allow it to rock your socks off as an Active Directory Domain Controller, some of the other Samba developers have been working just as hard on improving the existing Samba 3 codebase and adding features to that. This situation has caused tension between developers as well as technical problems in the past - code with the same purpose is being developed in parallel, libraries diverge because features are only added in one branch and not in the other, one codebase is considered "obsolete" by some and the other is considered only a playground for experimental features by others.

As of yesterday, we now have the two codebases living in one and the same git branch. This should make it a lot easier for the two to use the same libraries. Better yet, it should allow us to reconcile the copies of various libraries that exist in both codebases, all of which have diverged to some degree in the last few years.

After a few problems came up merging the two branches the easy way (they both have a directory called "source" and git doesn't deal well with renaming them to "source3" and "source4" respectively), we decided to replay the history of both branches . This has the disadvantage that all existing branches that are based on the Samba 3 and Samba 4 branches will have to be rebased against the new master branch, but it also means we keep the ability to run "git log" inside of our source directories and having it work right.

Other than the fact that this makes it possible to share more code between the two codebases, one of the ideas we have is also to see if it is possible to provide an Active Directory DC by glueing the best bits of Samba 3 and Samba 4 together (aka "Franky") before they are eventually merged completely.

Currently Playing: Phideaux - Formaldehyde

comments.

Oxegen '08

Oxegen was a bit expensive but awesome. Irish people rock.

update: Wilmer has details.

comments.

Microblogging

I've joined the microblogging hype; I'll be giving status updates on my coffee drinking habits and other uninteresting facts at http://identi.ca/jelmer.

comments.

bzr-svn push without file properties

Ever since bzr-svn started supporting "true push", people have been complaining about the extra file properties it sets.

The key thing about "true" push is that it preserves the exact revisions that were present in Subversion. This lets bzr behave on Subversion branches transparently using the same UI you also use for "native" Bazaar branches.

In other words, if I push to a Subversion branch from my machine, then that branch in Subversion contains enough information for somebody else to reconstruct the exact bzr branch I had.

Since some Bazaar metadata can not be represented in Subversion, it is stored in Bazaar-specific Subversion properties. Unfortunately, these file properties show up in email commit notifications and trac and so they tend to annoy people.

There are two ways around this:

Revision properties

Bazaar-specific metadata can be stored in in custom Subversion revision properties (these don't show up in commit notifications). Unfortunately, this requires Subversion 1.5 or newer to run on the server.

I hope to start setting revision properties instead of file properties when possible as of the next bzr-svn release.

less strict push

It's also possible to throw away any data that can not be represented in Subversion. Since this means that the remote branch won't end up an exact same copy of the local revisions, this isn't true push. The two branches will have diverged (no matter how slightly) after such a push so it is necessary to rebase on the remote branch after pushing.

This is similar to the way git-svn pushes data into Subversion - it calls it "dcommit".

Since this uses rebase it has the usual disadvantages of rebases, which I won't get into right now.

As of a couple of days ago, bzr-svn now also supports this mode of pushing using the "dpush" command, by popular demand.

Currently Playing: Brandi Carlile - The Story

comments.