On the way to Samba 4: Part 1

meta = {'category': samba, 'date': Wed 24 November 2010}

After Samba XP 2008 Andrew and I started keeping a wiki page with our bi-weekly goals and achievements for Samba 4. Because planning in a Free Software project is hard (time availability and priorities change over time, and other volunteers are equally unpredictable) we called this our “Fantasy Page”; it listed things we wanted to work on next (“fantasies”), but reality being what it is we would usually actually end up working on something entirely different. We discussed our progress and new plans in - what I would now call - a bi-weekly standup call.

There were several reasons for doing this. It gave us some sense of direction as well as a sense of accomplishment; a way to look back at the end of the year and realize how much we had actually achieved. Because Samba 4 is such a long term project (it is 7 years old at this point) it is easy to become disillusioned, to look back at a year of commits and to not see the gradual improvement, just the fact that there is no release yet.

We managed to keep this up for two years, much longer than I had anticipated, and eventually started to slip last year.

More recently Kai and Tridge have started to blog weekly about their efforts to make Samba 4.0 a reality and I’m going to join them by trying to blog regularly - every two weeks - about my contributions, even if there were none.

In the next two weeks I plan to work on finally getting alpha 14 of Samba 4 out and on fixing the daily builds of Samba 4 and OpenChange for Ubuntu on Launchpad after we did a massive reorganization of the private libraries in Samba 4.

Current Playing: Zero 7 - Somersault

comments.

Mumble and bluetooth

meta = {'category': misc, 'date': Sun 07 November 2010}

Mumble is an open source, low-latency, high quality voice chat application that we’re using at Canonical, and which Samba has recently also adopted.

After I busted the cable of my cabled headset and inspired by Matt’s post about Mumble and Bluetooth I bought a cheap Nokia Bluetooth headset that works with my laptop as well as my phone. By using BlueMan I even found that the headset worked out of the box on Maverick.

A nice feature in Mumble is that it is controllable using D-Bus. Blueman supports running an arbitrary command when the answer button is pressed. Combining these two features, it is possible to automatically toggle mute when the answer button is pressed:

#!/usr/bin/python
import dbus
bus = dbus.SessionBus()
mumble = bus.get_object("net.sourceforge.mumble.mumble", "/")
is_muted = mumble.isSelfMuted()
mumble.setSelfMuted(not is_muted)

To use this script, set its path in the configuration tab for the “Headset” plugin in blueman.

The only remaining problem with this setup is that I can’t keep the headset on during my work day, as I don’t have a way to put it in standby mode automatically. This means that my battery runs out pretty quickly, even when nothing is happening on Mumble.

Currently Playing: Red Sparowes - Finally As That Blazing Sun Shone

comments.

OpenChange server and SOGo

meta = {'category': openchange, 'date': Tue 26 October 2010}

There’s more good news on the OpenChange front. Julien has been working together with Wolfgang and Ludovic from Inverse to leverage the server-side support in OpenChange to provide native Exchange server support in SOGo.

A couple of days ago we announced that there now is an initial version that allows the use of Outlook against a SOGo server through OpenChange.

There is a screencast up on youtube (there is also a .mov version of the screencast).

As far as I know, this is the first time it’s possible to actually use Outlook clients with a non-Microsoft Exchange-compatible server, without the need for plugins on the Outlook side. And it’s all Free Software. Of course, this is just a preview, and not something we’d recommend everybody to run in production just yet. But it’s exciting to finally see this come together.

We already have OpenChange packages in Debian and Ubuntu but I hope I can help get SOGo packaged for both distributions as well.

comments.

Samba 4 and OpenChange daily Ubuntu packages

meta = {'category': debian, 'date': Tue 28 September 2010}

1
2
3
4

# bzr-builder format 0.2 deb-version 4.0.0~alpha14~bzr{revno}~ppa{revno:packaging}+{revno:debian}
lp:samba
merge debian lp:~samba-team/samba/unstable
merge packaging lp:~samba-team/samba/4.0-ppa-maverick

1
2

$ apt-add-repository ppa:samba-team/ppa
$ apt-add-repository ppa:openchange/daily-builds

comments.

subunit usage in Samba

meta = {'category': samba, 'date': Fri 24 September 2010}

Both Samba 3 and Samba 4 are now using the “subunit” protocol inside their testsuite (aka “make test”). subunit is a streaming protocol used to report test results that is aimed at being simple to generate and parse as well as being human readable.

A very simple subunit stream might look like this:

1
2
3
4
5
6
7
8
9

test: samba4.tests.util.strlist.check_list_make
creating list...
list created!
success: samba4.tests.util.strlist.check_list_make
test: samba4.tests.util.strlist.check_list_make_shell
creating list...
xfail: samba4.tests.util.strlist.check_list_make_shell [
returned NT_STATUS_NOT_IMPLEMENTED
]

For those that are familiar with the TAP protocol used by Perl, it is similar to that, although it has a couple of features that TAP does not have. For example, it can report timestamps (useful for determining test duration) and has more flexible progress reporting.

Subunit is particularly useful for projects that use multiple programming languages as it allows a single tool to be used for test visualization or analysis rather than one per language. All that’s required per-language is a test runner that can spit out subunit streams.

selftest.pl, the main engine behind Samba’s test suite, has been using subunit internally since its creation a couple of years ago. Most other test tools we use can also report subunit, in particular our Python tests, blackbox tests, Perl tests (using tap2subunit) and smbtorture.

“make test” never displays raw subunit results, it always formats them using our format-subunit script. Samba 4’s “make test” stores the raw subunit output in st/subunit.

I’m attending SNIA SDC at the moment and a couple of people here have asked me about the tools I use to display and analyse test results. They are:

1
2

$ make test
$ tribunal-subunit st/subunit

1
2
3
4
5
6
7

$ testr init
# Run the full testsuite (1 hour goes by)
$ testr run
# Run those tests from the testsuite that failed in the previous run
# (this would be a lot shorter usually, depending on how many tests were
# failing)
$ testr run --failing

comments.

Working from home

meta = {'category': misc, 'date': Fri 25 June 2010}

For about 6 months now I’ve been working for Canonical on the Soyuz component of Launchpad. Like most other engineers at Canonical I don’t work at the office but from a desk at home, as our nearest office is in London, not really a distance that is feasible for a commute. I do work at regular hours during work days and stay in touch with my colleagues using IRC and voice over IP.

I did have some experience working on contracts and study assignments from home previously, but working a fulltime regular job has turned out to be a bigger challenge. It seems easy enough. No travel time, every day is casual Friday, being able to listen to obscure death metal all day without driving coworkers crazy. Awesome, right?

Well, not entirely. I can’t say I wasn’t warned beforehand (I was) but I still ran head-first into some of the common mistakes.

comments.

Samba Summer of Code

meta = {'category': samba, 'date': Thu 10 June 2010}

As I have done in previous years, I am again participating in the Google Summer of Code as mentor for the Samba project.

Last year I Andrew and I co-mentored three students with mixed results. In the end we had to drop one of our students but the other two did well. I’ve only taken on one student this year for various reasons.

The amount of time required to mentor a student varies wildly depending on the student and is hard to predict based on their application. Some students seem to require quite a lot of mentoring while others are self-motivated and self-learning. This has not just been my experience, I’ve heard similar stories from fellow mentors on other projects.

Last summer Ricardo worked on SWAT for Samba 4 and he is still actively working on the project, even after the Summer of Code has finished. I hope to find the time to package SWAT in time for Debian Squeeze. At the moment SWAT just supports managing shares but Ricardo is working on user management.

In 2009 Calin worked on the GTK+ frontends for Samba, in particular changing them to be Python-based rather than C-based. This year his work is going to be continued by Sergio, hopefully with the some user-ready tools as the end result.

Currently Playing: Gazpacho - 117

comments.

Proof of concept OpenChange server working

meta = {'category': openchange, 'date': Tue 08 June 2010}

Seeing this makes me very happy. It’s taken us a couple of years to get to this point but we’ve finally made it, mostly thanks to the dedication and persistence of Julien and Brad.

comments.

Thoughts on the Nokia N900

meta = {'category': misc, 'date': Sat 17 April 2010}

Yesterday I started writing up a quick review of my new phone/tablet, a Nokia N900. Unfortunately it has since broken down by what appears to be a hardware issue. It does still work to a certain degree, as the leds flash and I can access it as a USB mass-storage device, but the screen hasn’t shown anything since I woke up this morning. The timing could not have been worse, as I’m currently abroad and having a phone (as well as a GPS) would be useful, especially considering the ash cloud situation in Iceland.

Update: 4 weeks after dropping my phone off at the Nokia care center I have received a new N900; they weren’t able to tell me what was wrong with the old one.

Anyway, my thoughts about the N900. Please note that I haven’t used a lot of other smartphones, so I can’t really compare this with Android phones or iPhones.

comments.

Nostalgia: 10 Years of Samba Hacking

meta = {'category': samba, 'date': Thu 11 February 2010}

While searching for something else I happened to come across one of my first posts to the ntdom list in November 2000.

My post is a simple question about a Samba crash that I myself no doubt had introduced. I’m sure I could have found a solution to it by using Google - excuse me, AltaVista - but I still received a friendly reply from Jerry explaining me to use GDB. I’m not too embarrassed, at least I used proper punctuation and already wrote somewhat comprehensible English back then.

It’s also strange to realize it’s already been almost ten years since I started hacking on the Samba project.

comments.

Linux.Conf.Au 2010 - Day 3 - Wednesday

meta = {'category': misc, 'date': Thu 04 February 2010}

I went to Jonathan Corbet’s yearly update of the status of the Linux kernel. He talked about the various big changes that went into the kernel over the last year as well as the development processes. The Linux kernel is probably one of the largest open source projects, and very healthy - there are a lot of individuals and companies contributing to it. With this size comes a few interesting challenges coping with the flow of changes into Linus’ tree. Their current processes seem to deal with this quite well, and don’t seem to need a lot of major changes at the moment.

His talk also included the obligatory list of features that landed in the last year. The only one that really matters to me is the Nouveau driver, which I’m looking forward to trying out.

The second talk I went to in the morning was Selena Deckelmann’s overview of the Open Source database landscape. She mentioned there’s new projects started daily, but it was still a bit disappointing not to see TDB up there.

After lunch Rob gave a talk about Subunit, introducing to the ideas behind the Subunit protocol as well as presenting an overview of the tools that are available for it and the projects that have Subunitized as of yet. It’s exciting to see the Subunit universe slowly growing, I wasn’t aware of some of the projects that are using it. The recently announced testrepository also looks interesting, even though it is still very rudimentary at the moment.

In the evening Tridge, Rusty, Andrew, Jeremy, AJ and I participated in the hackoff as the “Samba Team”.

The hackoff was a lot of fun, and consisted of 6 problems, each of which involved somehow decoding the data file for the problem and extracting a short token from it in one way or another, which was required to retrieve the next problem. We managed to solve 4 problems in the hour that the organizers had allocated, and ended first because we were a bit quicker in solving the 4th problem than the runner-ups. No doubt the fact that we were the largest team had something to do with this.

I hung out with some of the awesome Git and Github developers in the Malthouse in the evening, and talked about Dulwich, Bazaar and Launchpad (“No really, I am not aware of any plans to add Git support to Launchpad.”).

comments.

Linux.Conf.Au 2010 - Day 2 - Tuesday

meta = {'category': misc, 'date': Wed 03 February 2010}

On Tuesday we had our “Launchpad” mini-conf, which featured talks from Launchpad developers and users alike. It wasn’t necessarily about hosting projects on Launchpad, but rather about how various projects could benefit from Launchpad.

I popped out of the Launchpad track for a bit to attend Andrews talk about the current status of Samba 4. He did a nice job of summarizing the events in the last year, the most of import one of course being the support for DC synchronization. I’m proud we’ve finally managed to pull this off - and hopefully we’ll actually have a beta out next year. We have been saying “maybe next year” for almost 4 years now when people asked us for estimates of a release date.

At the end of the day Aaron and I gave a quick introduction to Launchpad’s code imports and code reviews.

comments.

Linux.Conf.Au 2010 - Day 1

meta = {'category': misc, 'date': Wed 03 February 2010}

Linux.Conf.Au has a reputation for being one of the best FLOSS conferences in the world, and it more than lived up to my high expectations. The last one I attended was also in New Zealand, but further south - in Dunedin.

comments.

Build from branch

meta = {'category': misc, 'date': Mon 01 February 2010}

At the moment I am returning home after three very productive and awesome weeks in Wellington, Sydney and Strasbourg.

I spent the first week in the West Plaza in Wellington, working together with fellow Launchpad developers on getting the basics of building from branches working. We eventually managed to get something working at the end of Friday afternoon. We split the work up at the beginning of the week and then worked on it in pairs for a couple of days before integrating all work on Friday. At the end of the week William managed to get a basic source package build from recipe through the queue.

Pair-programming with Jono and Michael was very educational, I suspect I’ll be a fair bit quicker when I get back to hacking on Launchpad by myself. It’s scary to see how some people can make the changes that would take me a full day in a mere hour.

Tim picked up my initial work on support for Mercurial imports and completed and landed it during the sprint. Since the rollout on Wednesday it is possible to request Mercurial imports on Launchpad. Most imports (e.g. mutt, dovecot, hg) seem to work fine, with the main exception being the really large Mercurial repositories such as OpenOffice.org and OpenJDK. This is because of (known) scaling issues that will be fixed in one of the next releases of bzr-hg.

This was the first time I was back in Wellington since 2006, and the weather this year was exactly as I remembered it; showers and wind, with the occasional day of sunshine. For a capital the city centre is quite small, but it has its charm and the view from the various hills around the bay is amazing.

On the weekend I met up with Andrew and Kirsty and we did some hiking around Wellington (where the weather allowed it).

comments.

My first week as a Launchpad developer: impressions

meta = {'category': misc, 'date': Thu 10 December 2009}

Roughly a week ago I joined Julian, Muharem and Michael, working on the Soyuz component of Launchpad. For now I’ve been working on easy Soyuz bugs, as a way of becoming more familiar with the internals. I’m working from home but I had the chance to hang out with some of the other Launchpad developers, including the full Soyuz team, at UDS Lucid in Dallas.

Launchpad is different from most other FOSS projects I have worked on so far. Some things I noticed during my first week:

The codebase is big and well tied together. I don’t think I’ve ever used grep and ctags as often as I have in the last week. Fortunately, the directory structure makes it relatively easy to predict where to look for things.

Reviews are really quick - no long round-trips between author and reviewer trying to get a branch landed. This is a really really great thing.

It’s easy to find somebody familiar with a particular piece of code and it doesn’t take long to get an answer when you ask questions. I’m still getting used to this - I tend to ask questions sporadically because I have gotten used to having to wait a couple of days for an answer that’s actually useful.

Setting up the development environment takes some time. Or perhaps I’m spoiled by Bazaar where “bzr branch lp:bzr bzr && ./bzr/bzr selftest” is all you need to start hacking. And it seems like karmic is the only platform on which things work - I tried with Debian Sid and Lucid as well, but things broke in strange and unusual ways.

The test suite is heavy and takes long to start up, something that makes proper TDD too hard. I also managed to run into some unexplainable problems where the librarian wouldn’t shut down on one of my systems. Since there is only one instance of the database it is not really possible to run multiple instances of the testsuite at the same time unless you use chroots or something like that - this makes it hard to work on multiple branches at the same time, something which would especially be nice since the testsuite is slow (so you can run the testsuite in one branch, hack in another and alterate).

Doctests, while fast, a bit of a nuisance. Because of the setup/teardown overhead that is paid for every single test, doc tests are a lot faster than unit tests. On the other hand, pdb doesn’t play well with doc tests - it doesn’t show any context. Conceptually I also prefer small unit tests over doc tests, since they’re quicker to read, easier to understand and there’s less side-effects from previous instructions in the test that could affect the code that’s being tested.

And for those that know me well; yes, getting used to somewhat regular working hours was indeed a challenge, but I seem to have managed.

comments.