Posts by 'Jelmer Vernooij' – Page 6

Samba’s tdbdump reimplemented in Python

Less than 150 characters in Python, while the original implementation in C requires more than 2000 characters

1
2
3
4
5
6
import tdb, sys

db = tdb.Tdb(sys.argv[1])
for (k, v) in db.iteritems():
    print "{\nkey(%d) = %r\ndata(%d) = %r\n}\n" % (len(k), k, len(v), v)
}

comments.

GTK+ LDB Browser

As some may have noticed, a large portion of my Samba 4 work during the last few months has been focussed on adding Python bindings for our various public libraries and the refactoring necessary to make it possible to add Python bindings. So far, we have bindings for LDB and TDB but I intend to add bindings for most of our public API so it is possible to, for example, open Windows registry files, join domains, etc. from Python.

LDB is our LDAP-like embedded database, and is for LDAP what sqlite is for SQL. Last night I decided to see how hard it would be to write a graphical browser for LDB using Python, and it turned out to be quite easy, thanks to PyGTK. There is a screenshot of what it looks like here. Packages with the Python bindings for LDB are already in Debian.

The sources for gtkldb are available in the samba-gtk bzr branch at http://people.samba.org/bzr/jelmer/samba-gtk/trunk, along with some of the GTK+ frontends for Samba 4 I wrote earlier (gregedit, gwcrontab, gwsvcctl, gepdump and gwsam).

comments.

Building shared and static libs on different platforms

One of the main portability problems across different POSIX implementations is the way shared and static libs should be built.

Just came across a nice overview of the flags to use on the different platforms: http://www.fortran-2000.com/ArnaudRecipes/sharedlib.html

it would be nice if POSIX could standardise this at some point.

comments.

Epitech/OpenChange meeting

I’ve had a lot of time to practice my French skills this weekend while visiting Epitech and meeting up with Julien, Ali, Dan and the other OpenChange folks in Paris.

There was a forum at Epitech where student teams present the projects that they’ve been working on over the course of the last year. One of the Epitech student teams has done an excellent job on an OpenChange plugin for Evolution that now even appears to have been picked up by upstream. The other projects were also quite interesting as well and varied from a fun to play 3D racing game to a much improved LGPL’ed implementation of the ClamAv virus scanner or a personal logging application for diabetics. Afterwards there was a little bit of an Open Source conference, where Dan and I gave talks about Open Source and Samba, respectively.

comments.

OpenChange team member

As of today, I’m officially a member of the OpenChange development team. I already had access to the Subversion server and have contributed a number of patches in the last couple of years, mainly related to the build system.

comments.

Bazaar: Need for a “Product” object?

This is something that’s been lingering in the back of my head for the last year or so. I think I am missing something in the sequence of [Branch, Repository, WorkingTree]. Here are some of the reasons why I think this is the case:

  • Tags should ideally be shared amongst a set of related branches. This has come up often during discussions about where tags belong.
  • Management of sets of bzr branches is hard
    • It seems to make sense for the configuration of several plugins to be project-specific:
      • bzr-pqm-submit’s pqm address
      • bzr-email target address
      • bzr-cia’s project setting
      • …
    • may be useful to override whoami
    • having a way to group branches allows mass-pushes/pulls
  • I often find that the public_location I set is almost the same for related branches, with only the last part of the url differing and containing the branch nick
  • It would be nice if “bzr register-branch” could automatically determine what product to register as

I’m not looking for repositories: - repositories can contain data from multiple totally unrelated branches. a tag “1.0” could conflict because there are multiple unrelated projects that have it. - repositories are a storage optimization and I like them that way

although other projects (mercurial, git) seem to be using repositories to allow talking about a group of related branches.

I’m not looking for “just” directories: - There’s no place in a directory to store settings or tags - Having a long list of settings in ~/bazaar/locations.conf doesn’t scale and the settings won’t be able to propagate

Having another semantic object (”Product”?) on which options/tags can be set would help. Perhaps based on the root id (where available) ?

Currently Playing: Symphony X - The Odyssey

comments.

SerNet asks Microsoft for specs

Now that the European court has decided SerNet has asked Microsoft to publish specifications on SMB/CIFS.

Martijn has posted some photos of the concert we went to a couple of days ago.

comments.

Podcast with the Samba team

Last week was the annual Samba get-together in the US. Jeremy and Leslie from Google organized a somewhat more informal event than the SNIA CIFS interop lab that we usually attend.

Several Samba team members, including myself, took part in a podcast about our involvement in the Summer of Code afterwards.

comments.

Converting CSV files to QIF

Recently I’ve started using GnuCash for my personal finances. Eventually I got it running well, after I recompiled it myself, because the version in Debian was segfaulting on start-up (it is also very outdated).

My bank, PostBank only supports exporting transactions in it’s own custom CSV files which, of course, aren’t supported by Gnucash. I wrote a simple Python script that converts PostBank CSV files to QIF, which Gnucash can handle ok.

I’m publishing the script here for others interested in importing PostBank CSV files in their finance apps:

#!/usr/bin/python
# Very simple converter from PostBank mijn.postbank.nl .csv files to
# QIF files for use by (among others) gnucash.
#
# Written by Jelmer Vernooij <jelmer@samba.org>, 2007

import csv, sys

rows = csv.reader(sys.stdin)

header = rows.next()

assert header == (['Datum', 'Naam / Omschrijving', 'Rekening']
    + ['Tegenrekening', 'Code', 'Af Bij', 'Bedrag (EUR)']
    + ['MutatieSoort', 'Mededelingen'])

print '!Type:Bank\n'

for l in rows:
    if "-" in l[0]:
        p = l[0].split("-")
        print "D%s/%s/%s" % (p[1], p[0], p[2])
    else:
        print 'D%s/%s/%s' % (l[0][4:6], l[0][6:8], l[0][0:4]) # date
    if l[5] == 'Bij':
        print 'T%s' % l[6] # amount
    elif l[5] == 'Af':
        print 'T-%s' % l[6] # amount
    else:
        raise Error("Unknown field value '%s'" % l[5])
    print 'P%s' % l[1] # payee / description
    print 'M%s' % l[8] # comment
    print 'L%s' % l[7]
    print '^\n' # end transaction

comments.

Bzr-Svn

The next major release of bzr-svn, 0.4, has now been released. The main change in this release is that the behaviour of push is now intuitive. The big hack that allowed push to somewhat work in the previous release has been replaced by proper push which behaves the same way as it would against a Bazaar branch.

It’s now also possible to branch from non-standard branch locations such as /foo in a repository and not necessarily standard locations like /trunk or /trunk/foo. See the release announcement for a list of other changes.

It’s interesting to see what other people are saying about bzr-svn on their blogs:

Note that bzr-svn 0.4 has been tested on Windows and that branching schemes are now more flexible.

comments.

LUG Radio Live

Just got back to the hotel after LUG Radio Live and the afterparty event. I guess the best way to describe LUG Radio live is as a your average geek conference but with a bit of a rock concert flavor: lots of cheering on, groupies, swearing, simple humor and metal music being played in between between sessions.

The talks were very good; in particular I liked Nat Friedman’s talk about, among other things, the way the Oxford English Dictionary was developed (although he still doesn’t seem to understand what is so bad about the MS-Novell deal) and Aaron Seigo’s talk on KDE 4 (made my fingers itchy and want to contribute..). The short talk Szilveszter and I gave on bzr-gtk went ok, but was probably not all that entertaining right after Malcolm’s talk on zombies and Ubuntu.

The major thing I disliked is the fact that the wireless was very crappy. I got it to work on two occassions, just long enough to read a few emails. It took a while to get used to the weird accent with which they speak English around here and I seem to’ve picked up a few swearwords that I now have to unlearn again. Doh.

All in all really worth the trip from Amsterdam and definitely something to keep in mind for next year!

Erik and I decided to stay an extra day to see a bit of Wolverhampton and Birmingham, but it turns out this isn’t your typical tourist spot.

comments.

Ohloh - Statistics on Free Software projects

Ohloh is a nice web 2.0 site that contains stats on various Free Software projects. At the moment, they only support Subversion, CVS and Git. They’re open to feature requests though. If enough people ask for it, hopefully they’ll support Bazaar at some point.

comments.

OpenChange Interview

Linux Weekly News had a good interview with Julien Kerihuel, lead developer of OpenChange, two weeks ago. It’s now also available for those who are not subscribed.

comments.

Using a pqm with Subversion

One of the things that I’ve always missed in DVCS is the ability to refuse commits in a branch that’s shared by multiple people based on a test suite run. Sure, it’s possible to have a pre-commit hook - but that would mean that you’d have to wait for the full test suite to run until the commit finishes. With the time it takes to run the Samba testsuite, this is not really an option.

One of the things that would work is to have everybody work in a separate branch and then have some sort of tool that merges those revisions from everybody’s personal branches that worked ok. However, to my knowledge, there is no such tool for Subversion.

Bazaar uses a tool called PQM (Patch Queue Manager). PQM usually controls the main branch (for example for Bazaar, it controls bzr.dev), and waits for GPG-signed requests to merge a specific revision into that main branch. Before accepting such a revision, it will try to run the testsuite to make sure it passes. This guarantees that the main branch never contains broken code (as far as can be indicated by the testsuite).

Now that bzr-svn supports true push, it is possible to actually use a PQM with a Subversion branch. I’ve tried it on a smaller branch last week, and am now looking into using this for my Samba work.

comments.

London

I’m currently doing a bit of sightseeing in London, after attending the Bazaar sprint at the Canonical office. It was a good sprint, and quite different from the previous ones - in that there was only a limited amount of actual coding involved. The view from the Canonical office is magnificent, so we were even able to do some sightseeing while working…

Bazaars’ focus has previously mainly been on correctness and features. The first has always been one of our strengths, and we’re in pretty good shape regarding the second. Performance has been one of the main complaints from users about Bazaar and so we have recently tried to improve in that area.

Since 0.12, we have already tried to optimise some of the common code paths and some people have been working on a high performance smart server (to speed up remote operations).

During the first two-and-a-half days of the sprint, we analysed 20 of the most common use cases with Bazaar and determined what complexity they should ideally require to be able to work. After this analysis, we looked at ways to change our data structures to reach these goals.

I have mainly been a spectator during the latter parts of these discussion, but they were interesting to follow.

One of the things I worked on was support for true push in bzr-svn. This was one of the bugs that has bitten a lot of users of bzr-svn. The upcoming bzr-svn 0.4 now supports true push as well as commits in heavyweight checkouts. I hope to release 0.4 after adding nested tree and ignores support so that I don’t have to change the internal mapping mechanisms again.

And now, it is time for some more sightseeing. After that I hope to get back to the reason I’m doing all of this in the first place: Samba!

comments.