Posts by 'Jelmer Vernooij' – Page 11

Partially working upgrade paths

The one-way upgrade from Samba3 to Samba4 is now partially working! There are both a web-frontend and a console utility that can do an upgrade, given a libdir (usually /var/lib/samba) and a configuration file (usually in /etc/samba/smb.conf).

Last night, I rewrote most of the conversion code to be in EJS rather then in C. One of the positive things about this is that I can use embedded LDIF inside of the JavaScript code and that I can build on the existing Samba4 provisioning code (which is in EJS as well). Writing EJS wrappers for my existing C code was easier then I had thought.

One of the other things I did today is add a fallback mechanism for ldb_map where it can store the data that could not be mapped. There are now two types of records in ldb_map: those that are partially mapped (mapped, but might have one or two attributes stored in the fallback database) and those that are completely stored in the fallback database.

Below are some screenshots of the upgrade procedure in SWAT: /blog/uploads/screenshots/upgr1.png /blog/uploads/screenshots/upgr2.png /blog/uploads/screenshots/upgr3.png

comments.

LDIF output working

The sam.ldif and winbind.ldif generation code now works as well. I’ll be working on generating the new smb.conf next.

One of the things that worries me is the fact that I can generate so few of the Samba4 ‘user’ and ‘group’ attributes. Probably need to borrow some code from the current provisioning script to fix that up.

I’m wondering whether I should do some of the upgrade using EJS, so it can be integrated into SWAT easily…

comments.

ldb_map somewhat working

ldb_map and the samba3sam ldb module that is based on it work nicely now and appear to be memory-leak free. I have so far been testing with a TDB backend loaded with an LDIF from Samba3. Tomorrow will be the big day, testing against an OpenLDAP server.

Since the LDAP schema for Samba3 contains information that the Samba4 SAM database doesn’t store and vice versa, it will be interesting to find out whether the current mapping will be sufficient. If it is not, I might have to fall back to storing the Samba4 attributes I can’t store in the Samba3 LDAP database in a seperate database instead.

Another thing I did today is clean up the parameter list in Samba4 - I removed all unused parameters. This should hopefully make it easier to do the Samba3->Samba4 smb.conf conversion code.

TODO is down to:

  • smb.conf conversion
  • Finish sam.ldif / winbind.ldif from the TDB files
  • Test samba3sam module against OpenLDAP

comments.

First LDIF working

Today was a productive day. I just got the LDIF output for samba3dump working for wins.ldif and the various registry hives. TODO is down to:

  • sam.ldif
  • winbind.ldif
  • smb.conf
  • test and finish ldb_map and ldb_samba3

I hope to get the first three done tomorrow.

comments.

Starting on LDIF output

Next on my list is the generation of the LDIF’s for the one-way upgrade path. These will be generated by internally building ldb_ldif structs and then dumping them using the LDB helper functions. This would easily allow us to later on directly write to LDB database.

I’ll need to generate the following LDIF files:

  • hklm / hkcu / etc.ldif: from samba3_regdb
  • wins.ldif: from samba3_winsdb
  • sam.ldif: from samba3_secrets, samba3_policy, samba3_privileges, samba3_samdb
  • winbind.ldif: from samba3_winbind

I’ll also make a start on the smb.conf output.

comments.

ldb_map done

Finally got the ldb_map code done. This is a simple skeleton module that takes a mapping table of LDAP attributes and objectclasses and then maps between two different schema’s. Also got the initial part of ldb_samba3 compiling, which is the first module that actually uses ldb_map.

Yesterday evening, I also added support for reading smb.conf files and share_info.tdb files to the lib/samba3/ directory. Looks like my TODO-list is now down to:

  • testing (writing a couple of smbtorture tests, preferably)
  • code for generating new smb.conf and for generating LDIF from the various TDB’s

comments.

TDB reading done

I’ve now finished the ability to read the various TDB files. I’ve also started on the “mapping” LDB module that will allow a simple mapping from one objectclass to another. We’ll need this for the backwards compatible LDAP backend for Samba3.

This backend will never be used itself but instead have other backends based on it (e.g. ldb_samba3 for a Samba3-LDAP-to-samdb backend)

Each backend that uses this skeleton has to specify an array of struct objectclass_mapping’s (see ldb_map.h).

If no mappings are defined for an objectclass, it will be kept as is.

Will also keep cache of mapped dn’s so we don’t have to traverse the tree each time

So, TODO left:

  • finish ldb_map
  • fill ldb_samba3’s mapping table
  • support smb.conf parsing in lib/samba3/
  • support generating LDIF’s

comments.

Samba4 debian packages

Now that the Samba4 Debian packages appear to be somewhat working, I’ve set up a daily cronjob that generates packages for the latest SVN revision. Add the following line to your “/etc/apt/sources.list” to try them out:

1
deb http://jelmer.vernstok.nl/debian/autobuild .

Please note that these packages are still very much alpha, as is the Samba4 code. Comments are welcome

comments.

Status Update

My Summer of Code project is coming along nicely (although I haven’t been very actively blogging about it). I’ll be working on full-time until the deadline, I guess, but I think I’ll make it.

So far I’ve got the following done:

  • Reading of tdbsam
  • Reading of policy database (still need to add privilege support)
  • Reading of idmap database
  • Reading of registry file (still needs some work)
  • Reading of old smb.conf files
  • Reading of group mapping database
  • Initial work on ldb_map backend

The major lump of work now is the ldb_map backend and the conversion of the smb.conf.

comments.

Status Update

Althought I haven’t been writing my weekly Samba4 status updates for a while now, development on Samba4 certainly hasn’t been stalled. There have been a lot of improvements since June. I’ll try to sum up most of them here, hopefully not forgetting too much:

  • ldb_sqlite is done afaik, still need to try it
  • Deryck and Tridge have been working on SWAT, giving it a user-friendly interface
  • Lots of progress in the directory synchronisation interface by Brad (Summer of Code participant) and Metze
  • Andrew has done more work on Kerberos integration such as the kpassword protocol and the PAC
  • Yet more LDB updates from Simo, not sure what exactly :-)
  • Quite a couple of things have been converted to Javascript. ejs is now being used for some of the testing as well as the provisioning script. Mostly by Tridge and Mimir
  • pidl updates and build system updates from me

… and much, much, more

I’m personally still a bit skeptical on using ejs for things other then SWAT, but I can’t really describe why. Perhaps it’s the fact that when I used one of its early incarnations, Javascript was very hairy to use. I also don’t get that “warm fuzzy feeling” when I use Javascript that I get when I use C, C#, Haskell or Python, but I guess that’s not really a good argument. :-) The C interfaces to EJS are certainly very easy to use and EJS is a nice small library.

CIFS was good again this year. Samba.org has some pictures.

comments.

Ethereal parser generators

Together with a few others (Tim Potter, Alan De Kok) I have been working on parser generators in pidl for DCE/RPC interfaces over the last one-and-a-half years. However, none of our attempts got far enough to generate both clean and working code.

I sat down with Ronnie Sahlberg from Ethereal at the CIFS 2005 Conference and discussed some of the approaches he had taken in his idl2eth compiler (which is a big hack, but does generate nice code). With the help of Ronnie I was able to write a new module in about a day, taking to account some of the concerns the ethereal developers have.

See the new Parse::Pidl::Ethereal::NDR module for details. Hopefully Ethereal will adapt pidl with this new module for generating at least some of the DCE/RPC parser generators during the next month or so.

comments.

Participating in the Google Summer of Code!

This summer, I will be participating in the Google Summer of Code. The project I will be working on is the Samba3->Samba4 upgrade path.

In my original proposal, I split this project up into the following subtasks:

  • Each of the various password database backends (tdb, ldap, smbpaswd) available in Samba3 need to be accessible and converted to the SAM database in Samba4.
  • Printer database needs to be read from the Samba3 TDB and added to the apprioprate LDAP subtree.
  • The WINS database needs to be converted from both plain text file and TDB file to LDAP/LDB.
  • The account policy database needs to be read in from a TDB and applied to the data in LDAP/LDB.
  • The privilege database needs to be read from a TDB and added to the SAM database in LDB.
  • Group mappings need to be read from the TDB and added to the SAM database.
  • The share info database and the configuration file from Samba3 need to be read and converted to either xattrs or a TDB in Samba4
  • Secrets.tdb, containing the domain formation, needs to be merged into the SAM database.
  • Last, but not least, a Samba3 configuration file should be parsable in Samba4. This file can be used for producing a Samba4 smb.conf file as well as for updating the various databases mentioned above.

I will probably have to change this later on, but it’s a good list to start with.

comments.

Samba4 this week: Bug squashing

It has been a relatively quiet week for Samba4. Tridge has been working on supporting some more of the Windows LDAP (such as binary or and and operators). Andrew has merged the configuration loading and the command-line handling.

Most changes were bugfixes for existing code by Tridge, Andrew, Metze, Simo, Rafal and me.

comments.

Samba4 this week: “make test”, EJS Wrapper scripts, TDR and strings in pidl

It has been another interesting week in Samba4-land. We have now enabled the Samba4 test suite on all machines in the Samba buildfarm. I did the original (small) patch, metze has done a lot of nice work cleaning up the output, making sure it didn’t try to run the tests when smbd was down, etc and tridge has added options to smbd and smbtorture to make sure they don’t get stuck in an endless loop.

The SPOOLSS test now also works, thanks to continuing efforts from Metze over the past few weeks.

HotaruT and Derell have been working on, respectively, a PostGreSQL and a SQLite backend to LDB. I haven’t checked them out yet (they’re still mostly works-in-progress), but it’d be interesting to see whether they can beat TDB in performance.

I’m currently looking at implementing a new (very simple) data representation format in pidl, so that we don’t have to depend on NDR for, for example, the NetBIOS implementation. It’ll be a very simple format, not supporting pointers and stuff like that, so probably a bit more efficient as well.

Another thing I’m looking at is “proper” string support, e.g. similar to what MIDL has. Pidl’s current extensions (automatic conversion to UTF8, etc) could be supported using a couple of custom attributes.

Other changes this week included improvement of IRPC performance by tridge, EJS improvements by tridge and Tim, more heimdal fixes by Andrew Bartlett.

comments.

NUnit packages

I’m currently fixing the last few quirks in my NUnit Debian packages, after getting a some good feedback from the folks at #debian-mono. Just a few more hours before I’ll put into pkg-mono’s SVN. Until then, snapshots are available at:

1
2
deb ftp://autobuild.vernstok.nl/pub/releases/nunit/debian ./
deb-src ftp://autobuild.vernstok.nl/pub/releases/nunit/debian ./

I wonder what I’ll work on next. Perhaps nantcontrib or ndoc?

comments.