Building membase from the sources...

I thought I should share some information about my personal development model for membase.

I've set up a "sandbox" where I'm doing all of my development in with the following commands:

trond@opensolaris> pfexec zfs create -o mountpoint=/source rpool/source
trond@opensolaris> pfexec chown trond:staff /source
trond@opensolaris> mkdir /source/membase
trond@opensolaris> cd /source/membase
trond@opensolaris> git clone git://github.com/trondn/tools.git
trond@opensolaris> cd tools/membase

I like to keep my changes as separated as possible, to reduce the dependencies between them. Whenever I am fixing a bug report I would do something like:

trond@opensolaris> mkdir bugnnn
trond@opensolaris> cd bugnnn
trond@opensolaris> ln -s ../Makefile
trond@opensolaris> make

That would build the entire Membase stack and put the files in /tmp/membase-build. I would then change my working directory to the module where I'm going to fix a bug and (hopefully) fix it.

After fixing the bug (and writing a test case!) I would commit the change and push it for review with the following commands:

trond@opensolaris> git add -p   (and select the changes to include)
trond@opensolaris> git commit -m "bugnnn: blah blah blah"
trond@opensolaris> git for-review

The last command there would push the change to our review system, so that Dustin or someone else can read through the diffs and accept the patch if they like it.

If you look at the workflow above it looks pretty easy, there is however one little thing that is really annoying... That is that Membase is a cross platform project, so I need to ensure that the code compiles and works on all of our platforms. With the method above I would have to log into another system and set everything up and copy my change over to see that it works. For simple changes that only touch one module I could always use buildbot or Hudson to test it on all platforms, but that doesn't work if I do an interface change that affects all of our modules.

I'm kind of lazy so I don't want to do such boring work all of the time, so instead I wrote a script to set up the sources and create makefiles so that I can easily build the same source tree on all of my platforms.

In order for it to work you need to set up sharing on your filesystem:

trond@opensolaris> pfexec zfs set sharenfs=on rpool/source
trond@opensolaris> pfexec zfs set sharesmb=name=source rpool/source

To set up a tree for lets say bug 9999 I would run something like:

trond@opensolaris> ./setup.sh bug_9999
Download commit hook - Ok.
Checking out libmemcached (Bazaar) - Ok.
  Generate configure script - Ok.
Checking out bucket_engine (git) - Ok.
Checking out ep-engine (git) - Ok.
  Generate configure script - Ok.
Checking out libconflate (git) - Ok.
  Generate configure script - Ok.
Checking out libvbucket (git) - Ok.
  Generate configure script - Ok.
Checking out memcached (git) - Ok.
  Generate configure script - Ok.
Checking out moxi (git) - Ok.
  Generate configure script - Ok.
Checking out vbucketmigrator (git) - Ok.
  Generate configure script - Ok.
Checking out membase-cli (git) - Ok.
Checking out ns_server (git) - Ok.
Checking out memcachetest (git) - Ok.
  Generate configure script - Ok.
Configure build for SunOS

This will set up a build environemnt for Solaris that builds membase as a "multi isa" (both 32 and 64 bit) stack in /tmp/membase. But let's add support for my MacOSX, Ubuntu and Debian box as well. Since all of the code is located on my opensolaris box, I need to use the -s option to let it know where the source is located:

trond@opensolaris> ./setup.sh -s /net/opensolaris/source/membase/tools/membase -p Ubuntu bug_9999
Configure build for Ubuntu
trond@opensolaris> ./setup.sh -s /net/opensolaris/source/membase/tools/membase -p Darwin bug_9999
Configure build for Darwin
trond@opensolaris> ./setup.sh -s /net/opensolaris/source/membase/tools/membase -p Debian bug_9999
Configure build for Debian

So let's look inside the bug_9999 directory:

trond@opensolaris> ls -l bug_9999
total 15
drwxr-xr-x  13 trond    staff         14 Oct 12 13:35 Darwin
drwxr-xr-x  13 trond    staff         14 Oct 12 13:35 Debian
drwxr-xr-x  13 trond    staff         14 Oct 12 13:33 src
drwxr-xr-x   4 trond    staff          5 Oct 12 13:33 SunOS
drwxr-xr-x  13 trond    staff         14 Oct 12 13:35 Ubuntu

All of the sources are located in the src directory, and all of the makefiles for the various platforms will reference that code.

To build on all of my platforms I'm just executing:

trond@opensolaris> ssh ubuntu "cd /net/opensolaris/source/membase/tools/membase/bug_9999/Ubuntu && make" > ubuntu.log 2>&1 &
trond@opensolaris> ssh debian "cd /net/opensolaris/source/membase/tools/membase/bug_9999/Debian && make" > debian.log 2>&1 &
trond@opensolaris> ssh darwin "cd /net/opensolaris/source/membase/tools/membase/bug_9999/Darwin && make" > darwin.log 2>&1 &

but I've got that in a script of course:

trond@opensolaris> cat bug_9999/build.sh
#! /bin/ksh
cd SunOS && gmake > sunos.log 2>&1 &
ssh ubuntu "cd /net/opensolaris/source/membase/tools/membase/bug_9999/Ubuntu && make" > ubuntu.log 2>&1 &
ssh debian "cd /net/opensolaris/source/membase/tools/membase/bug_9999/Debian && make" > debian.log 2>&1 &
ssh darwin "cd /net/opensolaris/source/membase/tools/membase/bug_9999/Darwin && make" > darwin.log 2>&1 &
xterm -T SunOS -e tail -f sunos.log &
xterm -T Ubuntu -e tail -f ubuntu.log &
xterm -T Debian -e tail -f debian.log &
xterm -T MacOS -e tail -f darwin.log &

Unfortunately you can't start the membase we just installed in /tmp/membase, but I'm working on it!