Running Couchbase 2.1.1 on SmartOS

I assume most of my readers knows about my love to the Solaris operating system and its descendants such as SmartOS. I've been on vacation for a couple of weeks now, and during my vacation I noticed some comments on previous blog posts that people had tried to build Couchbase on their SmartOS system without success so I figured I should create a new blog post where I walked through the steps needed.

To make reproducible steps for people who are interested, I decided to ensure that the blog post includes all the steps needed (including creating the environment). The first thing we need to do is to log into our server and update the list of available datasets. I started off this morning by downloading smartos-20130725T202435Z-USB.img.bz2 and created a bootable USB stick and booted my server.

With my SmartOS server running the (as of today) latest bits I imported the dataset I was going to use for my build with the following command:

[root@smartos ~]# imgadm import 9eac5c0c-a941-11e2-a7dc-57a6b041988f

And created a the vm with the following setup:

[root@smartos ~]# cat | vmadm create
{
  "alias" : "couchbase",
  "autoboot": true,
  "brand": "joyent",
  "dns_domain" : "norbye.org",
  “resolvers” : [ "10.0.0.1" ],
  "image_uuid" : "9eac5c0c-a941-11e2-a7dc-57a6b041988f",
  "hostname" : "cbbuilder",
  "max_physical_memory": 4096,
  "nics": [
    {
      "nic_tag": "admin",
      "ip": "10.0.0.150",
      "netmask": "255.255.255.0",
      "gateway": "10.0.0.1"
    }
  ]
}

^D

Listing all of my vm's shows:

[root@smartos ~]# vmadm list
UUID                                  TYPE  RAM      STATE             ALIAS
1200e3a9-a9cc-49e5-b9f0-bed2ec3b005d  OS    4096     running           couchbase

The first thing I did was to log in and set the password for the root user and create my own user to use during the build process:

[root@smartos ~]# zlogin 1200e3a9-a9cc-49e5-b9f0-bed2ec3b005d
[Connected to zone '1200e3a9-a9cc-49e5-b9f0-bed2ec3b005d' pts/5]
Last login: Mon Aug  5 08:22:26 on pts/3
   __        .                   .
 _|  |_      | .-. .  . .-. :--. |-
|_    _|     ;|   ||  |(.-' |  | |
  |__|   `--'  `-' `;-| `-' '  ' `-'
                   /  ; SmartMachine (base64 13.1.0)
                   `-'  http://wiki.joyent.com/jpc2/SmartMachine+Base

[root@cbbuilder ~]# passwd root
[root@cbbuilder ~]# useradd -g 10 -s /usr/bin/bash \
-d /home/trond -m trond

[root@cbbuilder ~]# passwd trond
^D

Now that I've got my own user I logged in as that user over an ssh shell and became root and installed all of the packages I need to build Couchbase:

[trond@cbbuilder ~]$ pfexec su -
[root@cbbuilder ~]# pkgin -y in libtool-base autoconf \
                                automake scmgit-base gcc47 \
                                gnupg gmake libevent icu \
                                py27-expat snappy erlang \
                                subversion-base
[root@cbbuilder ~]# wget --no-check-certificate \
                         -O/opt/local/bin/repo \
        https://git-repo.googlecode.com/files/repo-1.19
[root@cbbuilder ~]# chmod a+x /opt/local/bin/repo

I'll be installing Couchbase to /opt/couchbase, so lets go ahead and create that:

[root@cbbuilder ~]# mkdir /opt/couchbase
[root@cbbuilder ~]# chown trond /opt/couchbase
^D

There are a few dependencies Couchbase use that don't exist in the pkgin repository. Let's go ahead and build them and install them into /opt/couchbase.

[trond@cbbuilder ~]$ wget --no-check-certificate \

https://gperftools.googlecode.com/files/gperftools-2.1.tar.gz

[trond@cbbuilder ~]$ gtar xfz gperftools-2.1.tar.gz

[trond@cbbuilder ~]$ cd gperftools-2.1

[trond@cbbuilder ~/gperftools-2.1]$ ./configure --enable-minimal \
--enable-shared \
--disable-static \
--prefix=/opt/couchbase

[trond@cbbuilder ~/gperftools-2.1]$ gmake install

[trond@cbbuilder ~/gperftools-2.1]$ cd ..

[trond@cbbuilder ~]$ wget --no-check-certificate -Ov8.tar.gz \

https://github.com/v8/v8/archive/3.19.0.tar.gz

[trond@cbbuilder ~]$ gtar xfz v8.tar.gz

[trond@cbbuilder ~]$ cd v8-3.19.0
[trond@cbbuilder ~/v8-3.19.0]$ gmake dependencies
[trond@cbbuilder ~/v8-3.19.0]$ gmake x64 library=shared -j 4
[trond@cbbuilder ~/v8-3.19.0]$ cp out/x64.release/lib.target/libv8.so \
/opt/couchbase/lib

[trond@cbbuilder ~/v8-3.19.0]$ cp include/* /opt/couchbase/include/

To avoid passing too many arguments when we're invoking make we can add them into ~/.couchbase/build/Makefile.extra:

[trond@cbbuilder ~]$ mkdir -p ~/.couchbase/build

[trond@cbbuilder ~]$ cat > ~/.couchbase/build/Makefile.extra

OPTIONS += CPPFLAGS="-I$(PREFIX)/include"

OPTIONS += LDFLAGS="-R/opt/local/lib -L$(PREFIX)/lib -R$(PREFIX)/lib"

OPTIONS += CXX="g++ -L/opt/local/lib -I/opt/local/include" 

OPTIONS += CC="gcc -I/opt/local/include -L/opt/local/lib"

memcached_EXTRA_OPTIONS += --enable-tcmalloc-minimal

^D

We need to "configure" git before we can start use it to download the source code:

[trond@cbbuilder ~]$ git config --global user.email "trond.norbye@localhost"
[trond@cbbuilder ~]$ git config --global user.name "Trond Norbye"
[trond@cbbuilder ~]$ mkdir compile && cd compile
[trond@cbbuilder ~/compile]$ repo init -u git://github.com/membase/manifest.git -m released/2.1.1.xml
[trond@cbbuilder ~/compile]$ repo sync

Unfortunately there is a problem with one of the exceptions being thrown in Couchbase that cause a crash on SmartOS, so we need to "patch" one file. Its not hard, just add the following 3 lines of code:

[trond@cbbuilder ~/compile/ep-engine]$ git diff
diff --git a/src/couch-kvstore/couch-kvstore.cc b/src/couch-kvstore/couch-kvstore.cc
index 931fb30..a48f271 100644
--- a/src/couch-kvstore/couch-kvstore.cc
+++ b/src/couch-kvstore/couch-kvstore.cc
@@ -515,6 +515,9 @@ void CouchKVStore::getPersistedStats(std::map &stats)
 {
     char *buffer = NULL;
     std::string fname = dbname + "/stats.json";
+    if (access(fname.c_str(), F_OK) == -1) {
+        return;
+    }
     std::ifstream session_stats;
     session_stats.exceptions (session_stats.failbit | session_stats.badbit);
     try {

With that in place we can build Couchbase with the following command:

[trond@cbbuilder ~/compile]$ gmake PREFIX=/opt/couchbase

When make completes /opt/couchbase should contain a successful build of Couchbase 2.1.1, and at this time you should probably go ahead and create your startup scripts etc. We can try to emulate a cluster by starting 2 nodes on the same machine by running the following command:

[trond@cbbuilder ~/compile]$ cd ns_server

[trond@cbbuilder ~/compile/]$ ./cluster_run -n 2

And in another terminal we can build the cluster by executing:

[trond@cbbuilder ~/compile/]$ ./cluster_connect -n 2

Now tune your browser to the IP address of your server at port 9000 and enjoy your cluster.

Happy hacking!

Trond