Friday, September 28, 2012

YANFS - Yet Another Network File System

In the last few blog posts we've been exploring different kinds of how you might want to utilize libcouchbase from your app. To me it has always been really hard to come up with good example apps, so I always end up creating pretty boring applications.

Those of you who attended CouchConf in San Francisco really missed out if you didn't watch Couchbase Labs demo sessions, where they showed off CBFS. I'm not going to go into any details about what CBFS is (hopefully Dustin Sallings will create a blog post about that), but you can think cbfs as a large object store analogous to S3 (but meant to run in your own environment).

CBFS is implemented in the go language (if you haven't looked at that yet, put that on your todo list!), and it's shipped with it's own client to perform operations (upload files, list files etc). This gave me a great idea for todays blog post! FUSE is a project that allows you to create a filesystem implementation in user space, so the topic for today is an example where we utilize libcouchbase in our implementation for our FUSE driver.

As usual I won't explain the entire code, but I'll try to comment on the important bits. You'll find the source code for the entire project at http://github.com/trondn/mount_cbfs

The readme in the project should tell you how to install fuse and build the stuff, so in this blog post I'm only going to focus on how it works.

Creating a full featured filesystem driver is a lot of work, so I figured it would be better to start with a minimal implementation with a lot of limitations we could fix up later on (or I would never get around building it ;))


  • It is fully single threaded. This should be pretty easy to fix up later on by using a pool of libcouchbase instances, no shared data etc..
  • Read Only. This limits the number of entry points we need to implement.
So let's start looking at the code and how it all fits together. In main we register the struct containing all of the function pointers to the functions our filesystem support:

static struct fuse_operations cbfs_oper = {
    .getattr = cbfs_getattr,
    .open = cbfs_open,
    .read = cbfs_read,
    .readdir = cbfs_readdir,
};

This means that whenever someone tries to open a file in our filesystem, cbfs_open is called etc. So let's walk through an example here: What is the flow when you're typing "ls -l" in your shell.

The first things that happens is that cbfs_getattr is called. Its responsibility is to populate a stats struct with information about the file. Let's skip the details for now, and just assume that we return that this is a directory. Now cbfs_readdir is called with the directory to receive all of the entries in the directory, before cbfs_gettattr is called for every file to get the details of the files.

So how does our cbfs_readdir work like (after all this blog is about libcouchbase, not fuse ;)). CBFS use a http interface so we can request the list of files in a directory through the following URL:

http://cbfsserver:8484/.cbfs/list/path

It returns a JSON document looking something like:

{"files":{"file1":{}},"dirs":{},"path":"/foo"}

All we need to do is to decode the JSON and populate the information to FUSE. So how do we do this through libcouchbase? Through the http interface:

static lcb_error_t uri_execute_get(const char *uri, struct SizedBuffer *sb) {
    lcb_http_cmd_t cmd = {
        .version = 1,
        .v.v1 = {
            .path = uri,
            .npath = strlen(uri),
            .body = NULL,
            .nbody = 0,
            .method = LCB_HTTP_METHOD_GET,
            .chunked = 0,
            .content_type = "application/x-www-form-urlencoded",
            .host = cfg->cbfs_host,
            .username = cfg->cbfs_username,
            .password = cfg->cbfs_password
        }
    };

    return lcb_make_http_request(instance, sb, LCB_HTTP_TYPE_RAW, &cmd, NULL);
}

Since I'm using the synchronous interface to libcouchbase that call will block until we've received the response from the server. In my response handler I just copy whatever data the server returned to me:

static void complete_http_callback(lcb_http_request_t req, lcb_t instance,
                                   const void *cookie, lcb_error_t error,
                                   const lcb_http_resp_t *resp)
{
    struct SizedBuffer *sb = (void*)cookie;

    if (error == LCB_SUCCESS) {
        /* Allocate one byte extra for a zero term */
        sb->data = malloc(resp->v.v0.nbytes + 1);
        sb->size = resp->v.v0.nbytes;
        memcpy(sb->data, resp->v.v0.bytes, resp->v.v0.nbytes);
        sb->data[resp->v.v0.nbytes] = '\0';
    }
}

Both cbfs_read and cbfs_getattr utilize the same function from libcouchbase, but they're only hitting different URL's to get their data.

Adding supports for write operations should be no worse than finding the correct URL and do a PUT etc.

Happy Hacking :)



Saturday, September 22, 2012

Extend a Couchbase Server with your own commands!

I was having a beer with Sergey after yesterday's CouchConf in San Francisco and we started discussing command extensions for memcached. I guess I haven't been very good at advertising about what I've been doing, because I added this to memcached a couple of years ago as part of the plugin extension framework.

I thought that it might be interesting to write up a small blog post that shows you how you may write your own commands to access a Couchbase cluster. Please note that what I'm going to show you in this blog post is abusing Couchbase Server, and nothing you should be doing in production. If you have special needs you should connect with us to let us work with you to make it happen (and ensure that you don't break stuff).

So lets imagine that you're storing relatively big objects with data into a cluster, and you only need fragments of each object at a time. The current SDK only allows you to read and write a complete item at a time, so we're going to extend our Couchbase cluster to be able to give us a fragment of the data at a time.

Luckily, I wrote this extension for memcached a couple of years ago, and it is actually bundled with the Couchbase Server. This does however not imply that it is a supported extension to be using from your Couchbase Server. So let's go ahead and enable the extension on the server:

# cd /opt/couchbase/bin
# mv memcached memcached.bin
# cat > memcached << ...
#! /bin/sh
exec `dirname $0`/memcached.bin \
            -X /opt/couchbase/lib/memcached/fragment_rw.so "$@"
...
# chmod +x memcached

If you are running a cluster you need to do this on all of your nodes (and you need to restart all of your nodes for them to load the module).

You don't learn anything unless we're walking through how the extension actually works. You'll find the module at https://github.com/trondn/memcached/tree/engine-pu/extensions/protocol in the files fragment_rw.c and fragment_rw.h.

So let's look into the details there. During startup memcached will load all shared objects specified with the -X parameter, and try to look up the function named memcached_extensions_initialize. This is where you should initialize your engine, and add it to the system. If you look at the signature for the method, you might wonder where you may specify the configuration. The "per engine" configuration is specified as part of the -X parameter. The fragment module allows you to specify the command id for read and write through "r" and "w", so we could write -X fragment.so,r=5,w=6 to tell it to use the command code 5 and 6 (please note that it would be a conflict with the existing commands ;))

Setting up a command extension happens in two different phases. The first thing you need to do is to registering your command descriptor:


server->extension->register_extension(EXTENSION_BINARY_PROTOCOL, &descriptor);

The descriptor for protocol extensions is pretty simple:

static EXTENSION_BINARY_PROTOCOL_DESCRIPTOR descriptor = {
    .get_name = get_name,
    .setup = setup
};

get_name is a function that just returns the name of the extension, whereas setup is the interesting function:


static void setup(void (*add)(EXTENSION_BINARY_PROTOCOL_DESCRIPTOR *descriptor,
                              uint8_t cmd,
                              BINARY_COMMAND_CALLBACK new_handler))
{
    add(&descriptor, read_command, handle_fragment_rw);
    add(&descriptor, write_command, handle_fragment_rw);
}

So what is happening here. The function is called with a parameter that is a function you may call in order to register handlers for various command codes. The first parameter is you should pass to this callback is the descriptor you registered. The second parameter is the command opcode you want to install a handler for, and the third parameter is the function that implements the logic.

So let's go ahead and look at how the function looks:

static ENGINE_ERROR_CODE handle_fragment_rw(EXTENSION_BINARY_PROTOCOL_DESCRIPTOR *descriptor,
                                            ENGINE_HANDLE* handle,
                                            const void* cookie,
                                            protocol_binary_request_header *request,
                                            ADD_RESPONSE response)

The descriptor is the descriptor you registered, the handle is the handle to the engine keeping all of the data (so that you may interact with the backing store). Cookie is a reference to the client who connected to you. Request is the complete package the client sent for the request, and response is a callback function you may call to send data back to the client. If you look at the implementation for handle_fragment_rw you'll see that it's not that hard to interact with the engine.


The only thing left for us now is to create create client functionality to connect to a server and send such a packet :) I leave that up to you folks for the moment (or if people would like me to post it here I'd be happy to do so. let me know).

Happy hacking!


Monday, September 17, 2012

libcouchbase is async, but my app isn't...

In the previous example I showed you how to hook libcouchbase into your own event loop, but not everyone wants to use it asynchronously. The best part of an asynchronous library is that it's pretty easy to make a synchronous library on top of it. All you need to do is to call the function and wait for it to complete, but in libcouchbase I went the extra mile. You may use it fully asynchronous, you may use it fully synchronous or you may use it somewhat in the middle.

The easiest way to use libcouchbase in a synchronous way is to toggle it's internals to the syncmode setting by calling:

lcb_behavior_set_syncmode(instance, LCB_SYNCHRONOUS);

One "drawback" with the syncmode setting is that you can't execute a chunk of commands concurrently. Lets look at the following example:

lcb_store(instance, NULL, 15, store_commands);
lcb_get(instance, NULL, 2, get_commands);

With syncmode enabled all of the store commands must have been executed and the response received before any of the get commands is sent (even if they don't hit the same servers). It would improve the latency of your application if you could shuffle as much data as possible over the network before blocking to wait for the result. That's when you want to set the syncmode to LCB_ASYNCHRONOUS (the default) and use lcb_wait() instead:

lcb_store(instance, NULL, 15, store_commands);
lcb_get(instance, NULL, 2, get_commands);
lcb_wait(instance);

when lcb_wait() returns we know that all of the above commands are executed (and the appropriate callbacks called).

So let's whip up a small program that shows you how to use the syncmode setting. To keep the example as small as possible I'm going to skip error recovery etc. Let's jump straight to the main() method:

int main(void)
{
    lcb_error_t error;
    lcb_t instance = create_instance();

    if ((error = lcb_connect(instance)) != LCB_SUCCESS) {
        fprintf(stderr, "Failed to connect to cluster: %s\n",
                lcb_strerror(instance, error));
        exit(EXIT_FAILURE);
    }

    set_key(instance);
    get_key(instance);

    lcb_destroy(instance);
    exit(EXIT_SUCCESS);
}

As you see, the program is pretty simple. First we create a libcouchbase instance and connect to the cluster; set and get a key before we clean up and exits.

Let's look inside create_instance:

static lcb_t create_instance(void)
{
    lcb_t instance;
    struct lcb_create_st copt;
    lcb_error_t error;

    memset(&copt, 0, sizeof(copt));
    copt.v.v0.host = "myserver:8091";

    if ((error = lcb_create(&instance, &copt)) != LCB_SUCCESS) {
        fprintf(stderr, "Failed to create libcuchbase instance: %s\n",
                lcb_strerror(NULL, error));
        exit(EXIT_FAILURE);
    }

    lcb_behavior_set_syncmode(instance, LCB_SYNCHRONOUS);
    lcb_set_error_callback(instance, error_handler);
    lcb_set_store_callback(instance, store_handler);
    lcb_set_get_callback(instance, get_handler);

    return instance;
}

This looks pretty much like how we normally do stuff except for the line I've marked as bold. From that line all of the lcb_ functions will block until we've received a reply from the server for the requested operation. You might be curious about how my handler functions looks like, and they are pretty boring:

static void error_handler(lcb_t instance, lcb_error_t err, const char *info)
{
    fprintf(stderr, "FATAL! an error occured: %s (%s)\n",
            lcb_strerror(instance, err), info ? info : "none");
    exit(EXIT_FAILURE);
}

static void store_handler(lcb_t instance, const void *cookie,
                          lcb_storage_t operation, lcb_error_t error,
                          const lcb_store_resp_t *resp)
{
    (void)cookie; (void)operation;

    if (error != LCB_SUCCESS) {
        fprintf(stderr, "Failed to store the key on the server: %s\n",
                lcb_strerror(instance, error));
        exit(EXIT_FAILURE);
    }

    if (resp->version == 0) {
        fprintf(stdout, "Successfully stored \"");
        fwrite(resp->v.v0.key, 1, resp->v.v0.nkey, stdout);
        fprintf(stdout, "\"\n");
    }
}

static void get_handler(lcb_t instance, const void *cookie,
                        lcb_error_t error, const lcb_get_resp_t *resp)
{
    (void)cookie;

    if (error != LCB_SUCCESS) {
        fprintf(stderr, "Failed to read the key from the server: %s\n",
                lcb_strerror(instance, error));
        exit(EXIT_FAILURE);
    }

    /* Validate that I read the correct key and value back */
    if (resp->version != 0) {
        fprintf(stderr,
                "WARNING: I don't support this version of libcouchbase\n");
        exit(EXIT_FAILURE);
    }

    fprintf(stdout, "I received \"");
    fwrite(resp->v.v0.key, 1, resp->v.v0.nkey, stdout);
    fprintf(stdout, "\" with the value: [");
    fwrite(resp->v.v0.bytes, 1, resp->v.v0.nbytes, stdout);
    fprintf(stdout, "]\n");
}

If we jump back to our main function the next instructions to execute would be:

    if ((error = lcb_connect(instance)) != LCB_SUCCESS) {
        fprintf(stderr, "Failed to connect to cluster: %s\n",
                lcb_strerror(instance, error));
        exit(EXIT_FAILURE);
    }

Now that we've enabled the LCB_SYNCHRONOUS mode in libcouchbase this will be a blocking call (I fixed that bug today Sept 17th), so that when the method returns we know the topology of the Couchbase cluster and we're about to store our object there in the set_key method that looks like:

static const char const *key = "mykey";
static const char const *value = "myvalue";

static void set_key(lcb_t instance)
{
    lcb_store_cmd_t cmd;
    lcb_error_t error;
    const lcb_store_cmd_t * const commands[] = { &cmd };

    memset(&cmd, 0, sizeof(cmd));
    cmd.v.v0.key = key;
    cmd.v.v0.nkey = strlen(key);
    cmd.v.v0.bytes = value;
    cmd.v.v0.nbytes = strlen(value);
    cmd.v.v0.operation = LCB_SET;

    if ((error = lcb_store(instance, NULL, 1, commands)) != LCB_SUCCESS) {
        fprintf(stderr, "Failed to store key: %s\n",
                lcb_strerror(instance, error));
        exit(EXIT_FAILURE);
    }
}


This looks exactly like how we would have done it in our asynchronous application, and here is a very important detail. The return code of the function is not the return code of the actual operation! It means the exact same thing as it used to: if we encountered any problems initiating the operation. It is the callback function that will tell you the result of the operation from the server!

To make the example complete let's look at the get_key function as well:

static void get_key(lcb_t instance)
{
    lcb_get_cmd_t cmd;
    lcb_error_t error;
    const lcb_get_cmd_t * const commands[] = { &cmd };

    memset(&cmd, 0, sizeof(cmd));
    cmd.v.v0.key = key;
    cmd.v.v0.nkey = strlen(key);

    if ((error = lcb_get(instance, NULL, 1, commands)) != LCB_SUCCESS) {
        fprintf(stderr, "Failed to get key: %s\n",
                lcb_strerror(instance, error));
        exit(EXIT_FAILURE);
    }
}

Pretty simple and straight forward.

Happy hacking!

Saturday, September 15, 2012

How do I use libcouchbase with my own event loop?

Writing asynchronous programs seems to be popular these days, so I thought I should whip up an example that shows you how you may utilize libcouchbase with your own event loop.

libcouchbase performs it's IO through an "IO handle". This is a plugin system so you should be able to use whatever mechanism you want. Adding support for a new system is nothing more than implementing a handfull of function calls and place them in a shared object. When I wrote libcouchbase I created a version for libevent and a small version that use select for Windows just to ensure that it should be possible to write such plugins. Later Mark Nunberg wrote the plugin we're using for node.js that is based on libuv. I'm not going to cover how to write your own plugin in this blog post (perhaps I'll get around to do that at a later time), but I'll show you how to let libcouchbase utilize the same event base from libevent that you're using for something else.

To keep the example as short as possible I'm only going to do the very basic error handling: print out an error message and terminate the program. This means that we can start writing our first function; our error handler:

static void error_callback(lcb_t instance, lcb_error_t error, const char *errinfo) {
    fprintf(stderr, "ERROR: %s %s\n", lcb_strerror(instance, error), errinfo);
    exit(EXIT_FAILURE);
}

I'm not very good at coming up with interesting examples, so today we'll just create a small program that connects to a Couchbase cluster, stores a key and then reads the key back out again.

So let's take a look at our main function:

int main(int argc, char** argv) {
    struct event_base *evbase = event_base_new();

    if (create_libcouchbase_handle(evbase) == -1) {
        exit(EXIT_FAILURE);
    }

    event_base_loop(evbase, 0);
    event_base_free(evbase);
    exit(EXIT_SUCCESS);
}


As you see there is nothing fancy there.. We're creating the event base that the rest of our application should use (I just don't use it for anything else in this example to make the example simple), before we create our libcouchbase handle and associate it with the newly created event base (we'll look at that shortly). We then start the event loop that will run the rest of the example through the callbacks.

So how does the create_libcouchbase_handle function look like:

static int create_libcouchbase_handle(struct event_base *evbase) {
    struct lcb_create_io_ops_st ciops;

    memset(&ciops, 0, sizeof (ciops));
    ciops.v.v0.type = LCB_IO_OPS_LIBEVENT;
    ciops.v.v0.cookie = evbase;

    lcb_io_opt_t ioops;
    lcb_error_t error = lcb_create_io_ops(&ioops, &ciops);
    if (error != LCB_SUCCESS) {
        fprintf(stderr, "Failed to create an IOOPS structure for libevent: %s\n",
                lcb_strerror(NULL, error));
        return -1;
    }

Let's stop there for a moment and look what we're doing. The first thing we're doing is that we're creating an instance of the lcb_create_io_ops_st structure. Like the rest of libcouchbase we're using a versioned struct for creating objects. The "constructor" for the IO handle on top of libevent allows us to pass the event base to use as the cookie.

We can now go ahead and create the libcouchbase instance "the normal way, with the only difference that we specify the io member in the create structure.:

    struct lcb_create_st copts;
    memset(&copts, 0, sizeof (copts));
    copts.v.v0.host = "localhost:8091";
    copts.v.v0.user = "Administrator";
    copts.v.v0.passwd = "secret";
    copts.v.v0.bucket = "default";
    copts.v.v0.io = ioops;

    lcb_t instance;
    if ((error = lcb_create(&instance, &copts)) != LCB_SUCCESS) {
        fprintf(stderr, "Failed to create a libcouchbase instance: %s\n",
                lcb_strerror(NULL, error));
        return -1;
    }

The next thing we'll do is to set up the different callbacks we're going to use in our program and call lcb_connect to initiate the connect sequence:

    lcb_set_error_callback(instance, error_callback);
    lcb_set_configuration_callback(instance, configuration_callback);
    lcb_set_get_callback(instance, get_callback);
    lcb_set_store_callback(instance, store_callback);

    if ((error = lcb_connect(instance)) != LCB_SUCCESS) {
        fprintf(stderr, "Failed to connect libcouchbase instance: %s\n",
                lcb_strerror(NULL, error));
        lcb_destroy(instance);
        return -1;
    }
}


You might be curious why I'm adding a "configuration callback"? This is actually a small trick :-) You might have tried to use libcouchbase yourself and had problems that your operations failed because you forgot to do a lcb_wait() after calling connect. The thing is that libcouchbase needs to know the topology of your cluster before it may perform any operations, and it cannot do that until it receives the first configuration from the server.

So how does this configuration callback look like in our example:

static void configuration_callback(lcb_t instance, lcb_configuration_t config) {
    if (config == LCB_CONFIGURATION_NEW) {
        // Since we've got our configuration, let's go ahead and store a value
        lcb_store_cmd_t cmd;
        const lcb_store_cmd_t * cmds[] = {&cmd};
        memset(&cmd, 0, sizeof (cmd));
        cmd.v.v0.key = "foo";
        cmd.v.v0.nkey = 3;
        cmd.v.v0.bytes = "bar";
        cmd.v.v0.nbytes = 3;
        cmd.v.v0.operation = LCB_SET;
        lcb_error_t err = lcb_store(instance, NULL, 1, cmds);
        if (err != LCB_SUCCESS) {
            fprintf(stderr, "Failed to set up store request: %s\n",
                    lcb_strerror(instance, err));
            exit(EXIT_FAILURE);
        }
    }
}


As the comment tells you, we've received the configuration from the server. This means that it's safe to start using the library. In a real world application you would probably have a more advanced logic here (please note that this callback will be called if you add/remove nodes etc, so it's not safe to assume that it will be called only once!)

When we receive the result for the store command from the server, our store_callback is called:

static void store_callback(lcb_t instance, const void *cookie, lcb_storage_t operation, lcb_error_t error, const lcb_store_resp_t *resp) {
    if (error != LCB_SUCCESS) {
        fprintf(stderr, "Failed to store key: %s\n",
                lcb_strerror(instance, error));
        exit(EXIT_FAILURE);
    }

    /* Time to read it back */
    lcb_get_cmd_t cmd;
    const lcb_get_cmd_t * cmds[] = {&cmd};
    memset(&cmd, 0, sizeof (cmd));
    cmd.v.v0.key = "foo";
    cmd.v.v0.nkey = 3;
    if ((error = lcb_get(instance, NULL, 1, cmds)) != LCB_SUCCESS) {
        fprintf(stderr, "Failed to setup get request: %s\n",
                lcb_strerror(instance, error));
        exit(EXIT_FAILURE);
    }
}

If the value was successfully stored on the server, we're issuing a single get command to the server to verify that it's there. When we receive the get result from the server, our get_callback is called and we're terminating the program:

static void get_callback(lcb_t instance, const void *cookie, lcb_error_t error, const lcb_get_resp_t *resp) {
    if (error != LCB_SUCCESS) {
        fprintf(stderr, "Failed to get key: %s\n",
                lcb_strerror(instance, error));
        exit(EXIT_FAILURE);
    }

    fprintf(stdout, "I stored and retrieved the key \"foo\". Terminate program");
    exit(EXIT_SUCCESS);
}

This wasn't the most exciting example, but it shows you the basics on how you may utilize libcouchbase with your own event loop. None of the above functions will cause your application to block waiting for socket IO. If that happens I'd be more than happy to fix it if you send me a bug report (unless you wrote your own IO plugin and forgot to enable nonblocking IO ;)

Happy hacking!

Friday, September 14, 2012

How do I utilize Couchbase from my node.js app?

In yesterdays post I told you that I had created an npm for our Couchbase driver, but I didn't give any examples on how you could utilize it so I figured I should create a short blog post about that. I'm the kind of guy who like real world examples instead of a lot of text, so I went ahead and created a small application for this purpose.

I didn't have any good ideas about what I could create, so I decided I should create a version of the vacuum example I did for libcouchbase a while back. The application is a small daemon that will move all JSON documents you store in a given directory into a Couchbase server by using the _id field in the JSON document as the key, and the entire file as the value. Its a pretty simple example, but should show us some of the concepts. You'll find the entire program (including installation notes) at https://github.com/trondn/vacuum.js.

I won't be going through the entire program, but only mention the specifics needed to talk to Couchbase.

The first thing we need to do is to create an instance of the Couchbase driver:

var couchnode = require('couchbase');
var cb = new couchnode.Couchbase(config.hostname,
    config.username,
    config.password,
    config.bucket);

Here we pass the address of the couchbase cluster (normally: hostname:8091), the username to log in with (normally the name of the bucket) with the corresponding password and the bucket we want to connect to. Unlike the C version we may start to use the instance immediately from JavaScript, and it will buffer all of our operations until it's connected.

If you look in the function process_file you'll find the first usage of the Couchbase instance:

cb.set(obj._id, String(data), 0, undefined, set_handler, fullname);

So what happens here? We try to store an object in Couchbase with the key being the content of the "_id" field in the JSON document. The current version of the driver only allows you to store Strings, so we need to create a String of the data. The next field is the expiration time of the object (0 means that the object will never expire). The undefined parameter is for the CAS field, which would allow us to say that we would only set this parameter if the object already exist in Couchbase with that identifier. The set_handler is the callback function when the operation is finished, and fullname is the absolute name of the file we just stored. But wait, why would the Couchbase driver need the absolute name of the file we just stored? The answer to that is that it doesn't. It is just a data field get back in the callback.

So let's look at the callback:

function set_handler(data, error, key, cas) {
    if (error) {
        console.log('Failed to store object: %s', key);
    } else {
        fs.unlink(data);
        process_next();
    }
}


It looks pretty simple? The first parameter is the "fullname" field we passed to the set parameter. It contains the absolute name of the file we wanted to store. The error parameter tells you if the operation succeeded or not. The key is the key we wanted to store, and the last field is the CAS identifier the server assigned to the object (which you may use in successive calls to set if you want to replace this exact version of the object.


Happy hacking!!!!

Thursday, September 13, 2012

couchnode meets npm

Earlier today I played around with npm to see if I could make your life easier when you're going to use couchnode to access your Couchbase cluster from your node app.

It turns out that it was pretty easy to create the package and upload it, but getting it to work the way I want to wasn't just as easy. Couchnode is built on top of libcouchbase, which means that you must have libcouchbase installed before you can try to install couchnode. I tried to figure out the recommended way to check for such dependencies in an npm module, but unfortunately I don't know yet (please drop me a note if you know how I should fix it!!!).

With that in mind, let's go ahead and install libcouchbase. The easiest would of course be to install this globally in /usr, but for the example of it I'll going to install it in my home directory under opt:

wget -O libcouchbase.tar.gz http://packages.couchbase.com/clients/c/libcouchbase-2.0.0beta.tar.gz
tar xfz libcouchbase.tar.gz
cd libcouchbase-2.0.0beta
./configure --prefix=$HOME/opt
make install

Since I didn't install libcouchbase into the "default location" for  libraries on the system I need to export some variables so that npm finds libcouchbase during npm install:

CPPFLAGS="-I$HOME/opt/include"
LDFLAGS="-L$HOME/opt/lib -Wl,-rpath,$HOME/opt/lib"
export CPPFLAGS LDFLAGS

You should now be able to install the couchnode driver with:

npm install couchbase

And you're good to go to use the couchnode driver from your web application :)