Tuesday, March 30, 2010

Building Memcached on Windows

I like to be able to compile my software on multiple platforms with different compilers, because it force me to write that complies to standards (if not you'll most likely have a spaghetti of #ifdefs all over your code). One of the bonuses you get from using multiple compilers is that you'll get "more eyes" looking at your code, and they may warn you on different things. Supporting multiple platforms will also make it easier to ensure that you don't have 'hidden bombs' in your code (byte order, alignment, assumptions that you can dereference a NIL pointer etc)...

Building software that runs on different flavors of Unix/Linux isn't that hard, but adding support for Microsoft Windows offers some challenges. Among the challenges are:

  1. Microsoft Windows doesn't provide the headerfiles found on most Unix-like systems (unistd.h etc).
  2. Sockets is implemented in a separate "subsystem" on windows.
  3. Win32 provides another threads/mutex implementation than what most software written for Unix-like systems use (pthreads).
  4. Microsoft doesn't supply a C99 compiler. One would think that it would be safe to assume that one could use a C standard that is more than 10 years old....

I am really glad to announce that we just pushed a number of changesets to memcached to allow you to build memcached on windows "just as easy" as you would on your favorite Unix/Linux system. This means that as of today users using Microsoft Windows is no longer stuck with an ancient version!

`The first thing you need to do in order to build memcached on Windows is to install the "fullversion" of msysgit. In addition to installing git, it also installs a compiler capable of building a 32 bit version of memcached.

So let's go ahead and build the dependencies we need to get our 32bit memcached version up and running!

The first thing up is libevent. You should download the latest 2.x release available (there is a bug in all versions up to (and including) 2.0.4, so unless there is a new one out there you can grab a development version I pushed to libevent-2.0.4-alpha-dev.tar.gz), and install it with the following commands (from within your msysgit-shell). Please note that I'm using /tmp because I've had problems using my "home directory" because of spaces in the directory name (or at least I guess that's the reason ;-) ):

$ cd /tmp
$ tar xfz libevent-2.0.4-alpha-dev.tar.gz
$ cd libevent-2.0.4-alpha-dev
$ ./configure --prefix=/usr/local
$ make all                    (this will fail with an error, but don't care about that.. it's is in the example code)
$ make install             (this will fail with an error, but don't care about that.. it's just an example)

It's time to start build memcached!!! Let's check out the source code and build it!

$ git clone git://github.com/trondn/memcached.git
$ git checkout -t origin/engine
$ make -f win32/Makefile.mingw

You should now be able to start memcached with the following command:

$ ./memcached.exe -E ./libs/default_engine.so

Go ahead and telnet to port 11211 and issue the "stats" command to verify that it works!

You don't need to install msysgit on the systems where you want to run memcached, but you do need to include pthreadGC2.dll from the msysgit distribution. That means that if you want to run memcached on another machine you need to copy the following files: memcached.exe .libs/default_engine.so and /mingw/bin/pthreadGC2.dll (Just place them in the same directory on the machine you want to run memcached on :-)

But wait, the world is 64 bit by now. Lets build a 64 bit binary instead

The msysgit we installed previously isn't capable of building a 64-bit binary, so we need to install a new compiler. Download the a bundle from http://sourceforge.net/projects/mingw-w64/files/ (I used mingw-w64-bin_i686-mingw_20100129.zip ). You can "install" it by running the following commands in our msysgit shell:

$ mkdir /mingw64
$ cd /mingw64
$ unzip /mingw-w64-bin_i686-minw_20100129.zip
$ export PATH=/mingw64/bin:/usr/local/bin:$PATH

So let's start compiling libevent:

$ cd /tmp
$ tar xfz libevent-2.0.4-alpha-dev.tar.gz
$ cd libevent-2.0.4-alpha-dev
$ ./configure --prefix=/usr/local --host=x86_64-w64-mingw32 --build=i686-pc-mingw32
$ make all
$ make install

The compiler we just downloaded didn't come with a 64-bit version of pthreads, so we have to download and build that ourself. I've pushed a package to my machine at: pthreads-w64-2-8-0-release.tar.gz

$ cd /tmp
$ tar xfz pthreads-w64-2-8-0-release.tar.gz
$ cd pthreads-w64-2-8-0-release
$ make clean GC CC=x86_64-w64-mingw32-gcc
$ cp pthread.h semaphore.h sched.h /usr/local/include
$ cp libpthreadGC2.a /usr/local/lib/libpthread.a
$ cp pthreadGC2.dll /usr/local/lib
$ cp pthreadGC2.dll /usr/local/bin

And finally memcached with:

$ git clone git://github.com/trondn/memcached.git
$ git checkout -t origin/engine
$ make -f win32/Makefile.mingw CC=x86_64-w64-mingw32-gcc

So how does it perform?

I'm pretty sure a lot of the "Linux fanboys" are ready to jump in and tell how much faster the Linux version is. I have to admit that I was a bit skeptical in the beginning if the layers we added to get "Unix compatibility" had any performance impact. The only way to know for sure is to run a benchmark to measure the performance.

I ran a small benchmark where I had my client create 512 connections to the server, and then randomly choosing a connection and perform a get or a set operation on one of the 100 000 items I had stored in the server. I ran 500 000 operation (33% of the operations are set operations), and calculated the average time. I can dual-boot one of my machines into Windows 7 and Red Hat Enterprise Linux, so my results shows the "out of the box"-numbers between Windows 7 and Red Hat Enterprise Linux running on the same hardware. I used my OpenSolaris box to drive the test (connected to the same switch). Posting numbers is always "dangerous", because people put too much into the absolute numbers. The intention with my graphs is to show that the version running on Microsoft Windows is pretty much "on par" with the Linux version.

256 byte userdata

512 byte userdata

1024 byte userdata

30 comments:

  1. Nice, compiled ok on XP.

    Slight snag was /tmp was mapped to a path with spaces which causes libevent to not install some of its headers.

    ReplyDelete
  2. Great job!!!

    Buuuuut..... the version from trondn/memcached.git is 1.3.3 and the last official version is 1.4.5 (http://www.memcached.org)....
    Using totalcommander I notice several differences between versions.
    And if I try to use the official repository instead of trondn's repository, the code wont compile.
    Also, the result exe wont work as a windows service or as a deamon either. This port does the trick but uses version 1.2.1 (Dec 23, 2006)...

    Never the less, the version 1.3.3 seems to work ok. Here is the compiled result from my Intel x86:
    http://fox.originsoftware.com.ar/memcached-1.3.3.zip

    Cheers,
    Lucas

    ReplyDelete
  3. Sorry, the memcached implementation of 1.2.1 that I was referring is:
    http://jehiah.cz/projects/memcached-win32/

    ReplyDelete
  4. I just found http://northscale.com/.
    They have a free implementation of 1.4.4!!!

    Cheers.
    Lucas

    ReplyDelete
  5. broken?! why?!
    fatal error: sys/utsname.h: No such file or directory
    compilation terminated.
    make: *** [.libs/memcached.o] Error 1

    ReplyDelete
  6. This works very well and I can wrap the exe around a windows service for easy management.


    What changes would be needed to be able to build the latest versions of memcached on Windows ? (1.4.5). As Irancez said.. it fails. I'm guessing it needs changes to the Makefile.mingw

    ReplyDelete
  7. We're currently working on cleaning up the next release.. I'll try to build a new 1.4.5 release that works on windows and push it somewhere... what do you folks want? a 64 bit binary?

    ReplyDelete
  8. 32 i 64 bit binary if it will be possible would be nice.
    But because PHP is still 32bit maybe 32 bit will be enough.

    ReplyDelete
  9. Microsoft Windows doesn't provide the headerfiles found on most Unix-like systems because Microsoft Windows, pre-built binaries are available in the form of a Microsoft installer file (.msi ).



    microsoft windows support

    ReplyDelete
  10. @Trond Norbye
    Thanks for the article. Will try.

    "what do you folks want? a 64 bit binary?"
    binaries for 32 and 64 bits :)

    ReplyDelete
  11. My windows machine just died, so I need to reinstall it before I can do anything else...

    ReplyDelete
  12. So just to clarify, this approach to building memcached neuters its BSD License by incorporating LGPL-based pthreads?

    ReplyDelete
  13. IANAL, but AFAIK the BSD license allows you to mix in other licenses, and the LGPL library allows you to link with other licenses.. You do however need to comply to the terms of both the licenses, so if you make modifications to the lgpl bits you need to make the source available to the people you give the binary... People should just stay away from *GPL and the world would be a better place...

    ReplyDelete
  14. Never worked with GIT before, and had one issue with your TUT.

    From your code:
    $ git clone git://github.com/trondn/memcached.git
    $ git checkout -t origin/engine

    The first line worked just fine, but when I tried to do the second line, it failed with a fatal error: fatal: Not a git repository (or any of the parent directories): .git

    I then cd-ed into memcached, and ran it again. It errored out differently, saying something to the effect that that branch didn't exist.

    I went to the github page, and found that there was an engine-pu branch. I then did $git checkout -t origin/engine-pu, and it seems to have worked. :)

    Might think of updating the TUT to have a cd memcached, and the new branch name. :)

    Cheers!

    ReplyDelete
  15. Hi.

    I've noticed that memcached project now has a win32 directory in their project.

    https://github.com/memcached/memcached/tree/engine-pu

    To clone it:
    git clone --branch engine-pu https://github.com/memcached/memcached.git

    Thanks for the tut. I've managed to build memcached with msysgit :)

    ReplyDelete
  16. when I use this command: make -f win32/Makefile.mingw ,something wrong.
    make win32/Makefile.mingw: No such file or directory
    make *** No rules to make target win32/Makefile.mingw . stop.
    I install msysgit in C:\msysgit\msysgit. The Tmp path is F:\tmp.

    please help to deal with it, tks.
    by the way , I am a chinese ,so my English is not well.

    ReplyDelete
  17. How to increase the max item size from 1 MB. I guess all I need to do it to update the POWER_SMALLEST (in default_engine.h file) to 2 (it is set to 1 now) if I want to to increase it to 2 MB. Right?

    ReplyDelete
  18. memcached supports the -I (capital i) to specify the max size of the object

    ReplyDelete
  19. Hi Trond,

    Any progress on compiling new memcached versions on Windows?

    Thanks
    Zab

    ReplyDelete
  20. Hello.
    Sorry my bad English.
    Trond Norbye, I try to build memcached from MB-6508 branch on msysgit 1.7.11 and I got error
    http://pastebin.com/fCuzE80v
    What is the dependency tcmalloc_minimal.dll and where I can get this dll?

    ReplyDelete
    Replies
    1. Hello,
      I managed to build some libtcmalloc_minimal.dll, but I still do not know where to put it in order to build the memcached. Shall it be registered somewhere?

      Delete
  21. Did anyone manage to build memcached using the latest binaries v1.4.15?

    ReplyDelete
  22. Hi Trond, first of all thanks for the tutorial!

    I am trying to build memcached now and get some problems linking to libevent.

    I get your git version of Memcached and use the win32/Makefile.mingw replacing some of the -lxxx with -Lpathtolibrary as teh whole path on my env is quite a mess.

    I downloaded the latest libevent code too and build it using nmake, which produced a bunch of .lib files. I copied the libevent.lib file and use it for the linking with -L on the memcached.exe recipe and copied the header files to the mingw include folder so they are found.

    Sadly when I try to build I get a lot of "undefined reference to event_*".

    I think the problem is that I am linking wrongly but I am not sure as I am not familiar with gcc and c any more, Im a .Net guy this days.

    If you can bring some light you will be more than appreciated. I found the following thread (https://groups.google.com/forum/#!topic/memcached/P4qMK4upyMk) where you answer some questions but the original problem wasn't hte same as mine.

    Thanks in advance!

    ReplyDelete
    Replies
    1. I've stopped using mingw on windows and I'm using MSVC instead.

      Delete
    2. Oh :(, thanks anyway!

      Any chance you know of a similar tutorial to build using MSVC? :)

      Delete
    3. Sure, when I'm done.. I'm currently working on making the compilation process of Couchbase better on windows. (please note that you may run Couchbase on Windows, and it does support memcached buckets...)

      Delete
    4. Thanks Trond.

      I have found this https://github.com/robertmeta/win32-memcache which is an older version of memcached (1.2.6) but that compiles out of the box, in case its help for anyone.

      Delete