Monday, January 9, 2012

Thank you Joyent, I love you!

I’ve always been a strong believer of that people should be able to choose the platform they feel suits their needs the most. If people want to make stupid decisions and not choose my beloved Solaris, I’m not going to stop them. This means that I need to ensure that the software I’m working on not only compiles, but also works on multiple platforms. In “the old days” I used to install a new os on my desktop box every time I upgraded my desktop (which always ran the latest bits of Solaris), but lately I’ve been using virtual machines to make it easier for myself ;)

You may wonder why I don’t just set up virtual machines in the cloud? The answer is pretty simple. I’m working from home, and I’ve had my share of problems with my ISP. I don’t want to end up in a situation where I can’t do my work just because my ISP fails to keep me connected with the rest of the world...
I guess it must be roughly a year ago since I decided to replace all of my VirtualBox instances running on a handful of old machines with KVM on top of Debian on my server. With the server running there I moved more and more stuff off my desktop box, and I ended up in a situation I really didn’t like. I had the services I “needed” running on top of a server without mirrored disks. To save money I had ordered that dell server with just a single disk. I’ve had enough disks dying on me over the years, so to me this feels like hiking in the middle of the freeway at midnight.. To get out of this situation I created an iSCSI share on my OpenIndiana box on top of a ZFS mirror that I connected to the Debian box. Now I could sleep a little bit better at night...

I had noticed the releases of SmartOS, and it looked really cool! Unfortunately for me I didn’t have a machine I could try it on, but luckily for me Debian gave me a helping hand in December to speed up the process! After upgrading packages with aptitude my box would no longer boot! I don’t have a keyboard/monitor attached to the box, so I had to bring the server into my office to “debug” the issue. I didn’t figure out why it wasn’t booting with the new kernel, but I was able to boot it with the old kernel and get the stuff I needed off the box. I could have spent more time trying to figure out why it was failing, but instead I took this as a golden opportunity to try out SmartOS. Doing so brought nothing but joy into my life!!!!!

After booting off the USB stick I had to answer a couple of questions to configure the system for the first time, and I was ready to create my first machine. I followed the instructions in "How to create a Virtual Machine in SmartOS" and I had my first VM up’n running in less than a minute! So simple, and yet so powerful!

Roughly a week later my box looks like:
[root@00-26-b9-85-bd-92 ~]# vmadm list
UUID                                  TYPE  RAM      STATE             ALIAS     
80658f56-0eb3-405f-a6eb-690461c2d9ce  OS    256      running           -       
9c441211-bb77-446f-abec-2291039aeca2  OS    256      running           smartos64 
b0e34876-cd7a-4922-ad6e-921452d34359  OS    512      running           jenkins   
13f4223b-a2c1-400a-b682-79372c3ba846  KVM   1024     running           solaris11 
7bad78f1-d202-4dfe-97f6-e421e8da8d58  KVM   1024     running           ubuntu64  
8e042001-bc79-48eb-a0bc-704ca64f20e0  KVM   1024     running           debian    
e1710b31-0270-43a3-89dc-71398ba3630a  KVM   1024     running           windows  
f165261a-6a27-4ee3-a6d3-f3814cf69bd6  KVM   1024     running           ubuntu32  

My "jenkins" vm is just running the Jenkins CI web application for, and it connects via ssh into the other VMs (and a couple of other machines) to build software there. This is part of my automatic build process. I've got it on my ever growing todo list to set up NIS/LDAP, but in the mean time I'm just sync'ing the user definitions around so that I can log into all of the machines.

But wait, I said I didn’t like to keep stuff on filesystems that isn’t mirrored, and choosing another OS doesn't change this? Right now I don’t care if the disk dies, because I can easily recreate all of the vm’s from scratch (i’ve got the descriptions I used to create the vm’s stored somewhere else). All “users” on the vm’s mount their home directory from my NFS server, so none of those files would get lost. I guess I could use zfs send/receive to back up the vm itself while I’m waiting for another disk for the box. 

So is there anything I miss from the current release, or is it perfect? There is one thing I really miss, and that is the ability to use the alias instead of the uuid when I’m using vmadm. I know that the alias doesn’t have to be unique, but if the alias is unique it would be a lot easier to use (instead of having to do a vmadm list first).

The second thing I’d love to see would be something like:

# vmadm bootinstall name iso-file

it would expand into something like:

cp iso-file uuid/root/cdrom.iso
vmadm boot uuid order=cd,once=d cdrom=/cdrom.iso,ide
vmadm info uuid vnc

But hey, if this is the biggest problems I’ve got with SmartOS I must be pretty happy with it. After all how often do you really create and install new VMs? The fact that I may use the tools I know and love (dtrace, zfs, smf etc) is just awesome!

Thank you Joyent for bringing this to my fingertips! 


  1. Trond, glad to hear you're enjoying vmadm and SmartOS! As for using aliases instead of UUIDs, the problem with this in the current system is that aliases are not guaranteed to be unique. When systems have multiple customers, it's very possible for there to be say 2 zones/VMs named 'mysql'. That said, the way I'd do this currently is something like: 'vmadm [action] $(vmadm lookup -1 alias=[alias])'. You can also do lookup that way for any of the parameters lookup supports (the man page for vmadm lists the listable options). I'll give some more thought to the alias problem and see if I can think up a solution that will work well for setups with and without multitenancy.

  2. Yeah, so what I was hoping for was that if I did:

    vmadm start mysql

    It would start the vm named mysql if there was only one named mysql, otherwise it would print out something like:

    Error: the name mysql refers to multiple instances.

    As I said I'm pretty damn lucky that this is the only problem I've got, because it's pretty minor ;)

  3. I know this is over a year old, but I put this together based the suggestion by Josh. Works well for my use case of a single customer (myself).