magpiebrain

Sam Newman's site, a Consultant at ThoughtWorks

Archive for ‘October, 2010’

Advertisements

I’ve been playing around with Vagrant (which just uses VirtualBox under the hood), with a view to making it easy for developers to bring up production-like systems on their local workstations. I fashioned a simple Vagrantfile cribbed from the examples on the web. I was specifically interested as to how fast it was to bring up VMs from scratch to understand how viable it was to incorporate this as part of a normal development workflow.

This system consists of 32-bit lucid boxes, each with default settings. All tests were done on my aging MacBook Pro, with 4GB of memory. All times captured using time. This test represents bringing up between one and four VMs from scratch, and applying a simple Chef recipe to install Apache (you can see the Vagrant file I used here). Note that simply suspending and restarting a VM is much faster – these times represent the worst case of firing these VMs up from scratch.

  • 1 VM – 2m15.543s
  • 2 VMs – 5m24.306s
  • 3 VMs – 10m53.824s
  • 4 VMs – 12m19.211s

I didn’t do enough sample points to predict any obvious trends, other than more VMs = longer startup time. This also discounts any startup time your application may have too. Nonetheless, bringing up a single VM from scratch appears to be fast enough to be usable, and even when dealing with more VMs to represent more complex topologies (useful for local DR testing for example) also seems viable.

I’m less sure as to how useful this is when provisioning environments to run functional tests – adding a couple of minutes to a build that you want to keep somewhere south of ten minutes in total is huge, but keeping a pool of vagrant clean VMs already provisioned could work around this issue (as could using better hardware). The idea of being able to bring up clean environments using the same production config and tear them down again afterwards as part of a CI build is compelling, but I need to give it more thought.

From http://www.flickr.com/photos/bigduke6/258262809/I’ve been playing around with both Chef and Vagrant recently, getting my head around the current state of the art regarding configuration management. A rather good demo of Chef at the recent DevOpsDays Hamburg by John Willis pushed me towards Chef over Puppet, but I’m still way to early in my experimentation to know if that is the right choice.

I may speak more later about my experiences with Vagrant, but this post is primarily concerning Chef, and specifically thoughts regarding repeatability.

Repeatability

Most of us I hope, check our code in. Some of us even have continuous integration, and perhaps even a fully fledged deployment pipeline which creates packages representing our code which have been validated to be production ready. By checking in our code, we hope to bring about a situation whereby we can recreate a build of our software at a previous point in time.

Typically however, deploying these systems requires a bit more than simply running apt-get install or something similar. Machines need to be provisioned and dependencies configured, and this is where Chef and Puppet come in. Both systems allow you to write code that specifies the state you expect your nodes to be in to allow your systems to work. To my mind, it is important therefore that the version of the configuration code needs to be in sync with your application version. Otherwise, when you deploy your software, you may find that the systems are not configured how you would like.

Isn’t It All About Checking In?

So, if we rely on checking our application code in to be able to reproduce a build, why not check our configuration code into the same place? On the face of it, this makes sense. The challenge here – at least as I understand the capabilities of Chef, is that much of the power of Chef comes from using Chef Server, which doesn’t play nicely with this model.

Chef Server is a server which tells nodes what they are expected to be. It is the system that gathers information about your configured systems allowing discovering via mechanisms like Knife, and also how you push configuration out to multiple machines. Whilst Chef Server itself is backed by version control, there doesn’t seem to be an obvious way for an application node to say “I need version 123 of the Web Server recipe”. That means, that if I want to bring up an old version of a Web node, it could reach out and end up getting a much newer version of a recipe, thereby not correctly recreating the previous state.

Now, using Chef Solo, I could check out my code and system configuration together as a piece, then load that on to the nodes I want, but I loose a lot by not being able to do discovery using Knife and similar tools, and I loose the tracking etc.

Perhaps there is another way…

Chef does have a concept of environments. With an environment, you are able to specify that a node associated with a specific environment should use a specific version of a recipe, for example:

name "dev"
description "The development environment"
cookbook_versions  "couchdb" => "11.0.0"
attributes "apache2" => { "listen_ports" => [ "80", "443" ] }

The problem here is that I think the concept of being able to access versions of my cookbooks is completely orthogonal to environments. Let’s remember the key goal – I want to be able to reproduce a running system based on a specific version of code, and identify the right version of the configuration (recipes) to apply for that version of the code. Am I missing something?