Automated Configuration Management With Opscode Chef: The Basic Moving Parts

by on June 8, 2010 · 6 comments

The Moving Parts

Managing your infrastructure with Opscode Chef involves a few moving parts you need to be aware of. As I found it quite hard to differentiate, I want to share the basics with you:

  • Chef server There you manage all your nodes and roles. The chef server distributes cookbooks to the nodes.
  • Chef clients (Nodes) Every client will be registered as a node at the chef server. It will receive cookbooks to run from the chef server.
  • Your workstation You will write cookbooks and role definitions on your local box. From there you will upload your cookbooks to the chef server, create roles at the server or manage the chef server using its command line tool knife.

The Opscode Chef Server

The Chef server is your central place where you find all the machines (nodes) to be managed by chef. In the chef server, you find a mapping of nodes to the roles they play. And, you upload all your cookbooks (definitions how to setup a server with a certain role) to the chef server, which distributes it to the nodes to run there.

You have two ways of getting a chef server: Either choose one of your machines to become the chef server and set it up accordingly. Or use the Opscode chef platform (in private alpha as of writing of this article) as your chef server. The advantage of the chef platform is that you have one moving part less to care about.

The Opscode Chef Clients

Every server you want to configure via chef needs to have the chef client software, be configured so it knows your chef server, and be registered at the chef server. The chef server will manage it as a node. You will then assign roles to the node. The roles define what will be installed on the node. E.g. a role called “rails_app” could define that you want to install “passenger_enterprise::apache2″ and “rails_enterprise” on every node with that role. Here’s an example role definition file:

name 'rails_app'
description 'The base role for systems that run Rails applications'
run_list "recipe[passenger_enterprise::apache2]", "recipe[rails_enterprise]"

Your Local Workstation

At your local workstation, you install and configure the knife command line tool to interact with the chef server. Additionally, you create all your cookbooks and role definitions on your local box and upload them to the chef server. To keep track of all your changes to your cookbooks (and to share them with your co-workers) it makes sense to store them in a version control system. It’s important to note that the chef server, not the chef clients, will pull it from there. You have to upload your changes from your local workstation to the chef server using the chef rake task rake upload_cookbooks. That is conceptually different e.g. from how capistrano deploys your application source code to the servers.

Mixing It All Together

After writing such a role file on your local workstation, you have to upload it to the chef server. There is a rake task for that. Just run rake roles within the directory where you manage your cookbooks (alternatively you could use knife to create the role, too). You can verify that the role is available on the chef server:

workstation $ knife role show rails_app
{
    "name": "rails_app",
    "default_attributes": {
    },
    "json_class": "Chef::Role",
    "run_list": [
      "recipe[passenger_enterprise::apache2]",
      "recipe[rails_enterprise]"
    ],
    "description": "The base role for systems that run Rails applications",
    "chef_type": "role",
    "override_attributes": {
    }
  }

If you already have a client called “app.example.com” registered as a node on the chef server, you can assign the “rails_app” role to it by running knife from your local workstation:

workstation $ knife node run_list add app.example.com "role[rails_app]"
{
  "run_list": [
    "role[rails_app]"
  ]
}

If you run the chef client app.example.com, the client will see that it has to apply the “rails_app” role and will retrieve and run all required cookbooks installing apache2, passenger, REE, rails, etc.

Have you ever used Opscode Chef? What are your experiences? Please let us know in the comments.

Did you enjoy this article? Get new articles for free by email:


Comments

  1. says

    Thanks for the post, Matthias.

    A few other nifty things you can do. If you are storing your cookbooks in git, you can automatically download cookbooks from the http://cookbooks.opscode.com site and track them over time with:

    $ knife cookbook site vendor ubuntu -d

    Where ‘ubuntu’ is the cookbook name. This will automatically create a vendor branch for you, and if you need to make local customizations, you can do them directly in-line. When new versions of the cookbook are released, just run the command again, and fix any merge conflicts.

    Let us know if we can answer any questions. :)

  2. Steve Conover says

    We’ve been very happy with Chef too. Notably, we use chef-solo, which we shell out to at deploy time using capistrano. This way we upgrade our code, data, and system config in concert.

    I have to say that I’ve never seen a use case in typical webapp development that I felt demanded the client-server model.

    -Steve

  3. Daniel Weaver says

    To expand on Steve’s comment, chef server mostly provides an interface for managing host specific configuration variables. This is a neat idea, but what we found is that (a) the interface for editing those variables is confusing/bad; (b) it introduces a new point of failure – spending time making our chef-server highly available is not development time well spent; and (c) it introduces a new place that isn’t really secure but is very tempting to store passwords.

  4. Steve Conover says

    “it introduces a new point of failure – spending time making our chef-server highly available is not development time well spent”

    This to me is the most damning part. Introducing a HA server into the world when running a script locally is just fine is just…strange, to say the least.

  5. says

    The other day we launched our new Chef Server as a service, where you can sign up to use the Chef Server hosted by Opscode for free for up to 5 nodes*. With that, you can have the benefits of a Chef Server (like stored node data and search indexes), without the additional system.

    And if you want to run a Chef Server, we (or our awesome community) will help you set that up as well through the mailing list (lists.opscode.com) or IRC (irc.freenode.net, #chef).

    * Actually for free during our beta period which is over the next ~60 days. More at http://www.opscode.com

Trackbacks

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>