The next step to get a better grip on your environment is figuring out exactly what kind of production configurations you have running out there. If you’ve ever caught yourself walking through the data center and wondering just what in the hell those servers in the back corner are for, this phase will be quite the eye-opener. Now that you have changes to this environment under better control you can start to invest some time in assessing what you actually have.
Archives for September 2008
As recommended by Ezra Zygmuntowicz, I’ve divided all layers of our web application into separate virtual machines using Xen. At a first glance, having virtual machines for every service sounds like quite some overhead. Isn’t it much simpler to just install the whole stack on one box and let it run? Why take the hassle of installing and maintaining an additional virtualization layer? Shouldn’t we keep everything simple and straightforward? After running this setup for nearly a year and scaling it from hundreds of thousands of clicks per month to a couple of million, I want to share with you the ways in which virtualization really pays off for us.
I had to port the client part of a PHP based client-server program, which received some XML data along with compressed images as binary data. As it cost me some time to inflate the received data in Ruby, I want to share what I found out about deflating and inflating data in Ruby and PHP.
As you can easily find out in the comments of the PHP gzdeflate manual entry, gzdeflate is using raw deflate encoding. Another important thing to note is that “PHP deflate” is not the same as “HTTP deflate” as the HTTP uses the zlib format along with the deflate compression.
Before we come to the solution of the original problem of inflating data, we take a short look at deflating data in Ruby and PHP.
Deflate Data in Ruby for PHP Destination
If you want to compress data exactly the same way like PHP does (e.g. to be able to send it to a PHP app and inflate it there again), you can use the solution described by eden li:
def gzdeflate (s) Zlib::Deflate.new(nil, -Zlib::MAX_WBITS).deflate(s, Zlib::FINISH) end
Note the necessary (but nonintuitive) usage of -Zlib::MAX_WBITS which avoids generating some headers not used by PHP.
Inflate Data in Ruby From PHP Source
But what to do if you have data which was deflated by a PHP app? In PHP you would simply write:
<?php $re = file_put_contents('5.jpg', gzinflate(base64_decode(encoded_and_compressed_image))); ?>
With this piece of Ruby code you are able to extract binary data, which was compressed by a PHP application:
require 'base64' require 'zlib' open("5.jpg", "wb") do |file| file.write( Zlib::Inflate.new(-Zlib::MAX_WBITS).inflate( Base64.decode64(encoded_and_compressed_image) ) ) end
First, you have to decode the data using the Base64 class. Then pass the encoded data to the inflate method of Zlib::Inflate.
Again, you have to use the -Zlib::MAX_WBITS parameter when constructing the Inflate object.
The -Zlib::MAX_WBITS is necessary because PHP uses a raw deflate algorithm in the gzdeflate method but ruby defaults to either gzip or zlib (but NOT the raw deflate format).
We’re currently using Capistrano not only to deploy our Ruby on Rails application, but also to setup and manage our physical and virtual (Xen based) servers. We have Capistrano recipes for adding users, installing packages like apache or mysql, configuring a Xen VM and more. Coming accross puppet, I started to wonder about the essential difference between the two. Puppet claims to enable the user to automate server management to a large extent, a goal which we already reached by implementing our custom Capistrano recipes. So, what are the differences between the two?
Fast forward to modern times. It’s no coincidence that popular sports teams typically number between five and twelve people. Could you imagine a football team with 25 players per side? The communication and coordination of such a large group would be much to inefficient for such fast paced game play. To fans in the stadium, their reactions would seem slow and imprecise and probably not very entertaining. Yet this is precisely how many companies have organized themselves in the last century.
As you might have already guessed, I’m constantly striving for the simplest yet most optimal process for running both an agile development team and agile web operations. People come first, then the procedures followed by the people and finally, the tools those people use.
Since tools supporting the agile development process have the lowest priority for me, they need to do their job well and get out of my way. It means they have to be as simple yet as powerful as I require. I soon realized this is not an easy requirement to fulfill.
Frustrated with the current tools for agile project management, I did a spike to find out whether it would be feasible to build my own tool and discovered some promising ideas. With my current workload, I’ve lowered the priority of building my dream tool, but I want to share my initial spike findings with you using the lighthouse API with ActiveResource. Simple and basic, it shows how easy it is to get started using a good API with a nice ruby wrapper. The example below didn’t take more than half an hour to build. Not too bad – especially as its already able to manage projects, users, and tickets (a.k.a Stories) including history and tags.
Get started with your basic ruby on rails app (using rails 2.1)
mysqladmin -u root create lightboard_development rails lightboard
Install the Lighthouse API Ruby wrapper as a plugin
script/plugin install http://ar-code.svn.engineyard.com/lighthouse-api/
Ouch, The Web Application Does Not Start
To avoid a problem like this:
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.1.0/lib/active_support/dependencies.rb:115:in `qualified_const_defined?': "Plugins::Lighthouse-api::Lib" is not a valid constant name! (NameError)
you have to rename the plugin from
lighthouse_api (mind the dash “-” vs. the underscore “_”)
mv vendor/plugins/lighthouse-api vendor/plugins/lighthouse_api
And as ActiveResource is now part of rails, you can get rid of the superfluous vendor directory
Configure Access to Your Lighthouse Account
Of course, you need an account at www.lighthouseapp.com first 😉
I put my account and token config into a custom inizalizer
Lighthouse.account = 'myaccountname' Lighthouse.token = 'myverylong-lighthouse-token'
Using The API is Trivial
script/generate controller story_board index
Then, your story_board_controller could use the API like so:
class StoryBoardController < ApplicationController include Lighthouse def index @projects = Project.find(:all) end end
Now you can show all the projects of your account in the view:
<%= render :partial => "project", :collection => @projects %>
<%= project.name %>
UI Would be The Most Complex Part to Build
Using the API as a backend, I get a lot of functionality for free. What I miss from the API is the support for managing attachments, but that's not the most important feature. The main efforts would lay in building the UI. Let's see when the pain with other tools becomes intolerable enough to finally tackle building my own 😉