Archive for the ‘Development’ Category

Self Identifying Software

22 September 2009 2 comments

How often has someone come up to you and asked you what build of your software is currently deployed in a specific environment?

How many times have you come across a .jar or .dll file and wondered what version it is? Especially when using Open Source Software?

The most frightening one for me is when I’ve looked at a cluster of production servers and noticed that the .war file for the application deployed on it is a different size on one of the nodes. Which one was the correct one to deploy? Luckily this happened to me a long time ago, but I know that people out there are still having this problem today.

The solution is what I call “Self Identifying Software”. Every build of your software needs to have something that tells you what version it is and how to get back to the source code that created it. Having a build label or release number visible in your application is a good start, but it does not make your software Self Identifying. Product companies have been doing this for ever. The problem is that for that number to be useful (particularly when you’re trying to access the source code to reproduce and fix a bug) you then need to refer to a build system or release notes to find out where the source code came from (if you’re lucky). It often also does not apply to development builds. To be truly Self Identifying you need to make sure that every build (including builds developers create on their workstations) also includes enough information from the SCM system so that anyone who has access to the source code can go right back to the exact source code that produced that binary. For example, if you use Subversion as your SCM then this will be a URL and a revision number.

This is not exactly a new concept, it’s something I (and others) have been doing for a number of years now. The reason I’ve decided to write about it now though is that recently I was showing a new guy around one of the projects I’m working on at the moment, and when I showed him how to determine which version of the app was deployed he was delighted.
Read more…

Podcast on Continuous Integration available

27 April 2009 Leave a comment

Last year at JAOO I had the chance to speak to Markus from Software Engineering Radio about the talk I gave there on Continuous Integration. It’s finally available now over here. The slides that go along with the talk are available from the JAOO site.

Introducing ESCAPE

13 February 2009 3 comments

Whenever I talk to new clients about Continuous Integration, and especially about using Build Pipelines to extend their CI process to cover testing things like the deployment of their application, there is always one question that is guaranteed to pop up – how do I manage the configuration of my application in all these environments?

This is always a hard one to answer. In the last place I worked as a System Administrator we used Split-Horizon DNS as one of the ways to solve connecting to the correct host for a service. Our applications had hard coded hostnames for key services such as db.internal.domain, loghost.internal.domain and mailhost.internal.domain. Depending on which environment they ran in, when the did a DNS lookup they would get an answer that pointed them to the correct server providing that service for the environment they were physically running in. Combine this with tools like an LDAP Server in each of these environments containing the rest of the system configuration and life is easy. Or is it?

DNS and LDAP servers are relatively straight forward for experienced Sys Admins to understand and maintain, but they still can’t be described as easy to use, which is probably why so few people do it. This limits our options for fixing things down to two general categories – fix DNS and LDAP admin tools so they are easy to use, or find another way to solve the problem.

We chose the path of least resistance and went for the second option – creating ESCAPE. In the Unix tradition, it’s an app that takes one problem and solves it simply and well (at least in our opinion). It provides a RESTful interface for both the setting and getting of environment configuration. GET requests will retrieve configuration, whereas POST/PUT requests will create new entries or update them.

The URL scheme we’ve decided on is most easily demonstrated by constructing an example URL:

URL Value returned in the body
http://escape/ User interface. No API available here.
http://escape/environments/ A JSON list of all the available environment.
http://escape/environments/production/ A JSON list of all the applications in the “production” environment.
http://escape/environments/production/mywebapp All keys and values for the “mywebapp” application in the “production” environment.e.g:
http://escape/environments/production/mywebapp/thiskey The value of “thiskey” for the “mywebapp” application in the “production” environment.

As the tool is now usable we’ve happily released version 0.1. Currently we’re busy putting a lot of work into making the management interface understandable and intuitive to use. The closest thing we currently have to a roadmap is our ToDo list.

Please join us on the discussion group with your feedback.

Retrospectively Breaching the Wall between Developers and Operations

26 November 2008 2 comments

I like to describe my job at ThoughtWorks as helping Developers and Operations realize that they’re playing on the same team. No matter how awesome your code is, how elegantly you’ve solved the problem at hand, how nice and readable the code is – if you can’t get it into production your software is just a collection of bits. Likewise, you can have the best network, the most scalable hardware, the neatest cable patching scheme – but it’s just a big fancy heater if it’s not running the code your business needs.

As I’m normally brought in with the developers, I’ve been trying to find efficient ways to engage with the client Operations teams. Normally I end up having one on one conversations with various members of the team, try to find out what their current processes are, what their major challenges are and what their concerns are regarding the project I’m involved in. I usually do this to keep the safety levels high. The problem though is that it takes quite a lot of time and effort to get things going and get some momentum going.

At my current client though I didn’t have the time or access to the people to do things the normal way. A meeting was arranged with the key Operations stake holders and I effectively had just 2 hours to explain our development process in general, and Continuous Integration and Build Pipelines in detail. While talking with Graham Brooks about what we wanted to cover, he came up with the idea of running it as a mini retrospective.

After the usual introductions, we gave them 15 minutes to list the Good, the Bad and the Puzzles of their current development and release process. We had good participation from the group and as expected had a high number Bad entries. After talking through the cards and grouping them into related sections, we then allowed them to vote on the ones they most wanted to talk about. Most votes went to the core pain points, and we spent the rest of the time talking about how our process would address those issues. It also helped a lot that most of the Good entries related to the automation they already have in place…

By the end of the meeting no was talking about the bad old days (lobbing releases over the wall). Everyone was engaged starting to get some spirit of collective ownership going in the whole delivery process and that breaking down the walls that exist between the various silos was high on the list of things to do. Rather than talk to them about our process and how we would like to interact with them, we had allowed them to lead the discussion on which elements from our toolbox would have the greatest value for them.

All I need to do now is learn how to be as good a facilitator as Graham was…

Categories: Build, Development, Software

Continuous Integration Server Comparison 2008

22 September 2008 2 comments

Early last year I did a Quick Comparison of some of the popular CI servers of the time. Things have moved on since then, and I’ve actually been involved with the Cruise development team since then. Now that Cruise has been released, a number of people both inside and outside ThoughtWorks have asked me to put together a follow up article – here it is.

The list of available products out there has grown a lot in the past 18 months, and the features that they support are really great. Since I did the last review I’ve actively avoided having a look at the other tools out there to keep a clear focus on what I wanted to see in Cruise. Doing this review has been a great way for me to see what everyone else has been up to.

Just having loads of features does not automatically make for a good tool though. Instead of having a shooting match between who does what, I’ve taken a little sample Java servlet that I use for demos and tried to get it working with all the tools. This project is hosted on a local subversion repository. I’m going to try set it up to simply run unit tests and create my distributable .war file. Areas that I’m going to look at are:

  • Installation (on Linux, OSX and Windows)
  • Setting up my existing project
  • What did the tool inspire me to try next

Tools that I’ll be trying out are (in alphabetical order):

Read more…

Categories: Build, Development, Software

Speaking at JAOO 2008

12 September 2008 Leave a comment

I’m busy adding the finishing touches to my talk for JAOO this year and I’ve just realised that I’ve not let you guys know that I’m going to be there. Consider yourself warned.

Last year at the conference I got a chance to show off the new UI we did for CruiseControl with Erik Dörnenburg. Afterwards I was having a chat with Martin Fowler, and I commented on how I’d like to see a whole track at JAOO dedicated to Building and Deploying software. While I’m sure I can’t take all the credit for it, this year there is going to be a short Build track with me talking about Continous Integration. Hope to see you there…

Categories: Build, Development, Software

Ant can be less verbose than Java

5 August 2008 Leave a comment

I had one of those rare opportunities today when I got to pair with one of the developers on our project. The story he’s working on has to do with generating usage documentation for a webservice at runtime. What he wanted was a list of source files to be available at runtime. The solution he came up with (which does work) was the following Java program:

package com.example.service.utils;

import java.util.Arrays;

public class ListControllers {
    public static void main(String[] args) throws IOException {
        String outputFile = args[0];
        String srcDir = args[1];
        File f = new File(srcDir);
        File serverDir = new File(f, "com/example/service/server");
        String[] entries = serverDir.list();
        BufferedWriter writer = new BufferedWriter(new FileWriter(outputFile, false));
        for(String entry : entries) {
            if(entry.endsWith("")) {
                writer.write(name + "\n");

This was then called by the following Ant snippet:

<mkdir dir="${target.dir}/war/WEB-INF/classes"/>
<java classname="com.example.service.utils.ListControllers" 
      classpath="${main.jar}" failonerror="true">
    <arg value="${target.dir}/war/WEB-INF/classes/controller.list"/>
    <arg value="${src.dir}"/>

This was then bundled up in the war file so that he could get his mits on it at runtime. It works, but is not without problems. My primary complaint is that it makes the build harder to understand. To figure out what’s going you need to find and open up the .java file, and be able to understand Java in the first place.

After a few minutes of pairing we managed to fix it with the following Ant snippet:

<mkdir dir="${target.dir}/war/WEB-INF/classes"/>
<pathconvert property="controller.list.propery" 
            <fileset dir="${src.dir}/com/example/service/server" 
    <map from="${src.dir}/com/example/service/server/" to=""/>
<echo message="${controller.list.propery}" 

Not only is it now easy to see what we’re trying to do in the build file, but we managed to replace 29 lines of Java + Ant with 10 lines of Ant. Sweet!

Categories: Build, Development, Software

CruiseControl and Buildix 2 at JAOO 2007

26 September 2007 Leave a comment

I’ve been at JAOO for the past few days, and while here I had a chance to do a presentation with Erik Doernenburg on Continuous Integration and CruiseControl. We used the new Beta version of Buildix 2 to show people the new CruiseControl Dashboard, and quite a few people were impressed with it. Favourite features were the CCTray integration, and the ability to see the status of a large number of projects at a glance.

As always, there were also people who were interested in hearing about how it can be used for non Java projects. I had a good chat to one person who is interested in using it on a mixed Common LISP and Erlang project he’s working on. I’m looking forward to hearing how it goes for him. Due to lack of experience on my part I could unfortunately not help him much with the darcs problems he’s having though. Some people have all the fun…

It was also quite useful to speak to people about the problems they’re currently facing when trying to use CruiseControl. A common theme is people trying to manage large numbers of builds, or trying to build products across large numbers of different platforms. These are problems the dedicated ThoughtWorks development team are currently working on, so it’s great to get the validation that we’re putting effort into the things people care about now.

Categories: Development, Linux, Software, Unix

Using CruiseControl for non-Java Projects

A lot of people are under the impression that CruiseControl is solely intended for Java projects. This is a common misconception. At my current client we’re using CruiseControl to build a 15 year old, 2.5 million line mixed C/C++ app using GNU make. We’re also using it to build and functionally test a new C app we’re writing as part of the project. I’ve also used it in the past on Python apps.

There are a number of ways to do this, and they all really depend on what you’re using Cruise for. If you’re simply using it for compilation, then you can simply use <exec> to call make (or whatever you’re building with). By default it will break the build if the command returns a non-zero return code, so it should be smart enough to do this out of the box. I know that some of our Ruby projects that use this method too.

My personal preference is to get CruiseControl to do an Ant build, which has some additional logic, including taging the source repository with the build label if the build passes. The approach is very similar in this case, and has the following steps:

  • Have ant do any additional work/checks you want to ensure the environment is how you want it.
  • Update my working copy to an exact known version that can be tagged at a later stage if the build is good (primarily a CVS problem, not really needed if you’re using Subversion).
  • Use the <exec> task from ant to run the build
  • If you have additional tests to run, use an <exec> within a <parallel> block to start your newly built binary in the background, then run the tests
  • If everything is good and passes, tag your source code with the build label.

So, although it is fairly easy to add build plugins for various other projects, you can save yourself a bit of pain simply using the one <exec> or shelling out of an <ant> builder.

Categories: Development, Software

Quick Comparison of TeamCity 1.2, Bamboo 1.0 and CruiseControl 2.6

21 February 2007 1 comment

One of the things I spend a lot of my life doing is looking after Continuous Integration environments. As I work for ThoughtWorks, and am one of the guys who created Buildix, it should come as no surprise that this is all done with CruiseControl. However, in the past few months there a few new Continuous Integration tools that have crept out onto the scene. The two that I hear the most about are TeamCity and Bamboo. I finally managed to get a few hours to test them out, and here’s my impressions.

One of the things I wanted to see was how quick and easy it was to get these tools up and running. With this in mind, I specifically decided in advance to pull down the .war bundle of each one and try that out. In my experience, this would be the one that most of the clients I’ve been at would have gone for.


  • This is very much a first impressions run of the products – I had a limited time window (2 hours total) to do this in.
  • People could quite fairly say that I have a biased view, but I’ve tried to keep this as fair as possible.
  • I only tested doing an Ant build of a simple sample webapp from a subversion repository.
  • You have to buy a license for TeamCity and Bamboo (I got evaluation licenses for both), but CruiseControl is still free…

Test Environment:

  • Fedora Core 4
  • Via C3 800 server with 512Mb RAM
  • Sun Java 1.6.0-b105
  • Apache Ant 1.7.0
  • Jetty 6.1.1
  • Subversion 1.2.3

Read more…

Categories: Development, Software