I like to describe my job at ThoughtWorks as helping Developers and Operations realize that they’re playing on the same team. No matter how awesome your code is, how elegantly you’ve solved the problem at hand, how nice and readable the code is - if you can’t get it into production your software is just a collection of bits. Likewise, you can have the best network, the most scalable hardware, the neatest cable patching scheme - but it’s just a big fancy heater if it’s not running the code your business needs.

As I’m normally brought in with the developers, I’ve been trying to find efficient ways to engage with the client Operations teams. Normally I end up having one on one conversations with various members of the team, try to find out what their current processes are, what their major challenges are and what their concerns are regarding the project I’m involved in. I usually do this to keep the safety levels high. The problem though is that it takes quite a lot of time and effort to get things going and get some momentum going.

At my current client though I didn’t have the time or access to the people to do things the normal way. A meeting was arranged with the key Operations stake holders and I effectively had just 2 hours to explain our development process in general, and Continuous Integration and Build Pipelines in detail. While talking with Graham Brooks about what we wanted to cover, he came up with the idea of running it as a mini retrospective.

After the usual introductions, we gave them 15 minutes to list the Good, the Bad and the Puzzles of their current development and release process. We had good participation from the group and as expected had a high number Bad entries. After talking through the cards and grouping them into related sections, we then allowed them to vote on the ones they most wanted to talk about. Most votes went to the core pain points, and we spent the rest of the time talking about how our process would address those issues. It also helped a lot that most of the Good entries related to the automation they already have in place…

By the end of the meeting no was talking about the bad old days (lobbing releases over the wall). Everyone was engaged starting to get some spirit of collective ownership going in the whole delivery process and that breaking down the walls that exist between the various silos was high on the list of things to do. Rather than talk to them about our process and how we would like to interact with them, we had allowed them to lead the discussion on which elements from our toolbox would have the greatest value for them.

All I need to do now is learn how to be as good a facilitator as Graham was…

Early last year I did a Quick Comparison of some of the popular CI servers of the time. Things have moved on since then, and I’ve actually been involved with the Cruise development team since then. Now that Cruise has been released, a number of people both inside and outside ThoughtWorks have asked me to put together a follow up article – here it is.

The list of available products out there has grown a lot in the past 18 months, and the features that they support are really great. Since I did the last review I’ve actively avoided having a look at the other tools out there to keep a clear focus on what I wanted to see in Cruise. Doing this review has been a great way for me to see what everyone else has been up to.

Just having loads of features does not automatically make for a good tool though. Instead of having a shooting match between who does what, I’ve taken a little sample Java servlet that I use for demos and tried to get it working with all the tools. This project is hosted on a local subversion repository. I’m going to try set it up to simply run unit tests and create my distributable .war file. Areas that I’m going to look at are:

• Installation (on Linux, OSX and Windows)
• Setting up my existing project
• What did the tool inspire me to try next

Tools that I’ll be trying out are (in alphabetical order):

• AntHillPro 3.5.3
• Bamboo 2.1.1
• Cruise 1.0
• CruiseControl 2.7.3
• Hudson 1.252
• Pulse 1.2.56
• TeamCity 3.1.2

AntHillPro

Installers

There are no native platform installers, but you can choose from 3 packages. Each package contains both the server and the agent components.

• Graphical Java Installer
• Windows Command Line Installer
• Unix Command Line Installer

Installation

I chose the Unix Command Line. Unpacked the tarball and ran the script. It’s unique in that all the script does is kick off an ant build that asks you questions and does the installation. The defaults all seemed pretty reasonable. Installing an agent is done separately and follows the same process. Once you start the processes, hit the web frontend and finish the setup.

Setting up my project

This is where things go interesting. There was no obvious “add your project here” button. A bit of clicking around, and I found something under the Admin tab that looked like a good place to start. Then it asked me if I want a Life-Cycle based project or a Non Life-Cycle based one? What’s a Life-Cycle in this context? I fuddled around and took random guesses, and then figured out that I needed to add a workflow to my project to actually do anything. Fair enough, only I was half way through configuring my new workflow when it asked for my repository, but only gave me a dropdown list to choose from. Back out, into the admin page, find out where to add new repositories, add a Subversion one. Once again try to finish configuring my new project (at least it remembered where I was), but then it all fell apart when trying to add the repository to the workflow. Kept on throwing up “AttributeMissingException: project is missing”. Decided that this had already taken up far too much of my time just to get to this point, and so I gave up.

Bamboo

Installers

A single installer that contains both the server and the agent.

• Windows .exe installer
• OSX .dmg file
• tarball

Installation

I picked the “Linux” tarball, even though I’m testing on OSX. I prefer doing this when evaluating products as I think it gives me a better feel for what’s actually going to be installed, and I’m guaranteed I can get rid of it efficiently and easily when I’m done playing with it. Unpackd the tarball, ran the start script. All looks good, but when you try use it it asks you to edit a file in the webapp path pointing to the installation directory. Nasty. Once that’s fixed and started again, you’re asked for your license key. There’s a handy link at the bottom to go get your key, copy paste and you’re away through the rest of the wizard.

Setting up my project

Clicked the Create Plan link. Still has my pet hate of wanting you to specify UPPER CASE KEYS for everything. Are we still using ancient Big Blue iron here that has such limitations? Wants me to specify an ant target which is also a little weird - I just want the default target! Other than that all very nice. I like how it goes right into the activity page once you’ve set up your project and you can see what’s going on, including SVN checkout.

I then wanted to play with

Agents. Installed agents on Ubuntu and WinXP (server running on OSX). It’s pretty easy really. Agent matrix shows you which projects will build on which agents. Also went to look at all the pretty graphs, but I don’t have much build history right now. Bamboo has built in Clover integration, but I use cobertura in my project for coverage. It looks like they have some interesting reports around code coverage, but I can’t run them…

Cruise

Disclosure – I wrote the installers for Cruise, so obviously I like them.

Installers

Separate installers for the server and the agent components.

• Windows .exe installer
• OSX .app packaged in a zip
• Linux RPM and Debian packages

Installation

This is one of the few products that does not ship a generic tarball or zip file. As I’m doing all this on a Mac, I pulled down the server and agent zip files, unpacked them, and dragged the .app’s over to my Applications folder. Double click on each and we’re up. Once the server is up it opens up the admin interface in your browser. The first thing you’re asked for is your license key.

Setting up my project

The very first thing you’re hit with (after you’ve put in your license key) is a wizard to set up your project. Follow the prompts, and fill in where your test results and artefacts are stored. It won’t let you create the project until it verifies that it can connect to your source repository, so make sure it’s up before you start.

I then wanted to play with

Pipelines. I presented one of the very first papers on Build Pipelines with Jez Humble (now product manager for Cruise) and Dan North at Agile 2006. When I first started implementing them I had to do all kinds of unholy hacks with CruiseControl, but the benefit of using CI like this was well worth the pain at the time. At the moment you have to jump into nasty XML to configure pipelines in Cruise, but the whole process should be wizarded up in the next release. Having lots of agents allows you to split up jobs in your pipeline stages quite nicely so they run in parallel and speed up your feedback loop.

The other great feature I like that product companies are always asking me about is the ability to run parallel builds for a stage on multiple platforms. Whether you’ve got a .Net app that you need to test across multiple versions of Windows, or a web app that needs to be tested from a whole matrix of different web browsers, Cruise makes it easy to run the same tests in all these different environments using the same build of your application. If the tests pass on all of them, your stage will pass, and you know that your product is playing nicely with all of them.

CruiseControl

Installers

• Windows .exe installer
• Generic zip file

Installation

More unzip and run script. It just comes up with the sample project.

Setting up my project

This is where the venerable Granddaddy of CI servers really started showing it’s age. I had to manually check out my project, then edit the xml config file to get it going. If I’d not been using it for almost 4 years I’d get pissed off and move along.

I then wanted to play with

Something else. If you’re using it already and it’s doing the job you want it to, then there is no real reason to change. If you’re new to CI and are looking for something easy to get you started, then I’m sad to say I suggest you look elsewhere…

Hudson

Installers

• A single war file for everyone.

Installation

The most basic of all, just run java -jar hudson.war or drop the war file into your favourite servlet container. Makes it kinda hard to know what version you have though, and by the look if it they release new versions quite regularly.

Setting up my project

Not too bad. Pointing at unit test and artefacts was a bit weird as it starts one level higher than I expected. I liked the weather reports. Because the build was failing (because it could not publish any artefacts or find test results), it had little storm clouds. Nice.

I then wanted to play with

The plugins. There’s quite a few, and I like how you can just download them and go.

Pulse

Installers

• Windows .exe installer
• tarball
• zip

Installation

Another unpack the tarball and run the start script. I see a trend here. Hit the web interface and it asks you for a data directory. Also needs a license key to get going, but has a nice link there to go get one.

Setting up my project

Nice start screen with list of things to do. Hit add project, then got confused about what the URL was it was asking for. Got a test button for your SCM - yay! Setting up the artefacts you want is easy, it all just works. Only thing I can moan about is that I can’t see what’s going on in the builds, and it feels slower than the rest for some reason.

I then wanted to play with

Personal builds! It’s not something I’ve ever played around with, but it looked kinda straight forward with pulse, and it is! Just install a little tool on your development workstation and use it to invoke a personal build. Does require a bit of extra data in the project, but it’s great.

It also has agents. I love agents. They have an interesting way of doing them though. You point the agent at the server. I suppose that’s one way of getting around the security implications of other people getting their hands on your code. Agents auto-discover what their hosts provide, and when configuring a project you get a drop down list of what’s provided already. You’ve still got the freedom to add your own stuff too.

It has stages, kinda like our build pipelines. To use them nicely though you need to start editing their XML, but they’ve got great help along the right for you to see what your options are. Their stages run in parallel, so it’s not really like our Cruise pipelines, but at least they got the parallel stuff going.

Team City

Installers

• Windows .exe installer
• tarball

Installation

Unpack the tarball, find and run the start script.

Setting up my project

Hit the Create Project button. Interesting that you create the project first, then add the config after. Test SCM button. Auto tagging and code coverage built in. Nice. Also agent based. Option to automagically run emma for you. Tried it, but because I already use cobertura it barfed. It lost my artefacts config somehow. Lots of options. Also by default does not trigger a build on check in - what’s with that?

I then wanted to play with

Integration with IntelliJ. I’m a big fan of IntelliJ and use it a lot. It makes sense in a developer centric world, because the IDE is the developers portal into the world of code. I like the way you can click on a failed test and it takes you to the test code, as well as the ability to take responsibility for fixing a broken build.

Summary

The world of CI servers has come a long way in the past 18 months. It’s certainly reinforced my belief that competition is a great driver for innovation. When I started at ThoughtWorks clients would often complain that they’d like to do Continuous Integration, but setting up a CI server was way too hard. Complexity of tool configuration is now no longer an acceptable excuse for not doing CI.

It’s also great to see the trend of agents being so popular. The days of having a big mother of a build server for a project or small group of teams is over. After one of the Cruise demos I did here in the UK our client commented that it was great that he could finally find a use for all the overpowered desktop machines their managers have – use them as build agents!

As a final note for those of you who are trying to decide on what CI server to use - remember to pick one that supports your process and what you want CI to do for you.

I’m busy adding the finishing touches to my talk for JAOO this year and I’ve just realised that I’ve not let you guys know that I’m going to be there. Consider yourself warned.

Last year at the conference I got a chance to show off the new UI we did for CruiseControl with Erik Dörnenburg. Afterwards I was having a chat with Martin Fowler, and I commented on how I’d like to see a whole track at JAOO dedicated to Building and Deploying software. While I’m sure I can’t take all the credit for it, this year there is going to be a short Build track with me talking about Continous Integration. Hope to see you there…

I had one of those rare opportunities today when I got to pair with one of the developers on our project. The story he’s working on has to do with generating usage documentation for a webservice at runtime. What he wanted was a list of source files to be available at runtime. The solution he came up with (which does work) was the following Java program:

package com.example.service.utils;

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.util.Arrays;

public class ListControllers {
    public static void main(String[] args) throws IOException {
        String outputFile = args[0];
        String srcDir = args[1];
        File f = new File(srcDir);
        File serverDir = new File(f, "com/example/service/server");
        String[] entries = serverDir.list();
        Arrays.sort(entries);
        BufferedWriter writer = new BufferedWriter(new FileWriter(outputFile, false));
        for(String entry : entries) {
            if(entry.endsWith("Controller.java")) {
                writer.write(name + "\n");
            }
        }
        writer.flush();
        writer.close();
    }
}

This was then called by the following Ant snippet:

<mkdir dir="${target.dir}/war/WEB-INF/classes"/>
<java classname="com.example.service.utils.ListControllers"
      classpath="${main.jar}" failonerror="true">
    <arg value="${target.dir}/war/WEB-INF/classes/controller.list"/>
    <arg value="${src.dir}"/>
</java>

This was then bundled up in the war file so that he could get his mits on it at runtime. It works, but is not without problems. My primary complaint is that it makes the build harder to understand. To figure out what’s going you need to find and open up the .java file, and be able to understand Java in the first place.

After a few minutes of pairing we managed to fix it with the following Ant snippet:

<mkdir dir="${target.dir}/war/WEB-INF/classes"/>
<pathconvert property="controller.list.propery"
             pathsep="${line.separator}">
    <sort>
        <path>
            <fileset dir="${src.dir}/com/example/service/server"
                     includes="*Controller.java"/>
        </path>
    </sort>
    <map from="${src.dir}/com/example/service/server/" to=""/>
</pathconvert>
<echo message="${controller.list.propery}"
    file="${target.dir}/war/WEB-INF/classes/controller.list"/>

Not only is it now easy to see what we’re trying to do in the build file, but we managed to replace 29 lines of Java + Ant with 10 lines of Ant. Sweet!

While we were still working on Buildix 2, people started asking about an AMI for Buildix on Amazons EC2. This didn’t seem to be such a big ask, but now that I’ve finally gotten around to working on this I’ve found it can be a bit fiddly! While there is a lot of good documentation in the various sections of the EC2 site, I still had a quite a few head scratching moments trying to create my own Ubuntu 7.04 Server image to load Buildix into.

The Buildix image is now available for public use as ami-e4ca2f8d.

Here’s a couple of tips to keep in mind when rolling your own:

Remeber it’s Xen

I had problems getting my first batch of uploads working on the network. They would boot without a problem, but the networking would not initialize (as I could see from the console output). Things got a lot easier when I started manipulating the image locally using my own Xen installation. By booting it up on my local Xen server I could see where the problems were and fix them with a lot less pain and suffering.

The other benefit you get from testing your image in Xen is that you can bundle the image directly (ec2-bundle-image) instead of having to scrape a running machine (ec2-bundle-vol).

There can’t be only one

As the name implies, EC2 is all about having a big cloud of images. Chances are there will be a lot of instances of each image running out there. This means that the image itself needs to pick up its identity on boot up. Amazon provide a nice RESTful API that can provide each instance with information about itself. The documentation for the meta-data that’s available is available at http://docs.amazonwebservices.com/AWSEC2/2007-08-29/DeveloperGuide/AESDG-chapter-instancedata.html. The stuff I needed the most was the public SSH key to allow to log in as root, and the hostname of the machine. Here’s what I added to /etc/rc.local to allow me to do this:

if [ ! -d /root/.ssh ] ; then
    mkdir -p /root/.ssh
    chmod 700 /root/.ssh
fi
# Fetch public key using HTTP
curl http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/my-key
if [ $? -eq 0 ] ; then
    cat /tmp/my-key >> /root/.ssh/authorized_keys
    chmod 600 /root/.ssh/authorized_keys
    rm /tmp/my-key
fi

hostname `curl http://169.254.169.254/latest/meta-data/hostname`

I’ve been at JAOO for the past few days, and while here I had a chance to do a presentation with Erik Doernenburg on Continuous Integration and CruiseControl. We used the new Beta version of Buildix 2 to show people the new CruiseControl Dashboard, and quite a few people were impressed with it. Favourite features were the CCTray integration, and the ability to see the status of a large number of projects at a glance.

As always, there were also people who were interested in hearing about how it can be used for non Java projects. I had a good chat to one person who is interested in using it on a mixed Common LISP and Erlang project he’s working on. I’m looking forward to hearing how it goes for him. Due to lack of experience on my part I could unfortunately not help him much with the darcs problems he’s having though. Some people have all the fun…

It was also quite useful to speak to people about the problems they’re currently facing when trying to use CruiseControl. A common theme is people trying to manage large numbers of builds, or trying to build products across large numbers of different platforms. These are problems the dedicated ThoughtWorks development team are currently working on, so it’s great to get the validation that we’re putting effort into the things people care about now.

Up until now I’ve managed to avoid ranting here, but I can no longer resist! After all these years of experience, why are the error messages in windows still generally meaningless? Is it because there are so many of them?

The problem I faced was trying to patch a game. Downloaded the patch, tried to apply it, but got the message “The application failed to initialize properly (0xc0000135)”. Much confusion. Downloaded from another source, tried again, same error. Tried on my laptop, and this time it worked perfectly. What’s the problem?

Turns out the patch is written in .Net, and I don’t have the correct version of the runtime installed on my desktop, but I do have it on my laptop. I only had .Net 1.0, and it needs a newer version. Good thing the error message told me that in the first place then, isn’t it…

A lot of people are under the impression that CruiseControl is solely intended for Java projects. This is a common misconception. At my current client we’re using CruiseControl to build a 15 year old, 2.5 million line mixed C/C++ app using GNU make. We’re also using it to build and functionally test a new C app we’re writing as part of the project. I’ve also used it in the past on Python apps.

There are a number of ways to do this, and they all really depend on what you’re using Cruise for. If you’re simply using it for compilation, then you can simply use <exec> to call make (or whatever you’re building with). By default it will break the build if the command returns a non-zero return code, so it should be smart enough to do this out of the box. I know that some of our Ruby projects that use this method too.

My personal preference is to get CruiseControl to do an Ant build, which has some additional logic, including taging the source repository with the build label if the build passes. The approach is very similar in this case, and has the following steps:

• Have ant do any additional work/checks you want to ensure the environment is how you want it.
• Update my working copy to an exact known version that can be tagged at a later stage if the build is good (primarily a CVS problem, not really needed if you’re using Subversion).
• Use the <exec> task from ant to run the build
• If you have additional tests to run, use an <exec> within a <parallel> block to start your newly built binary in the background, then run the tests
• If everything is good and passes, tag your source code with the build label.

So, although it is fairly easy to add build plugins for various other projects, you can save yourself a bit of pain simply using the one <exec> or shelling out of an <ant> builder.

One of the things I spend a lot of my life doing is looking after Continuous Integration environments. As I work for ThoughtWorks, and am one of the guys who created Buildix, it should come as no surprise that this is all done with CruiseControl. However, in the past few months there a few new Continuous Integration tools that have crept out onto the scene. The two that I hear the most about are TeamCity and Bamboo. I finally managed to get a few hours to test them out, and here’s my impressions.

One of the things I wanted to see was how quick and easy it was to get these tools up and running. With this in mind, I specifically decided in advance to pull down the .war bundle of each one and try that out. In my experience, this would be the one that most of the clients I’ve been at would have gone for.

Notes:

• This is very much a first impressions run of the products - I had a limited time window (2 hours total) to do this in.
• People could quite fairly say that I have a biased view, but I’ve tried to keep this as fair as possible.
• I only tested doing an Ant build of a simple sample webapp from a subversion repository.
• You have to buy a license for TeamCity and Bamboo (I got evaluation licenses for both), but CruiseControl is still free…

Test Environment:

• Fedora Core 4
• Via C3 800 server with 512Mb RAM
• Sun Java 1.6.0-b105
• Apache Ant 1.7.0
• Jetty 6.1.1
• Subversion 1.2.3

TeamCity 1.2

This is the one I’ve heard the most about, and the one I’ve been most keen to try out. Back in the days when I used to be a developer, our IDE’s were nothing much more than glorified syntax highlighters. I’d poked Eclipse a few times in the early days, and not really seen what the fuss was about. Then I spent 10 weeks in a small room Alistair Jones and James Lewis. They kindly took some time to show me all the funky thinks IntelliJ could do. If JetBrains can write an IDE that is so good, what could they do if they turned their minds to CI?

Getting the webapp up and running was pretty simple (as expected). I did notice some errors scroll past the Jetty console, but chose to ignore them initially as things didn’t seem too bad. Things started to get a bit wobbly when it came to getting the agent running though. There are some basic typos in their install documentation that had me scratching my head for a minute or two. Trying to get the agent to play nice with the webapp was more of a pain, lots of NullPointerExceptions all over the place. As it all seemed to be in XML parsing, I decided to gear back to Java 1.5 and this got all the NPE’s to go away. I now had an agent up and running! Which is where my first concern pops up. It appears anyone can just spark up an agent and point it at TeamCity. I can see all those afflicted with SarBox liking that.

There was not much that was configurable for the rest of the system. Mail and IM server details were pretty much it, so I decided to move on to trying to get it to do some builds.

I did manage to get a project configured, but things pretty much went downhill from there. I got lots of strange errors all over the place when trying to look at build reports. Bouncing the webapp seemed to trigger the agent into doing a build, but I could not get it to trigger a build from a checkin.

In short - quite a disappointment really. I’ll have to try out the release they bundle with Tomcat to see if that is any better. I also didn’t try get it to play with IDEA, which I’m starting to feel is what it’s all about really.

Bamboo 1.0

I’d only recently heard of Bamboo. I’ve used other Atlassian products (Jira and Confluence) in the past, and so was a bit apprehensive about what this would be like as I find their other products a bit heavy. Would this be more of the same?

Installation was a breeze (as it is with their other products). The only bump was having to unpack the war file (one of my pet hates) to open up a properties file to set the bamboo.home property. It was only once I edited the file that I saw this could be set as a system property. It would have made me a whole lot happier if the nice initial system check screen had told me I could set this as a property instead of telling me to unpack a .war.

There are quite a few interesting system options you can set. I quite liked having a global build expiry option, and the separate build queues concept (didn’t get to play with it though). Other options I quite liked were the global JVM and Builder lists. Time to move on to actually setting something up.

Creating a project is pretty simple, the only thing that annoyed me the typical Atlassian fixation for mainframe style descriptors. Each project needs to have a ALLCAPSNOSPACESDESCRIPTOR to identify itself. Most intuitive. Everything else was pretty self explanitory. Even selecting build artifacts was a joy. Once the project is created, it does an initial checkout and build.

Now to trigger a build. This is one area they still need a bit of work on. While Bamboo did trigger a build correctly when I made repository changes, the polling mechanism is a bit heavy. It seems to do an update, and then if anything has changed it will build. Finding details on what exactly triggered the build also takes some digging, it’s not that obvious. One thing I do like though is that it keeps a history of individuals activity. You can drill down to each user and see their checkin rate, build breaking rate, build fixing rate, etc. I can see the PHB types signing the cheque for this feature alone.

Other features I like are the way they show the ant log files for each build, and also how you can effectively tail the log file of a build in progress. The way it parses your JUnit results is also pretty nifty, going as far as building an English sentence for each of your tests if you name them in the recommended testShouldDescribeWhatTheTestIsDoing format.

Things that I felt were missing were auto refresh of your status screen, and the email reports didn’t trigger when I expected them too, and the format was pretty primitive. It also takes a long time for the app to start (probably because of the Hibernate over HSQL database it keeps everything in), and app response is pretty slow.

In short - not bad for a 1.0 release! Functional, but not sure if I’d use it…

CruiseControl 2.6

I’ve not had a chance, or the need, to upgrade any of the systems I look after to Cruise 2.6. They’re all running on 2.5 and doing pretty well. While I admit that using this on a daily basis gives it somewhat of an unfair advantage, I think it also means I’ve probably got less to say about it.

First off, installation is pretty easy if you’re using the embedded Jetty for your reporting. If you want to do what I’ve done with the others and run it in your own container, it’s not that much harder. The only things you need to do are edit the startup script, and edit the web.xml file in the webapp to point it to all the correct directories. As I said above, this is a pet hate of mine.

Configuring a new project is a real bitch after using the other two. Even with my experience, and even using the sample connectfour project as a template, I still managed to get a type in there. Enough said on the configuration front.

Things I still liked about it compared to the others was the speed. It’s small, simple, and fast. That’s a key thing for me as CI is all about fast feedback, and if your tools are taking their time, you’re reducing your feedback loop. The other important thing about feedback is the information. Although the others all look pretty, there is still some basic information they’re missing for the builds - who changed what and why.

The new kids on the block are really make the grand pappy of them all show its age. I don’t think this is a bad thing though. New ideas should always be welcome in any field. They also show how much they still have to learn.

Part of what we’re doing at my current client is writing an HTTP interface to allow read only access to a Tangosol cache. For the first month or so we’ve been using Jetty (6.0.2) for testing and development, because it’s free and easy. Everything has been going well, and we finally got the official ruling that our webapp will run in production under JBoss (4.0.3SP1). We did expect this, as current webapps they have in production run under JBoss, but we were not sure of which version we were going to end up on so we stuck with Jetty. We’ve been very carefull to keep our app as vanilla as possible, but when we deploy our happy app under JBoss, we start getting showered with NullPointerExceptions like these:

java.lang.NullPointerException
at org.apache.xalan.transformer.SerializerSwitcher.switchSerializerIfHTML(SerializerSwitcher.java:153)
at org.apache.xalan.transformer.TransformerIdentityImpl.startElement(TransformerIdentityImpl.java:991)

After a bit of poking around I found that the only nulls that existed in that area of our code was when we were calling startElement() and endElement() with nulls. A quick look at the JavaDocs for these methods says that you should pass “the empty string” if there is no Namespace URI or Namespace. Simply replacing calls like:

transformerHandler.startElement(null, null, ERROR_TAG_NAME, errorAttributes)

with

transformerHandler.startElement(”", “”, ERROR_TAG_NAME, errorAttributes)

sorted out the problem!

We’re only due to go live with this app in July, but one of the patterns that has saved me many times is to test early and often. The trick is not to limit your testing, but push it as far as you can go. In this case, testing in a production like environment 5 months before we’re due to go live has meant we only needed to change a few lines of code in two classes. Imagine how much more we would have to change in 5 months time…