Chris Read

One of the things I spend a lot of my life doing is looking after Continuous Integration environments. As I work for ThoughtWorks, and am one of the guys who created Buildix, it should come as no surprise that this is all done with CruiseControl. However, in the past few months there a few new Continuous Integration tools that have crept out onto the scene. The two that I hear the most about are TeamCity and Bamboo. I finally managed to get a few hours to test them out, and here’s my impressions.

One of the things I wanted to see was how quick and easy it was to get these tools up and running. With this in mind, I specifically decided in advance to pull down the .war bundle of each one and try that out. In my experience, this would be the one that most of the clients I’ve been at would have gone for.

Notes:

• This is very much a first impressions run of the products – I had a limited time window (2 hours total) to do this in.
• People could quite fairly say that I have a biased view, but I’ve tried to keep this as fair as possible.
• I only tested doing an Ant build of a simple sample webapp from a subversion repository.
• You have to buy a license for TeamCity and Bamboo (I got evaluation licenses for both), but CruiseControl is still free…

Test Environment:

• Fedora Core 4
• Via C3 800 server with 512Mb RAM
• Sun Java 1.6.0-b105
• Apache Ant 1.7.0
• Jetty 6.1.1
• Subversion 1.2.3

TeamCity 1.2

This is the one I’ve heard the most about, and the one I’ve been most keen to try out. Back in the days when I used to be a developer, our IDE’s were nothing much more than glorified syntax highlighters. I’d poked Eclipse a few times in the early days, and not really seen what the fuss was about. Then I spent 10 weeks in a small room Alistair Jones and James Lewis. They kindly took some time to show me all the funky thinks IntelliJ could do. If JetBrains can write an IDE that is so good, what could they do if they turned their minds to CI?

Getting the webapp up and running was pretty simple (as expected). I did notice some errors scroll past the Jetty console, but chose to ignore them initially as things didn’t seem too bad. Things started to get a bit wobbly when it came to getting the agent running though. There are some basic typos in their install documentation that had me scratching my head for a minute or two. Trying to get the agent to play nice with the webapp was more of a pain, lots of NullPointerExceptions all over the place. As it all seemed to be in XML parsing, I decided to gear back to Java 1.5 and this got all the NPE’s to go away. I now had an agent up and running! Which is where my first concern pops up. It appears anyone can just spark up an agent and point it at TeamCity. I can see all those afflicted with SarBox liking that.

There was not much that was configurable for the rest of the system. Mail and IM server details were pretty much it, so I decided to move on to trying to get it to do some builds.

I did manage to get a project configured, but things pretty much went downhill from there. I got lots of strange errors all over the place when trying to look at build reports. Bouncing the webapp seemed to trigger the agent into doing a build, but I could not get it to trigger a build from a checkin.

In short – quite a disappointment really. I’ll have to try out the release they bundle with Tomcat to see if that is any better. I also didn’t try get it to play with IDEA, which I’m starting to feel is what it’s all about really.

Bamboo 1.0

I’d only recently heard of Bamboo. I’ve used other Atlassian products (Jira and Confluence) in the past, and so was a bit apprehensive about what this would be like as I find their other products a bit heavy. Would this be more of the same?

Installation was a breeze (as it is with their other products). The only bump was having to unpack the war file (one of my pet hates) to open up a properties file to set the bamboo.home property. It was only once I edited the file that I saw this could be set as a system property. It would have made me a whole lot happier if the nice initial system check screen had told me I could set this as a property instead of telling me to unpack a .war.

There are quite a few interesting system options you can set. I quite liked having a global build expiry option, and the separate build queues concept (didn’t get to play with it though). Other options I quite liked were the global JVM and Builder lists. Time to move on to actually setting something up.

Creating a project is pretty simple, the only thing that annoyed me the typical Atlassian fixation for mainframe style descriptors. Each project needs to have a ALLCAPSNOSPACESDESCRIPTOR to identify itself. Most intuitive. Everything else was pretty self explanitory. Even selecting build artifacts was a joy. Once the project is created, it does an initial checkout and build.

Now to trigger a build. This is one area they still need a bit of work on. While Bamboo did trigger a build correctly when I made repository changes, the polling mechanism is a bit heavy. It seems to do an update, and then if anything has changed it will build. Finding details on what exactly triggered the build also takes some digging, it’s not that obvious. One thing I do like though is that it keeps a history of individuals activity. You can drill down to each user and see their checkin rate, build breaking rate, build fixing rate, etc. I can see the PHB types signing the cheque for this feature alone.

Other features I like are the way they show the ant log files for each build, and also how you can effectively tail the log file of a build in progress. The way it parses your JUnit results is also pretty nifty, going as far as building an English sentence for each of your tests if you name them in the recommended testShouldDescribeWhatTheTestIsDoing format.

Things that I felt were missing were auto refresh of your status screen, and the email reports didn’t trigger when I expected them too, and the format was pretty primitive. It also takes a long time for the app to start (probably because of the Hibernate over HSQL database it keeps everything in), and app response is pretty slow.

In short – not bad for a 1.0 release! Functional, but not sure if I’d use it…

CruiseControl 2.6

I’ve not had a chance, or the need, to upgrade any of the systems I look after to Cruise 2.6. They’re all running on 2.5 and doing pretty well. While I admit that using this on a daily basis gives it somewhat of an unfair advantage, I think it also means I’ve probably got less to say about it.

First off, installation is pretty easy if you’re using the embedded Jetty for your reporting. If you want to do what I’ve done with the others and run it in your own container, it’s not that much harder. The only things you need to do are edit the startup script, and edit the web.xml file in the webapp to point it to all the correct directories. As I said above, this is a pet hate of mine.

Configuring a new project is a real bitch after using the other two. Even with my experience, and even using the sample connectfour project as a template, I still managed to get a type in there. Enough said on the configuration front.

Things I still liked about it compared to the others was the speed. It’s small, simple, and fast. That’s a key thing for me as CI is all about fast feedback, and if your tools are taking their time, you’re reducing your feedback loop. The other important thing about feedback is the information. Although the others all look pretty, there is still some basic information they’re missing for the builds – who changed what and why.

The new kids on the block are really make the grand pappy of them all show its age. I don’t think this is a bad thing though. New ideas should always be welcome in any field. They also show how much they still have to learn.