Archive for the ‘Software’ Category

Real Clouds Don’t Have Logos

28 April 2009 1 comment

I’ve been doing even more reading than normal lately on the subject of Clouds lately as quite a few of us within ThoughtWorks who are going to be speaking on the subject next month are comparing notes. It follows that when I clicked the link for “The Wrong Cloud” I was not actually prepared for a delightfully entertaining paper that would contain my quote of the day:

“Today’s so-called cloud isn’t really a cloud at all. It’s a bunch of corporate dirigibles painted to look like clouds. You can tell they’re fake because they all have logos on them. Real clouds don’t have logos.”

As much as I enjoyed reading their paper, I must say that I disagree with a lot of what the guys at Maya are saying. Yes, there is a large dose of spin and hand wavey magic going on with the current leading fashion trend (that bit is totally true). Yes, it is very easy to tightly couple your application to a cloud vendor. The thing is though that it’s not that different from the tie in you get when selecting what language you use to develop your application in, which third party libraries you use or even what operating system(s) to target. The only real difference I can think of is that if for some reason the cloud vendor you’re backing stops running your app goes down, unlike all those mission critical OS/2 applications that are still running out there…

I’m pretty sure that you’d have fair warning before the plug was pulled though, especially if you’re still paying them money every month for their services.

The real questions you need to ask before doing anything on any cloud service are:

  1. What is the problem I’m trying to solve?
  2. Do any of the cloudy offerings actually help me solve that problem?
  3. What is the cost difference between deploying this app in the cloud vs our own infrastructure (assuming you have any of your own)?
  4. What is the point at which that will change? Is there a usage point where it would be cheaper for me to move off the cloud?
  5. If I chose platform X, how hard will it be to move to platform Y?

Cloud is not a magic silver bullet – such things don’t exist. As with any technology choice you make, you need to select the most cost effective one for the problem you have, and try your hardest to ignore the FUD.

Podcast on Continuous Integration available

27 April 2009 Leave a comment

Last year at JAOO I had the chance to speak to Markus from Software Engineering Radio about the talk I gave there on Continuous Integration. It’s finally available now over here. The slides that go along with the talk are available from the JAOO site.

Above the Clouds – This Sounds Familiar…

24 February 2009 Leave a comment

I found a link to Above the Clouds, a paper on Cloud Computing recently published by a quartet of UC Berkeley RAD Lab professors. I’ve been quite disappointed with publications on the subject of the latest buzzword taking the world by storm right now, so I was not expecting much when I first clicked on the link. The thing is, as I started reading through the Executive Summary it all sounded very familiar. The outline the give in the summary follows the same outline as a talk I gave in November last year at the ThoughtWorks London office for the London Java Community.

The only criticism I have is that they don’t put enough emphasis on one of my key reasons for why it’s suddenly taken off. Cloud computing is not a new idea – it’s an extension of the Utility Computing that John McCarthy talked about in 1961. Although they only make a passing remark in section 3, I think one of the most important reasons it’s taken off is that the services Amazon provide were the first that were not a “solution looking for a problem”. Earlier offerings by the likes of Sun, HP and Intel all created a solution that they tried to sell to clients. The problem was that there were remarkably few problems that their solutions solved. Amazon simply exposed services that they were using internally already. That’s not to say the other reasons they give are not valid, I totally agree with them. I think they just missed a good point.

One of the topics I only glanced over is covered cover quite well in section 6 – Cloud Computing Economics. They provide some interesting example cost calculations. Although the numbers are obviously US centric, they do provide a nice way for a company to approach making the old “build vs buy” comparison.

In summary, I highly recommend this paper for anyone who wants to get the head around what this Cloud stuff is all about and what they need to do to prepare for it.

Introducing ESCAPE

13 February 2009 3 comments

Whenever I talk to new clients about Continuous Integration, and especially about using Build Pipelines to extend their CI process to cover testing things like the deployment of their application, there is always one question that is guaranteed to pop up – how do I manage the configuration of my application in all these environments?

This is always a hard one to answer. In the last place I worked as a System Administrator we used Split-Horizon DNS as one of the ways to solve connecting to the correct host for a service. Our applications had hard coded hostnames for key services such as db.internal.domain, loghost.internal.domain and mailhost.internal.domain. Depending on which environment they ran in, when the did a DNS lookup they would get an answer that pointed them to the correct server providing that service for the environment they were physically running in. Combine this with tools like an LDAP Server in each of these environments containing the rest of the system configuration and life is easy. Or is it?

DNS and LDAP servers are relatively straight forward for experienced Sys Admins to understand and maintain, but they still can’t be described as easy to use, which is probably why so few people do it. This limits our options for fixing things down to two general categories – fix DNS and LDAP admin tools so they are easy to use, or find another way to solve the problem.

We chose the path of least resistance and went for the second option – creating ESCAPE. In the Unix tradition, it’s an app that takes one problem and solves it simply and well (at least in our opinion). It provides a RESTful interface for both the setting and getting of environment configuration. GET requests will retrieve configuration, whereas POST/PUT requests will create new entries or update them.

The URL scheme we’ve decided on is most easily demonstrated by constructing an example URL:

URL Value returned in the body
http://escape/ User interface. No API available here.
http://escape/environments/ A JSON list of all the available environment.
http://escape/environments/production/ A JSON list of all the applications in the “production” environment.
http://escape/environments/production/mywebapp All keys and values for the “mywebapp” application in the “production” environment.e.g:
http://escape/environments/production/mywebapp/thiskey The value of “thiskey” for the “mywebapp” application in the “production” environment.

As the tool is now usable we’ve happily released version 0.1. Currently we’re busy putting a lot of work into making the management interface understandable and intuitive to use. The closest thing we currently have to a roadmap is our ToDo list.

Please join us on the discussion group with your feedback.

Retrospectively Breaching the Wall between Developers and Operations

26 November 2008 2 comments

I like to describe my job at ThoughtWorks as helping Developers and Operations realize that they’re playing on the same team. No matter how awesome your code is, how elegantly you’ve solved the problem at hand, how nice and readable the code is – if you can’t get it into production your software is just a collection of bits. Likewise, you can have the best network, the most scalable hardware, the neatest cable patching scheme – but it’s just a big fancy heater if it’s not running the code your business needs.

As I’m normally brought in with the developers, I’ve been trying to find efficient ways to engage with the client Operations teams. Normally I end up having one on one conversations with various members of the team, try to find out what their current processes are, what their major challenges are and what their concerns are regarding the project I’m involved in. I usually do this to keep the safety levels high. The problem though is that it takes quite a lot of time and effort to get things going and get some momentum going.

At my current client though I didn’t have the time or access to the people to do things the normal way. A meeting was arranged with the key Operations stake holders and I effectively had just 2 hours to explain our development process in general, and Continuous Integration and Build Pipelines in detail. While talking with Graham Brooks about what we wanted to cover, he came up with the idea of running it as a mini retrospective.

After the usual introductions, we gave them 15 minutes to list the Good, the Bad and the Puzzles of their current development and release process. We had good participation from the group and as expected had a high number Bad entries. After talking through the cards and grouping them into related sections, we then allowed them to vote on the ones they most wanted to talk about. Most votes went to the core pain points, and we spent the rest of the time talking about how our process would address those issues. It also helped a lot that most of the Good entries related to the automation they already have in place…

By the end of the meeting no was talking about the bad old days (lobbing releases over the wall). Everyone was engaged starting to get some spirit of collective ownership going in the whole delivery process and that breaking down the walls that exist between the various silos was high on the list of things to do. Rather than talk to them about our process and how we would like to interact with them, we had allowed them to lead the discussion on which elements from our toolbox would have the greatest value for them.

All I need to do now is learn how to be as good a facilitator as Graham was…

Categories: Build, Development, Software

Continuous Integration Server Comparison 2008

22 September 2008 2 comments

Early last year I did a Quick Comparison of some of the popular CI servers of the time. Things have moved on since then, and I’ve actually been involved with the Cruise development team since then. Now that Cruise has been released, a number of people both inside and outside ThoughtWorks have asked me to put together a follow up article – here it is.

The list of available products out there has grown a lot in the past 18 months, and the features that they support are really great. Since I did the last review I’ve actively avoided having a look at the other tools out there to keep a clear focus on what I wanted to see in Cruise. Doing this review has been a great way for me to see what everyone else has been up to.

Just having loads of features does not automatically make for a good tool though. Instead of having a shooting match between who does what, I’ve taken a little sample Java servlet that I use for demos and tried to get it working with all the tools. This project is hosted on a local subversion repository. I’m going to try set it up to simply run unit tests and create my distributable .war file. Areas that I’m going to look at are:

  • Installation (on Linux, OSX and Windows)
  • Setting up my existing project
  • What did the tool inspire me to try next

Tools that I’ll be trying out are (in alphabetical order):

Read more…

Categories: Build, Development, Software

Speaking at JAOO 2008

12 September 2008 Leave a comment

I’m busy adding the finishing touches to my talk for JAOO this year and I’ve just realised that I’ve not let you guys know that I’m going to be there. Consider yourself warned.

Last year at the conference I got a chance to show off the new UI we did for CruiseControl with Erik Dörnenburg. Afterwards I was having a chat with Martin Fowler, and I commented on how I’d like to see a whole track at JAOO dedicated to Building and Deploying software. While I’m sure I can’t take all the credit for it, this year there is going to be a short Build track with me talking about Continous Integration. Hope to see you there…

Categories: Build, Development, Software