Is Agile Cheaper than Waterfall?

01/07/2011

Why would an organization decide to adopt a new way of building software? Common sense would tell you it’s because there’s a better way, that it can be done quicker, cheaper and produce superior results. After all, if what you’re doing currently works, why would you change it? If common sense does in fact prevail, the main reason for people adopting Agile over Waterfall is to save money and produce better results for less effort. But is that what happens when companies adopt Agile? It certainly wasn’t the case for an Agile project I took over last year, which at the time was well over budget and two months late. That’s in contrast to the following project I was managing using Waterfall that was on time and under budget. Does that mean Agile isn’t actually cheaper than Waterfall? Not necessarily, it’s not quite that simple.

Why Agile?

It helps to take a step back and remember why Agile first came about. Back in the good old days of the 20th century, software development had an even worse reputation than it does now for projects going over time and over budget. The 1995 Chaos Report by the Standish Group1 stated that only 9% of projects in large companies were successful (an average of 16.2% across all companies studied). With such a poor track record, clearly there had to be a better way to build software. A number of prominent people in the industry, Kent Beck, Ken Schwaber, Jeff Sutherland and Alistair Cockburn to name a few, started to tinker with Waterfall and came up with different ways of managing projects. Over time, stories spread about revolutionary approaches that flew in the face of tradition and actually produced better results. In time, the people behind these different approaches decided to meet and see if there was anything in common in how they were doing things differently. It was over a long weekend in Snowbird, Colorado in 2001 that these self-professed anarchists came together for a meeting of the minds and to produce the Agile Manifesto2.

What seems to have been forgotten though is that back in the 90’s, when the fathers of Agile were finding better ways of delivering projects, the scale and length of projects were different to the nature of projects now. I remember hearing the story of how one Agile methodology was formed by its founder Jeff Deluca (Feature Driven Development). Jeff had been brought in to review a project for a Singaporean bank that had been running for two years. All that had been delivered in those two years was a huge pile of use cases. Not a single line of code had been written. Fifteen months later, a working system had been delivered. The approach taken was a refinement of 20 years of experience that had been shaped into a repeatable way of delivering on time, on budget and with agreed function. Prompted to capture this by a colleague, Jeff wrote down what is now known as Feature Driven Development3. What’s important about this story and the stories behind other methods, such as the Chrysler project4 that was the genesis of XP, is the scale of the projects at hand. These were large, long, complex projects. The Agile movement came about because projects were taking so long to deliver anything, not because they were looking to save money. If you have a project that is going to be delivered in two months, the risks posed by analysis paralysis5 are greatly reduced and the benefits of Agile are less relevant than for larger projects.

These days, especially web based projects have shorter time- lines. There is a need to get to market quicker, the tools available to developers are better, and more can be done in less time. That means the driving force behind Agile, dealing with long overblown projects, is less relevant in 2011 than it was in 1990 when Agile was first being explored. That’s not to say the underlying premise of Agile “...uncovering better ways of developing software by do- ing it and helping others do it” is not just as relevant today as it was in 2001, but there’s nothing that states Agile is necessarily cheaper than Waterfall.

The Factors that Matter Most

The reasons why companies adopt Agile vary. Some feel they need to keep up with the times, some want a better way of doing things, some think it’s going to save them money, and some simply do it because it’s different. What isn’t taken into account is whether for the specific project at hand, Agile is going to be a better approach, whether it’s more efficient and saves money. What matters is understanding the factors that will help to decide if Agile is going to give you a better chance of delivering than Waterfall. When you break it down, it’s not that complex. There are only a few key factors at play: the team, the timeline and the complexity of the project.

The Importance of the Team

The first and most influential factor is the team itself. By team I mean all the people that will play a role in the project. Of these people, the key roles are the project sponsor, the project manager and the technical lead. These are the 3 roles that will influence the project the most and therefore determine its likely success. There are two aspects to the team that are critical in Agile to being more effective, and therefore cheaper than Waterfall. The first is experience, and the second is trust.

If a team is trying Agile for the first time, they are bound to make mistakes. Gerry Weinberg6 tells the story of how he managed to reach a particular level on his favourite pinball game, but no matter how many times he played, he couldn’t get past his high score. To get to the next level, he had to try a different approach. The problem was when he first tried a new approach, his score went down. He was doing worse than his tried and trusted approach. It took a while before he mastered the new approach and then was able to beat his high score and reach a new level. It’s the same story with Tiger Woods; he reached a particular level in his game, but when he changed his swing, initially he performed worse. It took a while before he was able to be competitive again. The morale of the story is that switching to Agile, without any experience, will almost inevitably lead to a worse result as the team learns how to master Agile. That’s not to say they won’t have some level of success. But if you’re trying Agile for the first time, it’s important to have realistic expectations. A team that is used to Waterfall and can successfully deliver using that approach is unlikely to achieve the same level of success using Agile, until they have done it a few times and learned from their mistakes.

The second aspect is trust. In Waterfall, analysis is completed and signed off before the build begins. This means that during the build there is less debate about what is to be done, there’s less change, less discussion and less interaction between the project sponsor and the developers. If the developers deliver what’s in the specification, they may never need to talk to the project sponsor. In Agile, there’s a lot more discussion, a lot more interaction, and change can happen daily. For example, in Scrum, planning for the next sprint is done based on the velocity of the previous sprint. It might be that a particular user story took longer than expected, which means less can be done in the next sprint, and it might mean there are things on the backlog that can’t be done at all. The project sponsor has to trust that the developers have done the best they can, and that the reason why that particular user story took longer than estimated was that it was harder and more complex than expected, rather than the developers’ lack of ability.

In Waterfall, the project sponsor can expect what’s in the specification to be built and rightly demand that it is delivered. In Agile, the project sponsor has to trust the developers are doing their best and accept the true velocity, and that may not be everything the project sponsor wants. It’s a very different type of relation- ship - Waterfall is more hands off, Agile is far more interactive and requires the team to work closely together. If there are conflicts, they will surface quicker and be more obvious. In Waterfall, some developers may never interact with the project sponsor, so conflicts never surface. Without trust and cooperation, Agile cannot work and will definitely not be cheaper than Waterfall.

Timelines

When it comes to timelines, Agile works best for medium to long term projects, i.e. 4 to 18 months. For shorter projects, taking an Agile approach can actually be counter-productive and lead to worse results than using a traditional Waterfall approach. To put this in context, I recently worked on a project that had to be delivered within 8 weeks. There was little point in taking a scrum approach in this case for a number of reasons. Firstly, to start the project, there needed to be enough analysis done for the user stories that were going to be in the first sprint. It was go- ing to take a couple of weeks to get this information together and break down the technical tasks required for the first sprint to commence. There was also a need for 2 weeks before launch to allow the client enough time to enter the content they needed into the system. The nature of the project meant there was a lot of overlapping functionality, which meant that it was impractical to just analyse some of the user stories upfront and leave the analysis for the rest to be done during the first sprint. All in all, using scrum would make the project harder to deliver in the timeframe. The compromise that we made with the client was to spend 3 weeks specifying the key risk areas before starting development, leaving some minor elements to be workout during development. It was a more effective way to manage the project.

For longer projects, Agile comes into it’s own. One of the risks of Waterfall is the time between the completion of the specification and the first time the project sponsor actually sees working code. When this period of time is many months, details behind decisions made in the specification can be forgotten and a piece of functionality may no longer make sense. When functionality is exposed earlier, there are two advantages: first, the team gets a sense of satisfaction in actually delivering something, it helps with morale. The second is the project sponsor is able to see what they are getting and confirm that it is in fact what they want. This is far more important than it sounds. With the best of intentions, project sponsors will sign off specifications of functionality that they firmly believe they need. For some reason, when that functionality is presented to them, on a webpage or a partially complete GUI, the project sponsor can either change their mind or come to the realisation that what they asked for doesn’t actually achieve what they are hoping. This is where an Agile approach is particularly effective. With Waterfall, waiting until everything is built means that change is much more expensive. With Agile, early exposure can prevent heading down a well meaning but misguided path. This doesn’t necessarily mean that Agile is cheaper, change is always going to incur some cost, but capturing it early reduces the cost and means the end result is more likely to be what the project sponsor is actually after.

The difficulty with timelines is for projects that are between 2 to 4 months in duration. The closer the project is to 2 months, the more likely a Waterfall approach is going to be more effective; the closer to 4 months, the more likely that Agile is going to be the better approach. The decision on which approach to take will depend on the nature and experience of the team. The best determining factor is what the team is more familiar with and experienced in . Making a team that is used to Waterfall use Agile for the first time, just because it’s a 4 month project, is unlikely to to achieve the best result.

Complexity

The final aspect is complexity. Let’s assume for the sake of argument that we’ve got a 6 month project which is supposed to deliver a website that allows people to sign up to various levels of membership, which provides various levels of access to content and features throughout the site. It’s impractical to even consider starting work on any of the features until there’s a clear understanding of the different types of users and the permission levels. If we are trying to take a scrum approach and have 2 week sprints, it might take more than 2 weeks to analyse the permission levels, determine the users and groups, and to technically determine how best to implement. It means a certain amount of analysis, both from a business and technical level, needs to be completed before a developer can practically start to work on a feature. Taking a purely Agile approach, and by that I mean picking a number of user stories as the first sprint, would lead to problems down the track as the permission levels evolve. It requires a middle ground between waterfall ( big design upfront approach (BDUF)), and the Agile (no design upfront (NDUF)), approach.. It lends itself better to the middle ground: just enough design upfront (JDUF), although I prefer the more aptly named JEDI – just enough design in front.

A Middle Path

Assuming that a project has to be either Agile or waterfall is part of the dilemma here. As the Buddhists would say, the best path is the middle path . There’s nothing to say that you can’t use elements of Waterfall and elements of Agile in the same project. For instance, you may choose to take a Waterfall approach for analysis and complete the specification before you start development. Yet, you might choose to take a scrum approach to development and work to deliver functionality in sprints. It’s an approach I’ve used on a number of medium sized projects quite effectively. It gives you the advantage of understanding the entire system up front as well as giving early exposure to the project sponsor and allow for adapting future sprints. The black and white view that it’s either Agile or Waterfall can prevent a team from using elements from either approach and deciding what’s going to work best in their particular situation, which ultimately is going to lead to the most effective and cheapest result.

In conclusion, asking whether Agile is cheaper than Waterfall is a misleading question. Sometimes it is, sometimes it isn’t, de- pending on the nature of the project, the team and the timeline. Waterfall, with the right team can be very effective, so can Agile. Similarly, in the wrong hands, Agile can be a disaster and there have been plenty of waterfall projects that have failed. A better question to ask is for this particular project and team, is it wiser, and therefore potentially cheaper to use Agile rather than Waterfall? That’s easier to answer. Agile can be cheaper than Waterfall when you have a team that is experienced with an Agile approach, that trusts each other, working on a project of 4 to 18 months with a medium level of complexity. So, rather than asking if Agile is cheaper than Waterfall, the question should be whether for this team, timeline and complexity of project, will Agile be the better approach, and that will always depend on the factors at play.

__________________

1 http://www.projectsmart.co.uk/docs/chaos-report.pdf
2 http://www.agilemanifesto.org
3 http://www.featuredrivendevelopment.com/
4 http://en.wikipedia.org/wiki/Chrysler_Comprehensive_Compensation_System
5 http://en.wikipedia.org/wiki/Analysis_paralysis
6 “Becoming A Technical Leader: An Organic Problem-Solving Approach ”, Gerald M Weinberg, Dorset House Publishing, 1986