How the fuck is this coming around again?
Every so often, someone will put forth the idea that agile practices are great and all, but they fail at certain things like providing an up-front budget, a well-researched scope before work begins, a full project timeline with milestones, etc. These sorts of things are such staples to the way most orgs approach their projects that there’s a certain tension to make agile practices better at providing them, and the path of least resistance to trying to supply these is to use the practices that used to provide them – typically some type of phase-gate approach like Waterfall. You hear things like this:
- We complete all the requirements and design up front, but then we do our development in an agile way.
- We do agile, but in a way that can be managed with Microsoft Project.
- We’re agile, except we do our testing at the end, but at least we release every six months.
And so on. The basic idea is that doing a project without these traditional Waterfall additions is somehow inherently more risky or will fail to provide a company what it needs to make intelligent project decisions.
I’ve already spent a lot of electronic ink on this blog talking about how the benefits Waterfall purports to offer are largely illusory (kind of like the illusion of security offered by being employed at a large company) and how various agile methods can actually do a better job at providing intelligence to the business, so I don’t want to rehash all that.
However, the point I want to make here is that Waterfall and Agile arose from very different systems and operate off very different assumptions about the work. For instance:
- The work to be done is mostly static. Changes are rare and always occasion for re-examination.
- Business needs / the market is not volatile; our priorities and methodologies today will be very similar three years from now.
- The work is extremely similar to work that we have done before. Almost everything is a known quantity.
- The work to be done changes often and progressively as more clarity is gained or needs evolve.
- Business needs and the market is volatile. The thing on the front burner today might be on the back burner in three months. Our methodologies are evolving.
- The work is to develop something new. Similar technologies and methods might be used, but the business problems we’re solving technically have not been solved before. Otherwise, we’d be using that.
The problems with Waterfall are not necessarily inherent to Waterfall as a methodology. On paper, it looks all right. The problem is that the set of circumstances under which it operates best is pretty rare in software development. Are there software development projects where the domain is pretty static, changes are rare, and the business changes very little? Sure, those do exist, and Waterfall would work just fine. In fact, it or something similar might be the best choice for those kinds of projects.
However, a lot of organizations believe Waterfall’s assumptions to be true of their situation, and they are not at all. Most real organizations have work that looks more like that list of Agile Assumptions, and in that context, agile methods will probably fare much better.
The issue with combining the methodologies isn’t a religious one; it’s that you’re trying to combine two things that are appropriate in two very different contexts. You’ll end up crippling Agile by trying to make it more Waterfallish because you’re trying to drag in a context that doesn’t apply. It’s like adding a sail to your car. Sails are pretty good in their context, but they’re not objectively good for all contexts.
The fact is, becoming agile will challenge a lot of illusions and assumptions we make about our work that are not accurate. If you’re hearing something like, “Agile makes it difficult to budget for the whole project up front,” that’s your cue to examine if budgeting a whole project up front is really the smartest thing to do as opposed to paying for value as you go. If you’re hearing, “Agile makes it hard to set project milestones,” that’s a good opportunity ask yourself why you feel you need project milestones, what value they offer, and could you get that value in better ways? Agile is meant to change your understanding, your values, and ultimately your practices at an organizational level. It is not designed to accommodate them.
So, before you decide to add a sail to your car, just give driving the car a shot. You might find that it doesn’t need that sort of improvement.