Feb 062014
 
Ed Sumerfield and Chris Nelson of Gaslight Sof...

Ed Sumerfield and Chris Nelson of Gaslight Software presenting “Why Agile Fails” (Photo credit: RubyNuby)

How the fuck is this coming around again?

George Westwater on Waterfall

Every so often, someone will put forth the idea that agile practices are great and all, but they fail at certain things like providing an up-front budget, a well-researched scope before work begins, a full project timeline with milestones, etc.  These sorts of things are such staples to the way most orgs approach their projects that there’s a certain tension to make agile practices better at providing them, and the path of least resistance to trying to supply these is to use the practices that used to provide them – typically some type of phase-gate approach like Waterfall.  You hear things like this:

  • We complete all the requirements and design up front, but then we do our development in an agile way.
  • We do agile, but in a way that can be managed with Microsoft Project.
  • We’re agile, except we do our testing at the end, but at least we release every six months.

And so on.  The basic idea is that doing a project without these traditional Waterfall additions is somehow inherently more risky or will fail to provide a company what it needs to make intelligent project decisions.

I’ve already spent a lot of electronic ink on this blog talking about how the benefits Waterfall purports to offer are largely illusory (kind of like the illusion of security offered by being employed at a large company) and how various agile methods can actually do a better job at providing intelligence to the business, so I don’t want to rehash all that.

However, the point I want to make here is that Waterfall and Agile arose from very different systems and operate off very different assumptions about the work.  For instance:

Waterfall’s Assumptions

  • The work to be done is mostly static.  Changes are rare and always occasion for re-examination.
  • Business needs / the market is not volatile; our priorities and methodologies today will be very similar three years from now.
  • The work is extremely similar to work that we have done before.  Almost everything is a known quantity.

Agile’s Assumptions

  • The work to be done changes often and progressively as more clarity is gained or needs evolve.
  • Business needs and the market is volatile.  The thing on the front burner today might be on the back burner in three months.  Our methodologies are evolving.
  • The work is to develop something new.  Similar technologies and methods might be used, but the business problems we’re solving technically have not been solved before.  Otherwise, we’d be using that.

The problems with Waterfall are not necessarily inherent to Waterfall as a methodology.  On paper, it looks all right.  The problem is that the set of circumstances under which it operates best is pretty rare in software development.  Are there software development projects where the domain is pretty static, changes are rare, and the business changes very little?  Sure, those do exist, and Waterfall would work just fine.  In fact, it or something similar might be the best choice for those kinds of projects.

However, a lot of organizations believe Waterfall’s assumptions to be true of their situation, and they are not at all.  Most real organizations have work that looks more like that list of Agile Assumptions, and in that context, agile methods will probably fare much better.

The issue with combining the methodologies isn’t a religious one; it’s that you’re trying to combine two things that are appropriate in two very different contexts.  You’ll end up crippling Agile by trying to make it more Waterfallish because you’re trying to drag in a context that doesn’t apply.  It’s like adding a sail to your car.  Sails are pretty good in their context, but they’re not objectively good for all contexts.

The fact is, becoming agile will challenge a lot of illusions and assumptions we make about our work that are not accurate.  If you’re hearing something like, “Agile makes it difficult to budget for the whole project up front,” that’s your cue to examine if budgeting a whole project up front is really the smartest thing to do as opposed to paying for value as you go.  If you’re hearing, “Agile makes it hard to set project milestones,” that’s a good opportunity ask yourself why you feel you need project milestones, what value they offer, and could you get that value in better ways?  Agile is meant to change your understanding, your values, and ultimately your practices at an organizational level.  It is not designed to accommodate them.

So, before you decide to add a sail to your car, just give driving the car a shot.  You might find that it doesn’t need that sort of improvement.

 

 

Enhanced by Zemanta

  2 Responses to “Improving Agile with Waterfall”

  1. “The issue with combining the methodologies isn’t a religious one; it’s that you’re trying to combine two things that are appropriate in two very different contexts.”

    Spot on Phil. I really admire this balanced take. Your point that Agile should change the org really has me thinking. I’ve struggled to answer just these questions in the past. When people want waterfall in front of Agile you’re in for long conversations with upper management. It’s a tough sell because you have to change minds first, then process. People are lazy and risk averse. And as a developer, I don’t have the time to sway these minds. The product teams I work with have little interest in rocking the boat as well, and frankly appear to have little interest in what methodology we use as long as the boss is happy.

    Few have the wisdom to discern the context you’ve laid out so clearly here. Even fewer the wherewithal to change. I continue to see organizations claiming get to be agile who are really doing nothing more than a scrum sandwich with waterfall on both sides.

    We have a long way to go but your post has given me clarity on how to move forward in these conversations. Well written.

    • Thanks, Cory. Coming from you, I consider that a high compliment.

      The relationship between the process and people’s minds is pretty reciprocal. It’s usually a matter of gaining just enough ground to be able to change the process and put points on the scoreboard, which establishes the credibility for more progress.

      And then, sometimes even empirical evidence of success isn’t enough depending on who’s in charge and what their commitments are. I don’t know. Over time, I think I’ve probably let go of any “You have to change X before you change Y” propositions. The reality is so much more interconnected and messy, it seems. I do know I tend to have more control over process than I do people’s minds or a company’s culture, though, so I tend to use process change more as a lever for affecting the other two if I can swing it.

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

(required)

(required)