Nov 052013
Industrial Engineering and Management building...

Industrial Engineering and Management building at the Technion in Haifa (Photo credit: Wikipedia)

I was talking with Lean/Agile Developer and overall smart guy Travis Dietz about requirements, and I brought up my old saw about requirements being inventory.  It’s an analogy that comes from the factory, which is where a lot of the ideas that inform Lean software development and Kanban come from.  Manufacturing was certainly Deming’s primary point of reference.  Principles, techniques, and even artifacts (control charts, flow diagrams) in this way of thinking most likely have their origins (or at least their ancestors) in manufacturing and industrial engineering.

How appropriate is this, really?

I mean, the Agile community has fought and is fighting hard to correct a misunderstanding about software development that we largely owe to NASA: the misconception that most software development projects are not fundamentally different from any other kind of building project.  You figure out what you want, an architect draws up all the plans, and the plans go to the workers to build out.  Because the plans have already been made, you can scale productivity just by adding numbers.  Also, past experiences should enable you to say how long a given development project will take if you can get a look at the plans.

In deterministic ways of building, this works all right.  If I have to make car doors, someone has drawn up the plans for a car door, and it’s my job to crank them off the line.  If my station averages 20 car doors a day, then when you ask me how long it’ll take to make a thousand car doors, I can give you a pretty good idea.  This environment – deterministic construction – is largely the environment much Lean thinking was generated in and is almost Deming’s whole world.

Software development, however, is non-deterministic.  Although you might have experiences that are similar at some level in the past, every problem is a new one.  Every bit of software you write has never been done before (because otherwise, you could buy it or just re-use the code).  Each feature is an act of problem solving and creativity, and we’ve all learned the hard way how quickly detailed plans become invalid.  Once you get to the battlefield, you have to adapt.   Software development is more like making a sculpture for a client than it is building them a car.  Given that the nature of the work itself is rather different than manufacturing, how appropriate is it to use optimizations from manufacturing?

Let me begin by saying that to dismiss insights from manufacturing because it’s manufacturing is pretty dumb.  Many advances in the field of software development did not come from software development, originally, and you should never close yourself off from wisdom that might come from a different field.  If you’re going to rule out every field that is different from software development, then you’ve pretty much ruled out every field, including the ones that you like and use all the time in your software development.  Math is a very deterministic field as well, but you’re going to have a hard time programming without it.

But the question remains, how far can we really take this?  For instance, I have pointed out some things from martial arts that I believe have relevance to software development, but I don’t think practicing takedowns will help us write better software.  I think the solution may be in the distinction between laws, principles, and techniques.

Laws are universal truths that everyone everywhere has to deal with.  For example, people have to sleep.  Therefore, no matter what field you’re in, you don’t want to schedule people for 24 hour shifts all month.  People can only be in one place at one time, therefore, we do not assign people to five separate teams and expect them to be fully available to each one all the time.  They’re laws built into the nature of the reality we all share, so dealing with these laws is automatically going to be something that spans all fields.  Any insight that comes from some fundamental law of reality is pretty much guaranteed transferable somewhere else.

Principles are also general truths, typically derived from laws, but they are not always universal to all places and all times.  “Avoid the Bubonic Plague” is a good principle, but generally speaking, it’s not really applicable to most people today.  On the other hand, we might consider a principle like, “Employees should be empowered with the tools and authority they need to do their work efficiently.”  Although you could probably find an edge case or two where this didn’t apply, it pretty much applies everywhere.  Maybe someday when we live in a telepathic hive mind, concepts like “employee” and “authority” will no longer serve a purpose, but at least in our current context, it’s a pretty good principle and relevant to all kinds of work.  When applied to many different fields, you see good results.  I’d say that the vast majority of the concepts and recommendations of Lean, Kanban, and the various thought leaders in that area operate at this level.  You have to understand how the principles apply in your context, but they are more or less truths that are bigger than a specific field.

Techniques are specific implementations/practices of something, usually derived from principles.  These are only valid insofar as they operate in accord with their principles and are usually heavily conditioned by specific context.  For instance, in most of the apostle Paul’s letters to churches, he tells them to greet each other with a holy kiss (Rom. 16:16, 2 Cor. 13:12, 1 Thes. 5:26).  In the first century near east, a chaste kiss was a culturally standard form of warm greeting.  It was a way to show that you were a community and had a level of affection for one another.  In modern, American churches, you are likely to be maced when trying this out.  In our time and culture, a kiss has romantic connotations, or at the very least suggests a particularly close affection like a relative or a very old and dear friend.

In modern America, the church members might shake hands, hug, smile, slap on the back – something that signifies warmth and community to us.  The principles of community and affection span both times and cultures, but the specific implementations of those principles are very culturally conditioned, and if you went around to American churches trying to kiss everyone because “Paul told you to,” someone would rightly argue that you had confused the principle of Paul’s instruction with the cultural particulars, once they had slapped you.

It is in this area where, in software development, it is appropriate to go through these other industries at the practice level and decide if they’re really helpful in our context or not and, if so, do they need to be modified to better suit non-deterministic work?  One good example is control charts.  We (software developers) generally do our control charts based on probabilities rather than standard deviations, because we don’t have a consistent completion time we can use.  The principle is the same – we are measuring our throughput so we can make informed projections, offer our customers service level agreements, and have conversations about things that look out of whack – but as for how that actually gets done, you can’t just lift the practice out of Toyota and drop it onto a software development team.  You have to figure out if the practice will actually help you embody the principle, or whether it might need to look different or be scrapped altogether.

So, I will continue to borrow insights from manufacturing.  That environment is much older than software development, and they have discovered principles about work, flow, quality, and how people perform their best that we’d be idiots not to consider just because it came from manufacturing.  But when it comes to practices (and principles to some extent), we need to make sure we are thinking critically about the nature of non-deterministic work and the best way to embody those principles for us.

Enhanced by Zemanta
May 212012
Requirements Paste

Requirements Paste (Photo credit: C_Knaus)

In his book Agile Management for Software Engineering, David Anderson draws a parallel between software production and manufacturing production.

In manufacturing, you start with raw inventory (say, steel) that has a certain cost. This inventory then goes through the chain of operations necessary to transform it into some product that has more value (throughput).

In software development, ideas/requirements/feature requests are the inventory. They are the raw materials that get transformed into something that has more value. This seems reasonably straightforward but, if accurate, it has some pretty big ramifications for businesses and how they handle their requirements.

This inventory is expensive.

When a manufacturing plant buys steel, this has a cost associated with it. If a plant wants to be profitable and generate a good ROI, they buy enough inventory to keep production flowing but no more. It’s a waste of money to buy raw materials you can’t put to use, and there are all kinds of associated costs with holding on to lots of inventory, such as storage, etc.

As I have noted earlier, requirements aren’t free. It costs money to generate them. It costs money for executives to have meetings to come up with ideas. It costs money to get those ideas into a format where they could be made into a product. It costs money to record them. It costs money to update them. In larger organizations, there may even be business analysts who also factor into that cost.

The fact is, it actually costs a great deal of money to generate and maintain requirements. Why most organizations don’t take this cost seriously, I’ll never know.

The inventory can only be processed at a given rate of speed.

If a manufacturing plant can only process 10 tons of a steel a week, it’s a huge waste of money to buy 50,000 tons of steel at a time. Not only am I spending a huge amount of money for that inventory, but I have to store it. Furthermore, there is no way all that steel will become a valuable product nearly fast enough to recoup my investment.

What I want to do, instead, is buy enough steel to match our capacity and a little bit more as a buffer so that my plant doesn’t go idle waiting for more steel. The ideal amount of steel to have on hand is the amount my plant can be making into valuable product at any given time.

Requirements are the same way. There is no reason to put together a hundred page requirements document and give that to the developers. It’ll take months and months to produce all those requirements. What I want to do is procure enough requirements to keep everyone busy and with a buffer so no one goes idle. Why spend all the money nailing down Requirement #197 when Requirement #197 won’t be built until a year from now, if at all?

This inventory degrades in value over time.

Steel isn’t a great analogy for requirements because steel lasts a long time and is just as good ten years from now as it is, today. You have to store it, but it doesn’t seriously degrade over time.

For software, the requirements do degrade over time. Business rules change. Markets change. Strategies and initiatives change. Priorities change. Regulations change. Technology changes. This quarter’s Super Important Project is next quarter’s back burner.

Therefore, as soon as a requirement comes into being, it’s a race against time to turn that into a valuable product, because the value of that requirement and the value of what you produce with it starts to drop almost immediately until, eventually, it’s not even worth turning the requirement into a product. It’s too stale and no longer reflects the business and market realities that drove it in the first place.

Maybe a better analogy for requirements as inventory would be like milk for an ice cream production plant. It’s just going to go bad over time. It needs to become ice cream in a hurry, because it’s not going to be a viable substance forever.

This fact alone speaks to the destructiveness of a traditional Waterfall model of building software or doing projects. The more time that goes between getting a requirement into throughput, the more value is lost. If you spend all that time getting all the requirements, doing design, etc., building all at once, then releasing all at once, the odds are very good many of your requirements have much less value or no value at all. ROI goes through the floor.

Don’t have too much inventory at any given time.

All of this seems to tell us that, if you want to waste the least amount of money and generate the highest value in your throughput, the best strategy looks something like this:

  • Only high-value features should be requested. If a feature isn’t worth $1000, don’t spend $1000 talking about it and documenting it.
  • Only generate enough requirements to keep the developers busy with a small buffer so they don’t have to wait for more things to do.
  • Get a requirement into releasable value as quickly as possible. Don’t wait for the other requirements to be done.