Nov 062013
 

Local staffing firm and winner of the Most Non-Ironically Orwellian-Sounding Twitter Handle Award tweeted this the other day:

All successful change initiatives start here

[sic]

That’s a pretty bold statement for a company that’s never done a change initiative.  All successful change initiatives start at the top.

This idea is actually not unheard of.  You hear it other places, mostly as the result of disillusionment.  You can’t be an agile consultant very long without having the surreal experience of a company hiring you to help them change, then deciding they don’t really want to change all that much.  As agility begins to cascade upward and outward threatening all extant presuppositions and practices, they stand, Gandalf-like, bellowing that YOU SHALL NOT PASS and breaking the bridge before all that hubbub you’re creating in their developer teams gets to accounting or whatever.  There was one organization in particular where I felt they were paying me a very good amount of money for the privilege of ignoring me altogether.  Like I said, it’s a very surreal feeling, especially when people can and do ignore me for free.

However, the truth is that change that begins at the worker level can and does work.  In fact, my most successful “agile transformation” started with a development team, and once people saw the points being put on the scoreboard, the changes cascaded throughout the organization, changing their culture, levels of trust, rate of delivery – all kinds of good stuff.  I’ve also seen change initiatives start at the top and fail miserably because, even though management was behind it, none of the workers were.  It was something forced down on them, and they could get with the program or get out.  People were resentful at worst and apathetic at best, confident in the knowledge that this was management’s new fad, and next year it would be something different.  Change initiatives that start at the top without the full buy-in and participation of the ground up have a high risk for failure, and I speak from experience.

So, is it the case that all successful change initiatives start from the ground up?  That’s also incorrect.  Nobody can destroy a company’s agility like their leadership, and I’m going to go along with Deming here and say that the vast, vast majority of a company’s performance problems rest with leadership (ironically, vast amounts of time and money are spent on structures trying to pinpoint failures in the work force – “We need to hold our people accountable” is usually followed by an unspoken “because the problem sure isn’t me”).  Your pilot project can outperform everything in company history, and management can still shut you down.  It would be nice if good results were the final word in these kinds of decisions, but anyone who thinks data is the final word in an argument has never been in a long-term relationship.  So, change initiatives that begin at the roots level without the full buy-in and participation of management also carry a high risk of failure.

If change initiatives that start at the top are prone to failure, and change initiatives that start at the bottom are prone to failure, then what are you supposed to do?  Start in the middle?

The first part of the solution is, as Deming says, “The transformation is everyone’s job.”  You have to be willing to address the organization as a whole.  Agility has ramifications for everyone, and can actually be harmful if you only optimize a particular segment.  Can you imagine what would happen if you could make the front driver-side tire on your car go 50mph faster than the other wheels?  A similar thing happens to organizations where agility is only increasing in one place.  If you want to transform an organization, everybody has to know that what you are doing is going to affect everyone from the CEO to the night janitor.  Naturally, this can cause concern, and this leads me to my second part of the solution.

The second part of the solution is that organizations are resistant to change because of the way we try to make them change.  When we blame an organization’s agile adoption difficulties on their industry, culture, or people, we haven’t gotten to the root cause.  The root cause is that your way of changing the organization is setting it off the way a cold virus sets off your body.  “What the hell is this thing?” says your body, and the fever goes up to try to boil it out of you, the snot spigots unload to try to flush it out of you, and sometimes other unpleasant things that tend to happen at inopportune moments.  This is what organizations do to alien matter introduced into their system.  And believe me, brothers and sisters, I am preaching to myself as much as anyone on this point, as I can cite instances in my career when I was happy to blame the transformational difficulties on the company instead of the way I was going about things.

Yosemite Sam

Yosemite Sam (Photo credit: Wikipedia)

Change initiatives have to be done with a respect for people and in a manner that drives out all fear of losing their job (Deming, again).  This is the genius of Kanban: the changes are incremental, evolutionary, and the ideas and decisions come from the people doing the work instead of something imposed on them from the outside.  You can’t just go in like Yosemite Sam hootin’ and hollerin’ with pistols blazing.  When people start responding with resentment or fear, that’s your cue that you have done something wrong; it is not a cue that something is wrong with them.  What kind of misanthrope would you have to be to start thinking about getting rid of people who are actually doing you a huge favor by reacting negatively to what you’re doing?

When you start with where workers are at, and you give them the tools to visualize their work, and you help them define what they’re doing, and you help them measure how efficiently that work is being done, and you ask them, “So, what do you think we should do?” you are on the road to people being valued, giving you buy-in, and coming up with better solutions than you could.  Do you sometimes need to bring in ideas from the outside to get people started?  Sure, sometimes – but even then, it’s something you want them to own, not something they have to comply with.

Change initiatives from the top can work; change initiatives from the bottom can work, but both are risky.  Successful change initiatives start with everyone, and they are done by listening and building up your people instead of forcing it on them or getting rid of them.

Enhanced by Zemanta
Nov 052013
 
Industrial Engineering and Management building...

Industrial Engineering and Management building at the Technion in Haifa (Photo credit: Wikipedia)

I was talking with Lean/Agile Developer and overall smart guy Travis Dietz about requirements, and I brought up my old saw about requirements being inventory.  It’s an analogy that comes from the factory, which is where a lot of the ideas that inform Lean software development and Kanban come from.  Manufacturing was certainly Deming’s primary point of reference.  Principles, techniques, and even artifacts (control charts, flow diagrams) in this way of thinking most likely have their origins (or at least their ancestors) in manufacturing and industrial engineering.

How appropriate is this, really?

I mean, the Agile community has fought and is fighting hard to correct a misunderstanding about software development that we largely owe to NASA: the misconception that most software development projects are not fundamentally different from any other kind of building project.  You figure out what you want, an architect draws up all the plans, and the plans go to the workers to build out.  Because the plans have already been made, you can scale productivity just by adding numbers.  Also, past experiences should enable you to say how long a given development project will take if you can get a look at the plans.

In deterministic ways of building, this works all right.  If I have to make car doors, someone has drawn up the plans for a car door, and it’s my job to crank them off the line.  If my station averages 20 car doors a day, then when you ask me how long it’ll take to make a thousand car doors, I can give you a pretty good idea.  This environment – deterministic construction – is largely the environment much Lean thinking was generated in and is almost Deming’s whole world.

Software development, however, is non-deterministic.  Although you might have experiences that are similar at some level in the past, every problem is a new one.  Every bit of software you write has never been done before (because otherwise, you could buy it or just re-use the code).  Each feature is an act of problem solving and creativity, and we’ve all learned the hard way how quickly detailed plans become invalid.  Once you get to the battlefield, you have to adapt.   Software development is more like making a sculpture for a client than it is building them a car.  Given that the nature of the work itself is rather different than manufacturing, how appropriate is it to use optimizations from manufacturing?

Let me begin by saying that to dismiss insights from manufacturing because it’s manufacturing is pretty dumb.  Many advances in the field of software development did not come from software development, originally, and you should never close yourself off from wisdom that might come from a different field.  If you’re going to rule out every field that is different from software development, then you’ve pretty much ruled out every field, including the ones that you like and use all the time in your software development.  Math is a very deterministic field as well, but you’re going to have a hard time programming without it.

But the question remains, how far can we really take this?  For instance, I have pointed out some things from martial arts that I believe have relevance to software development, but I don’t think practicing takedowns will help us write better software.  I think the solution may be in the distinction between laws, principles, and techniques.

Laws are universal truths that everyone everywhere has to deal with.  For example, people have to sleep.  Therefore, no matter what field you’re in, you don’t want to schedule people for 24 hour shifts all month.  People can only be in one place at one time, therefore, we do not assign people to five separate teams and expect them to be fully available to each one all the time.  They’re laws built into the nature of the reality we all share, so dealing with these laws is automatically going to be something that spans all fields.  Any insight that comes from some fundamental law of reality is pretty much guaranteed transferable somewhere else.

Principles are also general truths, typically derived from laws, but they are not always universal to all places and all times.  “Avoid the Bubonic Plague” is a good principle, but generally speaking, it’s not really applicable to most people today.  On the other hand, we might consider a principle like, “Employees should be empowered with the tools and authority they need to do their work efficiently.”  Although you could probably find an edge case or two where this didn’t apply, it pretty much applies everywhere.  Maybe someday when we live in a telepathic hive mind, concepts like “employee” and “authority” will no longer serve a purpose, but at least in our current context, it’s a pretty good principle and relevant to all kinds of work.  When applied to many different fields, you see good results.  I’d say that the vast majority of the concepts and recommendations of Lean, Kanban, and the various thought leaders in that area operate at this level.  You have to understand how the principles apply in your context, but they are more or less truths that are bigger than a specific field.

Techniques are specific implementations/practices of something, usually derived from principles.  These are only valid insofar as they operate in accord with their principles and are usually heavily conditioned by specific context.  For instance, in most of the apostle Paul’s letters to churches, he tells them to greet each other with a holy kiss (Rom. 16:16, 2 Cor. 13:12, 1 Thes. 5:26).  In the first century near east, a chaste kiss was a culturally standard form of warm greeting.  It was a way to show that you were a community and had a level of affection for one another.  In modern, American churches, you are likely to be maced when trying this out.  In our time and culture, a kiss has romantic connotations, or at the very least suggests a particularly close affection like a relative or a very old and dear friend.

In modern America, the church members might shake hands, hug, smile, slap on the back – something that signifies warmth and community to us.  The principles of community and affection span both times and cultures, but the specific implementations of those principles are very culturally conditioned, and if you went around to American churches trying to kiss everyone because “Paul told you to,” someone would rightly argue that you had confused the principle of Paul’s instruction with the cultural particulars, once they had slapped you.

It is in this area where, in software development, it is appropriate to go through these other industries at the practice level and decide if they’re really helpful in our context or not and, if so, do they need to be modified to better suit non-deterministic work?  One good example is control charts.  We (software developers) generally do our control charts based on probabilities rather than standard deviations, because we don’t have a consistent completion time we can use.  The principle is the same – we are measuring our throughput so we can make informed projections, offer our customers service level agreements, and have conversations about things that look out of whack – but as for how that actually gets done, you can’t just lift the practice out of Toyota and drop it onto a software development team.  You have to figure out if the practice will actually help you embody the principle, or whether it might need to look different or be scrapped altogether.

So, I will continue to borrow insights from manufacturing.  That environment is much older than software development, and they have discovered principles about work, flow, quality, and how people perform their best that we’d be idiots not to consider just because it came from manufacturing.  But when it comes to practices (and principles to some extent), we need to make sure we are thinking critically about the nature of non-deterministic work and the best way to embody those principles for us.

Enhanced by Zemanta
Aug 272013
 
Retrospective 5 (Pinestone)

Retrospective 5 (Pinestone) (Photo credit: .:fotomaf:.)

Kanban does not prescribe any particular meetings.  It leaves it up to the workers to decide what meetings have value and how often to do them.

One meeting that I’ve found has value nearly everywhere I go is the good ol’ Retrospective made famous by good ol’ Scrum.  As long as we think of retrospectives more generally as a regular time for the team to talk about improving itself, you’ll find retrospectives many places you find Kanban and/or Lean practices, including Toyota.

Having extensive experience with the traditional Scrum retrospective, though, I think there are some things that can be changed to make it flow better with Kanban’s intent and, in fact, I think it even produces better retrospectives for Scrum teams as well.

The traditional Scrum retrospective basically follows the format of each team member stating what things went well (things to keep doing), what things did not go so well (things to stop doing), and what things could benefit from change (things that could benefit from change).  The Scrum Master records all these and, ideally, the team selects some to work on for the next sprint, and they talk about these items as part of the next retrospective.

You don’t have to be a Scrum Master very long before you watch these meetings devolve into a dry, useless mirror of the daily standup where no real value is added or degenerate into an unfocused whine-fest.  The value of the classic retrospective tends to drop so quickly that, if you head over to LinkedIn and start rummaging through the threads in the various agile and Scrum groups, you’ll see tons of threads asking how to do retrospectives differently.  In these threads, hilarity ensues because 90% of the consultants and coaches only learned the practice but never really understood the purpose, so you get surreal ideas like, “I use Legos in my retrospectives.  I buy everyone ice cream.  I dress up as Sailor Moon.  I have my retrospectives underwater in SCUBA gear.” and so on.  Ideas that try to make the meeting more engaging instead of changing the meeting to better accomplish its intended purpose.

I would like to share with you what I think is a good way to change the meeting to better accommodate the way Kanban effects organizational improvement.  It is not the objectively best way, nor is it a good thing to do in every organization, but here it is.

The first part of the meeting is to go over the metrics – specifically, we review the Cumulative Flow Diagram and the “Control” Chart.  Together, we discuss flow.

Did you catch that?  We don’t talk about making low numbers higher or high numbers lower.  We talk about the flow of our work.  What does the data imply about where things are not flowing smoothly, and why might that be?  Are there communication issues?  Too many handoffs?  Is there a small part that is affecting the whole?  Is the whole thing screwed up?  We are concerned about the areas where work is not flowing smoothly and taking cues from areas where it is.  We talk about where we would like to be and how our situation is different than that.

Notice this is very similar to talking about what is going well, what isn’t going well, and what are areas for change.  The main difference is that we are talking about an objective situation as a team as opposed to navel gazing into our personal feelings.

Once we have talked about these things, we discuss as a team what we believe the one area is that needs the most love.  Usually, the data strongly suggests an area, so there is little debate.  Then, we brainstorm ideas of how to improve that area.  We vote on one idea, enact that, and see if it made a difference when we get together next time.

Here are the key points:

  • The discussion stays about our work, not things or people outside our work that we can’t control.
  • The discussion stays objective.  The metrics are what they are, regardless of what our individual feelings might be.
  • The metrics are a catalyst for discussion and inquiry, not a substitute for it.
  • The focus is on improving our flow, not going faster.  Take the kinks out of your hose, and the throughput will increase.
  • The focus area and the solution both come from the team, not from the outside, creating instant buy-in.
  • One and only one change is introduced so it can be measured.

One might argue these are the basic intents of the Scrum retrospective, and that may be, but I would then counter-argue that the traditional format is not the most efficient way to accomplish those intents in many cases.

Enhanced by Zemanta
Aug 262013
 
gray wolf

gray wolf (Photo credit: U.S. Fish and Wildlife Service – Midwest Region)

When I was in school, I bought a wolf adoption packet.  I don’t even remember what it was called, but the idea was to prevent the extinction of a specific kind of wolf.  You bought one of these packets that had a photo of your wolf and such, and your money went to an organization preventing their extinction.

When I told one of my friends about it, she said, “I can’t believe you spent money to save a wolf when there are so many children dying of starvation out there.”

You’ve probably seen something similar in your own life, when one person assists a cause, and that effort is criticized by someone on the grounds that there are other, more important causes out there.  Usually this criticism comes from someone who is doing nothing about either situation.

Obviously, there is the fact that people can care about more than one thing.  Of course I care about children dying of starvation.  Caring about one thing doesn’t mean you think it’s the most important thing, nor does it mean you can’t care about anything else.  But people do something for causes for a lot of different reasons.  In my case, someone presented an opportunity to me to do something about it in a way that made it easy to help.  People have different backgrounds, gifts, and preferences that draw them to certain causes, but I’d say the common factor in all of them is that people feel like they can actually make a difference in that area through their efforts.  This is why the Peace Corps has many members and the Giant Asteroid Hitting the Earth Prevention Society has so few.

In the Agile and Lean communities, we tend to talk a lot about processes – the pros and cons of different processes, how to tweak them, how to analyze them – and it usually isn’t long in such discussions before some wag pipes up, “Hey, guys, remember.  Individuals and interactions over processes and tools!”  Usually this is someone who is contributing nothing in either category.

I actually find this sort of thing both offensive and stupid – chronically short-sighted at the very least.

If the Saw movies have taught us anything, it’s that you can’t change a person.  People have to change themselves, and it’s something that takes a long time apart from some traumatic experience.  I can’t consult someone into a more agile mindset.  I can’t consult an organization into trusting each other more.  I might be able to point out these needs, but ultimately, I am very limited into what I can do about it.  I can’t change people by making them fall backwards into their co-workers’ arms, and I am unwilling to set up an elaborate chain of deathtraps to help them all see that they’re valuing the wrong things.  I can inform them, but that’s really where my power to change stops, more or less.

So, placing all my efforts in the people/culture bucket is kind of like joining the Asteroid Prevention Society.  It’s an important issue that I can do relatively little about – at least, directly.

At the same time, we all know that individuals and interactions are more important than processes and tools.  Absolutely.  100% agreed.  By a long shot.  So the question is, as a consultant, how are you best serving the individuals and interactions as opposed to being a process or tool vendor?  I believe the answer is in focusing on the things that will enable the people and the organization to change themselves – most notably, educating them, changing their behaviors (or processes if you will), and letting the results of that be the levers for creating the true people and culture changes that are the core of what will benefit an organization long term.

I have seen this path work several times – culture and people changes that cascade from education and creating more enabling processes.  It’s like irrigating dry soil, making it ready for the seeds of true growth that would inevitably die otherwise.  I have never seen lasting cultural change come from an initiative specifically designed to change the culture.

So, the next time you hear someone seeming to focus on processes or tools, don’t automatically dismiss that interest as misplaced, especially if they aren’t selling said process or tool.  It may be that they care about individuals and interactions far more than you know.

Enhanced by Zemanta
Aug 142013
 
Kanban in miniature

Kanban in miniature (Photo credit: jhritz)

For some years, now, David Anderson has vocally separated Kanban from “agile” or “agile methodologies” like Scrum, XP, and so on.  I never really understood why this was.  Although obviously Lean is found in several industries as is the use of various elements we find in Kanban, easily the most common place to find Kanban in the wild is in software development, and specifically software development teams trying to become more agile.

Also, you have to understand that I came to Kanban as a progression of sorts from Scrum.  Scrum helped me achieve a lot of the things I was trying to get out of a more agile software development experience, and when I came across Kanban, I saw it as (in my opinion) a better way to get at those same things.  But I didn’t see it so much as a different kind of thing from Scrum; just something that did some of the same things better.  So, in my head, Scrum, DSDM, XP, Kanban – these were all the same kind of thing, just with varying approaches and doing some things better than others.

But now, after having engineered the use of Kanban in a few different organizations and observed its power and value, as well as doing research trying to accumulate something intelligent to say about its differences with SAFe as it bulldozes its way into the market like a hyperactive wrecking ball that just demolished a Pixy Stix factory, I think I’m beginning to see what David’s been on about.

Agile and agile-ish methodologies can link their spiritual heritage to the concepts that became embodied in the Agile Manifesto – the signatories of which are names that you’d recognize in the field, like Ron Jeffries of XP fame and Darth Sidious of Scrum.  This Manifesto articulated principles and concepts of what agility was looking like in software development (and specifically software development, I might add – it says right there in the preamble).  People began to create and collate practices and systems that, in their minds, did good jobs of expressing in practice and technique those agile principles.  You do these things, and you are getting closer to expressing those principles more fully and more comprehensively.

What I have been discovering over time, though, is that Kanban actually makes nothing more agile, nor does it improve anything.

Rather, Kanban is an efficient, effective way to show you where you can improve, the best places to improve, and what the impact will be.  In other words, Kanban gives you everything you need to improve quickly, efficiently, predictably, and get the most bang for your buck.

When I was a kid, dentists would sometimes give you these red tablets for you to swirl around in your mouth after you brushed your teeth.  They tasted like they were made from Hell’s own brimstone and the ground up bones of dentists that had passed on, and perhaps they were, but when you spit the stuff out, there would be red areas on the surfaces of your teeth that you missed while brushing.  They didn’t clean your teeth; you had to do that.  But the red tablets drew your focus immediately and visibly to the areas that needed your attention and the severity of your lack of attention.

Kanban is like that.

Making your work visible, limiting your work in progress, measuring the flow of your work, making policies explicit, and managing to flow are all designed to give you exactly what you need to apply your improvement brainpower.  Where do these improvements come from?  From you, of course.  Nobody knows how to do the work better than the people doing the work; not an agile coach, not a consulting company, not a manager of the people doing the work – the actual people doing the work are your best source of ideas for improvement.

Organizations usually don’t lack opinions about how to improve; what they lack are ways to get their work situation under control so they can meaningfully study it, ways to sift through the good ideas from the not-so-good ones, ways to know what their priorities should be, ways to project the impact on the work, and ways to measure the actual impact to see where adjustments to the good ideas need to be made.

So, why is Kanban not agile?  Because Kanban doesn’t increase your team’s agility one single iota.  What it does it create the conditions and foundations necessary and  provide you with all the intelligence you need for kaizen – an ongoing journey of improvement, driven by the people doing the work, but always viewing how it improves the system as a whole.

Enhanced by Zemanta
May 242013
 
English: Simon Cowell at the National Televisi...

English: Simon Cowell at the National Television Awards at the Royal Albert Hall, London, October 2006. (Photo credit: Wikipedia)

How do you know if your chosen agile initiatives are working?  How do you know if you’re agile consultant is actually helping or just telling cute stories and quoting books they’ve read?  How do you know if your agile teams are actually agile or just call themselves that?

This is a dicey question, and there are a fairly large number of agile consultants who will tell you that you can’t measure such things.

Like most falsehoods, this has a seed of truth to it.  When you are dealing with complex systems that are primarily people-powered, you can’t quantify everything, nor should you try.  This is one of the many reasons why managing by the numbers is so wrong-headed.  You have to be in the midst of your people.  You can’t quantify things like communication, collaboration, trust, habits, or many of the multifaceted aspects of the way people work with other people.

It is also true that any change in business as usual will actually create a dip in productivity (and possibly morale) as people have to leave the familiar and struggle to form new ways of working together.  Unfortunately, this stage (sometimes called the “storming” stage) is where most people abandon their agile initiative because things got worse, and they never reap the rewards of riding through that dip and coming out the other side into the new normal of greater productivity.

However, it is my view that some aspects of agile effectiveness can be measured, and it basically comes down to this: are you delivering more value more quickly?

Companies increase their agility to make more money.  Full stop.  For all our fancy mission statements and talk about disruption and whatnot, every for-profit organization has the overarching goal of making money.  Things that help companies make money, save money, or protect money from risk are seen as improvements, while things that cause companies to lose sales, lose money, or put their money at risk are seen as detriments.

Agility is not exempt from this.  In fact, if agility isn’t helping you in this department, what’s the point in doing it?

There is no inherent value in conforming to “agile rules” or even the Agile Manifesto, so trying to measure that is a waste.  There’s no inherent value in conforming to a plan, so trying to measure that is also a waste.  That alone sweeps a ton of possible metrics off the table.  There’s no point in assessing agile effectiveness by things like if you’re doing Agile Practice X, if you’re following the Scrum rules, or how well your estimates match up to actuality.  None of that truly matters.

There is inherent value in delivering items of value, not wasting time reworking those items, and removing obstacles to delivering those items.  Therefore, if you want a measurement of the effective agility of your team or organization, here are the metrics I think are not only most important, but are actually measurable.

1. How much value are you delivering over time? (Throughput, Lead time)

2. How much rework are you having to do? (Quality, Time spent on bugs)

3. How are work items flowing through your system? (Flow, Amount of work over time in each stage, Amount of work coming into and leaving the system)

The things that help you deliver more over time without increasing rework are enhancing your agility.  The things that impair this dampen your agility.  The whole reason to become agile is to deliver more, deliver faster, and deliver higher-quality.  It isn’t to become better people, hack our corporate culture, or make software development more pleasant.  All those things might very well happen and are probably desirable, but they are means to an end.  Ultimately, it’s all about the Benjamins or whatever the kids are calling money these days, and that is the final criterion by which initiatives can be measured.

If you aren’t measuring these things, I encourage you to do so.  A lot of people think their diets work, too, until they step on the scale.  A lot of people think they can sing until they do it in front of Simon Cowell.  Find out, and when you’ve found out, don’t use those measurements to beat yourself up (or anyone else for that matter), but use them as a catalyst for conversations about improvement.

Enhanced by Zemanta
May 072013
 
Me presenting at KCDC 2013

Me presenting at KCDC 2013

Thirty-six hours before the first KCDC precompiler session began, I got a call from Kansas City’s premiere Lean/Agile expert asking if I could fill in for him at the Agile and Lean Workshop precompiler due to an emergency.  Twelve hours before the session began, the United States’ premiere Lean/Agile expert was delayed by snow and told me he couldn’t make it to the session until lunch.  That left me and this guy, or in other words, George and Ringo, and Ringo had a conference to run.

And it went great!  We had a really good group who gave us good questions to work with.  We played some games.  We talked about Scrum.  We talked about Kanban.  We talked about all the hot button Agile topics like estimation, bidding, and ongoing maintenance.  It was a good, energizing talk that reminded me how much I enjoy that sort of thing.

P.S. If you were at that workshop, please take a second and leave a rating for me at SpeakerRate.  Much appreciated.

Friday morning, Dan “Kan-Man” Vacanti gave a terrific keynote about how we serve our customers better with pull systems for doing our software development as opposed to push systems.  He also gave my own session a quick shout-out for which I am grateful, because without context, most people don’t get too fired up about a talk called “Lean Metrics.”

Next, I gave a talk called “Lean Metrics.”  Standing-room only with some people sitting in the aisles.  It was a huge fire hazard and OSHA violation, but it charged me up.

We talked about why estimation fails us and how keeping metrics can overcome some of the weaknesses of estimation, providing a better foundation for our planning and projections.  Then, I demonstrated control charts and cumulative flow diagrams, explaining how each one can help us in not only estimating, but also continuous improvement as well as challenging elements in the organizational culture that dampen agility.

Here are the slides for that presentation, and if you were there, it’d be cool if you left me a rating for that talk at SpeakerRate.

After this was over and I was carried out on the shoulders of screaming fans, I spent a little bit of time catching up with contacts from KCDC’s various sponsors.  This was an especially good KCDC for sponsors.  We had 850ish people there, which meant that, even between sessions, there was a continuous flow of activity to the various booths, unlike previous years where booth visits were pretty bursty in between sessions.

I attended an eye-opening talk on UX from John Alexander, a talk on the enterprise use of HTML5 and various JavaScript libraries from the Keyhole team, and a truly thought-provoking talk on what other languages have to teach us about object-oriented programming by Jessica Kerr.

That night, dinner and drinks with a huge crowd of people I usually only see once a year.  Also, Sugar Ray gave a concert nearby.  Nobody noticed.

I did not get to attend on Saturday, but I hear good things.

I think the venue worked out great, the crowd was numerous and diverse, and the variety of speakers and topics made sure there was at least something for everyone.  It may have been our most valuable KCDC yet in terms of investment.  If you didn’t go this year, dude, go next year.

Enhanced by Zemanta
Jan 282013
 
English: It shows that agile methods are focus...

English: It shows that agile methods are focused on different aspects of the software development life-cycle. (Photo credit: Wikipedia)

I haven’t had much time to write, lately, but I thought I’d assist the pithy Bob Marshall in preserving the following paper by Grant Rule.  It’s a paper that’s like pretty much completely right.  You can quote me on that.

What’s wrong with the project approach to software development?

January 11th, 2011 Author: pg_rule

Pretty much since “software” was first invented (60 years ago?), numerous folk have been promoting an ‘engineering-led’ approach to ‘software projects’. Yet this advice goes largely unheeded, with the result that the relative success of IT projects is poor, and has improved very little during all my years in IT (38 and counting). Given that such admonishments seemed to have had such little effect in all that time, I also find myself asking, “Do I think it likely that further exhortations to those involved in ‘software projects’ to change their project practices is likely to achieve improved value delivery to stakeholders?”

And I have to conclude that the answer is “No”.

Following Albert Einstein’s adage that, “The definition of insanity is doing the same thing over and over again and expecting different results”, it seems to me that we need an entirely new approach. A new approach which goes to the root causes of what actually goes wrong in the end-to-end process. Why are the honest endeavours of software developers often so disconnected from the delivery of customer and stakeholder value?

Observation of what actually happens in organisations suggests there are two root conditions to the problem:

  1. Those responsible for business strategy are disengaged from, and know relatively little about, software- intensive systems and technology. So they structure their organisations so that software & technology people are segregated into silos. In those silos, the inmates talk amongst themselves in whatever arcane language they choose. But importantly, they don’t communicate (or interfere) with the ‘real business’.
  2. Everyone conspires to pretend that software-related activities should be managed as ‘projects’. That is, as chunks of work that start and end (on more or less clear and agreed dates), that have more or less well- defined goals, that contain a list of activities (tasks) that are assigned by a ‘project manager’ to a project team to which ‘resources’ are assigned for a limited period.

The results of studies too numerous to mention shows that most software projects are ‘challenged’ or ‘fail’. One study suggested that the majority of experienced project managers (and I am sure, folk playing other roles) expect at least 1 in 3 of the projects they lead to fail! As systems become more complex, and larger, they employ more teams combining projects into programmes… which further reduces the likelihood of successful achievement of the overall goals.

My conclusion is that we need a complete change of mindset. We need to move away from the inherently batch & queue concept of the ‘project life-cycle’ (as promoted by organisations including BCS, APM, PMI, OGC, SEI, NCC, ISO, IEEE, IET, etc. etc.) to a different approach.

I suggest that the required new mindset will accommodate the ideas of flow production and lean systems thinking that first began to be developed systematically (in e.g. automotive engineering) around 100 years ago. (Of course, one can trace elements of flow production & lean at least back to Carthage c.300-200 BC, but let’s ignore the history for now.)

I posit that Tom Gilb’s Evo method, and other agile methods such as XP, Scrum, Flowchain, and software Kanban, etc., begin to achieve ‘better’ results compared to ‘traditional’ big-design-up-front, wholly batch & queue methods, precisely because they encourage workers to focus on smaller batches of stakeholder value. In other words, value in terms the software developer can get to grips with.

Agile methods are one or more steps nearer to the ideal of ‘single piece continuous flow’. BUT… they are inherently limited because they continue to create & disband teams, to establish & abandon value streams, to create & throw away know-how, at – it seems – every opportunity. And crucially, they allow the C- suite and ‘business-side’ managers to ignore their responsibilities for the system of work and for the desired outcome.

Flow production (toward which Evo, Flowchain and Kanban currently make the nearest approaches IMO) would:

A) Make the entire end-to-end, whole-life, ‘concept to consumption & retirement’ process of defining, deciding, acquiring, designing, developing, operating, supporting, maintaining & replacing software- intensive systems a visible, inherent part of normal business operations… forcing issues onto the management horizon so they can be addressed as business issues – and not just something technologists worry about.

B) Because it would be apparent that software & IT issues were causing interruption to (or even cessation of) the flow of value, C-suite executives would have to recognise the pressing need to engage with software & IT related issues just as much as they do with other kinds of business issue. Conversely, the engagement of the ‘systems and software engineers’ with ‘the business’ would also be stimulated, the role of each and the communication between them finally becoming acknowledged as a main artery of the organisation’s lifeblood.

Flow production can only work effectively with the active engagement of all involved. For this reason it is a far more sustainable business model than other, perhaps more familiar, approaches. It embeds the ability to flex and respond to market forces deeply into the organisational culture. The focus on ‘the unforgiving minute’ forces constraints and problems out into the open where everyone can see them. It won’t allow problems to be swept under the carpet, or passed from one department to another like the proverbial buck. Hidden problems will always result in debts in one or more of the five kinds of capital. And such hidden debts, whether financial, technical, intellectual, social or environmental, all too often bite you when you least expect it. They will injure or kill the project – and destroy stakeholder value. Even if the project avoids repaying its hidden debts, this usually means that the debt has been passed-off onto one or another unsuspecting stakeholder group (sometimes, the end-customer or taxpayer). Which must be judged as unethical behaviour.

Unfortunately, not only have most people in the software industry been taught to sit in their silos and focus largely on coding, they and their masters have developed a cultural love affair with the project concept. To the extent that everyone assumes that all work has to be compartmentalised into projects, the very epitome of batch & queue thinking. Tell me what you think. Has the software project had its day, or is there another way of revolutionising workpatterns in the software industry?

Enhanced by Zemanta
Sep 272012
 
English: Photograph of Steve Denning

English: Photograph of Steve Denning (Photo credit: Wikipedia)

This is a question addressed in Steve Denning’s article that uses it in the title.  I have already talked about why one should appreciate Denning for what he does, but also take him with a grain of salt.  I want to take a look at the actual article content.

First, it’s notable that these points aren’t actually Denning’s.  Everything he says about kanban comes from an e-book by Michael Sahota.  Michael Sahota is a Scrum coach who, to his credit, also has some experience with kanban.  This means he escapes the critique of 99.999999% of the Scrum dudes who critique kanban – that being that they have no experience with it.

But it’s interesting to note that Denning’s actual arguments essentially boil down to, “This other guy said these things about kanban, and that sounds probably right to me.”  This is what I mean about the grain of salt.  Agile coaches, consultants, and commentators will have made astounding progress when they stop proclaiming religiously what someone else has said and actually observed it for themselves.  If there was ever a field where it was necessary to become your own authority figure, it’s consulting, and there is no other initiation into that level than trial, tribulation, and victory.

If someone has been leading their company through Waterfall and has been kicking ass, I can’t argue with her.  Why?  Because she’s actually kicking ass.  I honestly can’t imagine this actually happening with Waterfall, but you see my point.  I see far too many agile “authorities” boldly proclaiming what does and does not work, what companies should and should not be doing, and what the agile community should and shouldn’t be doing, and it’s just ridiculous to me because I know they have no clue.

The crux of Sahota’s/Denning’s argument is that kanban can work with existing practices and, unlike Scrum, does not challenge an organization’s culture.  Sahota qua Denning points out that kanban can address cultural change, but it doesn’t intrinsically require it the way Scrum does.  He then goes on to discuss kanban’s usefulness as a sort of slow “gateway drug” to more rigorous agile practices, such as Scrum.

The first problem with this is a confusion between rate of change and lack of change.  An organization that moves to kanban will have to change – culture, practices, values – everything.  However, kanban does not require bulldozing everything an organization is currently doing.  Rather than crushing the rock like another rock, it erodes the rock like water – pointing out the sharp edges that need smoothing and, organically and over time, carving out the Grand Canyon.  It challenges a company’s practices and culture slowly and incrementally, but it does challenge them.  You can’t do kanban and refuse to deliver incrementally.  You can’t do kanban and measure success by conformity to up front estimates and requirements documents.  But we don’t uproot everything at once.

Scrum, by contrast, requires a pretty big displacement of practices.  You can’t really incrementally Scrumify, and this is generally discouraged.  The irony is that you can’t change culture overnight.  The idea that forcing an organization to adopt Scrum is going to instantly transform an organization’s culture is not only ridiculous but a key factor in many difficulties people have adopting Scrum.  They do all the Scrum stuff, but the culture hasn’t had time to change, difficulties ensue and… guess what, Scrum doesn’t work!  No, it does work.  It’s just that you tried to take people from 0 to agile in thirty seconds.

The second problem is that Sahota/Denning confuse leaving a value stream unchanged with leaving an organization’s culture/values unchanged.  This is ironic, given that one of the thrusts of the overall article is that a set of practices are not the same as a culture.  Kanban begins by mapping what an organization currently does to produce value.  It doesn’t recommend changes right off the bat; it just makes the value stream visible.

Have you ever noticed that almost no organizations have done this prior to kanban?  Have you ever noticed that it is rare for an organization to have a visual map of what their inputs are, what their customers want, and how they produce value?  Do you know why this is so rare?

Wait for it.

Because thinking critically about how you deliver value to a customer and all the things that hinder that is NOT USUALLY PRESENT IN THE CULTURE.

Mapping the value stream is a challenge to the culture.  Making work incremental is a challenge to the culture.  Identifying bottlenecks and constraints is a challenge to the culture.  Allowing work “phases” some slack and having them go idle if the downstream is full is a challenge to the culture.  If you don’t think so, walk on up to your manager and say, “Hey, I’m not going to develop any new features, today, because QA has their plate full.  I’m going to help QA, instead, to get that moving,” and see how that goes.  It’s a challenge to the culture!  Folks, I have been at this a while at a pretty good number of companies, and I can promise you that the kinds of questions and activities kanban raises are not ones that allow cultures to go unchanged.  Just because you begin by mapping the current process does not mean you will change nothing.

Finally, there is the idea that kanban is suitable for organizations that aren’t ready for a “real agile” methodology like Scrum.  It might be a lightweight gateway into them, however.

I have to admit that I find this completely puzzling in light of what actually happens in the industry.  Not only have I been at this a while, many of my friends and colleagues have as well.  I also actively read books, blogs, and interact regularly with agile coaches not just in America, but in other countries.  All that to say that, while there’s tons going on that I don’t know, I feel like I have a high level idea of what the trends are in our industry.  And I have never, not even once, heard someone say anything like, “Yeah, we mastered kanban, and once we were rockin’ along on that, then we knew we were finally ready for Scrum.”  No one does that, and if your organization has, they would have burned you as witches in Salem because of your freakish abnormalities.

By contrast, the stories of organizations moving from Scrum to kanban are legion.  That isn’t to say there are more orgs doing kanban than Scrum.  That’s not at all true.  Scrum is way more common and popular.  But when you hear about orgs switching from one to the other, it is virtually always people going from Scrum to kanban, not the other way around.  And as a CSM, I can tell you that doing kanban well is every bit as rigorous and challenging as doing Scrum well.

Is kanban agile?

Well, if agility means delivering valuable software more quickly, then yes.  If agility means incremental delivery of the most valuable items, then yes.  If agility means challenging the culture to value learning, observation, and adjustment to change over guesses, contracts, and trying to eliminate variation, then yes.  If agility means uprooting everything an organization is currently doing, then no.

Aug 282012
 
W. Edwards Deming in Tokyo

W. Edwards Deming in Tokyo (Photo credit: Wikipedia)

W. Edwards Deming was a man instrumental in helping Japan get back on her industrial feet after World War II. It’s hard to imagine the United States being able to tell someone else how to do things efficiently and without waste, but apparently we used to be in a position to do this without anyone laughing, and this guy did it.

Deming was a prodigious thinker, writer, and analyst, although once you worked through the core of what he had to say, a lot of it fell into categories like “Common Sense” and “No Seriously Why Aren’t You Guys Doing This Are You Touched In The Head Or Something.” One of these ideas was the Plan – Do – Check – Act flow, which is often shortened to PDCA because… well, actually, those are all one-syllable words, so I guess I have no idea why it gets shortened to PDCA.

Plan: What are you trying to accomplish and what do you think will get you there?

Do: Execute the Plan.

Check: Did the results of the Do correspond with the expectations in the Plan?

Act: Make adjustments if necessary (most often to the Plan).

As simple as the basic idea is, it’s astounding how many business processes, activities, and decisions leave out one of these components, or at least fly right through them as if it’s not important to do them carefully and well. Because I have no life to speak of, I got to thinking about the various ways this cycle occurs in software development.

PDCA at the project level

When we begin a project, we ideally have some strategic goal this project is supposed to accomplish. Maybe it’s to increase market share. Maybe it’s to cut costs by automation. Maybe it’s to give more control to our customers. This goal and this project have ideally been prioritized, and now its time has come.

In the Plan stage, we want to define what the project should look like (e.g. a feature list) and how we think we’ll get there (e.g. prioritized sequence, high level design and infrastructure, etc.). It’s important to note that we are planning at the Whole Project level. It’s just waste to start doing things like getting all the details about all the features at this point.

The Do stage, I’ll expand out in a moment, but this is basically building the application and getting it all into production.

In the Check stage, we make sure our intended features are in there and working. We also want to measure how this project is doing in terms of supporting the strategic goal that brought it to us.

In the Act stage, we either move on to the next project priority, or we enhance and rework this one to get closer to our intended feature list and strategic goal.

PDCA at the feature level

Now, we’re down to cranking out the features identified at the project level.

In the Plan stage, we want to take the top priority feature (or possibly features if you have enough developers to roll off onto a second feature) and we want to find out what it means for this feature to be complete. In other words, the business, testers, and developers work together to flesh out the Acceptance Criteria in an attempt to answer the question: How will we know when we’ve got this feature ready to deliver? It is a time of getting down business rules, various requirements, etc. This is also a good time to create any design or documentation artifacts that will actually help you deliver.

In the Do stage, we write automated tests around our acceptance criteria (I guess you could argue this goes in “Plan;” I won’t fight about it), then produce the stuff that will make those tests pass. Web pages, controllers, databases. We create according to the Plan, no more and no less.

In the Check stage, we’re going to see if our feature makes our acceptance tests pass. This is also when we want QA and the user to give it the once over, although hopefully they’ve been checking out our progress before now.

In the Act stage, we either make adjustments to accommodate the feedback we got from Check, or we get this feature out the door and start on the next priority.

NOTE: Feedback should be ubiquitous and, as such, it’s hard to find a nice, neat place to put it in diagrams like these. It’s great to have user and QA feedback as you go, for instance. It’s good to have a peer confirm your direction before you get halfway into something that won’t work. It’s good to have regular meetings where the team reflects on their own performance and things to improve. Those things don’t really fit in at a specific project, feature, or unit level, but they should definitely be liberally sprinkled throughout.

PDCA at the code unit level

When we code a feature, we have to write units of code to deliver that feature. Controller methods. Domain object properties. Service APIs.

At the unit level, the Plan stage involves making sure that we’re writing code for the specific purpose of satisfying an acceptance test. If this isn’t the case, then we screwed up. Either we didn’t capture a necessary requirement, or we’re writing something nobody asked for. We also want to write the automated unit test that our code will support. This helps us both design the code and write less of it.

The Do is basic – write the code that makes your unit tests pass.

The Check is mostly that the unit tests pass. It’s also a good idea to have some level of peer feedback, whether you do this by way of an official review or whether it just happens naturally from pairing and swarming. Having someone look at your code after every single unit might be annoying, but you want some mechanism to get feedback.

Based on whether or not your unit tests pass (and the feedback you get from other developers), you may need to rework your code. If the tests are passing and your colleagues agree your code isn’t draining their will to live, then check it in. I recommend checking in at the “my unit of code is making its test(s) pass” level, which probably means several times a day. Then, start your next unit of code, which also should be supporting an acceptance criterion.

I haven’t really fleshed all this out or even thought critically about it, but there are some interesting ramifications for looking at things in this way. For example, some people feel automated tests are waste, but when we look at them through Deming’s eyes, we see that they’re actually a mechanism for reducing waste (which in this case means code that does not work or adds no value to the customer). We find also that testing is done early and often, which addresses what is often pointed out as the highest risk in a waterfall approach. We also see that, in order to limit waste, planning needs to be level appropriate. A comprehensive UML diagram is completely inappropriate before you start work on a feature, for example.

More on this as it develops.