Mar 042014
Agility 2008 May 04

Agility 2008 May 04 (Photo credit: Cynthia Blue)

Mere days ago, Mike Cottmeyer wrote an article called “Managing Risk and Uncertainty in Agile.”  I think the main ideology is captured in the following quote:

I think [treating software as inestimable] is a mistake. If we keep pushing to treat everything like R&D, without understanding the delivery context we’re working within, the whole agile movement risks loosing [sic] credibility with our executives. If you remember from our earlier conversations, most companies have predictive-convergent business models. We may want them to be adaptive-emergent, but they aren’t there yet. We’ll [sic] can talk about how to move these folks, but for now we have to figure out how to commit. (Emphasis mine)

In other words, since most organizations aren’t ready to be agile, we need to ditch agile presuppositions and practices that would make them scared or uncomfortable and treat software development like deterministic work.  He also explains how the metaphor of building a house should inform our understanding of agile software development and what an impact that metaphor has had on his own understanding and practice.  Ultimately, what he wants agile teams to do is give up-front project commitments and estimates.  He (rightly) anticipates that people will say this is not agile, but answers this objection with something along the lines of, “Yeah, so?”

I certainly agree with Mike that just because something doesn’t line up with Ye Olde Agyle Orthodoxye, that’s not a reason to reject it out of hand.  “That’s not agile” is actually not a valid criticism of anything – you have to go on to explain how the idea or practice would hamper agility or has some other harmful impact.  But my interest here is not to refute the practice of up-front project estimates, commitments, or explain the dozens of ways in which developing software is completely unlike a house and why that is a very wrong-headed and misleading metaphor for software development.

My interest is in those emphasized statements: the idea that we have to learn to cater to an organization’s established, outdated worldview and practices until they change into an agile organization – somehow.  This kills me, and it’s disappointing to see it coming from a company that purports to train people in agility and transform their organizations (who’s transforming whom, exactly?).

It’s like that John Mayer song, “Waiting for the World to Change.”  That song has the worst message, which is, basically, “There’s a lot of institutionalized power opposing good social change, so I’m just gonna quit trying until something changes that situation – somehow.”  When you try to shape agility into the old processes and expectations, that’s basically what you’re saying.

Agility is a challenging and transformative force.  It’s meant to create pain points.  It’s meant to challenge processes and how we think about creating value.  This is what’s supposed to happen.  If greater agility is difficult to manage with an existing budgeting process, guess what’s supposed to happen!  The budgeting process needs to be examined.  If greater agility is difficult to manage with your established project management practices, then you need to look at how your PM processes should change.  Agility is something that affects an entire system.  You can’t just agile up a development team, leave everything else intact, and call it good.

So, I call upon agile coaches, consultants, advocates, and well-wishers to challenge rather than accommodate.  Don’t look for ways to make agility more palatable.  That doesn’t mean everything needs to be uprooted all at once, but it does mean that greater agility should be pushing outward to change existing structures, not the other way around.

If you make agility work in the ways an organization is used to, then you’ve made it irrelevant.  You are an enabler.  You are part of the problem.  They have cancer, and you’re trying to get them to smoke a nicer brand of cigarettes.

If you think you won’t be able to get clients if you rock their boat too much, then do us all a favor and let the market naturally select you out of it so people who can actually help your clients can pick them up.  Nobody needs you to institute daily standups and a backlog and allow them to think they’re ready for unbridled productivity.  Some of you are CONSULTANTS for Thor’s sake.  CONSULT!

All of you who, even now, are thinking of how to change your practices around to fit better with older presuppositions forged in an older business environment – all of you who think agility needs to learn to deliver what project managers expect in order to be viable – all of you who think scaling agile needs more command and control – all of you who have never stood up to a client because it’s in their best interests – I call you to courage.  I call you to challenge those presuppositions instead of accommodating them.  I call you to war against high-risk delivery, false and harmful metrics, misanthropic management philosophies, and waste rather than collaborating with the enemy.

As you work with your organizations, ask yourselves, “Am I doing this because it improves the value delivery process, or am I doing this because it’ll work more easily with the way things are?”  In most organizations, if it isn’t subversive, it probably isn’t agile.

Enhanced by Zemanta
Feb 062014
Ed Sumerfield and Chris Nelson of Gaslight Sof...

Ed Sumerfield and Chris Nelson of Gaslight Software presenting “Why Agile Fails” (Photo credit: RubyNuby)

How the fuck is this coming around again?

George Westwater on Waterfall

Every so often, someone will put forth the idea that agile practices are great and all, but they fail at certain things like providing an up-front budget, a well-researched scope before work begins, a full project timeline with milestones, etc.  These sorts of things are such staples to the way most orgs approach their projects that there’s a certain tension to make agile practices better at providing them, and the path of least resistance to trying to supply these is to use the practices that used to provide them – typically some type of phase-gate approach like Waterfall.  You hear things like this:

  • We complete all the requirements and design up front, but then we do our development in an agile way.
  • We do agile, but in a way that can be managed with Microsoft Project.
  • We’re agile, except we do our testing at the end, but at least we release every six months.

And so on.  The basic idea is that doing a project without these traditional Waterfall additions is somehow inherently more risky or will fail to provide a company what it needs to make intelligent project decisions.

I’ve already spent a lot of electronic ink on this blog talking about how the benefits Waterfall purports to offer are largely illusory (kind of like the illusion of security offered by being employed at a large company) and how various agile methods can actually do a better job at providing intelligence to the business, so I don’t want to rehash all that.

However, the point I want to make here is that Waterfall and Agile arose from very different systems and operate off very different assumptions about the work.  For instance:

Waterfall’s Assumptions

  • The work to be done is mostly static.  Changes are rare and always occasion for re-examination.
  • Business needs / the market is not volatile; our priorities and methodologies today will be very similar three years from now.
  • The work is extremely similar to work that we have done before.  Almost everything is a known quantity.

Agile’s Assumptions

  • The work to be done changes often and progressively as more clarity is gained or needs evolve.
  • Business needs and the market is volatile.  The thing on the front burner today might be on the back burner in three months.  Our methodologies are evolving.
  • The work is to develop something new.  Similar technologies and methods might be used, but the business problems we’re solving technically have not been solved before.  Otherwise, we’d be using that.

The problems with Waterfall are not necessarily inherent to Waterfall as a methodology.  On paper, it looks all right.  The problem is that the set of circumstances under which it operates best is pretty rare in software development.  Are there software development projects where the domain is pretty static, changes are rare, and the business changes very little?  Sure, those do exist, and Waterfall would work just fine.  In fact, it or something similar might be the best choice for those kinds of projects.

However, a lot of organizations believe Waterfall’s assumptions to be true of their situation, and they are not at all.  Most real organizations have work that looks more like that list of Agile Assumptions, and in that context, agile methods will probably fare much better.

The issue with combining the methodologies isn’t a religious one; it’s that you’re trying to combine two things that are appropriate in two very different contexts.  You’ll end up crippling Agile by trying to make it more Waterfallish because you’re trying to drag in a context that doesn’t apply.  It’s like adding a sail to your car.  Sails are pretty good in their context, but they’re not objectively good for all contexts.

The fact is, becoming agile will challenge a lot of illusions and assumptions we make about our work that are not accurate.  If you’re hearing something like, “Agile makes it difficult to budget for the whole project up front,” that’s your cue to examine if budgeting a whole project up front is really the smartest thing to do as opposed to paying for value as you go.  If you’re hearing, “Agile makes it hard to set project milestones,” that’s a good opportunity ask yourself why you feel you need project milestones, what value they offer, and could you get that value in better ways?  Agile is meant to change your understanding, your values, and ultimately your practices at an organizational level.  It is not designed to accommodate them.

So, before you decide to add a sail to your car, just give driving the car a shot.  You might find that it doesn’t need that sort of improvement.



Enhanced by Zemanta
Jan 292014
Anecdotes and Data DSC_9869

Anecdotes and Data DSC_9869 (Photo credit: Plashing Vole)

I’ve started to see a statement crop up more and more in blogs, on Twitter, and in person on discussions about what does and doesn’t work in becoming agile: Anecdotes are not data.  Typically, this happens because someone is explaining their views or practices as derived from their own experience.  You’d think that wouldn’t be a controversial move, but agile coaching/consulting is one of the most armchair-dominated fields out there.  It behooves people trying to make money in this field to cast aspersions on experience in favor of their “data.”  You can tell story after story about what you’ve run into in the field, but it’s not “data,” and therefore can’t be used to establish anything.

Here’s the catch, though: where does that “data” come from?

Let me tell you where it doesn’t come from.  It doesn’t come from a researcher, an organization, or a group of peers coming up with a hypothesis, defining the control parameters to test that hypothesis, then running experiments on a broad range of companies within the confines of those parameters, observing and recording the effects every day, then publishing those results after peer review.  So, whatever agile consultants mean by the word “data,” they don’t mean what the scientific method means.  They don’t mean actual empirical data.

Since no one has ever done an actual scientific analysis of various agile practices, what do people mean when they talk about data in this field?  Well, they mean things like the Chaos Report or various organizations’ “State of Agility” surveys.  In other words, they are referring to collated survey data, and unfortunately for the anti-anecdote wags, a survey response is anecdotal.  People are reporting on their own experiences, and the way in which those experiences are reported is further filtered and conditioned based on things like the questions asked, the questions not asked, the answer options, the quantification mechanism chosen, the response rate, and the survey base.

So, data in the agile community is typically nothing more than a collection of anecdotes.  One might argue that these collections of anecdotes yield more certainty than, say, one person’s collected anecdotes.  Maybe that’s so, but at what point do anecdotes magically transform from “Bob’s experience” to “actual data?”  Is it five people’s anecdotes?  How about twenty?  A hundred?  A thousand?

The only difference between anecdotes and data in this context is one of degree, but not one of kind.  Critiquing someone’s anecdotes because they aren’t data is like critiquing a full bathtub because it isn’t a swimming pool – maybe you can’t fit everyone into it, but there’s still a lot of commonality.  All these various reports and surveys are just collections of stories.  That doesn’t make them invalid, of course, because these stories are really all we have to go on until a hundred corporations volunteer themselves for experimentation by the agile community.  But it does mean personal experience has both meaning and weight.

Even in what we think of as the hard sciences, the experiences behind anecdotes carry value.  Do you think a heart surgeon who has never done surgery is just as good as a heart surgeon who has done a hundred surgeries because they both learned the same material?  If the hundred-surgery surgeon started giving advice to the no-surgery surgeon about what the real deal is like, do you think, “Sorry old man, but anecdotes aren’t data” is a valid response?

If someone is being hidebound and closed-minded because of their past experiences, I think a much better observation to make is, “Your experience isn’t normative.”  In other words, people may have experienced different things that lead them to different biases, actions, and even different ways of interpreting “data.”  For instance, not too long ago, I criticized the popular use of burndown charts on the grounds of what I’ve actually seen happen.  I would never, however, tell someone that burndown charts are universally inappropriate or should never be used under any circumstance.  Why?  Because my own experiences have led me to see them as a low-value activity, but another practitioner might have experienced a lot of benefit from them, and it serves the community as a whole for us to share those anecdotes and the views they espouse so that we can improve and the people who bother to listen can also improve.

Anecdotes are data, and if you discount what an experienced veteran says because it didn’t jive with someone’s survey, you’re taking a very risky path.

Enhanced by Zemanta
Jan 282014

Galaga (Photo credits: Giphy)

Thought I’d pick up a little of that sweet, sweet physics traffic with that title.  FYI, you can increase your change in position or decrease your time.

Now that’s out of the way, this is a question that crops up every so often in different forms.  I put it in the Scrum form, but it all basically boils down to, “How can I get my developers to move faster?”  This is a timeless question that has answers that generally revolve around adding people, forcing overtime, or instituting measures to make sure they aren’t all secretly playing Galaga.  I will contend, however, that if you really want more to get done more quickly, you’re asking the wrong question.

When you first start learning to play tennis, they don’t teach you how to hit the ball as hard as possible.  You can always tell when your opponent has decided power is the most important thing to the game, because the ball ends up wedged in the fence, over the wall, out of bounds 80% of the time, and generally anywhere on the court except the places that would actually enable them to score points.  But, boy, does it get there fast!  Tennis Coaching 101 is to teach control and placement.  Power follows as a logical consequence of easily and confidently putting the ball where it needs to go.  A less powerful player with consistent placement will beat Johnny Randomcannon every single time.  The key to greater power is greater consistency.

Instead of asking, “How can we go faster?” start by asking, “What keeps us from performing consistently?”  If you don’t have a predictable flow, it’s very hard to know if you’re improving.  If speed increases, but it’s unpredictable, who knows why you went faster?  Who knows if you’ll be faster next week?  Your results don’t mean anything because the variability is high.  Although it may sound counter-intuitive or just plain distasteful, it is better to be slow and predictable than fast and unpredictable, at least in terms of getting work done.  Obviously, the ultimate goal is to be fast and predictable, but one comes before the other.  Just like you wouldn’t focus on power before precision, focusing on speed before predictability can end up with you all over the place except the targets you’re actually trying to hit.

In the first place, most businesses prefer predictability to speed, despite what they say.  You can plan around predictability, even if the predictions aren’t what you’d like.  You can’t plan off variability no matter how fast you are.  If you offered most businesses a choice between a consistent amount of work delivered in a consistent time period versus a completely erratic delivery that could be much faster from time to time, they’ll pick the first one.

In the second place, there’s no way you can meaningfully measure if your improvements are helping if you’re already highly variable.  To use a Scrum example (I feel so dirty), let’s say I deliver 5 story points one sprint, 18 the next sprint, and 10 the next sprint.  My team decides on an improvement, and the next sprint, we deliver 14 story points.  Did our improvement help?  You have no freaking idea.  If you take that same scenario, but the team’s deliveries were more like 12, 16, and 14, then the next sprint you delivered 18 or 20, the odds are pretty good that your improvement is helping, or at least not hurting.

But how do we reduce variability in software development?  Isn’t it extremely variable?  Well, yes it is, and I have found the practice of trying to get all deliverables to the same “size” to do more harm than good.  It’s better to accept that some things will be big, some things will be small, and most things will gravitate to somewhere in the middle, relatively speaking.  You can’t control that.  What you can control is flow.

How fast does new work come to your team compared to how fast work gets delivered?  How many things is your team trying to finish at one time?  How often is something in progress stopped because of roadblocks?  How often is something not in progress started because of whiplash priority changes?  You can’t control the variability of what you have to deliver, but you do have quite a bit of control over the flow of these deliverables into and out of your team.

If you manage to flow instead of speed, the speed will come.

Enhanced by Zemanta
Jan 222014
Meryl Streep

Cover of Meryl Streep

Do you remember that movie “The Hours” with Meryl Streep?  Me, neither.

Last week, I was talking with some colleagues on LinkedIn about the utility of estimating tasks in hours.  For those of you with a Scrum background, you know it’s kind of tradition to estimate user stories in story points (or something like them), but then you break them into tasks.  The tasks are estimated in hours, and the total of all those hours forms the basis of your burndown chart.

I’m not saying my experience should govern everyone else’s, but I haven’t estimated tasks in hours in a very long time.

First of all, the whole thing begins with the premise that we can accurately estimate our tasks in hours, and this tends not to be the case.  There’s a common misconception that, if you are trying to estimate a smaller activity, that your estimates will be better.  This is not true.  It is true that the actual hard units of deviation will be smaller (being off by 2 hours is a smaller hard time span than 2 weeks), but there’s no reason to believe your estimate will be better just because the scope is smaller.  You might say at this point, “Well, ok, but it’s not really going to affect the project that much if I think it’ll take 2 hours to set up some database tables, and it takes 3.”  That’s a 50% deviation, but ok, sure, that extra hour may not affect much in the grand scheme of things.  But what if you’re off by an hour for all your tasks?

That’s being generous.  What is pretty common is to have a task that you think will take 4 hours, and it ends up taking 20.  You take these extremely common incidents of being wrong and you put them together, and you can easily end up with days or even weeks of error per sprint.  The absolute units of time give the appearance of precision, but it’s a false appearance – perhaps all the more dangerous for its compelling nature.

Because the units of time are small, this seems like a small problem, but that’s only because you’re thinking of one task.  When you look at the cumulative effect of this problem, it’s actually pretty serious.  It sets unrealistic expectations.  Your burndown chart whose entire raison d’être is to visually represent sprint progress is like a clock that loses 5 minutes every hour – it doesn’t take long before that clock is worse than worthless, because it’s misleading.  Have you ever had a burdown chart stay level or go up during a sprint?  More often than you’d like, right?  Maybe that’s a signal that your workflow isn’t being managed well, but those same things can just as easily happen for no reason whatsoever, just because your hours remaining started wrong and are correcting themselves.  This isn’t even taking into account the fact that, when people see hours, they expect those mini-deadlines to be hit.  Patience for spending 20 hours on a task wears pretty thin when you’ve told everyone it would take 4, initially.

This leads me to my next point – estimating in hours takes a long time to do.  All that time spent on being wrong could have been spent coding, getting acceptance criteria, or anything that would have moved toward delivery.  Hell, even using that time to make the world’s longest paper clip chain would have been at least as profitable.  Note, I’m not saying it’s a waste of time to break stories into tasks.  That process helps you understand what’s expected of you, ferrets out oversights, and helps in swarming.  But what are you getting by putting hours on those tasks?

“Well, the hours can help us verify if our sprint commitments are reasonable.”

Can they?  First of all, even the Scrum Guide doesn’t use sprint commitments, anymore, but let’s pretend that’s still a good idea.  If you can make some kind of equation between story points and hours, then why not just skip the whole story point process?  Just break everything into tasks, get your hours, and base your planning on that?

“Well, relative measurements and velocity are more accurate planning tools than hourly estimates.”

Exactly.  So how are those hours helping you, again?  You are using the less accurate tool to critique the more accurate tool?  Is there anything at all those estimated task hours help you do that relative measurements or empirical metrics can’t?

If you’re a Scrum sort, try an experiment next sprint just to see what happens.  Break your stories into tasks, but don’t put hours on them.  If you want to burn down, burn down story points (or whatever you use).  Just try it.  See if your workflow or end result is meaningfully impacted.  See if you miss them.  You might find that you’ve made a significant improvement in your practice.

Enhanced by Zemanta
Nov 062013

Local staffing firm and winner of the Most Non-Ironically Orwellian-Sounding Twitter Handle Award tweeted this the other day:

All successful change initiatives start here


That’s a pretty bold statement for a company that’s never done a change initiative.  All successful change initiatives start at the top.

This idea is actually not unheard of.  You hear it other places, mostly as the result of disillusionment.  You can’t be an agile consultant very long without having the surreal experience of a company hiring you to help them change, then deciding they don’t really want to change all that much.  As agility begins to cascade upward and outward threatening all extant presuppositions and practices, they stand, Gandalf-like, bellowing that YOU SHALL NOT PASS and breaking the bridge before all that hubbub you’re creating in their developer teams gets to accounting or whatever.  There was one organization in particular where I felt they were paying me a very good amount of money for the privilege of ignoring me altogether.  Like I said, it’s a very surreal feeling, especially when people can and do ignore me for free.

However, the truth is that change that begins at the worker level can and does work.  In fact, my most successful “agile transformation” started with a development team, and once people saw the points being put on the scoreboard, the changes cascaded throughout the organization, changing their culture, levels of trust, rate of delivery – all kinds of good stuff.  I’ve also seen change initiatives start at the top and fail miserably because, even though management was behind it, none of the workers were.  It was something forced down on them, and they could get with the program or get out.  People were resentful at worst and apathetic at best, confident in the knowledge that this was management’s new fad, and next year it would be something different.  Change initiatives that start at the top without the full buy-in and participation of the ground up have a high risk for failure, and I speak from experience.

So, is it the case that all successful change initiatives start from the ground up?  That’s also incorrect.  Nobody can destroy a company’s agility like their leadership, and I’m going to go along with Deming here and say that the vast, vast majority of a company’s performance problems rest with leadership (ironically, vast amounts of time and money are spent on structures trying to pinpoint failures in the work force – “We need to hold our people accountable” is usually followed by an unspoken “because the problem sure isn’t me”).  Your pilot project can outperform everything in company history, and management can still shut you down.  It would be nice if good results were the final word in these kinds of decisions, but anyone who thinks data is the final word in an argument has never been in a long-term relationship.  So, change initiatives that begin at the roots level without the full buy-in and participation of management also carry a high risk of failure.

If change initiatives that start at the top are prone to failure, and change initiatives that start at the bottom are prone to failure, then what are you supposed to do?  Start in the middle?

The first part of the solution is, as Deming says, “The transformation is everyone’s job.”  You have to be willing to address the organization as a whole.  Agility has ramifications for everyone, and can actually be harmful if you only optimize a particular segment.  Can you imagine what would happen if you could make the front driver-side tire on your car go 50mph faster than the other wheels?  A similar thing happens to organizations where agility is only increasing in one place.  If you want to transform an organization, everybody has to know that what you are doing is going to affect everyone from the CEO to the night janitor.  Naturally, this can cause concern, and this leads me to my second part of the solution.

The second part of the solution is that organizations are resistant to change because of the way we try to make them change.  When we blame an organization’s agile adoption difficulties on their industry, culture, or people, we haven’t gotten to the root cause.  The root cause is that your way of changing the organization is setting it off the way a cold virus sets off your body.  “What the hell is this thing?” says your body, and the fever goes up to try to boil it out of you, the snot spigots unload to try to flush it out of you, and sometimes other unpleasant things that tend to happen at inopportune moments.  This is what organizations do to alien matter introduced into their system.  And believe me, brothers and sisters, I am preaching to myself as much as anyone on this point, as I can cite instances in my career when I was happy to blame the transformational difficulties on the company instead of the way I was going about things.

Yosemite Sam

Yosemite Sam (Photo credit: Wikipedia)

Change initiatives have to be done with a respect for people and in a manner that drives out all fear of losing their job (Deming, again).  This is the genius of Kanban: the changes are incremental, evolutionary, and the ideas and decisions come from the people doing the work instead of something imposed on them from the outside.  You can’t just go in like Yosemite Sam hootin’ and hollerin’ with pistols blazing.  When people start responding with resentment or fear, that’s your cue that you have done something wrong; it is not a cue that something is wrong with them.  What kind of misanthrope would you have to be to start thinking about getting rid of people who are actually doing you a huge favor by reacting negatively to what you’re doing?

When you start with where workers are at, and you give them the tools to visualize their work, and you help them define what they’re doing, and you help them measure how efficiently that work is being done, and you ask them, “So, what do you think we should do?” you are on the road to people being valued, giving you buy-in, and coming up with better solutions than you could.  Do you sometimes need to bring in ideas from the outside to get people started?  Sure, sometimes – but even then, it’s something you want them to own, not something they have to comply with.

Change initiatives from the top can work; change initiatives from the bottom can work, but both are risky.  Successful change initiatives start with everyone, and they are done by listening and building up your people instead of forcing it on them or getting rid of them.

Enhanced by Zemanta
Nov 052013
Industrial Engineering and Management building...

Industrial Engineering and Management building at the Technion in Haifa (Photo credit: Wikipedia)

I was talking with Lean/Agile Developer and overall smart guy Travis Dietz about requirements, and I brought up my old saw about requirements being inventory.  It’s an analogy that comes from the factory, which is where a lot of the ideas that inform Lean software development and Kanban come from.  Manufacturing was certainly Deming’s primary point of reference.  Principles, techniques, and even artifacts (control charts, flow diagrams) in this way of thinking most likely have their origins (or at least their ancestors) in manufacturing and industrial engineering.

How appropriate is this, really?

I mean, the Agile community has fought and is fighting hard to correct a misunderstanding about software development that we largely owe to NASA: the misconception that most software development projects are not fundamentally different from any other kind of building project.  You figure out what you want, an architect draws up all the plans, and the plans go to the workers to build out.  Because the plans have already been made, you can scale productivity just by adding numbers.  Also, past experiences should enable you to say how long a given development project will take if you can get a look at the plans.

In deterministic ways of building, this works all right.  If I have to make car doors, someone has drawn up the plans for a car door, and it’s my job to crank them off the line.  If my station averages 20 car doors a day, then when you ask me how long it’ll take to make a thousand car doors, I can give you a pretty good idea.  This environment – deterministic construction – is largely the environment much Lean thinking was generated in and is almost Deming’s whole world.

Software development, however, is non-deterministic.  Although you might have experiences that are similar at some level in the past, every problem is a new one.  Every bit of software you write has never been done before (because otherwise, you could buy it or just re-use the code).  Each feature is an act of problem solving and creativity, and we’ve all learned the hard way how quickly detailed plans become invalid.  Once you get to the battlefield, you have to adapt.   Software development is more like making a sculpture for a client than it is building them a car.  Given that the nature of the work itself is rather different than manufacturing, how appropriate is it to use optimizations from manufacturing?

Let me begin by saying that to dismiss insights from manufacturing because it’s manufacturing is pretty dumb.  Many advances in the field of software development did not come from software development, originally, and you should never close yourself off from wisdom that might come from a different field.  If you’re going to rule out every field that is different from software development, then you’ve pretty much ruled out every field, including the ones that you like and use all the time in your software development.  Math is a very deterministic field as well, but you’re going to have a hard time programming without it.

But the question remains, how far can we really take this?  For instance, I have pointed out some things from martial arts that I believe have relevance to software development, but I don’t think practicing takedowns will help us write better software.  I think the solution may be in the distinction between laws, principles, and techniques.

Laws are universal truths that everyone everywhere has to deal with.  For example, people have to sleep.  Therefore, no matter what field you’re in, you don’t want to schedule people for 24 hour shifts all month.  People can only be in one place at one time, therefore, we do not assign people to five separate teams and expect them to be fully available to each one all the time.  They’re laws built into the nature of the reality we all share, so dealing with these laws is automatically going to be something that spans all fields.  Any insight that comes from some fundamental law of reality is pretty much guaranteed transferable somewhere else.

Principles are also general truths, typically derived from laws, but they are not always universal to all places and all times.  “Avoid the Bubonic Plague” is a good principle, but generally speaking, it’s not really applicable to most people today.  On the other hand, we might consider a principle like, “Employees should be empowered with the tools and authority they need to do their work efficiently.”  Although you could probably find an edge case or two where this didn’t apply, it pretty much applies everywhere.  Maybe someday when we live in a telepathic hive mind, concepts like “employee” and “authority” will no longer serve a purpose, but at least in our current context, it’s a pretty good principle and relevant to all kinds of work.  When applied to many different fields, you see good results.  I’d say that the vast majority of the concepts and recommendations of Lean, Kanban, and the various thought leaders in that area operate at this level.  You have to understand how the principles apply in your context, but they are more or less truths that are bigger than a specific field.

Techniques are specific implementations/practices of something, usually derived from principles.  These are only valid insofar as they operate in accord with their principles and are usually heavily conditioned by specific context.  For instance, in most of the apostle Paul’s letters to churches, he tells them to greet each other with a holy kiss (Rom. 16:16, 2 Cor. 13:12, 1 Thes. 5:26).  In the first century near east, a chaste kiss was a culturally standard form of warm greeting.  It was a way to show that you were a community and had a level of affection for one another.  In modern, American churches, you are likely to be maced when trying this out.  In our time and culture, a kiss has romantic connotations, or at the very least suggests a particularly close affection like a relative or a very old and dear friend.

In modern America, the church members might shake hands, hug, smile, slap on the back – something that signifies warmth and community to us.  The principles of community and affection span both times and cultures, but the specific implementations of those principles are very culturally conditioned, and if you went around to American churches trying to kiss everyone because “Paul told you to,” someone would rightly argue that you had confused the principle of Paul’s instruction with the cultural particulars, once they had slapped you.

It is in this area where, in software development, it is appropriate to go through these other industries at the practice level and decide if they’re really helpful in our context or not and, if so, do they need to be modified to better suit non-deterministic work?  One good example is control charts.  We (software developers) generally do our control charts based on probabilities rather than standard deviations, because we don’t have a consistent completion time we can use.  The principle is the same – we are measuring our throughput so we can make informed projections, offer our customers service level agreements, and have conversations about things that look out of whack – but as for how that actually gets done, you can’t just lift the practice out of Toyota and drop it onto a software development team.  You have to figure out if the practice will actually help you embody the principle, or whether it might need to look different or be scrapped altogether.

So, I will continue to borrow insights from manufacturing.  That environment is much older than software development, and they have discovered principles about work, flow, quality, and how people perform their best that we’d be idiots not to consider just because it came from manufacturing.  But when it comes to practices (and principles to some extent), we need to make sure we are thinking critically about the nature of non-deterministic work and the best way to embody those principles for us.

Enhanced by Zemanta
Oct 222013
Español: Socarrats Futbol Team

Español: Socarrats Futbol Team (Photo credit: Wikipedia)

All around enlightened guy Bob Marshall told me that he thought my last post was confusing.   What’s worse is that Bob is pretty familiar with the various elements and issues that circulate around Lean/Agile transformations, so if he thought it was confusing, then it’s probably just one notch up the food chain from gibberish depending on where you’ve been with this issue.

This is my attempt to be clearer.

Most organizations, when they transition to something more agile, adopt cross-functional teams as part of that initiative.  For those who don’t know, a cross-functional team is a team consisting of people from various “departments” who work together directly to deliver value.  In software development, this means that, instead of having a group of people gather requirements in this team over here, this group of people who do UI/UX over here, this group of people who do database stuff over here, this group of people who do QA/Testing over here… instead of all that, your teams are composed of one or more people who each contribute collaboratively to delivering an increment of value.

So, the UI/UX team, database team, and so on are replaced by Team Jedi Grizzlies which consists of a business analyst, a UI/UX person, 3 programmers, a database person, and a QA person (or something like that – you get the idea).  Then you have Team Frenzied Waffle Irons over here with a similar composition, and so on.  Instead of having teams siloed by job function, specialty, or skillset, your teams are cross-functional – they are interdisciplinary task forces that work together to deliver value incrementally.

In a football game, you don’t have all your linemen in one group and all your quarterbacks in another group and all your safeties in another group.  Instead, you have an offensive group, a defensive group, and special teams – all consisting of players with different skillsets and specialties necessary to score (or prevent it).

The first challenge to an organization’s culture is, first of all, the concept of team.  Most of our “teams” are really individuals that share a skillset or sit in the same part of the building, but they don’t depend on each others’ work.  We’re like individual silos.  So, like the gentleman in New York, one of the first hurdles to overcome is the idea that we actually need to work in teams.  Organizational success can’t afford to rely on a series of individual heroic efforts.  More throughput, higher quality, greater morale, less stress, more trust – these are the kinds of things that come from teams.

The second challenge is that the teams are cross-functional instead of being separated by function.  Most orgs are well-used to a siloed model and find a cross-functional model alien (and often impractical at first) to their way of thinking and doing.  A lot of procedural infrastructure has probably grown around the siloed model, complete with elaborate schemes of checks and balances to govern the hand-offs.

Having a team consisting of people with all different skill sets working together to deliver something, however, is disruptive.  It’s a huge growing pain, but the culture changes and consequent productivity changes are astounding.  Nonetheless, there are many things about the way you work, the way the company as a system operates, and a whole host of logistical issues that go into forming cross-functional teams.  But one of the key things that has to happen is the breakdown of traditional divisions, and each member of the team thinking of themselves as offering a service to the team – almost like a mini-business with the rest of the team as a customer.

The third challenge is primarily what I wanted to deal with in the last post, although I touched on the others.  As this process happens, inevitably the question comes up of how much crossover is appropriate among the team members.  It just so happens that, right now, there’s not a lot of database work.  So what’s your DBA team member supposed to do?  Could she be a member of another team and split her time?  Yes, that’s a solution that can work.  There are drawbacks to that, but that may be in everyone’s best interest.

But another solution is to have your DBA help with some other aspect of the team, like QA or coding or helping work through acceptance criteria or doing research for the next round of features or talking with customers to show them what you’ve got so far and ask if you’re on the right track.  To go back to the football analogy, if a cornerback intercepts a pass, he doesn’t drop it because scoring “isn’t his job.”  No, he runs with it as far as he can.  If he scores, great!  If not, great!  It was an amazing act of assistance that brought the team further.

Once we have bought into the idea of everyone helping everyone else out, the issue naturally arises of how far you take this.  Should your UI/UX guy learn programming?  Should your business analyst learn professional testing skills?  Should your QA person learn how to work on the UI?

And this is where it can get confusing because there is no one-size-fits-all answer that will apply to every team and every organization, so what I offered is that a focus on how best an individual can serve the team will begin to suggest these areas.  So, rather than everybody learning to do everything, look for the value you can offer to the team and see what opportunities that creates.  If you’re the DBA, for example, and you find that you frequently get done with the database stuff early, but your QA person is consistently swamped, then you might learn how to run test scripts or create automated tests.  If you’re a programmer, and you consistently wait for mockups from UX, then create a basic one in their tool or in HTML or on a napkin and start collaborating with them to help get the ball rolling.  The possible combinations are as unique as a team is.  I offered a way I like to give people on the team exposure to different areas.  The developer may not be a UI expert, but they can do in a pinch.  And so on.

But what I also wanted to get across is that you don’t have to sweat this principle.  Everyone does not need to be alarmed that they have to get into all kinds of additional areas in-depth in order to have an agile team.  By all means, keep working on the things that motivate you, but don’t forget that your role isn’t to become the world’s most productive programmer – it’s to help your team deliver value.  Sometimes, that means doing things outside your box.  Uncomfortable things.  Unpleasant things.  Horrific things.

Like test scripts, for instance.

Enhanced by Zemanta
Oct 212013
English: Jeff Webb, wide receiver with the Kan...

English: Jeff Webb, wide receiver with the Kansas City Chiefs. (Photo credit: Wikipedia)

The latest work in the How to Run an Agile Team from People Who Have Never Run an Agile Team genre came out last week, here, and opens with the following:

Many people in the Agile Community tout Strength Building as the best way to succeed. Strength Building suggests an Agile group should concentrate on exploiting and maximizing their strengths versus shoring up their weaknesses, assuming it is easier to gain productivity by doing more of what you are good at (i.e., head-down, following a scripted process) versus trying to become good at the things you are not (i.e., understanding and maximizing the human component in this sea of change). [emphasis mine]

The article claims that not only is this an idea that exists, but many people in the agile community say this is the best way to succeed.  I have never heard anyone claiming such a thing in my life, and I asked over on the blog if they could point me to any sources that actually claimed this.  They deleted the comment, so I’m assuming this is more drummed up marketing BS to give the appearance of credibility to a group of people who are about as agile as John Goodman on a flight of stairs.

But this is not another rant about frauds and wannabes claiming to be agile consultants in Kansas City.  Rather, despite the fact that the article’s claims are bogus, there is a real issue living in the same neighborhood with organizations trying to become agile that actually does come up a lot in actual experience.  That issue is: how far do we take the cross-functional team thing?

It’s a problem for organizations for a number of reasons.  Traditionally, organizations have been siloed along the lines of skill set or specialty.  Here’s accounting.  Here’s legal.  Here are the executives.  Here are the data enterers.  Here are the customer service reps.  Here are the programmers, and so on.  People are used to a workflow that revolves around these skill set centers processing a work item and handing it along to the next one.  Like many challenges in increasing agility, it’s entrenched in any number of processes and, ultimately, has it roots in certain ways of thinking.  So, there’s a lot of knot untying.

Also, even under the most open and collaborative of circumstances, people have their own strengths and preferences.  If I’m a QA analyst, should I learn to program?  What if I don’t want to?  If I’m a DBA, should I learn HTML?  How cross-functional do you have to be?  I even remember a very bold and honest declaration from a DBA in New York when I was consulting with ThomasNet.  He said, “I got into this job to work with machines, not other people.”

Obviously, there are some other issues there, but it illustrates the challenge.  People like what they like and don’t want to get into things they were never interested in.  So, what do you do?   Do you drag them kicking and screaming into areas they don’t want to work in?  Is this how we show respect for people?

No.  But also, yes.

I hope that clears it up for everyone.

One thing that I had to make very clear to my New York friend is that the days of of people working in isolation – especially in knowledge work and soft production – are dead or dying and probably never should have been given life to begin with (thank you, NASA).  When you compare the speed and quality of a highly-siloed, atomically isolated organization with a cross-functional, highly-collaborative one, the contrast is stark.  That doesn’t mean you have to be a “people person,” but it does mean that you have to recognize that organizational and personal success will depend heavily on a smoothly running team effort as opposed to individual heroics – the latter being an unsustainable model.

Ok, so we have to collaborate.  Great.  How much of Cathy’s job do I have to learn?

Probably not much.  The idea behind cross-functionality is not that everybody can do everything the team might need.  The idea of cross-functionality is that everyone who can add value to the work is working together so that handoffs are reduced, perspectives are shared, knowledge is held in common, and the system isn’t all kinked up by silos.

Now, the more individual team members learn about what other skill sets bring to the mix, the more that individual can contribute.  A QA analyst might not learn to program, but she might learn some basic HTML or JavaScript to be able to communicate her findings in more detail or perhaps even fix small errors.  A developer might never make a good QA analyst, but he might be able to run some test scripts.

The idea of cross-functional teams isn’t that everyone needs to be experts on everything or even novices at everything – it’s the philosophy that you as an individual provide a service to the people who need it (your team), and just as if you ran the business yourself, you’re going to do the things that allow you to deliver the most value.  If that means learning some things outside your comfort zone, then do so.  If it means making yourself more available, then do that.  If it means everyone sitting together with their teams instead of their separate departments, then do that.

One thing I like to do with my teams is to pick work items that allow us the luxury of getting someone out of their box.  Maybe on this feature, the database guy works on the UI.  Maybe on this other feature, one person does the development while the other developer runs it through a rigorous QA process.  We find ways to give people the opportunity to get experience outside their comfort zones so at the very least we begin to share some understanding (both existentially and facts about the job) and, in many cases, we’ve created future opportunities for that person to help when that area needs extra love.

That’s all kind of a long way to say this.  Don’t feel like, in order to be agile, everyone on the team needs to cross-train in everyone else’s job.  Really, the best advice I can give about being a good cross-functional team is for each person on that team to think of themselves as a little business having to compete for and serve their customers.  In the process of doing this, certain areas of overlap will organically arise, and when those opportunities arise, you seize them.  None of this “this is so and so’s job, not mine.”  It is your job to help the team deliver value.

Wide receivers are not running backs, but sometimes a wide receiver will pick up a fumble or take a shovel pass and run.  Defense is not offense, but the Chiefs are doing pretty well this year because our defenders will score points as opposed to dropping an interception and going, “This is really offense’s job.”  But free safeties don’t train to be wide-receivers, either.  That alchemy of everyone doing their personal job well in conjunction and alongside very different areas of specialty as well as the willingness to do something outside your comfort zone if it’ll bring your team success is what you’re looking for.

Enhanced by Zemanta
Oct 082013
organizational culture

organizational culture (Photo credit: nchenga)

In survey after survey, the #1 reason agile practitioners cite for the failure of an agile effort is that the company culture didn’t support it.

That’s probably true, but I think we might be letting ourselves off the hook, here.  For an agile consultant to say the failure of the initiative was due to the company culture is sort of like saying that a patient’s cancer treatments would have been effective if it weren’t for that cancer.  Or making a PSA to announce that death is the #1 killer in America and we should all take steps to avoid it.

In the interest of transparency, I have been guilty of this more than once.  I’d go to a place, get an uncharacteristic amount of resistance (it’s a weird phenomenon that organizations will sometimes pay for consultants and then ignore them), and after a certain period of time, decide that agile just wouldn’t work there because of the culture.  If I’d been really honest with myself, the truth would have probably been closer to, “I’m not wise and experienced enough to know how to handle this situation,” or maybe even, “I don’t want to put in the amount of effort and longsuffering it will take to weather this situation.”

A dysfunctional culture is exactly one of the things greater agility is supposed to challenge and gradually alleviate.  The kinds of things you do to empower people to turn from victims to self-improvers are the kinds of things that begin to dissolve cultural problems.  Command and control cultures begin to thaw when they see the gains that a high level of self-direction gives.  Trust and cooperation begin to build among traditionally hostile departments when teams become more dependable and transparent.  On and on.  This isn’t just me waxing hypothetical, either.  I have personally seen this cultural transformation several times at the hands of a burgeoning agility.

Only trying agile initiatives at companies who have the culture for it is like testing your hangover remedy with teetotalers.  A company culture that’s ideal for agile is inevitably going to be agile all by themselves.  You might still add value in helping them optimize here and there, but you won’t have substantially improved that organization.

It is the entrenched waterfall, hierarchy out the wazoo, big planning up front, command and control, whirlwind priorities, siloed organizations that need the gentle erosion of improvement, agility, and optimization the most.

Enhanced by Zemanta