Category Archives: Management of Uncertainty

Management of Uncertainty in Agile Projects

One of the biggest inherent advantages of an Agile approach is the management of uncertainty in Agile projects. Agile is based on an empirical process approach – the word “empirical” means “based on experiment or observation”. When you use an empirical process approach, you accept that you don’t know everything you might want to know about a project before you start and the process is designed to adapt both the solution and the process to discover the solution to learning as the project progresses.

A “Defined Process” is repeatable and doesn’t change significantly from one project to the next and is very predictable in the results it produces while an “Empirical Process” is specifically intended to favor adaptability over predictability. For that reason, an empirical process is much better suited for a project with high levels of uncertainty.

A very powerful concept for understanding uncertainty is the “Stacey Complexity Model” which is shown below:

Stacey Complexity Diagram

There are two dimensions of uncertainty in this model:

  • One dimension is requirements uncertainty – How well understood are the goals and requirements for the project and how certain are the customers that they know what they want?
  • The other dimension is technology uncertainty – How well understood is the technology solution to the problem and what is the level of risk associated with the technology solution?

This is a very important concept because the ability to handle uncertainty is so important in today’s most critical projects and heavily plan-driven projects are not well-designed to deal with high levels of uncertainty. What typically happens in a plan-driven project is the project manager tries to reduce the level of uncertainty to an acceptable level before starting the project by:

  • Trying to resolve any uncertainties in the requirements as much as possible before the project starts, and
  • Trying to eliminate as much technology risk as possible

This often results in using tried-and-proven technology and doesn’t push the envelope too far in terms of going into areas of new and undefined user requirements. The downside of that, of course, is that the technology approach may wind up being obsolete within a relatively short amount of time after it is released and it may also result in a very mediocre user experience with the solution.

Let me clarify what I mean by “managing uncertainty”. To some people, uncertainty is like a cancer that attacks a project and can cause it to fail and conventional project management thinking has reinforced this approach to reduce the risk and uncertainty in a project as much as possible. I don’t see it that way uncertainty can also represent opportunity to go beyond what is expected and the value a project produces is many times directly related to the risk and uncertainty in the project. If you force a project to fit into a plan-driven model by reducing the risk and uncertainty, you may be maximizing the predictability of the project to meet cost and schedule goals but minimizing the value that the project produces.

That doesn’t mean that uncertainty should be ignored and not managed at all and fortunately, this is not a black-and-white, either/or decision between a totally rigid, plan-driven approach with almost zero uncertainty and a totally adaptive approach with an extremely high level of uncertainty. The right thing to do is to fit the approach to the level of uncertainty in the project rather than force-fitting a project to some kind of canned approach (whatever it might be). It takes more skill to develop an intelligent approach for managing uncertainty – it requires:

  • The ability to objectively assess the level of uncertainty in a project
  • An understanding of both empirical and defined process models
  • A deeper understanding of the principles behind these approaches to know how to blend the two together to fit the situation

The ability to do that is the essence of what I believe is an effective Agile Project Management approach.

Learning to Live with Uncertainty with Agile

Learning to live with uncertainty with Agile as well as risks is probably the single-most important factor in the success of any project (Agile or not). Prior to Agile, there may have been a false sense of security from thinking that you could completely eliminate or significantly reduce uncertainty by developing a very detailed requirements document for a project upfront and then managing changes against those requirements to control the scope and direction of the project as the project was in progress. That sort of approach is often taken in order to gain some level of predictability over the costs and schedule of a project; however, it turns out in many cases that a good deal of that feeling of security may be an illusion and it is very difficult to accurately predict all the requirements, costs, and schedules of a software development project.

With Agile, in many situations, the pendulum has swung almost completely in the opposite direction – a lot of people in the Agile community seem to feel that any attempt to try to reduce uncertainty upfront in order to estimate the costs and schedules of a project is totally futile because there might be a lot of uncertainty involved and the requirements are only going to change anyway. For that reason, there is a tendency to not do any significant amount of upfront planning at all. I think that’s an over-reaction and it can lead to significant problems.

Here’s an example of the impact that it can have…I was recently involved in helping to rescue an Agile project that had gone on for about two years with no end in sight. One of the basic problems was that the project had started without a sufficient level of upfront planning to assess the scope of the effort and it was just assumed that it could be completed in some reasonable amount of time by a single agile team. After about two years into the effort, we stepped back and did some re-planning and found that the scope of the effort was much larger than a single Agile team could accomplish in any reasonable amount of time. If more upfront planning had been done prior to the start of the project, perhaps that probably would have been discovered much earlier.

There are two primary problems I have often seen in this area:

  1. Analysis Paralysis – Delaying making commitments or delaying making any decision at all until the information required to make the decision has a very high level of certainty about it and all of the risks associated with a project are well-known and completely understood. This is often associated with the traditional, “Waterfall” style of project management. Unfortunately, this mode of operation has been the norm in some companies that are very control-oriented and insist on accurately knowing the cost and schedule of a project before making a significant commitment to it.
  2. Cavalier Approach – The opposite of that can be to take a very “cavalier” approach to not worrying about managing uncertainty and risk at all, just start doing a project with very little or no upfront planning, and assume that whatever uncertainty and risk is inherent in the project will be discovered and somehow work itself out as the project proceeds. This is the approach I’ve commonly seen on a number of Agile projects.

Neither of those approaches is ideal and each can lead to significant problems – a better approach is to make informed decisions as best you can based on the level of risk and uncertainty in the situation. Very few decisions in life are based on either 100% known or 100% unknown information and learning to deal with uncertainty is a skill that any good Project Manager learns over time. That skill is still very important in an Agile environment and shouldn’t be ignored. (In other words, don’t throw the baby out with the bath water when you convert from a traditional project management approach to a more Agile approach)

You often have to make informed decisions based on very uncertain information because waiting for the information to become more certain might significantly delay the project or might not make sense at all. The approach that seems to work best is to separate the “known” from the “unknown”, make some hypotheses or assumptions about the unknowns if necessary, and then evaluate the risks of going forward based on those assumptions or hypotheses. If you do decide to go forward, you shouldn’t forget that those assumptions were only assumptions and may need to be revalidated later on. (Many people often forget to do that) In a traditional project management environment, a risk register or something equivalent to that might be used for that purpose – that level of formality may or may not be necessary in an Agile environment, but the principle is nonetheless important.

There’s an art and a skill associated with making decisions in an uncertain environment. Anyone who has done any gambling where you have to bet on the outcome of an unknown event (like what card you’re going to be dealt next) will understand that. Here’s an example: most people are familiar with the game of “Blackjack” where the players play against the dealer and both try to win by getting the highest possible hand without going over 21. Suppose I have been dealt two cards and have a total of 16 points in my hand and the dealer is showing one card as a “10” with the second card face down. There are several very significant things I don’t know:

  1. I don’t know what the dealer’s second card is because it is turned face-down
  2. I don’t know what card I might be dealt next

Both of those are significant unknown’s but I have to make a decision anyway. I can make an assumption that the dealer’s second card is either a “10” or a face card (Jack, Queen, or King) for a total of 20 points. That assumption seems to make a lot of sense because there are more ways for that to happen (Ten, Jack, Queen, or King) than any other individual card. If I make that assumption, I know that my 16-point hand is going to lose against his 20-point hand and the best course of action would be for me to take a hit and risk going over 21 because I’m going to lose anyway if I don’t take a card or I don’t do anything at all.

That’s an example of an informed decision – I separated the known’s from the unknown’s and then made a reasonable assumption about the unkown’s based on the risks I saw in the situation. The alternative would have been to make no decision at all because the unknown’s were too significant or to make a “shoot from the hip”, uninformed decision to do something without doing any real analysis or really thinking it through at all and without taking advantage of the facts I do know in order to make a decision. I think that one of the most important factors for doing that effectively is to make an honest and objective assessment of the situation. There are several errors I have seen often with this:

  1. Overestimate the level of confidence in what is known and make a poor decision based on incomplete information
  2. Make an assumption about incomplete information, make a decision based on that assumption, press forward, and then forget that it was an assumption and never retest or revalidate it later
  3. Ignore whatever known information is available for making a decision and just proceed forward based on an uninformed decision without taking advantage of what is known