Tag Archives: Level of Uncertainty

Why Is Planning Difficult? Is It a Waste of Time?

Why Is Planning Difficult?

Why is planning difficult? A lot of people seem to think that planning is very difficult and some think it is a waste of time because even very well-thought-out plans often don’t work out as expected. This issue is important because it is at the crux of selecting a methodology and planning approach for a project. Do I use Agile or plan-driven (aka Waterfall)?

Why Is Planning So Difficult?

Many people seem to think of planning as an “all-or-nothing” proposition – either you develop a highly-detailed plan or you do no planning at all. I don’t believe that to be the case. There seems to be two major problems associated with why people have this difficulty with planning:

Dealing With Uncertainty

Many people seem to have difficulty dealing with an uncertain environment – they want things to be crystal-clear, black-and-white and, in an uncertain environment, they think that it is a waste of time to do any planning at all.

Unrealistic Expectations

A related factor is that many people develop unrealistic expectations about planning.

  • If you develop a well thought-out plan, they expect that it should work every time.
  • Many people are also unrealistic that everything about a project will go absolutely perfectly all the time and
  • Murphy’s Law often contradicts that belief.

Planning Fundamentals

Let’s review some fundamentals about planning. We can learn a lot about how the military does planning.

Planning in the Military

Here are a few of my favorite quotes on planning in the military:

Plans are nothing; planning is everything”  – (Dwight D. Eisehhower)

What Eisenhower is saying is that documented plans should not be considered to be sacrosanct. When people document a written plan, the plan often takes on a life of its own and it becomes the “gospel” that everyone is expected to follow rigidly. However, that should not be the case and that’s not a reason to not do any planning at all. Here are a couple of examples:

  • It would be foolish for a military unit to go into battle without doing any planning at all. You should take advantage of whatever intelligence you might have about the enemy positions (as uncertain as it might be) but you shouldn’t lose sight of the uncertainty in that information.
  • In an American football game (or any other sport you want to choose), the coach prepares his team for what he thinks the strengths and weaknesses of the opposing team are; but, again, that planning is based on very uncertain information and lots of assumptions.

Here’s another quote along the same lines:

“No battle plan survives contact with the enemy”  (Helmuth von Moltke the Elder)

What Helmuth von Moltke is saying is that you shouldn’t expect that a plan won’t change at all once you start to execute it.

An Example

In World War II, Churchill and Roosevelt spent years planning the invasion of Europe that ultimately resulted in the Allied forces landing in Normandy on June 6, 1944. Both men knew that an invasion of Europe was the only way that the war could be won but there was a lot of uncertainty about:

  • Where should the invasion take place? How can the Allies preserve the element of surprise?
  • What’s the best way to do it? What are the odds of it being successful?
  • How many deaths and casualties can be expected?

This was a huge decision that had enormous impact:

  • The fate of Europe and most of the free world at that time depended on this being successful
  • Both men faced some level of opposition at home knowing the large number of casualties to be expected

It took a lot of courage to do this planning and make a decision under these circumstances knowing that there was a lot of uncertainty in the situation and the impact of the decision was so enormous. However, planning was essential – can you imagine what might have happened if the invasion was attempted without any planning at all?
Source: Churchill and Roosevelt Spent Years Planning D-Day

What Does This Have to Do With Agile?

This whole issue about planning is at the crux of the controversy about “Agile versus Waterfall”. Many people see the choice between Agile and Waterfall as a binary and mutually-exclusive choice and planning is at the center of this controversy:

  • Some people in the Agile community might say that its a waste of time to do planning in an uncertain environment and you should just use an Agile approach and let the project plan evolve as the project is in progress
  • Some people in the traditional plan-driven project management community might say that it would be foolish to attempt to do a project of any significant scope without a detailed plan for it to be successful

The truth is somewhere between those extremes – it’s not an “all or nothing” choice and you should adapt the methodology and the level of planning to fit the nature of the project. The level of uncertainty in the project is a big factor in making that choice. In an uncertain environment, it might be foolish to attempt to develop a highly detailed plan but uncertainty is never an absolute and it would be equally foolish not to take advantage of whatever information you might have.

How Do You Put This Into Practice?

A lot of people might see “Agile” and “Waterfall” as totally incompatible with each other. It’s like mixing oil and vinegar – they just don’t mix well. And, if you look at this from a mechanical process perspective, that might be true.

In order to learn how to blend these two approaches together in the right proportions to fit a given situation,

  • You have to understand both approaches at a deeper level and get past some of the strong stereotypes that exist about both “Agile” and “Waterfall”.
  • It’s really a choice of selecting a more plan-driven approach or a more adaptive approach based on the level of uncertainty in the situation and there is a whole spectrum of alternatives as shown below:
Levels of Agility

Rather than thinking of “Agile” and “Waterfall” as binary and mutually-exclusive choices and trying to force-fit a project to one of extremes, a better approach is to fit the methodology to the nature of the project and the level of uncertainty and the planning approach is a major factor in making that determination

Overall Summary

Here’s a summary of the key points I want to make in this article:

  1. Planning is not an “all or nothing” proposition
    • The planning approach should be directly related to the level of uncertainty in the environment
    • Planning in an uncertain environment can be difficult but that should not be an excuse for not doing any planning at all
    • However, to quote Eisenhower, “A plan is nothing – planning is everything” – don’t get locked into thinking that a written plan is sacrosanct and doesn’t need to be changed to adapt to the level of uncertainty in the environment
  2. The level of uncertainty and the planning approach is probably the most important factor in selecting a project management approach for projects
    • Rather than thinking about “Agile” and “Waterfall” as binary and mutually-exclusive alternatives,
    • Its more accurate and more objective to think of a spectrum of alternatives from heavily plan-driven at one extreme to heavily adaptive at the other extreme

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

How Do You Choose the Best Methodology for a Project?

I often see questions on “How do you choose the best methodology for a project?”

Choose the Best Methodology for a project

Is There a Best Methodology for a Project?

I don’t think that there is one single “best” methodology for a project that works for all projects.  A lot of people make the mistake of force-fitting a project to some standard methodology.

Traditional Plan-driven Approach (“Waterfall”)

Some project managers will try to use a traditional plan-driven methodology for a project (what many people loosely call “Waterfall”):

  • It may be the only project management approach that they know.
  • It has also been widely-accepted as the only way to do project management

Force-fitting a Project to a Methodology

Many other people have the misconception that there is a binary and mutually-exclusive choice between “Agile” and “Waterfall”. As a result, they will attempt to force-fit a project to one of those extremes

Fit the Methodology to the Project

The “best” approach is to go in the other direction and fit the methodology to the nature of the project. 

  • That takes more skill but it definitely can be done
  • You should also recognize that there is not a binary and mutually-exclusive choice between “Agile” and “Waterfall”
  • There is a whole spectrum of different approaches ranging from heavily plan-driven at one extreme to heavily adaptive (or Agile) at the other extreme
Range of Agility

Factors to Consider

There are a number of factors that influence the selection of the best approach for a particular project:

1. Level of Uncertainty

First and probably the most significant factor in choosing an approach, is the level of uncertainty in the project. A project with a high level of uncertainty would be best-suited for a more adaptive (Agile) approach. Attempting to force-fit such a project to a traditional plan-driven project management approach could be disastrous.

  • It would force you to make a lot of assumptions to try to resolve the uncertainty; and, in many cases, those assumptions may be wrong and require a lot of re-planning and possible re-work
  • The emphasis on planning and control in a traditional plan-driven project is not conducive to changes. That will make it difficult to maximize the value of the solution in an uncertain environment

2. Need for Creativity and Innovation

Next, in today’s competitive environment, creativity and innovation can be very important to create very competitive business solutions. An approach with a heavy emphasis on planning and control can stifle creativity and innovation.

3. Customer Relationship

Finally, managing customer expectations is probably one of the most critical aspects of any project.  If the results of a project are not consistent with customer expectations, the project will likely not be viewed as successful no matter how good you think it is.  The nature of the customer relationship can range between:

Contractual Style of Relationship

A contractual style of relationship is where there are very definite and well-defined customer expectations that must be met.  In this type of relationship, the customer may not take any responsibility for the success of the project.  The customer defines requirements and expects the project team to do whatever is necessary to meet those requirements. In this style of relationship, there is limited participation by the customer in the project

Naturally,  the contractual style of relationship is well-suited to a project with a relatively low level of uncertainty. It must be possible to define the customer’s requirements in some level of detail prior to the start of the project.  It would be much more difficult to make a contractual-style relationship work in a project with a high level of uncertainty.

Collaborative Style of Relationship

A collaborative style of relationship is where there is a shared responsibility for the success of the project. In this environment both the project team and the customer take an active role in defining the direction of the project as it is in progress

What’s the Right Style of Relationship?

Of course, the customer has to be amenable to whatever type of relationship you choose.  If the project has a high level of uncertainty, that would lean towards more of a collaborative relationship; however:

  • The customer has to be open to that kind of relationship for it to be successful. 
  • This is a big problem in many companies where it is difficult to break down organizational boundaries between organizations
  • It requires establishing truly collaborative relationships based on a spirit of shared responsibility, trust, and partnership.

3. Project Team Capabilities

The final major factor in selecting a project approach is, of course, the capabilities of the project team.   

  • An Agile approach requires a lot of training and skill and a hybrid Agile approach  can require even more training and skill. 
  • Naturally, it does not make any sense to choose an approach that the team is not capable of implementing.

Why Is Choosing the Best Methodology for a Project Important?

There are two major factors that require us to broaden the way we think about “project management” today:

  • Solutions tend to be much more complex and the level of uncertainty is much higher
  • Competitive pressures frequently require much higher levels of creativity and innovation

For those reasons, force-fitting all projects to a standardized plan-driven approach is not necessarily the best approach for all projects.

Overall Summary

Choosing the best methodology for a project can be a difficult thing to do. It is not necessarily a simple matter of choosing Agile or Waterfall. You have to fit the methodology to the nature of the project. A project should be focused on producing value for the customer and its very important to understand what “value” means to the customer:

  • First of all, meeting cost and schedule goals is only one component of value, and it may not be the most important component of value to the customer
  • Second, there have been many projects that have met their cost and schedule goals but failed to deliver an acceptable level of business value
  • Finally, “Value” can be a very elusive goal but, in any case, it’s what the customer thinks that “value” is that counts

There is no “best” process. Saying that “Agile is better than Waterfall” is like saying “A car is better than a boat”. Both have advantages and disadvantages depending on the environment you’re in:

When Does an Agile Approach Work Best?

When does an Agile approach work best? An Agile model tends to work best in projects where:

  • There is a relatively high level of uncertainty and a flexible and adaptive approach is needed to resolve the uncertainty as the project is in progress
  • Creativity and innovation are needed to maximize the business value of the solution

When Does a Plan-driven Approach Work Best?

When does a plan-driven approach work best? A traditional plan-driven model (what many people loosely call “Waterfall”)  tends to work best where:

  • There is a relatively low level of uncertainty, the solution is well-defined, and some level of planning and control are needed to maximize predictability of costs and schedules
  • The organization is not really  well-prepared to implement an Agile approach and/or the project team is not trained in Agile

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

In addition, you may also be interested in the following articles related articles:

Why Is Agile Important to Project Managers?

I think a lot of people are confused about “What is ‘Agile'” and the importance of Agile to project managers:

  • The word “Agile” has many different connotations
  • Many project managers think that it is something that only applies to software development and doesn’t apply to them at all.

For more detail on that, here’s an article with more detail on “What is Agile?”:

What is Agile?

Different Meanings of “Agile”

“Agile” means a lot of different things to different people. To some people:

  • It means just developing software faster, or
  • It means creating a more people-oriented project environment,
  • To others, it means making the project management process a lot more efficient by streamlining the whole process and eliminating unnecessary documentation

Those are only a few different connotations – there are many, many more. In addition, there are also many more stereotypes, myths, and misconceptions about what Agile is.

All of those things are potential outcomes of an Agile process but that’s not the fundamental essence of what an Agile process is all about in my opinion.  The fundamental essence of an Agile process is adaptivity.

What’s Wrong With the Typical Project Management Approach?

Many project management processes, as we know them today, were designed around what is called a “traditional plan-driven project management” model (what many people loosely call “Waterfall“). 

  • In this model, achieving predictability over the outcome of a project and the costs and schedule associated with achieving that outcome is very important
  • Therefore, it is also very important to have clearly-defined requirements as well as an adequate level of planning to be able to somewhat accurately predict the outcome, costs, and schedule of the project

That’s the predominant way that project management has worked since the 1950’s and 1960’s. A project was considered successful if it delivered what the requirements for the project within the defined budget and schedule.

That kind of predictability can be important. For large investments, it allows a company to:

  • Attempt to calculate with some level of certainty what the return on their investment is likely to be from a project, and
  • Make a go/no-go decision as to whether the project should be funded or not based on that information.

The primary problem with that approach is that it requires developing a fairly detailed plan for the project upfront. That is very difficult, if not impossible to do in projects with a very high level of uncertainty.

Why Is Adaptivity Important?

We live in a different world today from the world that existed in the 1950’s and 1960’s when formalized project management approaches were first defined. 

Higher Levels of Uncertainty

There is a much higher level of uncertainty:

  • Technology is rapidly changing,
  • Solutions are much more complex, and
  • The business environment that we operate in is dynamic and constantly changing.

In that kind  of environment,

  • Developing a detailed plan for a project with a lot of uncertainty upfront will typically require you to make a lot of assumptions.
  • And, many times those assumptions will be wrong  and may require significant re-planning and possible rework later.

Rather than force-fitting a project that has a high level of uncertainty to a traditional plan-driven approach;

  • it’s much better to fit the methodology to the nature of the project and that’s where a more adaptive approach really makes sense. 
  • That doesn’t mean that you don’t do any upfront planning; it means that you use a level of planning that is appropriate to the level of uncertainty in the project:
Traditional Plan-driven ApproachAdaptive (Agile) Approach
Atempt to develop a detailed set of requirements and a detailed project plan for the project upfrontLimit the amount of upfront planning based on the level of uncertainty in the project and use a “rolling-wave” planning approach to further define the requirements and plan for the project as the project is in place

Need for Creativity and Innovation

Another important factor in today’s environment is that there is a greater need for creativity and innovation to develop truly leading-edge products. An over-emphasis on planning and control can stifle creativity and innovation.

What’s an Example of a Project Requiring an Adaptive Approach?

A Simple Example

I use an example in my Agile Project Management training that is somewhat extreme but it gets the point across. The example I use is:

  • Suppose you were given the task to find a cure for cancer and you were asked to outline:
    • What the solution will be,
    • How long it will take to develop it, and
    • What the total cost of the research will be to develop the solution
  • In that situation, it would be ridiculous to attempt to develop a detailed project plan with accurate cost and schedule estimates – there is just way too many uncertainties to resolve
  • So what would you do? Give up and do nothing? That doesn’t make sense either

It’s important to recognize that we do know some things about finding a cure for cancer based on years of research that have gone into that area.

  • However, there are still way too many unknowns to develop a detailed project plan for a solution
  • What you would do is take advantage of what is known as much as possible and then take an iterative, trial-and-error approach to find a solution

Thomas Edison and the Light Bulb

That’s the way Edison invented the light bulb…here’s a quote from Edison on that subject:

“I speak without exaggeration when I say that I have constructed three thousand different theories in connection with the electric light, each one of them reasonable and apparently to be true. Yet only in two cases did my experiments prove the truth of my theory. My chief difficulty, as perhaps you know, was in constructing the carbon filament, the incandescence of which is the source of the light.”

(1890 Interview in Harper’s Magazine)

In a 1910 autobiography of Edison, Edison’s friend and associate Walter S. Mallory is quoted as asking

“Isn’t it a shame that with the tremendous amount of work you have done you haven’t been able to get any results?”  The book goes on to say that “

Edison turned on him like a flash, and with a smile replied:

“Results! Why, man, I have gotten lots of results! I know several thousand things that won’t work!”

What Makes This Kind of Project Different?

There are two things that make this kind of project fundamentally different:

  1. The level of uncertainty is very high and makes it impractical or impossible to develop a detailed plan for the project upfront
  2. Creativity and innovation required for finding a good solution are far more important than predictability

How Does an Agile Approach Solve This Problem?

An Agile process is built on an “Empirical” Process Control model. The word “empirical” means “based on observation”. In the context of an Agile development process, “Empirical” means that during the course of a project, both the product as well as the process to produce the product are continuously refined as the project is in progress. The goal is to produce the right product and to optimize the value of the product being produced.

Empirical Process Control Model

Why Is This Important to Project Managers?

You might ask “Why is this important to project managers?”

  • Isn’t Agile something that only applies to software development? (That’s a common misconception)
  • The truth is that all projects have some level of uncertainty associated with them

If you try to force-fit all projects to a traditional plan-driven project management approach, its just not going to work in many cases. Imagine, for example,

  • Trying to develop the next generation of the iPhone or any other new and innovative product
  • In that kind of project, creativity and innovation is just as important, if not more important, than predictability.

In this new environment, a project manager who only knows how to do a traditional plan-driven approach to project management will be at a serious disadvantage. What makes this even more difficult is that:

  • That this is not a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people seem to think.
  • It’s a matter of figuring out how to blend a traditional plan-driven approach with an adaptive (Agile) approach in the right proportions to fit a given situation.

Overall Summary

Agile will have a profound impact on the project management profession that will cause us to rethink many things we have taken for granted for a long time about what “project management” is.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

What are the Most Practical Ways to Do Project Planning?

I recently participated in an online discussion in response to a question that was asked on “What are the most practical ways to do project planning?” It’s a critical issue and it comes up a lot so I thought I would share some thoughts on this subject.

Ways to do Project Planning – Factors to Consider

In my opinion, there are two very significant factors in determining what planning approach would make the most sense for a particular project:

1. Level of Uncertainty

The level of uncertainty in the project is probably the most important factor. 

  • If there is a high level of uncertainty that cannot easily be resolved, it would probably be foolish to try to develop a highly-detailed, plan-driven approach. 
  • An example would be attempting to find a cure for cancer.  It would probably be foolish to try to develop a detailed plan for that effort. There’s just way too much uncertainty.

That doesn’t mean that you wouldn’t do any planning at all:

  • You would take stock of what you know and don’t know and try to develop at least a cursory plan based on that information and then
  • Continually revise the plan based on what you learn as the project is in progress

2. Customer Relationship

The second major factor is the relationship with the customer.  

  • If you have a contractual relationship with the customer where the customer is expecting a firm set of deliverables for a given schedule and cost, you might be forced into a planning model to try to effectively manage and satisfy those customer expectations. 
  • If there is more of a collaborative relationship with the customer, you probably have a lot more ability to optimize the approach based on the level of uncertainty in the project.

Ways to do Project Planning – Planning Quadrants

Obviously, there may be a conflict between these two factors.  I’ve created a diagram below to show some of the possible combinations of these two factors:

Ways to do Project Planning

Lets look at each of these quadrants individually:

1. Low Uncertainty, Contractual Relationship

This is the area that most project managers are most familiar with.

  • It is the area that is most well-suited for a traditional plan-driven project management model. 
  • In this area, the level of uncertainty may be low enough to permit developing a detailed plan.
  • That plan would be consistent with managing customer expectations in a contractual relationship model.

2. Low Uncertainty, Collaborative Relationship

If the level of uncertainty associated with a project is low, you might develop a more collaborative relationship with the customer; however, that might not be the most efficient way to do the project. If the level of uncertainty is truly low and it is relatively easy to define the project requirements upfront,

  • it may not make too much sense to engage the customer too heavily in a collaborative relationship
  • There is less of a need to further define detailed direction for the project as it is in progress.

3.. High Uncertainty, Collaborative Relationship

This is the area where an Agile approach makes most sense. In this area, instead of trying to develop a highly-detailed upfront plan prior to starting the project, you would probably:

  • Reach agreement with the customer on at least a vision for the project and at least some higher level objectives and requirements that the project must meet, and
  • Then further elaborate those requirements and the plan for meeting them as the project was in progress

It’s easy to see how this kind of planning model may not work well unless there’s a collaborative spirit of trust and partnership with the customer since:

  • It may be almost impossible to accurately define the costs and schedule for completing the project prior to starting the effort; and

  • Further defining the plan as the project is in progress requires a close working relationship with the customer.

4. High Uncertainty, Contractual Relationship

This area is the quadrant that is likely to be most problematic:

Impact of Uncertainty

The level of uncertainty will have a big impact. Attempting to do an Agile project with highly uncertain requirements without a collaborative relationship with the customer is not likely to work very well:

  • The customer may not be committed to actively engage in the project as it is in progress to help elaborate and resolve questions related to the requirements; and,
  • As a result, the project may either get stalled or go off in the wrong direction
Impact of Customer Relationship

Attempting to apply a contractual, plan-driven approach in this kind of environment is also likely to be problematic. There would likely be a very high risk associated with trying to develop a firm, plan-driven contractual relationship based on highly uncertain requirements. What is likely to happen is:

  • The project meets the defined requirements but the defined requirements were wrong, or
  • There are so many changes as the project is in progress that the scope of the project winds up being completely different from what the customer was expecting

It’s easy to see how either of these scenarios might present a very high risk.

Of course, this is somewhat of an over-simplification and all projects don’t fall very neatly into one of these quadrants.  There is a very broad range of scenarios in the middle of this diagram that call for some kind of hybrid approach.

What are the Most Practical Ways to Do Project Planning?

There are a number of conclusions that I think we can draw from an from understanding of this model:

1. There Is Not Just One Way to Do Project Planning

There is not just one way to do project planning. Project Managers who have been heavily indoctrinated in a traditional, plan-driven model and attempt to force-fit all projects to that kind of planning model are likely to not have optimal results.

2. This Is Not a Simple Binary and Mutually-exclusive Choice

This is not a simple binary and mutually-exclusive choice between an Agile and traditional plan-driven planning model. It is much more complicated than that.  There’s a whole spectrum of different possible scenarios and it is a multi-dimensional problem

3. Fit the Planning Model to the Nature of the Problem

The right approach is to fit the planning model to the nature of the problem based on the level of uncertainty and the relationship with the customer.  Attempting to use a single “one size fits all” planning model for all projects is not likely to work very well.

Overall Summary

This creates a big challenge for project managers to learn how to blend an Agile and traditional plan-driven approach in the right proportions to fit a given situation.  That’s not an easy thing to do. PMI still treats these two areas as separate and independent domains of knowledge with little or no integration between the two.  This is exactly the challenge that my Agile Project Management training is designed to address.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

What is the Real Essence of Agile? What Are the Real Advantages?

It’s apparent to me that a lot of people have gotten so heavily focused on the mechanics of how Agile is implemented that they’ve lost sight of the big picture of what the real essence of Agile is all about. 

  • The term “Agile” has taken on a number of different meanings today that are largely based on how it is implemented
  • For many people, “Agile” has become synonymous with Scrum and if you’re not doing Scrum and doing it “by the book”, you’re not really Agile at all

I think it is useful to step back and take a look at “What is the real essence of Agile”?

What is the Real Essence of Agile?

What is the Real Essence of Agile?

The real essence of “Agile”, in my opinion, is that:

  • It puts an emphasis on being adaptive to customer and business needs in order to maximize the value of the solution; rather than
  • Following a rigidly-defined plan with an emphasis on managing costs and schedules of delivering the solution.

For that reason, I like to use the terms “adaptive” and “plan-driven” rather than terms like “Agile” and “Waterfall”.

What’s the Truth About “Agile” vs “Waterfall”?

There’s a lot of myths, stereotypes, and misconceptions about “Agile” and “Waterfall” that we need to get past:

Are Agile and Waterfall Distinct and Unique Methodologies?

The terms “Agile” and “Waterfall” make it sound like you’re comparing two specific methodologies:

  • One called “Agile” and
  • One called “Waterfall”

and that’s not really accurate: 

  • “Agile” is not really a specific methodology or framework like Scrum;
  • It is much broader than that – it is a way of thinking defined by the Agile Manifesto

In It a Binary and Mutually-Exclusive Choice?

The “Agile versus Waterfall” kind of thinking leads people:

  • To think that there is a binary and mutually-exclusive choice between those two approaches and
  • That causes people  to try to force-fit a project to one of those extremes. 
  • The right approach is to go in the other direction and fit the methodology to the nature of the problem and sometimes that requires a blending the two approaches in the right proportions to fit the problem

Choosing the Best Approach

Here are some guidelines for choosing the best approach.

When Does Agile Work Best?

Agile works best in situations that have a high-level of uncertainty where it isn’t practical or possible to define the solution to the problem upfront.  In that kind of situation, it is much more effective to use an adaptive approach to incrementally and iteratively define the solution in more detail as the project is in progress rather than attempting to define detailed requirements for the solution upfront.

The example I like to use to illustrate this is finding a cure for cancer.  

  • If you set out to define a project to find a cure for cancer, it would be ridiculous to try to define a detailed plan upfront; there’s just far too much uncertainty. 
  • The only approach that is likely to work is an iterative, trial-and-error approach to find a solution.

Agile is based on what is called an “Empirical Process Control” model.  The word “Empirical” means “based on observation” and that means that in an Agile project, both the requirements for the solution AND the process for developing the solution are continuously refined as the project is in progress.

When Does a Plan-driven Approach Work Best?

That kind of empirical process control model approach is naturally not the most efficient approach if there is a low level of uncertainty and it is possible to easily define detailed requirements for the solution prior to the start of the project. In that situation, a trial-and-error approach really isn’t necessary and a plan-driven approach is much more appropriate and much more efficient.

The example I like to use to illustrate this scenario is building a bridge across a river. If you were defining a project to construct a bridge across a river, it would be ridiculous to say “we’ll take an adaptive approach, build the first span of the bridge, and decide how we will build the rest of the bridge after we’ve completed the first span. That wouldn’t make any sense.

This kind of situation calls for a plan-driven approach that is based on a defined process control model.

  • The key advantage of a defined process model is that it is predictable and repeatable and it is probably more efficient for projects with a low level of uncertainty.
  • If you designed a process for building a bridge and it has proven successful once, the same process or a variation of that process is likely to work in another similar situation with somewhat predictable results.

What if it is Between Those Extremes?

Very few real world situations are as extreme and clear-cut as the ones I’ve used of finding a cure for cancer and building a bridge across a river.

  • Most real-world situations fall somewhere between those two extremes.
  • There is some level of uncertainty but it’s not complete uncertainty.
  • You rarely start any project without at least some idea of what the solution is going to look like although the solution may be progressively refined in more detail as the project is in progress. There’s a range of approaches that look something like this:
Range of Agility

In this kind of situation, you have to tailor the approach to fit the nature of the project:

  • One of the biggest factors to consider is the level of uncertainty associated with the solution.
  • That requires more skill but it definitely can be done
  • It requires knowledge of a broader range of methodologies (both plan-driven and adaptive (Agile))
  • It also requires a deeper knowledge of the principles behind the methodologies in order to know how to tailor them to fit a given situation.
  • You can’t just force-fit a situation to some predefined methodology (whatever it might be) and do it mechanically.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

Management of Uncertainty in Agile Projects

One of the biggest inherent advantages of an Agile approach is the management of uncertainty in Agile projects. To understand this better, we need to first understand the difference between:

  • An Empirical Process Control Model and
  • A Defined Process Control Model

Empirical and Defined Process Models

A key thing to understand in this context is the difference between and Empirical Process and a Defined Process.

Empirical Process Control Model

Agile is based on an empirical process approach – the word “empirical” means “based on experiment or observation”.
When you use an empirical process approach,

  • You accept that you don’t know everything you might want to know about a project before you start and
  • The process is designed to adapt both the solution and the process to discover the solution to learning as the project progresses.

Defined Process Control Model

A “Defined Process” is repeatable and doesn’t change significantly from one project to the next and

  • Is very predictable in the results it produces while an “Empirical Process” is specifically intended to favor adaptability over predictability.
  • For that reason, an empirical process is much better suited for a project with high levels of uncertainty.

Uncertainty in Agile Projects

A very powerful concept for understanding uncertainty in Agile Projects is the “Stacey Complexity Model” which is shown below:

Uncertainty in Agile Projects

There are two dimensions of uncertainty in this model:

Requirements Uncertainty

One dimension is requirements uncertainty – How well understood are the goals and requirements for the project and how certain are the customers that they know what they want?

Technology Uncertainty

The other dimension is technology uncertainty – How well understood is the technology solution to the problem and what is the level of risk associated with the technology solution?

This is a very important concept because the ability to handle uncertainty is so important in today’s most critical projects and heavily plan-driven projects are not well-designed to deal with high levels of uncertainty.

Management of Uncertainty

What typically happens in a plan-driven project is the project manager tries to reduce the level of uncertainty to an acceptable level before starting the project by:

  • Trying to resolve any uncertainties in the requirements as much as possible before the project starts, and
  • Trying to eliminate as much technology risk as possible

This often results in using tried-and-proven technology and doesn’t push the envelope too far in terms of going into areas of new and undefined user requirements. The downside of that, of course, is that the technology approach may wind up being obsolete within a relatively short amount of time after it is released and it may also result in a very mediocre user experience with the solution.

What Does “Managing Uncertainty” Mean?

Let me clarify what I mean by “managing uncertainty”.

  • To some people, uncertainty is like a cancer that attacks a project and can cause it to fail. Conventional project management thinking has reinforced this approach to reduce the risk and uncertainty in a project as much as possible
  • I don’t see it that way uncertainty can also represent opportunity to go beyond what is expected and the value a project produces is many times directly related to the risk and uncertainty in the project
  • If you force a project to fit into a plan-driven model by reducing the risk and uncertainty, you may be maximizing the predictability of the project to meet cost and schedule goals but minimizing the value that the project produces

Here’s another article with more detail on this:

Overall Summary

Here’s a summary of some important points:

  • Uncertainty should not be ignored and not managed at all. Uncertainty often is directly associated with opportunity
  • Fortunately, this is not a black-and-white decision between:
    • A totally rigid, plan-driven approach with almost zero uncertainty and
    • A totally adaptive approach with an extremely high level of uncertainty.

The right thing to do is to fit the approach to the level of uncertainty in the project rather than force-fitting a project to some kind of canned approach (whatever it might be). It takes more skill to develop an intelligent approach for managing uncertainty; it requires:

  • The ability to objectively assess the level of uncertainty in a project
  • An understanding of both empirical and defined process models
  • A deeper understanding of the principles behind these approaches to know how to blend the two together to fit the situation

The ability to do that is the essence of what I believe is an effective Agile Project Management approach.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

Managed Agile Development Framework – A Hybrid Approach

I’ve seen many people ask a question like “should I use Agile or Waterfall for a project? That excludes the possibility that there is a hybrid approach that provides the benefits of both approaches. The Managed Agile Development Framework is an example of a hybrid approach that is very easy to implement

Background

A few years ago, I was responsible for managing a large government project.

  • The project required meeting some defined cost and schedule milestones
  • However, the customer wanted to take an Agile approach to defining the requirements.

In response to that project, I developed an approach which I call “The Managed Agile Development” framework that would satisfy those two seemingly inconsistent goals. The framework consisted of two levels:

LayerDescription
Macro LevelThe “Macro” level was the outer envelope of the project. It was focused primarily on managing overall contractual requirements
Micro LevelWithin that “macro level” envelope, we were still able to implement a fairly flexible Agile development approach at the “micro-level”
Managed Agile Development Framework

How Does It Work?

There are two layers in the framework as shown in the diagram above. The “Macro” layer is plan-driven; but it can be as “thick” or “thin” as you want it to be. The “Micro” layer can be any Agile development approach such as Scrum.

  • The macro-level framework is a plan-driven approach. It is designed to provide a sufficient level of control and predictability for the overall project. It defines the outer envelope (scope and high-level requirements) that the project operates within
  • Within that outer envelope, the micro-level framework utilizes a more flexible and iterative approach based on an Agile/Scrum approach. It is designed to be adaptive to user needs

Trade-offs to Consider

Naturally, there are tradeoffs between

  • The level of agility and flexibility to adapt to change at the “micro-level” and
  • The level of predictability and control at the “macro-level”.

It is important that both the client or business sponsor and the development team need to agree on those trade-offs. The framework provides a mechanism for making those trade-offs by making the “macro-level” as “thick” or “thin” as you want to fit a given situation.

Increasing Predictability and Control

Increasing the level of predictability and control requires:

  • Beefing up the macro-level,
  • Providing more detailed requirements at that level, and
  • Implementing at least a limited amount of change control

Increasing Agility

To increase the level of agility:

  • You can simply eliminate the macro-level altogether or limit it to only very high-level requirements
  • Other elements of the framework can be easily customized or eliminated depending on the scope and complexity of the project and other factors

Change Control

A question that often comes up is “How do you handle change control?”. The answer to that question is that:

  • You have to design enough slack into the milestones at the “macro” level to allow detailed elaboration of requirements to take place in the “micro” level.
  • However, when there is a significant enough change in the “micro” level that would impact achievement of the requirements in the “macro” level, that should trigger a change to the “macro” level milestones.

General Approach for Agile Contracts

This general approach can be used on almost any project. Check out this article for more detail on Agile Contracts:

Agile Contracts

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.