Why Is Planning So Difficult? Is It a Waste of Time?

A lot of people seem to think that planning is very difficult and some think it is a waste of time because even very well-thought-out plans often don’t work out as expected. This issue is important because it is at the crux of selecting a methodology and planning approach for a project. Do I use Agile or plan-driven (aka Waterfall)?

Why Is Planning So Difficult?

What’s the Issue?

Many people seem to think of planning as an “all-or-nothing” proposition – either you develop a highly-detailed plan or you do no planning at all. I don’t believe that to be the case. There seems to be two major problems associated with why people have this difficulty with planning:

  • Dealing with Uncertainty – Many people seem to have difficulty dealing with an uncertain environment – they want things to be crystal-clear, black-and-white and, in an uncertain environment, they think that it is a waste of time to do any planning at all.
  • Unrealistic Expectations – A related factor is that many people develop unrealistic expectations about planning. If you develop a well thought-out plan, they expect that it should work every time.

Planning Fundamentals

Let’s review some fundamentals about planning. We can learn a lot about how the military does planning. Here are a few of my favorite quotes on planning in the military:

Plans are nothing; planning is everything”  – (Dwight D. Eisehhower)

What Eisenhower is saying is that documented plans should not be considered to be sacrosanct. When people document a written plan, the plan often takes on a life of its own and it becomes the “gospel” that everyone is expected to follow rigidly. However, that should not be the case and that’s not a reason to not do any planning at all. Here are a couple of examples:

  • It would be foolish for a military unit to go into battle without doing any planning at all. You should take advantage of whatever intelligence you might have about the enemy positions (as uncertain as it might be) but you shouldn’t lose sight of the uncertainty in that information.
  • In an American football game (or any other sport you want to choose), the coach prepares his team for what he thinks the strengths and weaknesses of the opposing team are; but, again, that planning is based on very uncertain information.

Here’s another quote along the same lines:

“No battle plan survives contact with the enemy”  (Helmuth von Moltke the Elder)

What Helmuth von Moltke is saying is that you shouldn’t expect that a plan won’t change once you start to execute it.

An Example

In World War II, Churchill and Roosevelt spent years planning the invasion of Europe that ultimately resulted in the Allied forces landing in Normandy on June 6, 1944. Both men knew that an invasion of Europe was the only way that the war could be won but there was a lot of uncertainty about:

  • Where should the invasion take place? How can the Allies preserve the element of surprise?
  • What’s the best way to do it? What are the odds of it being successful?
  • How many deaths and casualties can be expected?

This was a huge decision that had enormous impact:

  • The fate of Europe and most of the free world at that time depended on this being successful
  • Both men faced some level of opposition at home knowing the large number of casualties to be expected

It took a lot of courage to do this planning and make a decision under these circumstances knowing that there was a lot of uncertainty in the situation and the impact of the decision was so enormous. However, planning was essential – can you imagine what might have happened if the invasion was attempted without any planning at all?
Source: Churchill and Roosevelt Spent Years Planning D-Day

What Does This Have to Do With Agile?

This whole issue about planning is at the crux of the controversy about “Agile versus Waterfall”. Many people see the choice between Agile and Waterfall as a binary and mutually-exclusive choice and planning is at the center of this controversy:

  • Some people in the Agile community might say that its a waste of time to do planning in an uncertain environment and you should just use an Agile approach and let the project plan evolve as the project is in progress
  • Some people in the traditional plan-driven project management community might say that it would be foolish to attempt to do a project of any significant scope without a detailed plan for it to be successful

The truth is somewhere between those extremes – it’s not an “all or nothing” choice and you should adapt the methodology and the level of planning to fit the nature of the project. The level of uncertainty in the project is a big factor in making that choice. In an uncertain environment, it might be foolish to attempt to develop a highly detailed plan but uncertainty is never an absolute and it would be equally foolish not to take advantage of whatever information you might have.

How Do You Put This Into Practice?

A lot of people might see “Agile” and “Waterfall” as totally incompatible with each other. It’s like mixing oil and vinegar – they just don’t mix well. And, if you look at this from a mechanical process perspective, that might be true.

In order to learn how to blend these two approaches together in the right proportions to fit a given situation, you have to understand both approaches at a deeper level and get past some of the strong stereotypes that exist about both “Agile” and “Waterfall”. It’s really a choice of selecting a more plan-driven approach or a more adaptive approach based on the level of uncertainty in the situation and there is a whole spectrum of alternatives as shown below:

Levels of Agility

The words “Agile” and “Waterfall” are two of the most loosely-used words in the English language. People use those words as if both “Agile” and “Waterfall” are specific and well-defined methodologies that can be easily compared and they are not:

  • “Agile” is not a methodology – “Agile” is actually a range of different approaches – Kanban and Scrum are both Agile methodologies (or frameworks) but they are very different. Kanban is more adaptive than Scrum and Scrum is more plan-driven than Kanban. Both approaches are very flexible and can be adapted to a range of different circumstances.
  • When people talk about “Waterfall”, they are generally not talking about the specific “Waterfall” process that was defined by Dr. Winston Royce in the 1970’s. As it is typically used, in practice, the word “Waterfall” really means any plan-driven methodology that is not totally agile. For example what about the Rational Unified Process (RUP)? That’s an iterative process but many people might include that in the realm of “Waterfall”.

Overall Summary

Here’s a summary of the key points I want to make in this article:

  • Planning is not an “all or nothing” proposition
  • The planning approach should be directly related to the level of uncertainty in the environment
  • Planning in an uncertain environment can be difficult but that should not be an excuse for not doing any planning at all
  • However, to quote Eisenhower, “A plan is nothing – planning is everything” – don’t get locked into thinking that a written plan is sacrosanct and doesn’t need to be changed to adapt to the level of uncertainty in the environment
  • The level of uncertainty and the planning approach is probably the most important factor in selecting a project management approach for projects
  • Rather than thinking about “Agile” and “Waterfall” as binary and mutually-exclusive alternatives, its more accurate and more objective to think of a spectrum of alternatives from heavily plan-driven at one extreme to heavily adaptive at the other extreme