Tag Archives: Agile and Waterfall

Why Is Planning So Difficult? Is It a Waste of Time?

Why is planning so difficult? A lot of people seem to think that planning is very difficult and some think it is a waste of time because even very well-thought-out plans often don’t work out as expected. This issue is important because it is at the crux of selecting a methodology and planning approach for a project. Do I use Agile or plan-driven (aka Waterfall)?

Why Is Planning So Difficult?

Why Is Planning So Difficult?

Many people seem to think of planning as an “all-or-nothing” proposition – either you develop a highly-detailed plan or you do no planning at all. I don’t believe that to be the case. There seems to be two major problems associated with why people have this difficulty with planning:

Dealing With Uncertainty

Many people seem to have difficulty dealing with an uncertain environment – they want things to be crystal-clear, black-and-white and, in an uncertain environment, they think that it is a waste of time to do any planning at all.

Unrealistic Expectations

A related factor is that many people develop unrealistic expectations about planning.

  • If you develop a well thought-out plan, they expect that it should work every time.
  • Many people are also unrealistic that everything about a project will go absolutely perfectly all the time and
  • Murphy’s Law often contradicts that belief.

Planning Fundamentals

Let’s review some fundamentals about planning. We can learn a lot about how the military does planning.

Planning in the Military

Here are a few of my favorite quotes on planning in the military:

Plans are nothing; planning is everything”  – (Dwight D. Eisehhower)

What Eisenhower is saying is that documented plans should not be considered to be sacrosanct. When people document a written plan, the plan often takes on a life of its own and it becomes the “gospel” that everyone is expected to follow rigidly. However, that should not be the case and that’s not a reason to not do any planning at all. Here are a couple of examples:

  • It would be foolish for a military unit to go into battle without doing any planning at all. You should take advantage of whatever intelligence you might have about the enemy positions (as uncertain as it might be) but you shouldn’t lose sight of the uncertainty in that information.
  • In an American football game (or any other sport you want to choose), the coach prepares his team for what he thinks the strengths and weaknesses of the opposing team are; but, again, that planning is based on very uncertain information and lots of assumptions.

Here’s another quote along the same lines:

“No battle plan survives contact with the enemy”  (Helmuth von Moltke the Elder)

What Helmuth von Moltke is saying is that you shouldn’t expect that a plan won’t change at all once you start to execute it.

An Example

In World War II, Churchill and Roosevelt spent years planning the invasion of Europe that ultimately resulted in the Allied forces landing in Normandy on June 6, 1944. Both men knew that an invasion of Europe was the only way that the war could be won but there was a lot of uncertainty about:

  • Where should the invasion take place? How can the Allies preserve the element of surprise?
  • What’s the best way to do it? What are the odds of it being successful?
  • How many deaths and casualties can be expected?

This was a huge decision that had enormous impact:

  • The fate of Europe and most of the free world at that time depended on this being successful
  • Both men faced some level of opposition at home knowing the large number of casualties to be expected

It took a lot of courage to do this planning and make a decision under these circumstances knowing that there was a lot of uncertainty in the situation and the impact of the decision was so enormous. However, planning was essential – can you imagine what might have happened if the invasion was attempted without any planning at all?
Source: Churchill and Roosevelt Spent Years Planning D-Day

What Does This Have to Do With Agile?

This whole issue about planning is at the crux of the controversy about “Agile versus Waterfall”. Many people see the choice between Agile and Waterfall as a binary and mutually-exclusive choice and planning is at the center of this controversy:

  • Some people in the Agile community might say that its a waste of time to do planning in an uncertain environment and you should just use an Agile approach and let the project plan evolve as the project is in progress
  • Some people in the traditional plan-driven project management community might say that it would be foolish to attempt to do a project of any significant scope without a detailed plan for it to be successful

The truth is somewhere between those extremes – it’s not an “all or nothing” choice and you should adapt the methodology and the level of planning to fit the nature of the project. The level of uncertainty in the project is a big factor in making that choice. In an uncertain environment, it might be foolish to attempt to develop a highly detailed plan but uncertainty is never an absolute and it would be equally foolish not to take advantage of whatever information you might have.

How Do You Put This Into Practice?

A lot of people might see “Agile” and “Waterfall” as totally incompatible with each other. It’s like mixing oil and vinegar – they just don’t mix well. And, if you look at this from a mechanical process perspective, that might be true.

In order to learn how to blend these two approaches together in the right proportions to fit a given situation,

  • You have to understand both approaches at a deeper level and get past some of the strong stereotypes that exist about both “Agile” and “Waterfall”.
  • It’s really a choice of selecting a more plan-driven approach or a more adaptive approach based on the level of uncertainty in the situation and there is a whole spectrum of alternatives as shown below:
Levels of Agility

Rather than thinking of “Agile” and “Waterfall” as binary and mutually-exclusive choices and trying to force-fit a project to one of extremes, a better approach is to fit the methodology to the nature of the project and the level of uncertainty and the planning approach is a major factor in making that determination

Overall Summary

Here’s a summary of the key points I want to make in this article:

  1. Planning is not an “all or nothing” proposition
    • The planning approach should be directly related to the level of uncertainty in the environment
    • Planning in an uncertain environment can be difficult but that should not be an excuse for not doing any planning at all
    • However, to quote Eisenhower, “A plan is nothing – planning is everything” – don’t get locked into thinking that a written plan is sacrosanct and doesn’t need to be changed to adapt to the level of uncertainty in the environment
  2. The level of uncertainty and the planning approach is probably the most important factor in selecting a project management approach for projects
    • Rather than thinking about “Agile” and “Waterfall” as binary and mutually-exclusive alternatives,
    • Its more accurate and more objective to think of a spectrum of alternatives from heavily plan-driven at one extreme to heavily adaptive at the other extreme

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

What is a “Hybrid Agile” Approach? Is There Such a Thing?

What is a hybrid Agile Approach? Is there such a thing? I recently came across an article on the Internet that was posted in several places entitled “The Moment of truth: There Is No Hybrid Agile“.

  • This article is so full of stereotypes and misconceptions about “Agile” and “Waterfall” that I felt that I had to respond to it. 
  • It is typical of many articles that position “Agile” and “Waterfall” as two binary and mutually-exclusive alternatives with no middle ground between the two.

What Are the Flaws in This Thinking?

Treating Agile and Waterfall as Discrete, Binary Opposites

The biggest flaw in this thinking is that this article and many others like it treat “Agile” and “Waterfall” as if they were individual, discrete methodologies. They also position “Agile” and “Waterfall”  as diametrical opposites of each other.  That’s not very accurate.

“Agile” and “Waterfall” are not really discrete, individual methodologies at all and both of those terms are used very loosely.  In common usage. Neither of those are individual, discrete methodologies:

  • Many people  may think of “Agile” as being synonymous with Scrum but that is not really accurate.  “Agile” is much broader than Scrum – it is a way of thinking defined by the Agile Manifesto
  • “Waterfall” is also not a single, discrete methodology. In today’s world, many people use the term “Waterfall” for any plan-driven methodology that is not Agile.  What about RUP and other iterative approaches that probably wouldn’t be considered to be Agile?  Is that “Waterfall”?

A Better Way of Thinking

Instead of thinking of what people commonly call “Agile and “Waterfall” as individual discrete methodologies, it is more accurate to see it as a continuous spectrum of approaches from heavily plan-driven at one extreme to heavily adaptive at the other extreme like this:

What is a hybrid agile approach?

If you think of it in that way, it is much easier to see the possibility for lots of approaches in the middle of that spectrum that blend the right level of plan-driven principles and practices with more adaptive principles and practices to fit a given situation.

Here’s what some methodologies would look like plotted on a spectrum of heavily plan-driven versus heavily adaptive:

Adaptive vs Plan-driven

As you can see from this diagram, “Agile” is not a single approach and there is not just one way to do “Agile”:

  • Kanban is more adaptive than Scrum, and
  • Even within Scrum you will find different styles of implementation from
    • Simple team-level projects which may tend to be more adaptive to
    • Larger more complex multi-team projects which may tend to be somewhat more plan-driven

Putting It Into Practice

The most important point to get out of this is that there is not a clear and well-defined boundary line between “Agile” and “Waterfall” as many people seem to think.

Fitting the Approach to the Nature of the Problem

Many people make the mistake of performing a methodology mechanically. They think they need to do it religiously and “by the book”(That’s true of both Agile and other non-Agile methodologies)

  • The right approach is to fit the methodology to the nature of the problem rather than force-fitting all problems to a given methodology (Agile or non-Agile)
  • It takes more skill to do that but it definitely can be done.
  • It requires understanding the principles behind the methodology and why they make sense in a given situation rather than doing a given methodology mechanically

If you think of methodologies as being rigid and prescriptive,

  • It will be difficult to see how two seemingly disparate methodologies could be blended together in the right proportions to fit a given situation.
  • On the other hand, if you understand the principles behind the methodologies at a deeper level, it is much easier to see how they could be complementary to each other rather than competitive.

Learning to be a “Chef”

It can take a lot more skill to learn how to blend different approaches together in the right proportions to fit a given situation. In my book on Agile Project Management, I use the analogy of a project manager as a “cook” and a project manager as a “chef”.

A Good “Cook”

“A good ‘cook’:

  • May have the ability to create some very good meals, but
  • Those dishes may be limited to a repertoire of standard dishes.
  • And, his/her knowledge of how to prepare those meals may be primarily based on following some predefined recipes out of a cookbook”.
A “Chef”

“A ‘chef’, on the other hand,

  • Typically has a far greater ability to prepare a much broader range of more sophisticated dishes using much more exotic ingredients in some cases.
  • His/her knowledge of how to prepare those meals is not limited to predefined recipes, and
  • In many cases, a chef will create entirely new and innovative recipes for a given situation
  • The best chefs are not limited to a single cuisine. They are capable of combining dishes from entirely different kinds of cuisine.

That’s the challenge for project managers and agile practitioners in today’s world – we need more chefs and fewer cooks.

What is a Hybrid Agile Approach?

In simple terms, a hybrid Agile approach is one that blends the plan-driven principles and practices with Agile (adaptive) principles and practices in the right proportions to fit a given situation.

Managed Agile Development Framework

An example of that is the Managed Agile Development framework that I created. It simply wraps an outer layer of project-level planning around an Agile development process.

Managed Agile Development Framework

The outer layer can be as thick or thin as necessary to fit the situation.

The Origin of This Approach

I originally developed this framework when I was managing a very large government program for a US government agency.

  • The government agency had to have some level of predictability over the costs and schedules of the program.
  • The program was so large that it actually had some level of congressional oversight so some level of predictability and control was essential
  • However, within that outer envelope, the government agency customer wanted to have flexibility in many of the detailed requirements.
  • We were able to find the right balance of control and flexibility to satisfy both needs.

What Are Examples of Hybrid Agile Approaches?

Some of the most common examples of hybrid Agile approaches are:

Agile Contracts

  • The government program I mentioned is a good example
  • I also have a case study in my book on General Dynamics UK, Ltd. They successfully used a hybrid Agile approach to manage a large defense contract for the ministry of defense in the UK
  • I just finished building a new house. I naturally had a contract with the builder that defined the cost and schedule for the home. However, the builder offered a lot of flexibility to make changes even as the construction of the house was in progress (He charges for changes, of course)

Large, Enterprise-level Projects and Programs

´╗┐It’s almost impossible to successfully implement some large complex enterprise-level projects and programs without integrating some level of project and program management.

  • A good example of that is the Harvard Pilgrim Healthcare case study that is written up in my latest book.
  • The project involved over 100 Agile teams and involved replacing almost everyone of HPHC’s most critical business systems over a period of time
  • The whole effort involved a lot of moving parts that had to be carefully planned and synchronized. It’s impossible to imagine how that could be done without a sufficient level of project and program management to guide and manage the overall effort

Other Business-driven Initiatives

Many people have the mistaken belief that you need to force the entire company to be agile in order to adopt an Agile development approach. That isn’t necessarily true.

Fitting the Approach to the Business

A business has to be designed around whatever critical success factors are most important for the business that they’re in. Becoming agile may not be the only factor and may not even be the most significant factor.

  • For example, some companies are in very cost-competitive industries and succeed primarily based on operational excellence to lower their costs as much as possible
  • Becoming more agile may play an indirect role in that but it isn’t necessarily the most important factor
Product Development Companies

On the other hand, in a company that is technology-driven that succeeds on bringing leading-edge products to market as quickly as possible, it’s much easier to see how a pure Agile approach might be a very strong and direct driver of the business

  • Agile was originally developed for companies that do product development and that’s where it works best.
  • In companies whose primary business is not developing products per se, there is typically more of a project-oriented approach.
  • The company has to typically evaluate a potential portfolio of projects to determine what mix of projects and programs is going to have the greatest impact on their business.
  • Then they need to monitor the execution of those projects and programs to determine if it is really delivering the expected returns.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.