Category Archives: Agile Planning

Why Is Planning So Difficult? Is It a Waste of Time?

Why is planning so difficult? A lot of people seem to think that planning is very difficult and some think it is a waste of time because even very well-thought-out plans often don’t work out as expected. This issue is important because it is at the crux of selecting a methodology and planning approach for a project. Do I use Agile or plan-driven (aka Waterfall)?

Why Is Planning So Difficult?

Why Is Planning So Difficult?

Many people seem to think of planning as an “all-or-nothing” proposition – either you develop a highly-detailed plan or you do no planning at all. I don’t believe that to be the case. There seems to be two major problems associated with why people have this difficulty with planning:

Dealing With Uncertainty

Many people seem to have difficulty dealing with an uncertain environment – they want things to be crystal-clear, black-and-white and, in an uncertain environment, they think that it is a waste of time to do any planning at all.

Unrealistic Expectations

A related factor is that many people develop unrealistic expectations about planning.

  • If you develop a well thought-out plan, they expect that it should work every time.
  • Many people are also unrealistic that everything about a project will go absolutely perfectly all the time and
  • Murphy’s Law often contradicts that belief.

Planning Fundamentals

Let’s review some fundamentals about planning. We can learn a lot about how the military does planning.

Planning in the Military

Here are a few of my favorite quotes on planning in the military:

Plans are nothing; planning is everything”  – (Dwight D. Eisehhower)

What Eisenhower is saying is that documented plans should not be considered to be sacrosanct. When people document a written plan, the plan often takes on a life of its own and it becomes the “gospel” that everyone is expected to follow rigidly. However, that should not be the case and that’s not a reason to not do any planning at all. Here are a couple of examples:

  • It would be foolish for a military unit to go into battle without doing any planning at all. You should take advantage of whatever intelligence you might have about the enemy positions (as uncertain as it might be) but you shouldn’t lose sight of the uncertainty in that information.
  • In an American football game (or any other sport you want to choose), the coach prepares his team for what he thinks the strengths and weaknesses of the opposing team are; but, again, that planning is based on very uncertain information and lots of assumptions.

Here’s another quote along the same lines:

“No battle plan survives contact with the enemy”  (Helmuth von Moltke the Elder)

What Helmuth von Moltke is saying is that you shouldn’t expect that a plan won’t change at all once you start to execute it.

An Example

In World War II, Churchill and Roosevelt spent years planning the invasion of Europe that ultimately resulted in the Allied forces landing in Normandy on June 6, 1944. Both men knew that an invasion of Europe was the only way that the war could be won but there was a lot of uncertainty about:

  • Where should the invasion take place? How can the Allies preserve the element of surprise?
  • What’s the best way to do it? What are the odds of it being successful?
  • How many deaths and casualties can be expected?

This was a huge decision that had enormous impact:

  • The fate of Europe and most of the free world at that time depended on this being successful
  • Both men faced some level of opposition at home knowing the large number of casualties to be expected

It took a lot of courage to do this planning and make a decision under these circumstances knowing that there was a lot of uncertainty in the situation and the impact of the decision was so enormous. However, planning was essential – can you imagine what might have happened if the invasion was attempted without any planning at all?
Source: Churchill and Roosevelt Spent Years Planning D-Day

What Does This Have to Do With Agile?

This whole issue about planning is at the crux of the controversy about “Agile versus Waterfall”. Many people see the choice between Agile and Waterfall as a binary and mutually-exclusive choice and planning is at the center of this controversy:

  • Some people in the Agile community might say that its a waste of time to do planning in an uncertain environment and you should just use an Agile approach and let the project plan evolve as the project is in progress
  • Some people in the traditional plan-driven project management community might say that it would be foolish to attempt to do a project of any significant scope without a detailed plan for it to be successful

The truth is somewhere between those extremes – it’s not an “all or nothing” choice and you should adapt the methodology and the level of planning to fit the nature of the project. The level of uncertainty in the project is a big factor in making that choice. In an uncertain environment, it might be foolish to attempt to develop a highly detailed plan but uncertainty is never an absolute and it would be equally foolish not to take advantage of whatever information you might have.

How Do You Put This Into Practice?

A lot of people might see “Agile” and “Waterfall” as totally incompatible with each other. It’s like mixing oil and vinegar – they just don’t mix well. And, if you look at this from a mechanical process perspective, that might be true.

In order to learn how to blend these two approaches together in the right proportions to fit a given situation,

  • You have to understand both approaches at a deeper level and get past some of the strong stereotypes that exist about both “Agile” and “Waterfall”.
  • It’s really a choice of selecting a more plan-driven approach or a more adaptive approach based on the level of uncertainty in the situation and there is a whole spectrum of alternatives as shown below:
Levels of Agility

Rather than thinking of “Agile” and “Waterfall” as binary and mutually-exclusive choices and trying to force-fit a project to one of extremes, a better approach is to fit the methodology to the nature of the project and the level of uncertainty and the planning approach is a major factor in making that determination

Overall Summary

Here’s a summary of the key points I want to make in this article:

  1. Planning is not an “all or nothing” proposition
    • The planning approach should be directly related to the level of uncertainty in the environment
    • Planning in an uncertain environment can be difficult but that should not be an excuse for not doing any planning at all
    • However, to quote Eisenhower, “A plan is nothing – planning is everything” – don’t get locked into thinking that a written plan is sacrosanct and doesn’t need to be changed to adapt to the level of uncertainty in the environment
  2. The level of uncertainty and the planning approach is probably the most important factor in selecting a project management approach for projects
    • Rather than thinking about “Agile” and “Waterfall” as binary and mutually-exclusive alternatives,
    • Its more accurate and more objective to think of a spectrum of alternatives from heavily plan-driven at one extreme to heavily adaptive at the other extreme

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

What are the Most Practical Ways to Do Project Planning?

I recently participated in an online discussion in response to a question that was asked on “What are the most practical ways to do project planning?” It’s a critical issue and it comes up a lot so I thought I would share some thoughts on this subject.

Ways to do Project Planning – Factors to Consider

In my opinion, there are two very significant factors in determining what planning approach would make the most sense for a particular project:

1. Level of Uncertainty

The level of uncertainty in the project is probably the most important factor. 

  • If there is a high level of uncertainty that cannot easily be resolved, it would probably be foolish to try to develop a highly-detailed, plan-driven approach. 
  • An example would be attempting to find a cure for cancer.  It would probably be foolish to try to develop a detailed plan for that effort. There’s just way too much uncertainty.

That doesn’t mean that you wouldn’t do any planning at all:

  • You would take stock of what you know and don’t know and try to develop at least a cursory plan based on that information and then
  • Continually revise the plan based on what you learn as the project is in progress

2. Customer Relationship

The second major factor is the relationship with the customer.  

  • If you have a contractual relationship with the customer where the customer is expecting a firm set of deliverables for a given schedule and cost, you might be forced into a planning model to try to effectively manage and satisfy those customer expectations. 
  • If there is more of a collaborative relationship with the customer, you probably have a lot more ability to optimize the approach based on the level of uncertainty in the project.

Ways to do Project Planning – Planning Quadrants

Obviously, there may be a conflict between these two factors.  I’ve created a diagram below to show some of the possible combinations of these two factors:

Ways to do Project Planning

Lets look at each of these quadrants individually:

1. Low Uncertainty, Contractual Relationship

This is the area that most project managers are most familiar with.

  • It is the area that is most well-suited for a traditional plan-driven project management model. 
  • In this area, the level of uncertainty may be low enough to permit developing a detailed plan.
  • That plan would be consistent with managing customer expectations in a contractual relationship model.

2. Low Uncertainty, Collaborative Relationship

If the level of uncertainty associated with a project is low, you might develop a more collaborative relationship with the customer; however, that might not be the most efficient way to do the project. If the level of uncertainty is truly low and it is relatively easy to define the project requirements upfront,

  • it may not make too much sense to engage the customer too heavily in a collaborative relationship
  • There is less of a need to further define detailed direction for the project as it is in progress.

3.. High Uncertainty, Collaborative Relationship

This is the area where an Agile approach makes most sense. In this area, instead of trying to develop a highly-detailed upfront plan prior to starting the project, you would probably:

  • Reach agreement with the customer on at least a vision for the project and at least some higher level objectives and requirements that the project must meet, and
  • Then further elaborate those requirements and the plan for meeting them as the project was in progress

It’s easy to see how this kind of planning model may not work well unless there’s a collaborative spirit of trust and partnership with the customer since:

  • It may be almost impossible to accurately define the costs and schedule for completing the project prior to starting the effort; and

  • Further defining the plan as the project is in progress requires a close working relationship with the customer.

4. High Uncertainty, Contractual Relationship

This area is the quadrant that is likely to be most problematic:

Impact of Uncertainty

The level of uncertainty will have a big impact. Attempting to do an Agile project with highly uncertain requirements without a collaborative relationship with the customer is not likely to work very well:

  • The customer may not be committed to actively engage in the project as it is in progress to help elaborate and resolve questions related to the requirements; and,
  • As a result, the project may either get stalled or go off in the wrong direction
Impact of Customer Relationship

Attempting to apply a contractual, plan-driven approach in this kind of environment is also likely to be problematic. There would likely be a very high risk associated with trying to develop a firm, plan-driven contractual relationship based on highly uncertain requirements. What is likely to happen is:

  • The project meets the defined requirements but the defined requirements were wrong, or
  • There are so many changes as the project is in progress that the scope of the project winds up being completely different from what the customer was expecting

It’s easy to see how either of these scenarios might present a very high risk.

Of course, this is somewhat of an over-simplification and all projects don’t fall very neatly into one of these quadrants.  There is a very broad range of scenarios in the middle of this diagram that call for some kind of hybrid approach.

What are the Most Practical Ways to Do Project Planning?

There are a number of conclusions that I think we can draw from an from understanding of this model:

1. There Is Not Just One Way to Do Project Planning

There is not just one way to do project planning. Project Managers who have been heavily indoctrinated in a traditional, plan-driven model and attempt to force-fit all projects to that kind of planning model are likely to not have optimal results.

2. This Is Not a Simple Binary and Mutually-exclusive Choice

This is not a simple binary and mutually-exclusive choice between an Agile and traditional plan-driven planning model. It is much more complicated than that.  There’s a whole spectrum of different possible scenarios and it is a multi-dimensional problem

3. Fit the Planning Model to the Nature of the Problem

The right approach is to fit the planning model to the nature of the problem based on the level of uncertainty and the relationship with the customer.  Attempting to use a single “one size fits all” planning model for all projects is not likely to work very well.

Overall Summary

This creates a big challenge for project managers to learn how to blend an Agile and traditional plan-driven approach in the right proportions to fit a given situation.  That’s not an easy thing to do. PMI still treats these two areas as separate and independent domains of knowledge with little or no integration between the two.  This is exactly the challenge that my Agile Project Management training is designed to address.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

Management of Uncertainty in Agile Projects

One of the biggest inherent advantages of an Agile approach is the management of uncertainty in Agile projects. To understand this better, we need to first understand the difference between:

  • An Empirical Process Control Model and
  • A Defined Process Control Model

Empirical and Defined Process Models

A key thing to understand in this context is the difference between and Empirical Process and a Defined Process.

Empirical Process Control Model

Agile is based on an empirical process approach – the word “empirical” means “based on experiment or observation”.
When you use an empirical process approach,

  • You accept that you don’t know everything you might want to know about a project before you start and
  • The process is designed to adapt both the solution and the process to discover the solution to learning as the project progresses.

Defined Process Control Model

A “Defined Process” is repeatable and doesn’t change significantly from one project to the next and

  • Is very predictable in the results it produces while an “Empirical Process” is specifically intended to favor adaptability over predictability.
  • For that reason, an empirical process is much better suited for a project with high levels of uncertainty.

Uncertainty in Agile Projects

A very powerful concept for understanding uncertainty in Agile Projects is the “Stacey Complexity Model” which is shown below:

Uncertainty in Agile Projects

There are two dimensions of uncertainty in this model:

Requirements Uncertainty

One dimension is requirements uncertainty – How well understood are the goals and requirements for the project and how certain are the customers that they know what they want?

Technology Uncertainty

The other dimension is technology uncertainty – How well understood is the technology solution to the problem and what is the level of risk associated with the technology solution?

This is a very important concept because the ability to handle uncertainty is so important in today’s most critical projects and heavily plan-driven projects are not well-designed to deal with high levels of uncertainty.

Management of Uncertainty

What typically happens in a plan-driven project is the project manager tries to reduce the level of uncertainty to an acceptable level before starting the project by:

  • Trying to resolve any uncertainties in the requirements as much as possible before the project starts, and
  • Trying to eliminate as much technology risk as possible

This often results in using tried-and-proven technology and doesn’t push the envelope too far in terms of going into areas of new and undefined user requirements. The downside of that, of course, is that the technology approach may wind up being obsolete within a relatively short amount of time after it is released and it may also result in a very mediocre user experience with the solution.

What Does “Managing Uncertainty” Mean?

Let me clarify what I mean by “managing uncertainty”.

  • To some people, uncertainty is like a cancer that attacks a project and can cause it to fail. Conventional project management thinking has reinforced this approach to reduce the risk and uncertainty in a project as much as possible
  • I don’t see it that way uncertainty can also represent opportunity to go beyond what is expected and the value a project produces is many times directly related to the risk and uncertainty in the project
  • If you force a project to fit into a plan-driven model by reducing the risk and uncertainty, you may be maximizing the predictability of the project to meet cost and schedule goals but minimizing the value that the project produces

Here’s another article with more detail on this:

Overall Summary

Here’s a summary of some important points:

  • Uncertainty should not be ignored and not managed at all. Uncertainty often is directly associated with opportunity
  • Fortunately, this is not a black-and-white decision between:
    • A totally rigid, plan-driven approach with almost zero uncertainty and
    • A totally adaptive approach with an extremely high level of uncertainty.

The right thing to do is to fit the approach to the level of uncertainty in the project rather than force-fitting a project to some kind of canned approach (whatever it might be). It takes more skill to develop an intelligent approach for managing uncertainty; it requires:

  • The ability to objectively assess the level of uncertainty in a project
  • An understanding of both empirical and defined process models
  • A deeper understanding of the principles behind these approaches to know how to blend the two together to fit the situation

The ability to do that is the essence of what I believe is an effective Agile Project Management approach.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

What is an Agile Business Analyst? How is the Role Different?

Many software development projects are moving rapidly to an Agile development approach:

  • That has dramatically impacted the role of a project manager in an Agile environment
  • Agile also has a significant impact on the role of a Business Analyst

In this article, we will explore “What is an Agile Business Analyst?” and “How is the role different?”.

What is an Agile Business Analyst?

Traditional Business Analyst Role

The traditional role of a Business Analyst is defined as follows:

“The business analyst’s primary objective is helping businesses implement technology solutions in a cost-effective way by determining the requirements of a project or program, and communicating them clearly to stakeholders, facilitators and partners.”
Business Analyst Job Description, Villanova Univ.

That’s the predominant role that has been played by Business Analysts for many years.  In some cases, the role has primarily focused on simply talking to users about their needs and writing requirements documents.

  • That it has been a very documentation-intensive role
  • However, the real value-added that a Business Analyst should provide is in helping to define innovative solutions to business problems
  • It should not be limited to just simply creating requirements documents. 

In a traditional plan-driven environment, documentation often takes on a life of its own. In that environment,

  • It is easy to lose sight of the fact that the goal is to create business value and
  • Documentation is only a by-product

What Are the Skills Needed by a Business Analyst?

In an Agile environment a Business Analyst needs to provide value beyond simply writing requirements documents. The Business Analyst has to have an understanding of:

Knowlege AreaRequirement
Business Domain KnowledgeIn-depth knowledge of the Business Domain that they work in (finance, healthcare, manufacturing, or whatever else it might be). That means that the BA understands:
  • What the major business processes are,
  • The metrics that are important to the business, and
  • How to improve the processes from a business perspective
Process Improvement Tools and TechniquesUnderstand how to:
  • Identify and define opportunities for improvement
  • Use process improvement tools and techniques such as Six Sigma and others to analyze an existing business process, and
IT Solutions PerspectiveKnow how to translate understanding of the business process improvement needs into IT solution requirements to create effective and innovative business solutions
Project Process UnderstandingNeed to understand:
  • How an overall project development process works, and
  • How their role fits into that process so that they can work most effectively as a member of a development team

These fundamental skills have not changed over the years. A good Business Analyst should be strong in all of these areas.

What Is Wrong With the Traditional Business Analyst Role?

The problems with the Traditional Business Analyst role are essentially the same as a Traditional Project Management role.

Level of Uncertainty

Today’s environment typically has a much higher level of uncertainty associated with defining software solutions.  The traditional approach of documenting detailed requirements prior to the start of projects just doesn’t work well with high levels of uncertainty: 

  • A lot of time might be wasted trying to define detailed requirements prior to the start of the project
  • Many of those requirements might be based on speculation and assumptions, and
  • Those requirements might then go through an enormous amount of change as the project is in progress.

Emphasis on Creativity and Innovation

The emphasis in many projects is shifting:

  • In the past, the primary emphasis in many projects has been on planning and control
  • A key goal was to achieve predictability of project costs and schedules
  • Competitive pressures today have created a need to put more emphasis on creativity and innovation to maximize the business value of the solution

An over-emphasis on planning and control can destroy creativity and innovation:

  • Can you imagine trying to develop the next generation of a very innovative product like a new iPhone using a traditional, plan-driven approach?
  • It would be impossible to do that based on developing detailed requirements for the product upfront.

How Do Agile Requirements Work?

To understand the role of a Business Analyst in an Agile environment, we first need to understand how an Agile requirements process works:

  • There are a lot of misconceptions about Agile. Some people may think that there are no documented requirements
  • That is not the case. The approach for developing requirements in an Agile project should be directly related to the level of uncertainty in the project

High Levels of Uncertainty

In a project with a high level of uncertainty, it might be foolish to try to develop detailed requirements prior to the start of the project:

  • In that environment, the requirements might be limited to an overall vision and high-level requirements that the project must meet
  • More detailed requirements would be further elaborated as the project is in progress. That elaboration would be based heavily on direct feedback and inputs from the users

Lower Level of Uncertainty

If there is a lower level of uncertainty, the project might go further towards developing more defined requirements.

  • That might be particularly true if there is a need for some level of predictability of the project costs and schedule
  • However, an Agile environment would still emphasize some level of flexibility and adaptivity to maximize the business value of the solution as the project is in progress

Key Differences

There are several key differences in an Agile approach:

1. Less Emphasis on Documentation

Agile requirements are considerably simplified and are typically written in terms of “user stories”. 

  • A “user story” is a brief and very succinct definition of a business need. It does not provide any detail about how it will be implemented, 
  • A “user story” is considered to be a “placeholder for conversation”. Further definition of how it will be implemented will be determined based on direct, face-to-face communications with the users. That will take place as the user stories are being implemented.

2. Changes Are Encouraged

An Agile approach is designed around flexibility and adaptivity to maximize the value of the solution.  For that reason, changes to requirements as the project is in progress are encouraged.

3. Direct Communication

There is a lot more direct communication between the business users and the project team. 

  • First, there is a Product Owner who represents the business. The Product Owner directly participates in the project to provide overall business direction and priorities
  • Second, the developers on the project team are expected to communicate directly with the business users. That is essential to further elaborate and define requirements as the project is in progress

In this kind of environment,

  • The role of a Business Analyst should be more than just an intermediary to talk to the users and document requirements
  • That role would inhibit direct communications with the project team. It would probably also limit the flexibility and adaptivity that is so important to an Agile approach

So, What is an Agile Business Analyst?

So, What is an Agile Business Analyst?

  • The role of a Business Analyst in an Agile environment is not well-defined
  • On small, simple Agile projects there may not be a need for a Business Analyst role
  • That is frequently not the case on large, complex enterprise-level projects

In an Agile environment,

  • It might be assumed that the Product Owner plays the role that might normally be played by a Business Analyst. However, the Product Owner responsibility goes well beyond the role of a Business Analyst, and
  • It can be very difficult for a Product Owner to perform that role without some assistance on very large, complex projects

Ways to Provide More Value-Added

The value-added provided by a Business Analyst in an Agile environment needs to go beyond simply writing requirements documents.  Here are some potential ways that a BA can provide a higher level of value:

1. Process Improvement

Many times, simply automating a process “as-is” does not provide a sufficient level of value. 

  • Rather than simply documenting a process “as-is”,
  • A Business Analyst can provide a much higher level of value by finding innovative ways to improve an existing process to make it more efficient or more effective

2. Analyzing a Broadly-Defined Area

In large, complex projects:

  • Using functional decomposition can be essential to create a well-organized, value-driven framework
  • That helps to align the Product Backlog with the required business value
  • It also makes it easier to understanding the relationship of the stories and epics to each other and for satisfying overall business goals

3. Writing Individual Stories

A Business Analyst can provide value by writing user stories that are very clear and concise and easy-to-understand and implement by the development team. 

Writing effective user stories is a skill that is often taken for granted.

  • In good stories the “why” or the “so that” clause that expresses the business value the story is intended to provide is important
  • A good BA can provide that perspective that is difficult for a developer to provide.

4. Identifying Related User Stories and Epics

In a large complex project, there is value in grouping stories into themes and epics as necessary. That can be essential to understand the interrelationship and associated business process flows. 

  • The interrelationship of user stories and epics should be well-understood and
  • The implementation of stories across different functional areas may require some planning and coordination so that they are consistently implemented.
  • This overall framework can provide a mechanism for easily identifying any inconsistencies and/or missing functionality.

5. Integration With Related Projects

On large projects, there may also be a need to:

  • Integrate the needs of related projects as well as
  • Integrate the needs of a number of different stakeholders to produce an overall solution.

Overall Summary

The key point is that if a Business Analyst is involved at all in an Agile project, he/she should add value in some way.

  • He/she should not be just an intermediary between the development team and the business users and stakeholders
  • The role should not be limited to creating requirements documents

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.