Category Archives: Understanding Agile

How Do You Choose the Best Methodology for a Project?

I often see questions on “How do you choose the best methodology for a project?”

choose the best methodology for a project

Is There a Best Methodology for a Project?

I don’t think that there is one single “best” methodology for a project that works for all projects.  A lot of people make the mistake of force-fitting a project to some standard methodology.

Traditional Plan-driven Approach (“Waterfall”)

Some project managers will try to use a traditional plan-driven methodology for a project (what many people loosely call “Waterfall”):

  • It may be the only project management approach that they know.
  • It has also been widely-accepted as the only way to do project management

Force-fitting a Project to a Methodology

Many other people have the misconception that there is a binary and mutually-exclusive choice between “Agile” and “Waterfall”. As a result, they will attempt to force-fit a project to one of those extremes

Fit the Methodology to the Project

The “best” approach is to go in the other direction and fit the methodology to the nature of the project. 

  • That takes more skill but it definitely can be done
  • You should also recognize that there is not a binary and mutually-exclusive choice between “Agile” and “Waterfall”
  • There is a whole spectrum of different approaches ranging from heavily plan-driven at one extreme to heavily adaptive (or Agile) at the other extreme
Range of Agility

Factors to Consider

There are a number of factors that influence the selection of the best approach for a particular project:

1. Level of Uncertainty

First and probably the most significant factor in choosing an approach, is the level of uncertainty in the project. A project with a high level of uncertainty would be best-suited for a more adaptive (Agile) approach. Attempting to force-fit such a project to a traditional plan-driven project management approach could be disastrous.

  • It would force you to make a lot of assumptions to try to resolve the uncertainty; and, in many cases, those assumptions may be wrong and require a lot of re-planning and possible re-work
  • The emphasis on planning and control in a traditional plan-driven project is not conducive to changes. That will make it difficult to maximize the value of the solution in an uncertain environment

2. Need for Creativity and Innovation

Next, in today’s competitive environment, creativity and innovation can be very important to create very competitive business solutions. An approach with a heavy emphasis on planning and control can stifle creativity and innovation.

3. Customer Relationship

Finally, managing customer expectations is probably one of the most critical aspects of any project.  If the results of a project are not consistent with customer expectations, the project will likely not be viewed as successful no matter how good you think it is.  The nature of the customer relationship can range between:

Contractual Style of Relationship

A contractual style of relationship is where there are very definite and well-defined customer expectations that must be met.  In this type of relationship, the customer may not take any responsibility for the success of the project.  The customer defines requirements and expects the project team to do whatever is necessary to meet those requirements. In this style of relationship, there is limited participation by the customer in the project

Naturally,  the contractual style of relationship is well-suited to a project with a relatively low level of uncertainty. It must be possible to define the customer’s requirements in some level of detail prior to the start of the project.  It would be much more difficult to make a contractual-style relationship work in a project with a high level of uncertainty.

Collaborative Style of Relationship

A collaborative style of relationship is where there is a shared responsibility for the success of the project. In this environment both the project team and the customer take an active role in defining the direction of the project as it is in progress

What’s the Right Style of Relationship?

Of course, the customer has to be amenable to whatever type of relationship you choose.  If the project has a high level of uncertainty, that would lean towards more of a collaborative relationship; however:

  • The customer has to be open to that kind of relationship for it to be successful. 
  • This is a big problem in many companies where it is difficult to break down organizational boundaries between organizations
  • It requires establishing truly collaborative relationships based on a spirit of shared responsibility, trust, and partnership.

3. Project Team Capabilities

The final major factor in selecting a project approach is, of course, the capabilities of the project team.   

  • An Agile approach requires a lot of training and skill and a hybrid Agile approach  can require even more training and skill. 
  • Naturally, it does not make any sense to choose an approach that the team is not capable of implementing.

Why Is Choosing the Best Methodology for a Project Important?

There are two major factors that require us to broaden the way we think about “project management” today:

  • Solutions tend to be much more complex and the level of uncertainty is much higher
  • Competitive pressures frequently require much higher levels of creativity and innovation

For those reasons, force-fitting all projects to a standardized plan-driven approach is not necessarily the best approach for all projects.

Overall Summary

Choosing the best methodology for a project can be a difficult thing to do. It is not necessarily a simple matter of choosing Agile or Waterfall. You have to fit the methodology to the nature of the project. A project should be focused on producing value for the customer and its very important to understand what “value” means to the customer:

  • First of all, meeting cost and schedule goals is only one component of value, and it may not be the most important component of value to the customer
  • Second, there have been many projects that have met their cost and schedule goals but failed to deliver an acceptable level of business value
  • Finally, “Value” can be a very elusive goal but, in any case, it’s what the customer thinks that “value” is that counts

There is no “best” process. Saying that “Agile is better than Waterfall” is like saying “A car is better than a boat”. Both have advantages and disadvantages depending on the environment you’re in:

When Does an Agile Approach Work Best?

When does an Agile approach work best? An Agile model tends to work best in projects where:

  • There is a relatively high level of uncertainty and a flexible and adaptive approach is needed to resolve the uncertainty as the project is in progress
  • Creativity and innovation are needed to maximize the business value of the solution

When Does a Plan-driven Approach Work Best?

When does a plan-driven approach work best? A traditional plan-driven model (what many people loosely call “Waterfall”)  tends to work best where:

  • There is a relatively low level of uncertainty, the solution is well-defined, and some level of planning and control are needed to maximize predictability of costs and schedules
  • The organization is not really  well-prepared to implement an Agile approach and/or the project team is not trained in Agile

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

In addition, you may also be interested in the following articles related articles:

What is Agile? How Would You Define Agile? What Does Agile Mean?

How do you define Agile? I’ve seen a lot of discussions where people have attempted to answer the question of “What is Agile?” with very different answers. 

Define Agile

Do You Really Mean Scrum?

First, there’s a lot of confusion about Agile and Scrum. Scrum is so widely-used as an Agile approach that when many people say “Agile”, they’re really talking about Scrum.

  • Agile is a general philosophy
  • Scrum is a specific Agile approach

Check out this article on What Is Scrum? What Is Agile? for more on that.

Feeling the Elephant

I’m sure that you could find at least 100 definitions of what “Agile” is on the Internet. It’s like the old fable of “The Blind Men Feeling the Elephant”:

“It was six men of Indostan
    To learning much inclined,
Who went to see the Elephant
    (Though all of them were blind),
That each by observation
    Might satisfy his mind[18]”

Each man came away with a different opinion depending on which part of the elephant he touched:

“Each in his own opinion concludes that the elephant is like a wall, snake, spear, tree, fan or rope, depending upon where they had touched”

Different Views of Agile

The way people define Agile is somewhat similar.  Depending on which part you touch, you may come away with a different impression of what Agile is:

  • Many people will define Agile in terms of how it is done. For example, many people will say that it is defined by the Agile Manifesto.  Here are a couple of examples of that:

“Agile is a set of methods and frameworks that embody the principles  and values of the Agile Manifesto”

“Agile is a term used to describe approaches to software development emphasizing incremental delivery, team collaboration, continual planning, and continual learning. The term “Agile” was coined in 2001 in the Agile Manifesto.”

  • Some people will say that it is just a mindset or way of thinking.  Here’s an example of that.

“Being ‘Agile Is a mindset. It’s about finding the right thing to build, faster (and not just building things faster)”

  • Some people will define it by comparing it to “Waterfall” to tell you what it is not.  Here’s an example of that:

“Agile is a time boxed, iterative approach to software delivery that builds software incrementally from the start of the project, instead of trying to deliver it all at once near the end.”

None of those definitions are incorrect, but it’s like “feeling the elephant”; they don’t really get to the real essence of what the “elephant” is, in my opinion.

How Do You Define Agile?

Personally, I like the definition that is published by the Agile Alliance:

“The ability to create and respond to change in order to succeed in an uncertain and turbulent environment.”

I think that is a better definition because it gets to what I consider the real essence of what Agile is.

“Agile is best suited for situations that have some level of uncertainty where creativity and innovation are important to maximize the business value of the solution as opposed to other situations with lower levels of uncertainty where planning and control to achieve predictability are more important.” (My own definition)

What Problems Does Agile Solve?

We should acknowledge that Agile is not a solution to every problem you might have. One way to define it is in terms of what problems it is useful for. 

  • If you accept that Agile is not a solution to every problem you might have, the first thing you would want to know is “what problems is it useful for?” 
  • You don’t need to get too far into the mechanics of how to do it until you’ve determined that it is a useful solution to the problem you’re trying to solve.

There is an interesting observation you might draw from this – people get very immersed in the mechanics of how to do Agile and that’s how they define it.  Isn’t it more important to know what problems it’s useful for solving before you get into the details of how to use it?

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

Why Is Agile Important to Project Managers?

I think a lot of people are confused about “What is ‘Agile'” and the importance of Agile to project managers:

  • The word “Agile” has many different connotations
  • Many project managers think that it is something that only applies to software development and doesn’t apply to them at all.

For more detail on that, here’s an article with more detail on “What is Agile?”:

What is Agile?

Different Meanings of “Agile”

“Agile” means a lot of different things to different people. To some people:

  • It means just developing software faster, or
  • It means creating a more people-oriented project environment,
  • To others, it means making the project management process a lot more efficient by streamlining the whole process and eliminating unnecessary documentation

Those are only a few different connotations – there are many, many more. In addition, there are also many more stereotypes, myths, and misconceptions about what Agile is.

All of those things are potential outcomes of an Agile process but that’s not the fundamental essence of what an Agile process is all about in my opinion.  The fundamental essence of an Agile process is adaptivity.

What’s Wrong With the Typical Project Management Approach?

Many project management processes, as we know them today, were designed around what is called a “traditional plan-driven project management” model (what many people loosely call “Waterfall“). 

  • In this model, achieving predictability over the outcome of a project and the costs and schedule associated with achieving that outcome is very important
  • Therefore, it is also very important to have clearly-defined requirements as well as an adequate level of planning to be able to somewhat accurately predict the outcome, costs, and schedule of the project

That’s the predominant way that project management has worked since the 1950’s and 1960’s. A project was considered successful if it delivered what the requirements for the project within the defined budget and schedule.

That kind of predictability can be important. For large investments, it allows a company to:

  • Attempt to calculate with some level of certainty what the return on their investment is likely to be from a project, and
  • Make a go/no-go decision as to whether the project should be funded or not based on that information.

The primary problem with that approach is that it requires developing a fairly detailed plan for the project upfront. That is very difficult, if not impossible to do in projects with a very high level of uncertainty.

Why Is Adaptivity Important?

We live in a different world today from the world that existed in the 1950’s and 1960’s when formalized project management approaches were first defined. 

Higher Levels of Uncertainty

There is a much higher level of uncertainty:

  • Technology is rapidly changing,
  • Solutions are much more complex, and
  • The business environment that we operate in is dynamic and constantly changing.

In that kind  of environment,

  • Developing a detailed plan for a project with a lot of uncertainty upfront will typically require you to make a lot of assumptions.
  • And, many times those assumptions will be wrong  and may require significant re-planning and possible rework later.

Rather than force-fitting a project that has a high level of uncertainty to a traditional plan-driven approach;

  • it’s much better to fit the methodology to the nature of the project and that’s where a more adaptive approach really makes sense. 
  • That doesn’t mean that you don’t do any upfront planning; it means that you use a level of planning that is appropriate to the level of uncertainty in the project:
Traditional Plan-driven ApproachAdaptive (Agile) Approach
Atempt to develop a detailed set of requirements and a detailed project plan for the project upfrontLimit the amount of upfront planning based on the level of uncertainty in the project and use a “rolling-wave” planning approach to further define the requirements and plan for the project as the project is in place

Need for Creativity and Innovation

Another important factor in today’s environment is that there is a greater need for creativity and innovation to develop truly leading-edge products. An over-emphasis on planning and control can stifle creativity and innovation.

What’s an Example of a Project Requiring an Adaptive Approach?

A Simple Example

I use an example in my Agile Project Management training that is somewhat extreme but it gets the point across. The example I use is:

  • Suppose you were given the task to find a cure for cancer and you were asked to outline:
    • What the solution will be,
    • How long it will take to develop it, and
    • What the total cost of the research will be to develop the solution
  • In that situation, it would be ridiculous to attempt to develop a detailed project plan with accurate cost and schedule estimates – there is just way too many uncertainties to resolve
  • So what would you do? Give up and do nothing? That doesn’t make sense either

It’s important to recognize that we do know some things about finding a cure for cancer based on years of research that have gone into that area.

  • However, there are still way too many unknowns to develop a detailed project plan for a solution
  • What you would do is take advantage of what is known as much as possible and then take an iterative, trial-and-error approach to find a solution

Thomas Edison and the Light Bulb

That’s the way Edison invented the light bulb…here’s a quote from Edison on that subject:

“I speak without exaggeration when I say that I have constructed three thousand different theories in connection with the electric light, each one of them reasonable and apparently to be true. Yet only in two cases did my experiments prove the truth of my theory. My chief difficulty, as perhaps you know, was in constructing the carbon filament, the incandescence of which is the source of the light.”

(1890 Interview in Harper’s Magazine)

In a 1910 autobiography of Edison, Edison’s friend and associate Walter S. Mallory is quoted as asking

“Isn’t it a shame that with the tremendous amount of work you have done you haven’t been able to get any results?”  The book goes on to say that “

Edison turned on him like a flash, and with a smile replied:

“Results! Why, man, I have gotten lots of results! I know several thousand things that won’t work!”

What Makes This Kind of Project Different?

There are two things that make this kind of project fundamentally different:

  1. The level of uncertainty is very high and makes it impractical or impossible to develop a detailed plan for the project upfront
  2. Creativity and innovation required for finding a good solution are far more important than predictability

How Does an Agile Approach Solve This Problem?

An Agile process is built on an “Empirical” Process Control model. The word “empirical” means “based on observation”. In the context of an Agile development process, “Empirical” means that during the course of a project, both the product as well as the process to produce the product are continuously refined as the project is in progress. The goal is to produce the right product and to optimize the value of the product being produced.

Empirical Process Control Model

Why Is This Important to Project Managers?

You might ask “Why is this important to project managers?”

  • Isn’t Agile something that only applies to software development? (That’s a common misconception)
  • The truth is that all projects have some level of uncertainty associated with them

If you try to force-fit all projects to a traditional plan-driven project management approach, its just not going to work in many cases. Imagine, for example,

  • Trying to develop the next generation of the iPhone or any other new and innovative product
  • In that kind of project, creativity and innovation is just as important, if not more important, than predictability.

In this new environment, a project manager who only knows how to do a traditional plan-driven approach to project management will be at a serious disadvantage. What makes this even more difficult is that:

  • That this is not a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people seem to think.
  • It’s a matter of figuring out how to blend a traditional plan-driven approach with an adaptive (Agile) approach in the right proportions to fit a given situation.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

What Does Waterfall Mean? How Do People Use That Word?

What does ‘Waterfall’ really mean? Have you ever thought about that?

  • The word “Waterfall” is often used in comparison to ‘Agile’ but do people know what they really mean when they compare ‘Agile’ to ‘Waterfall’?
  • I think the word ‘Waterfall’ is one of the most loosely-used words in the English language (the word ‘Agile’ is not far behind).

When people talk about ‘Agile’ and ‘Waterfall’,

  • it sounds like they’re comparing two very specific and well-defined methodologies that are binary and mutually-exclusive opposites of each other
  • However, when you dig into what the words ‘Waterfall’ and ‘Agile’ really mean, you quickly discover that’s a very inaccurate and misleading comparison
What Does Waterfall Mean?

What Does Waterfall Mean?

Strictly speaking, the word ‘Waterfall’ was originally defined in 1970 by Dr. Winston Royce in his very famous paper:

Dr. Winston Royce’s 1970 Waterfall Paper

Dr Royce described a model that consists of a sequence of phases. In this model, the outputs of one phase flow into the next phase like a “Waterfall”:

What Is Waterfall

The process was called ‘Waterfall’ because the results of one phase flow into the next phase like a waterfall.

What Was Life Like Prior to ‘Waterfall’?

it is useful to understand what life was like prior to Waterfall and what problems it tried to solve. What existed at that time was a lot of poorly-organized development efforts with little or no structure, discipline, and planning. Some of the major problems with the that approach were:

Coordinating Work of Large Development Teams

As projects grew in terms of scope and complexity with potentially much larger numbers of developers, it became apparent that a more planned and structured approach was essential to coordinate the work of large development teams

Cost and Schedule Overruns

The other major problem was that there was very limited predictability over the costs and schedules of software projects:

  • There were many and frequent very significant cost and schedule overruns, and
  • Business sponsors demanded some level of predictability

How Did the Waterfall Process Improve These Problems?

When the Waterfall approach was originally defined, it was a big improvement to go from practically no methodology at all to a very well-defined process. The new Waterfall process provided:

  • A “road map” to:
    • Coordinate the work of multiple developers as well as
    • Integrate the work with any other essential resources outside of the immediate development teams, and
  • A mechanism to gain control over the scope of software projects in order to get more predictability of project costs and schedules

Many younger people today don’t appreciate that history and just criticize Waterfall as being bad without understanding the problems it was intended to solve.

The “Pendulum Effect”

As with many things, there was a “pendulum” effect when the Waterfall approach was initially implemented. There was somewhat of an over-correction in many cases in going from no methodology to a very well-defined methodology. The pendulum swung in many projects from almost no control and discipline to very rigid control and discipline.

It Became Very Rigid and Inflexible

The initial implementation of the Waterfall process had a number of problems that even Dr. Royce recognized in 1970 when he first defined the process. Some of the most serious problems were:

  • The common practice when the Waterfall process was originally defined in 1970 was a very document-intensive and over-controlled process
  • You couldn’t exit a phase until all the documentation required to show that the work required for that phase had been completed, reviewed, and approved
  • The ultimate user of the software didn’t normally even see the software until all of the development and testing was complete and by that point in time; it was very difficult, if not impossible, to go back and make any significant changes
  • The emphasis on control of scope made the process very inflexible to any changes that might be needed to meet user needs and business goals in an uncertain environment

What Was the Impact?

As a result,

  • There have been many situations where the project may have met cost and schedule goals but failed to provide a sufficient level of business value
  • Another major problem was that a heavy emphasis on documentation and other overhead required for reviews and approvals made the whole process bureaucratic and not very cost efficient

It’s important to note that many of the problems associated with “Waterfall” are a result of how it is implemented and not necessarily inherent problems in the methodology itself.

Why Is the Agile versus Waterfall Comparison So Misleading?

A big reason why the typical Agile versus Waterfall comparison is so misleading is that the words “Agile” and “Waterfall” are so loosely-used:

How is the Word “Waterfall” Loosely-used?

Before Agile came into widespread use, many variations on the original Waterfall model were developed to create a more adaptive approach to solve some of these problems.

  • More iterative processes such as the Rational Unified Process (RUP) and a number of variants became widely-used in the 1990’s and early 2000’s
  • There has been a proliferation of a broad range of many different development models such as the Spiral model
  • Some of those have only a very limited resemblance to the original ‘Waterfall’ model as it was defined in 1970.

In spite of this evolution, people still loosely characterize all of those methodologies as ‘Waterfall’ as if it was one specific, unique and well-defined methodology called ‘Waterfall’ and that is not really the case


The common denominator of all the methodologies that people loosely call ‘Waterfall’ is that

  • They emphasize some level of upfront planning and control.
  • The goal is to try to achieve predictability over project scope, costs, and schedules.

For that reason, I think the word “plan-driven” is a more accurate and objective description of what people really mean when they say ‘Waterfall’.

How Is the Word “Agile” Loosely-Used?

The word ‘Agile’ is also loosely used. We all know that ‘Agile’ is not a specific methodology although many people equate ‘Agile’ with Scrum:

  • Scrum is not really a specific methodology, it is really a framework that is intended to be adaptable to a broad range of situations
  • Agile is not really equivalent to Scrum. There are other Agile methodologies such as Kanban

The common denominator of methodologies that people call ‘Agile’ is that:

  • They are flexible and adaptive and
  • Emphasize creativity and innovation in an uncertain environment

rather than:

  • Emphasizing planning and control to achieve predictability with lower levels of certainty.

For that reason, I prefer to use the word “adaptive” instead of the word ‘Agile’ when comparing it to ‘Waterfall’ (plan-driven).

Overall Summary

When people in the Agile community compare ‘Agile’ and ‘Waterfall’, it’s usually in the context of

  • Agile is good and Waterfall is bad and that’s really not accurate and objective.
  • Saying “Agile is better than Waterfall” is like saying “A car is better than a boat” – both have advantages and disadvantages depending on the environment that you are in.

The words “Agile” and “Waterfall” are very loosely used in practice and that causes a lot of confusion. They are used as if both “Agile” and “Waterfall” are unique, individual methodologies and that is not the case.

The word “Waterfall”, in particular, is very loosely-used.

  • When people use the word “Waterfall”, they’re not necessarily talking about the real “Waterfall model that was defined by Dr Winston Royce in the 1970’s
  • They’re really talking about any plan-driven methodology that is not completely agile.

A better way to think about “Agile” versus “Waterfall” is in terms of “adaptive” versus “plan-driven”. That’s a much more accurate and objective way of making that comparison.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

Agile and Six Sigma – Are They Complementary to Each Other?

I recently responded to a question on an online discussion that asked “Are there companies that use Agile and Six Sigma?”.  This raises an interesting question of “Are Agile and Six Sigma really complementary to each other?”. How would you go about blending the two approaches?

Are Agile and Six Sigma Really Complementary to Each Other?

Agile and Six Sigma – Potentially Conflicting Approaches

There are numerous approaches that might conflict with each other such as:

  • Agile and Six SIgma,
  • Lean and Agile, and
  • Agile and Waterfall

Those approaches have different objectives. If you pursued these approaches individually and independently of each other, the objectives of each approach might be somewhat contradictory. However, if you do it intelligently, it is very possible to blend these approaches in the right proportions.  

  • That requires a lot more skill and
  • It requires a different kind of thinking to see them as complementary rather than competitive approaches

The Fundamental Problem

There is a significant fundamental problem that must be overcome to see all of these approaches in a different perspective:

  1. Companies and individuals get enamored with a methodology like Agile or Six Sigma and see it as a “silver bullet” solution to any problem that they might have
  2. They attempt to mechanically force-fit their business to one of those methodologies without fully understanding the principles behind it

An Example With Six Sigma

When I published my first book in 2003, Six Sigma was very hot and everyone wanted to “jump on the Six Sigma bandwagon”.  At that time, I researched a number of companies that were doing Six Sigma and other process improvement methodologies. What I saw was this:

Successful Companies

In companies that seemed to do Six Sigma successfully:

  1. It wasn’t even obvious that it was Six Sigma and they might not have even called it “Six Sigma”:
    • The implementation wasn’t limited to Six Sigma
    • They understood the principles behind Six Sigma, and
    • Might have blended Six Sigma with other process improvement methodologies, and
  2. It was very well-integrated with their business:
    • It was just a tool that was part of their business
    • Rather than a program that was superimposed on their business
Less-Successful Companies

In other companies, I saw a much more superficial implementation of Six Sigma that didn’t last in many cases:

  • There was a lot of emphasis on the “mechanics” of doing Six Sigma,
  • There was a lot of “hoopla” about the ceremonies associated with Six Sigma. (green belts, black belts, etc.), and
  • They openly advertised that they were using Six Sigma to promote themselves

Does that sound familiar?  I think a similar thing is going on with Agile today.

The Key Factor for Success

What I learned from that some of the key factor for success are:

  • Don’t get overly enamored with any methodology (Six Sigma or anything else):
    • Don’t think of it as a “silver bullet” for any problem you might have.  
    • Be objective and recognize that any methodology has advantages and limitations depending on the problem you’re trying to solve,
  • Adapt the methodology to fit the problem and the business environment rather than force-fitting your business to some predefined methodology, and
  • Go beyond simply doing a mechanical implementation of any methodology (Agile or Six Sigma) and understand the principles behind it at a deeper level

Are Agile and Six Sigma Really Complementary To Each Other?

On the surface, Six Sigma and Agile would tend to pull you in different directions:

  • Agile emphasizes creativity and innovation as well as flexibility and adaptivity to maximize the business value of the solution
  • Six Sigma emphasizes process standardization and control of a process to minimize process variation

The key to seeing these approaches as complementary rather than competitive is to understand the fundamental principles behind the approach at a deeper level rather than getting lost in the “mechanics” of the approach.

The essence of Six Sigma is attempting to standardize processes and reduce variation in processes.  If you became obsessive about pursuing that goal, it would also not be very consistent with being Agile. However, there is absolutely nothing wrong with attempting to standardize processes to some extent as long as it is also done intelligently and in balance with other objectives.

The importance of “Systems Thinking”

A fundamental skill for doing this successfully is “Systems Thinking”.
Systems thinking is essential for seeing these seemingly contradictory approaches in a much broader context. It enables you to see how these objectives can interact in complementary ways rather than being competitive. Here’s an article with more detail on “Systems Thinking”:

Overall Summary

It is very possible to blend together different approaches that are seemingly in conflict with each other as long as it is done intelligently. It requires:

  • Understanding the fundamental principles behind each approach rather than getting lost in the mechanics,
  • Using a systems thinking” approach to see these seemingly contradictory approaches in a different perspective. Systems thinking enables you to see how they might actually be complementary to each other rather than competitive, and
  • Learning to fit the methodology to the problem rather than force-fitting a problem to any given methodology

This is exactly the approach behind the Agile Project Management online training courses I’ve developed.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

What is the Real Essence of Agile? What Are the Real Advantages?

It’s apparent to me that a lot of people have gotten so heavily focused on the mechanics of how Agile is implemented that they’ve lost sight of the big picture of what the real essence of Agile is all about. 

  • The term “Agile” has taken on a number of different meanings today that are largely based on how it is implemented
  • For many people, “Agile” has become synonymous with Scrum and if you’re not doing Scrum and doing it “by the book”, you’re not really Agile at all

I think it is useful to step back and take a look at “What is the real essence of Agile”?

What is the Real Essence of Agile?

What is the Real Essence of Agile?

The real essence of “Agile”, in my opinion, is that:

  • It puts an emphasis on being adaptive to customer and business needs in order to maximize the value of the solution; rather than
  • Following a rigidly-defined plan with an emphasis on managing costs and schedules of delivering the solution.

For that reason, I like to use the terms “adaptive” and “plan-driven” rather than terms like “Agile” and “Waterfall”.

What’s the Truth About “Agile” vs “Waterfall”?

There’s a lot of myths, stereotypes, and misconceptions about “Agile” and “Waterfall” that we need to get past:

Are Agile and Waterfall Distinct and Unique Methodologies?

The terms “Agile” and “Waterfall” make it sound like you’re comparing two specific methodologies:

  • One called “Agile” and
  • One called “Waterfall”

and that’s not really accurate: 

  • “Agile” is not really a specific methodology or framework like Scrum;
  • It is much broader than that – it is a way of thinking defined by the Agile Manifesto

In It a Binary and Mutually-Exclusive Choice?

The “Agile versus Waterfall” kind of thinking leads people:

  • To think that there is a binary and mutually-exclusive choice between those two approaches and
  • That causes people  to try to force-fit a project to one of those extremes. 
  • The right approach is to go in the other direction and fit the methodology to the nature of the problem and sometimes that requires a blending the two approaches in the right proportions to fit the problem

Choosing the Best Approach

Here are some guidelines for choosing the best approach.

When Does Agile Work Best?

Agile works best in situations that have a high-level of uncertainty where it isn’t practical or possible to define the solution to the problem upfront.  In that kind of situation, it is much more effective to use an adaptive approach to incrementally and iteratively define the solution in more detail as the project is in progress rather than attempting to define detailed requirements for the solution upfront.

The example I like to use to illustrate this is finding a cure for cancer.  

  • If you set out to define a project to find a cure for cancer, it would be ridiculous to try to define a detailed plan upfront; there’s just far too much uncertainty. 
  • The only approach that is likely to work is an iterative, trial-and-error approach to find a solution.

Agile is based on what is called an “Empirical Process Control” model.  The word “Empirical” means “based on observation” and that means that in an Agile project, both the requirements for the solution AND the process for developing the solution are continuously refined as the project is in progress.

When Does a Plan-driven Approach Work Best?

That kind of empirical process control model approach is naturally not the most efficient approach if there is a low level of uncertainty and it is possible to easily define detailed requirements for the solution prior to the start of the project. In that situation, a trial-and-error approach really isn’t necessary and a plan-driven approach is much more appropriate and much more efficient.

The example I like to use to illustrate this scenario is building a bridge across a river. If you were defining a project to construct a bridge across a river, it would be ridiculous to say “we’ll take an adaptive approach, build the first span of the bridge, and decide how we will build the rest of the bridge after we’ve completed the first span. That wouldn’t make any sense.

This kind of situation calls for a plan-driven approach that is based on a defined process control model.

  • The key advantage of a defined process model is that it is predictable and repeatable and it is probably more efficient for projects with a low level of uncertainty.
  • If you designed a process for building a bridge and it has proven successful once, the same process or a variation of that process is likely to work in another similar situation with somewhat predictable results.

What if it is Between Those Extremes?

Very few real world situations are as extreme and clear-cut as the ones I’ve used of finding a cure for cancer and building a bridge across a river.

  • Most real-world situations fall somewhere between those two extremes.
  • There is some level of uncertainty but it’s not complete uncertainty.
  • You rarely start any project without at least some idea of what the solution is going to look like although the solution may be progressively refined in more detail as the project is in progress. There’s a range of approaches that look something like this:
Range of Agility

In this kind of situation, you have to tailor the approach to fit the nature of the project:

  • One of the biggest factors to consider is the level of uncertainty associated with the solution.
  • That requires more skill but it definitely can be done
  • It requires knowledge of a broader range of methodologies (both plan-driven and adaptive (Agile))
  • It also requires a deeper knowledge of the principles behind the methodologies in order to know how to tailor them to fit a given situation.
  • You can’t just force-fit a situation to some predefined methodology (whatever it might be) and do it mechanically.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

Lean and Agile – Is Lean in Conflict with Agile?

I’ve participated in several discussions and presentations lately where the subject of Lean and Agile came up and I think the relationship of the two is very interesting.

  • If you pursued each of those approaches to the extreme and tried maximize what you would get out of both, they would tend to pull you in different directions.
  • Lean emphasizes reducing “waste” and Agile emphasizes flexibility and adaptivity to meet customer needs. Those two approaches are not totally compatible with each other; however, they are necessarily incompatible either. It just requires some skill to blend them together in the right proportions to fit a given situation.

Here’s an example, Michael Nir recently made a presentation at an Agile Boston meeting about “The Agile PMO” which was based on his book of the same name. Michael indicated that a key potential role of an Agile PMO is to reduce waste in an organization and that goal is very consistent with Lean. An example of that could be under-utilization of people in the organization.

How Does Lean Reduce Waste?

Michael indicated that a key potential role of an Agile PMO is to reduce waste in an organization and that goal is very consistent with Lean. For example,

  • Under-utilization of people in the organization,
  • Under-utilization of resources, or
  • Less than optimum utilization of resources

could certainly be a major source of waste in an organization. There are a number of ways that a PMO can reduce waste:

Utilization of Specialized Resources

If specialized resources that are not dedicated to project teams (such as DBA’s) are not well-planned and coordinated across teams:

  • Project teams may be idle waiting for these specialized resources, or
  • The specialized resources might not be fully-utilized waiting for work from project teams

Project Portfolio Management

If a project portfolio is not well-managed, allocation of resources to project teams may not be not well-aligned with company business goals and priorities

Project Management of Individual Projects

If individual projects are not well-managed and are allowed to go off track, the allocation of resources to projects may not be optimized to maximize the business results for the company

Development Process Definition and Training

If:

  • The development process is not well-defined,
  • Tools aren’t adequate to support the process, and/or
  • Project teams are not well-trained to execute the process

the execution of the process will not be consistent across teams and may not be as efficient and effective as it could be

In all of those areas, a PMO might add value by reducing waste but how far do you go with that? 

Can You Reduce Waste to Zero?

Carried to an extreme, a focus on simply reducing waste could easily become dysfunctional.  Michael mentioned that waste in some organizations could be as high as 95%.  What would happen if you attempted to reduce waste to 0%?

  • First, reducing waste to 0% is probably an unrealistic and impossible goal. No business is totally predictable where everything is known in advance to enable perfect prioritization, planning, and scheduling of resources
  • Second, putting too much emphasis on reducing waste would would mean superimposing a level of control and standardization on projects. That could easily be inconsistent with achieving the flexibility and adaptivity required by an Agile approach

What’s the Right Answer?

Given that conflict, what’s the right answer?  This is not necessarily an easy problem to solve. It will take some skill to figure out the right blend of:

  1. Focusing on lean and reducing waste and
  2. Preserving the flexibility and adaptivity required by an Agile approach. 

There clearly seems to be an optimum point between the two extremes of focusing on those two extremes individually. A PMO could probably perform a value-added role in helping an organization find that optimum point.

Finding that optimum point is yet another example of the need for “systems thinking”.  Here’s a previous post I wrote on that subject:

http://managedagile.com/2013/04/28/systems-thinking-and-binary-thinking/

People many times like to over-simplify what is really much more complex and reduce it to a simple, binary choice between two extremes: 

  • “Agile” versus “Waterfall” is one example of that and
  • “Lean” versus “Agile” is another example. 

Overall Summary

On the surface, Lean and Agile might appear to be in conflict with each other. If you pursued each approach individually and mechanically without really understanding the principles behind each at a deeper level, they could easily be in conflict. 

On the other hand, if you take take a systems-thinking approach to understand these seemingly disparate approaches at a deeper level. you will begin to develop a fresh new perspective to see them as complementary to each other rather than competitive.

Michael made a key point that it is a matter of focusing on value versus control and he’s absolutely right.  Here are some ways a PMO could add value:

  • Better defining processes and tools,
  • Providing training to people, and
  • Doing some level of project portfolio management and resource planning of people

Each of those can potentially add value; however, it does take some skill to determine the optimum point beyond which it stops producing value and starts to become dysfunctional.

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

What’s Next After Agile? Is There Something Else Coming Next?

I recently saw a discussion on an online forum where an individual raised the question of “What’s next after Agile?”

  • Someone speculated that the next big methodology might be Lean
  • I’ve also seen some people suggest that Kanban will become the next big methodology

I’ve seen this pattern before – I call it the “Program Du Jour” pattern.  When something new comes along, everyone wants to jump on the bandwagon.

What's Next After Agile
Bandwagon with kids on a white background

Here’s one of my favorite quotes on this subject:

The “Program du Jour” Effect

“Americans are our own worst enemy when it comes to new business concepts. We love novelty and newness. We become so enamored with new ideas, we burn through them the way a child rips through toys on Christmas morning – squeals of delight, followed by three or four minutes of interest, then onto the next plaything. That is our pattern with new management techniques, too.”

Barry Sheehy, Hyler Bracey, & Rick Frazier, Winning the Race for Value, American Management Association, 1996

The above quote was about business concepts and management techniques but the same thing can be said about methodologies.

An Example – Six Sigma

Here’s an example:

  • When Six Sigma came into vogue in the 1990’s and early 2000’s, it was really hot, everyone wanted to jump on the Six Sigma bandwagon, and
  • Any other earlier process improvement approach was considered obsolete and passé.  

I published my first book on Business Excellence in 2003 and I interviewed a number of companies for my book at that time.  What I saw was that:

Superficial, Mechanical Implementation

Many companies were doing Six Sigma very superficially and mechanically.

  • In these companies there was a lot of “hoopla” and very visible ceremonies about Six Sigma including Black Belts, Green Belts, etc. 
  • The implementation in many of these companies was not very successful because the company was looking for a “silver bullet” and
  • When it didn’t meet their expectations, the company tossed it out and started looking for the next “silver bullet”.

More Successful Implementation

In other companies where I thought Six Sigma was more successful and lasting, there was a big difference.  

  • Six Sigma was seen only as a tool and not a “silver bullet” or panacea,
  • People in the company understood Six Sigma at a deeper level, and
  • The implementation was not just mechanical and superficial.  
  • Six Sigma was so well-integrated into the way the company did business that it might not even have been very visible that it was Six Sigma and
  • They might not even have called it “Six Sigma”

How Does That Apply to Agile Today?

I see a similar pattern with Agile today. 

  • Many people today see “Agile” as a “silver bullet” or panacea for almost any problem you might have.
  • In many cases the implementation of Agile is superficial and mechanical; and,
  • When it doesn’t work, there’s a tendency to toss it out and look for something new to replace it. 
  • I think that kind of thinking has some serious flaws.

Rather than Agile being replaced by something new, what I hope that will happen is that:

People’s Understanding of Agile Will Mature

People’s understanding of Agile will mature. They will start to understand the principles and values behind it at a deeper level, and they will go beyond superficial and mechanical implementations

People Will Stop Seeing Agile as a Panacea

People will stop seeing Agile as a “panacea” or “silver bullet” for any problem you might have. Rather than force-fitting all problems to some particular methodology like Agile, they will recognize the need to go in the other direction and fit the methodology to the problem

Agile” Does Not Make All Other Management Approaches Obsolete

People will also recognize that “Agile” does not make all other management approaches obsolete and:

  • There’s a need to see Agile and more traditional plan-driven approaches in a fresh new perspective as complementary rather being competitive
  • Various Agile approaches such as Scrum, Kanban, and Lean are also complementary to each other rather than competitive

Trends in Agile Development and Project Management

Although I don’t see something else replacing Agile, there are a number of trends that seem evident to me:

1. Convergence

Traditional plan-driven project management is beginning to converge with Agile. Agile started out as a revolution against traditional plan-driven project management practices (what many people loosely call “Waterfall”). That pendulum is starting to swing back to the middle. 

  • People are beginning to recognize that there isn’t really a binary and mutually-exclusive choice between “Agile” and “Waterfall”.
  • Rather than force-fitting a project to one of those extremes, a better solution is to fit the methodology to the nature of the problem. That may require a blend of both approaches in the right proportions to fit the situation.

2. Hybrid Approaches

Learning how to blend those approaches together requires understanding a broader range of methodologies at a deeper level.

  • Many people today do Agile somewhat mechanically “by the book” without really understanding the principles behind it.
  • That results in a somewhat rigid approach to how to apply Agile. That is exactly the opposite of the adaptive approach that is intended for Agile.

3. Scaling Agile

Many companies and people are attempting to scale Agile to larger and more complex, enterprise-level projects. That will accelerate both of the above trends. Agile was originally designed around small, simple, single-team projects and it can be difficult to scale. Scaling Agile often requires thinking about how to blend it with typical enterprise-level management practices. Those practices include project/program management, project/product portfolio management, and overall business management.

4. Enterprise-level Agile Transformations

Sometimes, an attempt is made to force a whole company to be agile in order to adopt an Agile development approach. That just isn’t completely realistic or desirable in some cases. Becoming Agile is not necessarily a goal in itself. It has to be applied in the context of the company’s most critical business objectives. What problem will it solve and how will it solve it?

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.

What is an Enterprise Agile Coach?

What is an Enterprise-level Agile Coach? There are very different Agile Coach roles and when people use the term “Agile Coach” it is often not exactly clear what role that they are referring to.

Enterprise Agile Coach

Typical Team-level Agile Coach Role

Most often, what they’re talking about as an “Agile Coach” is what I would call a team-level Agile Coach. Someone in that role works at a tactical level with individual members of an Agile team to help them become more proficient in executing a Scrum process.

What Is an Enterprise Agile Coach?

The role of an Agile Coach at an enterprise-level needs to be better-defined and differentiated from a normal team-level “Agile Coach” role. Beyond the team-level Agile Coach role, an enterprise-level Agile Coach:

  • Helps companies design and implement an effective Agile transformation for their business
  • Works at a more strategic level to integrate an Agile development process with a company’s business. (See diagram above)

An enterprise-level Agile Coach should be able to see the need for an Agile transformation from a broader business perspective. Check out this article for more on that:

The Impact of the Business Environment

The problem is that there is a big difference between companies whose primary business is focused on product development and other types of businesses.

Product Development Companies

  • Agile works very well in companies that are in the primary business of developing products (particularly software products). Intuit is an example that develops TurboTax, Quicken, and QuickBooks).
  • In those companies, there is a strong and natural alignment between an Agile development process and the overall business goals of the company
  • It is very easy to apply an Agile development process in that environment.

Non-Product Development Companies

It is much more difficult to apply an Agile development process in a company that is not in the primary business of developing products. In that kind of business, the relationship of an Agile development process to the company’s overall business strategy is much more indirect.

In companies that are not in the primary business of developing products:

  • You can’t just force the company to be “Agile” in order to make the company more amenable to an Agile development process
  • The company’s overall culture and business strategy needs to be optimized around the critical success factors for that business

Fitting the Approach to the Business

An enterprise-level Agile Coach role can be very challenging. It’s important to fit the approach to the business rather than force-fitting the business to some arbitrary approach, Here’s an example:

  • If a company is in a business that requires operational excellence, it needs to focus its overall culture and business strategy primarily on efficiency of operations and reducing costs and
  • That doesn’t necessarily align completely with just becoming more “Agile”.
  • In that kind of environment, you have to develop a strategy that considers both the company’s business strategy and the requirements of an Agile development process to develop a well-integrated approach
  • The implementation of that strategy often requires fitting the approach to the company’s business environment rather than simply trying to force-fit the company to some kind of overall Agile approach

Blending Agile and Plan-Driven Project Management

The solution in that kind of environment could be a blend of Agile and traditional plan-driven management principles and practices. That is a lot more difficult thing to do and requires a lot more skill than a typical team-level Agile coach would normally have. It requires an understanding of:

  • Agile principles and practices; as well as
  • Traditional project management principles and practices
  • And a deeper understanding of the principles behind both of them to know how to blend them together as necessary to fit a given situation

Business Perspective

An enterprise Agile Coach should have the ability to look at a very complex, broad-based, enterprise-level business from both:

  • A more strategic high-level business management perspective as well as
  • A more tactical product development process perspective to develop a strategy for integrating the two.

Overall Summary

An Enterprise Agile Coach is a different kind of Agile Coach role:

  • Instead of working at a team-level on improving team performance,
  • He/she needs to work at a much higher strategic level to help a company fit an Agile approach to their business

Additional Resources

You will find much more detail on this in my Online Agile Project Management Training.