Category Archives: Hybrid Agile Methodolology

How to Make a Hybrid Agile Process Work and Not Fail

Have you given any thought to “How to make a hybrid agile process work”? I recently saw an article on LinkedIn entitled “Why Hybrid Agile-Waterfall Projects Fail” that caught my eye.  I’m not surprised at this article.  It takes some skill to make a hybrid Agile process work successfully and anyone who would literally try to combine an Agile methodology with a Waterfall methodology to create a hybrid methodology is asking for failure – that’s not the way to do it.

Blending Two Recipes

This should not be a matter of literally combining an Agile methodology and a Waterfall methodology. That would be like trying to take a recipe for Italian food and combine it with a recipe for Chinese food and literally trying to mix the steps and ingredients from the two recipes together to create “Italian-Chinese” food.  A good chef would never even attempt to do that but he/she might create an entirely new recipe that blends together some of the aspects of Italian food preparation with some aspects of Chinese food preparation.

It takes a lot of skill to create an effective hybrid approach and doing it effectively is like the difference between a “chef” and a “cook”.  If you presented a “cook” with a recipe for Italian food and a recipe for Chinese food and asked him to combine the two, I’m not sure what you would get if he/she literally tried to combine the two recipes.  It just doesn’t even make much sense. A “chef”, on the other hand, would take an entirely different approach to create a hybrid of the two.   A chef might not need recipes at all because he/she understands the art at a deeper level.

Creating a Hybrid Approach

When I talk about creating a hybrid approach, I try to avoid the words “Agile” and “Waterfall” if possible because it gives people the impression that you are literally combining two different methodologies together like combining two food recipes.   I prefer to think of a hybrid approach as the appropriate blend of an adaptive approach and a plan-driven approach.  The words “adaptive” and “plan-driven” convey an entirely different meaning than “Agile” and “Waterfall”.

  • It doesn’t mean that you’re trying to literally combine two very different methodologies by mixing them together
  • It means that you’re creating a blended approach with the appropriate amount of emphasis on being “adaptive” versus being “plan-driven”

An Example of a Real Hybrid Approach

Some years ago, I was responsible for managing a very large development program for a US Federal Government agency.  It had a fixed-price contract associated with it and a fairly aggressive delivery schedule but the customer wanted some level of flexibility in the details associated with defining requirements.  I created an approach for this situation that was very successful that I have called “The Managed Agile Development Framework“.

  • It does not attempt to literally combine an Agile methodology and a Waterfall methodology
  • It is simply a matter of wrapping a plan-driven layer at the “macro” level around a standard Agile/Scrum process at the micro level

Here’s a diagram that shows what it looks like:

Managed Agile Development Framework
You might say “That’s not a hybrid Agile-Waterfall model – it’s only a standard Agile development process with a plan-driven layer wrapped around it” and you would be correct.  There has been no attempt to actually mix-and match a phase-gate Waterfall methodology with an Agile methodology.

How To Make a Hybrid Agile Process Work

This is a very flexible model – that is why I call it a “framework” rather than a “methodology”.

  1. You would start out a project by at least developing high-level requirements with estimates of the costs and schedule for completing the project.  Those estimates can be as detailed as you want them to be – that’s what makes this model so flexible.
    • You can have a relatively “thin” layer of planning with very high-level requirements and less-detailed plans or
    • A “thicker” layer of planning with more detailed requirements and planning

    The most important thing is that there has to be a common understanding between the development team and the customer about how detailed the requirements and plan are.
    This model will not work without a spirit of trust and partnership between the customer and the project team.  The customer and the project team must agree to working collaboratively as the project is in progress to further elaborate requirements, continuously refining the plan for completing the project, and resolving any trade-offs that may be necessary to stay within the budgeted cost and schedule.

  2. As the project is in progress, any changes to the project requirements made at the “micro” level need to be fed back into the planning at the “macro” level. By agreement with the customer, there should be enough slack built into the plan so that minor changes can be absorbed easily without a change to the overall plan and only more significant changes might require trade-offs and adjustments. When trade-offs are needed to stay within the budgeted cost and schedule, the customer may need to either:
    • Adjust the planned cost and schedule as necessary, or
    • Reduce the scope of some of the requirements if necessary to fit within the planned cost and schedule.

This is a fairly simple model but it works and it can be easily adapted to a wide variety of projects; however, note that this is not a simple step-by-step cookbook approach with detailed checklists of what to do and fill-in-the-blanks document templates that some project managers are used to. It takes some skill to make this model work successfully.

Matching the Level of Uncertainty in the Project

A good strategy is to match the design of the hybrid approach to the level of uncertainty in the project:

  • A project with a lower level of uncertainty might lend itself to a more plan-driven approach particularly if predictability is important to the customer of the project.  That would mean putting a higher level of emphasis on developing a “thicker” layer of planning at the  macro-level
  • A project with a higher level of uncertainty would probably require a more adaptive approach to further elaborate the plan as the project is in progress.  That would mean that the level of planning at the macro-level would probably be much “thinner”.

What is a “Hybrid Agile” Approach? Is There Such a Thing?

What is a hybrid Agile Approach? Is there such a thing? I recently came across an article on the Internet that was posted in several places entitled “The Moment of truth: There Is No Hybrid Agile“. This article is so full of stereotypes and misconceptions about “Agile” and “Waterfall” that I felt that I had to respond to it.  It is typical of many articles that position “Agile” and “Waterfall” as two binary and mutually-exclusive alternatives with no middle ground between the two.

What Are the Flaws in This Thinking?

Here are some of the flaws in this thinking:

    1. This article and many others like it treat “Agile” and “Waterfall” as if they were individual, discrete methodologies and they position “Agile” and “Waterfall”  as diametrical opposites of each other.  That’s not very accurate.

“Agile” and “Waterfall” are not really discrete, individual methodologies and both of those terms are used very loosely.  In common usage, neither of those are individual, discrete methodologies:

      • Many people  may think of “Agile” as being synonymous with Scrum but that is not really accurate.  “Agile” is much broader than Scrum – it is a way of thinking defined by the Agile Manifesto
      • “Waterfall” is also not a single, discrete methodology – in today’s world, many people use the term “Waterfall” for any plan-driven methodology that is not Agile.  What about RUP and other iterative approaches that probably wouldn’t be considered to be Agile?  Is that “Waterfall”?

Instead of thinking of what people commonly call “Agile and “Waterfall” as individual discrete methodologies, it is more accurate to see it as a continuous spectrum of approaches from heavily plan-driven at one extreme to heavily adaptive at the other extreme like this:

What is a hybrid agile approach?

If you think of it in that way, it is much easier to see the possibility for lots of approaches in the middle of that spectrum that blend the right level of plan-driven principles and practices with more adaptive principles and practices to fit a given situation.

Here’s what some methodologies would look like plotted on a spectrum of heavily plan-driven versus heavily adaptive:

Adaptive vs Plan-driven

As you can see from this diagram:

      • “Agile” is not a single approach and there is not just one way to do “Agile”:
        • Kanban is more adaptive than Scrum, and
        • Even within Scrum you will find different styles of implementation from simple team-level projects which may tend to be more adaptive to larger more complex multi-team projects which may tend to be somewhat more plan-driven
      • What people commonly call “Waterfall” is also not a single well-defined approach.
        • At one extreme, the original phase-gate model originally developed by Winston Royce in 1970 was very plan-driven but that approach is not widely-practiced any more in that form for software development
        • In the 1990’s and early 2000’s more iterative approaches such as RUP became much more popular for software development and provided a higher level of adaptivity

The most important point to get out of this is that there is not a clear and well-defined boundary line between “Agile” and what people commonly call “Waterfall” as many people seem to think.

  1. Many people make the mistake of performing a methodology mechanically and think they need to do it religiously and “by the book”(That’s true of both Agile and other non-Agile methodologies)
    • The right approach is to fit the methodology to the nature of the problem rather than force-fitting all problems to a given methodology (Agile or non-Agile).
    • It takes more skill to do that but it definitely can be done. It requires understanding the principles behind the methodology and why they make sense in a given situation rather than doing a given methodology mechanically.

    If you think of methodologies as being rigid and prescriptive, it will be difficult to see how two seemingly disparate methodologies could be blended together in the right proportions to fit a given situation. On the other hand, if you understand the principles behind the methodologies at a deeper level, it is much easier to see how they could be complementary to each other rather than competitive.

    Learning to be a “Chef”

    It can take a lot more skill to learn how to blend different approaches together in the right proportions to fit a given situation. In my book on Agile Project Management, I use the analogy of a project manager as a “cook” and a project manager as a “chef”.

    • “A good ‘cook’ may have the ability to create some very good meals, but those dishes may be limited to a repertoire of standard dishes, and his/her knowledge of how to prepare those meals may be primarily based on following some predefined recipes out of a cookbook.”
    • “A ‘chef’, on the other hand, typically has a far greater ability to prepare a much broader range of more sophisticated dishes using much more exotic ingredients in some cases. His/her knowledge of how to prepare those meals is not limited to predefined recipes, and in many cases, a chef will create entirely new and innovative recipes for a given situation. The best chefs are not limited to a single cuisine and are capable of combining dishes from entirely different kinds of cuisine.

    That’s the challenge for project managers and agile practitioners in today’s world – we need more chefs and fewer cooks.

What is a Hybrid Agile Approach?

In simple terms, a hybrid Agile approach is one that blends the plan-driven principles and practices with Agile (adaptive) principles and practices in the right proportions to fit a given situation. An example of that is the Managed Agile Development framework that I created. It simply wraps an outer layer of project-level planning around an Agile development process.

Managed Agile Development Framework

The outer layer can be as thick or thin as necessary to fit the situation. I originally developed this framework when I was managing a very large government program for a US government agency. The government agency had to have some level of predictability over the costs and schedules of the program. The program was so large that it actually had some level of congressional oversight so some level of predictability and control was essential; however, within that outer envelope, the government agency customer wanted to have flexibility in many of the detailed requirements. We were able to find the right balance of control and flexibility to satisfy both needs.

What Are Examples of Hybrid Agile Approaches?

Some of the most common examples of hybrid Agile approaches are:

    1. Agile Contracts
      • The government program I mentioned is a good example
      • I also have a case study in my book on General Dynamics UK, Ltd. who successfully used a hybrid Agile approach to manage a large defense contract for the ministry of defense in the UK
      • I just finished building a new house and I naturally had a contract with the builder that defined the cost and schedule for the home; however, the builder offered a lot of flexibility to make changes even as the construction of the house was in progress (He charges for changes, of course)
    2. Large, Enterprise-level Projects and ProgramsIt’s almost impossible to successfully implement some large complex enterprise-level projects and programs without integrating some level of project and program management. A good example of that is the Harvard Pilgrim Healthcare case study that is written up in my latest book. The project involved over 100 Agile teams and involved replacing almost everyone of HPHC’s most critical business systems over a period of time. The whole effort involved a lot of moving parts that had to be carefully planned and synchronized and it’s impossible to imagine how that could be done without a sufficient level of project and program management to guide and manage the overall effort.
    3. Other Business-driven Initiatives

Many people have the mistaken belief that you need to force the entire company to be agile in order to adopt an Agile development approach. That isn’t necessarily true. A business has to be designed around whatever critical success factors are most important for the business that they’re in and becoming agile may not be the only factor and may not even be the most significant factor. For example, some companies are in very cost-competitive industries and succeed primarily based on operational excellence to lower their costs as much as possible. Becoming more agile may play an indirect role in that but it isn’t necessarily the most important factor. On the other hand, in a company that is technology-driven that succeeds on bringing leading-edge products to market as quickly as possible, it’s much easier to see how a pure Agile approach might be a very strong and direct driver of the business.

Agile was originally developed for companies that do product development and that’s where it works best. In companies whose primary business is not developing products per se, there is typically more of a project-oriented approach. The company has to typically evaluate a potential portfolio of projects to determine what mix of projects and programs is going to have the greatest impact on their business and then they need to monitor the execution of those projects and programs to determine if it is really delivering the expected returns.

When is Agile NOT Totally Appropriate?

I am often asked by my students “When is Agile NOT Totally Appropriate?”. 

When is Agile NOT Totally Appropriate?
Here’s a good example…

My wife and I are in the process of purchasing a new home that is now under construction.  This is the second new construction home we’ve bought in our lifetime.  The first time we built a new construction home, the builder did not allow much customization at all – there were very few choices that the builder allowed making in the house.   He gave us a price for the house and we had only a few minor choices to make.  It was clearly a traditional, plan-driven project management approach – changes were very limited and there was a very disciplined and well-documented process for managing any changes.

The builder that we’re currently working with is not that way – there are several different floor plans to choose from and almost anything can be customized.  For example, my neighbor wanted a 3-car garage instead of a 2-car garage and that was no problem (he had to pay for it of course).  Aside from customizations, there are also many detailed decisions that need to be made to select light fixtures, plumbing fixtures, floor tiles, cabinets, countertops, etc.

This is definitely more of an Agile approach that is adaptive to customer needs but there’s definitely a downside of allowing this level of customization without some level of control.  This particular builder has at least 20 houses in various stages of construction and all of them have some level of customization; so the builder has a lot of balls in the air at the same time and some problems have become apparent:

  • The first problem we experienced was when the foundation for the house was poured and we found that the floor plan was a mirror image of what we were expecting
  • Recently we did a walk-through of the house to see how it was coming along and we found several things that didn’t match what we thought we had agreed to with the builder
    • There was a window in the sunroom where there was supposed to be a sliding door
    • There was a door to a hallway where there should have been just an entranceway with no door
  • My next-door neighbors had problems with cabinets in their kitchen not fitting in as they expected them to fit

This is a perfect example of the need to blend Agile and traditional plan-driven project management practices in the right proportions to fit the situation.  This builder has been very responsive to customer needs for customization; however, there is still a need for a more disciplined approach to stakeholder management and  change control normally found in a traditional, plan-driven project management approach to manage all these detailed decisions and changes.  The builder corrected all of these problems but I’m sure that there was a cost involved in making those corrections that could have been avoided by doing it right the first time.

Another key point is that it is also not a binary and mutually-exclusive choice between a totally uncontrolled “Agile” approach and a rigidly planned and controlled “Waterfall” approach as many people seem to think.  It is very possible to blend a level of flexibility and adaptability to be responsive to customer needs and changes and still have a sufficient level of control to manage the project effectively.  It takes more skill to do that but it definitely can be done.  Having the right processes, systems, and tools to support that approach is also essential.   In this particular situation, there were some obvious problems:

      1. There was a contract defining what the builder will deliver; however, there have been multiple revisions of the contract still in circulation that have not been well-controlled so it is difficult to determine what has been agreed to and what has not been agreed to
      2. The process for making changes was not well-defined and was somewhat loose.   Many changes were often based on an email conversation or, even worse, on a simple hallway conversation that was not well-documented at all

The key lesson to be learned from this is that you need to fit the approach and the methodology to the nature of the situation rather than force-fitting all projects to either a pure “Agile” or pure “Waterfall” approach.  And, sometimes that requires blending Agile and traditional, plan-driven project management principles and practices in the right proportion to fit the situation.