What is the Real Essence of Agile?

It’s apparent to me that a lot of people have gotten so heavily focused on the mechanics of how Agile is implemented that they’ve lost sight of the big picture of what “Agile” is really all about.  The term “Agile” has taken on a number of different meanings today that are largely based on how it is implemented.  For many people, “Agile” has become synonymous with Scrum and if you’re not doing Scrum and doing it “by the book”, you’re not really Agile at all.  I think it is useful to step back and take a look at “What is the real essence of Agile”?

What is the Real Essence of Agile?

What is the Real Essence of Agile?

The real essence of “Agile”, in my opinion, is that it puts an emphasis on being adaptive to customer and business needs in order to maximize the value of the solution rather than following a rigidly-defined plan with an emphasis on managing costs and schedules of delivering the solution. For that reason, I like to use the terms “adaptive” and “plan-driven” rather than terms like “Agile” and “Waterfall”.

  • The terms “Agile” and “Waterfall” make it sound like you’re comparing two specific methodologies – one called “Agile” and one called “Waterfall” and that’s not really accurate.  “Agile” is not really a specific methodology or framework like Scrum; it is much broader than that – it is a way of thinking defined by the Agile Manifesto
  • The “Agile versus Waterfall” kind of thinking leads people to think that there is a binary and mutually-exclusive choice between those two approaches and that causes people  to try to force-fit a project to one of those extremes.  The right approach is to go in the other direction and fit the methodology to the nature of the problem and sometimes that requires a blending the two approaches in the right proportions to fit the problem

When Does Agile Work Best?

Agile works best in situations that have a high-level of uncertainty where it isn’t practical or possible to define the solution to the problem upfront.  In that kind of situation, it is much more effective to use an adaptive approach to incrementally and iteratively define the solution in more detail as the project is in progress rather than attempting to define detailed requirements for the solution upfront.

The example I like to use to illustrate this is finding a cure for cancer.   If you set out to define a project to find a cure for cancer, it would be ridiculous to try to define a detailed plan upfront; there’s just far too much uncertainty.  The only approach that is likely to work is an iterative, trial-and-error approach to find a solution.

Agile is based on what is called an “Empirical Process Control” model.  The word “Empirical” means “based on observation” and that means that in an Agile project, both the requirements for the solution AND the process for developing the solution are continuously refined as the project is in progress.

What’s the Alternative?

That kind of empirical process control model approach is naturally not the most efficient approach if there is a low level of uncertainty and it is possible to easily define detailed requirements for the solution prior to the start of the project. In that situation, a trial-and-error approach really isn’t necessary and a plan-driven approach is much more appropriate and much more efficient.

The example I like to use to illustrate this scenario is building a bridge across a river. If you were defining a project to construct a bridge across a river, it would be ridiculous to say “we’ll take an adaptive approach, build the first span of the bridge, and decide how we will build the rest of the bridge after we’ve completed the first span. That wouldn’t make any sense.

This kind of situation calls for a plan-driven approach that is based on a defined process control model. The key advantage of a defined process model is that it is predictable and repeatable and it is probably more efficient for projects with a low level of uncertainty. If you designed a process for building a bridge and it has proven successful once, the same process or a variation of that process is likely to work in another similar situation with somewhat predictable results.

What if it is Between Those Extremes?

Very few real world situations are as extreme and clear-cut as the ones I’ve used of finding a cure for cancer and building a bridge across a river. Most real-world situations fall somewhere between those two extremes. There is some level of uncertainty but it’s not complete uncertainty. You rarely start any project without at least some idea of what the solution is going to look like although the solution may be progressively refined in more detail as the project is in progress. There’s a range of approaches that look something like this:

Range of Agility

In this kind of situation, you have to tailor the approach to fit the nature of the project and one of the biggest factors to consider is the level of uncertainty associated with the solution. That requires more skill but it definitely can be done. It requires knowledge of a broader range of methodologies (both plan-driven and adaptive (Agile)) and it requires a deeper knowledge of the principles behind the methodologies in order to know how to tailor them to fit a given situation. You can’t just force-fit a situation to some predefined methodology (whatever it might be) and do it mechanically.

In my book on Agile Project Management, I use the analogy of a project manager as a “cook” versus a project manager as a “chef”. A “cook” knows how to prepare a few simple “recipes” by the book; a “chef” knows how to prepare a much broader range of recipes with more exotic ingredients and even knows how to improvise his/her own recipes when required. This clearly raises the bar for a lot of project managers – it is no longer sufficient in many situations to force-fit a project to some predefined approach (whatever it might be). You have to fit the approach to the nature of the problem and that’s exactly the challenge that my online Agile Project Management training is designed to address.

3 thoughts on “What is the Real Essence of Agile?”

  1. Agile can still be used for projects that you know the entire Scope of and assume there will never be any change to.

    It isn’t just about being adaptive, it allows you to design/code/test /show to the business in small increments.

    Using BDD, you will end up with tests that match acceptance criteria, produce living documentation – and the tests will be usable in Staging and Production environments (and for Regression Testing).

    Before my near 13 years of Agile, I did 17 years of Waterfall. One key problem was that if you test something months down the line, the memory of code (if it needs a change / defect fix) is not fresh in the minds of the coders – either that or they were contractors who had left. With Agile everything is recent/fresh – and the whole Team is on had to ensure it’s quality and acceptability to the Business.

    1. Hi Craig,

      I agree that an incremental development approach is a good approach that can work well even if there is a low level of uncertainty about the requirements; however, if the requirements for the project are truly frozen and are not expected to change at all for the entire duration of the project, many people would say that is not an “Agile” project, it is simply an incremental development approach; however, I think that’s primarily a problem with terminology. There are a lot of problems with terminology in this area that can be very confusing:

      • Many people seem to think that there is a binary and mutually-exclusive choice between “Agile” and “Waterfall”;
      • The terms “Agile” and “Waterfall” are also used very loosely in practice as if there were two discrete methodologies – one called “Agile” and one called “Waterfall”; and anything that is not “Waterfall” must be “Agile”

      The truth is that there are many different methodologies that are not limited to “Agile” and “Waterfall” and you have to fit the methodology to the nature of the problem rather than force-fitting all problems to “Agile” or “Waterfall”. For example, there are a lot of incremental and iterative development approaches that have been around for years that may or may not be considered to be “Agile” (RUP and the variations of RUP are an example). In the situation you described, I wouldn’t be too concerned about whether you call it “Agile” or “Waterfall” because the usage of those terms can be so misleading.

      Please note also that I didn’t say that Agile only works in situations that have high levels of uncertainty that require an adaptive approach; what I said is that an Agile (adaptive) approach works best in situations with high levels of uncertainty that require some level of adaptivity.

      Thanks,

      Chuck

Comments are closed.