I’m sure that you could find at least 100 definitions of what “Agile” is on the Internet. It’s like the old fable of “The Blind Men Feeling the Elephant”:
“It was six men of Indostan To learning much inclined, Who went to see the Elephant (Though all of them were blind), That each by observation Might satisfy his mind”
Each man came away with a different opinion depending on which part of the elephant he touched:
“Each in his own opinion concludes that the elephant is like a wall, snake, spear, tree, fan or rope, depending upon where they had touched”
Different Views of Agile
The way people define Agile is somewhat similar. Depending on which part you touch, you may come away with a different impression of what Agile is:
Many people will define Agile in terms of how it is done. For example, many people will say that it is defined by the Agile Manifesto. Here are a couple of examples of that:
“Agile is a set of methods and frameworks that embody the principles and values of the Agile Manifesto”
“Agile is a term used to describe approaches to software development emphasizing incremental delivery, team collaboration, continual planning, and continual learning. The term “Agile” was coined in 2001 in the Agile Manifesto.”
Some people will say that it is just a mindset or way of thinking. Here’s an example of that.
“Being ‘Agile Is a mindset. It’s about finding the right thing to build, faster (and not just building things faster)”
Some people will define it by comparing it to “Waterfall” to tell you what it is not. Here’s an example of that:
“Agile is a time boxed, iterative approach to software delivery that builds software incrementally from the start of the project, instead of trying to deliver it all at once near the end.”
None of those definitions are incorrect, but it’s like “feeling the elephant”; they don’t really get to the real essence of what the “elephant” is, in my opinion.
How Do You Define Agile?
Personally, I like the definition that is published by the Agile Alliance:
“The ability to create and respond to change in order to succeed in an uncertain and turbulent environment.”
I think that is a better definition because it gets to what I consider the real essence of what Agile is.
“Agile is best suited for situations that have some level of uncertainty where creativity and innovation are important to maximize the business value of the solution as opposed to other situations with lower levels of uncertainty where planning and control to achieve predictability are more important.” (My own definition)
What Problems Does Agile Solve?
We should acknowledge that Agile is not a solution to every problem you might have. One way to define it is in terms of what problems it is useful for.
If you accept that Agile is not a solution to every problem you might have, the first thing you would want to know is “what problems is it useful for?”
You don’t need to get too far into the mechanics of how to do it until you’ve determined that it is a useful solution to the problem you’re trying to solve.
There is an interesting observation you might draw from this – people get very immersed in the mechanics of how to do Agile and that’s how they define it. Isn’t it more important to know what problems it’s useful for solving before you get into the details of how to use it?
Many people have asked about “Applying Agile to non-software projects”. I’ve done a lot of that myself in using an Agile book publishing approach for publishing five books. I’ve also used similar techniques in designing and developing numerous online training courses. I thought I would summarize some of the techniques I’ve learned from doing Agile Book Publishing over the years.
Agile Common-Sense Principles
Here are some common-sense principles I’ve learned from using an Agile book publishing approach for publishing five books:
1. Just Get Started
One of the most important principles I’ve learned is “just get started”. When you’re faced with writing a book or developing a major online training course, it can be a daunting experience. Just getting started is sometimes the hardest part:
We’re not sure how the final result is going to come out
We’re not certain how the final result will be structured – what should come first, etc.
We don’t want to produce something that is going to be a failure
You have to stop worrying about all of that and just get started. I think of an Agile book publishing effort like developing a fine art sculpture. You start with a lump of clay and you just keep molding and shaping it until it becomes a work-of-art.
If you never get started, it will remain just a lump of clay.
It takes some courage and confidence in yourself to do this. You’ve got to have courage and confidence that if you just get started, that somehow the final result is going to come out OK if you keep working at it
It also takes patience and commitment because you may have to go through a large number of iterations to get something useful out of it. You may even have to throw something completely away and start over again
2. Use an Incremental and Iterative Approach
Many people don’t understand the difference between the words “incremental” and “iterative”. An Agile book publishing approach involves both:
“Incremental” means that you break up a solution into pieces and develop one piece (increment) at a time
“Iterative” means that if you’re not sure what a given piece should be, you develop something and then continue to refine that piece until you meet the customer’s expectations
Both of those are important:
Using an incremental approach is very important. In any large effort like writing a book or developing an online course, its best to break it up into “bite-sized pieces. If you try to take on too much at once, you’ll never finish it. The effort to write a book can easily take well over a year and it’s easy to get discouraged in that period of time that you will never finish if you don’t see progress in the work.
Taking an iterative approach is also important. A close corollary to “Just Get Started” is “Don’t Expect Perfection”. A major reason for not getting started sometimes is that we’re afraid to produce something that is less than perfect.
We have to accept that whatever we produce on the first iteration is certainly not going to be perfect. The final result may not be perfect either.
Get something done quickly and then continue to refine it as needed to meet customer expectations.
There’s also a saying in Agile that is used a lot called “just barely good enough”.
We shouldn’t try to “gold-plate” or over-design something. It should satisfy the need to provide value to the customer and nothing more. Keep it simple!
3. Know Your Customers and Listen to Them
When I first started writing books, I had a lot of people who helped me. I had a network of people on the Internet who provided me with lots of great feedback and inputs. I would write a chapter or two of the book and put it out to my network for feedback and comments. Part of doing that successfully is recognizing that you “don’t know what you don’t know”. You have to be humble, listen to other people, and respect their needs and interests. If you think that you know it all, you probably won’t be very successful.
In the online training I develop, I get lots of feedback and inputs from students and I listen to it and take action. As a result of that feedback, I have continuously improved all of my courses.
4. Refactor Your Work As You Go Along
I’ve done a fair amount of software development in my career and I’ve learned a lot from it.
I’ve learned the value of having clean and well-organized software because I have had to support a lot of the software I’ve developed
Organization and flow of the material is particularly important in books and online training as well
As a result, it is essential to take time to go back and clean-up your work as necessary as you go along
When I first write something, I get it done quickly but I may have to rewrite it and reorganize it 5-6 times before I’m satisfied with it
5. Work at a Sustainable Pace and Do a Little at a Time
When you’re doing a long project like writing a book, working at a sustainable pace is very important. That is especially true if it requires a lot of creativity and innovation.
You can easily get burned out by trying to do too much too quickly and when that happens, your creativity can go downhill quickly.
Sometimes you need to put it down, walk away from it for a while, and come back when you’re refreshed to start work again.
Agile Book Publishing – Overall Summary
I think all of this is just good, common-sense things to do – why do people have trouble doing this?
I think many people think of Agile as Scrum and also think about doing it mechanically. They aren’t sure how they would go about applying a Scrum process to this kind of effort
Agile is not just Scrum – it is a way of thinking. We need to understand the principles behind Agile. Don’t just do it mechanically and “by-the-book”. Take the time to interpret how it applies to your current situation and adapt it as necessary
What can Physics teach us about Agile Project Management? We can learn a lot from how the science of physics has evolved. I think there are a number of interesting similarities the way that Agile Project Management is evolving.
How Has the Science of Physics Evolved?
For many years until the late 1800’s and early 1900’s, physics was based on what is called “Classical Physics”.
What is Classical Physics?
“Classical physics is the physics of everyday phenomena of nature, those we can observe with our unaided senses. It deals primarily with mass, force and motion.
While its roots go back to the earliest times, to the Ancient Greeks such as Aristotle and Archimedes,
It later developed into a cohesive system with the contributions of Galileo, Kepler and Newton.
Classical physics achieved phenomenal success, as the Calculus of Newton and Leibniz gave it the tools to tackle even problems not imagined by its pioneers.”
How Has Classical Physics Evolved?
“Around 1900, give or take a decade, surprising new experimental evidence, primarily about atoms and molecules:
Showed us that these small-scale phenomena behave in ways not anticipated by classical theory
This ushered in a new era called “modern” physics.
New laws and methodology were developed to deal with the rapidly expanding experimental evidence.
Relativity and quantum mechanics added new tools to the study of nature.”
“These did not make classical physics “wrong”, for the old laws were working just as they always had, within their limited scope—which was the study of large objects (not atomic scale ones) moving relatively slowly (not near the speed of light). “
“So classical physics is still the starting point for learning about physics, and constitutes the bulk of the material in most introductory textbooks.
What Happened to Cause People to Rethink Classical Physics?
That notion of physics that was intended to define how the entire universe worked held together for a long time; however, serious weaknesses began to appear around the early 1900’s:
“By the end of the nineteenth century, most physicists were feeling quite smug. They seemed to have theories in place that would explain all physical phenomena. There was clearly a lot of cleaning up to do, but it looked like a fairly mechanical job: turn the crank on the calculator until the results come out. Apart from a few niggling problems like those lines in the light emitted by gas discharges, and the apparent dependence of the mass of high-speed electrons on their velocity”
Slavin, Alan J., “A Brief History and Philosophy of Physics”, Trent University
How is This Transformation Related to Agile Project Management?
Classical Physics Is Analogous to Traditional Plan-driven Project Management
Classical Physics is analogous to traditional, plan-driven project management. Similar to the laws of classical physics:
The traditional, plan-driven project management approach has been widely accepted as the only way to do project management for a long time
The way traditional, plan-driven project management is done hasn’t changed significantly since the 1950’s and 1960’s
It assumes a very predictable view of the world where it was possible to completely define a project plan with a fairly high level of certainty prior to the start of a project
Recognizing the Limitations
Physicists recognized the limitations of Classical Physics just as we are beginning to recognize the limitations of traditional plan-driven project management. The table below shows a comparison of how these two areas have evolved:
For many years, physicists believed that a model of the universe could be completely predicted based on some relatively simple and well-defined laws of classical physics
For a long time, we assumed that traditional plan-driven project management was the only way to do project management and that approach would work in any project
Beginning in the early 1900’s, modern physics began to evolve and the limitations in Classical Physics began to be much more apparent
In recent years, it is apparent that we are in a much more dynamic and more complex universe with much higher levels of uncertainty
In this new environment, Classical Physics still provides a foundation however, it is no longer a universal view of how the world works
In today’s world, we are beginning to recognize that a traditional plan-driven approach to project management is not the only way to do project management and it doesn’t work well in a very uncertain environment
What Are the Limitations of Physics and Project Management?
Traditional, plan-driven project management (just like Classical Physics) will never be totally obsolete and will continue to be a foundation for many areas of project management:
“…classical physics retains considerable utility as an excellent approximation in most situations of practical interest. Neither relativity nor quantum theory is required to build bridges or design cellphone antennas.”
The never-ending conundrums of classical physics, Trent University
Limitations of Classical Physics
However, it is important to recognize the limitations that are inherent in a traditional, plan-driven project management approach. Experienced physicists have learned to recognize the limitations of classical physics. It only works reliably in a certain range of situations as shown in the figure below:
“Classical Physics is usually concerned with everyday conditions: speeds much lower than the speed of light, and sizes much greater than that of atoms. Modern physics is usually concerned with high velocities and small distances.”
Limitations of Traditional Plan-driven Project Management
Similarly, project managers also need to recognize that a traditional, plan-driven project management approach only works reliably in a limited set of situations. In the project management world, this can be expressed with the Stacey Complexity Model:
In this model, there are two primary dimensions – one is requirements complexity and the other is technology complexity.
Traditional, plan-driven project management still works in areas of low complexity such as some construction projects. However, even in some of those areas, project managers have recognized a need for a somewhat more adaptive approach
As you get further out on either complexity axis, there is typically a need for more of an adaptive Agile approach. In that area, Agile is better suited for dealing with uncertainty but this is not a binary and mutually-exclusive proposition. There is a need to blend both approaches in the right proportions to fit the situation
The way that the science of Physics has evolved has some strong similarities to the evolution of Agile Project Management.
Classical Physics Is Like Traditional, Plan-driven Project Management
The foundation of Physics today is still Classical Physics, just as traditional plan-driven project management is still a foundation of project management today:
Classical Physics is the theory underlying the natural processes we observe everyday.
It is the key to understanding the motion of pulleys, machines, projectiles and planets.
It helps us understand geology, chemistry, astronomy, weather, tides and other natural phenomena”
Evolution of New Ways of Thinking
Just as new theories about Physics have significantly extended the notion of what “Physics” is beyond the Classical Physics, Agile will have a similar impact on project management. The way this will probably evolve is very likely similar to the way that Physics has evolved:
Project Management Evolution
In today’s world, there are people who specialize in Classical Physics There are also people who specialize in the more esoteric areas of Modern Physics
There will be project managers who continue to specialize in a traditional plan-driven approach to project management There will also be project managers who specialize in Agile
However, neither one of those areas can ignore the existence of the other area
Just as in Physics, neither one of those areas can ignore the existence of the other area
A truly broad-based Physicist has a fairly solid knowledge of both Classical and Modern Physics
A truly broad-based Agile Project Manager has a solid knowledge of both traditional plan-driven project management and Agile
PMBOK version 6 and the new PMI Agile Practice Guide signal a new direction for the future of project management. For the first time, PMI has started to integrate Agile and traditional plan-driven project management. What does that mean for the future of project management?
What’s the Impact?
I’ve written a number of articles on the future of project management and I get a lot of questions from project managers. Many are confused about the impact of Agile on project management and ask questions like “What Agile certification should I get?”.
Unfortunately, it’s not as simple as just going out and getting another certification like PMI-ACP
The PMI-ACP certification is a step in the right direction and it’s not an easy certification to get. However, it’s just a test of general Lean and Agile knowledge and is not aligned with a particular role.
In fact, the role of an Agile Project Manager Is not well-defined. There is even some controversy about whether there is a role for an Project Manager In an Agile environment.
Confusion Over Project Management Direction
It’s totally understandable why there would be a lot of confusion among project managers about how Agile might impact their career direction.
There are some project managers who are in “denial”.
They want to assume that traditional, plan-driven project management is the only way to do project management.
They assume that it will go on unchanged forever unchanged and Agile isn’t really a valid form of project management at all
On the other hand, there are people in the Agile community who believe that there is no need at all for traditional plan-driven project management. They believe that Agile is a solution to almost any problem you might have
An Objective, Pragmatic Viewpoint
I’m not an Agile zealot – I try to take a very objective and pragmatic approach.
In one of my courses, I have a slide that says “Saying Agile is better than Waterfall” is like saying “A car is better than a boat”. They both have advantages and disadvantages depending on the environment.
You have to be able to fit the approach to the problem rather than force-fitting all problems to one of those extremes.
Project managers who only know how to do traditional, plan-driven project management and try to force-fit all projects to that approach will be at a severe disadvantage relative to other project managers who know how to blend Agile and traditional project management in the right proportions to fit the situation.
What’s Wrong with Traditional, Plan-driven Project Management?
There’s nothing inherently wrong with the traditional, plan-driven approach to project management; the problem is in how its applied.
The primary problem with the traditional, plan-driven approach is that it works for situations where the requirements are well-defined. In that environment, the primary concern is planning and managing a project to meet those well-defined requirements within a given budgeted cost and schedule
That approach just doesn’t work well in situations where the requirements are much more uncertain. In an uncertain environment, the primary concern is not just managing costs and schedules but taking an adaptive approach to maximize the business results and value that the project produces.
In today’s rapidly-changing business environment the need for taking that kind of approach is becoming increasingly common.
The Future of Project Management
There’s essentially two sides of this equation: value and cost. In the past,
The value side has been assumed to be well-defined by a fixed set of requirements
Project managers only needed to worry about the cost side
In this new environment, that is no longer true. Project managers now need to worry about both maximizing value as well as managing costs and schedules. That’s a fundamental shift in thinking for many project managers – it means:
Taking a broader focus on maximizing the business value that a project produces
Using whatever methodology (or combination of methodologies) that makes sense to achieve those goals
Fitting the project management approach to the nature of the business problem rather than force-fitting all projects to a standard, plan-driven approach.
That raises the bar significantly for many project managers.
What Certification Should I Get?
Some people seem to think that it is only a matter of getting another certification. I’ve participated in several discussions lately where project managers were asking questions like:
“What certification should I get in order to get into Agile (CSM/PSM, CSPO, or ACP)?”
The answer to the question of “what certification should I get” depends on what role you want to play. It requires some thought because there is no well-defined role for a project manager in Agile at the team level
There are several possible career directions for project managers with regard to Agile. You may not:
Have to completely throw away your project management skills. However, you may ave to rethink them considerably in a very different context
Use some traditional project management skills very fully at all depending on the role you choose
Potential Agile Project Management Roles
There are several potential migration paths for project managers who want to develop into an Agile Project Management role:
1. Become a Scrum Master
A Scrum Master:
Ensures that the team is fully functional and productive
Enables close cooperation across all roles and functions
Shields the team from external interferences
Ensures that the process is followed, including issuing invitations to daily scrums, sprint reviews, and sprint planning
Facilitates the daily scrums
There’s a few project management skills that might be useful (at least indirectly) for that role. However, it doesn’t utilize much of the planning and management skills that a project manager typically has. For that reason, becoming a ScrumMaster may or may not make sense as a career direction for many project managers.
2. Become a Product Owner
The Scrum Alliance defines the primary responsibilities of a Product Owner as follows:
The product owner decides what will be built and in which order
Defines the features of the product or desired outcomes of the project
Chooses release date and content
Ensures profitability (ROI)
Prioritizes features/outcomes according to market value
Adjusts features/outcomes and priority as needed
Accepts or rejects work results
Facilitates scrum planning ceremony
The Product Owner role actually includes a lot of project management functions. However, it is actually much more similar to a Product Manager than a Project Manager. The major differences are that:
The Product Owner is a business decision-maker and requires some business domain knowledge that a project manager may not have.
The Product Owner role doesn’t typically include many team leadership skills. In an Agile environment, team leadership is more a function of the ScrumMaster and the team itself.
3. Hybrid Agile Project Management Role
For a lot of good reasons, many companies will choose to implement a hybrid Agile approach that blends the right level of traditional plan-driven project management with Agile.
This is a very challenging role for a project manager to play.
It requires a deep understanding of both Agile and traditional plan-driven project management to know how to blend these two seemingly disparate approaches together in the right proportions to fit a given situation.
4. Project/Program Management of Large, Complex Enterprise-level Agile Projects
There is a legitimate role for project managers in managing large, complex enterprise-level projects; however, there are several things to consider about planning your career in that direction:
This role is limited to large, complex projects that typically require multiple Agile teams
It also may require blending together some level of traditional plan-driven and Agile principles and practices in the right proportions to fit the situation
This role doesn’t exist at all on most small, single-team Agile projects
This role requires some very significant skills that can be very difficult to attain. Many people may assume that the PMI-ACP certification qualifies you to perform this role. It is a step in the right direction, but a lot more experience and knowledge is needed to perform this role including:
Knowing how to blend traditional, plan-driven principles and practices in the right proportions to fit a given project,
Adapting an agile approach to fit a business environment, and
Scaling Agile to an enterprise level.
You have to be a “rock star” Agile Project Manager to perform this role.
Agile will have a big impact on the future of the project management profession:
In many industries and application areas, the project management role associated with small, single-team projects may be completely eliminated by Agile
There may be some project managers who are not significantly impacted by this such as project managers in the construction industry, but even in those industries some knowledge of Agile principles and practices may be essential
This creates difficult choices for a Project Manager to make. Agile may force project managers to make some significant choices about their career direction. It isn’t as simple as just going out and getting another certification (like PMI-ACP).
PMI recently published PMBOK version 6 as well as a new document called “The Agile Practice Guide”. The Agile Practice Guide is a totally new kind of document for PMI and raises some questions about “What is the purpose of the new PMI Agile Practice Guide?”
For a long time, PMI has treated Agile and traditional plan-driven project management as separate and independent domains of knowledge with little or no integration between the two. A major goal of this guide s is to start to develop a more integrated view of these two areas. I think this is a major step forward to begin to close this gap.
A lot of people may have thought that integrating these two areas might be as simple as adding more content about Agile to PMBOK version 6. They might think that PMBOK version 6 would become a universal guide to both of these areas. I don’t believe that to be a realistic way to accomplish that goal at all. See my article on Does PMBOK Version 6 Go Far Enough to Integrate Agile?
What is the Purpose of the New PMI Agile Practice Guide?
Agile and traditional plan-driven project management are two radically different approaches to project management that each require significant individual focus; however, at the same time, we need to build a much more unified view of these two areas. I think that is exactly the role that the Agile Practice Guide attempts to fill. Here’s how I see these various documents fitting together:
Here’s how I see this all fitting together:
PMBOK has become well-accepted for many years as the “bible” for a traditional plan-driven approach to project management. It is very detailed and somewhat prescriptive. To some extent, some (not all) of the practices in PMBOK provide a foundation for a general project management approach
Agile documentation has a very different and less prescriptive format. It is primarily based on some very simple and succinct principles and values in the Agile Manifesto
Those two formats are very incompatible with each other in my opinion. However, there is some commonality and we need to start to develop a more unified view of these two different worlds. That is the major purpose that the PMI Agile Practice Guide attempts to serve in my opinion.
What Does This Mean for the Future of Project Management?
This strongly reaffirms what I’ve been saying for a long time. The way of the future seems very clear:
There is not a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people have seemed to think. Those two areas are actually complementary to each other rather than competitive.
There is a continuous spectrum of different approaches ranging from:
Heavily plan-driven (predictive) at one extreme to
Heavily adaptive (Agile) at the other extreme
The right approach is to fit the methodology to the nature of the problem rather than just force-fitting a problem to some predefined methodology (whatever it might be).
The project manager of the future needs to be proficient in both of these approaches and also know how to blend the two approaches as necessary to fit a given situation. In the not-too-distant future, any project manager who only knows how to do traditional plan-driven project management and attempts to force-fit all projects to that approach will be at a serious disadvantage.
Review of the Agile Practice Guide?
Here’s a brief summary of my review of the Agile Practice Guide:
Overall, I think this document is well-written and really helps to close the gap between Agile and traditional plan-driven project management. However, that is a huge gap and there is still a lot more work to be done to create a truly integrated project management approach.
Agile and traditional plan-driven project management are two very different ways of thinking and it will be very difficult to fully integrate the two. This is a great step in the right direction but it’s not the final step to close that gap.
Agile PM Role
I don’t think this document has gone far enough to address the real “elephant in the room”. That is, “What exactly is the role of a Project Manager in an Agile environment?”.
There are many project managers who are in denial about that.
They think that their project management role will go on indefinitely unchanged.
There is a need to address this issue more directly so that project managers can plan their future career direction.
In the back section of the document, in a number of different places, it says that the role and expectations of a project manager don’t change in an Agile environment. I don’t agree with that at all. The role of a project manager at the team level (if there is one at all) will likely change radically to more of a coaching and facilitation role than a traditional PM role.
The authors of The Agile Practice Guide made a decision to limit the scope of this document to project and team-level work. They excluded discussion of the context of implementing Agile at an enterprise and organizational level. I think that is serious a mistake.
This is much too limiting because most Agile implementations cannot be successful without some level of organizational transformation. Furthermore, the role of a project manager is either non-existent or very limited at the team level. That will force many project managers to move up to more complex enterprise-level projects.
The section on “Agile Mindset” is really important and probably could be beefed up a lot. There is a big shift in mindset that is needed but it’s not just a matter of a choice between adopting an “Agile Mindset” or a “Traditional Project Management Mindset”.
I n many cases, you need to blend the two approaches and take a broader view of what “project management” is.
That broader view should fully embrace both of those approaches.
Many people would not view “Agile” as “Project Management” because it doesn’t fit the normal stereotype of what “project management” is
However Agile is just a different form or “project management”. That’s a big mindset change that PM’s need to make – we need to rethink what “project management” is in broader terms that include all forms of project management including Agile.
Relationship of Lean and Agile
I don’t agree with the graphic on page 11 showing that Lean totally encompasses Agile. It does not – there is a lot of overlap between the two; however, taken to an extreme, each would tend to pull you in somewhat different directions. Both are focused on customer value but:
Lean is more heavily focused on efficiency where
Agile is more heavily focused on flexibility and adaptivity.
Agile versus Predictive
The document talks about a spectrum of alternatives with predictive at one end point and Agile at the other end point. The idea of a spectrum of approaches is right on. However, I don’t think that the use of the word “Agile” for an end point is the right choice. Agile should not be an end point because there is not just one way to do Agile. There are a range of choices for Agile. This spectrum should reflect different levels of planning and I think the end-points are “adaptive” and “plan-driven” (or “predictive”).
The section on hybrid approaches needs to be improved. This is a critical area for PM’s to understand. As it is currently written, this is too high level and not specific enough to help a PM understand how to really implement a hybrid approach.
I would like to see the discussion of team roles expanded. One particular subject that is not covered is how many project functions that might normally be performed by a project manager have been assimilated into other roles in an Agile environment. Agile uses a distributed form of project management.
If you are a PMI member, you can download a copy of the Agile Practice Guide from the following link:
I am very pleased to see the PMI Agile Practice Guide being published. It is definitely a step in the right direction and is very consistent with the integrated approach to Agile Project Management that I’ve developed in the Agile Project Management Academy.
One of the biggest changes in PMBOK® version 6 is that it has incorporated more guidance about Agile. Does PMBOK version 6 go far enough to integrate Agile?
I think that the release of PMBOK version 6 and The Agile Practice Guide is a huge step forward. It is a noble attempt to create a more integrated approach for integrating Agile and traditional plan-driven project management;
However, the full integration of Agile and traditional project management requires some very major shifts in thinking. It even involves something as fundamental as adopting a much broader definition of what “project management” is.
I don’t think that simply adding some words about Agile to PMBOK is going to be sufficient to bring about the kind of shift in thinking that is needed.
What is “Project Management?
The crux of the problem is that for many years the essence of what “project management” is has been centered on some very well-established stereotypes of what “project management is. Those stereotypes are based on achieving predictability and repeatability as shown below:
Traditional Project Management Emphasis
That’s the primary way people have thought about what “project management” is since the 1950’s and 1960’s. A successful project manager is one who could plan and manage a project to meet budgeted cost and schedule goals. That obviously requires an emphasis on planning and control.
The way to achieve predictability and repeatability has been to have a detailed and well-though-out plan and then control any changes to that plan.
Many people loosely refer to this approach as “Waterfall” because, in many cases, it has been implemented by using a sequential phase-gate process. However, I don’t believe that description is entirely accurate:
I prefer to refer to it in more general terms as “traditional, plan-driven project management”
PMI has started using the term “predictive” to describe this kind of project management approach because the emphasis is on predictability
What’s Wrong With That Definition?
In the 1950’s and 1960’s that approach worked well and it was particularly in high demand for large, complex defense programs that were well-noted for cost and schedule overruns. At that time, the primary goal was to achieve predictability. In fact, that approach has been so prevalent that it has essentially defined what “project management” is. Since that time, many project managers don’t see any other way to do project management.
The problem with that approach is it only works well in environments that have a fairly low level of uncertainty where it is possible to develop a fairly detailed plan prior to the start of the project.
Factors Driving Change
In today’s world, there are several major factors driving change:
The environment we live in today has a much higher level of uncertainty associated with it. That makes it very difficult, if not impossible, to develop detailed plans prior to the start of a project
Solutions are more complex and are much more difficult to design and optimize
Competitive pressures demand high levels of creativity and innovation in spite of the level of uncertainty in the environment. Producing high-value business results is more important than predictability in many cases.
The New Environment
This new environment demands a very different kind of project model that looks more like this:
Think of a typical new product today like the next generation of the iPhone. Do you think that a traditional plan-driven approach with an emphasis on predictability, planning, and control would work well to develop that kind of product?
How Are PMBOK and Agile Different?
The differences in how these two approaches have been defined and implemented in actual practice are very significant:
Traditional Plan-driven Approach (PMBOK)
Process Control Model
Based on what is called a “Defined Process Control Model”
Based on what is called an “Empirical Process Control Model”
The emphasis of is on planning and control to achieve predictability over project costs and schedules
The emphasis is on using an adaptive approach to maximize business results in an uncertain environment
Project management functions are typically implemented by someone with clearly-defined responsibility for that role called a “Project Manager”
The functions that might normally be performed by a “Project Manager” at the team level have typically been distributed among other roles
Following a well-defined plan and process are typically important
Reliant on the judgement, intelligence, and skill of the people doing the project to fit an adaptive approach to the nature of the project
Is the Agile approach shown above in the right-hand column not “project management? A lot of people would not recognize it as “project management” because it doesn’t fit with many of the well-defined stereotypes of what “project management” is. I contend that it is just a different kind of “project management” that will cause us to broaden our thinking about what “project management” is.
“Project Management” should not be limited to a particular methodology. A project manager should be capable of delivering results using whatever methodology is most appropriate to achieve those results.
Is One Approach Better Than the Other?
There are a lot of Agile enthusiasts out there who will advocate that Agile is a better approach for almost any problem you might have.
My opinion is that saying “Agile is better than Waterfall” is like saying “A car is better than a boat” – they both have advantages and disadvantages depending on the environment that you’re in.
An Agile approach works best in situations that have a relatively high level of uncertainty. In those situations, creativity and innovation to find an appropriate solution are more important than predictability. For example, if you were to try to find a cure for cancer, it would be ridiculous to try to develop a detailed plan for that effort.
A traditional plan-driven approach works well in situations that have a relatively low level of uncertainty and where predictability, planning, and control is important. For example, if you were building a bridge across a river, it would be equally ridiculous to say: “We’ll build the first span of the bridge, see how that comes out , and then we’ll decide how to build the remaining spans.”
Are These Two Approaches Mutually-Exclusive?
A lot of people have the mistaken belief that there is a binary and mutually-exclusive choice between “Agile” and “Waterfall”:
There has been a lot of polarization between the Agile and project management communities for a long time. Many people in these two communities have seen these two approaches in conflict with each other
PMI has treated these two areas as separate and independent domains of knowledge for a long time with little or no integration between the two
It takes a higher level of skill and sophistication to see these two approaches in a fresh new perspective as complementary to each other rather than competitive. It is a challenge to learn how to blend them together in the right proportions to fit any given situation but it definitely can be done.
Does PMBOK version 6 go far enough to integrate Agile?
I have ordered a final copy of PMBOK Version 6 and haven’t actually seen it yet; however, I have seen early preview editions and I think I understand where it is trying to go. I have several concerns:
As I’ve mentioned, I think that there is a huge and fundamental shift in thinking that is needed to rethink what “project management” is. I’m not sure that simply adding some words about Agile to PMBOK is going to be enough to help people make that shift in thinking. It requires seeing “project management” in a fundamentally and radically different perspective.
The whole concept of PMBOK does not seem to be very consistent with an Agile approach:
Agile is based on some very simple and succinct principles and values. It relies very heavily on the training and skill of the people performing the process to interpret those principles and values in the context of a project
The latest version of PMBOK is over 700 pages long. It’s supposed to be a “guide” but it seems to try to provide a detailed checklist of things to consider for almost any conceivable project management situation.
Putting those two things together is like trying to mix oil and vinegar. They just don’t blend together very well and attempting to blend the two approaches at that level doesn’t seem to make much sense.
What is the Solution?
This is definitely a challenging problem. Agile and traditional plan-driven project management are like two different religions – they both have a common goal of delivering business results but the way each approach goes about doing it is very different.
There are two significant components of the solution to this problem:
Developing an Integrated View of Project Management
Somehow, we have to create a much more unified view of what “project management” is. That view should fully embrace Agile as well as traditional plan-driven project management. However, modifying PMBOK to totally integrate Agile would be very difficult. Its like setting out to create a unified view of religion. A better approach might be to cross-reference the two sources to identify areas of similarity and then create an over-arching guide to blend the two approaches together to create a unified view of religion.
I believe that is essentially what PMI has attempted to do with The Agile Practice Guide. I discussed that in a separate article. For a long time, PMI has treated Agile and traditional plan-driven project management as separate and independent domains of knowledge with little or no integration between the two. The new Agile Practice Guide attempts to bridge that gap and show a more integrated approach to those two areas. I think that is the only reasonable strategy that makes sense for now.
Develop a New Breed of Agile Project Managers
This “raises the bar” significantly for the whole project management profession. In my Agile Project Management books, I have often used the analogy of a project manager as a “cook” versus a project manager as a “chef” that was originally developed by Bob Wysocki:
A good “cook” may have the ability to create some very good meals, but those dishes may be limited to a repertoire of standard dishes, and his/her knowledge of how to prepare those meals may be primarily based on following some predefined recipes out of a cookbook
A “chef,” on the other hand, typically has a far greater ability to prepare a much broader range of more sophisticated dishes using much more exotic ingredients in some cases. His/her knowledge of how to prepare those meals is not limited to predefined recipes, and in many cases, a chef will create entirely new and innovative recipes for a given situation. The best chefs are not limited to a single cuisine and are capable of combining dishes from entirely different kinds of cuisine
I have mixed feelings about the subject of “Is PMP certification still relevant in today’s world?”. On the one hand,
I am a PMP myself,
I have had a PMP certification since 2004, and
I’m proud to be a PMP, but I recognize the limitations of a PMP certification.
On the other hand, I can clearly see the limitations in the PMP certification.
Is PMP Certification Still Relevant?
What Are the Limitations of PMP®?
PMP is heavily based on a traditional plan-driven project management approach (what many people loosely call “Waterfall”). The world is rapidly changing today. There’s nothing inherently wrong with a traditional plan-driven approach to project management under the right circumstances. However, we definitely shouldn’t try to force-fit all projects to that approach.
A traditional plan-driven approach to project management works well in situations where there is a relatively low level of uncertainty and predictability is important
It does not work well in situations
With a high level of uncertainty or
Where an emphasis on creativity and innovation may be more important than an emphasis on planning and control to achieve predictability
In today’s world,
A project manager needs to be capable of using a broader range of methodologies to fit the nature of the project rather than
Force-fitting all projects to a traditional plan-driven approach.
Situations are becoming increasingly common that require a more flexible and adaptive approach due to very uncertain rapidly-changing technology and a very dynamic and very competitive business marketplace.
What Is the Impact of Agile on PMP?
For those reasons,
Agile is having a profound effect on the project management profession that will cause us to rethink the way we do project management.
That doesn’t mean that traditional plan-driven project management and PMP are obsolete.
However, we’ve got to think of project management in broader terms and
Recognize that traditional plan-driven project management is not the only way to do project management
PMI-ACP® certification is a step in the right direction but it doesn’t go far enough in my opinion:
It doesn’t really address the big challenge that many project managers face today of “how do I blend Agile and traditional plan-driven project management in the right proportions to fit a given situation?”
Unfortunately, PMI still treats Agile and traditional plan-driven project management as fairly separate and independent domains of knowledge with little or no integration between the two
What About PMBOK® Version 6?
The final edition of PMBOK version 6 was released in September 2017. One of the big changes is that it contains more references to Agile.
The changes to PMBOK v6 barely scratch the surface of what needs to be done to develop a more integrated approach
I can’t imagine that future extensions to PMBOK will solve this problem either
The whole concept behind PMBOK is not very compatible with an Agile approach. These are two very different ways of thinking:
PMBOK is based on the idea that you can develop a checklist of things to consider in almost every conceivable project management situation that you can imagine.
Agile requires a very different mindset. An Agile approach needs to be much more adaptive and it would be impossible to develop a checklist defining what to do in every conceivable situation you might find yourself in in an Agile environment.
PMBOK and traditional plan-driven project management are based on a defined process control model
Agile is based on an empirical process control model which means that both the product and the process to produce the product are continuously adapted based on observation throughout the project
PMBOK is over 500 pages long with lots of details on what to do or consider in various situations
Agile is based on some very brief and succinct principles and values without a lot of detail and expects you to figure out what to do in a given situation.
PMBOK is also based on compartmentalizing a project into distinct and well-defined process groups
Agile requires a much more holistic and integrated approach to project management
What is the Long-term Solution?
This is not an easy problem to solve.
In the long-term, the solution to this problem is likely to involve some very significant rethinking of both PMBOK and PMI certifications.
What is needed is to create a much more integrated approach for blending Agile and traditional plan-driven project management principles and practices.
However, that is a very difficult problem to solve and is not likely to happen for a while.
What Is the Short-term Solution?
In the short-term, here are some possible strategies:
If You Have a PMP Today
If you already have a PMP certification today, that knowledge is a good foundation to begin to develop a broader focus on an Agile Project Management approach. However, it does require a lot of rethinking on how to do project management and also requires a very different mindset.
If You Don’t Already Have a PMP Certification Today
If you don’t already have a PMP certification today and are early in your career as a project manager, you have a much more difficult choice to make between two directions:
1. Getting a PMP
You could make a significant investment in time and money to get a PMP certification and then perhaps move on to learn an Agile approach sometime later. If you are working in an industry or application area where traditional plan-driven project management is still the dominant way of working, that might be a reasonable choice.
2. Skipping PMP
If you’re not working in an industry or application area where traditional plan-driven project management is the dominant way of working, getting a PMP may not make sense. Certainly, some foundation of traditional plan-driven project management is worthwhile but you may not need a full PMP for that. An alternative is to skip getting a PMP and just focus on developing an integrated approach to Agile Project Management.
In my opinion, skipping PMP and developing a more integrated Agile Project Management approach may be a reasonable for anyone
Who doesn’t already have a PMP and
Is interested in an Agile Project Management role.
However, it is a very difficult path to follow in the short term because:
There is currently no certification built around an integrated approach to Agile Project Management and
The knowledge base is not well-developed either
For that reason, you have to be somewhat of a “pioneer” in choosing this direction and
Since there is no certification, “you don’t know what you don’t know”.
Is PMP Still a Good Foundation?
Some elements of PMBOK and PMP are definitely useful as a foundation for any kind of project management.
However, the depth of study and knowledge required for PMP certification tends to “brainwash” people into thinking that PMP/PMBOK is the only way to do project management and that is not the case
Someone who only wants a foundation of knowledge in traditional plan-driven project management principles probably doesn’t need that depth of knowledge
The full PMP certification would still be appropriate for any project managers who plan to specialize in traditional plan-driven project management. However, that depth of knowledge in plan-driven project management should not be needed for someone who wants to develop an integrated Agile Project Management approach.
“Agile” means a lot of different things to different people. To some people:
It means just developing software faster, or
It means creating a more people-oriented project environment,
To others, it means making the project management process a lot more efficient by streamlining the whole process and eliminating unnecessary documentation
Those are only a few different connotations – there are many, many more. In addition, there are also many more stereotypes, myths, and misconceptions about what Agile is.
All of those things are potential outcomes of an Agile process but that’s not the fundamental essence of what an Agile process is all about in my opinion. The fundamental essence of an Agile process is adaptivity.
What’s Wrong With the Typical Project Management Approach?
Many project management processes, as we know them today, were designed around what is called a “traditional plan-driven project management” model (what many people loosely call “Waterfall“).
In this model, achieving predictability over the outcome of a project and the costs and schedule associated with achieving that outcome is very important
Therefore, it is also very important to have clearly-defined requirements as well as an adequate level of planning to be able to somewhat accurately predict the outcome, costs, and schedule of the project
That’s the predominant way that project management has worked since the 1950’s and 1960’s. A project was considered successful if it delivered what the requirements for the project within the defined budget and schedule.
That kind of predictability can be important. For large investments, it allows a company to:
Attempt to calculate with some level of certainty what the return on their investment is likely to be from a project, and
Make a go/no-go decision as to whether the project should be funded or not based on that information.
The primary problem with that approach is that it requires developing a fairly detailed plan for the project upfront. That is very difficult, if not impossible to do in projects with a very high level of uncertainty.
Why Is Adaptivity Important?
We live in a different world today from the world that existed in the 1950’s and 1960’s when formalized project management approaches were first defined.
Higher Levels of Uncertainty
There is a much higher level of uncertainty:
Technology is rapidly changing,
Solutions are much more complex, and
The business environment that we operate in is dynamic and constantly changing.
In that kind of environment,
Developing a detailed plan for a project with a lot of uncertainty upfront will typically require you to make a lot of assumptions.
And, many times those assumptions will be wrong and may require significant re-planning and possible rework later.
Rather than force-fitting a project that has a high level of uncertainty to a traditional plan-driven approach;
it’s much better to fit the methodology to the nature of the project and that’s where a more adaptive approach really makes sense.
That doesn’t mean that you don’t do any upfront planning; it means that you use a level of planning that is appropriate to the level of uncertainty in the project:
Traditional Plan-driven Approach
Adaptive (Agile) Approach
Atempt to develop a detailed set of requirements and a detailed project plan for the project upfront
Limit the amount of upfront planning based on the level of uncertainty in the project and use a “rolling-wave” planning approach to further define the requirements and plan for the project as the project is in place
Need for Creativity and Innovation
Another important factor in today’s environment is that there is a greater need for creativity and innovation to develop truly leading-edge products. An over-emphasis on planning and control can stifle creativity and innovation.
What’s an Example of a Project Requiring an Adaptive Approach?
A Simple Example
I use an example in my Agile Project Management training that is somewhat extreme but it gets the point across. The example I use is:
Suppose you were given the task to find a cure for cancer and you were asked to outline:
What the solution will be,
How long it will take to develop it, and
What the total cost of the research will be to develop the solution
In that situation, it would be ridiculous to attempt to develop a detailed project plan with accurate cost and schedule estimates – there is just way too many uncertainties to resolve
So what would you do? Give up and do nothing? That doesn’t make sense either
It’s important to recognize that we do know some things about finding a cure for cancer based on years of research that have gone into that area.
However, there are still way too many unknowns to develop a detailed project plan for a solution
What you would do is take advantage of what is known as much as possible and then take an iterative, trial-and-error approach to find a solution
Thomas Edison and the Light Bulb
That’s the way Edison invented the light bulb…here’s a quote from Edison on that subject:
“I speak without exaggeration when I say that I have constructed three thousand different theories in connection with the electric light, each one of them reasonable and apparently to be true. Yet only in two cases did my experiments prove the truth of my theory. My chief difficulty, as perhaps you know, was in constructing the carbon filament, the incandescence of which is the source of the light.”
(1890 Interview in Harper’s Magazine)
In a 1910 autobiography of Edison, Edison’s friend and associate Walter S. Mallory is quoted as asking
“Isn’t it a shame that with the tremendous amount of work you have done you haven’t been able to get any results?” The book goes on to say that “
Edison turned on him like a flash, and with a smile replied:
“Results! Why, man, I have gotten lots of results! I know several thousand things that won’t work!”
What Makes This Kind of Project Different?
There are two things that make this kind of project fundamentally different:
The level of uncertainty is very high and makes it impractical or impossible to develop a detailed plan for the project upfront
Creativity and innovation required for finding a good solution are far more important than predictability
How Does an Agile Approach Solve This Problem?
An Agile process is built on an “Empirical” Process Control model. The word “empirical” means “based on observation”. In the context of an Agile development process, “Empirical” means that during the course of a project, both the product as well as the process to produce the product are continuously refined as the project is in progress. The goal is to produce the right product and to optimize the value of the product being produced.
Why Is This Important to Project Managers?
You might ask “Why is this important to project managers?”
Isn’t Agile something that only applies to software development? (That’s a common misconception)
The truth is that all projects have some level of uncertainty associated with them
If you try to force-fit all projects to a traditional plan-driven project management approach, its just not going to work in many cases. Imagine, for example,
Trying to develop the next generation of the iPhone or any other new and innovative product
In that kind of project, creativity and innovation is just as important, if not more important, than predictability.
In this new environment, a project manager who only knows how to do a traditional plan-driven approach to project management will be at a serious disadvantage. What makes this even more difficult is that:
That this is not a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people seem to think.
It’s a matter of figuring out how to blend a traditional plan-driven approach with an adaptive (Agile) approach in the right proportions to fit a given situation.
Have you given any thought to “How to make a hybrid agile process work”? Some people claim that hybrid Agile projects don’t work at all. For example, I recently saw an article on LinkedIn entitled “Why Hybrid Agile-Waterfall Projects Fail” that caught my eye. I’m not surprised at this article:
It takes some skill to make a hybrid Agile process work successfully and
Anyone who would literally try to combine an Agile methodology with a Waterfall methodology to create a hybrid methodology is asking for failure
That’s not the way to do it
Blending Two Recipes
This should not be a matter of literally combining an Agile methodology and a Waterfall methodology.
That would be like trying to take a recipe for Italian food with a recipe for Chinese food and literally trying to mix the steps and ingredients from the two recipes together to create “Italian-Chinese” food
A good chef would never even attempt to do that. However, he/she might create an entirely new recipe that blends together some of the aspects of Italian food with some aspects of Chinese food
Creating a Hybrid Approach
It takes a lot of skill to create an effective hybrid approach and doing it effectively is like the difference between a “chef” and a “cook”.
When I talk about creating a hybrid approach, I try to avoid the words “Agile” and “Waterfall” if possible. Those words give people the impression that you are literally combining two different methodologies together like combining two food recipes:
I prefer to think of a hybrid approach as the appropriate blend of an adaptive approach and a plan-driven approach
The words “adaptive” and “plan-driven” convey an entirely different meaning than “Agile” and “Waterfall”
Creating a hybrid Agile approach:
Doesn’t mean that you’re trying to literally combine two very different methodologies by mixing them together
Means that you’re creating a blended approach with the appropriate amount of emphasis on being “adaptive” versus being “plan-driven”
An Example of a Real Hybrid Approach
Some years ago, I was responsible for managing a very large development program for a US Federal Government agency:
It had a fixed-price contract associated with it and a fairly aggressive delivery schedule
However, the customer wanted some level of flexibility in the details associated with defining requirements
Does not attempt to literally combine an Agile methodology and a Waterfall methodology
Instead, it wraps a plan-driven layer at the “macro” level around a standard Agile/Scrum process at the micro level
Here’s a diagram that shows what it looks like:
You might say:
“That’s not a hybrid Agile-Waterfall model”
“It’s only a standard Agile development process with a plan-driven layer wrapped around it”
And, that would be correct. There has been no attempt to actually mix-and match a phase-gate Waterfall methodology with an Agile methodology.
How To Make a Hybrid Agile Process Work
This is a very flexible model – that is why I call it a “framework” rather than a “methodology”.
1. Project Planning
You would start out a project by at least developing high-level requirements with estimates of the costs and schedule for completing the project. Those estimates can be as detailed as you want them to be. That’s what makes this model so flexible.
You can have a relatively “thin” layer of planning with very high-level requirements and less-detailed plans or
A “thicker” layer of planning with more detailed requirements and planning
The most important thing is that there has to be a common understanding between the development team and the customer about how detailed the requirements and plan are:
This model will not work without a spirit of trust and partnership between the customer and the project team. The customer and the project team must agree to working collaboratively as the project is in progress to:
Further elaborate requirements,
Continuously refine the plan for completing the project, and
Resolve any trade-offs that may be necessary to stay within the budgeted cost and schedule.
2. Managing Changes
As the project is in progress, any changes to the project requirements made at the “micro” level need to be fed back into the planning at the “macro” level.
By agreement with the customer, there should be enough slack built into the plan so that minor changes can be absorbed easily without a change to the overall plan. Only more significant changes might require trade-offs and adjustments. When trade-offs are needed to stay within the budgeted cost and schedule, the customer may need to either:
Adjust the planned cost and schedule as necessary, or
Reduce the scope of some of the requirements if necessary to fit within the planned cost and schedule.
This is a fairly simple model but it works and it can be easily adapted to a wide variety of projects. However, note that this is not a simple step-by-step cookbook approach. It takes some skill to make this model work successfully.
Matching the Level of Uncertainty in the Project
A good strategy is to match the design of the hybrid approach to the level of uncertainty in the project:
A project with a lower level of uncertainty might lend itself to a more plan-driven approach particularly if predictability is important to the customer of the project. That would mean putting a higher level of emphasis on developing a “thicker” layer of planning at the macro-level
A project with a higher level of uncertainty would probably require a more adaptive approach to further elaborate the plan as the project is in progress. That would mean that the level of planning at the macro-level would probably be much “thinner”.
Dr Royce described a model that consists of a sequence of phases. In this model, the outputs of one phase flow into the next phase like a “Waterfall”:
The process was called ‘Waterfall’ because the results of one phase flow into the next phase like a waterfall.
What Was Life Like Prior to ‘Waterfall’?
it is useful to understand what life was like prior to Waterfall and what problems it tried to solve. What existed at that time was a lot of poorly-organized development efforts with little or no structure, discipline, and planning. Some of the major problems with the that approach were:
Coordinating Work of Large Development Teams
As projects grew in terms of scope and complexity with potentially much larger numbers of developers, it became apparent that a more planned and structured approach was essential to coordinate the work of large development teams
Cost and Schedule Overruns
The other major problem was that there was very limited predictability over the costs and schedules of software projects:
There were many and frequent very significant cost and schedule overruns, and
Business sponsors demanded some level of predictability
How Did the Waterfall Process Improve These Problems?
When the Waterfall approach was originally defined, it was a big improvement to go from practically no methodology at all to a very well-defined process. The new Waterfall process provided:
A “road map” to:
Coordinate the work of multiple developers as well as
Integrate the work with any other essential resources outside of the immediate development teams, and
A mechanism to gain control over the scope of software projects in order to get more predictability of project costs and schedules
Many younger people today don’t appreciate that history and just criticize Waterfall as being bad without understanding the problems it was intended to solve.
The “Pendulum Effect”
As with many things, there was a “pendulum” effect when the Waterfall approach was initially implemented. There was somewhat of an over-correction in many cases in going from no methodology to a very well-defined methodology. The pendulum swung in many projects from almost no control and discipline to very rigid control and discipline.
It Became Very Rigid and Inflexible
The initial implementation of the Waterfall process had a number of problems that even Dr. Royce recognized in 1970 when he first defined the process. Some of the most serious problems were:
The common practice when the Waterfall process was originally defined in 1970 was a very document-intensive and over-controlled process
You couldn’t exit a phase until all the documentation required to show that the work required for that phase had been completed, reviewed, and approved
The ultimate user of the software didn’t normally even see the software until all of the development and testing was complete and by that point in time; it was very difficult, if not impossible, to go back and make any significant changes
The emphasis on control of scope made the process very inflexible to any changes that might be needed to meet user needs and business goals in an uncertain environment
What Was the Impact?
As a result,
There have been many situations where the project may have met cost and schedule goals but failed to provide a sufficient level of business value
Another major problem was that a heavy emphasis on documentation and other overhead required for reviews and approvals made the whole process bureaucratic and not very cost efficient
It’s important to note that many of the problems associated with “Waterfall” are a result of how it is implemented and not necessarily inherent problems in the methodology itself.
Why Is the Agile versus Waterfall Comparison So Misleading?
A big reason why the typical Agile versus Waterfall comparison is so misleading is that the words “Agile” and “Waterfall” are so loosely-used:
How is the Word “Waterfall” Loosely-used?
Before Agile came into widespread use, many variations on the original Waterfall model were developed to create a more adaptive approach to solve some of these problems.
More iterative processes such as the Rational Unified Process (RUP) and a number of variants became widely-used in the 1990’s and early 2000’s
There has been a proliferation of a broad range of many different development models such as the Spiral model
Some of those have only a very limited resemblance to the original ‘Waterfall’ model as it was defined in 1970.
In spite of this evolution, people still loosely characterize all of those methodologies as ‘Waterfall’ as if it was one specific, unique and well-defined methodology called ‘Waterfall’ and that is not really the case
The common denominator of all the methodologies that people loosely call ‘Waterfall’ is that
They emphasize some level of upfront planning and control.
The goal is to try to achieve predictability over project scope, costs, and schedules.
For that reason, I think the word “plan-driven” is a more accurate and objective description of what people really mean when they say ‘Waterfall’.
How Is the Word “Agile” Loosely-Used?
The word ‘Agile’ is also loosely used. We all know that ‘Agile’ is not a specific methodology although many people equate ‘Agile’ with Scrum:
Scrum is not really a specific methodology, it is really a framework that is intended to be adaptable to a broad range of situations
Agile is not really equivalent to Scrum. There are other Agile methodologies such as Kanban
The common denominator of methodologies that people call ‘Agile’ is that:
They are flexible and adaptive and
Emphasize creativity and innovation in an uncertain environment
Emphasizing planning and control to achieve predictability with lower levels of certainty.
For that reason, I prefer to use the word “adaptive” instead of the word ‘Agile’ when comparing it to ‘Waterfall’ (plan-driven).
When people in the Agile community compare ‘Agile’ and ‘Waterfall’, it’s usually in the context of
Agile is good and Waterfall is bad and that’s really not accurate and objective.
Saying “Agile is better than Waterfall” is like saying “A car is better than a boat” – both have advantages and disadvantages depending on the environment that you are in.
The words “Agile” and “Waterfall” are very loosely used in practice and that causes a lot of confusion. They are used as if both “Agile” and “Waterfall” are unique, individual methodologies and that is not the case.
The word “Waterfall”, in particular, is very loosely-used.
When people use the word “Waterfall”, they’re not necessarily talking about the real “Waterfall model that was defined by Dr Winston Royce in the 1970’s
They’re really talking about any plan-driven methodology that is not completely agile.
A better way to think about “Agile” versus “Waterfall” is in terms of “adaptive” versus “plan-driven”. That’s a much more accurate and objective way of making that comparison.