Have you ever thought about “When an agile project fails, who gets blamed”? I thought it was a very interesting question.
How Does an Agile Project Fail?
It’s actually difficult for an Agile project to fail.
An Agile project typically does not have rigid cost and schedule goals that must be met and
An Agile/Scrum process has the capability to detect and correct potential failures early
Given that, an Agile/Scrum project should be self-correcting if it is done properly. For example, at the end of each sprint:
There is a Sprint Review to detect problems with the product
There is a Sprint Retrospective to detect and correct process problems
An Agile project should provide early warning of a potential failure with plenty of opportunity to correct any problems before the end of the project. At the end of each sprint, both the product and the process to produce the product are reviewed and corrected if necessary. If an Agile process fails, the process must have broken down somewhere; and rather than looking for an individual to blame, a more appropriate response would be to figure out what went wrong in the process to prevent it from happening again.
Fail Early, Fail Often
One of my favorite Agile mantras is “fail early; fail often”. People should not be afraid of failure and should see failure as an opportunity for learning. That is very important in an environment that is designed to support creativity and innovation. If senior management is looking for someone to blame, that’s not very consistent with an Agile culture.
How to Prevent Failure in an Agile Project
It’s relatively easy to prevent failure in an Agile project – its mostly a matter of:
Implementing the process effectively including Sprint Reviews and Sprint Retrospectives with an emphasis on continuously improving both the product as well as the process for producing the product as the project is in progress
Designing and implementing an enterprise-level transformation to align the Agile development approach with the company’s business and to create a culture that is supportive of an Agile approach
The important point is that their should be:
Everyone in the organization should have a spirit of shared ownership and partnership and be committed to the success of the project instead of an “arms-length” contractual relationship between the business and the project team
The business sponsors and users actively participate in the project in a spirit of partnership as the project is in progress to provide feedback and inputs as the project is in progress
A number of my students have requested some case studies that show applying Agile to non-software projects. As an example, I recently completed a home remodeling project which may seem simple and trivial; but believe me, it was not.
Applying Agile to Non-Software Projects
It is possible to apply Agile to almost any project but that doesn’t necessarily mean using Scrum. And, it certainly doesn’t mean just going through the rituals of doing Scrum mechanically. Applying Agile principles and figuring out how to apply them to non-software projects can be very challenging.
I have been a project manager for a long time. I’ve managed large, complex multi-million dollar projects; but nothing compared to a recent project to do a major remodeling of the kitchen in our house. The project involved:
Knocking down a wall that separated the kitchen from the rest of the house to create a more open environment
Ripping up the concrete floor to re-route electrical and plumbing connections
Replacing all of the existing kitchen cabinets and appliances
Installation of new lighting fixtures
Moving the entrance-way to the master bedroom to be more consistent with the new floor plan
Removing a pantry and replacing it with a new pantry cabinet which required knocking down a wall and moving an intercom system
Repainting the entire area and many other cosmetic enhancements
Agile Home Remodeling – Why was this project so difficult?
My wife was the major stakeholder in the project, she is a perfectionist, and she has a habit of changing her mind frequently about what she wants. (Her response to that is “She doesn’t change her mind, she just decides as she goes along”)
Multiple outside contractors did all of the work in this project
A major challenge was to try to manage the costs and schedule of this project within reasonable levels
Agile Contractor Selection
The first task was to select a contractor (or contractors) to do the work. I had several choices:
Contractor “A”was the most widely-known contractor in this area. They advertise widely on television and have a good reputation for delivering a high-quality result. They would also take full responsibility for the overall solution. However, their approach is fairly rigid and controlled. Once you sign a contract with them, it is very difficult to make any changes.
Contractor “B” was much less widely-known but offered much more flexibility and willingness to work with on a design that was customized to meet our needs. They would also take overall responsibility for managing the overall solution.
Contractor “C” offered the most flexibility to meet our needs but was actually two different contractors. It was not really possible for either of them to take overall responsibility for the overall solution
One contractor did the demolition and prep work including electrical and plumbing to prepare the new kitchen
Another contractor provided the kitchen cabinets and counter-tops. They installed them after the initial demolition and prep work had been completed
Following the installation of the cabinets and counter-tops, the original contractor returned to do the finish work. That work included final installation of new lighting fixtures and repainting of the entire area
Choosing a Contractor
Selecting a contractor was difficult:
Contractor “A” was probably the lowest risk choice from a traditional project management perspective. It would require less management on my part but offered little flexibility to adapt the solution to meet our needs
Contractor “C” was the highest risk and involved coordinating the work of two different contractors. However, they offered the most flexibility to meet our needs
Contractor “B” was a compromise between those two extremes. They had the advantage that they were a single contractor who would take overall responsibility for the solution. However, their costs were considerably higher than Contractor “C”
Final Contractor Selection
We chose contractor “C” because flexibility and adaptivity to meet our needs was so important; even though contractor “C” had the highest risk and might be the most difficult to manage. However, these two contractors had a history of working together successfully on other similar projects. I also had a good feeling that I could trust and partner with these individuals to manage the overall solution. That was a key difference:
With contractor “A”, we would have relied on a very clear and well-defined contract to deliver the solution. However, we would have little or no flexibility to make changes (That’s what many people might call “Waterfall”)
Contractor “C” had a statement of work but it was understood to be flexible and subject to change. The relationship relied on a spirit of trust, partnership, and collaboration (This relationship was much more similar to Agile)
How Did the Project Work Out?
This was a difficult effort to manage.
The scope of the project changed numerous times
My wife decided that we couldn’t remodel the kitchen without replacing all the living room furniture and carpets. And, of course, other changes to the rest of the house became necessary as well which included:
Repainting the master bedroom,
Replacing pictures and reupholstering other furniture. and
Enhancements to other areas of the house
Nailing down the design requirements was very difficult
As I mentioned, my wife changes her mind frequently, and insists on perfection in the end-result. We looked at many different kinds of granite counter-tops and many different floor tiles before making a final selection. There were also many times when a “final selection” changed before it really became a “final selection”
This Was a Project Management Nightmare
This was not a large project but it was one of the most difficult ones that I have ever had to manage. For a traditional plan-driven project manager, this would have been a nightmare attempting to control all of these changes. It is also very challenging to be caught in the middle between a very demanding stakeholder and contractors who have to deliver the work within a given cost.
However, this is a perfect example on a small scale of what an Agile Project Manager has to do. You have to learn how to balance flexibility and adaptivity to maximize the business value of the solution with some level of planning and control,
What Were the Results?
The project turned out to be enormously successful
The whole project was completed in a little over three weeks from the time the work started
It went over the budget that that we expected to spend but the costs were still at a reasonable level
Most importantly, my wife was delighted with the way it came out and she is the most important stakeholder I needed to satisfy.
Here’s a picture of what the finished kitchen looked like:
Here’s a couple of pictures taken during the work-in-progress leading up to finishing the kitchen:
How Does This Apply to a Business Situation?
I know this is an unusual situation but I like to use unusual situations. I think it encourages “out-of-the-box” thinking rather than viewing standard, stereotypical Agile case studies. The following is a summary of how I think these lessons-learned can be applied to a business situation:
Most businesses could not survive without some kind of contractual relationships with outside contractors. In addition, many businesses have significant supply chains that are critical to the success of their business.
Typically, a firm, fixed-price contract and a competitive bidding process among multiple bidders is used to get the lowest possible price.
That is a relatively low-risk approach from a cost-management perspective but doesn’t necessarily result in the best overall solution.
When there is a lot of uncertainty in the requirements, a different approach is needed to maximize the business value of the solution. It requires a collaborative partnership with a contractor to work together to maximize the value of the solution is essential
Developing that kind of relationship with contractors requires trust. For that reason, it will not be possible to develop that kind of relationship with just any contractor. That’s why it is important for a business to have strong relationships with a selected number of contractors who can be regarded as close partners.
I took a risk by going with contractors that I thought were the highest risk from a project management perspective. However, that risk paid off in terms of the overall quality of the solution. A similar thing is true in a business environment. Many times you have to take a risk to maximize the value of the solution.
The project was completed in a very short time once the work was started. That was largely due to the fact that I empowered the contractors to get the job done the best way they knew how and I didn’t attempt to micro-manage what they were doing. Conventional project management might attempt to more directly manage the work being done.
Here are some of the important conclusions and lessons learned from this project about applying Agile to non-software projects:
Agile principles and values can be applied to some extent to almost any project. However, it requires some skill to interpret these principles and perhaps combine them with traditional project management practices.
The overall value that the project delivers is what is most important. The key stakeholder determines “value”. Cost and schedule goals have some value but are only one component of value and not necessarily the most important,
A spirit of trust and partnership is important even in a contractual situation. Over-dependence on a traditional contractual relationship can severely reduce flexibility and impact the value that the solution provides.
Risk management is important but attempting to minimize and over-control risk can also impact the value of the solution. Taking risks may be necessary to maximize the value of the solution.
For another example of applying Agile to non-software projects, check out this article:
I have just released a new online training course called “Agile Project Management for Executives”.
The Agile Bandwagon
In many areas, “Agile” is becoming a hot new buzz word and everyone wants to jump on the “Agile bandwagon”. They may not fully understanding why they’re getting into it and exactly what they expect to get out of it. In addition, many companies also make the mistake of assuming that whatever is good for the development process is good for the business as a whole and that is not necessarily the case.
Agile Project Management for Executives Course Summary
Agile has huge potential benefits for a business; however, it is easy to get carried away with some of the hype that exists about Agile. To avoid that, it is important to develop an objective understanding of its benefits and limitations to know how and when to apply it successfully. The right approach is to not necessarily to just implement Agile for the sake of becoming Agile, but figure out how it’s going to help your business and what problems it will solve. The typical questions and challenges this poses for business managers and executives are:
How do I reconcile an Agile development approach with my existing business management and project management processes?
Do I need to unravel all of my existing management processes in order to adopt an Agile development approach?
This course will help you answer those questions. It also includes assessment tools and planning tools that are designed to help you develop a very effective Agile Project Management approach that is very well-aligned with your business.
Intended Audience – Agile Training for Managers
There are three potential audiences for this course:
1. Senior-level Executives
The first audience is senior-level executives who want to make their business more agile. The course will help develop a well-integrated approach to fit an Agile development process to their business
2. Business Sponsors
The next audience is Business Sponsors of an Agile initiative who want to learn more about Agile Project Management. The course will help them prepare to provide more effective leadership for the initiatives that they are responsible for
3. Product Owners
The final audience is for Agile Product Owners. Many of the people who are selected to perform that role are not well-prepared for what it requires and the role is not well-understood. The course will help them to better understand how to effectively perform the Agile Product Owner role
Why Is This Course Unique and Important?
For many years, many people have treated Agile as a development process. However, in recent years it has become apparent that the implementation of Agile as a well-integrated, enterprise-level business strategy is not well-understood.
1. Business Perspective
A lot of the Agile training that exists today is very focused on implementing Agile as a development process and on the “mechanics” of how to do Scrum. There is a relatively weak focus on Agile from a business perspective. For example, my own Certified Scrum Product Owner certification was heavily focused on the “mechanics” of how to do Scrum. It didn’t really directly address the role of the Product Owner as a business decision-maker at all.
2. Objective, Pragmatic Approach
This course is not a sales-pitch for Agile. It recognizes that there is not a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people seem to think. Instead, it objectively presents Agile and traditional plan-driven project management approaches as complementary to each other rather than competitive.
3. In-depth Training
This course is not a superficial seminar on how to implement Agile. It is a very substantive, university-level course that is over four hours long. It provides a very in-depth understanding of Agile from a business perspective
4. Complementary to Agile Project Management Approach
This course is also designed to complement all of my Agile Project Management courses. Implementation of Agile at an enterprise-level requires a collaborative partnership between a business executive and a senior-level Agile Project Manager. That relationship should be based on a mutual understanding of how an Agile approach might apply to their business.
“Distributed Project Management” is a new approach to project management. Here’s a brief overview of what it is all about.
What is Distributed Project Management?
“Distributed Project Management” is very important to help people see the relationship of “project management” and Agile in a very fresh new perspective. It has the potential to redefine many of the heavily-ingrained notions that we have about what “project management” is.
There are a number of people in the Agile community that believe that “project management” is not consistent with Agile. That opinion is based on:
A very narrow and stereotypical view of what project management is
An assumption that all project management functions are done by a single person called a “Project Manager”.
I think it is time to take a broader and more modern view of what “project management” is.
How Is Project Management Implemented in an Agile Team?
There is actually a lot of “project management” going on in an Agile environment, but many people won’t recognize it as “project management” because:
It’s a different kind of project management, and
The project management functions have been distributed among multiple people on the team
Let’s explore each of those areas individually.
1. It’s a Different Kind of “Project Management”
We need to broaden our thinking about what project management” is
The traditional view is based heavily on planning and control to achieve predictability over project costs and schedules.
A more modern and broader view of “project management” is based on delivering business value.
That doesn’t mean that meeting cost and schedule goals is unimportant.
Achieving cost and schedule goals is only one component of business value and not necessarily the most important component.
Creativity and innovation to maximize the value of the solution can be at least equally important
2. The Project Management Functions Have Been Distributed Among the Team
The functions that would normally be performed by someone called a “Project Manager” at the team level have been distributed among other members of the team. As a result, you typically may not find anyone at the team level in an Agile project called a “Project Manager”.
In an Agile team, everyone on an Agile team has some kind of responsibility that might normally be performed by someone called a “Project Manager”:
Project Management Function
Product Owner Role
The Product Owner comes closest to the overall responsibilities of a project manager. However, the Product Owner role actually goes beyond a project management role. The Product Owner is expected to:
Take overall responsibility for the success or failure of the project
Make any decisions or trade-offs that might be needed to meet the project goals
In an Agile environment, developers actually have responsibilities that might normally be done by a “Project Manager”. Developers are expected to:
Take responsibility for planning and completing the tasks that they are responsible for
Following through to meet commitments that they have made
Reporting progress and coordinating their work with other members of the team as necessary to produce an overall result
Scrum Master Role
The Scrum Master also has responsibilities that might normally be performed by a Project Manager. The Scrum Master is expected to:
Facilitate the work of the team and team meetings
Coaching the team in Agile practices and
Removing any obstacles that might be impeding the team’s progress
A project manager in a traditional plan-driven environment would normally perform those functions. A “Distributed Project Management” approach distributes these functions among multiple people.
Why Does Distributed Project Management Make Sense?
In an Agile environment:
Solutions can be much more complex and the level of uncertainty can be much higher. That makes it very difficult, if not impossible, to completely define a solution prior to the start of a project
That environment requires a much more flexible and adaptive approach
In that environment,
It is essential to further elaborate the requirements and the design of the solution as the project is in progress.
That calls for a very different approach to project management.
Distributing the project management functions among the different Agile team roles provides a much more dynamic approach:
Instead of centralized control where all decisions are made by a project manager,
Decision-making is more decentralized among the various roles on an Agile team
The team, as a whole, is self-organizing and empowered
That approach is very well-suited for an environment with a high level of uncertainty
Distributed Project Management is a new way of thinking about how to do project management:
Instead of the normal project management functions being performed by a single person called a “Project Manager”
Those functions may be distributed among other roles
This approach may be threatening to many traditional project managers because, in many cases,
It could eliminate the role of a project manager at the team level in an Agile project, and
It also could require a significant adaptation for many project managers who are used to being in control of a project
For more on that check out this article on The Future of Project Management:
For the project management profession to continue to thrive, we need to recognize this fundamental shift in thinking and develop a broader vision of what “project management” is.
Many people have asked about “Applying Agile to non-software projects”. I’ve done a lot of that myself in using an Agile book publishing approach for publishing five books. I’ve also used similar techniques in designing and developing numerous online training courses. I thought I would summarize some of the techniques I’ve learned from doing Agile Book Publishing over the years.
Agile Common-Sense Principles
Here are some common-sense principles I’ve learned from using an Agile book publishing approach for publishing five books:
1. Just Get Started
One of the most important principles I’ve learned is “just get started”. When you’re faced with writing a book or developing a major online training course, it can be a daunting experience. Just getting started is sometimes the hardest part:
We’re not sure how the final result is going to come out
We’re not certain how the final result will be structured – what should come first, etc.
We don’t want to produce something that is going to be a failure
You have to stop worrying about all of that and just get started. I think of an Agile book publishing effort like developing a fine art sculpture. You start with a lump of clay and you just keep molding and shaping it until it becomes a work-of-art.
If you never get started, it will remain just a lump of clay.
It takes some courage and confidence in yourself to do this. You’ve got to have courage and confidence that if you just get started, that somehow the final result is going to come out OK if you keep working at it
It also takes patience and commitment because you may have to go through a large number of iterations to get something useful out of it. You may even have to throw something completely away and start over again
2. Use an Incremental and Iterative Approach
Many people don’t understand the difference between the words “incremental” and “iterative”. An Agile book publishing approach involves both:
“Incremental” means that you break up a solution into pieces and develop one piece (increment) at a time
“Iterative” means that if you’re not sure what a given piece should be, you develop something and then continue to refine that piece until you meet the customer’s expectations
Both of those are important:
Using an incremental approach is very important. In any large effort like writing a book or developing an online course, its best to break it up into “bite-sized pieces. If you try to take on too much at once, you’ll never finish it. The effort to write a book can easily take well over a year and it’s easy to get discouraged in that period of time that you will never finish if you don’t see progress in the work.
Taking an iterative approach is also important. A close corollary to “Just Get Started” is “Don’t Expect Perfection”. A major reason for not getting started sometimes is that we’re afraid to produce something that is less than perfect.
We have to accept that whatever we produce on the first iteration is certainly not going to be perfect. The final result may not be perfect either.
Get something done quickly and then continue to refine it as needed to meet customer expectations.
There’s also a saying in Agile that is used a lot called “just barely good enough”.
We shouldn’t try to “gold-plate” or over-design something. It should satisfy the need to provide value to the customer and nothing more. Keep it simple!
3. Know Your Customers and Listen to Them
When I first started writing books, I had a lot of people who helped me. I had a network of people on the Internet who provided me with lots of great feedback and inputs. I would write a chapter or two of the book and put it out to my network for feedback and comments. Part of doing that successfully is recognizing that you “don’t know what you don’t know”. You have to be humble, listen to other people, and respect their needs and interests. If you think that you know it all, you probably won’t be very successful.
In the online training I develop, I get lots of feedback and inputs from students and I listen to it and take action. As a result of that feedback, I have continuously improved all of my courses.
4. Refactor Your Work As You Go Along
I’ve done a fair amount of software development in my career and I’ve learned a lot from it.
I’ve learned the value of having clean and well-organized software because I have had to support a lot of the software I’ve developed
Organization and flow of the material is particularly important in books and online training as well
As a result, it is essential to take time to go back and clean-up your work as necessary as you go along
When I first write something, I get it done quickly but I may have to rewrite it and reorganize it 5-6 times before I’m satisfied with it
5. Work at a Sustainable Pace and Do a Little at a Time
When you’re doing a long project like writing a book, working at a sustainable pace is very important. That is especially true if it requires a lot of creativity and innovation.
You can easily get burned out by trying to do too much too quickly and when that happens, your creativity can go downhill quickly.
Sometimes you need to put it down, walk away from it for a while, and come back when you’re refreshed to start work again.
Agile Book Publishing – Overall Summary
I think all of this is just good, common-sense things to do – why do people have trouble doing this?
I think many people think of Agile as Scrum and also think about doing it mechanically. They aren’t sure how they would go about applying a Scrum process to this kind of effort
Agile is not just Scrum – it is a way of thinking. We need to understand the principles behind Agile. Don’t just do it mechanically and “by-the-book”. Take the time to interpret how it applies to your current situation and adapt it as necessary
What can Physics teach us about Agile Project Management? We can learn a lot from how the science of physics has evolved. I think there are a number of interesting similarities the way that Agile Project Management is evolving.
How Has the Science of Physics Evolved?
For many years until the late 1800’s and early 1900’s, physics was based on what is called “Classical Physics”.
What is Classical Physics?
“Classical physics is the physics of everyday phenomena of nature, those we can observe with our unaided senses. It deals primarily with mass, force and motion.
While its roots go back to the earliest times, to the Ancient Greeks such as Aristotle and Archimedes,
It later developed into a cohesive system with the contributions of Galileo, Kepler and Newton.
Classical physics achieved phenomenal success, as the Calculus of Newton and Leibniz gave it the tools to tackle even problems not imagined by its pioneers.”
How Has Classical Physics Evolved?
“Around 1900, give or take a decade, surprising new experimental evidence, primarily about atoms and molecules:
Showed us that these small-scale phenomena behave in ways not anticipated by classical theory
This ushered in a new era called “modern” physics.
New laws and methodology were developed to deal with the rapidly expanding experimental evidence.
Relativity and quantum mechanics added new tools to the study of nature.”
“These did not make classical physics “wrong”, for the old laws were working just as they always had, within their limited scope—which was the study of large objects (not atomic scale ones) moving relatively slowly (not near the speed of light). “
“So classical physics is still the starting point for learning about physics, and constitutes the bulk of the material in most introductory textbooks.
What Happened to Cause People to Rethink Classical Physics?
That notion of physics that was intended to define how the entire universe worked held together for a long time; however, serious weaknesses began to appear around the early 1900’s:
“By the end of the nineteenth century, most physicists were feeling quite smug. They seemed to have theories in place that would explain all physical phenomena. There was clearly a lot of cleaning up to do, but it looked like a fairly mechanical job: turn the crank on the calculator until the results come out. Apart from a few niggling problems like those lines in the light emitted by gas discharges, and the apparent dependence of the mass of high-speed electrons on their velocity”
Slavin, Alan J., “A Brief History and Philosophy of Physics”, Trent University
How is This Transformation Related to Agile Project Management?
Classical Physics Is Analogous to Traditional Plan-driven Project Management
Classical Physics is analogous to traditional, plan-driven project management. Similar to the laws of classical physics:
The traditional, plan-driven project management approach has been widely accepted as the only way to do project management for a long time
The way traditional, plan-driven project management is done hasn’t changed significantly since the 1950’s and 1960’s
It assumes a very predictable view of the world where it was possible to completely define a project plan with a fairly high level of certainty prior to the start of a project
Recognizing the Limitations
Physicists recognized the limitations of Classical Physics just as we are beginning to recognize the limitations of traditional plan-driven project management. The table below shows a comparison of how these two areas have evolved:
For many years, physicists believed that a model of the universe could be completely predicted based on some relatively simple and well-defined laws of classical physics
For a long time, we assumed that traditional plan-driven project management was the only way to do project management and that approach would work in any project
Beginning in the early 1900’s, modern physics began to evolve and the limitations in Classical Physics began to be much more apparent
In recent years, it is apparent that we are in a much more dynamic and more complex universe with much higher levels of uncertainty
In this new environment, Classical Physics still provides a foundation however, it is no longer a universal view of how the world works
In today’s world, we are beginning to recognize that a traditional plan-driven approach to project management is not the only way to do project management and it doesn’t work well in a very uncertain environment
What Are the Limitations of Physics and Project Management?
Traditional, plan-driven project management (just like Classical Physics) will never be totally obsolete and will continue to be a foundation for many areas of project management:
“…classical physics retains considerable utility as an excellent approximation in most situations of practical interest. Neither relativity nor quantum theory is required to build bridges or design cellphone antennas.”
The never-ending conundrums of classical physics, Trent University
Limitations of Classical Physics
However, it is important to recognize the limitations that are inherent in a traditional, plan-driven project management approach. Experienced physicists have learned to recognize the limitations of classical physics. It only works reliably in a certain range of situations as shown in the figure below:
“Classical Physics is usually concerned with everyday conditions: speeds much lower than the speed of light, and sizes much greater than that of atoms. Modern physics is usually concerned with high velocities and small distances.”
Limitations of Traditional Plan-driven Project Management
Similarly, project managers also need to recognize that a traditional, plan-driven project management approach only works reliably in a limited set of situations. In the project management world, this can be expressed with the Stacey Complexity Model:
In this model, there are two primary dimensions – one is requirements complexity and the other is technology complexity.
Traditional, plan-driven project management still works in areas of low complexity such as some construction projects. However, even in some of those areas, project managers have recognized a need for a somewhat more adaptive approach
As you get further out on either complexity axis, there is typically a need for more of an adaptive Agile approach. In that area, Agile is better suited for dealing with uncertainty but this is not a binary and mutually-exclusive proposition. There is a need to blend both approaches in the right proportions to fit the situation
The way that the science of Physics has evolved has some strong similarities to the evolution of Agile Project Management.
Classical Physics Is Like Traditional, Plan-driven Project Management
The foundation of Physics today is still Classical Physics, just as traditional plan-driven project management is still a foundation of project management today:
Classical Physics is the theory underlying the natural processes we observe everyday.
It is the key to understanding the motion of pulleys, machines, projectiles and planets.
It helps us understand geology, chemistry, astronomy, weather, tides and other natural phenomena”
Evolution of New Ways of Thinking
Just as new theories about Physics have significantly extended the notion of what “Physics” is beyond the Classical Physics, Agile will have a similar impact on project management. The way this will probably evolve is very likely similar to the way that Physics has evolved:
Project Management Evolution
In today’s world, there are people who specialize in Classical Physics There are also people who specialize in the more esoteric areas of Modern Physics
There will be project managers who continue to specialize in a traditional plan-driven approach to project management There will also be project managers who specialize in Agile
However, neither one of those areas can ignore the existence of the other area
Just as in Physics, neither one of those areas can ignore the existence of the other area
A truly broad-based Physicist has a fairly solid knowledge of both Classical and Modern Physics
A truly broad-based Agile Project Manager has a solid knowledge of both traditional plan-driven project management and Agile
PMBOK version 6 and the new PMI Agile Practice Guide signal a new direction for the future of project management. For the first time, PMI has started to integrate Agile and traditional plan-driven project management. What does that mean for the future of project management?
What’s the Impact?
I’ve written a number of articles on the future of project management and I get a lot of questions from project managers. Many are confused about the impact of Agile on project management and ask questions like “What Agile certification should I get?”.
Unfortunately, it’s not as simple as just going out and getting another certification like PMI-ACP
The PMI-ACP certification is a step in the right direction and it’s not an easy certification to get. However, it’s just a test of general Lean and Agile knowledge and is not aligned with a particular role.
In fact, the role of an Agile Project Manager Is not well-defined. There is even some controversy about whether there is a role for an Project Manager In an Agile environment.
Confusion Over Project Management Direction
It’s totally understandable why there would be a lot of confusion among project managers about how Agile might impact their career direction.
There are some project managers who are in “denial”.
They want to assume that traditional, plan-driven project management is the only way to do project management.
They assume that it will go on unchanged forever unchanged and Agile isn’t really a valid form of project management at all
On the other hand, there are people in the Agile community who believe that there is no need at all for traditional plan-driven project management. They believe that Agile is a solution to almost any problem you might have
An Objective, Pragmatic Viewpoint
I’m not an Agile zealot – I try to take a very objective and pragmatic approach.
In one of my courses, I have a slide that says “Saying Agile is better than Waterfall” is like saying “A car is better than a boat”. They both have advantages and disadvantages depending on the environment.
You have to be able to fit the approach to the problem rather than force-fitting all problems to one of those extremes.
Project managers who only know how to do traditional, plan-driven project management and try to force-fit all projects to that approach will be at a severe disadvantage relative to other project managers who know how to blend Agile and traditional project management in the right proportions to fit the situation.
What’s Wrong with Traditional, Plan-driven Project Management?
There’s nothing inherently wrong with the traditional, plan-driven approach to project management; the problem is in how its applied.
The primary problem with the traditional, plan-driven approach is that it works for situations where the requirements are well-defined. In that environment, the primary concern is planning and managing a project to meet those well-defined requirements within a given budgeted cost and schedule
That approach just doesn’t work well in situations where the requirements are much more uncertain. In an uncertain environment, the primary concern is not just managing costs and schedules but taking an adaptive approach to maximize the business results and value that the project produces.
In today’s rapidly-changing business environment the need for taking that kind of approach is becoming increasingly common.
The Future of Project Management
There’s essentially two sides of this equation: value and cost. In the past,
The value side has been assumed to be well-defined by a fixed set of requirements
Project managers only needed to worry about the cost side
In this new environment, that is no longer true. Project managers now need to worry about both maximizing value as well as managing costs and schedules. That’s a fundamental shift in thinking for many project managers – it means:
Taking a broader focus on maximizing the business value that a project produces
Using whatever methodology (or combination of methodologies) that makes sense to achieve those goals
Fitting the project management approach to the nature of the business problem rather than force-fitting all projects to a standard, plan-driven approach.
That raises the bar significantly for many project managers.
What Certification Should I Get?
Some people seem to think that it is only a matter of getting another certification. I’ve participated in several discussions lately where project managers were asking questions like:
“What certification should I get in order to get into Agile (CSM/PSM, CSPO, or ACP)?”
The answer to the question of “what certification should I get” depends on what role you want to play. It requires some thought because there is no well-defined role for a project manager in Agile at the team level
There are several possible career directions for project managers with regard to Agile. You may not:
Have to completely throw away your project management skills. However, you may ave to rethink them considerably in a very different context
Use some traditional project management skills very fully at all depending on the role you choose
Potential Agile Project Management Roles
There are several potential migration paths for project managers who want to develop into an Agile Project Management role:
1. Become a Scrum Master
A Scrum Master:
Ensures that the team is fully functional and productive
Enables close cooperation across all roles and functions
Shields the team from external interferences
Ensures that the process is followed, including issuing invitations to daily scrums, sprint reviews, and sprint planning
Facilitates the daily scrums
There’s a few project management skills that might be useful (at least indirectly) for that role. However, it doesn’t utilize much of the planning and management skills that a project manager typically has. For that reason, becoming a ScrumMaster may or may not make sense as a career direction for many project managers.
2. Become a Product Owner
The Scrum Alliance defines the primary responsibilities of a Product Owner as follows:
The product owner decides what will be built and in which order
Defines the features of the product or desired outcomes of the project
Chooses release date and content
Ensures profitability (ROI)
Prioritizes features/outcomes according to market value
Adjusts features/outcomes and priority as needed
Accepts or rejects work results
Facilitates scrum planning ceremony
The Product Owner role actually includes a lot of project management functions. However, it is actually much more similar to a Product Manager than a Project Manager. The major differences are that:
The Product Owner is a business decision-maker and requires some business domain knowledge that a project manager may not have.
The Product Owner role doesn’t typically include many team leadership skills. In an Agile environment, team leadership is more a function of the ScrumMaster and the team itself.
3. Hybrid Agile Project Management Role
For a lot of good reasons, many companies will choose to implement a hybrid Agile approach that blends the right level of traditional plan-driven project management with Agile.
This is a very challenging role for a project manager to play.
It requires a deep understanding of both Agile and traditional plan-driven project management to know how to blend these two seemingly disparate approaches together in the right proportions to fit a given situation.
4. Project/Program Management of Large, Complex Enterprise-level Agile Projects
There is a legitimate role for project managers in managing large, complex enterprise-level projects; however, there are several things to consider about planning your career in that direction:
This role is limited to large, complex projects that typically require multiple Agile teams
It also may require blending together some level of traditional plan-driven and Agile principles and practices in the right proportions to fit the situation
This role doesn’t exist at all on most small, single-team Agile projects
This role requires some very significant skills that can be very difficult to attain. Many people may assume that the PMI-ACP certification qualifies you to perform this role. It is a step in the right direction, but a lot more experience and knowledge is needed to perform this role including:
Knowing how to blend traditional, plan-driven principles and practices in the right proportions to fit a given project,
Adapting an agile approach to fit a business environment, and
Scaling Agile to an enterprise level.
You have to be a “rock star” Agile Project Manager to perform this role.
Agile will have a big impact on the future of the project management profession:
In many industries and application areas, the project management role associated with small, single-team projects may be completely eliminated by Agile
There may be some project managers who are not significantly impacted by this such as project managers in the construction industry, but even in those industries some knowledge of Agile principles and practices may be essential
This creates difficult choices for a Project Manager to make. Agile may force project managers to make some significant choices about their career direction. It isn’t as simple as just going out and getting another certification (like PMI-ACP).
I have mixed feelings about the subject of “Is PMP certification still relevant in today’s world?”. On the one hand,
I am a PMP myself,
I have had a PMP certification since 2004, and
I’m proud to be a PMP, but I recognize the limitations of a PMP certification.
On the other hand, I can clearly see the limitations in the PMP certification.
Is PMP Certification Still Relevant?
What Are the Limitations of PMP®?
PMP is heavily based on a traditional plan-driven project management approach (what many people loosely call “Waterfall”). The world is rapidly changing today. There’s nothing inherently wrong with a traditional plan-driven approach to project management under the right circumstances. However, we definitely shouldn’t try to force-fit all projects to that approach.
A traditional plan-driven approach to project management works well in situations where there is a relatively low level of uncertainty and predictability is important
It does not work well in situations
With a high level of uncertainty or
Where an emphasis on creativity and innovation may be more important than an emphasis on planning and control to achieve predictability
In today’s world,
A project manager needs to be capable of using a broader range of methodologies to fit the nature of the project rather than
Force-fitting all projects to a traditional plan-driven approach.
Situations are becoming increasingly common that require a more flexible and adaptive approach due to very uncertain rapidly-changing technology and a very dynamic and very competitive business marketplace.
What Is the Impact of Agile on PMP?
For those reasons,
Agile is having a profound effect on the project management profession that will cause us to rethink the way we do project management.
That doesn’t mean that traditional plan-driven project management and PMP are obsolete.
However, we’ve got to think of project management in broader terms and
Recognize that traditional plan-driven project management is not the only way to do project management
PMI-ACP® certification is a step in the right direction but it doesn’t go far enough in my opinion:
It doesn’t really address the big challenge that many project managers face today of “how do I blend Agile and traditional plan-driven project management in the right proportions to fit a given situation?”
Unfortunately, PMI still treats Agile and traditional plan-driven project management as fairly separate and independent domains of knowledge with little or no integration between the two
What About PMBOK® Version 6?
The final edition of PMBOK version 6 was released in September 2017. One of the big changes is that it contains more references to Agile.
The changes to PMBOK v6 barely scratch the surface of what needs to be done to develop a more integrated approach
I can’t imagine that future extensions to PMBOK will solve this problem either
The whole concept behind PMBOK is not very compatible with an Agile approach. These are two very different ways of thinking:
PMBOK is based on the idea that you can develop a checklist of things to consider in almost every conceivable project management situation that you can imagine.
Agile requires a very different mindset. An Agile approach needs to be much more adaptive and it would be impossible to develop a checklist defining what to do in every conceivable situation you might find yourself in in an Agile environment.
PMBOK and traditional plan-driven project management are based on a defined process control model
Agile is based on an empirical process control model which means that both the product and the process to produce the product are continuously adapted based on observation throughout the project
PMBOK is over 500 pages long with lots of details on what to do or consider in various situations
Agile is based on some very brief and succinct principles and values without a lot of detail and expects you to figure out what to do in a given situation.
PMBOK is also based on compartmentalizing a project into distinct and well-defined process groups
Agile requires a much more holistic and integrated approach to project management
What is the Long-term Solution?
This is not an easy problem to solve.
In the long-term, the solution to this problem is likely to involve some very significant rethinking of both PMBOK and PMI certifications.
What is needed is to create a much more integrated approach for blending Agile and traditional plan-driven project management principles and practices.
However, that is a very difficult problem to solve and is not likely to happen for a while.
What Is the Short-term Solution?
In the short-term, here are some possible strategies:
If You Have a PMP Today
If you already have a PMP certification today, that knowledge is a good foundation to begin to develop a broader focus on an Agile Project Management approach. However, it does require a lot of rethinking on how to do project management and also requires a very different mindset.
If You Don’t Already Have a PMP Certification Today
If you don’t already have a PMP certification today and are early in your career as a project manager, you have a much more difficult choice to make between two directions:
1. Getting a PMP
You could make a significant investment in time and money to get a PMP certification and then perhaps move on to learn an Agile approach sometime later. If you are working in an industry or application area where traditional plan-driven project management is still the dominant way of working, that might be a reasonable choice.
2. Skipping PMP
If you’re not working in an industry or application area where traditional plan-driven project management is the dominant way of working, getting a PMP may not make sense. Certainly, some foundation of traditional plan-driven project management is worthwhile but you may not need a full PMP for that. An alternative is to skip getting a PMP and just focus on developing an integrated approach to Agile Project Management.
In my opinion, skipping PMP and developing a more integrated Agile Project Management approach may be a reasonable for anyone
Who doesn’t already have a PMP and
Is interested in an Agile Project Management role.
However, it is a very difficult path to follow in the short term because:
There is currently no certification built around an integrated approach to Agile Project Management and
The knowledge base is not well-developed either
For that reason, you have to be somewhat of a “pioneer” in choosing this direction and
Since there is no certification, “you don’t know what you don’t know”.
Is PMP Still a Good Foundation?
Some elements of PMBOK and PMP are definitely useful as a foundation for any kind of project management.
However, the depth of study and knowledge required for PMP certification tends to “brainwash” people into thinking that PMP/PMBOK is the only way to do project management and that is not the case
Someone who only wants a foundation of knowledge in traditional plan-driven project management principles probably doesn’t need that depth of knowledge
The full PMP certification would still be appropriate for any project managers who plan to specialize in traditional plan-driven project management. However, that depth of knowledge in plan-driven project management should not be needed for someone who wants to develop an integrated Agile Project Management approach.
“Agile” means a lot of different things to different people. To some people:
It means just developing software faster, or
It means creating a more people-oriented project environment,
To others, it means making the project management process a lot more efficient by streamlining the whole process and eliminating unnecessary documentation
Those are only a few different connotations – there are many, many more. In addition, there are also many more stereotypes, myths, and misconceptions about what Agile is.
All of those things are potential outcomes of an Agile process but that’s not the fundamental essence of what an Agile process is all about in my opinion. The fundamental essence of an Agile process is adaptivity.
What’s Wrong With the Typical Project Management Approach?
Many project management processes, as we know them today, were designed around what is called a “traditional plan-driven project management” model (what many people loosely call “Waterfall“).
In this model, achieving predictability over the outcome of a project and the costs and schedule associated with achieving that outcome is very important
Therefore, it is also very important to have clearly-defined requirements as well as an adequate level of planning to be able to somewhat accurately predict the outcome, costs, and schedule of the project
That’s the predominant way that project management has worked since the 1950’s and 1960’s. A project was considered successful if it delivered what the requirements for the project within the defined budget and schedule.
That kind of predictability can be important. For large investments, it allows a company to:
Attempt to calculate with some level of certainty what the return on their investment is likely to be from a project, and
Make a go/no-go decision as to whether the project should be funded or not based on that information.
The primary problem with that approach is that it requires developing a fairly detailed plan for the project upfront. That is very difficult, if not impossible to do in projects with a very high level of uncertainty.
Why Is Adaptivity Important?
We live in a different world today from the world that existed in the 1950’s and 1960’s when formalized project management approaches were first defined.
Higher Levels of Uncertainty
There is a much higher level of uncertainty:
Technology is rapidly changing,
Solutions are much more complex, and
The business environment that we operate in is dynamic and constantly changing.
In that kind of environment,
Developing a detailed plan for a project with a lot of uncertainty upfront will typically require you to make a lot of assumptions.
And, many times those assumptions will be wrong and may require significant re-planning and possible rework later.
Rather than force-fitting a project that has a high level of uncertainty to a traditional plan-driven approach;
it’s much better to fit the methodology to the nature of the project and that’s where a more adaptive approach really makes sense.
That doesn’t mean that you don’t do any upfront planning; it means that you use a level of planning that is appropriate to the level of uncertainty in the project:
Traditional Plan-driven Approach
Adaptive (Agile) Approach
Atempt to develop a detailed set of requirements and a detailed project plan for the project upfront
Limit the amount of upfront planning based on the level of uncertainty in the project and use a “rolling-wave” planning approach to further define the requirements and plan for the project as the project is in place
Need for Creativity and Innovation
Another important factor in today’s environment is that there is a greater need for creativity and innovation to develop truly leading-edge products. An over-emphasis on planning and control can stifle creativity and innovation.
What’s an Example of a Project Requiring an Adaptive Approach?
A Simple Example
I use an example in my Agile Project Management training that is somewhat extreme but it gets the point across. The example I use is:
Suppose you were given the task to find a cure for cancer and you were asked to outline:
What the solution will be,
How long it will take to develop it, and
What the total cost of the research will be to develop the solution
In that situation, it would be ridiculous to attempt to develop a detailed project plan with accurate cost and schedule estimates – there is just way too many uncertainties to resolve
So what would you do? Give up and do nothing? That doesn’t make sense either
It’s important to recognize that we do know some things about finding a cure for cancer based on years of research that have gone into that area.
However, there are still way too many unknowns to develop a detailed project plan for a solution
What you would do is take advantage of what is known as much as possible and then take an iterative, trial-and-error approach to find a solution
Thomas Edison and the Light Bulb
That’s the way Edison invented the light bulb…here’s a quote from Edison on that subject:
“I speak without exaggeration when I say that I have constructed three thousand different theories in connection with the electric light, each one of them reasonable and apparently to be true. Yet only in two cases did my experiments prove the truth of my theory. My chief difficulty, as perhaps you know, was in constructing the carbon filament, the incandescence of which is the source of the light.”
(1890 Interview in Harper’s Magazine)
In a 1910 autobiography of Edison, Edison’s friend and associate Walter S. Mallory is quoted as asking
“Isn’t it a shame that with the tremendous amount of work you have done you haven’t been able to get any results?” The book goes on to say that “
Edison turned on him like a flash, and with a smile replied:
“Results! Why, man, I have gotten lots of results! I know several thousand things that won’t work!”
What Makes This Kind of Project Different?
There are two things that make this kind of project fundamentally different:
The level of uncertainty is very high and makes it impractical or impossible to develop a detailed plan for the project upfront
Creativity and innovation required for finding a good solution are far more important than predictability
How Does an Agile Approach Solve This Problem?
An Agile process is built on an “Empirical” Process Control model. The word “empirical” means “based on observation”. In the context of an Agile development process, “Empirical” means that during the course of a project, both the product as well as the process to produce the product are continuously refined as the project is in progress. The goal is to produce the right product and to optimize the value of the product being produced.
Why Is This Important to Project Managers?
You might ask “Why is this important to project managers?”
Isn’t Agile something that only applies to software development? (That’s a common misconception)
The truth is that all projects have some level of uncertainty associated with them
If you try to force-fit all projects to a traditional plan-driven project management approach, its just not going to work in many cases. Imagine, for example,
Trying to develop the next generation of the iPhone or any other new and innovative product
In that kind of project, creativity and innovation is just as important, if not more important, than predictability.
In this new environment, a project manager who only knows how to do a traditional plan-driven approach to project management will be at a serious disadvantage. What makes this even more difficult is that:
That this is not a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people seem to think.
It’s a matter of figuring out how to blend a traditional plan-driven approach with an adaptive (Agile) approach in the right proportions to fit a given situation.
Have you given any thought to “How to make a hybrid agile process work”? Some people claim that hybrid Agile projects don’t work at all. For example, I recently saw an article on LinkedIn entitled “Why Hybrid Agile-Waterfall Projects Fail” that caught my eye. I’m not surprised at this article:
It takes some skill to make a hybrid Agile process work successfully and
Anyone who would literally try to combine an Agile methodology with a Waterfall methodology to create a hybrid methodology is asking for failure
That’s not the way to do it
Blending Two Recipes
This should not be a matter of literally combining an Agile methodology and a Waterfall methodology.
That would be like trying to take a recipe for Italian food with a recipe for Chinese food and literally trying to mix the steps and ingredients from the two recipes together to create “Italian-Chinese” food
A good chef would never even attempt to do that. However, he/she might create an entirely new recipe that blends together some of the aspects of Italian food with some aspects of Chinese food
Creating a Hybrid Approach
It takes a lot of skill to create an effective hybrid approach and doing it effectively is like the difference between a “chef” and a “cook”.
When I talk about creating a hybrid approach, I try to avoid the words “Agile” and “Waterfall” if possible. Those words give people the impression that you are literally combining two different methodologies together like combining two food recipes:
I prefer to think of a hybrid approach as the appropriate blend of an adaptive approach and a plan-driven approach
The words “adaptive” and “plan-driven” convey an entirely different meaning than “Agile” and “Waterfall”
Creating a hybrid Agile approach:
Doesn’t mean that you’re trying to literally combine two very different methodologies by mixing them together
Means that you’re creating a blended approach with the appropriate amount of emphasis on being “adaptive” versus being “plan-driven”
An Example of a Real Hybrid Approach
Some years ago, I was responsible for managing a very large development program for a US Federal Government agency:
It had a fixed-price contract associated with it and a fairly aggressive delivery schedule
However, the customer wanted some level of flexibility in the details associated with defining requirements
Does not attempt to literally combine an Agile methodology and a Waterfall methodology
Instead, it wraps a plan-driven layer at the “macro” level around a standard Agile/Scrum process at the micro level
Here’s a diagram that shows what it looks like:
You might say:
“That’s not a hybrid Agile-Waterfall model”
“It’s only a standard Agile development process with a plan-driven layer wrapped around it”
And, that would be correct. There has been no attempt to actually mix-and match a phase-gate Waterfall methodology with an Agile methodology.
How To Make a Hybrid Agile Process Work
This is a very flexible model – that is why I call it a “framework” rather than a “methodology”.
1. Project Planning
You would start out a project by at least developing high-level requirements with estimates of the costs and schedule for completing the project. Those estimates can be as detailed as you want them to be. That’s what makes this model so flexible.
You can have a relatively “thin” layer of planning with very high-level requirements and less-detailed plans or
A “thicker” layer of planning with more detailed requirements and planning
The most important thing is that there has to be a common understanding between the development team and the customer about how detailed the requirements and plan are:
This model will not work without a spirit of trust and partnership between the customer and the project team. The customer and the project team must agree to working collaboratively as the project is in progress to:
Further elaborate requirements,
Continuously refine the plan for completing the project, and
Resolve any trade-offs that may be necessary to stay within the budgeted cost and schedule.
2. Managing Changes
As the project is in progress, any changes to the project requirements made at the “micro” level need to be fed back into the planning at the “macro” level.
By agreement with the customer, there should be enough slack built into the plan so that minor changes can be absorbed easily without a change to the overall plan. Only more significant changes might require trade-offs and adjustments. When trade-offs are needed to stay within the budgeted cost and schedule, the customer may need to either:
Adjust the planned cost and schedule as necessary, or
Reduce the scope of some of the requirements if necessary to fit within the planned cost and schedule.
This is a fairly simple model but it works and it can be easily adapted to a wide variety of projects. However, note that this is not a simple step-by-step cookbook approach. It takes some skill to make this model work successfully.
Matching the Level of Uncertainty in the Project
A good strategy is to match the design of the hybrid approach to the level of uncertainty in the project:
A project with a lower level of uncertainty might lend itself to a more plan-driven approach particularly if predictability is important to the customer of the project. That would mean putting a higher level of emphasis on developing a “thicker” layer of planning at the macro-level
A project with a higher level of uncertainty would probably require a more adaptive approach to further elaborate the plan as the project is in progress. That would mean that the level of planning at the macro-level would probably be much “thinner”.