Tag Archives: Agile

Closing the Gap Between Agile and Traditional Project Management

Most people will agree that there is a need for closing the gap between Agile and traditional project management communities – Agile and traditional project management are essentially treated as two separate and independent domains of knowledge with little or no integration between the two and that has led to some polarization between those two communities:

  1. On the Agile side, many Agile people think of “project management” as a bad word and see no need for it in an Agile project. The fact is that although you may not find anyone with the title of “Project Manager” in an Agile project, there’s lots of “project management” going on – its just a different style of project management and the functions are distributed among a number of people:
    • The Product Owner in a Scrum project performs many project management functions by setting the direction and priorities for the project and making decisions as the project progresses
    • The Scrum Master performs some project management functions by facilitating the team and the process as well as resolving obstacles
    • Everyone on an Agile team performs some very basic project management functions in planning and managing their own work and the work of the team as a whole
    • The important lesson to learn from this is that:

      “(project) management is a function, not a role”.
      (Daniel Mezick originally coined that phrase, in this blog post).

      Just because you don’t see anyone with the formal title of “Project Manager” doesn’t mean that there is no project management going on.

  2. On the Project Management side, PMI has created the ACP certification that recognizes Agile as an alternative form of project management; however:
    • That certification doesn’t go far enough to define what Agile Project Management is and how someone would use it in a typical project that might require blending Agile and traditional project management principles and practices to fit the situation.
    • PMI needs to go a lot further to develop a broader concept of what “project management” is that fully embraces both Agile and traditional project management roles.

    Many of the definitions in PMBOK such as what a “project” is and what “project management” is are based heavily on a traditional, plan-driven approach to project management and really wouldn’t apply to an Agile project. For that reason, it’s very understandable why someone in the Agile community would have the perception that traditional project management principles and practices don’t apply to Agile at all.

In order to close this gap, I think it is essential to rethink some of the things we’ve taken for granted about project management for a long time to develop a new vision for “project management” that embraces both Agile and traditional plan-driven project management principles and practices. Agile and Traditional Project Management approaches need to be perceived as complementary rather than competitive approaches that are also capable of being blended together as necessary to fit a situation rather than being binary, mutually-exclusive choices.
I’ve recently drafted a document entitled “The Next Generation of Project Management” which outlines some of the more significant shifts in thinking that I think are needed and I’ve shared a preliminary draft of this document with some key people in PMI with some recommendations for what I believe needs to be done. That document has generated a lot of interest and some excellent comments and I’ve updated it to reflect the comments I received. Please take a look at it and let me know if you agree with this vision and recommendations:

“The Next Generation of Project Management”

What is a Project?

What is a project? I recently wrote a blog post on “What is Project Management?” that has generated some good comments on LinkedIn:

What is Project Management?

One of the comments from Wayne Mack on that post was that “the change is not merely to redefine ‘project management’ but to redefine ‘project'”. He is absolutely right.

PMBOK defines a project as follows:

“A project is a temporary endeavor undertaken to create a unique product, service, or result”. The temporary nature of projects indicates that a project has a beginning and an end.”

There are potentially several problems with applying this definition to an agile project:

  • An agile project may not have a well-defined beginning and end. An Agile project will have an end at some point in time, but the end may be indeterminate when the project starts.
  • An Agile project may be more of an ongoing development effort with an end that is not well-defined and the end, when it happens, may be a long time in the future. That kind of effort might not be considered to be “temporary”.
  • An agile project generally creates “a unique product, service, or result”; however, that “unique product, service, or result” might not be well-defined at the beginning of the project and the goal of the project may be more broadly-defined at the beginning of the project.

This definition is probably based on how people have seen the value provided by a project manager. The value provided by a project manager has traditionally been seen as planning and managing the activities of a project to meet well-defined requirements within expected costs and schedules. Obviously, you can’t do that unless the project has a beginning and an end and the product, service, or result the project is intended to create is relatively well-defined at the beginning of the project.

A more broadly-defined initiative to meet a business goal or objective that doesn’t have specific, well-defined requirements at the beginning of the project and may not have a well-defined end-date at the beginning of the project probably wouldn’t fit this definition of a “project”. In that situation, the value provided by a project manager is likely to be very different and may put more emphasis on guiding the people, process, and tools, to maximize the business value that the effort provides.

If we broaden the vision of what “project management” is, we also need to broaden the definition of what a “project” is. There are probably two things that are needed to develop a more general definition:

  1. Drop the second statement that a project has a well-defined beginning and an end
  2. Broaden the first statement to include the potential that a project may be designed to satisfy a more broad-based business objective

The new definition of a project would come out something like this:

“A project is a endeavor undertaken to satisfy a broad-based business objective/outcome or to create a unique product, service, or result”.

If we broaden the definition of “project Management” to embrace Agile as well as traditional plan-driven projects, we must also broaden the definition of what a “project” is as well.

What is Project Management?

I recently saw a LinkedIn post on the subject of “What is Project Management?” that suggested the standard PMI definition of project management from PMBOK:

“‘the application of knowledge, skills, tools and techniques to the project activities to meet project requirements’

I had to respond to that because it seems to me that this classic definition that so many people seem to take for granted is way out-of-date. It seems to be based on the narrow, traditional notion that a project manager is someone who manages the costs and schedule of a project to meet defined requirements. That implies that there is only one way to do project management and that is based on simply executing a project where the requirements have been defined in advance.

We need to significantly expand that thinking to embrace the idea that a project manager may play a role in a much more uncertain environment where the requirements are less-defined and the project requires a more adaptive (Agile) approach that is focused more on maximizing the value the project produces rather than being totally focused on managing costs and schedules to deliver well-defined requirements. The person performing that function may not even have the formal title of “Project Manager”. Many people will claim that is not “project management” because there have been so many well-established stereotypes that have developed over the years about what “project management” is. Here are a few of them:

  • Project managers are very “command-and-control” oriented – Project managers are noted for getting results and sometimes that means being assertive and somewhat directive to set goals and manage the performance of project teams. Many times that behavior is expected of a project manager by the businesses that they operate in. In many companies if a project team is underperforming, the Project Manager is the one held responsible and is expected to take corrective action to get the project on track.
  • Project managers are rigid and inflexible and only know how to manage by the “Waterfall” methodology – For many years, project managers have been held accountable for managing the costs and schedules of projects and we all know that in order to meet cost and schedule goals, you have to control the scope of the project. That, in turn, requires a disciplined approach to defining and documenting detailed requirements and controlling changes where changes become the exception rather than the norm. The emphasis on managing costs and schedules naturally leads to extensive use of plan-driven or “Waterfall-style” methodologies that are based on trying to define the project requirements in detail upfront before the project starts and controlling changes once the project is in progress.
  • Project managers are just not adaptive and cannot adapt to an agile environment – Like the other stereotypes, there may be some amount of truth in this stereotype, but it would be inaccurate to generalize and say that this is true of all project managers. Agile will require some considerable rethinking of the project management approach and some project managers are so heavily engrained in the traditional way of operating because it has been so widely accepted as the norm for such a long time that they may have a difficult time adjusting to an Agile project approach.

The definition of “Project Management” should also not be limited to someone who has a title of “Project Manager”. In an Agile environment, “Project Management” functions are often distributed among other roles, but just because they are being done by someone who has does not have a formal title of “Project Manager” doesn’t mean that they are not part of the functional discipline of “Project Management”.

In my opinion, we need to make some radical shifts in thinking about what “Project Management” is – it’s evident to me that the project management profession and PMI, in particular, seem to unintentionally perpetuate these stereotypes by not addressing these basic definitions that are published in PMBOK that many project managers have taken for granted for many years.

Here’s my suggestion for a broader definition of what “Project Management” is:

“Project Management is the application of knowledge, skills, tools, and techniques to maximize the value that the project produces.”

Project management is focused on maximizing business results within the context of the business environment that a project is part of in a way that is appropriate to the nature of the project.

I’m sure that definition could be fine-tuned, but the key point that I’m trying to get across is that there isn’t just one way to do project management and one of the greatest skills of a project anyone performing a project management function should be to select the right approach to fit the nature of the project within the context of the business environment the project is part of. I’m also not excluding the possibility that this function might be performed by someone who doesn’t have the title of “Project Manager” (for example, a Product Owner in an Agile/Scrum project).

Levels of Mastery in Agile

I came across the diagram shown below that I think nicely summarizes different levels of mastery in Agile:

Levels of Mastery in Agile

Many people don’t seem to realize that there are these three different levels of mastery and just learning the basic practices is only the beginning.

The three levels of mastery are:

  1. Practices (Doing) – This level is associated with learning the basic practices of Agile at a mechanical level. There are many people who are at this level of learning – they’ve received their CSM certification (or equivalent) and they may have had some practice in the real world and know how to do the basics. The danger is that many people think that this is all they need to know when they have mastered this level of learning. People who get stuck in this level of learning can become fairly ritualistic or dogmatic and insist that there is only one way to do Agile and that is doing it exactly by the book as they have learned to do it.
  2. Principles (Understanding) – People who have gone on to this level of learning have gained a deeper understanding of the principles behind Agile and why it makes sense. This deeper level of understanding gives people a broader perspective – instead of seeing Agile as a mechanical process that must always be done ritualistically “by the book”, people at this level recognize that there may be a need to customize and adapt the processes to fit a given situation. They are also able to see Agile in a much broader context beyond the basic team-level Agile implementation and recognize the need to make Agile work at much higher levels of complexity for large enterprise-level projects.
  3. Values (Being) – This is the highest level of mastery. People at this level of learning not only understand the principles at a deeper level, they also understand the values behind those principles and have internalized those values into the way they work. People at this level are becoming Masters and are at the “top of their game” – they are able to easily go beyond applying Agile to routine Agile project implementations and they are able to apply the principles and practices to much more demanding and difficult situations with much higher levels of consistency and success.

The three levels of mastery shown in this diagram correspond to the “Shu-ha-ri” levels of mastery from martial arts that I have previously discussed:

Agile and Lessons Learned From the Martial Arts

How does this relate to the idea of “Agile Project Management”? People who look at Agile and traditional plan-driven project management practices (what many people tend to call “Waterfall”) at a basic level of practices will typically see them as competitive, mutually-exclusive, binary alternatives that are totally incompatible with each other. This is a basic problem and has led to Agile and traditional plan-driven project management being perceived as separate and independent domains of knowledge with little or no integration between the two.

I believe that people who are at a higher level of learning and understand the principles behind these two disciplines will see things in a very different perspective. They will probably see them as much more complementary rather than competitive and recognize the reasons why one set of principles and practices makes sense in one situation and not another. They will probably not see them as totally incompatible and be able to easily blend the two sets of principles and practices together as needed to fit a given situation.

One of my favorite analogies for this that was originally created by Bob Wysocki is the difference between a “cook” and a “chef”:

  • A good “cook” may have the ability to create some very good meals, but those dishes may be limited to a repertoire of standard dishes, and his/her knowledge of how to prepare those meals may be primarily based on following some predefined recipes out of a cookbook.
  • A “chef,” on the other hand, typically has a far greater ability to prepare a much broader range of more sophisticated dishes using much more exotic ingredients in some cases. His/her knowledge of how to prepare those meals is not limited to predefined recipes, and in many cases, a chef will create entirely new and innovative recipes for a given situation. The best chefs are not limited to a single cuisine and are capable of combining dishes from entirely different kinds of cuisine.

This is the challenge that I believe we face in creating a more integrated approach for Agile Project Management. We need to develop more “chefs” who are capable of seeing both Agile and traditional plan-driven project management principles and practices in a very different light as complementary rather than competitive alternatives. My passion is in helping project managers achieve that goal and I will be teaching the first graduate-level course on Agile Project Management at Boston University this fall. I am also publishing a new book that will be used as a text book in that course and that text book will be available to other universities and also to individual project managers. If you want any more information on that, let me know.

Rescuing Failing Projects

Rescuing failing projects is an interesting topic. There’s a show on TV that I really like – it’s called “The Profit” and it features a guy named Marcus Lemonis who goes out and turns around failing businesses. It comes on right after “Shark Tank” which is another popular show – the two shows are similar. The difference is that Shark Tank is primarily about entrepreneur’s who need help in the startup phase of getting a business off the ground where “The Profit” is about turning around established businesses that are failing because of poor business management.

The star of the show is a billionaire named Marcus Lemonis. I think his approach is very unique and is very similar to the approach I try to use in turning around failing projects:

  • Marcus recognizes that there are systemic factors that make a business successful and if those systemic factors are not well-designed and aligned with the objectives of the business, the business is going to have lots of difficulty succeeding no matter how hard people try to make it work. His view of the systemic success factors for a business are:
    • People
    • Process
    • Product
  • When Marcus goes in to turn around a failing business, he usually buys a controlling interest in the business. He tells the people in the business something like “I’m not here as a consultant, I’m here to turn this business around and make money from it”. Sometimes the existing business owners aren’t happy with giving up a controlling interest in their business but he knows that without a controlling interest in the business, he will have a difficult time having the influence he needs to turn the business around.
  • The show has a broad variety of episodes in different situations with different kinds of businesses and it is very interesting to watch the dynamics of the people involved – sometimes Marcus gets into very heated arguments with the existing owners of the business because they are so set in their ways of how they’ve run the business for so long and resist making a change. There are actually one or two shows where he has walked away from a failing business and he wasn’t able to turn it around. Generally, in those situations the existing owners of the business resisted what he wanted to do to turn the business around.

I feel like “the Marcus Lemonis of project management” because I have been called upon to help save a number of failing projects and I have a similar approach for rescuing failing projects – the three systemic factors that I look at are:

  • People – Having the right people on the project where the resources are well trained in the process and technology and a team that is cohesive, cross-functional, empowered, and self-organizing
  • Process – Having the right methodology in place that is well designed and appropriate to the nature of the project and is also well integrated with the client for the project so that the client is fully engaged collaboratively in the process in a spirit of partnership and trust
  • Tools – Having the right tools in place to support the process and people is extremely important to maximize the efficiency of the people and process especially in cases that involve distributed teams that are not collocated and very fast-paced demanding projects

My experience is that many times when a project is failing, people just push harder to make it work by putting pressure on people to work harder and neglect to address these systemic factors that are at the core of making any project successful. The approach I use is based on the notion that if these three factors are addressed and are well-designed, the project has a much higher probability of success and people shouldn’t have to work so hard to make it successful. It’s a case of working smarter rather than just working harder.

This is a good illustration of how an Agile Project Manager is very different from that of a traditional Project Manager. A traditional Project Manager is noted for “ram-rodding” projects and sometimes over-controlling and pushing hard on people to make it successful. That is what is frequently known as a “Death March” project. An Agile Project Management approach is to put in place the right people, process, and tools and not micro-manage the effort any more than necessary to make it successful.

“The Profit” is on CNBC right after Shark Tank; however the episodes are already over for this season and you would have to watch a previously recorded episode to see it at this point. I highly recommend it – it’s one of my favorite shows on TV. You can read more about it here if you’re interested:

The Profit Show

Will PMBOK Ever Be Compatible With Agile?

Will PMBOK ever be compatible With Agile? I’ve been working with Boston University and a number of other universities on developing an Agile Project Management curriculum. In developing a curriculum like this, there are two aspects of the work to be done:

  1. Synthesizing or developing the body of knowledge that you need to communicate, and
  2. Designing the course materials to teach that body of knowledge

If the body of knowledge associated with Agile Project Management were available in some well-documented form like PMBOK(R) that integrated both traditional plan-driven project management knowledge with Agile, the task of developing a course to teach that body of knowledge would be much easier, but I’m not sure if the body of knowledge associated with Agile Project Management will ever exist in that form.

Some books have attempted to map PMBOK(R) to Agile principles and practices (Michelle Sliger’s Book, “The Software Manager’s Bridge to Agility” is an example). Although you can make some general comparisons, it’s like comparing apples and oranges, in my opinion – there may be some similarities, but the whole philosophy behind PMBOK(R) is very different from the philosophy behind an Agile approach which makes it very difficult, if not impossible, to combine the two. Here’s why:

The Difference Between Explicit and Tacit Knowledge

PMBOK and Agile are based on two very different approaches to knowledge management:

    • Explicit Knowledge – is “codified knowledge found in documents, databases, etc.”
    • PMBOK relies very heavily on explicit knowledge – it attempts to codify a checklist of things to consider in almost every conceivable project management situation

    • Tacit Knowledge – is “intuitive knowledge and know-how, which is:
      • “Rooted in context, experience, practice, and values”
      • “Hard to communicate – it resides in the mind of the practitioner”
      • “The best source of long-term competitive advantage and innovation”
      • “Is passed on through socialization, mentoring, etc.” – it is not handled well by systems that try to document and codify that knowledge.
      • Agile takes a very different approach which is more consistent with tacit knowledge – instead of providing a detailed comprehensive checklist of things to consider in a broad range of different situations, Agile provides some general, higher level principles and values that need to be interpreted in the context of the situation.

“Source: KMT – An Educational KM Site, http://www.knowledge-management-tools.net/different-types-of-knowledge.html

The Difference Between a Defined Process Model and an Empirical Process Model

That doesn’t mean that PMBOK is bad and Agile is good which is an inference that many people might jump to. That’s like saying “a car is better than a boat”. Neither one is inherently good or bad and each has advantages and disadvantages depending on the environment you’re in. An explicit knowledge approach such as PMBOK is very consistent with a “defined process model”. A “defined process model” looks something like this:

Defined Process Model

The characteristics of a defined process are:

  • The process is repeatable and doesn’t change from one project to the next
  • The process is predictable – given a similar set of inputs, it should produce a predictable set of outputs

Agile is based on more of a tacit knowledge approach which is more consistent with an empirical process model which looks something like this:

Empirical Process Model

The process is intended to be adaptive to fit the situation based on continuous improvement and learning. Scrum is a good example of an empirical process model.

Can PMBOK Be Modified to Integrate Agile Thinking?

There have been some attempts recently to make PMBOK(R) more compatible with adaptive project management approaches:

  • PMBOK(R) version 5 which has been released incorporates some more inclusion of adaptive thinking but it is still heavily oriented around a plan-driven approach
  • PMI(R) has introduced the “Software Extensions to the PMBOK(R) Guide” which has a little bit more mention of Agile but is still limited

I participated in a LinkedIn discussion group some time ago where someone made the comment that PMBOK was very adaptive to an agile approach and challenged me to defend why that was not so. My response was that almost anything can be adapted to a different purpose but that doesn’t mean it is really optimized for that purpose. I could say a piece of sheet metal is adaptable to be used for writing a book because I can write on it like a piece of paper but is it really optimized for that purpose? (I don’t think so) Extending PMBOK to be a single knowledge base to cover both Agile and traditional plan-driven project management would be like modifying a car so that it could also be used as a speed boat. You might be able to do it, but the results are not likely to be very optimal.

Patching up PMBOK(R) to make it more “Agile” is likely to have significant limits for that reason. Agile is a different way of thinking, in my opinion – it acknowledges that we don’t know everything we need to know and puts an emphasis on rapid and adaptive learning.

The Next Generation of Project Management

Background

What is the next generation of project management? What is the impact of Agile on the future of project management?

  • Does it mean that project managers who are heavily trained in a traditional plan-driven approach to project management will become obsolete over some period of time?
  • What do project managers need to do to adjust their career direction to adapt to the future direction of project management?

I believe that the project management profession is at a major turning point that requires broadening our view of what “project management” is and reshaping the direction of the project management profession for the future to fully embrace and integrate both Agile and traditional plan-driven project management as complementary approaches within an overall project management portfolio.

Reinventing Project Management

What sort of image comes to your mind when you think of the words “project management”? Does it have any relationship to Agile? My guess is that many people have a very well-ingrained image of what “project management” is and many people wouldn’t associate “project management” with Agile at all. In fact, many people still see those two disciplines as polar opposites. To see things differently, we have to broaden our thinking about what “project management” is and get past many of the well-established stereotypes of what “project management” is.

Long-lasting companies have learned to “reinvent” themselves from time-to-time to keep up with changes in technology and the business environment they operate in. Here’s an excerpt from Harvard Business Review on that topic:

“Sooner or later, all businesses, even the most successful, run out of room to grow. Faced with this unpleasant reality, they are compelled to reinvent themselves periodically. The ability to pull off this difficult feat—to jump from the maturity stage of one business to the growth stage of the next—is what separates high performers from those whose time at the top is all too brief.”

“The potential consequences are dire for any organization that fails to reinvent itself in time. As Matthew S. Olson and Derek van Bever demonstrate in their book Stall Points, once a company runs up against a major stall in its growth, it has less than a 10% chance of ever fully recovering. Those odds are certainly daunting, and they do much to explain why two-thirds of stalled companies are later acquired, taken private, or forced into bankruptcy.”

Source: “Reinvent Your Business Before It’s Too Late”, Harvard Business Review, January 2011,

Here’s another excellent article on that subject:

“A successful company is like a great white shark. In its prime, it chews up the competition, but if it dares to sit still for too long, it dies. Some of the world’s most profitable and enduring companies have achieved their long track record of success by constantly reinventing themselves.”

“Cell phone maker Nokia started off selling rubber boots. The oil giant Shell used to import and sell actual shells. But these companies and the eight others on our list adapted with the times, evolving their product lines and business strategies to stay one step ahead of their customers’ needs. In business, it’s better to be a chameleon than a great white.”

Source: How Stuff Works, “10 Companies That Completely Reinvented Themselves”

Check out the link above for some great examples of companies that have done that successfully. As the article points out, the trick is recognizing that you are at a “stall point” and taking action before you have stalled for very long and that can be a difficult thing to do.

Project Management History

To understand the transformation that is going on, its useful to look at the history of project management and how we got to where we are today:

  • Early History – Project Management could probably be considered to be one of the world’s oldest professions. Think of the Egyptian pyramids and the Great Wall of China. The level of “project management” at that time may have been very crude and they probably didn’t call it “project management” at all but large efforts like that don’t just happen without some kind of planning and organization behind them. In the US, the development of the Transcontinental Railway in the late 1800’s is another example of a very large effort that had to have some kind of planning and organization behind it.
  • Scientific Management Approach – Around the turn of the century, along came Frederick Taylor and his co-worker, Henry Gantt. Frederick Taylor started developing new theories on how to organize workers and Henry Gantt created his famous Gantt Charts to describe the order of operations in work.
  • World War II and the 50’s and 60’s – World War II resulted in the Manhattan project which was another huge effort and the 1950’s and 1960’s had more large scale efforts such as the Polaris missile program and the Apollo program to put a man on the moon. PERT and CPM were invented and then in 1969, PMI was founded.

The Next Generation of Project Management

The general approach for doing project management hasn’t changed significantly since that time and the big question is “What’s next?” and also “Why Now?” Has the project management profession reached its peak or is there yet another major phase of growth that is just beginning to take place? I believe it is the latter. Here’s why I believe it there is some level of urgency to rethink the way we think about “project management” – the diagram below shows how the adoption rate of new technologies has changed over the last century.

Consumption of New Technology Trends

“Source: Mulbrandon, Catherine, Visualizing Economics – Adoption of New Technology Since 1900, http://visualizingeconomics.com/blog/2008/02/18/adoption-of-new-technology-since-1900

This data only goes through 2005, but you can be sure this trend hasn’t slowed down since then. (Think of how quickly smartphones have evolved as an example) This rapid proliferation of new technology calls for a new approach to project management – the traditional, heavily plan-driven approaches of the past can’t keep pace with the speed that technology is changing in many areas.

This dynamic and rapidly changing environment calls for a more adaptive project management approach but that doesn’t necessarily mean that we need to throw out everything we’ve learned about traditional, plan-driven project management and start over again but it does create some significant challenges for individual project managers and for the project management profession, as a whole.

What’s the Impact on Project Managers?

This “raises the bar” for project managers significantly:

  • In the past, if you had a PMP certificate, that was as far as you needed to go for many project management roles.
  • PMI has now created the Agile Certified Professional (ACP) certification and that’s not an easy certification to get, but that’s only the beginning, in my opinion.
  • I think the ACP exam is good certification but it doesn’t go far enough. It is really mostly a test of terminology – it doesn’t really test whether you know how to integrate Agile and traditional project management principles and practices in the right proportions to fit a given situation and that’s the real challenge for project managers, in my opinion.
  • A good Agile Project Manager also needs to be a strong cross-functional leader – he/she cannot be just a coordinator or administrator. That means he/she needs to have some credible knowledge of the functions included in the project.

What Needs to be Done to Address These Challenges?

This is a huge challenge to transform the project management profession and broaden our thinking of what a “project manager” is and it will take some time. However, I believe that the alternative of ignoring these trends and continuing to think of a project manager in the narrow context of someone who only does traditional, plan-driven project management approaches will seriously degrade and undermine the project management profession over time. Here’s what I think needs to be done to address this challenge:

  • Acknowledge the Need to Make a Change – The first step in any twelve step program is to acknowledge that we have a problem – we cannot deny the impact of Agile on the project management profession and think that traditional, plan-driven project management approaches as we know them will go on forever and Agile is just a fad that will go away.
  • Get Past Stereotypes – There are many stereotypes about what traditional project management is and about what Agile is that we need to overcome and change our thinking to see both Agile and traditional project management approaches as complementary to each other rather than competitive.
  • Redefine Project Management – We have to better define and develop the concept of what an “Agile Project Manager” is and better define the role that an “Agile Project Manager” might play. In my view, an “Agile Project Manager” is not someone who only does Agile projects; it is someone who has a deep knowledge of both Agile and traditional plan-driven principles and practices and knows how to blend them together in the right proportions to fit a given situation.
  • Develop Agile Project Management Training – We need to develop training programs and resources to help project managers reach the goal of becoming an “Agile Project Manager”.
  • Influence Enterprise-level Management Practices – Project Managers are a product of the environment that they work in. For example, many project managers take a heavily plan-driven approach to controlling costs and schedules of a project because that’s what their organizations expect of them. To change the behavior of project managers, we change the expectations of what companies expect of project managers and that can require some significant changes in organizational culture.

How I’m Trying to Help

I am very passionate about this because I believe it is critical to the future of the project management profession. Here are some of the things I’m doing to try to help project managers address these challenges:

  • Graduate-level Course at Boston University – I’ve developed a new graduate-level course on Agile Project Management that will be offered at Boston University
  • New Book – I am currently finalizing a new book called “The Art and Practice of Agile Project Management” that will be released in February 2015. Check it out here:
    The Project Manager’s Guide to Mastering Agile
  • Online Training – I’ve created several online training courses to help project managers begin to make the transformation to an Agile Project Management approach. Check it out here:Online Training Courses

I encourage any project manager who needs help in making this transformation to check into my online training courses and to contact me directly if I can be of help.

Religious Fervor About Agile

There is a lot of religious fervor about Agile. I was reviewing one of Dean Leffingwell’s slide presentations and I particularly like a comment he made that “Agile is only a tool – it’s not a religion”. It inspired me to write this blog post.

There is a lot of religious fervor about Agile and there’s a lot of polarization between proponents of Agile and more traditional plan-driven project management approaches. There’s a lot of good reasons why Agile makes sense in many situations but some people seem to just do it mechanically – becoming Agile becomes a goal in itself without a real understanding of why it makes sense in a given situation.

Don Reinertsen has written a good book called “The Principles of Product Development Flow” which I think provides some valuable insight into the underlying principles of why Agile makes sense. Since the principles are general enough to apply to any product development effort (agile or plan-driven), it also provides an objective foundation for comparing Agile and alternative plan-driven approaches so that someone can make an intelligent and objective assessment regarding how these two seemingly competitive approaches.

The book goes a little too far in my view in developing a quantitative and mathematically sound basis for the principles in the book based on statistics. In the real world, I don’t think very many people would really apply that kind of rigor to analyzing these principles; but nonetheless, an understanding of the principles, at least at a high level, is very valuable and it does provide an objective foundation for understanding the benefits of Agile at a deeper level. I’ve summarized the eight principles here:

  1. Economics – Take an Economic View
  2. “Why do we want to change the product development process? The answer: To increase profits. As obvious as this might seem, this economic view of product development is surprisingly uncommon. Instead, product developers create a large number of proxy objectives: increase innovation, improve quality, conform to plan, shorten development cycles, eliminate waste, etc. Unfortunately, they rarely understand how these proxy objectives influence profits.”

    I’ve seen a number of instances where companies blindly pursue some of those “proxy objectives” such as “increasing innovation” without really understanding the economic impact. “Increasing innovation” should not be an end-in-itself and it reaches a point of diminishing returns at some point and it might begin to impact other proxy variables such as quality. For that reason, it is good to put things in an economic context. The economic context provides a framework for making intelligent decisions about how much to focus on these “proxy variables”.

    Here’s another example of how the economic context can provide a sound framework for decision-making: It is generally best to defer decisions on product features as long as possible; however, some decisions should be made early and should not be significantly deferred because if they have a significant economic impact.

  3. Queues – Actively Manage Queues
  4. “Queues matter because they are economically important, they are poorly managed and they have the potential to be much better managed. Queues profoundly affect the economics of product development. They cause valuable work products to sit idle, waiting to access busy resources.… queues hurt cycle time, quality, and efficiency.”

    Developing requirements far-in-advance that sit in a queue waiting for development can be inefficient and wasteful because:

    • The requirements may change prior to going into development and much of the effort involved in developing the requirements might have been wasted, and/or
    • Speculation in the requirements that are done too far into the future can result in erroneous assumptions that make their way into development without being questioned

    But, on the other hand, we shouldn’t blindly accept the notion that developing requirements far-in-advance never makes sense. There are a lot of reasons why at least developing high-level requirements early might make sense to guide the overall direction and architecture of the project and I’m sure that there are even some circumstances where developing more detailed requirements upfront also makes sense (you have to use the economic impact as a guide for making that decision).

  5. Variability – Understand and Exploit Variability
  6. “There is probably no aspect of product development that is more misunderstood than variability. Because variability sometimes leads to bad outcomes, it is inevitable that some observers will incorrectly generalize that variability always leads to bad outcomes.”

    Reducing variability will many times improve efficiency but that isn’t always the case. For example, Breaking up large requirements into smaller ones that are of a more uniform size reduces variability and can improve flow, however, at some point further attempts to reduce variability do not have economic value.

    For example, if we might use a rule-of-thumb that if a user story is greater than 13 story points, it needs to be broken down, but there’s probably little value in breaking down a story with less than 13 story points into smaller story points just to reduce variability.

  7. Batch Size – Reduce Batch Size
  8. “Ask a typical product developer what their batch size is, and why, and you will get a puzzled look…Product developers simply don’t think in terms of batch size. As a result, they underutilize one of the most important tools to improve flow… many of the most important improvements in product development such as concurrent engineering, rapid prototyping, and agile software methods, are recognizable as batch size reductions. However, because developers fail to recognize the underlying mechanism of action behind these improvements, they lose the chance to apply this principle more broadly throughout their development process.”

    Examples of batch size inefficiencies include:

    • Project scope – taking on more in a single project than is truly necessary
    • Project Funding – The entire project is conceived and funded as a single large batch proposal
    • Requirements definition – the tendency to define 100% of the requirements before the project starts
  9. WIP Constraints – Apply WIP Constraints
  10. Use Work in Process (WIP) constraints to manage overall flow. For example,

    • Control the number of projects taken on at any one time to avoid over-saturating development resources
    • Use specialized resources wisely to maximize their impact on overall flow
  11. Cadence, Synchronization, and Flow Control – Control Flow Under Uncertainty: Cadence and Synchronization
  12. “Cadence is the use of a regular predictable rhythm within a process. This rhythm transforms unpredictable events into predictable events. It plays an important role in preventing variability from accumulating in a sequential process…”

    Examples of cadence are using fixed-length sprints to establish a pace for the development effort. Examples of the use of synchronization include:

    • Concurrent development on multiple paths at the same time
    • Concurrent testing of multiple subsystems
  13. Fast Feedback – Get Feedback as Fast as Possible
  14. Fast feedback can lower the expected loss by truncating unproductive paths more quickly or raise the expected gain because we can exploit an emergent opportunity by rapidly redirecting resources.

    Fast feedback combined with selecting appropriate measures of performance enables rapid learning and ongoing continuous improvement.

  15. Decentralized Control – Decentralize Control
  16. “Sophisticated military organizations can provide very advanced models of centrally coordinated, decentralized control. There is an impression that military organizations seek robotic compliance from subordinates to the orders of superiors. In reality, the modern military focuses on responding to a fluid battlefield, rather than executing a predetermined plan. It views war as a domain of inherent uncertainty, where the side that can best exploit uncertainty will win.”

    Source: Reinertsen, Don, The Principles of Product Development Flow, Celeritas Publishing, 2009

    I recommend Don Reinertsen’s book for anyone who wants more in-depth understanding of these principles (this is just a very high-level overview). Many people get immersed in the mechanics of Scrum and forget about the values and principles of Agile. Don Reinertsen’s book goes even deeper into the principles of product development which provides a good basis for understanding both Agile (adaptive) approaches and Waterfall (plan-driven) approaches at a deeper level.

Learning to Live with Uncertainty with Agile

Learning to live with uncertainty with Agile as well as risks is probably the single-most important factor in the success of any project (Agile or not). Prior to Agile, there may have been a false sense of security from thinking that you could completely eliminate or significantly reduce uncertainty by developing a very detailed requirements document for a project upfront and then managing changes against those requirements to control the scope and direction of the project as the project was in progress. That sort of approach is often taken in order to gain some level of predictability over the costs and schedule of a project; however, it turns out in many cases that a good deal of that feeling of security may be an illusion and it is very difficult to accurately predict all the requirements, costs, and schedules of a software development project.

With Agile, in many situations, the pendulum has swung almost completely in the opposite direction – a lot of people in the Agile community seem to feel that any attempt to try to reduce uncertainty upfront in order to estimate the costs and schedules of a project is totally futile because there might be a lot of uncertainty involved and the requirements are only going to change anyway. For that reason, there is a tendency to not do any significant amount of upfront planning at all. I think that’s an over-reaction and it can lead to significant problems.

Here’s an example of the impact that it can have…I was recently involved in helping to rescue an Agile project that had gone on for about two years with no end in sight. One of the basic problems was that the project had started without a sufficient level of upfront planning to assess the scope of the effort and it was just assumed that it could be completed in some reasonable amount of time by a single agile team. After about two years into the effort, we stepped back and did some re-planning and found that the scope of the effort was much larger than a single Agile team could accomplish in any reasonable amount of time. If more upfront planning had been done prior to the start of the project, perhaps that probably would have been discovered much earlier.

There are two primary problems I have often seen in this area:

  1. Analysis Paralysis – Delaying making commitments or delaying making any decision at all until the information required to make the decision has a very high level of certainty about it and all of the risks associated with a project are well-known and completely understood. This is often associated with the traditional, “Waterfall” style of project management. Unfortunately, this mode of operation has been the norm in some companies that are very control-oriented and insist on accurately knowing the cost and schedule of a project before making a significant commitment to it.
  2. Cavalier Approach – The opposite of that can be to take a very “cavalier” approach to not worrying about managing uncertainty and risk at all, just start doing a project with very little or no upfront planning, and assume that whatever uncertainty and risk is inherent in the project will be discovered and somehow work itself out as the project proceeds. This is the approach I’ve commonly seen on a number of Agile projects.

Neither of those approaches is ideal and each can lead to significant problems – a better approach is to make informed decisions as best you can based on the level of risk and uncertainty in the situation. Very few decisions in life are based on either 100% known or 100% unknown information and learning to deal with uncertainty is a skill that any good Project Manager learns over time. That skill is still very important in an Agile environment and shouldn’t be ignored. (In other words, don’t throw the baby out with the bath water when you convert from a traditional project management approach to a more Agile approach)

You often have to make informed decisions based on very uncertain information because waiting for the information to become more certain might significantly delay the project or might not make sense at all. The approach that seems to work best is to separate the “known” from the “unknown”, make some hypotheses or assumptions about the unknowns if necessary, and then evaluate the risks of going forward based on those assumptions or hypotheses. If you do decide to go forward, you shouldn’t forget that those assumptions were only assumptions and may need to be revalidated later on. (Many people often forget to do that) In a traditional project management environment, a risk register or something equivalent to that might be used for that purpose – that level of formality may or may not be necessary in an Agile environment, but the principle is nonetheless important.

There’s an art and a skill associated with making decisions in an uncertain environment. Anyone who has done any gambling where you have to bet on the outcome of an unknown event (like what card you’re going to be dealt next) will understand that. Here’s an example: most people are familiar with the game of “Blackjack” where the players play against the dealer and both try to win by getting the highest possible hand without going over 21. Suppose I have been dealt two cards and have a total of 16 points in my hand and the dealer is showing one card as a “10” with the second card face down. There are several very significant things I don’t know:

  1. I don’t know what the dealer’s second card is because it is turned face-down
  2. I don’t know what card I might be dealt next

Both of those are significant unknown’s but I have to make a decision anyway. I can make an assumption that the dealer’s second card is either a “10” or a face card (Jack, Queen, or King) for a total of 20 points. That assumption seems to make a lot of sense because there are more ways for that to happen (Ten, Jack, Queen, or King) than any other individual card. If I make that assumption, I know that my 16-point hand is going to lose against his 20-point hand and the best course of action would be for me to take a hit and risk going over 21 because I’m going to lose anyway if I don’t take a card or I don’t do anything at all.

That’s an example of an informed decision – I separated the known’s from the unknown’s and then made a reasonable assumption about the unkown’s based on the risks I saw in the situation. The alternative would have been to make no decision at all because the unknown’s were too significant or to make a “shoot from the hip”, uninformed decision to do something without doing any real analysis or really thinking it through at all and without taking advantage of the facts I do know in order to make a decision. I think that one of the most important factors for doing that effectively is to make an honest and objective assessment of the situation. There are several errors I have seen often with this:

  1. Overestimate the level of confidence in what is known and make a poor decision based on incomplete information
  2. Make an assumption about incomplete information, make a decision based on that assumption, press forward, and then forget that it was an assumption and never retest or revalidate it later
  3. Ignore whatever known information is available for making a decision and just proceed forward based on an uninformed decision without taking advantage of what is known