Why Is Planning So Difficult? Is It a Waste of Time?

A lot of people seem to think that planning is very difficult and some think it is a waste of time because even very well-thought-out plans often don’t work out as expected. This issue is important because it is at the crux of selecting a methodology and planning approach for a project. Do I use Agile or plan-driven (aka Waterfall)?

Why Is Planning So Difficult?

What’s the Issue?

Many people seem to think of planning as an “all-or-nothing” proposition – either you develop a highly-detailed plan or you do no planning at all. I don’t believe that to be the case. There seems to be two major problems associated with why people have this difficulty with planning:

  • Dealing with Uncertainty – Many people seem to have difficulty dealing with an uncertain environment – they want things to be crystal-clear, black-and-white and, in an uncertain environment, they think that it is a waste of time to do any planning at all.
  • Unrealistic Expectations – A related factor is that many people develop unrealistic expectations about planning. If you develop a well thought-out plan, they expect that it should work every time.

Planning Fundamentals

Let’s review some fundamentals about planning. We can learn a lot about how the military does planning. Here are a few of my favorite quotes on planning in the military:

Plans are nothing; planning is everything”  – (Dwight D. Eisehhower)

What Eisenhower is saying is that documented plans should not be considered to be sacrosanct. When people document a written plan, the plan often takes on a life of its own and it becomes the “gospel” that everyone is expected to follow rigidly. However, that should not be the case and that’s not a reason to not do any planning at all. Here are a couple of examples:

  • It would be foolish for a military unit to go into battle without doing any planning at all. You should take advantage of whatever intelligence you might have about the enemy positions (as uncertain as it might be) but you shouldn’t lose sight of the uncertainty in that information.
  • In an American football game (or any other sport you want to choose), the coach prepares his team for what he thinks the strengths and weaknesses of the opposing team are; but, again, that planning is based on very uncertain information.

Here’s another quote along the same lines:

“No battle plan survives contact with the enemy”  (Helmuth von Moltke the Elder)

What Helmuth von Moltke is saying is that you shouldn’t expect that a plan won’t change once you start to execute it.

An Example

In World War II, Churchill and Roosevelt spent years planning the invasion of Europe that ultimately resulted in the Allied forces landing in Normandy on June 6, 1944. Both men knew that an invasion of Europe was the only way that the war could be won but there was a lot of uncertainty about:

  • Where should the invasion take place? How can the Allies preserve the element of surprise?
  • What’s the best way to do it? What are the odds of it being successful?
  • How many deaths and casualties can be expected?

This was a huge decision that had enormous impact:

  • The fate of Europe and most of the free world at that time depended on this being successful
  • Both men faced some level of opposition at home knowing the large number of casualties to be expected

It took a lot of courage to do this planning and make a decision under these circumstances knowing that there was a lot of uncertainty in the situation and the impact of the decision was so enormous. However, planning was essential – can you imagine what might have happened if the invasion was attempted without any planning at all?
Source: Churchill and Roosevelt Spent Years Planning D-Day

What Does This Have to Do With Agile?

This whole issue about planning is at the crux of the controversy about “Agile versus Waterfall”. Many people see the choice between Agile and Waterfall as a binary and mutually-exclusive choice and planning is at the center of this controversy:

  • Some people in the Agile community might say that its a waste of time to do planning in an uncertain environment and you should just use an Agile approach and let the project plan evolve as the project is in progress
  • Some people in the traditional plan-driven project management community might say that it would be foolish to attempt to do a project of any significant scope without a detailed plan for it to be successful

The truth is somewhere between those extremes – it’s not an “all or nothing” choice and you should adapt the methodology and the level of planning to fit the nature of the project. The level of uncertainty in the project is a big factor in making that choice. In an uncertain environment, it might be foolish to attempt to develop a highly detailed plan but uncertainty is never an absolute and it would be equally foolish not to take advantage of whatever information you might have.

How Do You Put This Into Practice?

A lot of people might see “Agile” and “Waterfall” as totally incompatible with each other. It’s like mixing oil and vinegar – they just don’t mix well. And, if you look at this from a mechanical process perspective, that might be true.

In order to learn how to blend these two approaches together in the right proportions to fit a given situation, you have to understand both approaches at a deeper level and get past some of the strong stereotypes that exist about both “Agile” and “Waterfall”. It’s really a choice of selecting a more plan-driven approach or a more adaptive approach based on the level of uncertainty in the situation and there is a whole spectrum of alternatives as shown below:

Levels of Agility

The words “Agile” and “Waterfall” are two of the most loosely-used words in the English language. People use those words as if both “Agile” and “Waterfall” are specific and well-defined methodologies that can be easily compared and they are not:

  • “Agile” is not a methodology – “Agile” is actually a range of different approaches – Kanban and Scrum are both Agile methodologies (or frameworks) but they are very different. Kanban is more adaptive than Scrum and Scrum is more plan-driven than Kanban. Both approaches are very flexible and can be adapted to a range of different circumstances.
  • When people talk about “Waterfall”, they are generally not talking about the specific “Waterfall” process that was defined by Dr. Winston Royce in the 1970’s. As it is typically used, in practice, the word “Waterfall” really means any plan-driven methodology that is not totally agile. For example what about the Rational Unified Process (RUP)? That’s an iterative process but many people might include that in the realm of “Waterfall”.

Overall Summary

Here’s a summary of the key points I want to make in this article:

  • Planning is not an “all or nothing” proposition
  • The planning approach should be directly related to the level of uncertainty in the environment
  • Planning in an uncertain environment can be difficult but that should not be an excuse for not doing any planning at all
  • However, to quote Eisenhower, “A plan is nothing – planning is everything” – don’t get locked into thinking that a written plan is sacrosanct and doesn’t need to be changed to adapt to the level of uncertainty in the environment
  • The level of uncertainty and the planning approach is probably the most important factor in selecting a project management approach for projects
  • Rather than thinking about “Agile” and “Waterfall” as binary and mutually-exclusive alternatives, its more accurate and more objective to think of a spectrum of alternatives from heavily plan-driven at one extreme to heavily adaptive at the other extreme

Agile Project Management for Executives

I have just released a new online training course called “Agile Project Management for Executives”. (It is actually an existing course that has been significantly redesigned)

Agile Project Management for Executives

The Agile Bandwagon

In many areas, “Agile” is becoming a hot new buzz word and everyone wants to jump on the “Agile bandwagon” without necessarily fully understanding why they’re getting into it and exactly what they expect to get out of it. Many companies also make the mistake of assuming that whatever is good for the development process is good for the business as a whole and that is not necessarily the case.

Agile Bandwagon

Course Summary

While Agile has huge potential benefits for a business, it’s important to not get carried away with some of the hype that exists about Agile and develop an objective understanding of its benefits and limitations to know how and when to apply it successfully. The right approach is to not necessarily to just implement Agile for the sake of becoming Agile; but figure out how it’s going to help your business and what problems it will solve. The typical questions and challenges this poses for business managers and executives are:

  • How do I reconcile an Agile development approach with my existing business management and project management processes?
  • Do I need to unravel all of my existing management processes in order to adopt an Agile development approach?

This course will help you answer those questions and includes assessment tools and planning tools to help you develop a very effective Agile Project Management approach that is very well-aligned with your business.

Intended Audience

There are three potential audiences for this course:

  1. Senior-level Executives – The first audience is senior-level executives who are interested in making their business more agile and want to develop a well-integrated approach to fit an Agile development process to their business
  2. Business Sponsors – The next audience is Business Sponsors of an Agile initiative who want to learn more about Agile Project Management in order to provide more effective leadership for the initiatives that they are responsible for
  3. Product Owners – The final audience is for Agile Product Owners who need to better understand how to effectively perform the Agile Product Owner role.  The Product Owner role in Agile is not well-understood and most of the people who might be selected to perform that role are not well-prepared for what it requires

Why Is This Course Unique and Important?

For many years, Agile has been treated as a development process but in recent years it has become apparent that the implementation of Agile as a well-integrated, enterprise-level business strategy is not well-understood.

A lot of the Agile training that exists today is very focused on implementing Agile as a development process and on the “mechanics” of how to do Scrum. There is a relatively weak focus on Agile from a business perspective. For example, when I got my CSPO (Certified Scrum Product Owner) certification more than five years ago, the course was heavily focused on the “mechanics” of how to do Scrum and didn’t really directly address the role of the Product Owner as a business decision-maker at all.

  • This course is not a sales-pitch for Agile – it recognizes that there is not a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people seem to think and it objectively presents Agile and traditional plan-driven project management approaches as complementary to each other rather than competitive.
  • This course is not a superficial seminar on how to implement Agile. It is a very substantive, university-level course that is over four hours long that provides a very in-depth understanding of Agile from a business perspective

This course is also designed to complement all of my Agile Project Management courses. The idea being that implementation of Agile at an enterprise-level requires a collaborative partnership between a business executive and a senior-level Agile Project Manager based on a mutual understanding of how an Agile approach might apply to their business.

Special Discount Offer

I’m anxious to have a limited number of people review this course and provide feedback and inputs. For that reason, I am offering a special discount offer to a limited number of people who are willing to to be a “beta tester” for this new course for only $10 (Normal list price is $195). If you’re interested in beta testing the course and providing feedback, please click on the link below:

Register for course for only $10

This is a very limited offer. Please don’t register for this course unless you are sincerely interested in learning and providing feedback and inputs.

What Are the Critical Skills of a Product Owner?

I recently participated in an online discussion on the question of “What are the Critical Skills of a Product Owner?” I think this is a very good question because the role of a Product Owner is not very well understood and the actual role that a Product Owner might play can vary significantly in the real-world depending on the nature of the company’s business and the scope and complexity of the projects that the Product Owner is responsible for.

What are the Critical Skills of the Product Owner?
Business chart.

What Does the Scrum Guide Say?

The role of the Product Owner is relatively well-defined in the Scrum Guide; however the Scrum Guide recognizes that how it is done might vary widely across organizations. Here’s what the Scrum Guide has to say:

“The Product Owner is responsible for maximizing the value of the product resulting from work of the Development Team. How this is done may vary widely across organizations, Scrum Teams, and individuals.

“The Product Owner is the sole person responsible for managing the Product Backlog. Product Backlog management includes:

  • Clearly expressing Product Backlog items;
  • Ordering the items in the Product Backlog to best achieve goals and missions;
  • Optimizing the value of the work the Development Team performs;
  • Ensuring that the Product Backlog is visible, transparent, and clear to all, and shows what the Scrum Team will work on next; and,
  • Ensuring the Development Team understands items in the Product Backlog to the level needed.

“The Product Owner may do the above work, or have the Development Team do it. However, the Product Owner remains accountable.”

Impact of the Company’s Business

The nature of the company’s business will have a very big impact on the role of the Product Owner. In this regard, there are two major types of companies that are most significant:

  • Product-oriented Companies
  • Project-oriented Companies

Product-oriented Companies

At one extreme are product-oriented companies that are in the business of selling products to external customers. In those companies, a Product Owner might have the full responsibilities of a Product Manager including profit-and-loss responsibility for the product being developed. 

In this world, the critical skills of a Product Owner are essentially the same as the skills of a good Product Manager

Project-oriented Companies

At another extreme, are companies that are not really in the product development business at all and the work is more project-oriented to develop projects for internal use inside the company. In that kind of environment, the role of a Product Owner is likely to be very different. What is typical in this role is the role of the Product Owner is somewhat of a combination of a “Business Analyst on steroids” and a “Project Manager on steroids”.

  • He/she has some attributes of a Business Analyst in representing the requirements of the business for the solution being developed but it is much more than an ordinary Business Analyst in that a Product Owner has decision-making responsibility on the requirements where a normal Business Analyst does not have that kind of decision-making authority. 
  • He/she also has some attributes of a Project Manager since he/she is responsible for the successful completion of the project but that responsibility is also much more than a normal Project Manager in that a Project Manager is normally only responsible for delivering defined requirements and doe not typically have responsibility for the overall business success of the project. 

In this world, the critical skills of a Product Owner are:

  • Some “Business Analyst” skills for succinctly and accurately defining requirements but also have the domain knowledge and business knowledge to be a decision-maker to determine and prioritize what those requirements should be
  • Some “Project Management” skills to make good risk-based decisions on managing the project to make it successful from an overall business perspective (not simply meeting defined requirements)

Impact of the Scope and Complexity of the Project

Beyond the nature of the company’s business discussed above, the role of the Product Owner can also vary widely due to the scope and complexity of the project.

  • At one extreme, you might have a small, single-team Agile project with a very limited scope and complexity
  • At another extreme, you might have a much larger and more complex enterprise-level project with multiple teams

Naturally, that will also have a very big impact on the role of the Product Owner.

What Does a Product Owner Really Do?

So, what does a Product Owner really do? Here’s a very good article on that subject:

The Top 10 Responsibilities of a Product Owner

What Are the Advantages and Disadvantages of Agile/Scrum?

I have often been asked “What are the advantages and disadvantages of Agile/Scrum?” so I thought it would be useful to summarize what I believe are the most important advantages and disadvantages.

What are the advantages and disadvantages of Agile/Scrum?

Advantages of Agile/Scrum

  • Flexibility/Adaptivity – An Agile/Scrum approach is best-suited for a relatively uncertain environment where it is very difficult, if not impossible, to accurately define the requirements and design for the solution in detail prior to the start of the project.  In that kind of environment, flexibility and adaptivity are essential to further define and elaborate the requirements and design of the solution as the project is in progress
  • Creativity/Innovation – In the highly competitive environment that we live in today, no one wants to buy average, run-of-the-mill products.  People expect a higher level of excellence and that requires creativity and innovation.  An Agile/Scrum approach emphasizes creativity and innovation to maximize the business value of the solution over planning and control which tends to stifle creativity and innovation.
  • Time-to-Market – An Agile/Scrum approach typically results in faster time-to-market due to shorter startup times and an incremental development effort that will allow early delivery of at least a portion of the solution without waiting for the entire solution to be 100% complete
  • Lower Costs – An Agile/Scrum approach can lower the costs of a project in several ways:
    • Significantly reduced overhead resulting from reducing unnecessary documentation and control requirements
    • Higher productivity of the project team
    • Reduced “feature bloat” from using an incremental development effort and prioritizing the requirements it will become apparent when the project begins to reach a point of diminishing returns where the value of the features no longer exceeds the development cost
  • Improved Quality  – In an Agile/Scrum project, quality is an integral part of the development process rather than a sequential activity.  The developers know that quality is not “someone else’s responsibility”
  • Customer Satisfaction – An Agile/Scrum approach should result in higher customer satisfaction and more effective solutions because the customer is heavily involved in providing feedback and inputs throughout the development process
  • Employee Satisfaction – An Agile/Scrum approach should also result in higher employee satisfaction from all employees that are engaged in the effort because they are much more engaged to take responsibility for their own work as part of an empowered team

Disadvantages of Agile/Scrum

  • Training and Skill Required – An Agile/Scrum approach requires a considerable amount of training and skill to implement successfully.  Many project teams don’t fully understand the need for training and skill or don’t want to put the effort into it and attempt to do Agile/Scrum mechanically without fully understanding the principles behind it and that is typically not very effective
  • Organizational Transformation – An Agile/Scrum approach may also require some level of organizational transformation to make it successful.  It require the business users to work collaboratively with the development team in a spirit of trust and partnership.  That may require breaking down some organizational barriers that make that difficult or impossible to do
  • Scalability – It can be difficult to scale an Agile/Scrum approach to large, complex projects.  There are some models for doing that (Scrum-of-Scrums, LeSS, and SAFe are examples) but none of those are a cookbook solution that is easy to implement.
  • Integration with Project/Program Management – An Agile/Scrum approach may not be totally appropriate for projects that require a more plan-driven approach; however, there are many ways to create a hybrid approach that blends a traditional plan-driven approach and an Agile/Scrum approach in the right proportions to fit the situation

Overall Summary

Agile is not a “silver bullet” and it is not a solution to every problem you might have; but if it is applied intelligently in the right situations, it has huge advantages and the advantages can easily outweigh the disadvantages.

What Is Distributed Project Management? Why Does it Make Sense?

What is Distributed Project Management?  I’ve just finished a new section in one of my Agile Project Management courses on that subject.   Here’s a brief overview of what it is all about.

What is Distributed Project Management?

“Distributed Project Management” is a new concept that I think is very important to help people see the relationship of “project management” and Agile in a very fresh new perspective.  It has the potential to redefine many of the heavily-ingrained notions that we have about what “project management” is.

What Is Distributed Project Management?
There are a number of people in the Agile community that believe that “project management” is not consistent with Agile and I’ve even heard someone say that “project management is antithetical to Agile”.  That opinion is based on a very narrow and stereotypical view of what “project management is that has been well-ingrained into our minds for years.  I think it is time to take a broader and more modern view of what “project management” is.

How Is Project Management Implemented in an Agile Team?

There is actually a lot of “project management” going on in an Agile environment but many people won’t recognize it as “project management”:

  • It’s a different kind of “project management” – it goes beyond the traditional view of what “project management” is.  The traditional view is based heavily on planning and control to achieve predictability over project costs and schedules.  A more modern and broader view of “project management” is based on delivering business value.  That doesn’t mean that meeting cost and schedule goals is unimportant but achieving cost and schedule goals is only one component of business value and not necessarily the most important component.  Creativity and innovation to maximize the value of the solution can be at least equally important
  • The project management functions have been distributed among the team –  The functions that would normally be performed by someone called a “Project Manager” at the team level have been distributed among other members of the team.  As a result, you typically may not find anyone at the team level in an Agile project called a “Project Manager”.

In an Agile team, everyone on an Agile team has some kind of responsibility that might normally be performed by someone called a “Project Manager”:

  • The Product Owner comes closest to the overall responsibilities of a project manager and has overall responsibility for the success or failure of the project.  However, his/her role actually goes beyond a project management role and is more like a product manager role with overall business responsibility for the project.
  • Developers have responsibility for planning and managing their own work, reporting on progress, and integrating their work with others on the team
  • The Scrum Master is responsible for facilitating the work of the team, coaching the team in Agile practices and removing any obstacles that might be impeding the team’s progress

All of those things are functions that might normally be performed by a project manager in a traditional plan-driven environment.  That’s the essence of what “Distributed Project Management” is all about.

Why Does This Make Sense?

In the environment we live in today, solutions can be much more complex and the level of uncertainty can be much higher which makes it very difficult, if not impossible, to completely define a solution prior to the start of a project.  That environment requires a much more flexible and adaptive approach to further elaborate the requirements and the design of the solution as the project is in progress and that calls for a very different approach to project management.

Distributing the project management functions at the team level among the different Agile team roles provides a much more dynamic approach that is very well-suited to a the flexibility and adaptivity that is needed in a very uncertain environment.  Instead of centralized control where all decisions are made by a project manager, decision-making is more decentralized among the various roles on an Agile team and the team, as a whole, is self-organizing and empowered.

What’s the Impact on Project Managers?

All of this may be threatening to many project managers because, in many cases, it will likely eliminate the role of a project manager at the team level in an Agile project.  And, it also can be a significant adaptation for many project managers who have been used to being in control of a project.

At the team level, if a project manager is involved in an Agile project at all, he/she may play more of a coaching and mentoring role rather than a management and control role.  There is a much more significant role for project managers for larger and more complex projects that require multiple teams and for projects that require a hybrid approach such as Agile contracts.

There are some people in the project management profession who are in denial about recognizing this impact and/or are resisting this transformation.    For the project management profession to continue to thrive, we need to develop a broader vision of what “project management” is.

How Do You Choose the Best Methodology to Fit a Project?

I often see questions on “How do you choose the best methodology for a project?”

How do you choose the best methodology for a project?

What’s the Best Approach for Projects?

I don’t think that there is one single “best” approach that works for all projects.  A lot of people make the mistake of force-fitting a project to some standard methodology.

  • Some project managers will try to use a traditional plan-driven methodology for a project (what many people loosely call “Waterfall”) because it is the only thing that they know and it has been so widely-accepted as the way to do project management
  • Many other people have the misconception that there is a binary and mutually-exclusive choice between “Agile” and “Waterfall” and will attempt to force-fit a project to one of those extremes

The “best” approach is to go in the other direction and fit the methodology to the nature of the project.  That takes more skill but it definitely can be done.  You should also recognize that there is not a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people seem to think.  There is a whole spectrum of different approaches ranging from heavily plan-driven at one extreme to heavily adaptive (or Agile) at the other extreme.

Range of Agility

How Do You Choose the Best Approach?

There are a number of factors that influence the selection of the best approach for a particular project.

    1. Level of Uncertainty – Probably the most significant factor in choosing an approach is the level of uncertainty in the project.  A project with a high level of uncertainty would be best-suited for a more adaptive (Agile) approach.  Attempting to force-fit such a project to a traditional plan-driven project management approach could be disastrous.
      • It would force you to make a lot of assumptions to try to resolve the uncertainty; and, in many cases, those assumptions may be wrong and require a lot of re-planning and possible re-work
      • The emphasis on planning and control in a traditional plan-driven project can create an environment that is not conducive to changes which will make it difficult to maximize the value of the solution in an uncertain environment
      • In today’s competitive environment, creativity and innovation can be very important to create very competitive business solutions and an approach with a heavy emphasis on planning and control can stifle creativity and innovation
    2. Customer Relationship – Managing customer expectations is probably one of the most critical aspects of any project.  If the results of a project are not consistent with customer expectations, the project will likely not be viewed as successful no matter how good you think it is.  The nature of the customer relationship can range from:
      • A contractual style of relationship where there are very definite and well-defined customer expectations that must be met.  In this type of relationship, the customer may not take any responsibility for the success of the project.  The customer defines what his/her expectations are and expects the project team to do whatever is necessary to meet those expectations with limited participation by the customer.
      • A collaborative style of relationship where there is a shared responsibility for the success of the project between the project team and the customer and the customer takes an active role in helping to define the direction of the project as it is in progress

      Naturally,  the contractual style of relationship is well-suited to a project with a relatively low level of uncertainty where it is possible to define the customer’s requirements in some level of detail prior to the start of the project.  It would be much more difficult to make a contractual-style relationship work in a project with a high level of uncertainty.

      Of course, the customer has to be amenable to whatever type of relationship you choose.  If the project has a high level of uncertainty, that would lean towards more of a collaborative relationship but the customer has to be open to that kind of relationship for it to be successful.  This is a big problem in many companies where it is difficult to break down organizational boundaries between organizations and establish truly collaborative relationships based on a spirit of shared responsibility, trust, and partnership.

    3. Project Team Capabilities – The final major factor in selecting a project approach is, of course, the capabilities of the project team.    An Agile approach requires a lot of training and skill and a hybrid Agile approach  can require even more training and skill.  Naturally, it does not make any sense to choose an approach that the team is not capable of implementing.

Why Is This Important?

There are two major factors that require us to broaden the way we think about “project management” today:

  • Solutions tend to be much more complex and the level of uncertainty is much higher
  • Competitive pressures frequently require much higher levels of creativity and innovation

For those reasons, force-fitting all projects to a standardized plan-driven approach that is oriented around planning and control to achieve predictability over project costs and schedules is not necessarily the best approach for all projects.

A project should be focused on producing value for the customer, meeting cost and schedule goals is only one component of value, and it may not be the most important component of value to the customer.  There have been many projects that have met their cost and schedule goals but failed to deliver an acceptable level of business value.  “Value” can be a very elusive goal but, in any case, it’s what the customer thinks that “value” is that counts.

Why is Change Management important for an Agile transformation?

Why is Change Management important for an Agile transformation? Implementing an Agile transformation can require a big shift in thinking and possibly also require changing a well-established corporate culture for many companies.  In general many companies need to move from an excessive emphasis on planning and control to a more flexible and adaptive approach with more emphasis on creativity and innovation in a very uncertain environment.  That requires:

  • Developing a spirit of trust and partnership among all parts of the organization at every level
  • Breaking down organizational barriers that might prevent a collaborative, cross-functional approach
  • Developing a clear, unifying vision for the organization that creates a common purpose that is well-aligned with the company’s business objectives

Those are not easy things to accomplish and creating lasting organizational change is difficult.  People resist change and will naturally revert back to the old, comfortable way of doing things if the change management effort is not successful.

Why is Change Management Important?

Kotter’s Change Management Principles

The best source on this is John Kotter’s book “Leading Change”. Kotter identifies eight errors that companies make in change management initiatives:

    1. “Allowing too Much Complacency” – This probably could be better stated…it isn’t necessarily a matter of “allowing too much complacency”. The real issue is that if there isn’t a feeling of urgency for making a significant change, it will be difficult to get support for making the change.
    2. “Failing to Create a Sufficiently Powerful Guiding Coalition” – Overall leadership is extremely important – all the key leaders have to be on board with making a broad-based, cross-functional change and providing the guiding leadership to make it happen.
    3. “Underestimating the Power of Vision” – A clear vision is needed for what the organization will look like after the change is complete. If there is no vision, it will be difficult to guide the change in the right direction.
    4. Under-communicating the Vision by a Factor of 10 (or 100 or Even 1,000)”. Major change is usually impossible unless most employees are willing to help, often to the point of making short-term sacrifices. But people will not make sacrifices, even if they are unhappy with the status quo, unless they think the potential benefits of change are attractive and unless they really believe that a transformation is possible.”
    5. “Permitting Obstacles to Block the New Vision” – The important points that Kotter brings up are:
      • Implementation of any kind of major change requires action from a large number of people
      • New initiatives fail when employees, even though they embrace a new vision, feel dis empowered by huge obstacles in their paths
      • Occasionally, roadblocks are only in peoples’ heads – In many cases, the blockers are very real
    6. “Failing to Create Short-Term Wins” Many times the best approach is to not take on too much at once but focus on some short-term wins that show progress.
    7. “Declaring Victory Too Soon”
      • People can be tempted to declare victory in a major change effort with the first major performance improvement
      • While celebrating a win is fine, any suggestion that the job is mostly done is generally a terrible mistake
      • Until changes sink down into the company culture(3-10 years) new approaches are fragile and subject to regression
    8. “Neglecting to Anchor Changes Firmly in the Corporate Culture” – Culture and values are extremely important – if people are simply mechanically implementing the change without any real change in corporate culture and values to support it, the change may be somewhat fragile and not long-lasting.

Three Most Critical Principles

All eight of the issues that Kotter has defined are important, but from my experience, there are three that are most critical that most frequently derail an Agile transformation.  These are:

  1. No “Burning Platform” – This is based on Kotter’s first error “Not Creating a Sense of Urgency”.  There needs to be something that creates some urgency about making a change because maintaining the current situation is very painful and untenable.The phrase “burning platform” originated with a fire on an oil-drilling platform off the coast of the United Kingdom some years ago.  The phrase can have several contexts –
    • One context was that there were serious safety issues with the platform that were not addressed until the platform caught fire because there didn’t seem to be a sufficiently urgent need to address them.
    • Another context was that once the platform was on fire, there was an urgent need for the employees to do something to save their own lives – staying on the platform at that point was an untenable situation.
  2. No Vision for the Future – This is based on Kotter’s third error “Underestimating the Power of Vision”.  Taking the time to define a vision for an Agile transformation is extremely important – many people make the mistake of using a one-size-fits-all approach to force-fit a company to some kind of “textbook” model.The best approach is to fit the model to the company’s business and a pure, top-to-bottom Agile approach may or may not be the best fit.  It may require more of a hybrid approach to achieve that kind of alignment, at least in the short-term.
  3. Failing to Show Short-Term Progress – This is based on Kotter’s sixth error “Failing to create short-term wins”.  There are always naysayers and skeptics who will remain on the sidelines waiting to see if the change is likely to be successful before they jump on the bandwagon.  That is why it is important to get started and demonstrate progress as quickly as possible.  From an Agile perspective, it is important to set clearly-defined and measurable goals that show progress and regularly communicate that progress to everyone involved.

Do Agile Methods of Management Make Any Sense?

I recently responded to a question on Quora that asked “Do Agile Methods of Management Make Any Sense?”. I thought it was an interesting question so I decided to make a blog post out of my response.Do Agile methods of management make any sense?

The “Program Du Jour” Effect

Agile has a similar pattern to many other new management approaches that have come before it.  This is what I call “The Program Du Jour Effect”:

  • Initially, there’s a lot of “hype” behind any new management approach that is fueled by the many consultants and trainers who have jumped on the bandwagon and want to sell their services
  • People start to see it as a universal solution for any problem you might have and all other previous management approaches are regarded as obsolete and are no longer relevant at all
  • The implementation becomes very superficial because people are looking for quick results without necessarily putting the effort that might be needed into really understanding it and making it work

I’ve Seen This Pattern Before

I’ve seen a similar pattern with Business Process Reengineering and Six Sigma. When I published my first book on Business Excellence in 2003, Six Sigma was very hot and everyone wanted to jump on the Six Sigma bandwagon. When I did the research for my book, I found that:

  • A number of companies did Six Sigma superficially, there was a lot of “hoopla” about it (Black Belts, Green Belts, etc.). That kind of effort wasn’t very effective and didn’t last long – they quickly threw out Six Sigma when it didn’t deliver magical results as quickly as expected and waited for the next management fad to come along.
  • In other companies, Six SIgma was simply a tool (and only one of many tools) that might have been used for process improvement, they might not have even called it “Six Sigma”, and it was so well-integrated into the way they managed their company that it might not have even been obvious that it was Six Sigma at all

Here’s a quote that I really like that I used in my first book on Business Excellence:

“Americans are our own worst enemy when it comes to new business concepts. We love novelty and newness. We become so enamored with new ideas, we burn through them the way a child rips through toys on Christmas morning – squeals of delight, followed by three or four minutes of interest, then onto the next plaything. That is our pattern with new management techniques, too.”
(Barry Sheehy, Hyler Bracey, & Rick Frazier, Winning the Race for Value, American Management Association, 1996  Page:  104)

Do Agile Methods of Management Make Any Sense?

If you understand the essence of what Agile is all about and use it appropriately in the context of how it is intended to be used, it has enormous value and makes a lot of sense. The essence of Agile is in being flexible and adaptive in a highly-uncertain environment and emphasizing an appropriate level of creativity and innovation in new product development and not simply emphasizing planning and control.

The alternative is to force-fit everything to a heavily plan-driven approach to management that heavily emphasizes planning and control only. That approach is clearly appropriate in some situations and not in others.  The key message is that:

  • We live in a complex world today with lots of uncertainty and a “one-size-fits-all” approach to project management no longer works well
  • You have to fit the project management approach to the nature of the project and that takes a lot more skill.  It is not simply a matter of picking up a standardized project management template with fill-in-the-blanks documentation

It is also not a simple matter of a binary and mutually-exclusive choice between Agile and Waterfall as many people seem to think.  It is  a matter of learning to blend those two approaches (and others) in the right proportions to fit a given situation.



Which Process Model is Best to Use In Software Product Development?

I recently saw a question on an internet discussion forum that asked “Which Process Model is Best to Use In Software Product Development?”.

Which Process Model is Best to Use In Software Product Development?

I’ve seen that kind of question come up over-and-over so it seemed like a good time to write a blog post to address it.

What’s Wrong With That Question?

There are at least a couple of things inherently wrong with that question to begin with:

    1. It implies that there is one process model that will work “best” in any possible software product development project

      If we just figure out which one is best, we can simply use that model for any software product development project you might imagine.  Wouldn’t that be wonderful if it were true?

      Unfortunately, software product development is not quite that simple.  You have to fit the methodology to the nature of the project  – you can’t just force-fit all software products to some universal development process.   And, that requires some good judgement, knowledge, and skill to know how to pick an appropriate model and adapt it as needed to fit a given project.  It isn’t as simple as having one universal model that works for all projects with a “cookbook style” checklist of what you need to do to make it work.

    2. It also implies that there is some way to easily make a determination that one model is inherently better than another.

      How would you make that judgement?  What would make one model “better” than another?  How can you make that determination objectively?

      There are many different factors that you might consider to determine which model is best but it certainly isn’t an easy determination to make.  It’s also difficult to create a side-by-side comparison of two models on the same project to really compare the performance objectively.

      That hasn’t stopped people from making a judgement that “Agile is better than Waterfall”. I’ve even seen studies that say that “Agile projects are statistically 2X more likely to succeed, and 1/3 less likely to fail than waterfall projects”.

      Reference to Standish Group 2018 Survey

      How do you make that determination?  How was “success” defined?  Was “success” really  defined the same way in both projects?

Which Process Model is Best to Use In Software Product Development?

So, what’s the real answer? Saying that “Agile is better than Waterfall” is like saying “A car is better than a boat”.  They both have advantages and disadvantages depending on the environment you’re in:

  • An Agile model tends to work best in projects where:
    • There is a relatively high level of uncertainty where a flexible and adaptive approach is needed to resolve the uncertainty as the project is in progress
    • Creativity and innovation are needed to maximize the business value of the solution
  • A traditional plan-driven model (what many people loosely call “Waterfall”)  tends to work best where:
    • There is a relatively low level of uncertainty, the solution is well-defined, and some level of planning and control are needed to maximize predictability of costs and schedules
    • The organization is not really  well-prepared to implement an Agile approach and/or the project team is not trained in Agile

The two factors that are making Agile so attractive in today’s environment are:

  1. A higher level of uncertainty in the environment
  2. Competitive pressures require more creativity and innnovation

However, that does not mean that there is no need for a traditional plan-driven model at all.  It would also be a mistake to think that there is a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people seem to think.

You have to fit the approach to the nature of the project and to the business environment.

Fit the approach to the project

There are many ways to create a hybrid model that blends Agile and traditional plan-driven project management principles and practices in the right proportions to fit a given situation.  And, sometimes it is necessary to do that to fit the model to the nature of the project rather than trying to force-fit a project to some arbitrary, predefined model.

What Are the Key Attributes of Post-Agile Development? What Problems of Traditional Agile Does It Try To Solve?

I recently responded to a post with the question: “What are the key attributes of post-agile development? What problems of traditional agile does it try to solve?”  I think people are looking for a new “silver bullet”

>What is Post-Agile Development?

What is Post-Agile Development?

I don’t really believe that there is such a thing as “post-Agile development”.  That implies that Agile will become obsolete and something really different will replace it.  However, I do believe that the use of Agile methodologies will continue to evolve and mature.

Trends in Agile Development and Agile Project Management

Some of the trends that are evident to me are:

    1. Traditional plan-driven project management is beginning to converge with Agile. Agile started out as a revolution against traditional plan-driven project management practices (what many people loosely call “Waterfall”) and that pendulum is starting to swing back to the middle.  People are beginning to recognize that there isn’t really a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people seem to think. Rather than force-fitting a project to one of those extremes, a better solution is to fit the methodology to the nature of the problem which may require a blend of both approaches in the right proportions to fit the situation.
    2. Learning how to blend those approaches together requires understanding a broader range of methodologies (both plan-driven and Agile) and understanding them at a deeper level.Many people today do Agile somewhat mechanically “by the book” without really understanding the principles behind it. That results in a somewhat rigid approach to how to apply Agile which is exactly the opposite of the adaptive approach that is intended for Agile.
    3. A very big trend is that many companies and people are attempting to scaleAgile to larger and more complex, enterprise-level projects and that will accelerate both of the above trends.Agile was originally designed around small, simple, single-team projects and it can be difficult to scale without thinking about how to blend it with typical enterprise-level management practices including project/program management, project/product portfolio management, and overall business management.
    4. In some cases, an attempt has been made to force a whole company to be agile in order to adopt an Agile development approach and that just isn’t completely realistic or desirable in some cases. Becoming Agile is not necessarily a goal in itself – it has to be applied in the context of the company’s most critical business objectives – what problem will it solve and how will it solve it?

The “Program Du Jour” Effect

I’ve seen this pattern before – I call it the “Program Du Jour” effect.  Everyone is looking for a silver bullet that will magically solve all of their problems.  When one thing doesn’t work, they toss it out and try something else.

I saw that with Six Sigma in the early 2000’s.  When Six Sigma was hot, everything that came before it was obsolete and no longer relevant.  And, some companies made a fairly superficial attempt to apply Six Sigma, decided it didn’t work, and tossed it out waiting for the next “silver bullet” to come along.  Here’s a quote from my original book on Agile Project Management on that subject:

“Agile methodologies have the potential to have an enormous impact; however, like many other new and hot methodologies:”

  • “Consultants tend to swarm all over them and sell it as a cure for almost anything that ails you, and
  • Many companies and managers want to jump on this bandwagon and that further builds the hysteria in the market.”

“The result of this can be:

  • “Jumping into agile looking for a ‘quick fix’ without fully realizing that it takes a significant commitment to make it successful; resulting in superficial implementations that are likely to fail
  • Attempting to implement agile methodologies in a business environment or organizational culture that is inconsistent with an agile approach
  • Attempting to use agile methodologies for projects that they are inappropriate for or failing to blend a sufficient level of agility with the level of control that is needed”

If Agile is done correctly, it is a very broad, flexible, and adaptive approach for solving a number of different problems; however, it is not a “silver bullet” – it won’t solve any problem you might have.  I hope we’re getting past the “silver bullet” phase and starting to put the right amount of thought into applying Agile intelligently and continuously improving and refining it.