Category Archives: Understanding Agile

What is ‘Agile’ and Why Is It Important to Project Managers?

I think a lot of people are confused about “What is ‘Agile'” and the importance of Agile to project managers.  The word “Agile” has many different connotations and many project managers think that it is something that only applies to software development and doesn’t apply to them at all.

What is ‘Agile’?

What is 'Agile'
“Agile” means a lot of different things to different people:

  • To some people it means just developing software faster,
  • To some people, it means creating a more people-oriented project environment,
  • To others, it means making the project management process a lot more efficient by streamlining the whole process and eliminating unnecessary documentation

Those are only a few different connotations – there are many, many more. There are also many more stereotypes, myths, and misconceptions about what Agile is. All of those things are potential outcomes of an Agile process but that’s not the fundamental essence of what an Agile process is all about in my opinion.  The fundamental essence of an Agile process is adaptivity.

What’s Wrong With the Typical Project Management Approach?

Many project management processes, as we know them today, were designed around what is called a “traditional plan-driven project management” model (what many people loosely call “Waterfall“).  In this model, achieving predictability over the outcome of a project and the costs and schedule associated with achieving that outcome is very important.  Therefore, it is also very important to have clearly defined requirements as well as an adequate level of planning to be able to somewhat accurately predict the outcome, costs, and schedule of the project.

That’s the predominant way that project management has worked since the 1950’s and 1960’s. A project was considered successful if it delivered what the requirements for the project called for within the defined budget and schedule. That kind of predictability can be important – for large investments, it allows a company to:

  • Attempt to calculate with some level of certainty what the return on their investment is likely to be from a project
  • Make a go/no-go decision as to whether the project should be funded or not based on that information.

The primary problem with that approach is that it requires developing a fairly detailed plan for the project upfront and that is very difficult, if not impossible to do in projects with a very high level of uncertainty.

Why Is Adaptivity Important?

We live in a different world today from the world that existed in the 1950’s and 1960’s when formalized project management approaches were first defined.  There is a much higher level of uncertainty:

  • Technology is rapidly changing,
  • Solutions are much more complex, and
  • The business environment that we operate in is dynamic and constantly changing.

In that kind  of environment, if you attempt to develop a detailed plan for a project with a lot of uncertainty upfront, it will typically require you to make a lot of assumptions, many times those assumptions will be wrong,  and that may require significant re-planning and possible rework later.

Rather than force-fitting a project that has a high level of uncertainty to a traditional plan-driven approach, it’s much better to fit the methodology to the nature of the project and that’s where a more adaptive approach really makes sense.  It doesn’t mean that you don’t do any upfront planning; it means that you use a level of planning that is appropriate to the level of uncertainty in the project:

Traditional Plan-driven Approach Adaptive (Agile) Approach
Atempt to develop a detailed set of requirements and a detailed project plan for the project upfront Limit the amount of upfront planning based on the level of uncertainty in the project and use a “rolling-wave” planning approach to further define the requirements and plan for the project as the project is in place

What’s an Example of a Project Requiring an Adaptive Approach?

I use an example in my Agile Project Management training that is a somewhat extreme but it gets the point across. The example I use is:

Suppose you were given the task to find a cure for cancer and you were asked to outline what the solution will be, how long it will take to develop it, and what the total cost of the research will be to develop the solution.

In that situation, it would be ridiculous to attempt to develop a detailed project plan with accurate cost and schedule estimates – there is just way too many uncertainties to resolve. So what would you do? Give up and do nothing? That doesn’t make sense either.

In that situation, we do know some things about finding a cure for cancer based on years of research that have gone into that area but there are still way too many unknowns to develop a detailed project plan for a solution. What you would do is take advantage of what is known as much as possible and then take an iterative, trial-and-error approach to find a solution.

That’s the way Edison invented the light bulb…here’s a quote from Edison on that subject:

“I speak without exaggeration when I say that I have constructed three thousand different theories in connection with the electric light, each one of them reasonable and apparently to be true. Yet only in two cases did my experiments prove the truth of my theory. My chief difficulty, as perhaps you know, was in constructing the carbon filament, the incandescence of which is the source of the light.” (1890 Interview in Harper’s Magazine)

In a 1910 autobiography of Edison, Edison’s friend and associate Walter S. Mallory is quoted as asking “Isn’t it a shame that with the tremendous amount of work you have done you haven’t been able to get any results?”  The book goes on to say that “Edison turned on him like a flash, and with a smile replied: Results! Why, man, I have gotten lots of results! I know several thousand things that won’t work!”

What Makes This Kind of Project Different?

There are two things that make this kind of project fundamentally different:

  1. The level of uncertainty is very high and makes it impractical or impossible to develop a detailed plan for the project upfront
  2. Creativity and innovation required for finding a good solution are far more important than predictability

How Does an Agile Approach Solve This Problem

An Agile process is built on an “Empirical” Process Control model. The word “empirical” means “based on observation”. In the context of an Agile development process, it means that during the course of a project, both the product as well as the process to produce the product are continuously refined as the project is in progress to produce the right product and to optimize the value of the product being produced.

Empirical Process Control Model

Why Is This Important to Project Managers?

You might ask “Why is this important to project managers?” Isn’t Agile something that only applies to software development? (That’s a common misconception) The truth is that all projects have some level of uncertainty associated with them and if you try to force-fit all projects to a traditional plan-driven project management approach, its just not going to work in many cases. Imagine, for example, trying to develop the next generation of the iPhone or any other new and innovative product. In that kind of project, creativity and innovation is just as important, if not more important, than predictability.

In this new environment, a project manager who only knows how to do a traditional plan-driven approach to project management will be at a serious disadvantage. What makes this even more difficult is that this is not a binary and mutually-exclusive choice between “Agile” and “Waterfall” as many people seem to think. It’s a matter of figuring out how to blend a traditional plan-driven approach with an adaptive (Agile) approach in the right proportions to fit a given situation.

That requires a lot of skill to do that but it definitely can be done and that is exactly what the Agile Project Management Training I’ve developed is designed to address.

Are Agile and Six Sigma Complementary to Each Other?

I recently responded to a question on an online discussion that asked “Are there companies that use Agile and Six Sigma?”.  This raises an interesting question of “Are Agile and Six Sigma really complementary to each other?”.

 

Are Agile and Six Sigma Really Complementary to Each Other?

The objectives behind numerous approaches such as:

  • Agile and Six SIgma,
  • Agile and Lean, and
  • Agile and Waterfall

are very different and if you pursued these approaches individually and independently of each other, the objectives of each approach are probably somewhat contradictory.  However, all of these approaches can be blended together in the right proportions if they are implemented intelligently in the right context.  That requires a lot more skill and it requires a different kind of thinking to see them as complementary rather than competitive approaches.

The Fundamental Problem

There is a significant fundamental problem that must be overcome to see all of these approaches in a different perspective:

  1. Companies and individuals get enamored with a methodology like Agile or Six Sigma and see it as a “silver bullet” solution to any problem that they might have
  2. They attempt to mechanically force-fit their business to one of those methodologies without fully understanding the principles behind it

An Example With Six Sigma

When I published my first book a long time ago in 2003, Six Sigma was very hot at that time and everyone wanted to “jump on the Six Sigma bandwagon”.  At that time, I researched a number of companies that were doing Six Sigma and other process improvement methodologies and what I saw was this:

  1. Successful Companies – In companies that seemed to do Six Sigma successfully:
    • It wasn’t even obvious that it was Six Sigma and they might not have even called it “Six Sigma”,
    • The implementation wasn’t limited to Six Sigma, they understood the principles behind Six Sigma, and might have blended Six Sigma with other process improvement methodologies, and
    • It was so well-integrated with their business, that it was just a tool that was part of their business rather than a program that was superimposed on their business
  2. Less Successful Companies – In other companies, I saw a much more superficial implementation of Six Sigma that didn’t last in many cases:
    • There was a lot of emphasis on the “mechanics” of doing Six Sigma,
    • There was a lot of “hoopla” about the ceremonies associated with Six Sigma – green belts, black belts, etc., and
    • They openly advertised that they were using Six Sigma to promote themselves

Does that sound familiar?  I think a similar thing is going on with Agile today.

The Key Factor for Success

What I learned from that some of the key factor for success are:

  • Don’t get overly enamored with any methodology (Six Sigma or anything else) and see it as a “silver bullet” for any problem you might have.  Be objective and recognize that any methodology has advantages and limitations depending on the problem you’re trying to solve,
  • Adapt the methodology to fit the problem and the business environment rather than force-fitting your business to some predefined methodology, and
  • Go beyond simply doing a mechanical implementation of any methodology (Agile or Six Sigma) and understand the principles behind it at a deeper level

Are Agile and Six Sigma Really Complementary To Each Other?

Any company who takes that kind of superficial, mechanical approach is going to have difficulty seeing how seemingly competitive and contradictory approaches might actually be complementary to each other. Here are a few examples:

  1. Agile and Six Sigma – On the surface, Six Sigma and Agile would tend to pull you in different directions:
    • Agile emphasizes creativity and innovation as well as flexibility and adaptivity to maximize the business value of the solution
    • Six Sigma emphasizes process standardization and control of a process to minimize process variation
  2. Agile and “Waterfall” – You could make a similar comparison about Agile and Waterfall:
    • Agile emphasizes flexibility and adaptivity and encourages changes as the project is in progress to maximize the business value of the overall solution
    • Plan-driven approaches (what many people loosely call “Waterfall”) emphasize planning and control and discourage changes as the project is in progress to maximize predictability
  3. Lean and Agile – You could also make a similar comparison about Lean and Agile:
    • Lean emphasizes standardizing processes and removing waste from a process to improve process efficiency
    • Agile requires a process to be flexible and adaptive

How Can These Approaches Be Complementary Rather Than Competitive?

How can all of these different approaches that have different objectives that seem to be contradictory to each other actually be complementary to each other rather than competitive? For example, there are many people who see “Agile” and “Waterfall” as binary and mutually-exclusive choices and don’t see any way that the two approaches could be combined.

The key to seeing these approaches as complementary rather than competitive is to understand the fundamental principles behind the approach at a deeper level rather than getting lost in the “mechanics” of the approach. For example:

  • “Waterfall” – The word “Waterfall” has some well-ingrained connotations associated with it that are not necessarily totally accurate.  It invokes an image of a very bureaucratic process with phases that are rigidly-controlled so that you don’t exit that phase until you have completed all of the onerous documentation requirements for exiting that phase.   That process that was originally documented by Winston Royce in the 1970’s and is rarely used in that form in practice today; however, the word “Waterfall” continues to be used very loosely for any process that is not Agile.
     

    The fundamental essence of what people call “Waterfall” is that it is a “plan-driven” approach – that is why I prefer to call it “plan-driven” instead of “Waterfall”.  A “plan-driven” approach does not necessarily have to have a rigid and inflexible plan and it also doesn’t necessarily have to have many of the other attributes of the original Waterfall approach such as excessive reliance on documentation and phases that have controlled phase transitions.

  • “Agile” – The word “Agile” has also taken on some very heavily-ingrained connotations in practice today.  When you say the word “Agile” many people associate it with Scrum and think that you can’t be “Agile” unless you are really following all the rituals and ceremonies associated with Scrum religiously.
     

    The fundamental essence of “Agile” is being flexible and adaptive to maximize the value of the solution that is being produced.  I prefer to use the word “adaptive” rather than “Agile” for that reason.  The word “adaptive” is a more general term that does not carry some of the heavily-ingrained connotations associated with the word “Agile”.

  •  

  • Lean – The essence of Lean is the reduction of waste in a process and there is absolutely nothing wrong with that as long as people don’t become obsessive about reducing waste at the expense of other objectives.  For example, in an Agile process, it is impossible to completely remove waste; and, at the same time, attempt to maximize the flexible and adaptive which is the essence of being Agile.  However, there is nothing wrong with trying to reduce waste in an Agile process as long as it is done intelligently and in the right balance with other objectives.
  •  

  • Six Sigma – The essence of Six Sigma is attempting to standardize processes and reduce variation in processes.  If you became obsessive about pursuing that goal, it would also not be very consistent with being Agile; however, there is absolutely nothing wrong with attempting to standardize processes to some extent as long as it is also done intelligently and in balance with other objectives.

The importance of “Systems Thinking”

A fundamental skill for doing this successfully is “Systems Thinking”. Here’s a definition of “Systems Thinking”:

“Traditional analysis focuses on separating the individual pieces of what is being studied; in fact, the word ‘analysis’ actually comes from the root meaning ‘to break into constituent parts’. Systems thinking, in contrast, focuses on how the thing being studied interacts with the other constituents of the system – a set of elements that interact to produce behavior – of which it is a part. This means that instead of isolating smaller and smaller parts of the system being studied, systems thinking works by expanding its view to take into account larger and larger numbers of interactions as an issue is being studied.”

Overview of Systems Thinking – http://www.thinking.net/Systems_Thinking/OverviewSTarticle.pdf

 

What is Systems Thinking?

This kind of thinking is essential for seeing these seemingly contradictory approaches in a much broader context of how these objectives can interact in complementary ways rather than being competitive.

Overall Summary

It is very possible to blend together different approaches that are seemingly in conflict with each other as long as it is done intelligently. It requires:

  • Understanding the fundamental principles behind each approach rather than getting lost in the mechanics,
  • Using a systems thinking” approach to see how these seemingly contradictory approaches might actually be complementary to each other rather than competitive, and
  • Learning to fit the methodology to the problem rather than force-fitting a problem to any given methodology or combination of methodologies

This is exactly the approach behind the Agile Project Management online training courses I’ve developed.

What is the Real Essence of Agile?

It’s apparent to me that a lot of people have gotten so heavily focused on the mechanics of how Agile is implemented that they’ve lost sight of the big picture of what “Agile” is really all about.  The term “Agile” has taken on a number of different meanings today that are largely based on how it is implemented.  For many people, “Agile” has become synonymous with Scrum and if you’re not doing Scrum and doing it “by the book”, you’re not really Agile at all.  I think it is useful to step back and take a look at “What is the real essence of Agile”?

What is the Real Essence of Agile?

What is the Real Essence of Agile?

The real essence of “Agile”, in my opinion, is that it puts an emphasis on being adaptive to customer and business needs in order to maximize the value of the solution rather than following a rigidly-defined plan with an emphasis on managing costs and schedules of delivering the solution. For that reason, I like to use the terms “adaptive” and “plan-driven” rather than terms like “Agile” and “Waterfall”.

  • The terms “Agile” and “Waterfall” make it sound like you’re comparing two specific methodologies – one called “Agile” and one called “Waterfall” and that’s not really accurate.  “Agile” is not really a specific methodology or framework like Scrum; it is much broader than that – it is a way of thinking defined by the Agile Manifesto
  • The “Agile versus Waterfall” kind of thinking leads people to think that there is a binary and mutually-exclusive choice between those two approaches and that causes people  to try to force-fit a project to one of those extremes.  The right approach is to go in the other direction and fit the methodology to the nature of the problem and sometimes that requires a blending the two approaches in the right proportions to fit the problem

When Does Agile Work Best?

Agile works best in situations that have a high-level of uncertainty where it isn’t practical or possible to define the solution to the problem upfront.  In that kind of situation, it is much more effective to use an adaptive approach to incrementally and iteratively define the solution in more detail as the project is in progress rather than attempting to define detailed requirements for the solution upfront.

The example I like to use to illustrate this is finding a cure for cancer.   If you set out to define a project to find a cure for cancer, it would be ridiculous to try to define a detailed plan upfront; there’s just far too much uncertainty.  The only approach that is likely to work is an iterative, trial-and-error approach to find a solution.

Agile is based on what is called an “Empirical Process Control” model.  The word “Empirical” means “based on observation” and that means that in an Agile project, both the requirements for the solution AND the process for developing the solution are continuously refined as the project is in progress.

What’s the Alternative?

That kind of empirical process control model approach is naturally not the most efficient approach if there is a low level of uncertainty and it is possible to easily define detailed requirements for the solution prior to the start of the project. In that situation, a trial-and-error approach really isn’t necessary and a plan-driven approach is much more appropriate and much more efficient.

The example I like to use to illustrate this scenario is building a bridge across a river. If you were defining a project to construct a bridge across a river, it would be ridiculous to say “we’ll take an adaptive approach, build the first span of the bridge, and decide how we will build the rest of the bridge after we’ve completed the first span. That wouldn’t make any sense.

This kind of situation calls for a plan-driven approach that is based on a defined process control model. The key advantage of a defined process model is that it is predictable and repeatable and it is probably more efficient for projects with a low level of uncertainty. If you designed a process for building a bridge and it has proven successful once, the same process or a variation of that process is likely to work in another similar situation with somewhat predictable results.

What if it is Between Those Extremes?

Very few real world situations are as extreme and clear-cut as the ones I’ve used of finding a cure for cancer and building a bridge across a river. Most real-world situations fall somewhere between those two extremes. There is some level of uncertainty but it’s not complete uncertainty. You rarely start any project without at least some idea of what the solution is going to look like although the solution may be progressively refined in more detail as the project is in progress. There’s a range of approaches that look something like this:

Range of Agility

In this kind of situation, you have to tailor the approach to fit the nature of the project and one of the biggest factors to consider is the level of uncertainty associated with the solution. That requires more skill but it definitely can be done. It requires knowledge of a broader range of methodologies (both plan-driven and adaptive (Agile)) and it requires a deeper knowledge of the principles behind the methodologies in order to know how to tailor them to fit a given situation. You can’t just force-fit a situation to some predefined methodology (whatever it might be) and do it mechanically.

In my book on Agile Project Management, I use the analogy of a project manager as a “cook” versus a project manager as a “chef”. A “cook” knows how to prepare a few simple “recipes” by the book; a “chef” knows how to prepare a much broader range of recipes with more exotic ingredients and even knows how to improvise his/her own recipes when required. This clearly raises the bar for a lot of project managers – it is no longer sufficient in many situations to force-fit a project to some predefined approach (whatever it might be). You have to fit the approach to the nature of the problem and that’s exactly the challenge that my online Agile Project Management training is designed to address.

Understanding Agile at a Deeper Level

Understanding Agile at a Deeper Level is important. One of the criticisms I’ve heard often about Agile/Scrum is that people do it “mechanically” – sometimes, they rigidly and dogmatically implement Scrum “by the book”. That’s very ironic because it’s the opposite of what was intended by the Agile Manifesto (remember “Individuals and interactions over processes and tools”). That shouldn’t be surprising – you can get a Certified Scrum Master (CSM) certificate by sitting through a two-day course and many people never go beyond that level of training.

In my opinion, to develop a high-performance Agile/Scrum approach that is dynamic and adaptable to a broad range of situations, you have to go beyond doing it “mechanically by the book” and understand the principles and values behind it at a deeper level. This becomes particularly important when you try to scale Agile/Scrum to larger and more complex enterprise-level projects.

I’ve developed a new online training course to help fill this need and I’m offering this course at a discounted price of $10 for anyone who wants to take it during the month of June. Here’s a brief video summary of this new online training course:

Understanding Agile at a Deeper Level Video Summary

You can find more information on this course plus the discount coupon code on this blog site training page:

Understanding Agile at a Deeper Level Course Information

If you’re interested in certification, this course should be excellent preparation for the Professional Scrum Master (PSM) certification. I think the PSM certification is more rigorous than CSM and it recognizes that training and development should be an ongoing process beyond simply sitting through a one-time, two-day training course.

Adaptive versus Plan-driven Project Management

I’ve written several articles on the subject of why the comparison of “Agile versus Waterfall” that is so widely used is so inaccurate and misleading. A better way to view it is to think of adaptive versus plan-driven project management. Check it out here:

Learn the Truth About ‘Agile versus Waterfall’

There’s a lot of reasons why that comparison isn’t very meaningful and can be misleading – What is Agile? What is Waterfall? What are you comparing to what? Is it really a meaningful comparison? (I don’t think so)

A much more objective and meaningful way of comparing different methodologies is needed and I think the comparison of “Adaptive” versus “Plan-driven” is much better-suited for that purpose. Here’s how those terms are defined:

  • Plan-driven – “Plan-driven software development is a more formal specific approach to creating an application. Plan-driven methodologies all incorporate: repeatability and predictability, a defined incremental process, extensive documentation, up-front system architecture, detailed plans, process monitoring, controlling and education, risk management, verification and validation” (Source: http://en.wikiversity.org/wiki/Plan-driven_software_development
  • Adaptive – An “Adaptive” approach, on the other hand is characterized by having a less well-defined plan that is more adaptable to fit the situation as the project progresses. It’s more of a rolling-wave planning approach where the plan evolves as the project evolves.

This is not a binary, all-or-nothing choice between 100% plan-driven and 100% adaptive:

  • You don’t typically find many situations that have an absolutely rigid plan down to the smallest detail that is not subject to any kind of change and
  • You don’t typically find situations that are totally unplanned where you just start doing something without any kind of plan and make up the plan as you go.

Most situations lie somewhere between those two extremes – there is a continuum of different approaches from a heavily plan-driven approach at one extreme and a heavily adaptive approach at the other extreme with lots of alternatives in the middle. If you were to plot different approaches on this continuum, here’s a sketch of what it might look this:

Adaptive versus Plan-driven Project Management

Here’s some notes on this diagram:

  • This is not meant to be an exhaustive and comprehensive diagram – it is simply an illustration showing a few examples
  • Each of these methodologies can be used in a range of situations – for example, Scrum can be used in a range of situations that have varying levels of adaptivity; however, it would be difficult to make Scrum work in a heavily plan-driven approach
  • However, by putting a plan-driven “wrapper” around Scrum, you can create a hybrid approach that provides a plan-driven shell at the project level but retains most of the flexibility of Scrum at the team level
  • Kanban is generally more adaptive than Scrum because it is typically used in more reactive situations that are less planned (like a customer service process)

There is no “good” and “bad” judgment in characterizing a methodology as “adaptive” or “plan-driven”. Neither an adaptive methodology or a plan-driven methodology is inherently good or bad just because it is adaptive or plan-driven. The problem comes up when they are misused – for example, when you try to use a heavily plan-driven methodology like Waterfall for a situation that has a lot of uncertainty in it and calls for a more adaptive approach.

Why is this important? The traditional comparison of “Agile versus Waterfall” is almost meaningless and very misleading. It frequently is used judgmentally that “Waterfall” is bad and “Agile is good. That leads people to think of those two approaches as binary, mutually-exclusive choices; and, as a result, people many times tend to force-fit a project to one of those extremes. A much better approach, in my opinion, is to view these approaches from a more objective perspective and fit the approach to the nature of the problem based on the characteristics of the methodology and the problem.

Religious Fervor About Agile

There is a lot of religious fervor about Agile. I was reviewing one of Dean Leffingwell’s slide presentations and I particularly like a comment he made that “Agile is only a tool – it’s not a religion”. It inspired me to write this blog post.

There is a lot of religious fervor about Agile and there’s a lot of polarization between proponents of Agile and more traditional plan-driven project management approaches. There’s a lot of good reasons why Agile makes sense in many situations but some people seem to just do it mechanically – becoming Agile becomes a goal in itself without a real understanding of why it makes sense in a given situation.

Don Reinertsen has written a good book called “The Principles of Product Development Flow” which I think provides some valuable insight into the underlying principles of why Agile makes sense. Since the principles are general enough to apply to any product development effort (agile or plan-driven), it also provides an objective foundation for comparing Agile and alternative plan-driven approaches so that someone can make an intelligent and objective assessment regarding how these two seemingly competitive approaches.

The book goes a little too far in my view in developing a quantitative and mathematically sound basis for the principles in the book based on statistics. In the real world, I don’t think very many people would really apply that kind of rigor to analyzing these principles; but nonetheless, an understanding of the principles, at least at a high level, is very valuable and it does provide an objective foundation for understanding the benefits of Agile at a deeper level. I’ve summarized the eight principles here:

  1. Economics – Take an Economic View
  2. “Why do we want to change the product development process? The answer: To increase profits. As obvious as this might seem, this economic view of product development is surprisingly uncommon. Instead, product developers create a large number of proxy objectives: increase innovation, improve quality, conform to plan, shorten development cycles, eliminate waste, etc. Unfortunately, they rarely understand how these proxy objectives influence profits.”

    I’ve seen a number of instances where companies blindly pursue some of those “proxy objectives” such as “increasing innovation” without really understanding the economic impact. “Increasing innovation” should not be an end-in-itself and it reaches a point of diminishing returns at some point and it might begin to impact other proxy variables such as quality. For that reason, it is good to put things in an economic context. The economic context provides a framework for making intelligent decisions about how much to focus on these “proxy variables”.

    Here’s another example of how the economic context can provide a sound framework for decision-making: It is generally best to defer decisions on product features as long as possible; however, some decisions should be made early and should not be significantly deferred because if they have a significant economic impact.

  3. Queues – Actively Manage Queues
  4. “Queues matter because they are economically important, they are poorly managed and they have the potential to be much better managed. Queues profoundly affect the economics of product development. They cause valuable work products to sit idle, waiting to access busy resources.… queues hurt cycle time, quality, and efficiency.”

    Developing requirements far-in-advance that sit in a queue waiting for development can be inefficient and wasteful because:

    • The requirements may change prior to going into development and much of the effort involved in developing the requirements might have been wasted, and/or
    • Speculation in the requirements that are done too far into the future can result in erroneous assumptions that make their way into development without being questioned

    But, on the other hand, we shouldn’t blindly accept the notion that developing requirements far-in-advance never makes sense. There are a lot of reasons why at least developing high-level requirements early might make sense to guide the overall direction and architecture of the project and I’m sure that there are even some circumstances where developing more detailed requirements upfront also makes sense (you have to use the economic impact as a guide for making that decision).

  5. Variability – Understand and Exploit Variability
  6. “There is probably no aspect of product development that is more misunderstood than variability. Because variability sometimes leads to bad outcomes, it is inevitable that some observers will incorrectly generalize that variability always leads to bad outcomes.”

    Reducing variability will many times improve efficiency but that isn’t always the case. For example, Breaking up large requirements into smaller ones that are of a more uniform size reduces variability and can improve flow, however, at some point further attempts to reduce variability do not have economic value.

    For example, if we might use a rule-of-thumb that if a user story is greater than 13 story points, it needs to be broken down, but there’s probably little value in breaking down a story with less than 13 story points into smaller story points just to reduce variability.

  7. Batch Size – Reduce Batch Size
  8. “Ask a typical product developer what their batch size is, and why, and you will get a puzzled look…Product developers simply don’t think in terms of batch size. As a result, they underutilize one of the most important tools to improve flow… many of the most important improvements in product development such as concurrent engineering, rapid prototyping, and agile software methods, are recognizable as batch size reductions. However, because developers fail to recognize the underlying mechanism of action behind these improvements, they lose the chance to apply this principle more broadly throughout their development process.”

    Examples of batch size inefficiencies include:

    • Project scope – taking on more in a single project than is truly necessary
    • Project Funding – The entire project is conceived and funded as a single large batch proposal
    • Requirements definition – the tendency to define 100% of the requirements before the project starts
  9. WIP Constraints – Apply WIP Constraints
  10. Use Work in Process (WIP) constraints to manage overall flow. For example,

    • Control the number of projects taken on at any one time to avoid over-saturating development resources
    • Use specialized resources wisely to maximize their impact on overall flow
  11. Cadence, Synchronization, and Flow Control – Control Flow Under Uncertainty: Cadence and Synchronization
  12. “Cadence is the use of a regular predictable rhythm within a process. This rhythm transforms unpredictable events into predictable events. It plays an important role in preventing variability from accumulating in a sequential process…”

    Examples of cadence are using fixed-length sprints to establish a pace for the development effort. Examples of the use of synchronization include:

    • Concurrent development on multiple paths at the same time
    • Concurrent testing of multiple subsystems
  13. Fast Feedback – Get Feedback as Fast as Possible
  14. Fast feedback can lower the expected loss by truncating unproductive paths more quickly or raise the expected gain because we can exploit an emergent opportunity by rapidly redirecting resources.

    Fast feedback combined with selecting appropriate measures of performance enables rapid learning and ongoing continuous improvement.

  15. Decentralized Control – Decentralize Control
  16. “Sophisticated military organizations can provide very advanced models of centrally coordinated, decentralized control. There is an impression that military organizations seek robotic compliance from subordinates to the orders of superiors. In reality, the modern military focuses on responding to a fluid battlefield, rather than executing a predetermined plan. It views war as a domain of inherent uncertainty, where the side that can best exploit uncertainty will win.”

    Source: Reinertsen, Don, The Principles of Product Development Flow, Celeritas Publishing, 2009

    I recommend Don Reinertsen’s book for anyone who wants more in-depth understanding of these principles (this is just a very high-level overview). Many people get immersed in the mechanics of Scrum and forget about the values and principles of Agile. Don Reinertsen’s book goes even deeper into the principles of product development which provides a good basis for understanding both Agile (adaptive) approaches and Waterfall (plan-driven) approaches at a deeper level.

Is Agile Project Management an Oxymoron?

Is Agile Project Management an Oxymoron? A few years ago, I attended a presentation by a well-known agilist who said that “Agile Project Management is an Oxymoron”. That kind of perception is based on a lot of stereotypes about Project Management that need to be changed. Here are some of those stereotypes that exist:

  1. Project managers are very command-and-control oriented
  2. Project managers are heavily associated with plan-driven “Waterfall” processes and document-centric methodologies where you need a plan for almost every aspect of any project and you also need a document, checklist, or a form for almost anything you do; and there is no role for a Project Manager in an Agile environment that requires a much more adaptive approach

There are certainly project managers who have some of those characteristics, but it’s unfair to make a judgment that all project managers are that way or to write-off the whole project management profession as being irrelevant to Agile because some project managers may be that way. However, we in the project management profession need to recognize the impact of Agile and recognize that a shift in thinking is needed to broaden our perspective on project management to correct these stereotypes that do exist. Here are some of those shifts in thinking that I believe need to take place.

  • Command-and-Control Orientation

Project managers are noted for getting things done and as a Project Manager, I’ve been in numerous situations where a customer has expected me to “ramrod” a project to get it done to meet time and budget constraints. That’s the way Project Managers have operated for many years and it naturally leads to a very assertive role where the Project Manager takes total responsibility for the success of the project and drives everyone on the project team to deliver results to achieve success. Both project managers and their companies need to recognize the limitations in that approach.

The Project Management profession needs to go through some changes that I saw the Quality Management profession go through a long time ago. In the early 1990’s I worked in the Quality Management profession. What I learned at that time was that it is self-defeating for a Quality Manager to take on too much ownership for quality. An effective Quality Manager has to be somewhat of a “missionary” and engage others in the effort to own responsibility for “quality” to be successful. If the only people who are responsible for “quality” are the people in the Quality Management department or the Quality Assurance department, it’s not going to be very effective.

The same thing also applies to project management. The project management responsibility for an Agile project should be shared by everyone on the team – even a developer has to be a little bit of a project manager to plan, organize, and manage the tasks that he/she is responsible for. Operating in this environment calls for more leadership than management – a leader sets the vision as well as goals and objectives to be accomplished; puts in place the people, process, and tools to get it done; and then steps out of the way as much as possible.

That approach is counter-cultural for many project managers because project managers are noted for getting things done and many companies expect that of their project managers. So it requires a shift in thinking in both project managers and the companies who manage them to bring about this change in orientation. There is also a natural resistance to this in some instances because if you do it successfully and really empower the teams you manage, you can eliminate the need for a project manager and put yourself out of a job and that might be the right thing to do – there is no defined role for a Project Manage in an Agile team that is working as it should be. The key thing is that it requires a change in thinking in project managers to let go of that role that they may have played in the past and move up to playing a higher-level role that provides more value-added. It’s ironic, that as I think back on some of the recent projects I’ve managed, if I’m asked what I did to make it successful, the answer is “I did as little as possible”.

  • Association with “Waterfall” Methodology

    Project managers are also heavily associated with plan-driven, document-centric approaches like the “Waterfall” methodology that can be extremely cumbersome and bureaucratic and there is no need for a project manager in an Agile environment. Unfortunately, I think this stereotype is somewhat legitimate and I know a number of project managers who are still holding on to a traditional PMBOK-style, plan-driven, document-centric, “Waterfall” approach to project management. I think there are a number of possible reasons for that:

    • Job Security – Many project managers have been heavily trained in traditional project management skills and PMBOK and it’s a big risk to give that up because that’s what their knowledge is based on and that’s the value-added that they have been trained to deliver. The role of an Agile Project Manager is also very undefined and uncertain at this point and might require a significant amount of retraining. It’s understandable that some project managers might not feel comfortable making that leap of faith to re-orient their career around Agile Project Management
    • Binary, All-or-nothing Thinking – A lot of people on both sides of the fence have the perception that it is a binary choice between a totally plan-driven and heavily controlled approach (e.g. “Waterfall” and a totally unplanned approach with no control (e.g. “Agile”) and that is not the case. I’ve previously written a couple of articles on that subject:

      Many project managers and many companies do not see that there is a lot of middle ground between extreme Waterfall and extreme Agile…those alternatives may not be well-defined and it can take more skill to figure out how to develop an approach that blends the right proportion of Agile and traditional project management principles and practices to fit the situation but it definitely can be done. Unfortunately, many people make the mistake of force-fitting their business and their projects to one of these extremes rather than fitting the methodology (or combination of methodologies) to their business and to each particular project.

What is the solution to this? This is basically a change management problem and I’ve learned over the years that there are three elements that are essential to any significant change management initiative:

  1. “Burning Platform” – There has to be a sufficient level of “pain” associated with the current situation to recognize that it is untenable and making a change is crucial. People have to get past the “denial” stage and recognize that Agile has a profound impact on the project management profession and in the not-too-distant future, project managers who only know how to do traditional project management PMBOK-style Project Management and don’t know how to take a more Agile and adaptive approach when it makes sense are going to be at a serious disadvantage in the job market.
  2. Vision for the Future – The next step in any successful change management initiative is that there has to be a vision for what the future looks like after the change has taken place. To do that, we need to better define what an Agile Project Manager is and what role he/she will play. I’ve tried to help develop that vision with the two books I’ve published but there’s still work to be done in that area to better define that vision. I’m currently working on developing a graduate-level course on Agile Project Management with a major university in the Boston area that I’m hoping will also help fill this need.
  3. Progress in the Right Direction – In any change management initiative, there will always be some hold-outs and laggards who will wait on the sidelines to see if the change is going to be successful or not before jumping on board. I’m sure that there are a number of project managers who don’t want to take the risk of becoming Agile Project Managers until that role is much more well-defined and proven and they will wait until more people have actually demonstrated success in that role.

That will obviously take time and isn’t going to happen overnight, but the first step is to recognize the problem. I hope this blog post helps others see this problem as I do. I welcome your comments and thoughts on this!

The Impact of “Tunnel Vision” in Agile

I am amazed at the amount of “tunnel vision” in Agile. Many people see it only as a development process for single teams and don’t pay much attention to the larger context that the development process operates in. An Agile project always exists inside of some business context and if you look at the project from that business context rather than looking at it only from a development perspective, you might see a very different picture.

A pure Agile approach is best suited for a company that is in the business of producing some kind of software product (an example would be Intuit and TurboTax or Quicken) or where the software plays a very direct role in leveraging the company’s primary business (an example would be Amazon.com). Where that is not the case and the company’s business is only indirectly leveraged by the Agile development process (an example would be an internal IT application development project), it can be a lot more difficult to implement an Agile development process because more adaptation may be required to fit the Agile development process to the company’s business environment.

Here’s the difference – in the product development company, the company typically budgets a fixed amount of development effort to produce and sustain the product and the business decision is primarily about prioritizing how that money is spent on creating new features for the product to get the most business impact. That kind of business decision fits very nicely with an Agile development process approach and it’s relatively easy to implement an Agile development approach in that kind of environment.

In a company that is in some other kind of business where the software development effort plays a more indirect role in leveraging the company’s business, there is typically a very different kind of decision process. An example would be a company whose business is focused on achieving operational excellence (like Walmart) and not heavily focused on product innovation as a primary business goal. In that kind of environment, there is typically some kind of Project Portfolio Management approach to determine how to allocate the company’s IT budget to get the most “bang for the buck”. The company needs to choose the projects that are going to provide the highest amount of leverage to the business and then monitor those efforts to see if they really do produce the desired effect.

At the level of a single team development process, the effort may look the same as a product development company, but the business context is entirely different and requires some adaptation. The primary difference is in the level of planning required. In a product development company, you might be able to use a highly adaptive approach with almost no Product Backlog defined in advance and you might be able to almost completely define the requirements for the project as it progresses looking no more than 2-3 sprints ahead into the future. That’s what a pure Agile approach looks like; however, that doesn’t work in environments where there is more of a need to plan the project in advance and estimate the schedule and costs of the project to support a higher-level project portfolio management decision process.

In those environments, there is a typically a need to at least define the Product Backlog to a sufficient level to define the scope of the project and to provide at least a high-level estimate of the costs and schedule for the project. Some people will say that it is futile to attempt to estimate the costs and schedule for an Agile project because the requirements are only going to change. I don’t totally buy that…this is not an all-or-nothing decision to have a project totally planned in detail (like the Waterfall approach) or not planned at all (totally adaptive). There are plenty of alternatives between those two extremes to pick a level of planning that is appropriate for the project, the level of uncertainty in the requirements, and the business environment that it has to operate in.

Some people will also say that you have to force the entire company to change its culture to become totally Agile in order to implement an Agile development process. A company has to define its culture around what makes the most sense for the business it operates in and that may or may not align very well with an Agile development process. In the product development company, it probably aligns very well and it makes sense for the company to become more Agile in delivering new and innovative product features to market quickly. In a company like Wal Mart whose business is leveraged primarily by operational excellence and lower costs, that is probably not the case and you can’t necessarily force the company to adopt a totally Agile culture.

The key message is that you have to consider an Agile development process in the context of the business it operates in and many times you need to adapt an Agile development process to fit the business environment. Within the development process itself, the process may be largely the same – the differences may come into play at a higher-level such as the project portfolio management layer that wraps around the project. Those higher-level layers could also be Agile (Dean Leffingwell’s Scaled Agile Framework is an example), but that kind of complete, top-to-bottom Agile transformation just doesn’t work in all business environments.

Agile Is a Lot Like Playing Golf – It Can be Very Difficult

Agile Is a Lot Like Playing Golf -there are a number of things about Agile that remind me of playing golf.

  • In the game of golf, if you didn’t have to get the ball in the hole, my golf score would be a lot better.  Imagine if the requirement in golf was only to get the ball somewhere near the green – if you didn’t actually have to get it in the hole, my score would be a lot better, but that would be very misleading, wouldn’t it?That’s equivalent to a team that doesn’t have a clear definition of “Done”. On the surface, it may look like they’re completing sprints successfully, but when you look deeper, you sometimes discover that there is not a good process to validate that the work is really complete and really meets the business need it is intended to fulfill.
  • The reason golf can be such a difficult and frustrating game to master is that it requires a fair amount of discipline, lots of practice, and lots of patience and persistence to be good at it – Agile is the same way.
  • Golf also requires some planning and strategy – the best players carefully consider every shot; They don’t just go out there and whack the ball around.