An understanding of the principles of product development flow and Agile can provide an objective basis for understanding how to maximize the efficiency of a product development process. There is a lot of “religious fervor” about product development methodologies and in particular Agile versus Waterfall. I still remember a quote from one of Dean Leffingwell’s slide presentations from years ago and I particularly like a comment he made:
“Agile is only a tool – it’s not a religion”
The choice of a methodology should be based on fitting the methodology to the nature of the problem. There are certainly a lot of good reasons why Agile makes sense in many situations but some people seem to just do it mechanically. People become committed to Agile like a “religion” and Agile becomes a goal in itself without a real understanding of why it makes sense in a given situation. We need a basis for comparison of Agile and other approaches that is objective and not based on religious fervor.
If you look at the whole product development process objectively and systemically, what you’re really trying to do is maximize the flow of that overall process.
Principles of Product Development Flow
Don Reinertsen has written a very good book called “The Principles of Product Development Flow” which I think provides some valuable insight into the underlying principles of why Agile makes sense. Since the principles are general enough to apply to any product development effort (agile or plan-driven), it also provides an objective foundation for comparing Agile and alternative plan-driven approaches so that someone can make an intelligent and objective assessment regarding how these two seemingly competitive approaches.
The book goes a little too far in my view in developing a quantitative and mathematically sound basis for the principles in the book based on statistics. I don’t think very many people in the real-world would really apply that kind of rigor to analyzing these principles; but nonetheless, an understanding of the principles at a high level, is very valuable. And, it does provide an objective foundation for understanding the benefits of Agile at a deeper level. I’ve summarized Don Reinertsen’s eight principles here:
1. Economics – Take an Economic View
“Why do we want to change the product development process? The answer: To increase profits. As obvious as this might seem, this economic view of product development is surprisingly uncommon. Instead, product developers create a large number of proxy objectives: increase innovation, improve quality, conform to plan, shorten development cycles, eliminate waste, etc. Unfortunately, they rarely understand how these proxy objectives influence profits.”Reinertsen, Don, The Principles of Product Development Flow, Celeritas Publishing, 2009
I’ve seen a number of instances where companies blindly pursue some of those “proxy objectives” such as “increasing innovation” without really understanding the economic impact. “Increasing innovation” should not be an end-in-itself, it reaches a point of diminishing returns at some point, and it might begin to impact other proxy variables such as quality. For that reason, it is good to put things in an economic context. The economic context provides a framework for making intelligent decisions about how much to focus on these “proxy variables”.
Here’s another example of how the economic context can provide a sound framework for decision-making: It is generally best to defer decisions on product features as long as possible; however, some decisions should be made early and should not be significantly deferred because they might have a significant economic impact.
2. Queues – Actively Manage Queues
“Queues matter because they are economically important, they are poorly managed and they have the potential to be much better managed. Queues profoundly affect the economics of product development. They cause valuable work products to sit idle, waiting to access busy resources.… queues hurt cycle time, quality, and efficiency.”Reinertsen, Don, The Principles of Product Development Flow, Celeritas Publishing, 2009
Developing requirements far-in-advance that sit in a queue waiting for development can be inefficient and wasteful because:
- The requirements may change prior to going into development and much of the effort involved in developing the requirements might have been wasted, and/or
- Speculation in the requirements that are done too far into the future can result in erroneous assumptions that make their way into development without being questioned
But, on the other hand, we shouldn’t blindly accept the notion that developing requirements far-in-advance never makes sense. There are a lot of reasons why at least developing high-level requirements early might make sense to guide the overall direction and architecture of the project and I’m sure that there are even some circumstances where developing more detailed requirements upfront also makes sense (you have to use the economic impact as a guide for making that decision).
3. Variability – Understand and Exploit Variability
“There is probably no aspect of product development that is more misunderstood than variability. Because variability sometimes leads to bad outcomes, it is inevitable that some observers will incorrectly generalize that variability always leads to bad outcomes.”Reinertsen, Don, The Principles of Product Development Flow, Celeritas Publishing, 2009
Reducing variability will many times improve efficiency but that isn’t always the case. For example, breaking up large requirements into smaller ones that are of a more uniform size reduces variability and can improve flow, however, at some point, further attempts to reduce variability do not have economic value.
For example, if we might use a rule-of-thumb that if a user story is greater than 13 story points, it needs to be broken down, but there’s probably little value in breaking down a story with less than 13 story points into smaller story points just to reduce variability.
4. Batch Size – Reduce Batch Size
“Ask a typical product developer what their batch size is, and why, and you will get a puzzled look…Product developers simply don’t think in terms of batch size. As a result, they underutilize one of the most important tools to improve flow… many of the most important improvements in product development such as concurrent engineering, rapid prototyping, and agile software methods, are recognizable as batch size reductions. However, because developers fail to recognize the underlying mechanism of action behind these improvements, they lose the chance to apply this principle more broadly throughout their development process.”Reinertsen, Don, The Principles of Product Development Flow, Celeritas Publishing, 2009
Examples of batch size inefficiencies include:
- Project scope – taking on more in a single project than is truly necessary
- Project Funding – The entire project is conceived and funded as a single large batch proposal
- Requirements definition – the tendency to define 100% of the requirements before the project starts
5. WIP Constraints – Apply Work in Progress (WIP) Constraints
Use Work in Process (WIP) constraints to manage overall flow. For example,
- Control the number of projects taken on at any one time to avoid over-saturating development resources
- Use specialized resources wisely to maximize their impact on overall flow
6. Control Flow Under Uncertainty – Cadence and Synchronization
“Cadence is the use of a regular predictable rhythm within a process. This rhythm transforms unpredictable events into predictable events. It plays an important role in preventing variability from accumulating in a sequential process…”Reinertsen, Don, The Principles of Product Development Flow, Celeritas Publishing, 2009
Examples of cadence are using fixed-length sprints to establish a pace for the development effort. Examples of the use of synchronization include:
- Concurrent development on multiple paths at the same time
- Concurrent testing of multiple subsystems
7. Fast Feedback – Get Feedback as Fast as Possible
Fast feedback can lower the expected loss by truncating unproductive paths more quickly or raise the expected gain because we can exploit an emergent opportunity by rapidly redirecting resources.
Fast feedback combined with selecting appropriate measures of performance enables rapid learning and ongoing continuous improvement.
8. Decentralized Control – Decentralize Control
“Sophisticated military organizations can provide very advanced models of centrally coordinated, decentralized control. There is an impression that military organizations seek robotic compliance from subordinates to the orders of superiors. In reality, the modern military focuses on responding to a fluid battlefield, rather than executing a predetermined plan. It views war as a domain of inherent uncertainty, where the side that can best exploit uncertainty will win.”Reinertsen, Don, The Principles of Product Development Flow, Celeritas Publishing, 2009
I highly recommend Don Reinertsen’s book for anyone who wants more in-depth understanding of these principles (this is just a very high-level overview). It provides a very sound way of looking at any product development process objectively. And, since the same principles apply to either Agile or traditional plan-driven projects, it can be used to provide an objective basis of comparison between the two.
Have you ever met someone who has what I call “Methodology Myopia”? The symptoms of this problem are that the person thinks that there is one particular methodology (whatever it might be – Agile, Waterfall, or something else) that is a universal solution for any kind of project you might have. This “one size fits all” approach many times results in people force-fitting a project to a methodology whether it fits or not and practicing that methodology rigidly without necessarily tailoring it to fit the situation. I believe that a better approach is to fit the methodology to the project and sometimes that might require blending elements of different methodologies together to fit the situation.
Why does this problem exist?
- Sometimes people only know one methodology. For example, for many years, project managers were heavily schooled in the Waterfall approach and there was no need to learn any other methodology.
- It takes a lot more skill to know how to tailor a methodology and/or mix-and-match elements of different methodologies to fit a situation. It requires knowledge of a broader range of methodologies and a deeper understanding of the principles behind the methodology rather than just the mechanics of how it is implemented.
- Sometimes people get so over-zealous about a particular methodology that they lose their objectivity and forget that any methodology has limitations and needs to be applied intelligently and tailored as necessary to fit a situation. For example, there are some agilists who are so zealous about Scrum that they completely reject any kind of more traditional plan-driven approach as obsolete and no longer relevant.
- Consultants often fan the flames of this fire by building the frenzy about a particular methodology or approach being the best thing possible because they are selling their services based on a particular approach or methodology.
Many people get immersed in the mechanics of Scrum and forget about the values and principles of Agile. Don Reinertsen’s book goes even deeper into the principles of product development which provides a good basis for understanding both Agile (adaptive) approaches and Waterfall (plan-driven) approaches at a deeper level.
You can find related articles on the topic of “Understanding Agile” here:
You will find much more detail on this in my Online Agile Project Management Training.