I recently participated in an online discussion on the topic of “Is Agile and Scrum Anti-Engineering?”. This question seems to be based on a number of popular stereotypes that people have about Agile and Scrum engineering practices. For example, a popular stereotype is that an Agile project consists of just a bunch of “cowboy software developers” who get together and start writing code without any plan and discipline about how it is done.
If you have that view, it’s easy to come to the conclusion that Agile is not a very sound approach to engineering at all because it doesn’t necessarily follow a well-planned and systematic engineering design approach but it would be a mistake to draw that conclusion. If it is well-executed, an Agile approach requires a considerable amount of discipline and skill and it’s not “anti-engineering” – it is just a different kind of engineering.
Importance of a Systematic Engineering Design Approach
Let’s step back and look at when and where a well-planned, systematic engineering design approach really makes the most sense. That kind of approach is most important for problems that have a high level of technical complexity and the aim is to find an optimum solution as quickly and efficiently as possible:
“A systematic engineering design process model aims at making it easier to find an optimal design for a product-to-be. To that end it is necessary to encompass the broadest range of solutions, that is, to search for solutions in a structured, systematic way. The breadth-first top-down strategy is adopted, which means first finding the largest possible number of abstract solutions (breadth-first) and then more concrete ones (top-down)”
https://www.designsociety.org/download-publication/26782/a_review_of_the_fundamentals_of_systematic_engineering_design_process_models
A good example of that might be a very data-intensive application that might start with a high-level, conceptual design phase, proceed to a logical design phase, and then finally wind up with a detailed physical design of the solution. To do that efficiently, you will want to stabilize the requirements as early as possible because each phase cascades into the next. You don’t want to get all the way into the detailed physical design and discover some of the critical assumptions made in the previous phases were wrong because it could require a significant amount of rework of the whole design.
However, there are many people who believe that a well-planned and systematic approach to engineering is the only way to do engineering. Here’s an example:
“The design of complex, complicated or a family of products is usually beyond the intuitive skills alone of a designer or design team. Gerhard Pahl and Wolfgang Beitz [1] have set out a strategy for the development of solutions which aims to increase the probability of technical and economic success of product design. This is done by creating a dependable approach which allows careful planning and systematic execution so that the whole design task reduces to a logical and comprehensible exercise and also allows recovery from inevitable errors. It also allocates a time schedule for the design stages which in turn leads to a predictable project timetable. Systematic design is general enough to be applied in any branch of engineering.”
http://theoriesaboutengineering.org/gerhard_pahl_and_wolfgang_beitz.html
When Does a Systematic Engineering Design Model Make Sense?
That well-planned, systematic approach to engineering design has been the predominant approach to engineering design for a long time. It has proven to be a reliable and efficient process for converging on a design solution where the requirements are relatively well-defined and the technology is also relatively stable as shown in the Stacey complexity model below:
The primary question that the design team must answer in this environment is:
“Is this solution the most optimum design solution to the requirements for this problem?”
It is primarily a pure engineering design decision.
Why Doesn’t That Approach Work in Many Areas Today?
The major problem with that approach is that we live in a much more complex world with much higher levels of uncertainty today. In terms of the Stacey complexity model, both the requirements and the technology associated with the solution might be much more uncertain. In that kind of environment,
- It is no longer a simple question of “Is this solution the most optimum solution to the requirements for this problem?” and
- Its no longer limited to relatively straightforward engineering design decisions because the requirements themselves might be wrong and the assumptions about the technology might also be wrong
In an uncertain environment, the approach needs to be much more adaptive and involve some amount of “discovery” to determine what the requirements for the solution should be. The approach should not be limited to just implementation of a predefined solution. There is also typically a lot of tradeoffs between the requirements for the solution and the technical implementation that optimizes both. Converging on a solution to those requirements without considering those tradeoffs might not yield an optimal solution.
When that kind of situation has arisen in the past, a typical approach has been to do some amount of upfront prototyping to try to converge on the requirements of a solution before actually starting on the full-scale design and development effort. That approach works in some cases but it isn’t the most efficient or the most effective approach if there is a large amount of uncertainty in the requirements. If there is a large amount of uncertainty in the requirements, it may take multiple iterations to converge on the requirements for the design; and, for larger and more complex projects, there may be a number of different areas of uncertainty that need to be resolved.
That’s why an Agile approach can make a lot of sense for projects with a high level of uncertainty. It’s not really “anti-Engineering”:
- It’s a different kind of engineering approach that is different from a typical, well-planned, systematic engineering design approach,
- It is much more appropriate for areas that have high levels of uncertainty
It’s important to recognize that Agile is not “cowboy engineering” – if it is done correctly it requires a lot of discipline and skill.
What Does This Mean For Project Management?
Project Management is not just an administrative task – project management needs to be designed around the most sensible engineering design approach for a given problem. Traditionally, the project management approach has been heavily-based on well-accepted engineering design practices that emphasize a well-planned, systematic engineering approach. As we rethink the way that engineering is done, we also need to develop new project management models that are also better aligned with higher levels of uncertainty.
Overall Summary
Here’s a summary of some key points from this article:
- There is an intimate relationship between project management processes and engineering design processes.
- Many people have forgotten about that relationship and think of project management as primarily an administrative management function to manage the execution of what many people loosely call “Waterfall”.
- Project management is more than just an administrative function – it should be geared to effectively manage an underlying engineering design process. And, that underlying engineering design process should be geared to the nature of the problem.
- The underlying engineering design approach that has been assumed to be well-established for so many years can no longer be taken for granted as the only way to do engineering design. It does not provide a sufficient level of flexibility and adaptivity for a highly uncertain environment.
- Project management must adapt as well – it is no longer viable to force-fit all projects to a project management that is based on an underlying engineering design approach that has serious weaknesses in an uncertain environment.
The challenge for project managers is learning how to blend an iterative and adaptive Agile approach with a more traditional plan-driven approach in the right proportions to fit the nature of the project and one of the biggest factors in choosing the right approach is the level of uncertainty in the project.
Related Articles
Check out the following related articles on “Agile Development Practices”:
- Is UML Still Relevant Today? How Is it Used in an Agile Environment?
- Agile and Scrum Engineering Practices
- What Is an Agile Developer? How Is the Role Different?
Additional Resources
Resources for Agile Project Management Online Training.