Why Is Automated Regression Testing Important in Agile?

Why is automated regression testing important in an Agile environment? An Agile development approach can dramatically improve the quality of software, but it requires a very different approach to software testing. Here’s a previous blog post with more on why an Agile testing approach makes so much sense:

What’s the Impact of Agile on Regression Testing?

Because the testing approach is so different, regression testing; and, in particular automated regression testing becomes a lot more important:

  • In a traditional plan-driven project management approach (Waterfall), all testing is typically done at the very end of the project
  • At that point-in-time, all software development is normally complete and the software is stabilized
  • In an Agile environment, software testing is done concurrently with development throughout the project
  • As a result, software is still changing as it is being tested

The difference is shown below:

Why Is Automated Regression Testing Important in Agile?

A big challenge this creates is this:

  • In a Waterfall-style development process, the software should be stabilized once it goes into testing
  • In an Agile environment, the software is continuously changing throughout the project. During each sprint, in addition to testing new functionality, it is essential to do regression testing. The purpose of regression testing is to verify that some new changes haven’t inadvertently broken something that was previously tested
  • In a large, complex project, the number of features requiring regression testing will grow significantly as the project is in progress. That makes it almost impossible to do manual regression testing. Attempting to do manual regression testing would slow down the progress of the project significantly if it can be done at all.

Overall Summary

For these reasons, it is important to do regression testing to ensure the quality and integrity of the software in an Agile environment. And, for a large, complex software project, automated regression testing becomes essential.

How Does Agile Testing Work and Why Does It Make So Much Sense?

Quality and reliability standards today for software are higher than ever and new approaches to software testing are essential to meet those goals. An Agile development approach provides a excellent approach to meet that challenge.

Agile Testing

How Is Traditional Software Testing Done?

In a traditional software testing approach, a separate and independent QA organization normally does the testing:

  • The development team typically turns over software to the QA organization for testing at the end of the project
  • That normally happens after all development is complete.

That method of testing is very consistent with a “Waterfall” style of development. A Waterfall project typically consists of phases that might look something like this:

That’s been the traditional way that software development and testing has been done for a long time.

How Is Agile Testing Different?

Agile takes a fundamentally different approach to the whole development process (not just testing).

  • First of all, Agile does not break up a project into phases as the Waterfall model does
  • It breaks up a project by chunks of functionality and uses a much more integrated approach to develop and test each “chunk” of functionality

There are two major differences that have a big impact on how testing is done:

  • Incremental Development Approach
  • Integral Testing Approach

We will discuss each of these in the following sections.

1. Incremental Development Approach

Instead of developing and testing the complete functionality of the project as one big effort:

  • The functionality is broken up into small increments called “user stories” and
  • The work to be done to develop those user stories is broken up into sprints. Each of those sprints are typically 2-4 weeks long
  • The work in each sprint includes testing as well as development

Ideally, at the end of each sprint, fully-tested software for that portion of functionality should be ready for release.

2. Integral Testing Approach

Instead of testing being done by a separate and independent QA organization:

  • The testing function is an integral part of the Agile team, and
  • As soon as software is sufficiently complete, it is tested

It is worthwhile to note that integration of testing with development does not necessarily mean that developers do their own testing. Within an Agile development team:

  • There is value in having people who are specialized and trained in testing
  • Having developers test their own code is generally not a very good idea

What Are the Advantages of the Agile Approach?

There are several significant advantages to the Agile testing approach.

1. Proactive Rather than Reactive Approach

Agile testing is a more proactive approach to eliminate defects at the source. A reactive approach to find and fix defects later can require a lot more rework.

2. Immediate Feedback to Developers

An Agile testing approach provides immediate feedback to the development team. If difficult software bugs go unresolved, they can compound themselves and that can make it much more difficult find and resolve the defects.

3. Quality is Not Someone Else’s Responsibility

Agile testing makes the quality of the product an integral part of the development process. The development team producing the product owns responsibility for the quality of the product they produce. It is not someone else’s responsibility.

In the traditional Waterfall-style development process, it is not uncommon for the development team to throw code “over the fence” to QA. They put the responsibility on QA to find any bugs.

The Roots in Total Quality Management (TQM)

In order to understand why this approach makes so much sense, it’s useful to understand the deeper roots that influenced it:

  • Agile is heavily based on the principles in Total Quality Management (TQM)
  • TQM was originally developed for manufacturing by Dr. W. Edwards Deming
  • The quality of the Japanese automobile industry was improved significantly as a direct result of TQM,
  • It is relatively easy to extend the TQM principles to software development.

TQM emphasized a major shift in thinking from

  • A reactive “Quality Control” approach that was heavily based on inspection to
  • A more proactive “Quality Assurance” approach to go upstream in the process to eliminate defects at the source

That makes sense for a number of reasons:

  • Resources required to perform the inspection are costly
  • Defects that are found after the work is already done can result in costly and unnecessary scrap and rework
  • Any inspection approach is based on sampling is not a thorough approach for finding defects. Some defects will likely slip through when you use sampling resulting in relatively low levels of quality

It is easy to see how these same factors apply to a software environment. Many software testing processes have been based on an old-fashioned quality control approach. which is based heavily on inspection.


Overall Summary

A major shift in thinking is essential to more effectively test software and to reach much higher levels of quality and reliability.

  • The principles of Total Quality Management (TQM) are a good foundation for this shift in thinking
  • Total Quality Management (TQM) dramatically improved the quality of products in the Japanese automobile industry
  • TQM principles can dramatically improve the quality and reliability of software

That shift in thinking that is based on TQM principles is well-integrated into an Agile/Scrum development and testing process

Additional Resources

This is only a very brief, high-level overview of Agile testing. If you want to learn much more detailed information, check out my Online Agile Project Management Training Courses. The first course is free!