Skip to content

What is Regression Testing and Why Is It Important? A Detailed Guide

Maria Homann

Maria Homann

When developers write and submit new code for an application, those pieces of code sometimes have unexpected effects on existing code, causing it to break.

It’s difficult to avoid this altogether, but it is possible to detect that broken code before it is released. This is done with regression testing.

Skip ahead to:

What is regression testing?

When to do regression testing

Examples of regression testing

Regression testing techniques

Regression testing at scale - how it's done

What is regression testing?

Regression testing is a series of tests that check that an application runs as intended. The idea is to run this series of tests every time new code is submitted.

It’s called regression testing because the verb regress means to return to a former state or condition, which in software, isn’t considered a good thing.

Regression testing is often confused with retesting, but the two differ on one key aspect: Regression testing is frequently run on the entire product, whereas retesting is used to check a specific feature, after you’ve fixed a bug in it. In other words. regression testing is designed to test for bugs you don't expect to be there, whereas retesting is designed to test for bugs you do expect to be there.

When to do regression testing

Regression testing is used to search for bugs to ensure that new updates or features that are added to software (involving new or changed code) don’t break any previously released updates or features (existing code), and to ensure that the tested software still performs as expected after a change.

Regression testing isn’t just a safety net; it's like a routine checkup for your system under test. The frequency of these tests will depend on the set-up of the business. Normally, it’s done before a software update or customization is introduced, or to check that integrations between applications work.

Examples of regression testing

If a customer application stops working, a business can lose revenue and their reputation is damaged. If an internal application like Dynamics 365 or Salesforce stops working, the business is at risk; revenue streams, customer experience and business reputation are just some of the things that are at stake when ERP and CRM processes break. Regression testing helps businesses minimize this risk.

In the bullets below, we have listed examples of what regression testing can look like across industries.

  • San Diego Superior Court uses a system called Odyssey for their Court Management. This system goes through 3-4 updates per year, each of which have to be tested. Before they introduced Leapwork, they would spend between 2-3 weeks on regression testing Odyssey. Today, they spend 40% less time on each release, and have a suite of 200 automated regression tests.
  • AdminContoluses test automation for their 2000+ strong regression testing suite. It’s integrated into their CI/CD pipeline, and they run end-to-end regression tests across web and mobile.
  • Dutch insurance companyVoogd & Voogd regression tests their business applications. 10,000 daily users rely on this application to complete their work. Their regression suite consists of testing each version release across a minimum of two browsers. Done manually, this would take eight hours and ≈115 tests. By introducing automation, they have been able to reduce this time to 20 minutes.

Regression testing techniques

You can bucket regression testing into two approaches. There’s manual regression testing and automated regression testing.

Manual testing is the process of letting people, often business experts from other departments who are closest to the business processes, run through the same regression tests. This process is very tedious, time intensive and is prone to error.

Automated testing is the process of building automated test cases once - and then adding them to your regression suite. Often this is done using a self-built framework or a commercial vendor.

Related reading: How to do regression testing

Many testers consider manual regression testing particularly tedious; it’s a bit like being given the task of searching through a haystack for a needle that probably isn’t there. And you don’t just have to do it once, you have to do it every time new code is created.

What’s more, the suite of regression tests grows with the product, meaning that over time, it becomes practically impossible to execute and maintain manually.

Technical-debt_diagram

For this reason, many QA teams choose to automate their regression testing for agility.

Regression testing at scale - how it’s done

Automated testing, in theory, can help a business build a scalable regression testing suite. However, not every solution is the same. Some take much longer to learn than others, and the maintenance of the tool will become larger and more time consuming the longer it is used. On top of this, not every solution will work with the technologies you need to automate. This prevents you from being able to run end-to-end tests.

When you’re searching for a scalable approach to regression testing, you want to keep these points in mind; the learning curve, the maintenance, and the technologies you can automate. You also want to make sure you're following the regression testing best practices.

New call-to-action