Regression testing is a core part of quality assurance and software testing. Yet, some testers rush the process or skip it entirely because it is time-consuming. But with the right approach, it doesn't have to be.
In fact, when done in the right way, it can save you time as a tester, and let you focus on more interesting tasks.
Before digging deeper into how regression testing is done fast and well, let’s first take a look at what regression testing is, and how you can approach it.
What is regression testing?
When a software system is built, it is released once the software company is confident that it works and performs as intended. They might then update the software to introduce new features or functionalities that will create additional value to the end-user.
Every now and then, however, updates go wrong. Despite rigorous testing of new features, introducing these might have caused old features to fail.
Naturally, as a software developer, you want to be able to update that software while maintaining its existing features. The last thing you want is to make your product perform worse – or to regress.
Regression: When you fix one bug, but you introduce several new bugs.
This is why you run regression tests. The process of testing existing features and looking for regressions is called regression testing.
In technical terms, regression testing, is the re-running of functional and non-functional tests to ensure that no new developed code causes bugs or breaks any existing functionality in the software.
The purpose is to detect any unwanted changes to ensure that the tested software still performs as desired after a change.
How to improve regression testing
There are many ways to approach regression testing. In this blog post we’ll focus on guiding you towards faster and more efficient regression resting.
Let’s start from the top.
1. Build your regression suite
To do regression testing, you as a tester must build a regression suite. A regression suite is essentially an overview of the different functionalities that your software consists of, and that you’ll therefore want to test. These are also called test scenarios.
To build your regression test suite doesn’t necessarily require a lot of work. The suite can be created from the existing test cases – functional tests, integration tests, unit tests, etc. – that you probably developed from day one. Anything that has previously successfully verified that the software functions as intended can be used in your regression suite.
Over time, as your software system grows or changes, your suite will grow as well, numbering into perhaps hundreds or thousands of regression test cases. This is why automation will inevitably become part of your testing process at some point.
2. Select a regression testing approach
But how much of your testing should be automated? The answer to this question largely depends on your approach to testing; will you write all your test cases up front and stick rigidly to these when testing, or will you take a more exploratory approach where your testing is more free and unstructured?
The approach you choose should ideally fall somewhere on a spectrum between the two.
Automation is in not just beneficial, but in fact essential, to the agile software development team, as it enables them to perform regression tests more efficiently. But it does have its limitations – and it requires an understanding of these to be able to reap the benefits of automation.
The limitation (which could also be viewed as a benefit) is that your robots will check exactly what you ask them to. Nothing more and nothing less.
As your software grows and changes, your testing requirements will inevitably grow and change as well, and the tests that you previously set up to test the product as it was at a given time, may no longer match what the product looks like at a later time.
This is where critical thinking becomes crucial to the testing process. It is up to the testing team to review and evaluate the results of the test.
The great thing about automation is that, when done right, it can free up testers' time and let them work on test design and improvement.
You can read much more about striking the perfect balance between automation and manual testing in our blog post: Manual Testing vs. Test Automation: 10 Considerations
However, in setting up your automated regression tests, it’s still up to you to decide what goes into the regression test suite, and what doesn’t.
So how do you select your test cases?
3. Select your test cases for the regression suite
As a starting point, if you haven’t built your test suite before, and you need help figuring out where to start, it can be useful to select your test cases based on the following prompts:
- Is the function core to your product? Is it essential for other functionalities to work? Core functions should always be tested.
- Is the feature new, and has it been tested against numerous other feature updates before? New code tends to be more vulnerable.
- Is the code sensitive to the environment being configured? Dependency on environment settings tend to be more vulnerable.
- Has the code been defect before? It may pay off to pay extra attention to code that has previously been faulty.
Once you’ve considered these elements, you can set up the regression suite and schedule your regression tests.
Remember that you can always adjust your testing suite. As your team develops and tests, you’ll be constantly learning and adapting, so it’s only natural that your test suite will too. It’s crucial, however, that the tool you use to set up your tests in provides a good overview and is simple to edit.
You can read more about tool selection in our blog post on how to choose your test automation tool.
4. Decide the frequency of your test runs
Once you’ve built your regression testing suite, you need to decide how often you want the regression tests to run.
The picture at the top of the page might be a good rule of thumb here: Every time you let a bug out the door, you’ll want to make sure you haven’t let a swarm of new bugs in the door.
That means regression testing should actually be performed whenever a change is made to the code. If your software system is large enough, this is only possible to do with automation.
Still, you'll need to decide the frequency of automation runs. Read our blog post How Frequently Should You Run Your Regression Tests? to learn how to hit the right regression testing frequency, and to learn about on-demand vs. scheduled automated tests.
Perform regression tests faster with automation (that doesn't require a massive time and resource investment)
In conclusion, if you want to perform your regression tests faster, you’ll have to automate the majority of your regression testing.
But how do you introduce test automation without spending hours and hours on setup and maintenance?
Read on in our whitepaper: How to do regression testing in agile teams to find out.