Regression testing is a core part of quality assurance and software testing. This type of testing is key to ensuring a great end-user experience. Yet, some testers rush the process or skip it entirely because it is time-consuming.
But it doesn’t have to be. In fact, when done in the right way, it can save you time as a tester, and let you focus on more interesting tasks.
Before digging deeper into how regression testing is done fast and well, let’s first take a look at what regression testing is, and how you can approach it.
What is regression testing?
When a software system is built, it is released once the software company is confident that it works and performs as intended. They might then update the software to introduce new features or functionalities that will create additional value to the end-user.
Every now and then, however, updates go wrong. Despite rigorous testing of new features, introducing these might have caused old features to fail.
Naturally, as a software developer, you want to be able to update that software while maintaining its existing features. The last thing you want is to make your product perform worse – or to regress.
Regression: When you fix one bug, but you introduce several new bugs.
This is why you run regression tests. The process of testing existing features and looking for regressions is called regression testing.
In technical terms, regression testing, is the re-running of functional and non-functional tests to ensure that no new developed code causes bugs or breaks any existing functionality in the software.
The purpose is to detect any unwanted changes to ensure that the tested software still performs as desired after a change.
How to do regression testing
There are many ways to approach regression testing. In this blog post we’ll focus on guiding you towards faster and more efficient regression resting.
Let’s start from the top.
To do regression testing, you as a tester must build a regression suite. A regression suite is essentially an overview of the different functionalities that your software consists of, and that you’ll therefore want to test. Some also call these ‘test scenarios’.
Building your regression suite
To build your regression test suite doesn’t necessarily require a lot of work. The suite can be created from the existing test cases – functional tests, integration tests, unit tests, etc. – that you probably developed from day one. Anything that has previously successfully verified that the software functions work as intended can be used in your regression suite.
Over time, as your software system grows or changes, your suite will grow as well, numbering into perhaps hundreds or thousands of regression test cases. This is why automation will inevitably become part of your testing process at some point.
Software regression testing: Two approaches
But how much of your testing should be automated? The answer to this question largely depends on your approach to testing; will you write all your test cases up front and stick rigidly to these when testing, or will you take a more exploratory approach where your testing is more free and unstructured?
The approach you choose should ideally fall somewhere on a spectrum between the two.
Automation is in not just beneficial, but in fact essential, to the agile software development team, as it enables them to perform regression tests more efficiently. But it does have its limitations – and it requires an understanding of these to be able to reap the benefits of automation.
The limitation (which could also be viewed as a benefit) is that your robots will check exactly what you ask them to. Nothing more and nothing less.
As your software grows and changes, your testing requirements will inevitably grow and change as well, and the tests that you previously set up to test the product as it was at a given time, may no longer match what the product looks like at a later time.
This is where critical thinking becomes crucial to the testing process. It is up to the testing team to review and evaluate the results of the test.
The great thing about automation is that, when done right, it can free up the time of resources, giving them more time to review and evaluate critically.
You can read much more about striking the perfect balance between automation and manual testing in our blog post on how to automate regression testing.
However, in setting up your automated regression tests, it’s still up to you to decide what goes into the regression test suite, and what doesn’t.
So how do you select your test cases?
Regression test selection: A framework
As a starting point, if you haven’t built your test suite before, and you need help figuring out where to start, it can be useful to select your test cases based on the following prompts:
- Is the function core to your product? Is it essential for other functionalities to work? Core functions should always be tested.
- Is the feature new, and has it been tested against numerous other feature updates before? New code tends to be more vulnerable.
- Is the code sensitive to the environment being configured? Dependency on environment settings tend to be more vulnerable.
- Has the code been defect before? It may pay off to pay extra attention to code that has previously been faulty.
Once you’ve considered these elements, you can set up the regression suite and schedule your regression tests.
Remember that you can always adjust your testing suite. As your team develops and tests, you’ll be constantly learning and adapting, so it’s only natural that your test suite will too. It’s crucial, however, that the tool you use to set up your tests in provides a good overview and is simple to edit.
You can read more about tool selection in our blog post on how to choose your test automation tool.
How frequently should the regression tests be performed?
Once you’ve built your regression testing suite, you need to decide how often you want the regression tests to run.
The picture above might be a good rule of thumb here: Every time you let a bug out the door, you’ll want to make sure you haven’t let a swarm of new bugs in the door.
That means regression testing should actually be performed whenever a change is made to the code. If your software system is large enough, this is only possible to do with automation.
Where traditional test approaches might only do so-called ‘sprint regression tests’ where each sprint cycle ends with a regression test of the most recent product features, followed by a so-called ‘end-to-end regression test’ to test all features, automation actually makes it feasible to run your entire regression suite after every build.
In an agile testing team, the goal is typically to get new updates and features tested and released as fast as possible. This means more releases and hence more regression tests. If the team follows a continuous integration approach, regression testing will be carried out each time new code is added to the software.
In fact, by automating regression tests, testers can work and update code continuously, while regression tests run in the background, ensuring that nothing breaks.
The agile team is by nature collaborative and will therefore also need to inform each other about changes made in order to perform the right regression tests.
Again, an automated regression testing tool can enable teams like this to collaborate on projects, improve flows, and overall optimize the regression testing processes.
Perform regression tests faster with automation
In conclusion, if you want to perform your regression tests faster, you’ll have to automate the majority of your regression testing.
Your test regression suite should ideally be created right from the beginning and held up to date by the testing team. This is particularly important in agile development teams, to ensure speed and change readiness.
Automating the regression suite makes it possible to schedule regression tests and run them as often as needed, at a high speed, error-free. Automated testing makes it possible to complete end-to-end regression testing every time a minor update is made.
It’s important to note, however, that the results of automating your regression tests highly depends on a good balance between manual and automated testing as well as the tool you choose. Read on in our blog post on automated regression testing to learn more.