How Frequently Should You Run Your Regression Tests?

Deciding how often to run your regression tests can be a challenge. Particularly when you've automated your regression suite, and you are more free to decide the frequency.

As a rule of thumb, you should run your regression tests as often as possible.

The picture below might be a useful analogy: Every time you let a bug out the door, you’ll want to make sure you haven’t let a swarm of new bugs in the door.

In other words, your regression suite should ideally be run whenever a change is made to the code. If your software system is large and complex, this is only possible to do with automation.

P.s. regression testing shouldn't be confused with retesting - here's the differences between retesting vs regression testing. 

This goes particularly for agile testing teams, where the goal is to get new updates and features tested and released as fast as possible.

There are two ways of ensuring that your tests are run at the desired frequency: You can either run them on-demand or on a schedule. 

Scheduled tests are ideal because they ensure consistent feedback on contributed code, and won’t be forgotten or overlooked as a result of other priorities. 

If, however, your test automation tool does not provide you with the option to schedule tests, you should as a minimum run your regression suite in connection with major releases, emergency fixes and incremental code changes. 

But remember that regressions don’t just occur when you submit a change. They can occur in connection with database or system updates, new browser versions, and other changes that are outside your knowledge, and perhaps also out of your hands. 

This is why finding a tool that allows you to schedule your tests is ideal.

To learn more about what to look for in a regression test automation tool, see our checklist for evaluating regression testing tools or download our whitepaper below to learn everything you need to know about regression testing in agile teams.