To keep up with the ever-increasing customer expectations to digital products and services, software providers are under pressure to be more and more thorough in their quality assurance of products. Learn how Continuous Testing can support those efforts.
An effective way to improve quality assurance (QA) is to “fail fast”. The purpose of failing fast is to catch and fix a bug as close to its origin as possible, before it makes its way further down the release pipeline. Because it’s much easier and less costly to fix an error in the context of which it was created, it’s always preferable for an error not to change hands. The traditional waterfall approach to software development does not support the objective of failing fast, as software testing is postponed until the final stages of a release. As such, another approach to QA is required. Enter Continuous Testing.
What is Continuous Testing?
The move towards a more agile production of software has introduced the concept of Continuous Delivery (CD). Author and long-time developer Jez Humble defines CD as “the ability to get changes of all types—including new features, configuration changes, bug fixes and experiments—into production, or into the hands of users, safely and quickly in a sustainable way.”
A common challenge when developing software following CD principles is the issue of constraint: You can only deliver as fast as your slowest asset. After a feature has been developed, but before it is released, what's most time-consuming is testing and all its related activities: bug fixing, deployment to test environments, etc.
There’s a rule of thumb in software development: You can release whenever you’re satisfied with the quality. Whereas agile development does not in itself ensure good quality, Continuous Delivery aims to help answer the question “What’s the quality of our software right now?” at any given point in the pipeline. Continuous Testing is a means to be able to answer that question, as this method is all about:
- testing at earlier stages of the release pipeline
- testing more often before release
- testing everywhere, that is, across environments and devices
Continuous Testing helps software teams approach the ideal of being able to evaluate quality at several stages of a pipeline where code is continuously developed, delivered, and deployed.
In a 2019 survey about Continuous Testing, 58% of respondents said that they would like for their IT organization to deploy daily – or even more frequently. This just shows that Continuous Delivery is here to stay, and testing needs to support this transformation.
Continuous Testing and Test Automation
In practice, Continuous Testing is achieved by having various tests (e.g. regression, functional, integration, and performance tests) being executed at their respective stages of the release pipeline, ideally without requiring human involvement to increase speed, lower costs, and reduce human error stemming from repetitive work. This is called orchestrated automation and automated testing is an essential part of that.
To adopt Continuous Testing successfully in a CD setup then, requires a test automation solution that’s a good fit for your pipeline. It’s important to note, however, that test automation does not in itself enable continuous testing, but it is absolutely a prerequisite for scaling testing – whether continuous or traditional.
Continuous Testing is about thorough, frequent, and reiterative execution of tests along the release pipeline. It enables early discovery of errors, and combined with other initiatives, such as Scrum, Continuous Testing helps you achieve agile development.