Continuous Testing in CI/CD: What, Why and How

Maria Homann

Automation Expert

Many software companies today are using Continuous Integration and Continuous Delivery (CI/CD) to release high-quality products at speed.

The CI/CD pipeline approach has emerged as a result of a tendency for today’s software users to be less and less tolerant towards defects and at the same time expect fast updates and fixes.

But building a CI/CD pipeline can be a challenge. And you cannot achieve CI/CD without the right automation strategy and tools.

What you'll find in this post: 

What is CI/CD?
What is continuous testing?
What does the CI/CD pipeline look like?
How to find the right CI/CD automation tool
Leapwork. An enabler of CI/CD

Watch on-demand webinar: Continuous Testing in Agile and Continuous Delivery Environments

What is CI/CD?

To explain what CI/CD is, it useful to start with what it’s not. Or rather, what was there before CI/CD and why we’ve now moved away from that towards a CI/CD strategy.

Most of you are probably familiar with the waterfall approach, a method in software development where you first develop, then test, then release. No processes overlap, and nothing happens in parallel. You start something and finish it before you go on to the next.

Waterfall

In many areas, this might not sound like such a bad idea. For example, it’s a good idea to finish your main course before you move on to dessert, and it’s also a good idea to finish your workout before you take a shower.

But in software development, things are a bit different.

To explain why, let’s add some context.

An example of continuous integration continuous development test

Imagine that you have to write all the code there is for an application. This might take you months or years. Once you’re done (let’s say it just took you 6 months) you pass it on to a tester.

The tester finds a piece of code that doesn’t work.

It just so happens to be some of the code you wrote at the very beginning of the project.

By this time, you’ve forgotten all about how and why you wrote the code like that. You have to re-write the code, which, by the way, impacts much of the remaining code.

Bugs start to appear, and the project ends up taking you years and costing you a lot.

This example is drawn from just one area of the software development pipeline – testing.

But software development is more than that, of course.

An example of the waterfall approach

This same ‘waterfall approach’ can take place even before the testing phase.

Let’s say we have multiple developers working on the same application. Each one is contributing units of code to certain features or bug fixes. As they write their units of code they’re pushing it into the same code repository.

At some point, however, a piece of code that one developer wrote doesn’t merge with the unit of code written by another developer. 

It's like three people building different sections of a puzzle. And when they try to merge the final puzzle, it doesn't quite fit. 

They now need to spend time on mending these units of code. This process can take more time than initially planned, halting the entire software development process.

As you probably now see, the waterfall approach is not great for developing software in a fast, efficient and lean manner.

CI/CD is faster than waterfall approaches

CI/CD, on the other hand, is great for delivering quality at speed.

CI/CD is an approach that is more circular than linear, meaning that processes are intertwined, and feedback loops are constant. Where a waterfall is perhaps most easily portrayed as, well… a waterfall, CI/CD is most easily portrayed as an infinity loop.

ci/cd vs waterfall

CI/CD is an automated chain where the idea is to involve as few human hands as possible, in order to avoid unnecessary manual work and error.

This doesn’t mean that humans aren’t involved in the process. It means that the chain should be able to run on its own. It should check for errors and notify testers and developers when something breaks or doesn’t merge. That way, changes can be made much quicker than would be possible in a waterfall approach.

CI/CD goes hand in hand with DevOps and agile methodologies, as these all work towards a common goal of delivering quality at speed through flexible approaches.

Due to its importance to the entire pipeline, testing is a key area in CI/CD. Let’s zoom in on this area next.

Related reading: How to Create a DevOps CI/CD Pipeline (With Example)

New call-to-action

What is continuous testing?

Testing is one process that can and should be automated to a great extent within the CI/CD pipeline.

Why? Because automated testing, which enables so-called continuous testing, ensures that bugs get found early and are fixed before the end-users experience any breaks. The idea is to 'fail fast' because it’s much easier to fix a bug when it’s found early, as the bug typically hasn’t traveled so far or changed hands, and it’s also a lot less costly.

Some types of tests that can be automated include regression tests, functional tests, integration tests, and performance tests.

Types-of-tests-web-automation

At the core, continuous testing is about three things:

  • testing at earlier stages of the release pipeline
  • testing more often before release
  • testing everywhere, that is, across environments and devices

You can learn much more about continuous testing in our blog post on the critical role of continuous testing in continuous delivery.

There’s more to CI/CD than testing of course, so next, let’s have a look at the entire CI/CD pipeline.

What does the CI/CD pipeline look like?

Let’s start with Continuous Integration (CI).

Let’s again say that we have multiple developers working on the same application. Each one has its own feature or bug fix that they’re working on while contributing to the application.

As they write their code their pushing it into the same code repository (typically GitHub), which keeps an overview of the different versions.

From there, a build system (typically Jenkins), takes it and builds it.

Since there are several developers contributing with code, the tricky part is to make sure that their contributions don’t introduce any bugs to the application.

This is where test automation comes in. As we’ve established, test automation is an essential piece of the CI/CD puzzle as it ensures that everyone is instantly notified in case something breaks within the application. Here, many agile teams choose Leapwork, as it requires zero coding, and is therefore easy to set up and maintain. 

Watch on-demand webinar: How to Overcome TestAutomation Challenges with Codeless Automation

Once the tests pass, the entire build cycle is considered successful and complete.

This construct of having multiple developers working on the same application at once, without experiencing any issues with merging code is what we call Continuous Integration.

CI/CD pipeline

Next is CD. Most talk about Continuous Delivery, though some also talk about Continuous Deployment.

So far, we’ve seen how the application is coded, built and tested within an automated pipeline using code repositories and build and testing systems.

Once the tests pass, the next step is to release the new version of the application. This means making it available to the end-users in some shape or form.

If this is done automatically through a pipeline, this is called Continuous Delivery.

The next step is to take the packaged application and automatically deploy it in a target environment. This is called Continuous Deployment.

How to find the right CI/CD test automation tool

By now, we've established that test automation is an enabler of CI/CD. However, some test automation tools are better enablers of CI/CD than others.

As the point of CI/CD is to deliver quality at speed in an agile matter, the test automation tool should be easy to work with so that it doesn’t take up an unnecessary amount of time from the tester.

If testers spend more time on programming or maintaining tests than they do on exploratory testing and test design, tests won’t be continuous. Similarly, if the automation tool doesn’t allow for easy hand-overs and collaboration by providing an easily understandable overview of test cases and troubleshooting therefore becomes a struggle, tests also won’t be continuous.

Related reading: 5 things you should look for in a CI/CD test automation tool.

Leapwork no-code automation: An enabler of CI/CD

If you’re on the lookout for a test automation tool that will work seamlessly with your CI/CD pipeline, you’re going to need one that is both easy to use and maintain and works with all your existing technologies.

Leapwork’s no-code automation platform offers a user-friendly approach to test automation that will integrate easily in your CI/CD pipeline.

To learn more about no-code test automation and to understand how it contributes to continuous testing and CI/CD, watch our on-demand webinar: Continuous Testing in Agile and Continuous Delivery Environments.

New call-to-action