The LEAP

Automation insights and productivity tips from LEAPWORK.

All Posts

How to Automate Functional UI Tests in a DevOps World

DevOps and automation

DevOps—the practice of bridging software development and software operations—is a means to releasing high quality software into production.

DevOps personnel facilitate collaboration and transparency between the different roles involved in the production of software: Development, Testing, and Operations. This includes defining the release pipeline, building or adopting the right tools for the team, and—perhaps most importantly—automating as much of the release pipeline as possible.

Automation is a prerequisite for success with DevOps. Especially test automation is a key ingredient when it comes to providing fast and accurate feedback to testers and developers enabling quick reactions to errors, bugs, and changing requirements.

 

Defining the release pipeline

The backbone of DevOps is the definition of the release pipeline. A release pipeline consists of various phases and checks (gates) that a piece of code, or software, must pass through as it travels from the individual developer’s computer into production. The pipeline is typically controlled by an “orchestrator” ensuring that things happen in the right order and the right checks are made between phases. Examples of pipeline orchestrators include Jenkins, Atlassian Bamboo, TFS/VSTO, and TeamCity.

Learn more about LEAPWORK integrations with pipeline orchestrators.

As illustrated in the figure below, a release pipeline typically includes:

  • the individual developer environment where the code is generated
  • a build phase where the code is prepared for release using Continuous Integration (CI)
  • a testing phase dedicated to various types of testing, incl. functional UI testing
  • a production environment

The Pipeline Orchestrator takes care of overall deployment and control of the gates between the different stages in the release pipeline, including the Developer phase, the CI-CD phase, the Test phase, and the Production phase. The orchestrator triggers third party platforms and applications to perform operations as part of the release pipeline.

To meet the requirements of a Continuous Delivery pipeline, automated functional UI testing is crucial. Read the post “How to Achieve Agile Testing by Automating Functional UI Tests” to learn why test automation is vital in this context.

But how do you integrate automated functional UI testing in a release pipeline? A common mistake when implementing automation is to settle for a tool that is not a good fit for the release pipeline. A pipeline should dictate the tool – not the other way around.

 

How to automate testing across the release pipeline

With the right setup, a test automation platform can be integrated with the release pipeline at several stages serving multiple purposes. Below are three examples:

1. Regression testing. This is the primary and most common target for automation; “classic” regression testing in a relatively stable test environment. Automated regression tests can be either triggered from the pipeline orchestrator or simply run on a scheduled basis as needed.

2. Build verification. The second area where test automation can support DevOps is to run a small, select set of test cases against CI servers. This will help answer the question: “Is the latest build from the CI servers testable?” In other words, is this build ready to be moved to the Test environment and worth spending actual tester resources on? This verification exercise provides developers with fast feedback in addition to running unit and integration tests.

3. Non-intrusive verification. The third and final application of test automation in the release pipeline is to verify single elements or parts of the software in a Production environment. The test cases running against Production should be non-intrusive, meaning that they shouldn’t leave any traces or interfere with the main purpose of the software. Instead, this kind of verification is intended for catching issues such as unwanted outages, malfunctions, etc. before users of the software experience them.

These three examples of how to utilize automated functional UI testing in a DevOps setup is shown in the figure below.

The Pipeline Orchestrator is integrated with a test automation platform which can automate build verification, regression testing, and smoke testing.

 

DevOps criteria for a test automation platform

So, from a DevOps perspective, which requirements should you be aware of when choosing a platform for automation of functional UI tests?

  • Integration. Make sure the platform is easy to integrate with your pipeline. A tool should come with relevant plugins for the most common pipeline orchestrators and/or well-documented APIs allowing the orchestrator to control the test automation platform.
  • Execution time. In a fully automated pipeline, the time needed for running tests is crucial. If it takes 48 hours to run all test cases, the pipeline is dysfunctional and will not work in practice. Make sure that the test automation platform can scale the execution of test cases to keep run-time at a minimum.
  • Control and flexibility. Successful automation requires a high degree of flexibility without compromising the feeling of being in control of the pipeline. A test automation platform should make it easy to define and execute different collections of test cases,
    • in various environments,
    • for various purposes,
    • and at various times.

For example, some environments are more stable than others, allowing for more frequent testing, while other environments are just used for quick verification and feedback.

The underlying vision of the LEAPWORK Automation Platform is based on the idea of the DevOps-pipeline. We experienced that the lack of a platform that “could take care of everything” was a recurring issue.

The LEAPWORK Automation Platform comes with a comprehensive public REST API allowing third party systems to trigger collections of test cases and retrieve a well-formatted response. Native plugins for the most common pipeline orchestrators (Jenkins, Atlassian Bamboo, TFS/VSTO, TeamCity, and more) are also available for LEAPWORK. This makes it very easy to include test automation as part of a release pipeline.

Read the LEAPWORK guide: Test Automation - A Winning Game PlanRead "Test Automation - A Winning Game Plan", LEAPWORK's guide to reducing risk, lowering costs, and driving value with test automation.

Kasper Fehrend
Kasper Fehrend
Senior Product Evangelist at LEAPWORK.

Related Posts

How to Analyze Test Automation Results

As test automation is introduced to the software delivery process, the amount of available test results explodes. Robots, or test execution agents, can run 24/7 without breaks, and, on top of this, the number of test cases accumulate during each sprint. As such, more results are produced to be managed and analyzed. This requires the right approach.

Test Automation Strategy: A Checklist

Too often, test automation is not handled as a strategic choice as there’s too much focus on putting out fires here and now. Test automation is a cornerstone in DevOps, and when implemented correctly, it helps increase output quality while containing costs. Not surprisingly, IT departments everywhere are realizing the importance of having an actual test automation strategy.

5 Failures in Test Automation – and Best Practices for Tackling Them

Too many test automation projects fail – and often for predictable reasons. To help you avoid these pitfalls, we have condensed the most common issues into this short guide along with related best practices for achieving success with automation.