What is Test Maintenance?

Test maintenance. It’s a tedious and time-consuming task. But without it, you’re opening your business up to major risk. And with it, your test automation costs skyrocket. Especially if you’re using code to build the automation that tests your graphical user interface (GUI). 

However, there still seems to be confusion as to what this term means. We’ll explain what test maintenance is, the challenges you should expect when you’re maintaining your tests, and the test maintenance best practices. 

What you can find in this post? 

What is test maintenance?
What are the challenges of test maintenance?
4 best practices to keep test maintenance low

What is test maintenance?

If you introduce any change to the software, even if it’s a minor change like a browser update, it can cause tests to break. Test maintenance ensures this doesn’t happen. 

So when you build an automated test for your application under test, it’s not always a one-time job. Sometimes the tests break.  

I know. It’s ironic. 

The very tests you built to check for errors break themselves.

The thing is, GUI tests are fragile because of how often a GUI changes. And this creates an expensive maintenance burden. But there is a way to keep these costs low for an ROI of test automation.

What are the main challenges of test maintenance? 

Well, there are a few:

• The system under test can be complex
• Not all testing tools are able to identify objects on a GUI
• The skills required to maintain and build certain test automation are not always available (here’s a post where you can read more about the skills gap issue)
• The effort it takes to maintain test automation can often outweigh the benefit. Here’s why:

If you’re managing a small number of automated test cases, the maintenance costs may not be large. 

But if you’re managing a big test suite spanning different technologies, this work quickly becomes unmanageable.

Let’s say the manual maintenance of every code-based automated test case of a GUI takes 10 minutes. If you have 300 test cases, that’s 50 hours (6.5 working days) of test maintenance per regression cycle. 

That’s what the image below illustrates. The time you save building automation doesn’t always save you time or money compared to the maintenance costs. 

To avoid this happening, you have to reevaluate the tools and practices you carry out for building test automation. 

4 best practices to keep test maintenance low

1. Don’t build automation for tests that aren’t repeatable. If you automate every test under the sun, you’re adding to the inefficiency of your automation. As a rule of thumb, automate the repeatable, high-value tests. If you’re aiming to create a scalable test automation strategy that extends beyond high-value, high-risk tests, you need to rethink the type of tool you’re using. One that’s easier to deploy, build and maintain by both business users and technical users. This creates room for scaling. 
   
   
2. Don’t repeat yourself. If you’re building the same test again and again, turn it into a reusable component. If you don’t do this early on - creating reusable components - you’re causing yourself a major headache later on. Using Leapwork, a visual test automation software, you can turn your tests into reusable components (we call them sub-flows). The maintenance of these tests becomes significantly lower. You only have to update one test, and the changes are automatically pushed out to every test case with that component. This approach makes for more reliable and scalable test automation.
   
   
3. Use visual debugging to find errors. With code-based tools like Selenium, finding a bug requires you to comb through lines and lines of code to find an error.  This task can quickly get out of hand, becoming time-consuming and costly.  With a visual debugger, you don’t have to comb through lines and lines of code to get to the route cause of a broken test. Don’t know what visual debugging is? It’s a capability unique to Leapwork. It shows you an error in three ways. A video recording. An activity log. And data-level insights. 
   
   
4. You don’t need to write code to automate. One of the problems with test maintenance, especially when tests for the GUI are coded, is that the person building the test isn’t always the person maintaining the test. Automation needs to be easy to understand for the person building and maintaining the test – whether or not they are a programmer. Even if you introduce coding standards and naming conventions, you can’t guarantee that the next person to build tests will automate the same way. The automation should be in a language/visual state that is meaningful to the user – not to the application you’re using. This creates less room for confusion.

Dorothy Graham and Mark Fewster from Experiences of Test Automation summarize it well:

“If testers without any programming knowledge can both write and run automated tests, then the automation will be of great use to the business domain and is much more likely to be widely used, further increasing the ROI”. 

To learn more about visual test automation, and how to build both maintainable and scalable test automation, download our guide to building maintainable and scalable test automation. If you’d like to see visual test automation in action, check out our no-code webinar.