The C-suite Guide to AI and Test Automation

Maria Homann

Automation Expert

The automation industry has seen a rise in AI capabilities. Artificial intelligence has presented itself as a brilliant way to augment the automation robots’ abilities, and so it is one of the most common requirements for automation tools nowadays.

But what about the applications of AI here and now? How can the CTOs, CIOs, and other executives leverage the technology today? 

Many businesses, small and large, are already using AI to drive real business value and power improvements in customer and employee experience, while others are scoping its potential.

This guide will take you through 5 steps of AI adoption that will increase the likelihood of success and help you determine the potential gains of adopting AI as a part of your test automation strategy:

    1. Start with the problem, not the technology
    2. Find examples of critical customer/employee touch points
    3. Select a pilot project, then scale
    4. Use artificial intelligence to augment human intelligence, not replace it
    5. Set realistic expectations and a timeline

For examples of AI automation use cases, see How to use AI in test automation today - and how not to.

This guide is for you if:

This guide is for you, if you are looking to learn about the potential value of AI in test automation, if you’re building a strategy around this, or if you are in the early stages of AI implementation.

There are many stages of AI maturity, spanning from a novel interest in the technology and project pilots, towards pervasive use and structural AI integration. So before you read this guide, consider your AI maturity level.

1. Start with the problem, not the technology

Every technological investment should start with the problem, e.g. whether it's to reduce internal resource spending, to reduce the number of bugs going live post-release, or to reduce the maintenance of your automation framework.

Technology - AI or not - should be adopted because it solves the problem in the most efficient way and brings actual business value, and not because it’s listed as a technological trend. 

“Take your specific use case, see how you can solve it, and then you find the most suitable technology for that.” - Florin Manole, Director of AI and Analytics at Leapwork

When it comes to AI, in particular, there’s a tendency to be blinded by its future potential and promises, and to forget if there is in fact a more efficient way of solving the problem here and now.

An example of such a case of AI in test automation is voice-assisted test creation using Natural Language Processing (NLP). In theory, this would allow us to tell a computer to automate a test with a simple voice command such as “automate login to SAP”.

But a computer wouldn’t understand such a command, because it cannot infer meaning without more context. 

So in practice, to use voice command for test creation, you would have to be extremely explicit in your commands. The directions would be more along the lines of: “Find SAP icon, double-click SAP icon, find login button, click login button, find username field, enter username” and so on. 

If you’re looking to test several login credentials in one test (which will allow you to scale that test) you’ve got a problem at this point: How will you tell the computer to pull this data and set the properties correctly?

A visual test automation solution would in this case be a quicker, and more efficient way of solving this problem.  


sap test automation flow


2. Find examples of critical customer/employee touch points

When implemented right, AI can augment human capabilities and free up time for your employees to focus on the most value-driving initiatives. 

A simple, yet widely applied use of AI for testers is capturing elements in the UI through Optical Character Recognition (OCR). Many applications can be difficult to test because the underlying code is either inaccessible or difficult to read. Examples of this include Mainframe and Citrix, or other types of legacy and virtual desktop software.

A bank, for example, might need to test that data moves as intended from their Mainframe application with a green screen interface to their customer-facing mobile application. Here, OCR can be used to access the green screen, capture an image, and translate that image into text.

The second step in integrating AI into your IT landscape is therefore to find the areas in your current setup that are critical for the customer/employee experience, where your current IT solution isn’t solving your problems, and to identify where you can enhance or improve these.


ocr test automation flow


3. Select a pilot project, then scale

There are many potential use cases for AI, like the one mentioned above, across industries. List these up front to evaluate if there’s enough to bring you a significant ROI. 

Regardless of how many you find, we recommend that you start with one, and then scale.

By focusing on the pilot, you can gather important insights and identify the resources needed for future scaling in advance, saving you time and resources down the road.

Keep scalability in mind during the pilot - to secure the ROI, the solution needs to be maintainable and scalable. Adoption of solutions that don’t scale is perhaps one of the biggest pitfalls in technology adoption in general.

Consider automation, for example. Many adopters of test automation seek popular open-source solutions such as Selenium to scope the potential of the technology, only to find that it is impossible to maintain due to the amount of coding required. 

Today, we see a rapidly increasing demand for no-code, because it gives businesses a much faster and larger ROI. We must apply these learnings to AI adoption as well.

4. Use artificial intelligence to augment human intelligence, not replace it

Your business is only as valuable as the people operating it. So when it comes to implementing technologies like AI and test automation, the purpose should never be to replace human testers. 

Testers and Quality Assurance personnel have irreplaceable skills. No one knows your business platforms better. AI and test automation are technological opportunities that enable you to use resources in the best way possible for maximum business value.  

A tester shouldn’t be performing tedious, repetitive tasks like regression testing. And a robot shouldn’t be tasked with critical, creative tasks such as exploratory tests.

If you manage to divide the tasks well, you will also conquer. The challenge here is to separate the future potential of AI and current application.

Recommended reading: How to use AI in test automation today - and how not to.

5. Set realistic expectations and a timeline

Being realistic about AI and its use shouldn’t be confused with being unambitious. 

There are many developments in this space that make it an intriguing one to follow, and competitive advantage can be gained by staying ahead of the curve.

But when push comes to shove, very few businesses will benefit from pioneering AI technologies that don’t have proven value.

For the majority of businesses, the race for efficiency gains will be won by implementing maintainable and scalable automation. 

AI will prove its value in the future when tools are more mature. For this reason, organizations seeking to adopt AI need to set realistic timelines that think resources and other priorities into the equation.

Is your organization looking into AI because there’s a hype, or because you have a problem that cannot be solved with your current toolset? 

Learn more about this topic in our webinar on AI and test automation with Florin Manole, Director of AI and Analytics at Leapwork. Watch the webinar here.

New call-to-action