Maria Homann
AI has been making its way into the automation space for a few years now, and businesses are increasingly looking to leverage the smart technology to enhance efficiency gains.
In test automation, Narrow AI exists already and contributes with real business value in several ways. In this article, we cover two types of AI that are often discussed in connection with test automation: Optical Character Recognition (OCR) and Natural Language Processing (NLP).
While one of these can be used to your advantage to optimize processes and drive efficiency, the other type isn’t quite there yet.
Optical character recognition (OCR)
Optical character recognition (OCR) lets bots recognize images and convert them to readable text. This technology allows you to validate visual elements and ensure that the UI appears as intended to the users.
Here are three use cases for OCR:
1. Validating elements on webshops
If you’re testing a web application, you will want to check that users can access their shopping basket. The basket button is probably an image of a cart or basket, so to test that button, you use OCR to capture and find it, as opposed to identifying it as an element in the html code.
2. Testing legacy software or virtual desktops
Another use case is testing legacy software or virtual desktops such as Mainframe and Citrix. In these instances, you might not be able to read the application structure or you don’t have access to the underlying metadata, making testing difficult. But OCR can be used to identify elements on e.g. greenscreens and Citrix interfaces making it possible to access elements for testing.
3. Testing desktop applications
A third example where OCR can be used is desktop application testing. Because OCR allows you to find elements on any interface, you can use the technology to access desktop applications that are difficult to test because they use an inaccessible application structure. Because OCR can convert images to text, you can use it for something as simple as capturing a name or number as an image in a CRM system and transferring it as text into Excel.
As AI technology is improving, automation tools are becoming better at recognizing elements despite small changes in e.g. color and size. This will pave the way for even more use cases and less maintenance of tests in the future.
Natural language processing (NLP)
Natural Language Processing (NLP) lets bots interpret human speech. There are two elements to this: First is the speech to text translation, and next is the processing of that text. This technology is used in chatbots, for example, where a bot is programmed to reply automatically to customer requests.
Learn how to build your own email classification robot with NLP in this step-by-step guide
This technology is useful for robotic process automation - but not yet for test automation.
Why? Because it’s not advanced enough yet for it to be helpful.
“Voice assisted tests are a relatively new area within automation. It is still debated whether this area of automation will pick up in the future.” - Florin Manole, Director of AI and Analytics at Leapwork
In theory, NLP can be used to follow a spoken command such as “Automate the creation and publication of an insurance policy”, which might be useful for an insurance company publishing policies for customers daily.
Unlike a human, a computer wouldn’t know what that means, so you would need to be more explicit. For example: “Log in to the front office application, read this excel file, create a new policy,” ect.
For a narrow AI, the type of AI we have today, such a command would have to be much more detailed for the computer to follow. The login process would in and of itself be quite complex to explain: “Find application X, click application X, find text field, enter username, enter password.” - even at this point it’s already getting complicated; if you want to pull the username and password data from an excel sheet and parameterize your test, how will you explain that to the AI?
At the end of the day, this test is easier to create with clicks on a screen than through speech.
“Voice assisted tests using natural language processing are still somewhat limited in terms of what benefits it can bring to an enterprise. This is because:
- It is not easy to define a test case in simple words
- It requires fine tuning - for example - when logging in with your username and password, it would require login information, and you would need to set five or ten different properties for this test to run properly.
- It requires time for a human to pass on information to the engine to perform an activity like logging into an ERP or CRM system.
- It is only applicable to narrow and simple test cases. When you have complex test cases that form hundreds of elements, it will not be possible to create these tests in a meaningful way.”
- Florin Manole, Director of AI and Analytics at Leapwork
Codeless automation solutions make test creation fast and easy to create and maintain, and are therefore the preferred solution today for test automation.
Smaller uses of AI for test creation are seen in some tools, for example, the suggestion of the next step in a test case, similar to the suggestion of your next word in a text message. But even this isn’t necessarily a useful feature - tools that enable you to reuse structures, such as the login process, can be just as helpful and let you save time on creation and maintenance of tests.
Want to learn more about how to use AI in test automation? Sign up for our webinar.
