Maria Homann
AI has, once again, been listed as a key technology trend for the year ahead. But what does this mean for test automation? In this post, we break down key terms and explain how QA can benefit from these technologies in 2022 and onwards.
Artificial Intelligence (AI) has gained a lot of traction in recent years due to its promise to allow organizations to operate in smarter, more efficient ways.
And it can, in fact, deliver on those promises. To many businesses it already does. The use of AI today, is, however, less about supercomputers replacing human workers in strategic decision-making, and more about contributing to processes with human-directed algorithms.
The type of AI used today is most often what we know as “Narrow AI”.
Narrow AI can be defined as tightly scoped machine learning solutions designed to perform a specific task. You don’t need a supercomputer to run these applications. Most of them can be hosted on a cloud server or even on your own laptop.
In test automation, such applications of Narrow AI already exist and are in use. Below are four examples.
1. Intelligent capturing of screen elements
Intelligent capturing means robots are programmed to capture information from sources with essentially no human intervention.
To achieve intelligent capturing, humans must pre-train algorithms to create capturing “strategies” for software robots. The robots use these strategies as guidance for how to find UI elements used in automation flows, such as buttons, text and text fields.
Intelligent capturing algorithms can be combined with visual recognition technology to create smart recording. This is a functionality that finds elements via visual recognition technology and captures them via intelligent capturing.
With the help of smart recording, sequential actions can be captured via the UI for automation. This is, to many, a much easier and more practical way of creating UI automation flows than, for example, writing automation scripts manually.
2. Self-healing, cross-technology automation flows
When applying intelligent strategies to automation, the execution becomes self-corrective, or self-healing. This reduces the burden of maintaining automation, and can be particularly helpful for businesses with large test automation suites, and with applications that undergo frequent change.
For example, if a button or other UI element changes appearance, a self-healing robot can correct itself to find the element regardless.
This is also useful for locating UI elements across technologies. If an element captured using visual recognition is mapped to other ways of identifying the same element in an application, a robot can rely on different technologies to find and interact with that element.
3. Intelligent text recognition
With trained algorithms, like the ones provided by ABBYY, it is possible to recognize text and images from screen pixels. Intelligent text and image recognition can be a real life saver in automation in general, as it lets you capture elements when other methods fall short.
This is useful for automation of virtual environments, such as Citrix applications and graphical-heavy applications used for games, 3D production, broadcasting, and more.
4. Natural language processing and sentiment analysis
A common application of AI is the processing of large amounts of texts or natural language, e.g. chat messages, to do sentiment analysis. This is possible with leading AI cloud services such as IBM Watson, Google AI, and Microsoft Azure ML.
With data-driven automation, you can feed information from AI cloud services to and into automation flows. When executing these flows, software robots can choose which tasks to perform based on e.g. the outcome of a sentiment analysis.
Separately, these examples of narrow AI might not seem like groundbreaking applications. But in combination, they are a great help when designing software automation. With ML-driven automation in their toolbox, testers can wield the power of AI without technical training.
Learn more about AI and its application in automation in our guide.
