Web automation is the concept of letting software robots perform actions, tasks, and processes that involves a web browser or web application.
Think of all the clicking and typing you do at your desk everyday that involve websites and web-based applications. All these actions can be automated with web automation. Examples of tasks include (inspired by Hacker Noon):
Filling out forms and fields
Scraping content from a web page
Extracting and transferring data between applications
Functional testing of websites and web applications across browsers and devices
And much more
Every day, in businesses of all sizes, there are scenarios where automating tasks like these would come in handy. For example: Importing contacts into a CRM system from a third-party service, testing the user experience and order handling of an online shop, integrating a web-based ERP-system with a document management platform in the cloud, and so on.
When talking about web automation, there is no way around Selenium, the world’s most popular open-source framework for automating browsers. It consists of several components, but here we will cover the widely used Selenium WebDriver. All components of the Selenium suite, are open-source tools for automation of websites and web applications.
WebDriver is an API that can automate any action that takes place in a web browser. It drives browser actions natively, meaning that it interacts with browser elements, like clicking on buttons, typing text in fields, etc., like a real user would.
Creating a Selenium script is a 7-step process as described in SauceLabs’ Selenium introduction:
Below is an example of how a Selenium test script can look like. In this case, the WebDriver is instructed to:
open a Chrome browser;
navigate to the web page amazon.com;
input the term “Lego” in a search field;
wait for the page to load results;
click on one of the items in the search results, and then;
conclude the test case by verifying the presence of a specified image.
As illustrated by this case, a considerable amount of programming proficiency is required to understand, let alone create, a relatively simple script.
The other widely used component of the Selenium products is Selenium Grid.
Selenium is one of the most widely used frameworks for web automation, with good reason. The software’s strengths include (Source: Guru99):
Open source, free to use, and free of charge
Can run cases across different browsers
Supports various operating systems
Supports mobile devices
Can execute cases while the browser is minimized
Can execute cases in parallel
Selenium WebDriver does come with some serious limitations (see table below), when used as a stand-alone tool. Instead, by using a fully-featured automation tool which has incorporated the WebDriver in its technology, like the LEAPWORK Automation Platform, you can enjoy the advantages of WebDriver without having to worry about the tool’s weakness:
See the video below for how a Selenium-based web automation case is built without writing scripts:
Web-based tasks and processes can be broken down into three overall types of actions:
Operating browsers to navigate the web
Interacting with web elements, both static and dynamic elements
Inputting/outputting data, text, and numbers
Imagine if each action and operation listed above was a building block that you could connect to make up visual workflows representing web-based processes. That’s the idea behind LEAPWORK’s Selenium-powered web automation blocks.
The example below automates the following process: Search for an item (“star wars lego”) in an online shop (amazon.com) and then compare the number of search results with a specified value (2500).
The automation flow outlined in this example is created by connecting just seven commands, or building blocks:
This approach of using visual building blocks to design processes enables automation of practically any web application and websites in any of the common browsers: Google Chrome, Mozilla Firefox, Microsoft Edge, and Internet Explorer.
As mentioned above, all web-based tasks and processes involve operating browsers, interacting with web elements, and inputting or outputting data. Examples include: typing credentials into an authentication form, dragging and dropping elements on a web page, uploading a file to a web application, etc. Some operations are more challenging than others to automate – at least, it requires a few more configurations to make a robot perform the operation correctly.
For example, with the LEAPWORK automation blocks, to find a specific image on a web page, you just need a simple “Find Web Element” operation, but to verify an ‘infinite scroll’ functionality on a web page you need to do a minor configuration of the same operation and then customize the strategy for locating a specific element. However, you can still instruct a LEAPWORK robot to perform even this slightly more complex example without having to write a single line of code.
Now that we have covered the ‘bricks’ that make up web automation, let’s look of how web automation is being used in different organizations to solve various challenges.
Web automation is applicable to processes involving any website and web application, both external and internal. Below are some real-life examples of how organizations are benefiting from web automation.
At a major utility company managing critical infrastructure for energy and gas, all service desk requests are managed with ServiceNow, a web-based business process platform.
ServiceNow is a great example of an application that can be automated with the codeless, Selenium-based automation approach outlined above.
Utilizing web automation, the IT Process Team at the utility company could automate the testing of business flows going across functions and departments involving multiple applications, including ServiceNow. This has allowed them to significantly reduce the time spent on testing in relation to system upgrades. This freed up time was then allocated to business-critical tasks, like customer support.
Back-office functions are increasingly being handled with web-based applications – even industry-specific ones, like Policy Management Systems (PMS) in the insurance sector and Transport Management Systems (TMS) at logistics companies. From an automation perspective, this move to web-based technologies simplifies things: Any application that can be operated using a web browser, can be automated with the approach described in this guide.
For example, an insurance company moved from quarterly to monthly releases of their policy management system, by automating the testing of the system based on the web application Pega.
Another example: a global logistics company automated their web-based online booking system which integrates with their TMS, reducing weeks of workload to the few clicks required to instruct robots to perform repetitive tasks.
E-commerce is practically an inevitable part of selling and purchasing products and services today: Digital and mobile storefronts, electronic transfer of funds, web-based inventory systems, and online marketing are just a few of the different aspects of e-commerce.
Automating front- and back-end processes for monitoring and testing purposes can help enhance the robustness and quality of an e-commerce channel. This involves:
ensuring availability of an online shop from an end-user perspective
continuous monitoring of the operational status of the entire channel
CRM systems have great automation potential: They are used for processes that go across departments and functions, they are often web-based, and a lot of the tasks involved are repetitive.
Salesforce, for example, is a top tier CRM solution for enterprises and organizations with popular cloud services for managing sales, marketing, help desk requests and much more. It's highly customizable and can be configured and extended to fit almost any requirement. The size and complexity of Salesforce means that many repeatable tasks are performed manually by sales reps, marketeers and managers over and over every day.
Web automation is not only for public systems. For example, a global manufacturer of medical devices automated a SharePoint-based service portal monitoring devices and systems at thousands of hospitals worldwide.
Web-based automation flows often involve repetitively inputting data, e.g. to fill out a form several times. There are several ways to input data to automation flows, either by using static data sources, like spreadsheets, or dynamic sources, like databases, web services, and command shells.
When it comes to executing web automation flows, there are two major concepts you need to know about:
Hosted environments: Execute automation flows across devices and systems using cloud-based services.
Parallelization: Run automation flows on several machines in parallel
To deliver a great end-user experience, web applications and websites must work across multiple browsers, browser versions, operating systems, and devices, including mobile. With all the possible combinations, the number of usage scenarios to be tested explodes! One provider of hosted environments for automated web testing, Sauce Labs, have identified more than 2,000 devices and more than 800 browser-OS combinations – and those are just the most commonly used.
From a testing perspective, a major challenge presents itself: Is it necessary for your organization to host all the possible combinations of browsers, operating systems, and devices in your own infrastructure to support automated web testing? This would come with additional costs and a massive administrative burden negatively impacting the business case for test automation.
Fortunately, you can outsource the hosting of test environments to vendors such as BrowserStack and Sauce Labs.
What follows is a walk-through of how to use hosted environments for web automation. Besides access to a provider of hosted environments such as setup only requires a few things: A suite of automated test cases ready to be executed, and a service running the test cases, called the Controller.
With this setup, a test team does not have to worry about preparing and maintaining the machines and devices on which the test cases are executed. For example, it is not necessary to purchase, maintain, and charging a bunch of different mobile devices.
Besides using environments hosted by cloud-based providers as described above, there are other ways to parallelize web automation, including execution agents and Selenium Grid.
An execution agent is a piece of software that is installed on every machine where automation cases are to run. Agents allow for a great deal of flexibility, as they can execute automation commands on both local, remote, and virtual machines. Agents can be installed in different and separate environments, including cloud servers, allowing for collaboration across teams and even across organizations, for example if you work with an automation partner.
Selenium Grid is a server for setting up different environments in which automation cases are executed. It is based on a Hub-Node concept and can be set up in your own infrastructure.
The Hub is the central machine running the automation cases, and the Nodes are different machines on which the actual browser automation takes place. The advantage of Selenium Grid is the ability to run multiple cases in parallel across different browsers, operating systems, and machines.
The different options for parallelizing web automation with LEAPWORK are illustrated below. The LEAPWORK Controller is the test runner service, and LEAPWORK Studio is where users design their automated test cases. Learn more about the different deployment options with LEAPWORK.
The two major cloud-based providers of hosted environments, Sauce Labs and BrowserStack, are both supported by the LEAPWORK Automation Platform. This makes it incredibly easy to set up hosted environments for automated web testing.