Automation insights and productivity tips from LEAPWORK.

All Posts

Manual Testing vs. Test Automation: 10 Considerations

Once a QA team has made the case for automation and is moving from doing all testing manually to introducing automated testing, the change process often comes with some challenges that need to be addressed.

A few examples of common challenges related to automation roll-out are:

  • If the automation of test cases is owned by only a few people, and not everyone on the test team, there is a risk that the ownership of product quality falls between the cracks.
  • If a DevOps team sends out new, “one-size fits all” directions and protocols for software releases, it might result in development and test teams losing equilibrium: They are still accountable for product quality, but they no longer have the power to do something about it.

To increase the chances of success when equipping a QA team with a test automation tool, it’s important to realize that manual testing and test automation complement each other, and that both have strengths and weaknesses. Below are ten aspects to consider when figuring out how your QA team can benefit from test automation.

CONSIDERATION #1: Flexibility in testing

Manual testing comes with the inherent benefit of offering testers full control and flexibility of how to execute single tests. Testing manually allows fast, ad hoc testing and is crucial for signing off new features during a sprint.

Test automation should only be applied to more stable features. This means that automation cases for features are built one or two sprints after the implementation of the features in question.

Read: How to Achieve Agile Testing with Test Automation

CONSIDERATION #2: Regression testing

Manual regression testing is incredibly time-consuming and comes with a range of issues:

  • Uncertainty about whether tests are executed the same way every time.
  • Executing the full regression suite manually creates a bottleneck in the release cycle making it rigid.
  • Due to time constraints, manual regression testing is usually not feasible every time the software changes.

For these reasons and more, regression testing is an ideal target for automation. Automated regression testing comes with following benefits:

  • Robots never sleep. This significantly extends the time window for running the regression suite.
  • Automated cases are executed the same way every time producing credible test results.
  • The time spent building an automated test case is a one-time cost.
  • An automated test case can be reused as many times as needed.

test automation vs manual testingCONSIDERATION #3: Scalability

Manual testing can only be scaled with more people and hours being allocated to the given project. More test cases require more hands, which doesn’t scale well.

Automation enables a high level of scalability as you can simply just add more test executors (i.e. “robots” or “agents”).

CONSIDERATION #4: The human touch in DevOps

DevOps is all about streamlining the release pipeline, and automation plays a key role in this. Test automation, specifically, is completely in line with DevOps principles, for various reasons:

  • Automated execution
  • Tests are triggered as part of a release pipeline
  • The development team is provided with feedback quickly
  • Reporting is highly structured

However, there are elements of the DevOps pipeline that require a human touch:

  • As mentioned earlier, testing of new features in a sprint manually comes with some advantages.
  • Analyzing and managing reasons for why automated test cases fail is still very much a task that requires a human mind.
  • Building automation cases is a manual task.

Read: How to Automate Functional UI Tests in a DevOps World

CONSIDERATION #5: Processes and ownership

When you apply automation to the DevOps pipeline, make sure that testers still have – and feel they have – a say in how the pipeline is defined. Testers are accountable for product quality and they should decide how often builds are pushed to test environments, when automated test should run, and how test results are distributed and formatted.

A DevOps pipeline should support the software development process, not the other way around. In the same way, a test automation platform should support, not dictate, the defined pipeline.

test automations role in software deliveryTest automation supports the DevOps pipeline which in turn
supports the software development process.


A test automation platform should come with plugins for pipeline orchestrators (Jenkins, Bamboo, TFS, etc.) and/or with APIs for executing test cases and to read test results.

Not two organizations or products are the same, so it’s important that a test automation platform has the necessary flexibility to support many types of software development processes.

CONSIDERATION #6: Change management

Introducing agile processes and DevOps is a big change for organizations, and management must equip their teams to being able to handle this transformation.

Testers should have a chance to take a step back and reflect on how things are done today and whether processes can be improved. In a QA team both developers and testers should have a say, and each team member should be empowered to make the necessary changes.

Some test automation platforms are restrictive in the sense that they force testers into one way of working. To avoid this be aware that the testers, not the tool, should decide on the processes related to test automation. This helps ensure ownership in a change process.


Introducing an automation tool comes with the risk that a long learning curve or technical challenges take up too much time and require a lot of support between team members. This takes away time and focus from testers’ primary tasks (building test cases, analyzing requirements, reporting, etc.).

When choosing an automation platform, make sure that the evaluation is done thoroughly. Every member of a QA team should be able to work with it, and the tool should be a good fit in three aspects: technology, processes, and organization.

From a technical perspective, you can use the following requirements for the evaluation:

  • Which applications are you automating? (Web, Desktop, Citrix, other?)
  • Should the automation platform be deployed on-premises or in the cloud, or a combination?
  • Should test cases run locally, on cloud-based services, or both?
  • How well does the platform integrate with the release pipeline?

Read: Getting Started with Test Automation

test automation tool fit

Criteria for evaluating a test automation tool:
Organizational fit, technology fit, and process fit.


The essence of testing is to guard and improve the quality of an application through a series of tasks: Test case definition, requirement analysis, test execution, regression testing, result reporting etc. All these disciplines are still relevant and required when test automation is introduced.

However, it is crucial to ensure that the team can reap the benefits of test automation. Otherwise it’s just another set of tasks added on top of the already existing tasks. Upgrade the skills of the test team through training and make sure to select an automation platform that matches the skills of the team.

When it comes to the skill sets required for test automation, there are basically two different paradigms. One is programming-based requiring testers to code when building automation cases. Automation tools of this paradigm are designed with a developer mindset and assume that testers have a deep understanding of how the underlying technicalities of the system under test.

The other approach is to let testers build automation cases by working with a visual designer. With this method, testers can fully utilize their domain knowledge (see below), without having to worry about an application’s underlying code.

Read: A LEAPWORK guide to reducing risk, lowering costs, and driving value with test automation.

CONSIDERATION #9: Domain knowledge

A deep understanding of an application never goes out of fashion! As mentioned above, the concept of knowing a software well is often confused with having to know the code the software is made of. To be clear:

Programming skills are not, and should not be, a default part of a tester’s skill set, but domain knowledge will always be part of the testing profession – automation or not.

Utilizing automation is based on the tester’s knowledge about an application and how it should be tested. Make sure to nurse and value domain knowledge as this is the basis for good software quality.

Automation success is dependent on the collective knowledge of the entire test team. A robot is not a sentient being – it does what humans instruct it to do. So, implementing automation is about utilizing a team’s collective knowledge in the development of automation cases and making sure that automation becomes part of the daily work routine – in the same way that writing test scripts is a natural part of manual testing.

Read: Will AI Take Over Test Automation?

CONSIDERATION #10: Organization

Test teams are often organized around products or projects for changing products.

With or without automation, this a valid and efficient way of organizing teams. It results in dedicated, relatively static teams with a well-kept pool of domain knowledge, which is exactly what is needed for creating and maintaining a regression suite for an application. Make sure to preserve all processes in which testers are picking up additional domain knowledge about the application in question.

Success with automation requires that the entire test team is on-boarded and receives proper training. Otherwise, the overall responsibility for the product quality is neglected.

All members of a test team play an important role in automation, but not necessarily the same role. For example, testers with some technical proficiency can take on organizing reusable components and working with DevOps to include test automation in the release pipeline. Testers with a high level of domain knowledge can oversee creating and maintaining test cases and monitoring results of automated tests.

technical testers, test management and domain specialist's role in test automationAll members of a test team, incl. domain specialists, technical testers, and test management play an important role in making test automation work.


SUMMARY: The 10 considerations in manual vs. automated testing


Manual testing considerations

Test automation considerations

1. Flexibility in testing

Manual testing allows for fast sign-off of features on an ad hoc basis. But flexibility decreases as the workload increases.

Gradually building automated cases on a sprint-by-sprint basis provides a high degree of flexibility.

2. Regression testing

Manual regression testing is time-consuming, causes uncertainty and creates bottlenecks. Inevitably the regression suite grows to a point at which it can’t be managed manually.

Automated regression tests can be executed 24/7, the same way every time. Building automation cases is a one-time effort, and once built, they can be reused indefinitely.

3. Scalability

Scaling manual testing requires more people and more hours.

Scaling automated testing is done by simply adding more test executioners (robots or agents).

4. The DevOps pipeline

Several aspects of the DevOps pipeline still require human cognitive skills.

Test automation perfectly supports DevOps principles of streamlining the pipeline.

5. Processes and ownership

Testers should have a say in defining the pipeline, esp. the areas related to test automation.

A test automation platform should support—not dictate—your DevOps pipeline.

6. Change management

In a QA team, both developers and testers should be empowered to make the changes required to improve the software development process.

Testers—not the automation tool—should decide on the processes related to test automation.

7. Tooling

Tools with long learning curves and unnecessary complexity will require that a lot of resources are spent on support.

An automation platform should be a good fit in three aspects: technology, processes, and organization.

8. Skills

The most important skill of the tester profession is to utilize one’s domain knowledge of the system under test when defining test cases, analyzing requirements, reporting results, etc.


Programming skills are not—and should not be—a default part of a tester’s skill set.

Some test automation tools require testers to code when building automation cases. Others let testers build automation cases by working with a visual designer. With the latter method, testers can fully utilize their domain knowledge, without having to worry about an application’s underlying code.


9. Domain knowledge

Domain knowledge will always be part of the testing profession – automation or not.

Implementing automation is about utilizing a team’s collective knowledge in the development of automation cases and making sure that automation becomes part of the daily work routine.

10. Organization

Organizing test teams around products results in dedicated, relatively static teams with a well-maintained pool of domain knowledge.

All members of a test team play an important role in automation – but not necessarily the same role. Tasks will vary depending on technical proficiency and level of domain knowledge.


Get the comprehensive guide on the 10 considerations that QA teams should think about when testing their software

Manual Testing vs Test Automation

Kasper Fehrend
Kasper Fehrend
Senior Product Evangelist at LEAPWORK.

Related Posts

How to Overcome Challenges in Cross-Platform Testing

With more businesses on the web, or moving to the web, it’s not often you see new desktop application releases. But the fact remains - many companies rely on a complicated web of desktop applications, web applications and legacy systems to perform business critical processes.

AI in Test Automation: 4 Types of AI that you’ll come across in 2021

AI has, once again, been mentioned as a key technology trend for the year ahead. But what does this mean for test automation? In this post, we break down key terms and explain how QA can benefit from these technologies in 2021 and onwards.

How to Automate Mobile Web Testing with Codeless Selenium

With brick and mortar stores closed during an on-going pandemic, websites, especially e-commerce websites, have to focus more than ever on creating quality customer experiences online. This has created a need for faster testing and new website functionality.