Book demo
Start trial
Book demo
Start trial


Automation insights and productivity tips from LEAPWORK.

All Posts

Test Automation Frameworks: Why They Fail

Test automation frameworks are basically recipes for how to build automated test cases. They include guidelines for testers on how they should do their job. The problem is, these frameworks don’t always work, as they tend to complicate test automation more than necessary.

When QA teams prepare for an implementation of automated testing, it often involves research on how to build frameworks and then trying to adopt these practices to the specific business-setting. It’s easy to be discouraged already in the research phase. Concepts like ‘scaffolding’, ‘coding standards’, and ‘meta-programming’ don’t necessarily help testers define guidelines for their daily work.

The problem with test automation frameworks

The idea of test automation frameworks rests on great intentions. They exist, at least in theory, to ensure benefits of automation, such as: Re-usability of code, higher coverage, low-cost maintenance, and easy reporting. However, when reality hits, frameworks don’t really work. Here’s why:

Frameworks don’t match the pace of development

Test automation frameworks are often supposed to support current and future software releases, but in reality, it’s impossible to match the pace of a development team, who produce new code continuously. The problem is, it takes a lot of time to implement code-based tests along with changes in the framework, and during sprints, there’s no time to implement automation while also taking care of testing. What usually happens is that, over time, testers come up with short-term fixes to the framework, which inevitably leads to frameworks starting to fall apart. And without a solid foundation, test cases stop working too. Centers of Excellence are then tasked with developing a “better, more robust, and future-proof” framework using a new technology – probably headed for the same disastrous end as the previous one.

It easily takes a year for a Center of Excellence to develop a first attempt at a framework. In the meantime, organizational conditions might have changed. For example: Major overhauls of the system under test and changes to the software release pipeline. There’s no way that you can program a framework that is flexible and adaptive enough to prepare for all these factors.

Frameworks are costly and take time away from testing

Script-based frameworks require development and maintenance. Skilled test automaton specialists who are also great at professional-level programming are difficult to find. This means QA teams can do one of two things: Either rely on costly developer resources, whose primary responsibilities lie elsewhere, or assign the task to (technical) testers who don’t have years of programmer experience, which would take time away from testers’ most important task: Testing!

Frameworks are difficult to scale and are limited in scope

Custom automation frameworks are usually project-specific and not geared for handling different types of applications. Most test automation frameworks are usually built for just one of the following application types: Web, desktop, or virtual, which doesn’t allow testers to use the same framework to test different applications across the organization.

Frameworks are person-dependent and have a short lifespan

Frameworks are often introduced by a single person in the organization, and they evolve around that same person as well. This means that frameworks are often incomplete and difficult to understand for other people who need to work with them. When the inventor of a custom framework changes jobs, or leaves the organization for other reasons, the framework is at risk of dying due to lack of proper hand-over, the use of sub-standard coding practices, and difficulties of deciphering the automated processes through code.

Overcome framework challenges with codeless UI automation

A testing tool is for the tester, what the spreadsheet application is for the finance manager. Does the finance manager in your organization ask programmers to write a script for how to work with her spreadsheet? Does she go through a long checklist each time she creates a new spreadsheet? Probably not.

Preferably, the tools you work with are designed in such a way, that by performing standard tasks and procedures in those tools, you are implicitly following best practices inherent to the application.

Automation frameworks are paradoxical: They are intended to hide code and complexity from the tester, but they still dictate that the tester should code. It takes another layer of abstraction to completely hide unnecessary complexity.

Codeless UI automation enables a level of agility that frameworks don’t. When all members of a QA team can take ownership of and collaborate on test automation, testing becomes agile and can match the pace of development. The maintenance workload of automated tests built with codeless UI automation is only a fraction of the workload related to script-based tests.

The benefits of codeless test automation

It is possible to achieve the intended benefits of automation, but not through code-based frameworks. It requires a codeless UI automation solution, like the LEAPWORK Automation Platform, that enables users to:

  • Match the pace of development by not wasting time on complex coding practices
  • Scale automation and lower maintenance workload by reducing dependency on developers and by relying on easy-to-design reusable components
  • Extend the scope of automation by testing across applications with one solution
  • Improve automation adoption by enabling collaboration on visual workflows
  • Easily document processes and changes with auto-generated logs and recordings
  • And much more.

For more details, check out our post: "7 Benefits of Codeless Test Automation”.

Test automation for more, meaningful work

Learn more about codeless test automation in the LEAPWORK guide to reducing risk, lowering costs, and driving value with test automation.CTA-StartTrial-2

Praveen Kumar
Praveen Kumar
Automation architect, test manager, and CI/CD expert.

Related Posts

How Does Behavior-Driven Development (BDD) Support Agile Development?

Behavior-Driven Development (BDD) was carefully developed to support agile development in the software industry. This is because agile cannot be fully deployed if the testing process still follows a traditional methodology.

Functional UI Testing Whitepaper: An Introduction to Codeless Test Automation

Faced with frequent product releases and system upgrades, there is increasing pressure on the resources used in software testing. One small change or upgrade can affect the whole software within seconds meaning that, if the functionality is not being continuously tested as per design, a single bug could have a tremendous effect on user experience.

Traditional Development vs. Behavior-Driven Development (BDD)

  Behavior Driven Development (BDD) is no new concept in the software development industry. Still, many companies don’t fully understand the benefits of this framework in the development process. In a world where traditional development methodologies have excluded team members in silos, BDD serves as a collaborative force that fosters agile development.