Book demo
Start trial
Book demo
Start trial

The LEAP

Automation insights and productivity tips from LEAPWORK.

All Posts

Test Automation Frameworks: Why They Fail

Test automation frameworks are basically recipes for how to build automated test cases. They include guidelines for testers on how they should do their job. The problem is, these frameworks don’t always work, as they tend to complicate test automation more than necessary.

When QA teams prepare for an implementation of automated testing, it often involves research on how to build frameworks and then trying to adopt these practices to the specific business-setting. It’s easy to be discouraged already in the research phase. Concepts like ‘scaffolding’, ‘coding standards’, and ‘meta-programming’ don’t necessarily help testers define guidelines for their daily work.

The problem with test automation frameworks

The idea of test automation frameworks rests on great intentions. They exist, at least in theory, to ensure benefits of automation, such as: Re-usability of code, higher coverage, low-cost maintenance, and easy reporting. However, when reality hits, frameworks don’t really work. Here’s why:

Frameworks don’t match the pace of development

Test automation frameworks are often supposed to support current and future software releases, but in reality, it’s impossible to match the pace of a development team, who produce new code continuously. The problem is, it takes a lot of time to implement code-based tests along with changes in the framework, and during sprints, there’s no time to implement automation while also taking care of testing. What usually happens is that, over time, testers come up with short-term fixes to the framework, which inevitably leads to frameworks starting to fall apart. And without a solid foundation, test cases stop working too. Centers of Excellence are then tasked with developing a “better, more robust, and future-proof” framework using a new technology – probably headed for the same disastrous end as the previous one.

It easily takes a year for a Center of Excellence to develop a first attempt at a framework. In the meantime, organizational conditions might have changed. For example: Major overhauls of the system under test and changes to the software release pipeline. There’s no way that you can program a framework that is flexible and adaptive enough to prepare for all these factors.

Frameworks are costly and takes time away from testing

Script-based frameworks require development and maintenance. Skilled test automaton specialists who are also great at professional-level programming are difficult to find. This means QA teams can do one of two things: Either rely on costly developer resources, whose primary responsibilities lie elsewhere, or assign the task to (technical) testers who don’t have years of programmer experience, which would take time away from testers’ most important task: Testing!

Frameworks are difficult to scale and are limited in scope

Custom automation frameworks are usually project-specific and not geared for handling different types of applications. Most test automation frameworks are usually built for just one of the following application types: Web, desktop, or virtual, which doesn’t allow testers to use the same framework to test different applications across the organization.

Frameworks are person-dependent and have a short lifespan

Frameworks are often introduced by a single person in the organization, and they evolve around that same person as well. This means that frameworks are often incomplete and difficult to understand for other people who need to work with them. When the inventor of a custom framework changes jobs, or leaves the organization for other reasons, the framework is at risk of dying due to lack of proper hand-over, the use of sub-standard coding practices, and difficulties of deciphering the automated processes through code.

Overcome framework challenges with codeless UI automation

A testing tool is for the tester, what the spreadsheet application is for the finance manager. Does the finance manager in your organization ask programmers to write a script for how to work with her spreadsheet? Does she go through a long checklist each time she creates a new spreadsheet? Probably not.

Preferably, the tools you work with are designed in such a way, that by performing standard tasks and procedures in those tools, you are implicitly following best practices inherent to the application.

Automation frameworks are paradoxical: They are intended to hide code and complexity from the tester, but they still dictate that the tester should code. It takes another layer of abstraction to completely hide unnecessary complexity.

Codeless UI automation enables a level of agility that frameworks don’t. When all members of a QA team can take ownership of and collaborate on test automation, testing becomes agile and can match the pace of development. The maintenance workload of automated tests built with codeless UI automation is only a fraction of the workload related to script-based tests.

The benefits of codeless test automation

It is possible to achieve the intended benefits of automation, but not through code-based frameworks. It requires a codeless UI automation solution, like the LEAPWORK Automation Platform, that enables users to:

  • Match the pace of development by not wasting time on complex coding practices
  • Scale automation and lower maintenance workload by reducing dependency on developers and by relying on easy-to-design reusable components
  • Extend the scope of automation by testing across applications with one solution
  • Improve automation adoption by enabling collaboration on visual workflows
  • Easily document processes and changes with auto-generated logs and recordings
  • And much more.

For more details, check out our post: "7 Benefits of Codeless Test Automation”.

Test automation for more, meaningful work

Learn more about codeless test automation in the LEAPWORK guide to reducing risk, lowering costs, and driving value with test automation.

 

Read the guide: "Test Automation: A Winning Game Plan"

Praveen Kumar
Praveen Kumar
Automation architect, test manager, and CI/CD expert.

Related Posts

The Critical Role of Continuous Testing in Continuous Delivery

To keep up with the ever-increasing customer expectations to digital products and services, software providers are under pressure to be more and more thorough in their quality assurance of products. Learn how Continuous Testing can support those efforts.

Comparison: Selenium IDE vs. LEAPWORK

Selenium IDE is a great open source tool for HTML-savvy users who want record and playback automation of simple web applications, but isn’t suitable for enterprises and has very limited functionality. This post compares Selenium IDE with LEAPWORK enterprise web automation.

Comparison: LEAPWORK vs. Selenium Web Automation

Selenium is a great tool for professional programmers who want to automate web applications, but for users who don’t (want to) write code, there are better options. This post compares code-based Selenium with LEAPWORK web automation. Side-by-side video comparison included.