Skip to content

How Mature is Your Testing Strategy?

Anna Thorsen

Anna Thorsen

Exactly how mature is your testing strategy? This can be difficult to determine, especially if you are in the early stages of your journey.

Whether you’re only dabbling in testing, or further on in your journey and already using test automation, this post is for you.

Related reading: What is test maturity?

We’ve compiled a short overview of the key characteristics of each level of maturity.

By taking the Test Maturity Model and our homegrown version, you’ll be able to categorize where you’re at, and how you can progress.

What is your testing strategy maturity?

There are lots of ways to define your test maturity level. The most common is the Test Maturity Model (TMM), created by the Illinois Institute of Technology (see below).

You can determine the maturity of your testing by following TMM. It lets you set a benchmark for where you need to be.

Stage

Objective

Current state

Initial

No structured processes
Ad hoc testing

Tests will follow no clear structure. It's likely there are no dedicated testers. Few or no processes and standards are in place.

The quality of software will be dependent on the person carrying it out.

How to improve: a manager should be assigned to create common testing processes and standards.

Managed

Test policy and strategy
Test planning
Test monitoring and control
Test design and execution
Test environment

Testing will be rudimentary. However, basic structures for testing will be in place.

Quality assurance will start to be worked into the everyday. However, new practices will often be abandoned because of high workloads.

How to improve: management needs a firm commitment to project management processes.

Defined

Test organization
Test training program
Test lifecycle and integration
Non-functional testing
Peer reviews

Testing becomes more organized. Those carrying it out are becoming committed to following processes. The right skills to get the job done are being identified.

Planning and quality assurance will be thought into all testing initiatives. The main focus becomes standardizing tests, integrations with the development pipeline, and documenting results.

How to improve: management needs to commit to running quality testing, and project management, even under time pressure.

Measured

Test measurement
Software quality evaluation
Advanced peer interviews

The testing performance is starting to be measured. Processes are adapting based on the outcome of measurements.

Approaches to testing may become more sophisticated, and external tools to aid testing may be considered.

How to improve: understanding and committing to increased productivity, cost reduction, and customer satisfaction.

Optimized

Defect prevention
Test process optimization
Quality control

Here, it is about continuously improving processes. Businesses may begin looking into innovating their practices with the help of new tools and technologies.

This level can be difficult to sustain and requires a strong toolset to maintain.

How to improve: benchmark the business against competition and continuously innovating your processes.


Digging deeper into test automation maturity

The maturity of your testing can be simplified even more. By taking a closer look at the stages above, we can categorize the type of testing a user is carrying out into four buckets.

These are:

No testing

Quality status: bad quality
Software risk: extremely high

By not testing your software (whether for internal or external software), you will have excessive costs, delayed product launches, a bad reputation, and dissatisfied users.

How to improve: improve quality with testing

Manual testing

Quality status: improving quality
Software risk: very high

Only carrying out manual testing takes more time and resources. It’s a less accurate method for testing that gives poor coverage.

How to improve: continue to increase quality with more efficient processes (e.g. automation)

Script-based automation

Quality status: improving resource constraints
Software risk: medium

At this stage, resource constraints have improved. The risk of costly bugs has gone down. However, code-based automation may open up new resource problems. It requires specialist resources and heavy maintenance both to use and to keep the automation running. This makes it difficult to scale.

How to improve: continue to improve resource constraints by reconsidering tooling, and becoming less reliant on tools that require technical resources.

Scalable and maintainable automation

Quality status: reconsider tooling
Software risk: low

Cost savings become visible and risk is significantly reduced. Productivity increases with the toolset and technologies you’re using.

How to improve: continue to adopt agile practices that reduce the maintenance burden. Adopt tooling that breaks down the barrier between machines and people.

 

So what do these stages mean for you and how can you build scalable and maintainable testing practices? We’ve put together a detailed guide on building scalable and maintainable test automation to help you on your journey.

New call-to-action