With a recent change from eight to seven service updates a year, testers are relieved from having to test during the holiday season. Even so, there is still a huge testing workload with every service release.
What does this change mean for businesses testing Dynamics? How can an organization ensure that its business-critical processes are working as intended after every update? And how can they avoid long, drawn-out testing that delays releases?
Because Dynamics 365 can be so heavily customized, testing the system becomes more complicated.
As mentioned above, when customizing any system, especially Dynamics365, it has to be tested to make sure that everything runs as intended. Any bug fix or update in a Microsoft system can impact unique configurations from business to business.
A business has to be in a position to accept these updates on a frequent basis. It’s possible to skip two, however it's recommended to follow the regular release cadence because of the complexity involved in the updates.
From a testing perspective, this is a big undertaking as test cases can often hit the thousands, giving a short time frame to plan and validate Dynamics updates.
It can quickly become demanding to identify the impact of these updates and get them tested and validated in a week or ten days. Realistically, this process can take 2-4 weeks when done manually.
Often, Dynamics functions as the backend to a front-end application. Using an example, an eCommerce company could have a website as their front-end application. The backend, which is used to manage everything related to customers, could be on Dynamics 365 CRM.
The probability that there are more applications than Dynamics in an organization's digital ecosystem is high. In the case that a business process extends beyond one application, these processes need to be tested end-to-end to make sure that integrations are functioning.
Automating an end-to-end test with code is complicated because applications can be built using different programming languages or extend across the web and desktop.
For example, if the frontend UI is tested for an account creation process, it could involve applications across web and desktop. You would test that details registered on the front-end have also been written back into the Dynamics CRM system, or (in some cases) a desktop-based CRM system.
You would need a tool that works on the web, on a desktop, and a combination of systems that could be written in different programming languages.
Additionally, automated testing within Dynamics and across their modules isn’t straightforward. A lot of testing takes place in the middle layer of Dynamics from 365 to other systems and logic applications.
Microsoft’s Regression Suite Automation Tool (RSAT) can perform testing within the Finance and Operations (FinOps) module, but outside FinOps, testing integrations with other modules require an alternative testing tool.
There is pressure to drive more agile deliveries for more releases, more updates, and faster release cycles to get better and faster at validating the impact of updates.
However, inefficient testing stops businesses from becoming agile. Manual testing consumes a lot of resources which increases spending and, on top of that, it is less accurate.
As a tester, you can get tired of testing the same processes month after month. Once you are used to how a system works, it is easier to miss or ignore problems that might occur outside of usual testing routes.
On the other hand, automated testing, when code-based, requires expensive developer resources to build and maintain scripts. Because of this, the initial investment in automation can often outweigh the benefits.
There is more than one way to test dynamics, and each approach takes varying degrees of time, energy, and resources.
When there is pressure from the top to become more agile, businesses find that they are continuously bogged down by testing. Manual testing or complicated automation frameworks are often the culprits.
So what do these testing approaches look like, and how can a business increase the speed of their testing without having to use complicated automation frameworks that require a lot of maintenance?
Testing approaches can be segmented into different areas:
Manual testing, while mostly ineffective, is still widely used by organizations. And that’s not necessarily a bad thing. Some environments call for more manual testing than others, for example, if you’re performing exploratory tests, or if you’re testing a legacy system (read here to learn how to actually automate a legacy system).
While there are some tests that cannot be automated, manual testing should be kept to a minimum.
Why? Our brains are not wired to run repetitively. We skip steps unknowingly or enter data wrong. Also, the tester can miss other bugs in the software if they stick to their usual ‘route’. This jeopardizes the quality and assurance of a Dynamics update.
Every time an update is implemented, it has to fulfill a standard. For businesses working in industries where regulatory compliance is particularly important (e.g. BFSI & Pharma), one wrong step can lead to a multi-million dollar mistake, or lead to legal and regulatory compliance issues.
For example, if you have a new release and you’ve made some changes in a pre-production environment and you want to push it out, it can take 2-4 weeks to get manual testers to go through all scenarios that have been mapped out and that edge cases have been covered too.
This is where automation comes in. It’s useful because it empowers people to execute their tasks in a productive way. Employees can focus their energy on more challenging and rewarding tasks, and the business can benefit from a more efficient process, and the ability to run the business more streamlined.
That being said, some automation is more effective than others. Let’s expand on that.
Transitioning from manual to automated testing? Here’s what to look out for.
Many businesses already have experience working with automation using a script-based tool. They’ve built their own framework, or they’ve adopted a commercial or open-sourced tool.
There’s just one problem. The tools are code-based, making them complicated to use if you don’t have a lot of coding experience.
One example is the SysTest framework. It comes with D365 and it is used to unit test the Finance and Operations module. It’s specific to d365 and it’s code-based. In order to automate with this tool, you must be reliant on a d365 developer.
SysTest has a task recorder that can walk you through different business processes, and make building test cases smoother. However, with heavy customizations and regular updates, the test cases require constant maintenance. For example, you’d have to go back and re-record everything when changes are made to the system.
Additionally, it doesn’t give insight into results. You can see if it’s a pass or fail, but you can’t see where it’s failing, why it’s failing, and how to go back and fix the bug.
From an organizational perspective, the number of people that can access and engage with the tool is limited. This creates a perception of automation as a time sink that requires a lot of expensive resources to build and maintain.
That’s not an attractive proposition for a business, and it’s not an attractive proposition for the person working with and building automation.
As a result, automation initiatives are often down-prioritized or canceled altogether.
An automation framework is essentially a recipe for how to build an automated test case. In practice, they can make test automation more complicated than necessary.
It’s costly because they generally require a person with strong programming knowledge and skills to build. While it may be easy for that person to maintain the framework, when they leave, it becomes difficult to maintain the standard of the code if you’re not the person that built it.
In combination with heavy customization in Dynamics, automation frameworks and script-based automation tools require a lot of work to make sure the script-based automation is still functional.
Learn more. Test Automation Frameworks: Why They Fail
In practice, for every release, update, and customization in Dynamics, each test has to be rewritten because locator strategies will change.
With a small number of automated tests, it’s doable. But when your tests run into the thousands, the maintenance completely outweighs the benefit of using automation.
Read more about bringing agility to dynamics testing
Dynamics applications are complicated, especially if they are automated with open-source automation tools like Selenium.
It takes a big initial effort to build locator strategies to find the correct elements in the different Dynamics applications. In WebDriver, this can create extra complexity to the testing process, having to deal with concepts such as “nested iFrames”, “deep object trees”, and “dynamic IDs for child windows” when trying to automate simple processes in the Sales application.
Automating different versions of Dynamics becomes even more complicated. From a testing perspective, the UI paradigms are not the same. This means that the entire markup forming the basis of the web applications changes – even though a great deal of the functionality remains the same.
If tests are automated with Selenium scripts, the codebase needs to be updated and verified. This is incredibly time-consuming and requires specialized resources that may not always be available on a test team. You don’t want to lock in developer resources to do test automation, as this would take time away from their primary tasks, such as developing your local Dynamics customization.
The good news is that testing Dynamics doesn’t have to be a slow and tedious process, and it can be automated with the right solution.
For organizations already dabbling in automation, it’s crucial to be able to spend as little time as possible having to re-do test cases with updates and avoid the maintenance that comes with manually writing scripts.
The tools that are available today are incredibly complicated, and they are based on writing and maintaining a ton of code. This makes the automated tests unusable to the people who need it - test managers, IT operations, business analysts, and product owners.
Testers working with Dynamics applications already possess and gather incredibly valuable information over time. They have in-depth knowledge about Dynamics functions and processes.
There is a way to tap into the potential of these resources, without having to manually test every test case or write and maintain automation test scripts.
The Leapwork Automation Platform uses Selenium as an engine for automating websites and web applications, but all the Selenium code for automating tests is generated ‘under the hood’. The tester does not, at any point, have to worry about reading or writing a single line of code. In fact, the tester doesn’t need to know anything about Selenium at all.
See the full comparison of Selenium vs. Leapwork.
A side-by-side comparison of Leapwork vs. Selenium automation
Instead of working with automation like this:
Example of test script for Dynamics 365
Testers can work with a simplified, visual process like this:
Example of a Leapwork flow doing data-driven automation of Dynamics 365.
The benefits of working with automation as a visual language rather than code are manyfold.
Easier to use. For one, business users can automate. You’re no longer reliant on expensive Dynamics365 developers to build and update complex automation scripts. You can test quicker because you’re no longer reliant on scripting automation. Instead, drag and drop blocks allow you to quickly and intuitively build an automation flow.
Faster validation. Easier and faster than manual. If you’re testing a purchase order/confirmation test, it can take 20 to 30 minutes to test. Within an automation tool like Leapwork, it can take 8 minutes to build, and 1 minute to execute. With the time saved, you can cover more test scenarios. This means your automation can be scaled beyond just 365.
Quick diagnosis. Diagnosing a problem is simpler. See what, where and why a test case returned with an error. With Leapwork, you don’t have to create a test report manually. The test report is automatically generated and uploaded to agile DevOps or any dashboard.
Better quality. With a robot performing the work, you can expect a more coherent and predictable output. It can provide a better audit trail. Because of the removal of human error, you’re less likely to miss bugs you would have otherwise not noticed. Higher worker satisfaction. The hours spent on manual repetitive test cases can be freed to work on more interesting work. This leads to a more productive, more satisfied employee.
Integrated with Azure DevOps. Codeless automation lets you test integrations end-to-end across different systems. Whether it’s a Microsoft-owned application, on-premise or in the cloud, virtual, legacy, desktop, or web environment. With Leapwork’s no-code automation platform, businesses can automate and test all Dynamics 365 web applications, as well as other applications within a companies’ wider IT landscape. Leapwork also integrates with your existing CI/CD pipeline.
The IT landscapes of today’s enterprises are complex constellations. Testing Dynamics 365 shouldn’t add additional complexity.
.svg "Leapwork")
