In this section, you will be introduced to automated testing. We will discuss why you would want to use it, what it exactly entails, and when you should use it.
I did finally get an agreement on writing a book on application test automationâone I had wanted to write for years already, as automated testing doesn't appear to be a love at first sight topic for many. And, like testing in general, in many implementations, it tends to be subordinate to requirements specifications and application coding, be it a project or a product. And, hell, who really loves testing? It is not typically what the average developer gets enthusiastic about. When raising the question during many of my workshops on automated testing, it often even takes a while for functional consultants to raise their hand.
Inevitably, I did pose the question to myself: do I love testing? And I answered it with a yes. A big YES. This subsequently led to additional questions, such as: what makes me love testing? Did I always love testing? Why do I love it while the rest of the world seems not to? Questions with answers that makes me go around the world evangelizing test automation, stubbornly sharing my findings, and pushing Microsoft to improve their tests to make them better and reusable. And all because I reckon that it is funâbig fun!
Having been an app tester in the former Dynamics NAV Global Development Localization (GDL) team at Microsoft, I surely got exposed to the testing virus. You could say that I had to learn to do the testing job, as I was paid for it. But it also suited me well, with me apparently having this specific DNA kind of thing that makes testers what testers are. Having a pride in breaking the thing and, meanwhile, loving to prove, hopefully, its robustness. And, not in the least, daring to break it, running the risk that your developer will no longer like you.
One afternoon at Microsoft, my developer team member walked into our office and stopped next to my desk. Him being very tall and me sitting, I had to turn my head up to look at him.
"What's up?" I asked.
"Don't you like me anymore?" he responded.
Me: ?
Him: "Don't you like me anymore?"
Me: "Nope, still like you. Why?"
Him: "You rejected my code; don't you like me anymore?"
Me: "Dude, I still like you, but concerning your code, my tests show it somehow useless."
Testing is not rocket science. Nor is automated testing. It's just another learnable skill. From a developer's perspective, however, it requires a change of mindset to write code with a totally different purpose than you are used to. And we all know that change is often not the easiest thing to achieve. Because of this, it's not unusual for attendees at my workshops to get to a certain level of frustration.
Application testing is a mindset and it needs a big dose of discipline tooâthe discipline to do what needs to be done: to verify that the feature was built right; to verify that the feature under test meets the requirementsâand the discipline when bugs are reported and fixed to execute the whole test run again, and again, with any new bug, to ensure that the verification is complete.
I tend to see myself as a disciplined professional. I have been quite a disciplined tester, with a high rate of bug reporting. But, did I always love testing? You know, in those days, all our tests were executed manually, and with each bug I found my discipline was challenged in some way. Imagine my mind running when executing the fifth test run after another bug fix. It's 4:00 PM and I am almost done. At breakfast, I promised my wife that I would be home on time for whatever reason. Let's pick a random one: our wedding anniversary. So, me being a disciplined tester, my promise to be home on time, 4:00 PM, and ⊠I ⊠hit ⊠another ⊠bug. Knowing that fixing it and re-running the tests will take at least a couple hours; how do you think my mind is running? Right: binary.
I had two options:
- Reporting the bug would keep me at work and make my wife highly disappointed, to say the least
- Not reporting the bug would get me home on time and save a lot of hassle at home
Had automated tests been in place, the choice would have been quite simple: the first option, resulting in no hassle at home and no hassle at work.
In this chapter, we will discuss the following topics:
- Why automated testing?
- When to use automated testing.
- What is automated testing?
If you prefer to read the what first, you might first want to jump to What is automated testing?
Plainly said: it all boils down to saving you a lot of hassle. There are no emotions or time-intensive execution will keep you from (re)running tests. It's just a matter of pushing the button and getting the tests carried out. This is reproducible, fast, and objective.
To clarify, automated tests are the following things:
- Easy to reproduce
- Fast to execute
- Objective in its reporting
If it was as straightforward as that, then why haven't we been doing this in the Dynamics 365 Business Central world all those years? You probably can come up with a relevant number of arguments yourself, of which the most prominent might be: we do not have time for that. Or maybe: who's going to pay for that?
Before elaborating on any arguments, pro or con, let me create a more complete why not? list. Let's call it the whys of non-automated testing:
- Costs are too high and will make us uncompetitive.
- Dynamics 365 Business Central platform does not enable it.
- Customers do the testing, so why should we bother?
- Who's going to write the test code? We already have a hard time finding people.
- Our everyday business does not leave room to add a new discipline.
- There are too many different projects to allow for test automation.
- Microsoft has tests automated, and still, Dynamics 365 Business Central is not bug free.
Dynamics 365 Business Central is not as simple as it used to be way back when it was called Navigator, Navision Financials, or Microsoft Business SolutionsâNavision. And the world around us isn't as one fold either. Our development practices are becoming more formal, and, with this, the call for testing automation is pressing on us for almost the same reasons as the why of non-automated testing:
- Drive testing upstream and save costs
- Dynamics 365 Business Central platform enables test automation
- Relying on customers to do the testing isn't a great idea
- Having a hard time finding peopleâstart automating your tests
- Test automation will free up time for everyday business
- Keep on handling different projects because of test automation
- Automated tests are code too
Regarding the costs, I am inclined to say that, on average, the Dynamics 365 Business Central project goes 25% over budget in the end, mainly due to after go-live bug fixing. I am not going to elaborate much on who's to pay for this, but my experiences are that it's often the implementation partner. The math is quite simple if assuming that to be the case. If you're spending 25% extra on your own account at the end of the line, why not push it upstream and spend it during the development phase on automated testing, building up a reusable collateral?
During my time at Microsoft in the 2000s, research had been performed on the cost of catching a bug in the different stages of developing a major release of a product. If my memory is not failing, the cost of catching a bug after release was found to be approximately 1,000 times higher than when catching the bug already at requirement specification.
Translating this to the world of an independent software vendor (ISV), this might roughly be a factor 10 lower. So, the cost of catching a bug all the way downstream would be 100 times higher than all the way upstream. In case of a value-added reseller (VAR) doing one-off projects, this could be another factor of 10 lower. Whatever the factors would be, any spending upstream is more cost effective than downstream, be it more formalized testing, better app coding, code inspection, or more detailed requirements specifying.
Note that people often do correct me, saying that the percentage of 25% is even on the low side.
In all honesty, this is a no-brainer, as this is the topic of this book. But it is worthwhile realizing that the testability framework inside the platform has been there ever since version 2009 SP1, released in the summer of 2009. So, for over nine years the platform has enabled us to build automated tests. Does it sound strange if I say that we have been sleeping for all that time? At least, most of us.
I agree that customers might know their features the best, and as such, are the presumable testers. But can you rely 100% that testing isn't squeezed between deadlines of an implementation, and, in addition, between deadlines of their daily work? And still, in what way does their testing contribute to a more effective test effort in the future? How structured and reproducible will it be?
Posing these questions answers them already. It isn't a great idea, in general, to rely on customers testing if you want to improve your development practices. Having said that, this doesn't mean that...