Test Automation

A manager's guide

Boby Jose

• Publisher: BCS, The Chartered Institute for IT
• Year: 2021
• ISBN:
  9781780175478
• Topic: Computer Science
• Subtopic: Quality Assurance & Testing
• Index: Computer Science

The practice of test automation is invaluable for modern software testers. Done well, it significantly increases software testing's efficiency, effectiveness and coverage and is essential for continuous testing and continuous delivery.This comprehensive guide for managers covers test automation in-depth for medium- to large-scale implementations, from the benefits of test automation to defining, building and implementing a test automation approach that is fit-for-purpose, to acquiring and executing test execution tools and frameworks. Industry examples are included throughout the book, templates and checklists can be found in the Appendix.

PART ONE
THE ‘WHATS’ AND ‘WHYS’ OF TEST AUTOMATION

1 TEST AUTOMATION: A STRATEGIC VIEW

Software testing is the process used to validate that the software solution or product meets requirements and expectations. Software testing also aims to find defects and demonstrate that the software is fit for purpose. There are numerous testing methodologies, types and techniques available to validate the functional and non-functional requirements.

Test automation is the process of using other software for automating manual testing or manual user actions performed in the application. Test automation helps to automate testing tasks to be performed with the help of tools known as test automation tools. It is a good idea to validate the software with the help of other software where manual testing is either not possible or time-consuming.

Automated testing is validating a software solution using a specialised software tool, and typically involves automating functions as part of the testing process.

This book explores the fundamentals of test automation and automated testing. It is written for senior test automation professionals, project managers, product owners, test managers and Scrum masters to assist with decision making in test automation. In this opening chapter, you will be introduced to the strategic view of test automation.

1.1 INTRODUCTION

Automated testing is the use of special software, separate from the software being tested, to control and execute tests, including the comparison and reporting of actual outcomes to the predicted outcome. The application is known as the application under test (AUT) or system under test (SUT), and the software used for testing is known as the automated testing tool (ATT).

Testing is necessary for all IT systems, and there are numerous instances of IT systems that have gone live without appropriate testing and ended up with defects, causing financial and reputational damage. Testing is a core activity in any IT solution development and is independent of the software development life cycle (SDLC) approaches such as DevOps, Lean, Agile or Waterfall.

IT projects and testing are carried out under three constraints: cost, time and scope. These three factors, commonly called ‘the triple constraints’, are represented as a triangle (see Figure 1.1). Any changes to the triple constraints create an associated impact on quality, which is measured through testing.

Testing strategies endeavour to create the optimal quality in the solution.

Figure 1.1 Triple constraints

• Scope. Scope reduction saves testing time; however, it implies that only a core set of product features are tested.
• Time. More time is required for more testing, and this will help to achieve confidence in quality but at an increased cost.
• Cost. Cost is directly proportional to time and scope.

This triple constraint is regularly visited from day one of any IT project due to delay of upstream dependencies, and test managers are often under pressure to shorten ‘time’. Risk-based testing (RBT) is one of the widely used approaches to optimise testing time and effort, test automation is another. Risk-based testing and test automation can be combined.

Risk-based testing is a good solution for managing quality risks due to the reduced time it takes. Test effort allocation based on the risk of failure is one of the efficient and effective ways to optimise testing. One of the key benefits of the risk-based test approach is ensuring that the maximum value is derived from the planned testing activities, even when the time available to complete test execution is reduced, by the late delivery of the solution, environment or resource availability problems and so on.

Risk-based testing is not about risk and issue management per se – it is about managing quality risks – but builds upon these processes to help identify critical areas that require testing focus. All testing activities need to be prioritised to ensure that the most important tests are completed early so that if time available for testing is reduced, the most critical tests can be completed.

A few methods to manage the triple constraints with minimal impact on quality and testing are listed here:

• Start testing or test preparation early, for example informal testing prior to the planned and scheduled testing.
• Involve the testing team from the beginning as part of the business case development or the design phase.
• Reduce duplicate testing such as common tests in system testing (ST) and user acceptance testing (UAT).
• Join up or merge testing cycles, for example the last cycle of system integration testing (SIT) and the first cycle of UAT.
• Introduce the quality assurance (QA) process as a proactive measure.
• Use testing techniques like exploratory testing to find defects.
• Introduce test automation.

Test automation is a software development project that includes most of the phases in the software development life cycle. Test automation is widely enhanced by test automation frameworks.

A test automation framework is a programming framework that includes a comprehensive set of guidelines to produce beneficial results from the test automation activity. An automation framework is either provided by the ATT or, in some cases, a customised tool, which manages test automation in order to produce a better outcome. A test automation framework generally provides a structure to automation tools that fit its purpose. Most test automation tools provide a default framework for automating the SUT; however, the automation tools can be customised for specific requirements, for example an automation framework that schedules various testing tasks as best fit a specific need, generating custom test reports. Test automation frameworks will be explained in detail through the later chapters of this book.

There are numerous ways to make software testing efficient, and test automation tops the list.

Test automation is a key approach in reducing testing effort, but it is not the panacea to all testing activities. Test automation can be introduced at different stages and phases of the test cycle, such as:

• Product development or solution implementation
• Test management
• Functional and regression testing
• Support (post-live) or run
• Test generation
• Test data generation
• Inspection and evaluation of test results
• Compliance

Test automation is widely used across various industries and applications, and it produces great results. Software development approaches such as DevOps, Agile, Waterfall and their different flavours widely use test automation to reduce cost, increase efficiency and accuracy, and speed up regression testing. Test automation is widely used to reduce cost in the long run. Figure 1.2 demonstrates the cost reduction by using test automation over time.

Figure 1.2 Manual testing versus test automation

Test automation is a strategic decision that is taken based on sufficient data and analysis. However, many projects and organisations approach test automation as simply a method for cost reduction and end up failing miserably. It is not a short-cut...