Practical Test Design

Selection of traditional and automated test design techniques

István Forgács, Attila Kovács

• Publisher: BCS, The Chartered Institute for IT
• Year: 2019
• ISBN:
  9781780174747
• Topic: Computer Science
• Subtopic: Quality Assurance & Testing
• Index: Computer Science

Reliable test design is important in software testing; without it, defects in software may remain undetected, making the software poor quality and causing dissatisfaction among users.This book presents the key test design techniques, in line with ISTQB, and explains when and how to use them, including in combination, with practical, real-life examples. Automated test design methods are also discussed. Tips and exercises are included throughout the book, allowing you to test your knowledge as you progress.

1 INTRODUCTION

WHY IS THIS CHAPTER WORTH READING?

This introductory chapter explains the importance of software testing, and covers what test design is, how we can put test design activities into the software development and testing life cycle, why tests should be designed, what factors affect test design and when test design should be done.

THE IMPORTANCE OF SOFTWARE TESTING

Software is a vital part of our everyday life. It helps in navigating to destinations, communicating with people, driving production, distributing energy resources, taking care of people’s health, and so much more. We use software for entertainment, for scientific research, for driving companies forward. There is embedded software in cars, trucks, locomotives, aeroplanes, mobile phones, electronic equipment and so on. Controlling and ensuring the quality of these software systems is vital. We need to check, that is test, software because things can always go wrong – humans make mistakes all the time. Despite this, the human aspects of producing quality software are indisputable even if artificial intelligence comes to the fore.

ISTQB certifies software testers worldwide; there are books, articles, documents, webinars, and blogs supporting the knowledge transfer of testing. In this book, we mainly draw on the terms of the ISTQB Glossary (2018a).

Let’s list a few goals that should be considered while testing software systems:

• ensuring the overall quality of the systems;

• customer satisfaction;

• reducing the risk of failures;

• cost-effective maintenance.

The most important goal of testing is to ensure good quality software by optimising the project costs in a way that all the parties involved gain confidence in the product – see Chapter 3. To be able to do this, a tester has to harvest information on system behaviour. One of the main reasons for gathering information is the execution of test cases. The test case is an essential notion in software testing: simply, ‘a test case is a set of preconditions, inputs, actions (where applicable), expected results and postconditions, developed on test conditions’ (ISTQB, 2018a). At this point the main questions are:

• How do we design the tests?

• Which test cases are the most appropriate for a given situation?

• How can test intensity be determined?

• How many test cases do we need?

• How can we validate the tests?

In answering these questions the process of test design is essential.

WHAT IS TEST DESIGN EXACTLY?

Test design is one of the most important prerequisites of quality. We design tests to support:

1. defining and improving quality-related processes and procedures (quality assurance);

2. evaluating the quality of the product with regards to customer expectations and needs (quality control);

3. finding defects in the product (testing).

Fortunately, or unfortunately, it is a creative process on its own, but also one that requires technical expertise.

More than the act of testing, the act of designing tests is one of the best bug preventers known. The thinking that must be done to create a useful test can discover and eliminate bugs before they are coded - indeed, test-design thinking can discover and eliminate bugs at every stage in the creation of software, from conception to specification, to design, coding and the rest.

(Beizer, 1990)

Test design depends on many factors. Figure 1.1 summarises the relevant entities of the traditional testing life cycle including test design activities.

Preceding test design activities – test planning and test analysis

In this subsection we describe the main activities that precede and influence test design.

Figure 1.1 Test design in the test software development life cycle (SDLC)

Test planning

The test planning process determines the scope, objective, approach, resources and schedule of the intended test activities. During test planning – amongst others – the test objectives, test items, features to be tested, the testing tasks, human and other resources, the degree of tester independence, the test environment, entry and exit criteria to be used, and any risks requiring contingency planning are identified.

The ISO/IEC/IEEE 29119-3 Standard for Software Testing – Test Documentation, recommends the necessary documents for use in defined stages of software testing.

A test policy document represents the testing philosophy of the company, selecting the frames that testing parties should adhere to and follow. It should apply to both new projects and maintenance work. The test strategy is defined as a set of guiding principles that influences the test design and regulates how testing is carried out.

The test approach defines how (in which way) testing is carried out, that is how to implement a test strategy. It can be proactive, that is the test process is initiated as early as possible to find and fix the faults before the build (preferable, if possible), or reactive, which means that the test design process begins after the implementation is finished. The test approach can be of different (not necessarily disjoint) types such as specification-based, structure-based, experience-based, model-based, risk-based, script-based, fault-based, failure-based, standard-compliant, test-first and so on, or a combination of them. For example, a risk-based approach is applied in almost all situations.

The proactive test-first approach starts with analysing the specifications, then applying risk analysis. Based on the result of this analysis one can select the appropriate techniques to design the test cases. The riskier code needs more techniques to be used in parallel. The test design can then be extended with structure-based testing, that is coverage analysis to create additional test cases for uncovered code.

Test monitoring and control

Test monitoring is an ongoing comparison of the actual and the planned progress. Test control involves the actions necessary to meet the objectives of the plan.

Although monitoring and control are activities that belong to the test manager, it is important to ensure that the appropriate data/metrics from the test design activities are collected, validated and communicated.

Test analysis

The test engineering activity in the fundamental test process begins mainly with test analysis. Test analysis is the process of looking at something that can be used to derive quality information for the software product. The test analysis process is based on appropriate project documents or knowledge, called the test basis, on which the tests are based.

The test analysis phase has three main steps before the review.

1. The first step is to analyse the test basis thoroughly. Possible test bases can take the form of business requirement documents, systems requirement documents, functional design specifications, technical specifications, user manuals, source codes and so on.

2. The second step is to identify the test objects (features, scenarios) and conditions by defining what should be tested.

A test condition is a statement about the test object, which can be true or false. Test conditions can be stated for any part of a component (or system) that could be verified by some tests, for example a function, transaction, feature, quality attribute or structural element.

A feature is an attribute of a component or system specified or implied by requirements documentation.

A test scenario is a business requirement to be tested, which may have one or more tests associated with it.

The advantages of using test scenarios are:

a. They help to achieve more complete test coverage.

b. They can be approved by various stakeholders.

c. They help to determine the end-to-end functionality of the test object.

Projects that follow Agile methodology – like Scrum or Kanban – usually do not use test scenarios: they apply user stories, sometimes use cases. We note that both are test objects, and a user story – which is an informal, natural language description of some system features – contains information about the test conditions.

A test object may consist of different test items. A test item is a specific element to be tested.

EXAMPLE FOR TEST OBJECT AND TEST ITEM

To clarify the notion, a test obje...