Pragmatic Software Testing

Becoming an Effective and Efficient Test Professional

Rex Black

• Publisher: Wiley
• Year: 2011
• Print ISBN: 9780470127902
• eBook ISBN: 9781118079386
• Edition: 1
• Topic: Computer Science
• Subtopic: Software Development
• Index: Computer Science

A hands-on guide to testing techniques that deliver reliable software and systems Testing even a simple system can quickly turn into a potentially infinite task. Faced with tight costs and schedules, testers need to have a toolkit of practical techniques combined with hands-on experience and the right strategies in order to complete a successful project. World-renowned testing expert Rex Black provides you with the proven methods and concepts that test professionals must know. He presents you with the fundamental techniques for testing and clearly shows you how to select and apply successful strategies to test a system with budget and time constraints. Black begins by discussing the goals and tactics of effective and efficient testing. Next, he lays the foundation of his technique for risk-based testing, explaining how to analyze, prioritize, and document risks to the quality of the system using both informal and formal techniques. He then clearly describes how to design, develop, and, ultimately, document various kinds of tests. Because this is a hands-on activity, Black includes realistic, life-sized exercises that illustrate all of the major test techniques with detailed solutions.

Part I
Goals, Strategies, and Tactics

In this Part

• Chapter 1: What Does It Mean to Be Pragmatic?
• Chapter 2: Triangle Test Exercise
• Chapter 3: Aligning Testing with the Project
• Chapter 4: Understanding Test Strategies, Tactics, and Design

Chapter 1
What Does It Mean to Be Pragmatic?

Let’s start at the beginning by exploring some obvious questions with some not-so-obvious and not-so-universally-accepted answers about pragmatic testing. From a pragmatic, or practical, standpoint, it involves being effective and efficient when testing software. What is effective software testing? What is efficient software testing? What is software testing, anyway? What is quality?

While these might seem like impractical, philosophical questions, in my experience, they are not. Your answers to these questions determine what you expect to do as a tester. Other people’s answers to these questions determine what they expect you to do as a tester. Having common expectations up, down, and across the organizational chart and throughout the project team is essential to success. Without such commonality, no matter what you do, someone’s sure to be disappointed. With common expectations, you can all strive for the same goals, and support others in their endeavors.

What Do Effective and Efficient Mean?

Webster’s dictionary defines the word effective as “producing a decided, decisive, or desired result; impressive.” So, to be an effective software tester, you must decide what results you desire from your testing efforts.

Likewise, Webster’s defines efficient as “productive of the desired effect; especially to be productive without waste.” So, to be an efficient tester, you must allocate resources (time and money) appropriately.

In the next few pages, you’ll get a closer look at each of these concepts.

First, though, note that testing usually occurs within the context of some larger project or effort. That larger project could be developing a new system, maintaining an existing system, deploying a system in a new or existing environment, or deciding whether to accept or reject a proposed system. So, when the testing subproject exists within the context of the larger project, you should look at test effectiveness and efficiency from the project perspective, not the test subproject perspective.

What Effects Do You Want?

On most projects I’ve worked on, I’ve typically wanted to do the following:

• Produce and deliver information to the project team about the most important aspects of the quality of the system under test.
• Produce and deliver information to the development team so they can fix the most important bugs.
• Produce and deliver information to the project management team so they can understand issues such as the current levels of system quality, the trends of those levels, and the level of quality risk reduction being achieved.
• Produce and deliver information that can help the technical support team, help desk, and other folks on the receiving end of the project deal effectively with the system they received, warts and all.

In short, I often find myself a producer and provider of useful information for the project team. This has some interesting implications, which you’ll examine in a minute.

Reading this list, you might have thought of some effects that you — or other people with whom you’ve worked — typically want from testing that are missing. Good! There is no one right set of answers for this question.

What Is the Right Level of Efficiency?

The definition of efficiency that was presented earlier included the ideas of productivity and a lack of waste. What does that mean?

Avoiding Redundancy

Of course, you don’t want to waste time, so you should try to avoid redundancy. You should try to run the right tests and run them as rapidly as possible. You should try to discover bugs as early as possible and find the more-important bugs before the less-important bugs.

That said, you’ve probably seen more than one project where the old cliché “more haste, less speed” applied. You rushed into things without planning and preparation. You didn’t attend to details. You abandoned carefully thought-out strategies at the first sign of trouble or delay. In short, in an attempt to save time, you made mistakes that cost you more time than you saved.

Reducing Cost

You also don’t want to waste money, so you should buy only the supplies — software and hardware tools, test environments, and so on — that you really need and that you have the time and talents to gainfully employ. You should carefully evaluate tools before you buy them and be ready to justify them with cost-benefit analyses.

When you do such analyses, you’ll often find that good tools and test configurations can pay for themselves — sometimes dramatically. On one project, a manager decided to “save money” by having us use a test server that was in a different country. This server cost about $20,000, and he said there was no money in the budget for another one. However, they paid more than $20,000 in lost time and productivity when people had to wait for the system administrator in the remote location to respond to their calls for help — sometimes for more than a day.

This matter of cost-benefit analyses brings us to another question: What is the cost-benefit analysis for testing? Is testing an investment that pays for itself?

While these questions are more a test management topic, let me briefly mention the concept of cost of quality. Cost of quality recognizes two types of quality-related costs that organizations pay:

• Costs of conformance: These are the costs you pay to achieve quality. They include costs of detection, which are the costs of testing, or looking for bugs. Conformance costs also include the costs of prevention, which are the costs of quality assurance, of improving the people, technologies, and process.
• Costs of nonconformance: These are the costs you pay when you fail to achieve quality. These costs include the costs associated with finding and fixing bugs, plus retesting those bugs and doing regression testing, before you release, deploy, or go live with the system. These are also called the costs of internal failure. The costs associated with finding bugs after that point are called costs of external failure. External failures can cost you in terms of finding, fixing, retesting, and regression testing, as well as the additional associated costs o...

Table of contents

• Cover
• Table of Contents
• Title
• Dedication
• Copyright
• About the Author
• Credits
• Preface
• Acknowledgments
• Introduction
• Part I: Goals, Strategies, and Tactics
• Part II: Risk-Based Testing
• Part III: Static Testing
• Part IV: Behavioral Testing
• Part V: Structural Testing
• Part VI: Appendixes
• Index
• End User License Agreement