Lessons Learned in Software Testing

A Context-Driven Approach

Cem Kaner, James Bach, Bret Pettichord

• Publisher: Wiley
• Year: 2011
• Print ISBN:
  9780471081128
• Edition: 1
• eBook ISBN:
  9781118080559
• Topic: Computer Science
• Subtopic: Software Development
• Index: Computer Science

Decades of software testing experience condensed into the most important lessons learned.

The world's leading software testing experts lend you their wisdom and years of experience to help you avoid the most common mistakes in testing software. Each lesson is an assertion related to software testing, followed by an explanation or example that shows you the how, when, and why of the testing lesson. More than just tips, tricks, and pitfalls to avoid, Lessons Learned in Software Testing speeds you through the critical testing phase of the software development project without the extensive trial and error it normally takes to do so. The ultimate resource for software testers and developers at every level of expertise, this guidebook features:
* Over 200 lessons gleaned from over 30 years of combined testing experience
* Tips, tricks, and common pitfalls to avoid by simply reading the book rather than finding out the hard way
* Lessons for all key topic areas, including test design, test management, testing strategies, and bug reporting
* Explanations and examples of each testing trouble spot help illustrate each lesson's assertion

CHAPTER 1
The Role of the Tester

What are testers supposed to do for a project? That’s the question we address in this chapter. Like many things about testing, the answer may seem obvious or trivial at first glance, but it’s not.

A role is a relationship. That means you don’t control your role, but you can negotiate it. People expect things from you that might not be reasonable. When you find yourself blamed for a low-quality product (and that will happen), whoever is blaming you probably suffers from role confusion. Maybe they think your job is to beat the product with the Magic Mallet of Quality before it ships, and they think you didn’t hit it hard enough.

When you’re clear about your role—when you have negotiated it—you have a foundation for setting expectations in any situation that may arise. However, even a clear and appropriate testing role is a demanding one.

You are the headlights of the project.

A project is like a road trip. Some projects are simple and routine, like driving to the store in broad daylight. But most projects worth doing are more like driving a truck off-road, in the mountains, at night. Those projects need headlights. As the tester, you light the way. You illuminate the road ahead so the programmers and managers, however they bicker over the map, can at least see where they are, what they’re about to run over, and how close they are to the cliff. The detailed mission of the testing group varies from company to company. Behind those details, though, is a common factor.

Testing is done to find information. Critical decisions about the project or the product are made on the basis of that information.

Your mission drives everything you do.

Your mission might depend on your industry, company project, or the personalities on the team. Test projects vary greatly from place to place. A challenge for the evolution of testing as a craft has been the difficulty of creating a conversation about test practices that will span the cultural and technical differences among us. Many of these differences amount to different missions for the test team. For instance, in some testing organizations, a test plan is just a tool to help the testers. It could be written on a napkin and still be effective. Other organizations create test plans as products that must be delivered along with the software. Their test plans may have to follow strict format and content guidelines.

Any of the following requirements might define your mission. Which ones are expected of you?

• Find important bugs fast.
• Provide a general assessment of the quality of the product.
• Certify that the product meets a particular standard.
• Help your clients improve product quality and testability.
• Assure that the test process meets accountability standards.
• Educate your clients about testing and how to work with testers.
• Follow a particular set of methods or rules.
• Help predict and control the costs of support.
• Help your clients improve their processes.
• Perform your work in a manner that minimizes cost, time, or undesirable side effects.
• Do whatever is necessary to satisfy particular clients.

If you spend time and effort on requirements that your clients don’t care about, you risk being treated as irrelevant or counterproductive. Negotiate your mission with your manager. Clarify it. If you can’t come to agreement on the mission, you won’t have a good foundation for anything you do.

What should you do when you don’t know what to do? One answer is review your mission. It identifies the core problems that you own. When you’re clear on your testing mission, you can defend your work and determine specifically what to do next. You can also explain your role to other people, in simple terms. If you can’t work toward your mission for some reason, take the matter to management right away.

What should you do when you know exactly what to do? Once in a while, revisit your mission to make sure that your clear plan hasn’t focused you so much on one part of the testing problem that you’ve forgotten about the rest.

You serve many clients.

Testing is a service role. Feel good about that. The service you provide is vital. Service implies clients—the people you serve. Your success is measured primarily by how well you serve your clients’ desires and best interests. That might not be so hard, except that testing has many clients. They all have their own needs, and their collective needs don’t necessarily align:

• The project manager. Project managers are entitled to know your process and influence it. You serve the project manager by reporting your status on demand, reporting important problems fast, and not being a bottleneck to the project. It’s the project manager’s prerogative to direct the project. It’s your job to tell him what you are able to do, what you can’t do, and what the impact on testing will be of any given decision or condition on the project.
• The programmer. You make the programmer’s job easier by providing good bug reports, as soon as possible. Strive to know your craft and know the product so you don’t waste the programmer’s time with mistaken or frivolous reports. If you can do that, you’ll have a lot more credibility, and that will translate into support and influence.
• The technical writer. Like you, the people who write the manuals and the online help get incomplete information about the product. You can help them understand how the product really works, and you can alert them to errors in the documentation. Writers can help you, too. As they do their research on the product and how the people who have to read the documentation will use the product, they will learn things that you don’t know. If you have a good relationship with the writers, they’ll alert you to new features, new uses, holes in your test plan, and to the bugs they find. Some of those bugs would never be reported unless a particular writer knows that a particular tester cares.
• Technical support. Whatever problems are left in the product become a burden for the people who provide technical support. You serve the support group by alerting them to aspects of the product that may trouble the user. If you work with them during development, sometimes the support staff will help you make the case that a bug should be fixed. You should also offer to help investigate difficult problems found in the field. Doing so will bring you into closer contact with support people and, therefore, with the customer.
• Marketing. Marketing needs to know whether anything in the product is inconsistent with the key benefits the product is supposed to provide to customers. A bug that seems minor to programmers might be critical to marketers. They might recognize that the bug makes it harder for the customer to do an important task. Also, by reviewing planned marketing documents or statements, you can help marketing promote an accurate account of the product’s capabilities.
• Top management and stockholders. You serve the business. That’s why you must be careful not to sound or act like a quality fanatic instead of a reasonable person. Especially near the end of the project, perform your work in a way that takes into account the short-term and long-term interests of the company. Express test status reports in crisp, operational terms, so that executives feel they have a basis on which to make decisions.
• The user. In your heart, you serve the people who will make use of the product. Their satisfaction is in the best interests of your project, of course. But there is also a special satisfaction that goes with being the primary user advocate on the project team.

This list is in no particular order, but your project may have a pecking order, so look into it. Find out who matters on your project. Discover whom you serve. This is the first step to great testing.

You discover things that will “bug” someone whose opinion matters.

Your group’s mission includes (or should include) informing clients about anything that threatens the value of the product, according to your clients’ definition(s) of value. If you can show that the product will not be valued even if it works as intended, it’s your duty to report your concerns. If your clients choose to dismiss the report, that’s their prerogative.

Find important bugs fast.

Most likely, your mission includes finding bugs that are important (as opposed to insignificant) and finding them quickly. If so, what does this mean in terms of the tests you run?...

Table of contents

• Title Page
• Copyright
• DEDICATION
• Brief Contents
• FOREWORD
• PREFACE
• ACKNOWLEDGMENTS
• CHAPTER 1 The Role of the Tester
• CHAPTER 2 Thinking Like a Tester
• CHAPTER 3 Testing Techniques
• CHAPTER 4 Bug Advocacy
• CHAPTER 5 Automating Testing
• CHAPTER 6 Documenting Testing
• CHAPTER 7 Interacting with Programmers
• CHAPTER 8 Managing the Testing Project
• CHAPTER 9 Managing the Testing Group
• CHAPTER 10 Your Career in Software Testing
• CHAPTER 11 Planning the Testing Strategy
• APPENDIX The Context-Driven Approach to Software Testing
• Bibliography
• Index