Effective Software Testing

A developer's guide

Maurizio Aniche

• Publisher: Manning
• Year: 2022
• ISBN:
  9781638350583
• Topic: Ciencia de la computación
• Subtopic: Desarrollo de software

Go beyond basic testing! Great software testing makes the entire development process more efficient. This book reveals a systemic and effective approach that will help you customize your testing coverage and catch bugs in tricky corner cases. In Effective Software Testing you will learn how to: Engineer tests with a much higher chance of finding bugs
Read code coverage metrics and use them to improve your test suite
Understand when to use unit tests, integration tests, and system tests
Use mocks and stubs to simplify your unit testing
Think of pre-conditions, post-conditions, invariants, and contracts
Implement property-based tests
Utilize coding practices like dependency injection and hexagonal architecture that make your software easier to test
Write good and maintainable test code Effective Software Testing teaches you a systematic approach to software testing that will ensure the quality of your code. It's full of techniques drawn from proven research in software engineering, and each chapter puts a new technique into practice. Follow the real-world use cases and detailed code samples, and you'll soon be engineering tests that find bugs in edge cases and parts of code you'd never think of testing! Along the way, you'll develop an intuition for testing that can save years of learning by trial and error. About the technology
Effective testing ensures that you'll deliver quality software. For software engineers, testing is a key part of the development process. Mastering specification-based testing, boundary testing, structural testing, and other core strategies is essential to writing good tests and catching bugs before they hit production. About the book
Effective Software Testing is a hands-on guide to creating bug-free software. Written for developers, it guides you through all the different types of testing, from single units up to entire components. You'll also learn how to engineer code that facilitates testing and how to write easy-to-maintain test code. Offering a thorough, systematic approach, this book includes annotated source code samples, realistic scenarios, and reasoned explanations. What's inside Design rigorous test suites that actually find bugs
When to use unit tests, integration tests, and system tests
Pre-and post-conditions, invariants, contracts, and property-based tests
Design systems that are test-friendly
Test code best practices and test smellsAbout the reader
The Java-based examples illustrate concepts you can use for any object-oriented language. About the author
Dr. Maurício Aniche is the Tech Academy Lead at Adyen and an Assistant Professor in Software Engineering at the Delft University of Technology.Table of Contents
1 Effective and systematic software testing
2 Specification-based testing
3 Structural testing and code coverage
4 Designing contracts
5 Property-based testing
6 Test doubles and mocks
7 Designing for testability
8 Test-driven development
9 Writing larger tests
10 Test code quality
11 Wrapping up the book

1 Effective and systematic software testing

This chapter covers

• Understanding the importance of effective, systematic testing
• Recognizing why testing software is difficult and why bug-free systems do not exist
• Introducing the testing pyramid

The developer community no longer needs to argue about the importance of software testing. Every software developer understands that software failures may cause severe damage to businesses, people, or even society as a whole. And although software developers once were primarily responsible for building software systems, today they are also responsible for the quality of the software systems they produce.

Our community has produced several world-class tools to help developers test, including JUnit, AssertJ, Selenium, and jqwik. We have learned to use the process of writing tests to reflect on what programs need to do and get feedback about the code design (or class design, if you are using an object-oriented language). We have also learned that writing test code is challenging, and paying attention to test code quality is fundamental for the graceful evolution of the test suite. And finally, we know what the common bugs are and how to look for them.

But while developers have become very good at using testing tools, they rarely apply systematic testing techniques to explore and find bugs. Many practitioners argue that tests are a feedback tool and should be used mostly to help you develop. Although this is true (and I will show throughout this book how to listen to your test code), tests can also help you find bugs. After all, that is what software testing is all about: finding bugs!

Most developers do not enjoy writing tests. I have heard many reasons: writing production code is more fun and challenging, software testing is too time-consuming, we are paid to write production code, and so on. Developers also overestimate how much time they spend on testing, as Beller and colleagues found in a nice empirical study with hundreds of developers in 2019. My goal with this book is to convince you that (1) as a developer, it is your responsibility to ensure the quality of what you produce; (2) that tests are the only tools to help you with that responsibility; and (3) that if you follow a collection of techniques, you can test your code in an effective and systematic way.

Note the words I used: effective and systematic. Soon you will understand what I mean. But first, let me convince you of the necessity of tests.

1.1 Developers who test vs. developers who do not

It is late on Friday afternoon, and John is about to implement the last feature of the sprint. He is developing an agile software management system, and this final feature supports developers during planning poker.

Planning poker

Planning poker is a popular agile estimation technique. In a planning poker session, developers estimate the effort required to build a specific feature of the backlog. After the team discusses the feature, each developer gives an estimate: a number ranging from one to any number the team defines. Higher numbers mean more effort to implement the feature. For example, a developer who estimates that a feature is worth eight points expects it to take four times more effort than a developer who estimates the feature to be worth two points.

The developer with the smallest estimate and the developer with the highest estimate explain their points of view to the other members of the team. After more discussion, the planning poker repeats until the team members agree about how much effort the feature will take. You can read more about the planning poker technique in Kanban in Action by Marcus Hammarberg and Joakim Sundén (2014).

John is about to implement the feature’s core method. This method receives a list of estimates and produces, as output, the names of the two developers who should explain their points of view. This is what he plans to do:

Method: identifyExtremes

The method should receive a list of developers and their respective estimates and return the two developers with the most extreme estimates.

Input: A list of Estimates, each containing the name of the developer and their estimate

Output: A list of Strings containing the name of the developer with the lowest estimate and the name of the developer with the highest estimate

After a few minutes, John ends up with the code in the following listing.

Listing 1.1 The first PlanningPoker implementation

public class PlanningPoker {  public List<String> identifyExtremes(List<Estimate> estimates) {   Estimate lowestEstimate = null; ❶  Estimate highestEstimate = null; ❶   for(Estimate estimate: estimates) {   if(highestEstimate == null ||  estimate.getEstimate() > highestEstimate.getEstimate()) { ❷  highestEstimate = estimate;  }  else if(lowestEstimate == null ||  estimate.getEstimate() < lowestEstimate.getEstimate()) { ❸  lowestEstimate = estimate;  }   }   return Arrays.asList( ❹  lowestEstimate.getDeveloper(),  highestEstimate.getDeveloper()  );  } }

❶ Defines placeholder variables for the lowest and highest estimates

❷ If the current estimate is higher than the highest estimate seen so far, we replace the previous highest estimate with the current one.

❸ If the current estimate is lower than the lowest estimate seen so far, we replace the previous lowest estimate with the current one.

❹ Returns the developers with the lowest and the highest estimates

The logic is straightforward: the algorithm loops through all the developers in the list and keeps track of the highest and lowest estimates. It returns the names of the developers with the lowest and highest estimates. Both lowestEstimate and highestEstimate are initialized with null and later replaced by the first estimate within the for loop.

Generalizing from the code examples

Experienced developers may question some of my coding decisions. Maybe this Estimate class is not the best way to represent developers and their estimates. Maybe the logic to find the smallest and highest estimates is not the best. Maybe the if statements could be simpler. I agree. But my focus in this book is not object-oriented design or the best ways to write code: rather, I want to focus on how to test the code once it’s written.

The techniques I show you throughout this book will work regardless of how you implement your code. So, bear with me when you see a piece of code that you think you could do better. Try to generalize from my examples to your own code. In terms of complexity, I am sure you have encountered code like that in listing 1.1.

John is not a fan of (automated) software testing. As is commonly done by developers who do not automate their tests, John runs the finished application and tries a few inputs. You can see one of these trials in figure 1.1. John sees that given the input in the figure (the estimates of Ted, Barney, Lily, and Robin), the program produces the correct output.

Figure 1.1 John does some manual testing before releasing the application.

John is happy with the results: his implementation worked from the beginning. He pushes his code, and the new feature is deployed automatically to customers. John goes home, ready for the weekend—but not even an hour later, the help desk starts to get e-mails from furious customers. The software is producing incorrect outputs!

John goes back to work, looks at the logs, and quickly identifies a case where the code fails. Can you find the input that makes the program crash? As illustrated in figure 1.2, if the developers’ estimates are (by chance) in ascending order, the program th...