Statistical Methods in Diagnostic Medicine

Xiao-Hua Zhou, Nancy A. Obuchowski, Donna K. McClish

• Publisher: Wiley
• Year: 2014
• ISBN:
  9781118626047
• Edition: 2
• Topic: Medicina
• Subtopic: Biostatistica

Praise for the First Edition

"... the book is a valuable addition to the literature in the field, serving as a much-needed guide for both clinicians and advanced students."—Zentralblatt MATH

A new edition of the cutting-edge guide to diagnostic tests in medical research

In recent years, a considerable amount of research has focused on evolving methods for designing and analyzing diagnostic accuracy studies. Statistical Methods in Diagnostic Medicine, Second Edition continues to provide a comprehensive approach to the topic, guiding readers through the necessary practices for understanding these studies and generalizing the results to patient populations.

Following a basic introduction to measuring test accuracy and study design, the authors successfully define various measures of diagnostic accuracy, describe strategies for designing diagnostic accuracy studies, and present key statistical methods for estimating and comparing test accuracy. Topics new to the Second Edition include:

• Methods for tests designed to detect and locate lesions
  
  

• Recommendations for covariate-adjustment
  
  

• Methods for estimating and comparing predictive values and sample size calculations
  
  

• Correcting techniques for verification and imperfect standard biases
  
  

• Sample size calculation for multiple reader studies when pilot data are available
  
  

• Updated meta-analysis methods, now incorporating random effects
  
  

Three case studies thoroughly showcase some of the questions and statistical issues that arise in diagnostic medicine, with all associated data provided in detailed appendices. A related web site features Fortran, SAS®, and R software packages so that readers can conduct their own analyses.

Statistical Methods in Diagnostic Medicine, Second Edition is an excellent supplement for biostatistics courses at the graduate level. It also serves as a valuable reference for clinicians and researchers working in the fields of medicine, epidemiology, and biostatistics.

0.1 PREFACE

Diagnostic tests play a pivotal role in medicine, often determining what additional diagnostic tests, treatments, and interventions are needed and ultimately affecting patients' outcomes. Given the importance of this role, it is critical that clinicians are given reliable data about the accuracy of the diagnostic tests they order. These clinicians need well-designed diagnostic accuracy studies and they need to understand how the results of these studies apply to their patients. The purpose of this book, then, is two-fold: to provide a comprehensive approach to designing and analyzing diagnostic accuracy studies and to aid clinicians in understanding these studies and in generalizing study results to their patient populations.

Since the first edition, we have updated each chapter with recently published methods. These updates include new methods for tests designed to detect and locate lesions (see Chapters 2, 3, and 9), recommendations for the type of covariate-adjustment needed (Chapter 3) along with new methods for covariate-adjustment (Chapter 8), estimating and comparing predictive values (Chapters 4 and 5) and calculating sample size for studies using predictive values (Chapter 6), sample size calculation for multiple reader studies when pilot data are available (Chapter 6), new methods for correcting for verification bias in estimation of ROC curves of continuous-scale tests (Chapter 10), and new methods for correcting for imperfect standard bias in estimation of ROC curves of ordinal-scale or continuous-scale tests (Chapter 11).

We have also added three case studies: a positron emission tomography (PET) study comparing the accuracy of three tests for detecting diseased parathyroid glands, a computer-aided detection (CAD) study of colon polyps, and a magnetic resonance imaging study of atherosclerosis in the carotid arteries (see Chapter 1). The data from these case studies are provided in the Appendix and are used throughout the book as illustrations of various statistical methods.

The book is organized such that the more basic material about measures of test accuracy and study design appear first (Chapters 2 and 3, respectively), followed by chapters on statistical methods of data analysis with real data examples to illustrate these methods. Chapters 4 and 5 illustrate methods for estimating accuracy and comparing tests' accuracies under a variety of study designs. Calculating the sample size required for a study is described in Chapter 6. Chapters 7 and 12 focus on the design and analysis of meta-analyses of diagnostic test accuracy. Chapters 8 and 9 look at models of diagnostic test accuracy for various patient subgroups and for multiple-reader studies, respectively. Corrections for estimates of test accuracy in studies with verification bias and imperfect gold standards are illustrated in Chapters 10 and 11. Chapters 1-3 are accessible to readers with a basic knowledge of statistical and medical terminology. Chapters 4-7 are geared to the data analyst with basic training in biostatistics. In Chapters 8-12 we provide more detailed statistical methodology for readers with more statistical training, but the examples in these chapters are accessible to all readers. The only needed change is to add mention of the books related Web site to the Preface. The authors have prepared a Web site (http://faculty.washington.edu/azhou/books/diagnostic.html) that contains links to some useful software.

0.2 ACKNOWLEDGEMENTS

We are thankful to many colleagues for supporting us during the writing and publication of both the first (2002) and second (2011) edition of this book. Their helpful critiques and suggestions about the first edition have led to this improved second edition. Particularly, we would like to thank Danping Liu and Zheyu Wang for their helpful comments on the manuscript and their computational assistance in implementing some of methods discussed in the book. We would like to thank Dr. Thomas D. Koepsell for his helpful comments on the manuscript.

We would also like to thank our families for their understanding and encouragement. Dr. Xiao-Hua (Andrew) Zhou thanks his wife, Yea-Jae, and their children, Vanessa and Joshua. Dr. Nancy Obuchowski thanks her husband, Dr. Ralph Harvey, and their children, Thcker, Eli, and Scout. Dr. Donna McClish thanks her husband, Tom, and their daughter Amanda.

PART I

BASIC CONCEPTS AND METHODS

CHAPTER 1

INTRODUCTION

1.1 DIAGNOSTIC TEST ACCURACY STUDIES

Diagnostic medicine is the process of identifying the disease, or condition, that a patient has, and ruling out conditions that the patient does not have, through assessment of the patient’s signs, symptoms, and results of various diagnostic tests. Diagnostic accuracy studies are research studies which examine the ability of diagnostic tests to discriminate between patients with and without the condition; these studies are the focus of this book.

A diagnostic test has several purposes: (1) to provide reliable information about the patient’s condition, (2) to influence the health care provider’s plan for managing the patient (Sox et al., 1989), and possibly, (3) to understand disease mechanism and natural history through research (e.g., the repeated testing of patients with chronic conditions) (McNeil and Adelstein, 1976). A test can serve these purposes only if the health care provider knows how to interpret it. Diagnostic test studies are conducted to tell us how diagnostic tests perform and, thus, how they should be interpreted. There are several measures of diagnostic test performance. Fryback and Thornbury (1991) described a hierarchical model for studying diagnostic performance for imaging tests. The model starts with image quality and progresses to diagnostic accuracy, effect on treatment decisions, impact on patient outcome, and finally costs to society. A key feature of the model is that for a diagnostic test to be efficacious at a higher level, it must be efficacious at all lower levels. The reverse is not true; for example, a new test may have better accuracy than a standard test but may be too costly (in terms of monetary expense and/or patient morbidity due to complications) to be efficacious. In this book, we deal exclusively with the assessment of diagnostic accuracy (level 2 of the hierarchical model), recognizing that it is only one step in the complete assessment of a diagnostic test.

Diagnostic test accuracy is simply the ability of the test to discriminate among alternative states of health (Zweig and Campbell, 1993). If a test’s results do not differ between alternative states of health, then the test has negligible accuracy; if the results do not overlap for the different health states, then the test has perfect accuracy. Most test accuracies fall between these two extremes. It’s important to recognize that a test result is not a true representation of the patient’s condition (Sox et al., 1989). Most diagnostic information is imperfect; it may influence the health care provider’s thinking, but uncertainty remains about the patient’s true condition. If the test is negative for the condition, should the health care provider assume that the patient is disease-free and thus send him or her home? If the test is positive, should the health care provider assume the patient has the condition and thus begin treatment? Finally, if the test result requires interpretation by a trained reader (e.g., a radiologist), should the health care provider seek a second interpretation?

To answer these critical questions, the health care provider needs to have information on the test’s absolute and relative capabilities and an understanding of the complex interactions between the test and the trained readers who interpret the imaging data (Beam, 1992). The health care provider must ask: How does the test perform among patients with the condition (i.e., the test’s sensitivity)? How does the test perform among patients without the condition (i.e., the test’s specificity)? Does the test serve as a replacement for an older test or should multiple tests be performed? If multiple tests are performed, how should they be executed (i.e., sequentially or in parallel)? How reproducible are interpretations by different readers? These sorts of questions are addressed in diagnostic test accuracy studies.

Diagnostic test accuracy studies have three common features: a sample of subjects who have, or will, undergo one or more of the diagnostic medical tests under evaluation; some form of interpretation or scoring of the test’s findings; and a reference, or gold standard, to which the test findings are compared. This may sound simple enough, but diagnostic accuracy studies are difficult to design. Here are three common misperceptions about diagnostic test accuracy.

The first misperception involves the interpretation of diagnostic tests. Investigators of new diagnostic tests sometimes develop criteria for interpreting their tests based only on the findings from healthy volunteers. For example, in a new test to detect pancreatitis, investigators measure the amount of a certain enzyme in healthy volunteers. A typical decision criterion, or cutpoint, is three standard deviations (SDs) below the mean of the normals. New patients with an enzyme level of three SDs below the mean of the healthy volunteers are labeled “positive” for pancreatitis; patients with enzyme levels above this cutpoint are labeled “negative”. In proposing such a criterion, investigators fail to recognize (1) the relevance of the natural distributions of the test results (i.e. are they really Gaussian [normal]?); (2) the magnitude of any overlap between the test results of patients with and without pancreatitis (i.e. are the test results from most pancreatitis patients 3 SDs below the mean?); (3) the clinical significance of diagnostic errors (i.e. falsely labeling a patient without pancreatitis as “positive” for the condition and falsely labeling a patient with pancreatitis as “negative”); and (4) the poor generalization of results from studies based on healthy volunteers (i.e. healthy volunteers may have very different enzyme levels than sick patients without pancreatitis who might undergo the test). In Chapter 2, we discuss factors involved in determining optimal cutpoints for diagnostic tests; in Chapter 4, we discuss methods of finding optimal cutpoints and estimating diagnostic errors associated with them.

Another common misperception in diagnostic test studies is the notion that a rigorous assessment of a patient’s true condition - with the exclusion of patients for whom a less rigorous assessment was made - allows for a scientifically sound study. An example comes from literature on the use of ventilation-perfusion lung scans for diagnosing pulmonary emboli. The ventilation-perfusion lung scan is a noninvasive test used to screen high-risk patients for pulmonary emboli; its accuracy in various populations is unknown. Pulmonary angiography, on the other hand, is a highly accurate but invasive test. It is often used as a reference for assessing the accuracy of other tests. (See Chapter 2 for the definition and examples of gold standards.) To assess the accuracy of ventilation-perfusion lung scans, patients who have undergone both a ventilation-perfusion lung scan and a pulmonary angiogram are recruited, while patients who did not undergo the angiogram are excluded. Such a design usually leads to biased estimates of test accuracy. The reason is that the study sample is not representative of the patient population undergoing ventilation-perfusion lung scans - rather, patients with a positive scan are often recommended for angiograms, while patients with a negative scan are often not sent for an angiogram because of the risk of complications with it. In Chapter 3, we define work-up bias, and its most common form, verification bias, as well as ...