Blinding as a Solution to Bias

Strengthening Biomedical Science, Forensic Science, and Law

Christopher T Robertson, Aaron S Kesselheim

• Publisher: Academic Press
• Year: 2016
• Print ISBN: 9780128024607
• eBook ISBN: 9780128026335
• Topic: Law
• Subtopic: Forensic Science
• Index: Law

What information should jurors have during court proceedings to render a just decision? Should politicians know who is donating money to their campaigns? Will scientists draw biased conclusions about drug efficacy when they know more about the patient or study population? The potential for bias in decision-making by physicians, lawyers, politicians, and scientists has been recognized for hundreds of years and drawn attention from media and scholars seeking to understand the role that conflicts of interests and other psychological processes play. However, commonly proposed solutions to biased decision-making, such as transparency (disclosing conflicts) or exclusion (avoiding conflicts) do not directly solve the underlying problem of bias and may have unintended consequences.Robertson and Kesselheim bring together a renowned group of interdisciplinary scholars to consider another way to reduce the risk of biased decision-making: blinding. What are the advantages and limitations of blinding? How can we quantify the biases in unblinded research? Can we develop new ways to blind decision-makers? What are the ethical problems with withholding information from decision-makers in the course of blinding? How can blinding be adapted to legal and scientific procedures and in institutions not previously open to this approach? Fundamentally, these sorts of questions—about who needs to know what—open new doors of inquiry for the design of scientific research studies, regulatory institutions, and courts.The volume surveys the theory, practice, and future of blinding, drawing upon leading authors with a diverse range of methodologies and areas of expertise, including forensic sciences, medicine, law, philosophy, economics, psychology, sociology, and statistics.- Introduces readers to the primary policy issue this book seeks to address: biased decision-making.- Provides a focus on blinding as a solution to bias, which has applicability in many domains.- Traces the development of blinding as a solution to bias, and explores the different ways blinding has been employed.- Includes case studies to explore particular uses of blinding for statisticians, radiologists, and fingerprint examiners, and whether the jurors and judges who rely upon them will value and understand blinding.

Section V

Blinding in Legal Institutions

Outline

Legal Applications of the Disclose-or-Blindfold Question

Chapter 15. Why Eyes? Cautionary Tales from Law’s Blindfolded Justice

Chapter 16. A Theory of Anonymity

Chapter 17. The Cases for and against Blindfolding the Jury

Chapter 18. The Compliance Equation: Creating a More Ethical and Equitable Campaign Financing System by Blinding Contributions to Federal Candidates

Chapter 19. Blinding Eyewitness Identifications

Chapter 20. Blind Appointments in Arbitration

Chapter 21. Psychological Obstacles to the Judicial Disqualification Inquiry, and Blinded Review as an Aid

Chapter 22. Masking Information Source within the Internal Revenue Service

Chapter 23. Blinding the Law: The Potential Virtue of Legal Uncertainty

Legal Applications of the Disclose-or-Blindfold Question

Saul Levmore

University of Chicago Law School, Chicago, IL, USA

Disclosure, that celebrated disinfectant of sunlight, is the conventional remedy for many questions in law and ethics, especially where there is a fear of bias. But an opposite strategy, of selective blindfolding, is hardly unknown. This book explores the choice between disclosure and blindfolding, or blinding as it is often labeled in scientific research and in the volume’s title, and suggests that law and medicine (including biomedical research) have some things to learn from one another. In thinking about the contributions that deal with law, it is useful to think about the extent of blindfolding that is desirable in legal settings, and then also to keep an eye on the ways in which law might learn from other disciplines.

Consider the situation where someone screens job applicants and sends you four files; you are to interview these four candidates and choose the winner. If the screener is related to one of these candidates, should the screener disclose this fact to you, disclose and identify the candidate, keep you blindfolded (and perhaps disclose after you select the apparent winner, but before any actual hiring), or simply not send the related candidate in the first place? This is a question of single blindfolding and, even here, different workplaces and readers have different policies and reactions. It is not exactly a legal application, but rather one about evolved norms, contractual choices, and simple preferences. Nevertheless, and as is often the case, the law draws on norms, and norms generally make more sense when viewed against a background of legal rules. I doubt there is a single, winning response to this example of disclose-or-blindfold. There is heterogeneity in our perceptions of our ability to be unbiased; the advantages of disclosure, or even of having four rather than three candidates, are not the same across all jobs; and blindfolding may be more or less attractive depending on what is known about the screener. In short, responses are apt to be contextual, and this suggests that general conclusions and rules will be difficult when it comes to the choice between disclosure and blindfolding, or other strategies for dealing with potential bias.

Law deals with many similar (but hardly identical) situations. Thus, a judge is often presented with a case that has some bearing on the judge’s own life. If a decision could directly affect the judge’s financial interests, then the judge is instructed to recuse herself from the case. Indirect interests are left to the judge’s judgment, and it is interesting that disclosure is virtually never used in this context, though it is the norm for academic writing. Thus, a judge’s decision in a matter of tax law is likely to have some distant impact on the judge’s own tax returns, but this is too trivial or impossible to control. A judge presented with the constitutionality of a statute regulating abortions is most unlikely to step aside (or disclose) on grounds that her own daughter might seek an abortion in the future. There is no blindfolding—for example, no attempt to have cases presented with the identities of the parties hidden by redaction, and no attempt to have abortion statutes ruled on by judges in another jurisdiction—and there is no opportunity to disclose and then proceed to decide the case with a direct conflict. In contrast, an academic would rarely recuse herself, but often disclose direct contact with a matter.

The importance of context is again clear. It seems unlikely that the answer one gives to the question about job candidates predicts the line one would draw where judicial behavior is concerned, even though both situations involve potential bias. The importance of context, or simply details, is even more obvious as we go farther afield. Imagine that one seeks a second opinion after a competent physician has recommended surgery. Would you advise blindfolding the second physician about the fact, identity, or advice of the earlier visit to another physician? And does the answer to that question cast any light on why we do not blindfold appellate courts regarding the identity of the lower-court judge whose opinion it reviews? A quick intuition is that while one case suggests the other, or makes it more interesting, there is little reason to think that a single approach works best in all these contexts.

When, however, the goal is not to intervene but rather to discover, there is unambiguous benefit in blindfolding, and the more the better—setting aside the transaction costs involved in the blindfolding itself. In most cases of intervention, blindfolding is a double-edged sword. When a second opinion is sought from a physician (or appellate court), for example, it can be efficient to disclose rather than to blindfold. The reviewer might better focus inquiry, because he knows something of the strengths and weaknesses of the first decision maker; he might also see ideas or conclusions in the first report (whether it is a physician’s diagnosis or a lower court opinion) that never would have occurred to him, but that now seem correct. On the other hand, a blindfolded reviewer would be free of bias, and hundreds of studies support the notion that bias can be hard to overcome, no matter how much one is aware of the problem it presents. It is thus apparent that there are costs and benefits to blindfolding in this context.

In contrast, consider the process of evaluating eyewitness testimony through lineups in which the witness is asked to choose someone from a group of persons or photographs. This example of blindfolding, and ways in which it might be improved, is the subject of a chapter by Brandon Garrett. The witness is blindfolded, in the sense of choosing from a lineup, rather than being asked to identify or disclaim a single suspect. Accuracy is improved by double-blinding; the police officer who runs the lineup should not know which person is the actual suspect. It might even be nice to triple-blind; perhaps the suspect as well the others in the lineup could be brought into view without their knowing why they were there. If so, there would be less room for conscious or subconscious behavior on their part that might provide information, which is to say bias, to the witness. Note that when shuffled photographs are used, rather than live suspects and confederates, there is this sort of extra blinding in the testing of the eyewitness. In any event, the point is that all this blindfolding sacrifices no information. It is quite different from the situation where a patient seeks a second opinion.

The question of how to increase blindfolding and its bias-reducing benefits is never far from the focus of the chapters found here on law and blindfolding, in part because the authors choose applications where law is looking for facts rather than where it is intervening or reviewing decisions. It is as if we know that the decision maker would or should prefer to be freed from bias, by choosing the blindfold rather than more disclosure. Thus, the chapter by Karie Davis-Nozemack suggests that blindfolded review by the Internal Revenue Service could eliminate an institutional bias against information provided by whistleblowers. Further work would then be needed to discover just how unreliable (or superior) whistleblower information turns out to be. Shari Seidman Diamond’s chapter provides an important counterweight by offering examples where more and better disclosure may be better than blindfolding, if only because blinded decision makers cannot help but make assumptions about facts withheld from them.

Other chapters dare us to imagine gains from further blindfolding. Yuval Feldman and Shahar Lifshitz help us see that law would often work better if those who are subject to it knew less about its content rather than more. Thus, debtors might (or might not) borrow from creditors, and then run their businesses more efficiently, if they did not know about the protections available under bankruptcy law should they eventually become insolvent. In Bertram Levine and Michael Johnston’s chapter, we are encouraged to think about campaign finance schemes that keep politicians ignorant of the source of donations to them. The plan may not be entirely workable because a politician who votes for (or against) a project, and then finds an unusual amount of money pouring in during the following week, will have enough information to be biased in the future. Still, the idea of blindfolding contributions, as we do votes, is surely worth examining. Note that the donor pool might shrink if this blindfolding works, as examined in Jeffrey Skopek’s discussion of anonymity and production incentives, but in the case of campaign contributions, the shrinkage might be desirable. Similarly, Sergio Puig suggests ways to eliminate “affiliation bias” in arbitration.

Judging by the chapters found here, the area of greatest promise for blindfolding in law is in the use of experts. Jeffrey Robinson, in the section of this anthology on forensic science, shows rather ingeniously how we might blindfold experts who review medical malpractice cases, by mixing these cases with others, and asking experts to review them all. Christopher Robertson forcefully argues that if we could simply blindfold litigants in their choice of experts, then we could safely maintain the adversarial system’s convention of...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright
• List of Contributors
• Foreword
• Section I. Introduction and Overview
• Section II. Blinding and Bias
• Section III. Biomedical Science
• Section IV. Forensic Science: Criminal and Civil
• Section V. Blinding in Legal Institutions
• Index