Discerning Experts

The Practices of Scientific Assessment for Environmental Policy

Michael Oppenheimer,Naomi Oreskes,Dale Jamieson,Keynyn Brysse,Jessica O’Reilly,Matthew Shindell,Milena Wazeck

• Publisher: University of Chicago Press
• Year: 2019
• ISBN:
  9780226602158
• Topic: Biological Sciences
• Subtopic: Science General
• Index: Biological Sciences

Discerning Experts assesses the assessments that many governments rely on to help guide environmental policy and action. Through their close look at environmental assessments involving acid rain, ozone depletion, and sea level rise, the authors explore how experts deliberate and decide on the scientific facts about problems like climate change. They also seek to understand how the scientists involved make the judgments they do, how the organization and management of assessment activities affects those judgments, and how expertise is identified and constructed. Discerning Experts uncovers factors that can generate systematic bias and error, and recommends how the process can be improved. As the first study of the internal workings of large environmental assessments, this book reveals their strengths and weaknesses, and explains what assessments can—and cannot—be expected to contribute to public policy and the common good.

CHAPTER ONE

The Need for Expert Judgment

INTRODUCTION

From ancient shamans, oracles, and diviners to the physicians of the World Health Organization and the scientists of the Intergovernmental Panel on Climate Change, there have long been individuals and groups with specialized knowledge who have been asked to provide judgment on issues that ordinary citizens and political and religious leaders felt unable to judge for themselves. In the twentieth century, the felt need for expert judgment grew, and institutionalized assessments of scientific knowledge for policy became a significant part of the landscape of scientific work and discourse. This book examines scientific assessments for public policy in the late twentieth and early twenty-first centuries. An obvious opening question is, Were there assessments before that time? If so, are recent assessments different? Exploring examples of expert judgment before the twentieth century, this chapter considers how assessments in the twentieth century can be distinguished from what went before.

THE PREHISTORY OF ASSESSMENTS

A vexing problem of the late medieval period was spirit discernment. Europeans in the Middle Ages faced the difficulty of distinguishing saints infused with the spirit of God from ordinary people possessed by demons. We might suppose that the two could scarcely be confused, but contemporaneous commentators agreed that the physical manifestations of divine and demonic possession were distressingly similar. Visions, trances, frenzies, levitation, the performance of miracles, feats of superhuman strength, xenoglossy, displays of stigmata, nudity, and other transgressions of social convention: these diverse manifestations were common to both. As historian Nancy Caciola has explained, the two kinds of spirit possession were “outwardly indistinguishable.”1 Moreover, this was no coincidence. As Paul had told us (2 Corinthians 11:14), Satan knew how to disguise himself as an angel of light.2

Divinely inspired prophets merited veneration, but false saints and demoniacs demanded condemnation, so it was essential to determine how to differentiate them. The challenge of discerning spirits was thus both epistemological and existential: epistemological because it involved questions of knowledge, existential because one’s fate could rest upon it. The arduous work needed to differentiate between the two called for experts who examined cases, created criteria of discernment, and wrote reports, generating a large literature on the subject. In the fourteenth century Brigit of Sweden became a test case for spirit discernment when she was examined by a six-man panel: an archbishop, three bishops, a theologian, and an abbot. A panel of knowledgeable ecclesiastics also examined the case of Catherine of Siena, who died in 1380 of self-imposed starvation, and who, like Brigit, was subsequently canonized.3

Archbishops and abbots were not scientists—indeed, it would be many centuries before the term “scientist” would be coined—but they were individuals who had specialized knowledge relevant to the problem at hand. In that sense, they were experts whose views might inform action, including such weighty matters as canonization.

The rise of the modern nation-state brought new concerns and ways of constructing expertise. In eighteenth-century France, a group of prototechnocratic military engineers proposed a new form of artillery that could be produced faster and more cheaply through the use of interlocking and interchangeable gun parts. While perhaps less accurate and less durable than the heavy cannons they replaced, these new weapons could be quickly moved and mobilized, allowing French military strategists to think beyond the established rules of siege warfare and improve France’s national security. This change meant rearranging the traditional relationships between the state, its armories, and the military. Armories now adopted a “systems” approach: traditional artisans were replaced by managers and planners who could arrange the work of machines and laborers into an organized whole. Social relationships were also restructured: titled lords who raised their own troops were now replaced by salaried professional officers who trained with designated artillery troops year round. Military leadership was now a career.4

A rival group of more traditionally minded military experts challenged these reforms. While no doubt motivated at least in part by a desire to protect traditional positions and privileges, they argued that the traditional techniques were more effective on the battlefield. To resolve this dispute, the minister of war convened a blue-ribbon panel of field marshals who had commanded French troops during the Seven Years’ War (which raged from 1756 to 1763 and cost about one million lives). The panel—along with members of the public—witnessed a set of field exercises designed to demonstrate the efficacy of the new approach. The marshals sided with the reformers, and state policy was formulated to embrace the new approach.5 Acknowledged expertise was now informing military policy.

THE RISE OF THE SCIENTIFIC AND TECHNOLOGICAL EXPERT

The nineteenth century witnessed a dramatic increase in activities that we now label scientific, as well as the increased visibility and social capital of savants who identified themselves by their disciplines—geologists, biologists, chemists, and the like. These “men of science” would soon come to be known collectively as scientists.6 With the growth of science as a professional activity, these experts were increasingly called upon to resolve disputes that were understood to be both scientific and social and to produce reports of their findings. Here we may identify what we might consider to be early forms of the modern scientific assessment. Assessment in this context would mean any attempt to review the state of expert knowledge in relation to a specific question or problem, judge the quality of the available evidence, and offer findings relevant to the solution of the problem.7

In France, Louis Pasteur and Felix Pouchet argued about the nature of life and the fixity of species, with the latter advocating the theory of spontaneous generation and the former challenging it. Pasteur is popularly regarded as having debunked the theory of spontaneous generation via a strict adherence to scientific method, but both sides at the time offered experimental evidence in support of their claims. The Académie des Sciences deemed it so important to resolve this issue that it formed not one but two special commissions to judge the two sets of experiments.8 While one Pasteur biographer bemoaned this approach as unsuitable to resolving a scientific dispute, the Académie did, in fact, settle the issue this way, siding with Pasteur and awarding him the Alhumbert Prize in 1862 for his experimental refutation of spontaneous generation theory.9

In the late nineteenth- and early twentieth-century United States, diphtheria was prevalent in urban areas, and recurrent outbreaks took the lives of many thousands of children. Historian Evelyn Hammonds notes that popular accounts generally suggest that the Pasteurian bacteriological model of disease led directly to new forms of medicine, including the use of antitoxins, but this is not in fact the case. When confronted with diphtheria epidemics in late nineteenth-century New York City, the medical community resisted both the bacteriological definition of disease and treatments whose justification rested on it. Partly, this was due to the medical profession’s interest in maintaining its authority over disease as well as physicians’ financial self-interest, but there was an epistemic issue at stake as well: at that time bacteriologists could neither account for nor control nonsymptomatic carriers of the diphtheria bacillus.10

Health department statistics showed a marked drop in mortality related to the use of diphtheria antitoxin, but many physicians remained skeptical. In 1896, the American Pediatric Society formed a committee to investigate antitoxin use.

The society’s commission drew upon the clinical experience of 613 private physicians in its membership. After reviewing 3,384 cases, the commission ruled in favor of antitoxin and recommended that it be used in all cases as early as possible.11 This still did not solve the problem of asymptomatic carriers but it did resolve the social question of whether physicians should embrace diphtheria antitoxin treatment, which at this point most did.

Vaccination was a major domain of expert assessment in the nineteenth century, because of both physician skepticism and public resistance. Historians and public health officials have noted that the British Vaccination Acts of the 1840s and 1850s—which mandated childhood vaccination for smallpox and outlawed variolation (the long-standing practice of exposing people to bodily fluids taken from a person with a live case of the disease)—were resisted by opponents who saw the acts as infringements on civil liberty. This resistance took the form of antivaccination leagues, protest, civil disobedience, and even riots. In response, a Royal Commission was established in 1885 to hear evidence for and against vaccines. The commission sat for seven years, during which it held 136 meetings, heard testimony from 187 witnesses, and examined two children suffering from ill health alleged to have been caused by smallpox vaccination; the final report extended to more than 500 pages.12 Among those who testified against vaccination was Alfred Russel Wallace, the codiscoverer of the theory of evolution by natural selection, who argued that the recent observed decreased smallpox mortality was largely due to improvements in sanitation, not vaccination.13

The committee’s charge was to consider both scientific and social questions regarding vaccination, though it did not sharply distinguish them in this way. The committee’s members were scientific men, such as professor of anatomy and physiology and fellow of the Royal Society Michael Foster, but its chair was Farrer Herschell, a lawyer and lord chancellor of England. Scientifically, the committee asked whether there was a theoretical basis for believing that smallpox vaccination would be protective and, irrespective of theoretical understanding, whether there was sufficient empirical evidence to conclude that it is. Socially, the panel recognized the reality of objection and noncompliance. To discern and comprehend these social realities, the committee solicited extensive testimony from those whose objections were moral, philosophical, or personal. This led to a broadly framed discussion that included questions of compulsion and penalties for noncompliance. Among the topics discussed were the harsh treatment of parents by magistrates and the unfairness that ensued when parents who continued to refuse to vaccinate their children were repeatedly fined for what was, in effect, a single infraction. The commission’s final report concluded that vaccines did protect against smallpox but recommended the abolition of penalties for noncompliance with the vaccination law. The new Vaccination Act of 1898 reflected this change and introduced a “conscience clause” allowing parents to decline vaccination on grounds of personal belief.14

As we will see in the chapters that follow, in the twentieth century, many scientists, legal scholars, and others would argue for a sharp separation between science and policy, but this distinction was not one about which the participants in the vaccination commission were unduly concerned.15 Another difference between the vaccination report and most twentieth-century assessments is the inclusion of a detailed dissenting opinion. The 1885 Royal Commission Report includes a report of over 150 pages by the “dissentients,” W. J. Collins and J. Allanson Picton.16 The former was a physician, the latter an independent member of Parliament who had been accused of heresy for his unorthodox religious (and perhaps, as well, his radical political) views. Like Wallace, they were not persuaded that vaccination was the principal reason for declining smallpox mortality and therefore argued that it would be “unwise to attempt to enforce vaccination on those who regard it as useless and dangerous”—a position that the rest of the panel essentially accepted, insofar as they recommended abolishing penalties for noncompliance. However, the “dissentients” went further, arguing that “it would be simpler and more logical to abolish compulsory vaccination altogether.”17

Social problems that required expertise to resolve often involved tensions over who had the relevant expertise and authority to lay claim to a particular domain. In Victorian-era Britain, Franz Mesmer’s theory of animal magnetism, popularly known as mesmerism, was fashionable. Mesmerists claimed to be able to exert mental control over the minds and bodies of others and to use this power to cure psychological illnesses and anesthetize patients for surgery, sometimes simply by the laying on of hands. Popular audiences welcomed these claims, but they potentially undermined the authority of emerging scientific and medical professionals.18 Moreover, the mid-nineteenth century was a time when a nonmedical perspective on madness—which saw it as a moral defect rather than a brain dysfunction—was threatening the medical profession.19 To counter this, doctors were eager to find new theories and therapies that could be integrated within their own naturalistic paradigm of madness. Phrenology, for example, became one route through which doctors could explain moral defects in physical terms.

The acceptance and use of what many would today consider pseudoscience by physicians in the service of maintaining their expert authority makes it difficult to characterize the mesmerism debate as science versus pseudoscience. In fact, the matter of where to draw the line between science and pseudoscience, medicine and quackery, was settled not so much by knowledge but by disciplinary boundary work: the drawing of expert boundaries in a manner that relegated mesmerism to the fringe—defining away the mesmerist’s expertise.20 Here we see illuminated one feedback dynamic of knowledge production (discussed further in chapter 5): professional expertise helps to resolve social problems and these resolutions help to define what constitutes pertinent expertise. Socially acknowledged problems become the contested space within which professional groups define their collective identity, stake their professional claims, and forge agreement on what constitutes knowledge.21

As the category of “scientist” became solidified in the late nineteenth and early twentieth centuries, it also became codified as a recognized locus of specialized knowledge on which society could draw to help resolve contested questions. Questions regarding madness and normalcy, disease and health, and technology and its impacts came to be viewed increasingly as the domain of science, and so society increasingly turned to scientists to answer questions about them. The industrialized nation-state needed diverse forms of technical expertise in order to run its affairs; experts with knowledge were becoming increasingly viewed as important, even essential.22 Scientists became the designated experts to help resolve a variety of societal problems, many of which were themselves consequences of science- and technology-inspired modernization. In the twentieth century, Lewis Mumford labeled these the questions of “technics and civilization.”23

Technics were the focus of numerous commissions in the nineteenth and twentieth centuries in Europe and the United States. Typically, the impetus was failure: problems with steam engines, boats, and railroads, and especially collapsing bridges. Exploding boilers were a persistent and deadly p...