Masters of Uncertainty

Weather Forecasters and the Quest for Ground Truth

Phaedra Daipha

• Publisher: University of Chicago Press
• Year: 2015
• Print ISBN: 9780226298689, 9780226298542
• eBook ISBN: 9780226298719
• Topic: Social Sciences
• Subtopic: Meteorology & Climatology
• Index: Social Sciences

Though we commonly make them the butt of our jokes, weather forecasters are in fact exceptionally good at managing uncertainty. They consistently do a better job calibrating their performance than stockbrokers, physicians, or other decision-making experts precisely because they receive feedback on their decisions in near real time. Following forecasters in their quest for truth and accuracy, therefore, holds the key to the analytically elusive process of decision making as it actually happens.

In Masters of Uncertainty, Phaedra Daipha develops a new conceptual framework for the process of decision making, after spending years immersed in the life of a northeastern office of the National Weather Service. Arguing that predicting the weather will always be more craft than science, Daipha shows how forecasters have made a virtue of the unpredictability of the weather. Impressive data infrastructures and powerful computer models are still only a substitute for the real thing outside, and so forecasters also enlist improvisational collage techniques and an omnivorous appetite for information to create a locally meaningful forecast on their computer screens. Intent on capturing decision making in action, Daipha takes the reader through engrossing firsthand accounts of several forecasting episodes (hits and misses) and offers a rare fly-on-the-wall insight into the process and challenges of producing meteorological predictions come rain or come shine. Combining rich detail with lucid argument, Masters of Uncertainty advances a theory of decision making that foregrounds the pragmatic and situated nature of expert cognition and casts into new light how we make decisions in the digital age.

One

The Weather Prediction Enterprise

Norm is in the process of publishing his forecast. The office bandwidth cannot handle sending out all thirty-nine forecast elements at once, so he has broken down the task into five chunks, “just to be safe.” He is already half an hour past the dissemination deadline. Just then, a request comes in from the neighboring office to the north via the chat room: “Can you change the speed of your sky cover so we come into better agreement tomorrow?” Norm pulls up the neighboring forecast on his left screen, muttering to himself, “Join us at the thirteenth hour, why don’t you.” He turns to look again at model guidance and satellite imagery on his right screen: “Where the hell is he getting that?” With a long sigh, he types in the chat room, “Can I do it? Yes. Do I think it’s going to happen? That’s a different answer. I’ll push the sky cover to 40% at [7:00 a.m.] and go from there.” He goes back into his forecast to “fix” the sky cover. Turns to me: “That’s some fine meteorology right there. I hope you’re writing all this down.” About twenty minutes later, over an hour past the dissemination deadline, he starts publishing his forecast all over again. Suddenly, out of nowhere, he bangs the table with both hands and exclaims at his computer screens, his face contorted with rage: “Come on, damn it, move!” He’s been really agitated with the new forecasting technology all day, but I have never seen him be violent before. Everyone on the operations deck turned around startled, then quietly resumed their duties.

The following day, “the Norm incident” is making the rounds in the office. Part of the fascination appears to be with how “thrown off” I had been by Norm’s behavior. Apparently, I jumped up in my seat and then got very quiet for the rest of the day. People tell me these kinds of outbursts are not out of the ordinary: “you’ll get used to them if you stick around.” Peter is filling in Simon on what happened, adding in a grave voice, “everybody deals with their frustration differently.”

Simon: [Shaking his head] It takes me, who knows the system inside and out, two hours to issue a warning. That’s not right! People are going to implode at this desk one day. We need more bandwidth. It’s ridiculous!

Peter: It’s not just the bandwidth, though. Or all the bugs. Or the collaboration issues. Some of the stuff we do with [this technology] is almost beyond us. There is no skill. There is no way of verifying. That hurts our credibility. Who are we supposed to be forecasting for with this thing?

* * *

To begin to understand the process of decision making, one must begin by understanding the more or less institutionalized field of action in which it is fundamentally embedded. What is at issue here is not that decision making is social action but that it is social action precisely because it is practical action. Meteorological decision making would not be possible, let alone successful, without recourse to some notion of what counts as a weather risk, specific tools and techniques for predicting such a risk, or a sense of what constitutes a good weather prediction. And the same holds true for any kind of problem-solving or decision-making task. Yet, under normal conditions, the character of decision making as a social phenomenon has a way of receding into the background. Even within organizations, constituted precisely in order to tackle complex tasks most efficiently, it is easy to lose sight of the fact that every decision represents a communal achievement. This is because much of that communal work is typically unremarkable and invisible, hidden in the accepted standards, technologies, and rules of thumb we use to guide our judgments and decision making. In fact, the more invisible and taken for granted the environment in which decision makers operate, the more far-reaching its role in decision-making practice. But the more far-reaching its role, the more streamlined, efficient, and masterful decision-making action is bound to be. It is thus a testament to the importance of the decision-making environment that it is typically hidden from view, the center stage occupied instead by individual actors intrepidly confronting challenging and risky situations with great skill and resourcefulness.

It takes an extraordinary event to reveal the ordinary, invisible work that decision-making standards do for a community of experts. And indeed, it took a crisis for me to properly appreciate the hold their tools have on NWS forecasters. This crisis was not precipitated by an accident or some other decision-making failure—it was precipitated by a change in equipment. Soon after I arrived at the Neborough office, NWS forecast offices switched from producing a text weather forecast to producing a graphical weather forecast. Already contentious before its official launch, the new forecasting technology now became a daily source of struggle and frustration, as the above excerpt from my field notes illustrates. So much were forecasters accustomed to particular established ways of doing and reasoning that the implementation of the new technology produced a profound disorientation, extending to the very logic of decision-making action. A closer look at the institutionalization of the new NWS forecasting routine, therefore, offers a rare view into the normally hidden environmental forces regulating the process of meteorological decision making.

It is to that task that I devote this chapter after situating NWS forecasting operations within the field of U.S. weather forecasting. Through an examination of the unfolding and eventual closure of the recent NWS controversy, I introduce readers to the institutionalized environment in which NWS forecasters operate and to the professional norms, practices, and technologies through which its logic becomes articulated on the ground. By analytically grounding the discussion in the experiences and point of view of Neborough forecasters as they underwent this by all accounts painful operational transition, the aim is to provide a balanced perspective on how institutional factors do, and do not, influence decision-making action.

Enter the National Weather Service

Historical studies of meteorology go a long way toward furnishing the necessary framework for understanding the forces that gave rise to and continue to shape weather forecasting as a system of expertise. The weather has of course always struck fear and awe in the hearts of men: efforts to reign in its power by making it predictable have existed since before the beginning of recorded time, with the earliest known effort to systematize and theorize atmospheric physics being Aristotle’s Meteorologica. But it was not until the late eighteenth century—and the spirit of Enlightenment sweeping Europe and the United States—that weather forecasting as an enterprise came into being (Golinski 2007). The catalyst was a relatively steady supply of weather reports as taking meteorological observations turned into a gentlemanly pastime (Janković 2000). Soon, local initiatives became more or less absorbed into scientific societies, such as the Royal Society of London or the Smithsonian Institution, and, aided by the adoption of the telegraph, there emerged a stable, albeit thin, network of weather observers spanning commercial telegraph stations, military hospitals, army posts, school academies, and colleges (Fiebrich 2009). It was the mounting demands for financial and human resources by these expanding weather observation networks around the globe that led to the establishment of dedicated, national weather agencies and the standardization of weather instruments and measurements in the mid-nineteenth century (Whitnah 1961; Hughes 1970; Fleming 1990). Data alone, however, did little to establish weather prediction as a science. If anything, the relative ease with which a wide variety of data on the weather could be collected intensified jurisdictional wars in the form of debates over the scientific merit of local weather versus global atmospheric systems, observation versus speculation, and reportage of unusual weather phenomena versus regular weather records (Jancović 2000; Anderson 2005). Nor was the development of numerical weather prediction models in the early twentieth century enough to legitimize weather forecasting as a science—such efforts were in fact greeted as premature and misguided at the time (Friedman 1989; Lynch 2006). It took until the midcentury, when observing networks and numerical weather prediction modeling were harnessed to the computing power of machines—all thanks to the close links between the meteorological community and the military galvanized during the Second World War—for meteorology to emerge as a scientific profession in its own right (Harper 2003, 2008; Lynch 2006). And it was not until it had risen to the status of a science that meteorology was able to unify the communities of observers, theorists, and forecasters under one discipline—that of predicting the weather (Nebeker 1995).

This laconically brief historical outline of the professionalization of weather forecasting hardly constitutes a complete explanation of the phenomenon, of course. Equally importantly, it glosses over the development of the distinctive national traditions of weather forecasting that in their interaction brought forth the current “epistemic culture” (Knorr Cetina 1999) of the discipline. It does, however, help highlight key interrelated regularities structuring the field: (1) a long-standing and extensive weather observing network buttressed by a project of “infrastructural globalism” (Edwards 2006, 2010), (2) quantitative forecast models generated by computers powerful enough to process the massive amount of available data, and (3) wide-ranging and keen stakeholders given the universal relevance and potential destructiveness of the weather. It is this last that has provided meteorologists with the collaborative data- and computer-intensive environment necessary for countering uncertainty, long before most other decision-making fields were able to muster similar resources and support. Nowhere are these three elements better aligned or more pronounced than in the operations of government weather organizations. Enter the NWS.

The NWS was established in 1870, just one year after the system of weather telegraphy—begun in the 1850s at the Smithsonian Institution but severely disrupted by the Civil War and a fire—was restarted once again at the Cincinnati Observatory (Fleming 1990, 141–62). Originally named “The Division of Telegrams and Reports for the Benefit of Commerce,” it was nonetheless assigned to the Army Signal Corps and placed under the Department of War because “military discipline would probably secure the greatest promptness, regularity, and accuracy in the required observations” (Cox 2002, 95). Twenty years later it became a civilian organization, renamed Weather Bureau, and transferred to the Department of Agriculture, where it remained until 1940, to be transferred again to the Department of Commerce because of its importance for the growing aviation industry. In 1965, the Weather Bureau took on its current name and was assigned to the Environmental Science Services Association, which, five years later, became NOAA, the National Oceanic and Atmospheric Administration (White 2006). Today, the NWS is the world’s largest meteorological organization, with about five thousand employees, a budget of close to a billion dollars (NWS 2012b), and an average of four hundred thousand weather bulletins a day.

Per its mission statement, the NWS “provides weather, hydrologic, and climate forecasts and warnings for the United States, its territories, adjacent waters and ocean areas, for the protection of life and property and the enhancement of the national economy. NWS data and products form a national information database and infrastructure which can be used by other governmental agencies, the private sector, the public, and the global community.” To fulfill this mandate, the NWS maintains 122 forecasting offices around the country assigned to—and located in—a specific geopolitical region. It is these offices that are responsible for assembling weather data, creating forecasts and warnings for a prescribed area of forecasting responsibility, and disseminating weather information as appropriate. And it is these offices that allow the NWS agency to call itself “The People’s Weather Service” and to justify its existence as “Your Weather Service” to governors and taxpayers alike. What is commonly understood as the NWS forecast, then, actually involves over a hundred separately issued and disseminated NWS forecasts, each catering to a particular area of forecasting responsibility but digitally stitched together into an apparently seamless national whole.

To be sure, forecasting at the NWS is as “big science” as it gets: the most modest and localized of pronouncements about the weather necessitates a massive coordination of people, resources, and technologies. Weather balloons are simultaneously launched twice daily from hundreds of locations around the world in order to sample the lower and upper atmosphere; hundreds of automated weather stations take minute-by-minute surface weather observations; hundreds of human volunteers continue to do the same at least once a day; over a dozen computer models are run multiple times daily, digesting the above and more observation data and transforming them into forecast guidance; hundreds of meteorologists at the NWS alone assimilate all this information to meet the varied weather needs of an entire nation.

The NWS solution to this massive coordination problem has been to surround its local forecasting operations with regional and national “Support Centers,” with the majority of resources being concentrated under the auspices of the National Centers of Environmental Prediction, such as the Environmental Modeling Center, the Hydrometeorological Prediction Center, the Storm Prediction Center, the National Hurricane Center, and the Ocean Prediction Center. A typical NWS forecast may thus be produced by a single meteorologist stationed at one of 122 field offices, yet it embodies a tremendous organizational achievement. Within such a scheme, what counts as a good forecast will not infrequently be at odds with what forecasters themselves might consider appropriate, so that NWS forecasting practice effectively embodies a ceaseless negotiation among various competing logics of telling the weather.

Against this background, the decentralized organizational solution adopted by the NWS looms large and demands further attention. For it is indicative of a profound and deep-seated mind-set, pervading the entire agency, that sees it as unavoidable, if not necessary, that forecast offices be allowed to self-determine how operational directives should actually be operationalized in their jurisdiction. So much so, in fact, that, upon comparing the cultures of three forecast offices, Fine (2007, 71) arrives at the conclusion that in “the local offices of the NWS, it is almost as if 122 organizational experiments are running simultaneously.” The rationale given by both NWS administrators and forecasters as the basis for this state of affairs is the same one presumably militating for the existence of field offices in the first place: the local particularities of weather and the particular weather requirements of local communities. Crucially, therefore, forecasters are deferred to not simply because they are presumed to be experts of the local indeterminacies of weather but also because they are presumed to be experts of its publics as well. Indeed, what gives them an edge over other government employees is not that they are scientists but that they are public scientists. That is why they ostensibly have the final word on the NWS forecast. That is why computer model forecasts but also forecasts from the Storm Prediction Center are expressly meant to serve as guidance only.

Not that NWS forecasters actually are public scientists. To be sure, NWS forecasting has a direct effect on the general public and as such is a “public-domain science” (Collins and Evans 2002, 2006). But it no longer satisfies the more restrictive definition of a “public science” because its primary audience is no longer the general public.1 While NWS forecasts and related data are—still—freely accessible over the Internet by any member of the public, it is in fact the media and the private weather industry that serve as the primary producers of the weather information actually consumed by the American public today (Pew Research Center 2011). Certainly it falls on the NWS, qua government agency, to provide the official weather story of what actually transpired, and only the NWS may issue public alerts about impending hazardous weather. Yet long gone are the days when the media simply served as mediators of the NWS word. With weathercasters now expected to have a college degree in meteorology,2 with a substantial amount of NWS meteorological guidance publicly available and most other model and observation data easily procurable at a nominal fee, it has become increasingly difficult to defend the claim that the weather forecasts featured on broadcast media are reproductions of the NWS forecast. Indeed, the television meteorologists I interviewed in the Neborough media market countered with bemused indignation any insinuation that they might be directly working off the NWS forecast, talking instead in great detail about their process of forecasting.

The shift from a quasi-monopolistic to a more competitive industry environment was already in progress in the late nineteenth century when private investors began realizing the economic value of weather forecasting services (Craft 1999). Private sector meteorology became properly established right after the Second World War, when, quite independently from each other, several former military meteorologists saw a business opportunity in offering to weather-sensitive industries the kind of very customized and specific forecasts they had been generating for weather-sensitive military operations (Spiegler 1996). But the private sector truly exploded with the “computer revolution” of the 1980s and the relatively cheap availability of meteorological instrumentation (Spiegler 1996, 432). Today, no self-respecting television news station does not have—and appropriately advertise—its very own state-of-the-art weather radar. Having solved the problem of tools that effectively served as the gatekeeper of this “big science” occupational field, the private weather sector has grown exponentially in the last decades. Selling the weather pays, per the success of the Weather Channel and the persistent trends in public opinion surveys. Even as weather companies and media outlets compete among themselves for the attention of the American public, the NWS forecast has been increasingly forced out of popular print and broadcast media, with its last direct foothold lost in 2002, when the Weather Channel decided to start producing its own forecasts for its “Local Weather on the Eights.” In 1995 the NWS divested its agricultural weather program and its (nonwildfire) fire weather program to commercial interests, the latest example of its policy obligation to give “due consideration” to the abilities of the private weather sector (NOAA 2006).

In light, then, of the general trend toward the propertization of public science (Nowotny 2005) and its particular manifestation in how the NWS may pursue its public science mandate, it becomes especially important for understanding the process of meteorological decision making at the NWS to follow the NWS forecast both upstream and downstream, during its verification but also during its consumption. Nonetheless, approaching the study of the NWS as if its forecasts are primarily consumed by the general public has much to recommend it. First, as a sensitizing concept, the notion of NWS forecasting as public science encourages a sociological interest in the changing landscape of doing science in the public domain and lays the groundwork for assessing and reformulating policy objectives. Although such a line of inquiry is beyond its scope, this book helps make the case that there are compelling reasons why the demarcation between private and public science, while constantly challenged, is tenaciously maintained. One can certainly not equate the two sides, however arbitrarily defined they may be. Second, the notion of NWS forecasting as public science serves as an important heuristic tool for appreciating the “politics of representation” (Mehan 1993; see also Goodwin 1994) through which NWS definitions of meteorological risk gain supremacy, indeed orthodoxy, over all others. As Fine (2007) shows in his discussion of how NWS forecasters become authors of—and hence authorities on—the storm, critical for the success of this project is the concurrence of doing science and doing government work. Unlike the fragile power of soft-money scientists who represent a reserve labor force for the government (Mukerji 1989), NWS forecasters’ epistemic authority has been bolstered and secured by virtue of directly working for the government to protect the nation from the vicissitudes of the weather. Downstream, the deep uncertainty surrounding weather predictions and the heavy burden of impression management attached to any fortune-telling enterprise are considerably mitiga...

Table of contents

• Cover
• Title Page
• Copyright Page
• Dedication
• Contents
• Introduction: Decision Making under Uncertainty
• 1 The Weather Prediction Enterprise
• 2 Working the Weather: A Shift in the Life of a Weather Forecaster
• 3 Distilling Complexity: Atmospheric Indeterminacy and the Culture of Disciplined Improvisation
• 4 Managing Risk: The Trials and Tribulations of Hazardous Weather Forecasting
• 5 Anticipating the Future: Temporal Regimes of Meteorological Decision Making
• 6 Whose Weather Is It Anyway? From the Production to the Consumption of Decisions
• 7 Toward a Sociology of Decision Making
• Acknowledgments
• Notes
• References Cited
• Index