Counterfactual Thought Experiments in World Politics

Logical, Methodological, and Psychological Perspectives

Philip E. Tetlock, Aaron Belkin, Philip E. Tetlock, Aaron Belkin

• Publisher: Princeton University Press
• Year: 2020
• ISBN:
  9780691215075
• Topic: Política y relaciones internacionales
• Subtopic: Historia y teoría política

Political scientists often ask themselves what might have been if history had unfolded differently: if Stalin had been ousted as General Party Secretary or if the United States had not dropped the bomb on Japan. Although scholars sometimes scoff at applying hypothetical reasoning to world politics, the contributors to this volume--including James Fearon, Richard Lebow, Margaret Levi, Bruce Russett, and Barry Weingast--find such counterfactual conjectures not only useful, but necessary for drawing causal inferences from historical data. Given the importance of counterfactuals, it is perhaps surprising that we lack standards for evaluating them. To fill this gap, Philip Tetlock and Aaron Belkin propose a set of criteria for distinguishing plausible from implausible counterfactual conjectures across a wide range of applications.
The contributors to this volume make use of these and other criteria to evaluate counterfactuals that emerge in diverse methodological contexts including comparative case studies, game theory, and statistical analysis. Taken together, these essays go a long way toward establishing a more nuanced and rigorous framework for assessing counterfactual arguments about world politics in particular and about the social sciences more broadly.

Part One

COUNTERFACTUAL INFERENCE: FORM AND FUNCTION

1

Counterfactual Thought Experiments in World Politics

LOGICAL, METHODOLOGICAL, AND PSYCHOLOGICAL PERSPECTIVES

PHILIP E. TETLOCK AND AARON BELKIN

THERE IS nothing new about counterfactual inference. Historians have been doing it for at least two thousand years. Counterfactuals fueled the grief of Tacitus when he pondered what would have happened if Germanicus had lived to become Emperor: “Had he been the sole arbiter of events, had he held the powers and title of King, he would have outstripped Alexander in military fame as far as he surpassed him in gentleness, in self-command and in other noble qualities” (quoted in Gould 1969). Social scientists—from Max Weber (1949) to Robert Fogel (1964)—have also long been aware of the pivotal role that counterfactuals play in scholarship on such diverse topics as the causes of economic growth and the diffusion of religious and philosophical ideas. Nevertheless, some contemporary historians still sternly warn us to avoid “what-might-have-been” questions. They tell us that history is tough enough as it is—as it actually is—without worrying about how things might have worked out differently in this or that scenario. Why make a difficult problem impossible? In this view (Fisher 1970; A. J. P. Taylor 1954), we do scholarship a grave disservice by publishing a volume on counterfactual reasoning. We are luring our colleagues “down the methodological rathole” in pursuit of unanswerable metaphysical questions that revolve around the age-old riddles of determinism, fate, and free will (Fisher 1970, 18).

The ferocity of the skeptics is a bit unnerving. Moreover, they are right that counterfactual inference is dauntingly difficult. But they are wrong that we can avoid counterfactual reasoning at acceptable cost. And they are wrong that all counterfactuals are equally “absurd” because they are equally hypothetical (Fisher 1970, 19). We can avoid counterfactuals only if we eschew all causal inference and limit ourselves to strictly noncausal narratives of what actually happened (no smuggling in causal claims under the guise of verbs such as “influenced,” “responded,” “triggered,” “precipitated,” and the like). Putting to the side whether any coherent and compelling narrative can be “noncausal,” this prohibition would prevent us from drawing the sorts of “lessons from history” that scholars and policy makers regularly draw on such topical topics as the best ways to encourage economic growth, to preserve peace, and to cultivate democracy. Without counterfactual reasoning, how could we know whether state intervention accelerated growth in country x, whether deterrence prevented an attack on country y, or whether the courage of a young king saved country z from sliding back into dictatorship? Counterfactual reasoning is a prerequisite for any form of learning from history (cf. Tetlock 1991). To paraphrase Robert Fogel’s (1964) reply to the critics of “counterfactualizing” in the 1960s, everyone does it and the alternative to an open counterfactual model is a concealed one.

This volume surveys the many roles that counterfactual arguments play in the study of world politics. A useful place to begin is by clarifying what we mean by counterfactual reasoning. A reasonably precise philosophical definition is that counterfactuals are subjunctive conditionals in which the antecedent is known or supposed for purposes of argument to be false (Skyrms 1991). As such, an enormous array of politically consequential arguments qualify as counterfactual. Consider the following rather representative sample of counterfactuals that have loomed large in recent scholarly and policy debates:

If Stalin had been ousted as general party secretary of the communist party of the Soviet Union, the Soviet Union would have moved toward a kinder, gentler form of communism fifty-five years before it actually did;

If Yeltsin had followed Sachsian fiscal and monetary advice in early 1992, Russian inflation in 1993 would have been a small fraction of what it was;

If the United States had not dropped atomic bombs on two Japanese cities in August 1945, the Japanese would still have surrendered roughly when they did;

If all states in the twentieth century had been democracies, there would have been fewer wars;

If Bosnians had been bottlenosed dolphins, the West never would have allowed the slaughter of innocents in the Yugoslav civil war to have gone on so long.

The contributors to this volume approach counterfactual inference from both normative/epistemological and descriptive/cognitive science perspectives. The normative issues—which we explore in the next two sections of this chapter—focus on how students of world politics should use and judge counterfactual arguments. We break these issues into two categories:

(1) In what ways do counterfactual arguments advance our causal understanding of political events? Are such arguments—as the skeptics insist—merely forms of rhetorical posturing? Or can such arguments sensitize us to historical and theoretical possibilities that we might otherwise have ignored? Although we do not doubt that true believers often use counterfactuals to justify predetermined conclusions, it is a mistake to dismiss all such arguments as thinly veiled tautologies. We advance a provisional taxonomy of live constructive functions of counterfactual arguments in world politics, illustrating each with examples drawn from chapters in this volume.

(2) Once we settle on the appropriate purposes of counterfactual inference, what criteria should we use to distinguish plausible from implausible, insightful from vacuous arguments? Although we recognize the need for somewhat different criteria for distinctive “ideal-type” functions of counterfactuals, we see an even more pressing need to be explicit about the standards that scholars use in evaluating competing claims. There is an unfortunate tendency in the scholarly literature to oscillate between the extremes of dismissing dissonant counterfactuals as hopelessly speculative and of proclaiming favorite counterfactuals as self-evidently true, indeed as factual. This reaction is understandable, but unhelpful. The choice is typically not dichotomous; as we shall see, counterfactuals vary along a plausibility (or, if you are a Bayesian, subjective probability) continuum. If debates over competing counterfactuals are not to reduce to expressions of theoretical or ideological taste, we need to articulate standards of evidence and proof that transcend rival schools of thought. In this spirit, we advance a provisional list of six standards for judging counterfactual claims, illustrating each standard with examples drawn from later chapters.

The final section of this chapter shifts the focus from “how should we generate, use, and judge counterfactual arguments?” to “how do we generate, use, and judge counterfactual arguments?” One key cognitive-science question concerns when people are prone to think about possible worlds. Of the infinity of past events that people could “mentally undo” and insert as antecedents into counterfactual arguments, why do they devote so much attention to certain causal candidates and so little to others (Kahneman and Miller 1986; Commentary 2, Olson, Roese, and Deibert)? A natural next question concerns when people are likely to be persuaded by counterfactual claims concerning the consequences of altering particular antecedents. Given that people have no way of directly determining what would have happened in these hypothetical worlds, why do they defer to some counterfactual arguments but disdain others (Commentary 1, Turner)? Finally, we explore the potential for double standards in so subjective a domain as thought experiments. Is there evidence of cognitive and motivational biases in how people judge claims about possible worlds, tendencies to raise standards of evidence and proof for dissonant counterfactuals but to lower standards for claims consonant with one’s beliefs and goals?

Normative Issues in Evaluating Counterfactual Claims

Our contributors generally agree that counterfactual reasoning is unavoidable in any field in which researchers want to draw cause-effect conclusions but cannot perform controlled experiments in which they randomly assign “subjects” to treatment conditions that differ only in the presence or absence of the hypothesized cause. Try though we do to control statistically for confounding variables in large-N multivariate studies or to find matching cases in comparative designs or to search for the signature of hypothesized causes in process-tracing studies, the potential causes are simply too numerous and too interrelated in world politics to permit complete escape from counterfactual inference. Researchers must ultimately justify claims that a given cause produced a given effect by invoking counterfactual arguments about what would have happened in some hypothetical world in which the postulated cause took on some value different from the one it assumed in the actual world (Fogel 1964; Fearon 1991).

The consensus among our contributors, however, begins to unravel beyond this point. They emphasize distinctive, albeit largely complementary, functions of counterfactual reasoning. The arguments they present have persuaded us to adopt a stance of epistemic pluralism that acknowledges the variety of ways in which counterfactual arguments can prove enlightening and the need for different standards in judging counterfactuals that serve different scholarly goals. We organize these distinct styles of counterfactual argumentation into five ideal types:

1. Idiographic case-study counterfactuals that highlight points of indeterminacy at particular junctures in history (reminding us of how things could easily have worked out differently and of how difficult it is to apply abstract hypothetico-deductive laws to concrete cases);

2. Nomothetic counterfactuals that apply well-defined theoretical or empirical generalizations to well-defined antecedent conditions (reminding us that deterministic laws may have been at work that were invisible to the original historical actors as well as to contemporary scholars who insist on a radically idiographic focus on the particular);

3. Joint idiographic-nomothetic counterfactuals that combine the historian’s interest in what was possible in particular cases with the theorist’s interest in identifying lawful regularities across cases, thereby producing theory-informed history;

4. Computer-simulation counterfactuals that reveal hitherto latent logical contradictions and gaps in formal theoretical arguments by rerunning “history” in artificial worlds that “capture” key functional properties of the actual world;

5. Mental-simulation counterfactuals that reveal hitherto latent psychological contradictions and gaps in belief systems by encouraging people to imagine possible worlds in which causes they supposed irrelevant seem to make a difference, or possible worlds in which causes they supposed consequential seem to be irrelevant.

Five Styles of Counterfactual Argumentation

1. Idiographic

Several authors use counterfactuals to explore “possibility-hood”—whether history had to unfold as it did. For instance, Breslauer (Chapter 3) explores the several junctures in the history of the Soviet Union that have sparked the most intense counterfactual debate within the expert community: Was the Bolshevik revolution inevitable given the Russian defeat in World War I? Was Stalinism inevitable given the vanguard-party legacy of Leninism? Was Gorbachevism inevitable given the repressive stagnation of Brezhnevism? And was the disintegration of the Soviet Union inevitable given the liberal reforms of Gorbachevism? Khong (Chapter 4) attempts to assess whether any conceivable British prime minister would have adopted a policy of appeasement toward Nazi Germany, at least up to March 1939. Herrmann and Fischerkeller (Chapter 6) examine several counterfactual controversies in which the positions taken by policy makers on “what would have happened?” shaped American policy toward Iran during the Cold War. Lebow and Stein (Chapter 5) construct an exhaustive inventory of the counterfactual beliefs that apparently guided American and Soviet policy during the Cuban missile crisis—the crisis during which, it is often asserted, the world “came closer” than ever before or since to nuclear war.

These diverse applications all use counterfactuals to focus on “conceivable” causes that could have easily redirected the path-dependent logic of events (cf. Hawthorn 1991; Chapter 2, Fearon). In each case, the investigators want to know what was historically possible or impossible within a circumscribed period of time and set of relations among political entities. To make this determination, they draw upon combinations of: (a) in-depth case-specific knowledge of the key players, their beliefs and motives, and the political-economic constraints under which they worked; and (b) general knowledge (nomothetic propositions) concerning cause-effect relations in human behavior and political-economic systems. Moreover, our case-study authors seem to agree that counterfactual speculation should be constrained by some form of “minimal-rewrite-of-history” rule that instructs us to avoid counterfactuals that require “undoing” many events—counterfactuals that, for instance, ask us to imagine a democratic Soviet Union at the end of World War II or Soviet possession of strategic nuclear superiority at the time of the Cuban missile crisis. A more fruitful way to proceed is to ask what could have worked out differently if we introduce easily imagined variations into the causal matrix of history. Might the murderous tyranny of Stalin have been averted if Trotsky had not gone duck hunting, caught a cold, and missed a key politburo meeting or if Bukharin had been a savvier politician? Might World War II have been nipped in the bud if British opponents of appeasement had had one or two additional cabinet seats during the Munich crisis? And might World War III have been triggered in October 1962 if Kennedy had followed the advice of his more hawkish advisors and immediately ordered air strikes against Soviet missile sites in Cuba?

These idiographic counterfactuals are not idle exercises in social-science fiction; they are a useful corrective to simple deterministic forms of theory. They compel us either to abandon determinism by acknowledging the role of chance or to abandon simplicity by acknowledging that factors outside the purview of our deterministic models—viruses, skillful or inept leadership, group dynamics, a well-timed or ill-timed persuasive argument—can decisively alter the course of events.

Beyond their heuristic contribution to social science theory, idiographic counterfactuals are an integral part of the process of passing moral judgment on individual leaders and even entire political systems such as Marxism-Leninism. We rely on them in attributing responsibility (Hart 1961). Would a reasonable person, confronted by these circumstances, have acted differently? Should a particular leader be praised for performance above the norm (spectacular prescience or courage) or condemned for performance below the norm (stubborn refusal to recognize trends apparent to others or cowardly failure to protest immoral conduct)? Neville Chamberlain, John Kennedy, Nikita Khrushchev, and Lyndon Johnson are all, in a sense, in the docket with their reputations as wise leaders hanging in the balance on counterfactual judgments of what they could or should have done at certain junctures in history.

2. Nomothetic Theory-Testing

Whereas idiographic investigators are interested in conceivable causes that they can readily imagine taking on different values within a specific historical context, nomothetic investigators usually show little or no concern for the plausibility of switching the hypothesized counterfactual antecedent on or off in any given context. From this perspective, counterfactuals are the inevitable logical by-products of applying the hypothetico-deductive method to an historical (nonexperimental) discipline such as world politics. Whenever we combine a well-defined Hempelian covering law (say, relating money supply to inflation) with well-defined antecedent conditions (the Russian economy in January 1992), we can deduce specific counterfactual conclusions (e.g., if the Russian central bank had adopted this or that monetary policy, then, ceteris paribus, inflation would have taken on this or that value). Note that these counterfactuals are in no way constrained by the historical plausibility of the Russian central bank adopting one or another policy. The counterfactual “predictions” follow from the context-free logic of macroeconomic theory, not from the context-bounded logic of what was psychologically or politically possible at that juncture in Russian history. Adopting Fearon’s (Chapter 2) terminology, these nomothetic counterfactuals invoke mira...