Multimethod Research, Causal Mechanisms, and Case Studies

An Integrated Approach

Gary Goertz

• Publisher: Princeton University Press
• Year: 2017
• ISBN:
  9781400888115
• Topic: Social Sciences
• Subtopic: Social Science Research & Methodology
• Index: Social Sciences

An innovative and accessible textbook on multimethod and case-study research Multimethod research has become indispensable to doing social science, and is essential to anyone who conducts large-scale research projects in political science, sociology, education, comparative law, or business. This authoritative and accessible book offers the first truly comprehensive approach to multimethod and case-study research, and is particularly aimed at students of qualitative methods in the social sciences.Walking step-by-step through these cutting-edge tools and techniques, Gary Goertz introduces a new integrated approach that unites three corners of a powerful research triad—causal mechanisms, cross-case causal inference, and within-case causal inference. He explains how the investigation of causal mechanisms and the making of within-case causal inference are the central goals of multimethod and case study research, and provides a logic for connecting case studies and causal mechanism analysis with cross-case analysis, whether they are statistical analyses, experiments, or QCA. In addition, Goertz analyzes how one can generalize using case studies, as well as systematically test game-theoretic and other models using multiple case studies.

• Provides a fully integrated approach to multimethod and case-study research
• An essential resource for students and researchers in political science, sociology, education, law, and business
• Covers constraint causal mechanism, game theory and case studies, QCA, and the use of case studies to systematically test and generalize theories
• An ideal textbook for a first-year graduate course in methods or research design

CHAPTER 1

Introduction

Singular causal claims are primary. This is true in two senses. First, they are a necessary ingredient in the methods we use to establish generic causal claims. Even the methods that test causal laws by looking for regularities will not work unless some singular causal information is filled in first. Second, the regularities themselves play a secondary role in establishing a causal law. They are just evidence—and only one kind of evidence at that—that certain kinds of singular causal fact have happened.

—Nancy Cartwright

THIS book addresses some of the core issues in doing (1) multimethod research, (2) causal mechanism analysis, and (3) case studies. Multimethod research has become very popular and almost a requirement for book-length studies. Multimethod can mean many things, but here it means combining case studies with statistics, qualitative comparative analysis (QCA), experiments, or game theory models. The purpose of case studies is to explore causal mechanisms at the heart of theories. One does case studies because cross-case methods give little purchase on the causal mechanisms (M_i) by which X produces Y. For multimethod researchers, showing a significant causal effect in a cross-case analysis is not sufficient; one needs to provide a causal mechanism and evidence for it. Demonstrating a causal effect is only half the job; the second half involves specifying the causal mechanism and empirically examining it, usually through case studies.

Causal mechanisms, cross-case analyses, and case studies form the research triad; see figure 1.1. This volume rests on the proposition that commitment to multimethod research is commitment to the research triad. Multimethod research typically is conceptualized as qualitative—within-case inference—along with quantitative cross-case inference. The research triad adds a third dimension to that, i.e., causal mechanisms. The research triad is an integrated approach because a commitment to multimethod research is also a commitment to a causal mechanism approach to explanation and social science research.

Figure 1.1: The research triad: causal mechanism, cross-case inference, and within-case causal inference.

One way to see the integration is by looking at research that focuses, e.g., 90 percent of the research effort, on just one corner of the triad. For many experimentalists most of the effort is devoted to determining the treatment effect. They usually do not talk about causal mechanisms per se. While they usually have theories and hypotheses, these all boil down to the one treatment effect. Game theorists provide an nice example of those who focus a lot of attention on the model, i.e., causal mechanism. It is quite possible to publish articles where essentially the whole article is the model (this is very true in economics). Often historians focus on single events and the whole focus of the article is explaining some individual historical event. They are not interested in generalization and maybe only very implicitly interested in causal mechanisms.

The integrated approach rests on a central claim:

As you move away from one-corner-only research you embrace the research triad.

For example, as a game theorist moves away from exclusive interest in the model, she begins to be involved in multimethod research, cross-case analyses, and within-case analyses. As soon as a case study researcher moves from one case to more cases, he is asking about generalization. Finally, as the Cartwright epigraph forcefully states, all statistical, experimental analyses and generalization imply individual case causal inference.

Good multimethod and causal mechanism research means a relative balance between the three corners of the research triad. If 90 percent of the effort is in demonstrating significant causal effects via cross-case analysis then the research is not serious multimethod or serious about the investigation of the causal mechanism. If the case studies are only “illustrations,” then there is little commitment to multimethod research. Conversely, research using case studies—e.g., the popular paired comparison—is weak on generalization. Doing five or six case studies—as is common in security studies books—deals poorly with cross-case analysis and generalization. The ideal is a fairly balanced effort on all three points of the triangle.

Case studies are often considered of questionable inferential value. Clarke and Primo illustrate this view of case studies. When they refer to case studies they almost always describe them as “exploratory”: “one can also design exploratory models with an eye toward explaining the events surrounding a specific case” (2012, 92). By “exploratory” they strongly imply the endeavor is not about causal inference. For example, their discussion of the analytic narratives project (Bates et al. 1998) falls into this category. When they talk about “empirical models” they mean statistical models. So a model of the US Senate—statistical or formal—is not exploratory, but a case study of the American Civil War in analytic narratives is exploratory. In contrast, the research triad emphasizes that case studies are about causal inference.

In everyday life—along with virtually all natural sciences—people successfully make individual-case causal inferences, e.g., origins of the universe, origins of the human species, why a given person died, why the Challenger shuttle exploded. None of these inferential successes relies on randomized treatments assigned to subjects, nor do they depend on conditional probabilities.

A core philosophy motivating this study is that we want to explain individual outcomes. Statistical analyses do not provide explanations: “There is little argument in political science that statistical models cannot serve as explanations in and of themselves. This belief manifests itself in the relegation of statistical models to devices for testing explanations” (Clarke and Primo 2012, 154).

The research triad assumes that one accepts the importance and value of causal mechanism analysis. One can find statistical methodologists who do not believe this is possible or important: “The importance of searching for causal mechanisms is often overestimated by political scientists, and this sometimes leads to an underestimate of the importance of comparing conditional probabilities. We do not need to have much or any knowledge about mechanisms in order to know that a causal relationship exists…. In general, as our understanding of an issue improves, studying individual cases becomes less important” (Sekhon 2004, 288–89; last sentence of the article). Gerring illustrates the skepticism about whether causal mechanism analysis is essential: “To clarify, this is not a polemic against mechanisms. It is a polemic against a dogmatic interpretation of the mechanismic mission. I argue that the analysis of causal mechanisms is best regarded as an important, but secondary, element of causal assessment—not a necessary condition” (Gerring 2010, 1500).

I do not survey or discuss how one goes about doing within-case causal inference in this book. There is a booming literature on the topic, e.g., Bennett and Checkel (2014), and Beach and Pedersen (2012; 2016) on process tracing, Mahoney (2012) on hoop and smoking gun tests, Goertz and Levy (2007), Levy (2008), and Harvey (2011) on counterfactuals. One can do statistical within-case causal inference. I take no position on how one does within-case inference. Similarly I do not cover how to do observational statistics, experiments, or QCA. As illustrated in figure 1.1 these methodologies provide input for the research triad but are not covered here.

“Qualitative” and “quantitative” are not very useful in describing or analyzing multimethod research. Instead of multi-method research as qualitative and quantitative, the research triad contrasts within-case causal inference (case studies) with cross-case causal inference (comparative case studies, statistical models, experiments, or QCA). This produces some surprising methodological bedfellows. Standard usage puts statistics in the quantitative category and set-theoretic approaches (e.g., QCA) in the qualitative. Experimenters spend a lot of time stressing the differences between experiments and observational research: here they are both cross-case methodologies. Similarly comparative case studies are cross-case analyses. I consider these all as versions of cross-case causal inference. In contrast, single case studies by their very nature are about what happens in individual cases. Case studies are fundamentally about within-case causal inference. The research triad means that multimethod research is multicausal inference analysis.¹ The causal inference techniques, procedures, and methodology of each type, cross-case and within-case, serve different but complementary goals.

The research triad works from a basic principle:

Multimethod work involves cross-case causal inference AND within-case causal inference.

Multimethod in this book means complementary causal inference methodologies. How one does cross-case inference or within-case inference is less important than the causal inference goals.²

To connect cross-case and within-case analysis means having a methodology for choosing cases for causal mechanism analysis. The practical problem of choosing cases runs as a bright red thread throughout the book. These decisions face all who connect case studies to other methodologies. In particular, I offer much specific guidance about case selection. This means a systematic set of guidelines for case selection including a list of criteria for getting to a final decision about which cases to choose.

McGuire (2010) illustrates typical multimethod research involving a statistical analysis and case studies. His dependent variable is health outcomes. The second chapter of his book is a large-N cross-national statistical analysis, which is followed by eight country case studies. These case studies focus on explaining outcomes in those countries, for example, the following:

What needs to be explained, then, is why Costa Rica’s infant mortality rate was so low in 2005 (why it attained a certain level); why it fell so fast from 1960 to 2005 (why it achieved a certain amount of progress), and why it fell faster during the 1970s than at other times within this period (why it evolved at a certain tempo). The sustained and effective public provision of basic health services to the poor, this chapter finds, goes a long way toward explaining why Costa Rica from 1960 to 2005 achieved a rapid decline (and, eventually, a low level) of infant mortality. (McGuire 2010, 66)

McGuire is making claims about what happened in Costa Rica and why it happened, i.e., within-case causal inference.

The standard rationale for multimethod work involves looking at causal mechanisms via case studies: “Despite some claims to the contrary in the qualitative methods literature, case studies are not designed to discover or confirm empirical regularities. However they can be quite useful—indeed, essential—for ascertaining and assessing the causal mechanisms that give rise to empirical regularities in politics” (Fearon and Laitin 2008, 773). This is exactly what the research triad proposes: cross-case analyses for “empirical regularities” and case studies for causal mechanisms.

Figure 1.2: Causal mechanisms and statistical multimethod research: democratic IGOs and democratic stability. Source: based on Pevehouse (2005), table 5.1.

McAdam and Boudet give a similar rationale in their study of environmental social movements: “We conceived of the project as an attempt to develop an alternative to the methodological conventions of social movement research. Equally dissatisfied with ‘thin,’ large-N studies of protest events and rich but nongeneralizable case studies of this or that movement, we sought a middle ground between these two modal ‘poles’ of social movement scholarship” (2012, 52).

Pevehouse (2005) provides a nice example of why people want to do multimethod research. He argues that democratic IGOs (intergovermental organizations) can help establish democracy, make it more robust, and encourage transitions to democracy. As illustrated in figure 1.2a, there is a causal connection proposed between democratic IGOs and democracy in states. He shows that there is a significant correlation between democraticness of the IGO and democracy in its member states. Multimethod research comes into play because he thinks there are multiple—and not mutually exclusive—causal mechanisms that explain this significant correlation. Figure 1.2b adds the causal mechanisms that produce this statistical effect: (1) acquiescence effect, (2) legitimization, (3) pressure, and (4) financial assistance (Pevehouse 2005, table 5.2, 153). Here one sees Pevehouse going around the research triad, from the statistical analyses to case studies to causal mechanism analysis.

Many hypotheses, experiments, and the like propose multiple causal mechanisms connecting the treatment to the outcome, as illustrated in figure 1.2. For example, Helfer and Voeten list three causal mechanisms whereby the European Court of Human Rights influences state policy: (1) preempting future international court litigation, (2) persuasive authority, and (3) agenda-setting at the national level. They argue—like Pevehouse—that “these three mechanisms may work separately or in tandem” (Helfer and Voeten 2014, 82). Hence, one role of case studies is to explore which of these mechanisms is actually at work.

A central role of case studies is combining within-case causal inference with ana...