Working Together

Collective Action, the Commons, and Multiple Methods in Practice

Amy Poteete, Marco A. Janssen, Elinor Ostrom

• Publisher: Princeton University Press
• Year: 2010
• ISBN:
  9781400835157
• Topic: Social Sciences
• Subtopic: Social Science Research & Methodology
• Index: Social Sciences

Advances in the social sciences have emerged through a variety of research methods: field-based research, laboratory and field experiments, and agent-based models. However, which research method or approach is best suited to a particular inquiry is frequently debated and discussed. Working Together examines how different methods have promoted various theoretical developments related to collective action and the commons, and demonstrates the importance of cross-fertilization involving multimethod research across traditional boundaries. The authors look at why cross-fertilization is difficult to achieve, and they show ways to overcome these challenges through collaboration.
The authors provide numerous examples of collaborative, multimethod research related to collective action and the commons. They examine the pros and cons of case studies, meta-analyses, large-N field research, experiments and modeling, and empirically grounded agent-based models, and they consider how these methods contribute to research on collective action for the management of natural resources. Using their findings, the authors outline a revised theory of collective action that includes three elements: individual decision making, microsituational conditions, and features of the broader social-ecological context.
Acknowledging the academic incentives that influence and constrain how research is conducted, Working Together reworks the theory of collective action and offers practical solutions for researchers and students across a spectrum of disciplines.

PART ONE

Introduction

CHAPTER ONE

Overcoming Methodological Challenges

QUESTIONS ABOUT the relative merits of alternative research strategies pervade the social sciences. What counts as an adequate explanation for social phenomena? How can we evaluate competing explanations? What standards should we apply when weighing evidence? How much and what types of evidence are convincing? Can social phenomena related to policy areas be studied scientifically? Some eminent scholars appear to agree on broad methodological goals or criteria (Brady and Collier 2004; Gerring 2001; Lieberman 2005). Explanations should be general yet precise, accurate, and well-specified. Evidence should be theoretically relevant and should identify mechanisms linking explanations to outcomes. Abundant evidence, if theoretically relevant, is valued because it enhances confidence in findings.

Despite the apparent common ground underlying the work of many scholars, methodological divides within the social sciences also run deep. As lamented by Mahoney and Goertz (2006) and E. Ostrom (2006), rival camps often cast aspersions on each other’s work rather than engage in constructive dialogue. The acrimony has several sources. The disagreements have been provoked in part by battles over induction versus deduction, poor methodological practice by some scholars, and a lack of sensitivity to diverse research goals. The stakes of the methodological debate are increased by the intertwining of methodological choice with ontological, normative, and theoretical positions, and with competition for professional status and resources (Moses and Knutsen 2007). These dynamics encourage intense and sometimes grossly unfair critiques.

The substantive focus of this book is on collective action and the commons. It is a field of research that utilizes multiple methods extensively, as well as being the one most familiar to the authors of this book. We believe that the discussion of the use of multiple methods in this research field, and the lessons we draw from our practical experiences, apply more broadly to social science in general. Therefore, we start this first chapter with a broader discussion on the methodological challenges in the social sciences.

Examples of poor methodological practice pervade social science research. Often, scholars follow “the rule of the hammer” and apply a single method indiscriminately, regardless of its suitability for a given research project. Harmonization of research goals, theory, data, and method does not, however, guarantee sound practice. One can find qualitative studies that overstate either the uniqueness or the generality of particular cases, fail to utilize relevant concepts and theories in the literature, or work with concepts that conflate multiple dimensions (Sartori 1991; compare Goldthorpe 1997). Quantitative studies sometimes use inadequate data and do not always use appropriate diagnostic checks and technical fixes (Jackman 1985; Scruggs 2007; Shalev 2007). Formal models often work with unrealistic assumptions without addressing the gap between assumptions and reality (Bendor 1988; Green and Shapiro 1994). No method is immune to poor applications.

Critics sometimes conflate methodological practice with the method itself, arguing that examples of poor application discredit the method. A method need not be abandoned because it has been poorly utilized; it makes more sense to encourage greater methodological awareness and better practices (Geddes 2003; Jackman 1985; King, Keohane, and Verba 1994; Scruggs 2007). Others fail to appreciate that research goals are varied and require diverse methods. More than three decades ago Robert Clark (1977, 10; emphasis in original) strongly warned against reliance on a single method:

A first rule should be to beware of one researcher, one method, or one instrument. The point is not to prove that the hypothesis is correct, but to find out something. To rely on a single approach is to be shackled.

Indiscriminate application of a method makes little sense, but complete rejection of a method because it is inappropriate in a particular setting or for a particular purpose is not more sensible. It is important for social scientists to recognize that all methods generate results that contain some level of uncertainty. While multiple scientific goals and trade-offs in achieving those goals are widely acknowledged (Coppedge 1999; Gerring 2001), little consensus exists on the relative importance of particular goals. Some scholars prioritize one or a few goals to such an extent that they dismiss as unscientific research that prioritizes other goals. For example, Goldthorpe (1997) includes generality as the most important criterion in his definition of causal explanation, rather than as one of several criteria (compare Gerring 2001). Consequently, he sees unique events and contingency as marking the limits of scientific inquiry. By this definition, analyses of such events are not scientific and cannot support causal inferences. Proponents of path-dependent explanations, analytic narratives, interpretive methods, and other approaches strongly disagree (Bates et al. 1998; Bennett and Elman 2006; Rogowski 2004; R. Smith 2004). As in this example, and as discussed further below, methodological controversies often reflect competition between research traditions.

Fortunately, social scientists increasingly recognize trade-offs across methods (Bates 2007; Brady and Collier 2004; Gerring 2001).¹ King, Keohane, and Verba (1994), for example, point out that all methodologies have limitations; scholars should be more aware of these limits and more transparent about the limits as well as the solid contributions of their work. To overcome the limits of any one method, one needs to draw on multiple methods (Bates et al. 1998; Coppedge 1999; Granato and Scioli 2004; Jackman 1985; King, Keohane, and Verba 1994; Laitin 2003; Lieberman 2005; Scharpf 2000; Tarrow 2004). If social scientists have shared standards, no single method fully addresses all standards. Methods offer different strengths and weaknesses. Rigorous research that combines complementary methods will be superior to research that relies on any single method (Gray et al. 2007).

The pragmatism and respect for diverse methodological traditions in these reflections are welcome. Too often, however, the challenges involved in using multiple methods are themselves overlooked. Proponents of mixed methods justify their preferred combination in logical terms and illustrate the approach with a few examples. With some exceptions (Lieberman 2005; Scharpf 2000), this literature offers few specific practical suggestions.

Practical challenges can be formidable. Not all methods are equally feasible or even appropriate for all research topics (Bennett and Elman 2006; Poteete and Ostrom 2008). Lieberman’s (2005) nested analysis, for example, involves large-N analysis prior to any case study work. There are many important topics for which broadly comparative data are scarce, difficult to access, or of dubious quality. Lieberman, however, does not address these challenges. Even if data availability is not a problem, the value of a multimethod approach requires sufficient command of multiple methods. Yet considerable investment is required to gain competency in any methodology, and the benefits of methodological specialization are substantial. While these challenges are sometimes acknowledged, few social scientists make practical suggestions to address them.

This book focuses on the practical challenges that influence methodological choice. We are particularly concerned with research on topics for which data are scarce, difficult to collect, and not readily comparable. These conditions affect research on a wide variety of topics, including those concerned with informal institutions, subnational organizations, and nonelite populations. We focus on collective action for the management of natural resources, an area of research in which all of these conditions apply. For such topics, data for large-N analysis are neither available nor readily accessible, and field research is unavoidable. Researchers often need considerable contextual knowledge even to recognize the phenomenon of interest. The need to conduct intensive fieldwork limits the potential for collecting enough data to support broadly comparative analysis.

We have become strongly aware of these challenges through our own work on collective action and natural resource management. We feel that the practical challenges of conducting rigorous social science research on topics for which data are scarce, or difficult to access or to interpret, have not received adequate attention in discussions about social science research. We have seen the benefits of collaboration and the combination of multiple methods in our research. We also have firsthand experience of the challenges involved in such research, and we will discuss these throughout this book.

In this chapter, we introduce four themes that recur through the book: (1) the interlinking of methodological debates with theoretical development, (2) the advantages and limitations of multiple methods and collaborative research, (3) practical constraints on methodological choices, and (4) the often problematic influence of career incentives on methodological practice. In this book, we explicitly acknowledge the practical challenges that affect methodological choices, evaluate several strategies for addressing these challenges, and direct attention to the influence of career incentives on methodological choices in social science research. We discuss a range of options for balancing competing methodological demands under the inevitable conditions of limited resources, including a variety of techniques that we feel have been underutilized in the social sciences. We discuss the merits and limits of each method, as well as the possibilities for and constraints on combining various methods. In our discussion of constraints on methodological choice, we hope to stimulate a debate about professional incentives and other structural aspects of academia that influence how research is conducted.

This book is more about methodological practice than about methodological ideals. We thus begin this chapter with a historical overview of methodological debates, highlighting interactions among methodological practices, changing theoretical orientations, and competition for professional status and resources. We then look more closely at issues surrounding research that uses multiple methods, an approach that has gained in acceptance in recent years. This leads to a discussion of constraints on methodological choice, both practical and professional. We then explain how our substantive focus—the study of collective action in natural resource management—helps us address our four thematic concerns. The chapter concludes with an outline of the rest of the book.

SOCIAL SCIENCE DEBATES OVER THE SUPERIORITY OF PARTICULAR METHODS

The history of the social sciences can be recounted with reference to major methodological shifts. An initial reliance on qualitative analysis gave way dramatically to quantification in the early to mid-twentieth century. When this transformation began, quantification largely meant statistical analysis of large-N data sets of public opinion surveys. The last third of the twentieth century saw a surge in the use of formal models as well. Debates about the relative merits of qualitative, statistical, and formal methods contributed to several developments in the late twentieth and early twenty-first centuries: refinements of quantitative methods that attempt to better match social conditions; the rise of formal models; greater appreciation for combining multiple methods; and the spread of post-positivist methods such as discourse analysis.

The qualitative orientation of the early social sciences can be seen in the emphasis on case studies and participant observation in sociology, ethnographic field-based research in anthropology, and descriptive and normative analyses of formal legal arrangements. In the early decades of the twentieth century, many scholars embraced quantitative methods as part of a drive to make the social sciences more scientific.² Quantitative methods began to gain currency across the social sciences in the 1920s and 1930s. The adoption of these methods accelerated at midcentury, as conveyed by references to the behavioral revolution.

The branches of the social sciences differed in their timing, pace, and preferred forms of quantification. Nonetheless, the methodological shift from qualitative to quantitative methods in the social sciences was dramatic. Psychology rapidly adopted experimental and statistical methods. Quantitative methods in economics encompassed formal models as well as experiments and statistics. For sociology, research activities during World War II marked the ascendance of survey research, experiments, and statistical forms of analysis (Platt 1986). Postwar political science shared the enthusiasm for survey research and statistical analysis, but formal modeling became widespread only in the 1980s and 1990s. In sociocultural anthropology, some interest was expressed in mathematical models in the early postwar period, but multivariate statistical analyses remained relatively rare until the 1970s (Chibnik 1985).

The role of quantitative methods in the social sciences has always been contentious. Current methodological debates echo those of a century ago, even if framed in somewhat different terms.³ Scholars concerned with methods have disagreed over (1) the goals of social research, (2) philosophical and theoretical issues, and (3) practical considerations, especially related to data quality. Methodological choices should be driven by theoretical and ontological assumptions (Hall 2003), but they also reflect underlying values and beliefs (Mahoney and Goertz 2006) and practical considerations (Platt 1986). The ontological and normative dimensions of methodological choices are not widely recognized (Mahoney and Goertz 2006). As a result, social science debates about methods involve frequent misunderstandings, with proponents of different approaches talking past each other (E. Ostrom 2006). Furthermore, because methodological discussions rarely acknowledge practical and professional considerations, they offer little guidance on how to address these constraints. In this section, we discuss controversies over the goals of social research, and how philosophical and theoretical issues interact with professional competition. We expand our treatment of practical and professional considerations in subsequent sections.

During the 1920s and 1930s, the social sciences became more institutionalized in North America. The social sciences sought recognition as sciences, and each discipline developed a more or less distinct professional identity (Guy 2003; Platt 1986). This process of institutionalization influenced methodological debates. During the prewar period, disagreements focused on the goals of social research. Should sociological research support social work to improve social conditions, seek subjective understanding of life experiences, or attempt to identify general patterns (Platt 1986)? Should the study of politics provide normative and practical guidance for administrators or objective understanding of political phenomena (Guy 2003; Lasswell 1951)? As universities set up schools of social work, public administration, and business administration alongside departments of sociology, political science, and economics, differences over goals were alleviated—but not really addressed—through the institutionalization of more focused programs of study.

Yet differences over the relative importance of theory and praxis cannot fully account for methodological debates. Scholars with common goals disagree over methods, and scholars draw on the same methods to pursue divergent goals. A lack of consensus on fundamental philosophical issues contributes to disagreements over methods. What counts as science? What model or models of causality and explanation make sense for social phenomena? In particular, do models of science and explanation developed in the natural, and especially the physical, sciences make sense for the social sciences?

Over the past century, some have embraced deductive models of science inspired by the natural sciences as a way to gain more reliable insights about social processes (King, Keohane, and Verba 1994; Przeworski and Teune 1970). Deduction involves the logical derivation of universalistic, lawlike statements of the sets of conditions associated with the outcome of interest from theoretical assumptions. Lawlike statements may be derived from formal or mathematical models, as in rational-choice...