![]()
Introduction
Growth and advancement in policy process studies depend not merely on developing better theories but also on developing better methods. In this volume, we use “methods” as an umbrella term to denote systematic techniques for applying the various approaches, frameworks, and theories of the policy process and systematically measuring their key concepts.1 Methods are ways researchers put theories into action, test them, and improve them. Methods involve strategies for actual application, including the research design (e.g., sampling approaches), conceptualization and measurement, data collection (e.g., experiments, field research, surveys), modeling (e.g., models of government innovation), and data analyses (e.g., quantitative or qualitative approaches). Methods also include precise ways in which researchers connect variables in explaining a phenomenon in context.
A focus on methods in the study of policy processes is notably absent. While some strands of research have best practices for applying a theory, these methods are rarely understood or communicated outside of a given research program. In other instances, some theories have existed for decades but have yet to develop standard and relatable methods for application. Whether the methods exist but are not communicated or yet developed, the result is the same – limited growth and advancement in the study of policy processes.
This volume supplements the well-established and widely used Theories of the Policy Process (Weible and Sabatier 2018). First published in 1999, Theories of the Policy Process has served as the primary compilation of the most established theoretical approaches in studying policy processes. It emerged from the need to develop better theories and to communicate these theories in a single volume for new and experienced researchers (Sabatier 1991).
Today, however, the intellectual landscape is very different. The increasingly global study of policy processes faces severe challenges with scholars applying theories in radically different national and cultural contexts (Tosun and Workman 2018). Questions increasingly arise about conducting policy process research comparatively among a global community of scholars. For example, scholars often interpret concepts differently and use different measures in applying the same theory. We should undoubtedly encourage diversity, experimentation, and creativity to advance the field. However, without some standardization, best practices, common templates, and general strategies for applications, the generation of our shared knowledge will be stunted, and many of the lessons learned will be overlooked or lost. We do not all have to do things the same way, but what we do and the logic underlying our approaches should be clear. Indeed, in advancing the study of policy processes, there has always been a necessary tension between recognizing contextual particularities and drawing appropriate generalizations.
This volume remedies this situation by bringing attention to methods of the policy process. Our intent is not to establish orthodoxy or impose a rigid way of researching on the policy process community. Instead, we offer the leading researchers of policy process theories the opportunity to share their best and established practices for theoretical applications. The goal is to communicate the diverse ways that we conduct theoretically based research toward better methods of the policy process.
Why This Book and Why Now?
For academia, the contributions to the field of policy processes will be nontrivial. Never has an edited volume sought to tackle issues of methods for theories of the policy process, and the need for such an emphasis has never been greater. In this respect, this volume has no peers. Many of these methods develop along parallel tracks within each of the theoretical traditions. This volume will allow academics and students, for the first time, the ability to see outside these tracks and be aware of the commonalities across the different approaches, as well as identify the key differences.
This volume serves as a reference for applying the theories for students at the undergraduate and graduate levels and experienced scholars. This compilation of theories captures the most established approaches with international research communities, ongoing empirical applications, efforts to push comparative insights, emphasis on being transparent and public, and continual growth in contributing knowledge about policy processes (Weible 2018).2 However, there are also many quality books and essential ideas in public policy that we teach in our courses and reference in our research that do not meet these criteria or fit in this volume, one of which is Peter John’s (2018) book How Far to Nudge? Public policy is associated with the fields of public administration, regulation, broader political science, and others that supplement theory-based policy process research and form valuable comparisons to approaches in public policy. Many of these related fields might also inform how to advance theory-based policy process research methods and methodologies. Where this is the case, we will discuss these areas of opportunity.
In addition to illuminating methods to spur innovation in public policy scholarship, broader trends in interdisciplinary science place a premium on making social science research methods accessible to students, researchers, and those in practice. Social and natural systems have become intertwined as human beings increasingly influence the systems that govern the natural world (Stromberg 2013; B. A. Jones et al. 2016). Climate change, food systems, and environmental justice are just a few of the myriad issues demanding interdisciplinary science for workable solutions. These problems beg for interdisciplinary science that not only sees social science as translational but as intrinsic to understanding and solving societal problems. Make no mistake about it; these problems are policy problems lying at the intersection of the natural world and governance systems.
The barrier to interdisciplinary science is language, or rather, jargon. Yet, in our experience, methods and methodologies offer a common language for interdisciplinary science. By shedding light on the methods of policy processes, this volume speaks to the range of possibilities and opportunities for collaborative research between natural scientists and policy scholars. These benefits push far beyond translating science and research to impactful social science on the front-end design of problem-oriented research and toward the betterment of societies.
The Approaches and Data Science
The approaches to research design and analysis presented in this volume represent unique data sciences and traditions. Thus, it is helpful to consider the emergent field of data science when thinking through the chapters to follow. As a subset to methods, we define data science as the set of processes for collecting, cleaning, organizing, storing, and analyzing data.
For the experienced scholar, the chapters lay out myriad possibilities for the types of data one might collect, innovative measures, and the many analytical techniques for testing and refining theories (e.g., statistical modeling or qualitative case studies). For the beginning scholar, these are second-order concerns. Often, the most significant barrier to applying theories is what data to collect and how to organize them for analysis. Thus, to the extent possible, the chapters attempt to give the inexperienced scholar a glimpse of what type of data is typically used, how it is collected, and how it is organized for analysis. For example, how should data be organized in a spreadsheet to be useful to the analyst (Broman and Woo 2018)? These questions seem basic, but in fields stretching from bio-medical data to business to public administration, data are often not collected and organized in a way that allows for analysis across platforms, theoretical traditions, or disciplines (see Workman 2020).
The chapters to follow offer an overview of how data are collected, organized, and analyzed in each of the theoretical traditions we cover. These are not compendiums on data science within the theoretical traditions, but the reader will leave with a good idea for getting started. Moreover, the theories covered in the chapters encapsulate not only standard operating procedures for data collection, organization, and analysis but also the norms and traditions within each theoretical approach. Thus, data science is a set of objective guidelines and perspectives on what works best, given data and context within theoretical traditions.
How Does This Book Relate to the Theories Book?
Distinctions between deductive and inductive research are common in courses on research design for budding social scientists and data analysts of all types. In general, deductive research proceeds with theory as the main driver and develops conjectures about the world for testing with empirical research designed for confirmation or falsification. On the other hand, inductive research begins with data and observation, developing descriptions and explanations of the world from the ground up.
While the distinction between deductive and inductive research is a useful pedagogical tool for understanding the relationship between data and theory building and testing, science is an iterative exercise in exploration, discovery, and explanation. The inductive scientist develops a theory from careful observation of the world and then uses that understanding to gather more data or undertake more observations to refine that theory. Likewise, the deductive scientist undertakes observation, data collection, and analysis that supports their working theory of the phenomena they study in whole or in part. The theory is then discarded or refined in light of the data or observations. All science is an iterative process of deduction and induction, differing only from how one starts. Discussions of research approaches often abbreviate the iterative process of data and theory, but the arc is much longer, involving observation, theory building, theory testing, evaluation of the evidence, and finally, feedback and theoretical refinement.
In this light, there is a clear relationship between this volume and Theories of the Policy Process. This volume closes the iterative loop in policy process research. The methods covered are the engines for the refinement of the theories in that volume. Whereas the Theories volume provides an overview of the state of the theoretical approaches to public policy and traces the development of the core ideas, this volume relates to conducting research within the theoretical approaches.
Elucidating the different methods and methodological approaches in policy process research also holds the promise of cross-fertilization. Some of the concepts across theoretical traditions are common or similar. Take the example of the concept of feedback – how does policy itself fuel resultant politics and decision-making in policy systems and relate to other concepts like learning or attention? Awareness of how different theories conceptualize and operationalize common concepts offers opportunities for innovations in and across multiple research programs and for bridging the silos that result from their decades of advancement.
Methods in the Context of Theory
This volume aims to provide a resource for those looking to research and contribute to these theoretical traditions. To that end, the authors have endeavored to be systematic and transparent without imposing any rigid set of rules for the beginning analyst. Some of the authors dig deeply into measurement and particular statistical techniques, while others offer a broad view of the techniques generally employed in the area, giving the reader a range of opportunities for engagement.
The theories embrace various methods for analyzing data and refining conjectures about the world. That last bit is important. The nature of scientific publishing means that we spend lots of time specifying, explaining, and justifying a method for a given piece of work. This is necessary, of course, to give the reader confidence in the findings. As a result, though, we tend to spend less time in any given project thinking about how that process should inform and refine our theory (usually leaving that to the literature review and theory in the next project). This volume offers that opportunity for those steeped in techniques and modes of analysis to evaluate those approaches, how they might be adjusted, and what needs reform. We return to the theme of building or creating knowledge from our methods and theories in the concluding chapter.
For the reader of this volume, we think two distinctions in the ways researchers use the approaches below are useful for understanding theory building, testing, and feedback in the traditions. The first distinction is whether the mode of analysis is qualitative or quantitative. In this volume, the modes of analysis span from advanced statistical analysis to qualitative case studies and comparison, process tracing, and more. There is no substantive advantage to quantitative or qualitative research – both can be systematic and transparent enough to follow their logic.
In the background, the distinction between deterministic and probabilistic models of the world lurks. Deterministic models explain the world and can predict the world accurately and precisely with minimal variance, given certain conditions. However, strictly speaking, nearly all social science is probabilistic – it explains the range of likely behaviors or outcomes under certain conditions. Certain behaviors and outcomes are more likely within this range, and our models attempt to attach probabilities to these, even when working qualitatively. The reasons for the probabilistic nature of social science are obvious – there is randomness in human behavior, and, thus, in social, political, and economic systems that distinguish them from natural systems. Additionally, policy and governance systems are complex and generally not linear and additive (e.g., more akin to the human body than a car). Finally, there is no “right” methodological argument or method, merely what is theoretically and logistically useful for building knowledge in the field.
We offer a roadmap for understanding and reasoning about the methods and methodologies to follow. The roadmap is a set of considerations addressing the research questions we ask, the research design, how the approaches treat data and measure concepts, and the tools for analyzing data. These are points of depa...
