The term âevidence-based policymakingâ (EBPM) is in common currency in media and social media. Generally, it is a vague, aspirational term, rather than a good description of the policy process. For some, it represents an ideal which governments fail to reach. A common allegation is that policymakers ignore or do not understand or act on the correct evidence. In other words, the identification of a problem by scientists should produce a proportionate solution, and policymakers should select the most effective, evidence-based solutionâbut they donât.
Policy studies provide more critical discussions of the EBPM concept, often suggesting that people are naĂŻve if they think that this kind of EBPM is a possibility and that they should not waste their time in wanting something that they canât have (Bastow et al. 2014a, b; Monaghan 2011: 6). In earlier post-war debates, Lindblomâs (1964: 157) famous analogy was between unaided versus mechanical human flight: any attention to the flight of fancy is time not spent on the more realistic aim.
Policy studies help us challenge two extreme views: first, the naĂŻve-EBPM view that there can and should be a direct and unproblematic link between scientific evidence, policy decisions, and outcomes; second, the âpolicy-based evidenceâ view that politics is so pathological that no decision is based on an appeal to scientific evidence if it gets in the way of politicians seeking election, or so messy that the evidence gets lost somewhere in the political process. Indeed, if we initially hold the naĂŻve-EBPM view, when we engage in politics, we may quickly become disillusioned and develop a more cynical view. Neither view is helpful to people trying to understand the role of evidence in the policy process and, in some cases, influence that process. Rather, we need two things: to define EBPM, and understand the policy process in which evidence is used.
The Meaning of EBPM: Letâs Go Beyond Unclear Definitions and Ill-Considered Aspirations
Our first requirement looks relatively simple: more clarity about what EBPM means, and should mean, in practice. Many scientists use EBPM to refer to something that should happen: there can and should be a direct and unproblematic link between âthe evidenceâ and policy decisions and outcomes. Further, the term âevidence-based policymakingâ or EBPM has a âbewitchingâ effect and seems like a valence issue: who could not want it? Yet, we cannot want something if we do not really know what it means, or what each word means.
Policy
There is no single accepted definition of policy. I use the working definition âthe sum total of government action, from signals of intent to the final outcomesâ to raise important qualifications: (a) it is problematic to conflate what people say they will do and what they actually do; (b) a policy outcome can be very different from the intention; (c) policy is made routinely through cooperation between elected and unelected policymakers and actors with no formal role in the process; and (d) policymaking is also about the power not to do something (Cairney 2012a: 24â5). It is also important to identify the many components or policy instruments that make up policies, including the level of spending, the use of economic incentives/penalties, regulations and laws, the use of voluntary agreements and codes of conduct, the provision of public services, education campaigns, funding for scientific studies or advocacy, organisational change, and the levels of resources/methods dedicated to policy implementation (2012a: 26). In that context, we are trying to capture a process in which actors make and deliver âpolicyâ continuously, not identify a set-piece event that provides a singular opportunity to use a piece of scientific evidence to prompt a policymaker response.
Policymakers
The intuitive definition is âpeople who make policyâ, but there are two important distinctions: (1) between elected and unelected participants, since people such as civil servants also make important decisions (see also BĂ©dard and Ouimet 2012: 628 on policy analysts) and (2) between people and organisations (both can be described as âactorsâ), with the latter referring to a group of people who make decisions collectively (âinstitutionâ describes the rules to which such actors make reference). These distinctions are crucial to remind us that advocates would miss something important if they focused their energies only on elected politicians. There are blurry dividing lines between the people who make and influence policy. Terms such as âpolicy communityâ suggest that policy decisions are made, in some sense, by a collection of people with formal responsibility and informal influence. Consequently, we need to make clear what we mean by âpolicymakersâ when we identify how they use evidence.
Evidence and Scientific Evidence
We can define evidence as an argument or assertion backed by information. Scientific evidence, therefore, describes information produced in a particular way. Some use the term âscientificâ broadly, to refer to information gathered systematically using recognised methods, while others refer to a specific hierarchy of scientific methods, with randomised control trials (RCTs) and meta-analysis/the systematic review of RCTs (published in high-status peer-reviewed journals) at the top (see Nutley et al. 2013). The latter definition is at the heart of EBPM debates in health and environmental policy, primarily because many people are unaware of, or unattached to, this hierarchy. Policymakers may not go as far as describing âevidence-based claimsâ, or a distinction between positivist and constructivist understandings of objective facts (by detached and value-free scientists) and subjective understandings (Marston and Watts 2003: 150â7). However, they will seek many kinds of information that scientists would not consider to be part of âthe evidenceâ.
Overall, these definitions donât take us very far. âEvidenceâ is assertion backed by information. âBasedâ is a metaphor. âPolicyâ is one of the worst defined words in politics. Policymaking implies there is a policymaker, but we donât always know who it is. This seems like a semantic discussion, but there is a lot of confusion in the EBPM literature because people begin by complaining that they donât have it without really saying what it is. A focus on phrases like âevidence informedâ doesnât always help because it is no less difficult to point to a policy and show how it relates to âthe evidenceâ.
Part of this confusion, among advocates of EBPM, relates to a lack of knowledge of key aspects of the policy process. For example, many debates confuse two different kinds of evidence-based activity relating to the size of the problem (e.g. the number of smokers and the link between smoking and ill health) and the effectiveness of the solution (e.g. the effect of higher taxes and health warnings on consumption). In the former, evidence may be used to prompt attention, and exhort an appropriate response, to a problem. In the latter, evidence may be used to generate knowledge on the effectiveness of solutions. Yet, in ill-disciplined debates, it is common to bemoan an insufficient link between the evidence on the size of a problem and the choice of solution, or to assert that the scientists who identify the problem are best placed to propose a solution.
More generally, scientists may be describing the extent to which they feel that policymakers listen to what they have to say, and act on that basis. In some cases, people are quick to say that a policy is ânot evidence basedâ if policymakers only listen to some of what they have to say and/or only adopt some of their recommendations. Yet, this is possibly the most they could reasonably hope for in a political system. Instead of suggesting that politicians mangle the evidence, we should recognise that elected policymakers are legitimate actors.
The idea that policymakers should base their decisions primarily on scientific evidence may seem intuitive, but it is problematic if we simply argue that the evidence should come first and bemoan the inability of policymakers to act accordingly. There are many other, equally defendable, roles for research, such as to inform solutions to a problem identified by elected policymakers, or to contribute to long-term public âenlightenmentâ (Weiss 1979). Scientists can help identify problems, and assess the effectiveness of solutions without feeling that they should be at the centre of a democratic policymaking system.
Letâs Understand the Policy Process to Recognise the Role of Evidence Within It
Our second requirement is less simple: we need to understand the policy process to explain how actors use evidence within it. That is the aim of this book. I take the naĂŻve-EBPM view as a starting point to argue that it represents an ideal type. An ideal type can be something to aspire to, but its main purpose is to help us compare an artificial situation with the real world and better describe and explain reality. This starting point is central to policy studies. We describe the ideal type, comprehensive rationality, in which policymakers are able to generate a clear sense of their preferences, gather and understand all relevant information, and make choices based on that information. Then, we describe bounded rationality, or what really happens when policymakers have unclear aims, limited information, and unclear choices. Our focus is on the ways in which policymakers understand information such as scientific evidence, taking into account the shortcuts they use when they have limited means to process it. This takes place in a policy environment which contains many other policymakers and pressures, and which influences how they act and how much control they have over the final outcomes.
The Value of Policy Theory
The policy literature contains theories and studies which can be adapted to explain how evidence processing fits into the wider policy process. These studies have two key points in common. First, we need to understand the psychology of policymaking. Policymakers have to make important decisions in the face of uncertainty, which is based on limited information, ambiguity, which is based on the fact that there are many ways to understand a policy problem (this kind of uncertainty cannot be solved by more information), and competition between actors to interpret information and draw conclusions (Zahariadis 2007: 66). They do so by drawing on policymaking shortcuts, such as by using information from sources they trust and by adapting that information to the beliefs they already hold. In other words, since policymakers do not have the ability to gather and analyse all information, we need to identify the heuristics they use to gather what they can. This may reveal their biases towards certain sources o...