Politics and Policy Knowledge in Federal Education

Confronting the Evidence-Based Proverb

Steven Putansu

• Publisher: Palgrave Macmillan
• Year: 2020
• Print ISBN:
  9783030383947
• eBook ISBN:
  9783030383954
• Topic: Politics & International Relations
• Subtopic: Educational Policy
• Index: Politics & International Relations

Policy knowledge derived from data, information, and evidence is a powerful tool for contributing to policy discussions and debates, and for understanding and improving the effectiveness, efficiency, and equity of government action. For decades, politicians, advocates, reformers, and researchers have simultaneously espoused this value, while also paradoxically lamenting the lack of impact of policy knowledge on decision making, and the failure of related reforms. This text explores this paradox, identifying the reliance on a proverb of using policy knowledge to supplant politics as a primary culprit for these perceived failures. The evidence in this book suggests that any consideration of the role of policy knowledge in decision making must be considered alongside, rather than in place of, considerations of the ideologies, interests, and institutional factors that shape political decisions. This contextually rich approach offers practical insights to understand the role of policy knowledge, and to better leverage it to support good governance decisions.

Part IConfronting the Evidence-Based Proverb

© The Author(s) 2020

S. PutansuPolitics and Policy Knowledge in Federal Educationhttps://doi.org/10.1007/978-3-030-38395-4_1

Begin Abstract

1. Policy Knowledge, Politics, and a Proverb of Decision Making

Steven Putansu¹

(1)

US Government Accountability Office, Washington, DC, USA

Steven Putansu

End Abstract

Without good data, information, and evidence, government decision makers cannot adequately identify problems, design solutions, or fully understand the impact of government action. This idea can be traced to James Madison’s (1787) argument for the virtue of a large republic over direct democracy in Federalist number 10. He argued in favor of “representatives whose enlightened views and virtuous sentiments render them superior to local prejudices and schemes of injustice.” In Madison’s view, general “enlightenment” was one factor that would help elected officials to make good decisions, and his argument implied that this enlightenment would protect decision makers from abandoning their virtuous sentiments and falling prey to baser, political differences. Madison believed that enlightenment was a necessary, but not sufficient, input into the political process (Bertelli and Lynn 2006). The “opposing forces,” as he described enlightenment and political sentiments, are balanced by elected decision makers in a republic. This book will examine the role that policy knowledge that constitutes enlightenment plays, alongside politics, to influence government action.

Politicians, public servants, and governance scholars have attempted to emphasize the value of enlightenment for nearly two and a half centuries. In 1946, Herbert Simon critiqued the study of administration as relying on “Proverbs” that were driven by inconsistent theory and insufficient evidence. He urged scholars and practitioners to place greater emphasis on better defining conceptual terms, focusing on decision making, and building models of theory and practice that could achieve stated results. Consistent with Simon’s call to action, this emphasis has been driven through reform, management practice, and research about the use of policy knowledge—data, information, and evidence about governance (Lindblom and Cohen 1979). These efforts have been associated with a variety of reforms, including performance budgeting (Robinson 2007; West 2011), portfolio budgeting (Meyers 2017), performance management (Hatry 2006), and a range of other calls for evidence-based policy (Haskins and Baron 2011). However, evidence-based reforms have also suffered from a lack of conceptual clarity, insufficient understanding of decision making processes, and limited success in generating models of evidence use that are tailored to specific purposes.

The first of these issues is a lack of conceptual clarity: The terms data, information, and evidence have been used in inconsistent, ambiguous, and sometimes inappropriate ways. To avoid confusion, the following will define how the terms will be used in this text. The phrase “policy knowledge” will be used as a broad umbrella term to cover all forms of data, information, and evidence used to inform the crafting and implementation of public policies. The foundation for this study’s definition of policy knowledge derives from Giandomenico Majone’s book, Evidence, Argument, and Persuasion in the Policy Process, which defines information in comparison with data and evidence. In this model, data represent an observation, information represents an interpretation of that observation, and evidence represents the selection and combination of information to make an argument (Ackoff 1989). As an example, these definitions could be applied to studies of the environment. Specifically, the average daily global temperature represents data, the upward trend evident in these data is one kind of information, and the associations between this trend and human factors represent evidence.

This study also shows that evidence-based reforms have paid insufficient attention to decision making. Reformers have promised to generate policy knowledge to guide decision making, and these promises wrongly treated enlightenment as a replacement for, rather than a balance against, political forces. Through this flawed assumption, reformers have overlooked important elements of decision making, which has resulted in inconsistent and limited success. Finally, by highlighting the relationships among data, information, and evidence, this text illustrates a critical relationship between the policy knowledge created by reforms and the specific goals it can support. Data provide the basis for the development of information, and information offers insights into potential evidentiary analyses. As such, all three elements must be considered in order to understand the potential value of specific sources and types of policy knowledge. This book will use the phrase “evidence-based governance” to refer to the production and use of policy knowledge to support, assess, and understand government, and the phrase “evidence-based reform” to refer to any effort that advocates for greater use of data, information, and evidence. These kinds of reforms are consistent in principle with Simon’s call for a stronger science of public administration. However, this book argues that lack of conceptual clarity, insufficient understanding of decision making processes, and limited success in generating models of evidence use that are tailored to specific purposes have doomed them to join his other Proverbs of Administration.

Advocates for evidence-based reform pursue increased production and use of policy knowledge, and increased transparency of its role in governance decisions. For example, starting in the 1960s, there have been repeated attempts to institute performance-based budgeting that explicitly links budget decisions to specific program performance information. At a broad level, these efforts are consistent with Madison’s emphasis on the ability of data, information, and evidence to improve decision making. However, these reforms have often experienced limited and uneven success, and a wealth of research has been dedicated to examining why political decision makers resist or ignore relevant policy knowledge (Gilmour and Lewis 2006a; Olsen and Levy 2004; Radin 2006; Shulock 1999). Both the framing of calls for reform and studies of their limited success reveal the evidence-based proverb at play. Specifically, both have treated the presence of political factors as support for claims that evidence-based reforms have failed, treating the two as an either-or proposition.

In performance-based budgeting reforms, one common issue is deciding whether a program that is not meeting performance targets should have its budget reduced, or if this is a signal that the program needs more resources to work. The answer to this question requires consideration of competing goals and alternatives that are influenced by ideological beliefs, engaged interests, and institutional arrangements. Rather than assessing the balance between policy knowledge and these political dimensions, reformers have reduced Madison’s advice to a proverb that policy knowledge can and should supplant politics in decision making. This book will show that reliance on this evidence-based proverb, coupled with restrictive definitions of policy knowledge, has repeatedly led to unfulfilled promises that data, information, and evidence will drive successful governance. Over time, these broken promises may lead decision makers to become more skeptical of policy knowledge, leading paradoxically to decreased trust in and influence of enlightenment. For example, a study of one government-wide performance-based budgeting reform found that congressional appropriations decisions were not significantly correlated with performance results (Gilmour and Lewis 2006a), ultimately leading to the information itself being challenged as political and the effort to generate it being abandoned.

This chapter provides a broad introduction to evidence-based governance. It will show that the “evidence” cited in related reform efforts is typically, whether explicitly or implicitly, limited to a single source or type of policy knowledge. It will show that, in each case, researchers and reformers have identified decisions, which they argue should have been guided by policy knowledge from evidence-based reforms, but which instead were driven by some combination of politics and “lesser” data, information, and evidence.

This chapter will explain that this limited use of policy knowledge is a clear and predictable result of three aspects of reforms. Specifically, reforms oversimplify the definition of policy knowledge, overpromise on the likely impacts of their reform, and ignore or undervalue the role of politics. As a result, they produce data, information, and evidence that ignore the reality of the political system and lack the contextual depth necessary to identify and successfully support a range of relevant governance decisions. Despite these challenges, the goal of linking policy knowledge to decisions continues to hold promise for improved decision making, and evidence-based reforms have remained popular. However, their uneven and limited success will continue unless the evidence-based proverb is abandoned. Reformers and scholars must adopt a more realistic view of the strengths and limitations of policy knowledge and a more contextually sophisticated view of decision making that recognizes the role and value of political factors. They must abandon a conception of the value of policy knowledge rooted solely in the strength of data, information, and evidence and must incorporate considerations of purpose and relevant political factors.

This book attempts to take a step toward this goal by offering frameworks and analyses that demonstrate how researchers and reformers can develop evidence-based reforms that are more responsive to the limitations addressed above. By examining the production, use, and impact of policy knowledge in federal education policy over the past 50 years, this study will explore how variation among the types and sources of policy knowledge, decision purposes, and political tens...

Table of contents

• Cover
• Front Matter
• Part I. Confronting the Evidence-Based Proverb
• Part II. Politics and Policy Knowledge in Federal Education
• Part III. Conclusions and Implications for Evidence-Based Governance
• Back Matter