Evidence-Based Public Management

Practices, Issues and Prospects

Anna Shillabeer, Terry F. Buss, Denise M. Rousseau

• Publisher: Routledge
• Year: 2015
• Print ISBN: 9780765624222
• eBook ISBN: 9781317470892
• Topic: Politics & International Relations
• Subtopic: City Planning & Urban Development
• Index: Politics & International Relations

Evidence-based management (EBMgt) derives principles of good management from scientific research, meta-analysis, literature reviews, and case studies, and then translates them into practice. This book is the first systematic assessment of EBMgt and its potential application in public management.

Part I

---

Background on Evidence-Based Public Management

1

The Emerging Field of Evidence-Based Public Management

Terry F. Buss and Anna Shillabeer

There is nothing a politician likes so little as to be well informed; it makes decision-making so complex and difficult.

—John Maynard Keynes

What Is Evidence-Based Management or Public Management?

It is a simple question to ask, but a complicated one to answer. Let us start with the purpose of evidence-based approaches: to improve public management and policy making by grounding decision making in evidence. This in turn leads to three contentious questions: What is evidence; how is evidence used in practice, policy, programs, and management; and what is new (or “emerging”) about the field?

Evidence

The first point of contention and confusion concerns the issue of what constitutes evidence. On one extreme are those who believe that evidence for decision making can, and in many cases should, come from any of a variety of sources—from experience and expert opinion through case studies to sophisticated experimental research—subject to some caveats, over which there is further disagreement: that evidence is of high quality, the best available, good enough, the only available, or compliant with widely accepted standards in a research, policy, or management field. At the other extreme are those who hold that only evaluations based on rigorous social science experimental research designs—either random control trials or, for some, quasi-experimental designs—can be evidence. In the middle ground between these polar opposites are those who try to classify evidentiary methods in either hierarchies or matrices, suggesting that some methods are better suited than others for answering certain questions. Finally, there are those who try to reconcile these divergent positions by suggesting that no evidentiary method is without serious flaws, and, therefore, evidence must be marshaled using a variety of methods and sources in an effort at triangulation: if multiple methods and sources point to the same conclusion, then managers and policy makers will have increased confidence in its veridicality. In the real world of management and policy making, most practitioners treat evidence as only one of several factors taken into account in decision making, the others being experience, feasibility, strategy, and politics. For practitioners, then, the question is not so much what is the best evidence, but rather what weight to give that evidence in their decision calculus. Chapter 2 lays out the controversies involving these issues.

How one views evidence is entirely a question of values and, in our view, is just as unresolvable as trying to decide which is best; chocolate or vanilla ice cream. What is clear is that disagreements about evidence trace their origins to decades-old debates in the philosophy of social sciences and about public management and policy methods. As such, it is unlikely that issues surrounding evidence-based management approaches will be resolved anytime soon. It is also likely the case that these long-standing issues raise questions in the minds of managers and decision makers who are not too keen about grounding decisions in such uncertainty.

Practice, Policy, Programs, and Management

Concerns about practice, policy, programs, and management create a second point of contention and confusion about evidence-based approaches. If one conducts a search on Amazon.com and Google, one finds that literature from all four approaches appears in the same search even after narrowing the search criteria: so the terms in use are ambiguous, overlapping, and inconsistent. Consider evidence-based practice (EBP). Much of the literature in EBP resides in the helping professions. How do social workers get the best outcomes for clients? Which treatment modalities work best for patients with chronic disease? In these contexts, practice is typically associated with management. Physicians, for example, use the phrase patient management strategy to lay out a treatment plan. But the concept of practice is also prevalent in public management. What is the best way to manage “talent” in the federal service? What steps should be taken to identify management and policy risk in public agencies? Evidence in this context means management (or practice) that complies or comports with widely accepted standards, guidance, lessons (irritating referred to as “learnings”), or best practices. Generally, they are a priori assumptions about what managers should do (Hewison 2004). They are nearly always prescriptive. These standards, and so forth, themselves are not necessarily (and likely not) derived from evidence-based studies. Chapter 8 sorts out the meaning and application of good management practice in government.

With respect to evidence-based policy (EBP)—which shares its acronym with evidence-based practice—one sees the application of every evidentiary method imaginable. Why? By public policy we mean a vision, goal, plan, or direction articulated by the executive, legislature, or courts that translates into programs, services, actions, or activities; attempts to solve a problem, make an improvement, or change a course of action; and tries to realize desired outcomes or impacts in the political arena. Public policy takes the form of legislation, regulation, executive orders, directives, and pronouncements. So, as a first cut, public policy is so broad that it legitimately employs every method in the social or policy sciences, and even the medical or hard sciences, often through metaphors and analysis.

But it is whether the focus ought to be on program performance that causes issues. Much of what government does involves assessing program performance, variously known as performance-based management and budgeting, results-based or results-oriented management, or management for results or outcomes (see Chapter 8 for a review and assessment of performance-based management and management reform in government and Chapter 16 for a modest proposal to the Obama administration to improve the federal performance management system). Evidence of good management, then, concerns the attainment of program goals against a set of quantifiable performance measures or indicators. So, a program’s performance is prima facie evidence of good or bad management.

Assessing program performance is not viewed as research, and hence many would exclude it from evidence-based management or evidence-based policy approaches, preferring to reserve “evidence” to refer to rigorous program evaluation, especially that grounded in experimental or quasi-experimental research design. So, some would exclude qualitative research as evidence or even science. While a compelling case can be made for experimental research designs, that approach does not apply to much of what public managers actually do. Some critics believe that a focus on experimentation is an attempt to elevate program evaluation to the same status as that enjoyed in medical research. What is interesting about this is that the New England Journal of Medicine, perhaps the most prestigious journal in the world, regularly publishes case studies as evidence.

To complicate matters, some include program evaluation and performance in the same management system, requiring that both be accomplished to demonstrate program effectiveness. If this were not enough, many believe that one cannot (or should not) separate management from policy. For them, evidence-based policy and management can be used interchangeably. So, for example, discussions of evidence in public management often use EBP and even medicine to make their points. We have done so in Chapter 2! In any case, Chapter 8 attempts to sort out these conundrums.

This leaves evidence-based management (EBMgt). Evidence in management, whether in the public or private sector, concerns the need to support management decision making with sound research. The impetus for EBMgt comes from the fact that much of what constitutes good management practice is likely based on faddism—personal memoirs, limited case studies, rank speculation, and many textbooks and the like that purport to be the latest in good management (Watson 2009; Moss and Francis 2007; Pfeffer and Sutton 2006; Brindle and Stearns 2001). Unfortunately, when examining these resources, one soon finds they have no scientific basis in fact, most are contradictory when examined side by side, a lot have led to organizational failure, and over time advice for managers dramatically changes as one fad replaces another.

So, the field is cluttered with terms of art that really do not represent a body of knowledge where there is much consensus. The field has no common view about what management is, what good management is, what evidence is, what good evidence is, what methods ought to be used, and perhaps what it all means. Every chapter in this volume touches on these issues in one way or another. This makes EBMgt a very interesting enterprise in which to work! We believe the work is well worth doing.

Origins

A third point of contention and confusion concerns whether evidence-based approaches, especially in management, are new, emerging, or long established. Most of the commentary on evidence-based approaches suggests that use of evidence in medical research and practice is the antecedent of and the inspiration for the approach. For our purposes, modern evidence-based medicine (EBM) emerged as a field in the 1970s with the work of Archie Cochrane (1992). Cochrane advocated for and convinced the medical research community they needed to demonstrate through experimental research that their practices and treatments were efficacious: patients have a way of getting well regardless of the treatment or lack thereof. Cochrane, who passed away in 1988, was honored when his colleagues launched the Cochrane Collaboration in 1993 with this rationale: “Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research” (http://www.cochrane.org/docs/ebm.htm).

In Communicating Social Science Research to Policymakers (1998), Vaughan and Buss decried the fact that policy analysts lacked a knowledge base about what works and what does not comparable to their counterparts in medical research and practice.

One irony of this fascination with EBM is that health care organizations, both public and private, have been unable to import advances in medical practice into the management of the organizations where medicine is practiced (e.g., see Chapter 13). Additionally, there is considerable disagreement about EBM as an approach in and of itself.

Although we think commentators on EBMgt are right to link its inspiration to medical practice, it is ironic that few seem to have recalled the long tradition of evidence-based activity (as broadly defined) in government. Consider just two. Aristotle, in the third century bc, served as a political adviser to Alexander the Great. Some scholars believe that Aristotle’s advice helped Alexander model his empire after the Greek city-state. Where did Aristotle’s knowledge come from? He conducted studies of different governance structures in various city-states, including their constitutions, to determine which features would lead to desirable political consequences. He published this in the Politics, perhaps the most famous policy book of all time. Machiavelli, in the fifteenth century, provided management and policy advice in the second-most famous policy book, The Prince, to Renaissance rulers in Florence. His methodology was to study policy decisions drawn from ancient Rome and Greece to derive lessons for his political masters. The list of contributions over time is legion and continues uninterrupted to the present.

A final point concerns whether public management needs its own acronym, perhaps EBPMgt, to be included among evidence-based medicine (EBM), practice (EBP), policy (EBP), and management (EBMgt). We vote no. As this set of acronyms illustrates, we are running out of options, and as the chapters in this book suggest, there is much commonality across these fields to suggest that EBMgt will do just fine (see especially Chapter 3).

What Are the Methods in Evidence-Based Approaches?

Contrary ...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• Foreword
• Acknowledgments
• Part I. Background on Evidence-Based Public Management
• Part II. Evidence-Based Public Management
• Part III. Applications and Research
• Part IV. Future Directions
• About the Editors and Contributors
• Index