Public Administration Research Methods

Tools for Evaluation and Evidence-Based Practice

Warren S. Eller, Brian J. Gerber, Scott E. Robinson

• Publisher: Routledge
• Year: 2018
• Print ISBN: 9781138059276
• eBook ISBN: 9781351672009
• Topic: Politics & International Relations
• Subtopic: Politics
• Index: Politics & International Relations

The best decisions made by public managers are based not on instinct, but on an informed understanding of what's happening on the ground. Policy may be directed by ideology, but it must also be founded on reality. The challenge of making the right decisions as a public manager is often, therefore, based on the need for rigorous, actionable research. Now in a thoughtfully revised second edition, this textbook shows students of Public Administration exactly how to use both qualitative and quantitative research techniques to give them the best chance to make the right decisions.

Uniquely, Eller, Gerber, and Robinson present research methodologies through a series of real-life case studies, with each chapter exploring situations where a public manager can use research to answer specific questions, demonstrating how that research can inform future policy. Taking readers through the key concepts, from research design and sampling to interviews, survey data, and more statistical-based approaches, this new edition provides a complete guide to using research in the public and voluntary sectors. New to this edition:

• To better orient the student, the second edition is thematically arranged. Five sections, each with a short essay, provide not only previews of the content of each section, but more importantly guide the reader through how the concepts and techniques covered relate to real-world use and application.
• A new chapter on applied quantitative analyses has been added to offer coverage of several commonly-used and valuable analytic techniques for decision making for policy and management: benefit-cost analysis, risk assessment, and forecasting.
• The second edition is accompanied by online materials containing suggested course plans and sample syllabi, PowerPoint lecture slides, and student support materials to illustrate the application of key concepts and analytic techniques.

Each chapter also includes discussion questions, class exercises, end of chapter review questions, and key vocabulary to provide students with a range of further tools to apply research principles to practical situations.

Section III
Estimating How the World Works: Testing Claims and Drawing Inferences

Section III Preview
Estimating How the World Works: Testing Claims and Drawing Inferences

Describing Those Data You've Collected

The previous section provided a useful discussion of the basic operations of gathering data. We covered how to structure a data collection effort (research design), how to measure social phenomenon (measurement), and how to draw a sample from a population of interest (sampling). We then moved on to explanations of data gathering and data use techniques, such as performing a case study or conducting interviews with subject matter experts. While all of that material is extremely important, here in section III, we are moving on to some even better stuff! You might think about where we are at this point in the text by considering this statement: ultimately we want to make claims about how the world is and how it works in the area of public affairs. We are at the point in this book where we move from data gathering to description and to inference. So we can say that one really good way of explaining how the world of public affairs works is to measure social phenomena that have importance to governance of the public sector. Once you collect measures, we are going to need to describe the nature of those variable distributions. Doing so is what descriptive statistics is all about: providing useful explanations of what distributions tell us about what we have collected. Effective descriptions are powerful: they give insight to what is happening with policies and programs.

Here’s a true, real-world story, that the authors of this book have direct personal knowledge of due to applied project work. Some of those charged with oversight for a state agency that distributed community development funds did not have full or precise awareness of the overall trend in how those funds had been geographically dispersed across the state. (That may seem a little hard to believe, but in abbreviated form, that was the situation.) A simple map showing the geographic patterns of where the development funds were delivered revealed important implications for how well the program was working in practice. In short, a simple mapping exercise—a visual display of data— produced a much clearer understanding of the nature of policy implementation; in turn the oversight board had a stronger basis for considering possible program adjustments. The lesson: do not underestimate the power of effective data description.

The first two chapters in this section cover the topics of coding and displaying data, as well as an introduction to the basics of descriptive statistics.

After Describing Your Distributions: Moving on to Testing and Drawing Inferences

After learning about how to describe and present variable distributions, our next step is to learn about inferential statistics. We would like to be fully honest here (given we’re honest people, more or less): this is where students often start to find the material challenging. It seems there tends to be two major hurdles in comprehension of the material. The first is understanding why we are employing a particular type of hypothesis test. The second is understanding what statistical significance means. Chapter 15 in this section gives you a clear and concise explanation of both, but before you get to that chapter, we would like to offer a quick word or two here to help with the intuition of what you are about to study.

If you think about observing outcomes in the real world, you might think of things in terms of a bell curve, which presumably everyone has seen. A bell curve shows in graphical form that most observations of a phenomenon occur around the central point of a distribution (the big middle chunk of the curve). Moving away from that central point (where most observations are found) we can think of those observations as more extreme and less likely to be seen (hence the tails of the curve get really small; hence the bell shape). Here’s a silly example: the average adult male in the United States is about 5 feet 9 inches or so. Say you enter a classroom for this course, and there are about 30 students enrolled and in the room. But as you walk in, you see a group of guys in the room, say four or five, taller than 6 feet 10 inches. Now, just due to random variation you’d expect absolutely to see in a classroom of 30 students some a little taller and some a little shorter than 5 feet 9 inches; that is, you wouldn’t expect all the males in the course to be exactly the average height of an adult male in the United States. That is important to recognize: some amount of variation around that average height is to be expected—but most people are going to be fairly close to the population average (which we’ve defined as about 5 feet 9 inches). But you really wouldn’t expect to see one-sixth of the classroom to be nearly 7 feet tall! There is a really low chance that you get that many tall people in a small group due to random chance; i.e., it is unlikely that your course of 30 students—just by wild coincidence—had five guys of such impressive and unusual height. So what do you make of the situation? Well, you might guess that for some reason, a group of members of the university’s men’s basketball team ended up enrolling in that course. That might be a good guess for what is going on because you are pretty sure it is unlikely to see that many tall people in a classroom of 30 students just by random chance.

While this is a silly little example, in essence it reveals what the process of hypothesis testing and ...

Table of contents

• Cover
• Title
• Copyright
• Dedication
• About the Authors
• Contents
• Section I The Purpose and Value of Evaluation and Analytic Skills and their Use in Evidence in Decision Making
• Section II The Basic Operations of Research: Setting Up a Project, Gathering Data
• Section III Estimating How the World Works: Testing Claims and Drawing Inferences
• Section IV Advanced Quantitative Techniques and Applied Decision Tools
• Section V Managing Projects, Completing Projects
• Appendix
• Index