Qualitative Inquiry in Evaluation

From Theory to Practice

Leslie Goodyear, Eric Barela, Jennifer Jewiss, Janet Usinger, Leslie Goodyear, Eric Barela, Jennifer Jewiss, Janet Usinger

• Publisher: Jossey-Bass
• Year: 2014
• Print ISBN: 9780470447673
• eBook ISBN: 9781118415252
• Edition: 1
• Topic: Education
• Subtopic: Evaluation & Assessment in Education

Integrate qualitative inquiry approaches and methods into the practice of evaluation

Qualitative inquiry can have a major effect on evaluation practice, and provides evaluators a means to explore and examine various settings and contexts in need of rich description and deeper understanding. Qualitative Inquiry in Evaluation: From Theory to Practice explores the most important considerations for both students and evaluation professionals. Using various evaluation theories and approaches as a springboard for real-world practice, this reference serves as an accessible text for beginning students and seasoned professionals alike. Readers are given an in-depth view of the key qualities and benefits of qualitative inquiry, which also serves as a crucial counterpart to quantitative analysis.

Chapters in part one focus on the foundations, core concepts, and intersection of evaluation theory and qualitative inquiry.

Part two contains contributions from leading evaluators whose design, implementation, and reporting strategies for qualitative inquiry are centered on common, real-world settings. These case-based chapters point to the strengths and challenges of implementing qualitative evaluations. Key competencies for conducting effective qualitative evaluations are also discussed.

• Explores the role of qualitative inquiry in many prominent approaches to evaluation
• Discusses the method's history and delves into key concepts in qualitative inquiry and evaluation
• Helps readers understand which qualities are necessary to be an effective qualitative evaluator
• Presents the viewpoints and experiences of expert editors and contributing authors with high levels of understanding on the topic

Qualitative Inquiry in Evaluation: From Theory to Practice is a vital tool for evaluators and students alike who are looking to deepen their understanding of the theoretical perspectives and practice considerations of qualitative evaluation.

Part 1

The Intersection of Qualitative Inquiry and Evaluation Approaches

Programs are designed by people; they are implemented by people; they affect the lives of participants and the viability of communities; and they often are supported by people who care deeply about the issues addressed by the program. Perhaps because the common denominator of any program is the human element, qualitative inquiry has been intentionally incorporated into the practice of evaluation over the past several decades. Program stakeholders, including people who financially support evaluations, often ask questions that go beyond simple assessments of program goals and objectives. Stakeholders also want answers to how and why questions. They are interested in improving the program, learning for whom and in what settings the program works best, and understanding how a particular program experience can inform policy. How and why questions require evaluation approaches that capture complexity and nuanced individual perceptions. Enter qualitative inquiry in the practice of evaluation.

A key to gaining confidence and prowess in the practice of evaluation, particularly evaluations that use qualitative inquiry, is to enter the field with a solid theoretical and methodological foundation. Part 1 of this volume begins with an overview of the origins and rationale of incorporating qualitative inquiry into the practice of evaluation. The remaining chapters focus on a few prominent evaluation approaches that provide critical direction and guideposts for the professional practice of evaluation. Although not an exhaustive examination of all the evaluation approaches that incorporate qualitative inquiry, each chapter illustrates the critical role that qualitative inquiry plays within a particular evaluation framework, including some evaluation approaches that are not necessarily associated with qualitative inquiry.

Chapter 1
The Origins, Meaning, and Significance of Qualitative Inquiry in Evaluation

Thomas A. Schwandt and Timothy J. Cash

---

Key Ideas

• The definition of qualitative evaluation has been constantly contested. Several narrative accounts exist of the genesis of qualitative inquiry; each sets a particular framework for the introduction of qualitative inquiry and establishes its role in the development of program evaluation.
• The move to incorporate qualitative evaluation into the lexicon was, in some ways, a response to educational researchers' failures to demonstrate program effects using experimental designs and came on the heels of Cronbach's (1963) call for evaluators to “reconceptualize evaluation.”
• Influenced by anthropology and sociology, qualitative evaluation brought new approaches to understanding human actions and meaning making to evaluation.
• Qualitative evaluation relies on methods used to generate qualitative data (e.g., interviewing, observation, focus groups, document review) and makes use of such reporting conventions as narratives, stories, and case studies.
• Qualitative evaluation prioritizes value pluralism and considers both the stakeholders' values and the evaluator's values.
• Within qualitative evaluation, an evaluation is more about social communication than about technical reporting.

---

This chapter sets the stage for subsequent chapters that discuss how qualitative inquiry is related to prominent evaluation approaches and how those who conduct and are committed to the practice of qualitative inquiry view several critical issues in evaluation practice. The chapter is meant as preliminary in the sense of a beginning or general orientation to key issues involved with the origins, meaning, and significance of qualitative evaluation; it is not an exhaustive examination of these issues. The chapter begins with two brief sections—the first presents a perspective on ways in which qualitative evaluation originated; the second discusses the contested definition of the term qualitative evaluation. The third section identifies what, in our view, are important contributions of the extensive literature in qualitative inquiry in evaluation to shaping the practice of evaluation.

The Genesis of Qualitative Evaluation

Some notable narratives over the past three decades describe the advent and development of qualitative inquiry in the field of evaluation in several unique ways (e.g., Campbell, 1984; Conner, Altman, & Jackson, 1984; Guba & Lincoln, 1987; Madaus & Stufflebeam, 2000; Patton, 1975; Rossi & Wright, 1987; Scriven, 1984). Madaus, Scriven, and Stufflebeam (1983) argued that qualitative evaluation was one of many new conceptualizations of evaluation that arose in the late 1960s and early 1970s, primarily in the field of education. These developments followed on the heels of Cronbach's (1963) call for educational evaluators to “turn away from their penchant for post hoc evaluations based on comparisons of norm-referenced test scores of experimental and control groups” and to “reconceptualize evaluation—not in terms of a horse race between competing programs but as a process of gathering and reporting information that could help guide curriculum development” (cited in Madaus, Scriven, & Stufflebeam, 1983, p. 12). Similarly, Guba (1969) pointed to what he called the failure of educational evaluation, arguing “the application of conventional experimental design to evaluation situations . . . conflicts with the principle that evaluation should facilitate the continuous improvement of a program” (p. 8).

For some scholars, the advent of qualitative inquiry in evaluation was inspired in large part by the failure of attempts to demonstrate the effects of Title I projects (funded by the Education and Secondary Education Act of 1964) using experimental designs. For Rossi and Wright (1987, p. 59), the introduction of qualitative evaluation was an “intellectual consequence . . . of [the] close-to-zero effects” of the social programs of the 1960s and 1970s. A strong critic of the national evaluation of Follow Through (an extension of the Head Start Program), begun in 1967 as a planned variation experiment, wrote:

We will not use the antiseptic assumptions of the research laboratory to compare children receiving new program assistance with those not receiving such aid. We recognize that the comparisons have never been productive, nor have they facilitated corrective action. The overwhelming number of evaluations conducted in this way have shown no significant differences between “experimental” and “control” groups. (Provus, 1971, p. 12)

Greene (2000) claimed that constructivist, qualitative approaches to evaluation emerged against the backdrop of several intellectual and social developments in the 1970s in the United States, including the “dethroning of experimental science as the paradigm for social program evaluation” (p. 992); a decline in the authority accorded social science theory; a decline in the authority of political figures in view of the Vietnam War, Watergate, and so on; and increased interest in value pluralism.

For Guba and Lincoln (1981, 1987, 1989; Lincoln & Guba, 1985; Guba, 1978) and Patton (1975), qualitative evaluation is about the birth (or perhaps discovery) in the 1970s of a new paradigm for evaluation derived from fieldwork methods in anthropology and qualitative sociology and from a strong interest in appropriating insights of the Verstehen tradition in German sociology. Scholars in that tradition held that understanding the actions of human beings as uniquely meaning-making creatures required methods different from those used to study the behavior of nonhuman objects. Lincoln and Guba (1985) initially called this new paradigm “naturalistic inquiry” and later refined it as responsive constructivist evaluation or “fourth-generation evaluation” (Guba & Lincoln, 1989). They claimed that three prior generations of evaluation (characterized as measurement, description, and judgment) were beset by several serious problems—a tendency toward managerialism, a failure to accommodate value pluralism, and an over-commitment to a scientific paradigm of inquiry. They argued that the fourth generation addresses these problems and offers a salutary alternative.

Each of these narratives presents a different understanding of the development of program evaluation and the role qualitative inquiry has played in that story. Each employs a particular framework for shaping its account of the introduction and development of qualitative evaluation. Each provides a partial perspective on how it is that qualitative approaches arose in the broad field of evaluation. The strongest reaction to the dominance of experimental and psychometric traditions in evaluation in the 1960s and 1970s came from scholars in education who were initially trained in those traditions, including Robert Stake, Egon Guba, Lee Cronbach, and others.

The history of qualitative evaluation has often been portrayed as a struggle between different methodologies and methods or of fundamental epistemological disagreements between, for example, strong empiricists and interpretivists or post-positivists and social constructionists. These accounts are accurate to the extent that they reflect the dominance of experimental methods and the hypothetico-deductive paradigm found in texts discussing evaluation research in the late 1960s and 1970s, (e.g., Bernstein & Freeman, 1975; Reicken & Boruch, 1974; Rossi & Williams, 1972; Suchman, 1967).

Guba and Lincoln are unique in interpreting the appearance and development of qualitative evaluation as a narrative of progression or generations (although a similar idea has been advanced by Denzin and Lincoln [1994] regarding the development of qualitative research in the social sciences more generally). We are more skeptical of this way of viewing the genesis of qualitative evaluation, for there is a modernist narrative of progress implicit in the movement from one “generation” of evaluation to the next. Our thinking about evaluation may indeed develop over time—for example, an enlargement on, improvement in, rejection of, or expansion on concepts and ideas—but earlier generations of evaluation thinking are still very active and still very much in dialogue with one another. One generation has not ceased to exist or completely given way to another.

An engaging, intellectual history of the advent and development of qualitative inquiry in the field of evaluation in the United States has yet to be written. Such a history would have to account for more than the methods wars or paradigm wars characteristic of several explanations. It would trace the influence that debates both within and outside the social sciences had on how the field of evaluation took shape, developed its multiple perspectives on what constitutes legitimate approaches to evaluation, wrestled with the politics of knowledge construction, and defined the role of professional evaluation expertise in contemporary society. Developing such an account is not our purpose here. After a brief discussion of definitions of qualitative evaluation, we offer a modest version of three sets of ideas that have their origins in the...

Table of contents

• Cover
• Title Page
• Copyright
• Figures and Tables
• Introduction
• About the Editors
• About the Contributors
• Acknowledgments
• Part 1: The Intersection of Qualitative Inquiry and Evaluation Approaches
• Part 2: Tales from the Field of Qualitative Evaluation
• Index
• End User License Agreement