Evaluation Theory, Models, and Applications

Daniel L. Stufflebeam, Chris L. S. Coryn

• Publisher: Jossey-Bass
• Year: 2014
• ISBN:
  9781118870228
• Edition: 2
• Topic: Education
• Subtopic: Evaluation & Assessment in Education

The golden standard evaluation reference text

Now in its second edition, Evaluation Theory, Models, and Applications is the vital text on evaluation models, perfect for classroom use as a textbook, and as a professional evaluation reference. The book begins with an overview of the evaluation field and program evaluation standards, and proceeds to cover the most widely used evaluation approaches. With new evaluation designs and the inclusion of the latest literature from the field, this Second Edition is an essential update for professionals and students who want to stay current. Understanding and choosing evaluation approaches is critical to many professions, and Evaluation Theory, Models, and Applications, Second Edition is the benchmark evaluation guide.

Authors Daniel L. Stufflebeam and Chris L. S. Coryn, widely considered experts in the evaluation field, introduce and describe 23 program evaluation approaches, including, new to this edition, transformative evaluation, participatory evaluation, consumer feedback, and meta-analysis. Evaluation Theory, Models, and Applications, Second Edition facilitates the process of planning, conducting, and assessing program evaluations. The highlighted evaluation approaches include:

• Experimental and quasi-experimental design evaluations
• Daniel L. Stufflebeam's CIPP Model
• Michael Scriven's Consumer-Oriented Evaluation
• Michael Patton's Utilization-Focused Evaluation
• Robert Stake's Responsive/Stakeholder-Centered Evaluation
• Case Study Evaluation

Key readings listed at the end of each chapter direct readers to the most important references for each topic. Learning objectives, review questions, student exercises, and instructor support materials complete the collection of tools. Choosing from evaluation approaches can be an overwhelming process, but Evaluation Theory, Models, and Applications, Second Edition updates the core evaluation concepts with the latest research, making this complex field accessible in just one book.

Part One

Fundamentals of Evaluation

Part One provides information on the foundations of evaluation. In the following three chapters we give an overview of the evaluation field, analyze the state of theory in the field, and describe the field's guiding principles and standards. These chapters afford an appreciation of the history and status of the evaluation discipline and address some of the key theoretical and professional issues facing its theoreticians and practitioners.

Chapter 1
Overview of the Evaluation Field

---

Learning Objectives

In this chapter you will learn about the following:

• The distinction between formal and informal evaluation
• The potential contributions and limitations of formal evaluation
• Evaluation as a profession and its relationship to other professions
• Conceptual and operational definitions of evaluation
• Key criteria for evaluating programs, including merit and worth
• The roles of values clarification and setting standards in reaching evaluative conclusions
• Four main uses of evaluation
• Distinctions between formative evaluation and summative evaluation
• Distinctions between research and evaluation
• Historical milestones in the development of professional evaluation

---

Evaluation is perhaps society's most fundamental discipline; it is an essential characteristic of the human condition; and it is the single most important and sophisticated cognitive process in the repertoire of human reasoning and logic (Osgood, Suci, & Tannenbaum, 1957). It permeates all areas of human activity and has important implications for maintaining and improving services and protecting citizens in all areas of interest to society. Evaluation is a process for giving attestations to such matters as reliability, effectiveness, cost-effectiveness, efficiency, safety, ease of use, and probity. Society and individual clients are at risk to the extent that services, products, and other objects of interest are of poor quality. Evaluation serves society by providing affirmations of worth, value, progress, accreditation, and accountability—and, when necessary, a credible, defensible, nonarbitrary basis for terminating bad programs or, conversely, expanding good programs.

What Are Appropriate Objects of Evaluations and Related Subdisciplines of Evaluation?

In general, we refer to objects of evaluations as evaluands. When the evaluand is a person, however, we follow Scriven's recommendation to label the person whose qualifications or performance is being evaluated as the evaluee (Scriven, 1991). Objects of evaluations may be programs, projects, policies, proposals, products, equipment, services, concepts and theories, data and other types of information, individuals, or organizations, among others. Although the practice of evaluation largely concentrates on program evaluation, one can refer to a range of other areas of evaluative inquiry, such as personnel evaluation, product evaluation, portfolio evaluation, performance evaluation, proposal evaluation, and policy evaluation. The scope of evaluation applications broadens greatly when one considers the wide range of disciplines, activities, and endeavors to which evaluation applies. One can speak, for example, of educational evaluation, social and human services evaluation, arts evaluation, consumer product evaluation, human resources development and evaluation, city planning and evaluation, real estate appraising, engineering testing and evaluation, hospital evaluation, drug testing, manufacturing evaluation, science policy evaluation, evaluation of international development and international aid, agricultural experimentation, and environmental evaluation.

Are Evaluations Enough to Control Quality, Guide Improvement, and Protect Consumers?

The presence of sound evaluation does not necessarily guarantee high quality in services or that those in authority will heed the lessons of evaluation and take needed corrective actions. Evaluations provide only one of the ingredients needed for quality assurance and improvement. There are many examples of defective products that have harmed consumers not because of a lack of pertinent evaluative information, but because of a failure on the part of decision makers to heed and act on rather than ignore or cover up alarming evaluative information. The continued sales of the Corvair automobile after its developers and marketers knew of its rear-end collision fire hazard provides one clear example (see also Nader, 1965). Here we see that society has a critical need not only for competent evaluators but for evaluation-oriented decision makers as well. For evaluations to make a positive difference, policymakers, regulatory bodies, service providers, and others must obtain and act responsibly on evaluation findings. The production and appropriate use of sound evaluation constitute one of the most vital contributors to strong services and societal progress.

Evaluation as a Profession and Its Relationship to Other Professions

As a profession with important roles in society, evaluation has technical aspects requiring thorough and ongoing training. It possesses an extensive and rapidly developing professional literature containing information on evaluation models and methods and findings from research on evaluation (Christie, 2011; Coryn & Westine, 2013). Its research material evolves from, and is closely connected to, the wide range of evaluations conducted in all fields. Evaluation has many professional organizations, including the American Evaluation Association (AEA) and other state and national evaluation associations. Among the earliest known professional societies were the May 12th Group, Division H of the American Educational Research Association (AERA), the Evaluation Network (E-Net), and the Evaluation Research Society (ERS), all of which originated in the late 1960s and early 1970s.

In 1995 there were only five evaluation organizations worldwide, including AEA (ensuing from the merger of E-Net and ERS in 1986), the Canadian Evaluation Society (CES), the Australasian Evaluation Society (AES), the European Evaluation Society, and the Central American Evaluation Society. By 2006 there were more than fifty national and regional evaluation organizations throughout the world, most in developing countries (Segone & Ocampo, 2006). There are also university training programs in evaluation, among them the Interdisciplinary PhD in Evaluation (IDPE) program and the Evaluation, Measurement, and Research (EMR) program at Western Michigan University (Coryn, Stufflebeam, Davidson, & Scriven, 2010), as well as other evaluation graduate programs at Claremont Graduate University, the University of Illinois, The Ohio State University, the University of Minnesota, the University of North Carolina, the University of Virginia, and the University of California at Los Angeles (for historical trends in graduate training in evaluation, see LaVelle and Donaldson [2010]). In addition, the field has developed recognized standards for evaluation services, including the Joint Committee on Standards for Educational Evaluation's standards for evaluating programs, personnel, and students (1981, 1988, 1994, 2003, 2009, 2011) and the U.S. Government Accountability Office's Government Auditing Standards (U.S. General Accounting Office, 2002; U.S. Government Accountability Office, 2003 2007), plus AEA's Guiding Principles for Evaluators (2004).

To communicate and disseminate developments in, thinking about, and critiques of evaluation theory, methods, and practice, professional journals and other types of publications dedicated exclusively to evaluation scholarship and practice began to appear in the 1970s (Coryn, 2007a). One of the field's earliest publications, which first appeared in 1974, was the journal Evaluation and Program Planning. This was followed in 1975 by the journal Studies in Evaluation; in 1976 by Evaluation Review: A Journal of Applied Social Research; some years later by the American Journal of Evaluation (formerly published under the titles Evaluation News, prior to 1986, and Evaluation Practice, between 1986 and 1997); New Directions for Evaluation (formerly New Directions for Program Evaluation) and Evaluation & the Health Professions, both of which appeared in 1978; and Educational Evaluation and Policy Analysis, which first appeared in 1979.

The 1980s were marked by the appearance of the Canadian Journal of Program Evaluation, which emerged in 1986; the Journal of Personnel Evaluation in Education (now published under the title Educational Assessment, Evaluation, and Accountability), which was first published in 1987; and Practical Assessment, Research and Evaluation, which was launched in 1988.

In the 1990s several additional journals appeared, including Research Evaluation in 1991, which is published in the Netherlands; Evaluation: The International Journal of Theory, Research and Practice, which is published in the United Kingdom; and the Journal of Evaluation in Clinical Practice, the last two having first been published in 1995. In the next decade several more scholarly journals devoted to evaluation emerged, including the Evaluation Journal of Australasia, which was first published in 2000, and the Journal of MultiDisciplinary Evaluation, which first appeared in 2004.

Despite the burgeoning number of scholarly evaluation journals, many evaluation scholars and practitioners disseminate their work in discipline-specific journals, including those found in education, health and medicine, philosophy, psychology, and sociology, to name but a few. In addition to publishing in evaluation and discipline-specific journals, other evaluation scholars publish their work in subject-specific areas, such as measurement, research, and statistics.

As a distinct profession, evaluation is supportive of all other professions and in turn is supported by many of them; no profession could excel without evaluation. Services and research can lead to progress and stand up to public and professional scrutiny only if they are regularly subjected to rigorous evaluation and shown to be sound. Also, improvement-oriented self-evaluation is a hallmark of professionalism. Program leaders and all members of any profession are obligated to serve their clients...