Evaluating Public and Community Health Programs

Muriel J. Harris

• Publisher: Jossey-Bass
• Year: 2016
• Print ISBN: 9781119151050
• eBook ISBN: 9781119151081
• Edition: 2
• Topic: Medicine
• Subtopic: Public Health, Administration & Care

A practical introduction to participatory program evaluation

Evaluating Public and Community Health Programs provides a comprehensive introduction to the theory and practice of evaluation, with a participatory model that brings stakeholders together for the good of the program. Linking community assessment, program implementation, and program evaluation, this book emphasizes practical, ongoing evaluation strategies that connect theory with application. This updated second edition includes new discussion on planning policy change programs using logic models and theory of change, plus expanded coverage of processes, outcomes, data collection, and more. Each chapter includes classroom activities and group discussion prompts, and the companion website provides worksheets, lecture slides, and a test bank for instructors. Mini cases help illustrate the real-world applications of the methods described, and expanded case studies allow students to dig deeper into practice and apply what they've learned.

Accurate and effective evaluation is the key to a successful program. This book provides a thorough introduction to all aspects of this critical function, with a wealth of opportunities to apply new concepts.

• Learn evaluation strategies that involve all program stakeholders
• Link theory to practice with new mini cases and examples
• Understand the uses, processes, and approaches to evaluation
• Discover how ongoing evaluation increases program effectiveness

Public and community health programs are a vital part of our social infrastructure, and the more effective they are, the more people they can serve. Proper planning is important, but continued evaluation is what keeps a program on track for the long term. Evaluating Public and Community Health Programs provides clear instruction and insightful discussion on the many facets of evaluation, with a central focus on real-world service.

CHAPTER 1
AN INTRODUCTION TO PUBLIC AND COMMUNITY HEALTH EVALUATION

---

LEARNING OBJECTIVES

• Identify the uses and approaches of evaluation.
• Describe preassessment evaluation.
• List the principles of participatory evaluation.
• Describe the links among community assessment, program implementation, and program evaluation.
• Explain the ethical and cultural issues in evaluation.
• Describe the value and role of stakeholders in evaluation.

---

Public health may be assessed by the impact it has on improving the quality of life of people and communities through the elimination or the reduction in the incidence, prevalence, and rates of disease and disability. An additional aspect of public health is to create social and physical environments that promote good health for all. The Healthy People 2020 goal describes health as being produced at multiple levels: households, neighborhoods and communities. In addition, it describes the importance of social and economic resources for health with a new focus on the social determinants of health. Its overarching goals are as follows:

1. Attain high quality, longer lives free of preventable disease, disability, injury, and premature death.
2. Achieve health equity, eliminate disparities, and improve the health of all groups.
3. Create social and physical environments that promote good health for all.
4. Promote quality of life, healthy development, and healthy behaviors across all life stages. (http://www.healthypeople.gov/2020/About‐Healthy‐People)

Public health, therefore, has an obligation to improve conditions and access to appropriate and adequate resources for healthy living for all people, and it includes education, nutrition, exercise, and social environments. Public health programs and policies may be instituted at the local, state, national, or international level.

The Committee for the Study of the Future of Public Health defines the mission of public health as “fulfilling society's interest in assuring conditions in which people can be healthy” (Institute of Medicine, 2001, p. 7). Public and community health programs and initiatives exist in order to “do good” and to address social problems or to improve social conditions (Rossi, Lipsey, & Freeman, 2004, p. 17). Public health interventions address social problems or conditions by taking into consideration the underlying factors and core causes of the problem. Within this context, program evaluation determines whether public health program and policy initiatives improve health and quality of life.

Evaluation is often referred to as applied research. Using the word applied in the definition lends it certain characteristics that allow it to differ from traditional research in significant ways.

• Evaluation is about a particular initiative. It is generally carried out for the purposes of assessing the initiative, and the results are not generalizable. However, with the scaling up of programs to reach increasingly large segments of the population, and with common outcome expectations and common measures, evaluations can increase their generalizability. Research traditionally aims to produce results that are generalizable to a whole population, place, or setting in a single experiment.
• Evaluations are designed to improve an initiative and to provide information for decision‐making at the program or policy level; research aims to prove whether there is a cause‐and‐effect relationship between two entities in a controlled situation.
• Evaluation questions are generally related to understanding why and how well an intervention worked, as well as to determining whether it worked. Research is much more focused on the end point, on whether an intervention worked and much less on the process for achieving the end result.
• Evaluation questions are identified by the stakeholders in collaboration with the evaluators; research questions are usually dictated by the researcher's agenda.

Some approaches to evaluation, such as those that rely on determining whether goals and objectives are achieved, assess the effects of a program; the judicial approach asks for arguments for and against the program, and program accreditations seek ratings of programs based on a professional judgment of their quality and are usually preceded by a self‐study. Consumer‐oriented approaches are responsive to stakeholders and encourage their participation. This book focuses on the evaluation of public health programs primarily at the community and program level.

OVERVIEW OF EVALUATION

Rossi et al. (2004) describe evaluation as “the use of social research methods to systematically investigate the effectiveness of social intervention programs in ways that are adapted to their political and organizational environments and are designed to inform social action to inform social conditions” (p. 16). In addition, these authors caution that evaluation provides the best information possible under conditions that involve a political process of balancing interests and reaching decisions (p. 419).

Evaluation is the cornerstone for improving public health programs and is conducted for the purpose of making a judgment of a program's worth or value. Evaluation incorporates steps that specify and describe the activities and the process of evaluation; the initiative and why it is being evaluated; the measures needed to assess the inputs, outputs, and outcomes; and the methodology for collecting the information (data). In addition, an evaluation analyzes data and disseminates results in ways that ensure that the evaluation is useful.

This definition of evaluation as adopted by the social sciences and public health reflects a long tradition of evaluation that takes different approaches to evaluation and are applied across a wide field of study. Each has its own criteria, and the evaluator chooses the approach that best suits their field, their inclination, or the purpose for which the evaluation is being conducted.

The next section provides a brief overview of the most widely used approaches. These evaluation approaches include the consumer‐based, decision‐based, goal‐free, participatory, expertise‐oriented, and objectives‐based.

Consumer‐Based Approach

In the consumer‐based evaluation approach, the needs of the consumer are the primary focus and the role of the evaluator is to develop or select criteria against which the initiative or product is judged for its worth. The focus of this evaluation is on the cost, durability, and performance of the initiative or product being evaluated.

Decision‐Based Approach

This approach adopts a framework for conducting evaluation that includes the context, inputs, process, and product. It is also referred to as the context, input, process, and product (CIPP) approach. In including the context in the evaluation, this approach considers both the problem that is being addressed and the intervention that addresses it. In the context of public health, adopting this model requires understanding the public health problem being addressed and the program or policy intended to address it. The community or needs assessment forms the basis for developing the intervention. The input components of the evaluation assess the relationship between the resources available for the program and the activities identified to address the problem. Process evaluation, which is the third component of this model, asks the question, “Is the program being implemented as planned?” The last component, the product, assesses the extent to which goals and objectives have been met.

Goal‐Free Approach

A goal‐free approach to evaluation is just that. The evaluation does not start out with any predefined goals or objectives related to the initiative being evaluated. It is expected that the initiative will have many outcomes that are not necessarily related to the objectives that may have been crafted when the initiative was initially conceived and started. Therefore, not having defined objectives allows the evaluator to explore a wide range of options for evaluation.

Participatory Approach

The participatory approach to evaluation adopts an approach that values and integrates stakeholders into the process. Stakeholders in this process are the beneficiaries of the initiative's interventions. In this case, the evaluator serves as technical advisor allowing the stakeholders to take responsibility for most aspects of the evaluation process. The aim of this approach is to transfer skills in a co‐learning setting and to empower stakeholders to become evaluators of their own initiatives.

Expertise‐Oriented Approach

The expertise‐oriented approach expects the evaluator to be a content expert who draws on his life experience to judge a program's worth. It may or not be accompanied by specified clearly defined and explicit criteria. This approach is often used in judging competitions and in public health and other fields in accreditation. However, in accreditation, such as the accreditation of schools of public health, although the institution provides the self‐study narrative based on predefined criteria, the judgment of the program's merits and the decision to grant accreditation is made by the accrediting body.

Objectives‐Based Approach

The objectives‐based evaluation is the most commonly used in public health practice especially recently as responses to calls for proposals for funding now invariably require the applicant to include objectives. The objectives for an initiative are developed following the community assessment, and form the bases on which the initiative is developed focusing on risk or protective factors that would have an impact on the problem being addressed. Additional obj...

Table of contents

• Cover
• Title Page
• Copyright Page
• Table of Contents
• Dedication
• Preface
• Acknowledgments
• CHAPTER 1: AN INTRODUCTION TO PUBLIC AND COMMUNITY HEALTH EVALUATION
• CHAPTER 2: THE COMMUNITY ASSESSMENT: AN OVERVIEW
• CHAPTER 3: DEVELOPING INITIATIVES: AN OVERVIEW
• CHAPTER 4: PLANNING FOR EVALUATION: PURPOSE AND PROCESSES
• CHAPTER 5: DESIGNING THE EVALUATION: PART 1: DESCRIBING THE PROGRAM OR POLICY
• CHAPTER 6: DESIGNING THE EVALUATION: PART 2A: PROCESS EVALUATION
• CHAPTER 7: DESIGNING THE EVALUATION: PART 2B: OUTCOME EVALUATION
• CHAPTER 8: COLLECTING THE DATA: QUANTITATIVE
• CHAPTER 9: COLLECTING THE DATA: QUALITATIVE
• CHAPTER 10: ANALYZING AND INTERPRETING QUANTITATIVE AND QUALITATIVE DATA: QUANTITATIVE (PART 1)
• CHAPTER 11: REPORTING EVALUATION FINDINGS
• CHAPTER 12: CASE STUDY: THE COMMUNITY ASSESSMENT
• CHAPTER 13: CASE STUDY: PROCESS EVALUATION
• APPENDIX A: MODEL AGREEMENT BETWEEN EVALUATION CONSULTANT AND TEAM MEMBERS
• APPENDIX B: MODEL PREAMBLE FOR ADULT INDIVIDUAL INTERVIEWS
• APPENDIX C: MODEL DEMOGRAPHIC SHEET
• APPENDIX D: MODEL FIELD NOTES REPORT
• APPENDIX E: MODEL INTERVIEW REFUSAL REPORT
• APPENDIX F: DATA COLLECTION TRAINING MANUAL TEMPLATE
• APPENDIX G: GUIDELINES FOR COMPLETING AN EVALUATION REPORT
• REFERENCES
• INDEX
• End User License Agreement