Program Evaluation in School Counseling

Improving Comprehensive and Developmental Programs

Michael S. Trevisan, John C. Carey

• Publisher: Routledge
• Year: 2020
• Print ISBN: 9781138346574
• eBook ISBN: 9780429793394
• Edition: 1
• Topic: Psychology
• Subtopic: Mental Health in Psychology
• Index: Psychology

Program Evaluation in School Counseling is the first book on program evaluation that looks to the field and literature of program evaluation and then relates methods, procedures, and practices back to the practice of school counseling.

Written by two accomplished authors who teamed up to build evaluation capacity among school and school-based counselors internationally, the book highlights their interdisciplinary work, covering many years and several continents. Based on the authors' model for teaching program evaluation and their research on school counselor competence in program evaluation, this concise, clear, and practical guide supports the continuing professional development of school counselors through training, workshops, and self-study. This book addresses the program evaluation knowledge, skills, and understandings that school-based counselors are expected to use in line with the CACREP 2016 Standards.

The book is intended as a companion text for university courses in research methods and/or in the organization and administration of counseling services. It is also appropriate as a self-study guide to help practicing school counselors develop expertise in evaluation.

1

Toward a Conceptual Understanding of Evaluation for School Counselors

Questions to Consider

•What is the definition of evaluation?

•What are the purposes for evaluation?

•What is the connection between evaluation and the school counseling profession?

•What are the components to the school counseling evaluation framework?

•What are standards in program evaluation, and how can they be used?

•What is culturally responsive evaluation, and why is it important?

The Idea of Program Evaluation

This book is about the use of program evaluation by school counselors to help shape improvements of their program and services and demonstrate value to school personnel, families, and other stakeholders. When program evaluation is done well, improvement of the program and services is possible, informed by the information provided by the evaluation. Positive student outcomes as a result of the school counseling program are more likely. And school counselors can gain strong, sustained support for their program. In short, everyone benefits.

This book is unique because it makes connections between the discipline/profession of evaluation and the discipline/profession of school counseling. In fact, we are advocating an interdisciplinary approach to program evaluation that enhances the profession of school counseling and increases the relevance for the field of evaluation, particularly for school counselors.

Evaluation as a discipline and profession is relatively young, with perhaps little more than 50 years in existence. As a consequence, many of the concepts, approaches, essential characteristics, and definitions are still emerging. As evaluators confront new challenges and develop new ways of addressing these challenges, new approaches replace old ones. This competition for ideas about evaluation and the rapid pace of development of the discipline make this an exciting time to do evaluation work.

Michael Scriven, one of the most prominent thinkers in evaluation, maintains that evaluation is a trans-discipline (Scriven, 2008). That is, while maintaining disciplinary features, evaluation is applied in a variety of other fields, such as school counseling. As different fields incorporate evaluation into the repertoire of professional practice, different ideas are developed about what constitutes evaluation and for what purposes it should be used. However, each profession defines evaluation somewhat differently, which can prevent broad understanding of the essential characteristics and purposes for evaluation.

The main goal of this book is to bring together the best thinking about evaluation and integrate these ideas with current practices of the school counseling field. We believe that evaluation has much to offer the professional school counselor. The concepts and ideas presented in this book are designed to provide the basis for school counselors to effectively evaluate their program and services.

As a starting point, the Joint Committee on Standards in Educational Evaluation (JCSEE) provides a comprehensive definition of program evaluation that has shown utility across a variety of fields, disciplines, and evaluators. Program evaluation is

•The systematic investigation of the quality of programs, projects, subprograms, subprojects, and/or any of their components or elements together or singly.

•For purposes of decision making, judgments, conclusions, findings, new knowledge, organizational development, and capacity building in response to the needs of identified stakeholders.

•Leading to improvement and/or accountability in the users’ programs and systems.

•Ultimately contributing to organizational or social value (Yarbrough, Shulha, Hopson, & Caruthers, 2010; p. xxv).

This definition emphasizes that program evaluation is systematic. It is based on a clearly articulated evaluation plan that includes rigorous data collection protocols. In addition, program evaluation results must be used in some way. Evaluation information can be used to guide improvement of services, inform decisions about the merit or worth of a program, or provide accountability data to the general public. In the end, program evaluation should contribute something of value to society. It should lead to social betterment.

There is a lot packed into the aforementioned definition. Understanding what it all means for school counselors and learning how to accomplish its stated ends are the purposes of this book. One feature of the JCSEE Program Evaluation Standards important to note is that it was developed by a joint committee of representatives from a variety of disciplines and professions (Yarbrough et al., 2010). The American Counseling Association (ACA) has been a member of this joint committee for all three versions of the JCSEE Program Evaluation Standards. In short, a key counseling professional organization has recognized the importance of evaluation to the school counseling profession and provided input to develop evaluation standards that will meet the professional needs of school counselors. We will discuss the JCSEE Program Evaluation Standards in more detail later in this chapter.

Differences between Research and Evaluation

Further understanding of what evaluation is and is not, found in the literature, is to compare and contrast evaluation with social science research (e.g., Mathison, 2008; Walser & Trevisan, 2019). In many disciplines, evaluation is often confused with research. For school counseling students, this confusion is further compounded because evaluation is often taught in research methods courses. As a consequence, students can come away with the idea that evaluation is another form of research. Further, some authors argue that evaluation is applied research. That is, evaluation is simply the use of social science research methods in a particular context, (e.g., the evaluation of a school counseling program, intervention, or activity). After all, evaluation uses research methods to collect data that answer evaluation questions. So, it “looks and feels” like research.

However, the idea of program evaluation being simply another form of research falls short of an accurate and nuanced understanding of program evaluation, as it doesn’t capture the many important features that are different from research. Most anyone deeply involved in evaluation today knows that stakeholders play an essential part and should have an integral role in developing some or all aspects of the evaluation. Research, applied or basic, is und...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Contents
• Preface
• Authors
• 1. Toward a Conceptual Understanding of Evaluation for School Counselors
• 2. Evaluation and the School Counseling Program and Profession
• 3. Involving Stakeholders
• 4. Developing Theories of Action and Logic Models
• 5. Developing School Counseling Program Evaluation Questions
• 6. The Evaluation Design and Methods
• 7. Data Analysis for Quantitative Data
• 8. Data Analysis for Qualitative Data
• 9. Reporting Evaluation Results and Using Findings
• 10. Evaluation Knowledge Skills and Practice
• Glossary
• Index