Evaluating Student Learning Outcomes in Counselor Education

Casey A. Barrio Minton, Donna M. Gibson, Carrie A. Wachter Morris

• Publisher: American Counseling Association
• Year: 2016
• ISBN:
  9781119292074
• Edition: 1
• Topic: Pedagogía
• Subtopic: Desarrollo profesional

This timely text describes the role of program evaluation in counselor education and provides step-by-step guidance for faculty seeking to develop comprehensive Student Learning Outcome (SLO) evaluation plans to meet accountability expectations. It serves as a blueprint for demystifying the SLO process and making the switch from an input-based measure of productivity that focuses on what counseling programs do, to an outcome-based approach that concentrates on the quality of learning through evidence-based assessment of students' knowledge and skills.

The first and second parts of the book lay the foundation for the SLO process and provide practical guidance for identifying and developing direct and indirect measures of student learning. Part III offers strategies for creating measures; collecting, managing, and reporting student data; and using data to ensure competence. In Part IV, counselor educators across the country offer hands-on application through a wide variety of SLO activities and rubrics linked to each of the curricular and specialty areas of the 2016 CACREP Standards.

*Requests for digital versions from ACA can be found on www.wiley.com.

*To purchase print copies, please visit the ACA website *Reproduction requests for material from books published by ACA should be directed to [email protected]

Part I
Context

If you are reading this book, you have likely found yourself navigating expectations within the third decade of accountability in higher education. Perhaps you are here because you are enthusiastic about the possibility of learning creative ways to find what your students know and can do. If so, that is wonderful, and we welcome you! We believe that student learning outcome (SLO) assessment can be surprisingly meaningful and rewarding if done well. More likely, you cracked this book open because you understand your need to show evidence of student learning for some reason or another. Maybe there is an expectation that you generate effectiveness data for promotion, tenure, or merit purposes. Or maybe you are helping to establish an SLO assessment plan to meet expectations of administrators, accrediting bodies, or key stakeholders. If so, you are in good company.

At its heart, SLO assessment involves going beyond traditional, input-based measures of productivity (e.g., how many courses and students we taught, what syllabi we provided, whether students were satisfied with their experiences) to outcome-based measures of effectiveness. These measures of effectiveness help us understand what students know and can do as a result of their educational endeavors (Council for Higher Education Accreditation, 2011; Urofsky, 2009).

Part I includes three chapters designed to lay a foundation for understanding the SLO assessment process. In Chapter 1, we provide a brief overview of the SLO movement. In Chapter 2, we outline the bigger picture of academic program evaluation. Chapter 3 includes practical strategies for identifying SLOs and mapping them to curricula in intentional ways. After you have reviewed these foundations, you will be ready to turn your attention to Part II, where we provide concrete and practical guidance for developing direct and indirect measures of student learning. Part III is focused on technical assistance for gathering, storing, reporting, making meaning of, and implementing changes on the basis of SLO data. Finally, Part IV includes several assignments and rubrics that your colleagues have developed to help you think of options for assessing student learning throughout core areas of the counselor education curriculum. Because we are counselor educators who are writing to an audience largely composed of counselor educators, we looked to the Council for Accreditation of Counseling and Related Educational Programs (CACREP) 2016 Standards to frame the presentation of our work and organization of the application; at the same time, we trust you will find the material relevant even if you are in a program that is not accredited or that is accredited by a specialty body other than CACREP.

Chapter 1
The SLO Movement

The emphasis on evaluating SLOs in higher education began with a focus on institutional accountability and overall undergraduate program improvement (New Leadership Alliance for Student Learning and Accountability, 2012). Over time, key players have shifted from state government to federal government to accrediting bodies. The now-famous Commission on the Future of Higher Education (also known as the Spellings Commission) admonished higher education institutions for not being more forthcoming about SLOs, which marked a pivotal point in the accountability movement in higher education. The Commission on the Future of Higher Education (2006) wrote the following:

We are disturbed by evidence that the quality of student learning at U.S. colleges and universities is inadequate and, in some cases, declining . . . Colleges and universities must become more transparent about cost, price, and student success outcomes, and must willingly share this information with students and families. (pp. 3–4)

Today, accreditation is the “primary vehicle for quality assurance in American higher education and the major driver of learning outcomes assessment” (Kuh & Ewell, 2010, p. 23), and faculty members across programs are becoming more involved in the process of assessing SLOs and using data to promote learning (Ewell, Paulson, & Kinzie, 2011; Kuh, Jankowski, Ikenberry, & Kinzie, 2014). Indeed, after examining assessment practices across countless institutions, Kuh et al. (2014) concluded the following:

U.S. higher education has turned a corner in the assessment of student learning. Carrying out this important work is no longer primarily an act of compliance but more appropriately and promisingly is driven by a balance of compliance and institutional desire to improve. (p. 5)

In this chapter, we provide a brief review of accountability and SLO assessment movements in higher education to provide context for the tasks that we discuss in the remainder of the book. First, we review key developments and stakeholders in this movement. Later, we provide a synopsis of the current state of SLO assessment procedures, challenges, and opportunities. We end the chapter with resources that you may find helpful for understanding the greater context of assessment and accountability in higher education.

A Brief History

In a reflection on major changes over the first 25 years of the SLO assessment movement, Ewell (2009) noted several major changes and strides nationwide. First, over time, faculty members have increased buy-in and improved perspectives regarding the legitimacy of the movement. Second, there is ongoing and increased government and policy concern regarding the importance of transparency and attention to learning outcomes in higher education. Third, although accountability expectations began with mandates from state governments, expectations related to SLO assessment procedures have transitioned to regional and specialty accrediting bodies operated for and by educators. Finally, the last two decades have brought rapid advances in technology and resources to facilitate the SLO assessment process.

While reflecting on trends in the first 25 years of the SLO movement, Ewell (2009) called attention to four main players:

1. State governments. Although funding models are changing, state governments traditionally held the purse strings for public higher education institutions. The accountability movement in higher education began with short-lived, state government mandates for standardized testing and reporting.
2. Federal government. Traditionally, the federal government had limited involvement in higher education, although it plays an important role in financial aid. The formation of the Commission on the Future of Higher Education marked an important turning point in the role of federal government in higher education accountability.
3. Accrediting bodies. Over time, regulation of accountability and expectations for SLO assessment has transitioned to regional accrediting bodies. Program or specialty accrediting bodies are now quite active in setting expectations for SLO assessment.
4. Customers, public opinion, and press. Ultimately, institutions of higher education are accountable for providing the products and processes that constituents need to thrive in a changing world.

With a focus on demonstrating that higher education programs are effective, accreditation standards of agencies and professional accrediting organizations provide the primary motivation to engage in evaluation. These happen on two primary levels: regional accreditation and program accreditation. The Council for Higher Education Accreditation, which is the accrediting body for accrediting bodies, stated in 2010 that accreditors must “require institutions or programs routinely to provide reliable information to the public on their performance, including student achievement as determined by the institution or program” (Policies and Procedures Standard 12.B.1, p. 5).

In the United States, seven regional, nonprofit, self-regulatory accrediting commissions are responsible for determining accreditation status of most institutions of higher education. These regional accrediting bodies are more alike than different when it comes to expectations for SLO assessment. Provezis (2010) found that all seven regional accreditors (a) were following the Council of Regional Accrediting Commissions' (2003) Principles for Good Practices related to SLO assessment, (b) expected institutions to design and assess SLOs, and (c) expected public disclosure of findings. All but one accrediting commission expected faculty involvement in the SLO assessment process. All accrediting commissions reported concerns regarding the state of SLO assessment, all but one accrediting commission provided support for institutions in the SLO assessment process, and none of the accrediting commissions prescribed how institutions were to go about the task.

Higher education personnel acknowledge that recognizing only the accomplishments of programs and students is not adequate for growth...