Student Growth Measures in Policy and Practice

Intended and Unintended Consequences of High-Stakes Teacher Evaluations

Kimberly Kappler Hewitt, Audrey Amrein-Beardsley, Kimberly Kappler Hewitt, Audrey Amrein-Beardsley

• Publisher: Palgrave Macmillan
• Year: 2016
• Print ISBN:
  9781137539007
• eBook ISBN:
  9781137539014
• Topic: Education
• Subtopic: Education General

This book examines the intersection of policy and practice in the use of student growth measures (SGMs) for high-stakes purposes as per such educator evaluation systems.  The book also focuses on examinations of educators' perceptions of and reactions to the use of SGMs; ethical implications pertaining to the use of SGMs; contextual challenges when implementing SGMs; and legal implications of SGM use. The use of student test score data has been the cornerstone of the recent transfiguration of educator evaluation systems in forty-two states and the District of Columbia. Three leading voices on SGMs—Sean Corcoran, Henry Braun, and David Berliner—also serve as section and concluding commentators.  

© The Author(s) 2016

Kimberly Kappler Hewitt and Audrey Amrein-Beardsley (eds.)Student Growth Measures in Policy and Practice10.1057/978-1-137-53901-4_1

Begin Abstract

1. Introduction: The Use of Student Growth Measures for Educator Accountability at the Intersection of Policy and Practice

Kimberly Kappler Hewitt¹ and Audrey Amrein-Beardsley²

(1)

University of North Carolina at Greensboro, Greensboro, NC, USA

(2)

Arizona State University, Tempe, AZ, USA

End Abstract

Introduction

In the early 1990s, William Sanders, who was then a researcher and adjunct statistics professor at the University of Tennessee, Knoxville, introduced his value-added methodology, a model appropriated from the fields of economics and agriculture and based on the theoretical work of C.R. Henderson and David A. Harville, to the Tennessee legislature after piloting it in several school districts in Tennessee. Tennessee legislators were smitten and included what became (and what is still currently known as) the Tennessee Value-Added Assessment System (TVAAS) in the state of Tennessee’s landmark Education Improvement Act of 1992 (Ceperley & Reel, 1997).

Originally, TVAAS served to hold schools and districts accountable for student growth. it was not until the fall of 1996 that the state began providing teacher-level, value-added reports to teachers and their administrators, two years later the state began using teacher value-added data as part of teachers’ evaluation (Tennessee Department of Education, 2011). ¹ While in the past, value-added data accounted for up to 8% of a teacher’s evaluation in Tennessee (The Center for Greater Philadelphia, 2004), the legislature increased the value-added component to 35–50% of a teacher’s evaluation in order to be competitive for Race to the Top (RttT) funds from the US Department of Education (Eckert & Dabrowski, 2010). Tennessee’s RttT plan, dubbed First to the Top, was awarded $500 million in the second round of RttT funding.

Today, 42 states and the District of Columbia have similar teacher evaluation systems in place that incorporate measures of teacher effectiveness for consequential purposes (Doherty, Jacobs, & National Council on Teacher Quality, 2013). Likewise, these states are also “now grappling with the practical realities” (p. i) of these policies. While Tennessee was the first state to increase emphasis on teacher accountability, as well use student growth measures (SGMs) to evaluate teachers, the other states followed with the adoption of a variety of SGMs.

SGMs involve the quantification of student progress over time, using student test scores and other student test and demographic data aggregated at the teacher level to determine effectiveness, or more descriptively teacher effects. SGMs come in numerous varieties, including value-added models (VAMs), Student Growth Percentiles (SGPs), and Student Learning Objectives (SLOs), which are discussed later in this chapter. This chapter begins with a discussion of the growing interest in teacher evaluation as a lever for reform, after which we examine the current state of affairs, as well as the theory of action (ToA) underpinning more contemporary teacher evaluation policies largely based on SGMs. This chapter concludes with an overview of the rest of the volume as well as an overview of our perspectives as the volume’s editors.

Growing Interesting in Teacher Evaluation

The current flurry of interest in teacher accountability is a function of the confluence of several occurrences, including research on factors most influential to student performance, increasing attention to limitations of traditional teacher evaluation, and neoliberal influences, as discussed in the following sections.

Teachers Matter Most

An important element in the seismic shift to using SGMs for teacher accountability is research that points to teachers as the single most influential school-related factor in student learning, with teachers also having lasting impacts on students into adulthood (Chetty, Friedman, & Rockoff, 2011, 2013; Rivkin, Hanushek, & Kain, 2005; Wright, Horn, & Sanders, 1997). Paired with these findings are researchers who have found high variability in teacher effectiveness within schools (Rivkin et al., 2005). Looking across these studies, it is clear that teachers vary in their effectiveness, and this effectiveness has a profound impact on student learning. Thus, teachers have been increasingly seen as the most promising lever for improving student learning and achievement (McCaffrey, Lockwood, Koretz, & Hamilton, 2003), and this has added impetus to the aforementioned educational policies pushing for teacher-level accountability.

Limitations of Traditional Teacher Evaluation

Another key element underpinning the increase of teacher accountability is the recognition that many traditional teacher evaluation systems were lacking in the ability to distinguish strong from weak teachers (e.g., Weisberg, Sexton, Mulhern, & Keeling, 2009). In their “Widget Effect” study, for example, Weisberg et al. (2009) found that over a three-year period, only 10 % of failing schools in Denver assigned at least one rating of unsatisfactory to a teacher with tenure (p. 12). Thus, as teachers have increasingly been seen as the lever for improving student achievement, evaluation systems, and more broadly human capital management initiatives, have been increasingly seen as the fulcrum to educational reform.

To this point, authors of multiple studies have suggested that replacing ineffective teachers with effective teachers could have substantial impacts on student achievement. For example, Hanushek (2009) argues that replacing the bottom 6–10 % of teachers could increase student achievement by 0.5 standard deviations, bringing performance in the USA in line with that in Canada. Other researchers (e.g., Winters & Cowen, 2015), however, suggest that evaluation policy designs that incorporate SGMs would require the cutoff percentile for dismissal to be set quite high, up to the 27th percentile, for designs that involve dismissing a teacher after two consecutive ineffective ratings. Likewise, an adequate and available labor supply of teachers would be needed to ensure that student achievement increases as a function of removing and replacing teachers with those who are likely less experienced (Winters & Cowen, 2013).

The Neoliberal Influence

In addition to the influence of Sanders and research findings that teachers are profoundly important and vary substantially in their effectiveness, and that traditional teacher evaluation systems have been inadequate, the rise of teacher accountability is also being influenced by neoliberal ideals. Educational policy in the USA is strongly steeped in neoliberal ideals (Hursh, 2007), which emphasize economic competitiveness and promote accountability, high-stakes testing, and market competition. These ideals underpin assumptions underlying teacher accountability, one of which is that:

[T]o reform American’s public schools, we must treat educational systems as we would market-based corporations. Educational corporations produce knowledge, the quality of which can be manipulated by objectively measuring knowledge-based outcomes. (Holloway-Libell & Amrein-Beardsley, 2015, p. 4)

This neoliberal influence is omnipresent in policy action related to teacher accountability and is reflected in the ToA that underpins such policy.

Theory of Action

The ToA that underpins the role of SGMs in teacher accountability is illustrated in Fig. 1.1, as predicated upon the use of multiple measures in educator evaluations. A number of recent works have centered on the nature and purpose of educator evaluations, including books by Darling-Hammond (2013), Lavigne and Good (2014), Kelly (2011), Marzano and Toth (2013), Popham (2013), and Silverberg and Jungwirth (2014). The two main purposes of evaluation are to inform performance improvement and personnel decisions, including hiring, dismissal, tenure, and compensation. The aforementioned tomes promote the use of multi-measure evaluations, which generally include an observation- or practice-based component and an SGM component. Some also include other data, such as student response surveys, which are used in Austin Independent School District in Texas (Silverberg & Jungwirth, 2014), for example, and are increasingly used elsewhere (e.g., Bill & Melinda Gates Foundation, 2013).

Fig. 1.1

Theory of Action underpinning teacher accountability. This articulation of the ToA is informed by Amrein-Beardsley (2014), Harris and Herrington (2015), and Holloway-Libell and Amrein-Beardsley (2015)

According to the ToA, teacher evaluation systems that involve SGMs will primarily affect teacher quality by promoting the voluntary and involuntary exit (dismissal) of ineffective teachers and motivating remaining teachers to work harder and smarter. The result would be increased teacher quality, which will lead to increased student learning, as reflected in increased student achievement, which will elevate US education and increase the nation’s global competitiveness. The ToA has a prima facie logic to it, yet it assumes that a) SGMs can reliably and accurately distinguish effective from ineffective teachers, b) that these data, together, will be appropriately used to inform personnel (dismissing, hiring) decisions, c) that remaining teachers will be motivated to work harder and smarter in response to the data (as opposed to moving to other grade levels, schools, or leaving teaching altogether), and d) that changes in test scores as a function of these policies will reflect increases in student learning and—more importantly—increase the type of student learning that will make the USA more globally competitive. The legitimacy of the ToA will ultimately be determined in the years and decades to come as the effects of teacher accountability policies become clearer.

Current State of Affairs

Policy

Now, since the turn of the millennium, SGMs have been at the forefront of recent policy emphases surrounding teacher accountability and the transformation of teacher evaluation systems. For the reasons discussed earlier, the use of SGMs, in fact, is likely the most momentous change in educational accountability policy since the passage of No Child Left Behind (NCLB) in 2002, whereby all states were required by federal law to develop/implement large-scale, criterion-referenced, standardized tests now being used within many SGMs.

Incentivized by the federal government via the $4.5 billion in federal RttT funds, and NCLB waivers granted to states agreeing to hold educators accountable for student learning using SGMs, 42 states and the District of Columbia now have teacher evaluation systems in place that incorporate SGMs for consequential purposes (Doherty et al., 2013). However, this policy trend may be shifting. In late 2015, President Obama signed into law the Every Student Succeeds Act (ESSA), the seventh reauthorization of the Elementary and Secondary Education Act (ESEA) of 1965. ESSA now prohibits the federal government from dictating teacher evaluations and from requiring SGMs for teacher accountability. This new legislation disrupts prior federal policy trends promoting the use of SGMs. Thus, it will be left to states to determine the future of SGMs for teacher accountability.

Research

While there is a growing body of scholarship on the technical elements in the use of SGMs, there is, however, little on the intersection of policy and practice in the use of SGMs for high-stakes purposes, especially for teacher accountability purposes (Harris, 2011; Harris & Herrington, 2015; Jiang, Sporte, & Luppescu, 2015). More specifically, very little is known about the impact of such policies on the realities of those at their receiving ends, including, in particular, teachers, school leaders, and students (see also Collins, 2014; Harris & Herrington, 2015). This volume attempts to address this gap.

Existing research on the intersection of policy and practice. While the intersection of teacher accountability policy and practice is a nascent field, there have been some important studies, especially in the area of educators’ perceptions of pay-for-performance and evaluation systems that utilize SGMs. Springer et al. (2010), in a three-year study of pay-for-performance based on VAMs in Nashville, found that two-thirds of teachers involved in the study perceived that the VAM could not accurately discriminate between effective and ineffective teaching. Similarly, Amrein-Beardsley and Collins (2012), in a study of the high-stakes use of the SAS corporation’s Education Value-...

Table of contents

• Cover
• Frontmatter
• 1. Introduction: The Use of Student Growth Measures for Educator Accountability at the Intersection of Policy and Practice
• 1. Part I
• 2. Part II
• Backmatter