High Stakes Testing

New Challenges and Opportunities for School Psychology

Louis J. Kruger, David Shriberg, Louis J. Kruger, David Shriberg

• Publisher: Routledge
• Year: 2018
• Print ISBN: 9780789035219
• eBook ISBN: 9781136864506
• Topic: Education
• Subtopic: Education General

Meet the challenges of high stakes testing in the practice of school psychology School psychologists can be a positive influence on how students, teachers, parents, schools, and communities cope with the challenges and opportunities associated with high stakes testing. Unfortunately, there has been a significant lack of literature to guide school psychologists and related school-based practitioners on this topic. High Stakes Testing: New Challenges and Opportunities for School Psychology is a timely groundbreaking book that provides useful and thought-provoking information to help psychologists meet the challenges of high stakes testing and create new roles for themselves in helping children succeed. This book discusses practical ways to help provide academic support to facilitate student success on high stakes tests, reduce the impact of stress associated with high stakes testing, assess the data from the tests to improve programs, and take a leadership role in the appropriate use of the tests. The No Child Left Behind Act of 2001(NCLB) and its accountability provisions has helped create and sustain a climate where student performance on state-created achievement tests often has high stakes implications for students, families, and schools. High Stakes Testing: New Challenges and Opportunities for School Psychology provides important background information about high stakes testing, including the legal, historical, and political context of high stakes testing, pertinent psychometrics, and a review of research on academic and non-academic outcomes as it relates to high stakes testing. Using this information as a foundation, the book then identifies new roles and opportunities for school psychologists with respect to high stakes testing. This book is comprehensively referenced. Topics in High Stakes Testing: New Challenges and Opportunities for School Psychology include:

• advocating for the appropriate use of state-wide assessments
• the influence of item response theory (IRT) on the development of high stakes tests
• whether the accountability system of NCLB is truly improving student's learning
• the impact of high stakes tests on classroom instruction and student motivation
• strategies for helping students succeed on high stakes tests
• available resources to cope with the stress of high stakes testing
• and more

High Stakes Testing: New Challenges and Opportunities for School Psychology is a thought-provoking, horizon-expanding resource for school psychologists, public school educators, administrators, school counselors, curriculum coordinators, and special education teachers involved in organizing, administering, and preparing students to take high stakes tests.

Section 1:
The Challenges, Problems, and Dilemmas Associated with High Stakes Testing

Large-Scale Assessment, Rationality, and Scientific Management: The Case of No Child Left Behind

Andrew T. Roach Georgia State University Jennifer L. Frank Vanderbilt University

SUMMARY. This article examines the ways in which NCLB and the movement towards large-scale assessment systems are based on Weber’s concept of formal rationality and tradition of scientific management. Building on these ideas, the authors use Ritzer’s McDonaldization thesis to examine some of the core features of large-scale assessment and accountability systems. According to Ritzer, McDonaldized systems and routines are characterized by four central features: (a) a pursuit of efficiency; (b) emphasis on calculability or quantification of outcomes; (c) predictability and uniformity; and (d) control through nonhuman technologies. Strengths and shortcomings of each of these features for schools and educators are discussed. The article concludes with ideas and strategies for school psychologists interested in maximizing the benefits and minimizing the negative outcomes of large-scale assessment and accountability systems. doi:10.1300/J370v23n02_02 [Article copies available for a fee from The Haworth Document Delivery Service: 1-800-HAWORTH. E-mail address: <[email protected]> Website: <http://www.HaworthPress.com> © 2007 by The Haworth Press, Inc. All rights reserved.]

Keywords.

Large scale assessment systems, No Child Left Behind, accountability, scientific management, rationality

Henry Ford created a world-class company, a leader in its industry. More important, Ford would not have survived the competition had it not been for an emphasis on results. We must view education the same way. Good schools do operate like a business. They care about outcomes, routinely assess quality, and measure the needs of the children they serve.

–U.S. Secretary of Education Rod Paige, October 2003

Although the No Child Left Behind Act of 2001 (NCLB) did not introduce the concept of large-scale assessment systems to the nation’s schools, teachers, and students, it has highlighted the centrality of assessment systems as a policy tool in state and federal educational reform efforts. The current legislative embrace of large-scale assessment systems is based on a theory of action that assumes increased information about student achievement coupled with salient incentives for increased performance, and corresponding punishments for lack of improvement, will motivate educators and produce improved student outcomes (Baker & Linn, 2002). Researchers have dubbed this the “new accountability” and have outlined the following additional characteristics of these systems: (a) the use of student achievement data as an indicator of system and educator functioning; (b) public reporting of student performance data; and (c) utilization of the school as the unit of analysis for change efforts (Elmore, Abelman, & Fuhrman, 1996; O’Day, 2002). Large-scale assessment and accountability systems assume the existence of hierarchical, linear relationships between parts of the educational system (e.g., the US Department of Education, state and local educational agencies, and school-based practitioners). From this structural policy design and traditional educational planning perspective, large-scale assessment programs have a direct influence on the educational system and its agents (e.g., school administrators, teachers, and students).

In many ways, the foundations for NCLB and accompanying large-scale assessment systems can be found in the work of Max Weber and Frederick Taylor. Weber (1864-1920) was a German economist and sociologist whose work dealt with formal rationality and its effect on society, organizations, and individuals. He suggested the movement towards formal rationality results in the development of practices and interactions intended to facilitate efficiency or calculation rather than to promote tradition, morality, or aesthetics (Lippman & Aldrich, 2003). Moreover, Weber conceptualized bureaucracy as the organizational form that maximized the influence of rationality on individual and group behaviors (Bolman & Deal, 1997). With its emphasis on structure and delegation, rules and regulations, and hierarchical power relations, Weber believed bureaucracy allowed organizations to complete a large number of tasks in a highly efficient and predictable manner. In addition, Weber indicated bureaucratic organizations generally embrace quantification of tasks because it allows for easier operationalization and measurement of success and productivity (Ritzer, 2000, p. 23).

A contemporary of Weber’s, Frederick Winslow Taylor (1856-1915) was an American engineer who undertook a series of time-and-motion studies to determine the “one best way” of completing various manufacturing tasks. Taylor is associated with the concept of scientific management, which aims to increase performance and results by making work more rational and workers more efficient (Evans, 1996). As conceptualized by Taylor, scientific management emphasized (a) discovering efficient and effective ways of working via the use of scientific methods; (b) identifying and recruiting the best, most skillful workers to complete tasks; (c) providing training and professional development to improve the efficiency of these workers; and (d) closely monitoring workers’ attainment of clearly identified goals and standards (Evans, 1996; Morgan, 1986; Ritzer, 2000).

NCLB is not the first attempt to apply Weber and Taylor’s work to education. At the beginning of the twentieth century, administrative progressives (including Ellwood Cubberley and Edward Thorndike) posited that there were “scientific” solutions that could rectify the inefficiencies in the American educational system. Using Taylor’s principles of scientific management, these educational leaders reorganized schools to place students in classes by age and ability, hired professional administrators to oversee the work of teachers and schools, constructed tests to monitor student mastery of academic and vocational skills, and used the resulting test scores to compare schools and districts to one another (Cuban, 2004).

In his work The McDonaldization of Society, Ritzer (2000) indicated we have entered an era in which the majority of the organizations and systems have embraced and enacted the basic tenets of rationality and scientific management. He identifies a familiar modern organization– McDonald’s–whose structures and practices illustrate and exemplify Weber and Taylor’s ideas in action. Ritzer suggests that McDonald’s (and the “McDonaldization” of other modern organizations) is not a new or novel phenomenon, but the product of rationalization processes that have been occurring throughout the last century (influencing educational, governmental, and commercial organizations). McDonaldized organizations have four common features or purposes:

• Calculability, or an emphasis on the quantitative aspects of products and services offered;
• Efficiency, or the optimum method of getting from one point to another;
• Predictability, the assurance that products and services will be the same over time in all locales; and
• Control over people who enter the organizations through non-human technology (Lippmann & Altman, 2003).

Because most readers are undoubtedly familiar with McDonald’s and other McDonaldized organizations, this article will use Ritzer’s four components of McDonaldizaton as a framework for considering the influence of formal rationality and scientific management in the development of NCLB and large-scale assessment systems.

The no Child Left Behind Act of 2001

NCLB is a revision and expansion of the Elementary and Secondary Education Act (ESEA). This act is the largest federal education funding program in U.S. history, and has provided a new array of requirements, incentives, resources, and challenges for states. In enacting NCLB, the federal government took on a broader and stronger role in education than ever before. The Act builds on earlier school reform legislation and the standards-based education movement by establishing challenging academic standards, but it is far more than a simple revision of existing law. NCLB marks a significant change in federal and state responsibilities in education. The objectives of NCLB are far reaching and ambitious: to ensure that all children become “proficient” in reading, mathematics, and science and to close the achievement gap.

In May 2006, four years after NCLB was signed into law by President Bush, the US Department of Education released a document entitled No Child Left Behind Is Working that reviewed student performance on standardized test programs and indicated that “because of NCLB’s accountability provisions, schools and parents are getting the information and help they need to focus attention and resources on the children who need it most–and it’s working” (USDOE, 2006). Using Ritzer’s McDonaldization concept as a framework for considering NCLB and some of its provisions may provide us with (a) an understanding of the ways in which NCLB “works” and (b) insights into how school psychologists can practice within the constraints of this policy to facilitate improved outcomes for all children.

Calculability: Nclb's Embrace of Large-Scale Assessment

Ritzer suggests that calculability–the emphasis on quantitative aspects of work undertaken and products completed–is the linchpin that supports all the other aspects of McDonaldization. For example, a quantitative index of production or results makes it easier to monitor and evaluate the efficiency of workers (or educators and students). Moreover, when quantity becomes a substitute or proxy for quality, highly predictable and uniform modes of work are more likely to be promoted and embraced, resulting in greater predictability and increased control of worker behavior (Ritzer, 2000).

Under NCLB, the results of states’ large-scale assessment systems are used to provide a quantitative index of educational outcomes or productivity. Although previous versions of ESEA required states to test students in at least one elementary, middle school or junior high school, and high school grade annually (e.g., grades 4, 8, and 10); NCLB is more expansive, and explicit, in defining assessment and accountability than previous legislation. Among the most important changes is the mandate for annual testing in reading and mathematics of every student in grades 3-8 and at least one high school grade. In addition, annual science assessment in at least one grade in elementary, middle school, and high school is to be initiated by the 2007-2008 school year.

NCLB also requires states to identify schools not making Adequate Yearly Progress (AYP), and specifies a series of consequences for schools failing to meet AYP for two or more consecutive years. AYP is defined as progress toward meeting the goal of 100% of all children in a state meeting state proficiency standards by 2014. At a most basic level, NCLB requires states to develop assessments closely aligned with state content standards. Although local school or districts may use non-standardized testing components to address NCLB requirements, these forms of assessment may not meet NCLB’s technical quality and comparability requirements. Currently, NCLB requires state assessments to be constructed in such a way that results can be reported in terms of percentages of students in at least three performance categories, and that results are able to be disaggregated by subgroups based on gender, race/ethnicity, poverty level, English-language proficiency, and disability status.

Other indicators (e.g., attendance) may also be used to track progress, but achievement as measured by standardized tests is considered the most salient outcome.

In addition, schools are required to track (and attain) AYP for identifiable subgroups, including groups defined by race/ethnicity, poverty, gender, disability, and English proficiency. States also must ensure that their assessment program achieves at least 95% participation for the students in each of these subgroups.

The Promise of Calculability

Ritzer (2000) suggests the emphasis on calculability brings with it many advantages, especially the ability to obtain data on a large number of individuals and services at relatively little cost. This cost-effectiveness is one of the central appeals of standardized testing programs. In comparison to more personalized classroom-based methods of assessment (e.g., portfolio, observation, performance assessments), the multiple-choice and constructed-response standardized tests used in most states’ assessment programs provide a wealth of information on student performance with relatively small financial outlays needed to support implementation, scoring, and reporting. Moreover, although many educators and other stakeholders assert that testing programs take an inordinate amount of instructional time, students tested annually spend less than 1% of their school time taking tests (Elliott, Braden, & White, 2000). Although the time dedicated to test preparation is likely to vary across states, schools, and teachers, the actual instructional time lost to mandated assessment requirements is arguably quite small.

In addition, NCLB’s emphasis on annual assessment coupled with the inclusion of students with disabilities and English language learners in assessment systems means that there is more available information on overall student achievement than at any point in our nation’s history. Historically, there has been low partici...

Table of contents

• Cover
• Half Title
• Title
• Copyright
• Contents
• About the Contributors
• Introduction and Overview: High Stakes Testing
• SECTION 1: THE CHALLENGES, PROBLEMS, AND DILEMMAS ASSOCIATED WITH HIGH STAKES TESTING
• Large-Scale Assessment, Rationality, and Scientific Management: The Case of No Child Left Behind
• Measurement Issues in High Stakes Testing: Validity and Reliability
• High-Stakes Testing: Does It Increase Achievement?
• The Unintended Outcomes of High-Stakes Testing
• SECTION 2: NEW ROLES FOR SCHOOL PSYCHOLOGISTS
• Providing Academic Support for Teachers and Students in High Stakes Learning Environments
• Coping with the Stress of High Stakes Testing
• Using Data from High-Stakes Testing in Program Planning and Evaluation
• The School Psychologist as Leader and Change Agent in a High-Stakes Era
• Index