The International Handbook of School Effectiveness Research

David Reynolds, Charles Teddlie

• Publisher: Routledge
• Year: 2002
• eBook ISBN: 9781135715274
• Topic: Education
• Subtopic: Education General

What constitutes quality schooling? What are the implications for educational practice and administration? The text looks at these questions and examines international research evidence and reform initiatives with particular emphasis on North America, UK, Australasia and the Third World. It offers a synopsis of the Third World School Effects Research (SER). The authors claim that the challenges now facing educational leaders is to find a balance between SER and the other school movements and to ask more demanding questions of our educational systems.

Section 1
The Historical and
Intellectual Foundations of
School Effectiveness Research

1 An Introduction to School Effectiveness Research

David Reynolds and Charles Teddlie, with Bert Creemers, Jaap Scheerens and Tony Townsend

Introduction

School effectiveness research (SER) has emerged from virtual total obscurity to a now central position in the educational discourse that is taking place within many countries. From the position 30 years ago that ‘schools make no difference’ that was assumed to be the conclusions of the Coleman et al. (1966) and Jencks et al. (1971) studies, there is now a widespread assumption internationally that schools affect children’s development, that there are observable regularities in the schools that ‘add value’ and that the task of educational policies is to improve all schools in general, and the more ineffective schools in particular, by transmission of this knowledge to educational practitioners.

Overall, there have been three major strands of school effectiveness research (SER):

• School Effects Research—studies of the scientific properties of school effects evolving from input-output studies to current research utilizing multilevel models;
• Effective Schools Research—research concerned with the processes of effective schooling, evolving from case studies of outlier schools through to contemporary studies merging qualitative and quantitative methods in the simultaneous study of classrooms and schools;
• School Improvement Research—examining the processes whereby schools can be changed utilizing increasingly sophisticated models that have gone beyond simple applications of school effectiveness knowledge to sophisticated ‘multiple lever’ models.

In this chapter, we aim to outline the historical development of these three areas of the field over the past 30 years, looking in detail at the developments in the United States where the field originated and then moving on to look at the United Kingdom, the Netherlands and Australia, where growth in SER began later but has been particularly rapid. We then attempt to conduct an analysis of how the various phases of development of the field cross nationally are linked with the changing perceptions of the educational system that have been clearly visible within advanced industrial societies over the various historic ‘epochs’ that have been in evidence over these last 30 years. We conclude by attempting to conduct an ‘intellectual audit’ of the existing school effectiveness knowledge base and its various strengths and weaknesses withinvarious countries, as a prelude to outlining in Chapter 2 how we have structured our task of reviewing all the world’s literature by topic areas across countries.

SER in The United States

Our review of the United States is based upon numerous summaries of the literature, many of which concentrated on research done during the first 20 years of school effectiveness research (1966–85). A partial list of reviews of the voluminous literature on school effects in the USA during this period would include the following: Anderson, 1982; Austin, 1989; Averch et al., 1971; Bidwell and Kasarda, 1980; Borger et al., 1985; Bossert, 1988; Bridge et al., 1979; Clark, D., et al., 1984; Cohen, M., 1982; Cuban, 1983; Dougherty, 1981; Geske and Teddlie, 1990; Glasman and Biniaminov, 1981; Good and Brophy, 1986; Good and Weinstein, 1986; Hanushek, 1986; Levine and Lezotte, 1990; Madaus et al., 1980; Purkey and Smith, 1983; Ralph and Fennessey, 1983; Rosenholtz, 1985; Rowan et al., 1983; Sirotnik, 1985; Stringfield and Herman, 1995, 1996; Sweeney, 1982; Teddlie and Stringfield, 1993.

Figure 1.1 presents a visual representation of four overlapping stages that SER has been through in the USA:

• Stage 1, from the mid-1960s and up until the early 1970s, involved the initial input-output paradigm, which focused upon the potential impact of school human and physical resources upon outcomes;
• Stage 2, from the early to the late 1970s, saw the beginning of what were commonly called the ‘effective schools’ studies, which added a wide range of school processes for study and additionally looked at a much wider range of school outcomes than the input-output studies in Stage 1;
• Stage 3, from the late 1970s through the mid-1980s, saw the focus of SER shift towards the attempted incorporation of the effective schools ‘correlates’ into schools through the generation of various school improvement programmes;
• Stage 4, from the late 1980s to the present day, has involved the introduction of context factors and of more sophisticated methodologies, which have had an enhancing effect upon the quality of all three strands of SER (school effects research, effective schools research, school improvement research).

We now proceed to analyse these four stages in detail.

Stage 1: The Original Input-Output Paradigm

Stage 1 was the period in which economically driven input-output studies predominated. These studies (designated as Type 1A studies in Figure 1.1) focused on inputs such as school resource variables (e.g. per pupil expenditure) and student background characteristics (variants of student socio-economic status or SES) to predict school outputs. In these studies, school outcomes were limited to student achievement on standardized tests. The results of these studies in the USA (Coleman et al., 1966; Jencks et al., 1972) indicated that differences in children’s achievement were more strongly associated with societally determined family SES than with potentially malleable school-based resource variables.

Figure 1.1 Stages in the evolution of SER in the USA

The Coleman et al. (1966) study utilized regression analyses that mixed levels of data analysis (school, individual pupil) and concluded that ‘schools bring little influence to bear on a child’s achievement that is independent of his background and general social context’ (p.325). Many of the Coleman school factors were related to school resources (e.g. per pupil expenditure, school facilities, number of books in the library), which were not very strongly related to student achievement. Nevertheless, 5–9 per cent of the total variance in individual student achievement was uniquely accounted for by school factors (Coleman et al., 1966). (See Chapter 3 for a more extensive description of the results from the Coleman Report). Daly noted:

The Coleman et al. (1966) survey estimate of a figure of 9 per cent of variance in an achievement measure attributable to American schools has been something of a bench mark, despite the critical reaction.

(Daly, 1995a, p.306)

While there were efforts to refute the Coleman results (e.g. Mayeske et al., 1972; Mclntosh, 1968; Mosteller and Moynihan, 1972) and methodological flaws were found in the report, the major findings are now widely accepted by the educational research community. For example, the Mayeske et al. (1972) re-analysis indicated that 37 per cent of the variance was between schools, but they found that much of that variance was common to both student background variables and school variables. The issue of multicollinearity between school and family background variables has plagued much of the research on school effects, and is a topic we return to frequently later.

In addition to the Coleman and Jencks studies, there were several other studies conducted during this time within a sociological framework known as the ‘status-attainment literature’ (e.g. Hauser, 1971; Hauser et al., 1976). The Hauser studies, conducted in high schools in the USA, concluded that variance between schools was within the 15–30 per cent range and was due to mean SES differences, not to characteristics associated with effective schooling. Hauser, R., et al. (1976) estimated that schools accounted for only 1–2 per cent of the total variance in student achievement after the impact of the aggregate SES characteristics of student bodies was controlled for statistically.

As noted by many reviewers (e.g. Averch et al., 1971; Brookover et al., 1979; Miller, 1983), these early economic and sociological studies of school effects did not include adequate measures of school social psychological climate and other classroom/ school process variables, and their exclusion contributed to the underestimation of the magnitude of school effects. Averch et al. (1971) reviewed many of these early studies and concluded that student SES background variables were the only factors consistently linked with student achievement. This conclusion of the Averch review was qualified, however, by the limitations of the extant literature that they noted, including: the fact that very few studies had actually measured processes (behavioural and attitudinal) within schools, that the operational definitions of those processes were very crude, and that standardized achievement was the only outcome measure (Miller, 1983).

Stage 2: The Introduction of Process Variables and Additional Outcome Variables into SER

The next stage of development in SER in the USA involved studies that were conducted to dispute the results of Coleman and Jencks. Some researchers studied schools that were doing exceptional jobs of educating students from very poor SES backgrounds and sought to describe the processes ongoing in those schools. These studies are designated as Type 2A links in Figure 1.1. In these Type 2A studies, traditional input variables are subsumed under process variables (e.g. available library resources become part of the processes associated with the library). These studies also expanded the definition of the outputs of schools to include other products, such as were measured by attitudinal and behavioural indicators.

The earlier studies in this period were focused in urban, low-SES, elementary schools because researchers believed that success stories in these environments would dispel the belief that schools made little or no difference. In a classic study from this period, Weber (1971) conducted extensive case studies of four low-SES inner-city schools characterized by high achievement at the third grade level. His research emphasized the importance of the actual processes ongoing at schools (e.g. strong leadership, high expectations, good atmosphere, and a careful evaluation of pupil progress), while the earlier studies by Coleman and Jencks had focused on only static, historical school resource characteristics.

Several methodological advances occurred in the USA literature during the decade of the 1970s, such as the inclusion of more sensitive measures of classroom input, the development of social psychological scales to measure school processes, and the utilization of more sensitive outcome measures. These advances were later incorporated in the more sophisticated SER of both the USA and of other countries in the 1990s. The inclusion of more sensitive measures of classroom input in SER involved the association of student-level data with the specific teachers who taught the students. This methodological advance was important for two reasons: it emphasized input from the classroom (teacher) level, as well the school level; and it associated student-level output variables with student-level input variables, rather than school-level input variables.

The Bidwell and Kasarda (1980) review of school effects concluded that some positive evidence was accumulating in studies that measured ‘school attributes at a level of aggregation close to the places where the work of schooling occurs’ (p.403). Their review highlighted the work of Murnane (1975) and Summers and Wolfe (1977), both of which had painstakingly put together datasets in which the inputs of specific teachers were associated with the particular students that they had taught. (The Bidwell and Kasarda review also concluded that studies including information on curricular track (e.g. Alexander, K., and McDill, 1976; Alexander, K., et al., 1978; Heyns, 1974) were providing valuable information on the potential impact of schooling on achievement.)

The research of Summers and Wolfe (1977), Murnane (1975), and others (e.g. Armor et al., 1976; Winkler, 1975) demonstrated that certain characteristics of classroom teachers were significantly related to the achievement of their students. The Summers and Wolfe (1977) study utilized student-level school inputs, including characteristics of the specific teachers who taught each student. The researchers were ableto explain around 25 per cent of the student level variance in gain scores using a mixture of teacher and school level inputs. Summers and Wolfe did not report the cumulative effect for school factors as opposed to SES factors, but the quality of college the teachers attended was a significant predictor of students’ learning rate (as it was in Winkler, 1975).

Murnane’s (1975) research indicated that information on classroom and school assignments increased the amount of predicted variance in student achievement by 15 per cent in regression models in which student background and prior achievement had been entered first. Principals’ evaluations of teachers was also a significant predictor in this study (as it was in Armor et al., 1976) (see Chapter 3 for more details regarding both the Summers and Wolfe and the Murnane studies).

Later reviews by Hanushek (1981, 1986) indicated that teacher variables that are tied to school expenditures (e.g. teacher-student ratio, teacher education, teacher experience, teacher salary) demonstrated no consistent effect on student achievement. Thus, the Coleman Report and re-analyses (using school financial and physical resources as inputs) and the Hanushek reviews (assessing teacher variables associated with expenditures) produced no positive relationships with student achievement.

On the other hand, qualities associated with human resources (e.g. student sense of control of their environment, principals’ evaluations of teachers, quality of teachers’ education, teachers’ high expectations for students) demonstrated significantly positive relationships to achievement in studies conducted during this period (e.g. Murnane, 1975; Link and Ratledge, 1979; Summers and Wolfe, 1977; Winkler, 1975). Other studies conducted in the USA during this period indicated the importance of peer groups on student achievement above and beyond the students’ own SES background (e.g. Brookover et al., 1979; Hanushek, 1972; Henderson et al., 1978; Summers and Wolfe, 1977; Winkler, 1975).

These results led Murnane (1981) to conclude that:

The primary resources that are consistently related to student achievement are teachers and other students. Other resources affect student achievement primarily through their impact on the attitudes and behaviours of teachers and students.

(Murnane, 1981, p.33)

The measures of teacher behaviours and attitudes utilized in school effects studies have evolved considerably from the archived data that Summers and Wolfe (1977) and Murnane (1975) used. Current measures of teacher inputs include direct observations of effective classroom teaching behaviours which were identified through the teacher effectiveness literature (e.g. Brophy and Good, 1986; Gage and Needels, 1989; Rosenshine, 1983).

Another methodological advance from the 1970s concerned the development of social psychological scales that could better measure the educational processes ongoing at the school and class levels. These scales are more direct measures of student, teacher, and principal attitudes toward schooling than the archived data used in the Murnane (1975) and Summers and Wolfe (1977) studies.

As noted above, a major criticism of the early school effects literature was that school/classroom processes were not adequately measured, and that this contributed to school level variance being attributed to family background variables rather than ed...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• List of Figures
• List of Tables
• Preface
• Acknowledgments
• Section 1: The Historical and Intellectual Foundations of School Effectiveness Research
• Section 2: The Knowledge Base of School Effectiveness Research
• Section 3: The Cutting Edge Issues of School Effectiveness Research
• Section 4: The Future of School Effectiveness Research
• Notes On Contributors
• References