The Dynamics of Educational Effectiveness

A Contribution to Policy, Practice and Theory in Contemporary Schools

Bert Creemers, Leonidas Kyriakides

• Editore: Routledge
• Anno: 2007
• ISBN:
  9781134168385
• Edizione: 1
• Argomento: Education
• Categoria: Education General

This book brings together the current thinking and research of two major investigators in the field of educational effectiveness. After defining educational effectiveness, the authors analyse the various theories and strands of research within educational effectiveness, especially with respect to the comprehensive model developed by Creemers.

Written by one of the worlds leading experts in the field, this book will both elucidate our current understanding of educational effectiveness and carry the discipline forward by proposing profound changes to accepted views.

Part I
Background to educational effectiveness research

Chapter 1
Towards the development of the theoretical framework of educational effectiveness research

Introduction

This introductory chapter offers a definition of educational effectiveness. We also refer to theory and research in educational effectiveness and especially to the importance of establishing a new theoretical framework which, among other things, takes into account the new goals for education, the existence of differential effects of education upon students, and the dynamic relations between the multiple factors of effectiveness. It will be argued that the development of this framework will contribute to the improvement of research, evaluation and practice. Finally, the aims of the book and its structure will be outlined.

Stringfield (1994) defines effectiveness research as the process of differentiating existing ideas and methods of schooling along dimensions deemed to be of value. Educational effectiveness research (EER) does not attempt to invent new ideas or programmes; rather, it aims to concentrate on understanding the lessons to be drawn from existing practices. In this way, EER attempts to establish and test theories that explain why and how some schools and teachers are more effective than others. It is important to note here that three terms – school effectiveness, teacher effectiveness, and educational effectiveness – are used inconsistently in the literature but are interrelated. We are taking school effectiveness to mean the impact that school-wide factors, such as school policy for teaching, school climate, and the school’s perceived mission, have on students’ cognitive and affective performance. On the other hand, teacher effectiveness refers to the impact that classroom factors, such as teacher behaviour, teacher expectations, classroom organisation, and use of classroom resources, have on student performance.

Teddlie (1994) argued that most teacher effectiveness studies have been concerned only with processes that occur within the classrooms, to the exclusion of school-wide factors, whereas most school effectiveness studies have involved phenomena that occur throughout the school with little emphasis on particular teaching behaviours within individual classrooms. Only a few of them have looked at both school and classroom effectiveness simultaneously (e.g. Mortimore et al., 1988; Teddlie and Stringfield, 1993), but this weakness is being addressed in recent studies (e.g. de Jong et al., 2004; Kyriakides, 2005a; Reynolds et al., 2002; Opdenakker and Van Damme, 2000). This can be seen as a significant development in the field, since joint studies on school and teacher effectiveness revealed that neither level can be adequately studied without considering the other (Reynolds et al., 2002). In this context, we are using the term ‘educational effectiveness’ rather than the terms ‘teacher’ and/or ‘school effectiveness’ to emphasise the importance of conducting joint school and teacher effectiveness research which can help us identify the interactions between the school, classroom, and individual student levels, and their contributions to student performance. Finally, it is important to note that EER refers to the functioning of the system as a whole, and can therefore be used to develop models of effectiveness (e.g. Creemers, 1994; Scheerens, 1992; Stringfield and Slavin, 1992) that ultimately explain why educational systems perform differently.

The origins of EER stem from reactions to the work on equality of opportunity undertaken by Coleman et al. (1966) and Jencks et al. (1972). These two studies from two different disciplinary backgrounds – sociological and psychological, respectively – came to almost the same conclusion in relation to the amount of variance that can be explained by educational factors. After taking into consideration student background characteristics, such as ability and family background, not much variance in student achievement was left. This pessimistic feeling of not knowing what, if anything, education could contribute to society was also fed by the failure of large-scale educational compensatory programmes, such as ‘Head Start’ and ‘Follow Through’, both conducted in the United States, which were based on the idea that education in schools would compensate for initial differences between students. Similar results have been reported for the effects of compensatory programmes conducted in other countries (e.g. Driessen and Mulder, 1999; MacDonald, 1991; Schon, 1971; Taggart and Sammons, 1999).

Thus, the first two effectiveness studies, undertaken independently by Edmonds (1979) and Rutter et al. (1979) during the 1970s, were concerned with examining evidence and making an argument about the potential power of schooling to make a difference to students’ life chances. This was an optimistic point of view, because many studies published in that period showed that teachers, schools, and maybe even education in general did not make much of a difference at all. The early existence of two independent research projects in different countries asking similar questions and drawing, to a certain extent, on similar quantitative methodologies demonstrated the potential for establishing a scientific domain dealing with effectiveness in education (Kyriakides, 2006a). Thus, the publications by Brookover et al. (1979) and Rutter et al. (1979) were followed by numerous studies in different countries on school effectiveness (Teddlie and Reynolds, 2000). During the same period, based on expectations that effective knowledge can be used to improve education, studies on school improvement efforts were conducted (Edmonds, 1979), and contributed to the establishment of links between school effectiveness and school improvement research (Townsend et al., 1999).

In the past 25 years, EER has improved considerably by responding to the criticisms on research design, the sampling, and statistical techniques. Methodological advances have enabled more efficient estimates of teacher and school differences in student achievement to be obtained (Goldstein, 2003). Anyone now attempting to measure the effects of educational systems, schools, and teachers has already encountered two methodological imperatives: ‘collect longitudinal data’ and ‘pay attention to the multilevel organisational structure’ in which education occurs. This can be attributed to the fact that school and teacher effects are dynamic things that occur within multilevel, hierarchical organisational structures. Their dynamic character is essential both because school effects modify children’s growth and because the schools that produce them are changing (Kyriakides and Creemers, 2006a). Longitudinal data are, therefore, needed to measure the effect of schools and teachers on student achievement gains. Longitudinal data also enhance the validity of causal inferences in non-experimental research by providing a basis for assessing the direction of causation between two variables and by making possible some control over selection effects (Cook and Campbell, 1979). The multilevel, hierarchical character of school and teacher effects is crucial, because such effects occur only when policies and practices implemented at the school and/or teacher level affect learning measured at the student level. Incorporating multilevel effects into statistical analyses helps the researcher avoid a variety of errors of statistical inference, discussed in more detail in Chapter 2 (see also Aitkin and Longford, 1986; DeLeeuw and Kreft, 1986; Goldstein, 2003; Kyriakides and Charalambous, 2005; Raudenbush and Bryk, 1986). Finally, there is substantial agreement regarding appropriate methods for estimating school or teacher differences or effects and the kinds of data required for valid comparisons to be made (Teddlie et al., 2000; Gray et al., 1995; Goldstein, 1997).

As far as the theoretical aspects of the field are concerned, progress was made by more precisely defining the concepts used and the relations between the concepts (e.g. Creemers, 1994; Levine and Lezotte, 1990; Mortimore et al., 1988; Scheerens, 1992; Slavin, 1987a). However, there is a shortage of well-developed theoretical models from which researchers in the area of effectiveness can build theory. The problem is aggravated by the infrequent use of existing models (Scheerens and Bosker, 1997). As a consequence, most of the studies on educational effectiveness are atheoretical and are concerned with the establishment of statistical relationships between variables rather than with the generation and testing of theories that could explain those relationships (Creemers, 2002). There are several reasons to argue for the need to develop and test models of effectiveness that could help us explain differences in student learning results by specifying the relationships between the components in the models and student outcomes. First, a model serves to explain previous empirical research parsimoniously. Second, the establishment and testing of models of educational effectiveness could help us generate a guide to the field to prevent new entrants from reinventing the wheel by repeating existing research. It also maps a series of avenues for future research that may help us expand our knowledge base of educational effectiveness. Finally, a model may provide a useful road map for practitioners, and indeed there are hints that, in part, it has been the absence of educational effectiveness theory that has hindered the uptake of effectiveness knowledge by practitioners in schools (Creemers et al., 2000). It is therefore argued in this book that the next step of EER is to establish models that can be used in policy and practice for improvement.

Characteristics of the proposed theoretical framework of educational effectiveness research

The first part of this section is therefore concerned with the importance of generating a theoretical framework for EER that is able to establish stronger links between EER and improvement practice. A critical review of the main findings and of the theoretical models of educational effectiveness is also provided and helps us identify what the essential characteristics of this new theoretical framework of EER should be.

First of all, it is important to mention that researchers in the 1990s attempted to develop models of educational effectiveness by integrating the findings of school effectiveness research (SER), teacher effectiveness research (TER) and the early input–output studies (e.g. Creemers, 1994; Scheerens, 1992; Stringfield and Slavin, 1992). The studies conducted to test the validity of these models, and especially those studies that tested Creemers’ model (i.e. de Jong et al., 2004; Kyriakides et al., 2000; Kyriakides, 2005a; Kyriakides and Tsangaridou, 2004), revealed that the influences on student achievement were multilevel. This finding is in line with the findings of most studies on educational effectiveness conducted in various countries during the past two decades (Teddlie and Reynolds, 2000), and provides support for the argument that models of educational effectiveness should be multilevel in nature. The analysis of the results of these studies also reveals that in addition to the multilevel nature of effectiveness, the relationship between factors at different levels might be more complex than is assumed in the integrated models of educational effectiveness. This is especially true for interaction effects among factors operating at the classroom and student levels, which reveal that effective teachers are expected to provide different learning support systems to different groups of students in order to help them achieve different types of objectives.

Second, one significant methodological criticism of EER is that most effectiveness studies have examined the magnitude of teacher and school effects overall and have paid very little attention to the extent to which teachers and schools perform consistently across different school groupings. Thus, the concepts of teacher and school effectiveness have been developed in a generic way by drawing up a ‘one-size-fits-all’ model, in which the assumption is that effective teachers and schools are effective with all students, in all contexts, in all aspects of their subjects, and so on. Such conceptualisation of effectiveness has led policy-makers and the general public to a simplistic dichotomy between effective and ineffective teachers or schools (e.g. the idea of schools falling behind) and has eschewed the possibility that teachers may have strengths and weaknesses in their professional practice and some schools may be more or less effective for some groups of students. One consequence is that it becomes difficult to use teacher or school effectiveness research findings to measure such strengths and weaknesses, and therefore difficult for the findings to become a source for conducting formative teacher or school evaluation (Kyriakides and Campbell, 2003). Another consequence is that researchers have attempted to develop generic models of educational effectiveness even though there is some evidence in support of differential educational effectiveness. It is important to acknowledge that the term ‘differential effectiveness’ is often used in the literature to indicate variation in effectiveness at teacher or school level for different groups of students (e.g. Jesson and Gray, 1991; Nuttall et al., 1989). However, the term here is used in a broader sense to indicate that the effect of a factor may be different not only for different groups of students but also when effectiveness is measured through different outcomes of schooling or in different contexts (Kyriakides and Tsangaridou, 2004). Therefore, this concept is used to emphasise the importance of differentiation not only in teaching but also in the functioning of different types of schools and educational systems. This weakness of the current integrated models of educational effectiveness (e.g. Creemers, 1994; Scheerens, 1992; Stringfield and Slavin, 1992) is aggravated by the fact that there is some evidence that low- and high-ability students and students of low and high socio-economic status (SES) respond to different teacher behaviours and styles (e.g. Brophy, 1992; Walberg, 1986a). Although causal relations between teacher behaviour and student achievement have been demonstrated, resulting in a description of effective teaching practice, many of the characteristics of effective teaching vary according to student SES and ability, grade level, or teachers’ objectives (see Campbell et al., 2004; Kyriakides and Creemers, 2006b). Therefore, researchers should attempt to develop models of educational effectiveness that are not only multilevel in nature but also able to demonstrate the complexity of improving educational effectiveness by taking into account the major findings of research on differential effectiveness.

Third, ther...