The Routledge International Handbook of Educational Effectiveness and Improvement

Research, policy, and practice

Christopher Chapman, Daniel Muijs, David Reynolds, Pam Sammons, Charles Teddlie, Christopher Chapman, Daniel Muijs, David Reynolds, Pam Sammons, Charles Teddlie

• Publisher: Routledge
• Year: 2015
• Print ISBN: 9780415534437
• eBook ISBN: 9781317394310
• Topic: Education
• Subtopic: Education General

The International Handbook of Educational Effectiveness and Improvement draws together leading academics and researchers in the field to reflect on the history, traditions and the most recent developments in this dynamic and influential field. This handbook provides a comprehensive overview of:

• the foundations of the field
• the evolution of educational effectiveness theory and methodology
• the links with other research disciplines
• the links between policy and practice

In conclusion, the handbook sets out a new agenda for future educational effectiveness research.

This handbook is an essential resource for those interested in the effectiveness of educational systems, organisations and classrooms. It offers academics, researchers, students and policy-makers new insights into the latest thinking and evidence about educational effectiveness.

1

Educational effectiveness and improvement research and practice

The emergence of the discipline

Christopher Chapman, David Reynolds, Daniel Muijs, Pam Sammons, Sam Stringfield, and Charles Teddlie

Educational effectiveness and improvement research and practice (EEI) has shown a rapid growth in the three decades since its emergence in the form of three distinctive disciplines in the 1970s. It is generally recognised as evidencing one of the most rapidly growing and successful generations of an intellectual, research-based field in the whole discipline of educational research over the time since the end of the Second World War – a fact acknowledged by critics and proponents alike. Early reviews of the literature across a range of nations were modest in scope, reflecting a ‘modest in scope’ discipline: Reynolds’ (1982) early review had only 100 references; in 2000, more than 1,200 publications formed the basis of the first International Handbook of School Effectiveness Research (Teddlie & Reynolds, 2000). This new International Handbook of Educational Effectiveness and Improvement shows an established field, with robust, valid, and reliable knowledge bases across multiple areas of the discipline.

The roots of EEI lie in the emergence of a number of areas of research in the 1970s and 1980s, specifically those of school effectiveness (SE), school improvement (SI), and teacher effectiveness (TE).

• School effectiveness research has sought to establish what makes schools ‘good’ or enables them to add value to their students, seeking to describe all of the factors within schools in particular and educational systems in general that might affect the learning outcomes of students in both their academic and social areas.

• Teacher effectiveness research has sought to investigate the properties, characteristics, and behaviours of the ‘good’ teachers who add value to their students.

• School improvement research has sought to establish how both schools and teachers can be made ‘good’, as it were, by means of studying how these knowledge bases relate to processes within schools and classrooms to improve student academic and social outcomes.

These three research areas have historically had very different characteristics in terms of their methodologies, their values bases, and their general purposes (see the 1996 School Effectiveness and School Improvement special issue on country reports, 7(2), for further elaboration of these). School effectiveness research has been mostly quantitative in its methods, collecting a large volume of data to establish the ‘value added’ of schools and the characteristics of the more ‘effective’ schools that added the greatest value. The data on school organisational factors focused upon the school’s formal, organisational, and administrative features, rather than on the cultural ones. This disciplinary area assumed that the outcomes of education were mostly a ‘given’ and concentrated heavily upon the academic outcomes, which themselves were the value commitment of state education systems and their personnel (see reviews in Chapman et al., 2012; Reynolds, 2010a; Townsend, 2007a).

By contrast, the school improvement area has been predominantly qualitative in orientation, seeking to collect naturalistically the data that explored teacher and student perspectives on schooling. The collected data on schools and classrooms focused much more on educational ‘culture’ in all its complexity, and the disciplinary area encouraged a critical perspective on – indeed, often a debate about – the utility of ‘official’ state education measures as indicators of educational worth (see reviews in Hopkins & Reynolds, 2001; West, 2012).

It was no surprise that proponents of these two effectiveness and improvement disciplinary areas spent much time attacking each other (see examples in Teddlie & Reynolds, 2001; Thrupp, 2001), while many who researched the boundaries called for a merger of the disciplinary areas (Reynolds et al., 1996).

The third disciplinary area – that of teacher effectiveness – evolved independently from and generally earlier than the other two. Early material from North America in the 1970s and 1980s exhibited many of the intellectual characteristics of school effectiveness research, save for the concentration upon the classroom rather than the school (Creemers, 1994). Indeed, much of the more nearly rigorous study of school effectiveness of the late 1980s into the 2000s grew in part out of the work of several people now associated with the ‘Invisible College for Research on Teaching’ informally sponsored for two decades by such teacher effectiveness luminaries as Jere Brophy, N. L. Gage, Carolyn Evertson, Tom Good, and Jerry Freiberg. Bert Creemers and Sam Stringfield were two examples of school effectiveness scholars who were shaped by the teacher effectiveness traditions of the Invisible College. Further, in the United States, the teacher effectiveness field produced more successful change projects in its early years than did school effectiveness (see, for example, Good & Grouws, 1979; Stallings, 1980; for a review, see Rosenshine & Stevens, 1986).

The idea of ‘teacher effects’ was, for many societies (such as the United Kingdom, United States, Australia, and continental Europe), more ‘naturalistic’, appreciative, and focused upon understanding teacher perspectives (Muijs & Reynolds, 2011). The teacher professional development literature was also much less focused upon the transmission of ‘good’ practice to teachers than was the school effectiveness paradigm and much more on encouraging teachers, as professionals, to generate their own knowledge for use.

Over time, the three component parts of EEI – SE, SI and TE – have begun to develop from their initially different disciplinary stances to encompass a range of common perspectives about methodology, scientific orientation, and scientific purpose, which now can be said to comprise the EEI paradigm. The emergence of that paradigm, and its characteristics, are what we outline now.

We should be clear, first, that there are strong elements of continuity in the progress over time from SE, TE and SI towards the EEI paradigm and strong commonalities in the trends over time. The historic, initial three disciplines were all possessed of:

• a strong commitment to the use of a scientific approach concerned with the gathering of empirical data to generate ‘known to be valid’ knowledge about schools, classrooms, and other educational settings;

• a commitment to the use of this knowledge – which can be called science with a social purpose – to improve educational settings for all children, and especially for children from disadvantaged homes and communities;

• a commitment to resourcing the teaching profession with this knowledge to improve practice; and

• a continued emphasis upon equity issues in terms of addressing the needs of disadvantaged children, as well as wanting to improve the educational outcomes and chances of all children. The means of addressing this issue may have been a concentration on research in the school ‘sites’ of the socially disadvantaged in the 1960s to 1980s, and may now be the use of sophisticated techniques to disaggregate the student group across all schools into those with lower prior attainment and the remainder, but the commitment to reducing the differences between the disadvantaged and the rest remains a constant between the old paradigms and the new EEI one.

In these respects, EEI is partially a continuation of an educational paradigm, as well as the manifestation of a new one.

We should note that the generation of the EEI paradigm was aided by the setting up of an organisational structure that made the emergence of a new paradigm possible: the International Congress for School Effectiveness and Improvement (ICSEI) in 1988, together with its journal, School Effectiveness and School Improvement (SESI) in 1990. New organisational structures, such as the Society for Research on Educational Effectiveness (SREE) and its Journal of Research on Educational Effectiveness (JREE), and the evolution of several Special Interest Groups (SIGs) within the American Educational Research Association (AERA) in the United States, the SIGs on Educational Effectiveness of the British Educational Research Association (BERA) and the European Association for Research on Learning and Instruction (EARLI), and the Educational Effectiveness and Quality Assurance network of the European Conference for Educational Research (ECER), have also continued to make cross-paradigmatic development possible. The ICSEI has evolved a team of researchers involved in developing a specialist group on the methodology of educational effectiveness (MORE) and other journals, such as the Journal of Educational Effectiveness, Effective Education, School Leadership and Management and, in Germany, the Journal for Educational Research, have also played their part.

This organisational structure led over time to a marked internationalisation of the field, as shown in the international reach of most of the literature reviews (see Teddlie & Reynolds, 2000, and Townsend, 2007a, for reviews of the field) and in the increasing proportion of research work that has been multinational (see Reynolds et al., 2002, for example).

It also led to a merger or synergy of approaches generated by having, for example, school effectiveness researchers in close intellectual proximity to school improvement researchers and practitioners. Examples of some large-scale studies from within the EEI period include: those of Brandsma and Knuver (1988) on primary schools, and those of Bosker and van der Velden (1989) on secondary schools in the Netherlands; the studies of Grisay (1996) on secondary schools in France; those of Hill and Rowe (1996) on primary and secondary schools in Australia; Stringfield, Reynolds, and Schaffer (2008) in Wales and England; and those of van Damme et al. (2006), de Fraine et al. (2007), and Verachtert et al. (2009) in Flanders.

The international opportunities for networking and for joint research in multiple countries, and the powerful effects of different research and country traditions in EEI that were offering new learning opportunities, choices, and possibilities, meant that the field developed rapidly. Also, after calls for a merger of SE and SI concerns (for example, Reynolds, Hopkins, & Stoll, 1993), many SE researchers became more comfortable with SI’s qualitative methodology, its commitment to more ‘cultural’ views of school processes rather than the reified formal organisational factors that had been the SE commitment, and its commitment to the importance of seeing teachers as something other than mere ‘empirical/rational’ educational actors.

We now proceed to look in detail at the evolving EEI paradigm as it stands currently in terms of its characteristics.

Characteristic one: A commitment to theory generation

The EEI field has been committed to the generation of theories that explain its empirical findings, since possessing theories that explain the relationships between variables – such as those between schools and students – is essential for any successful field. Theories ‘organise’ findings in ways that help new entrants to a field. They provide clear explanations for people inside and outside a field, and provide rationales for any practitioner or policy take-up of findings. Having ‘theory’ that moves beyond associations to explanations is an important issue throughout the social and behavioural sciences, of course, as well as in EEI.

In its early stages, EEI had only theoretical perspectives that were the results of the borrowing of theories from other disciplines – such as contingency theory (used to discuss contextual variation), or coalition building taken from political science (used to discuss successful leadership in effective schools), together with some preliminary attempts to ‘causally order’ the educational effectiveness factors (Bosker & Scheerens, 1994; Scheerens & Bosker, 1997).

However, systematically explaining, in an integrated theoretical fashion, the associations between variables has been the focus of the recent dynamic theory of educational effectiveness of Creemers and Kyriakides (2008). Their theory is comprehensive in nature and looks simultaneously at all the different levels of the educational system: the student, the classroom, the school, and the school context. Crucially, it is dynamic and seeks to place the study of change at its heart, since its proponents rightly believe that the lack of appropriate models of change has hindered the uptake of EEI by practitioners in schools. Longitudinal research is obviously favoured, because this makes the study of change easier, and each educational factor is argued to possess five dimensions: frequency, focus, stage, quality, and differentiation. Factors at different levels are seen as having both direct and indirect effects upon student outcomes. There is also a particular focus upon the classroom and upon teachers’ behaviours. The theory is being tested in multiple studies (for example Kyriakides, 2008; Kyriakides & Creemers, 2008a, 2008b, 2009), with promising results. Other interesting theoretical speculations of a different kind are in Luyten, Visscher, and Witziers (2005) and van de Grift (2009).

The historical criticism of the work of the SE and SI traditions as ‘atheoretical’ carried some purchase, particularly given the production, historically, of simple empirically generated ‘checklists’ of effective practices that characterised some of the early studies (such as Edmonds, 1979). These criticisms have been blunted by the conscious attempt in EEI to employ theoretical positions already in existence and to generate novel positions, as evidenced in the van de Grift and Houtveen (2006) study of underperformance in Dutch primary schools, and Muijs’ (2004) employment of three theoretical positions from contingency theory, compensation hypotheses, and additionality hypotheses in work on improving English schools in low socio-economic status (SES) context areas.

It should be noted that the proponents of SI showed greater interest in theoretical modelling from the outset (with attention being given to Kurt Lewin’s theorising, for example), but have probably not maintained this interest. Models of improvement have long existed in this area both in terms of acknowledgement of phases of improvement and types/levels of improvement, but little attempt has been made to move from ‘meso’-level heuristics to a more general ‘theory’ of improvement, although school improvement has drawn quite extensively on ‘meta’ theories of change (see, for example, Langley et al., 2009).

Characteristic two: A commitment to methodological sophistication

In the last decade, it is clear that EEI has generated considerable methodological advances. A SESI special issue was devoted to this topic (Sammons & Luyten, 2009), and Creemers, Kyriakides, and Sammons (2010) have also discussed methodological issues and new advances in depth. There were a number of important methodological achievements in EEI, particularly related to the use of multilevel models and large-scale longitudinal research, which recognised the complexity and hierarchical structure of most educational systems. The advent of accessible software packages, such as HLM for hierarchical linear (and non-linear) modelling (HLM) and MLWin for multilevel modelling (MLM), encouraged improvements in the size and scale of statistical approaches used during the late 1980s and 1990s. (See, for example, work examining both teacher and school effects by Hill & Rowe, 1996, 1998, which demonstrated not only that teacher effects tend to be larger than school effects, but also that, in combination, teacher and school effects could account for a substantial proportion of the variance in student outcomes.)

Improvements have taken place in the modelling of measurement error and interaction effects, cross-classified models that examine multiple institutional membership, and regression discontinuity that studies the size of the schooling effect and its variation in the absence of longitudinal data. These provide examples of recent developments in EEI that are proving fruitful.

Further refinements in multilevel approaches include multilevel meta-analysis, which has the potential to provide better estimates of the size and variation in educational effectiveness for a range of outcomes, phases of education, and contexts (Hox & de Leeuw, 2003). Meta-analysis uses statistical results from a range of studies that address a similar research question, and often seeks to establish an average effect size and estimate of the statistical significance of a relationship (Cooper, Hedges, & Valentine, 2009). In EEI, this might be the effects attributable to a particular approach to teaching or of a school reform programme. This can be seen as a major advance in the field. It has promoted the refining of theory, and enables researchers to identify generic and more specific factors, the impact of which is dependent on the educational setting in which they are operating (Scheerens & Bosker, 1997).

Multilevel modelling is also important, with its usefulness further demonstrated by its rapid spread across many different scientific fields. It became popular in many different countries in the early 1980s onwards because it allowed researchers simultaneously to explore individual- and group-level influences. It is now extensively used in disciplines such as demography (for example, Sacco & Schmidt, 2005), biology (McMahon & Diez, 2007), medicine (Diez-Roux, 2000), and non-educational general social science (Jones, Johnston, & Pattie, 1992).

In EEI, the use of MLM was clearly essential to represent faithfully the reality of educational systems in which students learned or not, in accordance with variation in their background characteristics, and in which they attended classes nested in schools, which in turn were nested in districts/local authorities, which in turn were nested in regions and nations. Multilevel modelling makes possible much more than the simple historical analysis of a ‘means on means’ variety that the multiple regression methodologies of the early studies had utilised; it makes possible the handling of multiple variations, differential effects, cross-level interactions, and reciprocal effects, and the simultaneous study of primary and secondary school effects, with additional neighbourhood effects.

Also increasingly sophisticated has been EEI’s understanding of the interrelationships between levels and stages. Early MLM approaches had tended...

Table of contents

• Cover Page
• The Routledge International Handbook of Educational Effectiveness and Improvement
• The Routledge International Handbook Series
• Title
• Copyright
• Contents
• List of Figures
• List of Tables
• List of Contributors
• List of Abbreviations
• Preface
• 1 Educational Effectiveness and Improvement Research and Practice: The Emergence of the Discipline
• 2 Methodological and Scientific Properties of School Effectiveness Research: Exploring the Underpinnings, Evolution, and Future Directions of the Field
• 3 Effective School Processes
• 4 The Scientific Properties of Teacher Effects/Effective Teaching Processes
• 5 School Improvement and System Reform
• 6 Theory Development in Educational Effectiveness Research
• 7 Quantitative Methods
• 8 Qualitative Methods in Educational Effectiveness and Improvement Research
• 9 Educational Effectiveness Research in New, Emerging, and Traditional Contexts
• 10 Comparative Educational Research
• 11 Educational Effectiveness and Improvement Research and Educational Policy: The Rise of Performance-Based Reforms
• 12 Educational Effectiveness Research and System Reconstruction and Change
• 13 Leadership Development and Issues of Effectiveness
• 14 Educational Effectiveness and Improvement Research, and Teachers and Teaching
• 15 The Challenges of Globalisation and the New Policy Paradigms For Educational Effectiveness and Improvement Research
• 16 Critical and Alternative Perspectives On Educational Effectiveness and Improvement Research
• 17 Conclusions: The Future of Educational Effectiveness and Improvement Research, and Some Suggestions and Speculations
• References
• Index