Understanding PISA's Attractiveness

Critical Analyses in Comparative Policy Studies

Florian Waldow, Gita Steiner-Khamsi, Florian Waldow, Gita Steiner-Khamsi

• Éditeur: Bloomsbury Academic
• Année: 2019
• ISBN:
  9781350057302
• Édition: 1
• Sujet: Éducation
• Sous-sujet: Éducation comparative

Understanding PISA's Attractiveness examines how policy makers and the media interpret the results of PISA league-leaders, losers, and slippers in ways that suit their own reform agendas. As a result, a myriad of explanations exist as to why an educational system is high or low performing. The chapters, written by leading scholars from Australia, Austria, Denmark, Finland, Germany, Norway, Singapore, South Korea, Spain, Sweden, Taiwan, the UK and the USA, provide a fascinating account of why results from PISA and other international large-scale assessments are interpreted and translated differently in the various countries. The analyses in this book bring to light the wide array of idiosyncratic projections into these international tests. In some countries, these tests are also used to scandalise one's own educational system and to generate quasi-external reform pressure. Compiled by two leading scholars in comparative education, Florian Waldow and Gita Steiner-Khamsi, this book offers a truly global perspective on the uses and abuses of PISA and will be of great interest to students and academics working in educational policy, comparative education and political science and those working on large-scale data sets.

1

Introduction: Projection in Education Policy-Making

Florian Waldow

International large-scale assessments and education policy-making

Starting from humble beginnings in the late 1950s, international large-scale assessments (ILSAs) in education have undergone a tremendous expansion in number, frequency, and scope over the past few decades (Kamens and McNeely 2010; Pizmony-Levy 2013; Addey 2018). While in the beginning they were mostly driven by scientific curiosity, over the years ILSAs have increasingly developed into a tool of governance and accountability (Addey 2018). In addition, ILSA results have become an important point of reference and a legitimating argument in education policy-making.

By now, there is a substantial body of research looking at how large-scale assessments, especially PISA, are being used in education governance and policy-making (to name but a few important publications in this field: Grek 2009; Martens and Niemann 2013; Meyer and Benavot 2013; Bieber et al. 2014; Sellar and Lingard 2014; Hamilton, Maddox, and Addey 2015). Among other things, this body of research has shown that countries do not simply participate in ILSAs because they are interested in the results. Rather, according to Steiner-Khamsi (2017), participating in ILSAs can serve purposes of

(1) coalition-building or, more precisely, . . . generating or alleviating reform pressure, (2) mobilising financial sources for the education sector, (3) demonstrating ‘internationality’ and, finally, (4) learning technical skills of how to measure student outcomes and system performance. (Steiner-Khamsi 2017, 444; see also Addey and Sellar 2017)

The fact that there is such a diverse range of purposes for participating in ILSAs means that the processes initiated by ILSAs in national education systems and education policy-making are a good deal more complicated than mere “lesson-drawing” from successful ILSA-league leaders.

This volume aims to take a fresh look at some of the changes the rise of ILSAs has brought about in education policy-making. It understands policy-making in a wide sense, also comprising media discourse and policy-relevant academic writings, which ultimately may feed back into the policy process proper. The contributors to this book focus on why and how national policy actors use the results from international large-scale student assessments for setting policy agendas, evaluating reform options, and formulating new policies. The analytic lens for this fresh look is provided by the concept of projection, which will be outlined below.

Reference Societies, Scandalization, and Glorification

Before the concept of projection is introduced, it is necessary to revisit the concept of “reference societies,” which the former concept builds on in a number of ways. The concept of “reference societies” was originally coined by the macro-sociologist Reinhard Bendix (1967, 1978). Bendix modeled his “reference societies” on the concept of “reference groups” as used in social psychology and sociology (see Hyman 1942; Merton and Rossi 1968).¹ Usually, the term “reference society” is used in the sense of a model nation from which to borrow elements (e.g., by Schriewer 1990; Arbeitsgruppe Internationale Vergleichsstudie 2003; Lingard and Rawolle 2011), that is, the concept denotes positive reference to another country.² However, arguably Bendix’s (1978) definition of the concept allows for a wider analytic use of “reference societies.” In his book Kings or people, he gives the following definition: “I shall use the term reference society whenever intellectual leaders and an educated public react to the values and institutions of another country with ideas and actions that pertain to their own country” (Bendix 1978, 292, emphasis in original). Bendix’s definition leaves open whether reactions to “values and institutions of another country” are positive or negative, accepting or rejecting. Indeed, these reactions can take both a negative and a positive form (Waldow 2016). Reference societies can function as a way of presenting either a desirable model or an “anti-model.” Both types of reference societies are frequently encountered in education policy-making and will feature in this book.

Reference societies are frequently invoked in the context of “scandalization” or “glorification” (see Steiner-Khamsi 2003), respectively. Gita Steiner-Khamsi (2003) defines “scandalization” as “highlighting the weaknesses of one’s own educational system as a result of comparison” and, conversely, “glorification” as “highlighting the strengths of one’s own educational system as a result of comparison” (Steiner-Khamsi 2003). It should be noted that “highlighting the weaknesses of one’s own educational system as a result of comparison” is not the same as “highlighting the weaknesses of one’s own educational system as a result of low scores.” Comparison is a complex process of meaning-making, and scandalization can also occur if ILSA results are average, but lower than anticipated (see Martens and Niemann 2013). Scandalization can even occur when ILSA results are very good, for example, if there is a perception that these results have been bought at too high a price, as claimed by some observers, for example, in Korea (see Chapter 11 by Lee and Sung in this volume) or Shanghai (see Chapter 8 by Reyes and Tan in this volume).

When speaking of reference societies, Bendix speaks of “countries,” that is, he tacitly assumes that reference societies are contained within the boundaries of nation states. However, this is not necessarily the case. Reference societies can encompass several nation states deemed to be similar or part of a region that is perceived as culturally homogeneous, for example, when the “Scandinavian” or “East Asian” education systems are invoked as a reference (e.g., see Chapter 3 on Germany and Chapter 4 on Australia in this volume).

What turns a society into a reference society for others? Researchers have identified various factors. The educational philosopher and historian R. Freeman Butts (1973), who used the term “reference society” for the first time in connection with education, identified countries that were particularly quick to modernize as potential reference societies for others. Bendix (1978) himself demonstrated how economic competitors and military rivals can become reference societies, citing among others Meiji-era Japan as an example. More recently, scholars have identified achieving high scores in ILSAs as a factor that can turn a country into a reference society for others (Wiseman 2010; Lingard and Rawolle 2011). Reference societies have also been actively promoted with reference to ILSA results by the Organisation for Economic Co-operation and Development (OECD) itself. For instance, OECD (2011) presents “[l];essons from PISA for the United States” by presenting a range of “[s]trong performers and successful reformers” (OECD 2011, title page). These not only include PISA top scorers such as Finland, Hong Kong, and Shanghai but also countries that are not top scorers, but supposedly show improvement and are presented as models for successful reform, such as Germany and Brazil. Finally, non-state commercial actors such as McKinsey & Company also promote reference societies in connection with high ILSA results (Barber and Mourshed 2007).

It is important to note that economic competition, military rivalry, and success in ILSAs are “in the eye of the beholder,” that is, they are not objectively given, but the result of perception and interpretation. This means that there can be different perceptions and interpretations in different places. ILSA success can lead to countries becoming reference societies in many other countries and lead to the emergence of new semi-global reference societies, Finland being a case in point. However, ILSAs and the data they produce resonate differently with different nationally or locally preexisting structures and discourses. As mentioned above, the processes involved are more complicated than merely promoting league leaders to the status of global models. How ILSA participation and results (of one’s own country and other countries) are processed and interpreted and which consequences they have (if any) depend heavily on the context in which this takes place (see Martens and Niemann 2013; Baird et al. 2016). These processes of perception and interpretation and the broad range of reactions to them are an important part of what this book is about.

Projection

Having clarified what reference societies are, we can now turn to the concept of “projection.” In the following, it will be argued that some of the processes occurring in connection to the use of reference societies in education policy-making can usefully be conceptualized as “projection.” The notion of projection has popped up now and then in the literature on education policy-making and educational transfer (e.g., Meyer 1986; Zeng and LeTendre 1999; Smithers 2004; Takayama 2010; Waldow 2010, 2016; You and Morris 2016), although researchers have often used terms other than “projection” to describe the phenomenon understood here as “projection” and have rarely discussed the phenomenon systematically, a gap this book attempts to fill at least to a certain extent.

Reference as Projection

The concept of projection in education policy-making grows out of a tradition of research on education policy borrowing and lending grounded in the work of, among others, Schriewer (1990), Steiner-Khamsi (2004) and Zymek (1975). One important insight from this line of research is that references to “elsewhere” depend mostly on the perspective prevalent in the context from which the referring is done, not the context that is being referred to. The concept of projection takes this insight one step further, stressing that actual conditions in the place that is being referred to are often of minor importance; the important thing is what observers want to see, to the extent that what is observed may not even exist in the place that is being referred to. Conceptions of “good” and “bad” education are being projected onto other countries or regions like a slide or a film are projected onto a projection screen. The ensuing image is mostly determined by the “slide,” not by the “screen.” The main function of these projections is the legitimation and delegitimation of educational policies and agendas in the place from which the projection is made.

The fact that the “slide” is more important for the image presented than the “screen” means reference societies will usually be depicted in a very selective way. Certain aspects may be inflated out of proportion, complex and contradictory situations may be presented in a simplified way, and important aspects of context may be neglected. In any case, projection radically reduces the complexity of actual educational conditions and situations.

In the most extreme case, the things that are visible on the projection screen will not even exist in the reference society that is being used as a projection screen. An example from the German media discourse may illustrate this: in an article published about half a year after the German “PISA shock,” that is, the publication of the first round of PISA results in December 2001, the German newspaper Die Welt celebrated “all-day schooling” (Ganztagsschule) as the reason for Finnish PISA success and recommended this as a policy recipe for Germany to emulate (Gatermann 2002). At the time, there was an intensive discussion about all-day schooling in Germany. The fact that “all-day schooling” did not exist as a widespread phenomenon in Finland at the time the article was published (see Matthies 2002) was apparently unknown to the journalist, nor did presumably many readers notice that the journalist was projecting his own idea of a trait of the “good school” onto the Finnish case.

Not every potential reference society is equally suitable as a projection screen, a point that will be elaborated below. Established stereotypical assumptions and beliefs about reference societies play a role here. For purposes of projection, it may even be helpful if not too much is known about the reference society that is used as a projection screen, since a blank screen is particularly suitable for projection. In Chapter 2 in this volume on the Chilean case, Parcerisa and Verger note that other Latin American countries are conspicuously absent as reference societies in educational matters in Chile. They suspect that hidden local rivalries may play a role here. In addition, countries from the vicinity of one’s own country may not be as suitable as countries that are further away because observers feel they know a lot about the former; therefore, the projection screen is not blank and it is more difficult to project (positive or negative) images on to it than in the case of a country that is widely unknown in the context from which the projection is made.

Also, different slides can be projected onto the same projection screen. Again, the chapter...