Impact of international organisations' work on assessment
International organisations in education generally follow developments in the research and policy communities but, once they have embraced the development, they can exert a powerful influence, often shaping further developments. A good example is work in educational assessment and, more particularly, in monitoring levels of student learning (e.g., OECD, 2001).
When educational assessment was exclusively norm-referenced, individual student's performances were judged in comparison with the performances of others, most notably with the average (norm), and countries or systems were judged on the performances of their students in comparison with the performances of students in other countries or systems. In both cases, the judgements depended crucially on whom the others were. When educational assessment became standards or criterion-referenced, performances could be related to an achievement scale. Improvement no longer needed to be at the expense of others. Absolute change could be monitored as well as relative change (Lord & Novick, 1968; Rasch, 1960).
These developments in educational measurement can be seen in national assessment programs such as the US National Assessment of Educational Progress, commenced in 1969 (National Center for Education Statistics, 2020) and Australia's National Assessment Program: Literacy and Numeracy (NAPLAN) (Australian Curriculum, Assessment and Reporting Authority, 2020). They are also evident in the work of an international non-government organisation, the International Association for the Evaluation of Educational Achievement (IEA) which gathered international comparative data on student achievement for the first time in 1960 (International Association for the Evaluation of Educational Achievement, 2020).
The Organisation for Economic Co-operation and Development (OECD) has gathered statistics on a broad range of policy domains since its foundation in 1961. In education, its collections focused on inputs and outcomes such as educational attainment, measured as the number of years of education completed. It was only under pressure from the United States as one of its Members, that it extended the coverage to measures of student achievement, measured for the first time in 2000 for 15-year-olds in reading, mathematics, and science in its Program for International Student Assessment (PISA) (OECD, 2001).
OECD's PISA built on IEA's long tradition of international, comparative surveys of student achievement but, probably due to OECD's status as an intergovernmental body, the PISA surveys quickly attracted considerable interest and exerted considerable interest. One indicator is the extending nature of its coverage. The 1960 survey involved 28 of the then 29 OECD Member countries and four others. The most recent PISA survey in 2019 involved all 35 current OECD Member countries and 42 others (OECD, 2019b, pp. 57–58).
The international, inter-governmental work on educational measurement followed the academic research work and, by and large, national developments. The first PISA in 2000, for example, assessed only the traditional domains of reading, mathematics, and science. The second PISA in 2003 assessed these same domains but added an assessment of problem solving, though it was a limited view of problem solving since it was largely a test of analogical reasoning (OECD, 2003).
Problem solving in PISA 2003 was seen largely as a generic skill though, well before then, cognitive psychology research on the differences between experts and novices had distinguished generic and domain-specific problem solving. For example, Vos et al. (1983) had shown that high-level researchers in chemistry performed like novices not experts when seeking to solve a political science problem.
More recently, PISA has increasingly drawn on research on cognitive skills and learning to broaden the range of educational outcomes measured. PISA 2015 included an assessment of collaborative problem solving (OECD, 2017), PISA 2018 included an assessment of global competence (OECD, 2019a), and PISA 2021 will add an assessment of creative thinking that draws on the cognitive science research on the differences between domain-specific and general competences (OECD, 2019c, p. 8). PISA is now driving ahead of developments in many countries and stimulating further national developments.
OECD's work on brain research and learning science
In applications of brain research and learning science to education, OECD similarly built upon a substantial amount of work already underway in the scientific community. In this field, OECD could not play to its traditional strength as a source of powerful, comparative data. Rather it became essentially a knowledge broker through its Center for Educational Research and Innovation (CERI) within its Directorate for Education. The head of the Centre, Jarl Bengtsson, recognised the potential significance of this new field and obtained support for a new project, Learning Sciences and Brain Research, from OECD and external sources including the US National Science Foundation and the UK Lifelong Learning Foundation and the Japanese Ministry of Education, Culture, Sports, Science and Technology (OECD, 2002, pp. 3–4).
OECD's entry into this field capitalised on its convening power. It commenced with three High-Level Forums which it organised jointly with important research institutes and to which it attracted significant researchers in the field. The first, on Brain Mechanisms and Early Learning, was held in June 2000 at the Sackler Institute in New York City, the second, on Brain Mechanisms and Youth Learning, was held in February 2001 at the University of Grenada in Spain and the third, on Brain Mechanisms and Learning in Ageing, was held at the RIKEN-Brain Science Institute in Tokyo, Japan. OECD's first publication in the field (OECD, 2002) derived from these forums. It reviewed the research field, speculated about how cognitive neuroscience could inform education policies and practices and charted a way ahead involving the establishment of three research areas, Brain development and literacy, Brain development and numeracy, and Brain development and learning over the life cycle. Each area was then addressed by an international research network.
Advancing its brokering role, OECD-CERI produced an online collection, Brain and learning – Resources, that included links to “Brain Primers,” “Brain Maps,” “Brain Glossary,” and articles of interest that identified articles in research journals and popular magazines of potential interest to people in education following the project (OECD-CERI, 2020). OECD also published on its website a series outlining six neuromyths that were misrepresentations of the science that can generate misleading claims about implications of neuroscience for education. They can be found at https://www.oecd.org/education/ceri/neuromyth1.htm.
OECD's Learning Sciences and Brain Research project ran from 1999 to 2007. It culminated with the publication of a final report that synthesises progress on the brain-informed approach to learning, and uses this to address key issues for the education community. It offers no glib solutions nor does it claim that brain-based learning is a panacea (OECD, 2007, p. 13).
Reflecting on the work, key players in the project subsequently wrote;
Education research is gradually accumulating a knowledge base linking educational policies and practices with learning outcomes. However, we often lack detailed explanations of how and why these outcomes arise. Studies are often based on correlations, making a policy or practice with a certain outcome while leaving the process in a “black box.” Brain research allows us to look deeper into the underlying learning processes and shed more light on causal relationships (Hinton, Miyamoto, & della Chiesa, 2008, p. 87).
A year later, they were somewhat more sanguine, writing:
This transdisciplinary project brought many challenges. Within the political community, participation in the project varied, with some countries resisting approval of the project altogether, in the beginning. In the neuroscientific community, participants struggled to represent their knowledge in a way that would be meani...
