Chapter 1 Inside the Black Box Revisited
We noted in the Preface that as an approach to teaching, assessment for learning (AfL) has developed considerably over the last two decades in many parts of the world. In this chapter, we revisit the ideas from Black and Wiliamâs review (Inside the Black Box, 1998b) that have made such an impact on educatorsâ thinking about teaching, learning, and assessment. Letâs begin by defining AfL.
AfL is a range of practices that encourages both teachers and learners to seek evidence of the ways in which students are developing their knowledge, skills, and understanding with the intention of using the evidence to inform learning. AfL requires a dynamic approach to assessing learning with assessment opportunities embedded into the ongoing learning activities and interactions in the classroom. Evidence of learning for both teachersâ and studentsâ use is generated from these classroom activities, is explored, and then acted upon during the learning. This process stands in contrast to teachers administering tests, taking studentsâ papers away to grade, or giving feedback at a later date.Assessment used at the point of learning is more likely to reveal what is needed to move forward so that teachers can make judgments about what the evidence shows and act in the immediacy of learning to secure progress for each student.
A New Way of Thinking About Assessment
While the practices that we now identify as AfL have always been within the repertoire of good teachers, the process of actively seeking evidence of learning during classroom activities and intentionally using that evidence to make decisions about next steps has only really come to the fore since Paul Black and Dylan Wiliamâs 1998 review. Prior to this, assessment in many schools and classrooms, both in the UK and the US, had generally been thought of as a separate activity from the teaching and learning process, often conceptualized as a quiz, test, or examination coming at the end of a unit of study.
Black and Wiliamâs (1998b) review was written at a time when accountability for schools had been high on the agenda of the UK government for the previous decade. This political background had led many schoolsâand particularly teachers in England and Wales, where a range of legislation and reform had led to a more centralized system with less autonomy for schoolsâto focus almost exclusively on performance in examinations. Perhaps the most pernicious of the UK governmentâs actions was the introduction of published League Tables that compared examination results across schools. Unsurprisingly, this action resulted in a backwash effect that concentrated teachersâ, studentsâ, and parentsâ perceptions of classroom assessment as the means to produce scores on achievement tests. Even though some year groups, known as grade levels in the US, were not nationally tested, this more summative approach to assessment was rampant across all years of schooling.
Black and Wiliamâs review presented teachers with a new way of thinking about assessment. Rather than the predominant view of assessment as the means to compare and rank students and schools, their review conceptualized assessment as a method of providing qualitative insights, guidance, and feedback during the process of learning. They also helped teachers to recognize that students were actively involved in the process, receiving feedback about their learning and how they could improve from teachers, peers, and from their own internal self-assessment.
Of great significance, Black and Wiliamâs review also presented evidence that a more formative approach to assessment raised achievement. Such a goal was desirable to all stakeholders, but for teachers, a formative approach just made sense in terms of helping children learn. This approach to assessment eventually changed the way that classrooms worked by enabling teachers to recognize and respond to learning as it unfolded, rather than simply measuring it at the end of a more or less extended period of learning for summative purposes, and using this measure for grading, reporting, or ranking.
A Systematic Review
Black and Wiliam produced a systematic review, which provides a broader and more detailed analysis of a research field than a meta-analysis (in which only quantitative results are compared). Consequently, systematic reviews generally give a better indication of what actually happened in the classrooms being studied and provide clearer and more informed guidance for teachers and schools on what such practice looks like in action. Black and Wiliamâs review consisted of a detailed analysis of some 250 academic articles, book chapters, and books from an initial trawl of 560 publications that focused on assessment during the learning process and which resulted in raised achievement or improved learning behaviors. Some of these publications described quantitative studies where control and experimental classes or groups were compared pre- and postintervention. The majority of the publications in the review, however, focused on more qualitative studies that informed the classroom practices that ultimately resulted in improvement.
Black and Wiliamâs extensive systematic review was first published as an academic paper, âAssessment and Classroom Learning,â in the journal Assessment in Education: Principles, Policy and Practice (1998a). To support and encourage teachers to engage with the ideas in the review, Black and Wiliam also published a twenty-page booklet, Inside the Black Box (1998b), which conveyed the key messages from their review. A US version of this booklet was published later that year in Phi Delta Kappan (1998c). At the heart of Inside the Black Box was the message that learning is driven by the interactions between teachers and students in the classroom. We will return to this powerful idea later in the book.
What Is the Black Box?
The analogy of the black box was derived from systems engineering, casting the 1998 UK educational system as focused only on inputs and outputs (and, indeed, the same could be said of the US). Black and Wiliam wanted to explore how the system itself worked. In this analogy, the black box was the classroom and the focus of the review was the teaching and learning that went on inside that box. By investigating what goes on inside the classroom, we can assess whether, how, and in what ways the inputs (teaching, resources, support) truly affect the outcomes we desire for our students.
Because of the more summative approach to assessment that had prevailed in the UK prior to 1998, Black and Wiliam made clear that the form of assessment they were interested in was separate from testing and had a formative purpose. It involved teachers directly collecting evidence of learning and progress from observing learners during activities and reflecting on the conversations and interactions that arose in the classroom.
Through this approach, Black and Wiliam advanced the idea that assessment can happen naturally as teachers and students go about their classroom activities and that assessment allows teachers to see both what students are learning as well as how they are managing their learning. Studentsâ own self-assessment informs them of how well they are doing in their learning, helps them to recognize what they find easy and what they find difficult, and so provides students with guidance on where they need to focus their effort. Both the teachersâ and studentsâ assessment experiences create feedback situations that inform future teaching and learning.
The Assessment Reform Group, a collaboration of assessment experts in the UK, took the message that there was a need for change and a necessity to focus on a more formative approach to assessment as highlighted by the Black and Wiliam review. It was this group that coined the term assessment for learning (AfL) as a description of the classroom practices that created a more formative approach to assessment, distinguishing these practices from assessment of learning, which referred to examinations, tests, and quizzes that measured performance rather than informed teaching and learning.
Getting Into the Details
AfL entails teachers noting and reflecting on observations of students working individually or in groups, analyzing and assessing artifacts they produce in activities, and listening carefully to the many learning interactions in the lesson. With the evidence they obtain through these means, teachers can recognize and pinpoint areas of partial understanding, which then enable them to respond to and put in place support for the next steps in learning as they arise. By intentionally noticing specific pieces of evidence, teachers are more able to meet the needs of individual learners and respond to these in the activities that follow.
Through taking an iterative approach of collecting evidence and making judgments, the process only becomes formative when there is a response either by the teacher, the student, or both that affects future learning. The response might be returning to previous ideas to ensure they are more consolidated before moving on. Often, the response means diverting from the main activity path to ensure students strengthen a particular skill or acquire a piece of knowledge to take their current learning forward. Occasionally, a response requires teachers to put aside the planning they have done and move students on to a more complex activity because the evidence indicates that most students have understood better and more quickly than the teacher had anticipated.
To elaborate further, letâs consider how teachers might move from considering evidence to taking action in the classroom. For instance, if the evidence from a foreign language classroom indicates that several students are struggling to locate the appropriate tense to use in their writing in Spanish, the teacher might introduce some activities that focus on strengthening this language aspect before students return to their writing. In another classroom, perhaps the teacher recognizes from the evidence collected that specific students have become very adept at choosing similes and metaphors and so might use these as good examples with the whole class or perhaps pair up these adept students with other students who need some coaching in using or devising similes and metaphors. Perhaps a teacher notices in science investigations that students need practice in recognizing trends in data before moving on to more complex inquiries. Here, the teacher might provide some sets of secondary data within that topic area and model how analyzing the data, using graphs or bar charts, can suggest relationships between particular variables. These relationships could then be tested with students deciding how to control and manipulate the variables in their investigation. In each case, AfL encourages teachers to be responsive to their studentsâ needs; sometimes their response might require consolidation of earlier learning while, at other times, their response might incorporate the strengths identified in learning to move ideas forward.
Three Research Questions
In their review, Black and Wiliam asked three research questions:
- Is there evidence that improving formative assessment improves standards?
- Is there evidence that there is room for improvement?
- Is there evidence about how to improve formative assessment?
Their analysis of the large body of evidence they accumulated concluded that the answer to all three questions was a firm yes. What follows is a description of how Black and Wiliam came to their conclusion in relation to each question, which gives us a better understanding of the AfL pedagogy that has developed in the years since the review and which we will discuss more specifically in subsequent chapters.
Is There Evidence That Improving Formative Assessment (AfL) Improves Standards?
To answer this question, Black and Wiliam selected twenty or so quantitative studies that demonstrated significant, and often substantial, learning gains in response to classroom interventions where the ways in which learners received feedback were changed. These studies were from several countries and were wide-ranging in terms of the size of the student cohort, the age of the...