Assessment of Learning

Wynne Harlen

• Publisher: SAGE Publications Ltd
• Year: 2007
• Print ISBN: 9781412935197, 9781412935180
• eBook ISBN: 9781446223123
• Edition: 1
• Topic: Education
• Subtopic: Education General

This book takes a critical look at how students? achievements are assessed for a range of purposes, from reporting progress to selection and qualification. It considers the relationship between what is taught, and how, and what and how learning outcomes are assessed. The impact of using assessment results for setting targets and evaluation of provision for learning is also discussed.

The pros and cons of using tests and examinations and alternatives based on the judgments of teachers are considered in terms of four key criteria: validity, reliability, impact and required resources. Evidence from research and examples of current practice in different countries within and outside the UK support the case for making more and better use of teachers? judgments in assessment of learning. In this way assessment of learning (summative assessment) can be compatible with assessment for learning (formative assessment).

PART

1

Assessment of Learning in Context

1	Meanings, processes and properties of assessment

This chapter lays the ground for later chapters by setting out the meaning of terms that are used to describe, analyse and evaluate assessment procedures and systems. In particular it makes clear that the word ‘assessment’ is used to refer to the process of gathering, interpreting and using evidence to make judgments about students’ achievements in education. The term ‘evaluation’ is reserved for this process of using evidence in relation to programmes, procedures, materials or systems. It also makes explicit the meaning of ‘assessment by teachers’ or ‘teachers’ assessment’, terms that will feature frequently throughout this book.

A framework of variables within seven main components of assessment is offered as a way of describing different ways of conducting assessment. Finally, properties that need to be considered in making decisions about how to conduct summative assessment are proposed: validity, reliability, impact and use of resources.

Introduction

Any discussion of assessment inevitably involves reference to concepts that are special to the subject – jargon to those not familiar with the terms. Whilst trying to avoid this as far as possible, it is important to use words with some precision in developing arguments relating to topics as complex as assessment. So, although it makes a rather dry start to the book, the matter of terminology used in discussing assessment cannot be avoided. Words such as assessment, evaluation, testing, performance, achievement, formative, summative and so on will have to be used in this book and, without intending to suggest what is correct or incorrect usage, it is essential that their meaning as used here is clear and consistent. It is best to get this done sooner rather than later and to be candid about how difficult it is to be precise.

As part of this clarification, the second section of this introductory chapter looks at how types of assessment can be described in terms of the various ways in which it can be carried out. In the third section, we consider some key properties that ought to be taken into account when evaluating the effectiveness and value of any type of assessment. These properties are revisited throughout the book, in the course of providing evidence and arguments for proposing reform in assessment systems that rely heavily on external tests and examinations.

Meanings

Assessment and Evaluation

Assessment and evaluation both describe a process of collecting and interpreting evidence for some purpose. They both involve decisions about what evidence to use, the collection of that evidence in a systematic and planned way, the interpretation of the evidence to produce a judgment, and the communication and use of that judgment. The evidence, of whatever kind, is only ever an indication or sample of a wider range that could be used.

The terms ‘evaluation’ and ‘assessment’ in education are sometimes used with different meanings, but also interchangeably. In some countries, including the USA, the term ‘evaluation’ is often used to refer to individual student achievement, which in other countries including the UK is described as ‘assessment’. In the UK ‘evaluation’ is more often used to denote the process of collecting evidence and making judgments about programmes, systems, materials, procedures and processes; ‘assessment’ refers to the process of collecting evidence and making judgments relating to outcomes, such as students’ achievement of particular goals of learning or teachers’ and others’ understanding. The processes of assessment and evaluation are similar but the kinds of evidence, the purpose and the basis on which judgments are made, differ. Our concern here is assessment in this latter sense; that is, the evidence is about what students can do, what they know or how they behave and the judgments are about their achievements.

Assessment Systems

‘System’ implies a whole comprised of parts that are connected to each other. In the case of assessment the system will include: procedures for collecting evidence, its use for different purposes, and how it will be used for individual reporting, certification and selection; system monitoring at local and national levels; the use of measures of performance of students in the accountability of teachers and schools; the role of teachers in assessment, both formative and summative; the moderation of teachers’ judgments; and the way in which evidence from different sources, such as assessment by teachers and external tests, is combined.

We need look no further than either side of the border between Scotland and England to appreciate the difference between assessment systems. In Scotland summative assessment of pupils up to the age of 15 or 16 is for internal school use only. There is a nation-wide programme for implementing formative assessment. Achievement is reported to students and parents about twice a year in terms of levels, but there is no central collection of these results and they are not used for national monitoring or for creating league tables of schools (although school results are published for the external testing at age 15–16). Individual student assessment is based on teachers’ judgments, moderated by the use of external tests administered by teachers when they judge students as able to pass a test at a certain level. National monitoring is conducted through a separate programme of testing which involves samples of pupils at certain ages.

By contrast, in England a combination of national testing and teachers’ judgments is used for internal summative assessment and national test results are used for monitoring performance of students at age 7, 11 and 14 year on year. National test results are also to evaluate the performance of schools, local authorities and the country as a whole. Test and examinations results are also used to create targets for schools and give rise to league tables. (There is more detail on these and other systems in Chapters 6 and 7.)

Differences between the ways in which certain components of a system are carried out matter. Changes made in one part of a system have implications for how other parts can function. It is not difficult to point to examples of this interaction. For example, there is research evidence that

• when school accountability is based on external summative assessment data, this impacts on the way that teachers conduct their own internal summative assessment and on how they use assessment formatively (see for example Pollard et al., 2000);
• how teachers carry out formative assessment can influence their own internal summative assessment practice (Black et al., 2003);
• how moderation of teachers’ judgments is carried out can affect the evidence they gather and report about students’ achievements (Donnelly et al., 1993).

Often unwanted effects in one part of the system on another arise from piecemeal changes in policy that ignore the relationship of parts within the whole. These interactions, and those between assessment and the curriculum and teaching methods (pedagogy), are indeed at the heart of arguments made here for change in systems where student assessment is largely based on tests.

Before embarking on making these arguments, however, we take a closer look at the components and clarify some of the terms used in describing the various ways in which these can be carried out.

Components of an Assessment System

The components of a system can be described in terms of combinations of the variables set out in Figure 1.1 under seven headings: purpose, use, type of task that provides evidence, who makes the judgment, the basis for judging the evidence, the way in which the results are reported and any moderation procedures that are needed. Some of the main variables in relation to these seven aspects are also indicated in Figure 1.1. (A far more detailed taxonomy of just one type of assessment, summative assessment by teachers, was developed by Wilmut, 2004.)

The variables in Figure 1.1 can be used to describe whole systems, as well as particular types of assessment within them. For example, the profile of these aspects for formative assessment (assessment for learning) would be:

Purpose formative
Use helping learning
Type of task regular work and some tests/tasks created by the teacher
Agent of judgment teacher and students combined
Basis of judgment criterion-referenced (detailed and task specific) and student-referenced
Form of report or feedback comment or oral feedback.

Moderation does not apply to formative assessment, but in the case of the focus of this book – assessment of learning – all seven headings do apply and Figure 1.1 makes clear the very large number of ways in which assessment for this purpose can be carried out using different combinations of the variables. Not surprisingly, how this is done matters and has a considerable impact on those involved, quite apart from any reaction to the result of the assessment. The choice of variables and how they are combined has implications for what evidence is included (validity), the accuracy of the result (reliability), the impact on those involved, and the cost of the operation in time and other resources. These four properties will be briefly discussed later in this chapter, but also revisited throughout the book as they underpin the main arguments for reform in assessment policy.

First we look briefly at each of the seven aspects in Figure 1.1 and in particular at the different ways in which they can be put into practice in summative assessment.

Figure 1.1 Components and variables of an assessment system

Purposes of Assessment

There are two main purposes for assessing students: to inform decisions about learning experiences and to report on what has been achieved. ‘Formative’ means that the assessment is carried out in order to help learning. It is detailed and relates to specific learning goals. It is essentially part of an approach to teaching and learning in which information about what students have achieved is used to inform decisions as to how to make progress. For this reason it is also called ‘assessment for learning’ and although sometimes a distinction between ‘formative’ and ‘assessment for learning’ is forged (Black et al., 2002), both terms are widely used as meaning:

the process of seeking and interpreting evidence for use by learners and their teachers to decide where the learners are in their learning, where they need to go and how best to get there. (ARG, 2002b)

Summative assessment is carried out for the purpose of reporting the achievement of individual students at a particular time. It relates to broader learning goals that can be achieved over a period of time. It can be conducted by giving a test or examination at that time, or summarizing achievement across a period of time up to the reporting date. Each of these approaches can take a number of different forms and their relative pros and cons, impacts and costs are explored in some detail later.

Although the main focus here is on summative assessment (assessment of learning), we shall also look at the relationship between formative and summative assessment. This includes examining the extent to which there is evidence to support the different claims that ‘Any attempt to use formative assessment for summative purposes will impair its formative role’ (Gipps, 1994: 14) or the conclusion that there are ways of using summative tests formatively (Black et al., 2003). The reason for concern about how these two purposes relate to each other is the evidence that summative assessment, particularly when conducted through testing and when there are high stakes attached to the results, can inhibit the use of assessment formatively. Indeed, this concern was a major reason for bringing together evidence about the properties and impacts of summative assessment conducted in different ways. Since both formative and summative purposes are important in education – it is not a matter of ‘formative, good’, ‘summative, bad’ – it is essential to discuss how summative assessment can be conducted most effectively and without negative consequences for formative assessment. (A more detailed discussion of these matters is in Chapter 8.)

Use of Assessment Results

For formative assessment there is one main use for the data – to help learning. Indeed, this use defines formative assessment and if the information about students’ learning is not used to help that learning, then the process cannot be described as formative assessment. By contrast, the data from summative assessment are used in several ways, some relating to individual students and some to the aggregated results of groups of students.

For individual students, the uses include reporting to parents, other teachers, tracking progress, and selection, certification or accreditation by an external body. These can be grouped under two main headings of ‘Internal’ and ‘External’ to the school community:

• ‘Internal’ uses include using regular grading, record keeping, informing decisions about what courses to follow where there are options within the school, reporting to parents and to the students themselves.
• ‘External’ uses include certification by examination bodies or for vocational qualifications, and selection for further or higher education.

In addition to these uses, which relate to making judgments about individual students, results for groups of students are used in evaluating the effectiveness of the educational provision for students. The main uses of aggregated results are

• Accountability – for the evaluation of teachers, schools, and local authorities, although the extent to which this should depend on measures of students’ achievement is problematic.
• Monitoring – within and across schools in particular areas and across a whole system for year-on-year comparison of students’ average achievements, but whether aggregated individual data are the most useful is contested.

In both cases, there are issues about using the results of individual students in these ways. Information yielded is restricted and does not meet the needs of users. (We return to these matters in Chapter 9.)

Type of Task

In theory, anything that a student does provides evidence of some ability or attribute that is required in doing it. So the regular work that students do in school is a rich source of evidence about the abilities and attributes that the school aims to help students develop. This evidence is, however, unstructured and varies to some degree from class to class, or even student to student. These differences can lead to unfairness in judgments unless assessment procedures ensure that the judgments made are comparable and equivalent work is assessed in the same way. One way to avoid this problem entirely is to create the same conditions and tasks for all students, that is, to use tests.

Testing is a method of asse...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Table of Contents
• List of figures and tables
• Acknowledgements
• Introduction
• Part 1: Assessment of learning in context
• Part 2: Using teachers’ judgments for assessment of learning in practice
• Part 3: Changing assessment practice and policy
• References
• Author index
• Subject index