Testing: Friend or Foe?

Theory and Practice of Assessment and Testing

Paul Black

• Publisher: Routledge
• Year: 2002
• eBook ISBN: 9781135715137
• Topic: Education
• Subtopic: Education General

Owing to daily work pressures and concerns, many teachers have little opportunity for considering and furthering their understanding of different issues surrounding assessment. Written in a user-friendly, jargon-free style, this text provides the reader with points of growth or change in the field of assessment. Each chapter in the text ends with a section on questions/exercises and further reading.

Chapter 1
Introduction

The Aim of this Book

As I write this introduction, there are about fifty books on assessment on and around my desk, and this is only a fraction of the number on the library shelves. So a book like this one can only be an introduction to the academic study of assessment. It is designed to be a comprehensive introduction. My strategy is to attempt to convey a grasp of the most important concepts, and to then convey some introductory acquaintance with the many conceptual, technical and practical issues that follow. Where the treatment has to be so short that it is superficial, I hope that the bibliography and references will guide readers who look for more thorough treatments.

My own experiences, as a university teacher and examiner; as an A-level examiner with two GCE boards; as a member of the board and of the research committee for one of those boards; as one of the directors of the science monitoring of the Assessment of Performance Unit (APU); and as chair of the 1987–88 Task Group on Assessment and Testing (TGAT), have all fashioned my particular perspective on this subject. However, whilst I draw on these resources, particularly from the APU work, to illustrate the ideas, analyses of these particular experiences lie outside the scope of this book. Thus, for example, readers will have to look elsewhere for my attempts to reflect on the experiences of the APU work (Black, 1990) or of the TGAT venture (Black, 1993, 1994, 1997). In the same spirit, this book does not attempt to focus closely on any analysis of the developments in national assessment in the UK following from the 1988 Education Reform Act, except to refer to some of the developments as illustrations of basic issues and, of course, to give reference to fuller treatments by others. The discussion does draw upon the experience of other countries, partly because this broadening of perspective shows up some of the issues more clearly, and partly because it is important to question and explore what one has come to take for granted about one’s own experience.

Structure and Sequence

In any study of curriculum and pedagogy in education, the issues involved in assessment and testing cannot be understood as marginal extras. As the arguments in this book should make clear, assessment and testing should be the heart of any such study. As the TGAT report (DES, 1988 paragraph 3) declared:

Figure 1: An outline of the book—three main strands

Promoting children’s learning is a principal aim of schools. Assessment lies at the heart of this process. It can provide a framework in which educational objectives may be set, and pupils’ progress charted and expressed. It can yield a basis for planning the next educational steps in response to children’s needs.

However, there is far more involved. Clear and compelling principles are not enough to guide the establishment of assessment practices. These practices can only be understood in relation to the historical, cultural and political contexts within which they are worked out, and they also entail complex issues both of technique and of principle.

The way in which these issues are worked through in this book is illustrated by Figure 1. There are three main themes or strands, represented by the three vertical columns. The left hand strand is the general social context. Here, some historical and cultural foundations are explored in Chapter 2 by a presentation of two brief and contrasting histories—of assessment and testing in England and in the USA respectively. The central strand is concerned with purposes. Chapter 3 sets the scene for the core exploration of the issues, both of principle and of technique, with a discussion of the three main purposes of assessment—the formative, the summative and the accountability purposes respectively.

The right hand strand is concerned with technical issues, which are covered by three chapters in succession. Chapter 4 discusses the concepts of reliability and validity, with particular reference to links between validity and assumptions about effective learning. In the light of these key criteria, Chapter 5 explores what assessment results are actually designed to tell us, an issue which underlies the technical issues of norm referencing, and of criterion and domain referencing. Chapter 6 then looks more closely at the practicalities with a survey of the different methods which can be used for assessment—ranging from the various types of questions used in written test papers to the recording of classroom performance.

Chapter 7 comes back to the links between purposes and general policy with a discussion of teachers’ roles in assessment, giving particular emphasis to formative assessment but also leading in to debates about teachers’ roles in summative assessment. Chapter 8 takes this same exploration further by looking closely at pupils’ roles in assessment, in particular at the development of self-assessment by pupils.

The summative aspects, which serve the certification and accountability purposes, are then discussed more fully in Chapter 9. Thus, this chapter adds to the discussions of purposes, but the ways in which these purposes are achieved can only be appraised by looking at national systems within which policies for assessment, curriculum and accountability are interrelated. For this reason, this chapter gives examples from several different countries. In doing this, and probing more deeply into the issues surrounding accountability and the rhetoric of standards, it brings the discussion back to the broader perspectives introduced in Chapter 2. Chapter 10 then gives a closing summary, emphasizing particularly the need for a systemic approach to any reform of assessment.

Organization of the Chapters

Each chapter has both an introduction, to explain the structure and sequence of its sections and sub-sections, and a summary which draws together the main points that have been explored.

The links which will help you to explore the literature are given in two forms. At the end of each chapter there is both a bibliography and a set of references. The bibliography lists books or review articles which would be useful sources to follow up the issues of the chapter in more detail, and also to see them presented from different perspectives. Where appropriate, some indication is given of the chapters in each book which are particularly relevant.

Whilst the bibliography gives a general guide to treatments by other authors, the set of references listed are very specific guides for following up particular points made in the present text. These are therefore linked to explicit references in the text. Where the reference is to a particular chapter or set of pages this will be indicated either in the text or in the reference list.

The bibliography and reference lists for each chapter are self-contained, in that they do not depend or draw upon the corresponding lists from other chapters. So, for example, some books will appear in the bibliographies for several chapters—each chapter referring to different parts of the same book, whilst others may only appear for the one chapter for which their particular treatment gives a useful expansion.

Words, Words, Words

Most difficulties about terminology have been dealt with as the need to introduce words with unusual or difficult meanings arises. However, two particular problems arise throughout and need particular introduction.

The first is presented by the terms ‘assessment’ and ‘testing’. These overlap, can often be taken to mean the same thing, and carry different overtones, ‘testing’ being hard, rigorous, inflexible and narrow-minded, ‘assessment’ being soft, sensitive, and broad- or woolly-minded. Other authors have met the same difficulty:

Note that I use the term test when referring to traditional, standardised developmental and pre-academic measures and the term assessment when referring to more developmentally appropriate procedures for observing and evaluating young children. This is a semantic trick that plays on the different connotations of the two terms. Technically, they mean the same thing. (Shepard, 1994)

It is to be hoped that the overtones can be set aside, but the position adopted here is that whilst the two terms overlap, they do not mean the same thing. The terms are to be used according to the following definitions provided in the glossary of the TGAT report:

Assessment: A general term embracing all methods customarily used to appraise performance of an individual pupil or group. It may refer to a broad appraisal including many sources of evidence and many aspects of a pupil’s knowledge, understanding, skills and attitudes; or to a particular occasion or instrument. An assessment instrument may be any method or procedure, formal or informal, for producing information about pupils: for example, a written test paper, an interview schedule, a measurement task using equipment, a class quiz.

Test: Strictly, any assessment conducted within formal and specified procedures, designed to ensure comparability of results between different test administrators and between different test occasions. For some it implies a set of written questions, externally prescribed, with written responses marked according to rigid rules; for others, any of a broad range of assessment instruments with standardised rules of administration and marking which ensure comparability of results. This report uses the term in this latter, broader sense, (see the Glossary in DES, 1988)

The second problem concerns the words used to refer to those involved in the assessment process. I shall use teachers throughout: the main emphasis here is on assessment in schools. Much of what is being said will be applicable to higher education and to employment, although the particular needs and problems in those spheres are not directly addressed.

I shall also use the terms ‘pupils’ and ‘students’ to refer to those who enjoy or suffer assessment or testing practices. To use ‘students’ for very young children in primary or pre-primary education could be confusing, whereas to use ‘pupils’ for A-level candidates seems inappropriate. However, in principle these terms will be interchangeable.

Statistics

It is not difficult to find works on assessment which are replete with algebraic equations and heavy with the technical language and concepts of statistics. Whilst there are sound reasons for the deployment of this machinery in assessment studies, very little of it is required for this book’s purpose. A qualitative understanding of the following terms will be assumed:

Mean	Normal distribution	Spread
Standard deviation	Correlation	Regression

Explanation of these terms for the beginner is given in the Appendix, together with some useful references to other texts.

Attitudes and Outlooks

All who read this book will have experienced assessment and testing—at least as their recipients (victims?) and probably as their agents. Few will regard either of these sets of experiences as happy memories. Thus, it is to be expected that a book about the subject can hope for no more than to arouse morbid fascination, rather than to give pleasure. Being an optimist, this author hopes for more. The subject is inescapably central to any educational enterprise. Those who can grasp the concepts that underpin it, and who can explore the alternatives for practice that are possible, will be better equipped for all educational work, and might even be able to turn at least some experiences of assessment into enjoyable aspects of learning.

References

BLACK, P.J. (1990) ‘APU Science—the past and the future’, School Science Review, 72, 258, pp.13–28.

BLACK, P.J. (1993) ‘The shifting scenery of the National Curriculum’, in O’HEAR and WHITE, J. (eds) Assessing the National Curriculum, London: Paul Chapman, pp. 57–69.

BLACK, P.J. (1994) ‘Performance assessment and accountability: The experience in England and Wales’, Educational Evaluation and Policy Analysis, 16, pp. 191– 203.

BLACK, P.J. (1997) ‘Whatever happened to TGAT?’, in Cullingford, C. (ed.) Assessment versus Evaluation, London: Cassell, pp. 24–50.

DEPARTMENT OF EDUCATION AND SCIENCE (1988) Task Group on Assessment and Testing—National Curriculum: A Report, London: Department of Education and Science. SHEPARD, L. (1994) ‘The challenges of assessing young children appropriately’, Phi Delta Kappan, November, pp. 206–10.

Chapter 2
History

Why Bother?

It may seem strange to choose a historical review for the first substantive chapter of this book. A first justification is that in gene...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Series Editors’ Preface
• Acknowledgments
• Chapter 1: Introduction
• Chapter 2: History
• Chapter 3: Purposes
• Chapter 4: Confidence In Assessment—Reliability and Validity
• Chapter 5: What to Test—Norms, Criteria and Domains
• Chapter 6: How to Assess—Methods and Instruments
• Chapter 7: Teachers’ Roles In Assessment and Testing
• Chapter 8: Pupils and Assessment
• Chapter 9: Certification and Accountability
• Chapter 10: Conclusions
• Appendix: A Little Statistics