From Testing to Assessment

English An International Language

Clifford Hill, Kate Parry

• Publisher: Routledge
• Year: 2014
• eBook ISBN: 9781317893387
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

From Testing to Assessment: English as an International Language provides a critical review of conventional and alternative approaches to the assessment of English literacy skills in various parts of the world. It presents empirical studies conducted in three major settings: in countries such as Japan and Brazil where English functions as the language of international commernce; in multilingual countries such as Nigeria and Zimbabwe where English is the national language of education and government; and in such countries as Canada, the United Kingdom and the United States where English is the dominant language.

The book opens with a discussion of language assessment in relation to debates about the nature of literacy; it concludes with a discussion of policy implications, which is grounded in literacy theory as well as in practical constraints such as available human and material resources.

PART I

THEORETICAL CONSIDERATIONS

A central theme of this volume is that a reading test, like any other text, is embedded in a social and ideological context. Accordingly, we begin Chapter 1 by examining the historical development of reading tests in both Britain and the United States. We show the social and ideological origins of the two traditions as well as the intellectual and pedagogical concerns that have informed test development within each. We then illustrate the differences by presenting and analysing a sample of test discourse from each tradition.

The differences are, however, largely superficial, since both traditions are premised on similar assumptions about text and how it works. We characterise these assumptions collectively as an autonomous model of literacy and we examine this model with reference to text, of readers and writers, and the skill of literacy itself. We claim that this model is misleading for it underplays the social dimensions of written communication.

We then develop an alternative model, which we characterise as a pragmatic model of literacy. We present this model with reference to the same three parameters, demonstrating that communication is necessarily social in character, whether it is achieved by spoken or written language. We present this model both as a tool for critically examining reading tests and as a guide for developing alternative methods of assessment. It thus serves as a theoretical basis for the remaining parts of this book.

1 Models of literacy: the nature of reading tests

Clifford Hill and Kate Parry

Those who have taught or studied English in different parts of the world may well have come across two distinct kinds of reading test. One consists of a relatively small number of passages with tasks that require students to write their own responses. It is generally included in a larger English Language paper, and is found in the school certificate exams of many countries where English is an official language; it will be familiar, too, to anyone educated in Britain before the mid-1980s. The other consists of a larger number of shorter passages, with tasks that require students to select a response from a given set. This kind of test is often administered on its own, and it is a basic educational tool in the United States and in countries that have come under American influence. These two kinds of test represent distinct traditions of assessing literacy skills in English, the British and American, and they have quite different social and ideological origins.

Let us first examine the British tradition, which is the older of the two.¹ It originated in the late 1850s with the foundation of the first public examination boards – the Oxford Delegacy of Local Examinations and the Cambridge Local Examinations Syndicate – and in the opening of London University’s matriculation exams to the general public. At this point the three universities were working with a somewhat élitist model of education: all three were attempting to provide support for ‘schools for the middle classes’ (Bruce 1969, p. 78) and were focusing on the end-point of students’ school careers, the exams being intended to demonstrate that a student had acquired knowledge of a range of subjects. For the population that took these exams, the ability to read was hardly in question: where concern was expressed about language, it centred on the ability of students to write about what they had read. In 1858, for example, the Oxford Delegacy’s examiners reported:

There was often a tolerably wide range of information, and sometimes no small amount of original thought: but candidates who showed both these, frequently showed little power of putting their information together, and still less power of expressing it in clear language. (Bruce 1969, p. 77)

In 1917, these university exams, together with those of various professional bodies, were incorporated into a single set of School Certificate exams, the regulations for which required that students should pass in a range of subject areas. While there was a certain amount of latitude within this range, all candidates were required to pass in English Language, an exam designed to address the problem noted by the first Oxford examiners. This exam did include reading components -a comprehension section and an exercise in summary – but the emphasis, again, was not on students’ ability to read as such, but rather on their ability to express their understanding of what they had read. Thus, the British tradition examined reading and writing together, and this feature distinguishes it most sharply from the testing tradition that developed in the United States.

Since 1917 many changes have been made in the British examination system, reflecting an increased concern with a wider spectrum of the population. In 1947 the School Certificate was superceded by the General Certificate of Education (GCE), which allowed both teachers and students more latitude. But although its proponents had liberal intentions, the GCE retained the élitist character of its antecedents: it was meant to be taken only by those who were likely to pass, and since such candidates were overwhelmingly middle-class, working-class students often got no certificate at all. Later, an alternative Certificate of Secondary Education (CSE) was introduced to address the needs of less academically inclined, and more often working-class, students; but this led to a two-tiered system in which the GCE unquestionably represented the upper tier. In recent years both these certificates have been superceded by the General Certificate of Secondary Education (GCSE), a qualification which presents a more radical departure from tradition: it is open to students of a wider range of abilities, and it depends less on exams and more on the assessment of projects completed over an extended period.

These changes did not affect those countries that were British colonies or protectorates at the time when the School Certificate was developed. As early as 1862, the Cambridge Local Examinations Syndicate received appeals to make its exams available in the colonies, and the first Cambridge overseas exams were held in Trinidad in 1864. With the introduction of the School Certificate, the Cambridge Syndicate instituted School Certificate exams in the colonies, and then, as independence approached, it fostered the development of regional examination boards to take over its responsibilities. The West African Examinations Council, described in Chapter 4, is one such board. These regional boards have revised their syllabi to reflect local conditions and interests, but the basic examination model has remained much the same, especially with respect to English Language. In some countries this model has been extended to other levels of the education system: in Chapter 6, Allen describes its use at the junior secondary level in Zimbabwe, and we have observed its influence on exams for primary teacher-training institutions and universities in Nigeria. Thus, despite recent changes in Britain, this older tradition flourishes in many parts of the world; and it has continued not only to examine reading and writing in one paper but to require candidates to write their own responses to reading comprehension tasks. We present material from the British tradition in Figure 1.1; it is taken from the West African School Certificate and is quite typical of what are called comprehension tests throughout the British Commonwealth.

The American tradition of testing springs from a more egalitarian conception of education: however inconsistently applied in practice, the American ideal has always, since the time of Jefferson, been of a public education system that would make literacy available to all (Graff 1987). In the late nineteenth century, however, as changing patterns of immigration produced a larger and more diverse school population, politicians and educators became increasingly aware of the system’s inadequacies. The problem was diagnosed as one of efficiency. Edward L. Thorndike, in a government-commissioned report, complained in 1907 that

many pupils are held back unduly. … The work which they are given to do but fail to do is unsuited to them. (Quoted in Jonçich 1968, p. 301)

He suggested that the solution lay in

special classes, careful regulation of promotion, the substitution of industrial and trade schools or courses for the regular school, and the like. (Quoted in Jonçich 1968, p. 302)

But how could such differences in education be reconciled with the egalitarian ideal? The answer lay in making the necessary distinctions independent of wealth and social status – and also of teachers’ individual prejudices – by basing them instead on intrinsic ability; and this ability should be judged on ‘indices of merit that [were] fair and objective, standardized, competitive – and quantified’ (Jonçich 1968, p. 295).

The initial inspiration for developing such indices came from the tests developed by Alfred Binet, whose aim was to assess ‘a beautiful native intelligence [freed] from the trammels of the school’ (1908, quoted in Gould 1981, p. 151). His American followers seized upon the notion that intelligence was a separate faculty, independent of schooling and cultural background, that could be measured by a simple test. Unlike Binet, who used these tests only for diagnosing individual difficulties, psychologists such as Robert Yerkes and Lewis Terman developed tests to be used on a massive scale. During the First World War, intelligence tests were used in assigning nearly two million army recruits to specialised units. Similarly, after the war, the Stanford-Binet tests were developed and widely used to place elementary and high school students in what were considered to be homogeneous groups (Resnick 1982).

At about the same time, reading tests were developed along similar lines. Indeed, Thorndike, who was their leading proponent, claimed that reading and intelligence tests measured much the same thing, since, as he put it, ‘reading is reasoning’ (Johnston 1984, p. 148). Like Binet, he sought to develop instruments that would assess reading as an independent variable, and when he published his scaled tests of reading comprehension, he pointed out that not only were they easily scored, producing numerical results independently of individual judgements, but they minimised demands on test takers’ powers of expression (Thorndike 1915). Thorndike did not himself use a multiple-choice format but it was being developed at that time for use on intelligence tests by Arthur Otis who worked with Terman (Chauncy and Dobbin 1969); and it was not long before this format was extended to reading tests.

In the 1930s the arrival of machine scoring assured the triumph of the multiple-choice format. The consequence has been that in American tests reading and writing have been kept firmly separate. Tests of the two skills are independently developed and administered, and test takers, in demonstrating the one skill, do not, in principle, need to demonstrate the other (of course a writing test requires at least some reading, though it is common practice for given topics to be read aloud). Reading tests are much easier to administer and score and so have come to be viewed in public discourse as the primary indices of literacy. The basic format of these tests is illustrated in Figure 1.2. This material is taken from the University of Michigan’s Examination for the Certificate of Proficiency in English, a test used with non-native speakers throughout the world.

When test materials from the two traditions are placed together like this, the differences are readily discernible, and they often arouse hostile reactions. Those trained in the American tradition are likely to see the British tradition of test as élitist and irrelevant, while those trained in the British style may view the American form as reductionist and arbitrary; and students often experience difficulty in moving from one tradition to the other. For example, when well-educated Jamaicans enter an American university, they are sometimes misplaced in a remedial English course on the basis of exam performance; they complain that they are unable to decide among all the choices provided on a multiple-choice task. Similarly, students trained in the American tradition initially have considerable difficulty in composing responses to the British type of task. We have observed, however, that individuals who do reasonably well on one style of test ordinarily adapt rather quickly to the other. The reason, we suggest, is that beyond differences of format are basic similarities, both in the kind of text used and in what test takers are required to do with it.

If we consider the two sample passages apart from the tasks, we cannot readily tell which is American and which is British, while they deal with different subject matter – one with botany, the other with medical technology – the way the subject-matter is handled is strikingly similar. Both passages present information in a deliberately neutral way: while evaluative responses are described, there is no indication that they are shared by the writer. Nor is there any orienting material ...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• List of Contributors
• Editors’ Acknowledgements
• Publisher’s Acknowledgements
• Introduction
• Part I Theoretical Considerations
• Part II Reading Tests
• Part III Literacy Assessment
• Part IV Policy Implications
• Bibliography
• Index