Testing Second Language Speaking

Glenn Fulcher

• Publisher: Routledge
• Year: 2014
• ISBN:
  9781317873679
• Edition: 1
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

The testing and assessment of second language learners is an essential part of the language learning process. Glenn Fulcher's Testing Second Language Speaking is a state-of-the-art volume that considers the assessment of speaking from historical, theoretical and practical perspectives.The book offers the first systematic, comprehensive and up-to-date treatment of the testing of second language speaking. Written in a clear and accessible manner, it covers:

• Explanations of the process of test design
• Costing test design projects
• How to put the test into practice
• Evaluation of speaking tests
• Task types for testing speaking
• Testing learners with disabilities

It also contains a wealth of examples, including task types that are commonly used in speaking tests, approaches to researching speaking tests and specific methodologies that teachers, students and test developers may use in their own projects.

Successfully integrating practice and theory, this book demystifies the process of testing speaking and provides a thorough treatment of the key ethical and technical issues in speaking evaluation.

Chapter 1
The history of testing second language speaking

It is often remarked that language testing is one of the youngest fields of research and practice in Applied Linguistics (Alderson, 1991a). And the theory and practice of testing second language speaking is the youngest subfield of language testing. During the heyday of modern language testing research through the First World War and into the 1920s, little if any attention was paid to testing speaking. Despite some notable exceptions, it was not until the Second World War that the testing of second language speaking became a focus of interest (Fulcher, 1997).

In this first chapter we will first consider the testing of speaking up to the Second World War, look at how the war changed the way in which speaking was taught and tested, and then trace the movement of innovation in a military context to universities and schools. This chapter shows that it is important for us to understand the intimate connection between the development of speaking tests and political/military language needs; political and military events have had a deep impact upon the form and scoring of many modern speaking tests. We will also see that the problems encountered with speaking tests from the very early days have not disappeared. By placing our consideration of testing speaking into a historical context we can more fully appreciate why speaking tests that we are familiar with today look the way they do.

Testing Speaking Before 1939

In the United States where most of the language testing research was being conducted, there was an overwhelming concern with achieving reliable scores from tests. ‘Reliable’ meant that the scores from tests were consistent over a number of administrations, assuming that the conditions are the same and there is no impact on the scores from learning, familiarity or fatigue. It was argued that the testing of speaking could not produce reliable scores because the process relies upon judges who will be swayed by many uncontrollable factors (see Chapter 6). Learners would therefore get different, or inconsistent, scores depending on the person making the judgement and the range of factors that may impact upon the speaking test at the time (Heyden, 1920). It was also considered impractical when testing large numbers of learners.

Although the phrase ‘oral test’ appears in language testing prior to the war, it did not mean that learners were required to speak in the test. Rather, it referred to the testing of pronunciation, usually requiring the test taker to write down the pronunciation of a written word using phonetic script. For example, in 1913 the Association of Modern Language Teachers instituted a committee to consider university admissions tests in French, German and Spanish that had an aural and oral component (Spolsky, 1995: 35). A test of speaking was abandoned because of problems with reliability. It was admitted that ‘no actual oral test is included in this examination’. But it was also claimed that ‘no candidate could pass it who had not received abundant oral, as well as aural training’ (Committee on Resolutions and Investigations, 1917: 252). The aural component of the test was a 10-minute dictation, and writing answers to questions spoken by the examiner. Similarly, in his pioneering work on language tests for New York Schools, Wood (1927: 96) acknowledges the importance of using ‘conversational materials’ that can be used to hold individual speaking tests with students as a valid method of assessing students, but remained with paper and pencil tests because speaking tests would be ‘subjective’ and logistically difficult to put into operation.

We will return to these two problems again throughout this book. The concerns of early language testers have not disappeared. Reliability and practicality issues áre major drivers of research into semi-direct tests of speaking, and the automatic (machine) scoring of speech samples today.

Language testing practitioners avoided the complex problem of testing speaking until 1926. Instead they concentrated on the ‘new-type’ multiple choice tests as reliable, objective measures of language ability. The first true speaking test used in North America was The College Board’s English Competence Examination, introduced in 1930 for overseas students applying to study at US colleges and universities (College Entrance Examination Board, 1929). The 1930 test was specifically designed by the College Board at the request of the Association of College Registrars in 1927 (Spolsky, 1995: 55), and the request was made because of the 1926 requirement of the United States government (Commissioner of Immigration) that schools and colleges indicate the ‘exact knowledge of English language a student must have before he can be admitted’. The College Board realised that it was essential to test whether a student was capable of speaking intelligibly in an academic context, and their solution was the first academic speaking test in a foreign language.

This format of the whole test was:

1. Reading I
   Four short passages, the first a simple narrative, the second a historical passage or a text of some topical interest, the third a critical discussion of a social issue, and the fourth a scientific text. Questions were primarily true/false.
2. Reading II
   One longer passage on a critical or theoretical topic that would present the opportunity to test understanding of key ideas.
3. Dictation
   One regular dictation, and one to be produced from memory after listening.
4. Speaking
   A conversation with ten topics prepared for the examiner. The criteria for assessment were:

• Fluency
• Responsiveness
• Rapidity
• Articulation
• Enunciation
• Command of construction
• Use of connectives
• Vocabulary and idiom

The examiner graded each criterion on a three-point scale of: proficient, satisfactory and unsatisfactory. Interestingly, the examiner was also asked to record whether the test taker was shy. This is an early sign of interest in individual differences that may be a threat to the valid interpretation of test scores.

• 5. Writing
  A 250-300 word essay on a selected topic.

The purpose of section 4 of the 1930 test was to ensure that a potential student in a US university could express him or herself intelligibly in spoken English. The criteria for assessment were not articulated, and we do not know anything about the training provided to examiners. Nevertheless, we may observe the interest in attempting to define what is important about speaking - defining the construct through enumerating the key features for assessment - and considering at least one factor that may affect scores but which is not related to the construct (construct irrelevant variance).

This innovative test design was not only the result of a very practical requirement to know whether a student would be able to speak once enrolled on an academic programme in a North American university. It was also designed to play an important role in US immigration policy.

Inter-war immigration policy was itself influenced by intelligence testing. During the First World War over 1.75 million men had taken the Army Alpha or Beta tests (the world’s first mass-produced intelligence tests), or individual assessments for those whose intelligence was too low to attempt the Beta tests. The results were published by race (Yerkes, 1921), and the conclusions drawn were that:

• (a) The average mental age of white Americans was in danger of decline because of breeding with more feeble-minded peoples. Yerkes calculated the average mental age as approximately 13. This estimate is just slightly higher than the definition of a ‘moron’:

Going down the scale of mental development it has become customary to call those mental defectives who have a mentality of from eight to twelve years, morons; those who range from three to seven, imbeciles; and those of two or under, idiots.

(Goddard, 1919: 60)

• (b) Many European immigrants, particularly those from Southern or Eastern Europe, had extremely low innate intelligence compared with those from Northern Europe.
• (c) Coloured people are the least intelligent, their level of intelligence being directly correlated with the deepness of the colour of their skin.

Despite serious problems with the tests themselves, data collection and interpretation (see Gould, 1981), the First World War experiment with testing had a major political impact in the United States. On 6 December 1923 Calvin Coolidge said that:

American institutions rest solely on good citizenship. They were created by people who had a background of self-government. New arrivals should be limited to our capacity to absorb them into the ranks of good citizenship. America must be kept American. For this purpose, it is necessary to continue a policy of restricted immigration. It would lie well to make such immigration of a selective nature with some inspection at the source, and based either on a prior census or upon the record of naturalization. Either method would insure the admission of those with the largest capacity and best intention of becoming citizens. I am convinced that our present economic and social conditions warrant a limitation of those to be admitted. We should find additional safety in a law requiring the immediate registration of all aliens. Those who do not want to be partakers of the American spirit ought not to settle in America.

He reiterated the view that America should remain American when he signed the 1924 Immigration Act, designed to keep the unintelligent foreigner out of the United States. He was merely responding to the fear of the day, the popular conception that allowing additional immigration would reduce the intelligence of the American population. This view was most clearly expressed by Brigham (1923, quoted in Gould, 1981: 260):

The decline of American intelligence will be more rapid than the decline of the intelligence of European national groups, owing to the presence here of the negro. These are the plain, if somewhat ugly, facts that our study shows. The deterioration of American intelligence is not inevitable, however, if public action can be aroused to prevent it. There is no reason why legal steps should not be taken which would insure a continuously progressive upward evolution.

The steps that should be taken to preserve or increase our present intellectual capacity must of course be dictated by science and not by political expediency. Immigration should not only be restrictive but highly selective. And the revision of the immigration and naturalization laws will only afford a slight relief from our present difficulty. The really important steps are those looking toward the prevention of the continued propagation of defective strains in the present population.

Brigham may not have won the battle to ‘prevent the propagation of defective strains’ of human being in the United States, but he did get a restrictive immigration law that was based on strict eugenic principles (Brigham later repudiated these views when he moved to Princeton University). The Act was the first to introduce an immigration quota law, establish a preference quota system, non-quota status and consular control system. Immigration to the United States from the target nationalities fell swiftly. But there was a loophole in the 1924 Immigration Act. It allowed visas to be issued to non-quota applicants who had been accepted to study at a school, college or university in the United States. As a result the number of applications for student visas immediately began to rise dramatically. Spolsky (1995: 55) reports that in 1926 the Commissioner General of Immigration wrote that:

The experience of the bureau in the past two and one half years is to the effect that many non-quota immigrant students gain admission to the United States totally unfit, because of insufficient knowledge of the English language.… Therefore, it is requested that all schools indicate in the certificate of admission the exact knowledge of the English language a student must have before he can be accepted.

The College Entrance Examination Board was given the task of providing this test by the American Association of Collegiate Registrars. It was used effectively until 1935, when funds to operate the test ran out, and the item pool had been over-exposed.

The fact that immigration needs led to the development of the1930 speaking test shows the link that often exists between developments in language testing and political initiatives. These pressures still exist today, in the development and use of tests to regulate immigration, or determining the residential status of asylum seekers (Hawthorne, 1997; Shohamy, 2001).

In the United Kingdom the story was somewhat different. While testing speaking in the United States was frequendy seen as ‘desirable but not feasible’, the University of Cambridge Local Examinations Syndicate (UCLES) were not hampered by concerns over reliability or measurement theory. A sub-test of spoken English was included in the Certificate of Proficiency in English at its introduction in 1913 (Roach, 1945). The examination consisted of:

1. Written
   • (a) Translation from English into French or German (2 hours)
   • (b) Translation from French or German into English, and questions on English Grammar (2½ hours)
   • (c) English Essay (2 hours)
   • (d) English Literature (The paper on English Language and Literature [Group A, Subject 1] in the Higher Local Examination) (3 hours)
   • (e) English Phonetics (1½ hours)
2. Oral:
   Dictation (½ hour)
   Reading and Conversation (½ hour) (marked only for pronunciation)

Unlike in the United States, the primary purpose of an examination in the United Kingdom was to support the syllabus and encourage good teaching and learning (Brereton, 1944). As speaking was valued, it was therefore included in the examination on these grounds alone. When the Lower Certificate (now referred to as the First Certificate in English, or FCE) was introduced in 1939 there was a speaking t...