Assessing Chinese Learners of English

Language Constructs, Consequences and Conundrums

Guoxing Yu, Yan Jin, Guoxing Yu, Yan Jin

• Publisher: Palgrave Macmillan
• Year: 2016
• Print ISBN: 9781349553976, 9781137449771
• eBook ISBN: 9781137449788
• Topic: Languages & Linguistics
• Subtopic: Teaching Languages
• Index: Languages & Linguistics

This volume gathers researchers from around the world endeavouring to better understand a number of perennial issues in assessing Chinese learners of English, covering topics such as students' test performances, interactional competence and lexical knowledge, students' motivation, teachers' attitudes and assessment policy changes.

1

Assessing Chinese Learners of English: The Language Constructs, Consequences and Conundrums – An Introduction

Guoxing Yu and Yan Jin

1.1 Introduction

In this chapter, we introduce the context and the rationale for the edited volume on assessing Chinese learners of English as a foreign language. In specific, we will discuss the constant challenges and conundrums in understanding the language constructs, the various assessment methods, Chinese learners’ preparation for and performance on English language tests, as well as the wide-reaching consequences of assessing Chinese learners of English. This introduction chapter also presents the logic of the sequence of the individual chapters and the overall organisation of the edited volume. The central question that we keep asking ourselves throughout this edited volume – What have we learned from research on assessing Chinese learners of English? – helps us to draw together, though very much tentatively, the implications of the findings of the studies reported in this volume which represents our collective endeavours as researchers to contribute to solving part of the conundrums.

1.2 The context and rationale

Understanding how Chinese students are being tested, how they are preparing or being prepared for different purposes, at different educational levels, and for different tests, will lend some insight into not only the validity of the tests per se but also the wider issues in relation to local and global impacts of the tests. English language assessment as a social practice is hugely complex in terms of assessment policies, practices and hence its impacts at different educational levels. The uses, misuses and abuses of English language assessment transcend the traditional studies focusing exclusively on the reliability and validity of tests. The policies and practices of assessing Chinese learners of English as a foreign language are intertwined with the social, political and educational systems in which the tests operate; as a result, the impacts of English language tests are social, political and educational in nature. As Ross (2008) rightly pointed out: “Language assessments for high-stakes purposes invariably involve policy making at some level. Language assessment policy analysis requires an appreciation of the social, economic, and historical contexts in which assessment policies are introduced, modified, extended, or abandoned” (p. 5).

To understand the current status of English language assessment in China, it is imperative and inevitable that first and foremost we take into account the history of Chinese imperial examinations and the impact of the examinations on the present social, political and education systems. It is widely accepted that China is the origin of large-scale examinations of individuals’ abilities for selection purposes (Bowman, 1989; Martin, 1870). Although the system of imperial examinations was abolished in 1905, its influence is still permanently embedded in the present education and assessment systems in China. Sit for the exam and fight for the rank – was and still is not only a manifestation of the nature of competitiveness in all aspects and levels of educational assessment in China but also one of the key mechanisms used by the Chinese government to manage resources and social mobility. Issues in educational access, equity and quality (Davey, Lian, & Higgins, 2007; Hannum, An, & Cherng, 2011; Rong & Shi, 2001; Wang, 2008), social justice and political centralism (Feng, 1995) are the main criticisms of the selection purposes of education assessment in China (see Yu & Jin, 2014).

Compared to imperial examinations, English language assessment, which probably started in the 1860s in China (Cheng, 2008; Fu, 1986), is relatively a “small baby” in terms of its history. However, in terms of its size, scope and reach of influence, English language assessment is colossal; it now permeates every aspect and moment of Chinese society. A phenomenal number of Chinese learners of English, from nursery to higher education institutions and beyond, are taking English language tests. English is the compulsory school subject from year three almost everywhere in China, rural and urban. English is one of the three key subjects (the other two being Chinese and mathematics) in Gao Kao – the national university entrance examinations. College English Test (CET) has millions of test takers every year, e.g., in 2012 alone it had 18 million test takers. There has been a substantial increase in the number of Chinese taking international English language tests. In 2010 there were over 300,000 Chinese who took International English Language Testing System (IELTS), and a similar number of Chinese taking TOEFL iBT (Test of English as a Foreign Language, internet-based test). Educational Testing Service, the owner of TOEFL iBT, reported a 19% increase of Chinese test takers in 2011 from 2010, and a further 32% increase in 2012 from 2011. According to a recent ETS publication (Liu, 2014), Chinese test takers represent about 20% of the TOEFL iBT population. Test preparation courses, especially for TOEFL iBT and IELTS, have been the major income sources of some public listed Chinese companies such as New Oriental at NYSE and Global Education and Technology at NASDAQ which was purchased by Pearson in December 2011. To gain a sense of the scale of English language learning and assessment, this TED video by Jay Walker is particularly telling:

http://www.ted.com/talks/jay_walker_on_the_world_s_english_mania.html

English language assessment affects not only millions of people within China but also has far-reaching global effects, academically and financially, on recruitment and education of Chinese students in English-speaking universities. According to the UK Council for International Student Affairs (UKCISA), there were 428,225 international students in UK higher education institutions in the 2010–2011 academic year; they made up of 48% of full-time research degree students, 70% of full-time taught postgraduates, and 14% of full-time first degree students. Several UK universities recruited a substantial percentage of their students from overseas (e.g., LSE 66%, Imperial College 40%, UCL 38%, Cambridge 30%, Warwick 30%, and Edinburgh 28%). In the USA, there were 723,277 international students in colleges and universities in 2010–2011 academic year. In Australia, there were 184,830 international university students enrolled as of July 2012. In New Zealand, there were 22,811 international university students enrolled as of April 2012 (around 13% of university enrolments). China is the leading place of origin for international students enrolled in the aforementioned countries; and the number of Chinese students has been increasing substantially year on year. For example, UK higher education institutions enrolled 17% more Chinese students from mainland China in 2011/12 than 2010/11 (Source: UKCISA). As a well established but highly debatable, global practice, universities use students’ English language test results as one of the most important admission criteria (Rea-Dickins, Kiely, & Yu, 2007). As a result, we witness an increasing number of Chinese taking TOEFL iBT and IELTS year on year as we described above. These English language tests shape and are shaped by the globalising higher education sector. The English language abilities of Chinese students have an impact on the extent to which the students can access and benefit from their higher education experiences, and affect their lives as students and the overall quality of higher education.

In addition to Chinese from the mainland, there are similarly a large number of Chinese learners and test takers of English in Hong Kong and Taiwan who share in many aspects the cultural, linguistic and educational traditions and values as their mainland Chinese counterpart. In this edited volume, we use Chinese or China as terms associated with the Chinese language and people, rather than as a political entity, unless otherwise stated explicitly.

Among policy makers, curriculum designers, material writers, English language instructors, and assessment professionals, at all educational levels, there are substantial and sustainable interests in understanding the issues surrounding the assessment of Chinese learners of English. A number of academic publications have recently appeared or are under preparation to address these issues. For example, Researching Chinese Learners: Skills, Perceptions and Intercultural Adaptations, (Editors, Jin & Cortazzi, 2011, Palgrave), English Language Assessment and the Chinese Learner, (Editors: Cheng & Curtis, 2009, Routledge), English Language Education and Assessment: Recent Developments in Hong Kong and the Chinese Mainland, (Editor: Coniam, 2014, Springer Singapore). Assessment in Education (Taylor and Francis) published a special issue on the assessment of Chinese learners of English, edited by Yu and Jin (2014). Language Assessment Quarterly (Taylor and Francis) published a special issue on English language assessment in Taiwan (Guest Editor, Vongpumivitch, 2012). Another special issue on high-stakes English language testing in China is under preparation by Professors David Qian (a contributor to this edited volume) and Alister Cumming (OISE, University of Toronto), to be published by Language Assessment Quarterly. Together, these publications make incremental contributions to understanding the constructs and consequences of assessing Chinese learners of English.

1.3 The chapters

Given the nature and scope of the complexity of the issues in assessing Chinese learners of English, no single volume would be able to capture all. This edited volume is intended to provide some insights into language constructs of assessment, various assessment methods and innovations, Chinese students’ preparation for and performance on a number of English language tests, and consequences of assessment. These chapters are arranged broadly in line with the fundamental questions that have been continuously challenging the field of language assessment: who, what, how and why to assess.

• What are the characteristics of Chinese learners we are assessing?
• What makes Chinese learners of English different from learners of other first languages?
• To what extent do the social, political and educational systems in China affect the students’ learning motivation and test preparation strategies?
• How are Chinese learners being assessed?
• What are the underlying language constructs of assessment?
• What is Chinese learners’ performance in English tests, and what affects their performance?
• What are the consequences of assessment?
• What are the policy and pedagogical implications of requiring students to reach a certain English language proficiency level before they are allowed to graduate?
• How do different stakeholders cope with assessment policy changes? For example, how do teachers implement formative assessment in response to government assessment mandate?

These are the main questions that the research studies reported in this edited volume endeavour to address, from different perspectives. The authors of the chapters come from Australia, mainland China, Hong Kong and Taiwan, UK and USA. Some are seasoned researchers who have published widely on language assessment, and some are recent PhD graduates; however, it is our shared experience in assessing and working with Chinese learners of English that brings us together to address collectively a number of perennial issues in assessing Chinese learners of English.

Below we briefly introduce the focus of each chapter.

In Chapter 2, Hamp-Lyons, as one of the main architects of School-Based Assessment (SBA) in English in Hong Kong, reflected on the aims and structure of SBA, and the challenges and issues in developing and implementing SBA in this fervently examination-oriented society. SBA is a typical example of Hong Kong government’s initiative to address the dominant culture of summative assessment in schools. As a kind of teacher-based assessment, SBA is intended to serve both summative and formative purposes, in the high-stakes English Language examinations for secondary school students at age 15–16/16+. SBA was introduced by the government, seeking a balance between summative and formative assessment to make a major educational shift in assessment; however, it met with strong resistance from teachers initially. Hamp-Lyons explained that some of the cultural and political influences helped and hindered the effective implementation of SBA in Hong Kong. She argued that a rigorous teacher’s professional development programme and a carefully developed and validated set of assessment criteria and standards are two essential components for successful implementation of assessment innovations. Although SBA is increasingly being accepted by teachers and other stakeholders (including researchers), there are a number of issues that have remained problematic in the nearly ten years of this innovation. In this Chapter, Hamp-Lyons highlighted two of these issues. The first issue is related to the planning or preparation time for Group Interaction tasks in SBA English. Thanks to the “test prep” culture that is “ubiquitous” in Hong Kong, variation in planning time for the Group Interaction tasks could potentially pose threats to the validity of the tasks. The second issue has something to do with the different interpretations of “fairness” – fairness often viewed as equivalent to reliability in the examination-oriented societies, and fairness in terms of opportunity for learning, an opportunity for every student to develop and demonstrate their knowledge and ability to the best of their capabilities. This chapter clearly demonstrates what Ross (2008) argued, which is that language assessment policy analysis requires an appreciation of the broader social, cultural and political contexts in which educational assessment policies or innovations operate, but more importantly, Hamp-Lyons presented a very interesting and thought-provoking first-hand, first-person narrative of the challenges and issues that SBA English faced and still faces in the nearly ten years of implementation.

Following on the same topic, but from a more technical perspective of the implementation of SBA English, Lam reported in Chapter 3 a validation study on Group Interaction tasks. Lam observed that there was a considerable variation in the amount of planning or preparation time given to students for Group Interaction tasks – one of the two continuing challenges that SBA English faces as Hamp-Lyons pointed out in Chapter 2. He looked at how the task was implemented in schools and the authenticity of engagement in student interactions. Based on conversation analysis of student interactions and the stimulated recall interviews with students and teachers, Lam reported that the spoken discourse of the Group Interaction tasks exhibited some superficial features of authentic interactions and that the students’ pre-task planning activities revealed the “contrived and pre-scripted nature” of such interactions. The interactions observed were essentially a “staged performance of pre-scripted dialogues”, in other words, “the product of students acting out a composed dialogue based on their knowledge and perceptions of what interactional competence is, rather than students’ spontaneous performance of the competence that involves moment-by-moment monitoring of and contingent reaction to each other’s talk in real time”. The findings of this study can have important implications for designing SBA Group Interaction tasks and the assessment criteria. More generally, as group and paired speaking tasks aiming to assess students’ interactional competence often have “planning time” as a key task condition, the findings of this study offer further evidences on the effects of planning time on the features of interactions in such tasks. The next three chapters (4–6) continue the same topic on speaking assessment. Chapter 4 reports on the communication strategies used by test takers in computer-based and face-to-face discussion tasks, Chapter 5 on test takers’ use of single words and multi-word clusters in a paired speaking test, and Chapter 6 on test takers’ use of formulaic sequences (similar to multi-word clusters in Chapter 5) in a monologue story-retelling task.

In Chapter 4, Jin and Zhang reported a small-scale exploratory study investigating the comparability in test takers’ use of communication strategies in two different modes of speaking tasks. Data were collected from six pairs of test takers who sat both the computer-based and the face-to-face College English Test – Spoken English Test (CET-SET). Like Lam in Chapter 3, Jin and Zhang conducted conversation analysis of test takers performance in the two discussion tasks, and found a high level of similarity in both the quantity and variety of communication strategies used by the test takers. They also reported that test takers were generally capable of making effective turn-taking decisions in the computer-based discussion task. Furthermore, in both computer-based and face-to-face discussion tasks, test takers who were awarded a high score on communicative effectiveness made more frequent use of interaction strategies ...

Table of contents

• Cover
• Title
• Copyright
• Contents
• List of Figures, Tables and Appendices
• Foreword by Cyril J. Weir
• Acknowledgements
• Notes on Contributors
• 1 Assessing Chinese Learners of English: The Language Constructs, Consequences and Conundrums – An Introduction
• 2 Implementing a Learning-Oriented Approach within English Language Assessment in Hong Kong Schools: Practices, Issues and Complexities
• 3 Contriving Authentic Interaction: Task Implementation and Engagement in School-Based Speaking Assessment in Hong Kong
• 4 The Impact of Test Mode on the Use of Communication Strategies in Paired Discussion
• 5 Face-to-Face Interaction in a Speaking Test: A Corpus-Based Study of Chinese Learners’ Basic Spoken Vocabulary
• 6 Features of Formulaic Sequences Used by Chinese EFL Learners in Performing a Story Retelling Assessment Task
• 7 Assessing Incidental Vocabulary Learning by Chinese EFL Learners: A Test of the Involvement Load Hypothesis
• 8 Chinese Users’ Perceptions of the Use of Automated Scoring for a Speaking Practice Test
• 9 Project-Based Group Assessment in the Second Language Classroom: Understanding University Students’ Perceptions
• 10 Chinese EFL Students’ Response to an Assessment Policy Change
• 11 Students’ Voices: What Factors Influence Their English Learning and Test Performance?
• 12 Standard English or Chinese English? Native and Non-Native English Teachers’ Perceptions
• 13 The Power of General English Proficiency Test on Taiwanese Society and Its Tertiary English Education
• 14 Twenty Years of Cambridge English Examinations in China: Investigating Impact from the Test-Takers’ Perspectives
• Index