Handbook of Second Language Assessment

Dina Tsagari, Jayanti Banerjee, Dina Tsagari, Jayanti Banerjee

• Publisher: De Gruyter Mouton
• Year: 2016
• eBook ISBN: 9781501500862
• Edition: 1
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

Second language assessment is ubiquitous. It has found its way from education into questions about access to professions and migration. This volume focuses on the main debates and research advances in second language assessment in the last fifty years or so, showing the influence of linguistics, politics, philosophy, psychology, sociology, and psychometrics. There are four parts which, when taken together, address the principles and practices of second language assessment while considering its impact on society. Read separately, each part addresses a different aspect of the field.

Part I deals with the conceptual foundations of second language assessment with chapters on the purposes of assessment, and standards and frameworks, as well as matters of scoring, quality assurance, and test validation.

Part II addresses the theory and practice of assessing different second language skills including aspects like intercultural competence and fluency.

Part III examines the challenges and opportunities of second language assessment in a range of contexts. In addition to chapters on second language assessment on a national scale, there are chapters on learning-oriented assessment, as well as the uses of second language assessment in the workplace and for migration.

Part IV examines a selection of important issues in the field that deserve attention. These include the alignment of language examinations to external frameworks, the increasing use of technology to both deliver and score second language tests, the responsibilities associated with assessing test takers with special needs, the concept of 'voice' in second language assessment, and assessment literacy for teachers and other test and score users.

Dina Tsagari and Jayanti Banerjee

1The handbook of second language assessment

1Introduction

In stating the aims and scope of the series of which this volume is a part, the series editors describe Applied Linguistics as a field that “deals with the theoretical and empirical investigation of real world problems in which language and communication are a central issue” (Brumfit 1997: 93). They view it as a “problem-oriented and problem-solving activity” (Brumfit 1997: 93). These statements encapsulate the work of researchers and practitioners of second language assessment as it helps to solve various real world problems. For example, at the level of individuals such work identifies and certifies degrees of second language competence that can be used for decisions on further educational or professional careers. At the level of institutions like the education system it evaluates teachers. At the level of organizations it helps to select or promote staff with the necessary foreign language and communication skills. Finally, at the level of states it helps in the development of language tests as a basis for granting citizenship to immigrants. In many cases, assessment has a gatekeeping function. It therefore has to be done responsibly and with care.

Arguably, there is nothing more ‘real world’ than a second language assessment tool. For second language assessment is a process of gathering information about how much language a learner knows and can use. The first step in designing a language test is the definition of what it means to be proficient in the language (be it English, German, Korean, Igbo, or Bengali) and, as such, both language and communication are central to language testers’ work. As a result, language testing researchers are often at the leading edge of investigations into second language ability.

2History of second language assessment

The professional roots of the field of second language assessment can be traced to the publication of Robert Lado’s (1961) book Language Testing. At that time the approach to assessment was primarily structural. It reflected the way that language was taught – as a set of discrete elements. As views of language teaching and language use have evolved, the approach to language assessment has also changed. The psycholinguistic-sociolinguistic approach resulted in integrative, global measures such as cloze passages and dictation. These assessments required test takers to demonstrate their ability to use, in combination, the different aspects of language knowledge (grammar, vocabulary, spelling, etc.).

However, both the structuralist and psycholinguistic-sociolinguistic approaches to language and language learning were criticized for their restrictive view of how language works. Canale and Swain (1980) presented an alternative, the communicative approach. They showed how language is used to communicate meaning. This involves: knowledge of vocabulary, morpho-syntax, and phonology (grammatical knowledge); knowledge of the rules of language use, as demonstrated by register choice, and pragmatic inferences, and knowledge of the rules of discourse (sociolinguistic competence); and, knowledge of strategies, such as gestures, that can maintain communication (strategic competence). For the last thirty-five years, this view of language has informed approaches to language teaching and assessment. Tests are generally expected to provide context for the tasks and to engage the test takers in processing and producing realistic discourse.

As well as representing a long-standing approach to second language teaching and assessment, Canale and Swain’s (1980) paper is important because of the explicit way in which it draws in detail on major thought trends in linguistics at the time including Hymes’ (1972) discussion of language use as encompassing both grammaticality and sociolinguistic appropriacy as well as van Ek’s (1976) notion of language functions. This reflects a persistent relationship between linguistics and second language assessment. As Davies (1990) comments, language assessment is a “chief way of applying linguistics, perhaps more so than most other activities in applied linguistics, in that the selection of material for language tests and the statements made in terms of the results achieved [a]re always of linguistic import, making statements about the nature of language and language learning (p. 74).”

3Aims of the volume

Indeed, second language assessment is a transdisciplinary field that is influenced certainly by linguistics but also by politics, philosophy, psychology, sociology and psychometrics. For instance, in order to make a judgment about a learner’s vocabulary proficiency, we need to understand how words and formulaic phrases are learned, particularly learners’ sensitivity to patterns and sequences (Ellis, 2002) and the principles of Zipfian distribution (Sorrell, 2013). This seemingly straightforward aim of assessing vocabulary draws on lessons learned from psychology and mathematics.

In a very different example, the fields of politics and philosophy illuminate the agendas that motivate second language testing practices and show their consequences. McNamara and Roever (2006) expose in detail how language assessments can perpetuate unquestioned social values and policies. They show how language tests are used to exclude asylum seekers and to deny immigrants access to work opportunities.

The central aim of this volume is to capture the main debates and research advances in second language assessment over the last fifty or so years, showing the influence of linguistics, politics, philosophy, psychology, sociology and psychometrics. It provides:

• –an overview of current theories and research of language assessment by considering the philosophical and historical underpinnings of assessment practices,
• –a balanced evaluation of the major positions and approaches, including the examination of the increasingly important social and political context of language assessment,
• –practical applications of language assessment theory for practitioners, and
• –a theoretical basis for considering and analyzing language tests and understanding administration and training issues involved.

As such, the volume charts the new avenues that second language assessment has moved into by considering not only issues of design, administration and training involved in the assessment of second language but also the growing range of public policy, social and ethical issues. It also considers the various assessment and linguistic contexts while emphasis is put on the need for a careful consideration of various groups of test takers, e.g., young, adult and those with learning and other disabilities as well consideration of subtle dimensions of language proficiency such as the assessment of pragmatic competence.

The authors bring to their topics a high level of expertise and represent diverse methodological approaches, and geographic regions of the world. Each chapter includes a brief history of the topic to provide the context, discusses the major key issues, research approaches and findings, and ends with a forward-looking section where authors discuss what they view to be the most important challenges and research steps that their fields need to take.

4Overview of the volume

The volume is organized around four parts each comprising five to eight chapters. Part I, entitled ‘Conceptual foundations of second language assessment’ contains four chapters focusing on the history of language assessment, purposes, standards, frameworks and approaches and other characteristics of assessment such as quality, in terms of reliability, validity and impact.

Part II ‘Assessing second language skills/areas’ includes eight chapters focusing on the assessment of traditional skills such as listening, reading, speaking and writing as well as emerging language areas such as the assessment of pragmatic and intercultural competence, fluency and translation and interpretation. The five chapters in Part III, ‘Second language assessment contexts’, address exciting areas in the field such as learning-oriented assessment, dynamic assessment, assessment for the workplace and immigration, citizenship and asylum. Part IV explores new avenues by bringing together a collection of six chapters that draw on research and developments in a multitude of areas such as aligning language assessments to frameworks, using technology, assessing young learners and students with learning and other disabilities, the importance of ‘voice’ in second language assessment and teachers’ assessment literacy and professional development.

Part I begins with Liz Hamp-Lyon’s contribution (Chapter 2), which shows how, in recent years, the terms assessment, testing, examining, and evaluation have become increasingly intertwined. New terms such as alternative assessment, [X]-based assessment, and related concepts/terms such as learning-oriented language assessment, dynamic assessment and peer feedback have come to the fore. In her chapter Hamp-Lyons maintains a diachronic cast to these issues and builds up a picture of the present expanding view of the purposes of language assessment. Her chapter closes with a consideration of the opportunities and challenges for the future offered by these broader views of the purposes of language assessment for research and practice in language testing/assessment.

Given their increasing use by governments and international agencies as policy instruments, the need for clarity of ‘standards’ and ‘frameworks’ in language assessment is ever more pressing. This is the motivation behind the next chapter (Chapter 3) written by Glenn Fulcher entitled ‘Standards and frameworks’. Fulcher defines the terms, explores the relationship between models and frameworks, discusses standards-based testing as a new social phenomenon, and calls for a return to validation research that focuses on scores from tests and assessments, rather than standard-setting procedures.

In Chapter 4, Yasuyo Sawaki provides an overview of two key frames of reference for language test-score interpretation: ‘norm-referenced’ and ‘criterion-referenced’ testing (NRT and CRT respectively). Sawaki starts with a definition of the frames and their historical development. She then discusses aspects of measurement and quality through the provision of concrete examples and concludes with a discussion of issues, challenges, and future directions of language assessment research related to NRT and CRT.

The next two chapters in the volume (Chapters 5 and 6) focus on quality and impact in language assessment. Xiaoming Xi and Larry Davis (Chapter 5) begin by exploring the different ways in which test quality has been conceptualized such as validity, fairness, and test usefulness. They discuss frameworks for test validation, focusing particularly on the argument-based approach to test validation developed by Kane (2013).

Nick Saville and Hanan Khalifa (Chapter 6) explore the notion of impact by design in language assessment. They consider the effects and consequences resulting from the development and use of language assessments in societal contexts. Their discussion encompasses washback but expands the notion to deal with the wider effects and consequences which assessments have in society. Several recent case studies are used to illustrate the discussion and to present ways in which impact-related research can be integrated into operational validation procedures. The concluding section considers future challenges and ways in which this aspect of language assessment might develop in the future.

Part II presents eight impressive accounts of the assessment of language skills and areas. In the first of these chapters, Tineke Brunfaut (Chapter 7) describes the listening construct, a highly complex process, and gives an overview of different approaches to measuring listening ability. Different test techniques are illustrated with reference to existing listening tests, and factors that influence listening task difficulty such as text characteristics, task procedures and the interaction between listening texts and task responses as well as cognitive, affective and linguistic characteristics of the listener. The chapter concludes by discussing implications for listening assessment of the role of task- and listener-related characteristics, their interaction, and the resulting challenges for listening assessment practice and research.

In the following chapter, Fabiana MacMillan (Chapter 8), addressing the topic of assessing reading, draws on studies in written discourse analysis and second language reading and suggests a framework for the analysis of reading assessment items in terms of what specific reading skills are essential to successfully complete such tasks. MacMillan then discusses results of research applying this model to the analysis of large-scale standardized reading assessments. Her chapter concludes by showing that examining the number and type of lexical connections between reading comprehension questions and their keys is a helpful tool in evaluating the effectiveness of reading assessments.

In Chapter 9, Talia Isaacs discusses the challenges and inevitable trade-offs involved in assessing speech in ways that simulate test takers’ real world oral communication demands. Isaacs focuses particularly on how the construct of speaking ability has been defined and operationalized in assessment instruments, paying particular attention to sources of variability in L2 speaking performances as well as to the validity of human judgments and technology-mediated assessments.

Chapter 10, by Thomas Eckes, Anika Müller-Karabil and Sonja Zimmermann, provides a thorough review of the key conceptual and methodological issues involved in writing assessment. The chapter introduces basic concepts, components, and challenges of writing assessment, highlights a range of assessment purposes and settings, and discusses perspectives on issues of assessment design of the complex, multidimensional nature of the writing construct. The authors pay particular attention to the measurement issues concerning the role of human raters in scoring writing performances.

In Chapter 11, Kirby Grabowski presents several strands of the literature relevant to the assessment of pragmatic competence, a key aspect of communicative language ability. The chapt...

Table of contents

• Cover
• Title Page
• Copyright
• Preface
• Introduction to the handbook series
• Contents
• 1 The handbook of second language assessment
• I. Conceptual foundations of second language assessment
• II. Assessing second language skills/areas
• III. Second language asessment contexts
• IV. Issues in second language assessment
• Biographical notes
• Index
• Footnotes