Defining and Assessing Lexical Proficiency

Agnieszka Leńko-Szymańska

• Publisher: Routledge
• Year: 2019
• Edition: 1
• eBook ISBN:
  9781000712094
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

This comprehensive account of performance-based assessment of L2 lexical proficiency analyzes and compares two of the primary methods of evaluation used in the field and unpacks the ways in which they tap into different dimensions of one model of lexical competence and proficiency.

This book builds on the latest research on performance-based assessment, which has most recently pointed to the application of more quantitative measures to L2 data, to systematically explore the qualitative method of using human raters in assessment exercises and the quantitative method of using automatic computation of statistical measures of lexis and phraseology. Supported by an up-to-date review of the existing literature, both approaches' unique features are highlighted but also compared to one another to provide a holistic overview of performance-based assessment as it stands today at both the theoretical and empirical level. These findings are exemplified in a concluding chapter, which summarizes results from an empirical study looking at a range of lexical and phraseological features and human raters' scores of over 150 essays written by both L2 learners of English and native speakers. Taken together, the volume challenges existing tendencies within the field which attempt to use one method to validate one another by demonstrating their capacity to indicate very different elements of lexical proficiency, thereby offering a means by which to better conceptualize performance-based assessment of L2 vocabulary in the future.

This book will be of interest to students and researchers working in second language acquisition and applied linguistics research, particularly those interested in issues around assessment, vocabulary acquisition, and language proficiency.

The Open Access version of this book, available at http://www.taylorfrancis.com, has been made available under a CC BY NC ND 4.0 license.

1 Lexical Competence and Lexical Proficiency

1.1 Introduction

In the first 50 years of modern linguistics, vocabulary remained at the periphery of researchers’ interest and agenda. Lexis was traditionally conceptualised as a large and unstructured collection of individual items stored in memory, with no consequence for the system that language was considered to be. For example, Bloomfield (1933) famously defined the lexicon as “an appendix of the grammar, a list of basic irregularities” (p. 274). Although the main linguistic theories of the time—structuralism and generativism—recognised that words constituted the building blocks of language, they treated lexis as amorphous raw linguistic material and directed their attention to the structures and rules which held it together. An interest in the lexicon began to appear in the 1970s in the field of generative semantics (e.g. Katz & Fodor, 1963; Jackendoff, 1972) and a decade later in the area of applied linguistics, in particular of second language learning and teaching (Meara, 1980). Due to these influences vocabulary started to be perceived as a complex and organised linguistic module. It also started to play a more important role in linguistic research. Recently, it has featured prominently in the descriptions of language, which emphasise its equal status with other linguistic subsystems (e.g. pattern grammar, Hunston & Francis, 2000). It has also been vigorously studied in the field of psycholinguistics, which has explored mental representation, processing and acquisition of vocabulary (e.g. connectionism; Rumelhart, McClelland, & PDP Research Group (1986); Seidenberg & McClelland, 1989).

Recent models of language not only recognise the role of lexis in the overall linguistic system, but postulate its inseparability from other subsystems. The systemic functional approach advocates the complementarity of grammar and lexis. Its founding father, M. A. K. Halliday, observed that “The grammarian’s dream is … to turn the whole of linguistic form into grammar, hoping to show that lexis can be defined as ‘most delicate grammar’ ”. He further asserted that “grammar and vocabulary and not different strata [of language]; they are two poles of a single continuum, properly called lexicogrammar” (Halliday & Matthiessen, 2014, p. 24). According to Halliday, lexicogrammar forms the stratum of ‘wording’, which mediates between the lower stratum of ‘sounding’ (graphology/phonology) and the higher stratum of ‘meaning’ (semantics/discourse). It consists of a closed system of meaning-general grammatical structures and open sets of meaning-specific lexis, but also of intermediary stages between these two ends of the cline such as collocations and colligations.

The traditional distinction has likewise been challenged by cognitive linguistics which also postulates a continuum approach to the whole range of linguistic components rather than their modularity. Such a view was put forward by Ronald Langacker in his conception of grammatical structure, which he named cognitive grammar:

There is no meaningful distinction between grammar and lexicon. Lexicon, morphology and syntax form a continuum of symbolic structures, which differ along various parameters but can be divided into separate components only arbitrarily.

(Langacker, 1987, p. 3)

According to Langacker, these symbolic structures are associations of phonological and semantic units, i.e. forms and established concepts. Their simplest kind are morphemes; however, basic structures can combine and create increasingly larger symbolic structures, e.g. words and grammatical patterns, all of which function as pre-packaged unitary entities. Thus, lexical and grammatical structures (i.e. words and grammatical patters) differ from each other and other symbolic structures “not in kind, but only in degree of specificity” (p. 58). An offshoot of cognitive linguistics, construction grammar, also recognises words as one type from the pool of linguistic units, holistically termed constructions, other types of constructions being morphemes, idioms, phrases, partially lexically filled and fully general grammatical patterns. Goldberg (2003, p. 219) maintains that all constructions are “pairings of form with semantic or discourse function”. They create a large hierarchically structured network—construct-i-con—which encapsulates “the totality of our knowledge of language” (p. 219). The two approaches do not deny the existence of the lexicon as a discernible language constituent, but they assert that its distinctiveness from other types of linguistic elements is blurred and its precise delineation impossible (Langacker, 1987, p. 19). The mental representation of words is usage based and emergent (Bybee, 2006), and encompasses information on words’ various forms, syntactic patterns in which they occur, their different meanings and contexts of use.

This spur of interest in the lexical aspects of language and the new approaches to the status of lexis in the linguistic system have resulted in the multitude of definitions and models of vocabulary, which have been proposed in order to provide a theoretical framework for empirical studies, ranging from corpus-based explorations through psycholinguistic experiments to neurolinguistic imaging. Each of these attempts centred on lexical properties which suited its research agenda. Even within the more focused linguistic sub-discipline of Second Language Acquisition (SLA), there has been no agreement on the nature of vocabulary knowledge and on how it should be defined, described, measured and assessed. This lack of consensus has been aptly articulated by Read and Chapelle (2001):

An observation that emerges from a review of this literature is the ill-defined nature of vocabulary as a construct, in the sense that different authors appear to approach this from different perspectives, making a variety of—often implicit—assumptions about the nature and scope of the lexical dimension of learners’ language.

(Read & Chapelle, 2001, p. 1)

This chapter provides an overview of different approaches to and models of vocabulary that have been put forward in the literature on second language acquisition and assessment.

1.2 Preliminary Definitions

The precise definition of the construct of lexical command has far-reaching consequences for its measurement and assessment as well as for the interpretation of the results of these two procedures. However, there has been no agreement even regarding the terminology used to describe the lexical aspects of language. SLA researchers refer to the lexicon, vocabulary knowledge, lexical competence and lexical proficiency either employing these labels interchangeably or applying them to different, but poorly defined concepts. In fact, if these terms are ever elaborated on, it is usually through mentioning the component parts of the constructs which they refer to, rather than delineating the constructs themselves. For example, the seminal book on the mental lexicon Words in the Mind (Aitchison, 2003) offers only a rudimentary definition of its key term, which is “the word-store in human mind” (p. 4). A more elaborate explication can be found in Schwarz (1995), who writes:

The mental lexicon is a system in our long term memory (LTM), where all our knowledge about the words of our language(s) is stored.

(Schwarz, 1995, p. 63)

One of the few definitions of lexical competence, which can be found in the literature has been offered by the Common European Framework of Reference. The document specifies lexical competence as the “knowledge of, and ability to use, the vocabulary of a language” (Council of Europe, 2001, p. 110). This delineation may seem simplistic at first glance, but it emphasises two important aspects of vocabulary—knowledge and ability, as well as observable behaviour through which they are manifested—vocabulary use. It implies that the lexical competence does not only consist of a body of information but it also includes a capability of applying this information to perform a communicative act. In this way the authors of CERF relate the concept of lexical competence to larger models of language and frameworks of communicative competence, which were developed in the 1980s and 1990s.

1.2.1 Communicative Competence, Language Ability and Language Proficiency

For a long time preparing learners for the demands of communication in L2 was conceptualised as developing their knowledge of and about the second language, which was perceived as the only factor enabling learners to use language. This was a reflection of formal approaches to language dominant in linguistics for the first seven decades of the 20th century. Structuralism introduced the distinction between langue, an abstract and complex system of linguistic structures which existed independent of its users, and parole, concrete instances of the use of langue (de Saussure, Bally, Sechehaye, & Riedlinger, 1916). It was langue which constituted the focus of linguistic inquiries. A revolution in linguistics brought about Noam Chomsky (1957, 1965) and transformational-generative linguistics shifted the perspective by focusing on linguistic competence, i.e. “the speaker-hearer’s knowledge of his language” (1965, p. 4), but also discarded linguistic performance—defined as “the actual use of language in concrete situations” (p. 4)—as not worthy of scientific scrutiny. Although these major schools of thought were not directly concerned with second language acquisition and use, they exerted their influence on the SLA discipline. Thus, L2 knowledge was placed at the heart of L2 research, teaching and assessment and it was believed to be the driving force of L2 performance. It was not until Dell Hymes’s (1972) seminal paper that the direct link between knowledge and performance was challenged. Working in the context of sociolinguistics, Hymes introduced the concept of communicative competence which encompassed two separate components: knowledge of language and ability for use. He claimed that the actual language performance is a result of a complex interaction between underlying linguistic knowledge as well as more general cognitive and psychological mechanisms constituting ability for use.

Hymes’s model of communicative competence caught the attention of researchers in second language acquisition, teaching and testing, and in the ’80s and ’90s its several adaptations and elaborations were proposed by Widdowson (1979), Canale and Swain (1980), Bialystok and Sherwood-Smith (1985), Taylor (1988), Davies (1989), Bachman (1990) and most recently by Bachman and Palmer (1996). Each of these paradigms recognises the basic distinction introduced by Hymes, but differs in the way of conceptualising these two components and their constituents (see McNamara, 1996 for a detailed review).

One of the most influential accounts of these components and the patterns of interaction between them were proposed by Bachman (1990) and later modified by Bachman and Palmer (1996). In the more recent version, the authors introduce the term language ability which consists of language knowledge and strategic competence and which interacts with topical knowledge, personal characteristics and affective states and further with contextual factors in generating language use.

Bachman and Palmer (1996) describe their model in the following words:

Language use involves complex and multiple interactions among the various individual characteristics of language user, on the one hand, and between these characteristics and the characteristics of the language use or testing situation, on the other. Because of the complexity of these interactions, we believe that language ability must be considered within the interactional framework of language use. The view of language use we present here thus focuses on the interactions among areas of language ability (language knowledge and strategic competence, or metacognitive strategies), topical knowledge, and affective schemata, on the one hand, and how these interact with the characteristics of the language use situation, or the task, on the other.

(Bachman & Palmer, 1996, p. 62)

Bachman and Palmer further list the constitutes of the two components of language ability, which are presented in Figure 1.1 and Figure 1.2.

Figure 1.1 Areas of Language Knowledge

Source: Bachman and Palmer (1996, p. 68)

Figure 1.2 Areas of Metacognitive Strategy Use

Source: Bachman and Palmer (1996, p. 71)

The strategic competence is placed at the heart of the whole process of language use in Bachman and Palmer’s model. They define it as a set of metacognitive strategies which are executive processes regulating language use by managing the interaction of various components of the process, language knowledge being only one of them.

The interesting new development in this model of language use and language test performance consisted in including both topical knowledge and affective schemata, the latter defined as “affective and emotional correlates of topical knowledge” (p. 65) which determine the language user’s affective response to the language use situation and which can influence his or her linguistic reaction to it. Emotional response can have a facilitating or debilitating effect on the user. Yet, as pointed by McNamara (1996, p. 74) the understanding of the precise functioning of the affective schemata in relation to performance is still very crude.

The language ability, as conceptualised by Bachman and Palmer has also been referred to by other researchers as language proficiency. For example, Thomas (1994, p. 330, footnote 1) defines language proficiency “a person’s overall competence and ability to perform in L2”. A more in-depth elaboration of the concept of language proficiency was provided by Hulstijn (2011):

Language proficiency (LP) is the extent to which an individual possesses the linguistic cognition necessary to function in a given communicative situation, in a given modality (listening, speaking, reading, or writing). Linguistic cognition is the combination of the representation of linguistic information … and the ease with which linguistic information can be processed (skill)… . Linguistic cognition in the phonetic-phonological, morphonological, morphosyntactic, and lexical domains forms the center of LP (core components). LP may comprise peripheral components of a less-linguistic or non-linguistic nature, such as strategic or metacognitive abilities related to performing listening, speaking, reading or writing tasks.

(Hulstijn, 2011, p. 242; emphasis in original)

Clearly, all the notions already discussed: Hymes’s (1972) communicative competence, Bachman and Palmer’s (1996) language ability and Hulstijn’s (2011) language proficiency relate to the same construct which—according to the researchers—consists of two main components termed by the authors as: knowledge of language, language knowledge or linguistic cognition; and, on the other hand, ability for use, strategic competence, or strategic or metacognitive abilities. Yet, while for Hymes the two components seem equally important, Bachman and Palmer place the strategic competence in the centre of their model, and Hulstijn treats strategic or metacognitive abilities as peripheral constituents of the construct. Nevertheless, the researchers agree that this latter element of the construct is not a purely linguistic faculty and includes broader cognitive and affective factors. Its exact nature is not well understood, which was best commented on by McNamara (1996):

Language know...

Table of contents

• Cover
• Half Title
• Series
• Title
• Copyright
• Dedication
• Contents
• Acknowledgements
• Introduction
• 1 Lexical Competence and Lexical Proficiency
• 2 Lexical Assessment Methods
• 3 Performance-Based Assessment of Lexical Proficiency
• 4 Statistical Measures of Lexical Proficiency
• 5 Statistical Measures and Raters’ Scores of L2 Production—Review of Literature
• 6 The Study—Measuring and Assessing Lexical Proficiency of Advanced Learners
• Conclusions
• References
• Index