Assessing Second Language Pragmatics

S. Ross, G. Kasper, S. Ross, G. Kasper

• Publisher: Palgrave Macmillan
• Year: 2013
• Print ISBN: 9781137352132, 9781137003515
• eBook ISBN: 9781137003522
• Topic: Education
• Subtopic: Teaching Languages

The first book-length collection of studies on the assessment of pragmatic competencies in a second or foreign language. Grounded in theoretical perspectives on communicative and interactional competencies, it examines the reception and production of speech acts through a variety of assessment methods and quantitative and qualitative analyses.

1

Assessing second language pragmatics: An overview and introductions

Gabriele Kasper & Steven J. Ross

Pragmatics is a key domain in language assessment. For more than two decades, advances have been made in conceptualizing the domain, developing assessment instruments, and applying current methods of data treatment to the analysis of test performance. This book, the first edited volume on the topic, brings together empirical studies on a range of well-established and innovative strategies for the assessment of pragmatic ability in a second language. In this introductory chapter, we will first offer an overview of key concepts, situate theoretical models of pragmatic competence within the larger frameworks of communicative language ability and interactional competence, and consider the relationship between pragmatics and language testing. We will then introduce the chapters, organized into two Parts. The chapters assembled in Part I investigate assessment instruments and practices for a variety of assessment constructs, purposes, and contexts, guided by different theoretical outlooks on pragmatics. Part II comprises studies of interaction in different forms of oral proficiency interview, conducted from the perspective of conversation analysis.

1 Key concepts

1.1 Assessment

Mislevy and colleagues describe assessment as “a machine for reasoning about what students know, can do, or have accomplished, based on a handful of things they say, do, or make in particular settings” (Mislevy, Steinberg, & Almond, 2003, p. 4). In the context of applied linguistics, Chapelle and Brindley specify that “‘assessment’ refers to the act of collecting information and making judgments about a language learner’s knowledge of a language and ability to use it” (2002, p. 268). Assessment has a broader purview than testing. Assessment processes typically involve subjective interpretation in real time. For example, in classroom-based assessment (Rea-Dickins, 2008; Ishihara, this volume) and dynamic assessment (Lantolf & Poehner, 2011; van Compernolle, this volume), assessment is an integral part of the instructional process and serves to support student learning, based on the teacher’s or peers’ ongoing appraisal. In the case of oral proficiency interviews, the inter-subjectivity of the interlocutor and candidate is produced in a co-constructed discourse that is assessed later by a trained rater. Testing, in contrast, starts with subjective decisions about content sampling, as well as considerable subjectivity in the test moderation process before a test is deemed ready for objective scoring. Tests are generally of knowledge accessible to declarative memory, while assessments are of performances that integrate knowledge into action.

1.2 Second language

Following convention in applied linguistics, the category “second language” includes languages of any status in a person’s linguistic repertoire, whether foreign, second, heritage language, or lingua franca, regardless of the order in which the person has learned that particular language, how strong their command of the language is, how important it is for them, or how much and for what purposes they currently use it. Since many people learn multiple languages throughout their lives, some of them concurrently, it may be difficult to establish a meaningful order of a person’s various languages, and in some cases the “second language” may be the language they learned first. Depending on assessment context and purpose, it can be important to distinguish the status that the target language has in the test takers’ linguistic repertoire. To take an example from this volume, Youn and Brown investigate whether the status of Korean as a foreign language or as a heritage language influences the test takers’ performance of speech acts in Korean. Finally, languages may change status in the lives of societies and individuals.¹ In persons’ language biographies, what started to be learned as a foreign language becomes a second language when the person uses that language in the target environment. Two common settings for such shifts in the domain of education are study abroad contexts and study at English-medium universities. In this volume, Tominaga examines how a candidate in oral proficiency interviews changes his production of extended turns and stories in Japanese as a foreign language after study abroad in Japan. Seedhouse compares the interactional architecture of the IELTS Speaking Test, the most frequently used test of proficiency in spoken English for international applicants to British universities, with the organization of L2 classrooms and university content courses. For many of the candidates in the test, English is used as a foreign language in their home language classrooms, and the test examines whether they are prepared to use English as a second language to participate in instructional activities at the university.

1.3 Pragmatics

In the applied linguistics and language assessment literature, pragmatics is often described as the study of “meaning in context” or “language use in a social context”. On this definition, pragmatics includes such context-sensitive language use as the co-variation of postvocalic (r) with speaker’s social class, gender, and attention to form (Labov, 1966/2006) or the Northern cities vowel shift (e.g., Labov, 1994). Such a view is fully compatible with the scope of the pragmatics component in the Bachman and Palmer model (Bachman, 1990; Bachman & Palmer, 1996, 2010), the standard framework of language ability as a target construct for language assessment, about which we will have more to say below. As the examples indicate, the broad conception of pragmatics that many applied linguists favor renders pragmatics virtually co-extensive with sociolinguistics. In light of the history of pragmatics in North American applied linguistics, this is probably no coincidence. Before the second half of the 1980s, work that focused on language-mediated action sorted under sociolinguistics² and discourse analysis³ in North American second language research and education. Disciplinary boundaries are, of course, matters of institutional politics as much as of intellectual traditions, and this is not the place to engage with either. In the context of language assessment, formulating the global assessment construct as a person’s ability to use language for specifiable practical purposes coherently meets the overall goals of assessment, that is, to serve as a basis for accountable evaluation and decision-making (Bachman & Palmer, 2010).

But an argument can be made for a concept of pragmatics that is independent of any particular model of language ability as a target assessment construct, and that therefore can serve as a theoretical resource to develop alternative models of pragmatic ability in assessment contexts. For that purpose, it is useful to replace the underdefined description of pragmatics as “meaning in context” with a formulation that unfolds the notion into several constitutive dimensions. We will therefore repeat here David Crystal’s (1997) well-known definition. He describes pragmatics as “the study of language from the point of view of users, especially of the choices they make, the constraints they encounter in using language in social interaction and the effects their use of language has on other participants in the act of communication” (p. 301, italics added). Crystal references pragmatics to the semiotic theories of C. S. Peirce (1958) and Charles Morris (1946), in which pragmatics refers to the relation between the sign and its interpreters (users) in the semiotic triangle.⁴ The selections that speakers necessarily make from their semiotic (not limited to their linguistic) repertoire are not described as conscious or intentional, suggesting that consciousness and intentionality are not definitional to pragmatics. While the inclusion of choices highlights participants’ agency, these choices are subject to various constraints, first and foremost constraints in social interaction as the proximate environment for language use. The description of language use as a matter of interactionally constrained choices is compatible with Giddens’ (1984) dialectic relation of structure and agency in his structuration theory, and his very helpful notion of context as both “brought along” and “brought about” (Giddens, 1976). The constraints that impact semiotic choices may include social factors such as participants’ educational background, ethnicity, gender, sexual orientation, religion, political affiliation, occupational status, family status, age, and possibly other macrostructural influences; psychological factors, both cognitive and affective, the participants’ relationship, and the type of activity and wider social and cultural context. However, it is significant for Crystal’s vision of pragmatics that none of these categories of participant external and internal factors are specified. In fact, his formulation does not make any mention of “context”, something that readers across the epistemological spectrum might find deeply troubling. From a sociostructural and rationalist vantage point (Coupland, 2001; Kasper, 2006a), this absence is an underspecification and therefore a conceptual flaw. Poststructuralist and critical applied linguists as well will miss the appeal to the “usual macrosociological suspects” (McHoul, Rapley, & Antaki, 2008, p. 43). On the other hand, under a discursive constructionist and ethnomethodological perspective, Crystal can be understood to suggest that social and psychological context is not a given but becomes a legitimate consideration for the investigator only to the extent that the participants themselves invoke context through their interactional conduct (Schegloff, 1992a). The view of context as interactionally occasioned contrasts with the rationalist stance of Brown and Levinson’s politeness theory, which specifies power, social distance, and imposition as pre-given context dimensions, and its sociostructural transformation in speech act research (Arundale, 1999; Kasper, 2006a).

In Crystal’s vision of pragmatics, the key unit, the “act of communication”, is firmly embedded in social interaction, and the effect of a party’s action on other participants is expressly nominated as a topic for study. By including the consequences of action in interaction as part of pragmatics’ remit, Crystal invokes Austin’s notion of uptake and so offers a perspective on pragmatics that integrates both speaker and recipient as participants. In other words, Crystal does not limit pragmatics to the study of speaker meaning. His interactional perspective contrasts with Searle’s speech act pragmatics, in which the recipient’s uptake falls outside of the theory’s explanandum.⁵ Crystal’s view is compatible with discursive pragmatics (Bilmes, 1993; Kasper, 2006a), an effort to furnish pragmatics with a conversation-analytic foundation. But although Crystal’s description of pragmatics aligns with the study of social action in interaction, it does not confine pragmatics to talk. “Social interaction” can also be understood in a broader sense, extending to written and hybrid forms of communication (e.g., Widdowson, 1984). As a perspective on pragmatics that can inform language assessment, Crystal’s conceptualization of pragmatics is useful because it maintains the centrality of action that is in the name (πρᾶγμα ‘pragma’ ‒ ‘act’, ‘deed’) and specifies several constitutive components that lend themselves to further specification and operationalization in all modalities of language use.

Second language speakers’ pragmatic abilities entered the research agenda for language assessment by way of two sources: theoretical models of communicative language ability as a target construct in language assessment, and empirical research on second language pragmatics (Roever, 2011). Both sources are well-documented in the literature and discussed in several of the chapters in this volume. In order to situate the data-based studies reported in the chapters, we will review the most prominent theoretical frameworks in which the assessment of pragmatics is embedded.

2 Pragmatic competence as a target domain in language assessment

2.1 Communicative competence

Among several multicomponential models of communicative language ability (reviewed in Purpura, 2008), three proposals include pragmatic ability as one of their components: Canale and Swain’s (1980) original framework of communicative competence for language teaching and testing, Bachman’s model of communicative language ability (Bachman, 1990; in later versions, Bachman and Palmer’s model of language ability, 1996, 2010),⁶ and Purpura’s (2004) theoretical model of language ability. The more recent of the models elaborate the earlier proposals in various ways, and although they synthesize a wide range of theoretical resources from across the social sciences, Dell Hymes’ theory of communicative competence (1972) has shaped their fundamental outlook more than any other work. At the core of Hymes’ communicative competence theory is, of course, knowledge of “whether (and to what degree) something is appropriate (adequate, happy, successful) in relation to a context in which it is used and evaluated” (1972, p. 281, italics in original). The dimension of appropriateness has become most closely associated with pragmatics and is often treated as definitional of pragmatics. Social and cultural appropriateness anchors communicative competence firmly in the social world and enables relevant descriptions of the target use domain in language assessment contexts. Yet while communicative competence theory remedied Chomsky’s reductionist perspective on language competence, it maintained the competence‒performance dichotomy. By retaining the distinction between competence as a capacity of the individual mind and performance as the domain where competence is used in social life, Hymes adopts, in Halliday’s much-quoted formulation, an “intra-organism perspective on what are essentially inter-organism questions” (Halliday, 1978, p. 37).⁷ With their foundation in Hymes’ “psycho-sociolinguistics” (Halliday, 1978, p. 38), the three models of communicative language ability take on board the fundamental ontology that Hymes’ theory of communicative competence shares with Chomsky’s theory of language competence, namely that competence is located in the individual mind and is separate from performance.

Rival theories from sociology, anthropology, sociolinguistics, and linguistics that reject the competence‒performance dichotomy and a cognitivist perspective on language did inform the models of communicative language ability for language assessment, yet such alternatives were subsumed under the concept of language knowledge as an underlying individual trait. In addition, Bachman and Palmer’s (1996, 2010) framework carries forward an important elaboration of competence from Hymes (1972): the distinction between underlying language knowledge (in their model, organizational knowledge and pragmatic knowledge) and ability for use.

Bachman and Palmer conceptualize ability for use as strategic competence, defined as three sets of metacognitive strategies – goal setting, appraising, and planning. Plans are implemented through cognitive strategies, the component in the model that directly interfaces with the language use task. Taguchi (2012) recently proposed an empirically grounded multicomponential model of pragmatic competence with a similar structure. As do its predecessors, the model maintains the distinction between competence and performance, and it adopts the components of language ability from the Bachman and Palmer model. Drawing on Bialystok’s two-dimensional model of language proficiency (e.g., 1993), pragmatic competence is conceptualized as a two-component construct. Pragmatic knowledge refers to the ability to understand and produce “speech intentions” (Bialystok’s “analysis of knowledge”) and processing refers to the ability to use pragmatic knowledge efficiently in real time (Bialystok’s “control of processing”). As in her earlier research, Taguchi (2012) operationalizes pragmatic knowledge as accuracy of pragmatic comprehension and appropriateness of pragmatic (speech act) production, while processing is operationalized as fluency in comprehension and production. The processing dimension comprises both strategy categories from Bachman and Palmer’s model, metacognitive and cognitive strategies. In this vol...

Table of contents

• Cover
• Title
• 1 Assessing second language pragmatics: An overview and introductions
• Part I APPROACHES TO ASSESSING PRAGMATIC COMPETENCE
• Part II FROM PRAGMATIC COMPETENCE TO INTERACTIONAL COMPETENCE: INTERACTION IN ORAL PROFICIENCY INTERVIEWS
• Author index
• Subject index