Language, Gesture, and Space

Karen Emmorey, Judy S. Reilly

• Publisher: Psychology Press
• Year: 2013
• Print ISBN: 9780805813784
• eBook ISBN: 9781134779734
• Topic: Psychology
• Subtopic: History & Theory in Psychology
• Index: Psychology

This book brings together papers which address a range of issues regarding the nature and structure of sign languages and other gestural systems, and how they exploit the space in which they are conveyed. The chapters focus on five pertinent areas reflecting different, but related research topics:
* space in language and gesture,
* point of view and referential shift,
* morphosyntax of verbs in ASL,
* gestural systems and sign language, and
* language acquisition and gesture.

Sign languages and gestural systems are produced in physical space; they manipulate spatial contrasts for linguistic and communicative purposes. In addition to exploring the different functions of space, researchers discuss similarities and differences between visual-gestural systems -- established sign languages, pidgin sign language (International Sign), "homesign" systems developed by deaf children with no sign language input, novel gesture systems invented by hearing nonsigners, and the gesticulation that accompanies speech. The development of gesture and sign language in children is also examined in both hearing and deaf children, charting the emergence of gesture ("manual babbling"), its use as a prelinguistic communicative device, and its transformation into language-like systems in homesigners. Finally, theoretical linguistic accounts of the structure of sign languages are provided in chapters dealing with the analysis of referential shift, the structure of narrative, the analysis of tense and the structure of the verb phrase in American Sign Language. Taken together, the chapters in this volume present a comprehensive picture of sign language and gesture research from a group of international scholars who investigate a range of communicative systems from formal sign languages to the gesticulation that accompanies speech.

CHAPTER ONE

Theoretical Issues Relating Language, Gesture, and Space: An Overview

Karen Emmorey

The Salk Institute for Biological Studies

Judy S. Reilly

San Diego State University

The chapters in this volume arose from the Fourth International Conference on Theoretical Issues in Sign Language Research, held in 1992 in San Diego. Together, the chapters address a range of related issues regarding the nature and structure of sign language, gesture, and gesture systems. In this overview, we present a brief summary of each chapter, highlighting the questions addressed by the chapter and noting issues that permeate the volume. The volume is divided into five sections reflecting different, but related areas of research: Space in Language and Gesture, Point of View and Referential Shift, Morphosyntax of Verbs in ASL, Gestural Systems and Sign Language, and Language Acquisition and Gesture. Many ideas thread their way throughout these sections. Several chapters discuss similarities and differences between types of visual-gestural systems: established sign languages, pidgin sign language (“International Sign,” Supalla & Webb), “homesign” systems developed by deaf children with no sign input (Morford, Singleton, & Goldin-Meadow), novel gesture systems invented by hearing nonsigners (Singleton, Goldin-Meadow, & McNeill), and the gesture that accompanies speech (McNeill & Pedelty). The role and function of space in gesture and sign language are recurrent themes, investigated by several researchers along various dimensions. Within sign languages, space is analyzed at the discourse level (Winston), at a semantic/referential level (Emmorey, Corina, & Bellugi), at the morpho-syntactic level for verbs (Janis), and for referential shift (Engberg-Pedersen). In contrast, Liddell argues against a morphological or syntactic function of space.

The development of gesture and sign language in children is used as a context to understand their components and underlying nature. Several chapters investigate the production of gesture in both hearing and deaf children, charting the emergence of gesture (“manual babbling,” Meier & Willerman), its use as a prelinguistic communicative device (Volterra & Iverson), and its transformation into a language-like system (homesign, Wang, Mylander, & Goldin-Meadow).

Another set of papers provides theoretical accounts of the linguistic structure of different signed languages. Among the topics discussed are the analysis of referential shift (e.g., Lillo-Martin; Poulin & Miller), the structure of narrative (Bahan & Supalla; Winston), the structure of the ASL verb phrase (Metlay & Supalla), and the analysis of tense (Aarons, Bahan, Kegl, & Neidle). In the following, we introduce these and other issues in an effort to provide an overall integration of the chapters within this volume.

SPACE IN LANGUAGE AND GESTURE

Space is a unique resource available to the visual–manual modality. Both signs and the gestures that accompany speech are produced in space, and the chapters in this section address the function and structure of this spatial expression. In a dramatic departure from previous analyses, Liddell argues for three types of mental representations that underlie the use of signing space in ASL. He defines Real Space as a “mental space” (a type of mental representation, see Fauconnier, 1985) which consists of a person’s conception of what is physically real in the current, directly perceivable environment. In Real Space, location for pronominal reference and verb agreement is not lexically fixed but depends upon the physical location of the referent. Liddell argues that because the referent can be in an unlimited number of physical locations, there are no linguistic features or discrete morphemes that identify the location toward which a sign (pronoun or verb) is directed. When people or objects are physically present, grammatical reference to the person or object is deictic, rather than anaphoric, and it is signaled by non-discrete gestural means. Liddell draws the same conclusions for grammatical reference when Surrogate Space is used. Surrogates are full-sized, invisible entities that are often used during referential shift (see the following section). Signers conceptualize surrogate persons or objects as if they were present. As with Real Space, when a signer uses Surrogate Space, there is an unlimited number of distinct ways of directing a pronominal sign or agreeing verb (signers can imagine surrogates in an unlimited number of locations), and Liddell draws the same conclusions about grammatical reference when Surrogate Space is used: reference is deictic, not anaphoric, and location is not morphemic. Finally, Liddell argues that these properties hold for Token Space as well. Tokens inhabit Token Space and are three dimensional, invisible entities that are placed in signing space by the signer. Tokens differ from surrogates in their size (they must fit within the physical signing space) and in person role (tokens are limited to a third person role while surrogates can assume first, second, or third person roles). As before, Liddell argues that because there is an unlimited number of places in Token Space into which a token may be placed, location in signing space is neither listable nor morphemic. Liddell goes even further and argues that Token Space should not be considered a morpho-syntactic entity at all, just as Real Space is not so considered. Needless to say, Liddell’s proposal is quite controversial and differs radically from how linguists have characterized the use of space for grammatical functions.

In contrast to Liddell’s analysis, Emmorey, Corina, and Bellugi present a functional characterization of space in the service of linguistic expression. They distinguish two different functions, rather than types, of space, and they argue that space can have a purely referential function in which nominals are associated with spatial loci, and pronominals or agreeing verbs directed toward spatial loci are understood as coreferential with the nominal. In addition, they demonstrate how space can function topographically to express the location of persons and objects within a physically real or imagined space and how this ability can be impaired with brain damage. Emmorey and colleagues present the case of a patient with a right hemisphere lesion whose ability to express topographic relations in ASL is impaired. However, her ability to use space for grammatical reference remains intact. These data are complemented by on-line processing results that indicate that normal signers represent the association between nominals and spatial loci differently, depending upon whether space functions topographically or purely referentially. A spatial locus that conveys locative information may be strongly encoded in memory as part of semantic representation, whereas a locus that is used only for reference within a discourse or sentence is more likely to fade from memory. These authors argue that space is multifunctional in ASL, and that these functions (referential and topographic) reflect differential processing as well as distinctive neurological substrates.

Space also appears to be multifunctional for gestures that accompany speech. In their study of gesture in hearing adults, McNeill and Pedelty distinguish three spatial “regimes”: a concrete regime in which space is topographically equivalent to the space of events in a scene; a referential regime in which gesture space represents a character or event; and a structural regime in which space represents the structure of the narrative itself. The first two regimes have clear parallels with the linguistic functions of space discussed by Emmorey et al. for ASL, and the third is discussed in detail by Winston in her chapter on the discourse functions of space in ASL. In their chapter, McNeill and Pedelty also detail how gestural impairments in patients with brain damage mirror patterns of linguistic breakdown. Just as the topographic use of signing space can be impaired with right hemisphere damage as described by Emmorey et al., McNeill and Pedelty found that hearing speakers with right hemisphere damage do not use space topographically when they gesture. In these patients, left-side motoric neglect occurs only for iconic gestures in concrete (topographic) space, but neglect does not occur for abstract gestures—thus, substantiating the neural underpinnings for these distinctions. In addition, right hemisphere damaged patients also have difficulty with the referential and structural spatial regimes, failing to produce cohesive strings of gestures. McNeill and Pedelty elaborate on the contribution of the right hemisphere to narratives, and present evidence to show how gestural impairments in right hemisphere damaged patients highlight this role. Cases of patients with disconnection syndromes (alien hand and commisurotomy patients) are presented to show that the two hands must be coordinated for normal gesturing (and that the right hemisphere is critical for the coherent use of gesture within a narrative). This chapter illuminates some of the parallels and differences between gesture and sign language and how each system manipulates space for various communicative and linguistic purposes.

Another parallel between gesture and sign language can be seen in Winston’s discussion of the use of space as a cohesive devise in ASL discourse. She presents a fine-grained analysis of a text to demonstrate how spatial mapping functions at the discourse level to signal and maintain discourse frames.¹ At the discourse level, an area within signing space can be associated with an element from the mental representation of the signer. Similarly, McNeill and Pedelty (see also McNeill, 1992) show that speakers associate gestures with certain areas of space in a way that reflects their mental representation of a topic. However, for ASL, the use of space for discourse and narrative cohesion is governed by linguistic principles and properties. Spatial discourse maps in ASL can be used for either immediate or distant cohesion, and Winston presents evidence that discourse maps can be suspended and reintroduced throughout the text and that these spatial maps are contrasted with neutral space in a narrative. She also argues for a larger discourse level analysis of eyegaze, in contrast to Bahan and Supalla who investigate eyegaze at the level of the “line” in narrative. Finally, Winston presents a detailed analysis of how spatial discourse maps are patterned within the comparative discourse frame. It will be important to discover whether the patterns of spatial mapping that Winston has reported for ASL discourse hold for other signed languages and to what extent they parallel gestures that delineate narrative structure during speech.

POINT OF VIEW AND REFERENTIAL SHIFT

The linguistic mechanisms used to express point of view in signed languages appear to be much more explicit than in spoken languages. Both signers and speakers use linguistic devices to indicate whether utterances should be understood as expressing the point of view of the signer/speaker or of another person (or of the signer/speaker at a different point in time). Spoken languages have several different devices for expressing point of view: pronominal deixis (e.g., use of Ivs. you), demonstratives (here, there), syntactic structure (active vs. passive), and literary styles (e.g., “free indirect” discourse). Signed languages use these mechanisms as well, but in addition, point of view can be marked overtly (and often continuously) by a “referential shift.”² Referential shift is expressed by a slight shift in body position and/or changes in eyegaze, head position, or facial expression. Data from three different signed languages are examined in this section, and each chapter focuses on a different aspect of this unique expressive feature of signed language. Poulin and Miller begin to map out the semantics of point of view expressions in Quebec Sign Language; Engberg-Pedersen provides a detailed analysis of different types of referential shift within Danish Sign Language; Lillo-Martin presents a syntactic analysis of referential shift in ASL; and Bahan and Supalla analyze the use of eyegaze within a narrative for expressing the point of view of a character or the narrator by principled changes in gaze direction.

Poulin and Miller argue that the use of referential shift in Quebec Sign Language (LSQ) is used to express an internal point of view. That is, referential shift expresses the emotional state of mind or the internal feelings of the discourse participant with whom the signer empathizes. When the discourse is not marked with referential shift, then either a neutral point of view or that of the signer is expressed. These authors also note that the scope of the referential shift is critical to semantic interpretation. For example, if an affective facial expression falls within a referential shift, the expression is interpreted from an internal point of view (that of the character with whom the signer identifies); otherwise, the facial expression is interpreted as expressing the signer’s own emotional state. Also, they argue that neutral comments associated with an external point of view cannot be expressed within a referential shift. Overall, Poulin and Miller focus on the semantic constraints involved in the use of referential shift for expressing the point of view of different discourse participants within a narrative.

In contrast, Engberg-Pedersen focuses on the different mechanisms by which Danish Sign Language conveys point of view, as well as reported speech (both direct and indirect): “shifted reference” refers to the use of pronouns within a direct or indirect quote (e.g., John said, “I don’t want to go”); “shifted attribution of expressive elements” is found in reports of a person’s thoughts, actions, emotions, or observations, and its use is not restricted to reported speech. In spoken languages, shifted attribution is much more restricted and is expressed through literary styles, such as the “free indirect style” of narrative. Shifted attribution of expressive elements recruits both manual and non-manual behaviors, including affective facial expression and can, but does not necessarily, co-occur in contexts of shifted reference. Because of its spatial parameters, the third instance of shift, “shifted locus,” is unique to signed languages; it occurs when signers want to report their own (or somebody else’s) interaction with a referent associated with a spatial locus. The signer approximates a “canonical encounter” (see Clark, 1973) in which the signer faces the referent as if the referent were actually there. In his chapter, Liddell addresses this aspect of referential shift, but he treats the phenomenon as the use of Surrogate Space, rather than as a shift of spatial locus. Engberg-Pedersen argues for formal and functional independence for the three proposed subsystems of referential shift (i.e., shifted reference, shifted attribution of expressive elements, and shifted locus).

In a strictly syntactic analysis, Lillo-Martin offers a structural account of ASL shifted reference at the sentential level in which the shift itself is considered to be a predicate that signals a change in point of view (POV). The POV predicate takes a subject with which it agrees (i.e., the shift is produced toward the location of the subject). Lillo-Martin argues that first person pronouns produced within the POV predicate are logophoric pronouns that are interpreted as coreferential with the subject of POV. In spoken languages, logophoric pronouns are distinct pronouns that occur in embedded clauses and signal coreference with a matrix subject or object for certain verbs (particularly mental verbs—verbs like think or say which express point of view in spoken languages). Lillo-Martin presents evidence from various sources (e.g., WH-questions, reflexives) that POV is a predicate that takes an embedded clause as its complement. She leaves the cross-sentential discourse analysis of POV for further research (although she does suggest a few possibilities). It seems clear that a complete understanding of referential shift and point of view will occur when we understand their syntactic, semantic, and discourse properties as well as how these properties interact and constrain each other.

Bahan and Supalla add to our understanding of these issues with their investigation of eyegaze and its structured use in narrative. The literature on non-manual linguistic behaviors in ASL has grown considerably within the last 15 years (Baker-Shenk, 1983; Bergman, 1984; Liddell, 1980: Reilly, McIntire, & Bellugi, 1990). These studies have all focused on non-manual behaviors with lexical, clausal, or sentential scope. Bahan and Supalla extend this literature by tackling, for the first time, the complex issue of cross-sentential non-manuals in extended discourse. Their chapter discusses the structure of narratives in ASL using a model proposed by James Gee to analyze two signed narratives. The focus of this chapter is eyegaze and how it functions in ASL narratives to delineate the basic structural unit of narrative: the line. Bahan and Supalla describe different uses of eye gaze: eye contact with addressee; no contact with addressee, gaze directed towards hands; and no contact with addressee, narrator assumes character role. They go on to describe different distributional patterns of these behaviors, their scope, and how they interact with other non-manual behaviors (e.g., head nods and eye blinks). This chapter augments our basic understanding of how non-manual behaviors function in narrative discourse and begins to elucidate the role that gaze might play in establishing point of view.

Together these chapters provide new insight into how signed languages structure discourse to express point of view. The use of referential shift appears to be a unique grammatical mechanism available to signed languages; and its use to express reported speech and discourse point of view may turn out to be universal for signed languages. However, the semantic and grammatical properties of referential shift may differ across languages. It is also interesting to note that McNeill (1992) has found that point of view is represented in the gestures that accompany speech. McNeill characterizes certain gestures as either depicting an “observer viewpoint” or a “character viewpoint.” An example of a gesture that depicts a character viewpoint would be moving one’s hands up and down when talking about someone climbing up a ladder. A gesture with an “observer viewpoint” would be moving the hand upward...

Table of contents

• Cover Page
• Half Title Page
• Title Page
• Copyright Page
• Contents
• Contributors
• 1 Theoretical Issues Relating Language, Gesture, and Space An Overview
• Part I Space in Language and Gesture
• Part II Point of View and Referential Shift
• Part III Morpho-Syntax of Verbs in ASL
• Part IV Gestural Systems and Sign Language
• Part V Language Acquisition and Gesture
• Author Index
• Subject Index