Cognitive Models Of Speech Processing

The Second Sperlonga Meeting

Gerry Altmann, Richard Shillcock

• Publisher: Psychology Press
• Year: 2013
• Print ISBN:
  9780863773020
• eBook ISBN:
  9781134832934
• Topic: Psychology
• Subtopic: Cognitive Psychology & Cognition
• Index: Psychology

A comprehensive review for those interested in the range of theoretical concerns in speech and language processing.

1

Overview

Richard Shillcock

Centre for Cognitive Science, University of Edinburgh, UK.

Gerry T.M. Altmann

Laboratory of Experimental Psychology, University of Sussex, Brighton, UK.

Introduction

The chapters in this volume reflect many of the general theoretical concerns that have motivated research on speech and language processing over the past decade—modularity in language processing, interaction between bottom-up and top-down processing, the significance of connectionist modelling, the processing of languages other than English—as well as more specific concerns such as the nature of the lexical representations onto which the speech input is mapped, the mapping process itself, and the processes that operate over the lexical representations to yield ultimately the meaning of the spoken sentence. These issues of representation and process can be studied with reference to acquisition of language processing in infancy, its deployment by normal adults, and its breakdown in language-disordered individuals. Moreover, each can be studied with reference to production or perception, and to processing across different languages. A further perspective is provided by models of language behaviour, in terms either of implemented computational models or of formal theories of language. In recent years, we have seen that attempts to address these concerns have led to increasing interaction across different disciplines. One of the aims of this series on Cognitive Models of Speech Processing, has been to bring together examples of such interactions. Thus, in this second volume,¹ Marslen-Wilson draws on current phonological theory to inform the study of processing issues in lexical access; Nicol employs formal syntactic theory to develop and test hypotheses about parsing and sentence processing; Fodor, Barss, and Sag have as their main concern the relationship between formal syntax and the human sentence processor. And so on. In virtually every chapter we see the important distinction between representation and process. It is scarcely possible to proceed at all in modelling speech processes without critical assumptions concerning the most appropriate representations to use—syllables, phonemes, phonetic features, graphemes, morphemes, words … Nor is it possible to proceed very far without confronting implications for the more global aspects of cognition and the functional architecture that underpins it.

The purpose of this overview is to provide the reader with an illustrative summary of the concerns of each of the chapters, in chronological order; to furnish a sketch of some of the relevant background, issues, and connections; and to set the context within which cognitive models of speech processing are being developed. This chapter is not intended as an introduction to speech processing per se, but is intended instead to arm the reader with sufficient context by which to assess the individual contributions within the volume.

We have attempted to order the chapters to reflect some of the main commonalities between the papers, but, given the nature of language research, virtually any ordering might generate thought-provoking contrasts or comparisons. Sometimes the relationship between successive chapters stems from a common objective, experimental paradigm or field of study, and we have provided a general title for each such group of chapters in the list of contents. Sometimes one chapter was explicitly elicited as a commentary on the adjacent chapter(s), reflecting the roles of speakers and discussants in the original conference and the continuing interaction between the authors. The commentaries are sometimes short and applied, sometimes longer and more discursive. Any chapter may be read independently, but by reading adjacent chapters a broadening picture emerges of the overall progress in our understanding of speech and language processing.

Modelling Acquisition

A key concern of those researching the development of language skills has been to assess the actual perceptual abilities of infants at different ages, with the goal of determining how the language processor progresses from its biologically given starting point to its experientially defined adult state. Adult speakers of different languages employ, and are sensitive to, different collections of speech sounds. How is this sensitivity developed for certain contrasts between speech sounds, and lost for others? How does sensitivity change over time? And on what basis do these changes occur; are they irreversible, for instance? The notion of a biologically given starting point may be misleading; sensitivity to (low frequency) auditory stimuli begins in utero when the foetus is between six and seven months old. Further, the relationship between perceptual development and neural development in the early months remains to be explicated; the acquisition of speech processing capacity may well reflect a biologically given timetable of development, which weakens the notion of a biologically given starting point, and as a consequence different sorts of information may have different effects as development occurs. What is clear, however, is that very young infants show a preference for hearing the full complexity of natural connected speech (Mehler, Bertonocini, Barriere, & Jassik-Gerschenfeld, 1978), and that newborns are adept at discriminating between speech sounds (Streeter, 1976). Considerable changes in perceptual abilities occur over the first year and more, however. Of crucial importance in the adult processor is the ability to ignore acoustic-phonetic differences that are of no relevance in the specific language that has been learned. At this point in the development of the field, the principal concern is with establishing the timetable of events, the raw data for a theory of the development of speech processing.

Measures of discrimination in infant speech perception typically rely on increased sucking of a non-nutritive nipple to elicit ‘interesting’ (i.e. non-monotonous, contrasting) stimuli, or on the infant learning to orientate to a rewarding stimulus on perception of a specific cue. The use of such approaches has revealed surprising abilities of discrimination in very young infants; this ability to distinguish particular contrasts begins to be lost in the first year, under the influence of the ambient language, but the pattern of loss, and its potential reversibility, sheds light on the construction of the adult speech processor. For instance, the vulnerability to loss of a particular contrast depends in part on its relationship to the other contrasts present in the adult phonological system. Peter Jusczyk presents a valuable review of two decades of research into infant speech perception, addressing questions concerning the innate endowment that the infant brings to the problem, and the effect of exposure to the particular language of the community in which the infant is reared. He proposes the WRAPSA model as a theoretical framework in which to pursue these questions. The infant is seen as bringing general perceptual processes to bear on the acoustic input; with experience, these are attuned to the contrasts used in the ambient language. Juscyzk sees the syllable as the critical unit in the development of segmentation and pattern extraction (cf. Mehler, Dupoux, & Segui, 1990). Finally, and perhaps most at odds with the view of adult lexical access reported in many chapters in this volume, Jusczyk eschews any role in his model of the infant lexicon for abstract prototypical representations of the constituents of words, preferring instead a model in which multiple traces of a particular word reside in longterm memory. He proposes that ‘lexical access’ for infants involves matching to these multiple traces.

The theoretical concerns encountered in Jusczyk’s chapter recur in Janet Werker’s contribution, together with the methodological problems of assessing the perceptual capabilities of infants. In contrast to Jusczyk’s approach, however, Werker is more willing to consider the possibility of speech processing being special, as opposed to emerging from general perceptual abilities. The notion that humans are evolutionarily endowed for speech processing has often proved attractive, as in early claims made for the speech-specificity of categorical perception, for instance. A more conservative approach involves attempting to account for the data using more general perceptual mechanisms, while still allowing that the human may be evolutionarily adapted for language use (thus, profoundly deaf individuals taught to sign from birth may develop the same complexity of communicative skills as speaking individuals.) Werker demonstrates the value of cross-linguistic research when she describes experiments requiring infants and adults to categorise speech stimuli from a non-native language. After presenting evidence bearing on the level—acoustic, phonetic, phonemic—at which subjects’ responses are driven, she goes on to question some of the implications for the proposed modularity of processing; is phonetic analysis performed by a dedicated perceptual module? What makes Werker’s and Jusczyk’s contributions particularly relevant is that they converge on issues of representation and process, while employing very different sets of assumptions concerning the ‘speech is special’ debate.

In both Jusczyk and Werker’s chapters, various assumptions are made concerning the most appropriate levels of representation for the interpretation of the data. Recent work on the nature of the representations that are constructed prior to contacting the mental lexicon has stressed the role of the syllable and the potential for cross-linguistic comparison, both of which are topics that recur in the Jusczyk and Werker chapters. Recent work on speech segmentation strategies reveals that these two topics may be intimately linked, with the syllable being a more salient processing unit in some languages than in others (e.g. Cutler, Mehler, Norris, & Segui, 1986; Sebastian-Galles, Dupoux, Segui, & Mehler, 1992). A recurrent theme is that the study of apparently language-specific processing behaviour is allowing testable hypotheses to be made about language-independent universal processing structures; indeed, this issue is addressed explicitly by Anne Cutler, although the reader may ponder the potential for cross-linguistic comparison in each of the topics encountered in this volume.

Pre-Lexical Processing and Lexical Representation

Central to modelling speech processing is the need to establish the perceptual role of particular patterns in the input. Jusczyk, for instance, attributes an important role to the syllable during the acquisition stage. The role of the syllable is also central to recent work by Emmanuel Dupoux, who presents detailed evidence from a series of experiments aimed at establishing the perceptual status of the syllable. Using monitoring techniques, together with exhaustive manipulations of the consonant/vowel structure of the stimulus materials, Dupoux reports data that lead to a qualification of the original Syllabic Hypothesis—that there is prelexical segmentation of the speech stream into syllables (e.g. Mehler, Domergues, Frauenfelder, & Segui, 1981; Segui, Dupoux, & Mehler, 1990). He goes on to discuss alternative accounts of the hypothesis, principally involving the demisyllable. Dupoux demonstrates some of the problems of interpreting the relevant experimental results; in this analysis, absolute reaction times are taken into account in order to argue for differential interpretations of subjects’ responses. Of the range of experimental techniques described in this volume, perhaps phoneme-monitoring is the technique that has engendered the most experiments aimed, partly or wholly, at clarifying the technique itself, and elucidating the nature of the judgement being made by the subject. In contrast, the cross-modal priming technique, which features in many chapters, has been relatively free of this sort of attention and has thus accumulated fewer constraints on the nature of the stimulus materials that can be used with the technique.

Anne Cutler reviews cross-linguistic research using the syllable-monitoring task, and claims that the apparent contradictions within and between languages can be resolved by admitting language-specific processing differences in the context of a universal initial cognitive endowment. Specifically, Cutler claims that languages differ in terms of which aspect of the speech signal is the most appropriate in facilitating segmentation prior to lexical access; thus, although the syllable is taken to be a salient aspect of language structure in French, a syllable-timed language, this is definitely not the case in English, a stress-timed language (cf. Cutler, et al., 1986). This is a very different enterprise from seeking to determine language-independent ‘units of perception’, and it moves attention towards the principles that govern the emergence of language-specific differences and towards potential ‘parameter setting’ in infancy. Note that both enterprises assume some segmentation of the speech stream into an appropriate form for lexical access to be attempted; in this they differ from the connectionist approach of McClelland and Elman (1986), for instance, who see segmentation as a by-product of lexical access and not as a necessary goal of the processor.

Concern about prelexical representation and lexical access necessarily leads to concern about the form of the representations in the mental lexicon itself, a longstanding central issue in psycholinguistics. The orthodox approach has been to assume that each word has a unique lexical entry, access to which provides information concerning that word’s orthography, phonology, syntax, and semantics. This entry is seen as central, and modality independent. Much of the current debate revolves around an alternative to the orthodox approach—the connectionist claim that a lexical entry is no more than the ensemble of mappings between orthographic, phonological, and semantic representations (Seidenberg & McClelland, 1989). The study of the representation of complex words—words containing more than one morpheme—represents a challenge to both approaches. The formal study of morphology reveals the rule-governed structure of complex words, together with its interaction with other levels of description; how is this structure reflected in the storage and processing of these words, in their lexical entries?

Lorraine Tyler and colleagues consider this question from the orthodox perspective of abstract lexical entries with internal structure (stems and affixes) (see Seidenberg (1989) for a review of the reading of complex words from an alternative, connectionist, approach). In this general area progress has, until recently, been relatively limited, perhaps because of the lack of sufficiently sensitive experimental techniques, or perhaps simply because of the lack of the critical data necessary to distinguish between alternative models. Research generated by the ‘classical’ (i.e. non-connectionist) models has concentrated on derivational morphology, revealing a relationship in storage between related words (e.g. Bradley, 1980) and suggesting a variety of processing architectures to reflect this fact (e.g. Taft, 1991). Recently, further progress has been made, and this is exemplified by Tyler et al.’s contribution to this volume, and by Cristina Burani’s commentary. The former makes use of repetition priming: a cross-modal priming technique in which, for instance, the word friendly is heard and at its offset a lexical decision to the visual word friend is measured. Tyler et al. present data in the context of the Cohort model (e.g. Marslen-Wilson, 1987; Marslen-Wilson & Tyler, 1980; Marslen-Wilson, this volume), in which the processor attempts to isolate the discrete abstract representation of one (complex) word from the rest of the words in the lexicon—the most relevant competitors in the cases considered by Tyler being morphologically related words. At issue here are central, modality-independent lexical representations—the entries in the mental lexicon—rather than input representations that are specific to the spoken or written language.

The strength of Tyler’s approach is in the comprehensive manipulation of semantic transparency and phonological relatedness. Researchers approaching morphology from a connectionist perspective (e.g. Rumelhart & McClelland, 1986; Seidenberg & McClelland, 1989; see also Taft, 1991), and who eschew discrete abstract lexical entries, will find no initial comfort in some of the data presented here, notably the finding that there is no priming between derived words that share a stem (such as confessor and confession). At first sight this apparent insulation of the semantics of same-stem derived words is surprising, given that so much of their meaning is similar if not identical. From the perspective of diachronic language change, however, semantic drift between morphologically related words (committee, commission, commit, committal …) is common and we should perhaps expect processing to reflect or allow this. Tyler et al.’s model of the storage of complex words (that is, their central lexical entries, not the modality specific access representations) involves shared stems, and accommodates these data by assuming inhibitory connections between suffixes. Thus there is priming between words when one is a stem, but priming is disallowed when both are derived words. The speed and sensitivity of the processor are once again in evidence, with the apparently immediate deactivation of inappropriate semantic representations on the strength of a spoken suffix alone. These data establish the merits of the repetition priming technique in investigating morphology, and should provoke the interest of those concerned with formal accounts of morphology and its computational modelling. Crucial representational issues are also raised, such as the status of shared stems when different semantics are implied by the suffix.

Burani, in her commentary on the chapter by Tyler et al., provides a wide-ranging discussion of aspects of the relationship between formal linguistic accounts of morphology and psychological accounts of the processing of derived words. She considers, for instance, the relationship between the productivity of a particular suffix and the actual number of words in the lexicon manifesting that suffix; the relationship between the two is not simply predictable. Burani emphasises the useful...

Table of contents

• Cover
• Title Page
• Copyright
• Contents
• List of First Authors
• Acknowledgements
• 1. Overview
• Part I: Modelling Acquisition
• Part II: Pre-Lexical Processing and Lexical Representation
• Part III: Interaction and Variation in Lexical Processing
• Part IV: Sentence-Level Processing
• Part V: Issues of Process in Formal Linguistics
• Part VI: Processing Written Words
• Author Index
• Subject Index