Speech Perception By Ear and Eye

A Paradigm for Psychological Inquiry

Dominic W. Massaro

• Publisher: Psychology Press
• Year: 2014
• eBook ISBN: 9781317760450
• Topic: Personal Development
• Subtopic: History & Theory in Psychology

First published in 1987. This book is about the processing of information. The central domain of interest is face-to-face communication in which the speaker makes available both audible and visible characteristics to the perceiver. Articulation by the speaker creates changes in atmospheric pressure for hearing and provides tongue, lip, jaw, and facial movements for seeing. These characteristics must be processed by the perceiver to recover the message conveyed by the speaker. The speaker and perceiver must share a language to make communication possible; some internal representation is necessarily functional for the perceiver to recover the message of the speak.

Chapter 1: Scientific Framework for Psychological Inquiry

1. Modularity of mind

1.1 Speech perception

2. Research framework

2.1 Falsification

2.2 Strong inference

3. Two approaches to psychological inquiry

3.1 Psychophysics

3.1.1 Limitations of psychophysics

3.2 Information-processing theory

3.2.1 Limitations of current information-processing models

3.3 Integrating information and information processing

4. Pattern recognition

4.1 Multiple sources of information

5. Integrating information from multiple sources

5.1 Data and information

5.2 Brunswik's representative design

5.3 Limitations of representative designs

5.4 Information-integration framework and factorial designs

5.5 Bimodal speech

6. A theoretical framework for pattern recognition

6.1 Fuzzy logical model of perception (FLMP)

6.2 Model tests

6.3 Application of the FLMP

7. Speech perception by ear and eye

7.1 Binary oppositions

In face-to-face communication, the perceiver both sees and hears the speaker. The goal of the research reported in this book is to develop a psychological description of how the perceiver recognizes what the speaker says. Given the preceding century of psychological inquiry, it would not be inappropriate to bypass metatheoretical and methodological preliminaries and to get on with it. The hypotheses, experimental tests, and analyses stand on their own merit and reduce uncertainty about the processes involved in bimodal speech recognition. Preliminaries are necessary, however, because the relevance of the research stretches well beyond understanding speech perception into diverse domains of perceptual and cognitive performance. The goal of the book, therefore, is more ambitious than the goal of the research because the book confronts fundamental issues in experimental psychology and cognitive science, as well as the information and processes supporting speech perception. To set the stage for the research and to illustrate its relationship to extant issues, an important theoretical (or perhaps metatheoretical) principle is presented and evaluated.

1. Modularity of Mind

The present research addresses an engaging issue in extant psychological theory, defined as modularity by Fodor (1983). The thesis of modularity is apparent in a variety of domains ranging from Chomsky’s language organ (1975) and Pylyshyn’s (1984) impenetrable processes to Forster’s (1985) modular lexicon. I will follow Fodor’s presentation of the issue because it is the most complete and influential one to date. Fodor’s thesis is that there are independent psychological systems making up much of our mental life. These input systems are to be distinguished from more cognitive or central systems. We can and have learned a lot about input systems, but have not and cannot learn very much about general or central cognitive systems. What properties, then, do input systems have that make them good candidates for a successful psychological inquiry? Fodor gives nine properties that define input systems and distinguish them from central systems.

First, input systems are domain-specific, and Fodor manages to define about six of them—five corresponding to the traditional senses and one more for language. There are qualitative differences in processing among these input systems. For example, the psychological mechanisms in vision that mediate object perception would differ significantly from the processes contributing to sentence interpretation. Second, the operation of input systems is mandatory; we can’t help but read roadside signs. Third, there is only limited central access to the mental representations that input systems compute. Fourth, input systems are fast. The fifth and maybe most important property of input systems is that they are informationally encapsulated; the information available for processing by any input system is limited to specialized bottom-up properties. Sixth, the input analyzers have shallow outputs, or simply solve the task they were designed to solve. The seventh property of input systems is that they are associated with the fixed neural architecture. The eighth property is that input systems exhibit characteristic and specific breakdown patterns; specific brain injury creates specific deficits. Finally, in his ninth characteristic, Fodor argues for the innateness of input systems in which the ontogeny of input systems exhibits a characteristic pace and sequencing.

The nine properties of input systems are best understood when contrasted with the description of central systems. The main distinction here involves what the input systems compute as opposed to what the organism believes. When we use language to communicate our views, we draw on all of what we have seen, heard, remembered, or think. The cognitive mechanisms must have an interface among the outputs of all of the input systems. Perhaps it is this attribute of having available multiple sources of information that makes Fodor pessimistic about understanding central systems. The thesis of the present endeavor, however, is that input systems also have multiple sources of information available, and they should not be distinguished from central systems in terms of the number of sources of information. In speech, for example, there are multiple, audible, visible, and contextual sources of information supporting perception.

The contribution of the current approach has to do with the question of the uniqueness of the processes that have been uncovered in bimodal speech perception. The hypothesis is that the processes involved in bimodal speech perception are similar to those involved in a number of other domains of perceptual and cognitive functioning. After developing an understanding of speech perception by ear and by eye, I explore several domains and ask whether similar or analogous processes occur across these domains. The domains include person impression, learning of arbitrary categories, sentence interpretation, probability judgments of possible events, and judgments of category membership. Following Fodor’s classification, these domains include both input modules and central systems. It should not be possible to provide the same process description across these domains, especially a description developed from another unique input domain of bimodal speech perception. To the extent that the present framework can provide a unified account of this broad range of phenomena, modularity is not a reasonable guideline for psychological inquiry.

1.1 Speech Perception

Liberman and Mattingly (1985) have adopted Fodor’s modularity principle in their revision of the motor theory of speech perception. According to their view, a biologically distinct system (a module) is responsible for the perception of phonetic information without translation from preliminary auditory impressions. Speech perception results from the input module’s specialization for phonetic gestures. Given the assumed biological link between perception and production, listeners are prevented from hearing speech as ordinary sound. Articulatory gestures are perceived because of the lawful dependencies among gestures, articulatory movements, vocal-tract shapes, and signal. In a similar fashion, Eimas (1985) has proposed that infants arrive with innate perceptual mechanisms to detect discrete phonemic categories. These claims for specialization of speech would offer little hope for our research framework on two counts. First, approaching the study of bimodal speech perception from the perspective of prototypical pattern recognition must fail. By definition, a specialized process should not follow general principles. Second, the laws uncovered for bimodal speech perception should have little applicability to other domains of human performance.

To anticipate the outcome of the current enterprise, my claim is that both expectations from the modularity principle have failed to materialize. The research has succeeded in uncovering principles of bimodal speech perception; principles that are highly representative of bimodal speech perception prove to be relevant to a variety of behaviors ranging from person impression to the utilization of arbitrary categories. In each domain, the perceiver evaluates multiple sources of both bottom-up and top-down information, integrates these sources with respect to representations in memory, and classifies the pattern on the basis of the relative goodness of match among the relevant categories. The findings also offer a much more optimistic view of language use by the hearing-impaired. To the extent that speech perception is not specialized, we should be able to substitute the sounds of speech with other functionally-valid cues. Lipread information combined with transformation of acoustic signal into tactile, visible, or electrical forms should be sufficient to support language perception and understanding.

The remaining charge of the first chapter is to articulate the research paradigm and place it in the context of contemporary theory. Although familiar to most readers, the research strategies of falsification and strong inference are reviewed. The present use of the research strategy of testing between alternative hypotheses has succeeded in providing major constraints on any contending theory of speech perception. Two different approaches to experimental psychology are described and contrasted. The historical link to the current research is presented in terms of Egon Brunswik’s probabilistic functionalism and the contemporary reference belongs to Norman Anderson’s theory of information integration. There are also important relationships to the highly informative work on natural categories (Mervis & Rosch, 1981) and the descriptive framework of Fuzzy Set Theory (Zadeh 1965, 1984).

2. Research Framework

One framework for scientific endeavor has been expressed most succinctly by Karl Popper (1959). The central assumption is that hypothesis testing must follow deductive rather than inductive methods. Following Hume, Popper claims that we are not justified in inferring universal statements from singular ones. Any conclusion drawn inductively might always turn out to be false. Although we can generate many instances of positive results, the theory might still be exposed as false. As scientists, we should guard against trying to verify a particular hypothesis by demonstrating that it works in specific instances. Given that new instances can always falsify a given statement, no experimental observation can verify a hypothesis.

2.1 Falsification

Popper proposes that a hypothesis, once constructed, must be subjected to the following analysis. The investigator begins by comparing the conclusions derived from the hypothesis in order to determine whether they are internally consistent. An analysis of the conclusions will also indicate whether or not the hypothesis is testable. By contrasting this hypothesis with other hypotheses, the investigator determines whether the theory is unique and whether it would constitute a scientific advance should the hypothesis survive experimental tests. If the conclusions drawn from the hypothesis meet these requirements, it is worthwhile to subject its conclusions to experimental tests.

Experimental tests will decide how well a hypothesis or theory survives. If a theory survives the experimental tests, we should not discard it. On the other hand, if the experimental tests falsify conclusions drawn from the theory, then the theory should be rejected or modified accordingly. In a more recent contribution, Popper (1976) acknowledged that models could be modified indefinitely to incorporate inconsistent results. This prolongation of falsification is called immunization. Successive modification of a model keeps it alive and holds of...

Table of contents

• Cover
• Title Page
• Copyright Page
• Dedication
• Preface
• Acknowledgements
• Table of Contents
• Table of Phonetic Symbols
• 1. Scientific Framework for Psychological Inquiry
• 2. Single versus Multiple Sources of Speech Information: The Contribution of Visible Speech
• 3. Integration versus Nonintegration of Auditory and Visual Information in Speech Perception
• 4. Categorical versus Continuous Information in Speech Perception: The Relationship between Identification and Discrimination
• 5. Categorical versus Continuous Information in Speech Perception: Model Tests and Ratings
• 6. Independent versus Dependent Evaluation of Sources
• 7. Additive, Minimization, or Multiplicative Integration
• 8. Lifespan Changes in Speech Perception by Ear and Eye
• 9. Specificity versus Generality of the Findings
• References
• Author Index
• Subject Index