Connectionism and Second Language Acquisition

Yasuhiro Shirai

• Publisher: Routledge
• Year: 2018
• Print ISBN: 9780415528924
• eBook ISBN: 9781136307676
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

The latest title in the Cognitive Science and Second Language Acquisition Series presents a comprehensive review of connectionist research in second language acquisition (SLA). Second language researchers and the cognitive science community will find accessible discussions of the relevance of connectionist research to SLA. This important volume is key reading for any student or researcher interested in how second language acquisition can be better understood from a connectionist perspective.

1

OVERVIEW

Connectionism and Language Acquisition Research

This book presents a comprehensive review of connectionist research as it pertains to second language acquisition (SLA). Since the 1980s, connectionism has had a strong impact on cognitive science, and has transformed the way many cognitive scientists approach the task of understanding human cognition. Although the impact has not been felt as strongly in the field of SLA, a number of studies have been published during the past 30 years that are directly relevant to connectionism and second language acquisition. The purpose of this book is to present this body of research in a way that is accessible to second language researchers as well as to the cognitive science community in general, to inform the field of how SLA can be better understood from a connectionist perspective and of how SLA data can be relevant to cognitive science in general.

The book is organized as follows. This chapter (Chapter 1) introduces the connectionist approach to cognition and language. I explicate its historical development as well as its strengths and limitations. Chapter 2 discusses what connectionism offers second language acquisition research. Chapters 3 to 5 focus on specific issues surrounding connectionist approaches to second language acquisition. Chapter 3 reviews research on rules vs. connections, with special emphasis on the processing and acquisition of regular vs. irregular morphology, which has been the center of the theoretical debate surrounding connectionism and language since the 1980s (i.e. dual- vs. single-mechanism debate). Chapter 4 presents a review of connectionist research related to the critical period hypothesis, or age factor in second language acquisition, one of the most contentious issues in SLA and cognitive science. Chapter 5 focuses on research on SLA that offers new insights by utilizing the connectionist perspective, and its application to other areas of L2 research, such as second language teaching and assessment.

What Is Connectionism?

Connectionism is an approach to understanding human cognition that has its basis in a particular kind of computational modeling called neural networks. Neural networks generally use digital computers, but its computational algorithm tries to emulate how the brain stores and processes information.

Cognitive science is an interdisciplinary field that tries to understand human cognition from multiple perspectives, such as psychology, linguistics, computer science, anthropology, and so forth. The field emerged in the 1950s as a discipline, and, in its early days, computer science played a very important role. Computer modeling was (and still is) considered to be an important window to understanding human information processing. In the early days of artificial intelligence (AI) research, researchers hoped that computers would be able to simulate and/or emulate human cognition eventually, and a large amount of research funding was devoted to AI research. However, although much progress was made in this area, there still was a long way to go for a computer to attain human-like intelligence. Why would that be?

Symbolic Paradigm and Connectionism

One major problem in attaining human-like ability by computers was that although they can deal with some tasks with extraordinary facility, they fail miserably in other tasks that are very easy for humans, such as face recognition and language. This is generally considered to be due to the nature of a computation method largely dependent on symbol manipulation. I will outline some historical backgrounds that led to this realization.

By the end of the 1960s, what is often called the ‘cognitive revolution’ had put an end to behaviorism as a dominant paradigm in psychology. The stimulus– response theory of learning and cognition promoted by Skinner (1957) lost its legitimacy, mainly because of harsh criticisms from Chomsky (1959) and others, which showed that infinite creativity of language cannot be explained by associationist learning espoused by behaviorism. As a result, what is now called the ‘symbolic paradigm’ (e.g. Newell, 1980; Newell & Simon, 1976) has taken over as the dominant approach, which is often considered to be a paradigm shift (Kuhn, 1962) in cognitive science.

The problem, however, lay in this paradigm shift to some degree. The symbolic approach, in which symbols are manipulated through rules, has become a central approach in traditional AI, including natural language processing. The symbol system approach has its problem in that when faced with fuzzy data, it cannot handle them very well, even in cases where humans easily can. Thus, researchers began to wonder if it may be necessary to take a different approach (Allman, 1989).

Before this shift to the symbolic paradigm, there existed a computational model that can better handle fuzzy data. It is called ‘perceptron’, which was developed by Rosenblatt (1961), who built on earlier work such as Pitts and McCulloch (1947) and von Neumann (1956). These network models were used to simulate vision and other perceptual cognition, including learning, and had some important characteristics of the present day connectionist models, such as distributed representation and neural plausibility (to be discussed below). It was hoped that perceptrons could be utilized to help advance AI research. However, Minsky and Pappert (1969) practically nullified this hope by showing through mathematical reasoning that perceptrons cannot handle complex human cognition, after which mainstream AI research shifted to symbolic computers.¹

The symbolic paradigm has its roots in philosophical logic and computer science. It tried to model human cognition by symbols and rules. “If X, then Y”, for example, has X and Y as symbols, and “If . . . then” as rules. Phrase structure rules in linguistics such as “S -> NP VP”² is also an example. By manipulating symbols with rules, traditional AI tried to model human cognition. Although the perceptron-type network approach was pursued through the 1960s, the cognitive science community moved toward the symbolic approach, as noted above, and the 1970s were dominated by the symbolic AI.

The network approach, however, revived in the 1980s. Through the 1970s, researchers such as J. A. Anderson (1972), Kohonen (1972), and Grossberg (1976) continued network-based approaches to human cognition. The connectionist movement was also spurred by the fact that many connectionist researchers were at the University of California at San Diego around the early to mid 1980s, which resulted in the publication of the two-volume connectionist manifest, Parallel Distributed Processing (Rumelhart, McClelland, & the PDP research group, 1986; McClelland, Rumelhart, & the PDP research group, 1986).

The symbolic camp quickly reacted. In a special issue of Cognition, which was devoted to the criticism of connectionist models published in the two PDP volumes, Steven Pinker, Alan Prince, Jerry Fodor, Zenon Pylyshyn, and others pointed out the shortcomings of the connectionist models and their philosophy in general, and of Rumelhart and McClelland’s (1986) past-tense learning model (often referred to as the RM model) in particular. This, however, did not deter researchers from pursuing connectionism as a viable approach to understanding human cognition. Since then, it has become quite commonplace in cognitive science to incorporate connectionist simulation. If one goes to the Cognitive Science Society meetings, many researchers present both behavioral data collected from humans and connectionist simulation as a matter of fact. In many psychology and cognitive science departments in North America and the UK and Europe, it is now common to have a connectionist modeler among the faculty.

The Attraction of Connectionism

One of the reasons why connectionism was not abandoned even after concerted efforts by the symbolic camp to demote its appeal was the realization about the limitation of symbol manipulation. As briefly noted above, the symbolic AI is not very good at handling things that humans are good at. This had made researchers wonder whether it is its fundamental property that is not very suitable for simulating human cognition. In contrast, connectionist models do have important properties that are shared by humans, which made it very attractive, such as neural plausibility, satisfaction of soft constraints, graceful degradation, graded representation, and the capacity to learn from experience (Bechtel & Abrahamson, 2002; Gasser, 1990).

1. Neural plausibility: It is well known that connectionist models are modeled after how neurons work (hence the term neural networks). The precursors of perceptron (e.g. Pitts & McCulloch’s (1947) network mentioned earlier) was devised to model neurons. Activation of the unit is 0 or 1, which is supposed to model whether a neuron is in resting state or firing. The connections between units are to represent axons and dendrites which connect neurons in the brain. The brain also has massively connected neurons which perform parallel processing. These neurally plausible properties of the connectionist model gives researchers instant appeal as to its promise as a model to simulate human cognition.

2. Satisfaction of soft constraints: Rule-based systems generally rely on all-or-nothing rules, so if a rule applies it applies, and if not, it doesn’t. In contrast, connectionist models are better at handling gray areas. In connectionist models, when a unit receives excitatory activation from another unit, it fires, but if it also receives inhibitory activation at the same time from other units, then it may not, depending on how much activation a unit receives from different sources. Thus, it is better suited for handling soft constraints, which is abundant in real-life cognitive activities that we engage in.

3. Graceful degradation: Human behavior does not break down completely if some aspects of our cognitive system break down. When we are overloaded (with anxiety, fatigue, too much information, etc.), or the system is partially damaged (due to injury), we still do not break down completely but somehow carry out a task with subpar performance. A traditional symbolic system completely breaks down in such cases, which is often called ‘brittleness’ of the symbolic approach. In contrast, connectionist networks do not fail completely. Their performance does not significantly impair if a few connections or even units are destroyed.

4. Graded representation: While symbolic models represent concepts in all or nothing fashion, connectionist models handle categories in non-discrete way, some being better members than others. This type of representation is more compatible with most of the natural and linguistic categories that we have (such as bird, table, mug, decision, war, truth, past tense -ed, plural –s, nouns, verbs) that we deal with every day.

These properties of connectionism have made it attractive to many cognitive scientists and psychologist. One of the areas in which connectionism was extensively applied was language acquisition (Seidenberg, 1997).

Emergentism and Dynamical Systems Approach

One of the key notions of connectionist research is ‘emergent’. As noted above, the R&M model of past tense acquisition has shown that what is usually considered to be the result of the application of rules may in fact be a rule-like behavior, resulting from learning that comes from the pattern of network activation learned through patterns of input–output association, and thus emergent phenomena. In fact, this type of emergent phenomena that appear to be rule-governed is ubiquitous. When cars line up at tollgates, the length of each line is almost always equal. This does not mean, of course, there is such a rule or regulation. In fact, it is an emergent phenomenon coming from some other motivations (e.g. drivers want to go through the gate as fast as possible). Another example often cited is the hexagonal shape of each component used by honeybees to build a beehive. One wonders whether honeybees are innately preprogrammed to build a hexagonal shape, but it is, in fact, due to environmental necessity that they use the hexagonal shape—it is the most efficient way to build a beehive (e.g. Elman, Bates, Johnson, Karmiloff-Smith, Parisi, & Plunkett, 1996). Thus, what is often considered to be determined a priori by rules may, in fact, be an emergent phenomenon. The argument from connectionists is that such emergent phenomena are actually dominant processes characterizing human cognition, rather than (innate) rules specifying our behavior. In other words, it is the pattern of activation that is subserving rule-like behavior.³

How, then, do these rule-like (or sometimes random-looking) behaviors emerge? A new approach that has come to influence connectionist explorations is dynamical systems theory (DST, Port & van Gelder, 1991, 1995). A mathematical theory often used in physics, economics, motor control, biology and other disciplines, DST takes explanation of complex dynamic systems seriously, in which dynamic change over time is the target of explanation and the initial condition of the system is often important in the determination of long-term behaviors. Chaos theory is often used to explain such complex adaptive systems. This approach was argued to be a paradigm shift (Kuhnian revolution) in cognitive science by Tim van Gelder and Robert Port in the 1990s, which supplants, rather than complements, a computer-based approach to cognition. Traditional approaches in computational cognitive science, both classical symbolic AI and connectionism are similar in that cognitive science implicitly assumed that there is a computer in the mind/brain, when, in fact, the cognitive system “is comprised of nervous system, body and environment” (van Gelder & Port, 1995, p. 3). Despite their claim, however, the field has moved on to integrate connectionism and dynamical theory, as reflected in the 2009 edited volume by Spencer, Thomas, and McClelland (2009; see also McClelland et al., 2010). This may be done either by analyzing connectionist models’ representation from dynamical perspectives (for its non-linear changes in hidden-units representations, e.g. Elman, 1993), and/or by designing connectionist models more dynamically motivated (Freeman, 1987).

Connectionism and Language Acquisition Research

The goal of language acquisition research is to understand the mechanism of language acquisition: How it is possible to acquire language? What is the mechanism by which children can acquire the language they are exposed to despite its complex structures? To answer this question, one needs to understand both the nature of language (as target of acquisition) and the nature of learning (the mental processes that make it possible for us to know and use the language, from the stage of infancy without language ability to a mature, competent speaker).

This question has been approached in various ways for a long time, but the modern scientific inquiry started around the 1950s. At the beginning, however, the research was mostly based on speculative theoretical inquiries...

Table of contents

• Cover Page
• Connectionism and Second Language Acquisition
• Cognitive Science and Second Language Acquisition Series
• Title
• Copyright
• Contents
• List of Figures
• List of Tables
• Series Editor’s Preface
• Acknowledgments
• 1 Overview: Connectionism and Language Acquisition Research
• 2 Connectionism and SLA Research
• 3 Rules vs. Connections
• 4 The Critical Period Hypothesis
• 5 Connectionist Accounts and Applications
• 6 Conclusions and Future Directions
• Recommended Reading
• References
• Index