Cognitive Approach to Natural Language Processing

Bernadette Sharp, Florence Sedes, Wieslaw Lubaszewski

• Publisher: ISTE Press - Elsevier
• Year: 2017
• Print ISBN: 9781785482533
• eBook ISBN: 9780081023433
• Topic: Computer Science
• Subtopic: Natural Language Processing
• Index: Computer Science

As natural language processing spans many different disciplines, it is sometimes difficult to understand the contributions and the challenges that each of them presents. This book explores the special relationship between natural language processing and cognitive science, and the contribution of computer science to these two fields. It is based on the recent research papers submitted at the international workshops of Natural Language and Cognitive Science (NLPCS) which was launched in 2004 in an effort to bring together natural language researchers, computer scientists, and cognitive and linguistic scientists to collaborate together and advance research in natural language processing.The chapters cover areas related to language understanding, language generation, word association, word sense disambiguation, word predictability, text production and authorship attribution. This book will be relevant to students and researchers interested in the interdisciplinary nature of language processing.- Discusses the problems and issues that researchers face, providing an opportunity for developers of NLP systems to learn from cognitive scientists, cognitive linguistics and neurolinguistics- Provides a valuable opportunity to link the study of natural language processing to the understanding of the cognitive processes of the brain

1

Delayed Interpretation, Shallow Processing and Constructions: the Basis of the “Interpret Whenever Possible” Principle

Philippe Blache

Abstract

From different perspectives, natural language processing, linguistics and psycholinguistics shed light on the way humans process language. However, this knowledge remains scattered: classical studies usually focus on language processing subtasks (e.g. lexical access) or modules (e.g. morphology, syntax), without being aggregated into a unified framework. It then remains very difficult to find a general model unifying the different sources of information into a unique architecture.

Keywords

Aggregating by cohesion; Chunks; Delayed processing; “Interpret Whenever Possible” Principle; Segment-and-store; Segmentation operations; Working memory

We propose in this chapter to investigate the “interpret whenever possible” principle that consists of delaying the processing mechanisms until enough information becomes available. This principle relies on the identification of elementary units called chunks, which are identified by means of basic features. These chunks are segments of the input to be processed. In some cases, depending on the accessibility of the information they bear, chunks can be linguistically structured elements. In other cases, they are simple segments. Chunks are stored in a buffer of the working memory and progressively grouped (on the basis of a cohesion measure) when possible, progressively identifying the different constructions of the input. The global interpretation of a linguistic input is then not based anymore on a word-by-word mechanism, but on the grouping of these constructions that constitute the backbone of the “interpret whenever possible” principle.

1.1 Introduction

From different perspectives, natural language processing, linguistics and psycholinguistics shed light on the way humans process language. However, this knowledge remains scattered: classical studies usually focus on language processing subtasks (e.g. lexical access) or modules (e.g. morphology, syntax), without being aggregated into a unified framework. It then remains very difficult to find a general model unifying the different sources of information into a unique architecture.

One of the problems lies in the fact that we still know only little about how the different dimensions of language (prosody, syntax, pragmatics, semantics, etc.) interact. Some linguistic theories exist, in particular within the context of Construction Grammars [FIL 88, GOL 03, BLA 16], that propose approaches making it possible to gather these dimensions and implement their relations. These frameworks rely on the notion of construction, which is a set of words linked by specific properties at any level (lexical, syntactic, prosodic, etc.) and with which a specific meaning, which is often non-transparent or accessible compositionally (e.g. idioms or multi-word expressions), can be associated. Interestingly, these theories also provide a framework for integrating multimodal information (verbal and non-verbal). Interpreting a construction (i.e. accessing to its associated meaning) results from the interaction of all the different dimensions. In this organization, processing a linguistic production is not a linear process, but uses mechanisms for a global recognition of the constructions. Contrarily to incremental architectures (see, e.g., [FER 02, RAY 09]), the syntactic, semantic and pragmatic processing is not done word-by-word, but more globally, on the basis of such constructions.

This conception of language processing requires a synchronization procedure for the alignment of all the different sources of information in order to identify a construction and access its meaning. In natural situations (e.g. conversations), the different input flows can be verbal (prosody, syntactic, pragmatics, etc.) and non-verbal (gestures, attitudes, emotions, context, etc.); they are not strictly temporally synchronized. It is then necessary to explain how information can be temporarily stored and its evaluation delayed until enough information becomes available. In this perspective, the input linguistic flow (being read or heard) is segmented into elements that can be of any form, partially or entirely recognized: segments of the audio flow, set of characters, and also, when possible, higher-level segments made of words or even clusters of words. In this chapter, we address these problems through several questions:

1) What is the nature of the delaying mechanism?

2) What is the nature of the basic units and how can they be identified?

3) How is the delaying mechanism implemented?

1.2 Delayed processing

Different types of delaying effects can occur during language processing. For example, at the brain level, it has been shown that language processing may be impacted by the presentation rate of the input. This phenomena has been investigated in [VAG 12], claiming that when the presentation rate increases and becomes faster than the processing speed, intelligibility can collapse. This is due to the fact that language network seems to work in a constant of time: cortical processing speed is shown by the authors to be tightly constrained and cannot be easily accelerated. As a result, when the presentation rate increases, the processing speed remaining constant, a blocking situation can suddenly occur. Concretely, this means that when the presentation rate is accelerated, and because the processing speed remains constant, part of the input stream has to be buffered. Experiments show that the rate can be accelerated to 40% before reaching a collapse of intelligibility. This situation occurs when the buffer becomes saturated and is revealed at the cortical level by the fact that the activation of the higher-order language areas (that are said to reflect intelligibility [FRI 10]) drops suddenly, showing that the input signal becomes unintelligible.

This model suggests that words can be processed immediately when presented at a slow rate, in which case the processing speed is that of the sensory system. However, when the rate increases and words are presented more rapidly, the processing speed limit is reached and words cannot be processed in real time anymore. In such a situation, words have to be stored in a buffer, from which they are retrieved in a first-in-first-out manner, when cognitive resources become available again. When the presentation rate is higher than the processing speed, the number of words to be stored increases. A lock occurs when the maximal capacity of the buffer is reached, entailing a collapse of intelligibility.

Besides this buffering mechanism, other cues indicate that the input is probably not processed linearly, word-by-word, but rather only from time to time. This conception means that even in normal cases (i.e. without any intelligibility issue), the interpretation is only done periodically, the basic units being stored before being processed. Several studies have investigated such a phenomenon. At the cortical level, the analysis of stimulus intensity fluctuation reveals the presence of specific activity (spectral peaks) after phrases and sentences [DIN 16]. The same type of effect can also be found in eye-movement during reading: longer fixations are observed when reading words that end a phrase or a sentence. This wrap-up effect [WAR 09], as well as the presence of different timescales at the cortical level described above, constitute cues in favor of a delaying mechanism in which basic elements are stored temporarily, and an integration operation is triggered when enough material becomes available for the interpretation.

Figure 1.1 Illustration of the bottleneck situation, when the presentation rate exceeds the processing speed (reproduced from [VAG 12])

At the semantic level, other evidence also shows that language processing, or at least language interpretation, is not strictly incremental. Interesting experiments have been performed, which reveal that language comprehension can remain very superficial: [ROM 13] has shown that, in an idiomatic context, the access to the meaning of words can be completely switched off, replaced by a global access at the level of idiom. This effect has been shown at the cortical level: when introducing a semantic violation within an idiom, there is no difference between hard and soft semantic violations (which is not the case in a comparable non-idiomatic context); in some cases, processing a word does not mean integrating it into a structure. On the contrary, in this situation there is a simple shallow process of scanning the word, without doing any interpretation. The same type of observatio...

Table of contents

• Cover
• Title page
• Table of Contents
• Copyright
• Preface
• 1: Delayed Interpretation, Shallow Processing and Constructions: the Basis of the “Interpret Whenever Possible” Principle
• 2: Can the Human Association Norm Evaluate Machine-Made Association Lists?
• 3: How a Word of a Text Selects the Related Words in a Human Association Network
• 4: The Reverse Association Task
• 5: Hidden Structure and Function in the Lexicon
• 6: Transductive Learning Games for Word Sense Disambiguation
• 7: Use Your Mind and Learn to Write: The Problem of Producing Coherent Text
• 8: Stylistic Features Based on Sequential Rule Mining for Authorship Attribution
• 9: A Parallel, Cognition-oriented Fundamental Frequency Estimation Algorithm
• 10: Benchmarking n-grams, Topic Models and Recurrent Neural Networks by Cloze Completions, EEGs and Eye Movements
• List of Authors
• Index