Proceedings of the 2001 Fourth International Conference on Cognitive Modeling

Erik M. Altmann, Axel Cleeremans, Christian D. Schunn, Wayne D. Gray, Erik M. Altmann, Axel Cleeremans, Christian D. Schunn, Wayne D. Gray

• Publisher: Psychology Press
• Year: 2001
• Print ISBN: 9780805840421
• eBook ISBN: 9781135640798
• Topic: Psychology
• Subtopic: Cognitive Psychology & Cognition
• Index: Psychology

This book documents the proceedings of the Fourth International Conference on Cognitive Modeling (ICCM), which brought together researchers from diverse backgrounds to compare cognitive models; evaluate models using human data; and further the development, accumulation, and integration of cognitive theory. ICCM provides a worldwide forum for cognitive scientists who build computational cognitive models and test them against empirical cognitive data. New features of the conference included the Newell Award for best student paper, the Siegel-Wolf Award for best applied research paper, and a Doctoral Consortium which provided an opportunity for students to meet their peers and mentors and to explore their dissertation work in an intense but friendly, multi-approach environment. These new features, as well as a comprehensive view of the posters presented at the conference are offered in this volume.

Presented Papers

A Modular Neural-Network Model of the Basal Ganglia’s Role in Learning and Selecting Motor Behaviours

Gianluca Baldassarre ([email protected])
Department of Computer Science, University of Essex
CO4 3SQ Colchester, UK

Abstract

This work presents a modular neural-network model (based on reinforcement-learning actor-critic methods) that tries to capture some of the most-relevant known aspects of the role that basal ganglia play in learning and selecting motor behavior related to different goals. In particular some simulations with the model show that basal ganglia selects “chunks” of behaviour whose “details” are specified by direct sensory-motor pathways, and how emergent modularity can help to deal with multiple behavioral tasks. A “top-down” approach is adopted. The starting point is the adaptive interaction of a (simulated) organism with the environment, and its capacity to learn. Then an attempt is made to implement these functions with neural architectures and mechanisms that have a neuroanatomical and neurophysiological empirical foundation.

Introduction, Methodology, Empirical Evidence Addressed

What is the role that basal ganglia play in mammals’ sensory-motor behaviour? When organisms have different needs/goals, sometimes they have to associate slightly different behaviours to the same perception patterns, some other times they have to associate completely different behaviours to them. This work presents some simulations that suggest that in the former case the differences are dealt with within the same sensory-motor pathway (implemented by a neural module) while in the later cases different sensory-motor pathways are selected. In fact if the behavioral response to associate to a given perception were different with different needs/goals, using the same neural synapses/pathways would only cause interference. In this context basal ganglia could play a role in selecting different sensory-motor pathways when necessary.

This work follows a “top-down” approach, where the starting point is organisms’ behaviour and learning processes (cf. Meyer & Guillot, 1990). On this purpose it presents a simulation of an organism that has different needs (signals coming from the body and indicating a physiological unbalance, cf. Rolls, 1999) or, alternatively, different goals (desired states of body-world) associated to different positions in the environment (for example we can assume that these different positions are occupied by resources that satisfy different needs). The organism learns through classical and instrumental learning (Lieberman, 1993; in Baldassarre & Parisi, 2000, these two learning mechanisms are integrated in a comprehensive actor-critic model. Cf. Barto, 1995, and Sutton & Barto, 1998, for this model) to navigate in the environment in order to reach those positions. Given this behaviour as a starting point, the work attempts to yield it by building a neural-network controller that satisfies (some of) the constraints coming from the known empirical evidence about basal ganglia. Since the starting point of this approach is to simulate sophisticated organisms’ behaviours, sometimes there is no empirical data suggesting which mechanisms underlie them. In these cases some computational solutions are adopted that do not have a known empirical correspondent (they will be appealed as “arbitrary” in the rest of the paper). These solutions should be considered as a useful theoretical exercise, eventually suggesting interesting ideas to the empirical investigation, and should not be judged too severely on the basis of the neural evidence.

The anatomical and physiological evidence specifically addressed in this work is now illustrated. Chevalier & Deniau (1990) propose that a double-inhibition mechanism is the basic process of basal ganglia’s functioning. They report that in some experiments where monkeys have to carry out a delayed saccade to a remembered target, some striatal cells (usually mute) are induced to fire with local injection of glutammate. The striatal discharge inhibits (via GABAergic connections) a group of cells in the substantia nigra pars reticulata (usually tonically active) that release from (GABAergic) inhibition a subset of cells of the superior colliculus responsible for the saccade. In the case of skeletal movements the double inhibition releasing mechanism is implemented by the striatum-globus pallidus-thalamus pathway. The authors report that while in rodents this mechanism is sufficient to trigger movements, in the reported experiments the execution of a saccade requires temporal coincidence of basal ganglia disinhibition with command signals from other sources. This aspect is present in the model: basal ganglia select a particular sensory-motor pathway that then yields the detailed behavioral output.

Graybiel (1998), addressing the role that the basal ganglia’s neural modules play in human slow habit learning and animal stimulus response association, draws an abstract parallel between the striatum’s anatomical organization in partly interconnected zones, called “matrisomes”, and the modular architecture of the neural networks of Jacobs et al. (1991). As we shall see, the computational model presented here proposes a possible way to specify such parallel.

Houk, Adams, & Barto (1995) suggest a possible correspondence between the actor-critic models’ architecture and functioning (Barto, 1995; Sutton & Barto, 1998) and the architecture of the basal ganglia. In particular they propose that the circumscribed regions called “striosomes” (differently from matrisomes, they are identifiable for their chemical make-up and output connectivity) may implement the function of the critic (predicting future rewards and yielding a step-by-step reward signal in cases of delayed rewards) and the surrounding “matrix” regions may implement the function of the actor (selecting actions or, as in the model presented here, sensory-motor pathways). As we shall see, the actor-critic model is at the base of the model presented here.

Lots of other aspects of these contributions have been incorporated in the model, and will be presented in detail in the next section. The numerous brain-imaging studies of basal ganglia’s role in sequence learning are not directly addressed in this paper (see Graybiel, 1998, for some references).

Scenario and Model of Basal Ganglia

The environment used in the simulations is a square arena with sides measuring 1 unit (Figure 1). The organism cannot see the boundaries of the arena and cannot exit it. Inside the arena there are 5 circular landmarks/obstacles that the organism can see with a one-dimension horizontal retina covering 360 degrees with 50 contiguous sensor units. Each unit gets an activation of 1 if a landmark is in its scope, of 0 otherwise. The signals coming from the retina are aligned with the magnetic north through a compass. Before being sent to the controller, these signals are remapped into 100 binary units representing the image “contrasts”. Two contiguous retinal units activate one contrast unit if they are respectively on and off, another contrast unit if they are respectively off and on, no contrast units if they are both on or both off (cf. Figure 1). At each cycle of the simulation the organism selects one of eight actions, each consisting in a 0.05 step in one of eight directions aligned with the magnetic north (north, northeast, east, etc.). The outcome of these actions is affected by a Gaussian noise (0 mean, 0.01 variance). The organism’s task is to reach one of the three goal positions showed in Figure 1.

Figure 2 illustrates the main features of the organism’s controller and the possible brain areas and nuclei corresponding to the model’s components. Now a computational description of the controller is given, and its possible links to the mammal brain’s neural structures are ill...

Table of contents

• Preface
• ICCM Committee
• Invited Address
• Allen Newell Award for Student Paper
• Siegel-Wolf Award for Best Applied Paper
• Symposium
• Presented Papers
• Posters
• Doctoral Consortium Posters
• Author Index