Neural Network Models of Conditioning and Action

Michael L. Commons, Stephen Grossberg, John Staddon, Michael L. Commons, Stephen Grossberg, John Staddon

• Publisher: Routledge
• Year: 2016
• ISBN:
  9781317275961
• Edition: 1
• Topic: Psychology
• Subtopic: Cognitive Neuroscience & Neuropsychology
• Index: Psychology

Originally published in 1991, this title was the result of a symposium held at Harvard University. It presents some of the exciting interdisciplinary developments of the time that clarify how animals and people learn to behave adaptively in a rapidly changing environment. The contributors focus on aspects of how recognition learning, reinforcement learning, and motor learning interact to generate adaptive goal-oriented behaviours that can satisfy internal needs – an area of inquiry as important for understanding brain function as it is for designing new types of freely moving autonomous robots.
Since the authors agree that a dynamic analysis of system interactions is needed to understand these challenging phenomena – and neural network models provide a natural framework for representing and analysing such interactions – all the articles either develop neural network models or provide biological constraints for guiding and testing their design.

I MODELS OF CLASSICAL CONDITIONING

1 Memory Function in Neural and Artificial Networks

Daniel L. Alkon

National Institutes of Health,

Laboratory of Molecular and Cellular Neurobiology

Bethesda, Maryland

Thomas P. Vogl
Kim T. Blackwell
ERIM, Arlington, Virginia

David Tam

University of California, Irvine

ABSTRACT

The DYSTAL (Dynamically Stable Associative Learning) model is an artificial modifiable neural network based on observed features of biological neural systems in the mollusc Hermissenda and the rabbit hippocampus. In the DYSTAL network, synaptic weight modification depends on (a) convergence of modifiable “collateral” and unmodifiable “flow-through” inputs, (b) temporal pairing of these inputs, and (c) past activity of elements receiving the inputs. Modification is independent of element output. As a consequence, DYSTAL shows (a) linear scaling of computational effort with network size, (b) rapid learning without an external “teacher,” and (c) ability to complete patterns, independently associate different ensembles of inputs, and to serve as a classifier of input patterns.

Our memories are of complex patterns of stimuli—not of individual stimuli in isolation of each other. An image of a face is one such pattern, a melodic refrain is another. It is as if the pieces of a pattern are linked to or associated with each other so that awareness of a critical number of pattern elements or pieces activates the links to restore or recall the entire pattern. Attention to a distinctive scar can cause us to recall the appearance of a face encountered in the past. Hearing a few notes in a particular sequence may trigger a memory of a symphonic movement.

Mammalian memory formation may ultimately be reducible to link formation between stimulus elements in a pattern. Two lines of evidence from our laboratory provide support for this hypothesis. Our physiologic observations demonstrate remarkable similarities between molecular and biophysical mechanisms for learning links or associations in the snail Hermissenda and in the rabbit hippocampus (Alkon, 1989; Alkon, Quek, & Vogl, 1989). Association formation within a system of a relatively few Hermissenda neurons involves cellular substrates identical to those of hippocampal systems with vast arrays of neurons. Storage in such hippocampal arrays could arise, therefore, from multiplication of storage events within minimal neuronal nets such as the visual-vestibular network of Hermissenda. Induction of theoretical networks from these biological networks provides additional support for such an inference. My colleagues, Tom Vogl and Kim Blackwell, and I have started to construct artificial networks whose elements interact with each other and modify their weight according to principles abstracted from the Hermissenda and rabbit neural systems. A number of promising features of our artificial network, such as linear scaling, rapid operation, and lack of an external teacher, suggest that by simply increasing the number of elementary associatively modified links between neurons, nature may have enormously eased the computational burden on our brains during cognitive functions.

Our physiologic studies have focused on learned associations as a result of Pavlovian conditioning. Pavlovian conditioning results from a rudimentary temporal pattern: one stimulus followed after a constant delay by a second stimulus. We learn that a flash of lightening is followed by thunder. A snail learns that a flash of light is followed by rotation induced turbulence. For the snail, the first step in establishing a temporal link between a light stimulus and a rotation stimulus occurs when electrical signals in the snail’s visual pathway arrive at particular network locations together in time with electrical signals from the snail’s vestibular pathway. At these common visual-vestibular loci (one being a neuron called the type B cell) the electrical response, depolarization, is unique to the temporal relationship of the light and rotation stimuli. Chemical signals unique to the temporal pairing of light and rotation continue the linkage formation. Calcium flows through channels within the B cell outer membrane and becomes elevated within the cytoplasmic compartment. Our observations suggest that the linkage process extends its duration when calcium and diaclyglycerol together cause movement of protein kinase C (PKC) from the cytoplasm into the membrane. Observations implicating PKC in Hermissenda conditioning include:

1. Learning-specific differences in phosphorylation of PKC substrates and the duplication of such differences by exposure to phorbal ester.
2. Blocking of learning-induced biophysical changes by blocking PKC translocation with sphingosine and PKC-mediated phosphorylation with H7.
3. Production of learning-induced biophysical changes by activating PKC with phorbol ester, etc.
4. Learning-specific changes in the amounts of PKC protein substrates and the m-RNA which encodes for them.

The light rotation temporal link in Hermissenda is recalled when depolarization of the Type B cell elicits a markedly reduced K⁺ flow through specific channels. At the resting potential of the cell membrane, after the acquisition phase, there is no difference in K⁺ flow. Thus, recall depends on the voltage dependence of the K⁺ channel transformation. Pharmacologic blocking experiments indicate that this voltage dependence involves calcium flowing into the cell to activate a much more sensitive membrane-associated PKC. This PKC-regulated K⁺ flow remains reduced weeks after the training. This then is an entirely new time domain for biophysics, beautifully designed for associative memory and never encountered before in fully differentiated neurons. In Hermissenda, it was this biophysical difference that was causally implicated for storage of the associative memory. Furthermore, we found evidence of this same new biophysics for biological signaling in the rabbit hippocampus where causal implication was not possible but correlation was.

More permanent memory storage almost certainly involves changes of protein synthesis, most likely as an extension of second messenger function into more prolonged temporal domains. In Hermissenda, regulation of protein synthesis profoundly affects calcium-stimulated reduction of K⁺ ion flow, which occurs as a consequence of PKC translocation. That associative memory storage involves changes in the synthesis of specific proteins was more directly demonstrated by relating the learning behavior of living Hermissenda to protein and RNA metabolism of neurons functionally implicated in memory storage. Up to four days after the cessation of all training, the efficacy of memory storage was closely correlated with increases in m-RNA synthesis within the Hermissenda eye (whose only neuronal elements are the five photoreceptors that include 3 type B cells). There was also close correlation of memory storage with the quantity of specific proteins, one of which, the 20,000 kDa protein, a PKC and Ca²⁺/CAM-II kinase substrate, showed differences in phosphorylation shortly after memory acquisition. Furthermore, synthesis of m-RNA with distinct molecular weights was also closely related to memory storage. Again, the PKC substrate with 20,000 kDa m.w., which we found to have GTP-ase activity and binds GTP, appeared to correspond to an m-RNA species (of similar m.w.) that was clearly related to the efficacy of memory storage. (In addition, two other proteins, one a GTP-ase, the other most likely a structural constituent, were correlated with memory storage). Injection of the 20,000 m.w. protein, now known to be a GTP-binding protein, exactly simulated the effects of conditioning on the Type B cell. The number and quantity of species of m-RNA and proteins altered with memory storage suggest profound alterations of cellular metabolism. Such alteration might occur during substantial structural changes as can occur during growth and development.

Marked structural alterations were in fact bound to be specific to associative memory storage in Hermissenda. Blinded comparisons of type B cell among the three groups demonstrated a conditioning-induced focusing of terminal branches where synaptic interactions occur. The magnitude of branching volume was also clearly related to the magnitude of K⁺ flow reduction. Such learning-induced focusing may share properties with “synapse elimination” studied in developmental contexts.

In the snail Hermissenda, extension of memory storage from one major cellular compartment, the cell body, to another, the terminal branches, accompanies extension of the time domain from short-term to more permanent periods. An equally dramatic example of the same parallelism of temporal and spatial domains was found in the rabbit hippocampus. One day after the rabbit was learned to link a tone with an air puff to its eye, we found changes in the electrical properties of CAI pyramidal cells within the rabbit’s hippocampus. In slices isolated from conditioned vs. control animals there was reduced flow through the same K⁺ channels that we had previously found to be altered when the snail was conditioned. Just as with the snail neurons, drug-induced movements of PKC from the cytoplasm into the membrane of the CAI cells produced the same reduction of K⁺ ion flow. Furthermore, fractionation of the CAI cell region into membrane and cytosolic fractions revealed an unequivocal movement of the PKC from the cytoplasm into the membrane compartment only in conditioned animals.

Autoradiographic labeling experiments confirmed this conditioning-specific movement of PKC in the region of the cell bodies as well as the sites of synaptic interactions called dendrites (Fig. 1.1). The spatial temporal parallelism was most dramatically demonstrated in the distribution of the PKC label between cell body and dendritic layers. One day after conditioning, the cell body region had more PKC label than did the dendritic region. But three days after conditioning, this relationship was completely reversed. As the memory extended its time domain, the enzyme label moved its spatial domain from the cell body layer to the dendritic layer and also began focusing on fewer cells.

Why is it that we can see such dramatic changes with association formation? We might speculate that even input signals restricted to small compartments of th...