The Symbolic Foundations of Conditioned Behavior

Charles R. Gallistel, John Gibbon

• Publisher: Psychology Press
• Year: 2002
• Print ISBN:
  9780415654555
• eBook ISBN:
  9781135679293
• Topic: Psychology
• Subtopic: Experimental Psychology
• Index: Psychology

The goal of this book is to persuade students of animal learning that cognitive theorizing is essential for an understanding of the phenomena revealed by conditioning experiments. The authors also hope to persuade the cognitive psychology community that conditioning phenomena offer such a strong empirical foundation for a rigorous brand of cognitive psychology that the study of animal learning should reclaim a more central place in the field of psychology.

Chapter

1

Response Timing

The strengthening of an associative bond through repetitive experience is the basic idea in the associative conceptual framework. That idea is seemingly most directly evidenced in acquisition, where the conditioned response (CR) appears after some number of conditioning trials, as if something had been strengthened over successive trials. Thus, associative accounts of conditioning generally begin with simple acquisition. We begin, however, by considering the timing of the CR—when it occurs in relation to the onset of a conditioned stimulus (CS). The basic idea in timing models is that the animal learns the temporal intervals, and this knowledge determines its behavior. The fact that it learns the intervals in a protocol is most directly evident in the timing of the CR.

Our model for the timing of the CR introduces many of the important concepts in the timing framework. One concept is that remembered intervals have scalar variability. This means that the trial-to-trial variability in an interval retrieved from memory is proportional to the magnitude of that interval. The bigger a magnitude read from memory, the more that magnitude varies from one reading to the next. Another broad concept is that conditioned behavior is the result of simple decisions based on the comparison of mental magnitudes, like, for example, the mental magnitudes (signals in the brain) that represent intervals. The animal responds when a decision variable exceeds a threshold. The decision variable is itself created by means of a simple arithmetic comparison, usually between a currently elapsing interval and a remembered interval. A third concept is that decision variables are ratios, not differences. For example, the decision whether to respond at a certain latency is based on the ratio of the currently elapsed interval to a remembered interval. When that ratio exceeds a threshold the animal responds. Unifying these three concepts is the concept of the time-scale invariance of the conditioning process, which is seen to be a consequence of these principles. Empirically, time-scale invariance means that conditioning data are unaffected by the time scale of the experiment. For example, the form of the distribution of CRs about the reinforcement latency does not depend on that latency; the distributions observed with different reinforcement latencies differ only by a scaling factor.

The most common elementary Pavlovian conditioning protocol is diagrammed in Fig. 1.1. A CS (typically, a tone or light) is presented for a fixed interval T, at the end of which the unconditioned stimulus (US; also known as reinforcement) occurs. The presentation of the CS is called a trial. Because the US is delivered coincident with the termination of CS presentation, the trial duration and the reinforcement latency (delay between CS onset and reinforcement) are one and the same. The interval after a trial during which nothing happens is the intertrial interval. Test trials without reinforcement are given after a certain number of training trials to probe for the strength of the CR to the CS.

Fig. 1.1. Time line for simple classical conditioning. The duration of the CS is T, the reinforcement (dot) coincides with the offset of the CS. For reasons to be explained later, the duration of this reinforcement (the duration of the US) may be ignored. The other important interval is I, the interval between trials (CS presentations). The sum of the I and T is C, the cycle duration.

If one records the distribution of conditioned responses on test trials, one generally finds that they are maximally likely toward the end of a trial, that is, the mode (peak) of the distribution is close to the reinforcement latency (see Fig. 1.4). This means that the subject generally does not react to the CS when it is first presented (see Figs. 1.2 and 1.3). It reacts only as the time of expected reinforcement approaches. The longer the duration of a trial, that is, the longer the reinforcement latency, the longer the CR is delayed.

Fig. 1.2. A representative example of the conditioned eyeblink response on a single test trial in an experiment in which rabbits were conditioned to blink in response to a tone that signaled a puff of air to the sclera of the eye. The latency of reinforcement on training trials (time from CS onset to delivery of the air puff) was always either 400 ms or 900 ms, but this latency varied randomly (unpredictably) from trial to trial. The rabbit learned to blink twice on each trial, once at a latency of approximately 400 ms and once at a latency of approximately 900 ms. (Reproduced from Fig. 3, p. 289 of Kehoe, Graham-Clarke, & Schreurs, 1989. Copyright © 1989 by the American Psychological Association. Reprinted with permission.)

Fig. 1.3. Data from a one-trial contextual fear conditioning experiment. Rats were given a brief foot shock 3 min after being placed in an experimental chamber (the context). The next day, they were again placed in the chamber, and their freezing behavior (a manifestation of fear) was scored during an 8-min test. The percentage of rats observed to be freezing was maximal at the latency at which they had been shocked the previous day. (From Fanselow & Stote, 1995, reproduced by permission of the authors.)

Fig. 1.4. Scalar property: Scale invariance in the distribution of CRs. Panels on left: Responding of three birds on the unreinforced trials of the peak procedure in blocked sessions at reinforcement latencies of 30 s and 50 s (unreinforced CS durations of 90 s and 150 s, respectively). Vertical bars at the reinforcement latencies have heights equal to the peaks of the corresponding distributions. Panels on right: Same functions normalized with respect to CS time and peak rate (so that vertical bars would superimpose). Note that although the distributions differ between birds, both in their shape and in whether they peak before or after the reinforcement latency (K^* error), they superimpose when normalized (rescaled). (From Gallistel & Gibbon, 2000, by permission of the publisher.)

Pavlov termed this phenomenon the inhibition of delay. In his conception, delay of the US inhibited the initial occurrence of the CR. As the time to reinforcement grew shorter during a trial, this inhibition was released. Although the phenomenon has been known to experimentalists since the days of Pavlov, it is seldom emphasized in standard accounts of conditioned behavior, because the associative conceptual framework does not offer a ready explanation for it. In the timing framework, in contrast, it is the first thing to be noted about conditioned behavior, because it is direct evidence that the animals are in fact learning at least one of the temporal intervals in the protocol, namely, the reinforcement latency.

Another property of the distribution of CRs, whose empirical generality and theoretical importance was first emphasized by Gibbon (1977) is that its standard deviation is proportional to its mode: The longer the CR is on average delayed, the more variable is its latency. Thus, the coefficient of variation, which is the ratio of the standard deviation to the mean, is constant. The fact that the mean, mode, and standard deviation of CRs all increase in proportion to the reinforcement latency means that the temporal distribution of CRs is time-scale invariant.

Time-scale invariance is a profoundly important property of the conditioning process, whose many manifestations we repeatedly call attention to. What it means, speaking somewhat loosely, is that the data from a conditioning experiment look the same regardless of the time scale of the protocol, provided that the scale factors in the graphs that display the data are adjusted to match the time scale. Thus, if one repeats a conditioning experiment using intervals twice as long as the intervals originally used, the data obtained will look just like the data originally obtained, provided one doubles the temporal units employed in analyzing and displaying the data (doubling, for example, the numbers at each tic on the temporal axis of the graph). Equivalently, the data from experiments conducted at different time scales (with different reinforcement latencies) look the same if they are plotted as a function of the fraction of the reinforcement latency elapsed. Adjusting the scale in this way is called normalization. To say that conditioning is time-scale invariant is to say that normalizing the data renders the results from experiments done on different time scales superimposable, as demonstrated in Fig. 1.4.

The Peak Procedure

The data in Fig. 1.4 come from a conditioning procedure called the peak procedure. It is a standard conditioning protocol, modified in such a way as to reveal both the onset and the offset of conditioned responding. In this procedure, a trial begins, as usual, with the onset of the CS. When pigeons are the subjects, the CS is the illumination of a key on the wall of the pigeon’s chamber. The illumination of the key is followed on some trials by reinforcement, that is, the opening of the food hopper for a few seconds. As in Fig. 1.1, when reinforcement occurs, it always occurs at a fixed latency after CS onset, provided only that the bird is pecking the key when that latency elapses. On reinforced trials, the CS terminates at the moment of reinforcement, because when the hopper opens, the bird stops its pecking in order to eat the food. However, on 50% to 75% of the trials, there is no reinforcement; the hopper fails to open. On these unreinforced trials, the CS persists for three to four times beyond the point at which the hopper should have opened. These unreinforced trials constitute the probe or test trials, the trials from which the data on the distribution of the CR are gathered. On a typical test trial, the pigeon begins to peck the illuminated key before the expected reinforcement latency and stops pecking some while after the point at which the hopper should have opened but did not.

The time at which the bird begins to peck and the time at which it stops pecking vary from test trial to test trial. The rate of pecking once it begins is fairly constant (Church, Meek, & Gibbon, 1994). However, because of the trial-to-trial variability in the onset and offset of this steady pecking, the average rate of pecking at a given point in a test trial is a smoothly increasing and subsiding function of trial duration, as may be seen in Fig. 1.4. It is important to understand, that the smooth rise and fall of the function in Fig. 1.4 is an artifact of averaging across many trials. The behavior on an individual trial does not look like that. Rather, at some latency, the subject begins abruptly to peck and it stops just as abruptly later on in that trial.

Figure 1.4 shows the data from three different pigeons, when this experiment was repeated at two different time scales. In one case, the reinforcement latency was 30 s, and in the other, it was 50 s. The unnormalized data are shown in the panels on the left. The numbers on the x axis specify the amount of time elapsed since CS onset. Notice that the curve for the 50 s data is more spread out than the curve for the 30 s data. The normalized data are shown in the panels on the right. Here, the numbers on the x axis are the time elapsed since CS onset divided by the reinforcement latency (normalized time). And the numbers on the y axis are the average response rate divided by the peak response rate (normalized rate of responding). Dividing by the reinforcement latency adjusts the units on the x axis (rescales the time axis), so that a unit (interval between ticks) represents the same fraction of the reinforcement latency in both cases. Similarly, dividing by the peak response rate scales the y axis in such a way that each unit represents the same fraction of the peak rate. These rescaling operations render the results from the experiment with a 30 s reinforcement latency and the results from the 50 s reinforcement latency superimposable. This means that the curves in the panels on the left really have the same shape, they only appear to have different shapes because of the difference in the scale factors used to plot them.

Although there are a large number of published curves demonstrating the scale invariance of CR distributions shown in Fig. 1.4 (see, for further examples, Figs. 1.6 and 1.8), we use data in Fig. 1.4 for two reasons. First, the curves from the different subjects differ noticeably in their shapes, but the data from each bird are nonetheless scale invariant. Second, if you look closely at the data, you will notice that the curves for the top and bottom birds (4660 and 467...

Table of contents

• Cover
• Title Page
• Copyright Page
• Table of Contents
• Preface
• Series Preface
• Introduction
• 1. Response Timing
• 2. Acquisition
• 3. Cue Competition and Inhibitory Conditioning
• 4. Extinction
• 5. Backward, Secondary, and Trace Conditioning
• 6. Operant Choice
• 7. The Challenge for Associative Theory
• References
• Author Index
• Subject Index