Handbook of Learning and Cognitive Processes (Volume 2)

Conditioning and Behavior Theory

William Estes

• Publisher: Psychology Press
• Year: 2014
• Print ISBN: 9781848723900
• eBook ISBN: 9781317672371
• Topic: Psychology
• Subtopic: Cognitive Psychology & Cognition
• Index: Psychology

Originally published in 1975, Volume 2 of this Handbook looks at areas traditionally associated with learning theory such as conditioning, discrimination and behavior theory. It deals with concepts and theories growing principally out of laboratory studies of conditioning and learning. The intention was to treat mechanisms, processes, and principles of some generality – applicable at least to all vertebrates. It was becoming well understood that detailed interpretations of particular behaviors required the authors to take account of the way general principles operate in the context of species-specific behavioral organizations and developmental histories; but detailed consideration of just how these interpretations were accomplished for different animal forms was another enterprise. Here the authors limit their task to abstracting from the enormous literature facts and ideas which seemed general enough to be of interest and perhaps utility to investigators in other disciplines at the time.

Volume 1 presented an overview of the field and introduced the principal theoretical and methodological issues that persistently recurred in the expanded treatments of specific research areas that comprise the later volumes. Volume 3 looks at human learning and motivation, while the last 3 volumes range over the many active lines of research identified with human cognitive processes at the time.

1

Pavlovian Excitatory and Inhibitory Conditioning

Robert A. Rescorla

Yale University

I. INTRODUCTION

All organisms face a world composed of multiple events bearing various relations to each other. Most of those events are outside of the control of the organism, and they vary widely in biological significance. But any adaptive organism must show a sensitivity to at least some of these events and the relationships among them. It is clearly advantageous to change as a result of exposure to different relations among events in the environment, to learn about what Tolman and Brunswik (1935) called the “casual texture” of the world. The problem of interest for the psychologist is that of conceptualizing how the organism learns these relations and what relations he learns.

One way to view Pavlovian conditioning is an example of this kind of learning about environmental events over which the organism has little or no control. Thus, the historically prominent salivary experiments of Pavlov (1927) can be viewed as simply the most familiar examples of the laboratory study of interevent learning. Those experiments employed particular kinds of events and a rather narrow range of relations among them, but they nevertheless represent a beginning to the general problem. It is to be hoped that the principles derived from such experiments may be gen-eralizable well beyond the particular events and relations studied.

This is a somewhat liberal way of describing Pavlovian conditioning but it has the advantage of placing such studies in perspective within the general field of learning. It shifts attention away from many of the situationally specific issues with which conditioning has often concerned itself to the more general problem of studying the learning of relations. Consequently, this chapter ignores many of the issues historically associated with conditioning and concentrates instead on learning relations.

Two major conceptual tools have emerged in an attempt to understand how the organism organizes his learning about events in his environment, conditioned excitation and conditioned inhibition. Speaking casually, conditioned excitation has to do with the learning that two events are not independent but instead tend to cooccur in the environment; conversely, conditioned inhibition has to do with the learning that events tend to occur apart from each other. Most approaches to conditioning attempt to use just these two kinds of learning constructs to describe all of the information about relations among events which an organism retains. Consequently, attention here centers upon these two notions.

Historically, the notion of conditioned excitation has been considered primary. Indeed, for many psychologists excitatory conditioning is still synonomous with the general term “conditioning.” Normally, psychologists infer the presence of conditioned excitation whenever proximal presentation of two events results in a change in the behavior of the organism. In the classic case, presentation of a neutral conditioned stimulus (CS) contiguously with a food unconditioned stimulus (US) enables that CS to subsequently evoke a salivary response. It is the observation that behavior during the CS changes as a result of its temporal relation to the US that forms the basis for the inference of conditioned excitation. This description turns out to be unduly restrictive with regard to both the interevent relation and to the behavior from which excitation is inferred, but it will serve for the present.

Equally important to Pavlov, but slighted by many students of conditioning is learned inhibition. Historically, the conceptualization and specification of conditioned inhibition has been derivative of that of excitation. This subordinate role fits with an aspect of the intuitive meaning of inhibition—it acts not to generate its own effects but rather to modulate the effects of other processes. Thus, the common, although often inexplicit, procedure has been for theorists to begin by defining excitation in terms of operations and outcomes and only later to search out stimuli that might attenuate the action of excitation. Those stimuli are identified as inhibitors, and one may then raise the empirical question of what particular past experiences are necessary to endow stimuli with such inhibitory power.

The present usage of the terms “excitation” and “inhibition” will become clearer in subsequent sections, but two points should be made explicit here. First, those terms are used in many different ways by psychologists. This chapter by no means attempts to discuss all of those usages or even all of those usages within the framework of a conditioning experiment. Rather, since concern is with modification by experience, the discussion is confined to those examples of excitation and inhibition that involve learning, that is, are conditioned. Second, the current use of these terms is theoretical in character; inhibition and excitation are conceptual terms inferred from operations and outcomes, within the context of some more or less explicit theoretical structure. They are not, for instance, uniquely identified with a particular behavioral outcome, such as increase or decrease in the probability of some response. Indeed, little attention is paid here to the nature of the response changes from which excitation and inhibition are inferred. Instead this chapter emphasizes the nature of the relations learned and remains liberal in accepting a wide range of changes in behavior as indicating modification by exposure to a relation.

II. EXCITATORY CONDITIONING

Excitatory conditioning has been casually identified as the learning that two events are positively related in the environment. Further empirical and theoretical specification necessarily centers on two intertwined questions. First, what specific environmental relations are necessary and sufficient for generating such excitatory conditioning? Second, what is the nature of the learning so generated? How do we conceptualize the way in which experience with relations modifies the organism?

A. What Relation Produces Learning?

In addressing the first question, virtually all students of conditioning have agreed that contiguity in time is the primary condition for the establishment of excitation. Organisms learn that a CS and US go together when they occur contiguously in time. Such diverse theorists as Pavlov (1927), Tolman (1932), Hull (1943), Guthrie (1959), Spence (1956), and Konorski (1967) have been unanimous on this point. Furthermore, there is now a vast store of empirical information supporting the proposition that, as the contiguity between a CS and US is destroyed, the CS becomes a less adequate conditioned excitor. Studies of the so-called CS–US interval have been a popular activity of American psychologists and virtually all response systems studied thus far have supported this gross conclusion (see, for example, Black & Prokasy, 1972).

An important, and historically often troubling, limitation on that conclusion should be mentioned. Apparently, strict simultaneity in time does not maximize excitatory conditioning; rather, the CS should slightly precede the US. Although there are several results that suggest that conditioning can be obtained with CSs that follow USs, there can be little question that the sequence of events, as well as their proximity in time, dramatically affects conditioning.

It turns out, however, that in evaluating positive relations among events the organism is considerably more sophisticated than this initial, historically popular, description suggests. Although contiguity among events is important, in many conditioning situations the animal does much more than simply tabulate coincident occurrences of CS and US. Speaking casually, the organism seems to demand not only that the CS and US be contiguous but also that a CS provide “information” about the occurrence of the US in order to condition excitation to that CS.

Two examples will illustrate this point. Both examples come from fear-conditioning situations in which aversive foot shock is the US paired with neutral CSs in rat subjects. With such pairing, CSs typically come to elicit suppression of the ongoing behavior of the organism, indicating the development of excitatory fear conditioning. In the first example, Rescorla (1968) explored the effect of intermingling “intertrial” shocks among tone–shock pairings. He found that if such USs were presented with sufficient frequency in the absence of a CS, they severely disrupted the ability of CS–US contiguities to condition excitation to that CS. Notice that such “extra” USs left intact the CS–US contiguity but occurred with sufficient frequency to make the CS useless as a predictor of the US. Further parametric investigation suggested that the organism is sensitive to the relative rates of occurrence of the US in the presence and absence of the CS. The amount of excitatory fear conditioning to the CS was well predicted by the degree to which shock was more probable during the CS than in its absence; conditioning was poorly predicted by the simple number of USs during the CS. Such findings led to the suggestion that CS–US contingencies (or correlations), not simply contiguities, govern excitatory conditioning (Rescorla, 1967). The same contiguities apparently have quite different effects depending upon the context in which they occur.

In a related experiment, Kamin (1968, 1969) first paired a noise CS with a shock US in rat subjects. He then added a (redundant) light to the noise and continued the pairing with shock. Under those circumstances, the light acquired little conditioning despite its repeated contiguity with shock. Apparently, because the light provided no new information, conditioning did not occur. Again, this situation has been explored in great parametric detail and yields a data pattern generally consistent with the informational notion. Other examples of such findings are reviewed elsewhere (Rescorla, 1972a). Together, they suggest that animals demand a higher quality of evidence than simple coincidence to conclude that events are positively related and thus show excitatory conditioning.

There are now available several descriptions of excitatory conditioning which try to capture this sophistication without abandoning the primacy of temporal contiguity. These descriptions have been given in different languages and in different levels of specificity, but all share one assumption: the effectiveness of a US depends not simply upon its own physical properties but also upon the current status of excitatory conditioning. Put informally, they assume that conditioning is dependent upon the discrepancy between the US received and that anticipated; unanticipated, surprising USs are especially effective, whereas anticipated ones (ones derived when a substantial CR occurs) are reduced in effectiveness.

One specific version of such a theory, suggested by Rescorla and Wagner (1972), is described below. Conditioned excitation is specified in terms of the changes in associative strengths of stimuli, as a result of a contiguity with the US. Thus, if a single stimulus A is followed by a US, the change in its strength is specified by the equation

ΔV_A = α_Aβ(λ − V_A).

In this expression α and β are rate parameters acknowledging the individual features of the CS and US; λ is the asymptote of conditioning supportable by the US employed. The important feature is that increases in excitation (ΔV) depend upon the discrepancy between the asymptote (λ) and the current level of conditioning of the CS (V_A). For such a simple case, this formulation is simply an adaptation of the notions of Bush and Mosteller (1955) and of Hull (1943) and consequently predicts a variety of elementary findings, such as negatively accelerated acquisition curves.

The distinction between this formulation and those earlier ones becomes most apparent when a compound CS, AX, is followed by a US. In that case, the separate associative strengths of the elements A and X are changed according to the following equations:

ΔV_A = α_Aβ(λ − V_AX).

ΔV_X = α_Xβ(λ − V_AX).

The feature of interest is that the current strength of the total AX compound is relevant to the increment in strength which each stimulus receives. Since V_AX is assumed to equal the sum of V_A and V_X, this means that the current strengths of all stimuli present at a US occurrence modulate its effectiveness for conditioning each individual stimulus. It is this feature that enables such formulations to account for the “informational” character of conditioning.

The two examples mentioned above provide convenient examples of the power of such theories. Consider first the Kamin blocking experiment. In that experiment, A is first presented singly and paired with shock until it is asymptotic at V_A → λ. Then the redundant X is added and the AX compound reinforced. Notice that the quantity (λ – V_AX) determines the conditioning to X on such compound trials. Since V_AX is assumed to equal V_A + V_X, which in turn is close to λ, that quantity is close to zero, and so V_X receives little increment. Speaking casually, because the US was expected on the basis of A (i.e., V_A is high), it is ineffective in conditioning X. Thus, conditioning to X is blocked despite its presentation in contiguity with a physically potent US. A similar description can be given of Rescorla’s data. In that case, the role of A is played by situational cues and X by the experimenter’s stimulus. If the background stimulus is conceptualized as divisible into temporal units in the presence of which shocks may or may not occur, then the organism is exposed to two kinds of trials: A alone and AX (background plus CS). The distribution of shocks to A alone establishes some level of background conditioning; that level in turn blocks conditioning to X on the AX trials. Because the organism generally anticipates shock in the situation, the CS provides no special information, and so it receives no excitatory conditioning.

There are already available in the literature detailed discussions applying theories of this sort to a broad range of conditioning data (e.g., Wagner & Rescorla, 1972; Rescorla, 1972a). Such theories turn out to do a remarkably good job of integrating a wide variety of data which otherwise seem to demand sophisticated computational abilities on the part of the organism. They do so by providing an account not only of asymptotic performance but also of the trial by trial events which generate that performance...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Original Title Page
• Original Copyright Page
• Table of Contents
• Foreword
• Introduction to Volume 2
• 1. Pavlovian Excitatory and Inhibitory Conditioning
• 2. Learning as Adaptation
• 3. Adaptational Aspects of Conditioning
• 4. Classical Conditioning: Some Methodological–Conceptual Issues
• 5. The Classical–Instrumental Distinction: Reflexes, Voluntary Behavior, and Categories of Associative Learning
• 6. Perspectives on the Behavioral Unit: Choice Behavior in Animals
• 7 Stimulus Selection in Associative Learning
• 8. Stimulus Generalization
• Author Index
• Subject Index