Operant-Pavlovian Interactions

Hank Davis, Harry M. B. Hurwitz

• Publisher: Routledge
• Year: 2021
• eBook ISBN: 9781000363876
• Topic: Psychology
• Subtopic: History & Theory in Psychology
• Index: Psychology

The first important distinction between operant and Pavlovian conditioning was made in 1928 by Polish scientists Konorski and Miller. Unaware of their work, Skinner proposed a similar analysis in 1935 of the manner in which operant and Pavlovian conditioning might differ and interact. Konorski and Miller responded to Skinner's statement, and by 1937 the now-classic debate over "two types of conditioned reflexes" was in high gear.

In the years before publication, the attention of many learning theorists had returned to the fundamental question of whether there are identifiably different forms of learning. The present volume, originally published in 1977, contains chapters that reassess our basic learning paradigms of the time. They deal with the definitional problems of isolating operant and Pavlovian conditioning, as well as the attempt to analyze the inevitable interactions that follow. These issues are examined in a variety of settings: some authors deal with operant-Pavlovian interactions directly by devising procedures to generate them; others examine operant-Pavlovian interactions by examining their possible contribution to established conditioning paradigms.

1

On the Role of the Reinforcer in Associative Learning

R. G. Weisman

Queen’s University, Canada

DOI: 10.4324/9781003150404–1

INTRODUCTION

Weisman’s chapter addresses itself to what he terms a “paradigm shift,” or a change in the form that behavior analysis has recently taken. His view of learning attempts to account for animal behavior not only in terms of reinforcement but also in terms of innate responses and cognitive processes. The following questions are dealt with in Weisman’s chapter: (1) What is the evolutionary history of associative learning? (2) How are the events that generate reinforcement arranged? (3) What are the relative roles of practice effect and instinct in Pavlovian and operant conditioning? (4) What role do cognitive factors play in generating conditioned responses?

This chapter, as much a distillation of the literature as of my own beliefs, is an attempt to trace new outlines for the functions of the reinforcer in engendering behavior change in associative learning. The work has its historical impetus in the traditional distinction between learning and performance but its most important impetus is the modern view that behavior change resulting from associative learning involves, at least, a primitive but powerful knowledge acquisition system and some equally important deterministic performance rules that govern the conversion of stored knowledge into biologically relevant behaviors. In the present work the nature of the behavioral units, the roles of exercise and effect, and even the terminology of association are reassessed. Specifically, I have tried to show how innate, habituative, and associative processes interlock, one with the next, to generate the effect we attribute to reinforcers.

The Associative Process

Associative learning seems in large part a result of the ability of animals to extract correlations in time and space between events from nature. That is, animals are not merely affected by correlations between events, they learn about them. Animals can learn that the occurrences of two stimuli are positively or negatively correlated (Pavlov, 1927; Rescorla, 1968) or even that the occurrences of two stimuli are not correlated at all (Mackintosh, 1973). This ability may bg usefully conceptualized as cognitive (Neisser, 1967), not simply associative, and certainly not as itself a product of its consequences for individual animals. That is, animals more or less automatically produce some central representation of events that preserves their order in time whether one event is motivationally significant or both events are rather neutral.

Sensory preconditioning. For evidence that animals learn to associate essentially neutral stimuli one need only consult the often slighted literature of sensory preconditioning. In the sensory preconditioning experiment, the preconditioning phase correlates two motivationally trivial stimuli, say S₁ and S₂, usually in the literature a light and a tone; then the conditioning phase correlates S₂ with a reinforcer. Finally during a test phase Si is shown to control behavior conditioned to S₂. Of course, results obtained during the test phase should be compared against various controls to avoid mislabeling stimulus generalization, unlearned reactions, etc., as sensory preconditioning.

In a review of sensory preconditioning research Seidel (1959) concluded that the procedure generated small but reliable effects. Research conducted over the intervening 16 years (e.g., Prewitt, 1967; Rizley & Rescorla, 1972) has actively confirmed this conclusion (Mackintosh, 1974). Some further conclusions seem justified.

1. Overt behavioral mediation of association between S₁ and S₂ can be rejected as an explanation of the phenomena. Stimuli emanating from the same sound source have been associated (Kendall & Thompson, 1960) and a tone and light were associated even when rats were paralyzed by curare (Cousins, Zamble, Tait, & Suboski, 1971).

2. Motivationally powerful reinforcers are not necessary to associative learning as it occurs in the first phase of the sensory preconditioning experiment. The main consequence of preconditioning trials seems to be the construction of central representations of the stimuli.

Some sort of general associative process is integral to the present chapter and, indeed, to current thought concerned with information processing in animal learning. Nonetheless, as a verified instance of the general case sensory preconditioning remains deficient, because only evidence concerning learning about positive correlations has been reported. If sensory preconditioning is an example of the general case of which Pavlovian conditioning is an instance then animals must learn about negative and zero correlations between relatively neutral events.

It has not escaped notice that although sensory preconditioning experiments regularly produce highly reliable results, the magnitude of the difference between experimental and control groups is often quite small. Moreover, there is reason to believe that refinements in procedure, although otherwise useful, do not increase the magnitude of effect appreciably. The effectiveness of increasing the number of sensory preconditioning trials appears to reach a real limit, perhaps, imposed by the intervention of another process. Of course, one can press on, presenting many, many preconditioning trials. Indeed, Hoffeld, Kendall, Thompson, and Brogden (1960) administered up to 800 preconditioning trials to one of several groups of cats. However, across response measures four pairings of S₁ and S₂ resulted in several times as much avoidance responding to S₁ during the test phase as 800 pairings. Similar but less extreme results were reported by Prewitt (1967), who found S₁ a somewhat more effective conditioned suppressor of licking in rats after 16 preconditioning trials than after 64 trials. Thus, increasing the number of trials during preconditioning does not appear to result in ever more association between S₁ and S₂, although control of the conditioned response by S₁ remains much weaker than control by S₂.

The Habituative Processes

Implicated in the constraint of sensory preconditioning are what may be termed the habituative processes, habituation proper and latent extinction, which are at least as general as sensory preconditioning and latent learning but with opposite effects. Habituation, of course, is the waning of behavioral, and electrophysiotogical, responses with repeated stimulation. It occurs quite generally across behavioral categories affecting species-characteristic motor patterns to predators (Hinde, 1954) and conspecifics (Clayton & Hinde, 1968) as well as orienting reactions to tones and lights (Sokolov, 1960). There is good evidence that the neural correlates of tones and lights diminish with continued presentation, as in the preconditioning phase of the sensory preconditioning experiment (Sharpless & Jasper, 1956).

The neural correlates of repeatedly presented stimuli not only diminish but also enter into association with other stimuli less rapidly: that is, they show the effects of latent inhibition (Lubow & Moore, 1959). That nonreinforced preexposure to the conditioned stimulus retards later conditioning is well established (Lubow & Siebert, 1969; Rescorla, 1971: Siegel, 1972). It is therefore difficult to believe that repeated nonreinforced presentation of S₂ during preconditioning does not retard later learning to associate S₂ with a reinforcer. Association between S₁ and S₂ may be eventually retarded because neither is a reinforcer. Interaction between sensory preconditioning and latent inhibition has not been investigated. Perhaps, as Mackintosh (1973) suggests, animals simply learn to ignore stimuli that fail to predict reinforcers. It would appear that vague theoretical objections to sensory preconditioning research have obscured the more interesting questions this work poses. In particular, interlocking between the habituative and associative processes in sensory preconditioning experiments deserves more direct investigation than it has received so far. Covariation between latent inhibition, habituation proper, and sensory preconditioning must be studied within a single experimental setting.

Differential habituation. Learning to associate correlated events may occur automatically and without reinforcement, but not without opposition. Opposing the general associative process are habituative processes by which animals may learn to ignore events. For example, the waning of EEG arousal and the behavioral orienting response to a tone in cats is nearly complete after a total of only 1 or 2 min of exposure (Sharpless & Jasper, 1956), whereas comparable decrements in responses to a live owl by chaffinches (Hinde, 1954) or to their litters by female mice (Noirot, 1964) may require hours or even days of exposure. There seems to be general agreement that the habituative processes operate at different rates depending on the motivational relevance of the event (Lorenz, 1965, p. 56; Hinde, 1970, p. 296; Denny & Ratner, 1970, p. 534). One need hardly point out that the reinforcers most favored in Pavlovian conditioning experiments are powerful motivational events, if not themselves releasers of species-characteristic fixed action patterns.¹ This highly adaptive feature of reinforcers has evolutionary consequences for associative learning. For it is likely that associations between neutral events and reinforcers endure with repetition, whereas associations between merely neutral events do not, in large part because of the differentially greater effect of the habituative processes on the latter events.

1 The question of whether habituation and satiation, as in the repeated presentation of food or water, are allied processes is not within the scope of this chapter.

Some evolutionary considerations. Even a brief survey of the diversity found in evolutionary history prompts one to speculate that natural selection “experimented” with special associative processes prior to or over the course of the phylogenetic development of the general associative process. However evidence for the notion that special associative processes, one for taste, another for vision, etc., are the “general” case is noticeably less than overwhelming. Flavor aversion may be learned in one trial but so may conditioned suppression (Wearden, 1975). Flavor aversion may follow conditioning with long CS-US intervals but so may instrumental learning in a T maze (Lett, 1975). Moreover basic phenomena common to Pavlovian conditioning with, say, food or shock as reinforcers, for example, overshadowing, blocking, latent inhibition, discrimination, and extinction, are observed in flavor aversion experiments as well. Finally, quantitative differences in the speed at which animals learn to associate various CSs with various reinforcers should not be taken as strong evidence for qualitative differences in associative process. Actually, Waddington’s (1953) hypothesis for the genetic assimilation of phenotypic variation seems to predict selection pressures, rather than special associative processes, to account for important between stimulus-reinforcer, and response-reinforcer, variance in learning rates. For example, in nature rats learning flavor aversions in one trial are rather more likely to survive to bear progeny than rats requiring up to 10 trials to learn the same classically conditioned aversion. Also the likelihood of actual exposure to a toxic substance may be much less from merely being in the same place as the toxic substance. Accordingly, the association of places with toxic substances proceeds somewhat slowly. This suggests that we can apply artificial selection to produce very rapid place aversion with toxic reinforcers.

It is possible to muster other quite plausible evolutionary advantages of the interlocking of habituative and associative processes. Selection may favor the genotype of animals habituating more slowly to, say, the sight of food or a mate than to the repeated presentation of some biologically trivial event. Also, the general associative process seems the simplest adaptation sufficient to permit neutral events to acquire biological significance. Of course, without selective habituative processes a quite general associative process may well be an evolutionary disaster, causing animals to fill their heads with the neural correlates of literally every correlation between events they have encountered in nature. However, the interlocking of selective habituative processes with a general associative process has adaptive features not replicated by the development of more selective associative processes. For example, as new correlations between events arise animals may automatically learn about them; if after several repetitions these events possess or acquire no biological significance then their central correlates are quite simply and automatically diminished along with the orienting reactions they originally have elicited.

A general associative process, more fully than an array of special associative processes, provides a basis for the superior functioning attributed to humans and the other higher primates. The resulting picture of animal learning not only mirrors the one the behaviorist presents for us but also provides a cognitive basis of learning that no amount of S-R obscurantism can completely conceal. However, I shall not berate the S-R behaviorists too vigorously for their suspicions concerning a general associative process, which has appeared to leave animals quite busy but not very active. Even an outright mentalist, if one should be found, must admit that natural selection can only act on what animals do and not...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• New Preface to the Reissue of 2021
• Original Title Page
• Original Copyright Page
• Dedication Page
• Contents
• Preface
• Introduction
• 1 On the Role of the Reinforcer in Associative Learning
• 2 A Note on the Operant Conditioning of Autonomic Responses
• 3 Sensitivity of Different Response Systems to Stimulus-Reinforcer and Response-Reinforcer Relations
• 4 Performance on Learning to Associate a Stimulus with Positive Reinforcement
• 5 Behavioral Competition in Conditioning Situations: Notes Toward a Theory of Generalization and Inhibition
• 6 Pavlovian Second-Order Conditioning: Some Implications for Instrumental Behavior
• 7 The Safety Signal Hypothesis
• 8 Aversively Controlled Behavior and the Analysis of Conditioned Suppression
• 9 Response Characteristics and Control During Lever-Press Escape
• 10 Conditioning Food-Illness Aversions in Wild Animals: Caveant Canonici*
• Author Index
• Subject Index