The Use and Misuse of the Experimental Method in Social Psychology

A Critical Examination of Classical Research

Augustine Brannigan

• Publisher: Routledge
• Year: 2020
• eBook ISBN: 9781000209457
• Topic: Psychology
• Subtopic: Research & Methodology in Psychology
• Index: Psychology

This book critically examines the work of a number of pioneers of social psychology, including legendary figures such as Kurt Lewin, Leon Festinger, Muzafer Sherif, Solomon Asch, Stanley Milgram, and Philip Zimbardo. Augustine Brannigan argues that the reliance of these psychologists on experimentation has led to questions around validity and replication of their studies.

The author explores new research and archival work relating to these studies and outlines a new approach to experimentation that repudiates the use of deception in human experiments and provides clues to how social psychology can re-articulate its premises and future lines of research. Based on the author's 2004 work The Rise and Fall of Social Psychology, in which he critiques the experimental methods used, the book advocates for a return to qualitative methods to redeem the essential social dimensions of social psychology.

Covering famous studies such as the Stanford Prison Experiment, Milgram's studies of obedience, Sherif's Robbers Cave, and Rosenhan's exposé of psychiatric institutions, this is essential and fascinating reading for students of social psychology, and the social sciences. It's also of interest to academics and researchers interested in engaging with a critical approach to classical social psychology, with a view to changing the future of this important discipline.

1 The sunset on a golden age

Reflections on the gap between promise and practice

Human experimentation and causal explanations: promise versus performance

In his recollections of the history of social psychology, Albert Pepitone (1999:182) noted that, during the golden period, theory-driven research developed “the state-of-the-art-experiment in which the theoretical variables were created through artful manipulations of how subjects should interpret the situation.” Such stagings often entailed “fantastic scenarios and ‘cover stories’ that stretched credulity … From that initial postwar boom … into the late 1980s, marks a golden age … The golden age was quintessentially the age of experimentalism.” The experiments were designed to formally test hypotheses about behavior. But such “sharply focused experiments cannot themselves acquire comprehensive knowledge” (1999:182). In fact, it produced a crisis in confidence about just how far the experimental tradition was capable of contributing to the growth of knowledge. According to Zimbardo, that period drew to a close when persons with less appetite for extreme dramaturgy joined the profession and when institutions took meaningful steps to safeguard the treatment of human subjects. “That tradition is now dead and not mourned by those who hastened its demise, a cabal of some cognitive social psychologists, human subjects research committees, Protestants, and female social psychologists” (Zimbardo 1999).

In my view, there was a huge gulf between the appropriate use of experiments and what psychologists actually did in their laboratories, and this was the case not at the periphery of social psychology, but at its very core. In other words, there was a gap between the promise and the performance after experimentation dominated the arsenal of social psychologists in the 1950s and subsequent decades. In this chapter, I want to identify the scientific strength of experimentation and contrast this to how experiments actually developed in the golden age of experimental social psychology.

The promise

In the methodology of the social sciences, it is well accepted that experimentation is the key to objective knowledge, and is superior to rival methodologies, at least in principle.¹ The ideal design in scientific research is the true experiment, where subjects are randomly exposed to various treatment conditions and then tested to determine the effects of the different treatments on the outcomes. Since the designs are standardized, replication of results is typically quite straightforward. What made the experiment superior to other methods, such as cross-sectional surveys, ethnographies, or interviews, was, according to its proponents, its ability to combine certain features of inquiry: first, an association between two or more variables linking a potential cause and an effect; second, the ability to identify temporal precedence of the cause, that is, the appearance of the cause prior to the identification of the effect because of the time-ordering of the events in the experimental designs; and third, an ability to determine whether the connection between cause and effect is non-spurious. In addition, true experiments have three things associated with them: two comparison groups – minimally, an experimental group and a control group, variations in the independent variable before assessment of change in the dependent variable, and random assignment of subjects to the two (or more) groups.

In theory, this combination of factors is supposed to give some confidence in the validity of the causal connections between the “treatment” and the outcomes. And our confidence is further enhanced by two things: the identification of the causal mechanisms that underlie the observed changes and the experimenter’s control over the institutional context of the experiment. This said, it must also be acknowledged that not all experiments have a pure “exposure” and control or “no exposure” design. Sometimes a design will have several different kinds of exposures. Imagine looking at the effect of exposure to violent films, versus non-violent films, versus no films at all. The “no film” condition would be the true control group and the other types of exposures would be comparison groups across experimental treatments. In addition, true experiments do not actually require pre-tests on the variable or outcome of interest. If one was interested in the effects of certain types of films on attitudes (for example, propaganda and attitudes to certain minorities), it might be possible to get a pre-test measure of attitudes prior to the treatment. However, the logic of the design is that those in the control group are, in effect, a pre-test group because they have not received the treatment. Because they have been randomly assigned to the control condition, they are logically identical to the “before” group. The random assignment of subjects to various treatment groups avoids the potential artefact that arises from administering the same measures to the same subjects twice.

A related point has to do with randomization. Of course, it would be a mistake to believe that the people who end up in the treatment versus the control group are all exactly alike. They obviously are not. But what the logic of random assignment suggests is that the various salient things that might affect the outcomes have an equal probability of occurring in each group, so that their effect is neutral.

A different issue concerns random sampling versus random assignment. In a survey, we engage in a random sampling of a population to ensure that we can generalize from the persons in the sample to the larger population, since each S has the same probability of being selected for inclusion in the survey (Frankford-Nachmias 1999:481). By contrast, in experiments, randomization does not ensure generalizability, and neither is it designed to do so. It is designed to ensure internal validity. Internal validity covers a number of issues, but, for our purposes, suffice it to say that what it ensures is that the design gives us confidence that the only important difference between the control and the treatment group is the treatment itself. The issue of generalizability is perhaps the main Achilles’ heel of experimentation in social psychology. The logic of experimentation is that the sorts of things being investigated are of such generality that they are present in whatever sections of the population from which subjects are drawn. At least in theory, the lack of a careful selection process designed to ensure representativeness is irrelevant. Obviously, with these attributes, the experiment has earned a reputation as a powerful tool in the arsenal of social scientists. How did the experiment work out in practice? A rather different picture emerges.

The golden age of experimental social psychology: reflections from the Yosemite conference

In 1997, a group of senior American social psychologists gathered at Yosemite National Park to take stock of the growth of knowledge in experimental social psychology and to record some of their personal memories and professional reflections. They were the leading lights in psychology, who were active in creating the profession in the period of its heyday following the Second World War and in the decades thereafter.² All the participants took their doctoral training in the period 1948–1959, and were major contributors to the field during its impressive growth in the 1950s, 1960s, 1970s, and 1980s. Their post-mortem deliberations on the achievements of the field provide a rare window on the history of social psychology. They identify a number of ambiguities in the development of the field that appears to be associated with its commitment to the causal explanation of social behavior through the use of experimental methods.

What important conclusions emerged from this celebration of a century of research? There were recurrent observations that the golden age was over, that the field had not accumulated much reliable new knowledge, and that it had not achieved much consensus about important matters. The sociology of knowledge warns us to treat collective reflections about a field’s origins with a grain of salt, since often times these “memories” are myths about the turning points in history that eventuated in the current configuration of knowledge. Rodrigues and Levine (1999), who edited the proceedings, trace experimental social psychology to the work of Norman Triplett. In 1897, Triplett had published an experimental study of children’s task performance based on observations of bicycle racers. Triplett knew that the individual performance levels of racers are influenced by competition. Typically, racers “pace” each other before putting on the final sprint at the end of the race. Triplett measured children’s performance on a fishing reel undertaken alone and in competition. His work introduced the concept of “social facilitation” into individual productivity as well as the effects of the related concept of “rivalry.” What is ironic is that his research received rather mixed notice in the years that followed, and, aside from being experimental, created no research legacy.³ Indeed, in the early decades of the discipline, G. H. Mead criticized the use of experiments for social psychology, since he favored the method of introspection. However, social psychology gradually acquired scientific respectability, not because of its theoretical progress, but because of the adoption of the methods and logic of the hard sciences. The break between social psychology as a sociological discipline and social psychology as a psychological discipline emerged in the 1950s, when experimental methodology became the orthodox approach in professional psychology while the sister sciences remained relatively diversified in their approaches. Experimentation established itself as the gold standard because of its ability to link connections in a non-spurious, temporally informed fashion, and to explore relationships causally.

Several contributors at the Yosemite conference noted that social psychology had become a field in which practitioners appeared to know little of the history of their own discipline and had become alienated from cognate areas in sociology and anthropology. In this celebration of the discipline’s achievements, Aronson (1999:108) lamented the fact that contemporary social psychologists were ignorant of research prior to 1975, and Raven (1999:118) warned that new scholars were in danger of “reinventing the wheel,” or of failing to credit an idea to its originator because of disciplinary amnesia (Berkowitz 1999:161). There was a consensus that the field had become increasingly abstract, specialized, and divorced from issues of everyday life. There was also a sense that “the golden age” of experimentation had come to an end, a victim of the new institutional review boards instituted in the 1990s to protect human subjects from unethical conduct by experimenters.

The review boards were created in the 1970s to ensure protection of human subjects from harm and discomfort.⁴ While originally directed at medical research using human subjects, the boards may have sounded the death-knell for experimental studies of human psychology. In such studies, consent is often obtained from subjects through deception about the purpose of the research, a condition that renders the consent uninformed, and, hence, invalid. In addition, in the search for realism in the laboratory, some psychological experiments have entailed very detailed dramaturgical manipulations that have resulted in high levels of trauma among subjects. For example, in the disturbing study of obedience to authority, Milgram (1963) reported that many of his subjects experienced nervous fits and uncontrollable seizures. Zillmann and Bryant’s (1982) nine-week study of pornography reported that changes in callous attitudes among subjects were “non-transitory” (i.e., permanent). In their study of the dynamics of emotions, Schachter and Singer (1962) injected subjects with chlorpromazine (a medication used in the treatment of schizophrenia) or epinephrine (synthetic adrenaline) under the pretext of testing a new vitamin, “suproxin” (Schachter 1971). The passing of high-impact experimentation was noted by Zimbardo, mentioned earlier in this chapter. Zimbardo claimed that the ethics review boards “overreacted to the questionable ethics of some of the research by the oldies but goodies in experimental social psychology.” By imposing limits on what can now be done and said to research participants, the boards have provided safeguards “to the end of eliminating some of what could be called traditional experimental social psychology” (1999:138).

The dominant understanding about the defensible treatment of human subjects in experimental psychology has been that there has existed a trade-off between short-term deception and edgy manipulation of subjects on the one hand, and long-term benefits to science and society on the other. But there has never been a general meeting of the consumers of psychological knowledge to determine whether this investment was justified. In fact, Zimbardo’s own work raises some of the deepest questions. In the Stanford “prison study,” he reports that some of his mock guards assaulted the mock prisoners, and that many prisoners had to be released prematurely because of intense emotional trauma. In his 1972 account in Society, it appears he dragged his heels in terminating the “experiment” until it could be recorded on videotape by a local television station. But what was learned about prisons that we did not know? If the ethics boards overreacted in recent years, this may be due to an absence of effective internal self-regulation in the past. But that was not the only problem associated with experiments in the golden age. Evidence suggests that in many of the key studies, researchers would not take no for an answer, that the experiments simply became devices for demonstrating a relationship arrived at beforehand, and that the field could not grow because falsification of a hypothesis was virtually never recognized. In fact, “verification bias” has been identified as a leading cause of replication failures in contemporary experimental psychology.

Kurt Lewin and field theory

In my view, experimental social psychology began, not with Triplett in 1897, but with Kurt Lewin and his students in the mid- to late 1940s. Lewin was a German émigré whose “field theory” (1951) was based on the German gestalt tradition in which individual actions and attitudes were interpenetrated by socially based, cognitively coherent frames of reference. After the war, Lewin established the Research Center for Group Dynamics at the Massachusetts Institute of Technology. The Center moved to the University of Michigan after Lewin’s death in 1947, but not before Lewin had assembled an impressive group of graduate students and co-instructors. By all accounts, Lewin was an effective “tribal leader” (according to Deutsch 1999:9). He combined a commitment to the rigors of experimentation with an intellectual agenda that fostered practical engagement with everyday life, including the potential of social psychology for ameliorating social problems. Lewin’s own work demonstrated the greater effectiveness of democratic versus autocratic forms of leadership in laboratory studies. This in itself had a tremendous attraction among the graduate students who enrolled in social psychology after participating in the war against European and Japanese dictators. The field had a further caché since experimentation was the sole methodology in the social sciences expressly capable of suggesting not mere correlations, but causal connections. At the Yosemite meeting, Harold Kelley recorded the attitude at the time: “We were ‘real scientists,’ using the experimental method, drawing firm conclusions about cause and effect, and not fooling around with mushy correlational data” (1999:41). According to Gerard (1999:49) Cohen and Nagel’s Introduction to Logic and Scientific Method (1934) was standard reading for these scientific protégés.

The late 1940s and early 1950s were very consequential for the subsequent directions of the discipline. Lewin’s gestalt orientation appeared to lead psychologists away from a focus on stimulus–response behaviorism in their theoretical modelling, since it expressly celebrated the distinctive role of perception, recollection, and normative action in human social behavior. However, the commitment to the method of experimentation was subsequently to result in a narrow focus on cognitive mechanisms that were suitable for laboratory investigation, however remote they might be from pressing problems in everyday life. This narrow methodological focus was subsequently to stifle the search for general, integrative theories. By contrast, at the start of their careers, Deutsch, Gerard, Berkowitz, and Pepitone (among others) attested to the remarkable breadth of social psychology, and to the common definitions of problems in “sociological” and “psychological” social psychology. There was a tremendous interest in racism and discrimination, the related problems of school desegregation, and concerns over world peace. As the field developed, experimentalists became increasingly preoccupied with, in the words of Berkowitz (1999:162), “within-the-skin” versus “between-skins” phenomena. The “social” in social psychology became more associated with the idea of “information” and “information processing” than with meaning, culture, and context. And the focus on “cognitions” qua cognitions created “a spurious conceptual generality” (Pepitone 1999:191).

One of the casualties in the institutionalization of social psychology was psychoanalysis. Morton Deutsch, one of the earliest proponents of experimental studies of group behavior, was trained in psychoanalysis and enjoyed a long clinical career outside academic psychology. “The practice was personally rewarding. I helped a number of people, it enabled me to stay in touch with my own inner life, and it provided a welcome supplement to my academic salary” (Deutsch 1999:15). Similarly, Harold Gerard turned to “depth psychology” later in his career, entering psychoanalysis at age fifty-nine and becoming a psychoanalyst at age sixty-nine – thereafter switching his experimental work to a focus on “subliminal activation.” But these clearly were the exceptions.

As the field became more laboratory-oriented, such “soft methods” and general theories fell into disfavor. The single leading proponent of experimental social psychology – Leon Festinger – viewed such “applications” with suspicion. According to Aronson, “Leon was not interested in improving the human condition. Not in the least … Trying to understand human nature and doing good research (not doing g...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Dedication
• Table of Contents
• Preface
• Acknowledgements
• 1. The sunset on a golden age: reflections on the gap between promise and practice
• 2. Crisis and controversy in classical social psychology
• 3. Experiments as theater: the art of scientific demonstration in Sherif and Asch
• 4. Scientific demonstration in Milgram, Zimbardo, and Rosenhan: more evidence from the archives
• 5. Bystander research: plumbing the psyche of the indifferent Samaritan
• 6. Social psychology engineers wealth and intelligence: the Hawthorne and Pygmalion effects
• 7. A guide to the myth of media effects
• 8. Gender and psychology: from feminism to Darwinism
• 9. The failures of experimental social psychology in the classical period
• 10. The replication crisis: social psychology in the age of retraction
• Epilogue: looking forward: scientific life after experiments
• References
• Index