Natural Experiment

12 Key excerpts on "Natural Experiment"

• 

  eBook - PDF

  Advances in Experimental Political Science

  • James N. Druckman, Donald P. Green(Authors)
  • 2021(Publication Date)
  • Cambridge University Press

    (Publisher)

  3 Sekhon and Titiunik (2012) consider various exam- ples of Natural Experiments where this phenomenon occurs, including the study by Bhavnani (2009) cited above. this definition of a Natural Experiment is conceptually clear and its implementation relatively uncontroversial, it is not the focus of my discussion. Instead, my interest lies in another widely used definition that interprets a Natural Experiment as some sort of imperfect approximation to a randomized controlled experiment. According to this figurative definition, a Natural Experiment is a situation in which an external event introduces variation in the allocation of the treatment, and the researcher uses the external event as the basis to claim that the treatment is “as good as random” or “as-if random,” but no physical randomization device is explicitly employed by any human being. Scholars who employ this notion of Natural Experiments do not typically offer a formal definition of “as-if randomness,” but rather refer heuristically to an analogy or comparison with randomized experiments. Different versions of this analogy have been offered in political science, economics, public health, and other sciences. In political science, Dunning (2008) defines a Natural Experiment as a study in which the data come “from naturally occurring phenomena” (p. 282), where the treatment is not assigned randomly, but the researcher makes “a credible claim that the assignment of the nonexperimental subjects to treatment and control conditions is ‘as if’ random” (p. 283). In economics, Meyer (1995) defines a Natural Experiment as a study that investigates “outcome measures for observations in treatment groups and comparison groups that are not randomly assigned” (p. 151), and Angrist and Krueger (2001) define a Natural Experiment as a situation “where the forces of nature or government policy have conspired to produce an environment somewhat akin to a randomized experiment” (p. 73).

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Ethical Issues in Psychology

  • Philip Banyard, Cara Flanagan(Authors)
  • 2013(Publication Date)
  • Routledge

    (Publisher)

  The three ethical issues (consent, harm, privacy) discussed so far relate to participants’ rights. Kimmel (1996) also points out that there are wider issues for the profession and society. Once such studies become general knowledge (and a number of psychological studies do make their way into the popular press), then people start to think that a person collapsing on the subway could just be a confederate in a psychology experiment. Such deceptions bring psychology into disrepute and may alter the behavior of the general public so, for example, they do not help in an emergency situation.

  At their best, these studies give us an insight into how people behave in everyday situations; at their worse they are little more than a candid camera technique used to wind people up and then chuckle at their response.

  KEY TERMS

  Independent variable Some event that is directly manipulated by an experimenter in order to test its effect on another variable (the dependent variable).

  Natural Experiment A research method in which the experimenter cannot manipulate the independent variable directly, but where it varies naturally, and the effect on a dependent variable can be observed.

  Quasi-experiment Studies that are ‘almost’ experiments but lack one or more features of a true experiment, such as full experimenter control over the independent variable and/or random allocation of participants to conditions. This means that such studies cannot claim to demonstrate causal relationships.

  Natural Experiments

  There are behaviors that psychologists wish to study but where it would be impossible to manipulate an

  independent variable

  , however such ‘manipulation’ may occur naturally. These are called

  Natural Experiments

  or

  quasi-experiments

  : for example, observing the effects of child abuse on emotional development or exposing individuals to TV and observing what effect this has on their behavior. You cannot deliberately expose a child to abuse for obvious reasons and you cannot deliberately stop people watching TV because there are very few people in the world who have never watched TV. There are, however, some communities where TV has only recently been introduced, such as St. Helena (an island in the middle of the South Atlantic Ocean), and this gives psychologists the natural opportunity to observe the effects of TV (the independent variable) on, for example, antisocial behavior (a dependent variable) (Charlton et al.,

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Applied Child Study

  A Developmental Approach

  • Anthony D. Pellegrini, David F. Bjorklund(Authors)
  • 1998(Publication Date)
  • Psychology Press

    (Publisher)

  A major advantage of field experiments over those in a laboratory is that the children are being observed in a realistic and familiar setting. As a result, the conclusions made about children’s observed behavior also are likely to be supported by observations in similar, nonmanipulated, settings.

  The field experiment also has its disadvantages. Experimenters typically exercise control on an external institution, like the classroom, as they can in an experimental situation. For example, teachers in a field experiment, because they were often hired for reasons other than their ability to act as experimenters, may not be consistent in their use of a particular experimental technique. As a result, the actual experimental treatment may not be consistent with the ideal treatment. Experimenters who are trained and retrained to implement a specific procedure may be more reliable.

  “NATURAL” EXPERIMENTS

  Moving down the ladder of control from true laboratory experiments we come to the interesting case of “natural” experiments. Natural Experiments exist when important natural events occur which may have an effect on children’s behavior. Examples of such events include: a family moving from one neighborhood to another, a fetus’ in utero experience, or a group witnessing a traumatic event. We then gauge the effect of these events (considered independent, or predictor, variables) on theoretically relevant behavior. Take the case (reviewed in Rutter, 1985) of a child who is having behavior problems in one school. His parents move house and he attends another school, one with fewer boys having behavior problems. The boy’s problems then decline. We conclude that if we statistically control the effect of extraneous variables, such as income, the move and school change contributed to the decline.

  Another case: A mother is having pregnancy difficulties and her physician prescribes hormones (androgens). These male hormones represent a “treatment” on the fetus’ development and they typically result in girls exhibiting more male-typical behaviors than comparisons.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Experimental Design

  From User Studies to Psychophysics

  • Douglas W. Cunningham, Christian Wallraven(Authors)
  • 2011(Publication Date)
  • A K Peters/CRC Press

    (Publisher)

  For example, Jane Goodall became quite famous in the 1970s and 1980s for study-ing the naturally occurring behavior of chimpanzees in the wild. This form of experiment is the norm in many areas of science, from anthropology to as-tronomy to zoology. In some cases, such as with the natural behavior of wild chimpanzees, native tribes in the Amazon, or supernovas, it is impossible— even in principle—to cause the events that interest us. In such cases, observa-tional experimentation is the only option available. In other cases, such as in studying epidemic outbreaks, it is possible to generate the events of interest, but unethical to do so. Regardless of the reason, in an observational experi-ment we must observe the events or actions in the exact form in which they 3 4 1. What Is an Experiment? have arisen and wait for the specific events or variants that interest us to occur naturally. At the other extreme lies controlled research (also called experimental re-search). This form of experimentation relies on events or actions that are caused by the experimenter intentionally, and is the method of choice in most natural sciences, including psychology. Indeed, the ability to repeatedly and reliably produce a specific event, image, situation, etc. (referred to as the stim-ulus ), as well as finely controlled variants of it, is at the core of most perceptual research methodology. The reasons for this will become much clearer in Chap-ter 2. Increasingly, computer scientists are being required to conduct controlled, human experiments, generally using variants of established perceptual re-search methodology. Although experimental psychologists undergo several years worth of training in basic research methodology in addition to several years worth of supervised practical application of that methodology, computer scientists generally do not. Indeed, very few computer science programs offer even a single course on designing controlled, human experiments.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Microeconometrics

  • Steven Durlauf, L. Blume, Steven Durlauf, L. Blume(Authors)
  • 2016(Publication Date)
  • Palgrave Macmillan

    (Publisher)

  Natural Experiments and quasi-Natural Experiments The term ‘Natural Experiment’ has been used in many, often, contradictory, ways. It is not unfair to say that the term is frequently employed to describe situations that are neither ‘natural’ nor ‘experiments’ or situations which are ‘natural, but not experiments’ or vice versa. It will serve the interests of clarity to initially direct most of our attention to the second term – experiment. A useful, albeit philosophically charged definition of an experiment ‘is a set of actions and observations, performed in the context of solving a particular problem or question, to support or falsify a hypothesis or research concerning phenomena’ (Wikipedia, 2006). With such a broad definition in hand, it may not be surprising to observe a wide range of views among economists about whether or not they perform experiments. Vernon Smith, for example, in EXPERIMENTAL METHODS IN ECONOMICS, begins with the premise that ‘historically, the method and subject matter of economics have presupposed that it was a non–experimental y science more like astronomy or meteorology than physics or chemistry’ (emphasis added). As he makes clear, his observation implies that today , economics is an experimental science. Bastable’s article on the same subject in the first edition of The New Palgrave overlaps only superficially with Smith’s and divides experiments along the lines suggested by Bacon: experimenta lucifera, in which ‘theoretical’ concerns dominate, and experimenta fructifera, which concern themselves with ‘practical’ matters. In sharp contrast to Smith, Bastable concludes that experimenta lucifera are ‘a very slight resource’ (1987, p. 240) in economics. These two views of experiment, however, do not seem helpful in understanding the controversy regarding Natural Experiments. ‘Experiment’ in our context is merely the notion of putting one’s view to the most ‘severe’ test possible.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Social Science Experiments

  A Hands-on Introduction

  • Donald P. Green(Author)
  • 2022(Publication Date)
  • Cambridge University Press

    (Publisher)

  When studying naturally occurring experiments, the main challenges are reconstructing the exact procedure by which the random assign- ments occurred and tracking outcomes among those who won or lost the lottery. In other words, for a study to qualify as a naturally occurring experiment, a researcher must know which subjects were allocated to each experimental condition and must be able to verify that the allocation was random. 4 Having described some of the main types of experimental settings, let’s now delve more deeply into each category in order to get a sense of what each type of experiment looks like in practice. .    Since the 1940s, the lab has frequently been used to study how mass media shapes what people think about social and political issues. 5 In order to appreciate the details of how 1 See Parigi et al. (2017) for an overview of online experiments. Some noteworthy examples include Munger (2017) and Siegel and Badaan (2020), which study experimental interventions aimed at curbing inflamma- tory rhetoric by Twitter users, and Doleac and Stein (2013), which examines discrimination in online resale markets. 2 See Berinsky et al. (2012), Teschner and Gimpel (2018), and Coppock and McClellan (2019) on the behavior of experimental participants recruited from online labor markets and survey aggregators. 3 See Kuhn et al. (2011) on cash lottery prizes, Hall et al. (2019) on a land distribution lottery in the Antebellum South and its effects on slave ownership and fighting for the Confederacy, Clingingsmith et al. (2009) on visa lotteries allowing a pilgrimage to Mecca, and Finkelstein et al. (2012) on the Oregon Medicaid lottery. 4 Other scholars use the term “Natural Experiment” to include studies that do not involve random assignment. For example, studies that examine the effects of wars, treaties, famines, and natural disasters have all been called Natural Experiments despite the fact that none of the interventions are randomly allocated.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Causality

  Statistical Perspectives and Applications

  • Carlo Berzuini, Philip Dawid, Luisa Bernardinell, Carlo Berzuini, Philip Dawid, Luisa Bernardinell(Authors)
  • 2012(Publication Date)
  • Wiley

    (Publisher)

  18 ‘Natural Experiments’ as a means of testing causal inferences Michael Rutter MRC SGDP Centre, Institute of Psychiatry, London, UK 18.1 Introduction

  In this chapter many different forms of ‘Natural Experiments’ that can be helpful in testing causal inferences are considered. Although there are general principles for good research designs, they take rather different forms according to the nature of the sample and the purpose of the study. In discussing the science, it is crucial to specify the measures and the target population within which cause is to be inferred. In all cases, the analysis needs to complement and take advantage of the design. Accordingly, the presentation of specific examples is essential for considering what makes for better and for less justified causal inferences.

  18.2 Noncausal interpretations of an association

  There are several rather different reasons why statistically significant correlations or associations or differences cannot be assumed to imply a causative influence (Academy of Medical Sciences, 2007; Jaffee et al. , 2011 Rutter, 2007). Because different research strategies are needed to deal with each of the problems, it is necessary to begin with a brief summary of the nature of each hazard. Throughout this chapter, all of the examples come from the fields of psychology and psychopathology, but the principles apply across the whole of biomedicine (even though the relative magnitude of the hazard varies with different outcomes and different causal influences).

  A key issue concerns the possibility of reverse causation. Bell (1968) first raised this matter in relation to socialisation effects. He queried whether associations between, say, marked parental punishment and children's mental disorder reflected the influence of children's disruptive behaviour on parental disciplinary practices, rather than the other way round. The reality of child effects has been shown using a range of different research strategies (Bell and Harper, 1977), including experimental methods (Anderson et al. , 1986) and adoptee studies (Ge et al. , 1996; O’Connor et al. , 1998). Longitudinal studies, too, have shown that the rate of environmental hazards in adult life is strongly associated with prior behaviour in childhood (Robins, 1966; Champion et al. , 1995). Second, there is the possibility of social selection – meaning that the outcome reflects the origin of the putative risk factor, rather than its effects. Thus, is the marked increase in psychopathologic risk associated with being born to a teenage mother (Moffitt and The E-Risk Study Team, 2002) due to the quality of rearing provided by an adolescent parent or to the sort of person likely to have a child during the teenage years (Maynard, 1997)? Similar questions arise with respect to the association between social disadvantage and some forms of psychopathology (see Miech et al.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Research Methods

  A Tool for Life

  • Bernard C. Beins(Author)
  • 2017(Publication Date)
  • Cambridge University Press

    (Publisher)

  In fact, until the middle of the 1900s, psycholo- gists, like other scientists, referred to any research project as an experiment. Since then, however, psychologists have used the term in a specific way. An experiment is a methodology in which a researcher controls variables system- atically. The researcher alters the level, intensity, frequency, or duration of a variable and examines any resulting change in behavior. As such, research is experimental only when the investigator has control over the variable that might affect a behavior. By controlling and manipulating variables systematically, we can determine which variables influence behaviors that we are studying. Given that we recognize the advantage of the experimental approach, why would we bother to consider other types of research strategies? The answer is that ethical and practical considerations dictate the approaches we use. For example, suppose we wanted to know whether the amount of sleep a pregnant woman gets affects a baby’s weight at birth. It would be unethical to force a woman to get a certain number of hours of sleep each night. It would also be impossible to do. You can’t force people to sleep. In addition, in the course of living a life, people don’t always stick to the same schedule every day. There are too many inconsistencies in people’s lives to permit strict control over sleeping schedules. An investigator who wanted to see the relation between amount of sleep women get and their newborn babies’ weights would have two basic options: to experiment with non- human animals or to use a nonexperimental method. In some areas of psychology, the experimental approach predominates. In other domains, researchers choose other methods. In the end, the choice of research strategies depends on the practicalities of the project. Sometimes experiments are possible and fea- sible; sometimes they are possible but not realistic.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Social Research Methodology

  A Critical Introduction

  • Roger Gomm(Author)
  • 2008(Publication Date)
  • Bloomsbury Academic

    (Publisher)

  Sometimes experimenters do not so much opt for more natural-ism as have it thrust upon them, since there may be practical and ethical reasons why experimental control cannot be imposed on some variables in an experiment (see Chapter 7). This chapter looks first at four kinds of more naturalistic experiments, field experiments (Section 4.2) and decision-simulation experiments (Section 4.3), design experiments , and n-of-1 experiments (Section 4.5) and considers the benefits of relaxing experimental control in pursuit of naturalism and the kinds of problems which arise when experimental control is relaxed. The experiments discussed here are controlled experiments, even though the controls are weak in some regards. They would be termed quasi-experiments – not because they are more naturalistic but because true experiments always involve creating comparison groups by randomization (Chapter 3, Section 3.3). The remainder of the chapter considers strategies which might be used to judge how naturalistic, or unnaturalistic, an experimental situation was, in order to decide how far its results might apply more widely. In doing this, the chapter will also look at the use of experimental designs in applied social research in evaluating the effectiveness of medical, social work, educational or criminal justice interventions. 4.2 Field experiments There is no definitive meaning of the term ‘field experiment’. When I use it I am thinking about the kind of experiment where the researcher injects some control into an otherwise naturally occurring situation – for example, those experiments where two actors playing the role of job applicants with identi-cal CVs, but of different ethnicity, apply for the same real jobs, and covertly record what happens (Brown and Gay 1985). As a major example I shall use part of a famous study by David Rosenhan (1973) which has been reprinted many times (for example, 1996). I have deliberately chosen this because it is 86 • SOCIAL RESEARCH METHODOLOGY

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Developmental Psychopathology, Theory and Method

  • Dante Cicchetti(Author)
  • 2015(Publication Date)
  • Wiley

    (Publisher)

  As we indicated in the introduction to this chapter, several rather different methodological hazards can lead to misleading inferences from observational studies that involve no kind of experimental approach. The Natural Experiments that we have considered differ in which of these hazards they mainly seek to deal with. No strategy is equally appropriate for all the different hazards. Similarly, all research designs involve a range of crucial assumptions. Natural Experiments are no exception to that rule. We have sought to note some of the most important assumptions with the different designs and have indicated, where possible, what may be done to test the assumptions. Fortunately, the different forms of Natural Experiment vary in the assumptions they require. What this means is that conclusions are always likely to be more robust when the same answers are given by several quite different types of Natural Experiment. The consistent evidence, for example, that prenatal exposure to maternal smoking does not have a causal effect in leading to ADHD or conduct disorder well illustrates that point. Kendler and Gardner (2010) provided another example in which they use both co-twin control and propensity score matching to consider the possible causal impact of stressful life events on major depressive disorder. All Natural Experiments require appropriate statistical methods, but in some cases statistics are crucial in adding additional means of testing the process being investigated. The example of the effects of membership of a delinquent gang on an individual's engagement in crime constitutes a good example of this kind.

  A key limitation of almost all the Natural Experiments that we have discussed is that the designs are much stronger in showing whether or not there has been environmental risk mediation than they are in showing which particular aspect of the environment provided the mediation. Different designs and different forms of statistical analysis are required to take that absolutely crucial step from identifying a truly causal risk environment to a much more difficult issue of precisely which aspect of the environment provided the risk. That is especially the case when dealing with complex environments and when, as is usually the case, there are bidirectional effects that may vary over time. However, it should no longer be acceptable for researchers (or clinicians) to avoid considering whether a causal inference is justified. Both Rutter (2007) and Foster (2010) pointed out that a high proportion of the literature ignores this important question. Some deal with the problem by arguing that they are dealing only with statistical associations and not causation. However, why anybody should be in the least bit interested in statistical associations, if they carry no implications for causation, remains a mystery. Others make explicit that their research methods cannot test causation; nevertheless, in discussing findings at the end of the paper they suggest that the findings do mean causation. Natural Experiments do not provide anything approaching a complete answer to the dilemma of determining when correlation can rightly imply causation, but they do take the field forward in a most important way.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Psychology

  Modules for Active Learning

  • Dennis Coon, John Mitterer, Tanya Martini, , Dennis Coon, John Mitterer, Tanya Martini, (Authors)
  • 2021(Publication Date)
  • Cengage Learning EMEA

    (Publisher)

  The experimental method allows the careful control of con-ditions to bring cause-and-effect relationships into sharp focus (Stangor, 2015). Hence, it is generally accepted as the Experiment A study in which the investigator manipulates at least one variable while measuring at least one other variable. Like Chan, we all conduct little experiments to detect cause-and-effect connections. In a more formal way, that is exactly what psychologists do when they want to understand why we act the way we do. Let’s see why. Copyright 2022 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. PSYCHOLOGY MODULES FOR ACTIVE LEARNING 38 most powerful scientific research tool. To perform an ex-periment, you would do the following: 1. Directly vary a condition that you think might affect behavior. 2. Create two or more groups of participants. These groups should be alike in all ways except the condition you are varying. Usually, one group serves as a control group with which the other groups are compared. 3. Record whether varying the condition has any effect on behavior. Suppose that Chan just happened to be a psychologist who wanted to find out if using smart glasses while driving af-fects the likelihood of having an accident. First, he would form two groups of people. Then he could give the members of one group a test of driving ability while using their smart glasses. The second group would take the same test without using smart glasses. By comparing the average driving abil-ity scores for the two groups, he could tell if the use of smart glasses affects driving ability.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Experimental Design and Statistics for Psychology

  A First Course

  • Fabio Sani, John Todman(Authors)
  • 2008(Publication Date)
  • Wiley-Blackwell

    (Publisher)

  For instance, you could rely on the systematic observation of behaviour. This is what psychologists who are interested in animal behaviour tend to do. Basically, animal psychologists go where the animals live, or create an artificial environment in which animals are placed, and then they observe and record animals’ behaviour through the use of established procedures. For example, a psychologist who is interested in, say, the behaviour of chimps could use systematic observation to demonstrate that a high amount of time devoted to ‘grooming’, that is, reciprocal cleaning and brushing among a group of chimps, leads to more frequent cooperative activities in the group. However, the technique that is most often used by psychologists – as well as scientists in many other disciplines – is the experiment . Experiments constitute a very powerful technique for the investigation of causal links between different things, and this is why they are ideal for testing causal hypotheses. Experiments are typically run in laboratories (although it is possible to conduct them in more natural settings too). Because, as specified above, a hypothesis states that a specific change in one thing will produce (cause) a specific change in another thing, experiments are based on the creation of a situation in which a change in one thing is artificially produced, and the corresponding change in another thing is systematically observed. This book – with the exception of Chapter 10, in which we deal with non-experimental research – is entirely devoted to the use of the experiment as a method of hypotheses testing. To conclude this chapter, it is necessary to make a further observation on the research process.

  Sign up to read

  Learn more about book