Handbook of Psychology, Research Methods in Psychology

Irving B. Weiner, John A. Schinka, Wayne F. Velicer

• Publisher: Wiley
• Year: 2012
• Print ISBN: 9780470890646
• eBook ISBN: 9781118282038
• Edition: 2
• Topic: Psychology
• Subtopic: History & Theory in Psychology
• Index: Psychology

Psychology is of interest to academics from many fields, as well as to the thousands of academic and clinical psychologists and general public who can't help but be interested in learning more about why humans think and behave as they do. This award-winning twelve-volume reference covers every aspect of the ever-fascinating discipline of psychology and represents the most current knowledge in the field. This ten-year revision now covers discoveries based in neuroscience, clinical psychology's new interest in evidence-based practice and mindfulness, and new findings in social, developmental, and forensic psychology.

Part I

Foundations of Research Issues

Chapter 1

Experimental Design

Roger E. Kirk

Some Basic Experimental Design Concepts

Three Building Block Designs

Classification of Experimental Designs

Factorial Designs

Randomized Block Factorial Design

Factorial Designs With Confounding

Hierarchical Designs

Experimental Designs With a Covariate

References

Some Basic Experimental Design Concepts

Experimental design is concerned with the skillful interrogation of nature. Unfortunately, nature is reluctant to reveal her secrets. Joan Fisher Box (1978) observed in her biography of her famous father, Ronald A. Fisher, “Far from behaving consistently, however, Nature appears vacillating, coy, and ambiguous in her answers” (p. 140). Nature's most effective tool for confusing researchers is variability—in particular, variability among subjects or experimental units. Although nature can be duplicitous, Ronald A. Fisher showed that by comparing the variability among subjects treated differently to the variability among subjects treated alike, researchers can make informed choices between competing hypotheses in science and technology.

We must never underestimate nature—she is a formidable foe. Carefully designed and executed experiments are required to learn her secrets. An experimental design is a plan for assigning subjects to experimental conditions and the statistical analysis associated with the plan (Kirk, 2012, p. 1). The design of an experiment involves five interrelated activities:

1. Formulation of statistical hypotheses that are germane to the scientific hypothesis. A statistical hypothesis is a statement about (a) one or more parameters of a population or (b) the functional form of a population. Statistical hypotheses are rarely identical to scientific hypotheses—they are testable formulations of scientific hypotheses.

2. Determination of the experimental conditions (independent variable) to be manipulated, the measurement (dependent variable) to be recorded, and the extraneous conditions (nuisance variables) that must be controlled.

3. Specification of the number of subjects required and the population from which they will be sampled.

4. Specification of the procedure for assigning the subjects to the experimental conditions.

5. Determination of the statistical analysis that will be performed.

In short, an experimental design identifies the independent, dependent, and nuisance variables and indicates the way in which the randomization and statistical aspects of an experiment are to be carried out.

Analysis of Variance

Analysis of variance (ANOVA) is a procedure for decomposing the total variation displayed by a set of observations into two or more identifiable sources of variation. The procedure enables researchers to interpret the variability in designed experiments. The seminal ideas for both ANOVA and experimental design can be traced to Ronald A. Fisher, a statistician, eugenicist, evolutionary biologist, and geneticist who worked at the Rothamsted Experimental Station that is 25 miles northwest of London. According to Box (1978, p. 100), Fisher developed the basic ideas of ANOVA between 1919 and 1925. The first hint of what was to come appeared in a 1918 paper in which Fisher partitioned the total variance of a human attribute into portions attributed to heredity, environment, and other factors. The analysis of variance table for a two-treatment factorial design first appeared in a 1923 paper published with M. A. Mackenzie (Fisher & Mackenzie, 1923). Fisher referred to the table as a convenient way of arranging the arithmetic. In 1924, Fisher (1925) introduced the Latin square design in connection with a forest nursery experiment. The publication in 1925 of his classic textbook Statistical Methods for Research Workers and a short paper the following year (Fisher, 1926) presented all the essential ideas of analysis of variance. The textbook (Fisher, 1925, pp. 244–249) included a table of the critical values of the ANOVA test statistic in terms of a function called z, where

. The statistics

and

denote, respectively, treatment and error variance. A more convenient form of Fisher's z table that did not require looking up log values was developed by George Snedecor (1934). His critical values are expressed in terms of the function

that is obtained directly from the ANOVA calculations. He named it F in honor of Fisher. Fisher's field of experimentation—agriculture—was a fortunate choice because results had immediate application with assessable economic value, because simplifying assumptions such as normality and independence of errors were usually tenable, and because the cost of conducting experiments was modest.

Three Principles of Good Experimental Design

The publication of Fisher's Statistical Methods for Research Workers and his 1935 The Design of Experiments gradually led to the acceptance of what today is considered to be the cornerstone of good experimental design: randomization. It is hard to imagine the hostility that greeted the suggestion that subjects or experimental units should be randomly assigned to treatment levels. Before Fisher's work, most researchers used systematic schemes, not subject to the laws of chance, to assign subjects. According to Fisher, random assignment has several purposes. It helps to distribute the idiosyncratic characteristics of subjects over the treatment levels so that they do not selectively bias the outcome of the experiment. Also, random assignment permits the computation of an unbiased estimate of error effects—those effects not attributable to the manipulation of the independent variable—and it helps to ensure that the error effects are statistically independent.

Fisher popularized two other principles of good experimentation: replication and local control or blocking. Replication is the observation of two or more subjects under identical experimental conditions. Fisher observed that replication enables a researcher to estimate error effects and to obtain a more precise estimate of treatment effects. Blocking, on the other hand, is an experimental procedure for isolating variation attributable to a nuisance variable. As the name suggests, nuisance variables are undesired sources of variation that can affect the dependent variable. There are many sources of nuisance variation. Differences among subjects comprise one source. Other sources include variation in the presentation of instructions to subjects, changes in environmental conditions, and the effects of maturation, fatigue, and learning when subjects are observed several times. Three experimental approaches are used to deal with nuisance variables:

1. Holding the variable constant.

2. Assigning subjects randomly to the treatment levels so that known and unsuspected sources of variation among the subjects are distributed over the entire experiment and do not affect the subjects in just one or a limited number of treatment levels.

3. Including the nuisance variable as one of the factors in the experiment.

The third experimental approach uses local control or blocking to isolate variation attributable to the nuisance variable so that it does not appear in estimates of treatment and error effects. A statistical approach also can be used to deal with nuisance variables. The approach is called analysis of covariance and is described in the last section of this chapter. The three principles that Fisher vigorously championed—randomization, replication, and local control—remain the cornerstones of good experimental design.

Three Building Block Designs

In this section I describe three simple analysis of variance designs that can be combined to form more complex designs. They are the completely randomized design, the randomized block design, and the Latin square design. I call these designs building block designs.

Completely Randomized Design

One of the simplest experimental designs is the randomization and analysis plan that is used with a t statistic ...

Table of contents

• Cover
• Title Page
• Copyright
• Dedication
• Editorial Board
• Handbook of Psychology Preface
• Volume Preface
• Contributors
• Part I: Foundations of Research Issues
• Part II: Research Methods in Specific Content Areas
• Part III: Measurement Issues
• Part IV: Data Analysis Issues
• Author Index
• Subject Index