Research Design and Statistical Analysis

Third Edition

Jerome L. Myers, Arnold D. Well, Robert F. Lorch Jr

• Publisher: Routledge
• Year: 2013
• ISBN:
  9781135811631
• Edition: 3
• Topic: Psychology
• Subtopic: Research & Methodology in Psychology
• Index: Psychology

Research Design and Statistical Analysis provides comprehensive coverage of the design principles and statistical concepts necessary to make sense of real data. The book's goal is to provide a strong conceptual foundation to enable readers to generalize concepts to new research situations. Emphasis is placed on the underlying logic and assumptions of the analysis and what it tells the researcher, the limitations of the analysis, and the consequences of violating assumptions. Sampling, design efficiency, and statistical models are emphasized throughout. As per APA recommendations, emphasis is also placed on data exploration, effect size measures, confidence intervals, and using power analyses to determine sample size. "Real-world" data sets are used to illustrate data exploration, analysis, and interpretation. The book offers a rare blend of the underlying statistical assumptions, the consequences of their violations, and practical advice on dealing with them.

Changes in the New Edition:

• Each section of the book concludes with a chapter that provides an integrated example of how to apply the concepts and procedures covered in the chapters of the section. In addition, the advantages and disadvantages of alternative designs are discussed.
• A new chapter (1) reviews the major steps in planning and executing a study, and the implications of those decisions for subsequent analyses and interpretations.
• A new chapter (13) compares experimental designs to reinforce the connection between design and analysis and to help readers achieve the most efficient research study.
• A new chapter (27) on common errors in data analysis and interpretation.
• Increased emphasis on power analyses to determine sample size using the G*Power 3 program.
• Many new data sets and problems.
• More examples of the use of SPSS (PASW) Version 17, although the analyses exemplified are readily carried out by any of the major statistical software packages.
• A companion website with the data used in the text and the exercises in SPSS and Excel formats; SPSS syntax files for performing analyses; extra material on logistic and multiple regression; technical notes that develop some of the formulas; and a solutions manual and the text figures and tables for instructors only.

Part 1 reviews research planning, data exploration, and basic concepts in statistics including sampling, hypothesis testing, measures of effect size, estimators, and confidence intervals. Part 2 presents between-subject designs. The statistical models underlying the analysis of variance for these designs are emphasized, along with the role of expected mean squares in estimating effects of variables, the interpretation of nteractions, and procedures for testing contrasts and controlling error rates. Part 3 focuses on repeated-measures designs and considers the advantages and disadvantages of different mixed designs. Part 4 presents detailed coverage of correlation and bivariate and multiple regression with emphasis on interpretation and common errors, and discusses the usefulness and limitations of these procedures as tools for prediction and for developing theory.

This is one of the few books with coverage sufficient for a 2-semester course sequence in experimental design and statistics as taught in psychology, education, and other behavioral, social, and health sciences. Incorporating the analyses of both experimental and observational data provides continuity of concepts and notation. Prerequisites include courses on basic research methods and statistics. The book is also an excellent resource for practicing researchers.

PART 1
Foundations of
Research Design
and Data Analysis

Chapter 1
Planning the Research

1.1 OVERVIEW

---

There are three essential stages in carrying out an effective research project. In the first stage, the research is planned: objectives are stated; decisions are made about the treatments to be included, the measures to be obtained, and the type and number of subjects; the research design is determined; and possible patterns of results and their implications are contemplated. This stage is reflected primarily in the Method section of the final research report. In the second stage, the data are collected and analyzed: descriptive statistics are calculated; population parameters are estimated; and inferential tests are performed to determine whether any obtained effects are larger than those we would expect to occur due to chance. The outcomes of the analyses are presented in words, tables, and graphs in the Results section of the final report. The final stage is the interpretation of the results, which typically is presented in the Discussion section of the research report: What do the results tell us about the answers to the questions we initially asked? Answering the questions that motivated the research is, of course, our ultimate goal; however, correct conclusions about what our results mean are totally dependent upon the first two stages. If the study is designed in such a way that factors other than the independent variable may have influenced the results, we may be led to incorrect conclusions. Even if the design of the study is sound, our statistical tests may fail to reveal effects that are present in the population from which we have sampled if our measures are very variable, or if we collect too few data. Finally, despite a sound design and adequate procedures, data analyses that violate assumptions underlying the statistical procedures may lead us to incorrect inferences. We may think of the planning and analysis stages as providing input to the inferential stage and, as an old adage states, “garbage in, garbage out.”

In this chapter, we focus on the initial stage of planning the research. We will present an overview of the decisions that confront the researcher at the onset of a new project, and of the factors that should influence those decisions. In subsequent chapters, we will return to many of the issues raised in this chapter, explicitly linking the decisions made in the planning stage to aspects of the data analysis. Indeed, the decisions made in planning the research are the major factors influencing the results of statistical analyses, and subsequently influencing the conclusions that are drawn.

Chapter 1 is organized by the major decisions that must be made in planning a study:

• The independent variable. What is the question being asked in the research study? The question must be translated into an independent variable to be manipulated in an experiment or a predictor variable to be measured in an observational study.

• The dependent variable. What measure or measures should we use? In any study, there is a choice of measures. Different measures will provide different information and have different psychometric properties. Some measures may be better windows than others on the phenomenon we are studying. Some measures may be more sensitive than others to variation in the behaviors that are of interest. Some measures may be more reliable than others as indicators of some aspect of ability or performance. How should we balance these considerations in deciding among alternative measures?

• The subject population. Who is the target of the research question? Are we interested in healthy, elderly adults? Do we wish to study brain development in rats? And how should observations be sampled from the relevant population? Shall we sample from a diverse population, perhaps varying widely in attributes such as age, intelligence, and ethnicity? Or should our sampling process be more tightly constrained? What are the implications of our decisions about sampling for conclusions that we hope to be able to make?

• Nuisance variables. In addition to decisions about the independent and dependent variables, the researcher must carefully consider the possible influences of other variables in the research study. What other variables may plausibly influence the dependent variable? These other variables are potentially a nuisance in two very important respects. First, if they are not taken into account, they may confound the independent variable. In that case, it may be impossible to determine whether a difference in the mean scores across various conditions is due to an effect of the independent variable, or to effects of the nuisance variables that are correlated with the independent variable. Second, even if steps are taken to make certain that the independent variable is not confounded with other variables, nuisance variables contribute random variability to the data. This “error variance” is a concern because it can result in our failing to detect effects of the independent variable.

• The research design. In some circumstances, the research question demands an observational study; in other circumstances, the research question can be addressed by an experiment; in all circumstances, there are many options in designing the final study. The choice among the options must be informed by all of the questions considered to this point: What is the independent variable? What is the population to be studied? What measure has been chosen? What are the potential nuisance variables, and how—and how much—are they expected to influence the dependent variable?

• The statistical analyses. The statistical analyses should be planned before collecting the data for two reasons. First, planning the analyses has the healthy effect of forcing the researcher to specify the questions to be addressed by the data. Second, it enables the researcher to be sure that, given the planned research design, the targeted questions can be answered by a statistical analysis.

The decisions made in the planning stage are interrelated; therefore, any sequencing of those decisions is a bit arbitrary and may be somewhat misleading. Nevertheless, for expository purposes, we will sequence the six categories of considerations by the order in which the decisions are typically first confronted by the researcher.

1.2 THE INDEPENDENT VARIABLE

---

Research begins with a question. What is the best way to teach the logic of a simp...