Psychology
Meta Analysis
Meta-analysis is a statistical technique used to combine and analyze the results from multiple independent studies on a specific topic. It provides a more comprehensive understanding of the overall effect size and helps identify patterns or inconsistencies across studies. By synthesizing data from various sources, meta-analysis can offer more robust conclusions and insights into psychological phenomena.
Written by Perlego with AI-assistance
Related key terms
1 of 5
10 Key excerpts on "Meta Analysis"
- Brian S Everitt, Andrew Pickles(Authors)
- 2000(Publication Date)
- ICP(Publisher)
CHAPTER 10 Meta- Analysis 10.1. INTRODUCTION In the Cambridge Dictionary of Statistics in the Medical Sciences, meta-analysis is defined thus: A collection of techniques whereby the results of two or more inde- pendent studies are statistically combined to yield an overall answer to a question of interest. The rationale behind this approach is to provide a test with more power than is provided by the separate studies themselves. The procedure has become increasingly popular in the last decade or so, but is not without its critics, particularly because of the difficulties of know- ing which studies should be included and to which population final results actually apply. In essence, meta-analysis is a more systematic approach to com- bining evidence from multiple research projects than the classical review article. Chalmers and Lau (1993) make the point that both approaches can be biased, but that at least the writer of a meta- analytic paper is required by the rudimentary standards of the dis- cipline to give the data on which the conclusions are based, and to defend the development of these conclusions by giving evidence that all available data are included, or to give the reasons for not including the data. In contrast, the typical reviewer arrives at con- clusions that may be biased and then selects data to back them up. 280 Meta-Analysis 281 Chalmers and Lau conclude, “It seems obvious that a discipline which requires that all available data be revealed and included in an analysis has an advantage over one that has traditionally not presented anal- yses of all the data on which conclusions are based.” So meta-analysis is likely to have an objectivity that is inevitably lacking in literature reviews and can also achieve greater precision and generalisability of findings than any single study. Consequently, it is not surprising that the technique has become one of the great- est growth areas in medical research.
- Roger E Millsap, Alberto Maydeu-Olivares, Roger E Millsap, Alberto Maydeu-Olivares(Authors)
- 2009(Publication Date)
- SAGE Publications Ltd(Publisher)
18 Meta-Analysis Andy P. Field META-ANALYSIS Psychologists are typically interested in find-ing general answers to questions. Although answers to these questions can be obtained in single pieces of research, it is common for different researchers to address similar research questions. This replication makes it possible to answer research questions through assimilating data from a variety of sources. This process is known as meta-analysis . The use of meta-analysis has exploded over the past 30 years and even just a cursory scan of recent meta-analyses reveals the diversity of questions that it has been used to address: whether temperament differs across gender (Else-Quest, Hyde, Goldsmith, and Van Hulle, 2006), organizational wellness (Parks and Steelman, 2008); maternal employment and children’s achievement (Goldberg, Prause, Lucas-Thompson, and Himsel, 2008); marital discord as a predictor of domestic violence (Stith, Green, Smith, and Ward, 2008); the relationship between stress and depression (Stroud, Davila, and Moyer, 2008); and whether learning is stronger with static images or animation (Hoffler and Leutner, 2007). These arbitrarily selected recent meta-analyses show the breadth of topics to which the technique has been applied. However, I could have selected any number of others because meta-analysis has become ubiquitous in top ranking journals and is the technique of choice for reviewing research literatures. This was not always the case: search for meta-analytic studies 30 years ago and you would be hard pushed to find any. Figure 18.1 shows a rough estimate of the number of arti-cles discussing or using meta-analysis over the 30-year period from 1977 to 2007 based on a search for ‘meta-analysis’ as a topic in the web of knowledge, a major bibliographic database (http://www.isiwebofknowledge.com/) 1 . The data are split broadly by discipline: social science journals, where most psychological meta-analyses would be found, and science journals.
eBook - ePub- Robert Gifford(Author)
- 2015(Publication Date)
- Wiley-Blackwell(Publisher)
The term “meta-analysis” was coined and formalized by the statistician Gene Glass in 1976. Although Glass is widely recognized as the modern founder of this method, meta-analytic methodology pre-dates his work by several decades (e.g., Cochran, 1937), with the first true meta-analysis consisting of a review of 145 extra-sensory perception experiments published from 1882 to 1939, conducted by Pratt and Rhine and colleagues (see Bösch, 2004). More recently, evidence-based medicine has increasingly relied on this technique, and its popularity has grown among policy-makers and scholars in far-reaching disciplines, including environmental psychology. In an age of ever-proliferating knowledge, meta-analytic procedures offer an efficient means to assess the “overall” state of knowledge on a particular topic, as well as the potential to resolve inconsistencies among individual studies. Furthermore, meta-analyses can be helpful for designing interventions and guiding future research given that these procedures allow researchers to consider the current state of knowledge in an area through a broad lens. Researchers have even gone so far as to meta-analyze meta-analyses, such as Richard, Bond, and Stokes-Zoota’s (2003) analysis of 322 meta-analyses of social psychological phenomena – resulting in an exceptionally broad lens! In short, meta-analysis is truly a tool with many applications.Narrative Reviews versus Meta-Analyses
Narrative reviews
Before the 1990s, narrative reviews were the primary approach used to synthesize data from multiple, related studies. A narrative review is a non-statistical summary of findings; the researcher combines findings from related studies by counting the number of statistically significant studies compared to the number of statistically non-significant studies (in a process known as vote counting
eBook - ePub- Nikos Ntoumanis, Nicholas D. Myers, Nikos Ntoumanis, Nicholas D. Myers(Authors)
- 2015(Publication Date)
- Wiley(Publisher)
11 Application of meta-analysis in sport and exercise science Soyeon Ahn1 , Min Lu1 , G. Tyler Lefevor1 , Alicia L. Fedewa2 , and Seniz Celimli1 1 School of Education and Human Development, University of Miami, Coral Gables, FL, USA 2 Department of Educational, School, and Counseling Psychology, University of Kentucky, Lexington, KY, USAGeneral Introduction
Meta-analysis,1 or “analysis of analyses,” was coined by Gene V. Glass (1976) in his presidential talk at the annual meeting of the American Educational Research Association. Meta-analysis (see footnote 1) is a tool that provides a cohesive picture of a phenomenon of interest by statistically integrating study findings on the “same” or “similar” phenomenon across a large collection of studies. It boasts a number of strengths, such as an increase in statistical power by including more samples from selected studies, which have made meta-analysis popular in many disciplines, including sport and exercise science (Becker & Ahn, 2012).Stages of Meta-Analysis
The general procedure for conducting a meta-analysis is almost identical to that for primary research. Cooper (1984) conceptualized meta-analysis in five stages, which he further expanded to seven stages in 2010 (Cooper, 2010). Stage 1, problem formulation, involves formulating research questions and hypotheses. Stage 2, data collection, involves searching the literature and coding information from relevant studies. Stage 3, data evaluation, involves evaluating the quality of study and dealing with differences in the quality of studies. Stage 4, data analysis, involves analyzing and integrating effect sizes across studies. Stage 5, presentation, involves interpreting the cumulative evidence obtained through the analysis.In each of the five stages, Cooper outlined questions to be addressed that would “enhance or undermine the trustworthiness of the conclusion” (2010, p. 11). These questions allow a researcher to better ensure the validity of the subsequent meta-analysis. Though important, space prevents a detailed discussion of each stage in the present chapter. The interested reader is referred to several excellent sources that give a more detailed description of questions for consideration in each stage of meta-analysis (Borenstein, Hedges, Higgins, & Rothstein, 2009; Cooper, 1984, 2010; Hedges & Olkin, 1985; Wanous, Sullivan, & Malinak, 1989). In this chapter, our focus is primarily on Stage 4 as our main goal is to illustrate how to apply meta-analytic methods in sport and exercise science.
- Enilda Romero-Hall(Author)
- 2020(Publication Date)
- Routledge(Publisher)
To begin to understand and use meta-synthesis, many seminal manuscripts explain the process. In 2005, Walsh and Downe published a literature review describing meta-synthesis, its purpose, and the stages or process of doing meta-synthesis work. They provide an overview of the state of meta-synthesis work and include references to many articles written previously about meta-synthesis. These previous writing give information on what meta-synthesis is and how to go about using it to review qualitative research (Finfgeld, 2003; Sandelowski, Docherty, & Emden, 1997; Thorne, Jensen, Kearney, Noblit, & Sandelowski, 2004), as well as information on the issues involved in meta-synthesis work (Sandelowski et al., 1997) and reflections on using this method (Thorne et al., 2004). Since the 2005 literature review, additional articles have been published continuing to explain meta-synthesis and pushing the capabilities of the method, examined with a critical eye on the value and usefulness as an interpretive method (Zimmer, 2006), challenges associated with the method and its theoretical contributions (Bondas & Hall, 2007), clarifying the process and contributing to the development of the method (Finlayson & Dixon, 2008), and what, if any, generalizability and transferability elements are associated with the method (Finfgeld-Connett, 2010). Over some time, what meta-synthesis is, how to use it, challenges related to it, and the historical roots of the method are described.Meta-Analysis
Meta-analysis uses a statistical analysis evaluating studies together to gain an understanding of the magnitude of similarities and differences in the reported outcomes (Borenstein et al., 2009; Cooper, 2016; Glass, 1976, 2000). Building on the foundation of a narrative review, a typical meta-analysis has several standard features: search strategy transparency including inclusion and exclusion criteria, a clear problem statement indicating why a review is necessary, questions that the review is attempting to address, a critique of existing work, and the generation of new knowledge as an outcome of the review process (Boote & Beile, 2005; Torraco, 2016). From there, in a quantitative meta-analysis, a researcher will engage in structured coding of study features, beginning with calculating an effect size for each outcome. Generally, effect sizes are an effort to quantify the relationships between variables on a standard scale such as Pearson’s r or the differences between groups such as Cohen’s d , Hedges’ g , or an odds ratio. Researchers might also classify the research methods, threats to validity, and critical constructs in the area of literature. Each categorical, ordinal, or continuous variable is then used in subsequent analyses to look for summary effects or trends in the existing literature. Meta-analysis is becoming a standard means to quantitatively synthesize empirical literature (Lipsey & Wilson, 2001). Traditional and network meta-analysis are the two main types (Higgins & Green, 2008).Traditional Meta-Analysis (TMA)
In a traditional meta-analysis, the options include a fixed-effect model and a random-effect model comparing the differences between the treatment of interest, such as problem-based learning, and a common control group. For a fixed-effect, it is assumed that each study included is examining the same phenomenon and that a single true effect size exists. In a random-effect model, each study is considered to have its own effect size. Researchers might choose a random-effect model if studies compare a wide range of grade levels, ability levels, or variations in the treatment. In both cases, fixed and random-effect models result in a weighted mean of effect sizes. The more precise an individual effect size, the more weight it will be given in the mean (Borenstein et al., 2009).
eBook - PDFMeta-analysis in Medical Research
The Handbook for the Understanding and Practice of Meta-analysis
- Gioacchino Leandro(Author)
- 2008(Publication Date)
- BMJ Books(Publisher)
These results then form the basis for further theories and thus for other clinical trials. Medical practice is greatly influenced by the results of clinical studies, especially if they are brought to public attention through prestigious scientific journals or the mass media. In the scientific world, new therapies have a greater impact, so a greater number of publications are therefore produced. However, it is hard to define the quality and the importance of each study. Different studies on the same topic often provide discordant conclusions, giving the reader a confused message. In order to clarify the matter we need the help of experts who can provide a conclusive synthesis of the results of different studies. The growing number of invited reviews and the ‘state-of-the-art’ lectures is a clear example of the need for unequivocal communication for highly debated topics. Meta-analysis, when well designed and appropriately performed, is a pow-erful tool for synthesis. It is an analytical method where both independent and different studies are integrated and their results pooled into a single common result. The meta-analysis, when compared to other forms of re-viewing separate studies, has the great advantage of being less influenced by the personal opinion of the reviewer, and provides unbiased conclusions. Moreover, in a meta-analysis all the results of the single studies examined are reported, and the reader may easily recalculate the data and compare them with the conclusions derived by the authors. The term meta-analysis was coined in 1976. This identifies a process of analysis retrospectively performed on available published data on a specific topic. Introduction 3 When the analysis is done on individual data, it is called a meta-analysis of individual patients.
No longer available |Learn moreThe Psychology Research Handbook
A Guide for Graduate Students and Research Assistants
- Frederick T. L. Leong, James T. Austin(Authors)
- 2005(Publication Date)
- SAGE Publications, Inc(Publisher)
Meta-analysts’ ability to synthesize and draw causal inferences from large and unwieldy areas of primary research did not go unnoticed by pol-icy makers and practitioners. By the late 1980s, meta-analysis had become a needed tool for making sense of massive amounts of data from individual studies (Chalmers et al., 2002). Perhaps for this reason, meta-analytic strategies have garnered wide interdisciplinary appeal. The use of the technique spread from psychology and education to medicine and public policy. Today, meta-analysis is an accepted and respected tech-nique in the behavioral and social sciences, both theoretical and applied. P ROCEDURES OF M ETA -A NALYSIS Searching the Literature and Coding Studies Let us turn now to the more mundane issue of the procedures that constitute a present-day research synthesis. Suppose we enter the office of Dr. Polly C. Analyst, about to undertake a review of research assessing the effects of remedial education on the self-esteem of adolescents. In addition, suppose that Dr. Analyst has already completed her search for relevant studies. Her search strategy employed computer reference databases (e.g., PsycINFO), convention pro-grams, the reference lists of reports, a search of related journals, inquiries to government agen-cies, and letters to active researchers (see Chapter 3 of this volume). Notice that in conducting her search for relevant studies, Polly attempted to uncover both published and unpublished reports. This is important because although there are many reasons for a report not to be published, one reason is that the study failed to reject the null hypothesis (that is, the study did not find a statistically significant difference in self-esteem between participants who received remedial edu-cation and those who did not).
eBook - PDFImproving Learning
Meta-analysis of Intervention Research in Education
- Steven Higgins(Author)
- 2018(Publication Date)
- Cambridge University Press(Publisher)
The number of studies available to review in any area of education can be extensive, so techniques to aggregate and build up understanding of a field in terms of the impact of different interven- tions or approaches and what might explain variation so as to propose further research and test theories and hypotheses are invaluable. In fields like psychology and medicine meta-analysis is relatively uncon- troversial as a synthesis method, with nearly 40 years of development of the various principles and methods involved, despite its initial origins in education. 36 A Brief History of Meta-analysis Limitations and Challenges for Meta-analysis There are limitations, of course, and perhaps the most important is the assumption that the data from evaluations are equivalent or at least comparable across studies. Here the key issue in education is a conceptual one (Lipsey and Wilson, 2000). Are the studies which are being com- pared actually the same in terms of the way that they have defined or implemented a particular approach? This also relates to the nature of the question being addressed. Asking whether phonics interventions are effective for beginning readers to catch up with their peers is different from asking whether phonics approaches are the best approach for begin- ning readers (when compared with other approaches to teaching read- ing). Some studies would be included in both reviews, but in one it may be helpful to combine studies in different categories (phonics, whole word, comprehension-led, whole language, etc.), and clarity about definitions and outcomes (such as decoding words or comprehending sentences) would be essential. Another limitation is the so-called ‘file-drawer’ problem where stu- dies with null or negative effects are not reported or are less likely to be reported because they are not thought to be interesting or to add any- thing new to a field.
eBook - PDF- Michael Borenstein, Larry V. Hedges, Julian P. T. Higgins, Hannah R. Rothstein(Authors)
- 2021(Publication Date)
- Wiley(Publisher)
Some meta-analysts may make questionable judgments, and some critics may make unreasonable demands on similarity. We need to remember that meta-analyses almost always, by their very nature, address broader questions than individual studies. Hence a meta-analysis may be thought of as asking a question about fruit, for which both apples and oranges (and indeed pears and melons) contribute valuable information. One of the strengths of meta-analysis is that the consistency, and hence generalizability, of findings from one type of study to the next can be assessed formally. Of course, we always need to remember that we are dealing with different kinds of fruit, and to anticipate that effects may vary from one kind to the other. It is a further strength of meta-analysis that these differences, if identified, can be investigated for- mally. Assume, for example, that a treatment is very effective for patients with acute 416 Meta-Analysis in Context symptoms but has no effect for patients with chronic symptoms. If we were to combine data from studies that used both types of patients, and conclude that the treatment was modestly effective (on average), this conclusion would not be accurate for either kind of patient. If we were to restrict our attention to studies in only patients with acute symptoms, or only patients with chronic symptoms, we could report how the treat- ment worked with one type of patient, but could only speculate about how it would have worked with the other type. By contrast, a meta-analysis that includes data for both types of patients may allow us to address this question empirically. GARBAGE IN, GARBAGE OUT Criticism The often-heard metaphor garbage in, garbage out refers to the notion that if a meta- analysis includes many low-quality studies, then fundamental errors in the primary studies will be carried over to the meta-analysis, where the errors may be harder to identify.
eBook - PDF- W Paul Vogt(Author)
- 2011(Publication Date)
- SAGE Publications Ltd(Publisher)
There usually is an algebraic path from the reported statistics to a Pearson correlation coefficient or an approximation to one. Some signposts along the paths are set out in Table 3, where it is indicated how one might travel from particular forms of reported data to a product-moment correla-tion measure. Problems of Statistical Inference Whether the findings from a collection of studies are regarded as a sample from a hypothetical universe of studies, or they are in fact a sample from a well-defined population, problems of statistical inference arise. Significance tests or confidence intervals around estimates of averages or regression planes will indicate where the research literature is conclusive on a question and where the aggregated findings still leave doubts – at least insofar as sampling error is concerned. The inferential statistical problems of the meta-analysis of research are uniquely complex. The data set to be analyzed will invariably contain com-plicated patterns of statistical dependence. “Studies” cannot be considered the unit of data analysis without aggregating findings above the levels at which many interesting relationships can be studied. Each study is likely to yield more than one finding. An experiment comparing heterogeneous and 146 Meta-analysis homogeneous ability grouping might produce effect-size measures on three types of school achievement at four points in time; thus, 12 of the several hundred effect-size measures in an aggregate data set would have arisen from a single study. There is no simple answer to the question of how many independent units of information exist in the larger data set. One might attempt to impose some type of cluster or multiple-stage sampling frame-work on the data, but in the end this will probably restrict the movement of an imaginative data analyst. Two resolutions of the problem can be envi-sioned: one risky, the other complex.
Index pages curate the most relevant extracts from our library of academic textbooks. They’ve been created using an in-house natural language model (NLM), each adding context and meaning to key research topics.
