![]()
Contextualizing this book
How is the term research synthesis used in this book? What is the purpose of this book? Who is the intended audience for this book?
Analysis and synthesis of evidence in three modes can contribute to knowledge construction in research. These three modes are primary analysis, secondary analysis and research synthesis. Primary research involves going into an experimental situation, the field, archives or cyberspace to collect raw evidence or data to pursue one's own research questions. The analysis and interpretation of this raw data is referred to as primary analysis and the individuals conducting such research are called primary researchers. Secondary research involves re-analysing and re-interpreting raw evidence or data collected by other primary researchers for their own primary research. This re-analysis or re-interpretation could be pursued to address the same, or different, questions using different analytic tools or interpretive positions. In this book the term āprimary researchā is frequently used as an umbrella term for primary research and secondary research. Research synthesists are different from primary researchers and secondary researchers in the sense that they analyse or interpret primary research reports rather than collecting, analysing or interpreting any raw data or evidence. The evidence, methodological perspectives and techniques employed in each of the three modes of knowledge construction can be qualitative, quantitative or a combination of both (Suri 2011).
Typically, literature reviews form a part of every research report. Researchers conduct literature reviews to contextualize the rationale, theoretical framework, methods and findings of their own studies. Research synthesis is a special type of research review that is not only descriptive, informative and evaluative, but also connective (Mays et al. 2005). āSynthesis refers to making a whole into something more than the parts alone implyā (Noblit and Hare 1988, p. 28). The purpose of a research synthesis is to produce new knowledge by making explicit connections and relations between individual studies that were not visible before. It involves purposeful selection, review, analysis and synthesis of primary research on a similar topic. Many research syntheses also include evidence from the relevant secondary research and research reviews in the field. In a rigorous synthesis, readers are provided sufficient information about the synthesis process so that they can make informed decisions about the extent to which the synthesis findings may be adapted to their own contexts. This type of information about the process is frequently missing in typical literature reviews and some other forms of research reviews (Suri 2012).
Published literature on research synthesis methods
There has been a rapid growth in the number and variety of reports in educational research. Each topic tends to be examined by different researchers in diverse contexts, employing a wide range of methods, resulting in disparate findings on the same topic. Making useable sense of such complex bodies of research can be an overwhelming experience for most stakeholders with an interest in educational research. These stakeholders include policy-makers, administrators, teachers, funding agencies, researchers, students, parents and the wider community.
Research syntheses play an important role in disseminating research knowledge and in shaping further research, policy, practice and public perception. They are frequently cited in scholarly journals (Cooper and Hedges 2009). Issues of methodological rigour in research syntheses are as crucial as they are in primary research (Slavin 2008). During the last four decades, there has been a growing body of literature on research synthesis methods marked by several commendable methodological advances (Sandelowski et al. 2012).
Among early efforts of enhancing rigour in synthesizing research in education, the most well-known improvement has been the formal proposition of āmeta-analysisā as a statistical method for integrating quantitative summaries from individual primary research reports (Glass 1976, p. 3). As several meta-analysts have critically examined and refined various aspects of meta-analytic syntheses in the last four decades, these procedures have become more sophisticated and sensitive. The vast literature on meta-analytic methods has contributed substantially in systematizing and refining the entire process of research synthesis. Meta-analyses and meta-analyzable studies have a particular appeal among politicians because of their ability to provide quantitative generalizations. With the current thrust for Evidence-Based Education (EBE), randomized controlled trials (RCTs) and meta-analyses are being particularly favoured by several politicians, senior administrators and funding agencies (Wiseman 2010). However, meta-analyses can synthesize only a limited portion of contemporary research in most areas of education. Meta-analytic procedures can integrate only those findings that can be reduced to a common index, often an effect size, of a conceptually similar outcome measure. A political push for meta-analyses and meta-analyzable studies has evoked strong responses from several educational researchers who challenge the narrow conception of educational research and EBE that excludes valuable evidence from various qualitative research traditions (Luke et al. 2010).
In the last two decades, many researchers from education and health care have been adapting ideas and techniques from interpretive and critical traditions to inform research synthesis processes. Useful guidelines have been published for synthesizing qualitative research in education and social sciences from a critical interpretive perspective (e.g. Major and Savin-Baden 2010). A large number of publications, from health care and public policy researchers, have focused on synthesizing qualitative research or methodologically diverse research (Hannes and Macaitis 2012). Useful methodological lessons can be learnt from a number of published narrative syntheses of quantitative and qualitative research in top-tier journals like the Review of Educational Research (RER). Insightful commentaries have also been published by critical scholars, especially from education, on the politics, process, use and abuse of research reviews. Several synthesists are calling for increased involvement of consumers of research in formulating and refining questions and protocols for syntheses (e.g. Rees and Oliver 2012).
Each of these individual bodies of literature makes a worthy contribution by providing guidelines to synthesize particular types of primary research reports with certain types of synthesis purposes in mind (Kastner et al. 2012). However, there is a paucity of literature that establishes links and examines complementarities and incompatibilities between different existing research synthesis methods. Also, there is a relative dearth of literature that explores adaptation of a variety of qualitative research perspectives and techniques to the process of a research synthesis.
In recent years, there has been increased activity and interest in this space. As a reviewer of this book noted, there is a degree of similarity between this book and publications from the Evidence for Policy and Practice Information (EPPI) centre. This is not surprising as colleagues from the EPPI Centre and I have been grappling with the same issue of how to conduct more rigorous research syntheses that are inclusive of qualitative research since the late 1990s. This is evident from our various conference presentations and publications on this issue. However, even though we have engaged with the same broad question, we have approached it from different perspectives. The focus of EPPI centre's work has been on systematization of review methods as suggested by the title of their recent book, An introduction to systematic reviews (Gough et al. 2012). However, methodological inclusivity has been the focus of my work since 1998. Their book describes the current state of systematic reviews. This book takes the discussion further by contesting the prevalent discourse to engage researchers in pushing the boundaries of the prevalent conceptions of a research synthesis. It is this provocative engagement with the published literature on research synthesis methods and primary research methods that is a distinguishing feature of this book.
As suggested by the subtitle of this book, the goal here is to expand possibilities within research synthesis methods. It is the in-depth engagement, informed by critical sensibilities, with issues less represented in the published literature on research synthesis methods that sets this book apart from other texts on this topic. While the preliminary discussions of many issues in this book have been published before, this book takes these discussions further by raising new questions, contesting taken-for-granted practices and exploring possibilities many of which have not been discussed in previously published books on research synthesis methods. The extent to which the book engages with a variety of techniques and perspectives associated with a wider spectrum of qualitative research traditions makes it distinct. This is important as educational research is marked by a diversity of epistemological perspectives, methods and techniques. The sheer variety of lenses employed to expand possibilities in research synthesis methods, with an openness that invites further critiques and modifications, makes this book unique.
Methodologically Inclusive Research Synthesis (MIRS) framework
As the rhetoric of Evidence Based Education (EBE) is becoming popular, it becomes crucial to explore multiple ways of ājustifying evidence-based claimsā. This book is one such exploration.
Overarching question pursued in this book
Contemporary educational research is marked by diversity, complexity and richness of purposes, methods and perspectives. How can we accommodate and reflect such variety and complexity at the level of synthesizing educational research?
To address this question, we need an approach to research synthesis that is sufficiently inclusive and sensitive to accommodate and reflect the methodological and epistemological diversity in contemporary educational research. The Methodologically Inclusive Research Synthesis (MIRS) framework, presented in this book, is an attempt to respond to this need.
The Methodologically Inclusive Research Synthesis (MIRS) framework is intended to be a coherent conceptualization of research synthesis methods expressed through the identification of critical decisions and thorough discussion of varied options associated with each decision in the process of a rigorous research synthesis.
The MIRS framework is a synthesis of diverse theoretical and practical considerations to inform the theory and practice of research synthesis methods. It is not an alternative method for research synthesis. Rather, it is a framework developed to support critical reflection among producers and users of research syntheses. In developing this framework, methodological inclusivity has been considered from multiple angles:
ā¢by addressing issues arising from synthesizing methodologically diverse primary research;
ā¢by drawing on ideas and strategies from a variety of research synthesis methods and exemplary research syntheses;
ā¢by adapting ideas and techniques from a wide range of primary research methods, especially qualitative research methods, to the process of a research synthesis;
ā¢by taking into account how processes and politics of research synthesis may intersect with the interests of different stakeholders.
This book does not claim a comprehensive coverage of the possibilities within research syntheses for several reasons. First, contemporary educational research methods and perspectives are changing at such a rapid rate that any claim of comprehensiveness will be outdated by the time the work is published. Second, no single individual can rightfully claim expertise in all forms of research synthesis. Third, I humbly accept my limited experience and knowledge in various primary research methods and research synthesis methods. This book exemplifies a ānon-mastery approachā¦that can tolerate its own failure of knowledge and the detour of not understandingā. This book thus represents my āsituated, partial, and perspectival knowing that, while not knowing everything, does know somethingā (L...
