Integrating Analyses in Mixed Methods Research

Patricia Bazeley

• Publisher: SAGE Publications Ltd
• Year: 2017
• Print ISBN: 9781412961868, 9781412961851
• eBook ISBN: 9781526417169
• Edition: 1
• Topic: Social Sciences
• Subtopic: Social Science Research & Methodology
• Index: Social Sciences

Integrating Analyses in Mixed Methods Research goes beyond mixed methods research design and data collection, providing a pragmatic discussion of the challenges of effectively integrating data to facilitate a more comprehensive and rigorous level of analysis. Showcasing a range of strategies for integrating different sources and forms of data as well as different approaches in analysis, it helps you plan, conduct, and disseminate complex analyses with confidence. Key techniques include:

• Building an integrative framework
• Analysing sequential, complementary and comparative data
• Identifying patterns and contrasts in linked data
• Categorizing, counting, and blending mixed data
• Managing dissonance and divergence
• Transforming analysis into warranted assertions 

With clear steps that can be tailored to any project, this book is perfect for students and researchers undertaking their own mixed methods research. 

Part 1 A Foundation for Integrated Analysis

Mixed methods, as an approach to research, has both a long and a short history. During those histories, researchers across both natural and social sciences have instinctively embraced both numbers and words to understand and communicate their discoveries. They have fought paradigmatic and disciplinary condemnation and wars, there has been marriage and separation. But now a “mixed methods movement” has been birthed, or perhaps “rebirthed”, to the critical acclaim of researchers who seek legitimation of what is really an obvious choice – to use whatever combination of approaches and methods best suits their purposes and best answers their questions. This book focuses on the analytic work being done by those researchers who choose to combine methods of data collection and analysis in an approach now commonly referred to as mixed methods research. This first part of the book helps to prepare readers for mixed methods analysis by reviewing the foundations of the mixed methods movement and of mixing methods in research on the one hand, and on the other, by reviewing the preliminary steps to analysis that need to be considered if methods are to be effectively and productively mixed.

Integration of data sources and analyses during the course of undertaking a research project is key to mixed method analysis. Indeed, coherence in pulling together elements of theory, information gained, and insight is critical in any research project. Integration doesn’t necessarily just happen, however. Also integral to effective, productive, and integrated mixed methods research is thoughtful planning ahead by the researcher, while he or she also remains open and flexible about a design that might need to adapt to unexpected circumstances or interim results. Elements of design that impact on analysis are therefore included in this Part 1 review of preliminary steps to analysis.

Part 1 then concludes with consideration being given to the interpretive nature of analysis and the analytic process. Regardless of the specific methods used, thoughtful judgement must be exercised in assessing the meaning and significance attached to data, to what is learned through manipulating data, and in determining the next step to take in analysis. The claim of those using mixed methods is that by thoughtfully combining methods they will advance a more rounded, more complete, and better developed understanding of the phenomenon being studied than would otherwise occur. It is hoped this will be your experience also.

1 Mixed Methods in Context

Chapter overview

• Why mix methods?
• The emergence of a mixed methods movement
• Defining the mixed methods territory
  • Integration as a core component
  • Terminology
  • Qualitative and quantitative: dimensions of difference
• The challenge of integration
• What does mixing contribute?
  • Failure to integrate
• Philosophical foundations for integration in mixed methods analysis
  • Paradigms and mixed methods
  • Five approaches to managing paradigm issues
  • Seeing common ground
• Concluding remarks
• Further reading

Why mix methods?

In their report to the Mixed Methods International Research Association (MMIRA) on the future of mixed methods, Mertens et al. (2016a: 12) proposed that researchers live and act in a world that is faced with “wicked problems” and “grand challenges”. Wicked problems “involve multiple interacting systems, are replete with social and institutional uncertainties, and [are those] for which there is no certainty about their nature and solutions”, while grand challenges arise from multiple environmental, social, and political sources, or combinations of these. This complexity, evident across many fields of action and inquiry, demands methods able to investigate a problem from multiple viewpoints, with flexibility to adapt to changing situations, yet able to produce credible results convincing to diverse audiences.

Social and behavioural researchers employing mixed methods ask complex questions and engage with complex, real-world environments in a socially responsive and responsible way. They go beyond laboratory-based experimentation in which single variables are manipulated under tightly controlled conditions, or studies that observe and describe some small facet of life or practice. Rather, they flexibly employ a diversity of approaches to embrace the multiple perspectives that behavioural, social, and professional complexities demand. Laurel Richardson’s (2000) image of the multifaceted and ever-changing crystal, as a model of what happens when qualitative methods are used, seems even more appropriate for mixed methods approaches. Mertens et al. (2016b: 222) described mixed methods as kaleidoscopic with its “seemingly unpredictable patterns full of rich possibilities for diversity and potential to provide opportunities to see things that have not yet been seen”. Mixing methods is purposeful, however, rather than random: a “mixed methods way of thinking [intentionally invites] into the same inquiry space multiple ways of seeing and understanding and [engages] respectfully and dialogically with these multiple ways of knowing toward generative insights and better understanding” (Greene, 2012b).

This chapter briefly reviews the development of mixed methods as a recognised movement and approach to social research, key definitions with implications for the way methods are viewed in this book, and approaches to resolving the ontological and epistemological debates that arose when mixed methods began to be explicitly discussed as an approach to research.

The emergence of a mixed methods movement

The combination of multiple methods has an extended history in evaluation and sociological research, and an even longer one in the natural sciences (Maxwell, 2016). Mixed methods thinking and writing can be traced back at least to the eighteenth-century Enlightenment, while paradigm debates can be traced back to the classical Greek scholars, several hundred years BC (Johnson & Gray, 2010). The Philadelphia Negro, by W. E. B. DuBois, a field study published in 1899, may have been the first published report to explicitly integrate qualitative and quantitative methods in an attempt to overcome errors associated with either alone (Maxwell, 2016) – although DuBois did not use the terms qualitative or quantitative in describing his methods. The field studies of the Chicago School sociologists and social psychologists that followed in the early to mid-twentieth century freely combined multiple methods, for example, in studies of community life in America. Despite the dominance of experimentation and statistical testing in the mid-twentieth century, mixed methods continued to be used unproblematically by some sociologists as well as those in psychology and organisation studies during this period. In citing the psychological studies of Festinger, Riecken, and Schachter (1956) and Milgram (1974) as illustrative, Maxwell suggested that the work of these researchers has largely been ignored by more recent mixed methodologists. Their authors focused on the content area of the study and did not explicitly identify their studies as employing mixed methods, rather, presenting them more as quantitative studies, conforming to the trend at the time.

The 1970s and 1980s then heralded the construction and promotion of a paradigm incompatibility thesis among social scientists in which differing philosophical positions concerning the nature of reality and of knowledge construction became linked with methodologies and methods. This was prompted by Kuhn’s (1970) influential writing on the role of paradigmatic shifts in major scientific advances and spurred on by Lincoln and Guba’s (1985) promotion of an alternative constructivist paradigm supporting “naturalistic inquiry” using qualitative methods. Irreconcilable conflict was then presumed to exist between qualitative (inductive, observational, interpretive) and quantitative (deductive, experimental, statistical) approaches to social research. The ensuing debates, often referred to as the “paradigm wars”, drew attention to the practice of combining multiple methods within a single project, but they also had the effect of complicating and setting back the adoption of mixed methods in the social sciences and the professional disciplines, especially in the United States. While most mixed methods researchers have since worked out a satisfactory resolution of these issues that allows them to progress their own work, remnants of these debates still exist in subtle ways that continue to impede integration of methods for many. For example, the creation of a sharply drawn division between qualitative and quantitative methods, with rules built around this division, continues to limit researcher initiative in thinking about problem-focused sources of data and methods of analysis. Limited communication between communities of practice among mixed methods researchers across geographic continents and in different discipline areas, for example between those in management and organisation studies and those in health and education, also impeded shared development of strategies for integrating methods.

Thus, the practice of mixing methods is not at all recent, but the recognition of mixed methods as a third major approach to social science research, or as a methodological movement in addition to qualitative and quantitative approaches, is relatively recent, dating from the later decades of the twentieth century. The consequent burgeoning of theoretical literature and published research studies in handbooks, texts and new journals has both fed and been fed by increasing popularisation of mixed methods as an approach to the broad domain of research in the social and behavioural sciences and in education and health in particular. As mixed methods move into the mainstream, the danger is that some of the “creativity, inventiveness, and risk taking” characteristic of it as a “methodological innovation” will be lost (Fielding, 2012: 125).

Defining the mixed methods territory

Studies are infinitely variable in the way methods are used and combined. To clearly define and set boundaries for mixed methods as an approach to research is difficult if not impossible, and any attempt to do so is bound to be disputed. Johnson, Onwuegbuzie, and Turner (2007) listed and reviewed 19 definitions of mixed methods contributed by then leaders in the field in an attempt to arrive at a composite definition:

Mixed methods research is the type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the purposes of breadth and depth of understanding and corroboration. (Johnson et al., 2007: 123)

This definition covers what is being combined and the purpose of combination, yet agreement is not universal even with this limited formulation. The majority of researchers consulted by Johnson et al. considered that mixed methods necessarily involves a combination of both qualitative and quantitative approaches, but not all did so. Researchers can (and do) include mixes of methods without predefining each as qualitative or quantitative. Others deliberately include reference to combinations that would generally be considered as belonging to the same overall approach (e.g., grounded theory with ethnography or phenomenology; interviews with document analysis, an experiment combined with a questionnaire). Similarly, “combines” is very differently interpreted and practised by different researchers. There is general agreement that the combination of methods in a mixed methods study is designed to contribute insights and understanding beyond that derived from the component parts. Opinions differ, however, as to whether those components are able to make independent contributions as well as contribute to a mixed outcome (Creswell & Tashakkori, 2007), or if defining a study as mixed carries an inherent assumption that at least one of the components will be subservient to and incomplete without the other (Morse & Niehaus, 2009). Yin (2006) argued that for methods to be genuinely integrated rather than parallel, comparability and mixing is needed at all points in a study, from question setting, through design, sampling, and instrumentation, to analytic strategies.

Recently I reviewed a series of articles to evaluate trends in adoption of mixed methods among management researchers (Bazeley, 2015a). The boundary issues become very apparent when one attempts a task such as this. How much qualitative data and how much analysis and reporting of that data are necessary in a predominantly quantitative study to warrant classification as mixed method? Does inclusion of some numeric data and/or reports of counts (e.g., of themes) in a qualitative article make it mixed? When do studies that draw on different sources and types of data (possibly including coded text) to create variables used in a variety of statistical analyses cross the boundary into mixed methods? What about those that draw on varied sources and types of qualitative data using varied approaches to analysis to eventually build an integrated narrative account and/or model of an experience or process? When I had to make a decision about these, a critical consideration was not whether there were sources that could be defined somehow as qualitative and quantitative, but rather, whether a “conversation” between the different sources and/or methods used was evident within the analysis, continuing into the presentation of results and discussion of those results.

Do definitions and boundaries matter? Even the boundary between qualitative and quantitative methods lacks clear delineation. As long as we have a shared idea of the core of what is being discussed, then boundary issues are important only for methodological comparative reviews and for determining where to publish methodological (rather than substantive) articles. Paul Vogt (2008) regards these debates as a distraction from the more important issues of determining what methods will best answer the questions being asked, and of focusing on the phenomenon to be studied. He points to pain as an example of a subjective experience often measured with a scale, which might then be used in answering a categorical question (surgery or not?). “We believe that the quantitative-qualitative argument is essentially unproductive ... quantitative and q...

Table of contents

• Cover
• Half Title
• Publisher Note
• Title Page
• Copyright Page
• Contents
• Illustration List
• Table List
• Sidebar List
• Preface: A Focus on Analysis
• Acknowledgements
• Online Resources
• Permissions
• Part 1 A Foundation for Integrated Analysis
• 1 Mixed Methods in Context
• 2 Planning for Analysis
• 3 Interpreting Data
• Part 2 Integrative Analysis Strategies
• 4 Sequential Integration: Analysis Guiding Design and Further Analysis
• 5 Complementary Analysis of Varied Data Sources
• 6 Analysing Linked Data: Seeking Patterns and Contrasts
• 7 From Codes and Counts to Content Analysis and “Big Data”
• 8 Integration through Data Transformation 1: Qualitative Data to Statistical Variables
• 9 Integration through Data Transformation 2: Exploratory, Blended, and Narrative Approaches
• 10 Inherently Mixed, Hybrid Methods
• 11 Exploring Dissonance and Divergence
• Part 3 Negotiated, Warranted Inferences
• 12 From Integrative Analyses to Warranted Assertions and a Coherent, Negotiated Account
• References
• Index