Advances in Mixed Methods Research

Theories and Applications

Manfred Max Bergman

• Publisher: SAGE Publications Ltd
• Year: 2008
• Print ISBN: 9781412948098, 9781412948081
• eBook ISBN: 9781446241219
• Edition: 1
• Topic: Social Sciences
• Subtopic: Social Science Research & Methodology
• Index: Social Sciences

Advances in Mixed Methods Research provides an essential introduction to the fast-growing field of mixed methods research. Bergman?s book examines the current state of mixed-methods research, exploring exciting new ways of conceptualizing and conducting empirical research in the social and health sciences.

Contributions from the world?s leading experts in qualitative, quantitative, and mixed methods approaches are brought together, clearing the way for a more constructive approach to social research. These contributions cover the main practical and methodological issues and include a number of different visions of what mixed methods research is. The discussion also covers the use of mixed methods in a diverse range of fields, including sociology, education, politics, psychology, computational science and methodology.

This book represents an important contribution to the ongoing debate surrounding the use of mixed methods in the social sciences and health research, and presents a convincing argument that the conventional, paradigmatic view of qualitative and quantitative research is outdated and in need of replacement. It will be essential reading for anyone actively engaged in qualitative, quantitative and mixed methods research and for students of social research methods.

Manfred Max Bergman is Chair of Methodology and Political Sociology at the University of Basel.

Part I

THE THEORY OF MIXED METHODS DESIGN

1

The Straw Men of the Qualitative-Quantitative Divide and their Influence on Mixed Methods Research¹

Manfred Max Bergman

Qualitative researchers stress the socially constructed nature of reality, the intimate relationship between the researcher and what is studied, and the situational constraints that shape inquiry. Such researchers emphasize the value-laden nature of inquiry. They seek answers to questions that stress how social experience is created and given meaning. In contrast, quantitative studies emphasize the measurement and analysis of causal relationships between variables, not processes. Inquiry is purported to be within a value-free framework.

(Denzin and Lincoln, 1998: 8).

Quantitative research is based on observations that are converted into discrete units that can be compared to other units by using statistical analysis…. Qualitative research, on the other hand, generally examines people’s worlds and actions in narrative or descriptive ways more closely representing the situation as experienced by the participants… . These two paradigms are based on two different and competing ways of understanding the world… [which] are related in the way research data is collected (words versus numbers) and the perspective of the researcher (perspectival versus objective) [and] discovery versus proof

(Maykut and Morehouse, 1994: 2–3).

In social research, examples of methodologies are positivism (which seeks to discover laws using quantitative methods) and, of course, qualitative methodology (which is often concerned with inducing hypotheses from field research)

(Silverman, 1993: 2).

Introduction

Mixed method research design is one of the fastest growing areas in research methodology today. Its aims and benefits appear rather simple: take the best of qualitative (QL) and quantitative (QN) methods and combine them. However, many debates on mixed method research design are based on methodological arguments that, upon closer inspection, are difficult, if not impossible, to sustain. This is due in part to the way in which, particularly from the early 1980s, qualitative research methods were explicitly associated with constructivism,² while quantitative methods continued to be linked with positivism.³ This state of affairs led to the ‘Paradigm Wars’ and the ‘Incompatibility Thesis’⁴ (see Lincoln and Guba, 1985), which presents one of the hurdles to be overcome in order to make mixed methods designs ontologically and epistemologically viable. Many texts on mixed methods research tangentially touch upon this point but subsequently urge the researcher to be pragmatic (e.g. Creswell, 2003; Creswell and Plano Clark, 2007; Tashakkori and Teddlie, 1998; 2003). But if the differences between QL and QN methods are considered in detail, pragmatism is difficult to apply as an antidote to incompatibility. Maxcy (2003: 86; see also Maxcy, 1995), for example, suggests that

[p]ragmatism is not interested in explanations of anomalous cause-effect cases as in the ways in which practical intelligence may push toward full and free settlement of chaos and discord. Pragmatic-oriented social-behavioral researchers join hands with rationalists as they seek better reasons for educational policies and arguments. On the other side, they link with empiricists who support a ‘real world’ and some matters as ‘given’. Their unique contribution is to open up inquiry to all possibilities while tying that search to practical ends.

As a justification for mixing different elements within a mixed methods framework, such advice is vague and methodologically unsatisfactory. Thus, many researchers heeding such advice may incorrectly interpret pragmatism to mean that ‘anything goes,’ while others may wonder whether methodological concerns ought to preoccupy the researcher at all.

However, there are more elegant and consistent ways to explain apparent contradictions without having to gloss over some of the central ideas in research methodology. But even if this hurdle has been successfully overcome, it remains unclear why and how methods should be mixed. Will a mixed method design get us closer to objectivity? Should we mix different types of data or should we mix different findings? Is it possible to mix different theoretical approaches, or is it even possible to mix epistemologies? Considering these issues, one wonders whether mixing methods is indeed an improvement over mono method designs.

My aims in this chapter are to outline some of these issues in mixed method research design and then to transcend the limits of the current debates by, first, revealing inconsistencies in the literature and, second, presenting an alternative and, in my opinion, a more coherent approach to mixed methods research design. I will argue that the growth and exploitation of this fascinating research design has been hampered considerably by the ways in which QL and QN research methods have been ontologically, epistemologically, and habitually constrained by contemporary theory and application. A careful re-examination of the possibilities and limits will reveal that research design possibilities are far richer than expected, thus opening up the research process to a wealth of many new possibilities in relation to data collection and data analysis techniques. Answers to questions about why and how to mix methods will be more in line with contemporary practices. Serendipitously, this will also reveal both new and prematurely abandoned possibilities in applying mono method research designs.

Conventional divisions of labor between qualitative and quantitative methods

It is difficult to identify the origins of the idea that QL and QN methods represent fundamentally different approaches to the research process,⁵ leading to the suggestion that they are best understood as separate, Kuhn-inspired⁶ ‘paradigms’ (Guba and Lincoln, 1994). What can be observed in this regard, however, is that the focus on fundamental differences between QL and QN research methods has reached its zenith in the late 1980s and 1990s with the publication of an entire battery of influential texts (e.g. Brewer and Hunter, 1989; Danziger, 1990; Denzin and Lincoln, 1994; 1998; Flick, 1998; Lincoln and Guba, 1985; Maykut and Morehouse, 1994; Reichhardt and Rallis, 1994; Silverman, 1993; 1997; 1999). While written for different purposes and pursuing different lines of argumentation, e.g. what the differences are between QL and QN methods and whether they are compatible with each other, these texts structure our ‘there-are-two-kinds-of-research-methods’ perspective today and, as will be argued in this chapter, hamper a more systematic and theoretically grounded application of mixed methods design as a consequence.

Based on the heritage of these texts, numerous qualities are habitually attributed to qualitative research:

• A belief in a constructed reality, multiple (constructed) realities, or a nonexistent reality.
• An interdependence between the knower and the known, i.e. the impossibility to separate the researcher from the research subject.
• The inadvertent value-ladenness of the research process and its output, i.e. the impossibility to conduct research and interpret research findings objectively.
• The centrality of the context to the research process and findings, e.g. time-space, politics, specific situation during data production, interpretation, presentation, etc.
• The impossibility to generalize research findings beyond the limits of the immediate context.
• The impossibility to distinguish between causes and effects.
• The explicit focus on inductive, exploratory research approaches.
• The tendency to work with small, non-representative samples.
• The belief that research in this vein is or should be non-reductionistic, i.e. the belief in the ability to describe or explain in its entirety the complexity of phenomena under investigation.

In stark contrast to these qualities, the attributes of quantitative research ostensibly include:

• A belief in a single reality.
• The possibility and necessity of separating the knower from the known.
• The possibility and necessity of value-free research.
• The possibility to generalize findings beyond the contextual limits of the researched units and research situation.
• The pursuit of identifying universal, causal laws.
• The tendency to work with large, representative samples.
• An emphasis on deductive research via falsifiable hypotheses and formal hypothesis testing.

Drawing together the major distinctions between QL and QN methods, one has to wonder why these attributes are so diametrically oppositional, considering their shared subject space. In other words, should we not become suspicious by such clear and clean distinctions, especially if we reflect on the complex, messy, and compromiseladen research process itself? Examining these two lists of attributes, one ought to wonder whether the content of the lists is the result of an attempt at a truce between two factions, rather than a demarcation between two kinds of approaches. And if it is indeed a negotiated settlement between stakeholders, rather than a representation of the actual limits of the two approaches, what do QL and QN methods loose as a consequence and how does this settlement affect the possibilities and limits of mixed methods research design?

The precarious complicity of theories of mixed methods research design

Variants of such lists aiming to differentiate the so-called paradigms can be found in a number of influential books on mixed methods design or books that aim at differentiating QL and QN methods (e.g. Bryman, 1988; 2001; Creswell, 2003; 2007; Fielding and Fielding, 1986; Mertens, 2004; Tashakkori and Teddlie, 1998; 2003). They tend to reproduce previously published lists, whose elements are often categorized according to ontological, epistemological, and axiological concerns (e.g. Crotty, 1998; Denzin and Lincoln, 1994; Lincoln and Guba, 1985). As such, most theorists and practitioners concerned with mixed methods research design not only take the division of labor between the two paradigms as a given, but, more importantly, they build this division into their main raison d’être: whichever the prevailing definition and whichever variant is applied, mixed methods research designs are justified primarily by supposedly exploiting the strengths of each paradigm and by combining the respective strengths within one single research design.

Considering the qualities attributed to QL and QN methods more closely, however, it becomes clear how incompatible the paradigms seem to be. This is not surprising, given the main line of argumentation of its originators, i.e. an emphasis on the incompatibility thesis and the thus emergent paradigm wars (e.g. Denzin and Lincoln, 1994; Lincoln and Guba, 1985; 2000; Silverman, 1993; 1997; 1999). Accordingly, theorists and researchers engaging in mixed methods research design have to maintain a strangely schizophrenic position toward the division of labor between QL and QN methods: on the one hand, they must accept and emphasize the divergent qualities attributed to each paradigm, which, on ontological, epistemological, and axiological grounds, are clearly incompatible; on the other hand, they put forward the proposal that the strength of each paradigm can be combined fruitfully within one single research design. These two positions are irreconcilable with each other, and the fault lies not with the principle ideas behind mixed methods design but rather with its false premises, i.e. the strategic or naïve attribution of two distinct sets of qualities to the two large families of methods. Instead, QL and QN methods represent large and heterogeneous families of methods under convenient headings. The members of these two families vary tremendously within their own family to such an extent that it is difficult to identify a unique set of qualities that encompasses the characteristics of one family of methods, and that is clearly distinctive from the characteristics of the members of the other family. Most characteristics encompass either only a subgroup of members of the family or are also applicable to some members of the other family. Examining the variety of data collection and data analysis techniques habitually subsumed under the QL and QN methods label, it should be asked what use such labels have and why they have established themselves so strongly and detrimentally in narratives on methodology and methods application. I argue that we must rethink the division of labor between QL and QN methods in order to better understand the possibilities and functions of methods more generally, and to better justify and apply mixed methods design specifically.

Reconceptualizing the domains of qualitative and quantitative methods

Let us return to some of the items listed earlier, which ostensibly divide the two paradigms on ontological, epistemological, and axiological grounds (see Lincoln and Guba, 1985 for a good representative of this line of argumentation):

It is often claimed that QL research is based on the assumption that reality is either constructed or does not exist, while QN research supposedly assumes the existence of one single reality. In practice, however, this ontological proposition is often inconsistent with research applications. For example, Gilligan (1982) came to question Kohlberg’s stage theory of moral development, justice, and rights (e.g. 1969). According to her own hunches, she conducted a set of qualitative interviews with young men and women from a non-representative sample to explore alternative theories and explanations. While it is not possible to generalize such findings to a wider populations (e.g. men and women in general), Gilligan nevertheless was able to formulate an alternative explanation to a stage theory of moral development based on her interviews. There are plenty of other examples from interview and focus group research, which aim at revealing reported thoughts, behaviors, social processes, etc. In a similar vein, modeling identity constructions statistically based on sets of responses from survey data does not necessarily mean that true identities are at the base of such survey responses. Accordingly, it is methodologically acceptable to claim that emergent identity structures are based on the co-construction between the researchers’ selection and understanding of items in a questionnaire, their choice of analytic strategy, and their interpretation of the statistical output on the one hand, and the way the respondents interpreted the survey questions within the given social, political, historical, and economic context on the other hand. In contrast, it is equally acceptable to report that, based on exploratory interviews, most women who took part in a study and who are physically abused by their husbands report that the wellbeing of their children is the main reason for not leaving their partners. Here is another example against the claim that QL research must automatically reject the belief in a single reality: textual material (e.g. documents, interview transcripts, etc.) can be analyzed with regard to what was said by whom and in what particular context. The same material may be studied from a discourse or narrative analysis perspective, exploring, for instance discursive registers and strategies, agency, plots and subplots, meaning structures, etc. From a methodological perspective, it does not make sense to declare one approach more or less valid or valuable, scientific, etc. Instead, how to understand and analyze data must be based to a large extent on the consistency formed between how to understand data in conjunction with the specific research question, rationale, aims, etc. Only in connection with the specificities of the research goals does it make sense to delimit the nature of reality. Thus, in the context of their research undertaking, researchers decide (usually without being aware of it), which truth claims they make in relation to their data and findings. In other words, the research focus may well delineate ontological and epistemological constraints. A Foucauldian discourse analysis demands an interpretive approach to the research but it does not change the nature of the textual data. The same textual data can be used for other, ontologically and epistemologically different, positions, which are derived fr...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Contents
• Introduction: Whither Mixed Methods?
• Part I The Theory of Mixed Methods Design
• Part II Applications in Mixed Methods Design
• Index