Qualitative Methods and Health Policy Research

Elizabeth Murphy

• Publisher: Routledge
• Year: 2017
• eBook ISBN: 9781351495271
• Topic: Medicina
• Subtopic: Teoria, pratica e riferimenti medici

Qualitative researchers have traditionally been cautious about claiming that their work was scientific. The "right-on" schools have exaggerated this caution into an outright rejection of science as a model for their work. Science is, for them, outmoded; "an archaic form of consciousness surviving for a while yet in a degraded form" (Tyler 1986:200). Scientists' assertions that they are in pursuit of truth simply camouflage their own lust for power. There is no essential difference between truth and propaganda.The authors acknowledge that the boundary between science and propaganda has often been breached and some distrust of scientific claims may be healthy. They also question the claim that science creates disinterested and objective knowledge of an observer-independent world without concluding that science is impossible. The skeptics' reservations about qualitative research are based on the deep-rooted assumption among natural scientists, and some social scientists, that there is a world "out there," prior to, and independent of, their observations. This world can be known objectively in the sense that all observers will, if identically placed, see it in exactly the same way. If a suitable language were available, they would also all produce identical descriptions. From these observations they can work out the laws governing the world's operations. The authors try to resolve these contrary claims by asserting that science is a procedural commitment. It consists of openness to refutation, a conscientious and systematic search for contradictory evidence, and a readiness to subject one's preconceptions to critical examination. The devotion to truth as a regulative ideal is an essential difference between science and propaganda. This work is a unique and innovative defense of scientific method.

1

Qualitative Research and
Policy Science

What are your motives for reading this book? Are you a skeptic who believes that the only valid and reliable knowledge is that which comes in quantitative forms, and who wants to know why there is so much fuss about qualitative methods? Are you already a convert who believes that qualitative research is a way to bring romance back into a world that has been dulled by number crunching? Or are you an agnostic, who just wants to know whether there might be anything in this stuff that could be useful to you, your organization, and your patients? This chapter and the next are directed primarily at the first two readers. For the skeptic, they explain that qualitative research can be done in ways that are precise, rigorous, and scientific. For the romantic, they explain why many qualitative researchers have been reasserting the virtues of precision, rigor, and science against the recent fashion for subjectivity, empathy, and emotional politics. In the process, however, the agnostic will learn how we come to adopt the subtle realist foundations that underpin the remainder of this book.

Skeptical consumers frequently describe qualitative research with words like “soft,” “impressionistic,” “ideological,” and “anecdotal.” In context, these usually amount to a charge that the work is not scientific, as the skeptic understands that word. If qualitative research is not science, then it cannot contribute to a sound evidence base in health care policy and practice. Given this, it has nothing to offer to busy men and women concerned with the important practical issues of health service design, organization, and delivery. We disagree. We think that qualitative research can be done in a scientific fashion with rigor and precision. The means by which these are achieved may be unfamiliar to the skeptic but the objectives are identical.

As we set out a subtle realist program, however, we are conscious that this contradicts many features of some contemporary qualitative research that attract romantic consumers. They think this approach provides an element of color and humanity that has been eliminated by what they regard as the straitjacket of quantitative research. In their vocabulary, quantitative methods are “scientistic,” “positivistic,” “malestream,” “artificial,” “crushing of meaning,” and so on. Their enthusiasm is fueled by sections of the qualitative research community. For example, the editors of the influential Handbook of Qualitative Research claim, in the latest edition, that the history of qualitative research in North America can be divided into seven phases or “moments.” The current, sixth, moment is one in which “fictional ethnographies, ethnographic poetry, and multimedia texts are today taken for granted” (Denzin & Lincoln 2000:17). The barriers between scientific and other forms of writing, including journalism, fiction, and poetry, are being broken down (Ellis & Bochner 1996). Researchers are openly committed to “ideological research” that will contribute to the overthrow of patriarchy, neocolonialism, or global capitalism (Lather 1986). Qualitative research is to be understood as a “moral, allegorical, and therapeutic project” within which “the researcher’s story is written as a prop, a pillar that … will help men and women endure and prevail in the dawning years of the 21st century” (Denzin & Lincoln 2000:xvi). These approaches are claimed to be as acceptable as more traditional ones: “There can be no question that the legitimacy of postmodern paradigms is well established and at least equal to the legitimacy of received and conventional paradigms” (Lincoln & Guba 2000:164).

In the context of this romantic turn, the skeptics’ reaction to qualitative research is understandable, and we share a great deal of it. However, it makes our task more difficult, in that we must explain both why we think that qualitative research can offer useful knowledge for policy and practice and why we do not think that the search for alternative standards from the humanities is helpful. In the words of one early critic of this turn, we think that it is more important to be “right” than to be “right-on” (Strong 1988). This chapter looks at three issues where the pressure from romantics gives skeptics most cause for concern:

• Is qualitative research science?
• Can qualitative research reports be distinguished from journalism or fiction?
• Is qualitative research driven by a political agenda rather than by a quest for useful knowledge?

We shall show that realist qualitative researchers need not abandon a commitment to science or the search for authoritative knowledge. We do not believe that the dissolution of the boundaries between scientific and other kinds of writing is helpful. Finally, we believe that seeing research as primarily a political project confuses the roles of knowledge producer and activist in ways that are unhelpful and that undermine research’s potential contribution to practical social change.

Qualitative Research as Science

Qualitative researchers have traditionally been cautious about claiming that their work was scientific. The “right-on” schools have exaggerated this caution into an outright rejection of science as a model for their work. Science is, for them, outmoded, “an archaic form of consciousness surviving for a while yet in a degraded form” (Tyler 1986:200). Scientists’ assertions that they are in pursuit of truth simply camouflage their own lust for power. There is no essential difference between truth and propaganda. “Truth games” are represented as a form of terrorism (Rosenau 1992). The result is sometimes described as a crisis of legitimation, that there is no form of knowledge that is not arbitrary, subjective, and biased by interests (Lincoln & Guba 2000).

The boundary between science and propaganda has often been breached (Fay 1996) and some distrust of scientific claims may certainly be healthy (Sanders 1995). However, just because we can find examples of propaganda masquerading as science and of science being exploited as propaganda, it does not follow that propaganda and science are synonymous. Similarly, we can question the claim that science creates disinterested and objective knowledge of an observer-independent world without concluding that science is impossible.

The skeptics’ reservations about qualitative research are usually based on the deep-rooted assumption among natural scientists, and some social scientists, that there is a world “out there,” prior to, and independent of, their observations. This world can be known objectively in the sense that all observers will, if identically placed, see it in exactly the same way. If a suitable language were available, they would also all produce identical descriptions. From these observations they can work out the laws governing the world’s operations. Truth is simply a matter of correct description. Particular observations and statements of laws might contain errors but these will be corrected by further data or better observational techniques. Neither observations nor laws have any moral or ethical implication: they simply describe what is. The consumer of science is, ideally, presented with a structured set of facts and the laws that describe the relationship between these facts. If A is necessary for B and C to happen, then preventing A from happening will eliminate B and C.

If we take medicine as an example, this model would lead to the view that human diseases have always and everywhere been the same, at least once established in the species. If they have been described and classified in different ways by different cultures or at different historical periods, this is because there has been insufficient systematic data to establish their true nature. As this is collected, universally valid descriptions will be produced. In principle, then, any doctor seeing a patient with a particular disease will, if not today then at some foreseeable point in the future, see the same disease and describe it in the same terms. From being a “catchall” term for uncontrolled cellular growths, for instance, many specialists now talk of “cancer” as the aggregate of possibly several hundred different diseases, whose individual characters are gradually being captured and defined. We may look back in a hundred years’ time, as we now look back to the nineteenth-century disease of chlorosis, to a disease category that was once widely used and is now extinct (Figlio 1978). However, this is a sign of progress, of a closer approximation to the truth of disease replacing our current errors. Whatever names we give to these new disease categories, they have always been there: it is simply that we can now see them properly.

Although this approach has resulted in many valuable contributions to the welfare of humankind, it has never gone unquestioned. Ever since philosophers began debating the nature of science more than two thousand years ago, there has been a competing view that the world “out there” is shaped and organized, if not actually created, by the perceptions of the observer. In consequence, claims to know that world objectively must be treated with caution. Knowledge always rests on some point of view—on some mixture of the observer’s prior knowledge, experience, values, and motives with their biological and technological capacities. All facts are artifacts, products of the processes by which we decide what might be important to notice and record and of the concepts that frame those processes. In Fay’s words, “Descriptions always take place within a framework which provides the conceptual resources in and through which reality (or events and objects in it) is described” (1996:74). This framework is not, however, purely subjective. At the most basic level, one of the ways in which we constantly affirm our sanity is by seeing the world in the way expected of ordinary members of the social groups to which we belong (Goffman 1983). If we see a fuzzy road sign, we know that our vision is at fault rather than the sign (Pollner 1975). As scholars, we usually show our competence by demonstrating that we see the world in the way that people with our particular training and status would be expected to.

Natural scientists, then, have to learn a specific way of seeing the world in order to be accepted as competent in their field. This is enforced by the social processes of recruitment, organization, and control within the scientific community. If disease categories, such as “cancer,” vary over time, this does not show a progressive approximation to the essential truth of nature but the consequences of changes within the scientific community. Different generations use different investigative technologies and different classificatory criteria, associated with different therapies and different goals. The underlying biological structures and processes are seen through different frames, giving them a different appearance. The change from one frame to another over time is rarely a matter of truth correcting error but rather of changing ideas about what would count as truth and error. There might, indeed, be a real world out there somewhere: we can, however, only know it through a process that is subject to both social and psychological influences. The results can amalgamate statements of fact and statements of value. When we say that “X is a disease,” for example, we are not just describing X but also communicating a value judgment about X, that it is undesirable (Dingwall 2001). Conversely, a negative evaluation of X might lead to its, apparently factual, classification as a disease. Think, for example, of the long-running debate within the American Psychiatric Association about whether or not homosexuality should be defined as a disease that the profession should seek to “cure” (Bayer 1987).

Some qualitative researchers have gone on from this to conclude that they should give up any claim to be doing science and adopt some form of relativism (Ellis & Flaherty 1992; Lather 1993). Relativists assert that we decide what counts as “real” only through the linguistic and cultural resources of the groups to which we belong, which frame our interaction with the world (Fay 1996). Consequently, it is possible for many different realities to exist or even for there to be as many realities as there are persons (Smith 1984:386). Individual realities may contradict one another and yet still be equally true for those operating within them. We cannot test such realities against “objective facts” since “facts” are themselves produced by reference to conceptual frameworks. In a discussion of witchcraft and psychotherapy, Fay (1996) illustrates the difficulties that result. Relativists cannot distinguish between psychotherapy and witchcraft as means of dealing with strange behavior. The prior decision, whether to believe in witchcraft or in psychotherapy, shapes the very perception of what behavior will be counted as strange and how it can properly be explained. Claims about the world are only true, if the idea of truth has any meaning at all, within the frameworks adopted by those who make the claims. In that sense, all claims to truth are arbitrary.

The relativist position denies that there is any independent basis on which we can choose between different conceptual frameworks or the realities they produce. There is no possibility of a “God’s eye point of view” (Smith 1985). Standards of judgment are internal to particular conceptual schemes, so they cannot be applied across them. There is no way to evaluate the adequacy of one explanation or description against another. Relativists turn, instead, to moral, ethical, or political criteria. Truth claims rest on moral superiority or political expediency, on being “right-on” rather than being “right.” Research illustrates or justifies a prior position, which is itself placed beyond question. For romantic consumers of qualitative research, this is part of its attraction, that it can sustain what they already believe.

For skeptical consumers, however, such relativism further undermines the usefulness of qualitative research for practice (Greene 1996; Sanders 1995). If researchers’ only possible output is one more story, one more reality among an indefinite number of possible realities, what good are they? Why should they expect financial support in competition with novelists, poets, or artists (Strong 1983)? The public funding of research and scholarship rests on an implicit contract to produce knowledge that is in some sense relevant to the goals and values of a society (Hammersley 1995). Relativism undermines the foundations of that contract.

The relativists’ conclusion can be criticized in a number of ways. First, it is self-refuting. If the claim that all truths are relative is true, then this claim itself must be relative. The claim can only be true in terms of a particular set of assumptions that others may judge to be false. Second, it certainly underestimates the extent to which reality has a way of resisting our constructions. The world we observe has the crucial ability to “talk back” (Dawson & Prus 1995). While it may be true that any observation is irreducibly an interpretation of the world, it is not true that the world will bear any interpretation we care to put upon it. Garfinkel (2002:173–5) has recently characterized this as “natural accountability,” the challenge to produce descriptions that are above all disciplined by the local particulars of the “shop floor,” the material and cognitive environment in which real things happen. “The obdurate character of the empirical world” (Blumer 1969:22) can challenge our conceptual frameworks. Would you want to fly straight and level at five thousand feet from Denver to San Francisco with a pilot who thought the Rockies were a social construct? Even a postmodernist cannot play football with a broken leg. Third, it creates an implausible model of social organization. It leads to the claim that different people inhabit different and incommensurable worlds with no possibility of meaningful communication between them. If this were correct, human social interaction would be literally impossible, since there would be no common reference points.

Relativism is not the only possible response to the loose coupling between the world and our understanding of it. An alternative, which we argue is more appropriate for policy science, is what Hammersley (1992a) has called “subtle realism.” This acknowledges that researchers are constrained by the prior frames that they bring to their observations (Hammersley & Atkinson 1995). The observer’s knowledge is, however, always “a joint product of the referent and the cultural-biological lenses through which it [the phenomenon under study] is seen” (Campbell 1994:157, emphasis added). The subtle realist accepts that a world exists independently of its observers and constrains the observations that can be made. At the same time those observations are also constrained by the “cultural-biological lens” through which they are made.

Subtle realists accept that everything can be represented from a range of different perspectives, through different “cultural-biological lenses.” Several representations may coexist and be potentially true. Unlike the relativist, however, the subtle realist does not assume that all these representations are equally valid. Judgments can be made about their truth or falsity. We may never know with absolute certainty that a particular knowledge claim is true (Hammersley 1993). Nevertheless, claims can be rigorously tested and evaluated. We can make a judgment about whether they are adequately supported by evidence and argument. Dewey referred to this as “warranted assertability” (1938:7), while Phillips talks about “truth as a regulative ideal” (1987:23).

Science, in this view, is a procedural commitment. In practice, it consists of openness to refutation, a conscientious and systematic search for contradictory evidence, and a readiness to subject one’s preconceptions to critical examination. The devotion to truth as a regulative ideal is an essential difference between science and propaganda. Through its natural accountability, science is always capable of being changed by inconvenient data. Propaganda merely seeks to ignore, incorporate or explain away contradictory evidence. As such, objectivity is above all an attitude or “a state of mind,” which can characterize any kind of research. Qualitative research regulated by an ideal of truth should be capable of satisfying skeptical consumers that it meets their basic tests of science, even if the specific means adopted are unfamiliar. The next chapter will describe some of the general characteristics of the procedural commitments that we advocate for qualitative research.

Qualitative Research and Forms of Writing

Skeptical consumers frequently charge qualitative research reports with being indistinguishable from forms of writing like journalism and fiction. Ironically, many contemporary qualitative researchers would take this as praise rather than as criticism. As we have seen, they reject conventional forms of scientific writing as part of their program to break down the boundaries between science and the humanities (Ellis & Bochner 1996; Richardson 1988, 1992). These researchers have turned to alternative forms of writing in an attempt to escape the rhetoric, epistemology, and politics of conventional research reporting. Textual innovations include poetry (Austin 1996; Richardson 1992; Tillmann-Healy 1996), collage (Clifford 1981), personal narratives (Ronai 1992, 1996; Tillman-Healy 1996; Ellis 1996; Kolker 1996), dramatic presentations and constructed dialogues (BluebondLanger 1980; Ellis & Bochner 1992; Mienczakowski & Morgan 1993; Paget 1990), and polyvocal texts (Fox 1996). These experiments respond to what is described as the crisis of representation, because it arises from self-conscious questioning of what counts as an adequate representation of reality (Richardson 1988).

The advocates of these alternative writing forms are dissatisfied with conventional research reports on three grounds. The first is aesthetic: conventional reports are accused of being “dreary” (Richardson 1992), “formulaic” (Richardson 1988), or “boring, esoteric and parochial” (Ellis & Bochner 1996). In particular, they focus on the cognitive at the expense of the emotional (Ellis & Bochner 1996...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Contents
• Introduction
• I THE CONTRIBUTION OF QUALITATIVE RESEARCH
• 1 Qualitative Research and Policy Science
• 2 Three Myths about Qualitative Research
• 3 So What Is Different about Qualitative Research?
• 2 The Practice of Qualitative Research
• 4 Observation, Interaction Analysis, and Documents
• 5 Interviews in Qualitative Research
• 6 Selection and Sampling in Qualitative Research
• 7 The Analysis of Qualitative Data
• 8 The Ethics of Qualitative Research
• 3 Evaluating Qualitative Research
• 9 Judging the Quality of Qualitative Research
• Envoi
• References
• Index