PART I
THE BIG PICTURE
[ 1 ]
Standards for Qualitative Studies and Reports
JAMES W. DRISKO
Qualitative research continues to be accepted as a valuable approach in social work and in closely allied disciplines. In 1994, the Council on Social Work Education (CSWE) required that qualitative research methods be taught in all accredited bachelorâs and masterâs level social work programs, a requirement renewed in the Educational Policy and Accreditation Standards in 2002 and again in 2008 (CSWE 2002, 2008). Qualitative interest groups began and rapidly expanded within the Society for Social Work Research, the Council on Social Work Education, and through the interdisciplinary International Congress of Qualitative Inquiry. Juried papers and workshops on qualitative research methods and studies became common at social work conferences, if not more frequent in core professional journals. An international journal, Qualitative Social Work, with an emphasis on practice and research started up in 2002, and many new print and electronic journals and resources have been initiated in social work and allied fields.
Despite this growth in interest in qualitative research, there are few U.S. social work texts on the subject and fewer still that cover it in depth. Padgettâs (1998, 2008) text offers a sound introduction to several key methods of qualitative research. In contrast, most American social work textbooks offer a very narrow introduction to qualitative research. The texts fail to portray the range and variety of qualitative research as well as failing to offer much guidance on issues including epistemologies, ethical challenges, sampling, data analysis, and reporting qualitative studies. A few compilations of studies and opinion pieces are also available (Riessman 1994; Sherman and Reid 1994; Shaw and Gould 2002; Padgett 2004), offering a wider portrait of the types of qualitative research social workers undertake. However, little attention is given in these works to establishing standards for qualitative research to guide social work researchers, journal and conference reviewers, and social work educators at all levels. My own standards article (Drisko 1997) set forth a model specific to social work that I have expanded upon in several conference presentations (1999, 2000, 2007). Padgett (1998) identifies several methodological steps that can enhance the rigor of realist qualitative research. Drawing on both our works, Anastas (2004) has addressed standards for qualitative evaluation studies more narrowly. Several conference workshops have been offered on this topic (largely by these three researchers). Nonetheless, no consensus exists on standards for qualitative research in social work.
The lack of standards leads to difficulties at many levels. First, the lack of standards to orient qualitative research makes proposal development more challenging for students and faculty alike. To address this problem, the Group for the Advancement of Doctoral Education has organized discussion of standards for dissertations, many of which included specific expectations for qualitative research. The yield of these discussions has not been widely disseminated, nor widely adopted beyond the doctoral level. The National Institutes of Health, Office of Behavioral and Social Sciences Research (undated), developed a booklet outlining issues in proposal development and review for federally funded qualitative health research. The document, focused largely on ethnographic research, offers an outline of issues to be addressed for federally funded research. These standards, however, are not comprehensive to all the types of qualitative research social workers undertake, nor have they been systematically adopted within social work.
Second, faculty and doctoral students report that their qualitative manuscripts are frequently turned down by social work journals. In many cases, the prospective authors believe their work has not been evaluated using criteria suitable to the content and methods. Instead, they believe qualitative research is often âslammedâ by reviewers using inappropriate standards, most often those used for quantitative, statistical research. As Leininger (1994) notes, such reviewers apply no criterion other than a rejection of qualitative research. While rejection of a manuscript often causes authors to doubt the competency of the reviewers, the consistency of the core messageâreviews applying inappropriate standardsâsuggests a lack of clear standards for assessing qualitative research. Such reviews also indicate the lack of a sufficiently expert pool of reviewers for qualitative research. Given that qualitative research is minimally taught at the masterâs level (Drisko 2008) and is usually taught as a single course at the doctoral level, social work education offers little in the way of preparing future reviewers with deep qualitative expertise. Reviewers with extensive and varied training, research experience, and expertise in qualitative research are few in social work.
Third, the lack of standards combines with the lack of high quality published exemplars to make teaching qualitative research more difficult. Faculty and students are forced to find exemplars of qualitative research in disciplines other than social work. These exemplars may address topics of peripheral interest to the social work profession and they lack a social work perspective. A message is also sent that qualitative research is not highly regarded within the profession, despite the opposite message explicit in the CSWEâs accreditations standards.
Finally, all of these challenges jointly act to undermine the development of faculty expertise in qualitative research within social work. Given the lack of support and training for qualitative research, the cadre of knowledgeable and skilled faculty to train future leaders is small. Few opportunities to develop expertise in qualitative research are available, and very few opportunities to teach the content in depth are available at the bachelorâs and masterâs levels. Clear standards, strong materials and exemplars, and a path to develop talent in qualitative research are all needed.
While the focus of this chapter is on standards for qualitative research in social work, it is important to note that interest in standards is conspicuous in many other disciplines as well. Nursing (Burns 1989), education (Guba 1981; Guba and Lincoln 1981), psychology (Chenail 1997) medicine (Berkwits and Aronowitz 1995; Mays and Pope 2000; Barbour 2001), health sciences (Popay 1995; Popay, Rogers, and Williams 1998; Eakin and Mykhalovskiy 2003), linguistics (Lazarton 2003), and governmental social policy development (Spencer et al. 2003) each have recent publications on criteria for evaluating qualitative studies. Several formulations locate qualitative research as a perspective in contrast to âpositivismâ and/or âscienceâ (Burns 1989; Eakin and Mykhalovskiy 2003). That is, post-positivist and realist qualitative research is ignored or de-emphasized (Smith and Demmes 2000). Since much American qualitative social work research appears realist in epistemology, such standards would not fit well with some of the types of qualitative studies this chapter addresses. In a similar vein, many formulations are criticized as âproceduralistâ or âcriterionistâ: emphasizing checklist-style appraisal of particular procedures over broader, synthetic criteria (Barbour 2001; Eakin and Mykhalovskiy 2003). This chapter, in contrast, assumes an initial substantive appraisal of the research questionâs merits, then identifies a series of choice pointsâeach requiring careful appraisal and judgments about substance, procedure, and internal consistency of the research as a whole. Procedures are relevant as scaffolding, but implementation of them in service of fully meeting the study goals is the focus of evaluation. These criteria are meant to be heuristically useful, not as a checklist or as a straightjacket to creativity and innovation.
The Roles of Data and Interpretation in Qualitative Research
Qualitative research is a diverse terrain, unlike probabilistic quantitative research, which is a single, unified tradition. This diversity makes defining qualitative research challenging. Qualitative research may mean very different things to different people. Tesch (1990) offers a useful map. In describing the types of qualitative data analyses evident in published qualitative research and texts, she proposes a continuum heuristically divided into four sections. First, at the most structured end of the continuum, content analyses apply statistical methods to textual qualitative data. The linkage between the data or evidence to the conclusions is highly structured and mediated by a specified method.1 Somewhat less structured analyses constitute the middle two sections of Teschâs continuum of qualitative research. The second section centers on the discovery of patterns and regularities. In a similar vein, Crabtree and Miller (1999) call these research types template approaches as they often fit data to a priori categories determined before data collection and analysis. Patterns are closely tied to explicit evidence with minimal interpretation. Deductively generated analysis categories are common. Researchers in this section of the continuum apply such formal methods as logical analysis, matrix analysis and narrative analysis (Tesch 1990).
The third section of Teschâs (1990) continuum is yet more interpretive and centers on comprehension of actions and meanings. Crabtree and Miller (1999) label these âeditingâ approaches and note a priori codes and conceptualization are rarely applied. Here researchers emphasize meaning and intention, which may not include a simple one-to-one correspondence between evidence and interpretation, but a clear chain of inference is provided to justify the interpretation. Inductive analysis is emphasized (though occasionally deductive steps may be included). Data analysis is more interpretive but still follows formally described research methods. Grounded theory, other forms of analytic induction, semiotic analysis, hermeneutic analysis, and phenomenology are all examples of such meaning-focused research method (Tesch 1990). Fourth, at the far end of the continuum, Tesch (1990) locates âreflectionâ: Studies that use data as a springboard for interpretation with little to no explicit analytic structure. Crabtree and Miller (1999) call these immersion approaches. Linkage to empirical evidence may be quite loose or selective as the emphasis is on a personalized, situated analysis. Notably, no formal, named, research methods are necessarily applied. Such studies are often highly self-reflective (Eisner 1997). In social work, Hydeâs (1994) reflections on her own organizational qualitative research process are an example of a reflective research report. Similarly, Gilgun (2008) explored her reactions in a reflexive account of working with perpetrators of interpersonal violence. Other examples are found in the journals Qualitative Social Work, Qualitative Inquiry, and Qualitative Research. Within social work such interpretive work is likely to be directed away from traditional research-oriented journals.
Qualitative Research and Science
That some forms of qualitative research are highly interpretive raises the issue of the role of science in qualitative research. American social workâs professional norms have strongly emphasized a scientific approach to knowledge building and testingâas explicitly required by CSWEâs curriculum policy statements and accreditation standards. Though definitions of science vary, the core of science is the role of structured observation to test theories. If some forms of qualitative research (though rare in social work) emphasize interpretation with loose linkage to data, they do not seem to be âscientificâ as understood within social work. Indeed, some interpretive work is based upon a rejection of a scientific core as the optimal means to develop knowledge about lived, subjective experience. Hermeneutic and interpretive approaches are espoused instead, and vary in structure and formality of methods. This tension between scientific naturalism and hermeneutic approaches is a philosophical issue beyond the scope and purpose of this chapter (Hollis 1994). It does suggest, however, that some excellent qualitative research may purposefully not be scientific in order to achieve the goal of incorporating meaning making into the systematic, evidence-based study of people and their activities. On the other hand, other qualitative research may intentionally seek to be part of scientific understanding and evidence based practice (Popay, Rogers, and Williams 1998). Rubin (2008) argues that qualitative research may be a valuable part of evidence-based practice when the practice question centers on understanding the lived experience of persons with a specific difficulty.
While not all qualitative research is necessarily grounded in science, to date the vast majority of American social work qualitative research is so grounded. The problem most apparent in the published qualitative literature in social work is not a rejection of a scientific base, but the failure to produce work of sufficient clarity and rigor to make plain the linkage amongst question, methods, data, and conclusions. My own reviews of the published literature indicate that ethnography and grounded theory are the methods most widely cited by social work qualitative researchers (Drisko 2000a, 2000b, 2003, 2012). Yet Wells (1995) and Oktay (2004) found many deficiencies in the way that grounded theory methods were applied in these studies. Rodwellâs (1998) constructivist social work research model also centers on a clear and highly structured testing of conclusions against systematically collected evidence. More common in publications is a minimal description of purposes and methods, frequently including name-dropping of authors or methodsâwhich close inspection may reveal to be inconsistent or incompatible. For example, a grounded theory approach may be claimed, but realized via Miles and Hubermanâs (1984) quite different, matrix analysis model of data analysis. In American social work publications, problems with conceptual and methodological rigor are currently more apparent in qualitative reports than is a lack of fit with scientific purposes or methods. Rigorous qualitative research may indeed be empirical and scientific.
The hierarchy of research designs promoted by the evidence-based practice (EBP) movement actively devalues qualitative research. While experimental research is certainly a valuable method for testing theory and making cause and effect claims, it is always premised on a complex set of concepts, constructs, and theories that originated in discovery oriented qualitative research. EBP is one valuable form of applied research, but does not displace the need for innovative, basic research to explore and understand new trends and experiences. EBP addresses outcomes and generally promotes methods to enhance internal validity. I...