Managing Quality in Qualitative Research

Uwe Flick, Author

• Publisher: SAGE Publications Ltd
• Year: 2018
• Print ISBN: 9781473912021
• eBook ISBN: 9781526426192
• Edition: 2
• Topic: Social Sciences
• Subtopic: Social Science Research & Methodology
• Index: Social Sciences

Quality underpins the success (or failure) of any piece of qualitative research. In this book, Uwe Flick takes you through the steps in method and design to ensure quality and reliability throughout the entire research process. Showing hands-on what it means to ?manage? quality, this book puts the spotlight on practical questions and steps researchers can use to continually interrogate,  improve and demonstrate quality in your research. 

Chapter One How to Manage and Assess the Quality of Qualitative Research

Contents

• The relevance of quality issues in qualitative research 2
• Internal needs and external challenges 3
• Four levels of asking the question of quality 4
• The problem: how to assess the quality of qualitative research 6
• Ethics and quality of qualitative research 10
• Managing quality in qualitative research – what does that mean? 11
• Structure of the book 12

Chapter objectives

After reading this chapter, you should:

• see the relevance of the issue of quality for the further development and establishment of qualitative research;
• have an impression of the angles and levels at which this issue becomes relevant; and
• have an overview of the following chapters and of how they address the issue.

The relevance of quality issues in qualitative research

Qualitative research has come of age. The growing number of textbooks, journals, other publications and the extension of research practice across several disciplines demonstrate this. Perhaps another indicator of this development, or a need resulting from it, is the current relevance of the question of how to evaluate qualitative research, the plans, the methods and the results obtained with them. The phase of development in which researchers simply trust their methods according to what Glaser suggests: ‘Trust grounded theory, it works, just do it, use it and publish!’ (1998, p. 254), seems to be over. Instead of such (maybe somewhat naïve) optimism, we now find many articles devoted to criteria, to checklists, to standards, to quality, to rigour and evaluation of qualitative research.

In contrast to earlier stages in the development of qualitative research, questions about the quality of qualitative research are no longer raised mainly to demonstrate (from outside) that there is a lack of scientific quality in qualitative research. Rather, this question is increasingly raised from the inside with a ‘how to’ perspective: how to assess or evaluate what we are doing, how to demonstrate quality in qualitative research in an active and self-confident way? How to manage the issue of quality in the qualitative research process has become a topic of major relevance for the further development of qualitative research as a whole. Today it is less the acceptance of qualitative research as such (compared to quantitative research, for example) but the acceptance of specific procedures and results in a single piece of research (for reasons of funding or of publication, for example) that is at stake. Thus, the focus of the discussion about the quality of qualitative research has – not completely, but mainly – shifted from fundamental, epistemological and philosophical levels to more concrete and practical levels of planning, doing and presenting qualitative research.

Internal needs and external challenges

Nevertheless, discussions about the quality of qualitative research are located at the crossroads of internal needs and external challenges. The internal needs arise from the development and proliferation of qualitative research as a field. We find more and more alternatives for how to do qualitative research, a growing variety of methodological alternatives and epistemological and conceptual programmes. Qualitative research can no longer be associated with one or two specific methods. Rather, we find various research programmes with different backgrounds, intentions and strategies for how to do research. Although we can identify common features of qualitative research across the different programmes (see Flick, 2014a, 2014b (Chapters 2, 4 and 7), 2018c, 2018d, or Flick et al., 2004, for suggestions), we can see these programmes as in an internal competition. Thus, we can address issues of quality in qualitative research inside a single research programme: What is good grounded theory research (see Charmaz, 2014; Gibbs, 2018; Flick, 2018b)? What makes it distinct from bad examples? We can also address these issues inside qualitative research from a comparative perspective: What makes a specific grounded theory study a good example of qualitative research? What makes it more appropriate than a discourse analysis (see Rapley, 2018) about the same topic? In both cases, the issue of quality is raised from inside qualitative research and inside qualitative research practice, and thus internally.

External challenges to qualitative research linked to these issues become relevant once qualitative research becomes competitive with other research approaches. This is the case when qualitative research wants to enter fields traditionally dominated by other forms of research. When qualitative researchers in psychology, for example, want to publish their research in peer-reviewed journals, which were traditionally experimentally oriented, the need to demonstrate that the particular piece of research is a good example becomes a challenge. Or when, for example, medical sociologists want to publish a study using qualitative methods in a medical journal, where not only other disciplines but also the ideals of natural science are dominant, the quality issue becomes even more relevant for being convincing.

Another field of competition is funding of research in areas in which traditionally experimental or quantitative researchers are dominant. Here again, issues of quality are raised from outside in assessing the particular grant application, comparing it with grant applications from other backgrounds, or simply to have good arguments to reject proposals.

And finally, teaching and curriculum planning has become a field where qualitative research is in competition with other approaches for resources: What is the role of qualitative research in the curriculum for psychologists or sociologists, and what is its relation to more quantitative or experimental approaches?

In all these fields, the ability of qualitative research to demonstrate that there are criteria, strategies and approaches to distinguish good research from bad research and to enhance and assure the quality of qualitative research is an external challenge. The better qualitative researchers are able to present solutions to this problem, the more successful will they be in establishing themselves in these fields against competitors. As we will see later in this book, the issue of quality in qualitative research is not only a technical problem but also refers to the quality of results and insights from the research, or the study (what’s new in it?).

Four levels of asking the question of quality

Following from what has been said so far, the question of quality in (qualitative) research can be asked at four different levels and by four different groups of actors.

The researchers’ interests in knowing about how good or bad their own research is

Novices to research in particular will have an interest in judging how far they can trust their results and whether they applied their methods in a correct way – and more generally how good their own research is. How can I find out whether the interview I did is a good interview (see Brinkman and Kvale, 2018)? How can I find out how far to trust my findings from this or from my other interviews? And finally, what conclusions can I draw from them in a well-founded way? How can I be sure that insights from these interviews represent what the interviewee thinks or experienced? If I work with other researchers, how can we be sure that each of us proceeds in a similar way, so that the interviews and results are comparable at the level of the interviewees and not only at the level of the differences in the interviewers’ behaviour? In this context, quality criteria or strategies to assess and improve the quality of research will be seen as helpful to reassure oneself and to prepare for evaluation and critique by others (doctoral committees, for example). But also, questions of originality and novelty might puzzle the researchers concerning their study.

Funding institutions’ interest in evaluating what should be or has been supported

The issue of evaluation becomes relevant in the process of funding research at two points. First of all, a proposal has to be evaluated for its consistency and adequacy for what is to be studied and for the quality of the results that can be expected. Second, at the end of the funding, the research or the report has to be evaluated for which of the promises have been kept, how well the research was planned and carried out according to the plan, and in general what came out of the whole study. Here again, comparability becomes a relevant issue: how is the particular (qualitative) proposal comparable with other qualitative proposals, and how can it be compared with proposals coming from other disciplines such as natural or technology sciences? Review processes often pass several levels of committees before a final decision is taken and are mostly based on reviews from several scientists. Qualitative research, with its flexibility and openness in how the research is planned and practised, is often less compatible with such review processes than standardized or experimental research, where the planning is mostly done at the outset of the process.

These are reasons why considerable input to the internal discussion about how to assess the quality of qualitative research has recently come from funding institutions like the ESRC in the UK, the National Institutes of Health in the USA or the German Research Council. Their input has been to ask researchers from different qualitative traditions to set up lists of criteria and checklists allowing them to make judgements and decisions more rational and transparent (see Chapters 3 and 6). The results from such activities have led to a renewed intensification of the academic discussion about quality issues, as many researchers do not see their special approach, or qualitative research in general, represented in such (check-)lists.

Journal editors’ interest in deciding what to publish and what not to publish

A similar trend can be observed in the context of publishing qualitative research. The more the number of peer-reviewed journal articles becomes a general indicator for the scientific quality and merits of particular researchers, research groups, institutes, faculties and even universities, the more qualitative research has to be submitted and peer reviewed, too. Here again, we see the problem of how to make this process rational and transparent for those who submit a paper or are asked for reviews of qualitative papers. Again, we find attempts to set up checklists to evaluate the research that is reported and at the same time to evaluate the way in which the research is reported and made transparent (see Chapter 6). Here, the quality issue is in some way doubled. Consideration of rigour and criteria in the research is seen as essential if the research is to be published. The research in its presentation has to be linked back to existing literature, for example – which is a criterion at the level of presentation.

Readers’ interest in orientation about which research to rely on and which not to rely on

This brings us to the fourth level at which the quality issue becomes relevant. If you read an article about a study with interesting findings, you might like to know how far these findings are based on what was studied and what allows you to trust what you read. This is a more general description of the relevance of the quality issue on the part of the user of qualitative research. In standardized research, reliability, validity and significance tests also have the function of allowing a rather simple and fast credibility check about the study and its results. This will not be transferable to qualitative research (see Chapter 3), but criteria or checklists might be the way to answer the question of credibility. In any case, it would be helpful for the reception of qualitative research if a comparable ‘instrument’ were available and generally accepted here.

The problem: how to assess the quality of qualitative research

So far, I have outlined the relevance of taking up the quality issue in the discussion of qualitative research. In what follows, I will focus more on what the general problems of establishing instruments of quality assurance in qualitative research are like. These problems might be the reason why – unlike in quantitative research – a general canon of criteria has so far not been established and accepted in qualitative research.

Research evaluation based on standardization?

In quantitative and experimental research, we find a close link between evaluation of research and the standardization of research situations. In order to increase the internal and external validity of research and results, interfering conditions are controlled. If the interrelation of two variables is to be tested, the exclusion of interfering variables is a way to guarantee the internal validity of the interrelation that was measured. The exclusion of interfering variables can best be achieved by standardizing the research situation so that no uncontrolled influences may interfere. That is why psychological research ended up mainly in the experimental laboratory, where the chances of having such control are high. External validity means the generalization of results from the research situation (and case) to other situations and cases. Again, excluding interfering variables, for example biases in sampling, warrants this. Therefore, random sampling is applied in quantitative research, because it allows for excluding any biases in the sample. Similar examples could be given for the case of reliability (see also Chapter 2). What these examples show is that in quantitative and experimental research, criteria like validity and reliability are conceptuali...

Table of contents

• Cover
• Half Title
• Publisher Note
• Title Page
• Copyright Page
• Contents
• Sidebar List
• Illustration List
• Table List
• Editorial Introduction by Uwe Flick
• About this Book and its Second Edition
• Chapter One How to Manage and Assess the Quality of Qualitative Research
• Chapter Two Standards in Qualitative Research
• Chapter Three Criteria in Qualitative Research
• Chapter Four Using CAQDAS for Advancing the Quality of Qualitative Research
• Chapter Five Quality Indicators for Specific Methods and Approaches
• Chapter Six Checklists and Guidelines
• Chapter Seven Strategies for Managing Diversity in Qualitative Research
• Chapter Eight Relevance, evidence and quality of qualitative research
• Chapter Nine Quality, Creativity and Ethics Different Ways to Ask the Question
• Chapter Ten Managing Quality in Qualitative Research Process and Transparency
• Glossary
• References
• Index