Analyzing and Reporting Focus Group Results

Richard A. Krueger

• Publisher: SAGE Publications, Inc
• Year: 1997
• Print ISBN: 9780761908166
• eBook ISBN: 9781506339177
• Edition: 1
• Topic: Social Sciences
• Subtopic: Social Science Research & Methodology
• Index: Social Sciences

Richard Krueger offers a rich and valuable discussion of focus group analysis that is sure to become a major guide in future focus group efforts. Because analyzing focus group data is different from analyzing data collected through other qualitative methodologies, it presents new challenges to researchers. Analyzing and Reporting Focus Group Results offers an overview of important principles guiding focus group research and suggests a systematic and verifiable analysis strategy. Krueger is not doctrinaire, but instead offers multiple approaches and invites others to share their strategies for analysis. The book is helpful for academic audiences, focus group practitioners, and the occasional moderator. And the straightforward approach contains hundreds of helpful tips. The reader of this volume is bound to find delightful strategies that will improve analysis.

Part I

The Foundation

---

Commit yourself to the highest quality of analysis possible with the resources you have available. The analysis is a reflection of your ability as a focus group researcher. Begin analysis with the same level of dedication and seriousness as if you were the decision maker or client. If you were making the decision, shaping the policy, or designing the strategy, what would you invest in the effort to make it high quality?

During World War II, the U.S. government discovered that its parachutes failed to open 5 percent of the time. Clearly nothing less than zero defects was an acceptable level of quality. How do you tell paratroopers going on a mission that one out of twenty of the parachutes is not going to open? The problem was solved by requiring parachute packers and inspectors to put on one of their products occasionally and jump out of a plane. Parachute quality quickly improved from 95 percent to 100 percent. (LeBoeuf, 1985, p. 75)

Assume you are jumping out into the unknown, and only the results of your analysis will save you. Invest the amount of effort needed to be confident in your results.

In Part I, we will explore the foundation of focus group analysis. This foundation is critical, because it sets the stage for the adapting, revising, and modifying that occur in practice. The principles and approaches contained in the following chapters serve as a baseline to the analyst. The analyst can veer from these principles, but changes should be thoughtful, deliberate, and documented.

VOLUME 6

---

1

Introduction to Analysis

---

Overview

The Concept of Analysis

What Makes Qualitative Analysis Complex?

In qualitative research, we are open to different ways of seeing the world. We make few assumptions about how things work, and we are careful about attributing causation. We strive to be open to the reality of others. We seek to tell someone else’s story, but we must listen before we can understand. Analysis begins with careful listening.

We’ve been told that the scientific method is built on precision, careful formulas, and control over the environment. Yet, in qualitative inquiry, the researcher must begin at a different point with fewer assumptions and an openness to alternatives. This is disquieting for some researchers.

For some, analysis of focus groups comes easily. Others have great difficulty separating their personal views from what is said by participants, they key on trivial points, or they forget that they are seeking the needle and instead give us the haystack. We’ve seen teens conduct top-quality analysis, and we’ve seen Ph.D.s struggle and miss the mark. Education helps, but education is not the solution. Openness to new ideas, approaches, and concepts is essential.

Key characteristics of focus group analysis include a disciplined process, systematic steps, a defined protocol, verifiable results, and multiple feedback loops. The analyst must be ready to explain the analysis process. Failure to do so fosters mistrust and doubt.

---

Consider the Power of a Label

---

As you seek to describe your analysis strategy, consider calling it by name. Use a label or title to describe your strategy Labels used in current practice are often preferable to newly coined words. The label by itself can be influential. For example, successful healers have found that it is vitally important that a name be given to identify the disease, ailment, or condition. Without a name, the patient feels a sense of hopelessness and abandonment. The name conveys a sense of classification, of meaning, and, for the patient, a sense of hope. In a similar way, it is important for the qualitative analyst to have a name for the process used. Names are derived from authors, publications, or established concepts such as those described by Tesch (1990) or Patton (1990).

---

The focus group analyst will never be immune to criticism. When criticism of your analysis comes, be open to it and then strive to learn from this feedback. Indeed, criticism can be helpful.

The Concept of Analysis

Analysis starts by going back to the intent of the study. At times, the purpose of the study is narrow, and elaborate analysis is unneeded and inappropriate. The problem drives analysis. Difficulties emerge in both qualitative and quantitative analysis when there is a mismatch between analytical resources and the problem. This can result in elaborate analysis related to a trivial problem or inadequate analysis of a complex problem of major concern. The researcher must remember the intent of the study and regularly weigh options against two factors: available resources and the value of new information provided by the research.

In some respects, beginning the analysis is like standing at the entrance of a maze. Several different paths are readily apparent at the beginning, and as you continue, additional paths and choices emerge. You don’t know if a path will be productive until it has been explored, but the process of exploration requires an investment of effort, even if it is just to peek around the corner.

---

Respect for Our Ability to Observe

---

Survival requires a clear fix on the purpose of the study. Even experts can make mistakes if they are not prepared, caught unaware, or attend to the wrong stimulus. Here’s a classic example.

At a congress on psychology at Gottingen, during one of the meetings, a man suddenly rushed into the room chased by another with a revolver. After a scuffle in the middle of the room a shot was fired and both men rushed out again about twenty seconds after having entered. Immediately the chairman asked those present to write down an account of what they had seen. Although the observers did not know it at the time, the incident had been previously arranged, rehearsed and photographed. Of the forty reports presented, only one had less than 20 per cent mistakes about the principal facts, 14 had from 20 to 40 percent mistakes, and 25 had more than 40 percent mistakes. The most noteworthy feature was that in over half the accounts, 10 percent or more of the details were pure inventions. This poor record was obtained in spite of favourable circumstances, for the whole incident was short and sufficiently striking to arrest attention, the details were immediately written down by people accustomed to scientific observation.

(George, 1936; quoted in Beveridge; 1957, p. 131)

---

The story does not argue for the abandonment of observation but, instead, for a respect for our human strengths and weaknesses when engaged in research.

What Makes Qualitative Analysis Complex?

Consider the distinction between analysis of words and analysis of numbers. Analysis of numbers can be seductive, because the researcher gains a sense of accomplishment and confidence by knowing exactly the nature of the results. The results of quantitative inquiry come out the same each time the analysis is replicated—if they don’t, it’s time to fix the computer. Moreover, this analysis is firmly grounded in well-defined statistical procedures. In quantitative analysis, the respondents select numbers that best represent their position on a measurement scale. The researcher’s quest is built on the assumption that the scale and the question are appropriate instruments to measure the phenomenon under investigation. Unfortunately, elaborate statistical procedures cannot compensate for ambiguity in questions or responses.

On surveys, respondents are asked to select a number on a scale that represents their point of view. The number becomes a symbol of reality and the basis of analysis. The researcher may not know if the respondent really understood the question or if the available response choices were appropriate for the individual. In some situations, the respondent might find that the response choices are on target but that they apply in only certain cases. When these problems occur, the reliability of the results is jeopardized. The survey researcher attempts to eliminate these factors by pilot testing the instrument; nevertheless, surveys that reduce reality to numbers have inherent flaws in communication—some more than others. This does not mean that we should abandon statistical analysis but, rather, that we should recognize inherent assumptions and treat all data that measure human experiences with adequate humility.

Focus groups present another face of reality because open-ended questions allow participants to select the manner in which they respond. Furthermore, focus groups encourage interaction among the respondents and allow people to change their opinions after discussion with others.

The complexity of focus group analysis occurs at several levels. When a question is asked, two people may answer using different words, yet have the same meaning. The analyst needs to consider how to compare the different answers. Analysis begins with a comparison of the words. Are the words identical, similar, related, or unrelated? The analyst needs to consider other factors as well. What was the context of the comments? Were the two respondents really talking about the same thing when they answered? Did the discussion evolve so that the second respondent was keying her response to a different example? Also, the researcher must consider the emphasis or intensity of the respondent’s comment. Another consideration relates to the internal consistency of the comments. Did respondents change their positions later in the discussion? Still another consideration relates to the specificity of the responses in follow-up probes. Were the respondents able to provide examples or elaborate on the issue when probed? The researcher must take multiple dimensions into consideration when comparing responses.

The analysis process is like detective work. One looks for clues, but in this case, the clues are trends and patterns that reappear among various focus groups. The researcher’s task is to prepare a statement about what was found, a statement that emerges from and is supported by available evidence. In preparing the statement, the researcher seeks primarily to identify evidence that repeats and is common to several participants. However, some attention is also placed on the range and diversity of experiences or perceptions. The researcher must identify those opinions, ideas, or feelings that repeat, even though they are expressed in different words and styles. Opinions that are expressed only once are enlightening but should not form the crux of the report.

VOLUME 6

---

2

Critical Ingredients of Qualitative Analysis

---

Overview

Analysis Must Be Systematic

Analysis Must Be Verifiable

Analysis Requires Time

Analysis Is Jeopardized by Delay

Analysis Should Seek to Enlighten

Analysis Should Entertain Alternative Explanations

Analysis Is Improved by Feedback

Analysis Is a Process of Com...

Table of contents

• Cover page
• Title
• Copyright
• Brief Table of Contents
• Detailed Table of Contents
• Acknowledgments
• Introduction to the Focus Group Kit
• About This Book
• Part I: THE FOUNDATION
• Part II: DOING ANALYSIS
• Part III: SHARING RESULTS
• POSTSCRIPT: For Graduate Students Only
• References
• Index to This Volume
• Index to the Focus Group Kit
• About the Author