The Moderator's Survival Guide

Handling Common, Tricky, and Sticky Situations in User Research

Donna Tedesco,Fiona Tranquada

• Éditeur: Morgan Kaufmann
• Année: 2013
• ISBN:
  9780124047143
• Sujet: Diseño
• Sous-sujet: Diseño UI/UX

The Moderator's Survival Guide is your indispensable resource for navigating the rocky shoals of your one-on-one user research sessions. Inside, you'll find guidance for nearly 100 diverse situations (ranging from business-as-usual to tricky and sticky) that might occur during usability studies, contextual inquiries, or user interviews.

As a moderator, you are responsible for the well-being of the participant, your study, and your organization. You must be prepared for anything that may happen, from your technology failing to the participant quailing. Use this guide to identify your best next steps, react appropriately, and survive any challenges that comes your way.

• Practical, field-tested, and actionable tips for what to do and say—and what NOT to do or say—in each situation.
• Key patterns and extensive examples to sharpen your approach to the commonplace and prepare you for the unlikely.
• Illustrative "survival stories" contributed by numerous professionals on the front lines of user research.

Part 1

Your Moderation Toolkit

Outline

Chapter 1 Moderation Matters

Chapter 2 In the Trenches

Chapter 3 Mix and Match

Chapter 1

Moderation Matters

Power, Responsibility, and Style

Abstract

As the moderator of a user research session, you’re responsible for the competing needs of the participant’s emotional and physical state, your own emotional and physical state, your study’s research goals, overall management of the session, and the protection of your organization. Balancing all these needs requires empathy, flexibility, creativity, a sense of humor, and an aura of authority. These qualities combine with your moderating style to ensure that you can make your session as successful as possible, and that you can adapt as needed to any changing conditions.

Keywords

user research moderating style; new usability moderators; user research empathy; user research flexibility; user research creativity; user research sense of humor

1.1 “Are they laughing at me?”

1.2 Power and responsibility

1.3 The session ringmaster

1.4 The science and art spectrum

1.5 Your moderating style

1.6 Effective adaptation

1.1 “Are they laughing at me?”

I used to work at an institution of higher learning. We did a lot of usability testing, up to 120 studies a year. One particular day we had a client who wanted to test a website, but wanted their entire department staff to observe the testing. Our observation room sat 12 and it was mostly filled. Being late in the day, the observers were getting slaphappy after watching users identify the same usability problems over and over again. It was our last session of the day when we encountered a slightly more eccentric participant than normal.

As she was tackling the tasks, she was quite entertaining in some of her feedback, and you could hear, through the adjoining wall to the observation room, muffled commentary becoming louder. I proceeded to instant message my colleague in the room to have the observers keep it down, but by the time she read my note, it was too late. My participant’s eccentric nature had come out via an absolutely hysterical comment that I, as the moderator, had to literally bite my tongue to not laugh at. While I controlled my own reaction, a volcanic eruption of laughter could be heard through the wall. As my participant heard them, her face changed immediately, followed with “Are they laughing at me?”

To read about some factors to consider in a situation like Michael’s, see sections 9.7 and 15.8.

I no longer wanted to laugh. I was paralyzed with “What do I do?” I lied. I told her no, they’re not laughing at her, but rather that everyone who has come in here today has had a similar experience with this task, so we’ll have to redesign the website. It was the best I could do at the time, but I’m not sure she believed me. During the next several tasks, there were no sounds from the observation room; they were as silent as she had become. At the end, I walked her out and apologized on behalf of the observers.

—Michael Dutton

1.2 Power and responsibility

Michael’s story emphasizes how important a role you have as the moderator of a user research session. Your position on the session’s frontlines means that when tricky, sticky, and unexpected situations arise, you’re the one who has to decide what to do and then take action.

When you encounter an awkward or uncomfortable situation with a friend or someone else you know, you can inform your response with your knowledge about that person. You rarely have any additional knowledge to help you when the same situation happens during a user research session. And, of course, you have additional complications to consider. During a session, your reactions as a moderator must balance the following competing needs (Figure 1.1):

The participant’s emotional and physical states. Is she nervous or uncomfortable? Is she looking like she is unwell or experiencing physical discomfort?

Your emotional and physical states. Are you uncomfortable with anything happening in the session? Are you reacting to an allergen in the environment? Is there an external circumstance occurring that might threaten your physical safety or that of the participant’s?

The study’s research goals. Are you measuring an experience, looking for issues that need to be fixed, or trying to understand the participant’s process?

General session management needs. Are your remote session technology and recording equipment working correctly? Are your observers behaving well, especially if they’re in the same room as you?

Protection of your organization. Are you ensuring that the participant leaves with a positive experience so she doesn’t say nasty things about your organization on social media? Are you doing your best to get feedback from participants that will help improve the experience for future users of your organization’s products or services?

Throughout a session, this balance may shift and you, as the moderator, have to adjust accordingly. The way you handle a situation at the beginning of a session may be very different than how you’d handle the same situation later on, even with the same participant.

Figure 1.1 As the moderator, you’re balancing a lot of competing needs during the session.

Take the example of the usability study session from Michael’s story. At the beginning of the session, his duty as the moderator was mainly to elicit feedback that satisfied the study’s goals. Some minimal encouragement may have been needed along the way to keep the participant content, but his larger concerns were to stay neutral and keep the observers engaged in the session. But as soon as the participant heard laughter from the adjacent room, she became uncomfortable, embarrassed, and all too aware that her performance was being watched.

At this point, the balance of competing needs shifted away from the session goals and more toward the participant’s emotional comfort. Because the participant’s comfort was at risk, Michael’s attention as a moderator had to focus less on collecting feedback and more on making sure that she was okay and would leave the session feeling comfortable (rather than upset at him or his organization). Under normal conditions, telling the participant that everyone else had the same experience with the task might have been considered biasing. But Michael sacrificed that data integrity to help the participant relax and to avoid tarnishing the reputation of his organization. Also, he needed to ease tension between himself and the participant for his own emotional comfort.

This response...