Doorbells, Danger, and Dead Batteries

User Research War Stories

Steve Portigal

• Publisher: Rosenfeld Media
• Year: 2016
• ISBN:
  9781933820507
• Edition: 1
• Topic: Betriebswirtschaft
• Subtopic: F&D

User research war stories are personal accounts of the challenges researchers encounter out in the field, where mishaps are inevitable, yet incredibly instructive. Doorbells, Danger, and Dead Batteries is a diverse compilation of war stories that range from comically bizarre to astonishingly tragic, tied together with valuable lessons from expert user researcher Steve Portigal.

Photo by Steve Portigal

CHAPTER 1

The Best Laid Plans

Julia Thompson: For Want of a Shoe

Alicia Dornadic: Don’t Hate on a Tinkler

Dan Szuc: Shanghai Surprise

Sean Ryan: Pockets Full of Cash

Mary Ann Sprague: Be Prepared

George Ressler: Skyfall (or a View to a Kill)

Tamara Christensen: What the Hell? Don’t You Knock?

Jenn Downs: Burns, Bandages, and BBQ

Takeaways

Semper Gumby is an unofficial motto for the United States Marine Corps (and other military services). Adopted at a grassroots level, it references the official motto Semper Fidelis (Always Faithful), while invoking the built-in flexibility of that green clay character. It declares that successful military personnel must adapt to changes.

Some user researchers try to manage the complexity of research by crafting detailed plans and developing artifacts to support the act of planning, such as checklists, observation worksheets,¹ special notebooks for note-taking, and what’s-in-your-bag fieldwork kits.² I put together my own set of re-purposable documents³ to supplement Interviewing Users.

Planning, of course, is essential to successful fieldwork. And these artifacts can facilitate planning. But they can also imply, by their existence, if not their design, that things are going to go a certain way. Does the plan set you up to adapt, or are you in trouble when there’s a snafu?⁴ Sometimes these plans imply a simplicity to research, which may be more about selling the idea of research than actually empowering people to deal with the realities. Researchers learn—typically after losing some data—to bring extra batteries, to remind themselves to start the recording gear, to double-check they’ve brought along incentives and releases and interview guides and prototypes, and so on. But as these stories remind us, there’s no complete set of circumstances that can be fully planned for. Should Dan Soltzberg (from his story in Chapter 9, “People Taking Care of People”) have brought an extra pair of pants with him? There is no plan that can fully allow for the range of circumstances that can (and will) arise in and around fieldwork.⁵ Semper Gumby, indeed.

So you have to plan. And yet planning can never be sufficient and even worse may lull you into a state of blithe overconfidence. Instead, think about the total experience of fieldwork as a system, or “a set of elements or parts that is coherently organized and interconnected in a pattern or structure that produces a characteristic set of behaviors.”⁶

Typically, you use systems thinking when you consider your research data and when you design solutions, but what about the actual experience of doing fieldwork? It’s a complex system. Thinking about the interconnectedness between different elements will help you anticipate some points of failure (bring more than enough batteries), but it should also help you accept that failures of some form or another are inevitable (but don’t bring extra pants) and sometimes lead to other failures (a phenomenon known as cascading failure).

NOTE EVERYBODY NEEDS TO PLAN

I led a training workshop for a group of junior user researchers. I recommended using the field guide as a place to hold a number of critical process reminders (e.g., turn on the recording devices, sign the release form, etc.). One of the students reported, erroneously, to her boss, with an eye-roll and sigh, “He’s telling us to turn on the camera.” Her boss, a highly seasoned researcher, unaware of the higher order point I was attempting to teach, reminded her, “I’ve been doing this work for a long time, and I still forget to turn on the camera.”

In this chapter, Julia Thompson, Tamara Christensen, and Jenn Downs experience cascading failures. Alicia Dornadic and her team try—and fail—to plan for the most basic of human needs. Dan Szuc and Mary Ann Sprague find that a small mistake has significant consequences, and they each explore how to adapt. Sean Ryan fails to plan adequately, and he works in blissful ignorance without consequence and only realizes later that he might have taken a different approach. George Ressler anticipates a point of failure and designs his approach to prevent that, but still must adapt on the fly. Each of these researchers, flawed human beings all, negotiate the slippery, shifting balance between intentionality and flexibility.

Julia Thompson: For Want of a Shoe

It all started with a simple question from the dispatcher: “Do you want a call when your taxi arrives?” My nonchalant answer, “No thanks, I should be OK,” was the nail in my coffin. This was my first error in a series of cascading mistakes.

The next morning, I was heading out-of-country for in-home interviews. That night, in an effort to be as prepared as possible, I called to arrange a taxi for an early morning pickup. I hung up the phone and proceeded to pack my bags. I considered carefully what to pack. I visualized my next few days: What would the weather be like? What would be my mode of transportation? What clothing would be appropriate for the work—casual enough to fit into a home environment and dressy enough to fit into an office environment? I was sure that I had considered all the details. Unfortunately, the most important detail, my alarm, was what I missed.

Satisfied with my preparation, I went to bed and slept well. The next morning I awoke feeling refreshed. With birds chirping outside, sunlight filled the room. Yet something felt terribly wrong. What time was it? Why was it so light out? I picked up my phone, checked my alarm, and then checked the time. My stomach fell to the floor. My flight was leaving now.

Sheer panic overtook me. I couldn’t think straight. I had never missed a flight before. I felt like I was going to throw up. I was paralyzed. I had no idea what to do. I grabbed my phone and called our corporate travel agent. It felt like hours as I waited on hold to ask my pressing questions: Could I still make my interview? When was the next flight? Could I fly out of a different airport instead?

The sound of my heartbeat drowned out every noise as I sat there waiting, palms sweating, phone clutched. The agent came back on the line and said there was a flight leaving from another airport in two hours. Could I make it there in time? It’s almost rush hour. It’s an hour’s drive with no traffic. What about parking? Customs? Security? If I took the car, how would my husband get to work? On top of all that, the agent still wasn’t sure whether there was room on the flight.

We decided, together, that I should start driving, and I should stay on the line while she called the airline to confirm availability. I jumped in the car, with my phone on the passenger seat and that awful music taunting me as I continued to wait, on hold. I got about 10 minutes down the road when the agent told me to pull over and go home. That flight wouldn’t be mine. I would settle for another flight, hours later, and hours after my scheduled interview.

Later that day, as my plane came in for its landing, I just felt low. I was tired from the emotional rollercoaster of missing my flight, I was anxious knowing I’d have to tell the people I was working with what had happened, and I was sad that I had missed out on an interview and the opportunity to see firsthand into the life of one of our customers. The only thing saving me was the fact that I was the client, so even though I missed the interview, it still went ahead as scheduled.

The following day I awoke in the right place and at the right time, with a better perspective on life. Our local research partner was gracious enough to include me in an interview that day. I was thankful. I was relieved. But now, that meant there would be four of us attending this interview. Two consultants and two clients: two too many. The consultant had called ahead and confirmed with our interviewee that it would be OK if an additional person (me!) attended the interview. Our interviewee was very accommodating and agreed to have all four of us into her home. I was so preoccupied with resolving my own error that I didn’t consider, until later, how the dynamic of the interview would now be affected.

We all got to the interview, we all walked in, and we all sat down in the chairs offered to us by our interviewee. As everyone was setting up, I started to look around and take note of the environment. I noticed several pairs of shoes neatly arranged by the front door. I looked over at our host, I looked down: bare feet. My eyes darted around the room. I looked down at all our feet. All four of us had our shoes on, laces tied. Bah! We were the worst guests ever. Weren’t we all, as researchers, supposed to notice something so simple but so important?

PHOTO BY JULIA THOMPSON

I spent the next five minutes cursing myself, my missed flight, the totally wrong and overpowering dynamic of four researchers to one customer, and the miss on basic shoe etiquette. I had to shake it off—all the feelings of shame, all the feelings of doubt—and I had to focus. I had to be in the moment. I had to get the most I could out of the interview, and I had to show the interviewee the respect she deserved.

It ended up being a great discussion. It was, by no means, a textbook in-context interview, but we had a nice dynamic emerge nonetheless. My story is not one of a single epic fail, but instead of a series of errors with a cascading effect. “For want of a nail the shoe was lost, for want of a shoe the horse was lost . . . ” Here, we had not for want of a shoe, we had too many.

Alicia Dornadic: Don’t Hate on a Tinkler

Going to someone’s home for the first time to interview that person, especially in an unfamiliar culture and language, can be awkward. Showing up with two researchers, a cameraman, and a couple of clients in tow—all of whom are overcaffeinated and in need of a bathroom break—can make for a circus act. These were three-hour-long interviews, too. So, despite our best efforts to arrange feeding and peeing times before getting to the person’s home, we usually all had to pee at some point during the interview.

But our translator was the absolute queen of tinkling. The first day I was understanding. “Maybe she’s sick or nervous,” I thought. She would take two to four breaks during each interview, which left the rest of us smiling and pointing at things dumbly, trying to make conversation in her absence. By the end of the week, my patience was shot. I was ready to strap some adult diapers on her. I would glower at her every time she asked for water, tea, or a soda. “Really?” I thought, my eyes on fire, “Should you really be having that?” I’m not proud of this. But I couldn’t help being annoyed.

Finally, karma came to bite me on the ass. It was at the end of a long interview at the end of a long day, and I broke down and asked if I could use the restroom. Our host pointed to it, and I stumbled inside, missing the two-inch step down into it. There wasn’t a lot of light in the bathroom, and it was cluttered. I couldn’t find a switch. But no matter. I go. I reach for the toilet paper, and BOOM! CRASH! I take down the entire metal toilet paper rack off the wall, and it crashes onto the tiled floor. It was too dark to see how to fix it, so I had to come out and explain what I had done and apologize. Not only that, but my explanation and apology had to be translated! Translated and explained to two researchers, a cameraman, a couple of clients, and our participant. It ended up not being a big deal, but I was embarrassed. And I felt guilty for all my negative thoughts toward our translator. As much as I was annoyed at our tinkler friend, at least she didn’t break anything.

Dan Szuc: Shanghai Surprise

We were on the train in Shanghai on our way to visit a person in their home as part of a research project. Doing random checks of all equipment becomes second nature, ensuring that you have backups of backups, cables work correctly, sound is being recorded correctly, and video is working well.

“LE MÉTRO DE SHANGHAI” (CC BY-ND 2.0) BY CHRISTIAN MANGE

We all have specific roles on home visits. Hok and I capture both the interview and surrounds on film using Flip cameras, Jo is responsible for speaking with the person we are visiting to ensure that they are comfortable, and Hok also is our guy for ensuring that all the equipment is technically working well (and if something is not working well, he usually knows how to fix it).

So back to the train ride in Shanghai . . . the three of us were together, testing the recorder, cable, and microphone. We realized after conducting a few test recordings that there were clear breaks in the recording when playing it back. During the previous interview, we had gone through a security scanner at a train station with the participant (as part of the journey we were filming). The cable connecting the recorder and the bag were stretched going through security unnecessarily, possibly causing damage to the wires.

We tested various places where we thought the sound might be breaking up—the connectors, the microphone, and the cable itself. We wanted to get this right because the microphone clips onto the person we are interviewing and ensures that we have clear audio (in addition to the audio that’s captured on the video using the Flip cameras). We did not have time to go to an electronics store to get new equipment and were relieved that the audio recorder itself was working well and could serve as a (non-ideal) backup microphone.

Together, we needed to come up with a plan to ensure that...