Part I
FIELDS OF MOBILE MEDIA
1
OBSERVING MOBILE MULTIMEDIA
Ilpo Koskinen
Introduction
How do people use mobile multimedia? This was a seemingly simple question we posed in Helsinki early in 1999 after getting funding for a research project for the future of digital imaging. Even though multimedia messaging was not yet on the market, it had figured prominently in industry concepts for years, and we knew it was going to be reality in near future. The question for us was, however, how best to study it. This question led us to a series of questions about research methods and methodology, but more importantly, it also led us to several decisions on theory. The result of these investigations was pioneering studies on mobile media using a methodology that is still relevant for the research community.
This chapter describes two studies we did in Helsinki between 1999 and 2002. The first of these, Mobile Image, saw daylight in March 1999 and went on for about slightly over a year. The second study, called Radioiinja after the mobile carrier that funded the project, was done in (northern) summer 2002. Four researchers and numerous research assistants participated in these projects. The main researchers in Mobile Image were Ilpo Koskinen and Turo-Kimmo Lehtonen, two sociologists, and Esko Kurvinen, industrial designer; in Radioiinja, Koskinen and Kurvinen teamed up with Katja Battarbee, who is an industrial designer. These studies led to several follow-up projects. By 2007–2008, these studies had led to six books (including four Ph.D. theses) and several articles. In this chapter, we will not deal with these follow-up studies, though.1
Although we have been talking about design research, the studies were done in a design university, and our main inspiration for these studies was the idea of ready-mades; this chapter shows that there was more than art in these studies. This “more” can be found in their theoretical roots. In terms of methods, these studies built on two sources. First, ethnomethodology came into picture when we saw the first messages and realized that they are organized locally in a sequential context, much like conversation analysis claims.2 Messages 1 and 2, taken from actual screen shots, show how one sequence works.
Message 1. Greetings
July 2 2002 19:16 Tom to Ann Marie Greetings.
Message 2. Return Greeting
July 2 2002 19:33 Ann Marie to Tom Greetings back to you! Also from mom!
We also soon realized that looking at sequences of messages gave us a robust method for explication: we saw the same sequences again and again. There were greetings, questions and answers, teases and replies, riddles, and so on. In the language of ethnometh-odology, multimedia messages were, if not quite like conversation, designed with a very robust set of ethnomethods that had a moral basis as well. A greeting with no reply prompted demands for reply, or accounts for not replying very much like in conversation. Second, symbolic interactionism came into picture when we wanted to understand how experiences are co-construed. For instance, a sign like “Paris” functions in messaging not only locally, as ethnomethodology taught us, but also as a symbol used to define qualities not evident in messaging.3
Studying Messages as They Happen
Both frameworks posed similar demands for research methods. Namely, they told us to study messages as they happen. This is obvious for any ethnomethodologist; when we followed Blumer we wanted to focus not so much on how people construct experiences together, but how they modify and twist these experiences.4
This background thinking meant that we had to find a way to observe multimedia very closely. Things crucial to ethnomethodology required that we needed to get individual messages. Also, we had to see how recipients treat these messages. Furthermore, we needed to see how the senders of the original messages treat these responses—and so on. Things crucial to interactionism set similar requirements. To see how people create shared definitions of situation, we needed to see how they observe things, agree on these observations, and sink these agreements into an unquestioned background resource of action. Also, we had to see how these definitions are maintained and how people work their way out of them.
We needed to find ways to capture messages in sequence, which was our essential resource for interpretation. For example, a picture of a crying baby can be designed in many ways. It can become an endearment in the family, but it can also be a tease sent to a whining teenager or little brother. Whether it becomes an endearment or a tease, on the other hand, depends on how the recipient takes it. A response like “he is soooo CUTE when he is crying!!!” establishes a very different definition of the situation than “I’m NOT a baby.” Without seeing the actual messages we quite simply do not know what happens in messaging, our background theories told us.
In terms of methods, the key implication of our theoretical choices was that it rules out things like interviews and observations. People are notoriously bad at observing their own action as it happens, and most folk theories of interaction are barely more than bad psychology. Observing people sending messages was out of the question for even better reasons. By observing—designers usually talk about “shadowing”—people, we might have been able to see what they do when they capture something and how they design a message. What would have been impossible, however, was seeing what the recipient does. By relying on ethnographic observations, we would have lost a resource that was essential for interpretation, messages as they happen in their naturally occurring sequences.
Our theories also gave us analytic protocols. Both traditions rely more or less on “analytic induction,” as the sociologists Thomas and Znaniecki once called a method that builds up from observations into more abstract categories and finally to conceptual systems.5 This method starts from individual instances of data, sets them in sequence, describes these sequences, and then poses the question of what people achieve with these sequences. For example, our crying baby picture sent to a teenager may become an efficient tease if the teenager is insulted, leading to an exchange that actually strengthens the emotional bond between the sender and the receiver. It may also become entertainment to others who are CC’ed into the message sequence. After describing these sequences and their functions, the analysis proceeds into classifying them. Thus, we described sequences like greetings, good mornings, question-answer pairs, teases, riddles, and fights.
As this brief introduction shows, our theoretical choices led us away from the then-dominant methods of studying phones. Most early work on mobile phones was based on interviews or observations.6 One particularly important precedent had, however, collected actual messages. In a study of text messages Elisa Kasesniemi and her colleagues had collected actual messages and analyzed how they were performed, but their analysis built on ethnology rather than ethnomethodology, and was interested in studying the culture of text messaging rather than their delivery.7 For them, the sequence of messages was not important; for us, it was the key to understanding messages.
Our focus on messages in their naturally occurring sequential context was, as far as we know, the first of its kind in mobile media studies. By now, it is by no means unique in literature. Other researchers who deserve mention are Ditte Laursen, whose Ph.D. thesis looked at text messages; Ilkka Arminen and Sanna Raudaskoski, who have focused on Wireless Application Protocol, an early version of Mobile Internet; Christian Licoppe, who has focused on video in interaction with Marc Relieu, who has studied several experimental technologies like mobile drawing platforms; and Nancy van House and Marc Davis, whose study of how a community of students used its camera phones partly inspired the Helsinki studies.8 To our knowledge, in addition to us, only Licoppe, Relieu, and van House’s group have had access to actual messages by observing them through the Internet.
Research Design
In Mobile Image, we gave Nokia Communicators (9110) and a Casio digital camera to four groups for roughly two months, each to be used freely. Radiolinja (today Elisa), a Helsinki-based mobile operator, provided the telephone service. In 1999, the Communicator was the only phone on the market capable of storing, sending, and receiving photographs. With the Casio camera, people could take pictures, beam them into the phone with infrared, and send them as attachments in e-mail messages. Recipients could view these messages either in e-mail, or download them to their phone to be viewed on the screen, which used gray scale colors at that time. The whole process took place over a wireless connection (GSM).
We recruited three groups of friends and acquaintances, each group having five members. Table 1.1 describes the groups and gives details of their composition and the study period. Before the study, we organized a two-hour training session to teach people how to use their communicators and the camera, and how to troubleshoot issues like Internet Protocol (IP) addresses. We also interviewed them. After the study, we did a follow-up interview. During the study the male and the female groups sent a total of 371 e-mail messages (258+113). A single message contained between one and sixteen photographs.
Table 1.1 Groups in Mobile Image | Group | Dates | Participants |
| Pilot | 30.6.1999–31.12.2000 | Researchers of the University of Art and Design Helsinki, four men and one woman (three industrial designers, two sociologists) (born 1961–1973) |
| Male | 2.3.2000–8.5.2000 | A group of friends, university-level business and engineering students/graduates already in jobs (born 1973–1975) |
| Female | 17.5.2000–3.7.2000 | A group of friends, students of social sciences at the University of Helsinki (born 1973–1976) |
| Control | 14.12.2000–15.3.2001 | Friends and work acquaintances, designers working in a U.S. owned new media company (training in design and technology) (born 1977–1979) |
In Radiolinja, we followed three groups in the Finnish mobile phone operator Radi-olinja’s technology and service pilot, which took place July 11–20, 2002, and lasted about five weeks. Three mixed-gender groups with seven, eleven, and seven members respectively were studied. There were fifteen men and ten women in these groups. The average age was 28 (Md=30). Each user was given a multimedia phone. Seventeen participants had a Nokia 7650 with an integrated camera, and eight had a Sony Ericsson T68i with a plug-in camera. Out of the Radiolinja pilot, we selected groups to take into account gender difference, terminal types, and the city-countryside axis. Most participants lived in the He...