![]()
Part I
Events and representations
![]()
1 Telling tales, or journeys
Barbara Tversky
Years ago, Michel Denis gave us the secret of life. Life, after all, is a journey. And Michel gave us the instruction manual for a journey, the Denis Plan. According to the Denis Plan, the trick is to break it into steps, into segments, into parts. Segmenting is so crucial that it applies not just to space, but to time; not just to each alone, but to both together; after all, that's what a journey is. Indeed, this key step applies equally to any knowledge domain. But this is only the beginning. Each major segment can be further segmented, and the segments are not mere slices, they are integral parts, wholes at a finer level. For a journey: first a Start Point, then a Reorientation, then an Action, with progression along its path, and finally an End Point. Repeat until destination is reached. As he, Francesca Pazzaglia, Cesare Cornoldi, and Laura Bertolo have shown, this instruction manual is so good it even works in Venice (Denis, Pazzaglia, Cornoldi, & Bertolo, 1999). And, as should be evident by now, this is an instruction manual for life: Start somewhere. Reorient. Take an action. Arrive somewhere. Repeat.
Michel's life has included many journeys. Some have been to pursue research in imagery and spatial cognition, some have been to promote cognitive psychology in France and Europe, some have been to advance scientific psychology in the world. He has been an exemplary leader in each of those journeys, forging the way, seeking others to accompany him, encouraging them, coalescing communities, and finding resources to nurture them. I was fortunate to get picked up on one of his earliest journeys, into imagery and spatial cognition. His visits to Stanford and invitations to Paris allowed his work to inspire mine, and perhaps vice versa. He involved me in his efforts on behalf of the International Union of Psychological Sciences, first the inspiring International Congresses, and then the governance, where he served elegantly as Chair. What follows is a selective view of our work and his, the many places where our separate routes through cognitive psychology have intersected, and some where they diverged.
Cognitive maps
When I first met Michel, among other things, he had been working on scanning mental maps, using a paradigm of Steve Kosslyn (for a description of this paradigm, see chapters 2 and 9 , this volume), another scholar whom he had picked up on his virtual and real travels. Somewhat later, Holly Taylor and I began a different way to study maps and mental maps. We were interested in how people described environments they learned from maps and how people understood those descriptions. We found that people's spontaneous descriptions took one of two perspectives: either one from above, a survey perspective, or one from within, a route perspective. In a survey perspective, people describe landmarks relative to one another in terms of north, south, east, and west — for example, “The Eiffel Tower is west of the Louvre”. In a route perspective, people describe landmarks relative to a moving traveler, “you,” in terms of left, right, front, and back — for example, “As you travel up the Seine, you will see the Louvre on your right and, later, the Eiffel Tower on your left” (Taylor & Tversky, 1992, 1996). Despite claims that perspective should be consistent, half our participants mixed perspectives, often mid-sentence, rarely — if ever — signaling the switch. Rather than depending on some fixed mental representation, choice of perspective in descriptions depended on features of the environments; environments with landmarks on several size scales and with multiple routes received relatively more survey descriptions. Perspective is not just survey (or exocentric, the term favored by environmental psychologists, or absolute, the term favored by some linguists) or route (or egocentric or intrinsic or embedded). An embedded perspective can be yours or mine. In an interactive situation, we found that choice of perspective also depended on the relative cognitive loads of speaker and listener (Mainwaring, Tversky, Ohgishi, & Schiano, 2003). Participants were asked to describe the location of one object — say, a cache of gems — to a purported partner in espionage with a different perspective on the scene. Speakers spontaneously adopted the listener's perspective when the cognitive load of the listener was greater than that of the speaker, but speakers took their own perspective when their cognitive load was greater.
More surprisingly, we have found that people often spontaneously adopt another's perspective rather than their own even in a noninteractive situation (Tversky & Hard, 2009). Students sharing the same perspective as an experimenter were shown a photograph of a man seated at a table with a bottle to his left, reaching for a book on his right, and they were asked, “In relation to the bottle, where did he put the book?” More students answered from the perspective of the man in the photo, “on the right,” than from their own and the experimenter's perspective. That one's own perspective does not necessarily have primacy and that perspective-taking is FB02exible and consequently responsive to the circumstances suggests that mental representations of well-learned environments may be perspective-free. That this is the case is corroborated by companion research showing that spatial mental models of well-learned environments were more abstract than any specific perspective (Taylor & Tversky, 1992). In those experiments, students studied descriptions of a small town or a zoo or a convention center, each with 11–13 landmarks. Half studied descriptions with a route perspective and half studied descriptions with a survey perspective. Later, students verified true-false statements about the environments, taken verbatim from the descriptions or requiring inferences from the descriptions. Half the statements used a route perspective and half used a survey perspective. Students responded as quickly and accurately to inference statements from the perspective they had read as from the other perspective. This suggests that their mental models of the environments were perspective-free and allowed taking either perspective with equal ease. Evidence from neuroscience also shows that the experienced world is encoded in multiple perspectives simultaneously (e.g., O'Keefe & Nadel, 1978; Tipper & Behrmann, 1996).
Routes
Shortly thereafter, Michel became interested in route directions. Route directions are different from route descriptions, although both typically take an egocentric perspective. Route directions are meant to take someone from one place to another rather than describing an environment. Michel stopped dozens of people in the streets and asked them for directions from here to there. He painstakingly analyzed what they told him. He found that despite differences in style, accuracy, length, detail, and more, route instructions reduced to the structure described earlier — iterations of units consisting of start points, reorientations, progressions on a path, and end points (Denis, 1997). With others, he went on further forays and explorations of what makes route directions effective (e.g., Denis et al., 1999). I was so excited by Michel's analysis that I translated an earlier version of his 1997 paper into English for my students. Paul Lee and I then went out into the field and collected and compared route sketches to route descriptions (Tversky & Lee, 1998, 1999). We asked hungry students outside their dormitory if they knew how to get to a popular fast-food restaurant; if they did, we asked them to either sketch a route map or write directions to get there. We found the same four-step structure or syntax in route sketches as Michel had found in route directions as well as the same semantics. This meant that, despite external differences, besides differences in modality, the same mental model was used to generate both route sketches and route directions. Other research examining the order of sketching the maps indicated that the mental representations were organized hierarchically and that the hierarchy depended on specifics of the environments — for example, geographic features like mountains and rivers, or architectural features like entrances and paths (Taylor & Tversky, 1992). Despite similarities of semantics and even syntax, there were intriguing differences in the pragmatics of depictions and descriptions, in how the semantics and syntax of routes are used in one medium or the other. Descriptions could and did elide; they could and often did omit one or more of the segments that depictions of routes could not. For example, because the next start point is the previous end point, only one need be mentioned: the other can be inferred. By contrast, drawing a route requires continuity of end and start points for each segment. Indeed, it would be awkward to segment a route map, to draw separate sketches for each segment, and effortless to make it continuous.
Somewhat later, after a hiatus, we returned to route directions, combining two of Michel's interests: route directions and imagery. This was no mere coincidence; it was at the instigation of one of Michel's graduate students, Ariane Tom, whom he graciously sent to me for postdoctoral research. Ariane and Michel had found that when people recalled route directions, they remembered landmarks better than streets (Tom & Denis, 2004). In reexamining that work carefully, Ariane and I realized that the landmarks in the route directions were vivid (e.g., church rather than building), while the street names were rather bland (e.g., Williams St. rather than street lined with ornate gas lamps). We wondered if the superiority of landmarks was due to their vividness and distinctiveness rather than their status as a component of a route, and, indeed, it was (Tom & Tversky, in press). In one experiment, we reversed the effects, showing that streets were better remembered than landmarks when the streets were vivid and distinctive and the landmarks were bland. In a second study, we found that when both the streets and the landmarks in the route directions were vivid and distinctive, the directions were remembered better than when the streets and landmarks were nondistinctive. In addition, we found that people higher in imagery abilities both read the descriptions faster and remembered them better. We interpreted the findings as showing that mental model construction has two aspects: a spatial structure, in this case, the connectivity of streets and landmarks, and associative relations, the specific content of the streets and landmarks. Vividness seems to support both creating the associative relations and creating and remembering the spatial structure.
Assembly
We took a turn here, from one kind of directions to another, from directions to get from here to there, to directions for putting something together. Both tasks are familiar, and sometimes frustrating. Both have depictive and descriptive possibilities. We selected assembly of a TV cart because it is representative not only of other assembly tasks but also of a large class of learning tasks. That class includes any task where there are separate parts and relationships among them — for example, learning how things work or how to operate things or learning how characters interact in space and time in stories (Heiser & Tversky, 2010 submitted; Tversky, Agrawala et al., 2007). Putting together a TV cart was convenient because students could figure out how to do it and do it in a relatively short time. The task was simple: students were presented with the parts to be assembled and the picture of the assembled TV cart on the box. They were first asked to assemble the cart. After students assembled the TV cart, they were asked to produce instructions that would enable others to assemble it accurately and efficiently. Some were simply asked to produce instructions, so they could use a combination of words and sketches as they chose. Others were asked to create instructions by only using sketches or only using language; still others explained to a video camera, so they could and did use gestures.
As with route directions, we wanted to compare verbal and pictorial instructions to look for the parallels in semantics and syntax that would suggest the same underlying mental representation. But we had other interests as well: we wanted to characterize the nature of the depictions and descriptions, and we wanted to extract cognitive design principles for creating effective visual instructions. We were working with a group of whizzes in computer graphics who would instantiate the cognitive design principles in a program that generated assembly instructions from models of the objects to be assembled (Agrawala et al., 2003). Just as we were beginning this project, another of Michel's wonderful students arrived as a visiting researcher, Marie-Paule Daniel (we are back-tracking here, so common in narrative; Marie-Paule actually preceded Ariane; we are sacrificing a temporal path for a conceptual one). Marie-Paule had also worked on route directions with Michel. They had found that when participants were asked to give concise directions, participants retained the essential information, the information about actions, and omitted the less essential, more descriptive, information (Daniel & Denis, 2003). It seemed natural to add the same manipulation to our new experiments (Daniel, Tversky, & Heiser, 2010). Asking people to be concise would have the added advantage of informing us what information about assembly participants thought was critical.
The findings were rich. To begin with, spatial ability — as indexed by a common measure, the Vandenberg mental rotations task (Vandenberg & Kuse, 1978) — affected performance. Those high in spatial ability both assembled the TV cart faster and produced better pictorial instructions. We know their instructions were better because a new group of students rated them higher, and they helped yet another group of students to assemble the TV cart more efficiently. What made pictorial instructions effective? First, they were segmented into steps, where each new step corresponded to a new part to be attached. The steps were action-object pairs, analogous to turns at landmarks. Next, the highly rated instructions used perspective and showed the perspective of action. Finally, they used extra-pictorial devices, arrows, and guidelines. Arrows showed the actions that assemblers should take and were generally used for large parts; guidelines showed how parts fit together and were generally used for smaller parts. By contrast, those low in mental rotation skills made more errors assembling the TV cart and produced less effective diagrams. Often, those low in spatial ability only depicted menus of the parts, and they used words to describe assembly actions. When they showed assembly, their depictions were typically FB02at and imprecise. They rarely made perspective drawings and rarely showed action. The analyses of language corroborated the importance of conveying action in assembly instructions. That was the information that participants regarded as critical, and retained, even when asked to be concise. High spatial ability was expressed in language as well as in the visual instructions. Those high in mental rotation ability included more action information in their language than did those low in mental rotation ability. The key to instructions — whether to reach a destination or to assemble an object — are actions.
Cognitive design principles
Three major cognitive design principles for effective assembly instructions emerged from this project: proceed step-by-step, where each step is the addition of a part; depict the perspective of action; use arrows and guidelines to show actions. These were incorporated into the computer algorithm, and the resulting instructions enabled a new group of students to assemble the TV cart more efficiently and accurately than the instructions that came with the TV cart.
In fact, the earlier work on route maps had also yielded cognitive design principles. The route sketch maps that people produced ignored exact angle of turn, exact shape of road, and exact distances. Most turns were drawn as right angles, irrespective of actual angle; most roads were drawn as either straight or slightly curved; and small distances with many turns were exaggerated, whereas long distances without turns were minimized. These principles were instantiated into an algorithm that produces route maps on demand and almost instantly (Agrawala & Stolte, 2001).
Together, the projects suggest that cognitive design principles for effective visual instructions and explanations can be derived systematically from users' productions of verbal and visual instructions or explanations (Tversky, Agrawala, et al., 2007). The program for revealing cognitive design principles has several elements: production, preference, and performance. In the best cases, and routes and assembly were such cases, these converge.
Visual explanations
The depictive instructions, both for routes and for assembly, are examples of visual explanations. Diagrams of how something works or how to work something are also examples of visual explanations. The superior route and assembly depictive instructions, as well as the verbal in...
