Visual Thinking for Design

Colin Ware

• Publisher: Morgan Kaufmann
• Year: 2010
• ISBN:
  9780080558417
• Topic: Computer Science
• Subtopic: Human-Computer Interaction
• Index: Computer Science

Visual Thinking brings the science of perception to the art of design. Designers increasingly need to present information in ways that aid their audience's thinking process. Fortunately, results from the relatively new science of human visual perception provide valuable guidance.

In this book, Colin Ware takes what we now know about perception, cognition, and attention and transforms it into concrete advice that designers can directly apply. He demonstrates how designs can be considered as tools for cognition – extensions of the viewer's brain in much the same way that a hammer is an extension of the user's hand. The book includes hundreds of examples, many in the form of integrated text and full-color diagrams.

Experienced professional designers and students alike will learn how to maximize the power of the information tools they design for the people who use them.

• Presents visual thinking as a complex process that can be supported in every stage using specific design techniques
• Provides practical, task-oriented information for designers and software developers charged with design responsibilities
• Includes hundreds of examples, many in the form of integrated text and full-color diagrams
• Steeped in the principles of "active vision, " which views graphic designs as cognitive tools

Chapter 1 Visual Queries

When we are awake, with our eyes open, we have the impression that we see the world vividly, completely, and in detail. But this impression is dead wrong. As scientists have devised increasingly elaborate tests to find out what is stored in the brain about the state of the visual world at any instant, the same answer has come back again and again—at any given instant, we apprehend only a tiny amount of the information in our surroundings, but it is usually just the right information to carry us through the task of the moment.

We cannot even remember new faces unless we are specifically paying attention. Consider the following remarkable “real world” experiment carried out by psychologists Daniel Simons and Daniel Levin.^♦ A trained actor approached an unsuspecting member of the public, map in hand and in a crowded place with lots of pedestrian traffic, and began to ask for directions. Then, by means of a clever maneuver involving two workmen and a door, a second actor replaced the first in the middle of the conversation.

^♦ Daniel J. Simons and Daniel T. Levin. 1998. Failure to detect changes to people during a real world interaction. Psychonomic Bulletin and Review. 5: 644–669.

The second actor could have different clothing and different hair color, yet more than 50 percent of the time the unsuspecting participants failed to notice the substitution. Incredibly, people even failed to notice a change in gender! In some of the experiments, a male actor started the dialogue and a female actor was substituted under the cover of the two workmen with the door, but still most people failed to spot the switch.

What is going on here? On the one hand, we have a subjective impression of being aware of everything, on the other hand, it seems, we see very little. How can this extraordinary finding be reconciled with our vivid impression that we see the whole visual environment? The solution, as psychologist Kevin O’Regan^♦ puts it, is that “The world is its own memory.” We see very little at any given instant, but we can sample any part of our visual environment so rapidly with swift eye movement, that we think we have all of it at once in our consciousness experience. We get what we need, when we need it. The reason why the unwitting participants in Simons and Levin’s experiment failed to notice the changeover was that they were doing their best to concentrate on the map, and although they had undoubtedly glanced at the face of the person holding it, that information was not critical and was not retained. We have very little attentional capacity, and information unrelated to our current task is quickly replaced with something we need right now.

^♦Kevin O’Regan’s essay on the nature of the consciousness illusion brings into clear focus the fact that there is a major problem to be solved, how do we get a subjective impression of perceiving a detailed world, while all available evidence shows that we pick up very little information. It also points to the solution—just in time processing. J.K. O’Regan, 1992. Solving the “real” mysteries of visual perception: The world as an outside memory. Canadian Journal of Psychology. 46: 461–488.

There is a very general lesson here about seeing and cognition. The brain, like all biological systems, has become optimized over millennia of evolution. Brains have a very high level of energy consumption and must be kept as small as possible, or our heads would topple us over. Keeping a copy of the world in our brains would be a huge waste of cognitive resources and completely unnecessary. It is much more efficient to have rapid access to the actual world—to see only what we attend to and only attend to what we need—for the task at hand.

The one-tenth of a second or so that it takes to make an eye movement is such a short time in terms of the brain’s neuron-based processing clock that it seems instantaneous. Our illusory impression that we are constantly aware of everything happens because our brains arrange for eye movements to occur and the particularly relevant information to be picked up just as we turn our attention to something we need. We do not have the whole visual world in conscious awareness. In truth, we have very little, but we can get anything we need through mechanisms that are rapid and unconscious. We are unaware that time has passed and cognitive effort has been expended. Exactly how we get the task-relevant information and construct meaning from it is a central focus of this book.

The understanding that we only sample the visual world on a kind of need-to-know basis leads to a profoundly different model of perception, one that has only emerged over the last decade or so as psychologists and neurophysiologists have devised new techniques to probe the brain.

According to this new view, visual thinking is a process that has the allocation of attention as its very essence. Attention, however, is multifaceted. Making an eye movement is an act of attending. The image on the retina is analyzed by further attention-related processes that tune our pattern-finding mechanisms to pull out the pattern most likely to help with whatever we are doing. At a cognitive level, we allocate scarce “working memory” resources to briefly retain in focal attention only to those pieces of information most likely to be useful. Seeing is all about attention. This new understanding leads to a revision of our thinking about the nature of visual consciousness. It is more accurate to say that we are conscious of the field of information to which we have rapid access rather than that we are immediately conscious of the world.

This new understanding also allows us to think about graphic design issues from a new and powerful perspective. We can now begin to develop a science of graphic design based on a scientific understanding of visual attention and pattern perception. To the extent to which it is possible to set out the message of this book in a single statement, the message is this: Visual thinking consists of a series of acts of attention, driving eye movements and tuning our pattern-finding circuits. These acts of attention are called visual queries, and understanding how visual queries work can make us better designers. When we interact with an information display, such as a map, diagram, chart, graph, or a poster on the wall, we are usually trying to solve some kind of cognitive problem. In the case of the map, it may be how to get from one location to another. In the case of the graph, it may be to determine the trend; for example, is the population increasing or decreasing over time? What is the shape of the trend? The answers to our questions can be obtained by a series of searches for particular patterns—visual queries.

To find out which kind of fruit import is the largest by dollar value we make visual queries to find the tallest bar, then find and read the label beneath.

To find a fast route, we first make visual queries to find the starting and ending cities, then we make queries to find a connected red line, indicative of fast roads, between those points.

At this point, you may be considering an obvious objection. What about the occasions when we are not intensely involved in some particular task? Surely we are not continually constructing visual queries when we are sitting in conversation with someone, or strolling along a sidewalk, or listening to music. There are two answers to this. The first is that, indeed, we are not always thinking visually with reference to the external environment; for example, we might be musing about the verbal content of a conversation we had over the telephone. The second is we are mostly unaware of just how structured and directed our seeing processes are. Even when we are in face-to-face conversation with someone, we constantly monitor facial expressions, the gestures and gaze direction of that person, to pick up cues that supplement verbal information. If we walk on a path along the sidewalk of a city, we constantly monitor for obstacles and choose a path to take into account the other pedestrians. Our eyes make anticipatory movements to bumps and stones that may trip us, and our brains detect anything that may be on a trajectory to cross our path, triggering an eye movement to monitor it. Seeing while walking is, except on the smoothest and most empty road, a highly structured process.

To flesh out this model of visual thinking, we need to introduce key elements of the apparatus of vision and how each element functions.

THE APPARATUS AND PROCESS OF SEEING

The eyes are something like little digital cameras. They contain lenses that focus an image on the eyeball. Many find the fact that the image is upside-down at the back of the eye to be a problem. But the brain is a computer, albeit quite unlike a digital silicon-based one, and it is as easy for the brain to compute with an upside-down image as a right-side-up image.

Just as a digital camera has an array of light-sensitive elements recording three different color values, so the eye also has an array of light-sensitive cones recording three different colors (leaving aside rods^♦). The analogy goes still further. Just as digital cameras compress image data for more compact transmission and storage, so several layers of cells in the retina extract what is most useful. As a result, information can be transmitted from the 100 million receptors in the eye to the brain by means of only 1 million fibers in the optic nerve.

^♦The human eye actually contains four different receptor types, three cone types and rods. However, because rods function mainly only at low light levels, in our modern, brightly lit world we can for all practical purposes treat the eye as a three-receptor system. It is because of this that we need only three different wavelength receptors in digital cameras.

At the back of the eye is a mosaic of photo-receptors. Each responds to the amount of light falling on it. This example shows the central foveal region where three different cone receptors register different colors of light.

There is, however, a profound difference between the signal sent from the eye to the back of the brain for early-stage processing and the signal sent to a memory chip from the pixel array of a digital camera. Brain pixels are concentrated in a central region called the fovea, whereas camera pixels are arranged in a uniform grid. Also, brain pixels function as little image-processing computers, not just passive recorders.

Visual detail can only be seen via the fovea, at the very center of the visual field. Our vision is so good in this region that each eye can resolve about 100 points on the head of a pin held at arm’s length, but the region is only about the size of our thumbnail held at arm’s length. At the edge of the visual field, vision is terrible; we can just resolve something the size of a human head. For example, we may be vaguely aware that someone is standing next to us, but unless we have already glanced in that direction we will not know who it is.

The non-uniformity of the visual processing power is such that half our visual brain power is directed to processing less than 5 percent of the visual world. This is why we have to move our eyes; it is the only way we can get all that brain power directed where it will be most useful. Non-uniformity is also one of the key pieces of evidence showing that we do not comprehend the world all at once. We cannot possibly grasp it all at once since our nervous systems only process details in a tiny location at any one instant.

We only process details in the center of the visual field. We pick up information by directing our foveas using rapid eye movements.

At the edge of the visual field, we can barely see that someone is standing next to us.

The term brain pixel was introduced earlier by way of contrast with digital camera pixels. Brain pixels provide a kind of distorted neural map covering the whole visual field. There are a great many tiny ones processing information from central regions where we direct our gaze and only a few very large ones processing information at the edge of the visual field. Because of this, we cannot see much out of the corners of our eyes.

Strong eye muscles attached to each eyeball rotate it rapidly so that different parts of the visual world become imaged on the central high-resolution fovea. The muscles accelerate the eyeball to an angular velocity up to 900 degrees per second, then stop it, all in less than one-tenth of a second. This movement is called a saccade, and during a saccadic eye movement, vision is suppressed. The eyes move in a series of jerks pointing the fovea at interesting and useful locations, pausing briefly at each, before flicking to the next point of interest. Controlling these eye movements is a key part of the skill of seeing.

We do not see the world as jerky, nor for the most part are we aware of moving our eyes, and this adds yet more evidence that we do not perceive what is directly available through our visual sense.

This illustration is based on a slice through the head of the visible man at the level of the eyeballs. Its color has been altered to show the eye muscles and the eyeballs more clearly.

THE ACT OF PERCEPTION

The visual field has a big hole in it. Cover your left eye and look at the X. Move the page nearer and farther away, being sure to keep the X and the B horizontally aligned.

X

B

At some point the B should disappear. This is because the image of the B is falling on the blind spot, a region of the retina where there are no receptors at the point where the optic nerve and blood vessels enter the eye. We are unawa...