Perceptual Organization in Vision

Behavioral and Neural Perspectives

Ruth Kimchi, Marlene Behrmann, Carl R. Olson, Ruth Kimchi, Marlene Behrmann, Carl R. Olson

• Publisher: Psychology Press
• Year: 2003
• Print ISBN: 9780805838732
• eBook ISBN: 9781135647230
• Topic: Psychology
• Subtopic: Cognitive Psychology & Cognition
• Index: Psychology

This book presents an overview of different frameworks for understanding perceptual organization, and a state-of-the-art summary of the domain. It describes findings from visual search, illusory contours, and object recognition using electrophysiological measures.

I

Cognitive Approaches
to Perceptual Organization

1 Perceptual Organization
and Grouping

Stephen E. Palmer
University of California at Berkeley

The problem of perceptual organization is central to understanding vision. Its importance—and its difficulty—can perhaps be most easily appreciated by considering the output of the retinal mosaic at any moment in time as a numerical array in which each number represents the neural response of a single receptor. The main organizational problem faced by the visual nervous system is to determine the structure of the retinal image: What parts of this numerical array go together in the sense of corresponding to the same parts, objects, or groups of objects in the environment? These are such crucial issues in visual processing that it is hard to imagine how vision would be possible without organization.

Nevertheless, the topic of perceptual organization has had a rather uneven history. It was nearly unrecognized in the early days of perceptual inquiry. Gestalt psychologists first raised the organizational problem in its modern form during the first half of the 20th century. After the influence of Gestalt psychology diminished, however, organization in vision has received less attention than it deserves. This decline was precipitated in part by revolutionary developments in other areas of vision science, particularly single-cell recording techniques in primary visual cortex (e.g., Hubel & Wiesel, 1959) and the linear systems approach in visual psychophysics (e.g., Campbell & Robson, 1968). Vision scientists engaged in these programs of research largely ignored organizational issues until quite recently, apparently confident that organizational effects either did not matter or could be easily explained within the framework of their own theories.

Despite the abundance of knowledge uncovered by these research programs, it is now becoming evident that the theoretical constructs required to understand visual perception go beyond the limited vocabularies of linear systems and singlecell responses in area V1. Indeed, the next frontier of vision science will probably be to solve the problem of perceptual organization and to analyze its influences on other visual processes. Recent psychophysical work on organizational effects in basic visual processing has awakened new interest in a field that is now sometimes called mid-level vision (e.g., Adelson, 1993; Anderson, 1997; Gilchrist, Kossyfidis, Bonato, & Agostini, 1999; Nakayama & Shimojo, 1992). Organization is the key problem in mid-level vision, and it will not reveal its secrets unless we stop ignoring it in the vain hope that it will simply go away.

We should be clear, however, that solving the problem of perceptual organization will not be an easy task. If it were, we would surely have made more progress on it in the past 80 years than we have. Not only do we not have the answers yet, but it is not entirely clear what the proper questions are. I recently defined perceptual organization as “the processes by which the bits and pieces of visual information that are available in the retinal image are structured into the larger units of perceived objects and their interrelations” (Palmer, 1999, p. 255). This covers an enormous range of topics that can easily be construed to include almost all of vision. I will not attempt to address this entire domain but will focus instead on the more specific problem of visual grouping: determining what goes with what in the retinal image.

I begin with some general remarks about different kinds of theoretical approaches that one can take in understanding perceptual organization, with particular emphasis on grouping. These comments provide an historical backdrop not only for my own chapter but also for the contents of this entire volume. I then discuss different methodological approaches to studying grouping, with emphasis on a new paradigm that my colleagues and I recently developed. Next, I describe the evidence for some new principles of grouping that my colleagues and I identified using a variety of different methods. Then, I describe a series of experiments that we performed to discover the level of visual processing at which grouping is completed. I close by briefly considering the implications of these findings for my own theoretical views about the processes that underlie grouping and their relation to other processes of perceptual organization.

THEORETICAL APPROACHES

The theoretical issue I consider is how we are to understand organizational phenomena such as grouping. I briefly outline the four approaches that I find most important: structural, ecological, computational, and neural approaches. The first two concern the informational rationale for perceptual organization. The last two concern the metatheoretical level at which we seek explanations. I do not claim that these four approaches are mutually exclusive in any sense. Indeed, I will argue that they are largely compatible and complementary. Only by pursuing all four, as well as the relations among them, are we likely to reach a full and satisfying understanding of how visual perception is organized.

Structural Simplicity Approaches

Perhaps the most obvious theoretical approach to the problem of perceptual organization is the Gestalt approach, which is based on the notion of structural simplicity. Gestaltists believed that the key to understanding perceptual organization was to identify the kinds of structure in the retinal image to which the visual system was sensitive. Wertheimer’s (1923/1950) well-known laws of grouping, for example, are most easily understood in terms of the visual system being tuned to detect the sameness (or similarity) of perceptual elements in terms of certain salient properties, such as their location, color, size, orientation, motion, continuity, and so forth. Later theorists, such as Garner (1974), Leeuwenberg (1971), and Palmer (1983, 1991), have followed Gestaltists by analyzing the kinds of structural regularities to which the visual system is sensitive.

Why should structure be so important? The Gestalt answer was that structural simplicity was the driving force behind all of perception. They expressed this view in their famous principle of Prägnanz: Perception will be as “good” as the prevailing conditions allow. The prevailing conditions refer to stimulus constraints on the perceptual interpretation. In the case of vision, these constraints are provided by the structure of the retinal image. But because retinal constraints are not generally sufficient to uniquely solve the problem of perceiving the environment, Gestaltists proposed that additional constraints could be understood as arising from the maximization of “goodness” or structural simplicity—or, alternatively, the minimization of complexity. Wertheimer’s (1923/1950) original principles of grouping were very much along these lines, although he did not formulate them directly in terms of Prägnanz. The basic idea is that visual elements that are the same in color, motion, size, and so forth, are seen as grouped together because this organized perception is simpler than the alternative of seeing them as unorganized, independent elements. No further justification is required: These regularities simply are the particular kinds of stimulus structure to which the visual system is sensitive as a result of its underlying physiological mechanisms, whatever those might be.

Gestaltists never actually produced a well-defined theory of Prägnanz, however. A satisfying version of it was proposed several decades later by Dutch psychologist Emanuel Leeuwenberg in his coding theory of perception (Leeuwenberg, 1971), later renamed structural information theory (Buffart & Leeuwenberg, 1983; Van der Helm & Leeuwenberg, 1991). Leeuwenberg’s theory provided (a) a language for describing patterns, (b) a set of rules for simplifying these pattern descriptions by eliminating structural regularities, and (c) a metric for measuring the complexity of pattern descriptions based on the number of free parameters they contained. Together, these principles are able to predict, with reasonable success, the probabilities that people will see different interpretations of the image over a range of different tasks and conditions.

There are other computational theories that fit within the mold of structural simplicity. Many are couched in the language of minimization solutions, such as minimizing energy or minimizing the length of a symbolic description. The chapter by Lee in this volume discusses some of these approaches. Minimization is a general approach that can be used to solve underconstrained problems in the visual domain, such as completion of partly occluded surfaces and depth from texture. The basic idea is that when many different codings or interpretations are possible for the same sensory data, the visual system will pick the one that minimizes some relevant cost or energy function. Kellman and Shipley (1991), for example, claimed that amodally completed contours are constructed by finding the completion that will connect the visible edges of the partly occluded surface with the minimum curvature. In a very different application, Horn (1975) proposed a computational solution to the problem of depth from shading that involved finding the surface of minimum energy that would satisfy the shading constraints in the image. These approaches are related to Gestalt notions of structural simplicity but not as directly as Leeuwenberg’s coding theory.

Another modern computational approach that falls within the family of structural simplicity theories is minimal description length (MDL) coding (e.g., Rissanen, 1978). The underlying idea of the MDL approach to visual structure comes from two main sources. One is the conjecture, often attributed to Barlow (1961), Attneave (1954), or both, that the visual system recodes visual information optimally by eliminating redundancy in the structure of visual input. The other source of MDL theories is the mathematical theory of complexity, pioneered by Kolmogorov and Chaitin. Within this framework, the complexity of a given object or event can be measured by the length of the program required to generate it. Although this idea was initially applied to symbol strings (such as the 1s and 0s of numbers expressed in binary notation), it can be formulated to apply to twodimensional image structures as well. The hypothesis is that the visual system recodes image information into the shortest possible description.

Ecological Approaches

Structural simplicity theories of perceptual organization provide explanations of at least some kinds of perceptual organization. But they leave unanswered the important question of why the visual system is sensitive to those particular kinds of structure. In some sense, the most obvious answer is an ecological one: This sensitivity is helpful to the organism in discovering the structure of the external world. Perhaps the most important task of perceptual grouping is to find out which parts of the projected image belong to the same environmental objects. It seems that there are ...

Table of contents

• COVER PAGE
• TITLE PAGE
• COPYRIGHT PAGE
• PREFACE
• I. COGNITIVE APPROACHES TO PERCEPTUAL ORGANIZATION
• II. DEVELOPMENT AND LEARNING IN PERCEPTUAL ORGANIZATION
• III. NEURAL APPROACHES TO PERCEPTUAL ORGANIZATION
• IV. COMPUTATIONAL APPROACHES TO PERCEPTUAL ORGANIZATION