Face Recognition

The Effects of Race, Gender, Age and Species

James Tanaka, James Tanaka

• Publisher: Routledge
• Year: 2017
• ISBN:
  9781317650966
• Edition: 1
• Topic: Psychologie
• Subtopic: Histoire et théorie en psychologie

Although most people are good at face recognition, we are particularly good at recognizing the faces of individuals who share our race, gender, age and species. What factors might account for this type of bias in face recognition? This collection considers the issue of how our identity influences the type of perceptual experience that we have to faces, which, in turn, influences the processes of face recognition. Leading experts from cognitive psychology, neuroscience and computer science address a wide range of topics related to the neural and computational basis of the "own versus other" effect in face recognition, the impact of early experience in infant face recognition, the effect of laboratory training to reverse the other-race effect, cultural differences in expression recognition and the forensic and social consequences of "own versus other" face recognition. The combined work gives the reader a comprehensive overview of the field and an insider's perspective on the role that identity and experience play in the everyday process of face recognition.

This book was originally published as a special issue of Visual Cognition.

Neural perspectives on the other-race effect

Vaidehi Natu and Alice J. O’Toole

School of Behavioural and Brain Sciences, University of Texas at Dallas, Richardson, TX, USA

Psychological studies have long shown that human memory is superior for faces of our own-race than for faces of other-races. In this paper, we review neural studies of own- versus other-race face processing. These studies divide naturally into those focused on socioaffective aspects of the other-race effect and those directed at high-level visual processing differences. The socioaffective studies consider how subconscious bias and emotional responses affect brain areas such as the amygdala, anterior cingulate cortex, and parahippocampal gyrus. The visual studies focus on face-selective areas in the ventral stream, such as the fusiform face area (FFA). In both cases, factors such as experience, familiarity, social/emotional responses, cultural learning, and bias modulate the patterns of neural activity elicited in response to own- and other-race faces.

Over the last four decades, there have been hundreds of papers examining perceptual and memory components of the other-race effect with experimental behavioural approaches. This work began with the classic study of Malpass and Kravitz (1969). In perceptual terms, the other-race effect can be summed up with the adage, “They all look alike to me.” More formally, the phenomenon is characterized by a difficulty in differentiating among other-race faces, and thereby in perceiving the uniqueness of individual other-race faces. These perceptual problems give rise to difficulties in remembering individuals of other races. At the heart of these effects is one’s experience with people of other races. Social diversity (or lack thereof) sculpts face recognition capacity and ties perceptual and cognitive components of the effect to social and affective elements of face processing (Nelson, 2003; Quinn, Yarr, Kuhn, Slater, & Pascalis, 2002).

The purpose of the present paper is to consider the neural foundations of the other-race effect. In the context of the 40 years or more of psychological study of the topic, this perspective represents a relatively new approach. Indeed, when Malpass and Kravitz (1969) first reported evidence for the other-race effect, almost nothing was known about the neural processing of faces. Although neuropsychological case reports of prosopagnosia offered hints about the organization of face processing centres in the brain, few tools were available for more detailed study of unimpaired individuals. In 1969, functional neuroimaging methods such as positron emission tomography (PET) were not used for research, and functional magnetic resonance imaging (fMRI) had not yet been invented. These tools have been used widely in the last two decades to examine the complex neural network that underlies human face processing. The neural correlates of own- and other-race face processing make up a small, but expanding, part of this literature.

This review is organized as follows. The first section leads off with a brief introduction to the neural architecture of cortical regions that are functionally selective for faces. Next, we will review what is known about the neural correlates of own- and other-race face processing. The papers we consider can be divided into those focused on the neural processing of socioaffective responses to own- versus other-race faces and those concerned with visual processing differences. Throughout the paper, we emphasize the findings for understanding the challenges of creating representations that are shaped by experience with faces. Although the role of experience in the other-race effect for humans has been controversial, an understanding of the diversity of neural embodiments of experience may help to bridge theoretical gaps that are left open with behavioural approaches.

SYNOPSIS OF THE FACE PROCESSING SYSTEM

The neural architecture of the face processing system includes multiple regions in ventral temporal and superior temporal cortex that are “functionally selective” for faces. This selectivity is operationalized typically in functional neuroimaging experiments as a contrast between the strength of neural activity elicited in response to faces versus objects. There are three well-established face-selective areas in high-level visual cortex. The fusiform face area (FFA) is a small area on the lateral fusiform gyrus in the inferotemporal cortex (Hoffman & Haxby, 2000; Kanwisher, McDermott, & Chun, 1997; McCarthy, Puce, Gore, & Allison, 1997). The occipital face area (OFA) is in the occipital gyrus and likewise responds more vigorously to faces than to objects¹ (Halgren et al., 1999). Although some reports indicate that the FFA and OFA are localized more consistently in the right rather than left hemisphere, both of these areas are found bilaterally in many participants. The FFA and OFA are considered high-level visual regions in the ventral visual stream (DeYoe & Van Essen, 1988).

We digress briefly to note that although the face-selective properties of the FFA are well established, the exact nature of the specialized functions carried out by the FFA has been controversial (Gauthier, Tarr, Anderson, Skudlarski, & Gore, 1999; Grill-Spector, Knouf, & Kanwisher, 2004; Haxby et al., 2001; Kanwisher et al., 1997). The elements of this controversy reference a literature that is beyond the scope of this review, and so we simply state the three prominent perspectives. Kanwisher et al. (1997) originally posited the FFA as a modular face-specific processor. Haxby et al. (2001) argued for a representation of faces that includes the lateral fusiform gyrus, but is distributed more broadly across ventral temporal cortex. Gauthier et al. (1999) proposed that FFA is an area specialized for processing within-category object distinctions characteristic of expert processing. Despite the differences in perspectives, there is general agreement that the FFA is critical for processing facial identity from invariant feature- and structure-based information in the face. Moreover, a more detailed understanding of the function(s) of FFA is continually emerging in the literature as research techniques improve and as more data emerge on other brain structures important for face processing.

The role of OFA has been less controversial, with a recent review pointing to its role as a first stage processer for faces (Pitcher, Walsh, & Duchaine, 2011). This view is consistent with it responsiveness to face parts, including eyes, nose, and mouth (Pitcher, Walsh, Yovel, & Duchaine, 2007). Recent studies using transcranial magnetic stimulation (TMS) have indicated that OFA is critical for accurate face perception and that it plays its role at an early stage of processing (Pitcher et al., 2007).

In addition to the ventral temporal regions, an area in the posterior superior temporal sulcus (pSTS) responds strongly to facial movements, including expression, gaze, and facial speech (cf. Allison, Puce, & McCarthy, 2000; Hoffman & Haxby, 2000). This area is in the dorsal visual stream (DeYoe & Van Essen, 1988) and responds most consistently to moving faces (Fox, Iaria, & Barton, 2009; Pitcher, Dilks, Saxe, Triantafyllou, & Kanwisher, 2011).

An influential and now well-established neural framework for processing faces was proposed by Haxby et al. (2000). They posited a distributed neural network with two main subdivisions or “cores”, specialized to process the invariant and changeable aspects of faces, respectively. The former division includes the FFA and OFA in the ventral stream. These regions are thought to support the identification and categorization of faces. The latter division comprises face-selective regions in the pSTS in the dorsal visual stream and supports social communication functions through the analysis of facial motions (e.g., expression, gaze, facial speech).

In the Haxby et al. (2000) model, the core regions project to an extended system of other brain structures that utilize the results of the analysis. The FFA-OFA core projects to anterior temporal regions involved in coding personal identity, name, and biographical information. From the pSTS, the extended system includes connections to the intraparietal sulcus for spatially directed attention (e.g., from gaze), the auditory cortex for the processing of facial speech, and the amygdala, insula, and limbic system for the processing of emotional information from facial expressions.

The Haxby et al. (2000) system has been refined further to consider representations of familiar people (Gobbini & Haxby, 2007). Familiar person codes include semantic, episodic, and emotional associations with faces. Gobbini and Haxby (2007) also consider personal trait knowledge, mental state knowledge, and the assessment of intention as “theory of mind” processing. Although somewhat beyond the scope of the present paper, we sketch out very briefly the elements of the model as they may relate to the neural areas that activate in response to socioaffective components of own- and other race face processing.

According to the Gobbini and Haxby (2007) model, mental state evaluation is attributed to the anterior paracingulate cortex. Consistent with what is known about the neural areas involved in social cognition (cf. Allison et al., 2000; Haxby et al., 2000), intent judgements are assigned to pSTS and the temporal parietal junction (TPJ). The retrieval of biographical knowledge and related semantic knowledge activates areas in the anterior temporal cortex. Episodic memory retrieval is attributed to the precuneus and posterior cingulate areas. The emotional component of the extended face system is consistent with the model of Haxby et al. (2000) and includes the amygdala, insula, and the striatum (Gobbini & Haxby, 2007).

In neural studies of own- and other-race face processing, we will see that the focus is on the ventral core system for the visual studies and on the extended dorsal system (the amygdala, in particular) for the socioaffective studies.

NEURAL BASIS OF OWN- AND OTHER-RACE FACE PROCESSING

The architecture of the face processing system suggests a relatively stark division between the analysis of identity and the analysis of social/emotional information in the face. Neural studies of the other-race effect are likewise divided along these lines. We begin with fMRI studies of socioaffective differences in own- versus other-race face processing. These studies examine the neural basis of social and emotional behaviour associated with race and racial group memberships (Hart et al., 2000; Lieberman, Hariri, Jarcho, Eisenberger, & Bookheimer, 2005; Phelps et al., ). As such, they have focused on brain areas associated with emotion and conflict, and have characterized the relationship between neural activity and individual variability in the conscious and subconscious evaluation of race (e.g., Brosch, Bar-David, & Phelps, 2013; Phelps et al., 2000). Next, we will review visual processing differences for own- and other-race faces. These studies concentrate on understanding how visually based face recognition systems are affected by the race of a face (e.g., Feng et al., 2011; Golby, Gabrieli, Chiao, & Eberhardt, 2001; Natu, Raboy, & O’Toole, 2011). We have summarized the results of the socioaffective and visual processing studies in Tables 1 and 2, respectively.

SOCIOAFFECTIVE PROCESSING DIFFERENCES FOR OWN- VERSUS OTHER-RACE FACES

The rationale for focusing on the amygdala in socioaffective studies is that social bias and implicit negative processing for “other” racial groups should differentially modulate neural activity in brain regions associated with the encoding of emotional memories (e.g., Phelps, 2004). Consistent with the hypothesis that the amygdala responds to threat and novelty, Cunningham et al. (2004) found greater amygdala activation in response to other-race faces than to own-race faces, but only for participants who showed greater “implicit own-race bias”. Moreover, this amygdala activation difference occurred when faces were presented for a short-duration (30 ms), but not when they were presented with longer exposure times (525 ms). In fact, when Caucasian participants were given more time to process faces, differential neural activation was observed in the dorsolateral prefrontal cortex and anterior cingulate cortex, two areas associated with inhibition and conflict resolution. Cunningham et al. speculated that social memberships elicit an automatic emotional response associated with implicit racial bias, followed by an attempt to inhibit prejudice towards members of different racial groups.

Enhanced amygdala activation for other-race faces, however, has not been found consistently. Phelps et al. (2000) reported greater amygdala activation for other-race faces, but only when they were unfamiliar. Specifically, amygdala response to other-race faces varied as a function of social memberships and individual experience with faces. Phelps et al. found that measures of subconscious race-bias correlated with amygdala and anterior cingulate cortex response for unfamiliar own- versus unfamiliar other-race faces. No such differential response pattern was found for familiar faces. Phelps et al. conclude that familiarity with faces overrides social bias towards members of different races.

Lieberman et al. (2005) showed that amygdala responses varied across different encoding tasks related to processing racial information from faces. They did this by measuring differences in the amygdala response to African-American and Caucasian-American faces in both African-American and Caucasian-American participants. These measures were made while the participants performed perceptual and verbal encoding tasks. During the perceptual encoding task, participants matched faces based on race. During the verbal encoding task, participants judged whether the face presented was a Caucasian-American or an African-American face. In the perceptual encoding task, the amygdala activated more strongly to African-American than to the Caucasian-American faces, for all participants. In the verbal encoding task, Lieberman et al. found no differences. They interpreted the perceptual encoding results as a reflection of a culturally acquired social bias towards a racial group. The results also suggest that the mode of encoding race-based information (perceptual vs. verbal) may modulate the emotional responses of the amygdala.

In two other studies, differences in the amygdala activation to own- versus other-race faces vanished for tasks unrelated to race processing—even while participants viewed the faces (Brosch et al., 2013; Hart et al., 2000). In two separate scans, Hart et al. (2000) examined differences in the amygdala activation in African-American and Caucasian participants while they performed a gender categorization task for own- and other-race faces. Each face was presented on the screen for 1 s. During the first scan, the amygdala showed no differential activation to own- and other-race faces, inconsistent with the automated processing hypothesis (Cunningham et al., 2004). This suggests that during an unrelated task, the amygdala may be more responsive to the novelty of any face than to its race. In a second subsequent scan, Ha...