Cognitive and Computational Aspects of Face Recognition

Explorations in Face Space

Tim Valentine, Tim Valentine

• Publisher: Routledge
• Year: 2017
• ISBN:
  9781315516998
• Edition: 1
• Topic: Psychologie
• Subtopic: Geschichte & Theorie in der Psychologie

How can computers recognize faces? Why are caricatures of famous faces so easily recognized?

Originally published in 1995, much of the previous research on face recognition had been phenomena driven. Recent empirical work together with the application of computational, mathematical and statistical techniques have provided new ways of conceptualizing the information available in faces. These advances have led researchers to suggest that many phenomena can be explained by the structure of the information available in the population(s) of faces. This broad approach has drawn together a number of apparently disparate phenomena with a common theoretical basis, including cross-race recognition; the distinctiveness of faces; the production and recognition of caricatures; and the determinants of facial attractiveness. This title provides a state of the art review of the field at the time in which the authors use a wide variety of approaches. What is common to all is that the authors base the accounts of the phenomena they study or their model of face recognition on the statistics of the information available in the population of faces.

On publication this title was a comprehensive, up-to-date review of an important area of research in face recognition written by active researchers. It includes contributions from mathematics, computer science and neural network theory as well as psychology. It is aimed at research workers and postgraduate students and will be of interest to cognitive psychologists and computer scientists interested in face recognition. It will also be of interest to those working on neural network models of visual recognition, perceptual development, expertise in visual cognition as well as facial attractiveness and caricature.

Chapter 1

The development of face recognition

Robert A. Johnston and Hadyn D. Ellis

Infants may arrive into what James (1890) described as ‘booming, buzzing confusion’ but it is apparent that from a very early age they are able to respond differentially to faces. It has been demonstrated that babies aged less than 10 minutes old have a preference for following a moving schematic face rather than a blank head shape or one containing facial features in a jumbled configuration (Goren et al., 1975; Maurer and Young, 1983; Johnston et al., 1992). While there is still debate over exactly to what the neonate is responding, a face per se or some more fundamental sensory characteristic of the stimulus (e.g. phase, amplitude, etc.), the early preferential orientation towards such objects is not in doubt. It is clear, however, despite evidence of this early sensitivity, that children’s skill and efficiency in processing faces continue to improve throughout the course of childhood. Indeed, over the next forty-eight hours neonates learn to recognize their mother’s face – a quite remarkable cognitive feat for such an immature nervous system (Bushnell et al., 1989).

The earliest systematic studies of the later development of face recognition abilities were carried out by Goldstein and Chance (1964) on schoolchildren of different ages. Their initial experiments required children of 5, 8 and 13 years of age to perform a recognition memory task. Children were first shown a set of unfamiliar faces and then at a later stage were required to select these previously encountered faces from a larger set of unfamiliar faces. Performance in this task improved steadily across the three age groups tested.

Ellis (1991) has also shown that children below the age of 11 years find it particularly difficult to deal with simple age transformations or expression transformations of faces that would be trivial for adults. Children aged from 3–11 years were required to match simultaneous presentations of faces which differed in expression (e.g. smile, grimace or surprise): the performance of the children showed a clear developmental trend. Three year olds performed the worst and 11 year olds the best.

There is some disagreement when the performance of children reaches adult levels. Feinman and Entwistle (1976) reported that after 11 years of age subjects show little improvement in performance. Other workers, however, have shown a dip in performance at around 12 years of age (e.g. Carey et al., 1980; Flin, 1980). This dip in performance has been ascribed to a change in the processing strategies of children when recognizing faces (Carey, 1981). Below this age the processing of faces by children is assumed to be dominated by featural or piecemeal strategies rather than the more predominately configural recognition strategies customarily employed by adults. This idea is supported by a whole battery of experiments which appear to show that young children are particularly influenced by salient aspects of facial stimuli. Different faces may be erroneously judged to be the same if they share the same expression, if they possess the same piece of paraphernalia or if they are subjected to the same size transformations (Diamond and Carey, 1977; Flin, 1980; Ellis, 1990; 1991). This developmental inflection has been observed in the processing of other stimuli (e.g. voices: see Mann et al., 1979), which suggests that a more general developmental change may underlie any inflection in face processing.

There may be several explanations for the phenomena we outline above. Nevertheless, it is clear that strong evidence exists to demonstrate that a child’s ability to recognize faces continues to develop steadily across the school age years despite any early, probably innate, attunement to faces.

One potential explanation we would like to explore in the course of this chapter proposes that these effects mainly arise through an increasing ability to discriminate among faces. This may be due either to the way that faces are encoded when they are encountered or because of the way that representations of faces are stored in memory – indeed these options may be impossible to disentangle. It would not be surprising if to an untutored (or alien) eye human faces all looked identical. They all have the same component features, which are juxtaposed in more or less the same arrangement. The differences in the way these components are laid out are so small as to be considered insignificant if we were considering exemplars of another similarly homogeneous category (e.g. markings of dogs). The adult ability to discriminate among faces has been described as the acme of human visual perception (Ellis, 1981). Underlying this skilful perceptual ability must be an appropriately refined and discriminating process for storing representations of faces in memory – perhaps this ability is not present to the same level of sophistication in young children and only develops over time.

Ellis et al. (in preparation) have attempted to examine experimentally anecdotal reports of children’s tendency falsely to recognize strangers who are similar to familiar people. Parents of young children are often all too embarrassingly aware that their offspring make category-inclusive errors of facial recognition. If Uncle John has a round face and a bald head other men of similar appearance may be mistakenly identified as being him by pre-school children. This is an error typical of young children rather than adults. A diary study conducted by Young et al. (1985) analysed almost a thousand misidentifications which were made by adults when attempting to recognize people – very few of these errors were false positive identifications based on spurious facial similarity between the known person and the stranger. Keil (1987) provides a possible theoretical analysis of the way children of different ages make categorical judgements. He claims that younger children use wider boundaries and hence include more items within a category. As a child matures the critical, defining features for category membership become sharpened so that fewer objects are included. Ellis et al. (in preparation) propose a particular familiar face as a category and examine the preparedness of young children to accept other faces as versions of the target face (i.e. members of that category). In an experimental test of this hypothesis, distracter faces were selected to be either very similar or very dissimilar to one of two target faces. Ellis et al. examined identification performance across a range of ages (5, 8, 11 and 19 years) by looking at response latencies and accuracy for decisions as to whether a stimulus was or was not a specified famous person. As would be expected, accuracy was positively correlated with age and the time to make responses was negatively correlated with age. The interesting aspect of the results, however, relates to performance with similar and dissimilar distracters. While all subject groups produced similar error rates in classifying dissimilar distracters, the youngest group showed a unique difficulty with rejecting similar distracters, a tendency not shown by the other groups. When examining response latencies it was noted that the child groups were slower to reject similar distracters than dissimilar ones, but the 19-year-old group shows no difference in response times to reject either type of lure. Can we provide an explanation for these effects predicated on the manner in which faces are represented in memory for different age groups?

Let us first consider how adult faces may be stored in memory. Valentine has suggested that a useful heuristic for understanding how this is achieved is to view the adult face space as a multidimensional space (Valentine, 1991a; 1991b). Facial representations in memory can be viewed as locations within this multidimensional space. So far it has not been possible to specify the dimensions of this space but it would not be unreasonable to assume that they will be based on those that would best serve to discriminate among faces. Indeed there are many reasonable candidates which include such feature dimensions as face shape, hair length, hair colour, or perceived age derived from multidimensional scaling studies (e.g. see Shepherd et al., 1977).

The origin of the multidimensional face space will be the central tendency of the dimensions and it is assumed that the feature dimensions of faces experienced will vary normally around this point. Typical faces, by definition, are more often experienced than distinctive faces and so the density of points throughout this space will not be uniform (see Figure 1.1a). There will be a higher density of face representations around the central tendency (i.e. the region where representations of typical faces are located). The putative framework which Valentine has suggested allows us neatly to account for many effects described in the face recognition literature (e.g. distinctiveness vs. typicality effects, inversion phenomena and face classification effects).

Using this theoretical framework, Valentine has identified two specific models based on the multidimensional nature of face space. One version he describes as being a norm-based model (however, this label is intended to cover a variety of similar theoretical constructs). Specifically, the theoretical approaches included are the prototype hypothesis (Valentine and Bruce, 1986a; 1986b), the norm-based coding model (Rhodes et al., 1987), and the schema theory (Goldstein and Chance, 1980). All these accounts assume that storing representations of faces in memory entails the abstraction of something that can be called a face norm, prototype or schema. The norm-based model proposes that each individual face is stored in memory according to its deviation from a single, general face norm or prototype. This would be located at the origin of the face space. For an n-dimensional face space, an n-dimensional vector from the origin to a point representing the dimension values of a face would uniquely specify that face. The process of recognizing a face involves encoding the stimulus face as an n-dimensional vector and deciding if the resultant stimulus matches the stored vector of a face already encountered.

This is not the only way in which a multidimensional space could be described of course; for example, alternative formulations may not require a prototype face: instead these could be based on inter-stimulus similarity. In fact Valentine (1991a) has described such versions which he groups under the heading of exemplar-based models. It is appropriate in these models to think of faces as being encoded as points rather than vectors. Here, the origin of the space plays no role in encoding stimuli, it is simply the area of greatest exemplar density.

Valentine (1991a; 1991b) has shown how each of these models (norm-based and exemplar-based) can predict a particularly robust effect in the face recognition literature. Research employing faces in recognition memory experiments has shown that subjects are better at recognizing distinctive faces. A typical experiment would involve showing subjects a set of unfamiliar faces and subsequently asking them to identify these stimuli later when given a larger selection of unfamiliar faces. It has been demonstrated that subjects perform at a superior level with distinctive faces compared with typical faces. This differential performance can be demonstrated in a number of different ways. These include a higher recognition hit rate for previously seen faces if they are distinctive; and also faster recognition latencies. The distinctiveness advantage is also exhibited by the occurrence of fewer false positive decisions to distinctive distracter faces. An overall advantage for the recognition of distinctive as opposed to typical faces can often be demonstrated using a measure of sensitivity such as d’ or A’ ( Going and Read, 1974; Cohen and Carr, 1975; Light et al., 1979; Winograd, 1981; Bartlett et al., 1984; Valentine, 1991a; 1991b; Shepherd et al., 1991). Moreover, Ellis et al. (1989) demonstrated that, although when subjects searched for a target face using FRAME (a computerized mugshot retrieval system) they were as good at retrieving typical faces as distinctive faces; yet when other subjects used a traditional mugshot album to recognize faces there was an enormous advantage for distinctive faces, particularly when they occurred later in a series of 1000 mugshots.

There is some variation in how the above mentioned researchers refer to the faces we label as either distinctive or typical. Some share our nomenclature, while others instead use the labels ‘memorableness’, ‘uniqueness’ or ‘unusualness’. Vokey and Read (1992) even approach the dimension from the other direction and talk of typicality of faces where distinctive faces are considered atypical. Nevertheless, all researchers mean this to describe the range of variation present in ordinary faces. We do not intend the appellation ‘distinctive’ to conjure an image of a face that is deformed or has one eye or a huge scar.

As we mentioned earlier in this discussion, various workers have shown that the ability to discriminate novel faces from ones already encountered is a skill which improves steadily with age (e.g. Goldstein and Chance, 1964; Flin, 1980). However, compared to the abundance of work which has looked at this task with adult subjects when stimuli are controlled for distinctiveness, there is little research available on the performance of children. Ellis (1992) described some preliminary work on school-age children which suggested that the characteristic adult advantage for distinctive faces was absent in children aged around 6 years of age. He suggested that young children either fail to encode those aspects which make faces distinctive or that they store both typical and distinctive faces in the same manner. Ellis also demonstrated that even by the age of 13 years, subjects were not able to discount distinctive distracter faces more easily than typical distracters. In research employing adult subjects this advantage is a robust effect (e.g. Bartlett et al., 1984; Valentine and Endo, 1992), which suggests that the adult level of performance occurs after puberty.

Valentine’s multidimensional face space can readily accommodate the distinctiveness advantage shown by adults in recognition memory experiments. According to either norm-based or exemplar-based models, typical faces are located in the areas of highest exemplar density: so, when a typical face is presented for test it is more likely to resemble another face, which produces a situation with greater uncertainty and hence more opportunity for error: it will be reflected both in longer response latencies to typical faces and in more false positives. Is it possible that this framework can also account for effects observed with child subjects?

In order to answer this question we need to speculate on how the way faces are represented in children’s memory may differ from the adult arrangement. It is scarcely controversial to suggest that, in some way, a child’s face space would be smaller than an adult’s. This claim can be sensibly made on the basis that children have simply seen, and hence represented, fewer faces than adults. What is more contentious, however, is how this ‘smallness’ might be manifested.

One version is that the face space has the same general framework and parameters of the adult space but that it is less densely populated (Figure 1.1b) than the adult space (Figure 1.1a) and so, consequently, the density gradient of the space may be attenuated. Alternatively, the space might be based simply on a smaller volume than the adult space (Figure 1.1c). In this arrangement, relative difference in the density of face space enjoyed by typical and distinctive faces are preserved and, as the child experiences more faces, the volume of the face space increases to accommodate the additions. It is proposed that the space presented in Figure 1.1c has fewer dimensions than the adult space. (The face spaces displayed in Figure 1.1...