Our Biometric Future

Facial Recognition Technology and the Culture of Surveillance

Kelly Gates

• Publisher: NYU Press
• Year: 2011
• Print ISBN: 9780814732106
• eBook ISBN: 9780814732793
• Topic: Social Sciences
• Subtopic: Media Studies
• Index: Social Sciences

Since the 1960s, a significant effort has been underway to program computers to “see” the human face—to develop automated systems for identifying faces and distinguishing them from one another—commonly known as Facial Recognition Technology. While computer scientists are developing FRT in order to design more intelligent and interactive machines, businesses and states agencies view the technology as uniquely suited for “smart” surveillance—systems that automate the labor of monitoring in order to increase their efficacy and spread their reach.
Tracking this technological pursuit, Our Biometric Future identifies FRT as a prime example of the failed technocratic approach to governance, where new technologies are pursued as shortsighted solutions to complex social problems. Culling news stories, press releases, policy statements, PR kits and other materials, Kelly Gates provides evidence that, instead of providing more security for more people, the pursuit of FRT is being driven by the priorities of corporations, law enforcement and state security agencies, all convinced of the technology's necessity and unhindered by its complicated and potentially destructive social consequences. By focusing on the politics of developing and deploying these technologies, Our Biometric Future argues not for the inevitability of a particular technological future, but for its profound contingency and contestability.

1

Facial Recognition Technology from the Lab to the Marketplace

At the 1970s World’s Fair in Osaka, Japan, the Nippon Electric Company (NEC) staged an attraction called “Computer Physiognomy.” Visitors to the exhibit would sit in front of a television camera to have their pictures taken and then fed into a computer where a simple program would extract lines from the images and locate several feature points on their faces. In one last step, the program would classify the faces into one of seven categories, each corresponding to a famous person. As the computer scientist Takeo Kanade wrote in his doctoral dissertation, “the program was not very reliable,” but “the attraction itself was very successful,” drawing hundreds, probably thousands, of people to have their faces scanned and categorized in this new, state-of-the-art manner.¹

NEC’s largely forgotten World’s Fair attraction was part of the highly successful Osaka Expo ‘70 that showcased Japan’s postwar technological accomplishments and rapid economic growth.² The exhibit, like other attractions at World’s Fairs before and after, clearly aimed to marry technology with amusement, offering an “almost visceral pleasure. . . to sell the comparatively cold, abstract, and at times unappealing accomplishments of the technological.”³ It also married the archaic with the futuristic. It would have been naïve to believe that the program actually produced scientific results, since physiognomic analysis had long since been discredited as pseudoscience. But the experience of having one’s face analyzed by a computer program foretold a future in which intelligent machines would see the human face and make sense of it in a technologically advanced way. As Langdon Winner has observed, World’s Fairs have historically offered a vision of the technological future as an already accomplished fact, giving visitors a sense of being propelled forward by forces larger than themselves. “There were no pavilions to solicit the public’s suggestions about emerging devices, systems, or role definitions,” instead promoting a model of closed, corporate-sponsored research and development of which ordinary people need only be in awe.⁴ It is no stretch to suggest that the NEC Computer Physiognomy attraction conveyed this implicit message. One can imagine that in 1970, the experience of having a computer scan and analyze one’s own face might have impressed on the sitter a powerful sense of her own future subjection to information processing machines. At the very least, it demonstrated one of the many uses for which computers would be put: sorting human beings into distinct social classifications. It was hardly a new idea, but the computer seemed to give it a whole new edge.

The Computer Physiognomy exhibit also had a more direct use value, however, yielding a database of facial images for budding research on the problem of computer face recognition, images of “faces young and old, males and females, with glasses and hats, and faces with a turn, tilt or inclination to a slight degree.”⁵ Kanade used 688 of the photographs for his dissertation research in electrical engineering at Kyoto University in the early seventies, which built on early work in pattern recognition and picture processing to devise a facial feature extraction program. (Kanade would eventually become the head of the Robotics Institute at Carnegie Mellon University.) Many of the civilian and academic computer scientists working on the problem of computer image processing in these early days were motivated by the prospect of creating more intelligent machines, part of dedicated research teams working on the “quest” to create artificial intelligence.⁶ The earliest research on computer recognition of human faces was one small part of the research programs in computer vision and robotics, themselves arms of artificial intelligence research.⁷ In the United States, computer scientists working in the area of artificial intelligence made advances in computing technology—in computer vision as well as speech recognition and other areas—thanks in no small part to an enormous amount of military funding in the postwar period.⁸

But computer scientists and military strategists were not the only social actors with an interest in the technology. Computerized face recognition and other forms of automated bodily identification soon came to the attention of other social actors in both the public and private sectors who were recognizing an intensified need to “compensate for lost presences”—to deal with the problem of a proliferation of disembodied identities residing in databases and circulating over networks.⁹ The late 1960s was the beginning of the expansion of computer networking in the business and government sectors, and businesses like NEC saw even more expansive growth ahead.¹⁰ Indeed, electronics manufacturers, telecommunications companies, and other businesses were about to invest considerable effort into making it happen. As Dan Schiller has documented in painstaking detail, the big business users of expanding global computer networks spent the last three decades of the twentieth century laying proprietary claim to the network infrastructure, jettisoning many of the public service tenets that had earlier served to regulate telecommunications system development.¹¹ This “neoliberal networking drive” would require sweeping changes in telecommunications policy, to be sure. It would also require new divisions of labor among humans and computers, and new technologies of identification better suited to the demands of “network security.” In turn, biometric technologies would be envisioned and designed to meet the demand not only for more secure computer networks but also for more intensified, automated forms of surveillance, access control, and identification-at-a-distance in a wide range of settings.

This chapter examines how automated facial recognition and related technologies were envisioned and designed to serve a set of institutional priorities during the period of political-economic neoliberalization in the United States. Neoliberalism has not been an entirely unified program so much as an ad hoc set of governmental experiments that favor privatization, free-market principles, individualism, and “government at a distance” from the state system.¹² It has involved new ways of allocating government among state and non-state actors, a configuration that began to take shape in the United States and other developed nations in the 1960s and 1970s and gathered momentum in the 1980s and 1990s—precisely the period during which computerized forms of facial recognition and expression analysis became a possibility.¹³ In the United States, the effort to program computers to identify human faces began in the 1960s in research labs funded by the Department of Defense and intelligence agencies. By the 1990s, new companies were formed to commercialize the technology, searching for markets especially among institutions operating proprietary computer networks (like the finance industry and other business sectors) and large-scale identification systems (like passport agencies, state Department of Motor Vehicle offices, law enforcement, and penal systems). Across these sectors, biometric identification promised to enable what Nikolas Rose has referred to as the “securitization of identity,” the intensification of identification practices at a proliferation of sites—a priority that has gone hand in hand with political-economic and governmental neoliberalization.¹⁴

A close look at the early commercialization of facial recognition and other biometric technologies suggests that the turn to biometric identification at this particular juncture should not be understood as the inevitable result of seemingly inherent inclinations of the nation-state toward sovereign control of populations and territory, fueled by advancements in computer science and visual media technologies. Certainly technological advances and the persistence of ideas about state-centered forms of political power play their part. But the transition to biometric identification must likewise be understood as a response to a set of conflicting demands of both the state and the business system to individualize and to classify, to include and to exclude, to protect and to punish, to monitor and define parameters, and to otherwise govern populations in the face of their radical destabilization under the wrenching neoliberal reforms instituted in the United States and across the globe during the latter part of the twentieth and early twenty-first centuries. Building on existing communication and identification practices that gave pride of place to the face, facial recognition technology promised to play a unique role in this process. Although posing formidable technical and logistical challenges, functioning facial recognition systems promised to build on existing identification infrastructures to refashion face-to-face relations in networked environments, appropriating face-to-face forms of trust and recognition for official identification in mediated contexts. The primary aim of these systems would be to deliver concrete, practical benefits in the form of more accurate, effective, ubiquitous systems of facial identification that operated automatically, in real time, and at a distance. Although they seemed to simply recognize people the way most humans do in their everyday lives—by looking at one another’s faces—facial recognition systems in fact promised to enable more effective institutional and administrative forms of identification, social classification, and control.

Early Research on Facial Recognition Technology

Since its beginnings, research into computerized facial recognition in the United States has been a combined public-private venture funded and shaped to a significant extent by military priorities, a fact that makes it far from unique. Some of the earliest research on machine recognition of faces can be traced back to the 1960s at a private company called Panoramic Research Inc. in Palo Alto, California, one of many companies started up in the United States, post-Sputnik, to conduct government-funded research in computer science. The work at Panoramic Research was funded largely by the U.S. Department of Defense and various intelligence agencies, and so was unavoidably entrenched in the struggle for Cold War technological superiority. Although not all the computer scientists working in the postwar context were steadfastly dedicated to fulfilling military needs, they nevertheless had to emphasize the applicability of their work to Cold War priorities in order to secure funding from the Defense Advanced Research Projects Agency (DARPA) and its Information Processing Techniques Office (IPTO).¹⁵ Automated facial recognition in particular might eventually help the military identify, at a distance, specific individuals among the enemy ranks, in this way contributing to what Paul Virilio calls a “logistics of military perception.”¹⁶

Aside from the potential military applications of the technology, scientists working on early efforts to simulate face perception in computer programs were not doing so in response to immediate or well-defined social needs. In a manuscript of his dissertation published in 1977, Takeo Kanade speculated that the techniques of picture processing to which his research contributed might lend themselves to “sophisticated applications such as interpretation of biomedical images and X-ray films, measurement of images in nuclear physics, processing of a large volume of pictorial data sent from satellites, etc.”¹⁷ The techniques being developed for computer recognition of faces promised to address a set of vaguely defined problems concerning how to automatically process images and handle an expanding volume of visual information in medicine, science, and military intelligence. More broadly, early research and development of computerized facial recognition was part of a general effort to program computers to do what humans could do, or, better, what humans were incapable of doing. For some computer scientists and engineers conducting work on pattern recognition, as Manuel De Landa has noted, “the idea was not to transfer human skills to a machine, but to integrate humans and machines so that the intellectual skills of the former could be amplified by the latter.”¹⁸

This would be no simple undertaking. A look at the early work on machine recognition of faces at Panoramic Research underscores the formidable challenges ahead of computer scientists interested in creating computer programs that could identify faces in images, and the significant amount of human effort that would be required. Woodrow Wilson Bledsoe, one of the cofounders of Panoramic, headed up the research. Bledsoe is now recognized as a pioneer in the field of “automated reasoning” or automatic theorem proving, an arm of early artificial intelligence research. A member of the Army Corps of Engineers during WWII and a devout Mormon, he was a firm believer in incremental scientific advances as opposed to major leaps or paradigm shifts.¹⁹ Drawing on his research into computer recognition of letters, Bledsoe’s technique involved manually entering into a computer the positions of feature points in an image, a process known as “feature extraction.” A human operator would use a “rand tablet” to extract the coordinates of features such as the corners of the eyes and mouth, the top of the nose, and the hairline or point of a widow’s peak.²⁰ The name of the person in an image was stored in a database along with facial coordinates, and records were organized based on those measurements. The computer was then prompted to identify the name of the closest test image, given a set of distances between facial feature points. Bledsoe’s technique was labeled a “hybrid man-machine system” because a human operator was centrally involved in the process of extracting facial coordinates from the images.²¹ In addition to relying significantly on human intervention, Bledsoe’s man-machine program brought to light the fundamental difficulty that computer programs would have with facial image variability, especially in terms of “head rotation and tilt, lighting intensity and angle, facial expression, aging, etc.,” thus introducing the need for techniques of image “normalization,” or the manipulation of facial images to correct for angle, lighting, and other differences that confounded the matching process.²²

Following on the heels of this early research, computer scientists made small advances at programming computers to recognize human faces in images in the early 1970s. A very basic problem that proved difficult was programming a computer to simply locate a face in an image, especially if the image was visually cluttered, if faces were not represented in frontal view, or if faces were occluded with beards, hats, eyeglasses, or other objects. The earliest work to successfully program a computer to confirm the existence or absence of a face in an image, without human operator intervention, was conducted by three Japanese computer scientists and published in the journal Pattern Recognition in 1969.²³ Then in 1970, a doctoral student produced a landmark dissertation project at Stanford.²⁴ His technique enabled the computer to automatically extract the head and body outlines from an image and then locate the eyes, nose, and mouth, using three images of each individual: an image of the body, an image of the background without the body, and a close-up image of the head. In 1973, Takeo Kanade’s dissertation at Kyoto University in Japan made another significant contribution, reporting the same results as the Stanford research using only photographs of the face and a new “flexible picture analysis scheme with feedback,” consisting of a collection of simple “subroutines,” each of which worked on a specific part of the picture.²⁵ The recognition phase of Kanade’s project, which focused on the problem of automated extraction of face features, correctly identified fifteen out of twenty people in collections of forty photographs.²⁶

Experiments were not always designed with the goal of eliminating humans entirely from the process of recognizing faces. Instead, researchers had in mind creating synergistic relationships between human brains and computers. In 1971, scientists at Bell Labs worked on a system designed to locate and rank order a set of facial images from a file population based on a verbal description inputted into a computer.²⁷ In their report, titled “Man-Machine Interaction in Human-Face Identification,” they explained their aim “to design algorithms for optimizing the man-machine system so that we can take advantage of both the human’s superiority in detecting noteworthy features and the machine’s superiority in making decisions based on accurate knowledge of population statistics.”²⁸ The research had the more limited goal of producing a “population-reduction” system, a means of reducing large numbers of images to a quantity more manageable fo...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Dedication
• Contents
• List of Abbreviations
• Acknowledgments
• Introduction: Experimenting with the Face
• 1 Facial Recognition Technology from the Lab to the Marketplace
• 2 Police Power and the Smart CCTV Experiment
• 3 Finding the Face of Terror in Data
• 4 Inventing the Security-Conscious, Tech-Savvy Citizen
• 5 Automated Facial Expression Analysis and the Mobilization of Affect
• Conclusion
• Notes
• Bibliography
• Index
• About the Author