3D Face Modeling, Analysis and Recognition

Mohamed Daoudi, Anuj Srivastava, Remco Veltkamp

• Publisher: Wiley
• Year: 2013
• Print ISBN: 9780470666418
• eBook ISBN: 9781118592632
• Edition: 1
• Topic: Technology & Engineering
• Subtopic: Signals & Signal Processing
• Index: Technology & Engineering

3D Face Modeling, Analysis and Recognition presents methodologies for analyzing shapes of facial surfaces, develops computational tools for analyzing 3D face data, and illustrates them using state-of-the-art applications. The methodologies chosen are based on efficient representations, metrics, comparisons, and classifications of features that are especially relevant in the context of 3D measurements of human faces. These frameworks have a long-term utility in face analysis, taking into account the anticipated improvements in data collection, data storage, processing speeds, and application scenarios expected as the discipline develops further.

The book covers face acquisition through 3D scanners and 3D face pre-processing, before examining the three main approaches for 3D facial surface analysis and recognition: facial curves; facial surface features; and 3D morphable models. Whilst the focus of these chapters is fundamentals and methodologies, the algorithms provided are tested on facial biometric data, thereby continually showing how the methods can be applied.

Key features:
• Explores the underlying mathematics and will apply these mathematical techniques to 3D face analysis and recognition
• Provides coverage of a wide range of applications including biometrics, forensic applications, facial expression analysis, and model fitting to 2D images
• Contains numerous exercises and algorithms throughout the book

1

3D Face Modeling

Boulbaba Ben Amor,1 Mohsen Ardabilian,2 and Liming Chen2

1Institut Mines-Télécom/Télécom Lille 1, France

2Ecole Centrale de Lyon, France

Acquiring, modeling, and synthesizing realistic 3D human faces and their dynamics have emerged as an active research topic in the border area between the computer vision and computer graphics fields of research. This has resulted in a plethora of different acquisition systems and processing pipelines that share many fundamental concepts as well as specific implementation details. The research community has investigated the possibility of targeting either end-to-end consumer-level or professional-level applications, such as facial geometry acquisition for 3D-based biometrics and its dynamics capturing for expression cloning or performance capture and, more recently, for 4D expression analysis and recognition. Despite the rich literature, reproducing realistic human faces remains a distant goal because the challenges that face 3D face modeling are still open. These challenges include the motion speed of the face when conveying expressions, the variabilities in lighting conditions, and pose. In addition, human beings are very sensitive to facial appearance and quickly sense any anomalies in 3D geometry or dynamics of faces. The techniques developed in this field attempt to recover facial 3D shapes from camera(s) and reproduce their actions. Consequently, they seek to answer the following questions:

• How can one recover the facial shapes under pose and illumination variations?
• How can one synthesize realistic dynamics from the obtained 3D shape sequences?

This chapter provides a brief overview of the most successful existing methods in the literature by first introducing basics and background material essential to understand them. To this end, instead of the classical passive/active taxonomy of 3D reconstruction techniques, we propose here to categorize approaches according to whether they are able to acquire faces in action or they can only capture them in a static state. Thus, this chapter is preliminary to the following chapters that use static or dynamic facial data for face analysis, recognition, and expression recognition.

1.1 Challenges and Taxonomy of Techniques

Capturing and processing human geometry is at the core of several applications. To work on 3D faces, one must first be able to recover their shapes. In the literature, several acquisition techniques exist that are either dedicated to specific objects or are general. Usually accompanied by geometric modeling tools and post-processing of 3D entities (3D point clouds, 3D mesh, volume, etc.), these techniques provide complete solutions for 3D full object reconstruction. The acquisition quality is mainly linked to the accuracy of recovering the z-coordinate (called depth information). It is characterized by loyalty reconstruction, in other words, by data quality, the density of 3D face models, details preservation (regions showing changes in shapes), etc. Other important criteria are the acquisition time, the ease of use, and the sensor’s cost. In what follows, we report the main extrinsic and intrinsic factors which could influence the modeling process.

• Extrinsic factors. They are related to the environmental conditions of the acquisition and the face itself. In fact, human faces are globally similar in terms of the position of main features (eyes, mouth, nose, etc.), but can vary considerably in details across (i) their variabilities due to facial deformations (caused by expressions and mouth opening), subject aging (wrinkles), etc, and (ii) their specific details as skin color, scar tissue, face asymmetry, etc. The environmental factors refer to lighting conditions (controlled or ambient) and changes in head pose.
• Intrinsic factors. They include sensor cost, its intrusiveness, manner of sensor use (cooperative or not), spatial and/or temporal resolutions, measurement accuracy and the acquisition time, which allows us to capture moving faces or simply faces in static state.

These challenges arise when acquiring static faces as well as when dealing with faces in action. Different applications have different requirements. For instance, in the computer graphics community, the results of performance capture should exhibit a great deal of spatial fidelity and temporal accuracy to be an authentic reproduction of a real actors’ performance. Facial recognition systems, on the other hand, require the accurate capture of person-specific details. The movie industry, for instance, may afford a 3D modeling pipeline system with special purpose hardware and highly specialized sensors that require manual calibration. When deploying a 3D acquisition system for facial recognition at airports and in train stations, however, cost, intrusiveness, and the need of user cooperation, among others, are important factors to consider. In ambient intelligence applications where a user-specific interface is required, facial expression recognition from 3D sequences emerges as a research trend instead of 2D-based techniques, which are sensitive to changes and pose variations. Here, also, sensor cost and its capability to capture facial dynamics are important issues. Figure 1.1 shows a new 3D face modeling-guided taxonomy of existing reconstruction approaches. This taxonomy proposes two categories: The first category targets 3D static face modeling, while the approaches belonging to the second category try to capture facial shapes in action (i.e., in 3...

Table of contents

• Cover
• Title Page
• Copyright
• Preface
• List of Contributors
• Chapter 1: 3D Face Modeling
• Chapter 2: 3D Face Surface Analysis and Recognition Based on Facial Surface Features
• Chapter 3: 3D Face Surface Analysis and Recognition Based on Facial Curves
• Chapter 4: 3D Morphable Models for Face Surface Analysis and Recognition
• Chapter 5: Applications
• Index