Face Detection and Recognition on Mobile Devices

Haowei Liu

• Publisher: Morgan Kaufmann
• Year: 2014
• ISBN:
  9780124171282
• Topic: Ciencia de la computación
• Subtopic: Inteligencia artificial (IA) y semántica

This hands-on guide gives an overview of computer vision and enables engineers to understand the implications and challenges behind mobile platform design choices. Using face-related algorithms as examples, the author surveys and illustrates how design choices and algorithms can be geared towards developing power-saving and efficient applications on resource constrained mobile platforms.

• Presents algorithms for face detection and recognition
• Explains applications of facial technologies on mobile devices
• Includes an overview of other computer vision technologies

Face Technologies on Mobile Devices

Learn how computer vision algorithms are developed on mobile platforms with Face Detection and Recognition on Mobile Devices. This hands-on guide enables engineers to understand the implications and challenges behind every design choice. The author gives an overview of the field of computer vision and provides motivation about developing computer vision applications on mobile platforms. Using face-related algorithms as examples, the author surveys and illustrates how design choices and algorithms can be geared toward developing power-saving and efficient applications on resource-constrained mobile platforms.

Keywords

face recognition; computer vision; mobile hardware; power consumption; networking bandwidth

Face-related problems have long been studied in the field of computer vision, and many technologies have also been developed in the industry. Given that the face is a unique exhibition of each individual, face-related technologies are mostly developed for surveillance and security applications and, recently, for entertainment purposes. For example, large-scale surveillance systems [1,2] are built based on detected facial features to allow law enforcement to query for videos of individuals with specific facial attributes such as being bald or wearing a beard. Face-recognition technology is widely used as well. For instance, the FastAccess software developed by SensibleVision allows mobile users to log on with their credentials without typing. The technology was also depicted in the movie Mission Impossible: Ghost Protocol, in which Agent Hanaway (played by Josh Holloway) uses his cell phone to detect the faces of the incoming people and identifies the target through a remote server. In addition, avatar animations controlled by human facial expression are becoming more popular, as are the built-in face detector or “smile detector” features in numerous cameras and mobile phones. Through face detection and recognition, it is possible that you could use an application on your mobile phone that would allow you to take photos of your friends, tag them, and upload the photos to Facebook. In this chapter, we will give a brief overview about the principles behind these technologies.

Algorithms for Face Detection

Overview of Face-Detection Algorithms

Creating reliable face detection has long been a problem, and solving it is a crucial step toward other facial analysis algorithms such as face recognition, tracking, or facial feature detection (e.g., eyes or nose). It is a challenging problem as different face poses, camera orientations, lighting conditions, and imaging quality could make the detection algorithms fail. With that said, much progress has been made in the last decade, and numerous approaches have been proposed and developed.

Yang et al. [3] extensively surveyed these algorithms and divided them into four categories:

1. The top-down, rule-based approaches detect faces using rules derived from the researchers’ knowledge of faces. For example, the rules could be that the locations of the eyes have to be symmetric with respect to the center of the face or that the face region should be of uniform color. The drawback of a rule-based method is that it might not be that simple to derive precise rules from human knowledge. It is also difficult to extend the rules to cover all scenarios.

2. In contrast to the top-down approaches are the bottom-up, feature-based approaches. These approaches aim to find low-level features that signify the face region. They typically start with detecting features that could indicate face regions, such as regions with skin color, oriented edges that could indicate the lips, or textures that could indicate the cheeks. After the features are extracted, statistical models are built to describe their relationships and, later on, are followed by the verification of the existence of a face. Although these approaches enjoy a certain degree of success, they are sensitive to noise and occlusion. Shadows also have a big impact on the performance of a feature-based approach.

3. Template-matching approaches start by building a predefined template of the face. The template could be just an image of face silhouette or an edge map. It detects faces by correlating the template with the image, and regions that give the largest response are considered as face regions. However, a template-matching approach is inadequate, as it is not able to deal with variations in scale, poses, orientations, and shape effectively.

4. The appearances-based modeling approaches enjoy the most success. Different from the template-matching approaches, where the templates are handpicked by the researchers, an appearances-based approach attempts to “learn” a model from a large, labeled image dataset. The output of the learning process is a function the users can take to evaluate an image region. The function is called a classifier in the machine learning literature, and it will return whether or not the image region contains a face with a confidence value.

Many statistical models and machine-learning methods have been proposed for face detection, such as support vector machine, neural network, or Bayes classifier. However, the most widely used method is the one based on boosting, which we will touch on in the next section.

Viola–Jones Face-Detection Algorithms

The most popular and widely used face detection technique is the one developed by Viola and Jones [4]. It was developed as a generic object-detection framework and outperformed previously proposed approaches in terms of both performance and running time. The main challenge of face detection for these model-based or template-based approaches is the need to scan through every possible location and scale of an image in order to find the locations of the faces and reduce the false positives.

However, since an image usually contains only a few (0–5) faces, schemes like this end up spending lots of computing cycles on aggregating the information and evaluating candidates that are mostly nonface regions. The main contributions of the Viola–Jones face detector include a new image representation for fast feature computation, a boosting-base feature selection method, and a “cascade” classifier structure to quickly discard a large number of nonface regions. We will detail each of them next.

The Viola–Jones face detector starts with computing the features over a detection area on an intensity image. The features computed are rectangle features of different sizes. Three different kinds of rectangle features are used: two-rectangle features with two regions of the same size adjacent to each other horizontally and vertically. A three-rectangle feature consists of three neighboring rectangular regions, and a four-rectangle feature consists of two diagonal pairs of rectangular regions. For all cases, the feature value is computed as the difference between the sums of the pixel values within the black regions and the white regions. Figure 2.1 shows four examples of the rectangle features. You could think of these features as possible signatures for the face regions. The process of face detection is to look for the signatures, trim out nonface regions where the signatures do not exist, and identify the face regions with salient signatures.

Figure 2.1 Rectangle features for face detection. The feature values are computed by subtracting the sum of the pixel values within the white regions from the black regions. A and B show two-re...