Advanced Video Coding: Principles and Techniques

The Content-based Approach

K.N. Ngan, T. Meier, D. Chai

• Publisher: Elsevier Science
• Year: 1999
• Print ISBN: 9780444826671
• eBook ISBN: 9780080498737
• Topic: Computer Science
• Subtopic: Digital Media
• Index: Computer Science

In recent years, the paradigm of video coding has shifted from that of a frame-based approach to a content-based approach, particularly with the finalization of the ISO multimedia coding standard, MPEG-4. MPEG-4 is the emerging standard for the coding of multimedia content. It defines a syntax for a set of content-based functionalities, namely, content-based interactivity, compression and universal access. However, it does not specify how the video content is to be generated. To generate the video content, video has to be segmented into video objects and tracked as they transverse across the video frames. This book addresses the difficult problem of video segmentation, and the extraction and tracking of video object planes as defined in MPEG-4. It then focuses on the specific issue of face segmentation and coding as applied to videoconferencing in order to improve the quality of videoconferencing images especially in the facial region. Modal-based coding is a content-based coding technique used to code synthetic objects that have become an important part of video content. It results in extremely low bit rates because only the parameters needed to represent the modal are transmitted. Model-based coding is included to provide background information for the synthetic object coding in MPEG-4. Lastly, MPEG-4, the first coding standard for multimedia content is described in detail. The topics covered include the coding of audio objects, the coding of natural and synthetic video objects, and error resilience. Advanced Video Coding is one of the first books on content-based coding and MPEG-4 coding standard. It serves as an excellent information source and reference for both researchers and practicing engineers.

Chapter 1

Image and Video Segmentation

A. Eleftheriadis; A. Jacquin

Segmentation plays a crucial role in second-generation image and video coding schemes, as well as in content-based video coding. It is one of the most difficult tasks in image processing, and it often determines the eventual success or failure of a system.

Broadly speaking, segmentation seeks to subdivide images into regions of similar attribute. Some of the most fundamental attributes are luminance, color, and optical flow. They result in a so-called low-level segmentation, because the partitions consist of primitive regions that usually do not have a one-to-one correspondence with physical objects.

Sometimes, images must be divided into physical objects so that each region constitutes a semantically meaningful entity. This higher-level segmentation is generally more difficult, and it requires contextual information or some form of artificial intelligence. Compared to low-level segmentation, far less research has been undertaken in this field.

Both low-level and higher-level segmentation are becoming increasingly important in image and video coding. The level at which the partitioning is carried out depends on the application. So-called second generation coding schemes [1, 2] employ fairly sophisticated source models that take into account the characteristics of the human visual system. Images are first partitioned into regions of similar intensity, color, or motion characteristics. Each region is then separately and efficiently encoded, leading to less artifacts than systems based on the discrete cosine transform (DCT) [3, 4, 5]. The second-generation approach has initiated the development of a significant number of segmentation and coding algorithms [6, 7, 8, 9, 10], which are based on a low-level segmentation.

The new video coding standard MPEG-4 [11, 12], on the other hand, targets more than just large coding gains. To provide new functionalities for future multimedia applications, such as content-based interactivity and content-based scalability, it introduces a content-based representation. Scenes are treated as compositions of several semantically meaningful objects, which are separately encoded and decoded. Obviously, MPEG-4 requires a prior decomposition of the scene into physical objects or so-called video object planes (VOPs). This corresponds to a higher-level partition.

As opposed to the intensity or motion-based segmentation for the second-generation techniques, there does not exist a low-level feature that can be utilized for grouping pixels into semantically meaningful objects. As a consequence, VOP segmentation is generally far more difficult than low-level segmentation. Furthermore, VOP extraction for content-based interactivity functionalities is an unforgiving task. Even small errors in the contour can render a VOP useless for such applications.

This chapter starts with a review of Bayesian inference and Markov random fields (MRFs), which will be needed throughout this chapter. A brief discussion of edge detection is given in Section 1.2, and Section 1.3 deals with low-level still image segmentation. The remaining three sections are devoted to video segmentation. First, an introduction to motion and motion estimation is given in Sections 1.4 and 1.5, before video segmentation techniques are examined in Sections 1.6 and Sectio...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright page
• Dedication
• Preface
• Acknowledgments
• Chapter 1: Image and Video Segmentation
• Chapter 2: Face Segmentation
• Chapter 3: Foreground/Background Coding
• Chapter 4: Model-Based Coding
• Chapter 5: Video Object Plane Extraction and Tracking
• Chapter 6: MPEG-4 - Standard for Multimedia Applications
• Index