Academic Press Library in Signal Processing, Volume 6

Image and Video Processing and Analysis and Computer Vision

,

• Publisher: Academic Press
• Year: 2017
• Print ISBN: 9780128118894
• eBook ISBN: 9780128119006
• Topic: Technology & Engineering
• Subtopic: Digital Media
• Index: Technology & Engineering

Academic Press Library in Signal Processing, Volume 6: Image and Video Processing and Analysis and Computer Vision is aimed at university researchers, post graduate students and R&D engineers in the industry, providing a tutorial-based, comprehensive review of key topics and technologies of research in both image and video processing and analysis and computer vision. The book provides an invaluable starting point to the area through the insight and understanding that it provides. With this reference, readers will quickly grasp an unfamiliar area of research, understand the underlying principles of a topic, learn how a topic relates to other areas, and learn of research issues yet to be resolved. - Presents a quick tutorial of reviews of important and emerging topics of research - Explores core principles, technologies, algorithms and applications - Edited and contributed by international leading figures in the field - Includes comprehensive references to journal articles and other literature upon which to build further, more detailed knowledge

Section 1

Visual Information Processing

Chapter 1

Multiview video: Acquisition, processing, compression, and virtual view rendering

Olgierd Stankiewicz*; Gauthier Lafruit^†; Marek Domański* ^* Poznań University of Technology, Poznań, Poland
^† Brussels University (French wing), Brussels, Belgium

Abstract

This chapter presents an overview of the acquisition, processing, and rendering pipeline in a multiview video system, targeting the unique feature of rendering virtual views, not captured by the input camera feeds. This is called Virtual View Synthesis and supports Free Navigation similar to The Matrix bullet time effect that was popularized in the late 1990s. A substantial part of the chapter is devoted in explaining how to estimate the respective camera parameters and their relative positions in the system, as well as how to estimate/measure depth, which is an essential information in order to obtain smooth virtual view transitions with Depth Image-Based Rendering.

Keywords

DIBR—Depth Image-Based Rendering; View synthesis; Multiview plus depth

Acronyms

2D two-dimensional

3D three-dimensional

AVC advanced video coding

CAVE Cave Automatic Virtual Environment

DERS Depth Estimation Reference Software

DIBR depth image-based rendering

FTV Free viewpoint TV

HEVC high efficiency video coding

3D-HEVC high efficiency video coding with depth maps

HMD Head Mounted Device

IEC International Electrotechnical Commission

ISO International Organization for Standardization

ITU-T International Telecommunications Union-Telecommunication sector

JPEG Joint Photographic Experts Group

MPEG Moving Picture Experts Group

MVD multiview video plus depth

VCEG Video Coding Experts Group

VR Virtual Reality

VSRS View Synthesis Reference Software

Acknowledgments

This work was partially supported by the National Science Centre, Poland, within project OPUS according to the decision DEC-2012/05/B/ST7/01279, and within project PRELUDIUM according to the decision DEC-2012/07/N/ST6/02267, as well as project 3DLICORNEA funded by the Brussels Institute for Research and Innovation, Belgium, under Grant No. 2015-DS/R-39a/b/c/d.

The authors would also like to thank the fruitful discussions with members from the JPEG and MPEG standardization committees, and experienced colleagues having participated to the project DREAMSPACE funded by the European Commission.

1.1 Multiview Video

Ever since the success of three-dimensional (3D) games where the user selects his/her own viewpoint to a premodeled 3D scene, much interest has risen to be able to mimic this functionality also on real content. In 1999, the science-fiction movie The Matrix popularized the bullet time effect, showing a continuously changing camera viewpoint around the action scene. This was obtained by cleverly triggering and interpolating hundreds of camera views, cf. Fig. 1.1, which we refer as multiview video in the remainder of the chapter.

Fig. 1.1 Hundreds of cameras to capture The Matrix bullet effect. © Wikipedia.

Think of the possibilities if we could create such Free Navigation effect on the fly at low production cost, e.g., in sports events, where each individual TV viewer can choose his/her own viewpoint to the scene.

Moreover, synthesizing two adjacent viewpoints at any moment in time supports stereoscopic viewing, and closing the loop with a stereoscopic Head Mounted Device (HMD) that continuously renders the stereo viewpoints corresponding to the actual HMD position, brings Virtual Reality (VR) one step closer to a “Teleported Reality,” also called Cinematic VR.

Such topics are actively explored in many audiovisual standardization committees. Joint Photographic Experts Group (JPEG) and Moving Picture Experts Group (MPEG), for instance, are heavily involved in getting new multimedia functionalities off the ground, far beyond simple two-dimensional (2D) pictures and video. More specifically, 3D data representations and coding formats are intensely ...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright
• Contributors
• About the Editors
• Section Editors
• Introduction
• Section 1: Visual Information Processing
• Section 2: Computational Imaging and 3D Analysis
• Section 3: Image and Video-Based Analytics
• Index