The Essential Guide to Video Processing

Alan C. Bovik

• Publisher: Academic Press
• Year: 2009
• ISBN:
  9780080922508
• Topic: Tecnología e ingeniería
• Subtopic: Señales y procesamiento de señales

This comprehensive and state-of-the art approach to video processing gives engineers and students a comprehensive introduction and includes full coverage of key applications: wireless video, video networks, video indexing and retrieval and use of video in speech processing. Containing all the essential methods in video processing alongside the latest standards, it is a complete resource for the professional engineer, researcher and graduate student.

• Numerous conceptual and numerical examples
• All the latest standards are thoroughly covered: MPEG-1, MPEG-2, MPEG-4, H.264 and AVC
• Coverage of the latest techniques in video security

"Like its sister volume "The Essential Guide to Image Processing, " Professor Bovik's Essential Guide to Video Processing provides a timely and comprehensive survey, with contributions from leading researchers in the area. Highly recommended for everyone with an interest in this fascinating and fast-moving field." —Prof. Bernd Girod, Stanford University, USA

• Edited by a leading person in the field who created the IEEE International Conference on Image Processing, with contributions from experts in their fields
• Numerous conceptual and numerical examples
• All the latest standards are thoroughly covered: MPEG-1, MPEG-2, MPEG-4, H.264 and AVC
• Coverage of the latest techniques in video security

CHAPTER 1 Introduction to Digital Video Processing

Alan C. Bovik

Laboratory for Image and Video Engineering (LIVE), Department of Electrical and Computer Engineering, The University of Texas at Austin

In this age of dramatic technology shifts, one of the most significant has been the emergence of digital video as an important aspect of daily life. While the Internet has certainly dramatically changed the way in which we obtain information, how much more attractive is it because of the powerful medium of video? How many of us now watch the morning news on Yahoo! Video? Are there many readers who have not shared or viewed a Utube video? Indeed, the average person on the street has become an amateur video processor, now that handheld digital camcorders that record directly onto mini-DVD are standard, and software is available for simple film editing.

Developments in wireless and cellular telephony certainly represent the greatest recent sea change in global technology over the past five years. It is widely held that ongoing advances in handheld mobile devices will be dominated by easier wireless Internet access that will be enhanced by rich video and multimedia content. The great leaps in mobile bandwidths that are being engineered ensure that these developments will reach fruition. It is no mistake that the wildly popular Apple iphone has such a large display, a realization by visionary engineers that the future of wireless relies heavily on visual communication.

An increasing percentage of homes now own large-screen televisions receiving digital signals. Indeed, the age of analog TV has been declared dead. High-definition television (HDTV), heralded long ago but never realized until recently is now becoming widely available, and recent newspaper and Internet headlines herald the emergence of an agreed-upon HD DVD standard known as Blueray.

Beyond these notable examples, there are a wealth of other applications that involve digital video technologies. It is safe to say that digital video is very much the current frontier and the future of image processing research and development. Some of the notable digital video applications not mentioned above are the following:

Video teleconferencing using modern software codecs that run on your PC.

Medical video, where dynamic processes in the human body, such as the motion of the heart, can be viewed live.

Video telephony, an application coming to your cell phone.

Dynamic scientific visualization, such as viewable displays of galaxy formation over accelerated time.

Multimedia video, where video is combined with graphics, speech, and other sensor modalities for an enriched experience.

Video instruction, both in the classroom and on the Internet.

Digital cinema, not just in the making of graphical marvels such as Shrek but for delivery of movies to theatres in digital format.

With all these developments, it is a particularly exciting and rewarding time to be a video processing engineer. Digital video processing is the study of algorithms for processing moving images that are represented in digital format. Here, we distinguish digital video from digital still images, which are images that do not change with time – basically, digital photographs. A companion volume entitled The Essential Guide to Image Processing covers the processing and analysis of digital still images in great detail and offers material ranging from a very basic level to much more advanced topics and applications. For the novice to the field, it is recommended that The Essential Guide to Image Processing be studied first, since the field of digital video processing first builds upon the principles of still image processing before advancing significantly further.

A digital video is a moving picture, or movie, that has been converted into a computer-readable binary format consisting of logical 0s and 1s. Since video is dynamic, the visual content evolves with time and generally contains moving and/or changing objects. The information contained in video is naturally much richer than is contained in still images, since our world is constantly changing owing to the movement of people, animals, vehicles, and other objects. Indeed, the richness of video results in a significant data glut, which, until recently has been difficult to overcome. However, with faster sensors and recording devices, it is it is becoming easier to acquire and analyze digital video data sets.

However, it is important to realize that digital videos are multidimensional signals, meaning that they are functions of more than a single variable. Indeed, digital video is ordinarily a function of three dimensions – two in space and one in time, as depicted in Fig. 1.1. Because of this, digital video processing is data intensive: significant bandwidth, computational, and storage resources are required to handle video streams in digital format.

FIGURE 1.1 The dimensionality of video.

In the companion volume, The Essential Guide to Image Processing, a tutorial is given on the processes of digitization of images, or analog-to-digital conversion (A/D conversion). This conversion process consists of two distinct subprocesses: sampling, or conversion of a continuous-space/time video signal into a discrete-space/time video signal, and quantization, which is the process of converting a continuous-valued video that has a continuous range (set of values that it can take) of intensities and/or colors into a discrete-valued video that has a discrete range of intensities and/or colors. This is ordinarily done by a process of rounding, truncation, or some other irreversible, nonlinear process of information destruction. Quantization is a necessary precursor to digital processing or display, since the image intensities and/or colors must be represented with a finite precision (limited by word length) in a digital video processor or display device.

1.1 SAMPLED VIDEO

Just as with still images, digital processing of video requires that the video stream be in a digital format, meaning that it must be sampled and quantized. Video quantization is essentially the same as image quantization. However, video sampling involves taking samples along a new and different (time) dimension. As such, it involves some different concepts and techniques.

First and foremost, the time dimension has a direction associated with it, unlike the space dimensions, which are ordinarily regarded as directionless until a coordinate system is artificially imposed upon it. Time proceeds from the past toward the future, with an origin that exists only in the current moment. Video is often processed in “real time,” which (loosely) means that the result of processing appears effectively “instantaneously” (usually in a perceptual sense) once the input becomes available. Such a processing system cannot depend more than a few future video samples. Moreover, it must process the video data quickly enough that the result appears instantaneous. Because of the vast data volume involved, the design of fast algorithms and hardware devices is a major priority.

In principle, an analog video signal I (x, y, t), where (x, y) denote continuous space coordinates and t denotes continuous time, is continuous in both the space and time dimensions, since the radiation flux that is incident on a video sensor is continuous at normal scales of observation. However, the analog video that is viewed on display monitors is not truly analog, since it is sampled along one space dimension and along the time dimension. Practical so-called analog video systems, such as television and monitors, represent video as a one-dimensional electrical signal V(t). Prior to display, a one-dimensional signal is obtained by sampling I (x, y, t) along the vertical (y) space direction and along the time (t) direction. This is called scanning and t...