Image Super-Resolution and Applications

Fathi E. Abd El-Samie, Mohiy M. Hadhoud, Said E. El-Khamy

• Publisher: CRC Press
• Year: 2012
• eBook ISBN: 9781466597754
• Topic: Computer Science
• Subtopic: Computer Graphics
• Index: Computer Science

This book is devoted to the issue of image super-resolution-obtaining high-resolution images from single or multiple low-resolution images. Although there are numerous algorithms available for image interpolation and super-resolution, there's been a need for a book that establishes a common thread between the two processes. Filling this need, Image

Chapter 1

Introduction

High-resolution (HR) images are required in most electronic imaging applications. The high resolution means that the pixel density within the image is high, and therefore an HR image can offer more details than those obtained from a low-resolution (LR) image. HR images are of great importance in applications such as medical imaging, satellite imaging, military imaging, underwater imaging, remote sensing, and high-definition television (HDTV).

In the past, traditional image vidicon and orthicon cameras have been the only available image acquisition devices. These cameras are analog cameras. Since the 1970s, charge-coupled devices (CCDs) and complementary metal oxide semiconductor (CMOS) image sensors have been widely used to capture digital images. Although these sensors are suitable for most imaging applications, the current resolution levels and their associated prices are not suitable for future demands. It is desirable to have very HR levels with prices as low as possible. The demands for HR levels have been the motivations to find methodologies for increasing the LR levels obtained using the current image acquisition devices.

The direct solution to increase the resolution level is to reduce the pixel size in sensor manufacturing. Therefore, the number of pixels per unit area is increased. The drawback of this solution is that the amount of light available from each pixel is decreased. The decrement of light amount leads to the generation of shot noise that seriously degrades the image quality. Unfortunately, the pixel size cannot be reduced beyond a certain level (40 μm² for 0.35 μm CMOS processes) to avoid shot noise. This level has already been reached in the manufacturing process.

Another solution to the problem of resolution level increment is to increase the chip size with the pixel size fixed. This solution leads to an increase in chip capacitance. It is well known that the large capacitance limits the speeding up of the charge transfer rate. The slow rate of charge transfer leads to a great problem in the image formation process. Generally, all hardware solutions to this problem are limited by the costs of high-precision optics and required image sensors.

The most feasible solution to this problem is to integrate both the hardware and software capabilities to obtain the required HR level. Making use of as high an HR level as possible from the hardware can carry part of this task. The rest of the task is performed using software. This is the new trend in most up-to-date image capturing devices. Image processing algorithms can be used effectively to obtain HR images. Using a single LR image to obtain an HR image is known as image interpolation. On the other hand, when multiple degraded observations of the same scene are used to generate a single HR image, the process is known as image super-resolution.

1.1 Image Interpolation

Image interpolation is the process by which a single HR image is obtained from a single LR image. Interpolation can be classified as polynomial interpolation and interpolation as an inverse problem. Polynomial interpolation depends on the concepts of the sampling theory. In polynomial interpolation, estimated pixels are inserted between existing pixels using polynomial expansions. Different algorithms have been presented for polynomial image interpolation. The most famous of these algorithms is spline interpolation. Polynomial image interpolation depends on a finite neighborhood around the pixel to be estimated. Traditional polynomial image interpolation algorithms do not consider the LR image degradation model, and hence their performance is limited. Chapter 2 gives a discussion of polynomial image interpolation.

Adaptive variants of polynomial image interpolation have been presented in the literature. Some of these variants depend on distance adaptation without consideration of the LR image degradation model, while the others consider that this model yields better interpolation results. Chapter 3 is devoted to adaptive polynomial image interpolation, and Chapter 4 gives a neural modeling method for polynomial image interpolation.

Color image interpolation is a newly considered issue that makes use of simple polynomial image interpolation, but with color images. In a digital imaging process, not all color components are available in the acquired image, and hence there is a need for the interpolation of missing color components from the existing components of neighboring pixels. Chapter 5 is devoted to color image interpolation.

Polynomial image interpolation has found an application in pattern recognition. Instead of saving database images with their original sizes, decimation can be used to reduce their sizes. At the recognition step, image interpolation can be carried out to return images to their original sizes. The features extracted from the interpolated images must be robust to interpolation estimation errors. Chapter 6 is devoted to image interpolation for pattern recognition.

The limited performance of polynomial image interpolation led to the evolution of a new trend for image interpolation as an inverse problem. In this trend, the LR image degradation model is considered in the interpolation process. Four solutions have been developed for image interpolation as an inverse problem. The results obtained have shown a great success compared to polynomial image interpolation results. Chapter 7 gives an explanation of image interpolation as an inverse problem.

1.2 Image Super-Resolution

Image super-resolution is the process by which a single HR image is obtained from multiple degraded LR images. Image super-resolution can be carried out with or without a priori information. The problem of super-resolution reconstruction of images can be solved in successive steps: image registration, multi-channel image restoration, image fusion, and finally image interpolation, as shown in Figure 1.1.

Image registration aims at overlaying the LR degraded images prior to the super-resolution reconstruction process. This step is very important as it is responsible for the correct integration of the information in the multiple observations. Chapter 8 is devoted to image registration methodologies.

Image fusion is the process of integrating the information in multiple images into a single image. It can be used as a step in the super-resolution reconstruction process. Different algorithms based on transforms such as the wavelet and curvelet transforms can be used for image fusion and image fusion can be used to obtain HR images directly, which is the case in satellite image fusion. Image fusion can also be used with images of different modalities. Chapter 9 is devoted to image fusion and its applications in image super-resolution.

Image super-resolution can be carried out using some a priori information about the degradations in the available LR images, such as information about blurring, registration shifts, and noise. With this information available, the solution to the super-resolution reconstruction problem can be carried out easily. Chapter 10 is devoted to image super-resolution with a priori information. If no information is available, the problem is more difficult and it is known as blind image super-resolution. This is the topic of Chapter 11.

Figure 1.1 Successive steps of image super-resolution reconstruction.

Chapter 2

Polynomial Image Interpolation

2.1 Introduction

Image interpolation is a well known topic to most researchers who are interested in image processing. It pervades several applications. It is almost never a goal in itself, yet it affects both the desired results and the ways to obtain them. Interpolation may appear as a simple step in many image processing applications. Therefore, some authors give it less importance than it deserves. We try in this chapter to highlight the importance of image interpolation and cover the traditional work in this field.

Interpolation has been previously treated in the literature using different approaches [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 and 28]. Several authors have presented definitions for interpolation. One of the simples...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• Preface
• Acknowledgments
• Authors
• 1 Introduction
• 2 Polynomial Image Interpolation
• 3 Adaptive Polynomial Image Interpolation
• 4 Neural Modeling of Polynomial Image Interpolation
• 5 Color Image Interpolation
• 6 Image Interpolation for Pattern Recognition
• 7 Image Interpolation as Inverse Problem
• 8 Image Registration
• 9 Image Fusion
• 10 Super-Resolution with a Priori Information
• 11 Blind Super-Resolution Reconstruction of Images
• Appendix A: Discrete B-Splines
• Appendix B: Toeplitz-to-Circulant Approximations
• Appendix C: Newton’s Method
• Appendix D: MATLAB® Codes
• References
• Index