Image Restoration

Fundamentals and Advances

Bahadir Kursat Gunturk, Xin Li, Bahadir Kursat Gunturk, Xin Li

• Publisher: CRC Press
• Year: 2018
• ISBN:
  9781351832861
• Edition: 1
• Topic: Computer Science
• Subtopic: Computer Graphics
• Index: Computer Science

Image Restoration: Fundamentals and Advances responds to the need to update most existing references on the subject, many of which were published decades ago. Providing a broad overview of image restoration, this book explores breakthroughs in related algorithm development and their role in supporting real-world applications associated with various scientific and engineering fields. These include astronomical imaging, photo editing, and medical imaging, to name just a few. The book examines how such advances can also lead to novel insights into the fundamental properties of image sources.

Addressing the many advances in imaging, computing, and communications technologies, this reference strikes just the right balance of coverage between core fundamental principles and the latest developments in this area. Its content was designed based on the idea that the reproducibility of published works on algorithms makes it easier for researchers to build on each other's work, which often benefits the vitality of the technical community as a whole. For that reason, this book is as experimentally reproducible as possible.

Topics covered include:

• 
  
  

• Image denoising and deblurring
• Different image restoration methods and recent advances such as nonlocality and sparsity
• Blind restoration under space-varying blur
• Super-resolution restoration
• Learning-based methods
• Multi-spectral and color image restoration
• New possibilities using hybrid imaging systems

Many existing references are scattered throughout the literature, and there is a significant gap between the cutting edge in image restoration and what we can learn from standard image processing textbooks. To fill that need but avoid a rehash of the many fine existing books on this subject, this reference focuses on algorithms rather than theories or applications. Giving readers access to a large amount of downloadable source code, the book illustrates fundamental techniques, key ideas developed over the years, and the state of the art in image restoration. It is a valuable resource for readers at all levels of understanding.

Chapter 1

Image Denoising: Past, Present, and Future

XIN LI

West Virginia University

1.1 Introduction

Image denoising refers to the restoration of an image contaminated by additive white Gaussian noise (AWGN). Just like AWGN has served as the simplest situation in modeling channel degradation in digital communication, image denoising represents the simplest task in image restoration and therefore has been extensively studied by several technical communities. It should be noted that the study of the more general problem of signal denoising dates back to at least Norbert Wiener in the 1940s. The celebrated Wiener filter provides the optimal solution to the recovery of Gaussian signals contaminated by AWGN. The derivation of Wiener filtering, based on the so-called orthogonality principle, represents an elegant solution and the only known situation where constraining to linear solutions does not render any sacrifice on the performance. Therefore, at least in theory the problem of image denoising can be solved if we can reduce it to a problem that satisfies the assumptions behind the Wiener filtering theory. The challenge of image denoising ultimately boils down to the art of modeling images.

As George Box once said, “All models are wrong; but some are useful.” Under the context of image denoising, the usefulness of models heavily depends on the class of images of interest. The class of photographic images (a.k.a. natural images) are likely to be the most studied in the literature of image coding and denoising. Even though denoising research has been co-evolving with coding research, image models developed for one do not lend themselves directly to the other. The bit rate constraint and accessibility to the original image define the boundary of image coding differently from that of image denoising. Taking an analogy, image denoising behaves more like a source decoding instead of an encoding one — for example, the role played by the redundancy of signal representation is diametrically different in denoising and coding scenarios. An overcomplete representation — often undesirable and deemed “wrong” in image coding — turns out to be a lot more “useful” in image denoising.

Image models underlying all existing image denoising algorithms, no matter explicitly or implicitly stated, can be classified into two categories: deterministic and statistical. Deterministic models include those studied in functional analysis (e.g., Sobolov and Besov-space functions) and partial differential equations (PDE); statistical models include Markov Random Field (MRF), conditional random field (CRF), Gaussian scalar mixture (GSM) and so on. Despite the apparent difference at the surface, deterministic and statistical models have intrinsic connections (e.g., the equivalence between wavelet shrinkage and total variation diffusion). The subtle difference between deterministic and statistical models is highlighted by Von Neumann’s famous quote on randomness, “Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin.” Indeed, a theoretically optimal denoising algorithm (though of little practical value) is to recognize the deterministic procedure of simulating AWGN on digital computers. By reverse-engineering the noise simulation process, one can always perfectly remove it and reach zero errors!

The above reasoning raises another issue that has not received as much attention from the image processing community as image modeling — mathematical modeling of noise. Even though computer simulation of AWGN has become the gold standard of image de-noising, there is little justification that the contaminating noise in real-world images satisfies the AWGN assumption. In fact, noise sources in the physical world are often nonadditive (e.g., multiplicative) and non-Gaussian (e.g., Poisson). Nevertheless, algorithms developed for AWGN can often be twisted to match other types of noise in more general restoration tasks (e.g., involving motion or optical blur). As regularization strategies aim at incorporating a priori knowledge about either the image or noise source into the solution algorithms, we expect that mathematical modeling of the noise source is going to play a more important role in the recovery of images contaminated by real-world noise in the future.

The rest of this chapter is organized as follows. We first provide a historical review of image denoising in Section 1.2, especially its revival in the past decade. Due to space limitation, our review is concise and attempts to complement existing ones (e.g., [1]). Then we will work with a pair of popular test images — lena and barbara — and walk through a series of representative denoising algorithms in Sections 1.3 through 1.5. These two images — one abundant with regular edges and the other regular textures — serve to illustrate the effectiveness of incorporating complementary priori knowledge such as local smoothness and nonlocal similarity. Fully reproducible experimental results will be reported to help young minds entering the field get acquainted with the current state-of-the-art algorithms yet maintain a healthy skepticism toward authoritative models. We make some concluding remarks and discuss future research directions in Section 1.6.

1.2 Historical Review of Image Denoising

Signal denoising dates back to the pioneering work of Wiener and Kolmogorov in the 1940s. The Wiener–Kolmogorov filtering theory was the first rigorous result of designing statistically optimal filters for the class of stationary Gaussian processes. Its long-lasting impact has been witnessed in the past six decades, as we will elaborate next. In the 1950s, Peter Swerling — one of the most influential radar theoreticians — made significant contributions to the optimal estimation orbits and trajectories of satellites and missiles at the RAND Corporation, while the Soviet mathematician Ruslan Stratonovich solved the problem of optimal nonlinear filtering based on his theory of conditional Markov processes in 1959–1960. The next milestone was marked by Rudolf Kalman’s adaptive filtering, which extends the Wiener–Kolmogorov theory from a stationary to a nonstationary process. The capability of tracking changes of local statistics by Kalman filtering has led to a wide range of applications in space and military technology.

In the 1970s, two-dimensional signals such as digital imagery started to attract more attention. To the best of our knowledge, image denoising was first studied as a problem of statistical image enhancement by Nasser Nahi and Ali Habibi of the University of Southern California in [2,3]. Test images used in their study are apparently oversimplified from today’s standard, but given the limited computing power and memory resources, those early works were still visionary and it is not surprising that the USC image database is likely the most popular since then. By contrast, theoretic extension of Kalman filtering from 1D to 2D (e.g., [4]) had received relatively less attention partially due to the practical limitations at that time. The full potential of 2D Kalman filtering had to wait until advances in computing technology caught up in 1980s to make its implementation more feasible. The highly cited work of Jong-Sen Lee [5] on image enhancement/noise filtering by local statistics is a standard implementation of 2D Kalman filtering — namely, through the estimation of local mean/variance from a centralized window (the origin of image patches). Nevertheless, [5] was the first algorithmic achievement of applying local Wiener filtering to image denoising, and its conceptual simplicity (in contrast to mathematically more demanding state-space formulation in 2D Kalman filtering) greatly contributed to its impact on engineering applications.

The history of image denoising took an interesting turn in the late 1980s as wavelet theory was established independently by applied mathematicians, computer scientists, and electrical engineers [54]. Wavelet transforms rapidly became the favorite tool for various image processing tasks from compression to denoising. Simple ideas such as wavelet shrinkage/thresholding [7] became the new fashion; while orthodox approaches of applying local Wiener filtering in the wavelet domain (e.g., [8, 9, 10, 11, 12]) found themselves in an awkward position — they had to prove they work better than ad-hoc shrinkage techniques (e.g., [7, 13, 14, 15, 16]). Not to mention that some more sophisticated models in the wavelet domain (e.g., hidden Markov model [17,18] and Markov random field [19, 20, 21]) often achieve modest performance gain over local Wiener filtering while at the price of prohibitive complexity. The rapid growth of wavelet-based image denoising algorithms from the late 1990s to early 2000s might be the consequence of a bandwagon effect (unfortunately this author was also caught during his Ph.D. study). Hindsight reveals that what is more important than the invention of a tool (e.g., wavelet transform [22]) are the novel insights it could bear to a fundamental understanding of the problem. Good localization property of wavelet bases does indicate a good fit with the strategy of local Wiener filtering (even its more so...