Manifold Learning Theory and Applications

Yunqian Ma, Yun Fu, Yunqian Ma, Yun Fu

• Publisher: CRC Press
• Year: 2011
• Print ISBN: 9781439871096
• eBook ISBN: 9781466558878
• Topic: Economics
• Subtopic: Computer Science General
• Index: Economics

Trained to extract actionable information from large volumes of high-dimensional data, engineers and scientists often have trouble isolating meaningful low-dimensional structures hidden in their high-dimensional observations. Manifold learning, a groundbreaking technique designed to tackle these issues of dimensionality reduction, finds widespread

Chapter 1

Spectral Embedding Methods
for Manifold Learning

Alan Julian Izenman

1.1 Introduction

Manifold learning encompasses much of the disciplines of geometry, computation, and statistics, and has become an important research topic in data mining and statistical learning. The simplest description of manifold learning is that it is a class of algorithms for recovering a low-dimensional manifold embedded in a high-dimensional ambient space. Major breakthroughs on methods for recovering low-dimensional nonlinear embeddings of high-dimensional data (Tenenbaum, de Silva, and Langford, 2000; Roweis and Saul, 2000) led to the construction of a number of other algorithms for carrying out nonlinear manifold learning and its close relative, nonlinear dimensionality reduction. The primary tool of all embedding algorithms is the set of eigenvectors associated with the top few or bottom few eigenvalues of an appropriate random matrix. We refer to these algorithms as spectral embedding methods. Spectral embedding methods are designed to recover linear or nonlinear manifolds, usually in high-dimensional spaces.

Linear methods, which have long been considered part-and-parcel of the statistician’s toolbox, include PRINCIPAL COMPONENT ANALYSIS (PCA) and MULTIDIMENSIONAL SCALING (MDS). PCA has been used successfully in many different disciplines and applications. In computer vision, for example, PCA is used to study abstract notions of shape, appearance, and motion to help solve problems in facial and object recognition, surveillance, person tracking, security, and image compression where data are of high dimensionality (Turk and Pentland, 1991; De la Torre and Black, 2001). In astronomy, where very large digital sky surveys have become the norm, PCA has been used to analyze and classify stellar spectra, carry out morphological and spectral classification of galaxies and quasars, and analyze images of supernova remnants (Steiner, Menezes, Ricci, and Oliveira, 2009). In bioinformatics, PCA has been used to study high-dimensional data generated by genome-wide, gene-expression experiments on a variety of tissue sources, where scatterplots of the top principal components in such studies often show specific classes of genes that are expressed by different clusters of distinctive biological characteristics (Yeung and Ruzzo, 2001; Zheng-Bradley, Rung, Parkinson, and Brazma, 2010). PCA has also been used to select an optimal subset of single nucleotide polymorphisms (SNPs) (Lin and Altman, 2004). PCA is also used to derive approximations to more complicated nonlinear subspaces, including problems involving data interpolation, compression, denoising, and visualization.

MDS, which has its origins in psychology, has recently been found most useful in bioinformatics, where it is known as “distance g...

Table of contents

• Cover
• Title Page
• Copyright
• Contents
• List of Figures
• List of Tables
• Preface
• Editors
• Contributors
• 1 Spectral Embedding Methods for Manifold Learning
• 2 Robust Laplacian Eigenmaps Using Global Information
• 3 Density Preserving Maps
• 4 Sample Complexity in Manifold Learning
• 5 Manifold Alignment
• 6 Large.Scale Manifold Learning
• 7 Metric and Heat Kernel
• 8 Discrete Ricci Flow for Surface and 3-Manifold
• 9 2D and 3D Objects Morphing Using Manifold Techniques
• 10 Learning Image Manifolds from Local Features
• 11 Human Motion Analysis Applications of Manifold Learning
• Index