Variational Bayesian Learning Theory

Shinichi Nakajima, Kazuho Watanabe, Masashi Sugiyama

• Publisher: Cambridge University Press
• Year: 2019
• Print ISBN: 9781107076150
• eBook ISBN: 9781316998311
• Topic: Computer Science
• Subtopic: Computer Vision & Pattern Recognition
• Index: Computer Science

Variational Bayesian learning is one of the most popular methods in machine learning. Designed for researchers and graduate students in machine learning, this book summarizes recent developments in the non-asymptotic and asymptotic theory of variational Bayesian learning and suggests how this theory can be applied in practice. The authors begin by developing a basic framework with a focus on conjugacy, which enables the reader to derive tractable algorithms. Next, it summarizes non-asymptotic theory, which, although limited in application to bilinear models, precisely describes the behavior of the variational Bayesian solution and reveals its sparsity inducing mechanism. Finally, the text summarizes asymptotic theory, which reveals phase transition phenomena depending on the prior setting, thus providing suggestions on how to set hyperparameters for particular purposes. Detailed derivations allow readers to follow along without prior knowledge of the mathematical techniques specific to Bayesian learning.

Table of contents

• Cover
• Half-title
• Title page
• Copyright information
• Contents
• Preface
• Nomenclature
• Part I Formulation
• Part II Algorithm
• Part III Nonasymptotic Theory
• Part IV Asymptotic Theory
• Appendix A James–Stein Estimator
• Appendix B Metric in Parameter Space
• Appendix C Detailed Description of Overlap Method
• Appendix D Optimality of Bayesian Learning
• Bibliography
• Subject Index