Advances in Domain Adaptation Theory

Ievgen Redko, Emilie Morvant, Amaury Habrard, Marc Sebban, Younès Bennani

• Publisher: ISTE Press - Elsevier
• Year: 2019
• Print ISBN: 9781785482366
• eBook ISBN: 9780081023471
• Topic: Computer Science
• Subtopic: Computer Science General
• Index: Computer Science

Advances in Domain Adaptation Theory gives current, state-of-the-art results on transfer learning, with a particular focus placed on domain adaptation from a theoretical point-of-view. The book begins with a brief overview of the most popular concepts used to provide generalization guarantees, including sections on Vapnik-Chervonenkis (VC), Rademacher, PAC-Bayesian, Robustness and Stability based bounds. In addition, the book explains domain adaptation problem and describes the four major families of theoretical results that exist in the literature, including the Divergence based bounds. Next, PAC-Bayesian bounds are discussed, including the original PAC-Bayesian bounds for domain adaptation and their updated version.Additional sections present generalization guarantees based on the robustness and stability properties of the learning algorithm.- Gives an overview of current results on transfer learning- Focuses on the adaptation of the field from a theoretical point-of-view- Describes four major families of theoretical results in the literature- Summarizes existing results on adaptation in the field- Provides tips for future research

1

State of the Art of Statistical Learning Theory

Abstract

As we have previously seen, the general purpose of machine learning is to automate the process of acquiring knowledge based on observed data. Despite this general objective, the way in which this process is carried out may vary considerably depending on the nature of considered data and the final goal of learning. Thus, it can be characterized in many completely different ways. What remains unchanged, however, is that the nature of the learning paradigms. Despite their apparent variety, they can be roughly called “statistical”, which means that they are all related to revealing the statistical nature of the underlying phenomenon. To this end, the main goal of statistical learning theory is to analyze and formalize the process of knowledge acquisition and to provide a theoretical basis and an appropriate context for its analysis within a statistical framework.

Keywords

Gibbs classifier; Machine learning; No-free lunch theorem; PAC–Bayesian bounds; Rademacher bounds; Risk minimization; Statistical learning theory; Uniform stability; Vapnik–Chervonenkis bounds; VC dimension

As we have previously seen, the general purpose of machine learning is to automate the process of acquiring knowledge based on observed data. Despite this general objective, the way in which this process is carried out may vary considerably depending on the nature of considered data and the final goal of learning. Thus, it can be characterized in many completely different ways. What remains unchanged, however, is that the nature of the learning paradigms. Despite their apparent variety, they can be roughly called “statistical”, which means that they are all related to revealing the statistical nature of the underlying phenomenon. To this end, the main goal of statistical learning theory is to analyze and formalize the process of knowledge acquisition and to provide a theoretical basis and an appropriate context for its analysis within a statistical framework.

Statistical learning theory has a rich history and continues to constantly...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright
• Abstract
• Notations
• Introduction
• 1: State of the Art of Statistical Learning Theory
• 2: Domain Adaptation Problem
• 3: Seminal Divergence-based Generalization Bounds
• 4: Impossibility Theorems for Domain Adaptation
• 5: Generalization Bounds with Integral Probability Metrics
• 6: PAC-Bayesian Theory for Domain Adaptation
• 7: Domain Adaptation Theory Based on Algorithmic Properties
• 8: Iterative Domain Adaptation Methods
• Conclusions and Discussions
• Appendix 1: Proofs of the Main Results of Chapter 3
• Appendix 2: Proofs of the Main Results of Chapter 4
• Appendix 3: Proofs of the Main Results of Chapter 5
• Appendix 4: Proofs of the Main Results of Chapter 6
• Appendix 5: Proofs of the Main Results of Chapter 8
• References
• Index