This course aims at providing a mathematical perspective to some key elements of the so-called deep neural networks (DNNs).Ā Much of the interest on deep learning has focused on the implementation of DNN-based algorithms.Ā Ā Our hope is that this compact textbook will offer a complementary point of view that emphasizes the underlying mathematical ideas. We believe that a more foundational perspective will help to answer important questions that have only received empirical answers so far.
Ā
Our goal is to introduce basic concepts from deep learning in a rigorous mathematical fashion, e.g. introduce mathematical definitions of deep neural networks (DNNs), loss functions, the backpropagation algorithm, etc.Ā
WeĀ Ā attempt to identify for each concept the simplest setting that minimizes technicalities but still contains the key mathematics.
Ā
The book focuses on deep learning techniques and introduces them almost immediately. Other techniques such as regression and SVM are briefly introduced and used as a steppingstone for explaining basic ideas of deep learning.
Ā
Throughout these notes, the rigorous definitions and statements are supplemented by heuristic explanations and figures. The book is organized so that each chapter introduces a key concept.Ā When teaching this course, some chapters could be presented as a part of a single lecture whereas the others have more material and would take several lectures.
Ā
Ā