![]()
Chapter 1: Introduction to Deep Learning
Introduction to Neural Networks
Biological Neurons
Mathematical Neurons
Deep Learning
Batch Gradient Descent
Stochastic Gradient Descent
Introduction to ADAM Optimization
Weight Initialization
Regularization
Batch Normalization
Batch Normalization with Mini-Batches
Traditional Neural Networks versus Deep Learning
Deep Learning Actions
Building a Deep Neural Network
Training a Deep Learning CAS Action Model
Demonstration 1: Loading and Modeling Data with Traditional Neural Network Methods
Demonstration 2: Building and Training Deep Learning Neural Networks Using CASL Code
Introduction to Neural Networks
Artificial neural networks mimic key aspects of the brain, in particular, the brain’s ability to learn from experience. In order to understand artificial neural networks, we first must understand some key concepts of biological neural networks, in other words, our own biological brains.
A biological brain has many features that would be desirable in artificial systems, such as the ability to learn or adapt easily to new environments. For example, imagine you arrive at a city in a country that you have never visited. You don’t know the culture or the language. Given enough time, you will learn the culture and familiarize yourself with the language. You will know the location of streets, restaurants, and museums.
The brain is also highly parallel and therefore very fast. It is not equivalent to one processor, but instead it is equivalent to a multitude of millions of processors, all running in parallel. Biological brains can also deal with information that is fuzzy, probabilistic, noisy, or inconsistent, all while being robust, fault tolerant, and relatively small. Although inspired by cognitive science (in particular, neurophysiology), neural networks largely draw their methods from statistical physics (Hertz et al. 1991). There are dozens, if not hundreds, of neural network algorithms.
Biological Neurons
In order to imitate neurons in artificial systems, first their mechanisms needed to be understood. There is still much to be learned, but the key functional aspects of neurons, and even small systems (networks) of neurons, are now known.
Neurons are the fun...
