Advanced Deep Learning with R

Become an expert at designing, building, and improving advanced neural network models using R

Bharatendra Rai

• Publisher: Packt Publishing
• Year: 2019
• ISBN:
  9781789534986
• Edition: 1
• Topic: Computer Science
• Subtopic: Natural Language Processing
• Index: Computer Science

Discover best practices for choosing, building, training, and improving deep learning models using Keras-R, and TensorFlow-R libraries

Key Features

• Implement deep learning algorithms to build AI models with the help of tips and tricks
• Understand how deep learning models operate using expert techniques
• Apply reinforcement learning, computer vision, GANs, and NLP using a range of datasets

Book Description

Deep learning is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data. Advanced Deep Learning with R will help you understand popular deep learning architectures and their variants in R, along with providing real-life examples for them.

This deep learning book starts by covering the essential deep learning techniques and concepts for prediction and classification. You will learn about neural networks, deep learning architectures, and the fundamentals for implementing deep learning with R. The book will also take you through using important deep learning libraries such as Keras-R and TensorFlow-R to implement deep learning algorithms within applications. You will get up to speed with artificial neural networks, recurrent neural networks, convolutional neural networks, long short-term memory networks, and more using advanced examples. Later, you'll discover how to apply generative adversarial networks (GANs) to generate new images; autoencoder neural networks for image dimension reduction, image de-noising and image correction and transfer learning to prepare, define, train, and model a deep neural network.

By the end of this book, you will be ready to implement your knowledge and newly acquired skills for applying deep learning algorithms in R through real-world examples.

What you will learn

• Learn how to create binary and multi-class deep neural network models
• Implement GANs for generating new images
• Create autoencoder neural networks for image dimension reduction, image de-noising and image correction
• Implement deep neural networks for performing efficient text classification
• Learn to define a recurrent convolutional network model for classification in Keras
• Explore best practices and tips for performance optimization of various deep learning models

Who this book is for

This book is for data scientists, machine learning practitioners, deep learning researchers and AI enthusiasts who want to develop their skills and knowledge to implement deep learning techniques and algorithms using the power of R. A solid understanding of machine learning and working knowledge of the R programming language are required.

Section 1: Revisiting Deep Learning Basics

This section contains a chapter that serves as an introduction to deep learning with R. It provides an overview of the process for developing deep networks and reviews popular deep learning techniques.

This section contains the following chapter:

• Chapter 1, Revisiting Deep Learning Architecture and Techniques

Revisiting Deep Learning Architecture and Techniques

Deep learning is part of a broader machine learning and artificial intelligence field that uses artificial neural networks. One of the main advantages of deep learning methods is that they help to capture complex relationships and patterns contained in data. When the relationships and patterns are not very complex, traditional machine learning methods may work well. However, with the availability of technologies that help to generate and process more and more unstructured data, such as images, text, and videos, deep learning methods have become increasingly popular as they are almost a default choice to deal with such data. Computer vision and natural language processing (NLP) are two areas that are seeing interesting applications in a wide variety of fields, such as driverless cars, language translation, computer games, and even creating new artwork.

Within the deep learning toolkit, we now have an increasing array of neural network techniques that can be applied to a specific type of task. For example, when developing image classification models, a special type of deep network called a convolutional neural network (CNN) has proved to be effective in capturing unique patterns that exist in image-related data. Similarly, another popular deep learning network called recurrent neural networks (RNNs) and its variants have been found useful in dealing with data involving sequences of words or integers. Another popular and interesting deep learning network called a generative adversarial network (GAN) has the capability to generate new images, speech, music, or artwork.

In this book, we will use these and other popular deep learning networks using R software. Each chapter presents a complete example that has been specifically developed to run on a regular laptop or computer. The main idea is to avoid getting bogged down by a huge amount of data needing advanced computing resources at the first stage of applying deep learning methods. You will be able to go over all the steps using the illustrated examples in this book. The examples used also include the best practices for each topic, and you will find them useful. You will also find a hands-on and applied approach helpful in quickly seeing the big picture when trying to replicate these deep learning methods when faced with a new problem.

This chapter provides an overview of the deep learning methods with R that are covered in this book. We will go over the following topics in this chapter:

• Deep learning with R
• The process of developing a deep network model
• Popular deep learning techniques with R and RStudio

Deep learning with R

We will start by looking at the popularity of deep learning networks and also take a look at a version of some of the important R packages used in this book.

Deep learning trend

Deep learning techniques make use of neural network-based models and have seen increasing interest in the last few years.A Google trends website for the search term deep learning provides the following plot:

The preceding plot has 100 as the peak popularity of a search term, and other numbers are relative to this highest point. It can be observed that the interest in the term deep learning has gradually increased in popularity since around 2014. For the last two years, it has enjoyed peak popularity. One of the reasons for the popularity of deep learning networks is the availability of the free and open source libraries, TensorFlow and Keras.

Versions of key R packages used

In this book, we will use the Keras R package that uses TensorFlow as a backend for building deep learning networks. An output from a typical R session, used for the examples illustrated in this book, providing various version-related information, is provided in the following code:

# Information from a Keras R session
sessionInfo()

R version 3.6.0 (2019-04-26)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS 10.15

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

Random number generation:
 RNG: Mersenne-Twister 
 Normal: Inversion 
 Sample: Rounding 
 
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] keras_2.2.4.1

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.2 lattice_0.20-38 lubridate_1.7.4 zeallot_0.1.0 
 [5] grid_3.6.0 R6_2.4.0 jsonlite_1.6 magrittr_1.5 
 [9] tfruns_1.4 stringi_1.4.3 whisker_0.4 Matrix_1.2-17 
[13] reticulate_1.13 generics_0.0.2 tools_3.6.0 stringr_1.4.0 
[17] compiler_3.6.0 base64enc_0.1-3 tensorflow_1.14.0

As seen previously, for this book we have used the 3.6 version of R that was released in April 2019. The nickname for this R version is Planting of a Tree. The version used for the Keras package is 2.2.4.1. In addition, all the application examples illustrated in the book have been run on a Mac computer with 8 GB of RAM. The main reason for using this specification is that it will allow a reader to go through all the examples without needing advanced computing resources to get started with any deep learning network covered in the book.

In the next section, we will go over the process of developing a deep network model that is broken down into five general steps.

Process of developing a deep network model

Developing a deep learning network model can be broken down into five key steps shown in the following flowchart:

Each step mentioned in the preceding flowchart can have varying requirements based on the type of data used, the type of deep learning network being developed, and also the main objective of developing a model. We will go over each step to develop a general idea about what is involved.

Preparing the data for a deep network model

Developing deep learning neural network models requires the variables to have a certain format. Independent variables may come with a varying scale, with some variable values in decimals and some other variables in thousands. Using such varying scales of variables is not very efficient when training a network. Before developing deep learning networks, we make changes such that the variables have similar scales. The process used for achieving this is called normalization.

Two commonly used methods for normalization are z-score normalization and min-max normalization. In z-score normalization, we subtract the mean from each value and divide it by the standard deviation. This transformation results in values that lie between -3 and +3 with a mean of 0 and a standard deviation of 1. For a min-max normalization, we subtract the minimum value from each data point, and then divide it by the range. This transformation converts data to having values between zero and one.

As an example, see the following plots, where we have obtained 10,000 data points randomly from a normal distribution with a mean of 35 and a standard deviation of 5:

From the preceding plots, we can observe that after z-score normalization, the data points mostly lie between -3 and +3. Similarly, after min-max normalization, the range of values changes to data points between 0 and 1. However, the overall pattern seen in the original data is retained after both types of normalization.

Another important step in preparing data when using a categorical response variable is to carry out one-hot encoding. One-hot encoding converts a categorical variable to a new binary format that has values containing either 0 or 1. This is achieved very easily by using the to_categorical() function available in Keras.

Typically data processing steps for unstructured data, such as image or text, are more involved compared with a situation where we are dealing with structured data. In addition, the nature of data preparation steps can vary from one type of data to another. For example, the way we prepare image data for developing a deep learning classification model is likely to be very different from the way we prepare text data for developing a movie review sentiment classification model. However, one important thing to note is that before we can develop deep learning models from unstructured data, they need to be first converted into a structured format. An example of converting unstructured image data into a structured format is shown in the following screenshot, using a picture of the handwritten digit five:

As can be observed from the preceding screenshot, when we read an image file containing a black and white handwritten digit five with 28 x 28 dimensions in R, it gets converted to numbers in rows and columns, giving it a structured format. The right-hand side of the screenshot shows data with 28 rows and 28 columns. The numbers in the body of the table are pixel values that range from 0 to 255, where a value of zero represents the black color and 255 represents the white color in the picture. When developing deep learning models, we make use of some forms of such structured data that are derived from image data.

Once the data for developing the model is prepared in the required format, we can then develop the model architecture.

Developing a deep learning model architecture

Developing the architecture of a model involves defining various items, such as the type and number of layers for the network, the type of activation function, the number of units or neurons to use in the network, and also providing the data-related input/output values. An example of specifying a simple sequential model architecture using Keras in R is shown in the following code:

# Model architecture
model <- keras_model_sequential()
 model %>% 
 layer_dense(units = 8, activation = 'relu', input_shape = c(21)) %>% 
 layer_dense(units = 3, activation = 'softmax')

Note that a sequential model allows us to develop models layer by layer. As seen from the preceding code, two layers of densely connected networks have been added as a part of the sequential model. Two important decisions while choosing a model architecture involve the number and type of layers...