R Deep Learning Cookbook

Dr. PKS Prakash, Achyutuni Sri Krishna Rao

• Publisher: Packt Publishing
• Year: 2017
• ISBN:
  9781787127111
• Edition: 1
• Topic: Computer Science
• Subtopic: Neural Networks
• Index: Computer Science

Powerful, independent recipes to build deep learning models in different application areas using R librariesAbout This Book• Master intricacies of R deep learning packages such as mxnet & tensorflow• Learn application on deep learning in different domains using practical examples from text, image and speech• Guide to set-up deep learning models using CPU and GPUWho This Book Is ForData science professionals or analysts who have performed machine learning tasks and now want to explore deep learning and want a quick reference that could address the pain points while implementing deep learning. Those who wish to have an edge over other deep learning professionals will find this book quite useful.What You Will Learn• Build deep learning models in different application areas using TensorFlow, H2O, and MXnet.• Analyzing a Deep boltzmann machine• Setting up and Analysing Deep belief networks• Building supervised model using various machine learning algorithms• Set up variants of basic convolution function• Represent data using Autoencoders.• Explore generative models available in Deep Learning.• Discover sequence modeling using Recurrent nets• Learn fundamentals of Reinforcement Leaning• Learn the steps involved in applying Deep Learning in text mining• Explore application of deep learning in signal processing• Utilize Transfer learning for utilizing pre-trained model• Train a deep learning model on a GPUIn DetailDeep Learning is the next big thing. It is a part of machine learning. It's favorable results in applications with huge and complex data is remarkable. Simultaneously, R programming language is very popular amongst the data miners and statisticians.This book will help you to get through the problems that you face during the execution of different tasks and Understand hacks in deep learning, neural networks, and advanced machine learning techniques. It will also take you through complex deep learning algorithms and various deep learning packages and libraries in R. It will be starting with different packages in Deep Learning to neural networks and structures. You will also encounter the applications in text mining and processing along with a comparison between CPU and GPU performance.By the end of the book, you will have a logical understanding of Deep learning and different deep learning packages to have the most appropriate solutions for your problems.Style and approachCollection of hands-on recipes that would act as your all-time reference for your deep learning needs

Generative Models in Deep Learning

In this chapter, we will cover the following topics:

• Comparing principal component analysis with the Restricted Boltzmann machine
• Setting up a Restricted Boltzmann machine for Bernoulli distribution input
• Training a Restricted Boltzmann machine
• Backward or reconstruction phase of RBM
• Understanding the contrastive divergence of the reconstruction
• Initializing and starting a new TensorFlow session
• Evaluating the output from an RBM
• Setting up a Restricted Boltzmann machine for Collaborative Filtering
• Performing a full run of training an RBM
• Setting up a Deep Belief Network
• Implementing a feed-forward backpropagation Neural Network
• Setting up a Deep Restricted Boltzmann Machine

Comparing principal component analysis with the Restricted Boltzmann machine

In this section, you will learn about two widely recommended dimensionality reduction techniques--Principal component analysis (PCA) and the Restricted Boltzmann machine (RBM). Consider a vector v in n-dimensional space. The dimensionality reduction technique essentially transforms the vector v into a relatively smaller (or sometimes equal) vector v' with m-dimensions (m<n). The transformation can be either linear or nonlinear.

PCA performs a linear transformation on features such that orthogonally adjusted components are generated that are later ordered based on their relative importance of variance capture. These m components can be considered as new input features, and can be defined as follows:

Vector v' =

Here, w and c correspond to weights (loading) and transformed components, respectively.

Unlike PCA, RBMs (or DBNs/autoencoders) perform non-linear transformations using connections between visible and hidden units, as described in Chapter 4, Data Representation Using Autoencoders. The nonlinearity helps in better understanding the relationship with latent variables. Along with information capture, they also tend to remove noise. RBMs are generally based on stochastic distribution (either Bernoulli or Gaussian).

A large amount of Gibbs sampling is performed to learn and optimize the connection weights between visible and hidden layers. The optimization happens in two passes: a forward pass where hidden layers are sampled using given visible layers and a backward pass where visible layers are resampled using given hidden layers. The optimization is performed to minimize the reconstruction error.

The following image represents a restricted Boltzmann machine:

Getting ready

For this recipe, you will require R (the rbm and ggplot2 packages) and the MNIST dataset. The MNIST dataset can be downloaded from the TensorFlow dataset library. The dataset consists of handwritten images of 28 x 28 pixels. It has 55,000 training examples and 10,000 test examples. It can be downloaded from the tensorflow library using the following script:

 library(tensorflow) 
datasets <- tf$contrib$learn$datasets 
mnist <- datasets$mnist$read_data_sets("MNIST-data", one_hot = TRUE) 
 

How to do it...

1. Extract the train dataset (trainX with all 784 independent variables and trainY with the respective 10 binary outputs):

 trainX <- mnist$train$images 
trainY <- mnist$train$labels 

2. Run a PCA on the trainX data:

 PCA_model <- prcomp(trainX, retx=TRUE) 

3. Run an RBM on the trainX data:

 RBM_model <- rbm(trainX, retx=TRUE, max_epoch=500,num_hidden =900) 

4. Predict on the train data using the generated models. In the case of the RBM model, generate probabilities:

 PCA_pred_train <- predict(PCA_model) 
RBM_pred_train <- predict(RBM_model,type='probs')  

5. Convert the outcomes into data frames:

 PCA_pred_train <- as.data.frame(PCA_pred_train)
 class="MsoSubtleEmphasis">RBM_pred_train <- as.data.frame(as.matrix(RBM_pred_train))
 

6. Convert the 10-class binary trainY data frame into a numeric vector:

 trainY_num<- as.numeric(stringi::stri_sub(colnames(as.data.frame(trainY))[max.col(as.data.frame(trainY),ties.method="first")],2))
 

7. Plot the components generated using PCA. Here, the x-axis represents component 1 and the y-axis represents component 2. The following image shows the outcome of the PCA model:

 ggplot(PCA_pred_train, aes(PC1, PC2))+
 geom_point(aes(colour = trainY))+ 
 theme_bw()+labs()+ 
 theme(plot.title = element_text(hjust = 0.5)) 

8. Plot the hidden layers generated using PCA. Here, the x-axis represents hidden 1 and y-axis represents hidden 2. The following image shows the outcome of the RBM model:

 ggplot(RBM_pred_train, aes(Hidden_2, Hidden_3))+
 geom_point(aes(colour = trainY))+ 
 theme_bw()+labs()+ 
 theme(plot.title = element_text(hjust = 0.5)) 

The following code and image shows the cumulative variance explained by the principal components:

 var_explain <- as.data.frame(PCA_model$sdev^2/sum(PCA_model$sdev^2)) 
var_explain <- cbind(c(1:784),var_explain,cumsum(var_explain[,1])) 
colnames(var_explain) <- c("PcompNo.","Ind_Variance","Cum_Variance") 
plot(var_explain$PcompNo.,var_explain$Cum_Variance, xlim = c(0,100),type='b',pch=16,xlab = "# of Principal Components",ylab = "Cumulative Variance",main = 'PCA - Explained variance') 

The following code and image shows the decrease in the reconstruction training error while generating an RBM using multiple epochs:

 plot(RBM_model,xlab = "# of epoch iterations",ylab = "Reconstruction error",main = 'RBM - Reconstruction Error') 

2323__perlego__chapter_...