Python Machine Learning

Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow 2, 3rd Edition

Sebastian Raschka, Vahid Mirjalili

• Publisher: Packt Publishing
• Year: 2019
• eBook ISBN: 9781789958294
• Edition: 3
• Topic: Computer Science
• Subtopic: Natural Language Processing
• Index: Computer Science

Applied machine learning with a solid foundation in theory. Revised and expanded for TensorFlow 2, GANs, and reinforcement learning.Purchase of the print or Kindle book includes a free eBook in the PDF format.Key Features• Third edition of the bestselling, widely acclaimed Python machine learning book• Clear and intuitive explanations take you deep into the theory and practice of Python machine learning• Fully updated and expanded to cover TensorFlow 2, Generative Adversarial Network models, reinforcement learning, and best practicesBook DescriptionPython Machine Learning, Third Edition is a comprehensive guide to machine learning and deep learning with Python. It acts as both a step-by-step tutorial, and a reference you'll keep coming back to as you build your machine learning systems.Packed with clear explanations, visualizations, and working examples, the book covers all the essential machine learning techniques in depth. While some books teach you only to follow instructions, with this machine learning book, Raschka and Mirjalili teach the principles behind machine learning, allowing you to build models and applications for yourself.Updated for TensorFlow 2.0, this new third edition introduces readers to its new Keras API features, as well as the latest additions to scikit-learn. It's also expanded to cover cutting-edge reinforcement learning techniques based on deep learning, as well as an introduction to GANs. Finally, this book also explores a subfield of natural language processing (NLP) called sentiment analysis, helping you learn how to use machine learning algorithms to classify documents.This book is your companion to machine learning with Python, whether you're a Python developer new to machine learning or want to deepen your knowledge of the latest developments.What you will learn• Master the frameworks, models, and techniques that enable machines to 'learn' from data• Use scikit-learn for machine learning and TensorFlow for deep learning• Apply machine learning to image classification, sentiment analysis, intelligent web applications, and more• Build and train neural networks, GANs, and other models• Discover best practices for evaluating and tuning models• Predict continuous target outcomes using regression analysis• Dig deeper into textual and social media data using sentiment analysisWho this book is forIf you know some Python and you want to use machine learning and deep learning, pick up this book. Whether you want to start from scratch or extend your machine learning knowledge, this is an essential resource. Written for developers and data scientists who want to create practical machine learning and deep learning code, this book is ideal for anyone who wants to teach computers how to learn from data.

17

Generative Adversarial Networks for Synthesizing New Data

In the previous chapter, we focused on recurrent neural networks for modeling sequences. In this chapter, we will explore generative adversarial networks (GANs) and see their application in synthesizing new data samples. GANs are considered to be the most important breakthrough in deep learning, allowing computers to generate new data (such as new images).

In this chapter, we will cover the following topics:

• Introducing generative models for synthesizing new data
• Autoencoders, variational autoencoders (VAEs), and their relationship to GANs
• Understanding the building blocks of GANs
• Implementing a simple GAN model to generate handwritten digits
• Understanding transposed convolution and batch normalization (BatchNorm or BN)
• Improving GANs: deep convolutional GANs and GANs using the Wasserstein distance

Introducing generative adversarial networks

Let's first look at the foundations of GAN models. The overall objective of a GAN is to synthesize new data that has the same distribution as its training dataset. Therefore, GANs, in their original form, are considered to be in the unsupervised learning category of machine learning tasks, since no labeled data is required. It is worth noting, however, that extensions made to the original GAN can lie in both semi-supervised and supervised tasks.

The general GAN concept was first proposed in 2014 by Ian Goodfellow and his colleagues as a method for synthesizing new images using deep neural networks (NNs) (Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y., Generative Adversarial Nets, in Advances in Neural Information Processing Systems, pp. 2672-2680, 2014). While the initial GAN architecture proposed in this paper was based on fully connected layers, similar to multilayer perceptron architectures, and trained to generate low-resolution MNIST-like handwritten digits, it served more as a proof of concept to demonstrate the feasibility of this new approach.

However, since its introduction, the original authors, as well as many other researchers, have proposed numerous improvements and various applications in different fields of engineering and science; for example, in computer vision, GANs are used for image-to-image translation (learning how to map an input image to an output image), image super-resolution (making a high-resolution image from a low-resolution version), image inpainting (learning how to reconstruct the missing parts of an image), and many more applications. For instance, recent advances in GAN research have led to models that are able to generate new, high-resolution face images. Examples of such high-resolution images can be found on https://www.thispersondoesnotexist.com/, which showcases synthetic face images generated by a GAN.

Starting with autoencoders

Before we discuss how GANs work, we will first start with autoencoders, which can compress and decompress training data. While standard autoencoders cannot generate new data, understanding their function will help you to navigate GANs in the next section.

Autoencoders are composed of two networks concatenated together: an encoder network and a decoder network. The encoder network receives a d-dimensional input feature vector associated with example x (that is,

) and encodes it into a p-dimensional vector, z (that is,

). In other words, the role of the encoder is to learn how to model the function

. The encoded vector, z, is also called the latent vector, or the latent feature representation. Typically, the dimensionality of the latent vector is less than that of the input examples; in other words, p < d. Hence, we can say that the encoder acts as a data compression function. Then, the decoder decompresses

from the lower-dimensional latent vector, z, where we can think of the decoder as a function,

. A simple autoencoder architecture is shown in the following figure, where the encoder and decoder parts consist of only one fully connected layer each:

The connection between autoencoders and dimensionality reduction

In Chapter 5, Compressing Data via Dimensionality Reduction, you learned about dimensionality reduction techniques, such as principal component analysis (PCA) and linear discriminant analysis (LDA). Autoencoders can be used as a dimensionality reduction technique as well. In fact, when there is no nonlinearity in either of the two subnetworks (encoder and d...

Table of contents

• Preface
• Giving Computers the Ability to Learn from Data
• Training Simple Machine Learning Algorithms for Classification
• A Tour of Machine Learning Classifiers Using scikit-learn
• Building Good Training Datasets – Data Preprocessing
• Compressing Data via Dimensionality Reduction
• Learning Best Practices for Model Evaluation and Hyperparameter Tuning
• Combining Different Models for Ensemble Learning
• Applying Machine Learning to Sentiment Analysis
• Embedding a Machine Learning Model into a Web Application
• Predicting Continuous Target Variables with Regression Analysis
• Working with Unlabeled Data – Clustering Analysis
• Implementing a Multilayer Artificial Neural Network from Scratch
• Parallelizing Neural Network Training with TensorFlow
• Going Deeper – The Mechanics of TensorFlow
• Classifying Images with Deep Convolutional Neural Networks
• Modeling Sequential Data Using Recurrent Neural Networks
• Generative Adversarial Networks for Synthesizing New Data
• Reinforcement Learning for Decision Making in Complex Environments
• Other Books You May Enjoy
• Index