PyTorch Deep Learning Hands-On

Apply modern AI techniques with CNNs, RNNs, GANs, reinforcement learning, and more

Sherin Thomas, Sudhanshu Passi

• Publisher: Packt Publishing
• Year: 2019
• Print ISBN:
  9781788834131
• Edition: 1
• eBook ISBN:
  9781788833431
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

Hands-on projects cover all the key deep learning methods built step-by-step in PyTorch

Key Features

• Internals and principles of PyTorch
• Implement key deep learning methods in PyTorch: CNNs, GANs, RNNs, reinforcement learning, and more
• Build deep learning workflows and take deep learning models from prototyping to production

Book Description

PyTorch Deep Learning Hands-On is a book for engineers who want a fast-paced guide to doing deep learning work with Pytorch. It is not an academic textbook and does not try to teach deep learning principles. The book will help you most if you want to get your hands dirty and put PyTorch to work quickly.

PyTorch Deep Learning Hands-On shows how to implement the major deep learning architectures in PyTorch. It covers neural networks, computer vision, CNNs, natural language processing (RNN), GANs, and reinforcement learning. You will also build deep learning workflows with the PyTorch framework, migrate models built in Python to highly efficient TorchScript, and deploy to production using the most sophisticated available tools.

Each chapter focuses on a different area of deep learning. Chapters start with a refresher on how the model works, before sharing the code you need to implement them in PyTorch.

This book is ideal if you want to rapidly add PyTorch to your deep learning toolset.

What you will learn

Use PyTorch to build:

• Simple Neural Networks – build neural networks the PyTorch way, with high-level functions, optimizers, and more
• Convolutional Neural Networks – create advanced computer vision systems
• Recurrent Neural Networks – work with sequential data such as natural language and audio
• Generative Adversarial Networks – create new content with models including SimpleGAN and CycleGAN
• Reinforcement Learning – develop systems that can solve complex problems such as driving or game playing
• Deep Learning workflows – move effectively from ideation to production with proper deep learning workflow using PyTorch and its utility packages
• Production-ready models – package your models for high-performance production environments

Who this book is for

Machine learning engineers who want to put PyTorch to work.

PyTorch Deep Learning Hands-On

---

Table of Contents

PyTorch Deep Learning Hands-On

Why subscribe?

Packt.com

Contributors

About the authors

About the reviewers

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Download the color images

Conventions used

Get in touch

Reviews

1. Deep Learning Walkthrough and PyTorch Introduction

Understanding PyTorch's history

What is PyTorch?

Installing PyTorch

What makes PyTorch popular?

Using computational graphs

Using static graphs

Using dynamic graphs

Exploring deep learning

Getting to know different architectures

Fully connected networks

Encoders and decoders

Recurrent neural networks

Recursive neural networks

Convolutional neural networks

Generative adversarial networks

Reinforcement learning

Getting started with the code

Learning the basic operations

The internals of PyTorch

Summary

References

2. A Simple Neural Network

Introduction to the neural network

The problem

Dataset

Novice model

Autograd

Autograd attributes of a tensor

Building the graph

Finding error

Backpropagation

Parameter update

The PyTorch way

High-level APIs

nn.Module

apply()

cuda() and cpu()

train() and eval()

parameters()

zero_grad()

Other layers

The functional module

The loss function

Optimizers

Summary

References

3. Deep Learning Workflow

Ideation and planning

Design and experimentation

The dataset and DataLoader classes

Utility packages

torchvision

torchtext

torchaudio

Model implementation

Bottleneck and profiling

Training and validation

Ignite

Engine

Events

Metrics

Saving checkpoints

Summary

References

4. Computer Vision

Introduction to CNNs

Computer vision with PyTorch

Simple CNN

Model

Semantic segmentation

LinkNet

Deconvolution

Skip connections

Model

ConvBlock

DeconvBlock

Pooling

EncoderBlock

DecoderBlock

Summary

References

5. Sequential Data Processing

Introduction to recurrent neural networks

The problem

Approaches

Simple RNN

Word embedding

RNNCell

Utilities

Pad sequence

Pack sequence

Encoder

Classifier

Dropout

Training

Advanced RNNs

LSTM

GRUs

Architecture

LSTMCell and GRUCell

LSTMs and GRUs

Increasing the number of layers

Bidirectional RNN

Classifier

Attention

Recursive neural networks

Reduce

Tracker

SPINN

Summary

References

6. Generative Networks

Defining the approaches

Autoregressive models

PixelCNN

Masked convolution

Gated PixelCNN

WaveNet

GANs

Simple GAN

CycleGAN

Summary

References

7. Reinforcement Learning

The problem

Episodic versus continuous tasks

Cumulative discounted rewards

Markov decision processes

The solution

Policies and value functions

Bellman equation

Finding the optimal Q-function

Deep Q-learning

Experience replay

Gym

Summary

References

8. PyTorch to Production

Serving with Flask

Introduction to Flask

Model serving with Flask

A production-ready server

ONNX

MXNet model server

MXNet model archiver

Load testing

Efficiency with TorchScript

Exploring RedisAI

Summary

References

Another Book You May Enjoy

Leave a review - let other readers know what you think

Index

PyTorch Deep Learning Hands-On

Copyright © 2019 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Acquisition Editor: Andrew Waldron

Acquisition Editor - Peer Reviews: Suresh Jain

Project Editor: Tom Jacob

Development Editor: Joanne Lovell

Technical Editor: Gaurav Gavas

Proofreader: Safis Editing

Indexer: Rekha Nair

Graphics: Sandip Tadge, Tom Scaria

Production Coordinator: Sandip Tadge

First published: April 2019

Production reference: 1250419

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78883-413-1

www.packtpub.com

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

• Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
• Learn better with Skill Plans built especially for you
• Get a free eBook or video every month
• Mapt is fully searchable
• Copy and paste, print, and bookmark content

Packt.com

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.Packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.Packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Contributors

About the authors

Sherin Thomas started his career as an information security expert and shifted his focus to deep learning-based security systems. He has helped several companies across the globe to set up their AI pipelines and worked recently for CoWrks, a fast-growing start-up based out of Bengaluru. Sherin is working on several open source projects including PyTorch, RedisAI, and many more, and is leading the development of TuringNetwork.ai. Currently, he is focusing on building the deep learning infrastructure for [tensor]werk, an Orobix spin-off company.

I am indebted to a multitude of professionals who have influenced me and motivated me to write this book. Among them are my colleagues from CoWrks and my friends. I can't thank enough the technical reviewers and editorial assistants. Without them, I would not have been able to meet the deadlines. Last, and most importantly, I am indebted to my wife, Merin. Writing a book along with a day job is not easy, and without her, it would have been impossible.

Sudhanshu Passi is a technologist employed at CoWrks. Among other things, he has been the driving force behind everything related to machine learning at CoWrks. His expertise in simplifying complex concepts makes his work an ideal read for beginners and experts alike. This can be verified by his many blogs and this debut book publication. In his spare time, he can be found at his local swimming pool computing gradient descent underwater.

I would like to thank Sherin for this opportunity to be a co-author on this book. I would also like to thank my parents for their continuous support throughout the years.

About the reviewers

Bharath G. S. is an independent machin...

Table of contents

• PyTorch Deep Learning Hands-On