TensorFlow 2 Reinforcement Learning Cookbook

Over 50 recipes to help you build, train, and deploy learning agents for real-world applications

Praveen Palanisamy

• Publisher: Packt Publishing
• Year: 2021
• Print ISBN: 9781838982546
• eBook ISBN: 9781838985998
• Edition: 1
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

Discover recipes for developing AI applications to solve a variety of real-world business problems using reinforcement learning

Key Features

• Develop and deploy deep reinforcement learning-based solutions to production pipelines, products, and services
• Explore popular reinforcement learning algorithms such as Q-learning, SARSA, and the actor-critic method
• Customize and build RL-based applications for performing real-world tasks

Book Description

With deep reinforcement learning, you can build intelligent agents, products, and services that can go beyond computer vision or perception to perform actions. TensorFlow 2.x is the latest major release of the most popular deep learning framework used to develop and train deep neural networks (DNNs). This book contains easy-to-follow recipes for leveraging TensorFlow 2.x to develop artificial intelligence applications.

Starting with an introduction to the fundamentals of deep reinforcement learning and TensorFlow 2.x, the book covers OpenAI Gym, model-based RL, model-free RL, and how to develop basic agents. You'll discover how to implement advanced deep reinforcement learning algorithms such as actor-critic, deep deterministic policy gradients, deep-Q networks, proximal policy optimization, and deep recurrent Q-networks for training your RL agents. As you advance, you'll explore the applications of reinforcement learning by building cryptocurrency trading agents, stock/share trading agents, and intelligent agents for automating task completion. Finally, you'll find out how to deploy deep reinforcement learning agents to the cloud and build cross-platform apps using TensorFlow 2.x.

By the end of this TensorFlow book, you'll have gained a solid understanding of deep reinforcement learning algorithms and their implementations from scratch.

What you will learn

• Build deep reinforcement learning agents from scratch using the all-new TensorFlow 2.x and Keras API
• Implement state-of-the-art deep reinforcement learning algorithms using minimal code
• Build, train, and package deep RL agents for cryptocurrency and stock trading
• Deploy RL agents to the cloud and edge to test them by creating desktop, web, and mobile apps and cloud services
• Speed up agent development using distributed DNN model training
• Explore distributed deep RL architectures and discover opportunities in AIaaS (AI as a Service)

Who this book is for

The book is for machine learning application developers, AI and applied AI researchers, data scientists, deep learning practitioners, and students with a basic understanding of reinforcement learning concepts who want to build, train, and deploy their own reinforcement learning systems from scratch using TensorFlow 2.x.

Chapter 1: Developing Building Blocks for Deep Reinforcement Learning Using Tensorflow 2.x

This chapter provides a practical and concrete description of the fundamentals of Deep Reinforcement Learning (Deep RL) filled with recipes for implementing the building blocks using the latest major version of TensorFlow 2.x. It includes recipes for getting started with RL environments, OpenAI Gym, developing neural network-based agents, and evolutionary neural agents for addressing applications with both discrete and continuous value spaces for Deep RL.

The following recipes are discussed in this chapter:

• Building an environment and reward mechanism for training RL agents
• Implementing neural network-based RL policies for discrete action spaces and decision-making problems
• Implementing neural network-based RL policies for continuous action spaces and continuous-control problems
• Working with OpenAI Gym for RL training environments
• Building a neural agent
• Building a neural evolutionary agent

Technical requirements

The code in the book has been extensively tested on Ubuntu 18.04 and Ubuntu 20.04 and should work with later versions of Ubuntu as long as Python 3.6+ is available. With Python 3.6 installed along with the necessary Python packages as listed before the start of each of the recipes, the code should run fine on Windows and macOS X too. It is advised to create and use a Python virtual environment named tf2rl-cookbook to install the packages and run the code in this book. Miniconda or Anaconda installation for Python virtual environment management is recommended.The complete code for each recipe in this chapter will be available here: https://github.com/PacktPublishing/Tensorflow-2-Reinforcement-Learning-Cookbook.

Building an environment and reward mechanism for training RL agents

This recipe will walk you through the steps to build a Gridworld learning environment to train RL agents. Gridworld is a simple environment where the world is represented as a grid. Each location on the grid can be referred to as a cell. The goal of an agent in this environment is to find its way to the goal state in a grid like the one shown here:

Figure 1.1 – A screenshot of the Gridworld environment

The agent's location is represented by the blue cell in the grid, while the goal and a mine/bomb/obstacle's location is represented in the grid using green and red cells, respectively. The agent (blue cell) needs to find its way through the grid to reach the goal (green cell) without running over the mine/bomb (red cell).

Getting ready

To complete this recipe, you will first need to activate the tf2rl-cookbook Python/Conda virtual environment and pip install numpy gym. If the following import statements run without issues, you are ready to get started!

import copy

import sys

import gym

import ...

Table of contents

• TensorFlow 2 Reinforcement Learning Cookbook
• Why subscribe?
• Preface
• Chapter 1: Developing Building Blocks for Deep Reinforcement Learning Using Tensorflow 2.x
• Chapter 2: Implementing Value-Based, Policy-Based, and Actor-Critic Deep RL Algorithms
• Chapter 3: Implementing Advanced RL Algorithms
• Chapter 4: Reinforcement Learning in the Real World – Building Cryptocurrency Trading Agents
• Chapter 5: Reinforcement Learning in the Real World – Building Stock/Share Trading Agents
• Chapter 6: Reinforcement Learning in the Real World – Building Intelligent Agents to Complete Your To-Dos
• Chapter 7: Deploying Deep RL Agents to the Cloud
• Chapter 8: Distributed Training for Accelerated Development of Deep RL Agents
• Chapter 9: Deploying Deep RL Agents on Multiple Platforms
• Other Books You May Enjoy