Hands-On Deep Learning for Games

Leverage the power of neural networks and reinforcement learning to build intelligent games

Micheal Lanham

• Publisher: Packt Publishing
• Year: 2019
• Print ISBN: 9781788994071
• eBook ISBN: 9781788998765
• Edition: 1
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

Understand the core concepts of deep learning and deep reinforcement learning by applying them to develop games

Key Features

• Apply the power of deep learning to complex reasoning tasks by building a Game AI
• Exploit the most recent developments in machine learning and AI for building smart games
• Implement deep learning models and neural networks with Python

Book Description

The number of applications of deep learning and neural networks has multiplied in the last couple of years. Neural nets has enabled significant breakthroughs in everything from computer vision, voice generation, voice recognition and self-driving cars. Game development is also a key area where these techniques are being applied. This book will give an in depth view of the potential of deep learning and neural networks in game development.

We will take a look at the foundations of multi-layer perceptron's to using convolutional and recurrent networks. In applications from GANs that create music or textures to self-driving cars and chatbots. Then we introduce deep reinforcement learning through the multi-armed bandit problem and other OpenAI Gym environments.

As we progress through the book we will gain insights about DRL techniques such as Motivated Reinforcement Learning with Curiosity and Curriculum Learning. We also take a closer look at deep reinforcement learning and in particular the Unity ML-Agents toolkit. By the end of the book, we will look at how to apply DRL and the ML-Agents toolkit to enhance, test and automate your games or simulations. Finally, we will cover your possible next steps and possible areas for future learning.

What you will learn

• Learn the foundations of neural networks and deep learning.
• Use advanced neural network architectures in applications to create music, textures, self driving cars and chatbots.
• Understand the basics of reinforcement and DRL and how to apply it to solve a variety of problems.
• Working with Unity ML-Agents toolkit and how to install, setup and run the kit.
• Understand core concepts of DRL and the differences between discrete and continuous action environments.
• Use several advanced forms of learning in various scenarios from developing agents to testing games.

Who this book is for

This books is for game developers who wish to create highly interactive games by leveraging the power of machine and deep learning. No prior knowledge of machine learning, deep learning or neural networks is required this book will teach those concepts from scratch. A good understanding of Python is required.

Section 1: The Basics

This section of the book covers the basic concepts of neural networks and deep learning. We'll be looking at everything from the simplest autoencoder, generative adversarial networks (GANs), and convolutional and recurrent neural networks, all the way to building a working real-world chatbot. This section will give you the basic foundations for building your neural network and deep learning knowledge.

We will include the following chapters in this section:

• Chapter 1, Deep Learning for Games
• Chapter 2, Convolutional and Recurrent Networks
• Chapter 3, GAN for Games
• Chapter 4, Building a Deep Learning Gaming Chatbot

Deep Learning for Games

Welcome to Hands-on Deep Learning for Games. This book is for anyone wanting an extremely practical approach to the complexity of deep learning (DL) for games. Importantly, the concepts discussed in this book aren't solely limited to games. Much of what we'll learn here will easily carry over to other applications/simulations.

Reinforcement learning (RL), which will be a core element we talk about in later chapters, is quickly becoming the dominant machine learning (ML) technology. It has been applied to everything from server optimization to predicting customer activity for retail markets. Our journey in this book will primarily be focused on game development, and our goal will be to build a working adventure game. Keep in the back of your mind how the same principles you discover in this book could be applied to other problems, such as simulations, robotics, and lots more.

In this chapter, we are going to start from the very basics of neural networks and deep learning. We will discuss the background of neural networks, working our way toward building a neural network that can play a simple text game. Specifically, this chapter will cover the following topics:

• The past, present, and future of DL
• Neural networks – the foundation
• Multilayer perceptron in TensorFlow (TF)
• Understanding TensorFlow
• Training neural networks with backpropagation
• Building an Autoencoder in Keras

This book assumes that you have a working knowledge of Python. You should be able to set up and activate a virtual environment. Later chapters will use Unity 3D, which is limited to Windows and macOS (apologies to those hardcore Linux users).

You might be inclined to skip this chapter if you've already grasped deep learning. Regardless, this chapter is well worth reading and will establish the terminology we use throughout the book. At the very least, do the hands-on exercise—you will thank yourself later!

The past, present, and future of DL

While the term deep learning was first associated with neural networks in 2000 by Igor Aizenberg and colleagues, it has only become popular in the last 5 years. Prior to this, we called this type of algorithm an artificial neural network (ANN). However, deep learning refers to something broader than ANNs and includes many other areas of connected machines. Therefore, to clarify, we will be discussing the ANN form of DL for much of the remainder of this book. However, we will also discuss some other forms of DL that can be used in games, in Chapter 5, Introducing DRL.

The past

The first form of a multilayer perceptron (MLP) network, or what we now call an ANN, was introduced by Alexey Ivakhnenko in 1965. Ivakhnenko waited several years before writing about the multilayer perceptron in 1971. The concept took a while to percolate and it wasn't until the 1980s that more research began. This time, image classification and speech recognition were attempted, and they failed, but progress was being made. It took another 10 years, and in the late 90s, it became popular again. So much so that ANNs made their way into some games, again, until better methods came along. Things quietened down and another decade or so passed.

Then, in 2012, Andrew Ng and Jeff Dean used an ANN to recognize cats in videos, and the interest in deep learning exploded. Their stride was one of several trivial (yet entertaining) advancements which made people sit up and take notice of deep learning. Then, in 2015, Google's DeepMind team built AlphaGo, and this time the whole world sat up. AlphaGo was shown to solidly beat the best players in the world at the game of Go, and that changed everything. Other techniques soon followed, Deep Reinforcement Learning (DRL) being one, showing that human performance could be consistently beaten in areas where that was previously not thought of as possible.

While teaching their students about neural networks, there is a humorous and pertinent tale professors enjoy sharing: The US Army did early research in the '80s using an ANN to recognize enemy tanks. The algorithm worked 100% of the time, and the army organized a big demonstration to showcase its success. Unfortunately, nothing worked at the demonstration, and every test failed miserably. After going back and analyzing things, the army realized the ANN wasn't recognizing enemy tanks at all. Instead, it had been trained on images taken on a cloudy day, and all it was doing was recognizing the clouds.

The present

At present, at least at the time of writing, we are still in the midst of a deep learning explosion with debris and chaos, and it is our job as developers to make sense of all this. Neural networks are currently the basis for many DL technologies, several of which we will cover in this book. Except, it seems that every day, new and more powerful techniques emerge, and researchers scramble to make sense of them. Now, this rush of ideas can actually stall a technology, as researchers spend more and more time trying to replicate results. It is most certainly a cause for much of the earlier stalling that ANNs (deep learning) previously suffered from. In fact, many skeptics in the industry are pre...

Table of contents

• Title Page
• Copyright and Credits
• Dedication
• About Packt
• Contributors
• Preface
• Section 1: The Basics
• Deep Learning for Games
• Convolutional and Recurrent Networks
• GAN for Games
• Building a Deep Learning Gaming Chatbot
• Section 2: Deep Reinforcement Learning
• Introducing DRL
• Unity ML-Agents
• Agent and the Environment
• Understanding PPO
• Rewards and Reinforcement Learning
• Imitation and Transfer Learning
• Building Multi-Agent Environments
• Section 3: Building Games
• Debugging/Testing a Game with DRL
• Obstacle Tower Challenge and Beyond
• Other Books You May Enjoy