Deep Learning with Structured Data

Mark Ryan

• Publisher: Manning
• Year: 2020
• ISBN:
  9781638357179
• Topic: Informatique
• Subtopic: Traitement des données

Deep Learning with Structured Data teaches you powerful data analysis techniques for tabular data and relational databases. Summary
Deep learning offers the potential to identify complex patterns and relationships hidden in data of all sorts. Deep Learning with Structured Data shows you how to apply powerful deep learning analysis techniques to the kind of structured, tabular data you'll find in the relational databases that real-world businesses depend on. Filled with practical, relevant applications, this book teaches you how deep learning can augment your existing machine learning and business intelligence systems.Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the technology
Here's a dirty secret: Half of the time in most data science projects is spent cleaning and preparing data. But there's a better way: Deep learning techniques optimized for tabular data and relational databases deliver insights and analysis without requiring intense feature engineering. Learn the skills to unlock deep learning performance with much less data filtering, validating, and scrubbing. About the book
Deep Learning with Structured Data teaches you powerful data analysis techniques for tabular data and relational databases. Get started using a dataset based on the Toronto transit system. As you work through the book, you'll learn how easy it is to set up tabular data for deep learning, while solving crucial production concerns like deployment and performance monitoring. What's inside When and where to use deep learning
The architecture of a Keras deep learning model
Training, deploying, and maintaining models
Measuring performance About the reader
For readers with intermediate Python and machine learning skills. About the author
Mark Ryan is a Data Science Manager at Intact Insurance. He holds a Master's degree in Computer Science from the University of Toronto. Table of Contents 1 Why deep learning with structured data?2 Introduction to the example problem and Pandas dataframes3 Preparing the data, part 1: Exploring and cleansing the data4 Preparing the data, part 2: Transforming the data5 Preparing and building the model6 Training the model and running experiments7 More experiments with the trained model8 Deploying the model9 Recommended next steps

1 Why deep learning with structured data?

This chapter covers

• A high-level overview of deep learning
• Benefits and drawbacks of deep learning
• Introduction to the deep learning software stack
• Structured versus unstructured data
• Objections to deep learning with structured data
• Advantages of deep learning with structured data
• Introduction to the code accompanying this book

Since 2012, we have witnessed what can only be called a renaissance of artificial intelligence. A discipline that had lost its way in the late 1980s is important again. What happened?

In October 2012, a team of students working with Geoffrey Hinton (a leading academic proponent of deep learning based at the University of Toronto) announced a result in the ImageNet computer vision contest that achieved an error rate in identifying objects that was close to half that of the nearest competitor. This result exploited deep learning and ushered in an explosion of interest in the topic. Since then, we have seen deep learning applications with world-class results in many domains, including image processing, audio to text, and machine translation. In the past couple of years, the tools and infrastructure for deep learning have reached a level of maturity and accessibility that make it possible for nonspecialists to take advantage of deep learning’s benefits. This book shows how you can use deep learning to get insights into and make predictions about structured data: data organized as tables with rows and columns, as in a relational database. You will see the capability of deep learning by going step by step through a complete, end-to-end example of deep learning, from ingesting the raw input structured data to making the deep learning model available to end users. By applying deep learning to a problem with a real-world structured dataset, you will see the challenges and opportunities of deep learning with structured data.

1.1 Overview of deep learning

Before reviewing the high-level concepts of deep learning, let’s introduce a simple example that we can use to explore these concepts: detection of credit card fraud. Chapter 2 introduces the real-world dataset and an extensive code example that prepares this dataset and uses it to train a deep learning model. For now, this basic fraud detection example is sufficient for a review of some of the concepts of deep learning.

Why would you want to exploit deep learning for fraud detection? There are several reasons:

• Fraudsters can find ways to work around the traditional rules-based approaches to fraud detection (http://mng.bz/emQw).
• A deep learning approach that is part of an industrial-strength pipeline—in which the model performance is frequently assessed and the model is automatically retrained if its performance drops below a given threshold—can adapt to changes in fraud patterns.
• A deep learning approach has the potential to provide near-real-time assessment of new transactions.

In summary, deep learning is worth considering for fraud detection because it can be the heart of a flexible, fast solution. Note that in addition to these advantages, there is a downside to using deep learning as a solution to the problem of fraud detection: compared with other approaches, deep learning is harder to explain. Other machine learning approaches allow you to determine which input characteristics most influence the outcome, but this relationship can be difficult or impossible to establish with deep learning.

Assume that a credit card company maintains customer transactions as records in a table. Each record in this table contains information about the transaction, including an ID that uniquely identifies the customer, as well as details about the transaction, including the date and time of the transaction, the ID of the vendor, the location of the transaction, and the currency and amount of the transaction. In addition to this information, which is added to the table every time a transaction is reported, every record has a field to indicate whether the transaction was reported as a fraud.

The credit card company plans to train a deep learning model on the historical data in this table and use this trained model to predict whether new incoming transactions are fraudulent. The goal is to identify potential fraud as quickly as possible (and take corrective action) rather than waiting days for the customer or vendor to report that a particular transaction is fraudulent.

Let’s examine the customer transaction table. Figure 1.1 contains a snippet of what some records in this table would look like.

Figure 1.1 Dataset for credit card fraud example

The columns customer ID, transaction date, transaction time, vendor ID, City, Country, currency, and amount contain details about individual credit card transactions for the previous quarter. The fraud column is special because it contains the label: the value that we want the deep learning model to predict when it has been trained on the training data. Assume that the default value in the fraud column is 0 (meaning “not a fraud”), and that when one of our customers or vendors reports a fraudulent transaction, the value in the fraud column for that transaction in the table is set to 1.

As new transactions arrive, we want to be able to predict whether they are fraudulent so that we can quickly take corrective action. By training the deep learning model on the historical dataset, we will be defining a function that can predict whether new credit card transactions are fraudulent. In this example of supervised learning (http://mng.bz/pzBE), the model is trained by means of a dataset that incorporates examples with labels. The dataset that is used to train the model includes the value that the trained model will predict (in this case, whether a transaction is fraudulent). By contrast, in unsupervised learning the training dataset does not include labels.

Now that we have introduced the credit card fraud example, let’s use it to take a brief tour of some of the concepts of deep learning. For a more in-depth description of these concepts, see François Chollet’s Deep Learning with Python, 2nd ed. (http://mng.bz/OvM2), which includes excellent descriptions of these concepts:

• Deep learning is a machine learning approach in which multilayer artificial neural networks are trained by setting weights and offsets at each layer by optimizing a loss function (the delta between the actual outcome [the values in the fraud column] and the predicted outcome) through the use of gradient-based optimization and backpropagation.
• Neural networks in a deep learning model have a series of layers, starting with the input layer, followed by several hidden layers, and culminating with an output layer.
• In each of these layers, the output of the previous layer (or, in the case of the first layer, the training data, which for our example is the dataset columns from customer ID, date, time, vendor ID, City, Country, currency and amount) goes through a series of operations (multiplication by a matrix of weights, addition of an offset [bias], and application of a nonlinear activation function) to produce the input for the next layer. In figure 1.2, each circle (node) has its own set of weights. The inputs are multiplied by those weights, the bias is added, and an activation function is applied to the result to produce the output that is taken in by the next layer.
  

  
  
  Figure 1.2 Multilayered neural network
  The final output layer generates the prediction of the model based on the input. In our example of predicting credit card fraud, the output indicates whether the model predicts a fraud (output of 1) or not a fraud (output of 0) for a given transaction.
• Deep learning works by iteratively updating the weights in the network to minimize the loss function (the function that defines the aggregate difference between the predictions of the model and the actual result values in the training dataset). As the weights are adjusted, the model’s predictions in aggregate get closer to the actual result values in the fraud column of the input table. With each training iteration, the weights are adjusted based on the gradient of the loss function.
• You can think of the gradient of the loss function as being roughly equivalent to the slope of a hill. If you make small, incremental steps in the direction opposite the slope of the hill, you will eventually get to the bottom of the hill. By making small changes to the weights in the direction opposite to the gradient for each iteration through the network, you reduce the loss function bit by bit. A process called backpropagation is used to get the gradient of the loss function, which can then be appl...