Tree-Based Machine Learning Methods in SAS Viya

Sharad Saxena

• Publisher: SAS Institute
• Year: 2022
• Print ISBN: 9781954846630, 9781954846715
• eBook ISBN: 9781954846654
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

Discover how to build decision trees using SAS Viya!

Tree-Based Machine Learning Methods in SAS Viya covers everything from using a single tree to more advanced bagging and boosting ensemble methods. The book includes discussions of tree-structured predictive models and the methodology for growing, pruning, and assessing decision trees, forests, and gradient boosted trees. Each chapter introduces a new data concern and then walks you through tweaking the modeling approach, modifying the properties, and changing the hyperparameters, thus building an effective tree-based machine learning model. Along the way, you will gain experience making decision trees, forests, and gradient boosted trees that work for you.

By the end of this book, you will know how to:

• build tree-structured models, including classification trees and regression trees.
• build tree-based ensemble models, including forest and gradient boosting.
• run isolation forest and Poisson and Tweedy gradient boosted regression tree models.
• implement open source in SAS and SAS in open source.
• use decision trees for exploratory data analysis, dimension reduction, and missing value imputation.

Chapter 1: Introduction to Tree-Structured Models

Introduction

“Sometimes you make the right decision, sometimes you make the decision right.”

–Phil McGraw

A decision tree has many analogies in real life. In decision analysis, a tree can be used to represent decisions and decision making visually and explicitly. As the name suggests, it uses a tree-like model of decisions.

The adjective decision in decision trees is a curious one, and misleading. In the 1960s, originators of the tree approach described the splitting rules as decision rules. The terminology remains popular. This is ill-fated because it inhibits the use of ideas and terminology from decision theory. The term decision tree is used in decision theory to depict a series of decisions for choosing alternative activities. You create the tree and specify probabilities and benefits of outcomes of the activities. Software, including SAS, finds the most beneficial path. The project follows a single path and never performs the unchosen activities. The decider follows a path based on a set of criteria.

Decision theory is not about data analysis. The choice of a decision might be made without reference to data. The trees in this book are only about data analysis. A tree is fit to a data set to enable interpretation and prediction of data. An apt name would be data-splitting trees that would be used for supervised learning also called predictive modeling.

In supervised learning, a set of input variables (predictors) is used to predict the value of one or more target variables (outcome). The mapping of the inputs to the target is a predictive model. The goal is to create a model that predicts the value of a target variable by learning simple decision rules inferred from the input variables. The data used to estimate a predictive model is a set of cases (observations, examples) consisting of values of the inputs and target. The fitted model is typically applied to new cases where the target is unknown.

Decision Tree – What Is It?

There are several tree-structured models that include one or more decision trees. Decision trees are a fundamental machine learning technique that every data scientist should know. Luckily, the construction and implementation of decision trees in SAS Viya is straightforward and easy to produce.

A decision tree represents a grouping of the data that is created by applying a series of simple rules. Each rule assigns an observation to a group based on the value of one input. One rule is applied after another, resulting in a hierarchy of groups within groups. The hierarchy is called a tree, and each group is called a node. The original group contains the entire data set and is called the root node of the tree. A node with all its successors forms a branch of the node that created it. The final nodes are called leaves. For each leaf, a decision is made and applied to all observations in the leaf. The type of decision depends on the context. In supervised learning, the decision is the predicted value.

You use the decision tree to do one of the following tasks:

• classify observations based on the values of nominal, binary, or ordinal targets
• predict outcomes for interval targets
• predict the appropriate decision when you specify decision alternatives

The tree depicts the first split into groups as branches emanating from a root and subsequent splits as branches emanating from nodes on older branches. Figure 1.1 is an example decision tree predicting a nominal target Cause of Death using two binary inputs Weight Status and Smoking Status. The decision nodes include a bar chart related to the node’s sample target values and other details. The leaves of the tree are the f...

Table of contents

• Cover
• Copyright Page
• Contents
• About This Book
• About These Authors
• Acknowledgments
• Foreword
• Chapter 1: Introduction to Tree-Structured Models
• Chapter 2: Classification and Regression Trees
• Chapter 3: Growing a Decision Tree
• Chapter 4: Decision Trees: Strengths, Weaknesses, and Uses
• Chapter 5: Tuning a Decision Tree
• Chapter 6: Ensemble of Trees: Bagging, Boosting, and Forest
• Chapter 7: Additional Forest Models
• Chapter 8: Tree-Based Gradient Boosting Machines
• Chapter 9: Additional Gradient Boosting Models
• Appendix A: Practice Case Study
• References