Decision Trees

12 Key excerpts on "Decision Trees"

• 

  eBook - PDF

  Value-Added Decision Making for Managers

  • Kenneth Chelst, Yavuz Burak Canbolat(Authors)
  • 2011(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)

  10.1 Goal and Overview The goal of this chapter is to present Decision Trees as an analytic approach to making decisions involving uncertainty. It is a simpler, more transparent modeling tool than stochastic simulation discussed in the previous chapter. It is a normative decision-making tool that identifies the optimal decision based on expected value or expected utility. Stochastic simulation, in contrast, is primarily descriptive. Like simulation, the decision tree output includes a risk profile of all the alternatives. Decisions are first compared through expected value analysis; the optimal decision is the one with the best expected value. Further analysis enables the decision maker to understand and inter-pret the related strengths and weaknesses of the alternatives with the goal of developing an even better alternative that mitigates some of the risks. The tool is used in situations where the decision maker is faced with a discrete set of limited alternatives, and uncertainty plays a major role in the future outcome of these decisions. 278 Value Added Decision Making for Managers The first task in developing a decision tree is to identify key quantifiable uncertainties that directly impact the outcome of the decision. As a starting point, Table 10.1 presents some ran-dom variables and outcomes that arise in a variety of decision contexts. The key challenge is to explicitly account for the uncertainty upfront, not to drive the uncertainty out of the decision process. 10.2 Early Users of Decision Trees The oil industry was one of the first to use Decision Trees, incorporating available information to help determine the risks involved in various decisions. Significant uncertainty is commonplace in the industry, involving billions of dollars in costs and revenues. Uncertain variables include the potential yield of oilfields, the cost of extraction, political instability in various regions, and the fluctuating price of oil.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Decision Analysis for Management Judgment

  • Paul Goodwin, George Wright(Authors)
  • 2014(Publication Date)
  • Wiley

    (Publisher)

  They can help a decision maker to develop a clear view of the structure of a problem and make it easier to determine the possible scenar- ios which can result if a particular course of action is chosen. This can lead to creative thinking and the generation of options which were not previously being considered. Decision Trees can also help a decision maker to judge the nature of the information which needs to be gathered in order to tackle a problem and, because they are 162 Decision Trees AND INFLUENCE DIAGRAMS generally easy to understand, they can be an excellent medium for communicating one person’s perception of a problem to other individuals. The process of constructing a decision tree is usually iterative, with many changes being made to the original structure as the decision maker’s understanding of the problem develops. Because the intention is to help the decision maker to think about the problem, very large and complex trees, which are designed to represent every pos- sible scenario that can occur, may be counterproductive in many circumstances. Deci- sion trees are models, and as such are simplifications of the real problem. The simplifi- cation is the very strength of the modeling process because it fosters the understanding and insight which would be obscured by detail and complexity. Nevertheless, in rare circumstances highly complex trees may be appropriate and software developments mean that their structuring and analysis can now be facilitated with relative ease. For example, Dunning et al. 1 used software to apply a decision tree with over 200 mil- lion paths to a 10-year scheduling problem faced by the New York Power Authority. Similarly, Beccue 2 used a tree with around half-a-million scenarios to help a pharma- ceutical company make decisions relating to the development and marketing of a new drug.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Machine Learning with R

  Learn techniques for building and improving machine learning models, from data preparation to model tuning, evaluation, and working with big data, 4th Edition

  • Brett Lantz(Author)
  • 2023(Publication Date)
  • Packt Publishing

    (Publisher)

  A great benefit of decision tree algorithms is that the flowchart-like tree structure is not only for the machine’s internal use. After the model is created, many decision tree algorithms output the resulting structure in a human-readable format. This provides insight into how and why the model works or doesn’t work well for a particular task. This also makes Decision Trees particularly appropriate for applications in which the classification mechanism needs to be transparent for legal reasons, or if the results need to be shared with others to inform future business practices. With this in mind, some potential uses include the following:

  • Credit scoring models in which the criteria that cause an applicant to be rejected need to be clearly documented and free from bias
  • Marketing studies of customer behavior, such as satisfaction or churn, which will be shared with management or advertising agencies
  • Diagnosis of medical conditions based on laboratory measurements, symptoms, or rates of disease progression

  Although the previous applications illustrate the value of trees in informing decision-making processes, this is not to suggest that their utility ends here. In fact, Decision Trees are one of the single most widely used machine learning techniques, and can be applied to model almost any type of data—often with excellent out-of-the-box performance.

  That said, despite their wide applicability, it is worth noting that there are some scenarios where trees may not be an ideal fit. This includes tasks where the data has many nominal features with many levels or a large number of numeric features. These cases may result in a very large number of decisions and an overly complex tree. They may also contribute to the tendency of Decision Trees to overfit data, though as we will soon see, even this weakness can be overcome by adjusting some simple parameters.

  Divide and conquer

  Decision Trees are built using a heuristic called recursive partitioning . This approach is also commonly known as divide and conquer

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Knowledge Discovery from Data Streams

  • Joao Gama(Author)
  • 2010(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)

  Chapter 8 Decision Trees from Data Streams A decision tree uses a divide-and-conquer strategy. It attacks a complex prob-lem by dividing it into simpler problems and recursively applies the same strategy to the sub-problems. Solutions of sub-problems can be combined to yield a solution of the complex problem. The power of this approach comes from the ability to split the instance-space into subspaces and each subspace is fitted with different models. Decision Trees are one of the most used algorithms in the data mining community: they are distribution-free, and tree models ex-hibit a high degree of interpretability. These factors strongly contribute to their increasing popularity in the data mining community. 8.1 Introduction Formally, a decision tree is a direct acyclic graph in which each node is either a decision node with two or more successors or a leaf node . In the simplest model, a leaf node is labeled with a class label and a decision node has some condition based on attribute values. Decision Trees are one of the most used methods in data mining mainly because of their high degree of interpretability. The hypothesis space of Decision Trees is within the disjunctive normal form (DNF) formalism. Classifiers generated by those systems encode a DNF for each class. For each DNF, the conditions along a branch represent conjuncts and the individual branches can be seen as disjuncts. Each branch forms a rule with a conditional part and a conclusion. The conditional part is a conjunction of conditions. Conditions are tests that involve a particular attribute, operator (e.g., =, ≥ , ∈ , etc.) and a value from the domain of that attribute. These kind of tests correspond, in the input space, to a hyper-plane that is orthogonal to the axes of the tested attribute and parallel to all other axis. The regions produced by these classifiers are all hyper-rectangles.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Business Analytics

  The Art of Modeling With Spreadsheets

  • Stephen G. Powell, Kenneth R. Baker(Authors)
  • 2016(Publication Date)
  • Wiley

    (Publisher)

  The analysis of the interrelated decisions of two (or more) actors is quite difficult, and we will not discuss it at length except to mention that representing a competitor’s actions as random probably overstates the value we can extract from the situation. 13.3.3 Principles for Building and Analyzing Decision Trees Decision Trees can be built using a fairly standard procedure. Judgment is required, however, in determining which decisions and uncertainties are critical to the situation and therefore must be captured in the tree, as well as in selecting the specific choices and states to recognize. Beginners tend to draw overly complex trees. The more experienced analyst starts with a small number of nodes and a small number of outcomes and expands only when the results suggest that more detail is required. Here is a general procedure for constructing trees: 1. Determine the essential decisions and uncertainties. 2. Place the decisions and uncertainties in the appropriate temporal sequence. 3. Start the tree with a decision node representing the first decision. 4. Select a representative (but not necessarily exhaustive) number of possible choices for the decision node. 5. For each choice, draw a chance node representing the first uncertain event that follows the initial decision. 6. Select a representative (but not necessarily exhaustive) number of possible states for the chance node. 7. Continue to expand the tree with additional decision nodes and chance nodes until the overall outcome can be evaluated. Decision Trees can be used to develop a purely qualitative understanding of a situation, although they are usually used for quantitative analysis as well. This quantitative analysis can identify which decisions are best and can construct the probability distribution of results. To carry out this analysis, we need two types of information: the probability for each possible state and the overall value of arriving at the end of the tree along a particular path.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  A User's Guide to Business Analytics

  • Ayanendranath Basu, Srabashi Basu(Authors)
  • 2016(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)

  Based on data already in the database, and based on the clinical symptoms and ultimate outcome (death or survival) of the patients, a new patient may be classified into high-risk and low-risk categories. Decision Trees have very high applicability in industrial recommender systems. A recommender system recommends items or products to prospective buyers based on their purchase pattern. In many such cases, especially if the store does not have any information on the customer other than their previous purchases, or in the case of online stores, prospective buy- ers’ choices and click-stream responses, regression or any other model building procedures will not be applicable. Decision Trees are applicable in marketing to identify homogeneous groups for target marketing. Decision Trees segment the predictor space into several similar regions and use the mean of the con- tinuous response or mode of categorical response in each separate region for prediction. 277 278 A User’s Guide to Business Analytics 10.1 Algorithm for Tree-Based Methods Classification and regression tree (CART) is a recursive binary splitting al- gorithm introduced by Breiman et al. (1984). Even though there have been many improvements on this method, the core algorithm is still identical to that of CART. Consider the set of all predictors, continuous and categorical, together. CART does not make any differentiation between categorical and continuous predictors. CART is based on nodes and split. The full sample is known as the root node. At each instance of split, a variable and its level is selected, so that purity at each child node is the highest possible at that level. The basic idea of tree growing is to choose a split among all possible splits at each node so that the resultant child nodes are the purest. At each split the sample space is partitioned according to one predictor. Only univariate splits are considered, so that the partitions are parallel to the axes.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Machine Learning

  Hands-On for Developers and Technical Professionals

  • Jason Bell(Author)
  • 2020(Publication Date)
  • Wiley

    (Publisher)

  C H A P T E R 81 5 Do not be deceived by the decision tree; at first glance it might look like a simple concept, but within the simplicity lies its power. This chapter shows you how Decision Trees work. The examples use Weka to create a working decision tree that will also create the Java code for you. The Basics of Decision Trees The aim of any decision tree is to create a workable model that will predict the value of a target variable based on the set of input variables. This section explains where Decision Trees are used along with some of the advantages and limitations of Decision Trees. In this section, you also find out how a decision tree is calculated manually so you can see the math involved. Uses for Decision Trees Think about how you select different options within an automated telephone call. The options are essentially decisions that are being made for you to get to the desired department. These Decision Trees are used effectively in many industry areas. Financial institutions use Decision Trees. One of the fundamental use cases is in option pricing, where a binary-like decision tree is used to predict the price of an option in either a bull or bear market. Working with Decision Trees 82 Chapter 5 ■ Working with Decision Trees Marketers use Decision Trees to establish customers by type and predict whether a customer will buy a specific type of product. In the medical field, decision tree models have been designed to diagnose blood infections or even predict heart attack outcomes in chest pain patients. Variables in the decision tree include diagnosis, treatment, and patient data. The gaming industry now uses multiple Decision Trees in movement recog-nition and facial recognition. The Microsoft Kinect platform uses this method to track body movement. The Kinect team used one million images and trained three trees. Within one day and using a 1,000-core cluster, the Decision Trees were classifying specific body parts across the screen.

  Sign up to read

  Learn more about book
• 

  No longer available

  Artificial Intelligence and Machine Learning Fundamentals

  Develop real-world applications powered by the latest AI advances

  • Zsolt Nagy(Author)
  • 2018(Publication Date)
  • Packt Publishing

    (Publisher)

  5

  Using Trees for Predictive Analysis

  Learning Objectives

  By the end of this chapter, you will be able to:

  • Understand the metrics used for evaluating the utility of a data model
  • Classify datapoints based on Decision Trees
  • Classify datapoints based on the random forest algorithm

  In this chapter, we will learn about two types of supervised learning algorithm in detail. The first algorithm will help us to classify data points using Decision Trees, while the other algorithm will help us classify using random forests.

  Introduction to Decision Trees

  In Decision Trees, we have input and corresponding output in the training data. A decision tree, like any tree, has leaves, branches, and nodes. Leaves are the end nodes like a yes or no. Nodes are where a decision is taken. A decision tree consists of rules that we use to formulate a decision on the prediction of a data point.

  Every node of the decision tree represents a feature and every edge coming out of an internal node represents a possible value or a possible interval of values of the tree. Each leaf of the tree represents a label value of the tree.

  As we learned in the previous chapters, data points have features and labels. A task of a decision tree is to predict the label value based on fixed rules. The rules come from observing patterns on the training data.

  Let's consider an example of determining the label values Suppose the following training dataset is given. Formulate rules that help you determine the label value:

  Figure 5.1: Dataset to formulate the rules

  In this example, we predict the label value based on four features. To set up a decision tree, we have to make observations on the available data. Based on the data that's available to us, we can conclude the following:

  • All house loans are determined as credit-worthy.
  • Studies loans are credit-worthy as long as the debtor is employed. If the debtor is not employed, he/she is not creditworthy.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Decomposition Methodology for Knowledge Discovery and Data Mining

  Theory and Applications

  • Oded Maimon, Lior Rokach;;;(Authors)
  • 2005(Publication Date)
  • WSPC

    (Publisher)

  Chapter 2 Decision Trees 2.1 Decision Trees A Decision tree is a classifier expressed as a recursive partition of the instance space. The decision tree consists of nodes that form a Rooted Tree, meaning it is a Directed Tree with a node called root that has no incoming edges. All other nodes have exactly one incoming edge. A node with out-going edges is called internal node or test nodes. All other nodes are called leaves (also known as terminal nodes or decision nodes). In the decision tree each internal node splits the instance space into two or more subspaces according to a certain discrete function of the input attributes values. In the simplest and most frequent case each test considers a single attribute, such that the instance space is partitioned according to the attribute's value. In the case of numeric attributes the condition refers to a range. Each leaf is assigned to one class representing the most appropriate tar-get value. Alternatively the leaf may hold a probability vector indicating the probability of the target attribute having a certain value. Instances are classified by navigating them from the root of the tree down to a leaf, according to the outcome of the tests along the path. Figure 2.1 desc-ribes a decision tree that reasons whether or not a potential customer will respond to a direct mailing. Internal nodes are represented as circles, whereas leaves are denoted as triangles. Note that this decision tree incorporates both nominal and numeric attributes. Given this classifier, the analyst can predict the response of a potential customer (by sorting it down the tree), and understand the behavioral characteristics of the entire potential customers population regarding direct mailing. Each node is labe-led with the attribute it tests, and its branches are labeled with its corre-sponding values. 35 36 Decomposition Methodology for Knowledge Discovery and Data Mining Fig. 2.1 Decision Tree Presenting Response to Direct Mailing.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Discovering Knowledge in Data

  An Introduction to Data Mining

  • Daniel T. Larose, Chantal D. Larose(Authors)
  • 2014(Publication Date)
  • Wiley

    (Publisher)

  Chapter 8 Decision Trees

  1. 8.1 What is a Decision Tree?
  2. 8.2 Requirements for Using Decision Trees
  3. 8.3 Classification and Regression Trees
  4. 8.4 C4.5 Algorithm
  5. 8.5 Decision Rules
  6. 8.6 Comparison of the C5.0 and Cart Algorithms Applied to Real Data
     1. The R Zone
     2. References
     3. EXERCISES
     4. Hands-On Analysis

  8.1 What is a Decision Tree?

  In this chapter we continue our examination of classification methods for data mining. One attractive classification method involves the construction of a decision tree, a collection of decision nodes, connected by branches, extending downward from the root node until terminating in leaf nodes. Beginning at the root node, which by convention is placed at the top of the decision tree diagram, attributes are tested at the decision nodes, with each possible outcome resulting in a branch. Each branch then leads either to another decision node or to a terminating leaf node. Figure 8.1 provides an example of a simple decision tree.

  Figure 8.1

  Simple decision tree.

  The target variable for the decision tree in Figure 8.1 is credit risk, with potential customers being classified as either good or bad credit risks. The predictor variables are savings (low, medium, and high), assets (low or not low), and income (≤$50,000 or >$50,000). Here, the root node represents a decision node, testing whether each record has a low, medium, or high savings level (as defined by the analyst or domain expert). The data set is partitioned, or split, according to the values of this attribute. Those records with low savings are sent via the leftmost branch (savings = low) to another decision node. The records with high savings are sent via the rightmost branch to a different decision node.

  The records with medium savings are sent via the middle branch directly to a leaf node, indicating the termination of this branch. Why a leaf node and not another decision node? Because, in the data set (not shown), all of the records with medium savings levels have been classified as good credit risks. There is no need for another decision node, because our knowledge that the customer has medium savings predicts good credit with 100% accuracy in the data set.

  Sign up to read

  Learn more about book
• 

  Available until 25 Jan |Learn more

  Classification Methods for Remotely Sensed Data

  • Paul Mather, Brandt Tso(Authors)
  • 2016(Publication Date)
  • CRC Press

    (Publisher)

  183 6 Decision Trees Categorization of data using a hierarchical splitting (or top–down) mechanism has been widely used in the environmental and life sciences. The purpose of using a hier-archical structure for labeling objects is to gain a more comprehensive understand-ing of relationships between objects at different scales of observation or at different levels of detail. Its simplest representation takes the form of an inverted tree in which the different levels of classification are represented by the separate levels of a hierar-chy. When applied to multispectral image data, the design of a decision tree is based on knowledge of the spectral properties of each class and of the relationships among the classes. The main benefit of using a hierarchical tree structure to perform clas-sification decisions is that the tree structure can be viewed as a white box , which in comparison with artificial neural networks (ANN) (Chapter 3), is easier to interpret and understand the relation between the inputs and outputs. A hierarchical decision tree classifier is an algorithm for the labeling of an unknown pattern using a sequence of decisions. A decision tree is composed of a root node, a set of interior nodes, and terminal nodes called leaf nodes . The root node and interior nodes, referred to collectively as nonterminal nodes , are linked into decision stages. The terminal nodes represent the final classification. The clas-sification process is implemented by a set of rules that determine the path to be fol-lowed, starting from the root node and ending at one terminal node, which represents the label for the object being classified. At each nonterminal node, a decision has to be made about the path to the next node. Figure 6.1 illustrates a simple hierarchical decision tree classifier using a pixel’s reflectance value as input to identify that pixel as either trees, shrubs, swamp, or water.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Discovering Knowledge in Data

  An Introduction to Data Mining

  • Daniel T. Larose, Chantal D. Larose(Authors)
  • 2014(Publication Date)
  • Wiley

    (Publisher)

  CHAPTER 8 Decision Trees 8.1 WHAT IS A DECISION TREE? 165 8.2 REQUIREMENTS FOR USING Decision Trees 167 8.3 CLASSIFICATION AND REGRESSION TREES 168 8.4 C4.5 ALGORITHM 174 8.5 DECISION RULES 179 8.6 COMPARISON OF THE C5.0 AND CART ALGORITHMS APPLIED TO REAL DATA 180 THE R ZONE 183 REFERENCES 184 EXERCISES 185 HANDS-ON ANALYSIS 185 8.1 WHAT IS A DECISION TREE? In this chapter we continue our examination of classification methods for data mining. One attractive classification method involves the construction of a decision tree, a collection of decision nodes, connected by branches, extending downward from the root node until terminating in leaf nodes. Beginning at the root node, which by convention is placed at the top of the decision tree diagram, attributes are tested at the decision nodes, with each possible outcome resulting in a branch. Each branch then leads either to another decision node or to a terminating leaf node. Figure 8.1 provides an example of a simple decision tree. The target variable for the decision tree in Figure 8.1 is credit risk, with potential customers being classified as either good or bad credit risks. The predictor variables are savings (low, medium, and high), assets (low or not low), and income (≤$50,000 or >$50,000). Here, the root node represents a decision node, testing whether each record has a low, medium, or high savings level (as defined by the analyst or domain expert). The data set is partitioned, or split, according to the values of this attribute. Discovering Knowledge in Data: An Introduction to Data Mining, Second Edition. By Daniel T. Larose and Chantal D. Larose. © 2014 John Wiley & Sons, Inc. Published 2014 by John Wiley & Sons, Inc. 165 166 CHAPTER 8 Decision Trees Figure 8.1 Simple decision tree. Those records with low savings are sent via the leftmost branch (savings = low) to another decision node. The records with high savings are sent via the rightmost branch to a different decision node.

  Sign up to read

  Learn more about book