Decision Tree Method

10 Key excerpts on "Decision Tree Method"

• 

  eBook - ePub

  Fundamentals of Predictive Analytics with JMP, Third Edition

  • Ron Klimberg(Author)
  • 2023(Publication Date)
  • SAS Institute

    (Publisher)

  Chapter 10: Decision Trees Introduction

  The decision tree is one of the most widely used techniques for describing and organizing multivariate data. As shown in Figure 10.1 , a decision tree is one of the dependence techniques in which the dependent variable can be either discrete (the usual case) or continuous.

  Benefits and Drawbacks

  A decision tree is usually considered to be a data mining technique as well as a dependence technique. One of its strengths is its ability to categorize data in ways that other methods cannot. For example, it can uncover nonlinear relationships that might be missed by techniques such as linear regression.

  A decision tree is easy to understand and easy to explain, which is always important when an analyst has to communicate results to a nontechnical audience. Decision trees do not always produce the best results, but they offer a reasonable compromise between models that perform well and models that can be simply explained. Decision trees are useful not only for modeling, but also for exploring a data set, especially when you have little idea of how to model the data.

  Figure 10.1: A Framework for Multivariate Analysis

  The primary drawback of trees is that they are a high-variance procedure: growing trees on two similar data sets probably will not produce two similar trees. As an example of a low-variance procedure, consider that if you run the same regression model on two different (but similar) samples, you will likely get similar results: both regressions will have about the same coefficients. By contrast, if you run the same tree on two different (but similar) samples, you will likely get quite different trees.

  The reason for the variance is that, in a tree, an error in any one node does not stay in that node. Instead, the error is propagated down the tree. Specifically, if two variables (for example, Variable A and Variable B) are close contenders for the first split in a decision tree, a small change in the data might affect which of those variables is chosen for the top split. Splitting on Variable A might well produce a markedly different tree than splitting on Variable B. There are methods such as boosting and bagging to combat this issue by growing multiple trees on the same data set and averaging them. But these methods are beyond the scope of this text. The interested reader should consult the text by Berk (2008

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Value-Added Decision Making for Managers

  • Kenneth Chelst, Yavuz Burak Canbolat(Authors)
  • 2011(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)

  10.1 Goal and Overview The goal of this chapter is to present decision trees as an analytic approach to making decisions involving uncertainty. It is a simpler, more transparent modeling tool than stochastic simulation discussed in the previous chapter. It is a normative decision-making tool that identifies the optimal decision based on expected value or expected utility. Stochastic simulation, in contrast, is primarily descriptive. Like simulation, the decision tree output includes a risk profile of all the alternatives. Decisions are first compared through expected value analysis; the optimal decision is the one with the best expected value. Further analysis enables the decision maker to understand and inter-pret the related strengths and weaknesses of the alternatives with the goal of developing an even better alternative that mitigates some of the risks. The tool is used in situations where the decision maker is faced with a discrete set of limited alternatives, and uncertainty plays a major role in the future outcome of these decisions. 278 Value Added Decision Making for Managers The first task in developing a decision tree is to identify key quantifiable uncertainties that directly impact the outcome of the decision. As a starting point, Table 10.1 presents some ran-dom variables and outcomes that arise in a variety of decision contexts. The key challenge is to explicitly account for the uncertainty upfront, not to drive the uncertainty out of the decision process. 10.2 Early Users of Decision Trees The oil industry was one of the first to use decision trees, incorporating available information to help determine the risks involved in various decisions. Significant uncertainty is commonplace in the industry, involving billions of dollars in costs and revenues. Uncertain variables include the potential yield of oilfields, the cost of extraction, political instability in various regions, and the fluctuating price of oil.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Computational Intelligence in Software Modeling

  • Vishal Jain, Jyotir Moy Chatterjee, Ankita Bansal, Utku Kose, Abha Jain, Vishal Jain, Jyotir Moy Chatterjee, Ankita Bansal, Utku Kose, Abha Jain(Authors)
  • 2022(Publication Date)
  • De Gruyter

    (Publisher)

  Two CI approaches that have always piqued the interest of academics from many areas are neural networks and fuzzy systems. A comparison is made between these soft computing-based failure prediction approaches in order to determine their utility [12]. The next sub-section presents the overview of decision trees for better under- standing of the concept. 1.2 Decision tree A decision tree (DT) is a flow-chart like structure in which the internal nodes repre- sent dataset attributes/features, branches of the tree represent decision rules (usu- ally IF-THEN-ELSE Rules), and each leaf node provides the conclusion/result/ predictions. They are based on supervised learning technique that are employed to solve both regression and classification problems. It comes under non-parametric learning techniques. Because the building of a decision tree classifier does not necessi- tate domain expertise or parameter selection, it is well suited to exploratory knowledge discovery. A decision tree is a graphic representation of decision-making processes. We get a rule for each path from the decision node to the leaf node. The tree’s height should be kept low in order to achieve excellent accuracy. Some implications related to the use of decision trees include: a slight change in the data can result in a substan- tial change in the decision tree’s structure, resulting in instability. The training period for a decision tree is typically longer. At the same time, decision tree has proven the Decision tree–based improved software fault prediction 167 most powerful and popular tool for classification and prediction [13]. The following are some of the benefits of using a decision tree: – DTs are simple to use, interpret, and understand. – DT has the ability to work with both categorical and numerical data. – DTs are outlier-resistant, requiring minimum data preparation.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Pharmacoeconomics

  From Theory to Practice

  • Renee J. G. Arnold, Renee J. G. Arnold, Renee J. G. Arnold, Renee J. G. Arnold(Authors)
  • 2016(Publication Date)
  • CRC Press

    (Publisher)

  It is important to remember that the time frame does not include only those events directly related to the various strategies, but all of the future events implied by choosing each strategy. If a particular intervention increases the risk of a life-changing complication (stroke, heart attack, pulmonary embolism), the long-term effects of the complications need to be taken into account as well. 2.2.1 T YPES OF D ECISION M ODELING T ECHNIQUES Many methodologies and modeling types can be used to create and evaluate decision models, and the modeler should use the method most appropriate to the particu-lar problem being addressed. The choice is dependent upon the complexity of the problem, the need to model outcomes over extended periods of time, and whether resource constraints and interactions of various elements in the model are required. We will describe in detail the development of simple branch and node decision trees, which set the context for many of the other techniques. A brief review of several methodologies is then provided; more detailed descriptions of many of these tech-niques can be found in other chapters in this book. 2.2.2 D ECISION T REES The classic decision analysis structure is the branch and node decision tree, which is illustrated in Figure 2.1. The decision tree has several components that are always present and need to be carefully developed. A decision model comprises the model-ing structure itself (the decision tree), which represents the decision that is being made and the outcomes that can occur as the result of each decision, the probabilities that the various outcomes will occur, and the values of the outcomes if they do occur. Similar to any other research problem, the decision tree should start with a specific problem formulation, which in the figure is a choice between therapy A and therapy B in a particular condition.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Decision Analysis for Management Judgment

  • Paul Goodwin, George Wright(Authors)
  • 2014(Publication Date)
  • Wiley

    (Publisher)

  They can help a decision maker to develop a clear view of the structure of a problem and make it easier to determine the possible scenar- ios which can result if a particular course of action is chosen. This can lead to creative thinking and the generation of options which were not previously being considered. Decision trees can also help a decision maker to judge the nature of the information which needs to be gathered in order to tackle a problem and, because they are 162 DECISION TREES AND INFLUENCE DIAGRAMS generally easy to understand, they can be an excellent medium for communicating one person’s perception of a problem to other individuals. The process of constructing a decision tree is usually iterative, with many changes being made to the original structure as the decision maker’s understanding of the problem develops. Because the intention is to help the decision maker to think about the problem, very large and complex trees, which are designed to represent every pos- sible scenario that can occur, may be counterproductive in many circumstances. Deci- sion trees are models, and as such are simplifications of the real problem. The simplifi- cation is the very strength of the modeling process because it fosters the understanding and insight which would be obscured by detail and complexity. Nevertheless, in rare circumstances highly complex trees may be appropriate and software developments mean that their structuring and analysis can now be facilitated with relative ease. For example, Dunning et al. 1 used software to apply a decision tree with over 200 mil- lion paths to a 10-year scheduling problem faced by the New York Power Authority. Similarly, Beccue 2 used a tree with around half-a-million scenarios to help a pharma- ceutical company make decisions relating to the development and marketing of a new drug.

  Sign up to read

  Learn more about book
• 

  No longer available

  Artificial Intelligence and Machine Learning Fundamentals

  Develop real-world applications powered by the latest AI advances

  • Zsolt Nagy(Author)
  • 2018(Publication Date)
  • Packt Publishing

    (Publisher)

  5

  Using Trees for Predictive Analysis

  Learning Objectives

  By the end of this chapter, you will be able to:

  • Understand the metrics used for evaluating the utility of a data model
  • Classify datapoints based on decision trees
  • Classify datapoints based on the random forest algorithm

  In this chapter, we will learn about two types of supervised learning algorithm in detail. The first algorithm will help us to classify data points using decision trees, while the other algorithm will help us classify using random forests.

  Introduction to Decision Trees

  In decision trees, we have input and corresponding output in the training data. A decision tree, like any tree, has leaves, branches, and nodes. Leaves are the end nodes like a yes or no. Nodes are where a decision is taken. A decision tree consists of rules that we use to formulate a decision on the prediction of a data point.

  Every node of the decision tree represents a feature and every edge coming out of an internal node represents a possible value or a possible interval of values of the tree. Each leaf of the tree represents a label value of the tree.

  As we learned in the previous chapters, data points have features and labels. A task of a decision tree is to predict the label value based on fixed rules. The rules come from observing patterns on the training data.

  Let's consider an example of determining the label values Suppose the following training dataset is given. Formulate rules that help you determine the label value:

  Figure 5.1: Dataset to formulate the rules

  In this example, we predict the label value based on four features. To set up a decision tree, we have to make observations on the available data. Based on the data that's available to us, we can conclude the following:

  • All house loans are determined as credit-worthy.
  • Studies loans are credit-worthy as long as the debtor is employed. If the debtor is not employed, he/she is not creditworthy.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  A User's Guide to Business Analytics

  • Ayanendranath Basu, Srabashi Basu(Authors)
  • 2016(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)

  Based on data already in the database, and based on the clinical symptoms and ultimate outcome (death or survival) of the patients, a new patient may be classified into high-risk and low-risk categories. Decision trees have very high applicability in industrial recommender systems. A recommender system recommends items or products to prospective buyers based on their purchase pattern. In many such cases, especially if the store does not have any information on the customer other than their previous purchases, or in the case of online stores, prospective buy- ers’ choices and click-stream responses, regression or any other model building procedures will not be applicable. Decision trees are applicable in marketing to identify homogeneous groups for target marketing. Decision trees segment the predictor space into several similar regions and use the mean of the con- tinuous response or mode of categorical response in each separate region for prediction. 277 278 A User’s Guide to Business Analytics 10.1 Algorithm for Tree-Based Methods Classification and regression tree (CART) is a recursive binary splitting al- gorithm introduced by Breiman et al. (1984). Even though there have been many improvements on this method, the core algorithm is still identical to that of CART. Consider the set of all predictors, continuous and categorical, together. CART does not make any differentiation between categorical and continuous predictors. CART is based on nodes and split. The full sample is known as the root node. At each instance of split, a variable and its level is selected, so that purity at each child node is the highest possible at that level. The basic idea of tree growing is to choose a split among all possible splits at each node so that the resultant child nodes are the purest. At each split the sample space is partitioned according to one predictor. Only univariate splits are considered, so that the partitions are parallel to the axes.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Decomposition Methodology for Knowledge Discovery and Data Mining

  Theory and Applications

  • Oded Maimon, Lior Rokach;;;(Authors)
  • 2005(Publication Date)
  • WSPC

    (Publisher)

  Chapter 2 Decision Trees 2.1 Decision Trees A Decision tree is a classifier expressed as a recursive partition of the instance space. The decision tree consists of nodes that form a Rooted Tree, meaning it is a Directed Tree with a node called root that has no incoming edges. All other nodes have exactly one incoming edge. A node with out-going edges is called internal node or test nodes. All other nodes are called leaves (also known as terminal nodes or decision nodes). In the decision tree each internal node splits the instance space into two or more subspaces according to a certain discrete function of the input attributes values. In the simplest and most frequent case each test considers a single attribute, such that the instance space is partitioned according to the attribute's value. In the case of numeric attributes the condition refers to a range. Each leaf is assigned to one class representing the most appropriate tar-get value. Alternatively the leaf may hold a probability vector indicating the probability of the target attribute having a certain value. Instances are classified by navigating them from the root of the tree down to a leaf, according to the outcome of the tests along the path. Figure 2.1 desc-ribes a decision tree that reasons whether or not a potential customer will respond to a direct mailing. Internal nodes are represented as circles, whereas leaves are denoted as triangles. Note that this decision tree incorporates both nominal and numeric attributes. Given this classifier, the analyst can predict the response of a potential customer (by sorting it down the tree), and understand the behavioral characteristics of the entire potential customers population regarding direct mailing. Each node is labe-led with the attribute it tests, and its branches are labeled with its corre-sponding values. 35 36 Decomposition Methodology for Knowledge Discovery and Data Mining Fig. 2.1 Decision Tree Presenting Response to Direct Mailing.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Active Learning for Recommender Systems

  • (Author)
  • 2014(Publication Date)
  • Cuvillier Verlag

    (Publisher)

  (Nicholas & McCallum, 2001) replaces D test with D pool to estimate how candidate examples would reduce the test error. 7.3 Decision Tree or Decision Process? The way that decision trees are used in (Golbandi et al. , 2011) poses some ambiguities. In this section, we would like to clarify this ambiguity. In machine learning, decision trees are predictive models in which leaves represent the values of target variable and branches represent conjunctions of input features that lead to those values. There are two types of decision trees: classification trees and regression trees. Classification trees are used when the the target variable is a nominal variable and in the case of a numerical target variable, regression trees are used. Coming back to the new user problem in recommender systems, we can look at this problem from two different perspectives. In the first view, we define this problem as a decision-making problem. Namely, there is a user and we want to decide which item must be selected to ask its rating from that user. So, decisions are items. To learn the correct decisions, we use training data and measure the outcome of each decision and finally pick the best item. Obviously, this is not a decision tree that is used in machine learning because there is no target variable in the leaf nodes. In fact, it is a decision process that is used in data analysis to visually and explicitly represent decisions. In data mining, a decision tree describes data but not decisions and the resulting decision tree can be an input for decision making. Now let’s look at the new user problem from a different perspective. There is a new user who has not given any ratings. Therefore, recommender systems are not able to predict ratings for that new user because they are based on collaborative filtering and in collaborative filtering we need to compare the ratings of users to find similar users and then take the average over similar users.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Operations Research

  A Practical Introduction

  • Michael Carter, Camille C. Price, Ghaith Rabadi(Authors)
  • 2018(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)

  art of decision analysis is deciding which of these variables are likely to change the optimal strategy. For example, in a production problem, the likelihood of a union strike would have a significant impact on expected profit. However, it may have no impact on the best decision selection if the strike reduces all outcomes proportionally.

  An excellent example of the art of using decision trees is presented by Byrnes (1973). He describes an actual case study of a decision by a major soap manufacturer in England of whether to market a new deodorant bath soap in the 1960s, at a time when many companies were experimenting with the idea. The case is interesting because decision trees were used as a vehicle for understanding the problem. Although the final tree was used to predict expected profit, there was a sequence of decision trees that reflected the changes in the attitudes of management as they learned more about the decision at each stage. The case study describes each step, and, in particular, the mistakes and guesses that actually took place along the way.

  9.4      Utility Theory

  In Section 9.3 , we made the assumption that people will choose the alternative that exhibits the highest expected value. Such people will be called EMVers for their use of expected monetary value. If a particular decision is to be repeated many times, then the EMV approach is perfectly sound. In the long run, the actual profit will be very close to the EMV sum of the individual decisions. Unfortunately, most practical decision processes apply to a single decision-making event.

  For this reason, the vast majority of decision-makers do not rely solely on EMV, and will also make a subjective evaluation of the amount of risk involved in a decision. They will attempt to incorporate their attitude toward risk in a trade-off against the potential benefits of taking a chance.

  Sign up to read

  Learn more about book