Decision-Analytic Intelligent Systems

Automated Explanation and Knowledge Acquisition

David A. Klein

• Publisher: Routledge
• Year: 2013
• Print ISBN:
  9780805811056
• eBook ISBN:
  9781134768530
• Topic: Psychology
• Subtopic: Cognitive Psychology & Cognition
• Index: Psychology

This book presents a framework for building intelligent systems based on the mathematical decision models of Decision Analysis. The author provides new techniques for automated explanation and knowledge acquisition in formally sound systems that reason about complex tradeoffs in decisions. Also included are specifications for implementing these techniques in computer programs, along with demonstration applications in marketing, process control, and medicine.

Readers with an interest in artificial intelligence will gain a foundation for building formally justifiable, intelligible, modifiable systems for computing decisions involving multiple considerations, with applications across a variety of domains. Beyond decision models, the methodology of the work reported suggests a more general approach to employing formal mathematical models in transparent intelligent systems.

Decision-analysis experts will find a collection of methods for explaining decision-analytic advice to clients in intuitive terms, for simplifying parameter assessment, and for managing changing preferences over time. The book provides sufficient background material to promote understanding by readers who may be unfamiliar with artificial intelligence, with decision analysis, or with both fields, and such material is labeled to increase the well-versed reader's efficiency in skipping particular sections.

Chapter 1

Introduction

1.1Context

Judgment-intensive choices arise in diverse settings, from the commonplace (e.g., choosing an activity on a Saturday evening) to the technical (e.g., choosing an action to avoid a crisis in a computer complex). Significant research has been devoted to modeling such choices.

Models of choice generally accept as input a set of objective data and subjective judgments that characterize a choice among competing alternatives, and produce as output a recommended alternative. As in other modeling endeavors, computer systems are playing an increasingly important role in building, executing, and interpreting models of choice. Such systems generally vary with the formality of their specification, and with the transparency of their operation.¹ A formal system reflects a specific theory of operation that addresses a well-bounded class of applications; a transparent system provides an intuitive framework for interpreting its results and for systematically modifying its parameters.

Both formality and transparency contribute to the success of practical systems: Formality guarantees meaningful outputs for a clearly bounded class of inputs, and transparency lends credibility to the system’s results in the eyes of users while providing for the system’s adaptation to particular problems and user populations. Yet, previous approaches to modeling choices in a systems context generally have emphasized either one property or the other. Computer-based tools for decision analysis (DA), for example, produce meaningful results for any set of inputs that satisfies a well-defined set of constraints, but they usually lack intuitive facilities for justifying choices and for modifying choice parameters; these limitations render it relatively expensive to construct DA models, and to interpret the results of such models. Heuristic approaches to modeling choices in artificial intelligence (AI) systems, on the other hand, strive for intuitive appeal, but these approaches typically reflect the requirements of only particular domains, and they lack a foundation that permits prediction of their behavior outside the context of particular examples in those domains.

This book is concerned with providing computer-based models of choice that are at once formal and transparent.

1.2Approach and Contributions

We take the approach of providing a formal and transparent model by embellishing the transparency of a formal model from DA. The result, Interpretive Value Analysis (IVA), is the subject of this book. IVA is a framework for explaining and refining choices in the context of intelligent systems. More specifically, IVA lays the foundation for an array of potential systems that model multiattribute choices under certainty, ² in which multiple competing factors underlie choices (e.g., enjoyment vs. cost of an elegant dinner), and the outcomes of choices are assumed to occur with certainty (e.g., the dinner definitely will be expensive).³ IVA increases the transparency of multiattribute value theory, a formal model of value, by reformulating the theory and embedding it in a framework for explaining and iteratively refining value-based choices. IVA comprises the following components:

• An interpretation that provides an intuitive yet formally sound vocabulary of more than 100 terms for talking about value-based choices. The vocabulary is in part based on analyses of conversations about choices that were collected from both decision analysts and nonanalysts.
• A set of strategies for explaining value-based choices that organizes elements of the interpretation to provide insight into how choices are computed.
• A set of strategies for refining value-based choices that organizes model parameters for modification. These strategies build on elements of the ex planation strategies and of the interpretation.

IVA at once addresses open problems in AI and in DA. From an AI perspective, IVA provides a general foundation for building value-based systems that are formally justifiable, intelligible, and modifiable. Beyond decision models, the methodology underlying IVA’s development suggests a more general approach to employing formal mathematical models in transparent intelligent systems. From a DA perspective, IVA addresses problems of transparency. First, IVA can potentially increase the acceptance of decision-theoretic advice by providing methods for justifying that advice in intuitive terms. Second, IVA provides an approach to managing bias in parameter assessment; the framework provides users with an opportunity to observe the step-by-step effect of a parameter value on the final result, so that users’ responses may potentially be influenced less by the fashion in which parameter-assessment questions are posed. Third, IVA can potentially reduce the demands on parameter-assessment methods by providing for the incremental repair of model parameters. Finally, the framework provides an approach to the problem of managing changing preferences over time.

Because the book lies at the intersection of theory and practice, it necessarily reports on a number of demonstration systems. Many of the elements of IVA are implemented in VIRTUS,⁴ a shell for building value-based systems. VIRTUS implements a view of model construction as an iterative argument with a machine. Aided by a knowledge engineer,⁵ the user begins by supplying VIRTUS with an initial model of his preferences. VIRTUS then computes a choice, and justifies that choice with an explanation. If the user finds the explanation convincing, then model construction is complete; if not, the user can request additional explanations, or can initiate the process of refinement to correct a suspicious component of the explanation (i.e., component of the underlying model), and to generate a new explanation that reflects his modification. This process is repeated until convergence is achieved.

VIRTUS is a domain-independent, architecture-independent module that can be used in isolation or in concert with other representations. Three practical value-based systems have been constructed using VIRTUS: JESQ-II (Klein & Short-liffe, 1990b) is a system that chooses among competing alternative actions in managing a large computer complex; RCTE (Klein, Lehmann, & Shortliffe, 1990) is a program for evaluating clinical research in medicine; and ES-SHELL is an application for choosing among competing expert-system shells. These applications demonstrate the capabilities of IVA-based systems, and underscore the domain-independence of our approach.⁶

To frame the work more clearly, we mention explicitly objectives that we do not attempt to satisfy in the book. First, although IVA is based in part on observations of human discourse, the book makes no contribution to discourse analysis or to natural language processing (in the usual sense of these terms). For example, VIRTUS is not designed to replicate human discourse and employs simple methods for parsing input and for generating text. Second, VIRTUS is not intended as an ideal for any particular system; rather, VIRTUS is a demonstration vehicle for IVA, which represents a general specification for a broad range of potential value-based systems. For example, VIRTUS implements redundant techniques for research-level tasks, and omits the implementation of functions that would be desirable in a production setting, but that are not expected to yield scientific or engineering insights.

1.3VIRTUS Example

VIRTUS can be used in an interactive mode and to generate reports.⁷ The interactive mode includes facilities for generating intuitive justifications for choices and for assisting users in modifying model parameters. The reporting mode draws on a number of IVA’s explanation strategies to produce intuitive reports about decisions.

The following is an interactive dialog with VIRTUS that demonstrates a subset of the implemented capabilities.⁸ In this example, the user is a computer-operations manager who is verifying and correcting an existing model of value for automatically managing a large computer complex. More specifically, there is a large dataset waiting for a user-requested printer that currently is disabled, and the manager employs VIRTUS to explain and refine its choice among the following actions for removing this dataset from the overloading print queue:

1. COPY: Copy the dataset to tape, and print it later when the requested printer has been repaired
2. EXPENSIVE-PRINTING: Print the dataset on a faster printer that uses forms more expensive than those requested by the user
3. CHEAP-PRINTING: Print the dataset on a slower printer that uses forms of lower quality than those requested by the user
4. DELETE: Delete the dataset from the queue
5. INSTALL: Connect a duplicate of the requested printer for use until the requested printer has been repaired
6. FICHE: Deliver the dataset to the user on microfiche
7. CARDS: Deliver the dataset to the user on punched cards
8. DASD: Transfer the dataset to the user’s private disk storage, so that the user can transfer it back to the queue for printing after the printer has been repaired

Which alternative is best?

DASD is the best with regard to overall queue space management effectiveness.

What does DASD involve?

DASD involves transferring the user’s dataset to his private disk storage for later printing. This encompasses:

• .1 minutes of additional operator time
• .5 dollars of additional cost
• 1 minute of problem resolution time
• 32.1 minutes of additional turnaround time
• 1 subjective unit of similarity to the user’s requested form

Which alternatives are very similar to DASD with regard to overall queue space management effectiveness?

Close ...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright
• Dedication
• Contents
• Preface
• Chapter 1. Introduction
• Chapter 2. Background, Motivation, and Problem Statement
• Chapter 3. IVA and VIRTUS: Overview and Applications
• Chapter 4. Interpretation
• Chapter 5. Explanation
• Chapter 6. Refinement
• Chapter 7. Implementation
• Chapter 8. Evaluation, Experience, and Observations
• Chapter 9. Summary, Contributions, and Future Work
• Appendix: VIRTUS Report
• References
• Author Index
• Subject Index