Evaluating Explanations

A Content Theory

David B. Leake

• Publisher: Psychology Press
• Year: 2014
• Print ISBN: 9781138969162
• eBook ISBN: 9781317782445
• Topic: Psychology
• Subtopic: Cognitive Psychology & Cognition
• Index: Psychology

Psychology and philosophy have long studied the nature and role of explanation. More recently, artificial intelligence research has developed promising theories of how explanation facilitates learning and generalization. By using explanations to guide learning, explanation-based methods allow reliable learning of new concepts in complex situations, often from observing a single example.

The author of this volume, however, argues that explanation-based learning research has neglected key issues in explanation construction and evaluation. By examining the issues in the context of a story understanding system that explains novel events in news stories, the author shows that the standard assumptions do not apply to complex real-world domains. An alternative theory is presented, one that demonstrates that context -- involving both explainer beliefs and goals -- is crucial in deciding an explanation's goodness and that a theory of the possible contexts can be used to determine which explanations are appropriate. This important view is demonstrated with examples of the performance of ACCEPTER, a computer system for story understanding, anomaly detection, and explanation evaluation.

1 Explanation and Understanding

Two basic problems face understanders in real-world domains: recognizing what needs to be explained, and deciding whether a given explanation is sufficient. We discuss these problems and sketch the major points of the theory that this book develops to address them.

1.1 Fashioning Beliefs

When people are given new information, they seldom accept it blindly. If a politician pledges to lower taxes, or a car advertisement trumpets the high quality of a brand known for problems, people are skeptical. One possible explanation for their skepticism is that they compare new information to standard patterns and question information that conflicts. By questioning anomalous information, they can avoid mistakenly abandoning correct conclusions. Knowledge refined over long experience should not be discarded because of a freak incident: Seeing a magician levitate his assistant makes people wonder how they were fooled, rather than making them believe that the magician’s supernatural power overcomes the law of gravity.

However, always clinging to old beliefs can be as dangerous as too easily abandoning them, because conflicts between old beliefs and new information can signal flaws in the old beliefs that need to be repaired. For example, suppose that someone with an unreliable car decides to replace it. In order to decide which brands to consider, he might try to judge the likelihood of various brands having problems. One heuristic people use for deciding likelihoods is the availability of instances with a given property: If most of the cases that can be recalled have the property, they assume that the property usually holds [Tversky & Kahneman, 1982]. If the car buyer uses this heuristic, his bad experience might make him generalize that the manufacturer of his car has bad quality control. Later, he might receive information contradicting that belief: A newspaper story could state that the manufacturer’s quality control is much better than that of the competition. If he simply clung to his first generalization he would continue to avoid his original brand despite the report—and possibly buy a car that was worse.

In everyday understanding, it is impossible to avoid imperfect beliefs. Exigencies of the current situation often force conclusions to be drawn from partial or uncertain information, and even correct conclusions may be invalidated by changes in the world. Consequently, a system must be able to notice and repair those flaws in its knowledge that affect its performance.¹ In an understander, such flaws are revealed by failures to understand: by the anomalies that arise when new information conflicts with the understander’s prior beliefs and expectations.

By noticing and explaining anomalies; people repair flawed knowledge and determine how to respond [Schank, 1986]. By noticing and explaining the discrepancy between his prior generalization and the newspaper story, the car buyer determines whether to revise his level of confidence in the newspaper or in his prior generalization, making his world model more accurate; he also determines which car to buy.

Likewise, Artificial Intelligence systems dealing with complex and dynamic domains must have the capacity to notice anomalous situations and react appropriately. This requires both recognizing when anomalies occur and finding a good explanation. These are difficult tasks in real-world domains, in which available information and processing resources may be limited, and where the ramifications of new information depend strongly on context—both the situation in the world, and the relationship of that situation to the system’s current goals.

This book investigates when to trigger explanation, how to guide the search for explanations, and how to decide the adequacy of candidate explanations. This chapter surveys some of the territory the book will cover, beginning by discussing major issues in maintaining accurate beliefs in real-world domains. It then presents the highlights of a theory addressing those issues—a theory of how to detect potential belief problems, how to direct explanation towards resolving them, and how to assure that the resultant explanations give the information the system needs.

1.1.1 Belief in Real-World Domains

In real-world domains, understanding systems need ways to maintain reasonable views of the world, despite partial and unreliable information. Much research in story understanding has focused on how to generate a reasonable interpretation of events from partial information, but an equally important companion problem has received little attention: retracting beliefs that turn out to be incorrect [Charniak, 1978; Granger, 1980; O’Rorke, 1983]. The capability to recognize and learn from faulty beliefs is vital to the success of any real-world understanding system. No matter how good its strategies for interpreting events may be, it cannot be completely immune to being led astray by the following problems:

• Lack of applicable knowledge: No real-world understander can have a library of rules describing all features of the world that will ever be relevant; sometimes it will encounter situations outside of its prior knowledge. For example, a child going to a fancy restaurant for the first time may expect his parents to pay when they order, as they do when they take him to McDonald’s.

• Failure to notice that a situation merits attention: Even if an understander has appropriate rules, it may not recognize that they apply. For example, even if the stock market shows classic signs of weakening, wishful thinking may make an investor miss its warning signs.

• Application of inappropriate knowledge: No rule about the real world can specify all the factors that affect its applicability—there are simply too many implicit conditions. Consequently, there is always a possibility that applied knowledge will not be appropriate. For example, clocks in airports and train stations are usually kept quite accurate, but this generalization may fail without warning—in one airport, many passengers missed their flights because the clocks had not been reset after the previous night’s time change.

• Changes in the world: Even if an understander applies the correct knowledge when it first interprets a situation, the situation may change. For example, a luxurious house may change overnight from a good investment to a bad one, if the state decides to put a freeway through its front yard.

Because no real-world system can maintain perfect beliefs, the best that can be hoped for is to notice any problems that arise and to generate appropriate repairs. These are the belief maintenance issues faced by understanders of everyday events.

1.2 Maintaining Beliefs During Understanding

Belief maintenance depends on two things: deciding when beliefs need to be updated, and deciding how to update them. We consider these problems in the context of an understanding system that must maintain accurate beliefs and expectations despite partial information. In addition, we assume that the understander is part of a system with some overarching task (e.g., planning) that places additional requirements on the understanding process through task-specific needs for information. Thus our theory is one of belief maintenance in goal-driven understanding.

Our theory of goal-driven understanding involves four component processes: (a) routine understanding, (b) anomaly detection, (c) construction of explanations, and (d) evaluation of candidate explanations, to assure that the explanation selected provides sufficient information for the explainer to accomplish system goals.

Before looking at each of these processes in detail, we give a simple example of how the processes interact in a goal-based understander. Suppose Mary is waiting for the 12:20 bus to the airport, and the time is now 12:40. Failure of the bus to arrive is a failure of Mary’s expectations and shows that her beliefs need to be revised. However, the importance of the failure does not arise from the abstract goal to have an accurate world model. Instead, it is important because it may affect her concrete plans and goals: She may be in danger of missing her plane. The explanation effort is motivated by her need to understand the situation in order to get to the airport on time.

Obviously, in order to respond to the unexpected problem, her routine understanding must include anomaly detection. If she did not notice the anomaly of the bus’s failure to appear and simply waited at the bus stop, she might miss her flight. When she notices the delay, she needs to construct an explanation, in order to determine the appropriate response. If she explains the delay by a recent schedule change by the bus company, but knows that the bus should arrive in 5 minutes, she may wait. If she explains it by a bus strike, she needs to take a taxi. If the delay is caused by roads being closed to let a presidential motorcade cross the city, a taxi may not be faster, and she needs to take the subway instead.

As she looks for an explanation, she needs to be able to evaluate candidate explanations, to reject implausible candidates. For example, if a passerby explains the bus’s failure to appear by the transit authority’s cancellation of that route, she should reject that explanation if she saw a bus stop there earlier in the day. In addition, she must elaborate explanations that fail to give sufficient information. For example, if a passerby tells her that the bus broke down a few blocks away, it would be useful to know the cause of the breakdown, to estimate the delay before the bus could restart its route. Thus each of the four component processes in goal-driven understanding plays a crucial role.

1.3 Routine Understanding

Routine understanding is simply the integration of new information into prior knowledge. Recognizing the coherence of ideas is part of this integration: A program does not understand “Mary won a million dollars and took a trip around the world” unless it knows that her new wealth probably enabled the trip. The construction of chains representing such causal links is an important way to establish textual coherence [Schank, 1975].

One way to facilitate forming causal connections between events is to pre-package common chains in schemas or knowledge structures that can be applied as a unit to guide routine understanding [Schank & Abelson, 1977]. Once a knowledge structure has been selected to package an event, the appropriate causal connections are available with very low inference cost. Many AI models of understanding consider it primarily a process of finding an appropriate knowledge structure into which to fit the event, and applying that knowledge structure [Charniak, 1978; Cullingford, 1978; DeJong, 1979; Kolodner, 1984; Lebowitz, 1980; Minsky, 1975; Schank & Abelson, 1977]. A very simple example of this process is making sense of the sentence “Mary ordered lobster,” by recognizing it as part of a restaurant meal. Once the knowledge structure for restaurant meals has been activated, it provides connections between Mary’s ordering and likely prior events (we can assume that she entered the restaurant, and that she received a menu) as well as leading us to expect certain future events (such as her being served) without having to infer them from scratch.

As previously mentioned, much research effort has been devoted to the problem of selecting a knowledge structure to package an event, but little effort has been devoted to resolving problems that arise after a knowledge structure has been selected. These problems may be conflicts between external information and system beliefs and expectations, or conflicts between expectations that arise from multiple knowledge structures that are active simultaneously. For example, an understander might have a schema for the behavior of fraternity members at parties that includes excessive consumption of alcohol, as well as a schema for athletic training which prohibits such behavior. If both schemas were active simultaneously for a single person, accurate expectations would depend on noticing and resolving the conflict: deciding whether the person would refuse drinks, or would perform less well than normal in the next day’s game.

Understanding a fact requires more than placing it in a knowledge structure in memory: it depends on placing it in memory and reconciling it with other active knowledge.

Reconciling old and new information requires detecting when conflicts arise, in order to resolve them. This is the process of anomaly detection.

1.4 Detecting Anomalies

Intuitively, an anomaly is something that surprises us. Consequently, it might appear that being anomalous is a property of events in themselves. However, given any surprising event, we can imagine background knowledge that would render it mundane. It would usually be surprising for an elephant to trap someone in a telephone booth, but not within the context of a TV show like Candid Camera. In Chapter 3, we argue that being anomalous is a property of the interaction between events and context: Any particular fact can be anomalous or nonanomalous, depending on the situation and on the processing we are doing. A new fact is anomalous if we cannot reconcile it with other knowledge provided by the context that guides our understanding.

Anomalies are conflicts between aspects of our active knowledge.

The success of anomaly detection depends on two things. First, it depends on applying a wide enough range of checks to detect a large proportion of conflicts. Second, it depends on restricting verification effort so that anomaly detection does not overburden the system’s processing resources.

1.4.1 Anomaly Detection Must Consider Many Perspectives

Any event can be understood from many different perspectives; which one is appropriate depends on the active goals of the understander. If a friend buys something in a store, our explanation might focus on his motivation, to refine our expectations for his behavior—if he buys $10 worth of fishing lures, we may hypothesize that fishing is one of his hobbies. To the managers of a competing store, other aspects of the situation woul...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• Acknowledgments
• Overview and Reader’s Guide
• 1 Explanation and Understanding
• 2 Perspective on the Theory
• 3 Anomalies and Routine Understanding
• 4 Pattern-Based Anomaly Detection
• 5 Anomaly Characterization
• 6 A Vocabulary for Anomalies
• 7 Nonmotivational Anomaly Types
• 8 Evaluating Relevance and Plausibility
• 9 Focusing on Important Factors
• 10 Conclusions and Future Directions
• References
• Author Index
• Subject Index