Decision Making in Aviation

Don Harris

• Publisher: Routledge
• Year: 2017
• ISBN:
  9781351570039
• Edition: 1
• Topic: Technology & Engineering
• Subtopic: Industrial Engineering
• Index: Technology & Engineering

Decision making pervades every aspect of life: people make hundreds of decisions every day. The vast majority of these are trivial and without a right or wrong answer. In some respects there is also nothing extraordinary about pilot decision making. It is only the setting that is different - the underlying cognitive processes are just the same. However, it is the context and the consequences of a poor decision which serve to differentiate aeronautical decision making. Decisions on the flight deck are often made with incomplete information and while under time pressure. The implications for inadequate performance is much more serious than in many other professions. Poor decisions are implicated in over half of all aviation accidents. This volume contains key papers published over the last 25 years providing an overview of the major paradigms by which aeronautical decision making has been investigated. Furthermore, decision making does not occur in isolation. It is a joint function of the flight tasks; knowledge; equipment on the flight deck and other stressors. In this volume of collected papers, works from leading authors in the field consider all these aspects of aeronautical decision making.

Part I
Approaches to the Study of Aeronautical Decision Making

[1]

Focus Article: Taking Stock of Naturalistic Decision Making

RAANAN LIPSHITZ¹*, GARY KLEIN², JUDITH ORASANU³ and EDUARDO SALAS

¹University of Haifa, Haifa, Israel

²Klein Associated Inc., USA

³NASA Ames, USA

⁴University of Central Florida, USA

ABSTRACT

We review the progress of naturalistic decision making (NDM) in the decade since the first conference on the subject in 1989. After setting out a brief history of NDM we identify its essential characteristics and consider five of its main contributions: recognition-primed decisions, coping with uncertainty, team decision making, decision errors, and methodology. NDM helped identify important areas of inquiry previously neglected (e.g. the use of expertise in sizing up situations and generating options), it introduced new models, conceptualizations, and methods, and recruited applied investigators into the field. Above all, NDM contributed a new perspective on how decisions (broadly defined as committing oneself to a certain course of action) are made. NDM still faces significant challenges, including improvement of the quantity and rigor of its empirical research, and confirming the validity of its prescriptive models.

KEY WORDS naturalistic decision making; recognition-primed decisions; coping with uncertainty; team decision making; decision errors; decision training; research methodology

The study of decision making is studded by three-letter acronyms designating sub-disciplines which evolved partly as extensions of preceding sub-disciplines, and partly as a reaction to them: the once-popular CDM (Classical Decision Making), BDT (Behavioral Decision Theory), JDM (Judgment and Decision Making), ODM (Organizational Decision Making), and, most recently, NDM (Naturalistic Decision Making). The emergence of each sub-discipline can be conveniently traced to the publication of books or papers signifying the time at which theory and research pursued more or less in isolation gathered sufficient mass and coherence to attract wider attention. CDM can be traced to Bernoulli (1738) and, more recently, to Savage (1954) and von Neumann and Morgenstern (1944). BDT and JDM have their origins in Edwards (1954) and Meehl (1954). ODM can be traced to Simon (1957), March and Simon (1958), and Cyert and March (1963). Finally, NDM goes back to Klein et al. (1993). A decade has now passed since the conference that produced the last-named volume, a sufficiently long period to take stock of NDM: its essential characteristics, strengths, weaknesses, and future prospects. After drawing a historical sketch of NDM, we present its essential characteristics and examine critiques of its theoretical bases, methodology, and contributions, focusing on five areas: recognition-primed decisions, coping with uncertainty, decision errors, team decision making, and decision aiding and training. We close the paper by drawing some conjectures regarding the future directions of NDM.

A BRIEF HISTORY OF NDM

The NDM framework was initiated in 1989 in a conference in Dayton, Ohio, sponsored by the Army Research Institute. The conference enabled some 30 behavioral scientists working in academic and non-academic institutions to discover that they shared many common themes, regardless of domain. One theme was the importance of time pressure, uncertainty, ill-defined goals, high personal stakes, and other complexities that characterize decision making in real-world settings. Although these factors were difficult to replicate in the laboratory, they needed to be understood (Orasanu and Connolly, 1993). A second theme was the importance of studying people who had some degree of expertise; novices were never used in the study of the type of high-stakes tasks that were of interest (Pruitt et al., 1997). A third theme was that the way people sized up situations seemed more critical than the way they selected between courses of action (Klein, 1993).

In the past ten years there has been an increasing amount of interest in NDM. The 1989 conference (Klein et al., 1993) was followed by a second conference (Zsambok and Klein, 1997) held in 1994 and attended by approximately 100 researchers. A third NDM conference was held in Aberdeen, Scotland, in 1996 (Flin et al., 1997), and a fourth conference was held in Warrenton, Virginia, in 1998 (Salas and Klein, in press). In addition to the edited volumes emerging from each conference, Flin (1996) has written about the issues facing critical incident managers, Klein (1998) has described the work of his research group, and Cannon-Bowers and Salas (1998) edited a book describing the research program sponsored by the US Navy in the aftermath of the Vincennes incident. Finally, Beach (1997) surveyed NDM from the vantage point of his own work on Image Theory, a model that is aligned with the NDM framework. In addition to these publications, the Human Factors and Ergonomics Society established a technical group in 1995, called ‘cognitive engineering and decision making’, partly as an outlet for research and development along the lines of NDM. As of 1998 there were more than 500 members, making it one of the largest technical groups in the Society.

ESSENTIALS OF NATURALISTIC DECISION MAKING

NDM is an attempt to understand how people make decisions in real-world contexts that are meaningful and familiar to them. Fulfilling this ‘mission’ produced research marked by five essential characteristics: proficient decision makers, situation-action matching decision rules, context-bound informal modeling, process orientation, and empirical-based prescription. These particular characteristics were derived by locating the place of NDM in the study of decision making based on Rasmussen’s (1997) observation that

In several human sciences [including decision research], a trend is found in modeling behavior: Efforts are moving from normative models of rational behavior [e.g., CDM], through efforts to model the observed rational behavior by means of models of the deviation from rational [e.g., JDM], toward focus on representing directly the actually observed behavior [e.g., NDM], and ultimately to efforts to model behavior generating mechanisms [i.e., models of system constraints, opportunities and criteria, e.g., ODM] (p. 75, material in brackets added by us).

Granted that reconstruction of history inevitably finesses subtle twists and turns in the actual turn of events, Rasmussen’s sequence does fairly well (ODM in fact preceded NDM and fits into Rasmussen’s sequence only in terms of the move from individual to system-wide models). Our historical perspective suggests that one way of deriving the essential characteristics of NDM is to examine its differences from CDM, the preceding phase in Rasmussen’s sequence.

The essential characteristics of CDM were (1) choice (conceptualizing decision making as choosing among concurrently available alternatives, e.g. Dawes, 1988; Hogarth, 1987), (2) input–output orientation (focusing on predicting which alternative will, or should be, chosen given a decision maker’s preferences: Funder, 1987), (3) comprehensiveness (conceptualizing decision making as a deliberate and analytic process that requires a relatively thorough information search (Beach and Mitchell, 1978, Payne et al., 1990), particularly for optimal performance (Gigerenzer and Todd, 1999; Grandori, 1984), and (4) formalism (the development of abstract, context-free models amenable to quantitative testing, e.g. Coombs et al., 1971). The history of decision research consists of the gradual replacement of these characteristics, beginning with doubts regarding their effects on the descriptive validity of CDM and culminating in the replacement of all four by other characteristics for descriptive as well as prescriptive purposes in NDM.

Doubts regarding the validity of the rational choice model as a valid description of human decision making probably preceded the work of Simon and his associates at Carnegie-Mellon University. However, their contribution was seminal because it went beyond just pointing out that the informational requirements (i.e. comprehensiveness) entailed in the model exceed limited human cognitive capacities. Through the concept of bounded rationality, which points to attention as the scarce resource in human decision making, Simon et al. showed that people’s systematic deviations from the rational choice model make sense from an adaptive perspective: under bounded rationality thoroughgoing information processing is exhausting, and potentially futile. A second, and just as important though less publicized, proposition of the Carnegie School was an attack on the prescriptive validity of the Rational Choice model. As Simon (1978) suggested, real-world problems are typically loosely coupled, allowing decision makers with bounded rationality to attend to them effectively in a sequential fashion. Thus, effective adaptation does not require comprehensive analysis. Instead, all that are required are a modest intellectual capacity, an ability to detect and prioritize problems, and the ability to learn from experience.

JDM/BDT further undermined the descriptive validity of CDM, showing that people tend to deviate systematically from the rational choice model even when presented with relatively simple tasks which do not severely tax bounded rationality (Kahneman et al., 1982). However, JDM/BDT retained the essential characteristics of CDM and adhered to its normative models as standards for evaluating decision quality. Thus, Elimination by Aspects (Tversky, 1972), Prospect Theory (Kahneman and Tversky, 1979), and Einhorn and Hogarth’s (1986) Ambiguity Model, as three representative examples, are all formal choice models that describe which alternative is chosen from an available set of alternatives based on different comparison schemes. In addition, JDM/BDT texts prescribe Multi-Attribute Utility (MAU)-like and Subjective Expected Utility (SEU)-like procedures (Russo and Schoemaker, 1987) and ‘de-biasing’ procedures for correcting deviations from these models (Fischhoff, 1982).

Going beyond JDM/BDT’s criticism of CDM, NDM replaced all the four essential characteristics of the latter identified above. Comprehensive choice was replaced by matching, input-output orientation was replaced by process orientation, and context-free formal modeling was replaced by context-bound informal modeling. It is fair to say that these characteristics followed once researchers within the NDM framework embarked on the construction of descriptive models of proficient decision makers in natural contexts without relying on normative choice models as starting points. Following the emphasis on bounded rationality of the Carnegie School, NDM places the human (and hence boundedly rational) proficient decision maker at its center of interest and as its basis for prescription.

Proficient decision makers

In the decade since the first NDM conference in 1989, the definition of NDM has changed. It is marked by a shift in the relative emphasis placed on expertise and features of field settings in which decisions are made. The original definition proposed by Orasanu and Connolly (1993) emphasized the shaping features of the contexts in which many decisions of interest were made: ill-structured problems, uncertain, dynamic environments, shifting, ill-defined, or competing goals, multiple event-feedback loops, time constraints, high stakes, multiple players, and organizational settings. Expertise was included as a secondary factor.

By the time of the second NDM conference, an alternative definition had emerged. Zsambok (1997, p. 4) distinguishes NDM in terms of the decision maker, positing that ‘NDM is the way people use their experience to make decisions in field settings’.

Pruitt et al. (1997) went one step further, and concluded that the primary factor defining NDM studies is expertise:

[I]t is possible to answer the question of knowing an NDM study … by looking at how the study handles the subject’s prior experience … Does the study treat prior experience as a nuisance variable (one to be controlled, counterbalanced, or otherwise ignored) or does it view this variable as the focus of inquiry? We would argue that CDM [BDT, and JDM] do the former and NDM does the latter … [T]he strength of NDM is its emphasis on experience and knowledge which already is present in the subject. Looking back at the short definition of Zsambok [above] we believe that the inclusion of ‘in field settings’ is only secondary (pp. 37–38).

Still, we cannot ignore the influence of field settings because they establish the eliciting conditions for making decisions and shape decisions through their constraints and affordances. ‘Expertise’ is about these field settings.

Granted that NDM is concerned with proficient decision makers, namely people with relevant experience or knowledge in the decision-making...