Situation Awareness Analysis and Measurement

Mica R. Endsley, Daniel J. Garland, Mica R. Endsley, Daniel J. Garland

• Publisher: CRC Press
• Year: 2000
• ISBN:
  9781135691479
• Edition: 1
• Topic: Informatica
• Subtopic: Interazione tra uomo e computer

A comprehensive overview of different approaches to the measurement of situation awareness in experimental and applied setting, this book directly tackles the problem of ensuring that system designs and training programs are effective at promoting situation awareness. It is the first book to provide a all-inclusive coverage of situation awareness and its measurement. Topics addressed provide a detailed analysis of the use of a wide variety of techniques for measuring situation awareness and situation assessment processes. It provides a rich resource for engineers and human factors psychologists involved in designing and evaluating systems in many domains.

Part I
INTRODUCTION AND OVERVIEW

Chapter 1
Theoretical Underpinnings of Situation Awareness: A Critical Review

Mica R.Endsley
SA Technologies, Inc.

The enhancement of operator situation awareness (SA) has become a major design goal for those developing operator interfaces, automation concepts, and training programs in a wide variety of fields, including aircraft, air traffic control (ATC), power plants, and advanced manufacturing systems. This dramatic growth in interest in SA, beginning in the mid-1980s and accelerating through the 1990s, was spurred on by many factors, chief among them the challenges of a new class of technology.

One can easily see that SA has always been needed in order for people to perform tasks effectively. Prehistoric man undoubtedly needed to be aware of many cues in his environment in order to successfully hunt and keep from being hunted. For many years, having good SA was largely a matter of training and experience—learning the important cues to watch for and what they meant.

With the advent of the machine age, our emphasis shifted to creating a new class of tools to help people perform tasks, largely those physical in nature. The computer age and now the information age have followed rapidly on the heels of basic mechanization. The tools provided are no longer simple; they are amazingly complex, focused on not just physical tasks, but elaborate perceptual and cognitive tasks as well. The pilot of today’s aircraft, the air traffic controller, the power plant operator, the anesthesiologist: all must perceive and comprehend a dazzling array of data that is often changing very rapidly. I have taken to calling this challenge the information gap (Fig. 1.1).

FIG. 1.1. The information gap.

Today’s systems are capable of producing a huge amount of data, both on the status of their own components and on the status of the external environment. Due to achievements in various types of datalink and internet technologies, systems can also provide data on almost anything anywhere in the world. The problem with today’s systems is not a lack of information, but finding what is needed when it is needed.

Unfortunately, in the face of this torrent of data, many operators may be even less informed than ever before. This is because there is a huge gap between the tons of data produced and disseminated and the operator’s ability to find the necessary bits and process them together with the other bits to arrive at the actual information required for their decisions. This information must be integrated and interpreted correctly as well, frequently a tricky task. This problem is real and ongoing whether the job is in the cockpit or behind a desk. It is becoming widely recognized that more data does not equal more information. Issues of automation and “intelligent systems” have frequently only exacerbated the problem, rather than aided it (Endsley & Kiris, 1995; Sarter & Woods, 1995).

The criteria for what we are seeking from system designs have correspondingly changed. In addition to designing systems that provide the operator with the needed information and capabilities, we must also ensure that it is provided in a way that is usable cognitively as well as physically. We want to know how well the system design supports the operator’s ability to get the needed information under dynamic operational constraints (i.e., How well does it bridge the information gap?). This design objective and measure of merit has been termed SA.

WHAT IS SA?

Most simply put, SA is knowing what is going on around you. Inherent in this definition is a notion of what is important. SA is most frequently defined in operational terms. Although someone not engaged in a task or objective might have awareness (e.g., sitting under a tree idly enjoying nature), this type of individual has been largely outside the scope of human factors design efforts. Rather, we have been concerned mostly with people who need SA for specific reasons. For a given operator, therefore, SA is defined in terms of the goals, and decision tasks for that job. The pilot does not need to know everything (e.g., the copilot’s shoe size and spouse’s name), but does need to know a great deal of information related to the goal of safely flying the aircraft. Surgeons have just as great a need for SA; however, the things they need to know about will be quite different and dependent on a different set of goals and decision tasks.

Although the “elements” of SA vary widely between domains, the nature of SA and the mechanisms used for achieving SA can be described generically. (A determination of the elements of SA for different domains is discussed in more detail in chap. 8.) It is the goal of this chapter to provide a foundation for understanding the construct that is SA. This foundation is important both for creating systems that support SA and for creating tools that effectively measure SA.

Many definitions of SA have been developed. Some are very closely tied to the aircraft domain and some are more general. (See Dominguez, 1994, or Fracker, 1988, for a review.) That is, many are tied to the specifics of one domain, aircraft piloting, from whence the term originated. SA is now being studied in a variety of domains, however; education, driving, train dispatching, maintenance, and weather forecasting are but a few of the newer areas in which SA has been receiving attention.

A general definition of SA that has been found to be applicable across a wide variety of domains describes SA as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future” (Endsley, 1988, p. 97). Shown in Fig. 1.2, this definition helps to establish what “knowing what is going on” entails.

Level 1 SA: Perception. Perception of cues (Level 1 SA) is fundamental. Without a basic perception of important information, the odds of forming an incorrect picture of the situation increase dramatically. Jones and Endsley (1996) found that 76% of SA errors in pilots could be traced to problems in the perception of needed information (due to either failures or shortcomings in the system or problems with cognitive processes).

FIG. 1.2. Model of SA in dynamic decision making (from Endsley, 1995b).

Note. From “Toward a theory of situation awareness in dynamic systems” by M.R.Endsley. In Human Factors, 37(1), 32– 64, 1995. Copyright 1995 by the Human Factors and Ergonomics Society. Reprinted with permission.

Level 2 SA: Comprehension. SA as a construct goes beyond mere perception, however. It also encompasses how people combine, interpret, store, and retain information. Thus, it includes more than perceiving or attending to information; it includes the integration of multiple pieces of information and a determination of their relevance to the person’s goals (Level 2 SA). This is analogous to having a high level of reading comprehension as compared to just reading words. Twenty percent of SA errors were found to involve problems with Level 2 SA (Jones & Endsley, 1996).

Flach (1995) pointed out that “the construct of situation awareness demands that the problem of meaning be tackled head-on. Meaning must be considered both in the sense of subjective interpretation (awareness) and in the sense of objective significance or importance (situation)” (p.). A person with Level 2 SA has been able to derive operationally relevant meaning and significance from the Level 1 SA data perceived. As Flach pointed out, this aspect of SA sets it apart from earlier psychological research and places it squarely in the realm of ecological realism.

Level 3 SA: Projection. At the highest level of SA, the ability to forecast future situation events and dynamics (Level 3 SA) marks operators who have the highest level of understanding of the situation. This ability to project from current events and dynamics to anticipate future events (and their implications) allows for timely decision making. In almost every field I have studied (aircraft, ATC, power plant operations, maintenance, medicine), I have found that experienced operators rely heavily on future projections. It is the mark of a skilled expert.

Temporal Aspects of SA. Time, both the perception of time and the temporal dynamics associated with events, plays an important role in the formulation of SA. First, time itself has appeared as an important component of SA in many domains (Endsley, 1993b, 1994; Endsley, Farley, Jones, Midkiff, & Hansman, 1998; Endsley & Robertson, 1996; Endsley & Rodgers, 1994). A critical part of SA is often understanding how much time is available until some event occurs or some action must be taken. The phrase “within a volume of space and time” contained in the definition of SA derives from the fact that operators constrain the parts of the world (or situation) that are of interest to them based not only on space (how far away some element is), but also on how soon that element will have an impact on the operator’s goals and tasks. Time is a strong part of Level 2 SA (comprehension) and Level 3 SA (projection of future events).

The dynamic nature of real-world situations is a third important temporal aspect of SA. The rate at which information changes is that part of SA regarding the current situation that also allows for projection of future situations (Endsley, 1988, 1995c). A situation’s dynamic nature dictates that as the situation is always changing, so the person’s SA must constantly change or be rendered outdated and inaccurate. In highly dynamic environments, this forces the human operator to adapt many cognitive strategies for maintaining SA. Adams, Tenney, and Pew (1995) emphasized the importance of the dynamics of both situations and cognitive processes in their model of SA. Sarter and Woods (1991) also discussed the importance of the temporal aspects of the situation for SA.

SA and Decision Making

The Endsley model, shown in Fig. 1.2, shows SA as a stage separate from decision making and performance. SA is depicted as the operator’s internal model of the state of the environment. Based on that representation, operators can decide what to do about the situation and carry out any necessary actions. SA therefore is represented as the main precursor to decision making; however, many other factors also come into play in turning good SA into successful performance.

SA is clearly indicated as a separate stage in this model rather than as a single combined process. This is for several reasons. First, it is entirely possible to have perfect SA, yet make an incorrect decision. For example, battle commanders may understand where the enemy is and the enemy’s capabilities, yet select a poor, or inappropriate, strategy for launching an attack. They may have inadequate strategies or tactics guiding their decision processes. They may be limited in decision choices due to organizational or technical constraints. They may lack the experience or training to have good, well developed plans of action for the situation. Individual personality factors (such as impulsiveness, indecisiveness, or riskiness) may also make some individuals prone to poor decisions. A recent study of human error in aircraft accidents found that 26.6% involved situations where there was poor decision making even though the aircrew appeared to have adequate SA to make a correct decision (Endsley, 1995b). Conversely, it is also possible to make good decisions even with poor SA even if only by luck.

This characterization is not meant to dispute the important role of SA in the decision-making process or the integral link between SA and decision making in many instances, particularly where experienced decision makers are involved. Klein’s work in the area of recognition-primed decision making shows strong evidence of a direct link between situation recognition-classification and associated action selection (Klein, 1989; Klein, Calderwood, & Clinton-Cirocco, 1986). Where such learned linkages exist, they undoubtedly may be activated frequently in the decision process. Adams et al. (1995) and Smith and Hancock (1994) also discussed the integral relation between SA and decision making. Decisions are formed by SA and SA is formed by decisions. These are certainly views with which I agree. I nevertheless feel it is important to recognize that SA and decision making need not be coupled as one process and in practice frequently are not.

The human operator makes a conscious choice in the decision to implement the linked recognition-primed decision action plan or to devise a new one. This behavior can be seen in combat tasks, for instance, where people often wish to be unpredictable. In the many instances where no action plan is readily linked to the recognized situation, a separate decision as to what to do must take place. Just as easily as SA and decision making can be linked in practice, they can also be unlinked. Although this distinction may be purely theoretical, it is made here for the purpose of clarity of discussion. SA is not decision making and decision making is not SA. This distinction has implications for the measurement of SA.

Furthermore, the link between human decision making and overall performance is indirect in many environments. A desired action may be poorly executed due to physical error, other workload, inadequate training, or system problems. The system’s capabilities may limit overall performance. In some environments, such as the tactical aircraft domain, the action of external agents (e.g., enemy aircraft) may also create poor performance outcomes from essentially good decisions and vice versa. Therefore, for theoretical purposes, SA, decision making, and performance can be seen as distinct stages that can each affect the other in a circular ongoing cycle, yet which can be decoupled through various oth...