Designing Soldier Systems

Current Issues in Human Factors

John Martin, Laurel Allender, John Martin, Laurel Allender, Pamela Savage-Knepshield, John Lockett, John Martin, Laurel Allender, Pamela Savage-Knepshield, John Lockett

• Publisher: CRC Press
• Year: 2018
• Print ISBN:
  9781409407775
• eBook ISBN:
  9781317152071
• Topic: Technology & Engineering
• Subtopic: Philosophy & Ethics in Science
• Index: Technology & Engineering

This book focuses on contemporary human factors issues within the design of soldier systems and describes how they are currently being investigated and addressed by the U.S. Army to enhance soldier performance and effectiveness. Designing Soldier Systems approaches human factors issues from three main perspectives. In the first section, Chapters 1-5 focus on complexity introduced by technology, its impact on human performance, and how issues are being addressed to reduce cognitive workload. In the second section, Chapters 6-10 concentrate on obstacles imposed by operational and environmental conditions on the battlefield and how they are being mitigated through the use of technology. The third section, Chapters 11-21, is dedicated to system design and evaluation including the tools, techniques and technologies used by researchers who design soldier systems to overcome human physical and cognitive performance limitations as well as the obstacles imposed by environmental and operations conditions that are encountered by soldiers. The book will appeal to an international multidisciplinary audience interested in the design and development of systems for military use, including defense contractors, program management offices, human factors engineers, human system integrators, system engineers, and computer scientists. Relevant programs of study include those in human factors, cognitive science, neuroscience, neuroergonomics, psychology, training and education, and engineering.

PART I
Understanding Human Performance with Complex Systems

Chapter 1
Human Performance Challenges for the Future Force: Lessons from Patriot after the Second Gulf War

John K. Hawley and Anna L. Mares

Background

During the major combat operations phase of the Second Gulf War (Operation Iraqi Freedom (OIF), March and April 2003), US Army Patriot air and missile defense (AMD) units were involved in two fratricide incidents. In the first, a British GR-4 Tornado was misclassified as an anti-radiation missile and subsequently engaged and destroyed. The second fratricide incident involved a Navy F/A-18 Hornet that was misclassified as a tactical ballistic missile (TBM) and also engaged and destroyed. Three flight crewmembers lost their lives in these incidents. OIF involved a total of 11 Patriot engagements by US units. Of these 11, 9 resulted in successful TBM engagements; the other two (18 percent) were fratricides. Although significant in and of themselves, these fratricides opened the door for a unique look at the human performance problems introduced by increasing system and operational complexity in a major weapons system. The initial assessment was followed by a multi-year effort focused on remedying the problems identified during the initial incident investigation. Lessons and observations from the initial fratricide assessment and follow-on mitigation work set the stage for much of the discussion to follow. But first, we present a little background on the Patriot system.

Patriot is the Army’s first-line AMD system. The system has been in the active force since the mid-1980s, but has been upgraded numerous times since first fielded. Initially, Patriot was intended as a defense against conventional air-breathing threats (ABTs). However, since Operation Desert Storm (ODS), the First Gulf War in the early 1990s, the system has been used primarily against TBMs. Future usage scenarios portray the system being used against a spectrum of air threats, including TBMs, conventional ABTs, cruise missiles, and unmanned aircraft systems. The range of potential air threats in the contemporary operating environment, along with major changes in the system’s employment concept, have significantly increased the complexity of the air battle management¹ problem for Patriot and other AMD systems. Arthur (2009) argues that systems tend to become more complex as they are enhanced to meet new usage requirements.

Figure 1.1 presents an overview of the Patriot AMD system and its contemporary operating environment. To begin, the system itself is technically and tactically complex. Patriot currently employs more than 3.5 million lines of software code in air battle management operations. Number of lines of software code is sometimes used as a rough proxy for system complexity. An additional layer of operational complexity is added by a requirement for Patriot air battle management crews to coordinate with other systems, composing what is termed an “integrated air and missile defense system.” These associated systems include the Air Force’s Airborne Warning and Control System, the Army’s Joint Land Attack Cruise Missile Defense Elevated Netted Sensor and Terminal High Altitude Area Defense systems, the Navy’s Aegis sea-based missile defense system, and various space-based National Assets. The current Patriot as part of an integrated AMD system of systems illustrates the complexity associated with contemporary concepts for joint, network-enabled operations. The requirement to coordinate with the associated systems also means that from an analytical perspective, Patriot must be approached as a system of systems.

Figure 1.1 The Patriot AMD system and its operating environment

Given that Patriot is an existing system and has been in the Army’s inventory since the early 1980s, what do lessons from Patriot tell us about system complexity and its impact on future Army systems and future military operations? We think that in some respects Patriot provides a glimpse into the future of military systems and operations. As Patriot has evolved over the past 25 years, it has acquired features and characteristics that are more typical of systems the Army will employ in the future than those in the current inventory. In drawing a parallel between current Patriot operations and future Army systems, we are not suggesting that future systems or their performance demands will be identical to Patriot. Rather, we assert that many emerging systems may result in an operator and crew performance environment similar to Patriot’s. At a conceptual level, many of the performance demands placed on individual soldiers and crews will be similar to those encountered in Patriot.

When considering human performance topics in a system of systems setting similar to Patriot, one cannot avoid the issues of system complexity and the impact of complexity on system users. Complexity is a consistent theme in much of the discussion to follow. That said, a necessary first step is to define the term “complexity,” or, more specifically, the “state of being complex.” The dictionary definition of “complex” (Merriam-Webster 2009) refers to a system having many interconnected or related parts, or a system that has a complicated structure—not simple or straightforward. A second theme that emerges in an attempt to define complexity is the degree of orderliness or predictability of a system or process (Dekker 2005). An unpredictable, disorderly system or process, by definition, presents a high degree of complexity for users. Unpredictability is moderated by the amount of time available to effect control.

A definition that appears to encompass both of these definitional themes is given in Hollnagel and Woods 2005. Following these authors, system or process complexity is a function of (1) the number of parameters needed to fully define the system or process in space and time; and (2) the amount of information needed to fully comprehend the system or process and its operating environment. The impact of complexity on users revolves around the second portion of the Hollnagel and Woods definition: that is, the amount of information needed to comprehend the system and operating environment at any point in space and time. This requirement to fully comprehend the system and its operating environment is the key to maintaining effective human control and has significant implications for system design and user job preparation. The next part of this chapter begins our discussion of the impact of complexity on a system’s users. Again, the context for this discussion is an assessment of actual events involving the Patriot AMD system during the major combat operations phase of OIF.

The Patriot Vigilance Project

A team from the US Army Research Laboratory (ARL) began looking into the OIF Patriot fratricides and the more general issue of Patriot human–system performance at the invitation of the then Fort Bliss, TX, Commanding General, Major General (MG) Michael A. Vane. Fort Bliss was the site of the Army’s Air Defense Artillery (ADA) Center and School. In his own words, MG Vane was interested in operator vigilance and situational awareness (SA) as they relate to the performance of automated air defense battle management systems. (The generally accepted definition of SA is from Endsley, Bolte, and Jones (2003), who define it as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status in the near future). MG Vane was particularly concerned by what he termed a “lack of vigilance” on the part of Patriot air battle management crews, along with an apparent “lack of cognizance” of what was being presented to them on situation displays, with a resulting “unwarranted trust in automation.” The project team spent most of the summer and fall of 2004 reviewing the OIF fratricide incidents—reading documents, interviewing knowledgeable personnel in the Fort Bliss area, and observing Patriot training and operations. An initial report of findings was delivered to MG Vane in October 2004. ARL’s assessment was not intended to be just another exercise in “Monday morning quarterbacking.” Instead, the focus was to look into the deeper story behind the events leading to the OIF fratricides from a human performance perspective and to identify actionable solutions. MG Vane’s reference to lack of vigilance on the part of Patriot operators led to the effort being called the “Patriot Vigilance project.”

The ARL assessment team organized its presentation to MG Vane around two central themes, denoted (1) undisciplined automation during Patriot development; and (2) automation misuse on the part of Patriot crews. These themes and related contributors are shown graphically in Figure 1.2. A more detailed discussion of the findings of ARL’s initial fratricide assessment is provided in Hawley and Mares (2006). What follows is a summary of material from that source intended to set the stage for the ensuing discussion. The first theme, or contributing factor, was termed “undisciplined automation.” This is defined as the automation of functions by designers and subsequent implementation by users without due regard for the consequences for human performance (Parasuraman and Riley 1997). Undisciplined automation tends to define the operators’ roles as by-products of the automation. Every function that can be automated is automated. Operators are left in the control process to monitor the engagement process and respond to system cues. In the case of Patriot, little explicit attention was paid during system design and subsequent testing to determine (1) what residual functions were left for the operators to perform, (2) whether operators actually could perform these functions, (3) how operators should be trained to perform properly, or (4) the impact of potential operator training deficiencies on the system’s automated engagement decision-making reliability.

The downstream impact of undisciplined automation was exacerbated by two contributing or secondary factors: (1) unacknowledged system fallibilities, and (2) a fascination with and “blind faith” in technology. An unacknowledged system fallibility is a system deficiency that is known but not satisfactorily resolved. For example, a series of Patriot operational tests going back to the 1980s indicated that the system’s automated engagement logic was subject to track misclassification problems—system fallibilities. The Patriot system classifies acquired tracks as conventional air-breathing, rotary wing, ballistic missile, cruise missile, anti-radiation missile, or other relevant category on the basis of flight profiles and other track characteristics such as point of origin and compliance with Airspace Control Orders. A misclassification occurs when the system-generated category designation does not match the track’s actual status. These sources of automation unreliability were not satisfactorily addressed during system software upgrades, nor did information about them find its way into operator training, air battle management doctrine, crew procedures, or unit standard operating procedures. It should be noted that there are no technical quick fixes or easy solutions to such classifications problems. Research, experimentation, and operational experience suggest that any solution lies in the domain generally labeled situation-specific trust in automation (Hawley and Mares 2007). Operators and crews must have the technical and tactical expertise to determine when they can trust machine-generated solutions and when they should question such results. In the case of Patriot, system developers continued to pursue technology-centric solutions to automation reliability problems (e.g., increased use of artificial intelligence, non-cooperative target recognition, improved identification friend or foe query systems, etc.). When challenged concerning the system’s track classification accuracy, the claim was repeatedly made that a technical fix for these problems was “just around the corner.” But the basic problem remained unresolved: the system was not completely reliable in critical functional areas, most notably track classification and identification. To make matters worse, users were not informed regarding these reliability problems, or if they were informed, little if any effective responsive action was identified for them to take. Training enhancements and other crew-oriented solutions to the classification reliability problem were not considered.

Figure 1.2 Human performance contributors to the Patriot system’s fratricides during Operation Iraqi Freedom

In the aftermath of the First Gulf War (ODS), the air defense user community acquiesced in the developmental community’s apparent lack of concern for problems with Patriot’s track classification accuracy. Emboldened by Patriot’s seeming success in engaging the Iraqi TBM threat during ODS, Patriot’s organizational culture and command structure emphasized reacting quickly, engaging early, and trusting the system without question (Hawley and Mares 2006). The view that the system could be trusted without question persisted in spite of several “close calls” during ODS attributable to track classification problems and test results indicating that the system was not always accurate. This cultural norm was exacerbated by the air defense community’s traditional training practices, which were criticized in the Army’s post-fratricide review as emphasizing what were termed “rote drills” versus the exercise of high-level judgment. The Patriot user community approached training for air battle operations in much the same manner as less cognitively oriented tasks such as system movement and setup. The emphasis during training was on mastering routines (crew drills) rather than critical thinking and adaptive problem-solving (Hawley and Mares 2006). Inadequate individual and crew training were further impacted by the branch’s traditional methods of assigning personnel to air battle management crews. Traditional personnel assignment practices tended to place inexperienced personnel in key air battle management crew positions. Moreover, routine personnel administration practices tended to rotate crew members out of battle staff positions and to other jobs rather quickly. The result was that tactical crews were generally formed from the unit’s newest and least experienced personnel. System and operational complexity, inadequate training, and crew inexperience are a dangerous combination. As used here, the term “inadequate training” refers to training that is (1) too short to produce necessary levels of operator competence, (2) ill-focused, in the sense that training content does not address critical operator or crew skills, or (3) inappropriate, in that the instructional methods used are not suitable for the job’s skill content.

Before the first missile round was fired during OIF, the stage was thus set for the second primary contributor, termed “automation misuse” (Parasuraman and Riley 1997). Automation misuse took the form of extensive automation bias on the part of Patriot crews. Automation bias is defined as unwarranted overreliance on automation, which has been demonstrated to result in failures of monitoring (vigilance problems) and accompanying decision biases (unwarranted and uncritical trust in automation—“let’s do what the system recommends”). Recall that these are the very concerns expressed by MG Vane in his kickoff discussion with ARL’s project staff.

One must be careful not to lay too much blame for these shortcomings at the feet of the Patriot crews involved in the OIF incidents. The roots of their apparent human performance shortcomings can be traced back to systemic problems resulting from decisions made years earlier by concept developers, software engineers, procedures developers, testers, trainers, and unit commanders. In hindsight, the most surprising aspect of ARL’s fratricide assessment is that there really were no surprises. The OIF Patriot air battle management crews did what they had been trained to do, what Patriot’s culture and command structure emphasized and reinforced, and what the automation literature suggested they likely would do in such circumstances.

Patriot is a very lethal system. It can be argued that the system was not properly managed during OIF. Driven by technological opportunity and mission expansion, the Patriot air battle management crew’s role changed from traditional operators to supervisory controllers, whose primary role is to supervise the subordinate automated engagement routines controlling the system (Sheridan 1992). That is, the subordinate systems automatically close a control loop on the task or process, and the crew is expected to monitor these systems for correct performance and intermittently respond to system cues when necessary. Control is indirect through the automated engagement system as opposed to direct as in traditional manual control. The supervising crew is thus “on” the control loop versus “in” the loop, as with traditional manual control. The term “on-the-loop” versus “in-the-loop” is becoming standard to reflect this role change (USAF 2009: 14 and section 4.6). This change in terminology might appear minor, but it is significant for design and training.

The Story Behind the Story

ARL’s report to MG Vane recommended two primary actionable items to address the problems identified during the fratricide incident assessment:

1. Reexamine air defense battle manag...

Table of contents

• Cover Page
• Designing Soldier Systems
• Copyright Page
• Contents
• List of Figures
• List of Tables
• About the Editors
• List of Contributors
• Preface
• List of Abbreviations
• PART I UNDERSTANDING HUMAN PERFORMANCE WITH COMPLEX SYSTEMS
• PART II OVERCOMING OPERATIONAL AND ENVIRONMENTAL CONDITIONS
• PART III ASSESSING AND DESIGNING SYSTEMS
• Index