Contemporary Issues in Human Factors and Aviation Safety

Helen C. Muir, Don Harris, Helen C. Muir, Don Harris

• Publisher: Routledge
• Year: 2017
• ISBN:
  9781351949187
• Edition: 1
• Topic: Tecnologia e ingegneria
• Subtopic: Aviazione

Every issue of Ashgate's Human Factors and Aerospace Safety: An International Journal publishes an invited, critical review of a key area from a widely-respected researcher. To celebrate a successful first three years of the journal and to make these papers available to a wider audience, they have been collated here into a single volume. The book is divided into three sections, with articles addressing safety issues in flight deck design, aviation operations and training, and air traffic management. These articles describe the state of current research within a practical context and present a potential future research agenda. Contemporary Issues in Human Factors and Aviation Safety will appeal to both professionals and researchers in aviation and associated industries who are interested in learning more about current issues in flight safety.

Section Two

Operations and Training

6 A review of the benefits of aviation human factors training

Graham D. Edkins

This paper was first published in Human Factors and Aerospace Safety 2(3), 2002, pp. 201–216

Abstract

This paper reviews the available evidence for the benefits of aviation human factors training. Despite the proliferation of human factors training programmes across the aviation industry since the 1980s there are few published studies that demonstrate positive shifts in attitude or behaviour following the introduction of such training. Those studies reporting benefits suffer from a number of methodological weaknesses including the failure to use control groups, lack of longitudinal evaluation and small sample sizes. Of significant concern is a lack of cost effectiveness data demonstrating a return on investment from human factors training. Recommendations for future research are made in the light of consolidating existing evidence on the commercial benefits of human factors training.

Introduction

Over the past twenty years, there has been an increasing recognition that human factors training can have a significant impact on safety, particularly within high-risk domains such as transport, mining and energy. The aviation industry has been at the forefront of this trend. Human error has been consistently identified as contributing to the majority of air crashes since the late 1970s (FAA, 1995; Johnson, 2000a; Salas, Prince, Bowers, Stout, Oser and Cannon-Bowers, 1999b). In most cases, these human errors occurred despite the operators’ technical competence. Highly trained people committed what seemed to be trivial errors. Usually these errors involved factors such as breakdown in communications, poor decision-making and failures in leadership (Helmreich, 1996) and in some cases resulted in tragic loss of human life, as well as huge financial and public relations costs for those organisations involved.

To minimise human error, the aviation industry introduced a series of cockpit resource management training programmes in the early 1980s. These resource management programmes have typically raised awareness about human factors concepts, such as communication, decision-making and teamwork, within the flight crew working environment. Airlines throughout the world have progressed from cockpit orientated training programmes to Crew Resource Management (CRM) and Advanced Crew Resource Management (ACRM). Subsequently, these programmes have expanded into a plethora of different forms across a wide variety of industries. For example, the air traffic control domain has adapted airline CRM training into a set of Team Resource Management programmes (Andersen and Bove, 2000). Outside aviation, CRM-style training has been applied within the nuclear industry (Harrington and Kelso, 1991), the medical fraternity (Davies and Helmreich, 1996; Fisher, Phillips, and Mather, 2000; Howard, Gaba, Fish, Yang and Sarnquist, 1992), for maritime crews (Andersen, Soerensen, Weber and Soerensen, 1996; Bydorf, 1998), and in the offshore oil industry (Flin, 1995; Flin and O’Connor, 2001). Variants of CRM have also been used to enhance decision-making and team skills training for emergency services personnel. Similarly, team based safety behaviour training, which is analogous to human factors training, has been incorporated into industries such as construction (e.g. Raya, Bishop, and Qi, 1997). Regardless of industry application, these human factors training programmes have similar foci on safety and human behaviour.

Despite the widespread use of human factors training programmes, there is a lack of longitudinal studies examining their effectiveness. For example, do human factors training programmes have a significant impact on error management? Is there a relationship between human factors training and improved safety performance? The answers to these questions are particularly important for organisations questioning whether such programmes are a sound investment in times when training budgets may be scarce.

The purpose of this paper is to critically review the current state of human factors training within the aviation industry with specific regard to: studies that claim a successful link between human factors training and improved safety performance; and programmes that appear to demonstrate a return on investment. In particular, attention is paid to those studies reporting benefits from human factors training application in non-flight crew areas. It is hoped that this review may provide a précis of the status of human factors work on this subject as well as offer a useful resource for organisations to build a stronger human factors business case.

An empirical analysis of human factors training

While human factors training is becoming established across many industries, it remains most widespread in aviation. Successive generations of training programmes have refined the concepts and added to their scope. However, from the outset, doubts have been expressed about the validity of programmes. Initial doubters described the courses as ‘charm school’ and irrelevant to aviation (Helmreich, 1996). Since then, programme content has matured and the training methods have become increasingly sophisticated, but the doubters persist. Steps to overcome these doubters’ resistance have usually been fruitless. But do the doubters have the evidence on their side?

At first glance, the best method to evaluate human factors training programmes would appear to be simple: rates of accidents. This approach seems logical given that the first programmes were introduced to reduce the incidence of human error in accident rates. Yet despite twenty years of human factors training, human error continues to be involved in the majority of aviation accidents. Johnson (2000a) recently found that while CRM style courses have been used in many airlines, there has not been any noteworthy reduction in rates of incidents and accidents resulting from crew teamwork and communication failures.

However, using accident rates to evaluate human factors training is futile, due to significant changes over this period in the way aviation accidents are analysed. Aircraft safety occurrences are now investigated in much greater detail, with ‘multiple causality’ now typically identifying a chain of events that leads to an accident (ICAO, 1992; Reason, 1992). Detailed systemic investigations almost inevitably lead to the identification of human error at one or more points along the chain. Such detailed analysis allows greater understanding of the causes of accidents, but it also means that the rate of accidents classified as ‘human error’ has increased.

With the increasing sophistication of investigation techniques, accident rates cannot be used as a broad method of evaluating training success. In addition, because accident rates in commercial aviation are already low, the infrequency of events makes any detection in improvement difficult even with identical accident investigation techniques (Helmreich, Chidester, Foushee, Gregorich and Wilhelm, 1990; Salas et al., 1999a). Accident rates could only provide valid comparisons if pre-training and post-training accident rates are available, and if the methods of analysing cause and the error classification system are identical. Without such data, human factors training has to be evaluated in other ways.

However, there are examples of when human factors training has been cited as preventing accidents. For example, potential disasters with two United Airlines flights³ were averted and attributed to successful training in communication and teamwork (Salas, Rhodenizer, and Bowers, 2000). While such incidents are suggestive of benefits, anecdotal evidence is difficult to evaluate scientifically. Without rigorous controls, it cannot be determined if human factors training prevented the occurrence of accidents in these or other cases.

A review of the literature reveals that there are two approaches to evaluating the effectiveness of human factors training programmes; attitudinal and behavioural.

Attitudinal based methods of evaluating human factors training effectiveness

In the early development of human factors training, emphasis was often placed on attitudes. This was typically true in both training and in evaluation. In the aviation industry, formative courses sought to modify trainee’s attitudes toward teamwork and other human performance issues. The effectiveness of these programmes was usually evaluated by measuring employee attitudes after training (Helmreich, 1999). This approach has been typical of human factors programmes in other industries as well. For example, air medical programmes have been evaluated by questionnaires, which aimed to measure individuals’ attitudes towards team awareness and effective communication (Fisher et al., 2000). Similarly, crisis resource management training in medical operating rooms has been assessed by questionnaires, where participants where asked to rate themselves on their performance (Holzman, Cooper, Gaba, Philip, Small and Feinstein, 1995).

This reliance on attitude measurement does not provide sufficient evidence for the effectiveness of human factors training. Surveying attitudes is relevant to some parts of programme evaluation, such as judging employee reactions and detecting potential morale problems. However, when evaluating the benefits of a training programme, attitudes are an unsuitable measure, for two reasons. Firstly, it is a debatable assumption that attitudes have a direct effect on behaviour. It is possible that trainees have simply learned the right thing to say and are responding appropriately. Even if trainees have absorbed the concepts being taught, attitudes give no indication of whether they know how to apply what they have learned. Secondly, and more importantly, attitudinal assessment is not appropriate for an objective evaluation of what an individual has learned. When psychologists wish to test a person’s intelligence, they do not give them a questionnaire asking them how intelligent they are. Instead, they use a set of performance items, which are compared to objective and standardised normative data points. Such behavioural methods of evaluation are applicable for human factors training programmes.

Despite such flaws, many human factors training programmes continue to rely on attitudes as a prime measure of programme effectiveness. However, some sections of the aviation industry are shifting to behavioural methods of training and evaluation (see Flin and Martin, 2001 for a review). This illustrates another significant problem; namely that human factors training programmes have not developed as a unified field with standardised methods. Rather, the area can be characterised as a diverse range of programmes and concepts developed by separate airlines and government agencies (Salas et al., 1999b).

As a result, any evaluation of the effectiveness of human factors training needs to consider the diversity in the field, including the impact of different organisational cultures, the operating environments and the types of programme conducted. Studies that claim to demonstrate the apparent failure or success of human factors training often need to distinguish which form or type of programme is being used. All forms of human factors training are not created equal and without further probing of the type of courses participants underwent, a well-intentioned evaluation may not provide substantive evidence against the field as a whole.

The use of behaviour based evaluation criteria

In recent years, some human factors training programmes have started to develop more rigorous criteria for training and evaluation. Behavioural competencies have begun to replace training for attitudes and awareness (e.g. van Amermaete and Krujisen, 1998). The development of behavioural competencies allows trainees to learn specific skills, which can be employed in work settings. Furthermore, the evaluation of specific competencies allows greater accuracy in assessing those areas in which individual trainees perform well, and those areas they need to improve. While behavioural human factors training is not universally applied, its acceptance appears to be spreading.

In contrast to the initial development of CRM style programmes, characterised by their wide variety, different assumptions, different training methods, and a lack of common content, the recent growth of behavioural human factors training has been accompanied by a shift toward standardised programmes. The involvement of regulatory bodies such as the Joint Aviation Authority (JAA) has led to increased standardisation of training methods and course content.

In the United States, the Federal Aviation Administration (FAA) has sponsored the development of Advanced Crew Resource Management (ACRM). This system is not a single package for all airlines, but a basis for individual airlines do develop their own programmes. Significantly, the ACRM programme includes skill-based, behavioural measures of crew performance, as well as standardisation of trainers.

In Europe, the Joint Aviation Authority has produced the NOTECHS system, an amalgamation of existing methods to measure non-technical skills. In this system, non-technical skills are defined as ‘attitudes and behaviours in the cockpit not directly related to aircraft control, system management, or standard operating procedures’ (van Avermaete and Krujisen, 1998). Non-technical skills are divided into four categories: cooperation, leadership and/or management skills, situation awareness, and decision-making. The NOTECHS system also includes five principles, which are intended to provide objective assessment. The first requirement is that only observable behaviour is assessed. Secondly, for behaviour to be rated unacceptable, it is a requirement that there be a threat to flight safety. The third requirement is that unacceptable behaviour must be repeated during a check to determine if there is a substantive problem. Fourthly, each behaviour must be rated as either acceptable or unacceptable. Finally, an explanation is required for each unacceptable rating.

While the shift from attitude based evaluation to a focus on behavioural training and assessment of flight crew skills is promising (Flin and Martin, 2001), human factors training is now being increasingly applied to other areas of aviation such as maintenance (e.g. Johnson, 2000b; Stelly and Poehlmann, 2000). With the broader application of human factors programmes, it has become even more imperative to evaluate the evidence for t...