Formal Papers
Using high fidelity simulator for aircraft inspection training: development and evaluation
Anand K. Gramopadhye*, Brian J. Melloy, Stacey Chen, Xiaochun Jiang**, Mohammad T. Khasawneh***, Sittichai Kaewkuekool and Shannon R. Bowling
Abstract Research in aircraft inspection and maintenance has revealed the criticality of human inspection performance in improving aviation safety. Training has been identified as the primary intervention strategy in improving the quality and reliability of aircraft inspection performance. If training is to be successful, it is clear that we need to provide aircraft inspectors with tools to help enhance their inspection skills and improve performance. In response to this need, a high fidelity inspection-training simulator was developed: (ASSIST: Automated System of Self-Instruction for Specialized Training) - a specialised inspection training software focused on improving aircraft inspection performance. Following the development a detailed performance and usability evaluation was conducted and is reported as a part of this paper.
Introduction
The aircraft inspection/maintenance system is a complex one (Drury, Prabhu and Gramopadhye, 1990; Drury, 1991) with many interrelated human and machine components. One of the major factors contributing to this complexity is the ageing fleet. Scheduled repairs to an older fleet account for only 30% of all maintenance compared to the 60-80% for a newer one. This difference can be attributed to the increase in the number of age-related defects (Drury, 1991). In such an environment the importance of inspection cannot be overemphasised. It is critical that these visual inspections be performed effectively, efficiently, and consistently over time. Moreover, because 90% of all inspection in aircraft maintenance is visual in nature and is conducted by inspectors, inspector reliability is fundamental to an effective maintenance system.
Since it is difficult to eliminate errors altogether, continuing emphasis must be placed on developing interventions to make inspection and maintenance more reliable and/or more error tolerant. Training has been identified as the primary intervention strategy in improving inspection performance. If training is to be successful, we need to provide inspectors with training tools to help enhance their inspection skills. Existing training for inspectors in the aircraft maintenance environment tends to be mostly on-the-job training (OJT). However, this method may not be the best one (FAA, 1991; Gordon, 1994) because feedback may be infrequent, unmethodical, and/or delayed. Moreover, in certain instances feedback is economically prohibitive or impractical because of the nature of the task. Because the benefits of feedback in training have been well documented (e.g. Weiner, 1975), and for other reasons as well, alternatives to OJT are sought.
More importantly, training for improving the visual inspection skills of aircraft inspectors is generally lacking at aircraft repair centres and maintenance facilities even though the application of training knowledge to enhance visual inspection skills has been well documented in the manufacturing industry where training has been shown to improve the performance of both novice and experienced inspectors (Weiner, 1975; Drury and Gramopadhye, 1990). Visual inspection skills can be taught effectively using representative photographic images showing a wide range of conditions with immediate feedback on the trainee's decision (Weiner, 1975; Blackmon and Gramopadhye, 1996), a combination of training methods that has also been shown to be superior to OJT alone (Latorella, Gramopadhye, Prabhu, Drury, Smith and Shanahan, 1992). A case study presented by Gramopadhye, Bhagwat, Kimbler and Greenstein (1998) showing how photographic images and feedback were used to develop a computer-based training program for a contact lens inspection task supports the findings of the Latorella et al. (1992).
The success of off-line training/retraining with feedback suggests that this method can play a role in aircraft inspection training. One of the most viable approaches for delivering training, given the many constraints and requirements imposed by the aircraft maintenance environment, is computer-based training. Computer-based training offers several advantages over traditional training approaches: it is more efficient white at the same time facilitating standardisation, and supporting distance learning. One recent example is the Aircraft Maintenance Team Training (AMTT) Program that is specifically designed to teach aircraft maintenance technicians basic team skills (Gramopadhye, Krause, Pradeep and Jebaraj, 1995) using a multimedia approach with interaction opportunities between the user and the computer. With computer technology becoming cheaper, the future will bring an increased application of advanced technology to training.
In the domain of visual inspection, the earliest efforts to use computers for offline inspection training were reported by Czaja and Drury (1981). They used keyboard characters to develop a computer simulation of a visual inspection task. Similar simulations have also been used by other researchers to study inspection performance in a laboratory setting (e.g. McKernan, 1989). Since these early efforts, Latorella et al. (1992) and Gramopadhye, Drury and Sharit (1994) have used low fidelity inspection simulators using computer-generated images to develop off-line inspection training programs for inspection tasks. Similarly, Drury and Chi (1995) studied human performance using a high fidelity computer simulation of a printed circuit board inspection. Another domain, which has seen the application of advanced technology, is that of inspection of x-rays for medical practice (Kundel, 1990).
However, most of the work in the application of advanced technology to inspection training has focused on developing low fidelity simulators for running controlled studies in a laboratory environment. Thus, research efforts need to be extended in order to take full advantage of today's computer technology. Moreover, advanced technology has found limited application for inspection training in the aircraft maintenance environment. Presently, most of the applications of computer technology to training have been restricted to the defence industry for complex diagnostic tasks. It is only recently that we have seen the use of high fidelity simulation (e.g. virtual reality) for aircraft maintenance and inspection (http://www.aitram.de/; Vora et al., 2002). The message is clear: we need more examples of the application of advanced technology to training for industrial inspection tasks that draw upon the principles of training which we already know will work. In this vein, this paper describes a university/industry collaborative research effort to develop an off-line computer based inspection training system for aircraft inspectors. The specific objectives of this research were as follows: develop, validate, and conduct usability and performance evaluations of the aircraft inspection training system. The paper describes a generic methodology for developing and evaluating a computer-based training system, which can be used by practitioners in the field. As an example, the authors describe how this generic methodology was used, and the evaluation efforts related to the computer-based inspection training program entitled 'Automated System of Self Instruction for Specialized Training' (ASSIST). The results of the usability and performance analyses conducted following the development of the system are also outlined.
Training
Training methods
Patrick (1992) has identified training content, training methods and the trainee as the important components of the training program. Drury (1992) includes the training delivery system as another component. Training methods that have been used effectively for inspection training (Drury and Gramopadhye, 1990; Gramopadhye, et al., 1997) are described below.
Pre-training Pre-training provides the trainee with information concerning the objectives and scope of the training program. During pre-training, pre-tests can be used to measure the level at which trainees enter the program and the cognitive or perceptual abilities that can be used later to gauge the training performance/progress. Advanced organisers or overviews, which give the trainee an introduction to the program and facilitate the assimilation of new material fulfil the elaboration theory of instruction which proposes that training should be imparted in a top-down manner with the general level being taught before the specifics.
Feedback A trainee needs rapid, accurate feedback in order to know whether a defect was classified correctly or a search pattern was effective. Gramopadhye, et al. (1997) classifies feedback as either performance or process feedback. Performance feedback typically consists of information on search times, search errors and decision errors. Process feedback, on the other hand, informs the trainee about the search process, such as areas missed. Feedback with knowledge of results coupled with some attempt at performing the task provides a universal method of improving task performance (Wiener, 1975) which can be applied to learning facts, concepts, procedures, problem solving, cognitive strategies and motor skills. A training programme should start with rapid feedback, which should then be gradually delayed until the ...