Cockpit Displays: Test and Evaluation

Richard L. Newman, Kevin W. Greeley

• Publisher: Routledge
• Year: 2017
• eBook ISBN: 9781351950831
• Topic: Technik & Maschinenbau
• Subtopic: Maschinenbau

Cockpit Displays is an in-depth examination of the design rationales, test philosophy and test procedures for cockpit systems. Whilst its main emphasis is on cockpit displays, it also includes an important discussion of flight management systems and mission computers. Areas covered include: the cockpit design process, test techniques for flight displays and equipment, and situation awareness testing. Comparing civil and military requirements, it is an important analysis of the lessons learned from test and evaluation and will be of interest to cockpit systems design engineering staff at major airframe manufacturers, procurement executives and program managers at military aircraft program offices and flight test engineers and test pilots.

1   Introduction

The Need for Integrating Design and Test and Evaluation

There have been a number of papers and articles written about operational difficulties with modern display and other cockpit systems.* As we see it, the problem has been a series of discontinuities between the users and the designers, between the designers and the testers, and between the users and the testers.

As a result of the first discontinuity, between users and designers, inadequate design requirements are established. This is particularly unfortunate as systems can be (and are being) designed with greater and greater capabilities in terms of automatic flight and guidance and flight control. Without adequate requirements, it is hardly surprising that there are problems encountered in operational use.

The second discontinuity, between designers and testers, reduces the opportunities for feedback to the designer. In fact, with today’s economic setting, many systems are practically committed to production by the time they reach flight test. Only if there is very serious problems, will these systems be corrected.

The third discontinuity, between users and testers, results in inadequate test criteria. As a result of this discontinuity, we are left with highly subjective criteria which vary from tester to tester. Or we have inappropriate criteria.

Essential Features of a Test and Evaluation Philosophy

The essential features of our test and evaluation (T&E) philosophy are easy to state.

Feedback to the design team

Valid and objective test completion criteria

The ultimate tester is the user

We are testing displays, not pilots

Feedback to the Design

There should be prompt and early feedback to the design team. The development process should allow for early proof-of-concept designs with early test feedback. This is the only way we will obtain improved systems which meet the needs of the user.

Objective Test Completion Criteria

There should be valid and objective test completion criteria. The use of subjective criteria should be avoided. We feel that objective test completion criteria makes the ultimate procurement specification. The objective criteria must be based on performance measurements. The final tests (and to some extent the important tests) are mission scenarios and situation awareness tests. All tests to that point are for risk reduction or proof-of-concept.

This is important enough to repeat… The test criteria should be based on objective performance measures.

Ultimate Testers

The ultimate tester is the user. A frequent question is “Test pilots or operational pilots?” We strongly believe that the objective performance measures mentioned above must be achieved by operational pilots — the users. For this reason, we feel that test pilots should not act as evaluators during mission and situation awareness testing.

Final mission testing should use customer pilots. Test pilots should certainly participate in this testing, but as test directors, not as evaluators. If the test pilot community has concerns with aspects of a cockpit system, they need to design test scenarios to address these concerns.

Testing Displays, not Pilots

We are testing displays, not pilots. Finally, the purpose behind this book it articulating better ways of evaluating displays and related cockpit systems. While some of the methods and techniques discussed can be used for psychological testing of human subjects, the primary purpose of the discussion is how best to ensure that our displays and cockpit systems aid in the performance by the using pilot — at the sharp end of the airplane.

Statement of the Problem

Put to its simplest form, the problem is lack of requirements in many design efforts, lack of test requirements, and minimal feedback to the designer.

__________________________

Note

*These operational difficulties will be covered later in Chapter 2 (page 4) and Chapter 7 (page 53)

2 Lessons Learned

Cockpit Automation

The most common question in modern cockpits is reported to be "What’s it doing now?” There are anecdotal reports of pilots turning the automated system off when air traffic control (ATC) changes the landing runway because it’s easier to do without the “help” from the automation than to reprogram it.*

Cockpit automation has come under fire recently because of a series of operational incidents.(1–2)† Mode confusion was reported to be a serious problem by the NTSB. Increasing mode complexity is prompting one operator to modify its current flight management systems (FMSs) to reduce the number of operating modes.

Recently, a study by the Australian Bureau of Air Safety Investigation published a report (3) on a survey of Pacific rim pilots flying FMS-equipped airplanes. Among the troubling results are the numbers of design induced errors (mode selection and transposition of heading-select and course-select functions) and the large number of “work-arounds” necessary to manipulate the FMS. These are most often reported during descents. It was not clear if these were the result of incompatible ATC procedures or of poor software design.

The problem seems to be one of pilots frequently suffering from a loss of situation awareness (LOSA) because of the increasing complexity of FMS modes. A recent review article was titled “How in the World Did We Ever Get into That Mode?”(4) This article, by Sarter and Woods, suggests that current cockpit automation makes it more important and more difficult for pilots to remain aware of the status of the system’s different modes of operation. They cite research where pilots made critical errors during non-standard situations, such as leaving autothrottles engaged during aborted takeoffs.

The likelihood of mode errors increases when the operating rules change from mode to mode.(5) This would be true of the mode changes are not obvious or well-annunciated. Sarter and Woods cite the FCS in a modern fly-by-wire (FBW) transport which has a two vertical modes which differ in their speed control methods. Of these, the non-standard OPEN DESCENT mode can be entered inadvertently and was thought to have been a factor in an approach accident.(6)

Billings (7) observes that present automation reduces the workload during normal operations but increases it during abnormal operations. He argues that the systems should be designed to reduce workload and error during abnormal or emergency operations. The normal operational case is well within pilot capabilities.

Billings (8) also suggests that cockpit automation should adequately convey status information to the flight crew and the systems should be accountable, comprehensible, and informative.

Some electronic attitude indicators remove unneeded information, such as mode annunciation, during extreme attitudes — a process designed to enhance the ability of the pilot to use the display for recovery without distraction. This removes mode awareness from the pilot and was thought to have contributed to the accident to the Airbus A-330 at Toulouse.*

Cockpit Displays

Symbol Choices

With any electronic aircraft display, head-up, head-down, or helmetmounted, there are two divergent forces. On the one hand, there is a great clamor for standardization of symbology. At the same time, there is an extraordinary desire to make every aircraft application different. Any student of head-up display (HUD) history will testify to this.

Electronic displays can be developed in almost any format. In spite of this, they have often mimicked existing conventional panel instruments. Similarly, HUD symbology often mimics head-down displays. This has resulted in confusion over control te...

Table of contents

• Cover Page
• Halftitle
• Dedication
• Title Page
• Copyright
• Table of Contents
• List of Figures
• List of Tables
• Acknowledgments
• Abbreviations
• 1 Introduction
• 2 Lessons Learned
• 3 A Review of Cockpit Design Guides
• 4 The Cockpit Design Process
• 5 Requirements
• 6 Test Sequence Leading to Airworthiness
• 7 Situation Awareness
• 8 Flight Test
• 9 Test Techniques for Flight Displays and Equipment
• 10 Initial and Collateral Tests
• 11 Rotary-Wing Mission Task Elements
• 12 Fixed-Wing Mission Task Elements
• 13 Mission or Operational Test and Evaluation
• 14 Situation Awareness Testing
• 15 Strawman Test Schedules
• 16 Summary
• Definitions
• References
• Index