Flight Simulation

Virtual Environments in Aviation

Alfred T. Lee

• Publisher: Routledge
• Year: 2017
• ISBN:
  9781351936378
• Edition: 1
• Topic: Technology & Engineering
• Subtopic: Aviation
• Index: Technology & Engineering

Advances in computer, visual display, motion and force cueing and other technologies in the past two decades have had a dramatic effect on the design and use of simulation technology in aviation and other fields. The effective use of technology in training, safety investigation, engineering and scientific research requires an understanding of its capabilities and limitations. As the technology has as its primary goal the creation of virtual environments for human users, knowledge of human sensory, perceptual, and cognitive functioning is also needed. This book provides a review and analysis of the relevant engineering and science supporting the design and use of advanced flight simulation technologies. It includes chapters reviewing key simulation areas such as visual scene, motion, and sound simulation and a chapter analyzing the role of recreating the pilot's task environment in the overall effectiveness of simulators. The design and use of flight simulation are addressed in chapters on the effectiveness of flight simulators in training and on the role of physical and psychological fidelity in simulator design. The problems inherent in the ground-based simulation of flight are also reviewed as are promising developments in flight simulation technology and the important role flight simulators play in advanced aviation research. The readership includes: flight simulation engineers and designers, human factors researchers and practitioners, aviation safety investigators, flight training management and instructors, training and instructional technologists, virtual environment design community, and regulatory authorities.

Chapter 1
Visual Scene Simulation

Introduction

The simulation of the visual scene that presents itself through the window of an aircraft has been one of the most significant technical challenges in the development of flight simulation technology. It is, in fact, one of the elements of flight simulators that did not begin to see to major progress until the 1980s and the onset of rapid growth in computer processing power. Prior to that decade, highly detailed visual scene simulation was largely delivered by the scanning of cameras over detailed terrain models.¹ Early attempts at scene simulation by computer-generation resulted in sparsely detailed images, which were of very low resolution. In the decades since the explosive growth of computer technology began, the ability to display real-time imagery over a large field-of-view (FOV) has become commonplace in modem flight simulators. This chapter explores the technology of visual scene simulation and its role in flight simulation as well problems in the design and implementation of the technology.

Aircraft Control by Visual Reference

Flying an aircraft² by use of the visual scene provided out of the cockpit window can be divided into three basic parts: flight control, navigation, and collision avoidance. Positive flight control can be further divided into attitude, speed, and altitude control. Control of aircraft attitude for the pilot is in three basic dimensions: pitch, roll and yaw. A simplified example of these three control axes in relation to the visual scene is seen in Figure 1.1. An aircraft pitches up and down when rotated about its y or lateral axis. Pitch attitude cues are derived from movement of the visual scene elements vertically up and down the aircraft wind screen. The most important visual scene element to the pilot using the visual scene for attitude control is the visual horizon. Even the most sparsely detailed visual scene can provide the necessary cues to control of aircraft pitch if a visible horizon is available. Indeed, provided one knows which part of the scene is sky and which part is ground, a visual scene devoid of all details but the visual horizon is adequate for aircraft pitch control.

Figure 1.1 Visual Cues to Pitch (a), Roll (b), and Yaw (c)

Aircraft control in the roll axis is also dependent, to a large extent, on the existence of a visible horizon. Roll or rotation about the aircraft's longitudinal or x axis rotates the visual horizon around the point where the aircraft is headed. The angle of the visual horizon is the aircraft's angle of bank and the rate of rotation is the aircraft's rate of roll. Early simulator's with primitive visual simulation systems often provided only the visual horizon represented by a line segregating two halves of white (for the sky) and black or gray (for the earth). This was adequate for providing limited pitch and roll control provided the aircraft's pitch attitude was not so extreme as to completely void the display of the visible horizon. In these situations, if no artificial visual horizon was available, loss of control was likely Gust as it is in the aircraft). The third axis of aircraft control which uses external visual reference, control of the yaw or z-axis, can be achieved only by reference to an object or reference point which can be displaced horizontally, left or right, across the windscreen. A simple line segregating earth and sky is therefore not adequate to provide feedback for control in the lateral axis. The image needs to include an object or other detail that will move in conjunction with aircraft yaw or rotation around the z-axis. Translational motions along the z-axis also result in horizontal displacements, though such motion is minimal in fixed wing aircraft (but may be much greater in other aircraft such as helicopters). Without these additional visual scene details, early visual simulations were extremely limited in the usefulness of the visual information they could provide and, therefore, in the use of the simulator for training aircraft flight control by means of visual reference.

Increased object detail in visual scene simulation is essential when an aircraft operates at altitudes at less than a thousand feet above ground level. At these heights, ground reference cues become increasingly important in the judgment of height and in the judgment of both rate of altitude change and the angle at which the aircraft is approaching the ground. Two essential elements of visual scene simulation play a role under these conditions. The first of these elements is the display of individual objects in the visual scene and the second is the display of details within each of the displayed objects.

Object details such as size are essential in order to provide the pilot with valid and reliable cues to depth and distance and to the changes in depth and distance that support aircraft control, particularly control of airspeed and control of altitude when close to the ground. The importance of accurate object rendering is related to the basic human visual processing of object details. Objects in our world typically have a known size. These include natural terrain elements such as trees as well as cultural features such as highways and houses. Because the objects change in perceived (or retinal) size as the distance to them changes, the pilot interprets the difference in perceived size as a change in depth or distance. This is possible because an object's size is normally invariant and changes in its size can only be due to changes in the distance of the object from the pilot. This size-distance relationship of objects is a powerful visual cue for the interpretation of height above ground, for the distance from another aircraft, for the rate of change in altitude and for many other piloting tasks that occur close to the ground or to other airborne objects.

Other visual processes of the pilot are also supported by accurate rendering of object details. These include object detection as well as object identification and classification. Object detection is determined by the retinal or perceived size of the object, the relative contrast of the object against its background, and the position of the object in the pilot's visual field. Identification of an object occurs when sufficient object detail is available in order for the object to be classified in some way. Often this is a matter of the pilot's ability to perceive key classification characteristics of the object that allow identification to take place. For example, classifying an airborne object as an aircraft involves the ability to determine whether the object's shape and size corresponds to any known aircraft type. Further classification processes may include determination of the aircraft's aspect ratio, that is, the direction of another aircraft's flight path relative to one's own. These visual detection, identification and classification processes are critical to flight tasks such as airborne collision avoidance and, in military aircraft operations, to air combat maneuvering. Object detection and discrimination processes are also used for objects on the ground in those flight applications where ground-based target acquisition and classification are important. Discrimination of ground targets is an important tool in pilotage where navigation is dependent upon the identification of key ground objects such as roads, lakes, rivers, buildings, and other features which can be used in conjunction with maps and charts to identify key waypoints and visual approach points to airports and in locating controlled mrspace.

It should be evident from the above descriptions of piloting tasks that accurate rendering of object details in visual scenes may have a significant impact on the utility of flight simulators. Distorted rendering of objects in flight simulator visual displays or the absence of key image details can not only limit the usefulness of flight simulators but, perhaps more significantly, alter the way in which pilot's learn key flight skills.

Monocular Cues to Depth and Distance

All objects in a simulator visual scene are displayed on a two-dimensional surface. They therefore lack the normal binocular distance cues that are provided in real life, 3-dimensional (3-D) imagery. These distance cues are the product of binocular disparity effects resulting from the display of slightly different images to each of the two eyes. However, these binocular disparity cues are only significant at ranges below 2 m. With rare exceptions, aircraft do not operate this close to other objects either in the air or on the ground. For this reason, visual scene simulations in flight simulators will normally provide only monocular cues to depth and distance.

Linear Perspective

One of the most compelling monocular cues to distance is linear perspective. Simply put, linear perspective is produced by the visual merging or convergence of parallel lines as the points on the lines recede from the viewer. The more obvious examples of linear perspective cues for the pilot are provided by airport runways and taxiways. When a third dimension, altitude, is added, the linear perspective of the resulting object forms a powerful visual cue, particularly during the approach and landing phase. This cue varies in relation to the changes in an aircraft's approach angle and distance from the runway (see Figure 1.2).

Improper rendering of the runway object in a simulator visual scene can occur for a variety of reasons, including poor display resolution or inadequate vertical display field-of-view. In poor resolution displays, display aliasing can introduce artifacts into the visual scene such as foreshortening of the departure end of the runway so that the runway edges appear to join in the distance. Lack of vertical field-of-view can also cut off the runway end of long runways and thus fail to provide a rendition of the runway object shape as it would appear in the real visual scene.

Aerial Perspective

Typically, flight simulator image displays provide visual scenes that encompass great distances both horizontally and vertically. In real world operations, phenomena such as water particles, smoke and other atmospheric contaminants can dramatically alter what the pilot actually sees at these distances. Collectively, these phenomena create a visual effect called aerial perspective. As with linear perspective, aerial perspective provides a strong cue to distance and depth. This is because atmospheric particulates scatter and diffuse light emitted or reflected from an object. The scattering and diffusion of light increases as the distance of the observer from the object increases. Furthermore, the color (hue) of objects becomes increasingly desaturated or ‘washed out’ with increasing distances between the pilot and the object.

Figure 1.2 CGI Image of Runway on Final Approach

(Image Courtesy of Adacel, Inc.)

While aerial perspective cues will tend to correlate with the distance at which objects can be seen, the effect can be dramatically altered by the position and intensity of light sources. An example of aerial perspective in a CGI display system is shown in Figure 1.3. Note how the aerial perspective increases with distance in the image and eventually obscures the visible horizon.

Texturing and Texture Gradients

Other monocular cues to depth and distance are available to the pilot in real life. Among the most powerful of these is image texture or, more accurately, the texture gradient. A texture gradient is simply the change in density of texture elements as a function of distance from the viewer. As the density of texture increases as a direct function of distance from the viewer it provides the pilot with a cue to depth and distance. Objects in the distance which have a texture gradient in the foreground of the visual field will, therefore, always be perceived as further away than an object of the same size with less or no texture gradient in the foreground. Texture gradients are also useful cues to judging the relative slant or slope of terrain. Typically, flat terrain surfaces in, for example, forested areas will reveal the typical texture gradient characteristic as the distance from the observer. As the slope of terrain changes so will the texture gradient of the surface objects on the terrain.

Figure 1.3 Aerial Perspective in a CGI Display

(Image Courtesy of Adacel, Inc.)

Texture gradients may have their greatest utility in aiding a pilot during the landing flare. It is at this stage of the landing process where judgment regarding the aircraft's height above ground and the aircraft's rate of sink to the ground is most important. The landing flare is one of the most difficult tasks for student pilots to master. It is also a skill that can deteriorate rapidly if not refreshed either in a flight simulator or in an aircraft. Providing texturing of runway surfaces as it occurs in the real world visual scene is therefore essential to the successful training of the landing flare maneuver.

The absence of texture in a flight simulator display of an airport runway can lead to the development of compensatory skills. These skills are developed in flight simulators as a means of overcoming the limitations or improper design of a simulator. While many such skills may be harmless, others may inadvertently introduce safety problems into flight operations. In a study by Mulder, Pleisant, van der Vaart, and van Wieringen (2000), trainee pilot performance in the landing flare was compared with and without runway texturing in a flight simulator. Pilots in the non-texture condition developed the strategy of using runway edge markings as a means to judge height above the ground while pilot trainees in the texture condition used the runway's texture as a flare cue instead. For pilots in the non-texture condition, developing the strategy of controlling the landing flare by using the distance between runway edges would only be useful if all runways are of the same width. As runways vary widely in width in the real world, pilots in the non-textured display condition acquired a skill that will often be inappropriate in real world operations. When applied to real aircraft landings, these pilots may initiate the landing flare too early or too late. In this example, the absence of runway texture in the simulated visual scene required the pilot to develop a compensatory skill which could adversely affect aircraft safety.

Other Monocular Cues to Depth and Distance

A number of other monocular cues to depth and distance are available to the pilot. These include object overlap or occlusion, where objects nearer to the pilot overlap portions of objects further away. The occluded object will always be interpreted by the pilot as being further away than the occluding object. Object occlusi...