The Aesthetics of Virtual Reality

Grant Tavinor

• Publisher: Routledge
• Year: 2021
• ISBN:
  9781000452129
• Edition: 1
• Topic: Philosophy
• Subtopic: Aesthetics in Philosophy

This is the first book to present an aesthetics of virtual reality media. It situates virtual reality media in terms of the philosophy of the arts, comparing them to more familiar media such as painting, film and photography.

When philosophers have approached virtual reality, they have almost always done so through the lens of metaphysics, asking questions about the reality of virtual items and worlds, about the value of such things, and indeed, about how they may reshape our understanding of the "real" world. Grant Tavinor finds that approach to be fundamentally mistaken, and that to really account for virtual reality, we must focus on the medium and its uses, and not the hypothetical and speculative instances that are typically the focus of earlier works. He also argues that much of the cultural and metaphysical hype around virtual reality is undeserved. But this does not mean that virtual reality is illusory or uninteresting; on the contrary, it is significant for the altogether different reason that it overturns much of our understanding of how representational media can function and what we can use them to achieve.

The Aesthetics of Virtual Reality will be of interest to scholars and advanced students working in aesthetics, philosophy of art, philosophy of technology, metaphysics, and game studies.

1 The Virtual Turn

DOI: 10.4324/9781003107644-1

1.1 An Introduction to Virtual Reality

Once I had everything set up, with the numerous cords and connections properly plugged in and a space cleared in front of me so I would not trip over, I turned on the console, entered the disc, and put on the headset.

Oh, this is what virtual reality is like.

Of course, I knew what virtual reality was in an abstract sense—it presents visual scenes that give the impression that you occupy an alternative place—but the first-hand experience felt a bit like a revelation. It really does feel like I am within this place. And then I turned my head to discover that the space also existed behind me. To discover that I could reveal the world by simply turning my head to see my surroundings, was another surprise. I could not stop looking around, taking pleasure in inspecting the apparent illusion that I occupied another place. And this was just the menu screen for the demo disc! Virtual reality (VR) gets a lot of hype, but some of it surely is deserved. Putting on the headset for the first time, the sensory impression of being in another space is immediate and undeniable.

Soon afterwards I received a copy of the survival horror game Resident Evil VII as a sort of sadistic gift from my brother. Resident Evil is a series of videogames in which the player finds themselves trapped and helpless in some mysterious and threatening scenario. There are zombies, oversized rabid dogs, that kind of thing. I had played Resident Evil games and others from its genre previously, but while challenging and frustrating, I never found them particularly scary. In such cases it is all too easy to look away from the screen and distance yourself from the horror. I found the VR version of Resident Evil to be an altogether different story. The game begins with the player entering an apparently abandoned and dark country house, slowly searching through the house to solve a mystery. But even the first hallway was utterly terrifying for me to experience. I stood in the entrance for a long time wondering what was ahead. I did not want to walk down the hallway knowing that I would have to open a door at its end, and who knew what was behind that door?!

As a result, my progress in the game was incredibly slow, because each time I entered a new area I would stop and carefully look and listen for threats in the environment. Every time I inspected an object in the world, I worried that because my attention was on the item some threat in the house would sneak up and do me some horrid evil. I also did a lot of stopping and looking over my shoulder to make sure I was not being followed. I felt vulnerable. When I finally did meet the inhabitants of the house, the involuntary terror that overtook me in that moment was so complete that I did something I cannot remember ever doing while playing a game before: I literally screamed. This must have all seemed hilarious to my wife who observed from the other side of the room, but she was also now quite reluctant to enter the virtual house herself. After that I had to take off the headset for a little while. It was a terrifying experience; I can thoroughly recommend it.

Virtual reality has been with us for at least 40 years, both as a preoccupation of technologists and futurists, and by existing as a viable though cumbersome and creaky technology usually designed for games or technical applications. Virtual reality has also had an undeniable influence on popular culture, particularly as a subject of films such as Tron (1982) or The Matrix (1999) and the Holodeck on the TV show Star Trek: The Next Generation (1987–1994). But it is only very recently that the medium itself has become widely available for home use. Several virtual reality products are now commercially available, including dedicated VR headsets from PlayStation 4, Valve, HTC and the virtual reality forerunner Oculus, and a number of mobile headsets that use smartphones as their screen, such as Samsung Gear VR. There are signs, perhaps, that virtual reality is not the fleeting novelty it could have been, and that many critics predicted it would be. And there are now many popular games that utilize the technology. No Man’s Sky (2016) for example, has an often-beautiful virtual reality mode that allows players to explore galaxies of procedurally generated planets. The venerable videogame series Half-Life has received a VR sequel in the form of Half-Life: Alyx (2020). Because of its recent successes, it is increasingly common to hear that “Virtual reality is finally here.” This may be premature, because there are still deep challenges to the medium and its adoption by a truly mass audience, but the practical realization of the concept already seems before us.

These events are, foremost, a media development. What are the key features of this advancement in media? I noted above that the basic understanding of virtual reality is that it “presents visual scenes as though you occupy a place within them,” but in technical terms, what does this involve? Leaving aside a lot of detail and theory for the following chapters, in the most general terms, PlayStation VR and other commercial virtual reality systems comprise three key elements: the depiction of a sensory environment; a means of tracking and depicting the user’s apparent position within this environment; and, finally, a means of providing for user interaction within this virtually depicted space. These, as I will refer to the phenomenon here, are the principal elements of virtual reality media (or VR media for short).

The depiction of the VR environment is predominantly visual and is most frequently achieved via a stereoscopic headset or head mounted display (HMD). For example, the PlayStation VR headset includes a single small organic light-emitting diode panel that sits in the headset close to the user’s eyes. Two small lenses are placed before the screen, magnifying and softening the images, allowing for a wider field of vision on the depicted scene, and reducing the visual prominence of the surface of the pixel array. The images that are depicted on the panel are produced in such a way—and this is where the sophisticated rendering algorithms of the software play a role—that the apparent displacement of the two images, combined with the binocularity of the lenses, mimics our natural visual situation in the real world. The resulting visual environment—such as a darkened room within a decaying house—gives a very strong visual impression that the viewer is situated within that environment.

While the visual elements of VR media predominate, the depiction of the sensory environment is not restricted to this visual modality, and at a minimum VR media usually involve stereophonic, or even 3D virtual audio, to place the user within an aural space. The spatial effect is achieved in a formally similar manner to the stereoscopic depiction of the visual environment: the spatial cues of native hearing—the displacement of the two ears and the brain’s ability to use the resulting difference in timing and intensity of the received sounds to identify the spatial location of sound sources, that is, the ability of sound localization—is utilized in VR to mimic the acoustic spatiality of natural hearing.

Sometimes a means of haptic or kinesthetic representation is also used to convey a sensory engagement with the virtual environment; the senses in question being touch, proprioception and the spatial sense provided by the vestibular system. Haptic gloves, which provide tactile feedback when you touch or grasp objects in the virtual environment, are now being developed by several companies. But despite these developments, these modes of sensory representation remain less common and quite limited compared to the visual and auditory modes introduced above.

This apparent experiential vantage point is not passive or fixed, as VR systems typically allow for the movement of the user to be tracked and for this movement to be represented in the user’s apparent experiential orientation on the virtual environment. In PlayStation VR, this tracking partly involves small light emitting diodes (LEDs) fixed to the exterior of the headset which are captured by a camera placed in front of the user. The relative position of the LEDs is used to calculate the orientation of the user’s head so that head movement can be replicated in the depictive viewpoint displayed by the stereoscopic headset. The HTC Vive goes further and allows for the user’s body movements to be tracked and depicted within a 15-foot radius. Other VR systems such as Oculus and Magic Leap have developed eye tracking technology so that the experiential effects of eye movement—particularly on focus in the visual field—might be replicated. For each of these means of tracking to be effective, a low latency between the tracking and the display is crucial for creating a realistic impression, and this has been a significant technical hurdle in the design of effective VR systems.

While many VR media applications comprise “experiences” where the user’s agency is limited to changing their orientation on the depicted virtual spaces, most applications, including all VR games, allow the user the ability to interact with objects within the virtual space. So, for example, in the opening scene of Half-Life: Alyx, players find themselves on a balcony looking over a dystopian city dominated by a vast Citadel, built by the Combine, the antagonist force of the game series. Combine craft move menacingly through the air and stalk the streets of City 17. On the balcony, the player can interact with various objects depicted in the world of the game: a radio can be tuned to various stations, cans of soda can be picked up, crushed, and thrown (perhaps at a pigeon perched on the balcony rail). In VR, such control is typically achieved either through standard gaming controllers or purpose-built peripherals, but again the position and orientation of these devices needs to be tracked by the VR system so that the user’s movements and interactions can be realized in the virtual space.

Together, these media elements may give the users of VR a compelling sense of inhabiting and interacting with alternative spaces. And these alternative experiential spaces are somewhat more diverse than is frequently acknowledged. These media may give rise to the fear and trepidation that I felt while playing Resident Evil VII, but they may liberate us to explore fantasy spaces such as in No Man’s Sky where the predominating feeling need not be one of vulnerability, but instead one of curiosity. VR media also allow us to explore and learn about the landscapes and cities of our actual world as in Google Earth VR, which utilizes the mapping technology with which we are now so familiar, to present navigable VR depictions of our world’s geography. We may even step into historical spaces to experience ancient cities such as in the independent developer Steven Lou’s VR Rome (2018). VR may also free us from our human-sized spatial constraints so that we can experience the very large and the very small, including the VR experience Discovering Space (2017) which allows users to explore the solar system, and the virtual microscopy VR app arivis VisionVR, in which users can manipulate and section microscopic items such as cells. And indeed, VR media may allow for the disruption of our conception of space, as in Superliminal (2020), a VR game that employs forced perspective and other visual devices to create challenging puzzles. Finally, VR may affect the spaces in which art takes place, as in the VR painting application Tilt Brush, which can be used to create paintings that because of their three-dimensional appearance seem like a kind of virtual sculpture.

1.2 The Aesthetics of Virtual Reality Media

These virtual reality media are a single aspect of a widespread technocultural trend of recent times. So many of the artefacts, activities, institutions, and relationships which in previous times we experienced in the actual world, we now encounter often, if not principally, in a virtual way. You might, like me, spend your leisure time playing virtual reality games or exploring virtual worlds. Virtual stores, a novelty just twenty-five years ago, are now ubiquitous features in our lives, and some of us will do most of our shopping through such online means. You may now interact with many of your “friends” in a principally virtual way by posting on social media or “liking” their posts and activities. Some of these virtual friends you may not have even encountered face to face in the actual world. Our economies are now in a significant part distributed in digital worlds, distant from the facts of labor, products, and services, with many items of trade seeming hardly tangible at all. Our politics has also become increasingly virtualized, in that our leaders may communicate with us through virtual means, utilizing the same social media we use to chat with our friends, to communicate with or influence their followers. We might refer to this trend as the virtual turn.

While acknowledging that my focus is part of a broader set of technocultural changes, and that a basic examination of the concept of virtuality needs to be given, I will for the most part avoid discussion of this wider context. Rather, I will address one specific aspect of the virtual turn: the way that virtual reality media allow their users the impression of experiencing environments other than those in which they actually exist. Moreover, my interest will be in the philosophical and aesthetic questions raised by such virtual media. The kinds of questions I ask, and issues I discuss, will derive from my disciplinary orientation as a philosopher of the arts.

The philosophy of the arts, or what is also frequently called philosophical aesthetics, is an active and robust subdiscipline of philosophy that enquires into the nature of the arts and aesthetic experience. In the last fifty years in particular, the discipline has produced important work on a wide range of topics and has dramatically reconfigured the philosophical understanding of issues such as pictorial representation, fiction and the imagination, the relationship between art and emotion, artistic ontology, the value of art, and the aesthetic appreciation of everyday practices and the natural environment. This body of theory and philosophical practice easily lends itself to the current issue. VR is a genuinely fascinating media development, and one that is of obvious interest to philosophers generally, and philosophers of the arts specifically. But while the academic and scientific literature on VR is large and ever growing, and there is also a reasonable amount of philosophy on virtual reality, there has been very little attention paid to the medium from within the field of philosophical aesthetics. Perhaps this lack of attention is partly explained by the very newness of VR; but also, I think, because aestheticians are not yet convinced about the relevance of this technological development to their typical concerns. I hope to change this.

Because this is principally an investigation borne out of the philosophy of the arts, the theoretical orientation of this book and the literature that I draw on to explore the topic will be somewhat different to most other accounts of virtual reality. Virtual reality will here be considered as a medium to be understood alongside of painting, film, photography, as a means of configuring representations of real and imaginary spaces, as conveying a sense of spatial experience itself, and as allowing for distinctive aesthetic and artistic practices. Thus, I will situate VR in terms of several key issues in the philosophy of the arts that relate to such media. I will be asking about the representational or depictive nature of VR; the historical and conceptual relationship of VR media to paintings, photographs, and other forms of picturing; the apparent realism of VR media and how it might relate to other “illusionistic” works such as trompe l’oeil paintings; the striking affective experiences that VR media conveys or evokes; and how the interactivity inherent in VR may reconfigure our experience of art.

It is worth introducing just a few of these issues at the outset of this study to give the reader a sense of how the consideration of VR media might affect long-standing philosophical issues. Pictorial realism was a significant issue in aesthetics in the 20^th Century, often centering around whether the technique of linear perspective, developed in Florence in the 15^th Century, really counted as more realistic than alternative ways of depicting space. Much has been written either defending pictorial realism or subjecting it to criticism. Adopting a favorable stance, Ernst Gombrich (1960) contended that linear perspective comprised an incremental refinement of depictive naturalism by employing the process of “schemata and correction” to approximate natural experience and perception. But there was also a lot of doubt about the naturalism of the technique, with Nelson Goodman contending that for a picture utilizing the technique of linear perspective to be seen as reproducing the geometry of the scene depicted, “the picture must be viewed through a peephole, face on, from a certain distance, with one eye closed and the other motionless” (Goodman, 1976: 12). For Goodman linear perspective was an artificial and moreover conventional means of spatial depiction, and he concluded that “the behavior of light sanctions neither our usual nor any other way of rendering space” (1976: 19). But virtual reality media loosen Goodman’s constraints on the viewer, who may now, with both eyes open, move freely towards and around pictured objects in a way that, at first blush, gives a great impression of our natural experience of a visual scene. In this way VR seem to revitalize the potential for pictorial realism.

VR may also alter our emotional connection to the arts, and our capacity to be emotionally affected by what we find depicted within them. In traditional artistic media such as painting or film, the apparent objects, people, and events of our appreciation seem separated from us, existing in another space that almost never registers our presence as a viewer or allows us to influence what occurs within that space. But VR media give precisely the appearance that its users occupy the same space as the objects or events they perceive. The persisting issue of the reality or fictionality of the emotions we have for fictions—for example, feeling for Anna Karenina’s plight or fearing a green slime slithering toward you “through” a cinema screen (Radford, 1975; Walton, 1978)—should surely be reconsidered given the development of a medium in which you have the seeming potential to occupy the same space as the objects of these emotions. A green slime might be worrisome on the cinema screen, but what if it is in the same room, slithering not just towards the screen, but towards you? I would imagine it could be utterly terrifying!

And finally, the ideas of interaction and interactivity have been the topic of a great deal of philosophical...