Augmented and Mixed Reality for Communities

Joshua A. Fisher, Joshua A. Fisher

• Publisher: CRC Press
• Year: 2021
• ISBN:
  9781000378474
• Edition: 1
• Topic: Informatik
• Subtopic: Human-Computer-Interaktion

Using mixed and augmented reality in communities is an emerging media practice that is reshaping how we interact with our cities and neighbors. From the politics of city hall to crosswalks and playgrounds, mixed and augmented reality will offer a diverse range of new ways to interact with our communities. In 2016, apps for augmented reality politics began to appear in app stores. Similarly, the blockbuster success of Pokémon Go illustrated how even forgotten street corners can become a magical space for play. In 2019, a court case in Milwaukee, Wisconsin, extended first amendment rights to augmented reality.

For all the good that these emerging media provide, there will and have been consequences. Augmented and Mixed Reality for Communities will help students and practitioners navigate the ethical design and development of these kinds of experiences to transform their cities. As one of the first books of its kind, each chapter in the book prepares readers to contribute to the Augmented City. By providing insight into how these emerging media work, the book seeks to democratize the augmented and mixed reality space.

Authors within this volume represent some of the leading scholars and practitioners working in the augmented and mixed reality space for civic media, cultural heritage, civic games, ethical design, and social justice. Readers will find practical insights for the design and development to create their own compelling experiences. Teachers will find that the text provides in-depth, critical analyses for thought-provoking classroom discussions.

Part 1
The Body in the XR Community

Chapter 1
Against the Instrumentalization of Empathy: Immersive Technologies and Social Change

Rebecca Rouse

University of Skövde, Kanikegränd 3a, 541 34 Skövde, Sweden

Email: [email protected]

Introduction: Technology as Solution

“It’s a machine, but inside of it, it feels like real life, it feels like truth.”

–Chris Milk (2015)

“A continual retreat from the discomfort of authentic racial engagement results in a perpetual cycle that works to hold racism in place.”

–Robin DiAngelo (2011) p. 66

Technologies have a long history of being sought after as panaceas for complex human problems. Recently, I had a front row seat to this type of thinking at the previous university where I taught. In the midst of a discussion on the development of a new AR/VR lab on campus, the Dean of Engineering asked me, “Could you develop a VR game to teach our faculty diversity and inclusion?” The context for his question was increasingly visible activism from Black student groups on our campus in response to systemic racism, hostile faculty and administrators, and a lack of structural support. It was an open secret that the predominantly white campus lacked in faculty training resources, and that many faculty held racist and other deeply prejudicial views that were expressed in their teaching and interactions with students, negatively impacting the retention rates of students, faculty, and staff of color. While the administration had begun a small effort to provide occasional optional training, the Dean was clearly reacting to the great need with his suggestion to develop a VR game to ‘solve the problem.’ As an out lesbian (albeit white and cisgendered) I was one of the few ‘diverse’ games faculty, and the only one actively involved in campus diversity and inclusion work, and therefore an obvious candidate in the Dean’s mind to do this work and design a VR game to enlighten recalcitrant faculty members. After recovering from my initial shock at the question, I did my best to describe why I felt the project would not succeed in the way he seemed to imagine.

My colleague’s incredible faith in technology has good company. Think, for example, of the work of immersive journalists and documentary filmmakers such as Chris Milk, Nonny de la Peña, the Be Another research collective, and others. Milk famously described VR as an “empathy machine” in his 2015 TED Talk, stating that by using VR “[…] we become more compassionate, we become more empathetic, and we become more connected. And ultimately we become more human” (Milk, 2015). Milk’s claim—which may seem glib at first—has been astutely critiqued by many (Clune, 2016; Murray, 2020; Nakamura, 2020). It is worth further examination, however, because it is emblematic of a pervasive perspective on technology today often encountered in the technology industry, STEM disciplines, and even popular culture at large. The seeming simplicity of this perspective belies the many layers of assumption underneath, many of which I believe hamper designers in achieving meaningful impact with their works. In this chapter the layers of assumption in the empathy machine perspective are examined, and an alternative way of structuring the human-technology relationships that occur in the design process is suggested.

First, let us examine the tacit assumptions that underlie statements about VR as an empathy machine. There are two main parts to this larger claim. One part is a claim about what empathy is, and how it can influence people and society. A second claim centers on the nature of interactive, immersive media as particularly conducive to fostering this specific type of empathy in viewers. What is meant by empathy in the usage from Milk and other journalists and documentary makers working in immersive media today should be made explicit. Empathy in this case is used to denote a positive outcome in the viewers, due to change in attitude or belief, with a likelihood that this change may result in pro-social behavior and even actions advancing justice. It is worth examining how this colloquial understanding of empathy holds up against scholarship on the topic, which will be discussed below. More deeply embedded in this definition is an implied mental model of how social change works, and so the question also arises as to whether not this empathy-based model of social change is accurate. At the foundation of this set of assumptions is an idea about the ontology of technology itself, as a labor saving device that is capable of simplifying complexity.

Viewed from this perspective, it becomes easy to see why this idea of the ‘empathy machine’ could be so attractive to many across the technology industries, STEM disciplines, and popular culture in the West. The reasoning may go something like this: if only we can use AR/VR to unlearn prejudice and inspire action, then the hard, painful work of the emotional and intellectual labor of coming to terms with prejudicial beliefs and attitudes could be made easier. There is a connection to be made here with Robin DiAngelo’s concept of white fragility as discussed in her original paper (2011). DiAngelo developed the concept of white fragility to understand and name the defensiveness, fear, misdirected anger, and inaction often displayed by white people when confronted with their participation and complicity in perpetuating oppressive racist systems and these systems’ devastating costs to others. The idea of using a technology, such as AR or VR, to ‘change minds’ via empathy (understood as an almost involuntary, emotional response) plays into a fantasy that neatly aligns with a privileged positionality, seeking quick, easy, and relatively painless methods of mitigation that fall far short of actual change.

This perspective on technology as labor-saving device fits well with dominant so-called common-sense but wrongheaded ideas about technologies as neutral tools that can take on the burden of labor from humans and increase efficiency, notably critiqued by Langdon Winner (1980), Susan Leigh Star (1999) and others. Other methods for prejudice reduction, such as in-person intergroup dialogue training, require many resources (mostly trained practitioners and teachers, who must be paid) and quite a lot of time—even years. The technological fantasy of the immersive media empathy machine imagines a mass medium that can seamlessly reach scores of individuals, necessitate no actual interpersonal contact with the ‘others’ these individuals seek to empathize with, and smoothly and easily change minds in minutes. Unfortunately, these foundational ontologies of both technology and social change are at odds with key concepts in scholarly research on social change, and the media artifacts produced within this imaginary are more likely to serve to assuage the privileged person’s guilt and enrich the artifact’s creators (through money or social capital, or both) rather than effect true social transformation. This ironic moral pitfall is discussed in depth by Kate Nash (Nash 2018), in her examination of the difficulties of designing an effective experience in VR for witnessing the suffering of others.

De la Peña’s 2012 Hunger in Los Angeles provides just one example of the many immersive media projects that function in this manner and fall prey to what Nash describes (Nash, 2018). In De la Peña’s 2012 piece, a spatial 3D rendering created in the Unity game engine displays a reenactment of people standing in line at a food bank in L.A., using actual audio recorded from the real food bank line. The situation becomes increasingly stressed as one man succumbs to a diabetic seizure and collapses on the sidewalk. The VR interactor, however, is prevented from taking any action let alone an ethical one to help the man and can only continue as a passive voyeur in the scene. But, the question is opened, even if the interactor could make choices to help the sick man or the many hungry people, how meaningful would that be? Ultimately, the work presents a simulation with little functional connection to reality, transforming the very real suffering of people struggling with poverty and disease into a simulated spectacle.

In an interview with Lizzy Goodman (2012) de la Peña discusses interactors’ reactions to the piece as emotionally intense, citing interactors’ tears and aggrieved facial expressions following the experience. Remarkably, de la Peña expresses her complete surprise that interactors would want to help the stricken man, stating “It’s shocking to me the number of people who were so upset that they couldn’t help this guy.” This statement reveals far more about de la Peña’s understanding of people than the reactions of the interactors who experienced the VR piece. It is also important to note that throughout the interview de la Peña refers to the sick man as “this guy” and “some guy,” but never by name, which suggests she does not know the man and made no effort to involve him in the design of the work, or seek his consent. In terms of the value of the VR work, the article mentions it was prominently exhibited at the prestigious Sundance film festival, and de la Peña discusses the need to “think about experience as part of your overall business plan […] The idea of commodifying experience is not new but this is like commodifying emotion” (Goodman, 2012). No positive impacts for the actual people struggling with hunger and illness are mentioned, and so it seems the largest positive impact of the work was for de la Peña herself. As discussed by Joshua Fisher (2017) it is all too common in the case of this genre of VR work that the interactor ends up empathizing most closely with the VR designer, and not the subject (Fisher, 2017). This brings to mind the many examples of mediated witnessing that lack authentic or meaningful engagement with the person suffering but bring value to the designer, such as photographer Dorothea Lange’s cursory interaction with and ultimate exploration of Florence Owens Thompson, better known as ‘Migrant Mother’ (1936).

Empathy, Ethics, and Social Change

“What [white Americans] see is a disastrous, continuing, present condition which menaces them, and for which they bear an inescapable responsibility. But since, in the main, they appear to lack the energy to change this condition, they would rather not be reminded of it.”

–James Baldwin (1966) p. 173

What exactly is empathy? Is it an emotion, a media-induced response, a personality-based positioning, or something else? There is a wealth of scholarship on empathy and related phenomena such as compassion, identification, and transference. Steve Larocco’s research provides an important perspective on the ethical complexity of empathy, which is relevant to our discussion given the focus on social change. Larocco understands empathy as a type of positioning of the self towards the other, as opposed to an emotion or feeling (Larocco, 2018). Larocco underscores the uncertainty around the potential of this empathic positioning, as there are many possibilities along a spectrum, all the way from authentic identification with another to selective empathy that seeks to misconstrue the other as similar to the self, or identifies only with aspects of the other perceived as similar to the self. Larocco points out that due to this wide range of empathic responses, it is a mistake to draw a causal relationship between empathy and compassion.

While empathy is commonly thought of as a positive, warm emotional response, it can also be directed and instrumentalized or used as a tool in a predatory manner. A fictional example that nicely illustrates this usage of empathy is the interrogation scene in Star Wars: The Force Awakens (2015). The villainous Kylo Ren seeks to understand resistance fighter Rey in order to probe for weaknesses in her psyche so that he may more easily defeat her. This capacity to weaponize empathy has been important in th...