Multimodal Semiotics and Rhetoric in Videogames

Jason Hawreliak

• Publisher: Routledge
• Year: 2018
• eBook ISBN: 9781351659710
• Edition: 1
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

This book merges recent trends in game studies and multimodal studies to explore the relationship between the interaction between videogames' different modes and the ways in which they inform meaning for both players and designers. The volume begins by laying the foundation for integrating the two disciplines, drawing upon social semiotic and discourse analytic traditions to examine their relationship with meaning in videogames. The book uses a wide range of games as examples to demonstrate the medium's various forms of expression at work, including audio, visual, textual, haptic, and procedural modes, with a particular focus on the procedural form, which emphasizes processes and causal relationships, to better showcase its link with meaning-making. The second half of the book engages in a discussion of different multimodal configurations and user generated content to show how they contribute to the negotiation of meaning in the player experience, including their role in constructing and perpetuating persuasive messages and in driving interesting and unique player decisions in gameplay. Making the case for the benefits of multimodal approaches to game studies, this volume is key reading for students and researchers in multimodal studies, game studies, rhetoric, semiotics, and discourse analysis.

Part I
Setting the Stage

1
Play, Videogames, and the Problem of Meaning

We all play occasionally, and we all know what playing feels like. But when it comes to making theoretical statements about what play is, we fall into silliness.

—Brian Sutton-Smith

Players… interact with videogame content in highly individualised, multi-linear ways which are never exactly the same from one person to the next.

—Astrid Ensslin

As noted in the introduction, a primary motivation of this book is to outline a methodological approach that will be useful for scholars, practitioners, and anyone else interested in understanding how meaning is ultimately negotiated between player and (video)game at a particular moment in time. This is, after all, a primary area of concern of game studies as an academic discipline in the Humanities: What does X mean in this particular video-game and how might it help us understand each other and the world around us? The challenge is not insubstantial. Indeed, although videogames as a medium are hardly alone in resisting simplistic, easily digestible interpretations, they nevertheless possess certain characteristics which make semiotic and rhetorical analysis particularly challenging. As Ian Bogost succinctly puts it, “videogames are a mess” (2009). Bogost does not use the term ‘mess’ in a pejorative sense, but instead, borrows John Law’s usage which sees messiness as “a methodological concern, one that resists creating neat little piles of coherent analysis” (qtd. in Bogost 2009). Videogames, regardless of genre, length, or platform, are complex artifacts that are inevitably imbued with meaning potentials from seemingly countless sources both within and outside of the text. In this chapter I will briefly outline some of the chief culprits that make interpreting games particularly messy and then offer some ideas as to how principles of multimodality might help us address these challenges.

Play and Games

The first problem encountered when approaching meaning in videogames is that it is difficult to identify a satisfactory definition of ‘play.’ Every year I ask my undergraduate students to think about the nature of play and to identify a definition that is

1. Unique to play and play alone; and
2. Capable of including all the myriad forms of play.

Despite our best efforts we are yet to find success. Criterion one is necessary for ensuring that a definition of play cannot be so broad that it can be applied to other concepts, as doing so limits its usefulness as a definition. Criterion two serves the opposite function, namely, that a definition of play must be robust enough to account for the varied nature of activities we typically think of as play. Both criteria pose problems for identifying a satisfactory definition.

For example, in their highly influential book, Rules of Play, Katie Salen and Eric Zimmerman define play as “free movement within a more rigid structure” (2004, 373). Play certainly does operate in this way. If I am playing tag with friends, I can move left, right, backward, or forward, move quickly or slowly, tag someone on the shoulder or the leg, but all this can only occur within the confines of the playground and within the rules of the game. So it is a fine definition and we all understand what is meant here, but it is quite broad for the purposes of definition. The same definition also applies to many things that are not play, such as driving a car, cooking, social interactions, or school assignments.

On the opposite side of the spectrum, some definitions of play are too restrictive. In what I will call the ‘classical’ conception of play, Johan Huizinga (1936, 1955) and later Roger Caillos (1961) both describe play in roughly the same manner, namely, that it is a free and voluntary activity done for its own sake, which temporarily separates the player or players from the ‘real world’ spatially, temporally, and psychologically. For Huizinga, play as a conceptual entity resides “within an artificial mental sphere governed by rules of its own and temporarily encompassing all conduct in a voluntarily accepted system of action. A conventional proceeding takes the place of the direct pursuit of utility or pleasure” (1936, 176). In the classical definition, play is ‘pure,’ and its purity is tainted by extrinsic motivations such as money, politics, or grades. Professional sports, for instance, are anathema to this definition of play since it converts or reduces play to the worldliness of industry. The problem with this definition is that someone could conceivably play under coercion, such as playing basketball in gym class as a child, or play a game while being paid for it. In both cases there is no reason why the individual under coercion may not also be playing. People play for all sorts of reasons, and the Platonic ideal set forth in the classical definition is too restrictive to be acceptable as an all-encompassing definition.

Brian Sutton-Smith persuasively argues that part of the problem in finding a satisfactory definition for play lies in the sheer diversity of play and that however one decides to define it is probably governed by their disciplinary biases. We use the term ‘play’ to refer to a number of seemingly disparate entities, such as the act of participating in a game, a theatrical performance, the clever use of language like a ‘play on words,’ and so on. Merriam-Webster lists 21 different definitions or uses of the word and it is not exhaustive. As Sutton-Smith observes, “The diversity of play is well illustrated by the varied kinds of play that are to be found within the larger menagerie of the ‘play’ sphere. Almost anything can allow play to occur within its boundaries” (2001, 3). As soon as there are boundaries (rules), the possibility of play opens up within them. In his book Play Anything: The Pleasure of Limits, the Uses of Boredom, and the Secret of Games, Ian Bogost writes that while play may often be associated with freedom and the care-free, it is precisely rules and limitations which create opportunities for play. Ultimately, Bogost argues that play is “the operation of structures constrained by limitations.… Play is a way of operating a constrained system in a gratifying way” (2017, x–xi). Every system, of course, is constrained in some way. With all its many forms and variations, one is hard pressed to find any activity or concept that does not possess at least the potential for playfulness.

To bring some order to the chaos, one framework I have found useful in the contexts of both game studies and game design is Roger Caillois’ four categories of play as outlined in the 1961 book, Man, Play and Games. The four categories he proposes are agon (competition), alea (chance), mimicry (role-play), and ilinx (vertigo). Competitive forms of play include sports, chess, most videogames, and so on; the object of competitive games is to demonstrate one’s skill and superiority over others. Games of chance, on the other hand, have less to do with the player’s skill and more about luck; roulette, dice, playing the lottery, and so on all fall into this category. Mimicry is about ‘playing-as’ something and includes all forms of role-playing from deeply complex and codified table-top role-playing games (TTRPGs) like Dungeons and Dragons to cosplaying at a convention to children playing out scenes from their favorite television show. Unlike the first two genres of play, there is not necessarily a win condition in mimicry; it has more to do with the pleasure of playing as someone or something else. Finally—and it is an oddball—there is ilinx, which is the pleasure one feels from temporarily disrupting their perceptual and vestibular systems. Children spinning around until they get dizzy just for the fun of it, riding a roller-coaster, or skydiving are all examples of ilinx.

Clearly there are many instances where these categories overlap. Poker, blackjack, or Monopoly, for instance, all require luck—of cards or dice— but they are also competitive games and players can use strategy to increase their chances for winning. Likewise, a TTRPG like Dungeons and Dragons has players role-play as a character (mimicry), roll dice for attacking and other actions (alea), and although it is largely a collaborative game there may be unofficial competitive elements, like who can craft the most ridiculous character (agon). These categories are useful from an analytical and heuristic perspective because each one has its own expectations, possibilities, and constraints. From the perspective of game design, they are also useful because they each provide their own types of experiences, challenges, and emotional resonances. The joy of winning a game of oneon-one basketball, for instance, is very different from the joy of winning a game of chance like roulette, even if no actual money is at stake. At a high level, one of the first questions game developers often ask themselves is, What kind of experience do I want to deliver? and Caillois’ four categories of play are a fine starting point for answering that question.

Caillois helps bring some order to the issue of play, but it is clearly not enough. Perhaps the most satisfying conceptualization of play comes from the philosopher Bernard Suits, who argues that, among other things, what is essential to playfulness is a ‘lusory attitude’ (1973/2005). The lusory attitude can be described as a psychological or attitudinal approach to an activity. To use one of Suits’ examples, why would someone willingly experience the frustration of trying to hit a tiny ball into a slightly larger hole with pieces of metal or wood (i.e. golf) when the same result could be achieved much easier by simply picking up the ball and placing it into the hole? For Suits, the answer is attitudinal: Individuals make a decision to accept certain conditions and constraints (i.e. rules) for the purposes of playing. In Suits’ words, the lusory attitude provides “an explanation of that curious state of affairs wherein one adopts rules which require one to employ worse rather than better means for reaching an end” (2005, 52). One implication here is that the quality of playfulness does not rest in any object or activity but in the mind of the individual engaging in the activity. In this view, two people could conceivably engage in the same activity—say, golf—but one could be playing and the other doing something else entirely (e.g. working) depending on the attitudinal approach of each.

The concept ‘game’ is a little more solid than ‘play’ but not by much. Drawing on (and truncating) Jesper Juul’s (2005) ‘classic’ definition, games are often conceived as a particular form of play with a number of fairly typical (though not absolute) criteria. Games typically have

• Rules which dictate what is and is not permitted;
• Victory and fail conditions;
• Unpredictable outcomes.

In other words, players need to know what is required of them to win or lose a game, and the outcome of the game cannot be predetermined. Similarly, Salen and Zimmerman define ‘game’ as “a system in which players engage in an artificial conflict, defined by rules, that results in a quantifiable outcome” (2004, 5). There are many other definitions of games that resemble this and they often fall into what is known as the ‘formalist’ approach to game ontology (Yang 2013; Keogh 2014; Lantz 2015). In this view while it is accepted that there are many different forms of games, a necessary requirement is that they offer quantifiable outcomes (win, lose or tie) which are negotiated by navigating clear, predefined rule-sets. Furthermore, rules are granted ontological and representational priority when compared to other components of a game—like art or sound—which are seen as secondary (Juul 2005). While the formalist approach is relatively easy to grasp and may even offer a useful design heuristic, there are some significant problems with this approach.

Brendan Keogh notes that formalism is ultimately reductionist and that “formalist approaches that insist that certain elements of game design are more fundamental than others marginalize those videogames that do not meet certain arbitrary criteria” (2014). So while formalism priori-tizes rules and quantifiable outcomes it excludes narrative based games or games without explicit win or fail conditions. This ontological gatekeeping is more than pedantry; it has deleterious effects on both game design as a discipline and, more importantly, on marginalized groups who have traditionally been left out of gaming culture (Gray 2014; Vossen 2018). As game designer and scholar Anna Anthropy observes,

Limiting the creation of games to a small, exclusive group leads not only to creative stagnation, but also to the alienation of anyone outside that group… [Historically, games have been] designed by a small, male-dominated culture and marketed to a small, male-dominated audience, which in turn produces the next small, male-dominated generation of game designers. It’s a bubble, and it largely produces work that has no meaning to those outside that bubble.

(2012, 12–3)

While the landscape is arguably getting better, with marginalized peoples appearing in—and more importantly—making games now more than ever, gaming culture is still tremendously noxious, particularly to these same marginalized groups (Cross 2014; Vossen 2018).

Exclusionary definitions of ‘game’ thus run the risk of further silencing and oppressing already marginalized voices. By saying ‘X is not a game because of Y,’ we exclude games and experiences that mainstream gaming as an industry and discipline sorely needs. As Robert Yang notes in his discussion of formalist approaches to games,

That line of inquiry has been a long favored tool of well-intentioned oppression, because these arguments often masquerade as thoughtful discourse but function as a weapon of de-legitimization, that argue these personal games can’t really fit a formal definition of game. The emotional leap is that these people can’t really fit a formal definition of people. Adding, ‘it’s okay if it’s not a game’ comes off as sounding like, ‘it’s okay if you’re not a person.’

(2013, para. 3)

The barrier to entry for creating games has been significantly reduced in the past decade with user-friendly engines like GameMaker, Construct, Twine (for interactive narrative) and the proliferation of online tutorials for robust, free to use engines like Unity and Unreal. This means that the typical demographic of gamers and gam...

Table of contents

• Cover
• Half Title
• Title
• Copyright
• Dedication
• Contents
• List of Figures
• List of Tables
• Acknowledgments
• Introduction: Deepening the Conversation Between Multimodal Studies and Game Studies
• PART I Setting the Stage
• PART II Modal Configurations
• PART III Players, Mods, and Future Directions
• Index