Who Understands Comics?

Questioning the Universality of Visual Language Comprehension

Neil Cohn

• Éditeur: Bloomsbury Academic
• Année: 2020
• ISBN:
  9781350156050
• Édition: 1
• Sujet: Literature
• Sous-sujet: Literary Criticism Theory

**Nominated for the 2021 Eisner Award for Best Academic/Scholarly Work** Drawings and sequential images are so pervasive in contemporary society that we may take their understanding for granted. But how transparent are they really, and how universally are they understood? Combining recent advances from linguistics, cognitive science, and clinical psychology, this book argues that visual narratives involve greater complexity and require a lot more decoding than widely thought. Although increasingly used beyond the sphere of entertainment as materials in humanitarian, educational, and experimental contexts, Neil Cohn demonstrates that their universal comprehension cannot be assumed. Instead, understanding a visual language requires a fluency that is contingent on exposure and practice with a graphic system. Bringing together a rich but scattered literature on how people comprehend, and learn to comprehend, a sequence of images, this book coalesces research from a diverse range of fields into a broader interdisciplinary view of visual narrative to ask: Who Understands Comics?

CHAPTER ONE

An Assumption of Universality

Sequential images are so pervasive in contemporary society that we may take their understanding for granted. Here, I broadly refer to sequences of images bound by meaningful connections, including the ubiquitous instruction manuals and signage. Visual narratives are a particular type of sequential image, often drawn, which typically convey meaning through a continuous event sequence, particularly to tell a story. Visual narratives often appear in contexts combined with writing, as in comics or picture stories, among other contexts. These latter types of visual narratives are among the first literature that children engage with, while reading storybooks with caregivers. All this makes sequences of images appear simple to understand, and this feeling of transparency has underlined a broad assumption that sequential images are universally comprehended.

Theorists of comics have followed these notions. In his treatise Understanding Comics, Scott McCloud stated that “Pictures are received information. We need no formal education to ‘get the message’” which contrasts with his statement that “writing is perceived information. It takes time and specialized knowledge to decode the abstract symbols of language” (McCloud 1993: 49). Echoes of these sentiments have carried over statements by scholars studying comics, who widely presume the transparency of their sequential images in contrast to language (e.g., Miodrag 2013; Szawerna 2017). Others have also stated without evidence that learning to understand or produce visual narratives is akin to other deliberately learned skills like playing an instrument, unlike the naturalistic acquisition of language (e.g., Kowalewski 2018).

Similar thinking motivates the growing advocacy for using comics in a range of communicative contexts. The past decades have seen a surge of advocates promoting the use of comics in educational settings (Cary 2004; Sousanis 2015). Recent work has also pushed for using comics for health (Green and Myers 2010) and science communication (Bach et al. 2017; Farinella 2018). Similar notions have underscored efforts to use visual narratives in humanitarian settings for decades (Cooper et al. 2016; Fussell and Haaland 1978; Stenchly et al. 2019). Indeed, recent partnerships by the United Nations, UNICEF, and NGOs aim to teach sustainable development using “the universal visual language and transformative power of comics to educate people in every corner of the globe” (comicsunitingnations.com).

These studies have advocated for the use of visual narratives like comics in education and communication, and indeed various empirical studies have suggested such benefits in these contexts (Mayer 2009; Nalu 2011; Wong et al. 2017). Nevertheless, many of the motivations behind such activism indicate tacit assumptions that sequences of images combined with text will be universally understood and thus confer an advantage to comprehension by sidestepping the need to decode the visual information while allowing it to supplement or aid the verbal.

Similar assumptions of universality pervade psychology studies. Researchers have used visual narratives in intelligence (IQ) testing and clinical assessments for decades (Kaufman and Lichtenberger 2006; Wechsler 1981), and sequential images are widely used as stimuli in experimental and anthropological research. These studies assume that visual narratives will be universally understood, including for children and/or illiterate populations, thus narrative sequential images have been used in tasks to explore a range of cognitive capacities. These include social intelligence (Campbell and McCord 1996), Theory of Mind (Baron-Cohen, Leslie, and Frith 1986; Sivaratnam et al. 2012), action planning and event sequencing (Tinaz et al. 2006; Tinaz, Schendan, and Stern 2008), sequential reasoning (Zampini et al. 2017), temporal cognition (Boroditsky, Gaby, and Levinson 2008), and discourse comprehension (Gernsbacher, Varner, and Faust 1990), among others.

1.1 Why Might They Be Universal?

There are several reasons sequential images might be presumed to be understood universally. We can start at the level of single pictures. Images are iconic—they look like what they represent (e.g., Peirce 1931)—and naïve beliefs about drawing hold that they represent what is seen either by vision or a mental image, not culturally constrained and learned schematic patterns (like language). If drawings represent what people see, they should be universal, since all people ostensibly have the same perceptual capacities. Differences in producing drawings thus reduce to a matter of “talent,” despite the assumed universality in understanding images (for review, see e.g., Arnheim 1978; Cohn 2014b; Willats 2005; Wilson 1988).

Similar assumptions extend to sequential image understanding: if comprehending events and actions in general should be universal, and images simply depict perception, images depicting events should also be transparent. This renders sequential image understanding as a matter of visual perception and event cognition (Loschky et al. 2020), which otherwise requires no learning, decoding, or specialized knowledge. Under this view, simply seeing and knowing events provides enough basis to understand sequences of drawn images.

Beliefs about the simplicity and universality of sequential images are no doubt reinforced by their ubiquity across cultures and history. Like most drawings, sequential images appear in diverse historical and cultural contexts recorded as far back as cave paintings (McCloud 1993; Petersen 2011). In contemporary industrialized societies, sophisticated visual narratives appear in comics, picture books, and storyboarding, in addition to sequential images in instruction manuals and signage. This ubiquity appears to span across human cultures and history without relying on appropriation from a particular origin—i.e., visual narratives do not seem to have a place they were “invented” and then spread across the world. Rather, the production of graphic images, including in sequence, appears to be a “universal” potential of human communication and cognition.

Thus, the idea that sequences of images are transparent and universally understood is widespread and pervasive. Though the cross-cultural ubiquity of sequential images supports them as a “universal” aspect of human communication, it does not mean that they are understood universally. Indeed, these ideas are merely an assumption, guided largely by the intuitions of adults with longstanding familiarity with visual narratives. As such, I will call this belief the Sequential Image Transparency Assumption (SITA).

How would we know whether sequences of images are universally understood? To show that sequential images are not universally transparent, we would need evidence where people could not construe their meaning, or have difficulty comprehending visual narratives. As it turns out, there are several examples where sequences of images cannot be construed. One example comes from cross-cultural cases where individuals had not been exposed to Western-style visual narratives. Another case is from children, who have difficulty understanding visual sequences while they are below certain ages (typically under the age of 4). In fact, comprehension of visual sequences varies even among people with different frequencies of reading comics and visual narratives. Also, individuals with various neurodiversities sometimes have challenges processing image sequences, including people with Autism Spectrum Disorder, Schizophrenia Spectrum Disorder, and Developmental Language Disorder.

All of these cases provide evidence that the SITA is not supported. This research has been dispersed across many different fields, including comics studies, anthropology, linguistics, art education, and including the subfields of developmental, cognitive, and clinical psychology. This wide dispersion implies two things. First, the diverse and multifaceted perspectives across these fields all recognize that visual narratives are an important aspect of human expression and communication. Yet, second, because they are so distributed, no clear field studying the comprehension of these materials has been consolidated. As a result, most of this research has remained unknown across disciplines, rendering the perception that such scholarship has not been undertaken.

This book seeks to rectify this widespread dispersion by synthesizing this literature on the understanding of sequential images, particularly visual narratives. Indeed, this wealth of research suggests that sequential images are not simplistic or universally transparent. Rather, the understanding of image sequences seems to require a fluency—i.e., a proficiency acquired through exposure to and practice with a system of visual narrative. Such fluency is comparable to the natural, extensive, and often passive exposure and practice required to comprehend language.¹

Although language is a cognitively “universal” and “innate” system in the sense that all typically developing human brains have the cognitive structures necessary to speak languages (Jackendoff 2002), language fluency is not developmentally inevitable and requires exposure to and practice with an external system. For spoken or signed languages, only in unfortunate circumstances do individuals fail to receive the requisite exposure and practice necessary to learn a language (Goldin-Meadow 2003). For visual narratives, a lack of drawing skill may be more widespread and viewed as culturally permissible, particularly since they are less integrated into everyday interactive communication (although, for a cross-cultural counterpoint, see Wilkins 1997/2016). However, this does not excuse visual narrative fluency from the same interaction between nature and nurture as fluency in language, despite different cultural assumptions and practices.

1.2 Visual Language Theory

This notion of fluency applied to sequential image understanding is in line with Visual Language Theory (VLT), a framework arguing that creating and understanding graphic images, particularly those in sequence, taps into analogous and/or overlapping structures and cognition as language (Cohn 2013b). A language is made up of a set of patterns in a person’s mind/brain. The patterns for an individual constitute their own idiolect, which is developed from the patterns they are exposed to and practice with across their lifespan. To the extent that a person’s idiolec...