International Perspectives on Digital Media and Early Literacy

The Impact of Digital Devices on Learning, Language Acquisition and Social Interaction

Katharina J. Rohlfing, Claudia Müller-Brauers, Katharina J. Rohlfing, Claudia Müller-Brauers

• Publisher: Routledge
• Year: 2020
• eBook ISBN: 9781000198492
• Edition: 1
• Topic: Education
• Subtopic: Education General

International Perspectives on Digital Media and Early Literacy evaluates the use and impact of digital devices for social interaction, language acquisition, and early literacy. It explores the role of interactive mediation as a tool for using digital media and provides empirical examples of best practice for digital media targeting language teaching and learning.

The book brings together a range of international contributions and discusses the increasing trend of digitalization as an additional resource in early childhood literacy. It provides a broad insight into current research on the potential of digital media in inclusive settings by integrating multiple perspectives from different scientific fields: (psycho)linguistics, cognitive science, language didactics, developmental psychology, technology development, and human–machine interaction. Drawing on a large body of research, it shows that crucial early experiences in communication and social learning are the basis for later academic skills. The book is structured to display children's first developmental steps in learning in interaction with digital media and highlight various domains of early digital media use in family, kindergarten, and primary schools.

This book will appeal to practitioners, academics, researchers, and students with an interest in early education, literacy education, digital education, the sociology of digital culture and social interaction, school reform, and teacher education.

Part 1
Learning and interaction with digital devices

Chapter 1
Promising interactive functions in digital storybooks for young children

Adriana G. Bus, Kathleen Roskos, and Karen Burstein

Introduction

When first introduced about 30 years ago, digital picture storybooks seemed so promising. Stories enlivened with animations, music, and sounds seemed to be an exciting development that opened up new possibilities for actively engaging children with stories. Digital storybooks, however, have yet to realize their promise, although the ubiquity of digital media virtually guarantees their growing presence in early childhood. This confronts us with a pressing question: Which digital book format immerses children in stories and benefits not only children’s meaning-making of narratives but also their sheer enjoyment of literature?

In this chapter, we address this question by sorting out the interactive affordances of digital book reading. We first look at interactivity in early first-generation digital books in which digital enhancements were enjoyable, but additions often missed the mark of supporting children’s comprehension of the story. We then describe some theoretical drivers of interactivity that may help explicate the greater efficacy of some techniques over others. Building on this work, we explore efforts to improve and advance interactive designs in digital storybooks that increase engagement and deepen children’s early literacy and literary experiences. We close the chapter with a long view of the potential benefits of digital books in children’s literacy development and a call for collaboration that will bring together book app creators/authors and researchers to build a stronger knowledge base around digital books in early childhood.

One aside before we start: Digital books are indeed new, innovative books. Hence, as has become current practice, it is risky to simply migrate or recycle paper books to interactive app formats – an effort akin to the horse designed by a committee that turns out to be a camel. Linda Labbo (personal communication) compares the new books with kitchen machines – shapeshifters that can slice, dice, or mash. Thus, there are a variety of possibilities that make the outcome uncertain and sometimes unrecognizable as a narrative experience.

Digital storybooks version 2.0

Interactivity in first-generation digital books

Early digital storybook designs were often additive, the objective being to add technology to the paper book, mostly by including user-activated visual and sound effects as mechanisms for engaging children. The user taps an object, animal, or person to elicit a sound (e.g., snore, laugh, scratch) or perform an action (e.g., fall, jump, laugh). In many cases, this interaction had no relevance to the story, except that it allowed the children to “interact” with the illustration (Zhao & Unsworth, 2017). In brief, the design goal was to include interactive functions that invite the child to explore these mechanisms as objects in the book. However, the risk is that performing these actions will lead to less exploration of narrative meanings. Those with computer expertise had a heavy hand in the early designs of these books, evidenced by their (naïve) belief that by making reading sessions more interactive, children would be more motivated – an assumption that probably guided their choices of which technology to add to books. First-generation digital storybooks such as Just Grandma and Me (Mayer, 1983) and P. B. Bear’s Birthday Party (Davis, 1994) are examples of this design approach. In Just Grandma and Me, the successive screens present the same defining features of paper picture books: on each screen, a section of text and an illustration. When opening a new screen, the text is read aloud automatically, and story events explained in the narration are animated. However, this does not occur simultaneously. After the oral narration finishes, we see what the narration explained – for instance, a boy building a sandcastle and a big wave destroying the castle. After the oral rendition and animation, the user can activate ten or more hotspots in the frozen screen and elicit visual and/or sound effects, almost all incongruent with the storyline: a blob in the sand turning into a fountain; a little beetle hurrying away through the sand; a prehistoric spear thrower in bearskin running by.

Although such interactive enhancements are designed to be motivating (Ricci & Beal, 2002), adding these to a multimedia presentation may distract children from the storyline and diminish meaning-making. Albeit novel and exciting, irrelevant information quickly overloads the human capacity to learn (Renninger, 1992). Outcomes of studies testing the effects of these early digital book designs on children’s narrative comprehension were indeed generally negative. Labbo and Kuhn’s (2000) case study of Arthur’s Teacher Trouble (Brown, 1994) concluded that children trying to integrate all available information in the narration – hence, also the incoherent gimmicks – fail to make meaning of the story. For example, they do not succeed in integrating the prehistoric spear thrower into the story of Little Critter on the beach with his grandma. Furthermore, quantitative studies support the adverse effects of cognitive overload on comprehension. Instead of immersing young children in stories, the designs seem to distract them from the storyline and result in lower scores on comprehension (Parish-Morris, Mahajan, Hirsh-Pasek, Golinkoff, & Collins, 2013). Bus, Sarı, and Takacs (2019) concluded that activating many hotspots unrelated to the storyline distorted meaning-making. Focusing on the number of pages on which the story that the child retold was similar to the original story, children’s score decreased by more than 25% when unrelated hotspots were added to the storyline. A meta-analysis of these and other early studies reaffirmed the design pitfalls of digital additions that introduce extraneous information. Nonetheless, there were not enough studies to make a clear-cut distinction between problematic and possibly promising interactivity in stories (Takacs, Swart, & Bus, 2015).

In brief, research on first-generation digital books yielded some valuable insights into the design of interactivity in digital storybooks. The interactions that digital books afford can work for or against story comprehension as a cornerstone of early literacy. Realizing the digital promise in New Age storybooks therefore requires a more fine-grained analysis of digital additions that promote literate thinking in storybook reading. A closer inspection of what works and why it works will allow us to identify which digital storytelling techniques support the design and development of quality digital storybooks in early childhood literature.

Theory and principles of interactivity in 2.0 digital books

Multimedia learning theories

Second-generation (version 2.0) digital books shift the design focus to digital additions with potential for supporting young children’s narrative comprehension skills (Bus, Takacs, & Kegel, 2015; Bus et al., 2019; Korat & Shamir, 2008; Parish-Morris et al., 2013; Roskos, Brueck, & Lenhart, 2017; Roskos, Brueck, & Widman, 2009). We refer to these broadly as digital storytelling techniques that may include adding materials such as a dictionary, embedding motion and camera movements in illustrations, adding music and sounds, or asking story-related questions. In our own research, we aim to develop interactive 2.0 designs that are more explicitly theory-driven than first-generation digital books (see Figure 1.1 for an overview of the theoretical cornerstones essential to version 2.0 designs).

Figure 1.1 Theoretical cornerstones, didactic conclusions, and exemplary design features

Grounded in assumptions that underlie theories of multimedia learning (Mayer, 2009), the following theories are particularly relevant:

• Active processing: Humans are not passive processors who seek to add as much information as possible to memory (Mayer, 2005a). They are active processors who seek to make sense of new information. This typically includes paying attention, selecting from all incoming information, and integrating information with their prior knowledge (Gopnik & Meltzoff, 1997). The child applies cognitive processes to incoming narrative and other information to make sense of the storyline (Glenberg, Gutierrez, Levin, Japuntich, & Kaschak, 2004). We do not want children to lean back and listen; we want them to actively make attempts to interpret the presented information – the main reason to add interactive functions. Interactivity is a strength of digital media that, if intertwined with a narrative, may be a strong tool to promote effective problem solving.
• Capacity theory: However, the interactive embedded content may come at a cost for performance. Because the human information processing system has a limited capacity, distributing cognitive resources across both the narrative and the embedded content may be problematic. A person’s ability to process both simultaneously depends on how much capacity separate sources require. Adverse outcomes may result from limited resources being available for processing both narrative and embedded content (Kahneman, 1973). When the embedded content is tangential to the narrative, the two parallel processes of comprehension compete for limited resources in working memory. The result may be that the narrative cannot be processed as deeply as it might otherwise be, and that children may attend to only part of it, thus potentially distorting understanding of the narrative content and resulting in less detailed representations (Yokota & Teale, 2014).
• Distance theory: However, outcomes may be different when interactive functions are intertwined with the narrative. By studying educational television content, Fisch (2000) introduced the idea that distance between the narrative and embedded content will contribute to clear-cut distinctions between problematic and promising embedded content, suggesting that if the distance between the narrative and the embedded content in digital books is small, these elements can complement one another rather than compete for resources, thereby functioning to increase meaning-making of the narrative.

Do’s and don’ts of interactive quality design

Version 2.0 books may draw on several design principles derived from the assumptions underlying theories of multimedia learning in order to achieve beneficial interactivity. In the following sections, we briefly describe four principles that seem particularly important for interactive digital books. The do’s and don’ts of digital storytelling techniques are intended to support children’s active engagement in digital books and meaning-making. Previous publications (e.g., Bus et al., 2015; Sarı, Asûde Başal, Takacs, & Bus, 2019) describe the temporal contiguity and modality principles. Both are demonstrably relevant for digitizing picture storybooks but are not related to this chapter with its focus on interactive functions.

• Coherence principle: It is important that children are involved only in the interactive functions striking the very core of the narrative, and that we bypass interactive functions related only to story themes that would risk introducing extraneous material that might distract children from the storyline. For instance, we may expect adverse effects on story comprehension when an on-demand dictionary provides word definitions. However useful word definitions may be for vocabulary learning, cognitive load theory warrants concern when embedding dictionaries in narratives because additional processing may reduce the amount of attention available for understanding narrative content. Processing word definitions reduces the available resources in working memory for processing the narrative, thereby raising the risk that children will not understand the story or miss important details. Only when interactive functions highlight the or...

Table of contents

• Cover
• Half Title
• Series Page
• Title Page
• Copyright Page
• Contents
• List of figures
• List of tables
• List of contributors
• Introduction to international perspectives on digital media and early literacy
• Part 1 Learning and interaction with digital devices
• Part 2 (Early) literacy learning with digital media
• Index