Imagery and Text

A Dual Coding Theory of Reading and Writing

Mark Sadoski, Allan Paivio

• Publisher: Routledge
• Year: 2013
• ISBN:
  9781136623295
• Edition: 2
• Topic: Personal Development
• Subtopic: Writing & Presentation Skills

Imagery and Text, Second Edition extends the first edition's unified theory of cognition in literacy from the perspective of Dual Coding Theory (DCT), one of the most influential and empirically sound theories of cognition ever developed. This theory provides a comprehensive, systematic account of all major aspects of literacy including decoding, comprehension, and response in reading and composing in writing. The Second Edition updates DCT as a scientific theory, a cognitive theory, an embodied theory, and a constructivist theory of literacy. New content includes a detailed account of the decoding process and its integral connection to comprehension, a new program of research on DCT in composing text, a review of neuroscientific support, and increased attention to multimedia literacy, socio-cultural influences, and recent educational applications. More than any other theory, DCT explains how both verbal and nonverbal cognition are woven together through all aspects of literacy. Written in concise chapters with illustrative examples, Imagery and Text is approachable for both students and advanced scholars in the field of literacy.

1
A UNIFIED THEORY OF LITERACY

This book presents a unified theory of literacy that derives from one of the most well-established general theories of cognition yet developed, Dual Coding Theory (DCT). Few such unified theories of literacy have ever been proposed, and even fewer with the historical, empirical, and practical support this theory provides. This chapter will introduce DCT as it relates to the practice of science and to contemporary theoretical perspectives relevant to literacy. In this chapter, we will characterize DCT as: (a) a scientific theory, (b) a cognitive theory, (c) an associationist or connectionist theory, (d) an embodied theory, (e) a constructivist theory, and (f) a theory of literacy.

DCT as a Scientific Theory

DCT shares all the characteristics of other theories in the realm of science. Scientific theory contrasts with other uses of the term theory such as literary theory, critical theory, postmodern theory, and so on. While these schools of thought enrich the literacy scene and often cite scientific studies for support, they are primarily ideological, not scientific. Scientific theory follows the general principles of science. We have provided our perspective on scientific theorizing as it applies to literacy in some detail (Sadoski & Paivio, 2007). We briefly summarize it here.

Science is a cycle of observation and theory. Science involves the close and careful observation of phenomena, the attempt to find consistent patterns in those observations, and the formulation of general theoretical constructs and principles that explain and predict the patterns. Those constructs and principles are then tested through more close and careful observations, modified as necessary, and the cycle continues until a well-defined, empirically-based, and testable theory is established. This may begin on a limited scale, but the practice of scientific theorizing is necessarily expansive, attempting to see how smaller patterns fit into still larger patterns in an ever-expanding quest for more comprehensive understanding. As Kenneth Goodman (2005, p. 15), a pioneer in reading theory, commented:

The job of the scientist is not to find simple causal relationships in reduced and controlled contexts. It is to build a theory of the underlying structures and processes of the reality being studied and then to test that theory against reality again. In doing so, the theory changes and improves but there are always new layers of reality revealed. The more we know, the more we realize how much more there is to be known.

Science does not claim access to ultimate truth, and it can be wrong and even misused. But it employs the qualities of imagination, skepticism, and self-correction that have resulted in the most valuable source of knowledge yet developed. All scientific theories are tentative, evolving toward more accurate, inclusive, and precise theories in an ongoing, asymptotic approach to understanding. Limited theories yield to larger and better theories, with the end goal being the explanation of phenomena in a given domain under a common, parsimonious set of principles.

Of course, that goal has yet to be reached in any scientific field including the physical sciences. Also, the social sciences differ in complexity, measurability, and replicability from the physical sciences. Moreover, scientific theories of literacy are still in their infancy. The first volume dedicated to scientific theories of reading was published little more than forty years ago (Singer & Ruddell, 1970). We readily acknowledge that all current scientific theories of literacy fall well short of comprehensiveness, but we maintain that DCT is the strongest current candidate for a unified scientific theory of literacy (Sadoski & Paivio, 2007). The rest of this book is devoted to explaining how DCT principles can scientifically explain the varying aspects of the domain of literacy to varying degrees.

DCT as a Cognitive Theory

DCT began as part of the “cognitive revolution” of the mid-twentieth century. That revolution came as a reaction to radical behaviorism and heralded a return to the study of inner mental processes after decades of exclusive attention to external stimuli and our external responses to them. Cognitivism shifted attention to the inner world of mental experience; it focused attention on internal mental representations and processes in addition to external stimuli and responses.

DCT originated in observable, everyday phenomena in thought and memory and has developed to account for a complex variety of both everyday phenomena and specialized laboratory phenomena. It is therefore an empirically-based theory; its complexity is determined by our ever-widening knowledge of cognitive behavior. The constellation of testable predictions derived from its basic principles has been programmatically explored through decades of studies. By the late twentieth century, DCT had become recognized as “one of the most influential theories of cognition this century” (Marks, 1997, p. 433). The seminal volumes explaining DCT constructs and principles and their research record include Paivio (1971, 1986, 2007), Paivio and Begg (1981), and Sadoski and Paivio (2001). The DCT research program continues to investigate new and unresolved issues and new questions arise almost daily.

Although the cognitive revolution re-introduced mental imagery as a mental phenomenon of scientific interest, much cognitive theory in the second half of the twentieth century was dominated by the view that the mind was a computer or computer-like. The operational model was computer software, the programs for input, internal processing, storage, and output that control a computer’s operations. Such theories interpret mental representations and processes in terms of abstract entities such as abstract phonemes, abstract lexical nodes, propositions, or schemata. Some of these theories proposed that a single, abstract, computerlike code and its computations ultimately explain all of our inner experience. Others distinguished between verbal and nonverbal representations at the superficial level of input or output, but assumed that deeper levels of cognition were carried on exclusively through an abstract code. Now, in the twenty-first century, computational theories are showing increasing signs of wear as newer findings in behavioral science and neuroscience question their foundations.

DCT has always differed fundamentally from abstract computational theories of cognition in postulating two distinct mental codes, a verbal code for human language in all its sensory forms, and a nonverbal code of our mental images in all their sensory forms. DCT has never postulated any abstract mental representations; all mental representations in DCT are specific to a sensory modality (i.e., modality-specific as opposed to amodal). The inclusion of nonverbal aspects of cognition such as mental imagery is a distinctive aspect of DCT in modern cognitive theory and especially literacy theory, but DCT provides a comprehensive account of the verbal, linguistic aspects of cognition as well. This will be explained in more detail in Chapter 3 and elsewhere.

DCT as Associationist or Connectionist Theory

DCT is an associationist or connectionist theory in that it postulates the mind as a loose web of connected units. Some parts of the web become densely and intricately connected as our experience and knowledge grow, while other areas remain sparsely connected where experience and knowledge are lacking. This web has a degree of stability, but it grows and changes with experience so that connections are dynamic, flexible, and subject to contextual shifts. The size of this web ultimately becomes enormous as connections grow.

The associational characteristics of the theory bear comparison with older associationist theories (e.g., Osgood, 1953) and newer connectionist theories (e.g., Seidenberg & McClelland, 1989). However, DCT differs from all other members of both classes in its emphasis on the functional difference between the verbal and nonverbal codes. Early verbal associationist theories did not take imagery into account. Newer connectionist computational theories either discount the difference between verbal and nonverbal encoding, or recognize it without attributing any functional significance to it. In DCT, the number and strength of the connections between units is important, but the difference in the qualities of the verbal and nonverbal units that get connected is also important. These qualitative differences have important consequences for the kinds of mental structures that are built and how they perform their functions.

We emphasize again that this network is not an assemblage of abstract schemata. (For more extensive discussions of the differences between DCT and schema theory, see Sadoski, Paivio, & Goetz, 1991; and Krasny, Sadoski, & Paivio, 2007). In DCT, modality-specific language units such as letters or written words are connected to other modality-specific language units such as phonemes or whole-word pronunciations. Words or phrases can be verbally associated with other words or phrases such as synonyms, antonyms, or paraphrases. Words and phrases can be connected to nonverbal units as well, as when mental images spontaneously occur in reading words, sentences, or longer texts describing scenes and events. In DCT, this network is assumed to have a neurological basis in modality-specific representations and their connections. Such associations or connections are formed through life experiences with language and the world to which it refers, and are predictable within personal and sociocultural situations.

DCT as an Embodied Theory

The view that cognition is inherently embodied is rapidly replacing mind-as-computer models. Theories of embodied cognition have in common an assumption succinctly stated by Thelen, Schöner, Scheier, and Smith (2001, p. 1):

To say that cognition is embodied means that it arises from bodily interactions with the world. From this point of view, cognition depends on the kinds of experiences that come from having a body with particular perceptual and motor capacities that are inseparably linked and that together form the matrix within which memory, emotion, language, and all other aspects of life are meshed.

In an influential work, Lakoff and Johnson (1999) maintained that all knowledge and reason is inherently embodied – rooted in sensual experience. Our conceptual system is grounded in, and shaped by, our sensorimotor systems. Conceptualization is therefore fundamentally concrete; abstract conceptualization and reasoning are based in metaphors that are rooted in sensorimotor experience (e.g., time is like movement in space, cause is like physical force). Other cognitive theorists have taken similar positions (e.g., Barsalou, 1999; Glenberg, 1997; Wilson, 2002). Literacy theorists are likewise turning toward embodied theories. For example, Zwaan (2004) has proposed an embodied theory of reading comprehension, citing DCT as one influence.

Embodied principles have been part of DCT since its inception. It bears resemblance to the work of earlier theorists including Piaget, Hebb, and others who acknowledged the bodily, sensory nature of experience and learning. As explained earlier, mental representations and processes in DCT are assumed to be derived from sensory experience and stored in modality-specific rather than amodal form. DCT and all other embodied theories differ from computational theories of cognition in this important way.

However, DCT differs from other embodied theories in at least one important way. Embodied theories in general do not distinguish the qualitative differences in the verbal and nonverbal codes that predict and explain phenomena such as the differences in the way we process concrete and abstract language, picture-language effects in multimedia learning, and so on. Also, some embodied theories still include abstract schematic representations at some level (e.g., image schema), clouding the theoretical picture. As we have noted, DCT does not include any abstract mechanisms. The embodied cognition movement is evolving rapidly, and the perspectives it is opening or re-opening are introducing exciting new directions. Throughout this book, we will emphasize the embodied nature of DCT as well as its uniqueness in this family of theories.

DCT as a Constructivist Theory

Constructivism is a popular metaphor in cognition and literacy. The general constructivist view maintains that we are not passive receivers of experiential information, but active builders of our experience into knowledge structures through which we then interpret and navigate the world. Two general variants of constructivism have gained prominence in literacy: cognitive constructivism and social constructivism.

Cognitive constructivism is largely noncontroversial in literacy theory and education (but see Stanovich, 1994; and Phillips, 2000). The history of the early impact of cognitive constructivism on the field of literacy has been well reviewed by Spivey (1987, 1997) among others. DCT is clearly a part of the cognitive constructivist movement although it has important differences from some other theories in the movement (e.g., schema theory) as noted earlier. In DCT, the reader or writer constructs texts and their interpretations from sensory-based, modality-specific, verbal and nonverbal mental structures as affected by external contexts of various kinds including social contexts.

Social constructivism (and related schools of thought such as sociocultural constructivism, social constructionism, and so on; see Hruby, 2001) is generally based on the idea that individual knowledge and thought is strongly mediated by social influences including community and culture. The social constructivist perspective most influential in literacy has been that of Lev Vygotsky (1962, 1978). Vygotsky’s ideas about cognition are inconsistent with DCT because his theory held that all thought took the form of inner speech even at very young ages, a position very similar to that of the radical behaviorists of the early twentieth century. The DCT position is that thought takes the form of nonverbal mental images as well as inner speech. However, DCT allows for the effects of social influences in constructing inner mental representations and even the self.

In this regard, the social constructivist philosophy of George Herbert Mead (1934) is perhaps more consistent with DCT. Mead, a pragmatist philosopher and colleague of John Dewey, is often heralded as the founder of social psychology. Mead maintained prominent roles for both language and mental imagery in thought, action, and the development of the self in a social world. He employed the terms significant other and generalized other, the latter term meaning our mental impressions of common beliefs, thoughts, and language use within a social system that define us as an acting member of that social system. Because DCT emphasizes constructive sensory experience, and because we live in a social world, we become social beings by definition. For a discussion of Mead’s views in relation to DCT in literacy, see Sadoski (1992). For a broader discussion of the DCT perspective on the role of social interactions in the evolution of the mind and the development of the self, see Paivio (2007). For a philosophical discussion of the relevance of Mead’s views to social constructivism in science and education, see Bredo (2000).

DCT as a Theory of Literacy

Accounting for all aspects of literacy under the aegis of a general theory of cognition is a consummate goal from both the perspectives of scientific theory and educational practice. Ultimately, the receptive and productive facets of literacy must be interpreted as complementary parts of the same cognitive system, explained by the same general constructs and principles. This is a normal part of scientific progress, but such an approach to literacy has considerable practical value for education as well, where the development of complementary cognitive capabilities could efficiently serve the development of both reading and writing.

DCT emerged as a fully realized theory of cognition that involved both language and mental imagery in the 1970s. However, the role of imagery in reading had been of interest to researchers in literacy and education for decades before (e.g., Durrell & Murphy, 1963). One of the earliest theorists to propose a central role for imagery and the nonverbal in reading was Huey (1908). Huey was unable to resolve the apparent impasse in his theory created by the difficulty of imaging to relational words such as prepositions and conjunctions. However, others extended Huey’s theory, suggesting that such words are imaged when associations are brought together in phrases and sentences (e.g., Paivio, 1971). Imagery could be evoked by contexts as well as individual words.

Interest in imagery in reading and education burgeoned throughout the 1970s and into the 1980s. Pressley (1977) reviewed dozens of studies on imagery and children’s prose learning done by various researchers between 1970 and 1977. DCT captured some early attention as a potential explanation of text comprehension and individual differences in reading (Jacob, 1976). The potential of DCT to explain reading gained more direct attention in the 1980s, with some literacy researchers invoking it specifically as an explanation for their results (e.g., Sadoski, 1983, 1985). An analysis of the use of both imagery and verbal processes in reading was presented by Paivio and Begg (1981, Chapter 14).

A fuller extension of the theory to reading came about when it was posed as an alternative to schema theory in explaining reading comprehension (Sadoski et al., 1991). Shortly after, the dual coding view of reading was proposed more comprehensively to include comprehension, learning, and reader response (Sadoski & Paivio, 1994). It was also introduced as a plausible theory of written composition (Sadoski, 1992). More complete treatments and updates followed (Sadoski & Paivio, 2001, 2004, 2007). This book summarizes and updates the DCT account of literacy and its empirical record.

In summary, DCT can be classified a scientific theory, a cognitive theory, an associationist or connectionist theory, an embodied theory, a constructivist theory in both the cognitive and social senses of the term, and a theory of literacy. However, it is a special case in each of these categories because its theoretical constructs and processes differ significantly from other members of the classes.

Overview of the Rest of the Book

Chapter 2 summarizes how the fundamentals of dual coding have been of interest throughout intellectual history from ancient...