Languages & Linguistics
Multimodality
Multimodality refers to the use of multiple modes of communication, such as language, images, gestures, and sound, to convey meaning. It emphasizes the interconnectedness of different modes and their combined impact on communication. In linguistics, multimodality is studied to understand how meaning is constructed and conveyed through various modes of expression.
Written by Perlego with AI-assistance
Related key terms
1 of 5
10 Key excerpts on "Multimodality"
eBook - PDFContemporary Stylistics
Language, Cognition, Interpretation
- Alison Gibbons, Sara Whiteley(Authors)
- 2018(Publication Date)
- Edinburgh University Press(Publisher)
As the prefix multi- indicates, Multimodality is the coexistence of multiple modes within any given context. Everyday conversations are an obvious example of multimodal interaction: when we talk, we rely on the modes of spoken language, intonation, and gesture (amongst others). Strictly speaking, there is no such thing as a monomodal text (a text that uses only one mode). Even a textbook like this one, which looks predominantly monomodal, exploits various semiotic resources of the visual system, including written language, 250 READING AS EXPERIENCE numerical signs, graphological emphasis such as bold and italics, dia-grams, and the conventions of textual layout. Nevertheless, analysts generally reserve the term ‘Multimodality’ for texts that more notice-ably use multiple modes. A stylistic approach to Multimodality considers how multimodal texts are composed and how the various modes interact to produce meaning and influence interpretation. 19.2 Analysing multimodal literature Digital printing technologies reduced the publication costs of books with visual and coloured elements, and this has spurred a steady rise in the number of works of multimodal printed literature. Building on Gibbons (2012a: 2), some possible features of multimodal printed fictions include: • unusual textual layouts and page designs • concrete realisation of text to create images, as in concrete poetry • varied typography • images, such as photographs or drawings • use of colour in type and/or imagistic content • flipbook sections • textual deictic devices, drawing attention to the text’s materiality • footnotes and self-interrogative critical voices • genre-mixing, both in literary terms (such as horror) or in terms of visual effect (such as the inclusion of newspaper clippings or play dialogue). Note that the presence of illustrations does not necessarily make a liter-ary work multimodal.
eBook - PDFComputer-Assisted Language Learning
Diversity in Research and Practice
- Glenn Stockwell(Author)
- 2012(Publication Date)
- Cambridge University Press(Publisher)
The polysemy and semantic heterogeneity of this list make the word modality unusable as an operational concept. In the field of Multimodality research, a neighboring discipline to CALL, a consistent definition is available. Multimodality research seeks to understand how we make meaning through the diversity of communicative forms – lan- guage, image, music, sound, gesture, touch, and smell – that surround us. With the exception of smell, these can all be found within the experience of learning and teaching online, which is why I propose to draw some definitions from the work of Multimodality research’s main theorists, Kress and van Leeuwen, who Marie-Noëlle Lamy 110 developed a theory for understanding the meanings communicated to us by objects such as adverts and posters, which use language as one – but not as the main – resource for meaning-making. Let us start with the notion of “mode.” “Words processors,” these authors tell us, “must systematize such things as the thickness and positioning of the lines that separate sections of text, and develop a metalanguage, whether visual or verbal, for making these choices explicit” (2001, p. 79). Anyone who has had the opportunity to compare a page from a magazine in the English-speaking press with one from the French press knows that choice of typeface and positioning of headers, annotations, captions, and so forth are systematized differently in these cultures. The culture-dependent sys- tematization of graphic resources amounts to a specific “grammar” of visual communication, which Kress and van Leeuwen call a “mode.” The physical tool (a computer) changes the physical media (paper and ink) into a mode (a culturally intelligible page layout). Other, much more immediately obvious, culturally intelligible systems include language (written, spoken), the visual (figurative and non-figurative or coded, such as icons), sound (figurative and non-figurative such as music, or coded such as signals), and body-language.
eBook - PDFTransforming Literacies and Language
Multimodality and Literacy in the New Media Age
- Caroline M. L. Ho, Kate T. Anderson, Alvin P. Leong(Authors)
- 2010(Publication Date)
- Continuum(Publisher)
Part II Multimodality and Digital Narratives Multimodality offers a holistic perspective on communication by bringing into high relief the various modes used to make meaning, including visual, aural, gestural, spatial, temporal, and linguistic among others. Malinowski and Nelson illuminate the contentious questions of whether and how to prioritize the linguistic mode given the evanescent and increasingly multimodal textual landscape in education. Engaging the foundational linguistic concepts of “value” and “arbitrariness,” the authors illustrate how language is in a dialogic relationship with other modes, in which the meaning making potentials of different modes may be curtailed, expanded, redistributed, and transformed. Guo, Amasha, and Tan also consider shift-ing the focus from centralizing on language to considering other modes of meaning making as they discuss another dialogic relationship, that between formal and informal learning. The authors argue that teachers may need to expand their notion of learning to take into account the informal and the multimodal. The two chapters in this strand highlight the significance of multimodal communication as a lens for understanding meaning-making and learning beyond traditional, print-based literacy through the trans-formation of the role of language in literacy research and practice. This page intentionally left blank Chapter 3 What Now for Language in a Multimedial World? David Malinowski University of California, Berkeley Mark Evan Nelson National Institute of Education, Nanyang Technological University Introduction In an era in which communication, within and without school settings, is suffused with image-intensive books, icon-laden screens, and streaming videos, the ground that underlies the role of language in education would seem to be shifting.
eBook - PDF- Peter Hudson(Author)
- 2013(Publication Date)
- Cambridge University Press(Publisher)
To be literate in the 21st century demands a repertoire of literacy practices that permits texts to be read and produced for multiple contexts, in multiple modes, in multiple media, using multiple forms of technology; that is, multiliteracies (Cope & Kalantzis, 2000). Chapter 6 Multimodality and complex texts 95 Kress (2010) describes a mode as a semiotic resource for making meaning. Semiotics refers to the study of systems for making mean- ing – signs and symbols and their signification (that is, their meanings). His examples include images, alphabetic and/or pictorial writing, lay- out, music, gesture, speech, moving images, a soundtrack and 3D objects (Kress, 2010). Each mode offers a different potential for making meaning and adheres to different conventions and codes by drawing upon the var- ious semiotic systems: Linguistic: realised in oral and written language Visual: realised in still and moving images Gestural: realised in facial expression, body language and movement Aural: realised in music and sound Spatial: realised through layout and organisation of objects in space (Anstey & Bull, 2010, p. 10) When a text is multimodal, the various modes work together to produce a text that is understandable to its intended audience because it draws upon the appropriate com- bination of the conventions and codes of the five semiotic systems. Thus, a filmmaker producing a text is using the modes of moving image, colour, word, space and sound in such a way that meaning is being constructed by the audience. Being literate with multimodal texts demands an understanding of the possibilities and limitations of all modes, as well as familiarity with the conventions and codes of the semiotic systems upon which the text draws. The conventions and codes of each semiotic system are the socio- culturally accepted rules and patterns by which they work; that is, their grammar(s). Each mode can utilise different media. A medium is the means by which the text is realised.
- John Flowerdew, Li Wei(Authors)
- 2014(Publication Date)
- Bloomsbury Academic(Publisher)
The multimodal analysis reveals that discourse analysis based on language alone is insufficient for interpreting how meaning is created and negotiated today. As a result, the relationship between discourse, context and culture has to necessarily be redefined in multimodal terms. 2 Goals of the study Specifically, this study explores the implications of moving beyond language as an isolated semiotic system to language and semiotic resources as sets of interrelated systems which construe discourse, the situational context and culture. The multimodal (or multisemiotic) approach follows Halliday and Hasan ’s (1985: 4) view of culture ‘as a set of semiotic systems, a set of systems of meaning, all of which interrelate’. In this case, a multimodal approach provides a common theoretical platform for analysing discourse in context, in relation to both the ‘context of situation’ and the ‘context of culture’ (Halliday 1978). Significantly, the approach also provides a theoretical platform for analysing how discourses are resemioticized over time (e.g. Iedema 2001, 2003) with view to ‘tracing how semiotics are translated from one into the other as social processes unfold’ and investigating ‘why these semiotics (rather than others) are mobilized to do certain things at certain times ’ (Iedema 2003: 29). Resemioticization leads to the construal of new meanings which may function to reinforce and/or somehow negate earlier discourses, collectively referred to as the ‘recontextualization’ of meaning (see Iedema 2003). In other words, the multimodal approach is concerned with the dynamics of semiotic interactions and resemioticization and re-contextualization processes over space and time. The multimodal approach is demonstrated by investigating how business news networks contextualize certain events, social actors and social (inter)actions on the internet and television.
eBook - PDFNew Studies in Multimodality
Conceptual and Methodological Elaborations
- Ognyan Seizov, Janina Wildfeuer(Authors)
- 2017(Publication Date)
- Bloomsbury Academic(Publisher)
As a series of such events starts to form, the necessity for their proper documentation and dissemination becomes apparent, too. With each new event in the series (next: BreMM17 in September 2017), we discover new definitions, applications, iterations, operationalizations, and explorations of “Multimodality” as a theory, concept, method, and phenomenon. In setting the stage for this volume of new studies in Multimodality, we offer our working definition of the term, outline where the “new” most often lies, pay our respects to the timeless principles and thinkers that continue to shape multimodal research today, and offer greater insight into the individual chapters and their authors’ motivations and contributions to the field. 2 Defining Multimodality for the twenty-first century Multimodality has been defined variously as a phenomenon (Iedema 2003), a theory (Jewitt 2009b), a methodological application (Jewitt 2009b), and later yet as a field of application (Jewitt 2014; see in particular the differences in the two versions of the Routledge Handbook to Multimodal Analysis (Jewitt 2009a)). It is natural to strive to put any field on solid ground with concrete definitions and sharp delineations. However, we argue that this is counterproductive in the case of modern multimodal studies. At the current stage of development and in keeping with the goal of a grand unified approach to Multimodality, we propose to define it as a modus operandi for conducting research on human communication, both mediated and face to face. As such, it is more encompassing than a method and more palpable and pliable than a theory. Most importantly, it is inclusive and uniting rather than exclusive and divisive. While it falls short of the all-encompassing claim that “everything is multimodal,” it does not forbid the application of multimodal methodologies or theories to any instance of creating and communicating meaning per se.
eBook - PDFMultimodal Semiotics
Functional Analysis in Contexts of Education
- Len Unsworth(Author)
- 2008(Publication Date)
- Continuum(Publisher)
All of these modes of meaning-making interrelate and their totality might be thought of as a way of defining a culture (Halliday and Hasan, 1985: 4). 2 Multimodal Semiotics This conceptualization of language as one of many different interrelated semiotic systems, and hence the assumption that the forms of all semiotic systems are related to the meaning-making functions they serve within social contexts, indicates the strength of SFL in contributing to frameworks for the development of multimodal and intersemiotic theory. SFL-related theorizing proposes that the meaning-making functions of all semiotic systems can be grouped into three main categories, or metafunctions: ideational, interper-sonal and textual. These three kinds of meaning-making or metafunctions are related to three corresponding situational variables that operate in all commu-nicative contexts: field, tenor and mode. Any communicative context can be described in terms of these three main variables that are important in influencing the semiotic choices that are made. Field is concerned with the social activity, its content or topic; Tenor is the nature of the relationships among the people involved in the communication; and Mode is the medium and channel of communication. In relation to lan-guage Mode is concerned with the role of language in the situation – whether spoken or written, accompanying or constitutive of the activity, and the ways in which relative information value is conveyed. These situational variables are related to the three overarching areas of meaning, or metafunctions – ‘ideational’, ‘interpersonal’ and ‘textual’. For example, if I say ‘My daughter is coming home this weekend’, ideationally this involves an event, a participant and the circumstances of time and place associated with it. Interpersonally it constructs me as a giver of information and the reader/listener as a receiver (as well as perhaps suggesting I have at least some acquaintance with the listener).
eBook - PDFNon-discursive Rhetoric
Image and Affect in Multimodal Composition
- Joddy Murray(Author)
- 2009(Publication Date)
- SUNY Press(Publisher)
The Myth of Methodical Multimodality Just as Sharon Crowley and others have worked to dissuade scholars that the “methodical memory” reflected the “quality of authorial minds”—the more logical the writing, the more logical the mind that produced it—so too is there a myth of methodical Multimodality. Multimodality (or monomodality, for that matter) does not reflect the “quality of authorial minds”: there is no legitimacy to the notion that some of us are “more visual” or “more aural” than others when it comes time to create rhetorically appropriate texts for an audience—only, perhaps, that some of us are more practiced at it. By dispelling this myth, teachers and students cannot claim to “be less visual” or “be more visual” than others (and therefore more or less inclined toward composing multimodal texts). In fact, Multimodality is a compositional form that comes from processes based in images which, coincidentally, happens to be closer to the way humans think than the chaining together of concepts as demanded by discursive text. Part of the difficulty both students and teachers have who are unfamiliar with incorporating multimedia into their rhetorical texts stems from their inexperience in reading such texts. Just as any writing course stresses close reading as a way to improve writing, so must multimodal reading become a method of improving multimodal writing. As teachers of beginning film courses know, it takes some time to get students used to thinking about the intentionality of these texts. This requires practice in what Lanham calls “looking THROUGH” or looking AT” text: We are always looking first AT [the text] and then THROUGH it, and this oscillation creates a different implied ideal of decorum, both stylistic and behavioral.
eBook - PDF- Roger Beard, Debra Myhill, Jeni Riley, Martin Nystrand, Roger Beard, Debra Myhill, Jeni Riley, Martin Nystrand(Authors)
- 2009(Publication Date)
- SAGE Publications Ltd(Publisher)
When writing was the dominant mode, these media would vary in their writ-ing to construct different audiences. In the contemporary textbook, the whole range of lexico-grammatical and graphic resources is used to do so. On the web, such differentiation works differently. Often ‘educational’ websites have separate entries for students, teachers, and children. Not only do such sites allow learn-ers to choose themselves the text that they think is apt for their learning, but also allow learners to access all other texts – not only those for other year groups, but also those for teachers or for ‘experts’. ‘The Poetry Archive’, a website which ‘exists to help make poetry accessible, relevant, and enjoy-able to a wide audience’ may serve as an example (see www.poetryarchive.org.uk; retrieved 1 August 2007). The point we want to make repeats the point about the social generating semiotic forms, which we have made several times. In this case, we have a text-entity, which addresses a very different audience to that of the text-book with significant effects in all aspects of the multimodal text, writing included. 180 THE SAGE HANDBOOK OF WRITING DEVELOPMENT Outlook: writing in a multimodal communicational world What are the implications of Multimodality for a pedagogy of writing and for writing itself? The future uses, shapes, potentials of writing as well as conceptions of writing pedagogies need to be considered within a clear sense of social environments. Pedagogy is a specific instance of a larger-level social practice with its relations, processes, and structures, characterized by a focus on par-ticular selections and shaping of ‘knowledge’ (as ‘curriculum’) and learning (as engage-ment with and transformation of that ‘curri-culum’ in relation to the learner’s interest), in or out of institutions such as schools, univer-sity, and the like.
- Barbara Dancygier(Author)
- 2017(Publication Date)
- Cambridge University Press(Publisher)
Researchers then study the phonetic similarity between prime and response. Multimodality in Interaction 149 account for alignment. For lexical alignment, cumulative priming appeared to be the only significant factor: the more often a speaker uses a word, the higher the likelihood that the other speaker will use that word as well. For gestural alignment the factors distance and block proved to be significant: if two gestures are performed at the same time it is very likely those two gestures are identical; and the longer two interlocutors are engaged in interaction, the higher the chances of gestural alignment are. What these case studies demonstrate is that a multimodal approach is a valid if not an essential gateway into a better understanding of con- versational phenomena such as alignment. The Multimodality here does reside not only in the fact that multiple semiotic channels are considered in the analyses, but also in the fact that the interplay and relation between behavior at those multiple levels is the explicit topic of investigation. 9.3.3 Stance Taking: Expressing Obviousness In this final case study, we focus on the interplay between verbal and gestural markers as reported in empirical studies on the expression of obviousness. More specifically, our description concerns the role of co- speech gestures in utterances with a particular German modal particle (einfach) expressing obviousness. The use of modal particles such as ja, denn, doch, mal, halt, and eben counts as a typical characteristic of spoken German in an interactive setting. 7 These lexical elements are “used to add an (inter)subjective nuance to the utterance, in that they indicate the speaker’s position vis-a ` -vis the content of the utterance, the expected hearer reaction, or the relation of the utterance to the context” (Schoonjans 2013a: 1).
Index pages curate the most relevant extracts from our library of academic textbooks. They’ve been created using an in-house natural language model (NLM), each adding context and meaning to key research topics.
