Section 1
What are multimodal texts?
In this section:
- defining multimodal texts
- writing and multimodality
- affordance and design: using different modes and media
- texts that show and texts that tell
- surveying what children know about multimodality
- developing ways of talking about writing and multimodality
- Classroom Account: Planning and making picture books with Years 1/2
- popular culture and writing.
Defining and describing multimodal texts
Digital technology has brought significant changes to writing over the last 20 years or so. In everyday print-based communications like newspapers, information leaflets or advertising, words are now almost always accompanied by photographs, diagrams or drawings, and the print is enhanced by a variety of font sizes and shapes. Screens are much more familiar in shops, workplaces, schools and homes. Mobile phones transmit images and words as well as sound. Many everyday texts are now multimodal, combining words with moving images, sound, colour and a range of photographic, drawn or digitally created visuals.
But multimodality is not new. People of all cultures have always used a range of ways to represent ideas and communicate meaning through speech, writing, image, gesture and movement, music and sound. The ‘newness’ is the way that messages are relayed and distributed through different media of communication. Communication is possible not only through the traditional means of paper, in picture books, magazines, novels or information books but now also through the computer, as Internet information, emails and presentations, and via sound and visual media – radio, television, videos and DVDs.
Even the most familiar and everyday communications are made up of complex combinations of modes. Talk, whether in face-to-face meetings or viewed on screen, is accompanied by movement and gesture; print is often accompanied by pictures; and films and television programmes rely on sound effects and music to add atmosphere and effect. Any multimodal text might combine elements of:
- gesture, movement, posture, facial expression
- images: moving and still, real or drawn
- sound: spoken words, sound effects and music
- writing, including font and typography.
These elements will be differently weighted in any combination of modes; for example, there is usually no verbal dialogue in ballet, and novels are predominantly made up of words alone.
Children grow up in a highly multimodal environment. In the street, home and school, they are surrounded by texts on screen and on paper which merge pictures, words and sound. They expect to read images as well as print and, increasingly use computers in seeking information and composing their own texts. In school, developments in publishing mean that they are familiar with a wealth of picture books and information books presented in well-designed double-page spreads. These books and the texts children read on screen influence their compositions, acting as models and examples of possible ways to express ideas and information. This has implications for teaching. The texts that children are familiar with – including computer games and hypertext – often follow a different structure from sequential narrative, instruction or explanation. Presentational software and websites extend possibilities for hypertextual composition, and digital technology, with its facility for importing pictures and manipulating text, means that presentation of writing can be more varied, involving design features that paper-based writing does not allow.
However, the expansion of types of text does not mean that writing will become a thing of the past. Far from it. In fact, text messaging, emails and blogging may already have contributed to greater everyday experience of writing. As far as classroom writing is concerned, although handwriting will not disappear, there will be much more on-screen writing. It is also likely that the process of composing, editing and revising will expand to include screen-based presentations as well as writing.
One of the advantages of on-screen production of texts is that children will more easily see themselves as authors, with the responsibility to proofread and craft their writing. The use of interactive whiteboards (IWBs) in classrooms means that it is much easier for a group of learners to view a piece of writing and jointly discuss editing improvements. At first it is likely that they will read the work of published authors, considering use of language to create specific effects and analysing how an experienced writer crafts a piece of writing. From there it is a short step to reviewing their own composition with an analytical eye, examining their own use of language, style and sentence structure. A group of children composing on screen can readily amend work, so that composition benefits from collaborative support and the facilities of the computer. All of this experience forges strong examples of how an individual can gain satisfaction from crafting a piece of writing until it does the job the author wanted it to do. This process of apprenticeship to reading with a critical eye, editing and revising is equally relevant when children are writing or composing multimodal texts on paper and on screen.
Writing and multimodal texts
This book looks at writing both as part of multimodal texts and in its own right. Including multimodality in the literacy curriculum means learning to decide when to communicate in writing alone and when to use a multimodal form. The term ‘text’ is used specifically to describe any communication made up of an interwoven combination of modes. It is just not accurate to describe a designed leaflet, the double-page spread of an information book, or a screen displaying information on the Internet as ‘writing’. Each of these is made up of a combination of image, word, layout and sometimes sound. They are, in fact, multimodal texts. To avoid any confusion, we distinguish throughout this book between multimodal texts, which we term ‘texts’, and writing.
Decisions about whether it is better to use a combination of modes or a single mode are related to purpose and audience. If a message is to have maximum effect, it is important to choose the best form of communication. This will be influenced by the writer’s view of what the reader or audience will need to help them understand the meaning. For example, it may be better to use charts, pictures and even gesture alongside words to help explain a complicated process; on the other hand, creating a short story with words alone serves a different kind of purpose. The author selects particular combinations of modes for the job in hand. Similarly, specific media are better suited to certain types of communication: a novel is more easily read on paper than on a screen, and an IWB, with its facilities for moving text and images about, can be more helpful than paper when explaining a process in design technology or science.
Affordance and design
In teaching about choices of modes and media, children need to consider what different modes and media afford for making meaning. This is often tied up with the material of the medium. Reading a story in a printed book affords a different kind of experience from watching a television or film narrative. The fact that a book is made up of pages – the material of the book – which are easily turned by hand, makes it possible to skip descriptive passages, vary the pace of reading, and return to earlier pages to check details or recapture the narrative flow. With television or a film, ‘skipping’ or returning to earlier parts of the story is not possible unless it has been pre-recorded. Even with a recording, where it is possible to review and fast-forward, it is difficult to pick up on detail without considerable effort. The screen, disc or videotape afford a different set of reading possibilities from a magazine or book because of what they are made of and how they work.
Affordance is also related to differences in messages according to whether they are presented in writing or in words-plus-images. Writing is necessarily chronological – sequenced according to time: this event happened, then this, then this...