Systematically Working with Multimodal Data

Research Methods in Multimodal Discourse Analysis

Sigrid Norris

• Publisher: Wiley-Blackwell
• Year: 2019
• Print ISBN: 9781119168324, 9781119168317
• eBook ISBN: 9781119168348
• Edition: 1
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

A guide that offers a step-by-step process to data-driven qualitative multimodal discourse analysis

Systematically Working with Multimodal Data is a hands-on guide that is theoretically grounded and offers a step-by-step process to clearly show how to do a data-driven qualitative Multimodal Discourse Analysis (MDA). This full-color introductory textbook is filled with helpful definitions, notes, discussion points and tasks. With illustrative research examples from YouTube, an Experimental and a Video Ethnographic Study, the text offers many examples of how to deal with small to large amounts of data, including information on how to transcribe video data multimodally, including online videos, and how to analyze the data.

This textbook contains ample theory, directions for literature, and a teaching guide to help with a clear understanding of how to work with multimodal data.

• Contains new research data, exceptional illustrations and diagrams
• Offers step-by-step processes of working through examples, transcriptions and online videos
• Goes into great depth so that students can use the book as hands-on material to engage with their own data analysis
• Designed to be easy-to-use with color-coded definitions, tasks, discussion points and notes

Written for advanced undergraduate, graduate and PhD level students, as well as participants in research workshops, Systematically Working with Multimodal Data is an authoritative guide to understanding data-driven qualitative Multimodal Discourse Analysis.

Chapter 1
Introduction

1.0
Introduction to the Book

Multimodal discourse analysis is an area of research that is becoming more and more widely used in applied linguistics, sociolinguistics, education, psychology, anthropology, business and other applied social sciences. There are a number of multimodal approaches that have sprouted up over the past 20 years (Bateman 2008; Forceville 1994; Jewitt 2002; Kress and van Leeuwen 1996, 1998, 2001; van Leeuwen 1999; Mondada 2006; Norris 2002a, 2004a; O'Halloran 1999; O'Toole 1994; Scollon 1998, 2001a, b; Stöckl 2001). Yet, Multimodal (Inter)action Analysis (Norris 2002a, 2004a, 2011a, 2013a) is the only interdisciplinary approach that has been developed specifically for the analysis of multimodal action and interaction. In this approach, emphasis is placed upon the actions that people take as opposed to the language plus particular non‐verbal movements that they produce (Goodwin 1981; Mondada 2014; Scollon 1979), the cognitive work that they do (Anderson 1990; Collins and Quillian 1969; Fodor 1975; Kintsch 1988; Newell and Simon 1976; Tulving 1983) or the psychological expressions that they display (Ekman 1979; Ekman and Friesen 1969). Actions, of course, are embodied and cognitive, psychological and performed with language plus non‐verbal movements. In fact, all of the components, the verbal, non‐verbal, environmental, cognitive, and psychological come together in our approach to analyzing multimodal (inter)action (Norris 2013a, b). Through systematic analysis of actions and interactions, this approach allows us to gain new insight into human action and interaction in a holistic and comprehensive way, and this book demonstrates how to engage in systematic analysis of multimodal (inter)actions.

What This Book is About

This book illustrates the phases and steps used when engaging in a Multimodal (Inter)action Analysis. The Step‐by‐Step process outlined here is a guide that shows how you can systematically work with multimodal data. This systematic guide consists of five phases, four of which consist of a number of steps each and one of which consist of a great number of analytical tools. The book emphasizes our working with video data. However, multimodal (inter)action analysts also use this process when working with different kinds of data that include but are not limited to video data. This is shown to some extent in the chapter sections that demonstrate how to use the Step‐by‐Step guide with examples from an experimental study and a video ethnography. However, because this book focuses upon video data and the holistic ways to analyze it, other data such as interviews or observational notes may be alluded to, but not worked with here in detail. However, it is important to note that for us, observational notes, text messages, emails, interviews, and diary entries also often are a part of the data collected and analyzed. But now, let us turn to what this book is about.

This book is written for undergraduate and graduate students as well as for emergent and established researchers wishing to engage in multimodal discourse analysis in a theoretically founded and interdisciplinary manner, integrating the verbal and non‐verbal with object use and the embeddedness of people with the environment. Some sections of chapters are more geared towards undergraduate students, while other sections of chapters are more useful for Masters students, and again other sections are particularly important for PhD students and researchers wanting to engage in video ethnography. Some chapter sections are absolutely necessary to read for all readers, while other chapter sections are focused to this or that readership. Here, I would like to allude the reader to what is most necessary to engage in for whom to make reading choices and reading assignments easier and clearer for teachers and students alike. The gray boxes contain notes which give a quick overview of which chapters and sections are useful for whom and what to expect to find in them.

Systematically Working with Video Data: Phases I–V

Chapter 1: Systematically Working with Multimodal Data: Introduction

---

Note 1

Chapter 1 is a useful read, but not a must‐read for undergraduate or graduate students starting out to learn how to conduct a multimodal discourse analysis. PhD students will definitely want to read Chapter 1.1.

---

---

Note 2

Chapter 1 introduces the reader to the book overall, presents a chapter‐by‐chapter outline, and offers the keen reader a vast amount of references in some of its paragraphs. Without going into too much detail, these references can be used as a guide to delve deeper into the background literature alluded to here.

---

Chapter 1, besides first giving a brief overview of what you can find in the chapters to come, quickly sums up other multimodal research areas and the literature background of Multimodal (Inter)action Analysis. Her...

Table of contents

• Cover
• Table of Contents
• List of Figures
• List of Tables
• Acknowledgments
• About the Companion Website
• Chapter 1: Introduction
• 1.0 Introduction to the Book
• 1.1 Brief Introduction to Multimodal Discourse Analysis
• Chapter 2: Background
• 2.0 Philosophical and Theoretical Background
• 2.1 Development of Scollon’s Philosophical Thought
• Chapter 3: Systematically Working with Multimodal Data Phase I
• 3.0 Phase I
• 3.1 Systematically Working with Small Data Sets/Data Pieces
• 3.2 Systematically Working with Medium‐Sized Data Sets
• 3.3 Systematically Working with Large Data Sets
• Chapter 4: Systematically Working with Multimodal Data Phase II
• 4.0 Phase II
• 4.1 Systematically Working with Small Data Sets/Data Pieces
• 4.2 Systematically Working with Medium‐Sized Data Sets
• 4.3 Systematically Working with Large Data Sets
• Chapter 5: Systematically Working with Multimodal Data: Phase III
• 5.0 Phase III
• 5.1 Systematically Working with Small Data Sets/Data Pieces
• 5.2 Systematically Working with Medium‐Sized Data Sets
• 5.3 Systematically Working with Large Data Sets
• Chapter 6: Systematically Working with Multimodal Data Phase IV
• 6 Phase IV
• Chapter 7: Systematically Working with Multimodal Data Phase V
• 7 Phase V
• Chapter 8: Systematically Working with Multimodal Data
• 8 A Quick Guide for Instructors
• References
• Index
• End User License Agreement