Multimodality

Disciplinary Thoughts and the Challenge of Diversity

Janina Wildfeuer, Jana Pflaeging, John Bateman, Ognyan Seizov, Chiao-I Tseng, Janina Wildfeuer, Jana Pflaeging, John Bateman, Ognyan Seizov, Chiao-I Tseng

• Publisher: De Gruyter
• Year: 2019
• Print ISBN:
  9783110607987
• eBook ISBN:
  9783110608052
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

Multimodality's popularity as a semiotic approach has not resulted in a common voice yet. Its conceptual anchoring as well as its empirical applications often remain localized and disparate, and ideas of a theory of multimodality are heterogeneous and uncoordinated. For the field to move ahead, it must achieve a more mature status of reflection, mutual support, and interaction with regard to both past and future directions. The red thread across the disciplines reflected in this book is a common goal of capturing the mechanisms of synergetic knowledge construction and transmission using diverse forms of expressions, i.e., multimodality. The collection of chapters brought together in the book reflects both a diversity of disciplines and common interests and challenges, thereby establishing an excellent roadmap for the future. The contributions revisit and redefine theoretical concepts or empirical analyses, which are crucial to the study of multimodality from various perspectives, with a view towards evolving issues of multimodal analysis. With this, the book aims at repositioning the field as a well-grounded scientific discipline with significant implications for future communication research in many fields of study.

Part I:Introduction

Janina Wildfeuer, Jana Pflaeging, John A. Bateman, Ognyan Seizov, and Chiao-I Tseng

Multimodality: Disciplinary Thoughts and the Challenge of Diversity – Introduction

Abstract: In this introduction, we discuss the idea of establishing a discipline of multimodality, considering both how this might be defined and potential benefits and challenges of attaining such an independent status. This builds on previous rounds of discussion within the Bremen Conferences on Multimodality (BreMM) series concerning this issue, where diverse approaches to the study of multimodality have come together to create a broad-based set of agenda items where a more systematic engagement with the phenomena of multimodality is key.

Keywords: multimodality, discipline, interdisciplinarity, transdisciplinarity, multidisciplinarity, semiotics

1Conceptualizing Multimodality’s Disciplinary Status and Challenges

This book presents a collection of position articles and reports of empirical and theoretical results in multimodality drawn from the most recent of our series of international conferences on multimodality. The broad range of topics, approaches, and research objects covered by the contributions in this volume attests clearly to the growing diversity of what is typically subsumed under the label of multimodality. Not only does our volume showcase this remarkable diversity, including many innovative transdisciplinary and cross-national research projects, it also seeks to draw attention to the inclusivity and flexibility of current approaches in our field.

Be it in different ways, both of these characteristics encourage a more explicit and more thorough discussion of multimodality’s current disciplinary status, and any future developments that result from it: On the one hand, the overlap in research interests and similarity of approaches seems to justify thinking about a commonly shared name, concepts, or institutionalization. On the other hand, and perhaps more importantly, the prevalent diversity of approaches and perspectives suggests considering the unifying potential of a disciplinary ‘home’, which promotes an ongoing exchange of ideas and mitigate less productive ivergences and fragmentation.

As documented by this volume’s contributions, scholarship within the ‘broad context of multimodality’ has recently been expanding into a wide range of areas in which an even broader variety of methods is adopted. This developmental trajectory indicates that multimodality may indeed be at risk of fragmentation if (meta-)disciplinary guidelines are not adopted that fully embrace the necessity of associative and hierarchical approaches, and channels to communicate. As a field naturally poised for further growth, multimodality needs to be well-prepared to tackle a host of diverse theoretical and empirical challenges, both now and in the future. We believe that achieving a more mature status of critical self-reflection that will ultimately be able to offer better mutual support across research initiatives and methods, as well as strengthening interaction across such initiatives, will be instrumental in moving forward successfully.

This introduction, in particular, is entirely devoted to the pressing discipline-related questions pointed out, placing a particular focus on the diversity and breadth of multimodal research and critically conceptualizing the question of whether multimodality could, and perhaps even should, be seen as a discipline in its own right. In addition, some of the discussions led in particular chapters of this book also consider, either theoretically or by drawing on the results of empirical research, whether a move towards establishing multimodality as a stand-alone discipline is desirable and, if so, what would be necessary to accomplish it.

Our disciplinary thoughts have been heavily influenced by contributions from the international community of multimodality researchers, particularly those scholars that shared their work at one of the previous Bremen Conferences on Multimodality. These voices strongly support our assumption that the “broad context of multimodality research is now on its way toward forming its own fully acknowledged discipline” (Wildfeuer & Seizov, 2017, 279).

As our conferences as well as the many handbooks and introductions to the field continuously show, activities grouped under the term ‘multimodality’ constitute one of the most influential contemporary approaches to the study of all kinds of communicative artifacts and performances. Multimodal perspectives on such research objects can now be found in almost every discipline interested in the analysis of communication. This popularity even transcends such broad orientations as the ‘humanities’ and includes scholars and practitioners from diverse fields wherever the goal is to scrutinize combinations of communicative forms and the meaning-making mechanisms deployed. As a consequence, these efforts readily employ tools and frameworks that have evolved, and which continue to evolve, drawn from diverse research areas as well. In this sense, multimodality can already be seen to be far more multidisciplinary than many other fields of research; van Leeuwen (2005, 1), for example, even sees multidisciplinarity as an “essential feature” at least of social semiotics, upon which many approaches to multimodality build.

This situation brings both benefits and challenges. Among the benefits are precisely that broad range of different voices and empirical applications that become included from various areas concerned with communication. Conversely, however, among the challenges comes the task of dealing productively with often quite disparate conceptual anchorings, frameworks, and questions. Indeed, one of the most frequent and reoccurring doubts raised concerning multimodal research is that of whether such diverse voices can communicate at all. As a consequence, attempts to procure funding still often founder on the shoals of methodological doubt among reviewers relatively unversed in the radical multidisciplinarity that has long formed the everyday practice of many of those active in multimodal research.

Moreover, and as we have set out at length previously (see Wildfeuer, 2015; Seizov &Wildfeuer, 2017; Bateman et al., 2017), although the theories and methods of multimodality are indeed naturally heterogeneous, they have also nevertheless often remained confined to national and regional research communities, sometimes cross-cutting theoretical or philosophical lines of demarcation, sometimes aligning with them. This in turn continues to create problems of recognition both within multimodality and without. The current state of ‘multimodality’ is then one of a rather uncoordinated field that does not always grant its own wide-ranging scholarly community and the results they have achieved equal respect. Even appropriate mutual knowledge of related approaches is often wanting. It is particularly in response to issues of these kinds that we consider a more explicit orientation to the status of multimodality as a potential discipline not only timely, but also increasingly urgent.

The recent historical development of the diverse activities potentially contributing to multimodality has been complex. By the 1960s and 1970s, multimodality had already become an object of study in several disciplines and research directions, even if the term was not itself always explicitly used. Since then, and particularly after a first phase of development and theory formation in the 1990s and early 2000s, multimodal questions have increasingly taken on the role of an informing and supplementing research area incorporated by other disciplines or included in interdisciplinary projects, calling for practitioners and theorists alike to combine complementary approaches rather than seeking insufficiently motivated discipline-internal replacements or by beginning anew. Today, further developments in attitudes and awareness have begun to promote an openness that has increased both the motivations for, and the rewards of, broader inter- and transdisciplinary approaches (seeWildfeuer & Seizov, 2017). These not only focus on theory and method but also include a substantial growth in empirical applications and evaluations. Taken together, such developments have already widened horizons and opened up fundamental academic debates leading to more robust definitions and conceptualizations of the field’s main terms, and these, particularly, now invite explicit discipline-building explorations.

Although the current diversity and multidisciplinarity of multimodality could reasonably be seen as a significant constraint on sustainable development and advancement of the field, these properties have also created, on the other hand, a multiperspectivity which holds considerable potential for shaping a disciplinary status of a rather particular kind. The contributions to this book mainly reflect this potential of the diversity of disciplines and interests they represent. Their collection here serves to contribute further way stations requiring consideration when envisioning roadmaps for multimodality as the field reaches towards disciplinary status. Multimodality has clearly not achieved this status yet, however, and there are still serious concerns with regard to other disciplines and developing trends that make it difficult to assess whether this will happen in the near future. Nevertheless, below we further outline some relevant developments of the field to date which would serve as prerequisites for such disciplinary status, and we critically discuss how multimodality may benefit from further efforts in this direction.

As preparations for this, we explicitly included in the call for papers for BreMM17 several questions central when considering moves towards a discipline of multimodality. In Section 2 following, therefore, we begin by presenting recent relevant developments drawing on the conference contributions and presenting an overview of the chapters appearing in the volume. These all more or less respond to the questions raised in the call for papers by providing theoretical and methodological discussion or practical reports exemplifying the current status of multimodality. In the subsequent sections, we turn to consider the potential of these contributions as participants in a broader discipline-building endeavor. In Section 3, we collect criteria from more general characterizations of the features of disciplines as well as presenting a more particular assessment of one research field—the Digital Humanities—which is often argued to have gained the status of a fully-developed discipline despite being constituted by a similarly heterogeneous range of approaches and questions. In Section 4, this collection of discipline-specific characteristics is taken as a backdrop against which multimodality’s current disciplinary status can also be assessed. At the same time, this offers a frame of reference which makes apparent further prerequisites for the advances necessary for a discipline to emerge.

We then suggest an approach to a discipline of multimodality which, on the one hand, would provide a complex definition for the discipline, while, on the other hand, leaving appropriate room for extension. Finally, in Section 5, we round up the introductory discussion with some further reflections and an outlook on future directions relevant for establishing multimodality as a discipline in its own right.

2Towards a New Discipline: Developments to Date and This Book’s Contributions

In this section, we focus particularly on what steps have already been taken towards multimodality as a ‘stand-alone’ discipline, regardless of whether they were announced as such at the time or have only become identifiable in retrospect, and then situate the contributions to this book against this background.

2.1Developments to Date

Efforts to make multimodality a more homogeneous field or provide common foundations for a more unified theory or methodology are not new and can be found in various discussions early on, albeit often not in a particularly differentiated fashion. In the late 2000s, when the first handbooks and comprehensive overviews of the field were published, multimodality was described as “a theory, a perspective, a methodological application or a field of enquiry” (Jewitt, 2009a, 127)—a conceptualization that is not without ambiguity and vagueness. However, with calls for “a general project of multimodality” (Constantinou, 2005, 604) and the implementation of more “systematic rigour” (Forceville, 2010, 2607), the “need for methodological development” (Björkvall, 2012, 8) became increasingly recognized as a pressing issue.

Nevertheless, while experiencing rapid growth, as noted above, multimodality research remained extremely heterogeneous, exhibiting a broad diversity of theoretical and methodological developments. Indeed,

[t]his diversity is [...] one of the constituting features not only of the field and its contexts of investigation, but also of all the work combining under this keyword, which makes it a particularly challenging and exciting endeavour. (Wildfeuer, 2015, 21)

Moreover, there were (and still are) considerable differences between national and international approaches. Crossing disciplinary boundaries therefore poses a considerable challenge, as demonstrated with the example of the situation in Germany in comparison to the international context discussed in Wildfeuer (2015).

One of our main aims at that time, whichwe now see as a first initiative towards discipline-building, was to find relations between the various approaches engaging in multimodality research and to construct bridges between them as necessary, identifying similarities and differences within this inter- and multidisciplinary mix. Here it was necessary to emphasize that “multimodality is interdisciplinary in itself and there is absolutely no need for any discipline to claim the leading role for itself” (Wildfeuer, 2015, 22); it could already be observed that many different disciplines involved in multimodal research were “not compet[ing] against each other but rather gain from their reciprocal influence and mutual exchange” (Wildfeuer, 2015, 22). The contacts established between theories, methods, and empirical applications made visible further potential for developments that increasingly “help strengthen multimodality as a still to be fully established paradigm” (Wildfeuer, 2015, 30).

Parallel to our own initial efforts in the early 2010s, several other colleagues have made similar attempts to revisit multimodality by adopting a broader perspective and framing it as a discipline. Not least, several multimodality-related handbooks and textbooks published in recent years have been crucial for reflecting and neg...

Table of contents

• Cover
• Title Page
• Copyright
• Preface and Acknowledgements
• Contents
• Part I: Introduction
• Part II: Disciplinary Thoughts
• Part III: Diversity
• Part IV: More Disciplinary Thoughts
• List of Contributors
• Index