Multimodal Approaches to Chinese-English Translation and Interpreting

Meifang Zhang, Dezheng (William) Feng, Meifang Zhang, Dezheng (William) Feng

• Publisher: Routledge
• Year: 2020
• Print ISBN: 9780367331962
• eBook ISBN: 9781000320374
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

Nowadays, discourse analysis deals with not only texts but also paratexts and images; so do translation and interpreting studies. Therefore, the concept of multimodality has become an increasingly important topic in the subject areas of linguistics, discourse analysis and translation studies. However, up to now not much research has been done systematically on multimodal factors in translation and interpreting, and even less in exploring research models or methodologies for multimodal analysis in translation and interpreting.

This book aims to introduce and apply different theories of the multimodal discourse analysis to the study of translations, with case studies on Chinese classics such as the Monkey King, Mulan and The Art of War, as well as on interpretations of up-to-date issues including the Chinese Belt and Road Initiatives and Macao tourism.

The chapters reflect the first attempts to apply multimodal approaches to translation and interpreting with a special focus on Chinese-English translations and interpreting. They provide new understandings of transformations in the multimodal translation process and useful reference models for researchers who are interested in doing research of a similar kind, especially for those who are interested in looking into translations related to Chinese language, literature and culture.

1
Intersemiotic shifts in the translation of Chinese costume drama subtitles

A multimodal analysis approach

Qian Hong (Sunny) & Feng Dezheng (William)

1 Introduction

Audiovisual text (hereinafter AVT) is defined by Chaume (2004, 16) as “a semiotic construct comprising several signifying codes that operate simultaneously in the production of meaning” and is regarded as one of the most prominent polysemiotic text types. Subtitling, an indispensable part of AVT study, is defined by Díaz Cintas and Remael (2007, 8) as consisting of the presentation of

a written text, generally on the lower part of the screen, that endeavors to recount the original dialogue of the speakers, as well as the discursive elements that appear in the image (letters, inserts, graffiti, inscriptions, placards and the like), and the information that is contained on the soundtrack.

Over the past two decades, scholars (Zabalbeascoa 1996; Chuang 2006) have studied film translation from a polysemiotic perspective and claimed that “no text can be made entirely of verbal signs because such signs always need some sort of physical support” (Zabalbeascoa 1996, 338). Other scholars have started to explore the interaction between verbal and non-verbal elements in subtitle translation (Baldry and Thibault 2006; Gottlieb 2008; Kourdis 2015). However, up to now, scholarly attention in translation studies has mainly concentrated on verbal texts while the function of non-verbal elements such as intonation, layout, body language, and facial expression has been largely unnoticed (Zabalbeascoa 2008, 24). As Perez-Gonzalez (2014, 185) observes, “a large proportion of research efforts in audiovisual translation still revolve around elaborating taxonomies of different types of equivalence between short, decontextualized stretches of dialogue in the source and target language.”

Although research from the polysemiotic approach started decades ago, it is noticed that the aims of previous studies were rather general and the approaches were lack of theoretical support (Yu and Song 2017, 604; Lee 2018, 58; EI-Farahaty 2018, 37). When it comes to the study of AVT translation between Chinese and English, very few scholarly works could be found. A notable study is Chuang, who noted that “different semiotic modes contribute different kinds of meanings to the film text” (2006, 381). But the author did not give a thorough theoretical account of how those semiotic modes affect the Chinese-English translation. In order to provide further understanding about how the visual track impacts translation choices, this study proposes a theoretical framework to investigate Chinese-English AVT translation and applies it to the analysis of a Chinese costume drama.

Costume dramas, normally setting against historical background and unique conventions in ancient China, have always played an important role in the Chinese TV industry and social media. Recent years have witnessed the export of several popular costume dramas abroad such as Zhenhuan Zhuan (Empresses in the Palace), Yanxi Gong Lue (Story of Yanxi Palace), and Lang Ya Bang (Nirvana in Fire). They have attracted the attention of overseas audiences.

This chapter investigates the English translation of the Chinese costume drama Zhenhuan Zhuan by adopting a multimodal discourse analysis approach. Being the first costume drama of its kind officially exported overseas, it has aroused great interest from an international audience (Liu and Zhang 2017, 64). This drama is about the growth of the girl Zhenhuan in the harem of the Qing Dynasty, transforming from an innocent girl to a sophisticated empress dowager. Netflix, the largest media service provider in the United States, released its English-language version in 2014. Due to its high recognition, and the intersemiotic nature and translation quality assured by Netflix, this costume drama serves as good material for translation studies. In recent years, scholars have carried out research on the English translation of this drama. Shi and Fu (2016) took a practical approach and focused on the losses and gains in translating addressing terms and suggested several translation techniques like direct translation and addition to produce more effective translation products. Taking a sociological approach, Xiong (2016) discussed possible reasons for shortening the originally rather long drama into six episodes and commented positively on the translation provided by Netflix. However, very little research has been done on the abundant non-verbal modes of this drama, and even less in relation to translation. It is hoped that by investigating the interaction between verbal and non-verbal modes, this study will be able to systematically map out the intersemiotic shifts in this TV drama and examine the role of non-verbal modes in the translation process. Specific research questions of this study are as follows: (1) What intersemiotic shifts could be observed in the subtitle translation of Zhenhuan Zhuan? (2) Why do the shifts happen, and what are the roles of the non-verbal modes?

2 Multimodality: a networked system of choices

Multimodality is defined as “the use of several semiotic modes in the design of a semiotic product or event” (Kress and van Leeuwen 2001, 20). Regarded as an approach to investigate the integration between different modal elements, it can be applied to study various kinds of discourse such as discourses of advertisements, websites, museum exhibitions, textbooks, and comics. The term “mode” is an essential notion, which is defined as “a socially shaped and culturally given resource for making meaning” (Kress 2009, 54).

In the field of AVT, Stöckl is one of the first few scholars to have noticed the vital importance of non-verbal resources and integrated multimodal analysis in translation studies. He thinks that multimodality can be modelled “as a networked system of choices” (quoted in Gonzalez 2014, 191). In order to instantiate the meaning potential of an AVT, various specific modes could be chosen for better communication. He puts forward a specific framework to systemize different semi-otic resources, in which there are four core modes – SOUND, MUSIC, IMAGE, and LANGUAGE – and the core modes “need to be instantiated in a specific medial variant” (Stöckl 2004, 14).

To begin with, according to Stöckl (ibid.), each core mode consists of medial variants and is further realized through sub-modes. LANGUAGE, for example, has verbal signifiers such as lexico-grammar and syntax and three medial variants, which are speech para-verbal means, static writing, and animated writing. The medial variants, on the other hand, can be instantiated via volume, intonation, layout, font, and so forth. Speech para-verbal means help to realize LANGUAGE via manners related to sound or hearing. Bosseaux (2008) has realized that the voice, as the combination of prosodic features and phonetic markers of linguistic variation, has been acknowledged as central to perceptions of filmic performance and dramatic characterization. Static and animated writing help to realize LANGUAGE visually.

The core mode IMAGE is related to visual elements. Stöckl refers to the bound-togetherness of visual and verbal elements in AVT as the “language-image-link” (2004, 21). Perez-Gonzalez (2014, 191) argues that normally verbal and visual elements can add to a common mental image, which facilitates the audience’s understanding of the multimodal artifact. However, it does not always hold across languages due to linguistic and cultural differences. This actually has challenged professional translators, and solutions have to be found for a better cross-cultural communication. IMAGE includes static (still) images, whose sub-modes are color, lighting, and so forth; and dynamic (moving) images. For the latter, the sub-modes refer to camera panning, body language, and so on. In our study, non-verbal modes mainly refer to visual modes (IMAGE) and para-verbal means (Perez-Gonzalez 2014, 198) such as volume, intonation, and speed. The medial variants proposed in LANGUAGE and IMAGE will be applied to identifying translation cases involving the synergy of these semiotic resources.

3 Analytical framework

To understand how the verbal and non-verbal modes work together to contribute to the translation, theories from multimodal discourse analysis and translation studies will be drawn upon. An intersemiotic shifts model facilitating multimodal costume drama analysis will be adopted (Delabastita 1990; Lambert and Delabastita 1996).

Delabastita (1990) believes that audiovisual texts can transfer meaning via the combination of signs related with acoustic and visual modes: verbal (consisting of linguistic and para-linguistic signs), narrative, vestimentary, moral and cinematic (quoted in Perez-Gonzalez 2014, 114). Based on classical rhetoric, he proposed that semiotic shifts across codes cover three categories; Table 1.1 is a summary of the model.

Table 1.1 Semiotic shifts across codes (adapted from Delabastita 1990)

Semiotic shift	Description
Adiectio (Addition)	The translation of the source text involves the incorporation of additional signs, whether they are "new images, sounds, dialogue or spoken comments."
Detractio (Omission)	The translation results in a reduction of the verbal and non-verbal semiotics deployed in the source text.
Substitutio (Substitution)	Replacing one sign with a (more or less) equivalent one from a different code. It may involve a partial substitution of visual and nonverbal signs by verbal signs conveyed through the visual channel.

We argue that the model proposed by Delabastita, which consists of three kinds of intersemiotic shifts, is not sufficient to describe the translation process involving different kinds of modes. More specific subcategories of shifts and more methods are needed. Based on our preliminary categorization of the data and careful assessment of the diversified intersemiotic shifts in these cases, the following i...

Table of contents

• Cover
• Half Title
• Series Page
• Title Page
• Copyright Page
• Contents
• List of figures
• List of tables
• List of contributors
• Acknowledgements
• Introduction: multimodal approaches to Chinese-English translation and interpreting
• 1 Intersemiotic shifts in the translation of Chinese costume drama subtitles: a multimodal analysis approach
• 2 A multimodal study of paratexts in bilingual picturebooks on Mulan
• 3 Intersemiotic translation of rhetorical figures: a case study of the multimodal translation of The Art of War
• 4 Reshaping the heroic image of Monkey King via multimodality: a hero is back
• 5 “Dis”covering Hamlet in China: a case analysis of book covers of the Chinese Hamlet
• 6 Belt and Road Initiatives in texts and images: a critical perspective on intersemiotic translation of metaphors
• 7 A corpus-assisted multimodal approach to tourism promotional materials of Macao: a case study of three signature events
• 8 Effects of non-verbal paralanguage capturing on meaning transfer in consecutive interpreting
• Index