Cinesonica

Sounding film and video

Andy Birtwistle

• Publisher: Manchester University Press
• Year: 2021
• Print ISBN: 9780719095481
• eBook ISBN: 9781526162816
• Topic: Media & Performing Arts
• Subtopic: Film History & Criticism

Cinesonica: sounding film and video explores previously neglected and under-theorised aspects of film and video sound, drawing on detailed case study analyses of Hollywood cinema, art cinema, animated cartoons, and avant-garde film and video. Adopting an interdisciplinary approach to the study of the soundtrack, and breaking away from the focus on narrative and signification that has tended to dominate the study of film sound, the book examines the way in which sound's materiality figures within audiovisual experience. Through a close examination of sound-image relations in a range of film and video forms and genres – including Warner Bros. cartoons, scratch video, and artist's film and video – Cinesonica recasts the film and video text as the meeting point of audio and visual materialities, cultural practices and perceptual activity. The interdisciplinary approach adopted by the book makes its discussion of sound of interest to those studying and working in a range of subject disciplines, including film studies, sound studies, sonic arts, cultural studies, music and art history.

1

Introduction: sound, signification and materiality

This book attempts a coming-to-terms with what I propose are neglected aspects of film and video sound. This neglect, in one sense, manifests itself in the relative lack of critical literature on the sonic. This has been signalled elsewhere, and it is unnecessary for this critical absence to be rehearsed here, since the observation that much remains to be done on sound appears with regular frequency in the steadily growing body of literature on sound in general, and film sound in particular.¹ Rather, the issue addressed in my own study is exactly how the sonic dimensions of film and video might be auditioned, addressed, understood and discussed. The key concern of this book is to consider the ways in which we might come to terms with the materiality of film sound, both beyond and in relation to its semiotic or significatory dimensions, and what might be at stake in a critical engagement with this materiality.

Film sound merits study because it is an essential component of cinema. As a number of writers have demonstrated, cinema has always been audiovisual: before the general introduction of the recorded soundtrack in the late 1920s, the screening of films was accompanied by various forms of musical orchestration, narration, sound effects and performed dialogue.² Given the medium’s fun-damentally audiovisual nature, logic suggests that sound should make a substantial contribution to the effects and meanings created by cinema. However, the cultural bias towards the image has resulted in the mistaken belief that film is a visual rather than an audiovisual medium. As a consequence of this, sound has often been conceptualised as an add-on or supplement to the cinematic image, and the study of cinema has been, and continues to be, dominated by visual concerns. The challenge taken on by the first wave of modern film sound studies in the late 1970s and early 1980s was to address this cultural bias, and the resultant critical neglect that film sound had suffered. However, the body of literature that has developed since the early 1980s has now established the importance of the study of sound, and it is no longer as necessary as it once was to campaign for sound to be taken seriously. Rather, the task currently facing those working in film sound studies is to consider how thinking on the sonic dimensions of cinema might be developed further, and how areas of critical neglect within the discipline itself might be addressed.

While once there had been a fairly narrow focus on North American and European narrative cinema, the scope of film sound studies has now widened to include film and video emerging from other regional contexts, and an increasing variety of genres and forms. However, although the growing range of films being studied has served to develop understanding of film sound, the way in which the soundtrack has been conceptualised, and therefore the ways which it has been studied, have changed relatively little. Echoing the model of sound proposed by industrial processes of film production, writers tend to analyse the soundtrack according to its commonly accepted constituent elements – music, dialogue and sound effects – sometimes including silence, and quite often focusing exclusively on the role played by music. Unfortunately, in considering these elements of the soundtrack in isolation, the tendency is to neglect their interrelatedness, and more importantly, their changing relationships with one another and with the images on screen. Thus one of the challenges facing contemporary film sound studies is to engage with concepts, and to devise critical approaches, that allow for an increased consideration of the soundtrack as a whole.

Fundamental to the established approach to the soundtrack is the conceptualisation of film as a signifying text. Back in 1992 Rick Altman called for this dominant model of film to be replaced by the notion of cinema as event, thereby extending the range of critical discourse appropriate to the study of film sound (Altman, 1992: 2). While in the intervening period the work of Altman and others has provided alternatives to the formulation of film as text, this model nevertheless continues to serve as the dominant conceptual paradigm in the study of film sound aesthetics. The semiological project has had a profound influence on the landscape, and soundscape, of film studies, and while successive critical moves have distanced the study of film from the interventions made by the first wave of cinesemiology in the 1960s, the conception of film as signifying text has retained its influence on film theory and criticism.³

Although no longer completely commanding the centre ground of film studies, the complex of signification, meaning and representation that emerged from cinesemiology continues to serve as an important conceptual frame within which film or video are situated critically. Over the last three decades a concern with subjectivity has realigned the film text in relation to issues of reception and spectatorship, but nevertheless, what we term ‘the film’ seems to remain a clearly demarcated and unchanging entity – it remains a text that signifies. Concern with signification is no longer focused on the internal operations of the filmic text or the precise nature of the cinematic sign, as it was in the heyday of cinesemiology, but rather on the ways in which the signs contained within the film text relate to the various milieus in which they circulate: social, cultural, political, economic, psychological, historical and so forth. Despite the range of perspectives offered by the various strands of post-structural theory, film remains understood primarily as text to be read. The longevity of this formulation is, of course, testament to its usefulness.

There has undoubtedly been significant disenchantment with semiology itself as a means by which to engage with the significatory, and the body of post-structural theory can be seen both as a building on and a critique of structuralist modes of enquiry. However, since semiology had a profound impact on the development of the notion of film as text – a model that remains central to the way in which the soundtrack is conceptualised – it is still important to return to Saussure in order to identify what the limitations of a semiological modelling of sound might be.

The semiology of sound and the dematerialisation of the sign

Sound fares badly within Saussurian linguistics, constantly stripped from a project that privileges the seemingly stable, abstract, universal paradigm of language over individual concrete speech acts: langue in preference to parole. This approach to the study of signi-fication militates against engagement with concrete particularity, the material or the contingent. Thus, although the starting point for the Course in General Linguistics is a concern with speech – Saussure states, ‘in the lives of individuals and societies, speech is more important than anything else’ (Saussure, 1964: 7) – this concern is resolved through the model of language, which itself is identified as a ‘well-defined object in the heterogeneous mass of speech facts’ and therefore a ‘principle of classification’ (Saussure, 1964: 9). Saussurian linguistics consequently excludes the material and the contingent in favour of abstract paradigms, which as conceptual entities are necessarily divorced and absent from concrete phenomena, events and objects. Encountered within the conceptual frame of semiology, each individual act of signification is simply rendered a manifestation of an abstract paradigm.

A further distanciation from the sonic takes place when Saussure nominates writing as the means by which language can be successfully navigated: ‘we generally learn about language only through writing’ (Saussure, 1964: 23). In a telling use of imagery that figures chaos in terms of the oceanic, Saussure comments, ‘Whoever consciously deprives himself of the perceptible image of the written word runs the risk of perceiving only a shapeless and unmanageable mass. Taking away the written form is like depriving a beginning swimmer of his life belt’ (Saussure, 1964: 32). However, my primary concern with the Saussurian model of signification is not the fate of the sonic within a project that was obviously never concerned with it; rather, this brief return to Saussure serves to highlight a conceptual dynamic that hinders critical engagement with the concrete particularity of events, phenomena and objects situated within significatory networks, and circumscribes the discourse in which the materiality of sonic phenomena might be addressed. The conceptual mode proposed by Saussurian linguistics represses consideration of the material dimensions of the signifier beyond its ability to sustain difference, and thus to create or support meaning through negative differentiation. This is famously expressed in Saussure’s formulation, ‘in language there are only differences without positive terms’ (Saussure, 1964: 120). What this focus on difference means in terms of materiality is figured powerfully in an illustration given by Saussure in support of the synchronic study of language. In part, the Course in General Linguistics is motivated by Saussure’s objections to the diachronic study of language, which had dominated modern linguistics until this point. For Saussure, the study of language is served not by engaging with its evolution, but by removing it from a temporal frame: his interest is in the synchronic, the static, the science of language-states (états de langue). Accordingly he comments on language, ‘It is a system based on the mental opposition of auditory impressions, just as tapestry is a work of art produced by the visual oppositions of threads of different colours; the most important thing in analysis is the role of the oppositions, not the process through which the colours were obtained’ (Saussure, 1964: 33).

What this illustration demonstrates, beyond an obvious problematic dispensation with the temporal, is semiology’s inherent tendency to dematerialise the sign. Here there is no way to deal with the materiality of the signifier other than in terms of its ability to support and manifest difference; there is no discourse offered within which colour can be considered, other than its ability not to be other colours, and thus to differentiate itself in negative terms from others.

This dispensation with materiality takes its place within a broader conception of signification that renders both elements of the sign (signifier and signified) as psychological entities; the signifier is deemed by Saussure to be a ‘sound-image’ (the mental imprint of a sound), while the signified is a concept (Saussure, 1964: 66). Thus Saussure comments, ‘Everything in language is basically psychological, including its material and mechanical manifestations, such as sound changes’ (Saussure, 1964: 6). The clear inference here is that matter does not matter: the materiality of objects and events barely figures in a system founded primarily upon the notion of the arbitrary sign.⁴ Saussure makes this position clear when he states, ‘language is a form and not a substance’ (Saussure, 1964: 122). This dematerialisation of the sign must also be seen in relation to semiology’s privileging of the signified as the primary term of signification. While not dispensable, the signifier simply takes its place in the sign as that which supports the creation of meaning; the pay-off of signification is the concept that results, the sign’s terminal point.

One of the key problems of approaching film and video sound through the concept of signification is that it rather too neatly coincides with the way in which we casually formulate sounds in terms of the objects or events perceived as their source, describing sonic phenomena as the sound of something or other. The problem posed by this formulation is that it limits, to issues of representation, the ways in which we might come to terms with these sounds, while simultaneously ascribing to them a secondary status, situating them at the level of attribute, characteristic or effect. Thus an object represented on screen, perceived to be a sound’s source, seems to ‘explain’ the sound we hear, and if that source is absent from the screen, then sound seems to be explained by the fact that it signifies that source in its (visually unrepresented) absence.⁵

One of the concerns that emerges from this return to Saussure, and one of the key issues that dominates this book, is the question of how we might begin to come to terms with the materiality of film sound. Running alongside the significatory is a parallel universe of materiality, with ways of knowing sound, and ways of registering sonic presences that have little or nothing to do with the attribu-tion of meaning, and which cannot be understood through those reading techniques founded upon semiological models. How, then, can these sounds be mapped and negotiated beyond, and as well as in relation, to their significatory dimensions?

Affect and materiality

The work of a number of contemporary theorists suggests ways in which a non-significatory registration of sonic phenomena might be considered critically. Studies undertaken by Vivian Sobchack, Laura U. Marks and Brian Massumi have been formulated in ways that seek to engage with the sensory, both in relation to the significatory and as distinct from it. Vivian Sobchack’s work on film has drawn on phenomenology to explore the relationships between bodily experience and contemporary moving-i mage culture, and considers film in terms of the ways in which it represents and re-articulates our experiences of embodiment and vision (Sobchack, 1992, 2004). Laura U. Marks’s research on what she has termed ‘haptic visuality’ has considered how film signifies through its materiality, in ways that suggest the tactility of vision. In this way, argues Marks, film is able to evoke the sense of touch, smell and bodily presence (Marks, 1998, 2000). Negotiating intercultural cinema through the context of embodiment and the senses, Marks’s The Skin of the Film explores what can be represented by this form of transsensoriality, proposing that filmmakers working between cultures can engage the sensory to convey cultural experience and memory. And in Brian Massumi’s Parables for the Virtual (2002), movement, affect and sensation provide the key terms of reference for an examination of bodily and cultural processes that operate through multiple registers of sensation.

The increased interest in perception, embodiment and the senses has been seen by some as part of a broader ‘affective turn’ in literary and cultural studies. While the nature, magnitude and value of this ‘turn’ are hotly debated, the growing body of critical literature which explores notions of affect undoubtedly marks a departure from the text-based models of analysis that emerged from the ‘linguistic turn’ of the 1960s. What this brings to the study of film is an opportunity to examine, within a cultural framework, the spectator’s non-cognitive responses to the medium. Since the late 1990s, the interest in affect has been marked within film studies by the sheer number of books that have re-articulated or drawn upon the work of Gilles Deleuze and Felix Guattari, e.g., Ronald Bogue’s Deleuze on Cinema (2003), Barbara M. Kennedy’s Deleuze and Cinema: The Aesthetics of Sensation (2002), D. N. Rodowick’s Gilles Deleuze’s Time Machine (1997), Gregory Flaxman’s The Brain is the Screen: Deleuze and the Philosophy of Cinema (2000), Patricia Pisters’s Micropolitics of Media Culture: Reading the Rhizomes of Deleuze and Guattari (2001) and The Matrix of Visual Culture: Working with Deleuze in Film Theory (2003), Anna Powell’s Deleuze and Horror Film (2005), Elena Del Rio’s Deleuze and the Cinemas of Performance: Powers of Affection (2008...

Table of contents

• Cover
• Title
• Copyright
• Contents
• List of tables and figures
• Preface
• Acknowledgements
• 1 Introduction: sound, signification and materiality
• 2 Sound Source Object
• 3 The sound of technology
• 4 Strange sounds
• 5 Mickey-mousing
• 6 Organised sound
• References
• Selected filmography and videography
• Selected audio recordings
• Selected film and video distribution sources
• Index