Sound and Image

Aesthetics and Practices

Andrew Knight-Hill

• Publisher: Focal Press
• Year: 2020
• eBook ISBN: 9781000069945
• Edition: 1
• Topic: Computer Science
• Subtopic: Film & Video
• Index: Computer Science

Sound and Image: Aesthetics and Practices brings together international artist scholars to explore diverse sound and image practices, applying critical perspectives to interrogate and evaluate both the aesthetics and practices that underpin the audiovisual.

Contributions draw upon established discourses in electroacoustic music, media art history, film studies, critical theory and dance; framing and critiquing these arguments within the context of diverse audiovisual practices. The volume's interdisciplinary perspective contributes to the rich and evolving dialogue surrounding the audiovisual, demonstrating the value and significance of practice-informed theory, and theory derived from practice. The ideas and approaches explored within this book will find application in a wide range of contexts across the whole scope of audiovisuality, from visual music and experimental film, to narrative film and documentary, to live performance, sound design and into sonic art and electroacoustic music.

This book is ideal for artists, composers and researchers investigating theoretical positions and compositional practices which bring together sound and image.

1
Connected media, connected idioms

The relationship between video and electroacoustic music from a composer’s perspective

Diego Garro

Editor’s note

This chapter was originally presented at various international conferences between 2006–2008, but was never formally published. Its inclusion within this volume provides a permanent published record of the perspectives and ideas within and reflects the significant influence that the author had upon the editor as an undergraduate student, introducing him to the audiovisual.

This chapter is where this book began.

Introduction

As the enabling technologies for creative work with digital audio now reside on the same computers as those used for video applications, composers have the, relatively affordable, opportunity to ‘connect’ these two digital-media in their creative endeavors. Yet, despite the facilitating role of software interfaces and the inheritance of past experiences in mapping musical gestures to the moving image, the inherent difficulty of an interconnected audiovisual ‘idiom’ soon becomes apparent. The exposition and development of such an idiom represents a stimulating, yet daunting, challenge for both audiences and composers.

Electroacoustic music communities have treasured and pioneered technological advances in electronic and digital media tools. Sound and Image practices may find benefit in seeking to extend into the audiovisual domain the powerful and distinctive traits of a form of art that originally based itself, historically, culturally and aesthetically, on the primacy of the ear.

Strategies adopted by a new breed of – sometimes self-taught – audiovisual composers are informed by their experiences as Electroacoustic composers, sound designers and sound artists, but their very actions also throw the acousmatic paradigm into question. We will leave the darkness of the concert room behind us and reflect upon works articulated through the combination of shifting audible and visible morphologies.

Although this topic can be approached from the viewpoint of the connection between the two ‘physical’ digital media, it becomes apparent that simply connecting two media opens up several interesting questions, not only on the techniques and technologies, but, especially, questions on the ‘connected idioms’, notably how the sonic language and the language of the moving image relate to each other, and what (if any) new combined, integrated idiom they contribute to form. The authors position as a composer ensures that the following discussion is directed and informed by applied experience in practice.

A new trend has emerged within the electoacoustic community during the last twenty years: the combination of Electroacoustic Music soundtracks with video material to form audiovisual works in which the sound and images are more or less equally important. To illustrate the growing status of audiovisual composition within the Electroacoustic Music community I will cite a few historical precedents:

• The International Electroacoustic Music and Sonic Art Competitions of Bourges (www.imeb.net): this competition was founded in 1973 by Françoise Barrière and Christian Clozier and has been for many years one of the most important competitions in this field, certainly in Europe, attracting approximately 200 participants every year from 30 different countries. In the year 2001 for the first time the Bourges competition featured a category for multi-media works.
• The Computer Music Journal (MIT press), published since 1977, covers a wide range of topics related to digital audio signal processing and Electroacoustic Music. The first Video Anthology DVD, containing video works, was published in Winter 2003 as an accompanying disc to issue Vol. 27, Number 4 of the journal.
• The Computer Music Journal issue Vol. 29, Number 4, Winter 2005, was entirely dedicated to topics relevant to Visual Music. This issue also included a second Video Anthology DVD, with selected video works and video examples.
• The biannual Seeing Sound Conference/Festival, hosted by Joseph Hyde at Bath Spa University since 2009. An informal practice-led symposium exploring multimedia work which foregrounds the relationship between sound and image; exploring areas such as visual music, abstract cinema, experimental animation, audiovisual performance and installation practice through paper sessions, screenings, performances and installations, bringing together international artists and thinkers to discuss their work and the aesthetics of audiovisual practice.
• SOUND/IMAGE conference/festival hosted by Andrew Knight-Hill in Greenwich, running annually since 2015, bringing together international practitioners to share concepts and creative approaches to audiovisual composition in concert with exhibitions, screenings and performances. Colliding the worlds of experimental filmmaking and multichannel eletroacoustic composition.

Electroacoustic Music

For the sake of the ensuing discussion we should come, if at all possible, to an agreement of what constitutes ‘Electroacoustic Music’. The reader can refer to many recent writings on the subject as well as a critical review of at least some seminal works from the repertoire (see, for example, Knight-Hill 2020). But in the immediate context, we can take a giant leap forward to consider the following definition:

Electroacoustic Music is a form of art that is concerned with the technologically aided exploration of ‘sound’ in all its phenomenological aspects, and with the organisation of sonic material in time.

This definition follows closely Edgard Varèse’s idea of Music as ‘organised sounds’. Another definition could be:

The structuring and the articulation of time with the materials of sound patterns, which is both technically enabled and aesthetically informed by the capabilities of electro-acoustic transducers (microphones, loudspeakers) and by the opportunity to access and manipulate sound spectra by means of electronic analog and digital technologies.

(expanded from a definition of ‘Absolute Music’ by Brian Evans 2005)

Both these definitions are obviously incomplete, and they over-simplify a form of art which is complex, still evolving, and possibly still trying to define itself amongst the same tidal of globalised fragmentation that characterises all modern electronic arts (and perhaps all human endeavours).

Nevertheless, these definitions give us at least a starting point and some basic principles. More discursively, the concerns of Electroacoustic Music can be identified as follows:

• It has a concern with the ‘discovery’ of sound (recording, synthesis, processing).
• It is a time-based form of art, hence concerned with the evolution and organisation of sounds in time. This raises typical issues of all time-based media such as structure, balance, articulation, progression, direction, etc.
• It uses technology to ‘augment’ composers’ and performers’ control of sound material and audiences’ experience of sound stimuli in an artistic context.
• It assumes the ‘primacy of the ear’ promoted by members of the French school of Musique Concrète (Pierre Schaeffer and others) in the 1950s. This is a cultural attitude both for composers and for listeners. For composers, because they use their hearing ability as primary informant when they ‘compose the sound’ in the studio and when they ‘compose with sound’. For listeners, because in most cases they rely only on their hearing ability (no visual clues) to understand the artistic message in the music. The primacy of the ear is at the core of what we may label the ‘acousmatic paradigm’ of Electroacoustic Music culture.
• It draws together creators and audiences who share a deep fascination with the material of the acoustic space surrounding all of us. Not just the ‘music’ but, rather the ‘sound of it’.

These elements are important because they give us some clues on how to relate to the body of works for electroacoustic sounds and video discussed in this chapter.

The art of visible light – Visual Music

Electroacoustic Music developed alongside innovations and progress in audio technology, the realisation of a quest for a medium of artistic expression that utilised sounds freed from the expectations and cultural references of ‘music’, as we knew it, until the beginning of the 20th century. We could trace a similar path in the development of a non-narrative language of the moving image. In Germany, for example, pioneering film-makers such as Walter Ruttman, Viking Eggeling and Hans Richter developed a type of experimental cinema that articulated abstract shapes moving over time. Oskar Fischinger worked for over 30 years on this type of filmic language and is considered by many the father ofthis type of filmmaking, choreographing abstraction in connection with musical form (Evans 2005).

Experimental cinema of this kind is considered innovative, but was in effect a remediated version of practices that date back a couple of centuries further in time, to the pioneers of ‘Visual Music’.

Visual Music can be defined as time-based visual imagery that establishes a temporal architecture in a way similar to ‘absolute’ music. It is typically non-narrative and non-representational (although it need not be either). Visual Music can be accompanied by sound but can also be silent.

(Evans 2005)

Visual Music precedes even film. Early examples of gas-lamp Colour Organs date back to the 18th century. Colour organs were instruments that projected coloured light under the control of an organ-like keyboard and were used to provide a visual accompaniment to music performances (Peacock 1988). Such instruments became increasingly sophisticated, in terms of technology, control and visual sophistication, especially with the advent of electricity, but they responded to the same aesthetic quest as their predecessors. Interestingly, there has been a resurgence of interest in Visual Music with exhibitions, screenings and museum galleries in major cities in North America and Europe (see www.iotacenter.org and www.centerforvisualmusic.org for information and catalogues of works in the field, now available on DVDs¹).

Convergences

Media art histories of recent years may depict a parallel development of languages of sonic arts, with Electroacoustic Music at the forefront of this movement, and languages of the moving image with Visual Music and experimental non-narrative cinematography on the other side. We can see such a parallel development as an anticipation of an encounter between these disciplines, facilitated by certain key convergences:

Convergence in the type of media

Both the ‘art of sound’ and the ‘art of visible light’ are time-based media. They engage the viewer/listener in a revisited and augmented experience of chronometric time, accomplished through articulation of their time-varying stimuli (audible and visible respectively).

Technological convergence

Both the ‘art of sound’ and the ‘art of visible light’ can inhabit the digital domain, hence their materials can be stored and manipulated by computers. Nowadays the same relatively ‘inexpensive’ desktop computer and even laptops can handle digital audio and, with more difficulty, digital video data. In the solitary reclusion of the music project studio, enabling tools for digital audio-video experimentation reside in the same workstations used for modern computer-music endeavours. It is an opportunity that has been staring at Electroacoustic Music composers for the last 10 years.

Artistic and idiomatic convergences

Both the ‘art of sound’ and the ‘art of visible light’ developed a language that is very experimental, often abstract and breaks away from historically established forms (classical music, figurative art), concentrating on the exploration of the basic matter in the respective media and in the construction of temporal articulations of that matter for artistic purpose (see, for example, Whitney 1960).

Mapping between sound and images

Strategies to combine sound and images have been the subject of study and experimentation for centuries, from Isaac Newton and his corre...

Table of contents

• Cover
• Half Title
• Series Page
• Title Page
• Copyright Page
• Dedication Page
• Contents
• List of contributors
• Preface
• 1 Connected media, connected idioms: the relationship between video and electroacoustic music from a composer’s perspective
• 2 Sound/image relations in videomusic: a typological proposition
• 3 The question of form in visual music
• 4 Audiovisual spaces: spatiality, experience and potentiality in audiovisual composition
• 5 Rhythm as the intermediary of audiovisual fusions
• 6 The curious case of the plastic hair-comb: a rhythm-based approach to a parallel (sound-image-touch) theory of aesthetic practices
• 7 The spaces between gesture, sound and image
• 8 The gift of sound and vision: visual music as a form of glossolalic speech
• 9 Visual music and embodied visceral affect
• 10 The function of Mickey-Mousing: a re-assessment
• 11 Performing the real: audiovisual documentary performances and the senses
• 12 Blending image and music in Jim Jarmusch’s cinema
• 13 The new analogue: media archaeology as creative practice in 21st-century audiovisual art
• 14 Screen grammar for mobile frame media: the audiovisual language of cinematic virtual reality, case studies and analysis
• 15 Nature Morte: examining the sonic and visual potential of a 16mm film
• 16 Capturing movement: a videomusical approach sourced in the natural environment
• 17 Constructing visual music images with electroacoustic music concepts
• 18 Technique and audiovisual counterpoint in the Estuaries series
• 19 Exploring Expanded Audiovisual Formats (EAFs) – a practitioner’s perspective
• 20 Making a motion score: a graphical and genealogical inquiry into a multi-screen cinegraphy
• 21 The human body as an audiovisual instrument
• 22 Sound – [object] – dance: a holistic approach to interdisciplinary composition
• 23 Son e(s)t Lumière: expanding notions of composition, transcription and tangibility through creative sonification of digital images
• 24 Audiovisual heterophony: a musical reading of Walter Ruttmann’s film Lichtspiel Opus 3 (1924)
• Index