Audio-Vision: Sound on Screen

Michel Chion, Claudia Gorbman

• Publisher: Columbia University Press
• Year: 2019
• ISBN:
  9780231546379
• Topic: Médias et arts de la scène
• Subtopic: Histoire et critique du cinéma

Michel Chion's landmark Audio-Vision has exerted significant influence on our understanding of sound-image relations since its original publication in 1994. Chion argues that sound film qualitatively produces a new form of perception. Sound in audiovisual media does not merely complement images. Instead, the two channels together engage audio-vision, a special mode of perception that transforms both seeing and hearing. We don't see images and hear sounds separately—we audio-view a trans-sensory whole.

In this updated and expanded edition, Chion considers many additional examples from recent world cinema and formulates new questions for the contemporary media environment. He takes into account the evolving role of audio-vision in different theatrical environments, considering its significance for music videos, video art, commercial television, and the internet, as well as conventional cinema. Chion explores how multitrack digital sound enables astonishing detail, extending the space of the action and changing practices of scene construction. He demonstrates that speech is central to film and television and shows why "audio-logo-visual" is a more accurate term than "audiovisual." Audio-Vision shows us that sound is driving the creation of a sensory cinema.

This edition includes a glossary of terms, a chronology of several hundred significant films, and the original foreword by sound designer, editor, and Oscar honoree Walter Murch.

PART I

THE AUDIOVISUAL CONTRACT

1

PROJECTIONS OF SOUND ON IMAGE

The house lights go down, and the movie begins. Or at home or on a trip, we press PLAY. On the big or small screen, brutal and enigmatic images appear: a film projector running, a closeup of the film going through it, terrifying images of animal sacrifices, a nail being driven through a hand. Then, in more “normal” time, a mortuary. Here we see a young boy we take at first to be a corpse like the others, but who turns out to be alive—he moves, he reads a book, he reaches toward the screen surface, and under his hand there seems to form the face of a beautiful woman.

What we have seen so far is the prologue sequence of Bergman’s Persona (1967), a film that has been analyzed in books, in university courses, on internet sites. And the film might go on this way.

Stop! Let’s rewind Bergman’s film to the beginning and simply cut out the sound, try to forget what we’ve seen before, and watch the film afresh. Now we see something quite different.

First, the shot of the nail impaling the hand: played silent, it turns out to have consisted of three separate shots where we had seen one, because they had been linked by sound (figure 1.1). What’s more, the nailed hand in silence is abstract, whereas with sound, it is terrifying, real. As for the shots in the mortuary, without the sound of dripping water that connected them together we discover in them a series of stills, parts of isolated human bodies, out of space and time. And the boy’s right hand, without the vibrating tone that accompanies and structures its exploring gestures, no longer “forms” the face but just wanders aimlessly. The entire sequence has lost its rhythm and unity. Could Bergman be an overrated director? Did the sound merely conceal the images’ emptiness?

FIG. 1.1 Persona (1966). One of three shots of a nail hammered through a hand.

Next let us consider a well-known sequence in Tati’s Mr. Hulot’s Holiday (1953), where we laugh at the subtle gags taking place on a small beach (figure 1.2). The vacationers are so amusing in their uptightness, their lack of fun, their anxiety! This time, let’s cut out the visuals. Surprise: like the flipside of the image, another film appears that we now “see” with only our ears; there are shouts of children having fun, voices that resonate in an outdoor space, a whole world of play and vitality. It was all there in the sound, and at the same time it wasn’t.

Now if we give Bergman back his sounds and Tati his images, everything returns to normal. The nailed hand makes you sick to look at, the boy traces the shapes of faces, the summer vacationers seem quaint and droll, and sounds we didn’t especially hear when there was only sound emerge from the image like dialogue balloons in comics.

Only now we have read and heard differently.

Is the notion of cinema as the art of the image just an illusion? Of course: how, ultimately, can it be anything else? This book is about precisely this phenomenon of audiovisual illusion, an illusion located first and foremost in the heart of the most important of relations between sound and image, as I illustrated with Bergman: what I call added value.1

FIG. 1.2 Les vacances de Monsieur Hulot (1953). Suspicious looks onscreen, play and animated voices in the sound.

ADDED VALUE

By added value I mean the expressive and informative value with which a sound enriches a given image so as to create the definite impression, in the immediate or remembered experience one has of it, that this information or expression “naturally” comes from what is seen and is already contained in the image itself. Added value is what gives the (eminently incorrect) impression that sound is unnecessary, that sound merely duplicates a meaning that in reality it brings about, either all on its own or by discrepancies between it and the image.

The phenomenon of added value is especially at work in the case of sound/image synchronism, via the principle of synchresis (see chapter 3), the forging of an immediate and necessary relationship between something one sees and something one hears. Most falls, blows, and explosions on the screen, whether simulated or not, or created from the impact of nonresistant materials, only take on consistency and materiality through sound. But first, at the most basic level, added value is that of text, or language, on image.

Why speak of language so early on? Because the cinema is a vococentric, or, more precisely, a verbocentric phenomenon.

VALUE ADDED BY TEXT

Asserting that sound in the cinema is primarily vococentric is a reminder that it almost always privileges the voice, highlighting and setting the latter off from other sounds. During filming it is the voice that is collected in sound recording—which therefore is almost always voice recording—and it is the voice that is isolated in the sound mix like a solo instrument—for which the other sounds (music and noise) are merely the accompaniment.

I call vococentrism the process through which the voice spontaneously attracts and centers our attention in a mixture of sounds, in the same way that the human face directs our eyes in a movie shot. (Vococentrism can be diverted or attenuated by certain devices, to be discussed in chapter 8.) This is not to say that in classically vococentric films other sounds—noise and music—aren’t important, but they often act on a less conscious level.

By the same token, the historical development of synch sound-recording technology (for example, the invention of new kinds of microphones and sound systems) has concentrated essentially on speech, since of course we are not talking about the voice of shouts and moans but the voice as the medium of verbal expression. Thus what we mean by vococentrism is almost always verbocentrism.

Sound in film is voco- and verbocentric, above all, because human beings in their habitual behavior are as well. When in any given sound environment you hear voices, those voices capture and focus your attention before any other sound (wind blowing, music, traffic, a roomful of conversation). Only afterward, if you know very well who is speaking and what they’re talking about, might you turn your attention from the voices to the rest of the sounds you hear. So if these voices speak in an accessible language, you will first seek the meaning of the words, moving on to interpret the other sounds only when your interest in meaning has been satisfied.

If you are reading subtitles (which, in an effort to be concise and readable in a short time, cannot hope to match the original dialogue in style or completeness), they structure your vision, or rather your “audio-logo-vision.”

Subtitling plays an increasingly large part in film, for a variety of reasons. DVDs have menus that allow access to different languages, the internet circulates films throughout the world, and more and more films contain several languages that spectators need to distinguish in order to follow the story.

TEXT STRUCTURES VISION

An eloquent example that I used to use in teaching to demonstrate value added by text is a TV broadcast from 1984, a transmission of an air show in England, anchored from a French studio for French audiences by Léon Zitrone.2 Visibly thrown by these images coming to him over the wire with no explanation and in no special order, the valiant anchor nevertheless does his job as well as he can. At a certain point, he affirms, “Here are three small airplanes,” as we see an image with, yes, three little airplanes against a blue sky, and the outrageous redundancy never fails to provoke laughter in the classroom.

Zitrone could just as well have said, “The weather is magnificent today,” and that’s what we would have seen in the image, where there are in fact no clouds. Or: “The first two planes are ahead of the third,” and then everyone would have seen that. Or else: “Where did the fourth plane go?” and the fourth airplane’s absence, this plane pulled out of Zitrone’s hat by the sheer power of the Word, would have jumped to our eyes. In short, the anchor could have made fifty other “redundant” comments, but their redundancy is illusory, since in each case these statements would have guided and structured our vision so that we would have seen them “naturally” in the image.

The weakness of Chris Marker’s famous demonstration in his 1958 documentary Letter from Siberia, where he dubs voiceovers of different political persuasions (Stalinist, anticommunist, etc.) over the same sequence of innocuous images, is that through his exaggerated examples he leads us to believe that the issue is solely one of political ideology, and that otherwise there exists some neutral way of speaking. But the added value that words bring to the image goes far beyond the simple situation of a political opinion slapped onto images. Added value engages the very structuring of vision by rigorously framing it. In any case, the evanescent film image does not give us much time to look (until videocassettes and then DVDs and digital files did), unlike a painting on a wall or a photograph in a book, which we can explore at our own pace and more easily detach from their captions or their commentary.

Thus if the film or TV image seems to “speak” for itself, it is actually a ventriloquist’s speech. When the shot of the three small airplanes in a blue sky declares “three small airplanes,” it is a puppet animated by the anchorman’s voice.

VALUE ADDED BY MUSIC: EMPATHETIC AND ANEMPATHETIC EFFECTS

There are two primary ways for music in film to create a specific emotion in relation to the situation depicted on screen.3 On one hand, music can directly express its participation in the feeling of the scene by taking on the scene’s rhythm, tone, and phrasing; obviously such music participates in cultural codes for things like sadness, happiness, and movement. In this case we can speak of empathetic music, from the word “empathy,” the ability to feel the feelings of others. This is the well-known effect created by a kind of music that is, or appears to be, in harmony with the tone of the scene—dramatic, tragic, melancholic. It may have nothing at all to do with the music taken in itself and only be produced in the particular relationship between the music and the situation onscreen, in which it then has added value. For this reason, music, too, is not “redundant” in any way.

On the other hand, music can also exhibit conspicuous indifference to the situation, by progressing in a steady, undaunted, and ineluctable manner, like a written text or a machine that’s running: the scene takes place against this very backdrop of “indifference.” This juxtaposition of scene with indifferent music has the effect not of freezing emotion but rather of intensifying it, by inscribing it on a cosmic background. I call this second kind of music anempathetic (with the privative a-). The anempathetic impulse in the cinema produces those countless musical bits from player pianos, merry-go-rounds, music boxes, and dance bands, whose studied frivolity and naiveté reinforce the individual emotion of the characters and of the spectator even as the music pretends not to notice them.

To be sure, this effect of cosmic indifference was already present in opera, for example at the end of Bizet’s Carmen, when Don Jose stabs the heroine as we hear the joyous crowd cheering the toreador in the arena nearby. But on the screen, the anempathetic effect has taken on such prominence that it seems to bear an intimate relation to the essence of cinema, which is its (well-disguised) mechanical nature.

FIG. 1.3 Psycho (1960). Anempathetic sound of the shower, which continues after the murder.

For, indeed, all films proc...