Music Navigation with Symbols and Layers

Toward Content Browsing with IEEE 1599 XML Encoding

Denis L. Baggi, Goffredo M. Haus

• Publisher: Wiley-IEEE Computer Society Pr
• Year: 2013
• Print ISBN: 9780470597163
• eBook ISBN: 9781118494448
• Edition: 1
• Topic: Computer Science
• Subtopic: Computer Science General
• Index: Computer Science

Music is much more than listening to audio encoded in some unreadable binary format. It is, instead, an adventure similar to reading a book and entering its world, complete with a story, plot, sound, images, texts, and plenty of related data with, for instance, historical, scientific, literary, and musicological contents. Navigation of this world, such as that of an opera, a jazz suite and jam session, a symphony, a piece from non-Western culture, is possible thanks to the specifications of new standard IEEE 1599, IEEE Recommended Practice for Defining a Commonly Acceptable Musical Application Using XML, which uses symbols in language XML and music layers to express all its multimedia characteristics. Because of its encompassing features, this standard allows the use of existing audio and video standards, as well as recuperation of material in some old format, the events of which are managed by a single XML file, which is human and machine readable - musical symbols have been read by humans for at least forty centuries.

Anyone wanting to realize a computer application using IEEE 1599 -- music and computer science departments, computer generated music research laboratories (e.g. CCRMA at Stanford, CNMAT at Berkeley, and IRCAM in Paris), music library conservationists, music industry frontrunners (Apple, TDK, Yamaha, Sony), etc. -- will need this first book-lengthexplanation of the newstandard as a reference.

The book will include a manual teaching how to encode music with IEEE 1599 as an appendix, plus a CD-R with a video demonstrating the applications described in the text and actual sample applications that the user can load onto his or her PC and experiment with.

1

The IEEE 1599 Standard

Denis L. Baggi and Goffredo M. Haus

Summary: This chapter gives a brief and simplified technical overview useful in the understanding of all IEEE 1599 applications, in particular from I to VII, which exist and are complete, and VIII and IX, which have been studied but not yet fully realized.

1.1 Introduction

IEEE 1599 is both a standard and a technology to represent and encode music, in all its aspects, making music enjoyment a total experience akin to that lived with literary masterpieces and great movies, well beyond the rendering of a binary file through a micro earphone.

Every music lover has always known that music is much more than audio and noise. The musical experience is similar to the act of entering a new world, enjoying a new experience, understanding a narration, and recognizing descriptions, as in the case of seriously reading a work of literature such as Tolstoy’s War and Peace or Shakespeare’s Hamlet and as in a parallel reality. In addition, music offers the possibility of investigating how the whole is built from its technical standpoint, which is the object of musicology, the science of music.

For such an experience, music must be represented with something that goes beyond unreadable, binary standards for audio, such as WAV and MP3, which are not music standards—they are audio standards. Musical aspects beyond audio must be represented in human-readable form, such as symbols and characters. This has always been the case for music scores in classical music, and for music notation in all civilizations for at least 40 centuries (e.g., Babylonian tablets; see the Introduction). This is also the case with other symbols, such as the harmonic grid in jazz, and also in other written codes, as in non-Western music.

This is also true for the new standard for music encoding IEEE 1599, which uses Extensible Markup Language (XML) clauses [XML]. This work represents the culmination of decades of efforts of specialists in the field of computer applications to music and musicology, of which the “Plaine and Easie Code” [Brook 1970], Digital Alternate Realization of Musical Symbols (DARMS) [Erickson 1975], and SMDL [Newcomb 1991] are worth mentioning, while MusicXML [Recordare] and the Music Encoding Initiative (MEI) [Roland] are the direct ancestors of this technology. However, IEEE 1599 goes beyond such past efforts, as described next.

The characteristics of IEEE 1599 are described in detail in Chapter 2; however, for the purpose of this chapter at least two important ones will be mentioned: symbols and layers.

Important Features of IEEE 1599

The main distinguishing features of the IEEE 1599 technology are the use of symbols to represent music, and the concept of layers.

Every element of Common Western Notation (see Chapter 2) can be represented by XML clauses that can be nested as needed, as in the example:

<clef type=“G” staff_step=“2” event_ref=“c1”/>

This is described in several pages of Document Type Definitions (DTDs) listed in Appendix B and posted at http://standards.ieee.org/downloads/1599/1599-2008/, a site of the IEEE Standards Association. In addition, thanks to the inherent extensibility of XML, it is possible to add clauses for special needs—such as proprietary characters used by a particular music publisher or notation that is not yet standardized [Cage 1969].

In recognition of the different aspects of music, the concept of layers has been introduced [Haus and Longari 2005] and is an integral feature of the standard, as shown in Figure 1.1.

Figure 1.1. The layers in IEEE 1599.

The general layer provides a general description of the music work and groups information about all related instances, including titles, author, type, number, date, genre, and related items. The logic layer provides music description from a symbolic point of view and represents the core of the format. It contains the main time-space construct for localization and synchronization of music events, the description of the score with symbols, and information about a graphical implementation of the symbolic contents, as well as the spine with Logically Organized Symbols (LOS), a sorted list of music events. Again, this is described in more detail in Chapter 2.

All layers are related to each other, and possibly synchronized, as shown in Figure 1.2, which shows that standard IEEE 1599 allows representation, performance, and audible and visual fruition of a piece of music independently of the original standard or format with which it was previously encoded. It therefore supports existing formats and recognizes that music contents that are already encoded in pre-existing file formats cannot be ignored. Since no overall description has ever existed for all aspects of music in one single format, it provides a meta-language in XML to describe all related music elements as well as to link all corresponding media objects already encoded. Thus, music contents are either newly encoded in the proper layers (the general, logic, and structural layers actually store information) or they remain in their original format, with links from the corresponding layers to files (the Notational, Performance, and Audio layers contain mappings to external files). Hence, media files are handled as they are, and media contents are still available in their original encoding. The comprehensive format described in the standard uses the layers to represent the relationships between music events and their occurrence in media files, thus allowing an overall synchronization. In other words, it is possible to navigate music in all its aspects.

Figure 1.2. Relationship among layers in IEEE 1599.

The interaction and synchronization among these layers will become clear through the following examples.

Examples of Applications of IEEE 1599 to Increase Music Enjoyment

The following are examples of applications of the standard, which can be realized easily thanks to the symbolic representation of music and the presence of synchronized layers, thus opening up a new way for music fruition.

• An opera. A DVD of an opera allows the user to see the play on the screen, to hear the music, to see the score (including manuscripts), to read the libretto, and to choose alternative renditions (see item 5 below).
• Pieces by a jazz band. The harmonic grid is displayed and the name of the soloist pops up at the beginning of each solo—with didactical tools as presented in jazz textbooks [Baggi 2001] (see item 2 below).
• Music with a “program” or story. For example, Vivaldi’s Four Seasons come with poems by the composer that refer to segments of the music (see item 8 below).
• Music with no apparent meaning. For instance, free jazz of the 1960s–70s is perceived by many as a random collection of meaningless sounds, while an associated video, generated anew each time, may help the listener understand what is meant (see item 2 below).
• A fugue. The theme is highlighted as it gets passed among the different voices.
• A piece of Indian classical music. The scale of the raga is shown and the melodic development is highlighted.
• A piece of several drums, for example, as in African drumming, shows how the rhythmic pulse come from the fact that the hits do not fall together.
• Preservation of the music heritage from the past. To store documents in any media [Haus].
• Musicological study. Ease of queries, for example, all pieces utilizing the lowest note of a grand piano, and questions as to why a certain note is used in a given harmonic context.
• Books about the making of a masterpiece, for example, Kind of Blue, by Miles Davis and John Coltrane [Kahn].

The following is a list of existing and planned realizations of IEEE 1599:

I. “King Porter Stomp,” by New Orleans pianist and composer Jelly Roll Morton (1889–1941), with versions played by the composer, two different scores, seven versions (of which one is a video from a movie) for solo piano and for orchestra, and 10 related images.

II. “Crazy Rhythm,” a jam session with no score and a harmonic grid, with Coleman Hawkins and Alix Combelle, tenor sax; Bennie Carter and André Ekyan, alto sax; Stéphane Grappelli, piano; Django Reinhardt, guitar; Eugène D’Hellemmes, bass; and Tommy Benford, drums. Recorded in Paris, April 27, 1937, record Swing #1.

III. Tosca, by Giacomo Puccini, 1858–1924, application realized on the original manuscript of 1900, courtesy of Ricordi. With three versions, including one video.

IV. “Peaches en Regalia,” by Frank Zappa, an example of control at the symbolic and structural levels.

V. “Il mio ben quando verrà,” from Giovanni Paisiello’s Nina, o sia la pazza per amore. It allows the user a choice of instrument, voice, versions of the score and of the libretto.

VI. Brandenburg Concerto No. 3, by J.S. ...

Table of contents

• Cover
• IEEE
• Title page
• Copyright page
• Illustration of IEEE 1599
• Preface
• A Brief Introduction to the IEEE 1599 Standard
• Contributors
• 1: The IEEE 1599 Standard
• 2: Encoding Music Information
• 3: Structuring Music Information
• 4: Modeling and Searching Music Collections
• 5: Feature Extraction and Synchronization among Layers
• 6: IEEE 1599 and Sound Synthesis
• 7: IEEE 1599 Applications for Entertainment and Education
• 8: Past Projects Using Symbols for Music
• Appendix A. Brief History of IEEE 1599 Standard, and Acknowledgments
• Appendix B. IEEE Document-Type Definitions (DTDs)
• Appendix C. IEEE 1599 Demonstration Videos
• Index