Indexing Multimedia and Creative Works

The Problems of Meaning and Interpretation

Pauline Rafferty, Rob Hidderley

• Publisher: Routledge
• Year: 2017
• Print ISBN:
  9780754632542
• eBook ISBN:
  9781351928120
• Topic: Business
• Subtopic: Business Strategy
• Index: Business

Indexing and information retrieval work properly only if language and interpretation are shared by creator and user. This is more complex for non-verbal media. The authors of Indexing Multimedia and Creative Works explore these challenges against a background of different theories of language and communication, particularly semiotics, questioning the possibility of ideal multimedia indexing. After surveying traditional approaches to information retrieval (IR) and organization in relation to issues of meaning, particularly Panofsky's 'levels of meaning', Pauline Rafferty and Rob Hidderley weigh up the effectiveness of major IR tools (cataloguing, classification and indexing) and computerised IR, highlighting key questions raised by state-of-the-art computer language processing systems. Introducing the reader to the fundamentals of semiotics, through the thinking of Saussure, Peirce and Sonesson, they make the case for this as the basis for successful multimedia information retrieval. The authors then describe specific multimedia information retrieval tools: namely the Art and Architecture Thesaurus, Iconclass and the Library of Congress Thesaurus of General Materials I and II. A selection of multimedia objects including photographic images, abstract images, music, the spoken word and film are read using analytical and descriptive categories derived from the literature of semiotics. Multimedia information retrieval tools are also used to index the multimedia objects, an exercise which demonstrates the richness of the semiotic approach and the limitations of controlled vocabulary systems. In the final chapter the authors reflect on the issues thrown up by this comparison and explore alternatives such as democratic, user-generated indexing as an alternative . Primarily intended for third-year undergraduate and postgraduate information studies students, the breadth and depth of Indexing Multimedia and Creative Works will also make it relevant and fascinating rea

1 Information, retrieval, discourse and communication

INTRODUCTION

This book is about the indexing of multimedia objects. In the context of this book, the term indexing refers to the representation, in surrogate form, of information about:

• the physical characteristics of the multimedia object (for example, descriptive cataloguing)
• the intellectual characteristics of the multimedia object (for example, subject indexing).

Indexing is generally considered a crucial prerequisite for any effective information retrieval system. We use the term multimedia objects here to refer to a range of media within which information is stored. In this context information is assumed to be material, or in Michael Buckland’s terms, information is treated as ‘information-as-thing’ (1991).

In this book multimedia information retrieval is information retrieval which is used to manage still images, moving images (with and without soundtracks) and sound recordings. The main focus of this book is on concept-based information retrieval which uses text to index and retrieve multimedia objects. Other forms of multimedia information retrieval will be described briefly to provide a perspective of the range of systems.

We believe that it is useful to situate multimedia indexing within a broader context. Our use of the term multimedia suggests that the objects with which we will be concerned are not textual but non-textual objects, yet many of the issues relevant to the indexing of multimedia objects, for example, subject analysis, are issues which are also relevant to textual objects. Our contention is that the issues are similar, but the nature of the communicative medium (for example, photography rather than written language) might result in additional difficulties in the indexing of non-textual objects.

We see indexing itself within the broader context of communication, and in turn, communication sitting within the broader context of human history and society. We believe that reaching an understanding of the issues involved in the indexing of multimedia objects is best done within a broader discussion of communication and culture, and the philosophies of communication and culture.

There are many approaches to multimedia indexing which we believe are limited in their scope because they are built on the assumption that communication is a straightforward process. In this book we hope to demonstrate that the issue is more complicated than this by drawing on models of communication and meaning from the domain of semiotics and communication studies as well as describing and critiquing information retrieval models drawn from the domains of information science and computer science. This first chapter provides the reader with an overview of the range of issues that we believe are of significance in multimedia indexing. Subsections in this chapter will examine the following topics:

• traditional approaches to information retrieval
• information retrieval and subject analysis
• information retrieval and discursive forms
• information retrieval and subjective discourse
• meanings, codes and conventions
• Erwin Panofsky and levels of meaning.

TRADITIONAL APPROACHES TO INFORMATION RETRIEVAL

For many years, librarians and information workers have designed and implemented information retrieval systems to help information seekers find exactly what they are looking for. Information retrieval involves the identification and representation of the information content of and about documents using descriptive and analytical systems which allow specific user requests for information to be matched up with the relevant information source(s). In the retrieval of textual documents the assumption is that the human indexer is able to decode the textual document and construct a representation of the significant information content using the codes and conventions of cataloguing rules and indexing languages, for example classification codes and controlled vocabularies.

Traditionally in libraries, information workers have focused on the document as the information-carrying vehicle. Documents, chiefly books, are organized in libraries on shelves according to subject through the application of appropriate notation derived from a library classification scheme. This practice allows books to be shelved in relative order and facilitates open access policies in libraries. Catalogues are constructed to record information relating to the physical aspects of the document (descriptive cataloguing, author/title access points), and the intellectual aspects of the document (classification marks as indexing entries, controlled vocabularies).

The traditional process of information storage and retrieval depends upon human indexers and library users making decisions relating to the interpretation of, and representation of, meaning in media objects and indexing languages. These decisions can lead to success or failure in the implementation of the system and/or the subsequent search procedure. The human indexer must accurately describe the physical properties of the document using the appropriate cataloguing rules, and then form access points for author and title searches. How straightforward this part of the process is depends on how easy it is to identify relevant information and transpose it into the form demanded by the cataloguing code.

To subject index the document, the human indexer must first analyse the subject(s) of the document being indexed, and then choose the subject terms from the controlled vocabulary which best represents the subject of the document (the controlled vocabulary will be a standard, published externally or in house, which is often chosen or constructed by someone other than the indexer). To do this, the human indexer must interpret the meaning of the document, and interpret the meaning of terms in the controlled vocabulary.

The library users who seek information can perform an author search or a title search on the catalogue if they already have this information, but if the search is for an unknown item about a particular subject, then the search will take the form of a subject search. Such a search strategy is only possible because it is generally accepted that documents have fairly stable meanings which can be interpreted by readers and can be represented in other forms (cataloguing records, classification marks).

The information seeker has to decide which term in the controlled vocabulary best represents the subject of the information sought, and then hope that the subject of the document as interpreted by the indexer accurately represents the subject of the document as interpreted by the seeker. Thus analysed, it becomes clear that the information retrieval process is a communicative process in which meaning and interpretation are carried out by at least two agents, the object of interpretation having been encoded by at least one other agent, but probably more, given contemporary publishing practices.

The indexing process (Figure 1.1) as it applies to textual objects is considered to be one which needs specialist knowledge to implement successfully. This process has become professionalized in contemporary society. There are a number of textbooks devoted to information retrieval in general (for example, Chowdhury, 1999; Rowley and Farrow, 2000), and to specific aspects of the process (for example, Hunter, 2002; Langridge, 1989; Piggott, 1988). Computerized information retrieval has made it possible to focus search strategies on specific objects within documents (for example, text retrieval databases which facilitate searches for specific articles within journals, and hypertexts structures which allow access to specific nodes of information), but the principles of the process (analysis – representation – retrieval) are essentially similar.

Figure 1.1 The subject indexing process

INFORMATION RETRIEVAL AND SUBJECT ANALYSIS

Conventional information retrieval systems work on the principle that the textual object has meaning which has been encoded within the object by the author(s), and which can be decoded by the reader/indexer. The decoded meaning is assumed to be stable and ‘naturally’ transparent. The same principle works in relation to the indexing language which is also assumed to be ‘naturally’ transparent. Textbooks of librarianship have in the past given students scant advice about how to decode meaning in documents, and about how to deal with the perils of interpretation that lie before them. Derek Langridge, in Subject Analysis (1989), attempted a detailed exploration of the difficulties of establishing meaning of documents, and acknowledged that reading and interpreting meaning could very often require something more than the mechanistic ‘technical reading’ that Wyner referred to in passing (Wyner, 1980, p. 18). Langridge included many examples of difficult-to-interpret texts to illustrate the potential problems of interpreting meaning and deciding on primary and secondary subjects. The approach he took was to analyse and interpret these texts, thus showing the reader/student how it should be done. What is missing in this approach is the recognition that readers can interpret documents in a variety of ways. It may be that Langridge’s interpretation would not be shared by all other readers who may bring with them different knowledge and different interpretative agendas.

Langridge held that the most obvious philosophical influence on indexing at the time that he was writing was the influence of philosophers whom he labelled ‘the positivists’. He argued that the positivist definition of knowledge is a very narrow one, being limited to science and excluding a large part of written records (Langridge, 1989, p. 20). There are other philosophers who take a broader view about what counts as knowledge, but even those ‘positivists’ who favour a limited use of the term offer other terms such as ‘modes of experience’ or ‘apprehension’ to describe those written records which they do not view as knowledge. Langridge’s point is that:

[t]he fact that the positivists distinguish between the sciences and all else actually supports the view that there are different forms of experience on record, whatever name we use. Since they themselves are only interested in science, their writings can be no help in deciding how many distinct forms there are. (Ibid., p. 21)

There is much more of a debate about whether humans can ever claim that knowledge possessed at any given moment is immutable and absolute than is articulated in Langridge’s treatment of the matter, and the similarities and differences between various epistemological positions in the modern world are not always as clear cut as he suggests. Langridge accepted that like Plato, the ‘positivists’ are concerned with distinguishing ‘true knowledge from belief’ (ibid., p. 22) rather than with the forms that either might take. This issue is of some interest to information management because the information manager must consider the problem of forms in relation to both knowledge and belief because there are ‘written records’ pertaining to both aspects of human culture. These records have to be organized because people may want to retrieve them whether they are ‘true knowledge’ or ‘belief’. Subject coverage in conventional library classification schemes tends to be knowledge-based so that even aesthetic information objects are analysed and described from knowledge-based perspectives. The question that Langridge does not ask but might be of interest to multimedia indexing is whether there are approaches other than the knowledge-based academic discipline orientations of conventional retrieval tools. Could pleasure, ideology, or intertextuality form the bases of new approaches to indexing?

Langridge’s own philosophical preference was for the idealist approach offered by R.G. Collingwood in Speculum Mentis or, The Map of Knowledge (1924). He believed that Collingwood’s approach, which involved viewing different disciplines in their own terms because they are all ‘rival ways of conceiving the whole’ (Langridge, 1989, p. 22), and separating disciplines according to their own rules, offered an alternative way for librarians to think about ‘knowledge’ (ibid.). The various ‘forms of knowledge’ are treated as logically distinct because they are ‘the ultimate classes of knowledge beyond which – we can make no further reduction’ (ibid.).

Langridge’s approach to meaning in documents seems to be based on the view that the meaning of the text resides entirely in the text; in other words, there is no recognition that individual interpretations of text might depend on the codes and conventions, the belief systems and world views that the individual reader at a specific historical juncture brings to the text. For Langridge, what the librarian indexer must do is to learn to read, or to decode, in the correct way in order to properly determine the meaning of any given document. Post-structuralist linguists and communication theorists might question whether the process is always quite so straightforward, and whether totally transparent meaning is to be found in all types of documents.

INFORMATION RETRIEVAL AND DISCURSIVE FORMS

Many of the traditional information and library studies textbooks of the 1960s and 1970s referred in passing to subject analysis. It may be that the relative neglect of subject analysis is due to the belief on the part of the writers that the subjects of documents are generally easy to determine and to interpret, but this is not necessarily the case for all documents at all times. There are many different types of discourse with different purposes and audiences, and while some types of discourse are constructed with transparency in mind, there are other documents constructed with other motivations. Moreover, discursive forms change over time. Decisions about how transparent or opaque the meaning of a document will be might be chosen freely by the author, or, as in the case of an author such as the imprisoned Italian communist writer, Antonio Gramsci (1971), whose sometimes cryptic writings, smuggled out of prison, were always in danger of falling into the hands of his enemies, such decisions might be imposed by circumstances.

Michel Foucault (1991), in essays such as ‘What is an Author?’ drew attention to the constructed and historically contingent nature of the communicative operations of authorship and textuality. In The Order of Things (1970), Foucault argued that each historical age has its own episteme, that is, its own system of knowledge constituted by recurrent patterns of signs and symbols, ways of speaking and seeing things, which are ascribed with the status of knowledge. The episteme determines what can be known at that moment. Like the notion of the scientific paradigm, each episteme is historically contingent and is dislodged from its position of power by the proceeding episteme. From this perspective, ‘discursive practice’ is a body of ‘anonymous, historical rules’ in and through which ways of seeing and writing and speaking are determined. For our purposes, the notion of determined discourse is of some interest in modelling types of text, which in turn is of interest in mapping textual transparency and opacity in social, political and institutional contexts.

In the contemporary modern world, the academy sets rules, codes and conventions about how scientific writings should be presented which privilege accuracy, proof, evidence, transparency and knowledge. This type of writing is motivated towards the reader, but is often difficult for lay readers to interpret because scholarly conventions demand that writers use professional language or jargon, and demonstrate their familiarity with, and knowledge of, writers within the scholarly tradition. Other types of writing, for example modernist and postmodernist fiction, and poetry, are not so heavily concerned with transparency; indeed, the authorial intention in such writings may be to obscure or complicate meaning: to create the self-consciously ambiguous text.

Linguists, philosophers and literary theorists have long been interested in identifying different types of writing (Figure 1.2). Roland Barthes distinguished between the ‘readerly’ and ‘writerly’ text by which he meant:

• ‘writerly texts’ – texts which invite the reader’s involvement in interpretation
• ‘readerly texts’ – texts in which the writer keeps tight reign on the range of possible interpretations. The text tends to be representational. The reader is positioned as a relatively passive receiver and the text tends towards ‘a’ meaning (Barthes, 1974).

Umberto Eco differentiated between the ‘open’ and the ‘closed’ text (Eco, 1979). The point is that different discourses allow the writer different levels of freedom in establishin...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Contents
• List of figures
• List of tables
• Foreword
• 1 Information, retrieval, discourse and communication
• 2 An overview of information retrieval
• 3 Multimedia information retrieval
• 4 Using semiotics to analyse multimedia objects
• 5 Using multimedia indexing tools
• 6 Research issues
• Bibliography
• Index