Seeing the Past with Computers : Experiments with Augmented Reality and Computer Vision for History

Timothy J Compeau, Kevin Kee

• Publisher: University of Michigan Press
• Year: 2019
• eBook ISBN: 9780472900879
• Topic: Computer Science
• Subtopic: Computer Science General
• Index: Computer Science

Recent developments in computer technology are providing historians with new ways to see—and seek to hear, touch, or smell—traces of the past. Place-based augmented reality applications are an increasingly common feature at heritage sites and museums, allowing historians to create immersive, multifaceted learning experiences. Now that computer vision can be directed at the past, research involving thousands of images can recreate lost or destroyed objects or environments, and discern patterns in vast datasets that could not be perceived by the naked eye.Seeing the Past with Computers is a collection of twelve thought-pieces on the current and potential uses of augmented reality and computer vision in historical research, teaching, and presentation. The experts gathered here reflect upon their experiences working with new technologies, share their ideas for best practices, and assess the implications of—and imagine future possibilities for—new methods of historical study. Among the experimental topics they explore are the use of augmented reality that empowers students to challenge the presentation of historical material in their textbooks; the application of seeing computers to unlock unusual cultural knowledge, such as the secrets of vaudevillian stage magic; hacking facial recognition technology to reveal victims of racism in a century-old Australian archive; and rebuilding the soundscape of an Iron Age village with aural augmented reality.This volume is a valuable resource for scholars and students of history and the digital humanities more broadly. It will inspire them to apply innovative methods to open new paths for conducting and sharing their own research.

Page 11 →Chapter 1

The People Inside

Tim Sherratt and Kate Bagnall

Our collection begins with an example of computer vision that cuts through time and bureaucratic opacity to help us meet real people from the past. Buried in thousands of files in the National Archives of Australia is evidence of the exclusionary “White Australia” policies of the nineteenth and twentieth centuries, which were intended to limit and discourage immigration by non-Europeans. Tim Sherratt and Kate Bagnall decided to see what would happen if they used a form of face-detection software made ubiquitous by modern surveillance systems and applied it to a security system of a century ago. What we get is a new way to see the government documents, not as a source of statistics but, Sherratt and Bagnall argue, as powerful evidence of the people affected by racism.

In October 1911, the Sydney Morning Herald reported a local businessman’s complaints about his treatment by the Australian Customs Department. Charles Yee Wing, “a merchant of some standing, held in high esteem by Europeans and Chinese alike,” was planning a short trip to China.1 He had applied to the department for a certificate that would allow him to re-enter Australia on his return but was annoyed when officials insisted that he be photographed “in various positions” to document his identity. A naturalized British subject, respectable family man, and long-term resident of Sydney, Charles Yee Wing objected to being treated “just like a criminal.”

Today we are accustomed to being identified by our image. Passports, driver’s licenses, student cards—we readily submit to being photographed for a variety of purposes, and we carry the images with us as proof that we Page 12 →are who we say we are. The propagation and use of these likenesses has changed with the development of computer vision technology. Individual images can be discovered, analyzed, and compared across populations. The primary instrument of control has moved from document to database.

We are historians interested in bureaucratic systems for identification and control, and the impact of digital access on our understanding of how they worked. Kate’s research explores the social and familial worlds of Chinese Australians, particularly those of mixed race, in the late nineteenth and early twentieth centuries. Tim is a hacker who uses digital technologies to open cultural collections to new forms of analysis and exploration. Together we have been focused on the vast collection of records generated by Australia’s efforts to restrict non-European migration in the first half of the twentieth century. Among these records, preserved in the National Archives of Australia, are photographs and archival fragments documenting the life of Charles Yee Wing and thousands of others.

Computer vision can easily be used to find and recognize faces. Such technologies are often associated with the needs of law enforcement and national security, with the continued extension of systems for the identification and control of individuals. The latest facial recognition algorithms share a lineage with the thousands of immigration documents held by the National Archives. But can we use new technologies of identification to reveal the old? This chapter discusses an attempt to use facial detection technology to see archival records differently. What happens when instead of files and documents, systems and procedures, we see the people inside?

White Australia

Charles Yee Wing had a point in complaining about his treatment by the Customs Department. A century ago using portrait photography and fingerprinting to identify individuals was still fairly new, and until the early twentieth century, the most common official use of these technologies was to identify and manage criminals. Similar to law enforcement agencies in England, Europe, and the United States, the police in New South Wales, where Charles Yee Wing had lived since 1877, kept photographic gaol description books from around 1870.2 The gaol description books placed “mug shots” of convicted criminals alongside biographic information and a physical description to identify and keep track of convicted criminals.3

By the 1890s Australia’s colonial governments extended the use of these identification technologies to monitor and control the movement of peoplePage 13 → across their boundaries, and these practices were continued on a national scale after Federation under the Immigration Restriction Act 1901. But not all travelers were treated equally under this new law. Passports in the modern sense were not introduced until later, during World War I, and this earlier regime targeted certain groups whose presence was seen to be at odds with white Australians’ vision for their young nation.4 Charles Yee Wing’s photographs identified his race as well as his face.

The Immigration Restriction Act remained in force, with amendments and a slight change of name, until 1958. The Act was the legislative backbone of what became known as the White Australia policy—a discriminatory system founded on the conviction that a strong and self-reliant Australia must, of necessity, be “white.”5 Yet the Act itself said nothing about “color” or “race.” It was, by design, a fairly inoffensive piece of bureaucratic machinery that empowered the Commonwealth to reject certain classes of immigrant, including convicted criminals, the physically or mentally ill, or those who were deemed morally unfit. The history of colonial cooperation and the movement to Federation told the real story, however, and debates surrounding the passage of the Act, both in Parliament and in the press, made the context explicit—“color” was crucial. In the words of Attorney-General and future Prime Minister Alfred Deakin in 1901, “The unity of Australia is nothing, if that does not imply a united race.”6

The practices of discrimination and exclusion at the heart of the Immigration Restriction Act were elaborated gradually through regulations, reviews, precedents, notes, and guidelines. Between 1902 and 1911, the head of the Department of External Affairs issued more than 400 circulars about immigration restriction to Customs staff,7 and while the Act may have fudged its racial dimensions, such advice to government officials did not. For example, one memorandum from 1936 plainly stated: “In pursuance of the ‘White Australia’ policy, the general practice is not to permit Asiatics or other coloured immigrants to enter Australia for the purpose of settling here permanently.”8

The principal instrument of exclusion under the Immigration Restriction Act was the innocuous-sounding Dictation Test. This test required an arriving passenger to write down a passage that was read to them in a European (later, any) language; failing the test meant deportation. To remove any misunderstanding of those administering it, the test’s role was explained in a confidential note to Customs officials: “It is intended that the Dictation Test shall be an absolute bar to admission.”9 While the Act itself was silent on the details, officers were informed that all “persons of Page 14 →coloured races” who were not otherwise exempted from the provisions of the Act would be subjected to the Dictation Test—and they would fail. The deterrent effect of the Dictation Test was striking. In 1902, 651 arriving travelers were tested but only 33 passed. In 1905, 107 were tested and just 3 passed. In 1913, 71 were tested and all failed.10 Ultimately people just stopped trying to come.

The Dictation Test was clearly effective at preventing new arrivals, but the system also had to accommodate the thousands of “colored” Australians who, like Charles Yee Wing, needed to be able to return home to Australia after traveling overseas. The first national census held in 1911 counted over 40,000 people of “non-European race” (not including Indigenous Australians) in the country, around 25,000 of whom were Chinese.11 On their return they needed to prove their right to land by convincing Customs officials of their identity and of the validity of their claim to Australian domicile. Neither long-time residents, naturalized British subjects, nor the Australian-born could take for granted their right to re-entry if they looked “Asiatic” or “coloured.” They needed a piece of paper to prove it (see figure 1.1).

Some relied on naturalization papers or Australian birth certificates as proof, but most traveled after having applied and paid for an official certificate that would exempt them from the operations of the Dictation Test. The form of these certificates changed over time. In the earliest years of the Act, nonwhite residents could be granted a Certificate of Domicile. In 1905 this was replaced by the Certificate Exempting from Dictation Test (CEDT). Starting in 1903 these documents included photographs and handprints (later thumbprints), as well as a physical description, biographical information, and travel details.12

Nonwhite residents had to obtain a new CEDT for every journey. Two copies of the certificates were made—the traveler carried one, while a duplicate was retained by the Customs Department at the port of departure. On return the two copies were compared, the identity of the bearer was scrutinized, and officials decided if the traveler could stay or if the traveler would be deported as a “prohibited immigrant.” Many thousands of these certificates have been preserved. A growing number have been digitized and are available online. With portrait photographs and inky black handprints, these certificates are visually compelling documents.

Page 15 →

Figure 1.1. Charles Yee Wing’s Certificate Exempting from Dictation Test from 1908, when he traveled from Sydney to Fiji. NAA: ST84/1, 1908/301–310.

Page 16 →Records

The bureaucratic record-keeping system that underpinned the Immigration Restriction Act is preserved within the National Archives of Australia. As well as the exemption certificates, there are policy documents, departmental correspondence, case files, naturalization and birth records, reference letters, application forms, police reports, registers, indexes, and more. It is somewhat ironic that the records left by the bureaucracy of White Australia, an ideology that sought to marginalize and even deny the existence of “nonwhite” Australians, in fact document their lives in considerable detail and provide tangible evidence of Australia’s multiracial past.

Around th...

Table of contents

• Cover
• Title Page
• Copyright Page
• Contents
• Introduction
• Chapter 1. The People Inside
• Chapter 2. Bringing Trouvé to Light
• Chapter 3. Seeing Swinburne
• Chapter 4. Mixed-Reality Design for Broken-World Thinking
• Chapter 5. Faster than the Eye
• Chapter 6. The Analog Archive
• Chapter 7. Learning to See the Past at Scale
• Chapter 8. Building Augmented Reality Freedom Stories
• Chapter 9. Experiments in Alternative- and Augmented-Reality Game Design
• Chapter 10. Tecumseh Returns
• Chapter 11. History All Around Us
• Chapter 12. Hearing the Past
• Contributors
• Index