Communication in Vehicles

Cultural Variability in Speech Systems

Brion van Over, Ute Winter, Elizabeth Molina-Markham, Sunny Lie, Donal Carbaugh

• Publisher: De Gruyter
• Year: 2020
• Print ISBN: 9783110518917
• eBook ISBN: 9783110519150
• Edition: 1
• Topic: Computer Science
• Subtopic: Data Mining
• Index: Computer Science

New technology in vehicles is transforming the way people move around as well as what they do in their vehicles. How does one communicate with an in-car speech system and how does this vary by language or cultural community? This book explores this process by focusing on the communication practices that people engage in when using their in-car systems and when talking about their vehicles with co-passengers. Chapters present a robust theory and methodology for studying communication in cars, how tasks are begun and ended, how people switch between tasks, how non-task talk appears, what ways and styles of communication drivers prefer, and how they expect the system voice to respond, among other things. Particular attention is given to cultural preferences as they are evident in this communication; these preferences are found to ground various trajectories in the use and meaning of in-car communication practices. The book explores these matters with a focus on the United States and Mainland China. Implications are drawn for the design and utilization of in-car communication systems.

1Cultural Analyses of In-car Communication

1.1Introduction

When we sit with our laptop, phone, or television, we are involved in an interaction with communication technology. We have our own ways of thinking, our culturally based conceptions about that technology, about what it is, how it might be of use, and also, of course, about how we will indeed use it. This general arena of activity is called by some, the human−machine interface. This is of course a complex site of activity, and of study, as complexities of use lead to research reports with reports leading to the redesign of our technological devices or machines. This cycle of human use and research results in devices being the way that they are, and thereby sets the stage for end-users to use them in the complex and at times unexpected ways that we do.

One such device is being placed in the dashboard of cars or automotive vehicles generally. This sort of device allows drivers and passengers to adjust the temperature of the vehicle or the volume of speakers, to play a variety of radio stations, to play from personal music libraries, to make telephone calls, to navigate to destinations, and to conduct other such activities including the possibilities of messaging, conferencing, and other various forms of entertainment. Increasingly, multiple modes of interacting with such devices are being used. Earlier designs of these devices have relied mostly on push-button technologies. More often, now, a variety of modalities is integrated into a multimodal interface, such as touch screens or touch pads. When voice activation is used and when this is initially successful, for many tasks, users tend to like it.

The engineering of this technology for the dashboard of cars has been finely studied, as well as interactions between drivers and passengers in the car; however, the interaction between the people in the car and a dashboard device has been somewhat less studied, especially the cultural variety in its conception, use, and interpretation. For various reasons, among them the complexity of tasks and missing knowledge, a human−machine interface is developed for a large market, such as the US, and subsequently localized to a variety of other markets. For speech interface utterances of the machine, the responses to the user are often translated literally from one language to another, focusing on a generic dialog rather than cultural appropriateness, unconsciously violating cultural norms for communication. One observed example is the translation of the system English request ‘‘Please say your command’’ to the German ‘‘Bitte sagen Sie ein Kommando,’’ which ignores the different meanings of Kommando in German* where the word is used solely for military purposes or for training pets and implies a boss−underling relationship and thus a notion of arbitrariness. General Motors (GM) has decided to invest into knowledge about cultural communication practices in important markets to offer a compelling human−machine interface for more pleasant user experiences. The practical difficulties GM investigators needed to address, then, were cultural differences in not only uses but the recognition of languages including dialects, cultural differences in how errors were noticed then corrected, as well as cultural differences in the flow of in-car dialogue from task initiations to completions. What if new cars could adjust in-car devices to particular ways of speaking around the world? Investigators at GM Research and Development in Herzeliya, Israel, and in Warren, Michigan, have been pioneers in examining the human−machine interface and recently in noticing how cultural features like these were active at every stage but were not adequately being studied (Tsimhoni, Winter, & Grost, 2009). Dr. Ute Winter was quick to notice the need for basic research in this area and produced a call by GM for such research to address this variability. In it she wrote: ‘‘The goal of this research is to develop a framework of cultural dimensions and principles, which have influence on discourse and may lead to different perception of dialog success by conversation partners with different cultural background. This conceptual framework should enable GM to derive a method for empirical learning about culturally driven user expectations, decisions and behaviours, while interacting with speech applications in specific regions of the world.’’ Dr. Winter sent this call to Carbaugh whose research, reported in Cultures in Conversation (2005) she had noticed as perhaps relevant to the call. To cut a long story short, Carbaugh, his team, and Winter with her team collaborated to produce a theory and methodology for doing such work, and subsequently conducted field studies using it in the United States and China (Carbaugh, Molina-Markham, van Over, & Winter, 2012). At GM, as a part of this project, the results of the field work are transformed into design considerations and recommendations for communication with future in-car infotainment systems using speech among other modalities (Winter, Tsimhoni, & Grost, 2011).

1.2The Ethnography of Communication: Cultural Discourse Analysis

Derived from and indebted to the ethnography of communication (Hymes, 1972; Philipsen & Coutu, 2005), cultural discourse analysis is devoted to the description and interpretation of communication practice (Berry, 2009; Carbaugh, 2007; Scollo, 2011). A special focus has been the exploration of intercultural interactions, especially as this sort of variability in practice is active in specific cultural scenes, like, perhaps surprisingly, the automobile.

Given the need for attention to cultural variability in the human−machine interface, General Motors sought a perspective and methodology for examining such variability within in-car communication. Given the approach to its study, ethnographers of communication and cultural discourse analysts were well-suited to design and conduct such study.

The approach designed specifically for this project treated the automobile as a communication situation, with special attention to cultural sequencing of talk, uses of directives, opening and closing of tasks, and repair or corrective exchanges. In addition, special attention was given to the multimodal capacities of users as they used speech and touch. Similarly, our conceptualization was attentive to participant gaze as it moves from the road to the device and back again. The methodology for collecting data was designed to ensure as much comfort for participants as was possible. Each participant used their own vehicle which we equipped, in each case, with a tablet device running the in-car system with capacities for phone calling, radio-playing, access to a music library, and eventually navigation. The participant’s car was equipped also with three cameras, one focused on the tablet device, one focused on the user’s face, with the third offering a wide-angle view of the participant, the researcher in the front passenger’s seat, the tablet, and the road ahead. All user interaction with the device and others in the car was thus recorded for purposes of our eventual analyses. This theoretical stance and methodology have produced extremely rich corpora of data from the United States and China.

Benefits of these studies went in multiple directions. General Motors sought an approach specially designed for the study of cultural features and dimensions of incar communication that Carbaugh and Molina-Markham produced. A second stage of the project focused on the conduct of field studies in the US and in China. The research team, acknowledged and listed as co-authors of this book, then used the approach to conduct the field studies in each site. The benefits to the research team icluded support of this complex research project, collaboration with an interdisciplinary team which included linguists, engineers, human factor specialists, interaction designers, and others, access to GM’s process of developing in-car technology, support for conducting fieldwork in multiple sites, and of course the opportunity to bring such work to completion through several collaborative efforts.

1.3Criteria for Deciding to Participate

The process of deciding whether to assemble a team to respond to GM’s call was not a simple matter. Several considerations animated that process. A primary consideration was whether the primary participants, the principal investigators, in this case Winter and Carbaugh, were ‘‘on the same page’’ with regard to the intellectual problems needing addressed and the general approach desirable for addressing them. In detailed discussions and correspondence, Carbaugh was assured Winter was knowledgeable about and supportive of the research that was needed. Winter, in turn, was confident the University of Massachusetts team could design and execute the type of research she wanted and needed to get done.

Another set of concerns related to the logistical support available for doing such work. After initial discussions, Winter asked Carbaugh what was needed, and Carbaugh responded with a tentative list of items, which Winter then adjusted, and so on. This process involved both principal players in producing short- and long- term planning for the project, which, in the long run, allowed the work to be done with adequate support.

A further set of concerns had to do with access to data. Carbaugh had worried that GM might put limits on the availability of the data collected and the construction of research reports. Winter was quick to convey that such limits were minor at the most and access would be assured throughout the entire project for analysis of data and research serving GM. Any concerns Carbaugh had were expressed by him immediately and were addressed directly by Winter in a timely way (and vice versa). So, the criteria used to decide whether to participate had to do with the synchronicity among participant researchers of the intellectual problems being addressed, the approach needed for addressing them, confidence by each party of the other, congeniality in relations among the key participants, adequate financial and logistical support for the project so designed including not only conceptual development but field studies, access to the data gathered, and few if any restrictions on the production of research reports.

From Carbaugh’s view, from the start of the collaboration to the present day, the project provided an interesting set of problems to study, a unique set of communication practices to theorize about, and an opportunity to support graduate student researchers and colleagues, with Lie, Molina-Markham, and van Over falling into the former category, and Professor Libin Hang into the latter.

1.4Difficulties and Challenges

At the same time, there were difficulties and challenges the project has faced. As readers of this book and this section in particular are undoubtedly aware, research projects must undergo review typically by an institutional review board (IRB). When a project puts study participants behind the wheel of an automobile with advanced infotainment technology to use, a number of questions are raised. An added factor is that the studies here are naturalistic or ethnographic, relying predominantly upon a qualitative research design, rather than the quantification of variables, and one can guess that this approach was not typical for such a review. Suffice to say here that the IRB review process was detailed, complicated, but eventually successful!

Throughout the research process, and in particular during the fieldwork phase, it was clear that we were operating under a set of tensions that needed careful balancing. One such tension existed between our ethnographic commitment to naturalistic study, and the needs of GM to acquire data and produce analyses that they knew could lead to actual revisions of the technology. For instance, as noted earlier, each vehicle was outfitted with a touch screen device that served as the visual and tactile interface to the ‘‘brain’’ of the in-car system. After installing this system in the participant’s car, we provided them an opportunity to familiarize themselves with the system before going out on the road, and also during this time we collected data on how a new user of a system goes about learning what that system does and interacts with it.

The in-car system itself, provided by GM, posed a challenge on this undertaking. While the aim was to allow the user to formulate his intentions and desires from the infotainment system in his preferred natural language at any point of the interaction, and at the time of field research project, no in-car production system available on the market had these communication capabilities. This problem was solved by modifying an existing infotainment system, similar in design and manual-visual capabilities to the Cadillac CUE, and disabling the speech recognition capabilities in favour of an embedded Wizard, who would translate the user’s spoken utterances into touch sequences in a specially designed Wizard interface. This solution guaranteed control over the performance of the system, while still most of its parts were a machine with the user communicating with that machine.

There was variation in the initial exploration and learning process while using this system. This meant that users had significant variance in their understanding of the system’s abilities from the start. Some failed to discover that while doing one thing, listening to the radio for instance, they could initiate a speech command to do something else, like make a phone call. Such task-switching events are of interest to GM because they are scenarios in which misunderstanding is possible and in which preferences may exist for how the car accomplishes that task. Should the system mute the radio when a user asks to make a call, or simply dim the music? If a user never determines that this is possible and never decides to engage such a sequence, then these data will obviously not be collected. How then, do we allow users to use the system in whatever ways ‘‘naturally’’ make sense to them, which are data of great value, while also assuring that they have enough information about the system to try things they might want to try if only they knew they were possible?

Eventually, we decided that the best solution would be to break our observations of the drive into three phases. In the first phase, users explored with little to no input from researchers in a safe environment like a parking lot while the motor was idling. In the second phase, users attempted to employ what they discovered about the system’s abilities in the parking lot, while out on the road, and sometimes discovered new abilities along the way. Around the halfway point of the ride, by prearrangement, we had the driver pull off the road. If by that time the user still had not discovered one of the system’s primary functions, we asked prompting questions like, ‘‘Would you like it if the car were able to switch between radio and a phone call through a voice command?’’, ‘‘How might you ask the system to do something like that?’’, and ‘‘Do you want to give that a try?’’ In the third phase, then, users had the opportunity to drive back to the starting point, and, if so inclined, to try some ways of interacting with the system that perhaps they had not thought possible. These distinct phases allowed us to capture data of essentially different types and later analyze them as such, so as to exercise some minimal bracketing of our explicit influence. This tension played out in other arenas as well when questions arose about the extent to which we might manufacture problems to see how people would deal with them. Here the line between experiment and ethnography becomes problematically blurred and not crossing this line required vigilance and negotiation between members of the research team to assure all needs were ultimately met without compromising the integrity of the research, which all valued.

The human−machine interface, as represented in interactions with newly developing multimodal in-car systems, is a rich site of study for researchers interested in culturally distinctive communication practices. Automotive designers benefit as well from investigations into naturalistic usage of new designs. The work we presented in this chapter represents a collaboration between researchers and designers at various locations that led to the development of an approach for studying cultural features of in-car communication, data and reports from field studies in various sites, and design considerations and recommendations for future in-car systems. We discussed our participation criteria, as well as concerns and tensions that may arise when balancing commitments of naturalistic study with design development goals, emphasizing the importance of awareness, attentiveness, and cooperation in ensuring research integrity.

References

Berry, M. 2009. The social and cultural realization of diversity: An interview with Donal Carbaugh. Language and Intercultural Communication, 9:230−241. doi:10.1080/1470847090 3203058

Carbaugh, D. 2005. Cultures in conversation. Lawrence Erlbaum. (NJ)

Carbaugh, D. 2007. Cultural discourse analysis: Communication practices and intercultural encounters. Journal of Intercultural Communication Research 36:167−182. doi:10.1080/17475750701737090

Carbaugh, D., Molina-Markham, E., van Over, B., and U. Winter. 2012. Using communication research for cultural variability in human factor design. In: (N. Stanton, eds) Advances in human aspects of road and rail transportation. CRC Press. Boca Raton, (FL), pp. 176−185.

Hymes, D. 1972. Models for the interaction of language and social life. In: (J. J. Gumperz and D. Hymes, eds) Directions in sociolinguistics: The ethnography of communication. Blackwell. New York, pp. 35−71.

Philipsen, G., and L. M. Coutu. 2005. The ethnography of speaking. In: (R. Sanders and K. L. Fitch, eds) Handbook of research on language ...

Table of contents

• Cover
• Also of interest
• Title Page
• Copyright
• Acknowledgements
• Contents
• 1 Cultural Analyses of In-car Communication
• 2 A Model for Investigating Cultural Dimensions of Communication in the Car
• 3 “You can do it baby”: Non-Task Talk with an In-Car Speech Enabled System
• 4 User Interaction Styles
• 5 Taking Turns with a Machine: The cultural maintenance of interaction organization
• 6 “I had already started blabbing” – User Barge-in
• 7 Apologies in In-Car Speech Technologies
• 8 “Ok, talk to you later”: Practices of Ending and Switching Tasks in Interactions with an In-Car Voice Enabled Interface
• 9 Communication and Cultures in Cars: Reflections and Looking Forward
• 10 Appendix A