Research Methods in Human-Computer Interaction

Jonathan Lazar, Jinjuan Heidi Feng, Harry Hochheiser

• Publisher: Morgan Kaufmann
• Year: 2017
• ISBN:
  9780128093436
• Edition: 2
• Topic: Informatica
• Subtopic: Interazione tra uomo e computer

Research Methods in Human-Computer Interaction is a comprehensive guide to performing research and is essential reading for both quantitative and qualitative methods. Since the first edition was published in 2009, the book has been adopted for use at leading universities around the world, including Harvard University, Carnegie-Mellon University, the University of Washington, the University of Toronto, HiOA (Norway), KTH (Sweden), Tel Aviv University (Israel), and many others. Chapters cover a broad range of topics relevant to the collection and analysis of HCI data, going beyond experimental design and surveys, to cover ethnography, diaries, physiological measurements, case studies, crowdsourcing, and other essential elements in the well-informed HCI researcher's toolkit. Continual technological evolution has led to an explosion of new techniques and a need for this updated 2nd edition, to reflect the most recent research in the field and newer trends in research methodology.

This Research Methods in HCI revision contains updates throughout, including more detail on statistical tests, coding qualitative data, and data collection via mobile devices and sensors. Other new material covers performing research with children, older adults, and people with cognitive impairments.

• Comprehensive and updated guide to the latest research methodologies and approaches, and now available in EPUB3 format (choose any of the ePub or Mobi formats after purchase of the eBook)
• Expanded discussions of online datasets, crowdsourcing, statistical tests, coding qualitative data, laws and regulations relating to the use of human participants, and data collection via mobile devices and sensors
• New material on performing research with children, older adults, and people with cognitive impairments, two new case studies from Google and Yahoo!, and techniques for expanding the influence of your research to reach non-researcher audiences, including software developers and policymakers

Chapter 1

Introduction to HCI research

Abstract

Research in human-computer interaction (HCI) is fascinating because there are so many interesting research questions and so many changes over time due to technical advancements. At the time of this book going to press (2017), the field of HCI is approximately 35–40 years old. At the beginning, individuals researching HCI issues were primarily interested in how users without formal technical education interacted with office automation software, such as word processing, spreadsheeting, and database applications. Over time, many different topics have become of central interest to the HCI community, along with different types of research contributions, including empirical research, development of artifacts, contributions to theory, and contributions to methodology. New approaches to data collection, such as sensors, crowdsourcing, eye-tracking, and facial electromyography, have become available. Because HCI research takes place at the intersection of many different disciplines, it is important to always be aware of standards and expectations in the related disciplines. It is also important to consider who is the target audience for your research—other researchers? System developers? Public Policymakers? This chapter provides an overview of HCI research, including the history of HCI research, the disciplinary roots, the context of the existing research literature on a topic, the target audience for the research, and trends over time.

Keywords

Empirical research; Artifact contributions; Methodology; Theory; Trendlines; Big data; History of HCI; Datasets; Correlation; Causality; Interdisciplinary; Micro-HCI; Macro-HCI; Multidisciplinary; Audience-targeting; Policymaking; Public policy; Research lifecycle; Trade-offs; User-centered design

1.1 Introduction

Research in the area of human-computer interaction (HCI) is fascinating and complex. It is fascinating because there are so many interesting questions and so many changes over time (due to technical advancements). It is complex because we borrow research methods from a number of different fields, modify them, and create our own “standards” for what is considered acceptable research. It is also complex because our research involves human beings who are, to put it mildly, complex. It is important to understand the roots of the field, to understand the development of research methods in HCI, understand how HCI research has changed over time, and understand the multiple dimensions that must be considered when doing HCI research.

1.1.1 History of HCI

There is a general consensus that the field of HCI was formally founded in 1982. This is the date of the first conference on Human Factors in Computing Systems in Gaithersburg (Maryland, United States), that later turned into the annual ACM SIGCHI conference. So, at the publication time of this book (2017), the field of human-computer interaction (HCI) is around 35 years old. However, this is a deceptively simple description of the history of HCI. The field draws on expertise existing in many other areas of study. People were doing work before 1982 that could be considered HCI work. There is a fascinating article (Pew, 2007) that describes work on a project for the Social Security Administration in the United States starting in 1977. The work on this project could easily be described as HCI work, including task analyses, scenario generation, screen prototyping, and building a usability laboratory. Pew also describes presenting some of his work at the annual meeting of the Human Factors Society in 1979. Ben Shneiderman published Software Psychology, considered one of the first books on the topic of HCI, in 1980. The terms “office automation” and “office information systems” were popular in the late 1970s. At that time, you could find articles that could be considered HCI-related, in fields such as management, psychology, software engineering, and human factors. In an interesting article on the history of office automation systems, Jonathan Grudin describes 1980 as the “banner year” for the study of office automation systems, after which, the number of people studying the topic dwindled, and many of them refocused under the title of HCI (Grudin, 2006b). The computer mouse was first publicly demoed by Doug Engelbart in 1968 (Engelbart, 2016). Still others point to seminal papers as far back as Vannevar Bush's “As We May Think,” which looks surprisingly relevant, even today (Bush, 1945).

In the late 1970s and early 1980s, computers were moving out of the research laboratory and “secure, cooled room” into the home and the office. The use of mainframes was transitioning into the use of mini- and then microcomputers, and the more popular personal computers were making their debut: Apple II series, IBM PC/XT, and the Commodore/Vic. It was this move, away from large computers in secure rooms used only by highly trained technical people, to personal computers on desktops and in home dens used by nontechnical people in much greater numbers that created the need for the field of HCI. Suddenly, people were using computers just as a tool to help them in their jobs, with limited training, and personal computers became a product marketed to home users, like stoves or vacuum cleaners. The interaction between the human and the computer was suddenly important. Nonengineers would be using computers and, if there wasn't a consideration of ease of use, even at a basic level, then these computers were doomed to failure and nonuse. In the current context, where everyone is using computers, that may sound a bit odd, but back in the 1970s, almost no one outside of computing, engineering, and mathematics specialists were using computers. Personal computers weren't in school classrooms, they weren't in homes, there were no bank cash machines, or airline self check-in machines, before this shift towards nonengineering use happened. This shift created a sudden need for the field of HCI, drawing on many different fields of study.

1.2 Types of HCI Research Contributions

The field of HCI draws on many different disciplines, including computer science, sociology, psychology, communication, human factors engineering, industrial engineering, rehabilitation engineering, and many others. The research methods may have originated in these other disciplines. However, they are modified for use in HCI. For instance, techniques such as experimental design and observation from psychology, have been modified for use in HCI research. Because HCI draws on the work in so many different disciplines, people often ask “what is considered HCI research? What types of effort are considered research contributions?” In a recent article that we believe will become a classic read, Wobbrock and Kientz (2016) discuss seven types of research contributions:

Empirical contributions—data (qualitative or quantitative) collected through any of the methods described in this book: experimental design, surveys, focus groups, time diaries, sensors and other automated means, ethnography, and other methods.

Artifact contributions—the design and development of new artifacts, including interfaces, toolkits, and architectures, mock-ups, and “envisionments.” These artifacts, are often accompanied by empirical data about feedback or usage. This type of contribution is often known as HCI systems research, HCI interaction techniques, or HCI design prototypes.

Methodological contributions—new approaches that influence processes in research or practice, such as a new method, new application of a method, modification of a method, or a new metric or instrument for measurement.

Theoretical contributions—concepts and models which are vehicles for thought, which may be predictive or descriptive, such as a framework, a design space, or a conceptual model.

Dataset contributions—a contribution which provides a corpus for the benefit of the research community, including a repository, benchmark tasks, and actual data.

Survey contributions—a review and synthesis of work done in a specific area, to help identify trends and specific topics that need more work. This type of contribution can only occur after research in a certain area has existed for a few years so that there is sufficient work to analyze.

Opinion contributions—writings which seek to persuade the readers to change their minds, often utilizing portions of the other contributions listed above, not simply to inform, but to persuade.

The majority of HCI research falls into either empirical research or artifact contributions, and this book specifically addresses empirical research using all of the potential data collection methods utilized in empirical research. In their analysis of research papers submitted to the CHI 2016 conference, Wobbrock and Kientz found that paper authors indicated in the submission form that over 70% of the papers submitted were either empirical studies of system use or empirical studies of people, and 28.4% were artifact/system papers (it is important to note that authors could select more than one category, so percentages can add up to more than 100%). There were a fair number of papers submitted on methodological contributions, but submissions in all of the other categories of contributions were rare (Wobbrock and Kientz, 2016). This provides some empirical data for what we (as book authors) have observed, that most HCI research is either empirical or systems research (or sometimes, a combination of both, such as when you develop a prototype and have users evaluate it).

1.3 Changes in Topics of HCI Research Over Time

The original HCI research in the 1980s was often about how people interacted with simple (or not so simple) office automation programs, such as word processing, database, and statistical software. The basics of interfaces, such as dialog boxes, and error messages, were the focus of much research. Some of the classic HCI articles of the 1980s, such as Norman's analysis of human error (Norman, 1983), Carroll's “training wheels” approach to interface design (Carroll and Carrithers, 1984), and Shneiderman's work on direct manipulation (Shneiderman, 1983) are still very relevant today. Towards the late 1980s, graphical user interfaces started to take hold. In the late 1980s and early 1990s, there was growth in the area of usability engineering methods (and the Usability Professionals' Association, now known as UXPA, was founded in 1991). But there was a major shift in the field of HCI research during the early to mid 1990s, as the Internet and the web gained wide acceptance. New types of interfaces and communication, such as web pages, e-mail, instant messaging, and groupware, received attention from the research community. This caused an increased number of research fields to be included under the umbrella of HCI, especially communication. A recent article by Liu et al. (2014) on trends of HCI research topics, determined a big difference between research in 1994–2003, which focused on fixed technology, and research from 2004–13, which focused on mobile and portable computing (such as tablets and smart phones).

Around 2004–05, the focus of research shifted more towards user-generated content that was shared, such as photos, videos, blogs, and wikis, and later grew into research on social networking. On Dec. 26, 2006, Time Magazine famously named “You” as the “person of the year” for generating much of the content on the web. The topic of user diversity gained more attention, with more research studying how younger users, older users, and users with disabilities, interact with technologies. In the late 2000s, research increased on touch screens, especially multitouch screens, with studies on motor movement focused on pointing using fingers, rather than computer mice. It is important to note that while multitouch screens only entered common public use in the late 2000s, multitouch screens had been developed and researched as far back as the early 1980s (Buxton, 2016).

The research focus in the late 2010s (the publication date of the book) is no longer on something as simple as task performance in statistical software, but is now more focused on collaboration, connections, emotion, and communication (although, again, research on collaboration has existed since the early 1980s, even if it's now just gaining attention). The focus is not just on workplace efficiency any more, but is on whether people like an interface and want to use it, and in what environment they will be using the technology. Today's research focuses on topics such as mobile devices, multitouch screens, gestures and natural computing, sensors, embedded and wearable computing, sustainability, big data, social and collaborative computing, accessibility, and other topics (Liu et al., 2014). But, of course, that will change over time! The topics of HCI research continue to change based on factors such as technological developments, societal needs, government funding priorities, and even user frustrations.

1.4 Changes in HCI Research Methods Over Time

There are many reasons why, over time, research methods naturally evolve and change. For instance, tools for research that were originally very expensive, such as eye-tracking, sensors, drones, facial electromyography (EMG), and electroencephalography (EEG) are now relatively inexpensive or at least are more reasonable, allowing more researchers to afford them and integrate these tools into their research. New tools develop over time, for instance, Amazon's Mechanical Turk. New opportunities present themselves, such as with social networking, where suddenly, there are billions of pieces of text and multimedia that can be evaluated, looking for patterns. Or with personal health tracking, or electronic health records, which allow for analysis of millions of data points, which have already been collected. Some types of research are now fully automated. For instance, years ago, researchers would do a citation analysis to understand trends in research, but most of that analysis is now easily available using tools such as Google Scholar. On the other hand, automated tools for testing interface accessibility, are still imperfect and have not yet replaced the need for human evaluations (either with representative users or interface experts).

One important difference between HCI research and research in some of the other social sciences (such as sociology and economics), is that, large entities or government agencies collect, on an annual basis, national data sets, which are then open for researchers to analyze. For instance, in the United States, the General Social Survey, or government organizations such as the National Center on Health Statistics, the US Census Bureau, or the Bureau of Labor Statistics, collect data using strict and well-established methodological controls. Outside of the US, agencies such as Statistics Canada, and EuroStat, collect excellent quality data, allowing researchers to, in many cases, to focus less on data collection and more on data analysis. However, this practice of national and/or annual data sets, does not exist in the area of HCI. Most HCI researchers must collect their own data. So that alone makes HCI research complex.

Typically, HCI research has utilized sm...