Perspectives on Cognitive Task Analysis

Historical Origins and Modern Communities of Practice

Robert R. Hoffman, Laura G. Militello

• Publisher: Psychology Press
• Year: 2008
• Print ISBN: 9780805861402
• eBook ISBN: 9781136678301
• Topic: Psychology
• Subtopic: Cognitive Psychology & Cognition
• Index: Psychology

This volume is the first comprehensive history of task analysis, charting its origins from the earliest applied psychology through to modern forms of task analysis that focus on the study of cognitive work. Through this detailed historical analysis, it is made apparent how task analysis has always been cognitive.Chapters cover the histori

Part 1

History

1

Introduction to Section 1

History

That Was Then

Those who came before us were more aware of the key issues and arguments than we might give them credit for. Indeed, many modern researchers have reinvented the wheel and given it new names, often believing their work is distinct or special. In fact, Fernberger (1937) acknowledged that sporadic amnesia is inevitable:

Scientific ideas become public and common property in spite of anything one may do. … Examples of particular processes and modes of explanation become current in a laboratory and one does not remember whether one thought of them himself or whether they should be attributed to some one of one's colleagues. (p. xi)

One goal of this book is to draw connections from past approaches to current methods, tracing the historical roots of modern Cognitive Task Analysis (CTA), tracing the pendulum swings, the divergences and convergences. We hope that for a next generation this will reduce the need to reinvent the wheel and encourage efforts to improve on the wheels we already have.

Although we are hesitant to overly constrain the notion of CTA, we think it is necessary to place some boundaries around which approaches will be examined. First off, we need to look at the meaning of the word task.

What Is a Task?

The English word task has rich linguistic origins in the ancient Indo-European branch of language. The base of the word family is /tag-/, meaning “to touch” in Latin. From this base came many words, including the Latin taxare, meaning “to reprimand or blame.” Among the many English words that derived are tangible, attain, contingency, and dozens of others. In many of these words we can see concepts that we can relate to the notion of task as in the modern phrase “Cognitive Task Analysis.” Webster's New Collegiate Dictionary (G. and C. Merriam Company, 1979) links the word task to the Latin tasca, meaning a tax imposed by a feudal superior. From this comes the notion that

a “task” is a directive, given by a superior, that something is to be done or accomplished within a specified time, usually something hard or unpleasant that has to be done.

Industrial psychologists, human factors psychologists, and ergonomicists have studied proficient human performance in a great variety of domains and contexts (see Alluisi, 1967; Anastasi, 1979; Sanders & McCormick, 1992)—everything from the design of screwdriver handles to the design of information processing systems. The term task analysis, coined by R. B. Miller in 1953, has referred to methods that have been used in industrial psychology and ergonomics (Annett, Duncan, Stammers, & Gray, 1971; Eastman Kodak Company, 1983; Kirwan & Ainsworth, 1992; Meister, 1985; Wexley & Yukl, 1984). Task analysis can involve describing jobs in terms of individual physical actions (e.g., button pushing) or describing acts in terms of their higher level or functional foundations (e.g., goals, domain concepts, etc.).

Task analytic methods have been around since before World War I, after which they built on Frederick Taylor's Time and Motion Study approach to job analysis that was popular in the early 1900s. Applied psychologists have been studying aptitudes as a basis for personnel selection and training since the late 1800s. Industrial engineers have been studying work systems and how to make them more efficient since the Industrial Revolution, if not before. Certainly, ethnographers had been studying how history and culture affect work life long before the term CTA was ever coined.

A great variety of task analysis methods have been created and named so as to denote applicability in particular domains (e.g., operational sequence diagramming for industrial process control), the uniquenesses of particular methodologies (e.g., structured interviewing, hierarchical analysis, process tracing), or the goals of the procedure (e.g., fault tree diagnosis). The goals of task analysis can include the specification of entire occupations and jobs, the analysis of specific event sequences that lead to faults or accidents in industrial process control, the specification of ergonomic constraints on equipment design, and the development of training and remediational programs. (Lists of task analysis methods can be found in Chapanis [1996]; Drury, Paramore, Van Cott, Grey, and Corlett [1987]; Fleishman [1975]; Kirwan and Ainsworth [1992]; Meister [1985]; Nemeth [2004]; Salvendy [1987]; Shepherd [2001]; and Viteles [1922]. Diaper and Stanton [2004] provided a fine handbook on task analysis that includes overviews as well as chapters focusing on individual methods and approaches.)

Some researchers today hold that the task is not the proper unit of analysis because technology is always changing the work (Carroll, 1995) and therefore boundable tasks will be fleeting. Detailed, stepwise descriptions of tasks will be brittle—true today, gone tomorrow. Some define task as just the goals of particular activities or activity sequences. In part, this reduction of task to goal reflects the fact that in many jobs, worker activities cannot be completely (or well) described in terms of rulelike prescriptions or stepwise directives. Why? Because context, dynamics, and cognition are the keys. CTA methods are most often used in the study of adaptive work in complex contexts. Most commonly, these contexts include many people (teams) and many machines (computers), all acting as a system within a cultural, social, and organizational setting. Such contexts are referred to as complex cognitive systems (Hoffman & Woods, 2005), joint cognitive systems (e.g., Hollnagel & Woods, 2006), and sociotechnical systems (Clegg, 2000; Vicente, 1999). Generally, CTA has the goal of designing better training, better technologies, and better teams to support cognitive work and the achievement of proficiency. However, CTA also has the goal of enriching our basic understanding of human cognition, reasoning, and perceptual skill.

CTA is a suite of scientific methods and (as the saying sort of goes) it is an art. As the associationist philosopher Alexander Bain pointed out in his book Education as Science (1879), this is not an either-or.

The scientific treatment of any art consists partly in applying the principles furnished by the several sciences involved, as chemical laws to agriculture; and partly in enforcing, throughout the discussion, the utmost precision and rigour in the statement, deduction and proof of the various maxims or rules that make up the art. (Alexander Bain, 1879, p. 1).

The conduct of CTA requires practice, skill, and finesse.

CTA as Reflecting an Emerging Need

The phrase “Cognitive Task Analysis” began appearing in print in the late 1970s, primarily to express a need: How can we analyze the cognitive components of work? How can we help novices think and perform more like experts? The term emerged in the writings of instructional designers, expertise researchers, systems engineers, cognitive psychologists, and others. A number of researchers perceived a need for task analytic methods that would “define clearly what it is that an expert in a subject matter domain has learned” (Glaser & Resnick, 1972; Resnick, 1976, p. 209). Since about 1979 when the CTA phrase began appearing in print and the mid- to late 1980s when the phrase began appearing in the literatures of human factors psychology and instructional design, the phrase has come to be widely used as a designation for a number of different research and data analysis methods that all attempt to tap into or describe the cognitive processes that underlie proficient performance and its development, to reveal the patterns of human reasoning, problem solving, decision making, and collaboration and the content knowledge of domain expertise and skill. This emerging need is reflected in salient topics such as interface design and usability testing (e.g., Redish & Wixon, 2003). This emerging need is also reflected clearly in the advent of new communities of practice. Examples are the “Naturalistic Decision Making” community of practice (see chapter 9), the “Cognitive Systems Engineering” community of practice (see chapter 7), and the formation of the “Cognitive Engineering and Decision Making” Technical Group within the Human Factors and Ergonomics Society.

Workplace Changes

Recent decades have witnessed a shift in the sorts of things that task analysis is used to study and explain, moving from a traditional focus on relatively simpler and sometimes routine physical tasks—such as the assembly of vacuum-tube-based radios—to more complex problem-solving tasks, especially those that are involved in human interactions with complex information processing systems. Cognition has become the paramount aspect of the tasks performed by knowledge workers using information technology. Task analysis has placed increased emphasis on the specification of the cognitive elements of technical skill, especially the knowledge and reasoning components (Bailey & Kay, 1987; Bainbridge, 1979; Rasmussen, 1986b; Woods & Hollnagel, 1987). The goal is to understand complex interactions of humans and machines (see Gordon, Gill, & Dingus, 1987; Madni, 1988; Means & Gott, 1988; Moray, Sanderson, & Vicente, 1992; Sarter & Woods, 1995). If that is the case, it is argued, then one must engage in some sort of Cognitive Task Analysis (see Glaser et al., 1985; Gordon & Gill, 1997; Lesgold et al., 1986). There has been increasing demand for tools (including CTA methods and software support tools) aimed at helping researchers explore cognition in depth, unpack the influence of environmental factors on cognitive processes, and apply this in-depth understanding of cognition in real-world contexts to the design of technologies, interfaces, and training.

Emerging Interest in Studying Diverse Domains for Diverse Applications

Studies using CTA methods have been conducted on a great variety of domains—aircraft piloting, air traffic control, military command and control, industrial process control, reasoning in medicine, and electronics troubleshooting, to name just a few (see Hall, Gott, & Pokorny, 1995; Helander & Nagamachi, 1992; Klein, Orasanu, Calderwood, & Zsambok, 1993). Projects have had a great variety of purposes. CTA methods have been important for research and applications in such areas as training and remediation, the design of automated decision aids, the design of interfaces and workstations, the elicitation of expert knowledge for intelligent systems, the preservation of corporate knowledge, and the identification and mitigation of error (for examples, see Hall, Gott, & Pokorny, 1995; Hoc, Cacciabue, & Hollnagel, 1995; Hoffman, 1991a; Howell & Cooke, 1989; Klein, 1992; Marti & Scrivani, 2003; Merkelbach & Schraagen, 1994; Rasmussen, Pejtersen, & Goodstein, 1994; Redish & Wixon, 2003; Roth, 1997a; Woods, 1993). In addition to applications, CTA-based research has led to new models of decision making (see Klein, 1993a) and has enriched the ethnographic descriptions of the social construction of scientific knowledge (e.g., Collins, 1985; Hutchins, 1995a; Suchman, 1987).

CTA and CFR

The emerging need also involved the notion that the study of work cannot be conducted in isolation from the full, rich context in which tasks are performed, hence the need for what has come to be called Cognitive Field Research (CFR) (Hoffman & Woods, 2000 see DeKeyser, 1992; Rasmussen, 1992). The CTA community of practice emphasizes the study of complex cognitive systems in context. Hence, in this book we discuss CFR as well as CTA.

One purpose of the CTA–CFR distinction is to emphasize the fact that the analysis of tasks in terms of cognitive variables and functions can sometimes be somewhat divorced from empirical observation of actual activity in a realistic or real setting. That is, some CTA research is situated in laboratory or laboratory-like settings, whereas CFR is conducted in the place of work. These boundaries are of course fuzzy (see chapter 16 and Hoffman & Deffenbacher, 1992). Thus, a CTA interview procedure might be conducted in a meeting room next to the operations floor of a weather forecasting facility. That room would be essentially no different from a small cubicle in an academic psychology laboratory. So is it a laboratory, a natural laboratory, or a field setting? It is certainly laboratory-like: It is small, quiet, and includes a table and some chairs and a pile of research materials (instructions, note pads, data collection forms, etc.). But what if the interviewee weather forecaster sees a need to go back to his office and get some materials, say, satellite images, to refer to in the interview? When he comes back into the interview room, is that now a field setting?

The distinction regarding the location of the research (laboratory or field) may be a distraction. It may be more informative to consider the contrast between the objectives of CTA and CFR to retain as much real-world context as possible (even if the research must be conducted away from the work) and the traditional objective in laboratory research to control as many contextual variables as possible to isolate and examine a component of cognition. CTA and CFR embrace such notions as “cognition in the wild” (Hutchins, 1995a) and “situated cognition” (Clancey, 1997) (see chapter 10), both of which focus on explaining and studying cognition from an anthropological point of view in the context of work, culture, and political constraints. In this light, the assumption distinguishing CTA from CFR is the analysis of tasks in terms of cognitive variables and functions based on observation of real activity in the “real world,” regardless of whether the goal is to influence the shape of technology or further our understanding of the cognition of domain practitioners. (To ease the flow of the discussion, we use the single acronym CTA throughout the remainder this book to refer to both CTA and CFR, unless otherwise indicated.)

Perspectives on CTA

CTA in its present form is a relatively recent development. However, to many who read or hear about CTA for the first time, the core ideas seem somehow familiar. In fact, many of the elements of CTA have evolved over time in the context of different fields of study. Psychologists have been using self-report methods (variants of introspection), combined with ethnographic methods, for many years. The buzz phrases “Cognitive Task Analysis” and “Cognitive Field Research” seem to be both ironic and oxymoronic. To many psychologists, cognitive does not quite seem to go with task—it hearkens to the pendulum swing of behaviorism versus mentalism. To others, the phrases are critical in capturing the implicit claim that for most human activities, cognitions are not only relevant in the description of behavior but necessary to explain the behavior because they play a causal role. The notion of “field research” seems to be an oxymoron to the experimentalist who prefers factorialization and laboratory control to the uncertainties and complexities of the real world, arguing that it is difficult, if not impossible, to engage in rigorous hypothesis testing in any form of field research.

So goes a historical pendulum of ideas, reactions, and counterreactions—this forms a recurrent theme to this book. The irony is that things that were said about psychological research methods decades ago (e.g., Fernberger, 1937) can be said today now about CTA methods:

• CTA has existed in a variety of forms and has a variety of uses.

• The methodology of CTA, and indeed its very existence, is linked to historical trends in the philosophy, goals, and even disciplinary agendas of both applied and academic psychology.

• Like all methods, CTA has both strengths and weaknesses.

The power of CTA is in how the needs and historical trends have come together to be applied to the challenges of the information age: using qualitative and quantitative methods to study cognition and expertise in the context of work, to explore a work domain with the objective of uncovering cognitive complexity, and to apply this knowledge to the design of tools, technologies, and work systems.

The Focus and Organization of This Book

By the 1990s the term CTA was often used to describe methods or approaches to studying the cognitive components of work—attempts to meet the need articulated beginning in the 1970s. Not long thereafter, the conversation shifted to how to best define CTA, what is actually new or different about CTA, and how it relates to historical as well as modern ideas about and methods for the study of work. This book documents and expands that conversation. In the chapters of this book, we do no...

Table of contents

• Cover
• Halftitle
• Title
• Copyright
• Contents
• Dedication
• Series Editor’s Preface
• Preface
• Acknowledgments
• About the Authors
• Part 1 History
• Part 2 The “Perspectives”
• Part 3 Synthesis
• Notes
• References
• Subject Index
• Author Index