Cognitive Work Analysis: Coping with Complexity

Daniel P. Jenkins, Neville A. Stanton, Guy H. Walker

• Publisher: CRC Press
• Year: 2017
• ISBN:
  9781317164708
• Edition: 1
• Topic: Tecnologia e ingegneria
• Subtopic: Salute e sicurezza sul lavoro

'Complex sociotechnical systems' are systems made up of numerous interacting parts, both human and non-human, operating in dynamic, ambiguous and safety critical domains. Cognitive Work Analysis (CWA) is a structured framework specifically developed for considering the development and analysis of these complex socio-technical systems. Unlike many human factors approaches, CWA does not focus on how human-system interaction should proceed (normative modelling) or how human-system interaction currently works (descriptive modelling). Instead, through a focus on constraints, it develops a model of how work can be conducted within a given work domain, without explicitly identifying specific sequences of actions (formative modelling). The framework leads the analyst to consider the environment the task takes place within, and the effect of the imposed constraints on the way work can be conducted. It provides guidance through the process of answering the questions of why the system exists, what activities can be conducted within the domain as well as how these activities can be achieved, and who can perform them. The first part of the book contains a comprehensive description of CWA, introducing it to the uninitiated. It then presents a number of applications in complex military domains to explore and develop the benefits of CWA. Unlike much of the previous literature, particular attention is placed on exploring the CWA framework in its entirety. This holistic approach focuses on the system environment, the activity that takes place within it, the strategies used to conduct this activity, the way in which the constituent parts of the system (both human and non-human) interact and the behaviour required. Each stage of this analysis identifies the constraints governing the system; it is contended that through this holistic understanding of constraints, recommendations can be made for the design of system interaction; increasing the ability of users to cope with unanticipated, unexpected situations. This book discusses the applicability of the approach in system analysis, development and evaluation. It provides process to what was previously a loosely defined framework.

Chapter 1
Introduction

Humanity’s constant strive to do things better, faster and more efficiently continues to be a fundamental driving force for human development. In the modern world, this can be seen by the enormous number of products and services coming to market every single day. Not only do these new products and services allow us to do things better, faster and more efficiently they bring with them new affordances and capabilities; some of which, that were previously unthought-of. Human beings have an innate ability to adapt to these new opportunities and flourish; this allows us to conceive of new ways of using existing systems and designs, which in turn often leads to the design and development of new products and services. The idea of a ‘perfect product’ is a fallacy, as soon as a new product is exposed to human beings, new unanticipated uses are developed and improvements conceived of. As we as humans develop we expect, if not demand, our products and services to follow suit.

Figure 1.1 shows Carroll’s (1991) ‘task-artefact cycle’. One way to enhance the longevity as well as the usefulness of a product or service is to design for this innate curiosity and desire for improvement; this involves developing products and services that encourage and support rather than constrain this natural, formative, human behaviour.

Figure 1.1 The task-artefact cycle (adapted from Carroll, 1991)

This book aims to address how systems can be designed and developed to support, rather then discourage, this kind of formative behaviour. More specifically, this book explores the ability of Cognitive Work Analysis (CWA) to support the analysis, development and evaluation of complex sociotechnical systems.

This book builds heavily upon previous works on CWA and Work Domain Analysis (most notably; Rasmussen et al, 1994; Vicente, 1999; Naikar et al, 2005). Whilst there is concordance with Vicente’s (1999) view, that CWA is, and should be, non-prescriptive; this great strength of the approach, it flexibility, may also be perceived as one of its weaknesses; it is often perceived as being difficult to grasp. There is, therefore, perceived benefit in providing a more detailed description of one way of completing the process. Based upon the literature studied and the case studies completed, this book presents a detailed process for conducting a CWA; providing guidance to those new to the topic and hopefully consolidating the ideas of more experienced analysts. The presented approach is not intended to be the final word on the matter; it is readily acknowledged that the use of the framework and the choice of representation are likely to be heavily influenced by the domain and the required output of the analysis.

This book reinforces the importance of considering the framework in its entirety to gain an appreciation of the different constraint sets in existence in any given domain. It is contended that analyses that do not consider a range of constraint sets, captured within the framework, present a less complete model of the environment. Whilst discussing the problems of automation, Norman (1990) makes the following statement, which appears to be particularly applicable at this juncture:

…in design, it is essential to examine the entire system: the equipment, the crew, the social structure, learning and training, cooperative activity, and the overall goals of the task. Analyses and remedies that look at isolated segments are apt to lead to local, isolated improvements, but they may also create new problems and difficulties at the system level. (Norman, 1990 p.2)

In summary, this book presents a detailed description of how CWA can be applied to the analysis, design and evaluation of complex sociotechnical domains. The approach taken reinforces the importance of considering the framework in its entirety and provides a more complete description of how links between the relevant phases can be exploited.

Why CWA for Command and Control?

As the subsequent chapters in this book will discuss, the complexity embodied in command and control systems present significant challenges for modelling and analysis. Traditional reductionist modelling techniques, which decompose activity into a set of task sequences, can rarely be extended beyond stable and repeatable systems. An analysis or modelling technique is, therefore, required that can handle this inherent complexity and adaptability. One means of achieving this is to concentrate on the constraints shaping the way work is conducted in a given domain. In order to achieve this, it is advantageous to start by considering the environment independent of technology. As Moore’s (1965) law tells us, computing power doubles every 18 months bringing with it new capabilities and products. If we do not consider our models in a technologically agnostic way, it will soon become significantly outdated and, in turn, unrepresentative.

The question is not: how do we use new technologies to execute our current tactics and doctrine better? It is instead: how might the new technologies enable us to do things differently? Without new thinking, the new technologies are likely to increase efficiency in a way that is largely arithmetic and incremental in nature. They enable us to execute today’s tactics and operations in a quantifiably better way, but they still leave us with traditional, tightly controlled, and synchronized operations that are hierarchically planned and executed. (Smith, 2003 pp 74–75)

The domains within command and control are, clearly, both complicated and dynamic: the constituent parts of the system (including humans) are required to frequently change and adapt in time with the environment. Naikar (in press) points out that by focussing on constraints, rather than on particular ways of working, it is possible to support workers in adapting their behaviour online and in real time in a variety of situations, including unanticipated events. For these complex domains, an approach is required that models the conditions framing formative behaviour, allowing the examination of emergent, unpredicted, unanticipated actions. Cognitive Work Analysis (CWA) is presented as an approach that is particularly amenable for this domain. As previously stated, this book aims to address how CWA can be used to support the analysis, development and evaluation of complex sociotechnical systems. Due to their inherent complexity, specific attention will be placed on Command and Control systems. Through its focus on the system constraints, and its initial drive to be technologically agnostic, CWA supports revolutionary as well as evolutionary design. It is contended that; with formative thinking, based on approaches such as CWA, the exponential improvements described by Smith (2003) can be realised.

At this point in the book, it is important to temper the enthusiasm for the approach by acknowledging its limitations. The approach requires specific training for its application and understanding; the level of which will be explored later in the book (see the appendix to this document). The technique relies on the skill of the analyst to appropriately set boundaries for the analysts as well as make constant judgment calls to determine the level of the analysis. For these reasons, the outputs of the technique can vary significantly dependant on the skill of the analyst. As Chapter 3 will discuss, design is, essentially, an art dependant on creativity and the application of design principles. However, Chapter 3 will also show that the number of constraints, and the inherent flexibility, in these complex sociotechnical systems requires a structured approach. No approach exists, and probably never will exist, that can create an explicit link between problem and solution for complex sociotechnical systems. What this book sets out to demonstrate is that, through a series of developments, CWA can provide a structured framework for the modelling of constraints. This model of constraints forms the basis for the support of analysis, development and evaluation, provide designers and analysts with a common language and structure for the discussion and development of such systems.

Structure of the Book

This book has been constructed so that readers new to this specific subject area can read it linearly. An attempt has also been made to construct the individual chapters and case studies so that they can be read non-linearly, or independently of the early chapters, by those well versed in CWA with an interest in the extensions and proposed developments documented in this book. This means that there is some repetition in the description of CWA, in an attempt to create a resource for experts.

2 It’s a Complex World

The book commences with a scene-setting chapter explaining in more detail the challenges encountered by both analysts and designers when they are faced with complex sociotechnical systems, further justifying the need for this research. The CWA framework is introduced and the need for formative behaviour is explored in detail.

3 Interaction Design

This chapter introduces the basic principles of interaction design to the reader; placing particular attention on interface design. The position of design on the art-science continuum is explored, as is the requirement for a structured approach to support design for complex sociotechnical systems. The intension of this chapter is not to provide an exhaustive guide to interface design; rather, the aim is provide the reader with context for latter phases where design is applied. A case study of a battlefield-planning tool is used to highlight the pertinence of some of the discussed design considerations.

4 Application of CWA in Familiar Domains

In order to illustrate the CWA framework and principles, introduced in Chapter 2, a ‘simple’ familiar artefact is used. The examples, in this and subsequent chapters, have been selected because they convey a particular message, the individual examples are not intended to stand alone as perfect examples of how to complete a CWA, most are incomplete in either analysis, design or evaluation; however, they each convey a different message; each explaining a different part of the technique. It is in keeping with the non-prescriptive, formative nature of the approach that a conscious decision has been made to present a wider breadth of examples rather than fewer overly detailed cases.

5 Applications of CWA in a Complex World

In this chapter the CWA framework is applied to complex command and control case studies. Two examples are selected that in turn address, analysis, the extraction of design recommendations and system evaluation. The first example presented analyses a military rotary wing mission planning software tool. The second investigates battlefield-management and addresses the evaluation of the impact of new technology introduced to a domain. The chapter explores, in detail, the benefits of applying the entire CWA framework; further, a new, theoretically grounded, approach to system evaluation is presented.

6 Using CWA to Design for Dynamic Allocation of Function

This chapter explores the social organisational and cooperation phase of the framework in the context of a dynamic command and control paradigm. The chapter starts with a thorough analysis of the domain; this analysis is then used to inform the design and development of a series of interfaces; the rapidly reconfigurable interfaces are designed to support dynamic allocation of function. The developed interfaces are designed to be role-specific modelling the system at a functional and physical level.

7 Designing Interfaces Using CWA

Much of the criticism for CWA and other human factors techniques are formed around the difficulty analysts have in taking analysis products and applying them to create design solutions or design recommendations. This chapter is structured with the specific aim of redressing this concern; this chapter explores the benefits of considering complex military planning within the Abstraction Hierarchy framework. The example described discusses the development of a military decision-support software tool. Here, the evaluation approach introduced in Chapter 5 is extended to consider the resultant high order effects of manipulations to objects at low levels of abstraction. Key question relating to interface design are investigated though systematic experimentation.

8 Development of a CWA Software Tool

In order to expedite the process of analysis, development and evaluation, as well as the training of novices: The development of a CWA software tool is discussed. The chapter addresses the need for the tool as well as cataloguing its development. The benefits of the tool are compared against the paper-based approach along with standard graphics software.

9 Does the Tool Make the CWA Process any Quicker or Easier?

A military example is applied to evaluate the tool proposed in Chapter 8. The example is taken from a training video of a Battlegroup in the quick attack formation. From the analysis, additional requirements are extracted and discussed.

10 Conclusions

The final chapter concludes the study with discussion of the aims and objectives. Areas for further research are identified along with the key contribution to knowledge brought about by the research discussed in this book.

Appendix Can it be Taught?

A key factor in the success of any developed approach is its ability to be learnt and assimilated by others. The appendix asks the questions ‘can CWA be taught?’ and ‘how effective can the tuition be within a limited time period?’ An experiment was conducted using final year design students to assess how learnable Work Domain Analysis is. The findings of this experiment informed the development of a CWA desktop software tool.

Chapter 2
It’s a Complex World

Introduction

The aim of this chapter is to set the scene for the book, first the concept of complexity is introduced; this widely used term is explained in relation to sociotechnical systems. The specific example of command and control is used to highlight this. Cognitive Work Analysis is then presented as a framework to model these complex sociotechnical systems.

Complexity

Dependant on the context, the term complex has different meanings, whilst in mathematics it is used to describe a particular set of numbers, in this case we are considering complex by its definition of ‘something with lots of interrelated parts’; a system. According to Walker et al (2007) at a fundamental level, complexity relates to the amount of information needed to describe a phenomenon under analysis. The closer that the phenomenon under analysis approaches complete randomness, the more data is needed until it ‘cannot be described in shorter terms than by representing the [phenomenon] itself’ (Bar Yam, 1997). However, ‘something is complex if it contains a great deal of information that has a high utility, while something that contains a lot of useless or meaningless information is simply complicated’ (Grand, 2000, p. 140 cited in Bar Yam, 1997). Complexity is not a binary state of either ‘complex’ or ‘not complex’, the level of complexity lies on a non-numerical scale, in the past people have attempted to quantify this through a set of heuristics, Woods (1988) proposes the following four dimensions as a framework for assessing the complexity of a system:

• Dynamism of the system: To what extent can the system change states without intervention from the user? To what extent can the nature of the problem change over time? To what extent can multiple on-going tasks have different time spans?

• Parts, variables and their interconnections: The number of parts and the extensiveness of interconnections between the parts or variables. To what extent can a given problem be due to multiple potential causes and to what extent can it have multiple potential consequences?

• Uncertainty: To what extent can the data about the system be erroneous, incomplete, or ambiguous – how predictable are future states?

• Risk: What is at stake? How serious are consequences of users’ decisions?

Highly complex systems can be characterised by all, or most, of the four dimensions identified by Woods (1988). These domains are classified by Rittel and Webber (1973) as ‘Wicked problems’, wicked problems are not fully understood, they have fuz...