Building Intelligent Tutoring Systems for Teams

What Matters

Joan Johnston, Robert Sottilare, Anne M. Sinatra, C. Shawn Burke, Joan Johnston, Robert Sottilare, Anne M. Sinatra, C. Shawn Burke

• Publisher: Emerald Publishing Limited
• Year: 2018
• Print ISBN: 9781787544741
• eBook ISBN: 9781787544758
• Topic: Business
• Subtopic: Human Resource Management
• Index: Business

This volume presents multidisciplinary perspectives from leading scholars in the science of teams and intelligent tutoring systems (ITSs) on research needed to advance the state of the art of team ITSs. Our esteemed authors provide lessons learned to guide future research that will produce the technical capabilities needed to support team skills development. The introduction by Drs. Robert Sottilare and Eduardo Salas, who are leaders in the field of ITSs and team science, discusses the challenges and approaches to building ITSs for teams. The volume's first section introduces concepts for understanding team training such as team task analysis, team macrocognition, measurement strategies for dynamic processes, and effective team training methods to provide insights into ITS design. Section two presents recent advances in team assessment and feedback through unobtrusive assessments, modeling dynamic team interactions, neurodynamic scaffolding, and collaborative tutoring strategies. In the Volume's third section authors discuss lessons learned from past research, provide a discourse on the five disciplinary perspectives of engineering, learning sciences, team research, data analysis, and human computer interaction to create a framework for guiding team ITS developers, and examine the team ITS requirements for long term space travel. The final chapter summarizes and integrates lessons learned and provides recommendations for future research and development.

PART I
CONCEPTS FOR UNDERSTANDING
TEAM TRAINING

TEAM TASK ANALYSIS: CONSIDERATIONS AND GUIDANCE

C. Shawn Burke and Ryan Howell

ABSTRACT

The use of teams is ubiquitous in organizations, yet teams are not always effective. Much work has been conducted to understand those factors that facilitate effective team training. While much has been learned, there is no escaping the fact that team training is a complex, resource intensive endeavor. Recent advancements in the area of intelligent tutoring may provide a way forward as one method by which to reduce some of the ongoing resource requirements involved in team training. The current chapter relies on the science of team training to describe a tool, team task analysis, that should be considered in building ITSs that move beyond the training of individual tasks to those which are team-based. In that vein, an overview of team task analysis is provided, how it differs from individual task analysis, and what it may contribute to the design of ITS for teams. In doing so, we put forth five considerations that are somewhat unique as compared with traditional task analysis as well as some corresponding guidance from the literature in light of these considerations. It is our hope that this information will not only be useful to those building team-based ITSs, but spur future thought.

Keywords: Team training; team task analysis; teamwork; coordination demand analysis; team process; groups; collaboration

Over the last several decades the use of teams has become ubiquitous in organizations, emerging as a key organizational form. Teams that effectively work together are able to create a synergy that is not able to be accomplished by individuals. However, in many cases organizations fail to recognize that merely taking individuals who are experts at their jobs and placing them on a team does not guarantee an expert team – one that can effectively coordinate their actions. In this vein, much work has been conducted over the last 25 years that speaks to the factors that facilitate team effectiveness (e.g., LePine, Piccolo, Jackson, Mathieu, & Saul, 2008; Mathieu, Hollenbeck, van Knippenberg, & Ilgen, 2017; Mathieu, Maynard, Rapp, & Gilson, 2008), as well as the manner in which tools and methods combine to form the instructional strategies which have been shown to facilitate team development and, correspondingly, effectiveness (e.g., Salas & Cannon-Bowers, 2000; Salas et al., 2008).

While a great deal has been learned concerning the tools and methods by which to train members in the knowledges, skills, abilities, and other characteristics (KSAOs) that promote the shared knowledge, behavior, and affect (i.e., team coherence) shown to facilitate team performance, the development and implementation of team training is a resource intensive endeavor. As a result there have been calls by researchers and practitioners alike concerning instructional strategies (i.e., tools and methods) that can be used to reduce the burden on those charged with training teams. One of the tools that has been gaining momentum is the extension of intelligent tutoring systems (ITSs) traditionally designed for training individuals. However, teams are not only different than individuals, but their training is inherently more complex. Therefore, a number of hurdles and considerations must be overcome in order to build ITSs for teams that are scientifically grounded and systematically developed.

Teams differ from individuals in many ways, but perhaps the most apparent is the requirement for members to coordinate their actions in order to achieve collective team goals. In this vein, teams have been defined as two or more individuals who must work collaboratively and interdependently in the process of working toward a shared goal (Salas, Dickinson, Converse, & Tannenbaum, 1992). The requirement for interdependent action within teams drives the need for teams to master both taskwork as well as teamwork. Taskwork has been defined as the “task orientated skills that members must understand and acquire for task performance” (Salas et al., 1992, p. 10). It is this set of skills that are primarily targeted in most training programs within organizations. Conversely, teamwork skills reflect “the behavioral interactions and attitudinal responses that team members must develop before they can function effectively as a team” (Salas et al., 1992, p. 12).

Therefore, the purpose of this chapter is to describe a team task analysis method that can be utilized to uncover both the taskwork and teamwork skills needed for a given team and context. It is expected that this information can then be used to inform the content of the competency and pedagogical modules which reside within the architecture of team-based ITSs. We describe the nature of team task analysis, its importance in the systematic development of team training as seen through an ITS, as well as provide guidance into key decisions that must be made in its implementation.

TEAM TASK ANALYSIS – WHAT IS IT?

As traditional job analysis has been argued to be a critical foundation for many human resource functions, team task analysis is a critical foundation for those human resource functions related to teams (Arthur, Edwards, Bell, Villado, & Bennett, 2005). Team task analysis is a tool through which critical taskwork and teamwork KSAOs can be identified. Typically subsumed under the umbrella of “job analysis,” a task analysis is a critical part of the process whereby job tasks are defined in terms of objectively identifiable activities (McCormick, 1979). It is through this process that the KSAOs needed to successfully perform the task are identified. As there are different job analysis techniques that ascertain different data and yield different conclusions, there are also different techniques for analyzing tasks that vary in levels of description and specificity. However, regardless of job or task analysis technique, the main focus of both methods is at the task level. In conducting a task analysis various scales are used to assess specific jobs, and their relevant tasks are assigned ratings across a variety of dimensions, including (but not limited to) criticality, frequency, and time spent. Generally, scales used in both job and task analyses have been used to assess those functions at the individual level (Brannick & Levine, 2002).

As the inherent nature of a team is to consist of two or more individuals with specific role assignments and tasks, in addition to needing coordination and interaction to be successful (Baker & Salas, 1996), it is understandable that analyzing team tasks is significantly more complex than analyzing individual tasks. As teams research has seen its boom of interest in recent years (e.g., Brannick, Levine, & Morgeson, 2007; Kozlowski & Ilgen, 2006), much less attention has been devoted to the analysis of team tasks. When a literature search was conducted to determine the latest developments in team task analysis, the results were scant, returning fewer than 10 relevant articles, with most results analyzed at the individual level. At its core, a team-based job/task analysis is just an analysis of the job/task as performed by a team (Baker, Salas, & Cannon-Bowers, 1998).

While many of the initial steps which comprise a team task analysis mirror those done within a task analysis conducted at the individual level, there are some key differences. We will first briefly describe the initial steps which overlap with traditional task analysis. As with an individual-level task analysis, some of the initial steps within a team task analysis are to conduct a requirements analysis and identify the tasks which comprise the target job. A requirements analysis is the process through which the target job is defined, the conditions under which the job is performed are specified, and the specific tools utilized are identified. Once the target job is defined, this then guides the choice of the knowledge elicitation methods that will be used at later stages of the team task analysis. The literature has identified a wide variety of knowledge elicitation techniques, including but not limited to: observation, surveys, source documents, interviews, and work samples (see Goldstein & Ford, 2002).

The output of the requirements analysis not only serves to define the job but also facilitates the determination of who should serve as the subject matter experts concerning information pertaining to the tasks which comprise the job. Most often the subject matter experts identified at this stage are a combination of job incumbents and supervisors as both provide unique perspectives. The processes described within the requirements analysis mirror those found at the individual level and the outputs at this stage would most likely be found within the domain module of an ITS. While fairly straightforward the requirements analysis is important as it sets the stage for the remainder of the team task analysis.

After completing the requirements analysis, the next step is to identify the specific tasks which comprise the job. This typically entails a review of source documents combined with interviews with job incumbents in order to identify the full range of tasks completed within the job. This process is almost identical at the individual and team level, therefore the reader is referred to Goldstein and Ford (2002) for more information. This process is iterative in nature and usually results in a set of task statements written for each identified task. Task statements are worded such that they are direct, begin with a verb, and describe what the worker does – including how it is done, to whom it is done, and the purpose of the task (Goldstein & Ford, 2002). Next, we will highlight the unique factors that need to be considered when conducting a team task analysis.

UNIQUE CONSIDERATIONS IN IMPLEMENTING TEAM TASK ANALYSIS

While much of the early work on team task analysis used methods originating within traditional task analysis, later work suggested that not all methods and tools transfer across levels – there are some unique aspects to team task analysis (Arthur et al., 2005, 2012; Bowers, Baker, & Salas, 1994). Next, we highlight five such considerations that differ according to the level at which the task analysis is focused (i.e., individual or team).

Consideration 1: Does the Task Require Teamwork?

This first consideration represents the initial point at which team task analysis begins to diverge from traditional task analysis. Specifically, the focus at this stage of the team task analysis is to determine which of the tasks from the larger set of tasks identified for the job require teamwork. Put another way, which tasks require members to coordinate their work activities. As with many of the methods in team task analysis, there are a variety of resources that may be utilized to get the critical information. While at its simplest this may involve merely asking for each task whether there is a requirement to coordinate with others while working toward task accomplishment. However, recent work has argued that analysis should push beyond this superficial level of understanding. Specifically, Arthur et al. (2005) argue for the need to understand the specific type of interdependence required as this may impact the competencies, tools, and strategies used within an ITS.

The importance of the type of interdependence that characterizes team tasks was recognized long ago by Saavedra, Earley, and Van Dyne (1993). Specifically, Saavedra et al. (1993) characterized four levels of task interdependence that may occur in team tasks: pooled, sequential, reciprocal, and team intensive. Pooled interdependence represents tasks in which the workflow amongst team members is independent of other team members. In essence, this type of interdependence is characterized by an independent workflow whereby work enters and leaves the group with very little needed direct interaction amongst team members. Each team member makes a contribution to the work group, and performance is equal to the sum of the individual inputs. Under pooled interdependence members are only loosely considered a team as there is not much interdependent action.

A slightly more intensive team workflow is that of sequential interdependence which is characterized by unidirectional workflow whereby one member must act prior to another acting. Roles tend to be predefined in this form of interdependence and work flows in a prescribed manner. The most common example is an assembly line. Under conditions of reciprocal interdependence one member’s output becomes another person’s input and vice versa; workflow is dyadic but bidirectional. Different parts of the task are performed in a flexible manner, although role structure is imposed from outside the team. In the highest form of task interdependence, team members jointly “diagnose, problem solve, and collaborate to complete a task” (Saavedra et al., 1993, p. 63). Team interdependence refers to instances of simultaneous work interactions where group members have the freedom to determine the particular course of inputs and outputs amongst group members (Saavedra et al., 1993).

Leveraging this prior work, Arthur et al. (2005, 2012) developed and validated a scale that could be used to characterize the workflow pattern for a given task. Arthur et al. (2005) posited that any task can vary in the extent to which teams are necessary to perform the task, or as they put it, the “teamness” of the task at hand. In order to measure this, they developed scales for team relatedness (represents the amount of team interdependence) and team workflow (represents the type of interdependence) which can determine the varying level of interdependency that is inherent to any task. Specifically, team relatedness was rated on a 5-piont Likert scale for each identified task, while team workflow was indicated by choosing one of the workflow patterns as argued for by Saavedra et al. (1993). Workflow patterns ranged from 1 to 5, with pooled representing the low end of the scale and team interdependence representing the high end. Composites were created by averaging the two ratings, thereby providing a spectrum that defines levels of team interdependency along it. These scales have been shown to differentiate between levels of task interdependence (see Arthur et al., 2005, 2012).

Consideration 2. How to Operationalize Teamwork?

After the initial identification of which tasks require teamwork the next essential component is the identification of a teamwork taxonomy, as the components therein will serve to later guide the coordination analysis. More specifically, i...

Table of contents

• Cover
• Title Page
• Introduction
• Part I Concepts for Understanding Team Training
• Part II Team Assessment and Feedback
• Part III Team Tutoring Applications
• Part IV Summary
• Index