Early Warning Signs in Complex Projects

Ole Jonny Klakegg, Terry Williams, Derek Walker, Bjørn Andersen, Ole Morten Magnussen

• Publisher: Project Management Institute
• Year: 2010
• Print ISBN: 9781935589181
• eBook ISBN: 9781628251692
• Topic: Business
• Subtopic: Insurance
• Index: Business

Complex projects are often the most high-profile projects within an organization. How can early warning signs be identified and acted upon, so that problems are avoided and projects are successful in delivering the expected value for their owners and other stakeholders? What signals should we look for? Looking for early warning signs takes more than a keen eye. Collaborating with the Norwegian University of Science and Technology in Trondheim and the University of Southampton in the United Kingdom, Ole Jonny Klakegg, Terry Williams, Derek Walker, Bjørn Andersen, and Ole Morten Magnussen have expanded on their research of governance frameworks and guidelines as well as provided interviews with experts and case studies from Australia, Norway, and the United Kingdom. This international report identifies early warning signs in highly complex projects and offers tips on how to combat them.

1

Introduction

Project assessments are carried out worldwide. They are frequently referred to as reviews, audits, and health checks. The intent of these initiatives is to evaluate the extent to which the project documents are developed in accordance with expectations and formal criteria. The purpose is to support decision making and make sure that the projects are successful. One additional important aspect is, of course, to learn from past experience.

Still, we are often surprised at how projects turn out. Complex projects often do not behave in the way we expect, and in particular effects within complex projects are often time-delayed and take time to emerge. Project assessments need to look through the complexity of a given project in its context and identify the relevant early warning signs of project problems, project failure, under-performance, or cost overrun. Experience and current literature seems to indicate that we are not very good at picking up early warning signals.

Assessments are potentially important in securing the success of projects. To be effective, project assessment should be systematic and well adapted to the purpose and context of the assessment, as well as anchored in a governance framework established by the project owner/sponsoring organization. This study is designed to strengthen the basis for implementing effective project assessments.

Increasing complexity in projects and rapid changes makes the governance of projects increasingly challenging. Complexity in this context includes external complexities such as stakeholder relationships (Pryke & Smyth, 2006) and decision-making processes, and internal complexities such as technology and interfaces to existing systems (Williams, 1999). This increasing complexity suggests there may be a wide range of reasons why future projects may fail. In addition, the causal relations between early indications or incidents and later results are seldom obvious, but are often very complex. This makes it very difficult to know what to look for when assessing complex projects. This report investigates the practice of project assessments to identify how and to what degree the right early warning signs are identified.

1.1 Previous Work by the Research Teams

Previous work on governance frameworks has revealed how governance frameworks and project assessments are very closely linked. Existing governance frameworks include guidelines and instructions for performing project assessments and defining when and how they are used. Previous work by the research team has studied three existing frameworks in the UK and Norway (Klakegg, Williams, & Magnussen, 2009). By building on this work, extending it to Australia and the private sector, a wider platform of knowledge is established.

The previous work of Williams and associates covers a broad set of factors relating specifically to how to understand complexity (Williams, Eden, Ackermann, & Tait, 1995; Williams, 2005) and learn from projects (Williams, 2008). Understanding causal chains and the reasons for disruption and delay has been a major issue in their research (Eden et al. 2005). Further, the previous work of Andersen and associates on performance measurement adds to the methodological basis (Andersen & Fagerhaug, 2001). Earlier work by Walker and associates has built a strong platform for understanding the system sides of project assessments and diffusion of new knowledge (Walker & Rowlinson 2008). Also, work by Klakegg and associates from the Concept Research Programme in the governance of public projects and the performance of project assessments has added to the current knowledge in this field (Concept 2009).

1.2 Definitions and Basic Concepts

Most the Project Management Institute's general definitions of concepts are of relevance for this report (PMI, 2008). The authors have chosen the following specific concepts and definitions for this work. We use them throughout the report, although we also address other (more or less synonymous) words and concepts in the discussions and analysis.

Table 1-1 Definitions and Basic Concepts

Concept	Explanation
Assessment	In this report the term assessment is a wide concept, comprising all types of appraisals and examinations of project documents and practices in order to support decisions, learn from experience, or assure that expectations or formal criteria are met. A similar wide meaning seems to be used in the PMBOK® Guide—Fourth Edition (PMI 2008). We discuss more specific types of assessments (reviews, audits, etc.) in Chapter 2.
Assurance	Independent assessment and confirmation that a program^A as a whole or any of its aspects are on track, applying relevant practices and procedures, and that the projects, activities and business rationale remain aligned to the program's objectives (OGC, 2004). Further elaboration is given in Chapter 2.
Complexity	Complexity was first defined by Baccarini (1996) as “consisting of many varied interrelated parts,” operationalized in terms of differentiation (the number of varied elements) and interdependency (the degree of interrelatedness between these elements). Baccarini dealt with two types of project complexity: organizational complexity and technological complexity. Based on Baccarini's work and Turner and Cochran's goal-and-method matrix (Turner & Cochran, 1993), Williams developed the complexity concept further to include structure complexity and uncertainty—uncertainty in interactions, goals and methods (Williams, 1999). Complexity will be further discussed in Chapter 2 and the subsequent analysis chapters.
Early Warning Signs	An early warning sign is an observation, a signal, a message, or some other form of communication that is or can be seen as an expression, indication, proof, or sign of the existence of some future or incipient positive or negative issue. It is a signal, an omen, or an indication of future developments (Nikander, 2002, p. 49). Further elaboration is given in Chapters 2 and the subsequent analysis chapters.
Governance of projects	Governance of projects concerns those areas of governance (public or corporate) that are specifically related to project activities. Good project governance ensures that relevant, sustainable alternatives are chosen and delivered efficiently (Klakegg et al. 2009). Project assessments have to be anchored in an organization's governance framework, as discussed in Chapter 4.
Governance framework	Governance framework is an organized structure established as authoritative within an institution, comprising processes and rules established to ensure projects meet their purpose. A specific governance framework must be able to adapt to the given project and its context. Governance is a dynamic, multidimensional, interactive process involving a network of actors. The governance framework is time-dependant and self-organizing (Klakegg et al. 2009). We discuss the influence of governance frameworks on project assessments in Chapter 4.

^A Office of Government Commerce (OGC) focus program management. The meaning is similar when the word program is replaced by “project”. In this report, the concept of project is in focus.

1.3 Scope and Limitations

This study looks at how project assessments may be used to identify early warning signals in complex projects. This implies several aspects of scope definition for the research project and important limitations to the use of the results. The most important scope definitions are:

• Perspectives: This research represents a project owner's perspective (as seen from the organization or the firm). This is the strategic perspective—the view of a permanent parent organization. The success of a project has to be discussed in the owner's business environment and is a question of whether or to what degree the owner's intentions and purposes regarding the project are met. Success in terms of meeting targets within budget and time restraints (the supplier perspective) is still important but not enough to guarantee success (Atkinson, 1999; Jergeas, 2005).
• Countries/Sectors: The study includes guidelines, interviews, and cases from three countries: Australia, Norway, and the United Kingdom. These are the countries covered by the research teams involved. These are all wealthy, developed countries with a relatively well-established project tradition and culture. The results can be transferred to countries with a similar starting point in terms of economic thinking, culture, and administrative traditions. The study covers both public and private sector organizations in the countries included in the study.
• Project Types/Industries: The focus is on well-established industries with traditional investment projects including the development of some form of physical or digital infrastructure. Typically, this covers building and construction, energy, oil and gas, telecommunications, and information and communication technology (ICT). Research, product development, and defense projects are deliberately left out due to budget and time constraints, but references to interviews include those individual experts with experience from such projects.
• Phases/Timeline/Life cycle: Project management (PM) has traditionally focused on the execution phase. We acknowledge that the success or failure of any project may be determined well before the execution phase starts; consequently, this study focuses more on the early phases—the front-end. However, the entire timeline of a project and the complete life cycle of a project's results have to be considered when identifying early warning signals.

This study does not cover projects in developing economies where the governance framework is not developed or where a PM tradition and knowledge is not established. The study does not cover the aid and development projects of nongovernment organizations (NGOs).

The study is limited to a few categories of typical contexts (characterized by degrees of complexity) and only principal early warning signs, not detailed considerations of causality and intricate multiple influences. Classifying important signs identified in documents, interviews and case studies into context categories may give an indication of what signals are important in which contexts (for medium and high complexity projects).

1.4 Research Questions and Methodology

As mentioned in the introduction above, projects do not always turn out as they were originally intended. The obvious way to improve such a situation is to identify what may go wrong and then initiate a means to ensure that it will not happen. A wide range of analytical methods, computer tools, and project assessments have been developed in order to help avoid problems. Still, project owners and project managers seem to have limited success in picking up early warning signs. This report establishes a firm academic footing based on a literature survey, which serves to confirm that we understand the full background behind the causes of this phenomenon.

Traditionally, performance measurement in projects has been preoccupied with simple measures of time, cost, and quality. However, these factors are consequences of activities, incidents, and other conscious or unconscious actions, or even lack of actions. For the purpose of early warning in projects, there must be additional measures that can provide more relevant information. In the search for indicators that can serve as early warning signs in projects, we have turned to sources describing factors of project success and failure. Project management literature contains descriptions of so-called project success factors, or sometimes the inverse, project pitfalls. This topic has been extensively researched and is reported in Chapter 2.

In terms of factors that can be made subject to measurement and monitoring for early warning indications, several different avenues have been pursued. In IT projects, research findings indicate that people-related and process-related risks scored higher than product-related risks as dominant warning signs of IT project failure (Kappelman, McKeeman, & Zhang, 2006). Post-mortem analyses of unsuccessful IT-projects have shown that there are evident signs of symptoms well in advance of the final failure. We interpret this as an indication that improvement is indeed possible.

Syamil, Doll, and Apigian (2002) argued that behavior-related performance measures evaluating a project process are mediating variables affecting the extent to which the chosen process contributes to the overall project result. Supporting this view, Hoegl, Weinkauf, and Gemuenden (2004) found that collaborative processes during a project have predictive properties with regard to later team performance and can serve as early warning indicators. Balachandra and Raelin (1980) presented a model indicating that project success factors could be used for developing a model for early warning—an approach supported by Sanchez and Perez (2004). Investigating construction and building projects, Hanna and Gunduz (2005) found that a selection of variables from the pre-construction phase represented a project success rate of more than 70%. The model they developed could be used to define early warning signs for use in the tender phase, but did not lead to an explanation of how to conduct measurement and follow-up of the areas requiring attention. Jaafari and Jabiri (2006) and Jaafari (2007) proposed an approach to project diagnostics based on so-called project health checks, building partly on excellence models known in the quality management field and partly on maturity model principles. We will pursue these varying approaches further in Chapter 2.

In summary, many different approaches have been proposed in terms of what to measure as early warning signs. What is lacking seems partly to be empirical or practical validation of whether these approaches work, that is, whether they are truly capable of predicting emergent problems at an early stage. In addition, little work exists on how to use these measures in project organization. How does one collect the required data? Who should have access to the measurements? In which forums should they be discussed? What action should be taken? This report tries to shed some light on these areas.

The first research question is:

(i) How successful are current project assessment methods in trying to uncover early warning signs of problems?

Continuing our study of current literature, and expanding into governance frameworks and guidelines for assessments of projects we ask:

(ii) How do current project assessment methods try to uncover early warning signs of problems?

(iii) Why do methods fail to detect early warning signals?

This study of documented guidelines and instructions considers how and to what extent project assessments in the chosen countries/sectors are executed under established governance frameworks. The survey will indicate how frequently project assessments are reported in literature and public bodies such as National Audit Offices. The literature search has an international profile, whereas the study of public bodies is limited to the countries represented in this study.

Thus far, we have taken a positivist view to uncover what sort of problems there are. To look into solutions, we took a more phenomenological stance (Easterby-Smith, Thorpe, & Lowe, 1991). We considered the following three questions:

(iv) How can project assessments handle the complexity in projects and their context?

(v) Which practices seem more appropriate in which contexts?

(vi) What are the most important early warning signals to look for in the diffe...

Table of contents

• Cover Page
• Copyright Page
• Title Page
• Contents
• Executive Summary
• 1. Introduction
• 2. Literature
• 3. Reference Model
• 4. Guidelines for Project Assessments
• 5. Interview Analysis
• 6. Case Studies
• 7. Looking Through Complexity
• References
• Appendix: Interview Guide