Project Performance Review

Capturing the Value of Audit, Oversight, and Compliance for Project Success

Alexia Nalewaik, Anthony Mills

• Publisher: Routledge
• Year: 2016
• Print ISBN: 9781472461407
• eBook ISBN: 9781317075066
• Topic: Business
• Subtopic: Business General
• Index: Business

Project Performance Review focuses on evaluating projects efficiently and in context, identifying important improvement opportunities and leading project and organizational management practices. It advises how these can be put in place to give stakeholders confidence in the control and delivery of their projects without waste.

The authors explain not just the mechanism and objective of project performance reviews but also the ideal environment in which they are intended to be implemented. The shaping of this environment, by the stakeholders and technical team, is key to achieving your intended outcomes. Without the professional cooperation of all interested and informed parties, the effectiveness of any review may be compromised. Topics addressed include: introducing the project review method, engaging project stakeholders, ensuring project governance, conducting project risk assessments, improving accountability, providing project assurance, organizing and managing projects, optimizing review scope and approach, avoiding review pitfalls, meeting existing audit standards, and proposing alternate approaches to project evaluation.

1  Introduction

The method presented in this book is framed as an approach to project review and is based on the principles of performance audit. It is also designed to comply with audit standards. The method is ultimately intended to be used to improve the project and project organization.

What is performance audit?

Audit and review are two of the oversight mechanisms most commonly used to provide assurance and give confidence to stakeholders. There are many types of audits: internal audit, expenditure audit, compliance audit, systems audit, financial audit, and more. No matter which type of audit is performed, it can be generally defined as a review of an organization’s activities or accounts by either an internal department or independent external organization and a promise or guarantee (assurance) by the auditors that the accounts are true except as reported in the audit findings.

A performance audit, specifically, is intended to review the Efficiency, Economy, and Effectiveness of an organization in achieving a particular defined objective through the use of certain available resources and can be utilized as a vehicle for organizational improvement and maturity. Where traditional audits answer the question “Was it done right?” by verifying compliance with procedures, specifications, and systems and assuring or questioning the accuracy of accounting, the performance audit asks, “Was the right thing done?” and looks at the panorama of context, situation, intent of behavior, and actions undertaken in the course of achieving organizational goals. Performance audit and reviews complement other types of audit and are similar to operational reviews in that they consider not just the past and present but also the future.

The need for project performance reviews

Projects can experience dynamic change, which then results in overspending, delays in on-time delivery, reduced quality, and other such failings in achieving their intended objectives. Such failures can occur in any project type, sector, or industry. For most organizations, project funding is not unlimited, which creates a very real constraint on the project that is difficult to overcome when cost overruns occur. Projects are often created in response to a specific time-sensitive organizational need; where there is a hard deadline for the delivery of a product or opening of a building, this may be more important and critical to the organization than cost. If a project is performing well, reviews may be conducted at regular intervals as a preventative measure. However, any volatility or occurrence of an unrecognized significant risk creates an urgent need for immediate diligent oversight of projects, especially those that receive public funds or donations and thus require an even higher degree of transparency and accountability. An audit or review may be triggered in such circumstances.

Many types of projects and the teams that create them are unique, nonrepeatable, diverse, and a bit unpredictable; these projects change constantly, often to the extent that change becomes the norm. Herein, a project is defined as a temporary organizational entity created to realize a specific goal; it is important to note that, whereas the project is temporary, the executing (Owner/client) organization or enterprise might not be. Traditional control mechanisms work most effectively when tasks are repetitive and predictable; some types of manufacturing projects lend themselves to this static control situation, in which change is planned and occurs during iterations. The trouble is, the world of projects is now so deep and broad that detailed rules, processes, and policies and procedures cannot be developed to cover all possible occurrences for all possible types of projects.

Very few if any projects are perfectly repeatable. Projects and systems that change rapidly do not lend themselves to a template. On dynamic projects, stakeholders must rely on the skills, experience, and judgment of the project team to make robust decisions, provide guidance and assurance, and call on experts as needed to provide timely investigation and advice.

Due to the magnitude of expenditure on major and mega-projects, termination often becomes unacceptable even when their extreme cost and schedule overrun become visible and/or the project viability is in question. It may be difficult for the project team or organization to admit defeat, accept the sunk costs, and exert the necessary authority to cancel the project, especially for very public or otherwise high-profile projects. There may be such a demonstrated or perceived need (especially where there is a public need) for the project that it cannot be cancelled at any cost. As a result, many projects continue to limp forward in the face of adversity, change, newly discovered risky conditions, slipped milestones, and ever-rising costs toward an unhappy conclusion. Project shocks and failures of the type mentioned may trigger the need for a performance review, wherein an independent team can consider the objectives of the project and attempt to recommend solutions to the issues raised.

Fortunately (or unfortunately) for most organizations, history, short memories, and positive spin tend in the long term to erase the label of failure, and measures of success are ultimately identified and promoted, even for projects that missed cost, schedule, and performance goals. The Sydney Opera House comes to mind as an example.

Increased stakeholder awareness of expenditures, conservatism, risk awareness, demands for transparency and accountability, and a greater focus on controls and obtaining value for money mean that stakeholders more often are turning to audits and reviews as a mechanism for control, risk management, and trustworthy reporting of project and organizational status. Stakeholders (especially the general public) have certain expectations of Economy, Effectiveness, and Efficiency, legality, ethics, stewardship, and equity; consultants performing the project review can report back to stakeholders on all of these and provide a judgment of whether the expectations are reasonable. Project review is an effective tool that can be used to support management practices and controls. The performance review method discussed herein provides a necessary assurance and pertinent advisory function to project Owners and their stakeholders alike.

Types of projects that can be reviewed for performance

Review mechanisms are typically designed to be generic, such that they can be applied to many different types of organizations and situations, and performance review is no exception. Although the authors’ experience with performance audits and reviews is specifically in capital (construction) projects, the Nalewaik-Mills Performance Review Method presented here has been designed such that it can be applied to any type of defined and bounded public- or private-sector project, such as: government, information technology, construction, manufacturing, major events, and more. It can be used in any industry sector (e.g., education, healthcare, utilities, petrochemical, pharmaceutical, commercial, infrastructure) and in any country, county, state, city, or agency. It is flexible enough to be useful to projects, programs, and portfolios and the organizations that create them.

What makes this book different?

There is no shortage of project management literature available, each outlining various methods and best practices for managing projects and purporting to ensure successful projects. There are countless member-based professional organizations that provide support, guidance, forums, training, and certification to project managers and their ilk. Universities even offer academic degrees in project management. Project management is not a new profession, and yet projects continue to fail. Results seem to defy reason. A project that follows established policies and procedures precisely, monitors itself against metrics and benchmarks, and implements all the current best practices can flop. Similarly, a project that does little of this can, heroically, succeed. There are too many intertwined variables to claim that certain actions and activities will guarantee project success. The discipline of project management is evolving, but project assurance demands something more, and here performance review steps in.

Books written on performance management for project management professionals tend to focus on the use of key performance indicators and metrics. The majority of books and guidelines written on performance audit appear to be intended for persons in the accounting and audit professions, internal audit, or government or people who wish to be in those professions. Books on performance review do not specifically address the challenges of such reviews, specifically the performance of projects. Books on project audit tend to focus exclusively on expenditure and compliance auditing, even though compliance (with policies and procedures, laws, regulations, contracts, etc.) is not a guarantee of project success. While there are books on project and construction audit, some of those seem to be focused on teaching certified public accountants (CPAs) and auditors about key concepts in project management instead of actually describing how to audit projects. In the authors’ experience, it is easier and more effective to teach project management professionals how to audit than it is to teach project management to an auditor.

Currently, there exists no universal, comprehensive methodology for performance audits and/or reviews. Guidelines are insufficient, journal publications are scant, practitioners are unregulated, and formal education is unavailable (one cannot receive an academic degree in performance auditing, and audit is not typically included in project management curricula). As such, methodologies continue to evolve, diverge, and (occasionally) regress; approaches taken by practitioners vary widely and unpredictably, perhaps at the expense of their clients.

There are a few exceptions in which published guidance on project performance audit exists in the form of manuals or written policies and procedures, usually developed by individual government agencies, counties, or cities and which can sometimes be found freely available on the internet. These have been designed specifically for use by those entities, reflect the specific nature, obligations, bureaucracy, and politics of those entities, and may contain guidance that is unique to the challenges and risks faced by those entities and their stakeholders. That said, some of these manuals do contain useful guidance that may be portable to peer entities and possibly even portable to other types of projects and enterprises. Of note is the 2015 European Court of Auditors (ECA) performance audit manual, which specifically differentiates among performance, financial, and compliance auditing, recognizes that the discipline of performance audit relies on evaluation methods, and emphasizes that performance audit requires auditor skills, experience, and education that differ substantially from those required for financial audit. This book is not intended to supplant such performance audit manuals, but it may be used to supplement them.

The Nalewaik-Mills Performance Review Method presented in this book¹ was developed in response to requests from both audit clients and practitioners for a flexible and relevant method specific to reviewing the performance of projects. It builds on several decades of audit and project management experience, a doctoral dissertation borne from seemingly never-ending frustration with lackluster audit findings and stakeholder expectation gaps. It draws from numerous works previously published by the authors on the topics of project performance, audit, assurance, governance, quantity surveying, project controls, project management, risk management, stakeholders, and quality assurance. The model is based on performance audit concepts and designed to comply with audit standards. It is designed to be flexible and modular, such that the review can be responsible and responsive to project and organizational needs at every milestone and beyond. The review can be used to supplement existing project management methods or stand alone.

This method isn’t a collection of checklists, and this book does not teach people how to conduct a review via a step-by-step process. Rather, the modules presented herein can be used alone, combined, and applied on an as-needed basis in any order at any time. It is intended, specifically, to assist Owner entities and their consultants in developing the scope of the review, appropriately procuring project performance review services, and overcoming typical challenges encountered during project reviews.

The model presented here:

• utilizes strategic and organizational management concepts to fill the published guidance void
• incorporates key attributes of program evaluation, which bring together internal departments, internal and external stakeholders, best practices, and the achievement of long-term goals
• incorporates risk management and quality management techniques
• enables both systemic and substantive evaluation, resulting in both quantitative and qualitative findings
• simultaneously focuses attention horizontally across the life cycle and vertically through stakeholder and organizational levels on established goals and risks
• ties internal management processes and project oversight to desired outcomes, identifying opportunities for improvement, and linking decision making to strategic objectives

As such, it assumes a fairly mature enterprise and organization of project management, but it can also be used to bring an organization (especially start-ups and mergers) to a higher level of sophistication by identifying what is missing, recommending improvements, and streamlining processes to decrease variability, increase optimization, improve predictability, and reduce risk.

The book opens with a definition of performance audit in Chapter 1, and discusses the specific need for such audits and reviews of projects. It then engages the reader with a more in-depth discourse about how and why performance audit evolved over time and explains specifically why performance audit differs from financial and other types of audit and review. The authors describe in Chapter 2 some challenges they have experienced with past project performance audits and reviews and then offer advice on how best to scope and procure a project performance review in order to avoid some of those challenges. Chapter 4 presents the audit framework. Each module is then described at greater length and in context, with specific challenges identified, discussion of how those affect project performance, and a reveal of what other specific problems consultants might wish to look for during the review. The section closes with a discussion of what elements of review should be procured separately and not included in a performance review. The book then concludes with some thoughts on the ongoing evolution and future of performance review and a brief recap of what was covered.

Note

1 Nalewaik-Mills Performance Review Method, copyright by the authors, 2014.

2 Discussion

In the history of audit, performance audit and (specifically) the application of performance audits to projects are relatively new concepts which have evolved and developed in recent decades in tandem with the auditing and evaluation professions. This evolution was driven by shifts in social values and the growth of strategic management theory.

Chapter 2 describes how performance audits and reviews have evolved over time and then highlights some typical challenges and difficulties that are experienced when conducting performance reviews. The authors then offer advice on how best to scope and procure a project performance review in order to avoid some of those challenges.

The evolution of performance audits and reviews

A typical audit can be loosely defined as a review of an organization’s activities or accounts. Most often, when the term ‘audit’ is used, it (almost by default) brings to mind a financial audit of accounting and expenditures. The same term might also conjure a mental specter of a big, scary auditor with colored pencils, green eyeshades, calculator, and coffee breath. The practice of audit can be traced back to the stock market crash of the late 1920s. Historically, public-sector audits were focused on financial accountability, internal controls, answerability, and fiscal regularity. By the 1950s, accounting and internal controls were considered to be satisfactory if an internal or external audit demonstrated past transactions were performed legally, formally authorized, and accurately and completely reported according to standardized accounting principles. Internal audit functions became the norm in public-sector organizations and private-sector companies alike. To this day, financial audit remains an important factor in the review of organizational accounting, certifying that financial statements fairly reflect the entity’s financial situation and operations for the period reported and applying fixed standards and methodologies in the review of financial controls.

In the 1950s and 1960s, times of fiscal austerity led to public concerns about fraud, deception, waste, and integrity in government agencies, necessitating an evolution of audit and management strategy toward compliance, documentation, and internal controls for both detection and prevention of fraud. There grew the recognition that transactions, although formally authorized and performed legally, may be willfully and knowingly unnecessary or excessive. The 1960s saw an increase in efficiency auditing and, by the late 1970s, bureaucratic control was the public-sector norm. A desire for error prevention and fraud deterrence yielded internal controls developments such as codified policies and procedures for routine activities and segregation of duties, which could be readily audited for compliance. Regarding organizatio...

Table of contents

• Cover
• Title
• Copyright
• Contents
• List of illustrations
• Preface
• Foreword
• Acknowledgments
• Acronyms and abbreviations
• 1 Introduction
• 2 Discussion
• 3 The Nalewaik-Mills Performance Review Method
• 4 Overview of the Nalewaik-Mills Performance Review Method
• 5 Summary
• Further reading
• Index