Questioning Performance Measurement: Metrics, Organizations and Power

Guy Redden, SAGE Publications Ltd

• Publisher: SAGE Publications Ltd
• Year: 2019
• Print ISBN:
  9781526461858
• Edition: 1
• eBook ISBN:
  9781526462862
• Topic: Social Sciences
• Subtopic: Sociology
• Index: Social Sciences

Questioning Performance Measurement: Metrics, Organizations and Power is the first book to interrogate the organizational turn towards performance metrics critically. Performance measurement is used to evaluate a diverse range of activities throughout the private, public and non-governmental sectors. But in an increasingly data driven world, what does it really mean to measure 'performance'?

Taking a sociology of quantification perspective, this book traces the rise of performance measurement, questions its methods and objectivity, and examines the social significance of the flood of numbers through which value is represented and actors are held accountable.

An illuminating read for students, scholars and practitioners across Organization Studies, Sociology, Business and Management, Public Policy and Administration.

1 The Performance Revolution

In translating activity into numerical values amenable to analysis and decision making, performance measurement (PM) is now integral to contemporary organizations. Its techniques are evident almost everywhere throughout institutional life across the private, public and non-governmental sectors. These techniques shape the working practices of millions of people engaged in otherwise different fields. Monitoring, rating and ranking of processes and outcomes have become normalized, as have claims that the information produced promotes quality assurance, productivity, improvement, accountability and transparency.

As Gregory (1993: 281) notes, the desire to measure performance is not itself new. It is inherent to any organization concerned with the economical creation of value. Many of us are ‘unwittingly involved in performance measurement every day’ (Spooner 2002: 117). Performance review as a broader category involves a potentially wide range of methods and kinds of information, from direct observation or checking of work by supervisors (Ouchi 1979: 834) to more formalized kinds of periodic reckoning such as appraisal, audits or financial reports (Power 1997). Overall, performance review can be considered the act of asking ‘how well’ some kind of collectivity or members of it are doing against certain criteria. Managers review those persons and units for which they have oversight. Workers and units review themselves against goals and expectations. Shareholders and analysts review the performance of whole businesses. Governments are concerned with the performance of the overall economy and public services, while international organizations rate entire nations against criteria valued in international governance. It seems somewhat natural that we ask the ‘how well?’ question if we are concerned with doing ‘a good job’.

But while all organizational actors engage in some kind of performance review, my interest in this book is the striking rise and institutionalization, over the last three decades, of systematic measurement. Computing has made the collection and processing of large amounts of performance data cost-effective (Chapman 2005: 2), and more complex data sets have become evermore manageable (Eccles 1991: 134, Marchand and Raymond 2008: 670). Rudimentary interest in quantifying performance has been displaced by extensive elaboration, making it something ‘numerically’ visible, pervasive and manageable. Before the 1990s ‘performance measurement’ had little meaning outside a few academic specialisms concerned with modelling specific kinds of human, organizational and financial processes. What then is the social and cultural significance of its transformation into a general organizational practice? In this book, I propose that it is a form of statistical reasoning intimately tied up with social and epistemic developments in the era of its formation – an era which has seen a proliferation of metrics, if not the ‘datification’ of society (Beer 2016), and which demands interpretation of ‘social settings characterized by a high frequency of circulating numbers’ (Vollmer et al. 2009: 619).

PM could be considered entwined with the broader category of contemporary performance management (Fryer at al. 2009). Although the two should not be entirely conflated, performance management – itself the formalization of the desire to improve performance – makes much use of data derived from PM (Folan and Browne 2005: 674) and, in reality, performance measurement, performance reporting and performance management are deeply connected in contemporary organizations (Mizrahi 2017: 102). All depend on the idea that the abstraction ‘performance’ can be mapped and influenced in the multiple forms it may be seen to take. While appraisal directed at individuals and units has been a specific function of human resources since at least the 1960s, the urge to ‘manage’ performance across organizational dimensions and sites is now fundamental to management itself. The rapid proliferation of PM across institutional domains is nominally down to the fact that it offers a set of principles for quantification of performance that are seen as valid and transferable across contexts. Its principles are general but can be localized, customized and applied to anything. We must ask why and how the ‘need’ to measure performance has emerged, as well as what ends the practice may serve. However, as it promises nothing less than a means to evaluate how well human individuals and collectives act – with social consequences likely to follow – we must ask from a critical social perspective suspicious of the positivist tautology that may all too easily suggest performance ‘improves’ wherever it is measured. We must explore what this means outside of the technical terms often used to define and validate the effectiveness of PM.

What is performance measurement?

In a sense, performance measurement is very simple to define. For Neely et al., writing with regard to business, it ‘can be defined as the process of quantifying the efficiency and effectiveness of action’ (2005: 1229). Yet, beyond this generic level – which essentially just confirms that PM is about the measurement of performance – it is surprisingly hard to identify a set of core features. In setting out to define business PM, Franco-Santos et al. (2007) note how most publications in the area fail to offer definitions at all, while those that do so offer a wide range. ‘Performance’ can be deduced in almost any activity, but its identification depends upon the initial starting point of the given researcher, if not their intuition (Folan et al. 2007: 606). Richard et al. find that ‘organizational performance is the ultimate dependent variable’ in management research (2009: 719), appearing as a stated interest in a third of articles in two leading journals in the field. And yet there is very little consistency in its definition or measurement. Across the 213 papers in which it appeared as a dependent, independent or control variable, 207 different measures of ‘performance’ were used.

This apparent vagueness, or flexibility, with regard to PM’s central object is notable. Metrology in the natural sciences measures physical signals. However, the ontological status of the phenomena being quantified may be less clear when it comes to evaluating organizational outcomes. What exactly is countable – or rendered countable? The promise to produce hard data entails constructs of what performance is and how it is best measured that are ultimately determined in the dynamically changing social situations they are applied to by those with the ability to do so. As Lebas argues, what success in action or ‘good’ performance is construed to be is relative to whatever objectives one may have and what one may value: ‘Few people agree on what performance really means: it can mean anything from efficiency, to robustness or resistance or return on investment, or plenty of other definitions never fully specified’ (1995: 23). This suggests that the ostensibly ‘objective’ quantification of performance is very much tied up with processes of making meanings about potentially complex multi-dimensiona situations – situations that can be influenced by multiple actors and about which differing views might be possible.

For now we can only distil some of the core features present from all variants and approaches. PM is the use of statistical methods – however simple or sophisticated – to answer the ‘how well?’ question. In practice, it is to be understood as a concern to establish the ‘extent of value’ of what is done in organizations in such a way that performances can be understood through data collected against specified measurement scales. Accordingly, it revolves around a particular kind of performance information: quantitative data that is collected and interpreted to be a representation of how well things are going. In some cases, the quantification might seem simple, such as counting items produced or cases processed in a time frame. At other times, as in the example of measuring the ‘quality’ of social case work in challenging situations, the quantification of quality appears more challenging. Either way, measurement requires phenomena to be defined in certain measurable ways, and interpretation and subsequent use of data in social settings follow. Questioning the potentially complex social lives of numbers entailed in this, and their potential roles in the operation of power, are the fundamental interests of this book.

Long live the revolution

Although not all its methods are new or exclusive, PM’s proponents who helped it to become a normal aspect of organizational life transformed the common-sense idea that it is good to review performance into revolutionary zeal. Eccles, writing in his famous Harvard Business Review piece ‘The performance measurement manifesto’ (1991), proposed that managers should go beyond the limited existing diagnostic techniques of management accountancy and instead measure every dimension of an enterprise that matters. Only by building systematic frameworks that capture the value of all important activities can one master all the variables that make for success. And once this is achieved the ‘next step is to align the new system with the company’s incentives – to reward people in proportion to their performance’ (Eccles 1991: 135). The organization in this view becomes a dynamic, motivated, fully informed collection of beings. The alternative, in Eccles’ vision, is to be left behind by those who garner the advantages of such new improved performance data. Measure or perish.

Clearly advocates such as Eccles claim that PM is more than a dry exercise in descriptive statistics. It is a tool for social intervention. Generating and using performance data is seen as key to better-functioning organizations. To cite some indicative publication titles, it is variously a way of ‘increasing total productivity’ (Kaydos 1998), ‘maximising performance and maintaining results’ (Niven 2002) and promoting ‘organizational excellence’ (Moullin 2007). As there is nothing inherent to measurement itself that ensures improvement, it must be considered to provide critical information for organizational actors to act upon for such claims to be true.

It seems there was no turning back after the ‘revolution’ of the early 1990s. Metrics now dominate performance review and many organizations have adopted integrated measurement frameworks that report data from all key functions and at all organizational levels. As Neely et al. (2005) report, as of the early 2000s between 30% and 60% of companies had adopted the single most-widely known, off-the-shelf PM framework, the Balanced Scorecard. Frost (2007: 6) has the percentage at 80% of Fortune 500 companies in the USA in 2007. Micheli and Manzoni cite a report suggesting that ‘an average company with $1 billion sales spends over 25,000 person-days per year planning and measuring performance’ (2010: 466), and one Internet search revealed more than 12 million websites dedicated to business PM (Marr and Schiuma 2003: 680).

Meanwhile, public sector organizations’ reporting of performance data to government and the public has been the legally mandated norm since the 1990s in many countries. The reinventing government movement that swept through public administration in that decade led commentators to proclaim that ‘Performance measurement, the regular collection and reporting of information about the efficiency, quality, and effectiveness of government programs … is arguably the hottest topic in government today’ (Nyhan and Martin 1999: 348). It was widely touted as ‘the new way to ensure a focus on results and accountability’ in the provision of public services (Perrin 1998: 368). One text that shares a similar status to Eccles’ ‘manifesto’ in encapsulating and advocating the need for change was ‘The case for performance monitoring’ published in the Public Administration Review in 1992 (Wholey and Hatry 1992). In it leading figures in public sector measurement summarize the innovations of what was then an emergent practice, and make the case that it is needed at all levels of government for agencies to ‘provide timely information on the quality and outcomes of their major programs’ (604). Proclaiming that ‘The time is right for action’ (605), the authors outline the need for regular measurement and comparison of data to inform customer-focused, strategic management that demonstrates value for public money. There is every sign that the PM revolution in the public sector was as comprehensive as it was in business.

The new common sense suggested that measurement should be built intomanagement procedures and organizational routine rather than just conducted episodically (Cirincione 1998: 394). This differentiates PM somewhat from that other form of ‘checking up’ with which it otherwise shares a great deal in common: audit. However, changing approaches to audit mean that, although it remains associated with periodic review by external agents, it has become increasingly concerned with performance issues such as value for money rather than probity of procedures and accounts (Power 1997). There would be few kinds of audit today that do not draw upon organizations’ own routinely collected performance data. Indeed, performance auditing that uses performance indicators above and beyond financial information is one of the key variants of contemporary audit (Scott 2003).

Like audit, the pervasiveness of PM has given rise to a burgeoning technical field not only of practitioners but also of researchers and educators. PM is now a standard component and specialism within public administration, management and business education, fields that have themselves grown dramatically. Design and implementation of frameworks is also one of the heartland areas of contemporary management consultancy: questions of whether people are making their numbers, or what numbers they should make, are often deferred to outsiders. On reviewing the state of the literature in the 1990s, Neely (1999) discovered that 3600 academic articles were published on the topic between 1994 and 1996 alone, leading him to validate the scholarly existence of ‘the performance measurement revolution’ that Eccles had called for. Noting Neely’s statistic, Bititci et al. (2012) illustrated the scale of the academic literature through keyword searches using relevant terms including ‘performance measurement’, ‘performance management’ and ‘performance indicators’. The results identified that over 200,000 articles on the topic had been published over time, a finding consistent with Neely’s more restricted search period.

Although one review found that up to a quarter of PM texts concentrate on the public sector (Pongatichat and Johnston 2008: 207), studies oriented towards private enterprise account for the majority of published work. However, it is interesting to note that (at least in 2003) beyond a few ‘big names’, Marr and Schiuma found that ‘the field seems to be very diverse with over 95 per cent of all authors referenced only once or twice’ (2003: 681). The scale of the PM phenomenon should not be taken to mean that a common theoretical foundation has emerged. Rather, there is a ‘vast array of disparate information concerning PM’ due to ‘a proliferation of approaches … across a range of disciplines’ with limited cross-fertilization (Chenhall and Langfield-Smith 2007: 227).

Exactly how the ‘how well?’ question is posed and answered quantitatively varies greatly. Franco-Santos et al. (2007: 784) point out that areas as diverse as strategy management, operations management, human resources, organizational behaviour, information systems, marketing, and management accounting and control are contributing to the field. One of the issues raised by this diversity is that each has particular uses and aims. Marr and Schiuma (2003: 685) go as far as to say that scholars from different disciplines work on the topic in functional silos – PM for marketing, PM for human resources, and so on – rather than talking to each other across boundaries of specialism. While PM has diffused across social fields precisely because its core principles are transferable, it also appears everywhere in customized forms.

Why this book?

The flipside of its diversity is that PM has not been extensively considered as an overall practice from a critical social-studies viewpoint. Given the demand for technical know-how, it is to be expected that academic research develops principally to support design and implementation. Furthermore, other social research that identifies PM tends to view it as an aspect of organizational life in particular domains – such as the public sector and its subsectors, or specific kinds of business, or through the prism of issues in a particular discipline, for instance accountancy. Much of the practitioner literature is concerned with the forms in which and the conditions under which PM may or may not work, rather than its social, cultural or historical significance. In sum, most research into PM is either technical or contextual in character. Yet, in another sense, given its reach and use to evaluate people’s work, PM is everyone’s business – or should be.

In this setting the primary aim of this book is to conceptualize PM for readers who are interested in understanding organizations in their broader social and cultural contexts. For the reasons of scale and diversity just outlined, this does not involve an unrealistic bid to master the field and any of its aspects or disciplinary approaches, or to effect final arbitration between competing schools, claims or empirical research. Rather the aim is to generate ways of understanding PM in order to facilitate reflection on its socio-cultural significance. To do this I review languages that crop up in the practitioner and research literatures as a basis for conceptualizing PM and identifying critical issues. I seek to make connections between PM’s various manifestations so as to develop a deeper sense of it as a socio-historical formation, asking why and how it has developed, and not only whether ‘it works’ but potentially on what terms, in whose interests and to what ends. It should also be noted that my dependence on Anglophone literature is another reason why I offer indicative rather than definitive or universal answers.

My own interest in PM began as someone grappling with the effects of the ‘audit culture’ in education: that is, the recent tendency for assessing how well research and education institutions are performing against metrics derived from their activity, principally high-stakes tests in schools (Redden and Low 2012) and the counting and rating of the research outputs of universities (Redden 2008, 2013). In other words, I have been one of those scholars concerned with making sense of how PM operates in, and bears upon issues pertinent to, an organizational context. Yet I came to realize that the issues were not entirely local, representing only a shift in the logic of educational institutions. Rather what was going on in education was a manifestation of a broader turn to metrics across institutions, and potentially even broader shifts in management, governance and economy that made measurement of performance such an imperative. First this realization came from reading the literature about PM in the public sector, and then by coming to appreciate that, although performance review has always been present, systematic PM is also relatively new in all organizational sectors.

Of course, the many millions of people who are subject to its terms experience PM through the specific forms it takes. On the one hand it is a technical area with its own array of transferable methods. Yet on the other, for most workers it is hard to disentangle from what they do. As Giddens (1991) notes, in a world of expert systems everyone, including any expert, is a lay person relative to other fields of expertise. PM has filtered into organizational life often largely without being called upon to explain itself. It is a practice that ingresses into other practices, keeps watch over them and analyses their value. Yet for a non-expert it is hard to identify PM and its features because it always exists close up, in instituted forms. It largely manifests as the KPIs (key performance indicators) through which activity is evaluated. ...

Table of contents

• Cover
• Half Title
• Publisher Note
• Title Page
• Copyright Page
• Contents
• About the Author
• Acknowledgements
• 1 The Performance Revolution
• 2 What is performance measurement? Nuts, bolts and critical issues
• 3 Perspectives On Performance Measurement
• 4 Making The Numbers: Performance Measurement In Business
• 5 Magical Numbers: Performance Measurement And Public Goods
• 6 Rethinking Performance Culture
• References
• Index