Efficiency Measurement in Health and Health Care

Bruce Hollingsworth, Stuart J. Peacock

• Publisher: Routledge
• Year: 2008
• Print ISBN: 9780415569491
• eBook ISBN: 9781134487516
• Edition: 1
• Topic: Business
• Subtopic: Business General
• Index: Business

This book provides a concise synthesis of leading edge research in the theory and practise of efficiency measurement in health and health care. Whilst much of the literature in this area is confusing and impregnable, Hollingsworth and Peacock show the logical links between the economic theory underlying efficiency, the methods used in analysis and

1 Introduction

There has long been a perception that health services could be operating more efficiently. Concern over efficiency arises because health service resources are scarce, and given those scarce resources, health services cannot meet all the demands and needs of the population they serve. Some of those demands and needs will therefore go unmet, irrespective of how, when, where and to whom health services are provided. Inefficiency in the provision of health services will then result in greater levels of unmet need in the community and poorer levels of population health.

The commonly held view that health services could be operating more efficiently has been debated at the government level in the vast majority of countries. This debate has provided the impetus for radical health system reforms in many developed and developing countries in the last 20 years. Reform has occurred at different levels, for example at the state/local level in the USA and at the national level in the UK, and has taken many diverse forms, for example managed competition in the Netherlands and integrated care in New Zealand. However, despite the pervasive concern over efficiency and its role in driving health system reform, until relatively recently there have been few attempts to measure efficiency in health services, at least in terms recognizable to an economist.

Efficiency is a term widely used in economics, commonly referring to the best use of resources in production. From the 1950s onwards, following the seminal work of Farrell, economists have typically distinguished between two types of efficiency: technical efficiency and allocative efficiency (Farrell 1957). Technical efficiency refers to the maximization of outputs for a given level and mix of inputs, or conversely the minimization of input use for a given output level. Technically efficient behaviour can be mapped by plotting the different combinations of inputs that maximize outputs, which economists term the production frontier. Thus if an organization, such as a hospital, is technically efficient it is operating on its production frontier. Allocative efficiency refers to the maximization of outputs for a given level of input cost, or conversely the minimization of cost for a given output level. Allocative efficiency can be mapped by plotting the different combinations of inputs that minimize cost, which is termed the ‘cost frontier’. Similarly, allocative efficiency implies a hospital is operating on its cost frontier. When combined, technical and allocative efficiency comprise the ‘overall’ efficiency of an organization.

To measure the efficiency of an organization we therefore need knowledge of the production and/or cost frontier. In practice the frontier is made up of those organizations which are the most efficient in the sample of organizations under analysis. That is, the frontier consists of those organizations which produce a given level of output from the least inputs (or least cost), or produce the maximum output given a certain level of inputs (or cost). The level of inefficiency of organizations not lying on the frontier is estimated relative to these efficient organizations. Furthermore, efficiency changes from one period to the next – ‘technological or productivity change’ – can also be measured.

There are two main alternative empirical approaches to estimating frontiers: data envelopment analysis (DEA) and stochastic frontiers. These approaches have two fundamental differences. Stochastic frontiers, based on econometric regression techniques, are parametric and therefore require specification of a particular functional form. DEA, based on linear programming techniques, is non-parametric and does not require specification of the functional form. DEA is also non-stochastic, assuming that the distance an organization lies from the efficient frontier is due entirely to inefficient behaviour. Conversely, stochastic frontiers assume that the distance an organization lies from the frontier will be due a combination of random measurement error and inefficient behaviour. To date the application of stochastic frontiers to the analysis of health services has been somewhat limited. By contrast, DEA has been used extensively, with hundreds of published applications. This is perhaps because one advantage of DEA is that it is the only method available which easily allows the estimation of multiple input–multiple output models. Econometric methods usually require some degree of aggregation of the dependent variable.

This text defines efficiency clearly and its relationship to health and health care. There is confusion in this area at present. For example, The Handbook of Health Economics (Culyer and Newhouse 2000) is criticized by Rutten et al. (2001) for failing to address production and cost functions, and as an exercise the reader may wish to try and find a definition of efficiency in these volumes.

In what follows, we logically lead the reader through this potential maze, going on to a practical ‘how to do’ section, with a review of the software available to actually apply these methods. Following this exposition of theory and methods we give an up-to-date literature review of applications of efficiency measurement in health care, drawing out important methodological and policy implications of work undertaken so far. Following on from this and in part based on lessons learnt in the text we report new examples of practical applications of advances in this area. This is work which has been, and is being, undertaken by the authors, in collaboration with others. This includes a comparison of the different methods of analysis, with consequent policy implications; analysis of economies of scale and scope; modelling and consequences of restricting the weighting given to different variables; model specification; the efficiency of the production of health, as well as health care; and analysis in differing health-care settings.

Finally, we look to the future, based on our knowledge of what has been undertaken, what is currently being undertaken, and what needs to be done to advance this critical area in health economics. In summary, our text is a synthesis of theory, practice and leading edge research, separating the wood from the trees. It should appeal to a wide audience, including academics, practitioners and students, who find this area confusing and impregnable at present. This is especially important given the increasing references to the importance of efficiency in health services throughout the developed and developing world, from the level of the individual patient and the efficient production of health, through to the efficiency of entire countries and their health-care systems.

This chapter provides an introduction and description of the significance of the book, following the theme outlined above. It outlines the rationale for examining efficiency in the health sector, drawing on the established principles of economics and health economics. The chapter concludes with an outline of the book, and what each chapter seeks to achieve.

Why measure efficiency in the health sector?

Reinhardt (1998) castigates distinguished economists for misuse of the word ‘efficiency’ in the health-care environment. He singles out Nobel laureates Milton Friedman and Gary Becker as being ‘cavalier’ in using the term efficiency in normative statements regarding health policy. Reinhardt goes on to state that advocating one system above another, for example a market system as opposed to a system characterized by government intervention, based on efficiency as defined by certain economists is not based on what he calls ‘economic science’ and that comparing systems with different social goals makes no sense. This does not mean efficiency should not be measured, but that the term itself should not be ‘misused’.

In this book we draw on Reinhardt’s vision of economic science and attempt to avoid making normative statements as to what efficiency should be. We define efficiency in the strictest economic sense and go on to look at quantitative means of measurement in the context of well-specified models, founded on economic principles. Suffice to say the term ‘efficiency’ is frequently used inappropriately, and if economists as guardians of the term cannot use it appropriately, its misuse by others is inevitable. In particular, efficiency does not just mean operating at the lowest cost or achieving the best outcomes possible, regardless of costs. Both sides of the equation need to be examined together. As economics students learn at an early stage, production depends on the inputs to and outputs from the process. Looking at one side of the equation in isolation is ultimately meaningless for the efficiency analyst. Unfortunately this basic lesson is often forgotten, although it has been revived in some health economics texts recently (Folland et al. 2001; Rice 2002). Rice (2002) also points out the potential confusion in the view that ‘markets are efficient and governments are inefficient’. He argues that this view is wrong as it is based on ‘a misunderstanding of economic theory as it applies to health’. The economic assumptions for a market system to operate efficiently are not met in the health sector, which is why there is government intervention in all countries in this sector. This book does not seek to advocate one health system over another. We seek to clarify what efficiency is in economics terms, and how it is applied to the production of health care and health itself. It does not review each available system in terms of whether markets or their alternatives are ‘best’, or even critique these systems (others such as Rice (2002) have undertaken this comprehensively). We should also state at the outset that we are not writing a book about efficiency and equity in health systems, although of course we acknowledge that these considerations run hand in hand. There is a vast literature on this already (for example, see Chapters 9, 10, 34 and 35 of The Handbook of Health Economics (Culyer and Newhouse 2000)).

Our intention is to focus on the theory of and measurement of efficiency in health, which has received far less attention in health economics texts. We have produced a framework for how economists who ply their trade in the health sector can consider and measure efficiency based on economic theory. No more, no less.

Outline of the book

Health and efficiency

The concepts and definitions of efficiency adopted in health economics have often been confused, and have received relatively little attention in the literature. Chapter 2 introduces the reader to key concepts and definitions in studying efficiency in the health sector. Efficiency is often defined in a range of ways, and this has implications for both analysis and policy makers. We discuss the range of definitions of efficiency in health economics and the implications of this. The chapter offers a structured presentation of concepts and the theory of production from the economics and health economics perspective. Using the economic theory of production, we introduce readers to key concepts, including technical efficiency, cost minimization, allocative efficiency, and production and cost functions, drawing an important distinction between the production of health and the production of health care. Finally, we discuss relevant output measures and health economic efficiency concepts.

Efficiency measurement techniques

In Chapter 3, we discuss the theory and measurement of efficiency. The theoretical foundations are based on the work of Farrell (1957) and include the theory of production and cost frontiers and their relationship to production and cost functions, leading on to the measurement of technical and allocative efficiency using radial measures. We then describe three alternative approaches to measuring efficiency in the health sector: ordinary least squares (OLS) regression analysis, data envelopment analysis (DEA), and stochastic frontier analysis (SFA).

OLS draws on Feldstein’s seminal work on efficiency in the health sector (Feldstein 1967), which uses classical linear regression to estimate a cost/production functions for a sample of health-care providers. Residuals from these models can be used to tell us which providers are above or below average efficiency levels, as measured by the OLS average, and by how much. Criticisms of this approach include that OLS does not identify truly efficient behaviour as efficiency estimates are not related to a production frontier, but are based on average performance.

DEA creates a production frontier for a sample of providers using linear programming. It identifies efficient providers, which make up the frontier, and provides estimates of efficiency of all other providers relative to that frontier. The key features of DEA are described, including: non-parametric and non-stochastic estimation of the frontier; multiple inputs and outputs; and, input minimization versus output maximization variants of DEA models. Malmquist indices are then described, which are a means of measuring productivity over time using DEA. The index can be decomposed to show if changes are due to technology change (movements in the frontier from one year to the next), changes in efficiency (how far a provider moves from the frontier in each time period), and changes in scale of operation.

SFA estimates the production/cost frontier for a sample of providers using regression based techniques. The frontier is estimated by decomposing the error term into two parts – a one-sided error term that measures inefficiency and a more usual normally distributed error term that captures random influences. Key features of SFA are described, including: parametric and stochastic estimation of the frontier; choice of functional form; choice of distribution for the inefficiency term; testing of model assumptions; the treatment of casemix; estimation of economies of scale and scope; estimation of marginal costs; and, interpretation of rankings of efficiency. The chapter ends with a comparison of DEA and SFA as alternatives for frontier estimation.

Measuring efficiency in health services

Chapter 4 develops a framework for the practical modelling and measurement of efficiency in health services. Several issues are considered, from model specification to feeding back of results to those who may actually find them to be useful.

We structure this chapter around a number of questions: What is to be measured? Why? And for whom? In most cases the answer to the first question is relatively straightforward. We could be looking at the technical efficiency of a sample of hospitals, for example. The second and third questions are sometimes more difficult. Initially, we are usually concerned with increasing the amount of health care that can be delivered given certain resources, but increasingly factors such as quality of care are introduced, making analysis complex. Careful consideration of a range of other factors is important, for example in examining efficiency in the hospital sector how do we account for teaching and research, and the impact of this on health care? For whom we are measuring efficiency is also of interest. Studies can range from an academic exercise to advance a particular technique, to a study commissioned by health authorities to develop a practical measure for health service managers? Or, is the study to develop a ‘high level’ measure to be used to promote health system level efficiency?

To undertake an efficiency measurement study several practical steps need to be taken once the study perspective has been established. These include data collection, model specification, sensitivity analysis and reporting of results. So far ‘rules of thumb’ have often been used to guide choices for each of these steps. We examine this trend, and look more closely at validation techniques. The translation of empirical findings into policy tools is another area that has received little attention in the literature. We show how you can feedback study findings to decision makers and demonstrate some critical factors in translating results from efficiency measurement studies into practical policy instruments.

Importantly analysts need to choose an appropriate software package for data analysis. We review the current software available to undertake efficiency measurement analysis, ranging from complex to easy to operate, each with advantages and disadvantages.

Applications of efficiency measurement in health services

Next, Chapter 5 undertakes a comprehensive review of applications of efficiency measurement in health care. Papers are reviewed from the perspective of determining methods and data used, models specified, sensitivity analysis used, and validity and robustness of techniques. Results are summarized in a form of meta-analysis and some implications drawn.

This review contextualizes the lack of direction in this area, perhaps due in part to the lack of information available to researchers on what has been undertaken so far. It is important for a researcher in the field to examine the directions taken by their peers, in order to place ones own work in context. It is hypothesized that much work undertaken and published in this area is of the nature of ‘have software – will analyse’, perhaps setting a dangerous precedent, in terms of research that has a weak underlying basis in economic theory. This may mean ‘efficiency’ results being produced that potentially lead to policy changes based upon invalid models and unreliable information.

Drawing out the consequences of the literature helps us to set in place robust foundations and guidelines for a research agenda in this area. We also make the references we find in the review available in the form of a database, published as an appendix which summarizes comprehensively all our findings – a useful resource in itself.

Advanced applications and recent developments

Based on our in depth knowledge of current practice in this area, and our own research in progress, Chapter 6 covers some of the areas that are currently in deficit in research terms. For example, comparison of different methods of analysis, and potential policy implications. We compare SFA and DEA methods in terms of cost and production frontiers, testing robustness and properties of the efficiency measures generated. We critically explore the appropriateness of techniques under alternative study assumptions, settings and perspectives, and, importantly, over time.

We then examine the impact on efficiency of the size of health care ‘provider units’, and the scope of different services offered – is it more efficient to specialize, or diversify and jointly produce a selection of outputs?

Within efficiency modelling different variables may be perceived as more important than others, for example within a hospital it may be that teaching is seen as more important than providing a minor injury service. We explore whether the means of restricting the weights given to variables within the analysis, so that differing levels of importance are attached to different variable, are valid, and whether results can be used to inform policy choices.

Efficiency measurement to date has, in the main, estimated the production of health care. Health care is just one input into the production of health itself. We look at the efficiency of the production of health using data on oral health and health care in order to establish what the impact of the production of health care is on the production of health. Finally, we extend the application of efficiency measurement, introducing quality adjusted health care outcome variables.

Future directions in th...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• List of Fegures
• List of tables
• Foreword
• Acknowledgements
• 1 Introduction
• 2 Health and efficiency concepts
• 3 Efficiency measurement techniques
• 4 Measuring efficiency in health services
• 5 Application of efficiency measurement in health services
• 6 Advanced applications and recent developments
• 7 Future directions
• Notes
• Bibliography