Poor Numbers

How We Are Misled by African Development Statistics and What to Do about It

Morten Jerven

• Publisher: Cornell University Press
• Year: 2013
• eBook ISBN: 9780801467608
• Topic: Economics
• Subtopic: African History
• Index: Economics

One of the most urgent challenges in African economic development is to devise a strategy for improving statistical capacity. Reliable statistics, including estimates of economic growth rates and per-capita income, are basic to the operation of governments in developing countries and vital to nongovernmental organizations and other entities that provide financial aid to them. Rich countries and international financial institutions such as the World Bank allocate their development resources on the basis of such data. The paucity of accurate statistics is not merely a technical problem; it has a massive impact on the welfare of citizens in developing countries.Where do these statistics originate? How accurate are they? Poor Numbers is the first analysis of the production and use of African economic development statistics. Morten Jerven's research shows how the statistical capacities of sub-Saharan African economies have fallen into disarray. The numbers substantially misstate the actual state of affairs. As a result, scarce resources are misapplied. Development policy does not deliver the benefits expected. Policymakers' attempts to improve the lot of the citizenry are frustrated. Donors have no accurate sense of the impact of the aid they supply. Jerven's findings from sub-Saharan Africa have far-reaching implications for aid and development policy. As Jerven notes, the current catchphrase in the development community is "evidence-based policy," and scholars are applying increasingly sophisticated econometric methods—but no statistical techniques can substitute for partial and unreliable data.

1

WHAT DO WE KNOW ABOUT INCOME AND GROWTH IN AFRICA?

What do we know about income and growth in sub-Saharan Africa? The answer is: much less than we like to think. The data are unreliable and potentially seriously misleading. The question is of great importance. Economic growth rates or per capita income estimates are commonly used in statements about development in Africa. Sometimes the data are used to buttress a claim, and other times they are the starting point in defining a problem. If income and growth statistics in Africa do not mean anything, a great part of development analysis and policy targets are similarly meaningless.

The most pressing problem with the quality of data is ignorance among those who use the data. The scholars who are best equipped to analyze the validity and reliability of economic statistics are often data users themselves and are thus reluctant to undermine the datasets that are the bread and butter of scholarly work. When concerns with data are expressed, they are usually limited to some carefully phrased caveats in footnotes. International institutions are the main providers and disseminators of the data, and their programs and plans are often tied to targets and indicators. Thus the pragmatic approach is to accept the data at face value. Privately or in technical consultations advice may be given or direct pressure may be applied during the process of producing the data. Finally, on the domestic political scene, there is little to no transparent debate about the issue. The lack of economic literacy is a problem, and when statistics become the centerpiece of domestic debates, technical discussions give way to political agendas. Thus, the issue of data quality is doubly blurred.

At the same time, both the dependence on and demand for economic statistics is increasing. The aims of development are increasingly stated as quantifiable targets, as they are framed, for instance, by the Millennium Development Goals. The buzzword in the development community is “evidence-based policy,” and scholars are using increasingly sophisticated econometric methods, borrowing metaphors and methods from the medical sciences in their work, as if observations about economic development have the accuracy of laboratory experiments. The impression of measurability and accuracy is misleading, and that has broad implications across social science disciplines that deal with issues of African development.

This chapter starts by explaining the concept of national income accounting. It then presents a general picture of how national accounts are implemented in Africa. Empirical evidence shows that the quantitative basis for knowledge about African economic development is very fragile. Leading scholars know that the data are weak, but most data users are incapable of judging exactly how weak and how this weakness affects policy analysis.

What Is National Accounting?

National income measurement is governed by a global standard: the United Nations System of National Accounts (SNA). The foundations of this system were laid out by the Committee of Statistical Experts set up by the League of Nations in 1939. The committee produced a “Recommended System of Accounts,” a paper written by Richard Stone. The first version of the SNA was created by the National Accounts Research Unit of the Organization for European Economic Co-operation.1 This unit, chaired by Richard Stone, produced “A System of National Accounts and Supporting Tables” in 1953. The standards of national accounts have since been revised three times, so that there are four versions:2 in addition to the SNA 1953, there are also SNA 1968, SNA 1993, and SNA 2008. However, Michael Ward argues that “although they pay lip service to the subsequent revisions…many countries still adhere to the basic system and its corresponding accounting foundations as first set out.”3

Many people are not familiar with this system and may scarcely have heard of it, but it is the framework that generates most of our basic information about national economies. Its main product is the most important development indicators of them all: national income and economic growth. In theory, this global norm is now followed by all members of the UN member states. Data is regularly collected by the UN Statistical Office and disseminated by its agencies to rate and rank all the wealth and progress of the nations of the world. In the words of Yoshiko Herrera: “The scope of the SNA as an international institution and the level of cooperation and coordination that it demands are nothing short of heroic.”4

The resulting metric, and the object of study in this book, is gross domestic product (GDP), or gross national income (GNI), colloquially referred to as national income. This statistic is used to measure the size of an economy, and this is the data by which countries are ranked as developed or less developed. Economic growth is a measure of change in real GDP per capita. In theory, this measure is obtained by combining the value of all of the value-added activities in an economy throughout one year and dividing that total by the size of a country’s population in that year. This outcome is then deflated by a measure of price changes, and finally the result is compared with the equivalent figure for the previous year. This assumes that the data fully covers all activities and that the outputs and inputs within each activity are properly valued and quantified. It further assumes that the population is properly enumerated from year to year and that the deflation measure is timely and correct. In practice, this measure does not reach that assumed level of accuracy, even in countries with developed economies.5 Some economic activity is not measured, a population census is usually undertaken only once a decade, and the construction of comparable price indices involves compromises about which goods and services to include in the index. The disparity between the measure in theory and the measure in practice is the subject of this book.

National Accounts in Africa: The Main Problems

The central issue in national income accounting is deciding which economic activities and actors should and can be included in the official accounts.6 This is often referred to as the “production boundary.” Since the application of the United Nations Standard of National Accounts, there has been a discussion about where one should draw this line. In western economies, this means that the economic value of the activities of “housewives” are not accounted for. With specific reference to African economies, Brian van Arkadie noted that the “existence of a large amount of ‘subsistence’ activity (or, at least economic activity which does not result in a recorded marketed transaction) makes Pigou’s famous quip about the national accounting consequences of marrying your cook much more than a mere curiosity.”7 In other words, if you marry your cook, the value assigned to the activity of preparing meals will move outside the production boundary and the service provided will no longer be considered as part of the domestic production of goods and services.

In all economies a distinction between recorded and unrecorded economic activity exists. In “developed” economies, unrecorded activity consists of illegitimate economic activity and economic activity within the family household. In most African economies, the unrecorded economy is so large and therefore so economically important that to leave it unrecorded is unsatisfactory. However, its inclusion in the national accounts has been constrained by the availability of data. This has resulted in a variety of innovative accounting practices at the individual statistical offices. This section provides a general picture of some of the basic methods used at statistical offices throughout sub-Saharan Africa. A more nuanced picture, showing which sources and methods were used at different times and places across sub-Saharan Africa is discussed in chapter 2.8

In theory, there are three distinct ways of aggregating GDP: the income method, the expenditure method, and the production method. Again in theory, these are supposed to be reached independently. and their respective results should be balanced. The first approach adds up profits, rents, interest, dividends, salaries, and wages. In practice, this approach has not been suitable for estimating the GDP of African economies. The main component of the method would be profits earned by farmers, and this information is not directly available. The expenditure approach is more feasible, at least at first glance. Its components are private consumption, investment, government consumption, and the balance of exports and imports. The problem here is personal consumption and the part of capital formation related to rural and small-scale economic activities. The production method totals estimates of value added (output minus intermediate consumption) per sector (agriculture, mining, manufacturing, construction, and different services) to equal total value added, or GDP. This method has been preferred in official national income accounting in postcolonial Africa. While the System of National Accounts suggests that all three methods should be estimated independently, thus providing a check on the accuracy of each estimate, this practice is not often followed. Postcolonial national accounts have typically been estimated using the production method, while expenditure on private consumption has typically not been estimated independently but has been derived as what is called a “residual.” In practice, this means that instead of reaching an independent estimate of this important component, an estimate is reached by subtracting all other components of expenditures from the GDP estimate that was reached using the production approach.

GDP statistics from African countries, then, are best guesses of aggregate production. It is important to keep in mind that national income is a composite measure. Statisticians at the Kenyan central statistical office approach the issue pragmatically: “It is possible to use a number of criteria in order to assess the progress of the economy, but the usual measure of the rate of economic development is the estimate of gross domestic product. Estimates of domestic product are not, however, among those statistics which are a definite measure to which there can be only one precise measure comparable to the number of oranges in a bag. It is in fact an aggregation of numerous data which vary substantially in order of precision.”9 This was more clearly stated in an appendix to the national accounts for 1978 prepared at the statistical office in Lusaka. The report differentiated between two types of guesses; one asterisk indicated a “guestimate” and two asterisks meant a “guestimate with a weak basis.”10 These quotes highlight the importance of looking carefully at the individual components of this composite measure. The aggregate, here generically referred to as national income, is a result of pragmatic decisions at the statistical offices that are subject to the availability of data, financial resources, and political instructions.

The quality of a national income estimate is thus a result of the quality of the activities at a statistical office. National accounts divisions depend on data that are produced in different parts of a statistical office—particularly for data on population, agricultural and industrial production, and prices. The supply of data from these subdivisions is subject to the number of available data collectors and the level of funds available for collecting and processing data. Frequently statistical offices rely on data made available from other public and private bodies. For example, agricultural data typically comes from a ministry of agriculture or its equivalent. In some sectors that are dominated by a few large operators, such as construction, mining, electricity, water, finance, communications, and transportation, offices depend on the supply of data from these private or public entities.

There is a distinction between “survey data” and “administrative data.” A survey is a specific tool the statistical office uses to collect responses from individual agents. Whether or not a statistical office is able to conduct surveys depends on its access to specific funding, as the normal budget allowance typically covers only the basic operation costs of the office. The administrative data are collected by public bodies to facilitate day-to-day governance and reflect the ambitions and extent of the activities of the state. The availability of data, which varies from country to country and according to the circumstances at a given time, determines the quality of the final estimates.

The basic questions that determine the quality of GDP numbers are whether the statistical office has any data, how good those numbers are, and what the national accountants do when data are missing. The first step in the aggregation process is to create a baseline estimate or a benchmark year, which is year 1 in the statistical series. If everything is accounted for in year 1, one can later safely assume that any additions of people, goods, and services are additions and thus are progress or growth. The most exhaustive instrument is a census in which everything is about a “population” is recorded. This can be a census of population of the country, agricultural production, or the transport sector. If a census is not available, a survey may be used. Surveys contain some information about a sample of the total. If there ever was a census, the data compiler can aggregate its results, assuming that the sample is representative. If there is no total population to relate this survey to, the statistician will have to make a guesstimate, literally making up the missing information without any official guidelines. An example would be an informal sector survey. The survey will yield information on earnings of individuals in this sector but the statistician does not know the total number of participants in the sector. Often there is no data. When data on levels of economic activity are missing, GDP compilers have to rely on estimation by proxy, or assumed relationships. A classic example is when no data on food production exists and the statistician assumes a per capita intake of calories and then multiplies that by a guess of the farming population to get a measure of how much food is produced but not marketed in official channels or recorded markets. Data are usually missing for parts of the service sector, and a common method of estimating the value of that sector is to assume a proportional relationship with the production of other physical goods.

When an estimate of the level of national income for a given year has been reached, the wealth of the nation is measured. The next step is to measure economic growth, in order to monitor the progress of the nation. It is easy to get the impression that this would simply entail aggregating all available data once more and comparing the current year with previous years. However, the way this is done in practice is quite different. The estimates of levels for the individual sector form the starting point. In some categories of analysis, such as government expenditures and turnover for larger businesses, statisticians are able to compare the total for one year with another, but for large parts of the economy they usually rely on so-called performance indicators, or proxies. These indicators use annual data collected from public bodies and private businesses, supplemented by data on exports and imports. Typical examples of performance indicators use cement production and/or imports as a proxy for growth in a construction sector11 or the number of new official licenses as proxies for growth in a transport sector.

There is a basic distinction between the process of aggregating an estimate of income levels and the process of estimating economic growth. One can think of it in terms of weight. The value assigned to a weight may be inaccurate. If the degree of inaccuracy was consistent, it would not matter much in terms of measuring change. That is, even if a weight shows you to be too heavy, if the weight is equally skewed in the same direction the next year, you would at least know with accuracy how much weight they have gained or lost. There is one mathematical caveat to this: since change is measured in percentage, you will appear to be gaining weight at a faster rate if the weight showed you to be lighter than you really were. Following from these principles, one could expect the following: the more the level of GDP is an underestimation, the more the rate of growth will be an overestimation. However, this isn’t quite the way it works with complex statistical measures. It is mathematically true when one is measuring the weight of a person or the number of oranges in a bag but it is not automatically true when one is measuring GDP, because the GDP is a composite index with a base year.

The base year estimate is of crucial importance. It determines the proportional shares of different sectors of the economy. The issues that can follow when one uses composite indexes are generally referred the as the “index number problem.”12 The size of each individual sector in the base year determines the impact the growth in one sector has on the aggregate growth in the following years. In order to measure “real” economic growth, the economy is accounted for in the base year’s prices. This is done by either deflating a sector with a measure of inflation over time (this method is often used for data from service sectors) or by expressing output in the base year prices directly using volumes, which are multiplied with the prices in the reference and/or base year. Generally speaking, the less “normal” and the older the base year is, the more misleading the growth series will be. For ex...

Table of contents

• List of Illustrations
• Preface
• Acknowledgments
• Introduction
• 1. What Do We Know about Income and Growth in Africa?
• 2. Measuring African Wealth and Progress
• 3. Facts, Assumptions, and Controversy: Lessons from the Datasets
• 4. Data for Development: Using and Improving African Statistics
• Conclusion: Development by Numbers
• Appendix A. A Comparison of GDP Estimates from the World Development Indicators Database and Country Estimates
• Appendix B. Details of Interviews and Questionnaires
• Notes
• References