Introduction
We can achieve a sort of control under which the controlled, though they are following a code much more scrupulously than was ever the case under the old system, nevertheless feel free. They are doing what they want to do, not what they are forced to do. That’s the source of the tremendous power of positive reinforcement – there’s no restraint and no revolt. By careful cultural design, we control not the final behaviour, but the inclination to behave – the motives, desires, the wishes. (Skinner 1948: 246–7)
Many observers have commented on the spread of behavioural experiments in global policy. This new and emerging form of public policy often involves conditioning the receipt of welfare – in the form of cash transfers, goods or services – according to specific individual behaviour. Few however, have touched on the contextual differences in terms of the application of research and evidence governing policy, especially social assistance in the developing world and social security within the developed world. Subsequently, the debates on the progress of public policy in the global north and south are not as well connected as they might be.
In LMICs, social experiments with conditionality are part of the drive for evidence-based policy (Fiszbein et al. 2009). Conditional cash transfer (CCT) programmes usually have an a priori evaluation design built into their operation, that embraces experimental or quasi-experimental features and RCT designs, for example. By contrast, in the advanced liberal democracies, there has been less direct appeal to research evidence gained using from robust evaluation in order to secure major welfare reform: experimentation and evaluation with RCTs has been less of a priority. Consequently, welfare states were reformed on ideological grounds, with an appeal to political theory as Mead and Beem (2005) observe. Putting this in stark terms, welfare conditionality in the south is, arguably, being driven by an evidence-based policy-making agenda, whereas in the north, political philosophy is clearly driving welfare reform. This chapter seeks to shed new light on the relationship between evidence and evaluation within the different worldly contexts, by drawing out emerging arguments and counter-arguments about the ‘use’ and ‘misuse’ of evidence within public policy. The chapter is organized as follows: the first section examines the increase and nature of evidence-based policy for development; the second section considers how welfare policy has been transformed in the developed world; followed by a more detailed examination of some of the controversies in section three. Lastly, reflections and conclusions are drawn together in the fourth section.
Public Policy for Development: The Rise of Behavioural Economics
In the brave new world of the behavioural economist, achieving the Millennium Development Goals (MDGs) for health and well-being is ultimately about demonstrably changing people’s behaviour for the better. Being able to demonstrate the effectiveness or impact of an intervention is thus the keystone for policy development. As a result of the global research effort, it is now well established that there are certain (desirable) human behaviour and conditions of living that are beneficial for our health and well-being that, arguably, should form the basis for public policy (Dean 2010). This point, implicit in the literature, certainly needs to be made more explicit, as biomedical and social research continues to establish food and dietary requirements for good health, education, housing and living standards, along with certain behaviour and practices that reduce the risk of ill-health and disease (e.g. WHO 2002). For far too long, the criticism has been that acceptance of incontrovertible evidence into functioning policy has been slow, partial and unsystematic; resulting in health deficits, waste of human potential and other associated costs to society (WHO 2011). The second point, about clearly demonstrating success and impact in social policy addresses the need for robust evaluation. In order to consider the effectiveness of a particular policy or programme, one really needs to know what outcomes would have been achieved had the programme not been in place. This is often referred to as the ‘counterfactual outcome’ (OECD n.d.). One way of overcoming this evaluation problem is through the use of RCTs, which are considered to be the ‘gold standard’ of all the methods available to researchers (Young et al. 2002). In the trials, participants are usually randomly assigned to intervention or control groups (Sibbald and Roland 1998), researchers then compare the outcomes between groups (see figure 1.1). A systematic review of RCT results can usually be found at the very pinnacle of the research evidence hierarchy (figure 1.2), as it provides a way of pooling evidence from different studies to provide an overview of outcomes (White and Waddington 2012).1
It may not be surprising, therefore, to find an influential group of economists advocating social experiments and in particular RCTs, as the main tool for studying the effectiveness of policy in development settings. In a recent trial in Malawi, for example, poor families were given cash on the condition that they send their children to school (Baird et al. 2000a, 2000b). Some villages were randomly assigned to the social assistance programme and others were not (figure 1.3). Importantly, at the outset, each village had the same equal chance of being picked to receive the intervention; families were not told that they were part of an experiment, they were ‘blind’ or ‘blinded’: knowing this may affect their behaviour and the outcome of the study. The development economists – sometimes referred to as the ‘randomistas’ – argue that this type of randomized experiment is the only sure way of identifying impact, because it helps to eliminate bias and other confounding (hidden) factors. Other non-experimental methods found at the bottom of the evidence hierarchy are largely dismissed as they are considered unscientific and are best avoided (figure 1.2). The influence of the randomistas appears to be growing; non-governmental organizations (NGOs) (the World Bank, the International Monetary Fund, the World Trade Organization), philanthropic agencies and donors (e.g. DFID 2011b) are increasingly giving (explicit) preference to randomized designs and systematic reviews in evaluating public policy programmes and their impacts (Hickey et al. 2009; Hagen-Zanker et al. 2012).
According to behavioural economists like Banerjee and Duflo (2011), the major debates in international development can be boiled down to disagreements about the shape of a function in development theory (figure 1.4). The S-shaped curve on the left suggests poor people are ‘trapped’ in poverty and require a ‘conditional push’ to get out of their own poverty. The L-shaped curve on the other hand, suggests that poor people are gradually able to pull themselves out of poverty because they are not really ‘trapped’ at all. If we accept the premise of the development economist, the theoretical proposition and corresponding debate about which of the two graphs best represents the real world can only be settled experimentally, through the use of RCTs. Of course, not all social researchers agree with the world view depicted in these two charts or see the need for social experiments to solve the problem of global poverty. The randomistas, however, firmly believe that it is perfectly possible to make significant progress in tackling global social problems using experimentation; through the accumulation of small experimental steps, each well thought out, carefully tested and judiciously implemented. In effect, they claim to be saving lives with a well-placed behavioural ‘nudge’ and they offer plenty of evidence to support their position.
Saving lives with a well-placed nudge
Conditional social policy programmes are now firmly established across the developing world and shape the lives of millions of people. Many CCT programmes are large-scale and usually have an evaluation design built into their operation. Oportunidades in Mexico for example, covers about a quarter of Mexico’s national population (1.5 million households). In Brazil, some 11 million families – 46 million people – receive regular transfers under the Bolsa Família programme. Mexico’s Progresa (renamed Oportunidades in 2002), often seen as the seminal and model programme, began in 1997. This programme has demonstrated a range of benefits over the years, particularly in the areas of health and education (DFID 2006). For instance, some 70 per cent of families participating in the scheme have shown improved nutritional status and stunting has been reduced; ante-natal care increased by 8 per cent, contributing to a 25 per cent drop in the incidence of illness in newborns; immunization rates improved as a result of preventive healthcare appointments; and school enrolment rates increased by over 20 per cent for girls and by 10 per cent for boys. These social assistance programmes also have fiscal appeal, at least according to national governments and NGOs such as the World Bank. In terms of national budget, Mexico and Brazil commit only 0.5 per cent of gross domestic product to their programmes. With the evidence-base for CCT programmes growing, Guatemala is one of the latest Latin American countries to embark on reform with Mi Familia Progresa (my family is moving forward), introduced in 2008 (Gaia 2010). In lower-income contexts, especially in Africa, CCT programmes and experiments are often on a smaller scale – as well as familiar conditions relating to school attendance and the use of health services, other conditions continue to be tested, for example adult education, micro-credit, housing and accommodation schemes, also bed net schemes to help protect people against malaria-carrying mosquitoes (DFID 2011a).
The evidence suggests CCT programmes can be particularly effective in promoting health and education services in LMICs (Lagarde et al. 2007). Improvements in the use of health services (table 1.1) is particularly striking as the effects are concentrated among families who are least likely to use the services in the absence of conditional reward. School enrolment rates also show significant gains with conditionality in place (table 1.2). In addition, there are demonstrable impacts ...
