Information and Communication Technologies for Development Evaluation

Oscar A. García, Prashanth Kotturi

• Publisher: Routledge
• Year: 2019
• Print ISBN: 9780367785833
• eBook ISBN: 9780429647901
• Topic: Business
• Subtopic: Business General
• Index: Business

Written by a team of expert practitioners at the Independent Office of Evaluation of International Fund for Agricultural Development (IFAD), this book gives an insight into the implications of new and emerging technologies in development evaluation.

Growing technologies such as big data analytics, machine learning and remote sensing present new opportunities for development practitioners and development evaluators, particularly when measuring indicators of the Sustainable Development Goals. The volume provides an overview of information and communication technologies (ICTs) in the context of evaluation, looking at the theory and practice, and discussing how the landscape may unfold. It also considers concerns about privacy, ethics and inclusion, which are crucial issues for development practitioners and evaluators working in the interests of vulnerable populations across the globe. Among the contributions are case studies of seven organizations using various technologies for data collection, analysis, dissemination and learning.

This valuable insight into practice will be of interest to researchers, practitioners and policymakers in development economics, development policy and ICT.

1 Evaluation and the Sustainable Development Goals

Opportunities and Constraints

Marco Segone

We live in a world where a massive concentration of wealth and privilege exists in the hands of a few: the richest 1% of the population owns 40% of the world’s wealth, while the poorest 50% of the population owns only 1% of the world’s wealth. Human development indicators show that 793 million people are still malnourished and that one in three women will be beaten, raped, abused, or mutilated in their lifetimes. The world is already witnessing the impact of climate change on natural systems. Climate change is also projected to undermine food security, exacerbate existing health threats, reduce water availability and increase displacement of population. The unsustainable patterns of economic development have led to unequal distribution of fruits of economic growth and exacerbated concerns of environmental sustainability.

Is this the world we want? Or would we like to live in a world in which inequalities have been banished for everyone, everywhere, all the time? Most people would agree this is a common goal. So how do we get there? The good news is that the 197 countries that endorsed the 2030 Agenda for Sustainable Development recognize the importance of long-term, equitable and sustainable development, and more and more countries are implementing social and public policies to try to decrease the gap between those with the most and those with the least (Segone and Tateossian, 2017).

The ambitious 2030 Agenda for Sustainable Development, adopted in September 2015 by world leaders at a historic UN summit, calls for global transformation that focuses on ending poverty, protecting the planet and ensuring prosperity for all. In January 2016, the 17 Sustainable Development Goals (SDGs) intended to implement this agenda came into force. These new goals – built on the success and the unfinished agenda of the Millennium Development Goals (MDGs) – call on all countries to mobilize efforts to end all forms of poverty, fight inequalities and tackle climate change, while ensuring that “no one is left behind”. The innovative and transformational process and content of the SDGs increases our chances of reaching the goals. There are five fundamental differences between the SDGs and the MDGs.

First, the SDGs were formulated through a broad, inclusive process. For more than two years, governments, civil society groups, the private sector and thought leaders from around the world negotiated and discussed the SDGs. For the first time, eight million people voted on which of the global goals were most important to them. This inclusive and participatory process has also encouraged each country to adapt the SDGs to its own national contexts, increasing the sense of ownership of the goals (UNDG, 2016).

Second, the SDGs are universal. Unlike the MDGs, which had a strong focus on developing countries (with seven of the eight goals devoted to them), the SDGs are relevant to every country (Osborne, Cutter and Ullah, 2015). Rob D. van den Berg, president of the International Development Evaluation Association, has reminded us that “from the perspective of the SDGs, all countries are developing countries”.

Third, the SDGs are comprehensive and integrated. While the large number of goals (17) has led some to express concern, it also encourages sweeping transformation across a broad range of areas and encourages the use of partnerships to accomplish these goals. To improve communication and ensure that people understand the ultimate intent of the SDGs and Agenda 2030, the United Nations has clustered them into “five Ps”: people (human development), prosperity (inclusive economic development), planet (environment and climate change), peace (a key component of all development) and partnership (one of the few ways to achieve such sweeping transformation).

Fourth, the principle of “no one left behind” is the key principle informing every SDG and is mainstreamed throughout the structure of Agenda 2030. Achieving gender equality and reducing inequalities among and within countries are both stand-alone goals, and they are both mainstreamed through all SDGs.

Fifth, Agenda 2030 and the SDGs include a follow-up and review mechanism, operating at the national, regional and global levels. The principles for this mechanism are voluntary and country-owned; open, inclusive and transparent; support the participation of all people and all stakeholders; are built on existing platforms and processes; avoid duplication; respond to national circumstances; and are rigorous and evidence-based, informed by data that is timely, reliable and disaggregated. Most important for those in the evaluation community, the follow-up and review mechanism is expected to be informed by country-led evaluations. Consequently, the 2030 Agenda calls for strengthening national evaluation capacities, echoing the UN General Assembly resolution adopted in December 2014 on the same subject (UN, 2015a).

The Greatest Opportunity and the Greatest Challenge for the Global Evaluation Community

This is the first time in the history of international development that the world’s heads of state have committed to follow up and review mechanisms to assess the implementation of global goals. This assessment takes the form of voluntary national reviews (VNRs) to be undertaken by national governments of their progress on SDGs. One hundred and eleven VNRs have been presented at the UN High-Level Political Forum on Sustainable Development (HLPF) since 2016, with a further 51 due to be presented in 2019 (UNDESA, 2018). This high-level and far-reaching commitment could enable a surge in the demand for country-led evaluation. Key policymakers will hopefully demand their own national evaluation systems, so that they can produce high-quality evaluations to inform the national SDG reviews that countries will be presenting at the UN HLPF. This is therefore an unprecedented opportunity for the evaluation community. On the other hand, evaluation of these broad-reaching goals with a central focus on “no one left behind” presents several challenges:

• How do we assess whether development interventions are relevant, and are having an impact in decreasing inequality and improving the welfare of the worst-off groups?
• How do we carry out evaluation given the complexity of the SDGs? Are we going to evaluate complex and inter-dynamic environments with the traditional linear, simple and static logical framework (logframe) approach?
• How can we take advantage of new technologies to address the challenges above?
• Most importantly, how can we strengthen the capacities of governments, civil society organizations (CSOs) and parliamentarians to evaluate whether interventions are producing equitable outcomes for marginalized populations?

Below are some suggestions about how to address these challenges while capitalizing on the great opportunity the SDGs provide to all of us.

How Do We Assess Whether Development Interventions Are Relevant, and Are Having an Impact in Decreasing Inequality and Improving the Welfare of the Worst-Off Groups?

The 2030 Agenda made a commitment to ensure a systematic follow-up and review of the SDGs that would be “robust, voluntary, effective, participatory, transparent and integrated”, and that would “make a vital contribution to implementation and will help countries to maximize and track progress in implementing the 2030 Agenda in order to ensure that no one is left behind” (UN, 2015b). Country-led evaluations could play a central role in informing SDG reviews and, together with strong monitoring data, supporting national policy decision-making.

Gender equality and reducing inequalities between and among countries are central to the SDG principle of leaving no one behind (UN, 2017b). That means going beyond aggregate indicators, which only estimate the proportion of the population who have benefited from a particular intervention and can conceal the fact that some marginal or vulnerable groups are being left behind. In this context, the goal of the SDGs in reducing inequalities includes

• identifying groups that have been left behind,
• understanding why this has happened,
• identifying strategies to promote more inclusive approaches that will include these groups.

Strengthening national statistical systems is of paramount importance in order to produce disaggregated data that go beyond national averages. A data availability assessment, the establishment of national SDG indicators and benchmarks and a data ecosystem assessment are all elements that would provide the building blocks for the data inputs for the VNRs (ODI, 2018). Evaluators also have to explain why certain groups have been left behind and how this can be corrected. This is why equity-focused and gender-responsive evaluation is vital.

UN Women, the UN entity for gender equality and women’s empowerment, defines gender-responsive evaluation as having two essential elements: what the evaluation examines, and how it is undertaken. Gender-responsive evaluation assesses the degree to which gender and power relationships – including structural and other causes that give rise to inequalities, discrimination, and unfair power relations – change as a result of an intervention. This process is inclusive, participatory, and respectful of all stakeholders (rights holders and duty bearers). Gender-responsive evaluation promotes accountability regarding the level of commitment to gender equality, human rights, and women’s empowerment by providing information on the way in which development programmes are affecting women and men differently and contributing to the achievement of these commitments. It is applicable to all types of development programming, not just gender-specific work (UN Women Independent Evaluation Office, 2015).

UNICEF, the UN agency for children, defines equity-focused evaluation as a judgement of the relevance, effectiveness, efficiency, impact and sustainability of policies, programmes and projects that are concerned with achieving equitable development results (Bamberger and Segone, 2011). This approach involves using rigorous, systematic and objective processes in the design, analysis and interpretation of information in order to answer specific questions, including those of concern to the worst-off groups. It assesses what does work to reduce inequalities, and what does not, and it highlights the intended and unintended results for the worst-off groups, as well as the gaps between the best-off, average, and worst-off groups. It provides strategic lessons to guide decision makers and to inform stakeholders (Bamberger and Segone, 2011). The UN Evaluation Group, the professional network of evaluation offices of UN agencies, provides a valuable resource for all stages of the formulation, design, implementation, dissemination and use of the human rights and gender-responsive-focused evaluations (UNEG, 2014).

Equity-focused and gender-responsive evaluations use existing evaluation methods but bring a crucial perspective to how interventions are evaluated. In Figure 1.1, if the national policy to reduce poverty in a hypothetical country is evaluated, the main finding would be that the policy had a positive impact, reducing the percentage of the population living in poverty from 55% to 30% in 10 years. Therefore, the evaluation would recommend that the policy be continued.

However, if the same policy were evaluated with an equity-focused and gender-responsive approach, the findings and recommendations would be different. Data would be disaggregated by gender¹ as in Figure 1.2, and while the evaluation would acknowledge that the policy increased the national average, it would also find that the gap between males and females increased. The evaluation would also find that while the policy had an important positive effect in reducing poverty among males, it had a less positive effect on women. Therefore, the recommendation would be that the policy should be revised to decrease the inequality gap and have a more positive effect on the worst-off group.

Figure 1.1 National average reduction in income poverty in a hypothetical country.

Source: Segone (2017).

Figure 1.2 Reduction in income poverty in a hypothetical country, by gender.

Source: Segone ...

Table of contents

• Cover
• Half Title
• Series
• Title Page
• Copyright Page
• Table of Contents
• List of Figures, Tables and Boxes
• List of Contributors
• Acknowledgements
• Introduction
• 1 Evaluation and the Sustainable Development Goals
• 2 Information and Communication Technologies for Evaluation (ICT4Eval)
• 3 Big Data Analytics and Development Evaluation
• 4 Technology, Biases and Ethics
• 5 Technology and Its Implications for Nations and Development Partners
• Conclusions
• Index