Managing for Development Results

Relevance, Responsiveness, and Results Orientation

,

• Publisher: Asian Development Bank
• Year: 2012
• Print ISBN: 9789290926122
• eBook ISBN: 9789290926139
• Edition: 1
• Topic: Negocios y empresa
• Subtopic: Operaciones

This study assesses the achievement of the Asian Development Bank (ADB) in mainstreaming the Managing for Development Results agenda within ADB, with a view to track progress, identify lessons, and make recommendations for ensuring better development effectiveness of operations in the future.

Chapter 1

Introduction

A. Background

1. This special evaluation study (SES) has been prepared following the recommendation of the Independent Evaluation Department’s (IED) preliminary managing for development results (MfDR) assessment¹ done in 2007, which sought to provide timely feedback and help guide the evolution of MfDR within the Asian Development Bank (ADB). It presents an independent evaluation of the mainstreaming of the MfDR agenda in ADB (a detailed description of the MfDR agenda is provided in Chapter II). The timing of the SES was scheduled to make the study results available well in advance of the Asian Development Fund (ADF) Annual Consultation Meeting in December 2011, as well as to provide inputs into the comprehensive review of the ADB corporate results framework planned for 2012.

The special evaluation study on MfDR follows up the 2007 preliminary MfDR assessment

B. Objective and Scope of the Evaluation

2. The main objective of the SES is to assess independently ADB’s achievements in mainstreaming the MfDR agenda within ADB, with a view to track progress, identify lessons, and make recommendations for ensuring better development effectiveness of operations in the future. For the purposes of the study, the MfDR stakeholders include both in-country developing member country (DMC) clients as well as staff in headquarters. The period of coverage is from 1999 to 2010. However, some tables and appendixes provide more recent data.

The study focuses on evaluating the processes and institutional changes stemming from MfDR

3. In terms of scope, the evaluation covers the mainstreaming of MfDR across a number of dimensions, including the (i) MfDR key result areas (KRAs), (ii) MfDR layers, (iii) MfDR processes and tools, and (iv) results-chain links. See Chapter II for further discussion of each of these aspects. Given that this is a corporate evaluation and MfDR mainstreaming activities are still ongoing, the focus is more on evaluating the process and institutional changes stemming from MfDR, rather than evaluating actual results on the ground. However in some cases (e.g., for the assessment of achievements against the MfDR action plan), achievement of results has been assessed. Table 1 shows the coverage at each of the MfDR layers.

Table 1: Coverage at Each of the Managing for Development Results Layers

Layers	Areas Covered	Areas Not Covered and Reason
Corporate	Development Effectiveness Review (DEfR) reports	Private Sector DEfR—first report was issued only in late 2010 and, unlike the DEfR, uses only a partial balanced scorecard. Departmental and divisional results-based workplans—these were issued late 2010, so it is still too early to evaluate them.
Country/Sector	Country Partnership Strategy (CPS) results frameworks	Results frameworks in regional CPSs—these simply aggregate country CPSs in the region. CPS thematic roadmaps—these are no longer recommended to be prepared by the ADB Results Management Unit.
Project or Program	Public sector project design and monitoring frameworks (DMFs) were evaluated using data from existing reports	Given that many studies on project or program DMFs have been undertaken by IED, COSO, and other departments, the SES assessments on DMFs mainly used secondary information.

ADB = Asian Development Bank, COSO = Central Operations Services Office,

IED = Independent Evaluation Department, SES = special evaluation study.

Source: Managing for development results SES team assessments.

C. Approach, Methodology, and Limitations

4. The SES’s approach to performance assessment and rating is based on three evaluation criteria: relevance, responsiveness, and results orientation. This is in line with IED’s previous corporate evaluations, such as the 2007 SES on achievements of ADB’s Long-Term Strategic Framework (LTSF) 2001–2015.² Given that MfDR mainstreaming activities are still ongoing, the assessment of MfDR strategic responsiveness took account only of MfDR action plan achievements up to mid-2011. The SES rates both at an aggregated level as well as at the criteria level ADB’s performance in mainstreaming MfDR, and adopts four scale rating system (e.g., highly successful, successful, partly successful, and unsuccessful). The evaluation framework and the rating methodology are presented in Appendix 1.

Performance assessment and rating is based on three evaluation criteria: relevance, responsiveness, and results orientation

5. In terms of methodology, the SES comprised a number of study components, including (i) a progress review of MfDR mainstreaming efforts, (ii) an assessment of MfDR achievement vis-à-vis the latest MfDR action plan, (iii) a comparator assessment of other development agencies, (iv) questionnaire surveys of key external stakeholders (e.g., DMC officials serving on the Asia-Pacific Community of Practice for MfDR [APCOP-MfDR]) and internal MfDR stakeholders, and (v) key informant interviews (including selected field visits to DMCs).³ Given the limited availability of factual performance data, the evaluation findings are based partly on client perceptions elicited through the two perception surveys. To allow a comparison of progress over time, the ADB staff questionnaire survey was aligned with that of the 2007 preliminary MfDR assessment, which sought to provide timely feedback and help guide the evolution of MfDR within ADB.⁴ The various sources used for each of the study components were triangulated to confirm the validity of the findings.

6. Two limitations were recognized in undertaking the evaluation:

(i)	The assessment of the MfDR project results orientation subcriteria relied on assessments of a selection of project completion reports (PCRs) and project performance evaluation reports (PPERs) prepared over the past 5 years.
(ii)	Given the wide scope of the MfDR agenda, the evaluation covered the key areas under the three MfDR layers, as shown in the last column of Table 1. Based on the reasons given, other areas were not covered.

D. Organization of the Report

7. The rest of the report is organized as follows: Chapter II describes the MfDR agenda in detail, including initial mainstreaming activities, action plans and results frameworks, MfDR layers, MfDR processes and reporting tools, and findings of previous evaluations. Chapter III assesses the relevance of the MfDR agenda to the current international consensus on results-based management, ADB’s internal structure, high-level policies and strategies, the needs and priorities of ADB’s DMCs, and stakeholders’ perceptions. Chapter IV focuses on the effectiveness of ADB’s strategic and organizational response to the MfDR agenda. Chapter V discusses how MfDR has influenced ADB’s results at three MfDR layers: corporate, country/sector, and project. Chapter VI presents the overall assessment. Chapter VII presents the issues identified, lessons learned, and recommendations. Appendixes include additional information to support the main text. Further details are presented in supplementary appendixes, which are available upon request.

¹ IED. 2007. Special Evaluation Study on Managing for Development Results in the Asian Development Bank: A Preliminary Assessment. Manila:ADB. IED’s recommendation is also repeated in ADB. 2008. Asian Development Fund X Donors’ Report: Towards an Asia and Pacific Region Free of Poverty. Manila.

² IED. 2007. Special Evaluation Study on Long-Term Strategic Framework: Lessons from Implementation (2001–2006). Manila:ADB.

³ A copy of the draft SES was shared with relevant departments and offices of ADB, and their comments were incorporated, where relevant.

⁴ IED. 2007. Special Evaluation Study on Managing for Development Results in the Asian Development Bank: A Preliminary Assessment. Manila:ADB.

Chapter 2

ADB’s Managing for Development Results Agenda

A. Definition of MfDR

8. In recent years, several issues have emerged regarding the traditional manner of conducting developmental activities. One issue was that development assistance was too often delivered in a piecemeal manner and did not always respond to a given country’s priorities, development strategies, and objectives. Another issue was that development efforts were often fragmented and unsustainable. Yet another issue concerned with the perceived overemphasis on the funding of inputs and activities through resource transfers, losing sight of the achievement of broader development results or outcomes. In response to these issues, the MfDR agenda has emerged as an alternative “change management” approach that focuses on the who...

Table of contents

• Front Cover
• Title Page
• Copyright Page
• Contents
• Abbreviations
• Acknowledgments
• Foreword
• Executive Summary
• Management Response
• Chair’s Summary: Development Effectiveness Committee
• 1 Introduction
• 2 ADB’s Managing for Development Results Agenda
• 3 Relevance
• 4 Responsiveness
• 5 Results Orientation
• 6 Overall Assessment
• 7 Findings, Lessons, Issues, and Recommendations
• Appendixes
• Back Cover