Evaluation of the Technical Assistance Provided by the International Monetary Fund

International Monetary Fund. Independent Evaluation Office

• Publisher: INTERNATIONAL MONETARY FUND
• Year: 2005
• eBook ISBN: 9781589064249

NONE

1. INTRODUCTION

1. The purpose of this evaluation is to examine technical assistance (TA) provided by the IMF to its member countries and to derive operational recommendations that can enhance the contribution of TA to the overall IMF mandate.

2. The stated objectives of IMF TA were defined in the most recent policy statement on the subject. “In accord with the IMF’s first Article of Agreement, the objective of the IMF’s TA program is to contribute to the development of the productive resources of member countries by enhancing the effectiveness of economic policy and financial management. The IMF’s TA program works to achieve this objective in two ways. First, much of the IMF’s TA seeks to support the efforts of members to strengthen their capacity—in both human and institutional resources—to formulate and implement sustainable, growth-oriented and poverty-reducing macroeconomic, financial, and structural policies. Second, the IMF assists countries in the design of appropriate macroeconomic and structural policy reforms, taking account of the lessons learned by other countries in addressing similar economic policy concerns.”⁷

3. Thus, the definition of TA used within the IMF is quite wide. In practice, it includes not only activities to support long-term institution building in countries themselves, but also fact-finding and analytical activities that allow the IMF to advise countries on specific policies and in the design of IMF-supported programs. This wide definition presents special analytical challenges, both to identifying TA priorities as well as to evaluating its impact.

4. The IMF devoted about $80 million of direct resources to TA in FY2004 (excluding training), which is about 10 percent of the total administrative budget.⁸ The pattern of IMF TA activities has evolved considerably over time. In the early 1990s, a significant share of TA resources was allocated to the transition countries to help develop the policies and institutions for a market economy. Since the late 1990s, a greater share has focused on countries receiving assistance under the PRGF, including for improving governance and for capacity building. Crisis prevention and resolution, TA to post-conflict countries, and TA in connection with regional arrangements (such as the new TA regional centers), have also received increased attention.

A. Issues to Be Addressed

5. To assess whether IMF TA is achieving its stated objectives and, if necessary, to identify corrective steps, it is useful to unbundle the TA process into three stages:

•   Resource allocation. What are the factors determining the allocation of TA resources among countries? How are TA priorities selected at the country level and how well are they related to broader development strategies in the country to assure relevance? How is the balance between short-term TA (often to support the policy advice of the IMF) and TA to support (long-term) institution building established?

•   TA delivery. What can be learned about specific modalities of TA delivery and ways to interact with authorities during the implementation period so as to increase the effectiveness of TA?

•   Impact and evaluation. What has been the impact of TA and how is it tracked and monitored, including through ex post evaluation? What are the key factors that contribute to TA effectiveness?

How is TA allocated among countries and priorities identified within countries?

6. What patterns can be found in the allocation of IMF TA resources among countries? Are they responsive to indicators of low institutional capacity? How sensitive is the level of TA to different types of financial arrangements with the IMF? Are TA allocations to countries dominated by the level of external financing from donors (for example, trust funds)? The answers to these questions will help identify the major forces shaping cross-country allocations of TA.

7. Any process for allocating TA resources across competing uses will need to decide how priorities are set between and within countries and how resources are reallocated on the basis of evidence on outcomes (track records). The process would need the following two components:

•   An identification of the broad areas where the IMF would seek to maintain or develop “core competencies” in TA in the light of its small size relative to global TA efforts and the need to closely complement its other instruments, namely, lending in support of adjustment, surveillance, and related diagnostic activities, such as ROSCs and FSAPs.

•   A mechanism for prioritizing TA activities within these broad areas. Since the IMF’s definition of TA includes both short-term diagnostic activities and longer-term institution building, a major challenge will be to decide on the appropriate mix between the two types of activity in each country, and how they interact over time with each other and with other IMF instruments.

8. A key question to be addressed is how well this process is working today. Is TA provided to a country anchored in a medium-term strategy from which TA priorities can be coherently derived? Are Article IV consultations and the PRS process (in the case of low-income countries) providing such an integrated view? Or is TA the result of a process driven by the separate needs of programs, ROSCs and FSAPs? Given budget constraints, how are trade-offs managed in practice? How well are the authorities involved in the process?

How effective has been the process of TA delivery?

9. Achieving the final objectives of TA will also depend on the effectiveness of the means of delivery. Several questions can be raised: (1) How clear are the initial TORs and how involved are the authorities in their preparation? (2) Is there clarity at the outset regarding how progress will be measured? (3) What are the features for a successful resident expert? (4) Is the mixture between short-term missions and long-term resident experts (LTEs) appropriate given country circumstances? Should LTEs be in charge of monitoring program conditionality in their areas of expertise? (5) How effective has been the dialogue with the authorities during the delivery period in enhancing the ownership of TA recommendations?

What was the impact of TA and what factors contributed to its effectiveness? How was this tracked?

10. An assessment of the impact of TA and the factors influencing it—both country factors and those under the control of the IMF—is critical in allocating TA in the future and improving its design. Several questions emerge: (1) What was the impact of TA and what are the key factors determining impact? (2) How well is the IMF tracking intermediate and final outputs? (3) Does the IMF distinguish between the improved technical know-how of agencies receiving TA and their subsequent ability to apply that knowledge by carrying out their duties—the latter requiring political support from the authorities? and (4) How candid is the reporting to headquarters when effectiveness is impaired by political interference?

B. Methodology and Empirical Basis

11. The evaluation is organized around the three set of questions raised above—which also defines three consecutive stages of the result chain determining the impact of TA. They are addressed in this evaluation by using different types of empirical evidence, namely:

•   Internal IMF cross-country data on TA field activities (derived from the Travel Information Management System (TIMS)) to examine trends and patterns in the allocation of TA among countries and identify the forces behind these patterns.

•   A sample of 10 low-income countries to study the link between the PRSP and the identification of TA needs.

•   A sample of 18 countries to assess the link between Article IV surveillance documentation and TA activities.

•   A sample of 12 Technical Assistance Consultation exercises (TCs) which were piloted in an effort to enhance the link between Article IV surveillance and TA activities in a country.

•   In-depth analysis of 6 countries, including field visits to elicit views from local counterparts and authorities (Cambodia, Honduras, Niger, Ukraine, Yemen, and Zambia).

•   Interviews with IMF staff, including TA and training coordinators in both functional and area departments, as well as staff and consultants involved in TA activities in the countries above in the period examined.

•   A review of findings of 11 internal and external evaluations of thematic or regional TA provided by the IMF.⁹

12. Figure 1.1 describes how this evidence is used throughout the different stages of the evaluation. As we move to understand the impact of specific TA activities and its determinants, the evaluation relies on the in-depth case studies and qualitative evidence drawn from interviews. The measures used to assess impact, and their limitations, are discussed in more detail in Chapter 6.

Figure 1.1. Stages and Result Chains of TA Activities Supporting Institution Building

13. The report is organized as follows: Chapter 2 reviews briefly the results of, and follow-up to, the 1999 Fund-wide TA evaluation. Chapter 3 examines trends and patterns in the allocation of TA resources across subjects and countries; Chapter 4 examines how well the identification of TA priorities at the country level is working; Chapter 5 examines the process and modalities of TA delivery; and Chapter 6 examines the impact of TA, how progress is being tracked, and factors that may impair progress. At the end of each chapter, recommendations are presented.

14. This evaluation does not assess the role of the new RTCs because their experience is too recent for an effective evaluation. A review of preliminary experiences is at present being undertaken by OTM.

2. THE 1999 FUND-WIDE TA EVALUATION

15. In 1999, the IMF’s Office of Internal Audit and Inspection (OIA) carried out a comprehensive evaluation of IMF TA covering the Fiscal Affairs Department (FAD), the Monetary and Exchange Affairs Department (MAE, now Monetary and Financial Systems Department (MFD)), and the Statistics Department (STA). Several issues raised in Chapter 1 were addressed by this evaluation.

Major findings and recommendations from the 1999 evaluation¹⁰

16. The major findings were

• IMF TA is highly appreciated by members who want more of it.

• While IMF TA is rated highly for its advice and recommendations, obtaining country commitment and follow-up seem to be the weakest parts of the IMF’s advisory process; partly as a result, the impact of TA projects was less than satisfactory in about one-third of projects.

• In general, TA provided by staff missions appears more likely to be successful than that provided by LTEs.

• While implementation of advice and the impact of TA depend most importantly on the commitment of the recipient country, the probability of achieving high impact can be improved by actions within the control of the IMF.

• There is only a weak link between IMF surveillance and IMF TA, the orientation of which is generally reactive rather than proactive.

• There is no explicit IMF policy on TA, little evaluation, little reporting on results to management and the Executive Board, and little public dissemination of the lessons learned.

17. The evaluation suggested that those TA tasks with the greatest impact were characterized by strong government involvement and ownership, excellent communications between TA providers and recipients, placement of the activity in a broader policy framework, well-prepared expert focus and specific project preparation and recommendations, and attention to follow-up. Neither the existence of an IMF-supported program nor having an IMF resident representative in place appeared to make a statistically significant difference with respect to impact.

18. The major recommendations emerging from the 1999 evaluation were

• Link the assessment of TA priorities and allocation decisions more closely to IMF Article IV consultations. This would require a change in the orientation of the IMF’s TA from being largely curative to being more preventive.

• IMF staff should conduct specific TCs as part of Article IV consultations in order to assess progress in implementing past technical advice and discuss future needs. For countries requesting significant volumes of TA, a Technical Coopera...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Abbreviations and Acronyms
• Summary of Major Findings and Recommendations
• 1. Introduction
• 2. The 1999 Fund-wide TA Evaluation
• 3. Trends and Patterns in the Allocation of TA Resources
• 4. Identifying TA Needs and the Allocation of TA Resources
• 5. The Process of TA Delivery
• 6. Measuring the Effectiveness of TA: Tracking Implementation and Factors Influencing Progress and Implementation
• Boxes
• Bibliography
• Footnotes