IMF Interactions with Member Countries

International Monetary Fund. Independent Evaluation Office

• Publisher: INTERNATIONAL MONETARY FUND
• Year: 2010
• eBook ISBN: 9781589069435

NONE

CHAPTER 1 Introduction

1. This report presents the evidence and findings of an evaluation of the effectiveness of IMF interactions with member countries. It is being issued at a critical juncture for the international monetary system, when the IMF has adopted a more flexible approach to lending, and been given important new responsibilities and a major injection of resources to help members deal with the global financial crisis. Implementation of the new roles will present major challenges, as will maintaining traction when the crisis subsides, and with it the demand for the Fund’s quick response role, in which it has traditionally been effective.

2. Against this background, this report highlights the evaluation’s findings and lessons learned most relevant to the tasks that lie ahead for the Fund. It does so through the lenses of the evaluation’s two main questions: (i) whether interactions between the IMF and its member countries were effective and (ii) whether they were well managed. It focuses on interactions during 2001–08, with special attention to 2007–08. It covers interactions with the entire membership; for analytic purposes, it uses three main country groups—advanced economies, emerging economies, and Poverty Reduction and Growth Facility (PRGF)—eligible countries.

3. In conducting the evaluation, the team examined evidence covering a wide variety of IMF experience and country circumstances, triangulating across data sources where possible (see Box 1). Interview and documentary evidence was gathered and analyzed for 49 case study countries, including interviews with officials and IMF staff working on those countries. Surveys polled country authorities and civil society in member countries, and IMF staff. Special studies explored selected themes, and in the course of their work, evaluation team members visited selected countries to follow up on issues that had surfaced in preliminary interviews with country officials at IMF headquarters or in written responses to questionnaires. Interviews also were conducted in several non-case-study countries in conjunction with IEO visits for different purposes. The team drew on the evidence of previous IEO evaluations.

4. The structure of this main report is as follows. Chapter 2 summarizes the evidence on the effectiveness of the IMF’s interactions with country authorities. Chapter 3 summarizes the evidence on interactions with other in-country stakeholders, and Chapter 4, the evidence on whether interactions were well managed. Chapter 5 draws conclusions and makes recommendations. Annex 1 profiles the country groups used in the analysis.¹ Companion papers on the three main country groups consider the evaluation’s evidence and analysis in more depth.

Box 1. Evaluation Building Blocks

Motivation for the evaluation. This evaluation is motivated by the central importance of the IMF’s interactions with member countries to the institution’s ability to achieve its goals, and by the persistence of criticisms from country authorities, staff, and outside observers.¹ Also relevant, aspects of interactions and their shortcomings have been recurring themes in past IEO evaluations. As the nature of interactions with member countries has evolved in recent years, and more changes are in prospect, it is timely to review past practices and lessons learned as inputs into future strategies and actions.

Definition of interactions. For purposes of the evaluation, interactions are defined to include exchanges of information, analysis, and views between IMF officials and country authorities, or other people or entities in member countries. They include the policy dialogue between the authorities and staff in the context of surveillance and financial and monitoring programs, as well as capacity building. They also include informal contacts with the authorities that can build mutual understanding and trust. They involve interactions with others, including parliamentarians and civil society, as they affect that key relationship between the authorities and staff.

Evaluation scope. In defining its scope, the evaluation starts with the IMF’s results chain, which includes as critical ingredients: (i) the quality of the Fund’s analysis, advice, and assistance; (ii) its interactions with member countries; and (iii) its impact on country policies, and in due course, on outcomes. Within this results chain, the evaluation focuses on the middle stage of interactions. In so doing, the evaluation does not ignore the other two stages of the results chain—the quality of analysis and impact on policy directions and outcomes—but it does not address them in depth. The evaluation also looks at three instruments in the management of interactions—strategy, staffing, and relationships; it asks how and how well the Fund calibrated each to promote effective interactions.

Definition of effectiveness. The evaluation considers interactions to be effective to the extent that they contribute to the overall purposes of the IMF as established in the Articles of Agreement and Executive Board policies. Those policies countenance a range of roles for interactions, from the identification of risks to external stability for the benefit of the international community as a whole, to the provision of advice and related services for the benefit of individual countries. The evaluation does not equate effectiveness with maintaining smooth and harmonious relations at all times.

Measurement of effectiveness and related properties. The evaluation polled the country authorities and IMF staff on their perceptions of the overall effectiveness of IMF interactions in each country. It also polled them on the relevance and effectiveness of ten different IMF roles—in contributing, for example, to the development of domestic policy frameworks or to international policy coordination—and about the quality of various aspects of interactions. The resulting data were used to construct composite indicators of interactions for comparing the Fund’s role relevance, role effectiveness, quality, strategic alignment, and overall perceived effectiveness across five country subgroups—the large and other advanced economies, the large and other emerging economies, and the PRGF-eligible countries.

Evidentiary sources. The evaluation relied on three main data sources—surveys, interviews, and internal documents. Each of these three sources is covered in a companion background technical document which sets out how the data were obtained and analyzed, and catalogues its findings, albeit in very summary form with a view to protecting confidential sources. Survey responses were received from representatives of the authorities in 129 countries, and civil society representatives in 159 countries. Eight hundred and thirty staff members responded to the survey working on 170 countries. The interview evidence, which was focused on the 49 case-study countries, was gathered in face-to-face and telephone meetings with about 300 country officials and stakeholders and IMF staff members. The document review involved the reading of internal documents delivered to IEO by the IMF’s five area departments for the 49 case-study countries for the entire 2001–08 evaluation period. To manage possible interpretation and measurement risks associated with individual pieces of evidence, the evaluation triangulated across the individual sources of evidence and applied judgment when different sources suggested different answers.

¹ See IEO (2008).

CHAPTER 2 Were Interactions with Country Authorities Effective?

5. In assessing the effectiveness of interactions between the IMF and the authorities of member countries, the evaluation focused on the perceptions of country officials and individual Fund staff members working on those countries.² Evidence on these perceptions was gathered through surveys of the whole membership, and interviews focused on 49 countries that explored a number of aspects of effectiveness, which were then considered in tandem with the evaluation’s documentary evidence. This chapter explores what the evaluation’s evidence has to say about the effectiveness of this interface, looking at it from a substantive perspective. The strategic, stylistic, and relationship management issues associated with the management of interactions are taken up in Chapter 4.

6. The evaluation took the view that general perceptions of overall effectiveness, to be meaningful, needed to be grounded in a common understanding on the part of the authorities and the IMF staff of what interactions were supposed to achieve, and in evidence that agreed roles were performed effectively and were of high quality. With this in mind, the evaluation framework developed measures for different aspects of perceptions of interactions. It also provides a systematic basis for considering the evaluation’s other evidence (from interviews, documents, and case studies) in forming its overall judgments.

7. On this basis, overall, the evidence is mixed. While one may be tempted to take solace from relatively high perceptions of overall effectiveness in some country groupings, such reaction needs to be tempered by clear evidence of lack of agreement between the authorities and staff on the scope of interactions in some cases, and of widely varying effectiveness in particular roles. Interactions were effective in a program and technical assistance context, and, in general, in contributing to a good exchange of views and in providing objective assessments. However, in other areas, including in the international dimensions of its surveillance and other work, where one would expect the IMF to excel, effectiveness and quality were not rated highly.

8. In turn, these findings translate into the evaluation’s broader implications about effectiveness across country groups—that the Fund has been most effective with the PRGF-eligible countries and the smaller emerging economies. They were the least effective with the advanced and large emerging economies, together accounting for about 90 percent of global GDP, where there also have been continuing differences between the authorities and staff on the Fund’s role and relevance in interacting with them.

9. Against this background, the chapter starts with cross-cutting issues—setting out first the big picture, drawn primarily from the survey evidence, and several key themes that the evidence highlights. It complements that discussion with a brief exploration of the particular issues arising by country group, drawing from the evaluation’s case studies of 10 advanced economies, 23 emerging economies, and 16 PRGF-eligible countries, which are discussed in greater detail in the three companion papers on the country groups.

A. Cross-Cutting Issues

10. Measurement framework and key themes. This section sets out the measurement framework developed by the evaluation for structured discussion of different aspects of effectiveness. It then explores four Fund activities (basic country assessment of surveillance and other Fund activities, international dimensions of Fund country analysis, policy dialogue, and country programs and technical assistance) using the framework’s building blocks.

Indicators of interactions

11. Box 2 presents composite indicators of interactions, derived from the authorities’ perceptions as recorded in the evaluation survey. They cover the authorities’ ratings for (i) role relevance, (ii) role effectiveness, (iii) quality, (iv) strategic alignment with staff views, and (v) overall perceived effectiveness—all as defined in the box. Figure 1, shown later in the chapter, presents the same indicators from the IMF staff’s perspective.

Figure 1. Composite Indicators of Interactions (Staff Results)

¹Inverted scale. The average absolute percentage point difference across the ten purposes between how much the authorities wanted the IMF to fulfill each purpose, and how much staff aimed to do so (“a fair amount” or “very much”).

• Taken together, the data show that PRGF-eligible countries were satisfied in important respects with the substance of their interactions with the Fund—and to a lesser extent so were the other emerging economies. This does not mean there were no problems in interactions with these countries or that there was not much to improve, but it does show the importance of relevant products and services for effectiveness. These results are reinforced by the interview and documentary evidence, and also by the staff survey.
• But for the advanced and large emerging economies, the indicators are less favorable to the Fund. Role effectiveness was rated the lowest by the authorities of the large emerging economies, while role relevance and quality were rated the lowest by the authorities of the large advanced economies. Low marks also came from staff working on large advanced economies with respect to overall perceived effectiveness and other dimensions of effectiveness.

12. Key features of Box 2’s indicators are discussed below.

• Role relevance. For the advanced and the large emerging economies, the low level of this indicator in part reflects these authorities’ limited interest in programs and technical assistance. But that is not the full story. The underlying data also point to limited interest in the Fund’s policy advice (including on operational aspects) and related outreach, and very limited interest by the large advanced economies in a contribution by the Fund to international policy coordination.³ ...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Contents
• Foreword
• Abbreviations
• Executive Summary
• 1 Introduction
• 2 Were Interactions with the Country Authorities Effective?
• 3 Were Interactions with Other Stakeholders Effective?
• 4 Were Interactions Well Managed?
• 5 Conclusions and Recommendations
• Annex 1. Country Group Profiles
• References
• Boxes
• Companion Papers
• Background Technical Documents
• Statement by the Managing Director, Staff Response, IEO Comments on Management and Staff Responses, and the Acting Chair’s Summing Up
• Footnotes