1 Introduction
The global development of evaluation is remarkable. Starting from North America, during the past 50 years, evaluations are conducted in a steadily increasing amount of countries, policy fields, types of organisations and technical forms. In the course of an administrative modernisation that can also be observed worldwide, evaluations are used to make measures, projects and programmes more effective and efficient, to supply decision makers with information and assessments in order to enable rational decision-making processes. Evaluations serve to measure success, accountability, legitimise political and administrative action and sometimes alsoāand this is the other side of the coināby means of evaluations decisions already taken are to be rationalised and justified at a later date. Evaluations are used ex ante for regulatory impact analysis, on-going for implementation adjustment or for impact analysis. In recent decades, the evaluation portfolio has continued to diversify both horizontally and vertically.
This goes hand in hand with a professionalisation process that can be observed worldwide. More and more training courses are offered at universities, academic and non-academic training centres, inside and outside public and private organisations, andānot to forgetāin the internet. Furthermore, the number of evaluation journals, textbooks, articles and studies has increased in tsunami terms. Quality standards for evaluations have been developed in many countries, associations and networks have been established and certification systems are being developed (Stockmann 2013). After almost 50 years of development history, it is now time to take some global stocktaking, because research on evaluation is still underdeveloped (Coryn et al. 2017) and merely concentrated on North America. Hence, many parts of the world are not well integrated into the scientific publishing system (Altbach 2003) because most of the important journals are in English and publications in other languagesāeven in other universal languages like Chinese, French, German, Portuguese, Russian or Spanishāare almost not recognised (Jacob 2005; Widmer et al. 2009). Studies on evaluation are, therefore, clearly biased and incomplete. This is even true for the most important work, the āInternational atlas of evaluationā edited by Jan-Eric Furubo, Ray C. Rist and Rolf Sandahl (2002). While many European countries are included (Denmark, Finland, France, Germany, Ireland, Italy, the Netherlands, Norway, Spain, Sweden, Switzerland and United Kingdom), almost all African (with the exception of Zimbabwe) and Asian countries (with the exception of China, Japan and South Korea) and all Latin American and Central Eastern European Countries are missing. Nevertheless, from a European perspective the atlas (and its update by Jacob et al. 2015) is a wonderful reference for āThe institutionalisation of evaluation in Europeā.
Again, the focus in this volume is on Europe (including Eastern Europe). But this volume marks the beginning of a book series that also focuses on the other continents. This is all the more important as evaluation is developing particularly dynamically there. For this reason, the Europe volume will be followed by three further volumes that examine the institutionalisation of evaluation in America, Africa and Asia.
The reason for this order is just pragmatic: for the editors, the access to authors was easier (and faster) here than in other areas and an early start at the European Evaluation Society (EES)-conference in Maastricht 2017 was possible.
All four publications are based on the same theoretical and methodological concept, which leads to a common analysis grid. In this way, a high degree of systematic comparability across countries and continents is to be ensured.
The chapter is structured in such a way that the current state of research on the institutionalisation of evaluation is briefly discussed before the theoretical and methodological concept applied to all four volumes is presented. Afterwards, the selection of countries for the European volume and the challenges to be mastered with the production of the volume will be discussed. A total of 16 countries and the role of the European Union (EU) will be covered. The Europe volume concludes with a synthesis chapter in which the most important results are worked out in cross-section and, as far as possible, attempts are made to explain the different degrees of institutionalisation of evaluation.
2 State of Knowledge
As already mentioned above, Furubo, Rist and Sandahl provided the first systematic overview about evaluation systems in 21 different countries and three international organisations. The purpose of the research was to describe the countriesā individual systems as well as global trends and developments, and to explain these developments regarding possible driving forces and consequences. The methodological approach was to conduct country case studies oriented on nine indicators. In 2015 an update was presented by Jacob, Speer and Furubo,1 offering a cautious longitudinal comparison. The finding in this longitudinal comparison was a general āliftā effect of institutionalisation in all countries, or like Furubo and colleagues state it āevaluation culture has matured over the last decadeā (Jacob et al. 2015, p. 23). While the former forerunner countries from 2001 (United States of America [USA], Canada, Australia and Sweden) remained quite static at the high level they achieved previously, others fulfilled huge efforts to catch up and thereby managed to establish themselves at the rankings top (such as Spain, New Zealand and Japan). This ranking can be considered as very useful to get a first impression about the actual state of evaluation in a country, but two points lead to a limited comparability, namely that evaluation, evaluation culture and the institutionalisation of evaluation lack a clear definition, and the fact that in the update from 2015 different experts have been questioned than in 2002, which might have induced a shift in the results. Besides that, it seems that no theoretical deduction of the indicators took place and an indicator measuring training and education is missing. The focus lays, like already mentioned above, clearly on the European and North American continent, while all Latin American countries and nearly all African and Asian countries were left out. Still the findings presented by Furubo and colleagues are important contributions towards how evaluation is institutionalised. It has been possible to work out three different internal driving forces for the development of evaluations. Those are the political constellation, the fiscal situation and constitutional features. External driving forces might be significant external pressures by donor countries or organisations (cf. Stockmann and Meyer 2014; Furubo et al. 2002; Jacob et al. 2015).
Another publication dealing with the institutionalisation of evaluation was presented by Widmer et al. (2009). This aims to capture the development of evaluation in a systematic way in 10 different topic areas, thereby comparing Switzerland, Germany and Austria. It covers the institutionalisation of evaluation, focusing on the constitutional and other legislative foundation of evaluation as well as the anchoring of evaluation in parliament, government and administration, different topic areas and the use of evaluations as well as sectoral and national trends. The obvious difference in comparison to Furubo and colleagues is the focus on specific sectors. Each chapter is written by a different expert from the specific sector in the specific country. Each thematic rubric ends with a comparison of all three countries. Thereby it is possible to identify differences and similarities of the countries and sectors. To enable the already mentioned comparisons the authors set a strict framework for the chapters and defined evaluation as systematic, transparent and data based. The authors had to follow this definition. It can be criticised that the methodological approach of the single chapters is not systematicāsome relied only on their own expertise in this area while others carried out written surveys or systematic analysis of databases (cf. Widmer et al. 2009). Still the book provides a detailed description of the differences and similarities of all three countries regarding the institutionalisation of evaluation.
Two studies about National Evaluation Policies have been conducted by Rosenstein in 2013 and 2015. In 2013 Rosenstein carried out an internet research of 115 countries and found that 20 already possess a written, legislated evaluation policy, while others are either developing a policy (23) or conducting evaluation routinely without a policy (34). 38 did not provide any information indicating that they are developing one at the moment (Rosenstein 2013). These findings have to be interpreted very cautiously due to the fact that the study of Rosenstein was performed solely in the internet, which can lead to a false categorisation, if one is not familiar with the countryās political system, language or other issues. The cross-country comparison about different legislation of evaluation can be seen as a starting point for further research, but systematic assessment of the quality of governance as well as the development of evaluation will be necessary (cf. Stockmann and Meyer 2016).
In the āFuture of evaluationā, by Stockmann and Meyer (2016), more than 30 different authors from 20 countries placed in all continents provide an overview about the professionalisation of evaluation in their countries. The focus lays, like the title expresses, on the future of evaluation, meaning how evaluation will develop in different countries around the world. Topics covered in this book are challenges, which might be able to weaken, stop or reverse the increase of evaluation or the question, if there will be a globalisation process leading towards āoneā evaluation culture, or if there will be differentiation of the evaluation culture according to the political culture of various countries and their specific use of evaluation. The strength of the publication clearly lays in the provision of a new viewpoint towards evaluation, also including countries that are normally not on the radar of evaluation research. But although the editors provided indicators to ensure comparability, these were not systematically explored in all contributions. Therefore, also this publica...