Smart Evaluation and Integrated Design in Regional Development

Territorial Scenarios in Trentino, Italy

Grazia Brunetta

• Publisher: Routledge
• Year: 2016
• Print ISBN: 9781472445834
• eBook ISBN: 9781317054245
• Edition: 1
• Topic: Politics & International Relations
• Subtopic: Local & Regional Planning Public Policy
• Index: Politics & International Relations

Smart Evaluation and Integrated Design in Regional Development puts forward an alternative approach to evaluation in spatial planning - one that focuses on 'territory' and 'landscape'. The book introduces an innovative evaluation approach, namely Territorial Integrated Evaluation (TIE), a meta-evaluation methodology for designing regional development scenarios. A research team from the Politecnico di Torino applied this methodology experimentally to the practices of spatial planning in Trentino in order to aid the Province in a process of institutional innovation that is still going on today. TIE defines territorial scenarios serving the need for regional economic development as well as the conservation of nature and landscape. A cross-border region, Trentino has a special need to harmonize economic development with the exceptional and internationally renowned value of its landscape which includes the Dolomites, a UNESCO World Heritage Site. Therefore TIE set out to design regional development scenarios that integrated various topics - retail, tourism, infrastructures, nature and landscape. By testing out TIE in practice in this extraordinarily dynamic institutional context, the book makes a significant contribution to the discussion about newly emerging approaches to spatial planning that involve multidisciplinary vision, new paradigms in regional development, and institutional learning and capability in decision-making.

Chapter 1

Territorial Integrated Evaluation Methodology

Grazia Brunetta

1.1 Aims of the Evaluation Process

Territorial Integrated Evaluation (TIE), the methodology being proposed and tested here, primarily aims to go beyond a merely one-dimensional approach to the regional planning of retail territorial development. In order to do this, it furnishes planners with information, thus laying more a solid foundation for the planning and design of territorial development scenarios in which the retail policy¹ could play on a convincing role in the enhancement of urban areas. The Department of Economic Development and Work of the Autonomous Province of Trento (APT), handling retail and tourism, can use this institutional and technical tool as part of their decision-making process in regional policy. TIE methodology is not meant to take the place of compulsory assessment procedures, such as Environmental Impact Assessment (EIA) and Strategic Environmental Assessment (SEA). Rather, it is meant to be a voluntary act of institutional evaluation to help in decision-making for planning and designing territorial transformation and development.

This entire initiative comes out of the conviction that Italy needs to change its approaches to evaluation in spatial planning. In other words, evaluation is something that is significant not merely as a technical-scientific procedure subject only to rigid regulations (such as the SEA). Rather, evaluation requires something more than, obviously, a solid supportive foundation in regulations. It requires that there be a thorough administrative and technical capacity to reorganize the practices of the stakeholders who take part in the processes of territorial planning.

Evaluation is even more worthwhile when it is interpreted as one of the many potential strategies to support the procedures of governance, so that decisions can be made that are more readily accepted and the spatial planning process can be strengthened (Alexander, 2006a; Lichfield et al., 1998; Miller and Patassini, 2005).² Thus strategic evaluation takes on the role of a technical process of learning accompanying the governance of decisions (Alexander, 2006a; Lichfield, 1996). It is not only a kind of dialogue conducted in order to reach decisions but also a process through which institutions can learn and accumulate more and more ‘usable knowledge’ (Lidblom, 1959; 1979).

Theoretically, TIE is a scientific methodology that builds innovative technical knowledge, supports local planning processes and innovates territorial governance, aiming ultimately to link regional development policies to different levels of planning (Archibugi, 2007; Brunetta, 2006; Khakee et al., 2008). Accordingly, TIE assumes that there is a general idea of evaluation in planning, and proposes this idea, one that articulates what the strategic dimension of evaluation means when it is applied to regional planning processes in order to strengthen the institutional design of the evaluative processes in the territorial planning, thus fostering innovation (Alexander, 2005). TIE is a process of meta-evaluation intended to act as a new capacity-building institutional framework for various territorial policies and plans. TIE’s main strategic action is that it emphasizes technical communication as a learning experience for institutional authorities, seeking out the highest possible integration of evaluation with the decision-making and territorial planning process. The rationale for this kind of evaluation is the need to construct, make specific, and reach an agreement beforehand on the relative importance of the various criteria used in the decision-making process.

TIE aims to work out an overall framework of shared principles and criteria for planning territorial development scenarios in Trentino. This would be a framework to refer to when proposing cooperative agreements and stabilizing existing ones, and when defining satisfying and profitable aims for all the stakeholders involved in decision-making. On the practical level, TIE integrates evaluation into the system of territorial planning as an institutional learning process. TIE shares the open and performance characteristics of a limited rationality approach and therefore requires new methods that can reveal the strategies that are competing to use certain resources and identify which actions are most effective for making decisions. This approach helps orient subsequent decision-making phases (in itinere and ex-post) and guarantee stability at the first level of cooperation that had been reached. Over time, it can increase consensus by making its positive results visible, leading to other potential forms of agreement. Such evaluation methodology also lends to heighten the level of participation and responsibility among the various parties involved in making decisions, which become the product of a process of social interaction (March and Olsen, 1989). Each party thus participates in the process of value definition and becomes part of an organizing ‘construction’, which is a fundamental resource for the planning of territorial development strategies.

TIE necessarily overturns the classic paradigm of evaluation, where ‘the evaluation depends on the values’, assuming on the contrary that ‘the values depend on the evaluation’ (Archibugi, 2006: 77). That values depend on evaluation is a principle, one that derives from the meaning of evaluation as a process aimed at attributing social values, which have no fixed or intrinsic meanings.³ Values do not depend on the objects in themselves, but take on variable properties over time that depend on socio-economic, territorial, institutional and regulatory factors – that is, the conditions, opportunities, circumstances and expectations within the territorial system (Alexander, 2009; Brunetta, 2013; Brunetta and Voghera, 2008; Moroni, 2006).⁴ In short, ‘the consensual valuing construction that is the outcome of the evaluation subject to continuous reconstruction (change) including refinement, revision, and, if necessary, replacement’ (Guba and Lincoln, 1989: 263).

Therefore values, evaluation and decisions are limited and partial actions that come out of the specific circumstances of the issues and objects being evaluated. These are issues and objects that, in turn, come out of specific social and institutional contexts made up of precise resources, opportunities, questions and expectations that change over time. It follows that TIE helps the values that are held to be ‘stable’ come forth, values that can be redefined and designed in order to support regional policy and territorial planning choices. The core of the TIE approach to evaluation is the construction of a shared knowledge and technical framework through the re-composition and progressive integration of resources, selection criteria and stakeholders.

Evaluation is designed in planning while it is being done. Afterwards, it is updated incrementally, as answers are being sought out to the problems that are arising in the course of the action. This strategic approach to evaluation is focused on the design of a shared technical and knowledge framework.⁵ This approach introduces meta-criteria that progressively recompose the positive effects produced over the course of the process of the making the decisions, meta-criteria that promote potential new forms of agreements (March and Olsen, 1995). The application of TIE in Trentino shows that meta-criteria – such as sustainable development, bio-based economy, and green and integrated transport – play roles that adopt themselves according to their positions during the act of evaluation in the planning process in order to help lay out the path of planning and motivate the direction it takes in the various territories of the Province (see Chapter 4).

TIE articulates the strategic logic of evaluation as a process with three main features:

• It is a gradual learning process that is continuously evolving. As such, an integral part of TIE is the design of improved monitoring systems that follow the implementation of territorial development;

• It is oriented towards the construction of policies (it is not just analytical). It is a means for innovating design, verifying effectiveness during the implementation phase, correcting errors and improving performances;

• It is a form of dialogue among institutions and between the institutions and citizens, the aim of which is to improve the level of cooperation and subsidiarity.

We should think of evaluation as a knowledge framework inside which we can go on to ‘re-construct’ the interrelations among resources, stakeholders and strategies progressively. The strategic logic of evaluation and spatial planning is something that goes beyond the technocratic dimension of the act of evaluation. Instead, this logic includes an approach directed towards proactive institutional learning that is initiated through a tailor-made approach that adapts itself to the planning approach in each territorial context.⁶ TIE plays a role in guiding and designing planning process based on a continuous institutional learning process. In this sense, the process of defining principles and values becomes an indispensable step in the decision-making process. In fact, it could facilitate the ‘strategic navigation’ of future development trajectories in spatial planning⁷ (Hillier, 2011).

1.1.1 The Meaning of Integration

The process of TIE in designing territorial planning scenarios for retail settlement policies is accompanied by the procedures of legally mandated evaluation. A primary example of these is the EIA, which is made during the final phase of the decision-making process and oriented mainly towards checking whether the project conforms to the planning aims. TIE goes beyond this prevailing logic of evaluation. It is not concerned with estimating the specific impact of any specific settlements project. As a result, this integrated territorial evaluation approach does not concern an estimate of the specific impacts determined by a precise settlement project (as in the case of the EIA).

TIE has a different emphasis. It focuses on combinations of the cumulative territorial effects of the potential retail development processes of Gross Leasable Areas (GLAs). In dealing with these effects, TIE is conceptually and operationally different from Cost-Benefits Analysis (CBA) and EIA, the two assessment methods for retail settlement projects. CBA uses a variety of countable evaluation techniques in order to propose a quantification of the various impacts. EIA uses market values rather than estimates of costs to relate the kinds of environmental impact that it identifies, proposing qualitative estimates of these impacts. EIA aims to check whether a project conforms to planning restrictions in order to estimate the extent of any adjustments to be made to protect environmental resources. Chiefly, both CBA and EIA estimate the impact – CBA, economic, and EIA, environmental – of establishing a new activity, using variables and values that are assumed to be definitive. Both CBA and EIA are based on the assumption that it is already clear to everyone involved what the resources at stake are – environmental, territorial and landscape-related – and that these resources will end up being used only at the end of the decision-making process, after the framework of the project’s variables (subject to assessment) has been defined.

For CBA and EIA, assessment is essentially a compliance check based on the assumption that the evaluation criteria and their relative importance have already been defined within the context of the general aims of the territorial planning choices. This sequence ideally traces a model of abstract decisional rationality. It is, however, broadly recognized that the process described above is not implemented in the planning practice according to a linear framework linking knowledge, decision and action. Rather, what usually happens is that public actors interact and construct agreements that result in setting up the relative ranking of the criteria for decision-making. These rankings are not a priori data. In the step from the regional policy level to that of local planning, it will become necessary to translate the framework of conditions and criteria to support the design of shared development choices in the various areas of Trentino.

When planners are trying to make decisions about establishing GLAs, it is impossible for them to follow linear/sequential hierarchy models that have been established in advance and pay no attention to the specific nature of each context. Rather, they present, compare and contrast various points of views and expectations.⁸ Significantly, TIE takes an approach that is not a neutral one, as the other approaches allege to be. TIE, instead, considers the ethical perspective in designing the process. TIE emphasizes that there is no unilaterally determined solution. In other words, there is no a priori solution that can be conveyed through allegedly neutral evaluation approaches, such as estimating the sum total of the individual preferences. Rather, a solution can also be the result of sharing principles and criteria that are not a priori, but emerge from the evaluation process itself (Lichfield, 1988; 1996). In this way, TIE can become an instrument that helps construct the values needed for decisions. Planning, of course, involves sector-focused decisions, those that are based on the priorities and points of views of a particular sector of society or the economy. From the perspective of TIE, these sector-focused decisions may be valid, but need to be compared, assessed and made to be consistent with a multidimensional point of view (Brunetta and Peano, 2003; Khakee, 1998b; Lichfield et al., 1998; Miller and Patassini, 2005a; Palumbo, 2001; Patassini, 2006; Voogd, 1998).

In this view, Alexander (2005: 296–7) also suggests the following:

the main problems with plan and project evaluation today cannot be solved with consciousness-raising methods: they demand deliberative and effective institutional design. Current planning paradigms give planners, evaluators and analysts a scientifically objective position or a socially benign role in a neutral evaluation process. The norms are demonstrably incompatible with our actual life-world […] Reality is different: evaluation methods and frameworks are weapons in a planning process that is a partisan debate between conflicting interests, and evaluation is usually a contested, non a consensual proc...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Contents
• List of Figures
• List of Maps
• List of Tables
• Notes on Contributors
• Acknowledgements
• List of Abbreviations
• Introduction Smart Evaluation, the Aims and Stages of a Work in Progress
• 1 Territorial Integrated Evaluation Methodology
• 2 The Design of Territorial Integrated Evaluation1
• 3 The Experimentation of Territorial Integrated Evaluation
• 4 The Analytic Network Process Method for Territorial Integrated Evaluation
• 5 A Mathematical Model for Territorial Integrated Evaluation
• 6 Smart Evaluation: The Agenda Emerging from Experimentation of the Territorial Integrated Evaluation Methodology
• Index