Introduction
Theories of Change (ToC) is an evaluation approach emerging in the context of community change initiatives in the USA (Connell and Kubisch 1998). It has since been used more widely in other countries and sectors, for example health (Sullivan et al. 2002; Barnes et al. 2003; Mackenzie and Blamey 2005; Sullivan and Stewart 2006; Breuer et al. 2016), education and higher education (Hart et al. 2009a, b; Levy 2012; Richards et al. 2016), community development (Archibald et al. 2016), crime (Hopkins and Wickson 2013), and agriculture (Mayne and Johnson 2015; Thornton et al. 2017). Although ToC has evolved in the discipline of evaluation theory and practice, it is not restricted to this purpose. The approach involves facilitated development of models that can form the basis for planning implementation and evaluation activity, and reflecting on the results of evaluation , to inform decisions about further improvement. Those familiar with soft operational research (OR) and problem structuring methods will recognise elements of the approach. However, it is anticipated that it will be new to practitioners more familiar with project management methodologies.
The examples used in this chapter to illustrate the application of ToC are from educational development activity in the context of a large UK university . The term educational development is used to mean â systematic and scholarly support for improving both educational process and practices and capabilities of educatorsâ (Stefani 2003, p. 10). UK universities are expected to undertake a process of systematic continuous improvement of their educational provision (Higher Education and Research Act 2017) guided by a quality code (Quality Assurance Agency 2017). However , there continues to be debate about methods and measures to inform improvement efforts (Gibbs 2010). There has been much criticism of these efforts focusing on a narrow range of sector-wide quality assurance measures (Harvey and Williams 2010a, b).
The approach illustrated here is intended to help practitioners understand how their change strategies are working in the specific contexts of application. However, it is adapted with some systems thinking to improve connection with the wider environment and higher-level strategies. Systems thinking involves exploring a situation of interest âas if it wereâ a complex adaptive system . The theoretical basis for this is explored in the next section. This is followed by a description of ToC applications in educational development projects, and finally a personal critical reflection on the learning from these applications.
Theoretical Background
The theoretical model of organisational learning typically relied on to underpin quality enhancement processes in higher education is that of the reflective practitioner (Schön 1983; Kolb 1984). From this perspective, improvement action is assumed to be informed by practitioners actively engaged in attempting to understand how and why their implemented teaching strategies work (or not) in specific contexts of implementation. It is argued that through their everyday activity practitioners develop mental models about the complex dynamics of the situations in which they practise. These models have been termed âtheories of changeâ (or variations on this), and in reflective practice these theories are consciously and continuously tested and revised through learning cycles of planning, action, evaluation , and reflection. More recently it has been argued that there is a need to develop improved and explicit ToC (Trowler et al. 2014) with joined up thinking about the connection between change at different levels of organisation (Trowler et al. 2005, 2014). This organisation-wide learning and change requires this process to be undertaken collectively (Biggs 2001, Vince 2002) and through rigorous action research (Argyris and Schön 1996; Kember 2002; Marks-Maran 2015). However, a more formal and collective approach to educational action research has been acknowledged to present significant challenges in terms of the complexity of the social and political processes (Trowler et al. 2005). For example in motivating and engaging participants (Greenbank 2007), establishing shared goals and vocabulary for collaborative work (Jacobs 2016), and producing outputs that are more widely transferable and usable (Saunders 2012).
Programme evaluation aims to improve ToC about how intervention programmes work in practice in specific contexts and is used to inform decisions and actions to improve these situations (Funnell and Rogers 2011; Patton 2012). Typically this type of approach involves some sort of âmodellingâ or âproblem structuringâ of the situation of interest, which helps in framing the planning of data generation, analysis, and interpretation. The approach is not prescribed, and there is much debate about how decisions about the evaluation design affect what is learnt and how evaluation is used to influence improvement. In addition to decisions about, for example methods, data, and participants, there are core decisions about whose theories are tested, and whose questions are answered. These choices can reflect fundamentally different assumptions about the complexity of improving organised activity and learning about how this can be achieved. For example the reflective practitioner or action research mode of inquiry described earlier is often considered to lack rigour because the investigator is too closely involved in the situation, and has a vested interest in the findings and outcomes , which may bias their interpretation of them. The use of external âexpertsâ is often used to introduce this rigour. In the OR literature, Franco and Montibeller (2010) identify this expert mode as the most common and traditional approach to OR intervention. In this mode, the assumption is that an (external) âexpertâ can straightforwardly define âsuccessâ and use objective and scientific methods of inquiry to measure success, discover how activity and other factors are influencing success, and therefore recommend solutions. However, the risk associated with this mode is that the prioritisation of the expertâs definition and criteria of success will lead to findings not thought to be relevant or useful by other stakeholders, and will not be used (Patton 1986). This approach therefore does not appear compatible with enhancement processes, where improvement needs to be understood from the perspective of a wide range of stakeholders and is dependent on the actions of many actors. From a utilisation-focused perspective (Patton 1986), the inquiry should help decision makers and others that have the ability to influence change in a situation to arrive at their own judgements about, and commitment to, the improvement needed. This suggests a more facilitated and developmental approach is needed. In the facilitated mode of OR intervention (Franco and Montibeller 2010), consultants facilitate a participative process of problem structuring with stakeholders to guide their intervention and inquiry. It is accepted that different stakeholders and actors involved will have different notions of success and how to achieve it, based on their previous experiences, learning, values, motivations, and the information to which they have access. Cause-effect relationships in these situations are therefore understood more as producer-product relationships that are socially constructed, that is through peopleâs actions based on their own mental models of their effects in the complex contexts in which they act. Models are probabilistic rather than predictive, and the more complex the situation, the more uncertain the outcomes . In developmental approaches (Fetterman 1994; Patton 1994), the facilitator also helps in capacity building for o...