School Improvement Through Performance Feedback

A.J. Visscher, R. Coe

• Publisher: Routledge
• Year: 2013
• Print ISBN: 9789026519338
• eBook ISBN: 9781134381173
• Topic: Education
• Subtopic: Education General

Internationally there is an increasing trend to publish and feed back information to schools and teachers on their functioning. School improvement is often the central goal, though accountability and the promotion of parental/student school choice also play an important role.Practical initiatives and research in this field have often failed to recognize the full complexity of improving schools through school performance feedback. Relevant questions are, for example, to what extent: the information fed back to schools is valued and understood by them; the school performance feedback enables appropriate diagnosis and remediation to take place in schools; the use of the feedback generates conflicts and tensions for teachers and administrators; the features of the implementation process and school organization influence the utilization of school performance feedback systems; schools do actually improve as a result of performance feedback; and the feedback systems have unintended effects. This book fills the gap by including: a thorough conceptualization of school feedback as a performance enhancing mechanism, and the prerequisites for utilizing this information in schools; a typology of school performance indicators; a review of the research on the impact of feeding back performance information to individuals and to organizations; evidence on the process, problems and impact of school performance feedback from an international variety of contexts (Australia, the Netherlands, United Kingdom, USA); and reflections on these experiences within the theoretical framework, leading to recommended strategies for school improvement through performance feedback.

PART 1

Theoretical Introduction

1

Evidence on the Role and Impact of Performance Feedback in Schools

Robert Coe

University of Durham, England

1.1 Introduction

There can be few statements in social science more likely to gain popular agreement than the claim that giving feedback can improve a person’s performance on a task – and few topics that have been the subject of more research. The use of performance feedback in schools and other organisations is becoming more widespread every year and often presented as needing no justification. However, the evidence about feedback effects is mixed, complex and not well understood. Research results indicate that feedback can be beneficial to future performance, but it can also do harm. Moreover, the relative lack of evidence derived specifically from school contexts makes it hard to predict confidently what the effects will be in any particular case.

This chapter sets out to review what is known about feedback effects, drawing on empirical evidence and theoretical understandings from education, psychology and organisational behaviour. After a brief consideration of the complexity of feedback research, it will attempt to clarify what is meant by ‘feedback’. A number of the theories that have been proposed to account for the mechanisms of feedback effects will be examined. The conceptualisation of ‘performance feedback’ also requires that different kinds of ‘performance’ be defined, and research from three particular contexts will be examined: feedback in learning, feedback on performance in organisational settings and, most relevant to the present context, school performance feedback. The chapter will then attempt to summarise the evidence about the effects of performance feedback in relation to specific characteristics of the feedback and of the task. Finally, it will consider some of the reasons for the difficulty of making predictions about the impact of feedback, particularly in school contexts.

1.1.1 A Complex Picture

Published research on feedback effects is extensive in terms of both its quantity and the length of its history. Much of this research seems to be characterised by an apparent clarity about the benefits of feedback, despite often seeming very unclear about precisely what ‘feedback’ is or what kinds of ‘performance’ may be helped by it. For example, Ammons’ (1956) influential review was already able to draw on over 50 years of research and concluded:

Almost universally, where knowledge of their performance is given to one group and knowledge is effectively withheld or reduced in the case of another group, the former group learns more rapidly, and reaches a higher level of proficiency. (p. 283).

However, many of the studies reviewed by Ammons, and subsequently by others, did in fact contain inconsistencies with this belief, but were regarded as anomalies or ignored (Kluger & DeNisi, 1996). Indeed, the plausible view that feedback generally enhances performance is still prevalent in the literature (e.g. Neubert, 1998). Nevertheless, a closer examination of the evidence reveals a far more complicated picture: feedback is by no means always beneficial in its effects, and identifying the conditions under which it may be expected to improve performance is far from straightforward.

Probably the most significant step forward in untangling this complexity is Kluger and DeNisi’s (1996) meta-analysis. In analysing 131 studies¹ (607 effects) on the effects of ‘Feedback Interventions’, Kluger and DeNisi found that although the average effect was moderately positive (weighted mean effect size² 0.41), over 38% of the effects were negative and the mode of the distribution of effect sizes was zero. They concluded:

F[eedback] I[ntervention]s do not always increase performance and under certain conditions are detrimental to performance. (p. 275).

Similar results have been found in other recent reviews and metaanalyses (e.g. in Bangert-Drowns et al., 1991; Locke & Latham, 1990, Balcazar et al., 1985).

1.1.2 Towards a Conceptualisation of ‘Feedback’

Perhaps inevitably, ideas about precisely what ‘feedback’ is do not always seem to coincide. It is important, therefore, to clarify exactly what is meant by ‘feedback’ in this context.

Much of the early literature refers not to feedback but to ‘Knowledge of Results’ (KR) or ‘Knowledge of Performance’ (KP). These correspond to information about the outcome of the task undertaken, such as performance on a test, the development of a motor skill, compliance with a behavioural injunction, job productivity, etc. Excluded from this definition of feedback, however, would be any information about the process of how one undertook the task, as, for example, the message “you do not use your thumb for typing” (Kluger & DeNisi, 1996, p. 255).

Kluger and DeNisi adopt a definition of feedback that is similar to the KR/KP concept, but their focus is more specifically on feedback as an intervention that can be manipulated. Thus, they define Feedback Interventions (FIs) as “actions taken by (an) external agent(s) to provide information regarding some aspect(s) of one’s task performance”. They therefore exclude ‘natural’ feedback arising without external intervention, feedback that arises directly from the task, feedback that is not task-focused and feedback that is actively sought.

Black and Wiliam (1998a) provide a useful conceptualisation of the feedback mechanism by identifying four elements of a feedback system:

• data on the actual level of some measurable attribute;

• data on the reference level of that attribute;

• a mechanism for comparing the two levels and generating information about the gap between the two levels;

• a mechanism by which the information can be used to alter the gap.

For Ramaprasad (1983), “Feedback is information about the gap between the actual level and the reference level of a system parameter which is used to alter the gap in some way” (p. 4). This definition is clearly more restrictive in that it requires not only that the information must point out a gap between performance and some standard, but even that it must lead to action to address it. If the information is not actually used in altering the gap, then there is no ‘feedback’.

This seems to limit unnecessarily what may be classified as feedback and rather to pre-empt the empirical/theoretical question about how feedback works. Nevertheless, the concept of a ‘gap’ between performance and some standard seems to be important in conceptualising feedback effects.

Kluger and DeNisi (1996) list four possible ways a person may respond to the gap: one can seek to alter the performance or the standard, abandon the standard or reject the feedback. Each of these four options can be seen to be the likely choice in certain circumstances, and their Feedback Intervention Theory (FIT) sets out to predict which of them will be chosen in a particular situation. This theory is described in more detail below. However, they point out the need to take account of the fact that in most situations people have multiple goals or standards and that performance itself – especially in complex tasks – can also be multidimensional.

Some writers have sought to identify what may be important dimensions along which the characteristics of feedback may vary. For example, in a review of the effects of feedback on performance in organisational settings, Ilgen et al. (1979) point out that feedback can originate from three kinds of sources: other individuals, the task environment and one’s self. Prue and Fairbank (1981) identify a further four dimensions: the mechanism used to transmit the feedback (e.g. written, spoken), the content of the feedback message (e.g. comparison with others or with own previous performance), the unit of feedback receipt (individual or group) and the frequency with which feedback is provided.

Kluger and DeNisi (1996) provide something a little more systematic in their analysis of significant feedback characteristics. They coded the studies in their meta-analysis using the following variables, making predictions from their FIT about the expected direction of effect:

• feedback sign (positive or negative);

• feedback content:

- identifies performance as right/wrong

- gives correct solution

- gives attainment level

- compares with previous performance

- compares performances with others’

- provides information on performance of others

- designed to discourage

- designed to praise

- provided verbally (spoken)

- provided in writing

- provided graphically

- provided from a computer

- made public

- refers to group performance;

• feedback frequency.

The results of their moderator analysis will be considered below.

It seems appropriate, for the purposes of this chapter, to take a broad and inclusive definition of feedback and then to investigate the empirical evidence about the conditions under which certain effects will be found. We may therefore loosely define feedback as “information relating to some aspect of task performance”.

1.1.3 Development of Theory

Perhaps the most influential theory in the development of understanding about feedback effects is Thorndike’s (1913) ‘law of effect’. According to this, positive feedback is seen as reinforcement, negative feedback as punishment. The former reinforces the correct behaviour; the latter punishes the incorrect. However, as Kluger and DeNisi (1996) point out, this theory was never well enough specified to make clear predictions in any but a narrow range of contexts, and perhaps more crucially, it was inconsistent with many empirical results. Despite this, it had a substantial influence on research on feedback until at least the 1960s.

Subsequently, a number of other theories have made predictions about feedback effects, or have focused on the concept of feedback, though generally without aspiring to the same breadth and scope as the law of effect. Four of these in particular will be mentioned briefly.

First among these is control theory or cybernetics (Carver & Scheier, 1981; Podsakoff & Farh, 1989). This assumes that individuals’ reactions to feedback are determined by their desire to close the gap between their actual level of performance and their internal standards. Where the feedback shows no discrepancy, subsequent goals and effort will remain stable.

Second is the theory of goal setting (Locke & Latham, 1990), which claims that task performance can be enhanced by setting specific, challenging goals. Goal setting theory predicts that people are likely to strive to improve their performance, provided the goal is clear and they have high commitment to it – commitment being determined by a combination of the desirability of the goal and a belief in its attainability. Feedback moderates the effect of goal setting by pointing out the discrepancy between goals and actual performance.

Third is the theory of learned helplessness (Abramson et al., 1978; Mikulincer, 1994), according to which the repeated experience of failure leads individuals to adopt maladaptive performance strategies.

Fourthly, social cognitive theory (Bandura, 1991). According to this, individuals’ reactions to feedback depend on two processes: a comparison between the performance feedback and their own performance standards (any discrepancy leading to dissatisfaction which motivates greater effort), and a judgement of self-efficacy. Beliefs about self-efficacy are both a product of feedback and a factor in determining responses to it. Negative feedback may reduce self-efficacy, but if it does not (for example, if previous experience in similar tasks has been of success), it is likely to lead to increased effort to reduce the gap.

These four theories clearly overlap in the extent of their coverage, the evidence on which they draw and the predictions they make in a given situation. There are also other theories which have been developed with more specific applications, for example models of the feedback process as it relates to learning (Kulhavy & Stock, 1989; Bangert-Drowns et al., 1991) or organisational behaviour (Balcazar et al., 1985).

Perhaps the most comprehensive review and synthesis of this vast literature is by Kluger and DeNisi (1996), who attempted to build on these theories in so far as they relate to feedback, synthesising and improving as necessary. The result is their Feedback Intervention Theory (FIT), which has five basic propositions, and will be briefly summarised.

The first is that behaviour is regulated by comparisons of feedback to standards, with the four possible choices outlined above for removing the ‘gap’. This argument is shared by both control theory and goal setting theory, but while the former sees people as primarily motivated to reduce the gap, the latter accounts for behaviour in terms of achieving a goal, rather than simply eliminating a discrepancy. When individuals lack belief that their actions are likely to lead to success (perhaps as a result of repeated and extremely negative feedback), they may be expected to exhibit ‘learned helplessness’ (Dweck, 1986; Mikulincer, 1994) and are most likely to abandon the standard against which they are being judged. If abandoning the standard completely is not possible or desirable, it may be lowered instead; alternatively, if feedback suggests that performance exceeds the standard, the latter may be raised. Finally, there is some evidence that the feedback is more likely to be rejected if it is negative (Ilgen et al., 1979) or comes from a source lacking in credibility or status (Brinko, 1990, 1993). The likelihood of rejecting the feedback message may also depend on cultural and individual personality factors.

The second of Kluger and DeNisi’s (1996) propositions is that goals are organised hierarchically. Within ...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Contents
• Introduction
• PART 1 Theoretical Introduction
• PART 2 Evidence on School Performance Feedback
• PART 3 Conclusions, Reflections and Recommendations
• Index