Now, I would like to turn to the first issue referred to above: the educational evaluation of computer-based practice. Because of my concern with social psychological themes, it is natural to begin with some observations focusing on the people caught up in this innovation. In the first of the following two sections, I will summarise difficulties associated with the reaction of practitioners to the implementation of new technology in education. In the second section, I shall summarise evaluative research on learning outcomes associated with computer-based instruction where it has been implemented.
Evaluation of implementation strategies
When we reflect on experience within schools and colleges over the past ten years, it is inevitable that our judgements about progress will be influenced by what was expected when investment and training began. These expectations were often extravagant. As Maddux (1989) observed in this context, pessimism is the familiar enemy of innovators, so a degree of vigorous optimism was natural enough in the early period of computer diffusion. Even commentary on the very first, and most modest, examples of such educational intervention could be fired with enthusiasm (the title of a review by Feldhusen and Szabo (1969) refers to computer-assisted instruction as the āeducational heart transplantā). A more recent judgement that is often cited occurs in an article by Bork (1980). He comments:
We are at the outset of a major revolution in education, a revolution unparalleled since the invention of the printing press. The computer will be the instrument of this revolutionā¦. By the year 2000, the major way of learning at all levels, and in almost all subject areas, will be through the interactive use of computers. (p. 53)
This prediction has more time to run but, in my view, it now looks to have misjudged something significant in the relation between education and new technology. It seems that the diffusion of this technology has not been as dramatic as was expected in the early period of microcomputer development. Certainly, within recent years, a number of commentators (themselves sympathetic to computer-based learning) have felt obliged to remark on the problems associated with getting computers into active use within education (e.g. Bliss, Chandra and Cox, 1986; Collis, 1987; Cox, Rhodes and Hall, 1988; Cuban, 1986; Hanson, 1985; Heywood and Norman, 1988; Holden, 1989; Lepper and Gurtner, 1989; McCormick, 1992; Plomp, Pelgrum and Steerneman, 1990).
Consequently, actual classroom usage remains limited. A recent British government report suggests that only about 20 per cent of teaching time is making use of computers (DES, 1989b). Similar limits on uptake are apparent in other countries (Dillon, 1985; Plomp and Pelgrum, 1991). Becker comments on the findings of one large-scale US survey: āin spite of the changes that computers have brought to schools, only a small minority of teachers and students can be said to yet be major computer usersā (Becker, 1991).
What are the obstacles? It is natural to seek them within the attitudes or strategies adopted by the teachers who manage this technology ā by looking in a focused way at what is being done at the classroom chalk-face. However, this would be too narrow a view. McCormick (1992) characterises problems arising from a widespread failure to develop a whole-school strategy. McInerney (1989), Plomp et al. (1990) and Wild (1991) identify a whole range of issues at the institutional level that need to be confronted to make this innovation work.
So, progress may depend, to an important extent, upon action organised at the level of institutional practices. Research that is directed more at the classroom level tends to dwell on teachersā lack of self-assurance when using this technology. For example, Heywood and Norman (1988) highlight obstacles to good practice arising from a shortfall in (primary) teachersā self-confidence. In Britain at least, there was limited anticipation of how difficult it might prove for staff unfamiliar with computers to assimilate them into their practices. On reflection, the combination of circumstances characterising many teachersā first encounters with this technology should have been fairly explosive. Early configurations of classroom microtechnology were tedious and time-consuming to prepare (often requiring the loading of programs off small audio cassettes). Educational software could be of very dubious quality. All sorts of occasions were possible where the computer would appear to fail ā leaving the teacher exposed as having lost control (the childrenās more spontaneous enthusiasm having been undermined in the process).
Politicians and educational administrators were sensitive to this problem ā if not to its scale. Certainly, some of the extra financial support for priming this innovation was given over to staff development. Many formal courses of in-service training (Inset) were offered. Yet the feeling often expressed within the profession (in Britain, at least) is that it was not enough and, often, not of the right character. It is now popular to challenge the faith of early policy makers (e.g. Fothergill, 1984) that in-service provision was the quickest way to create an impact. A cascade model underpinned much of this thinking: the hope was that those who received training on intensive short courses would go back to their institution and pass on their expertise. For one reason or another, if they gained any expertise, it looks as if they often kept it to themselves (Boyd-Barrett, 1990).
The contemporary view is that a better strategy would have been to concentrate more effort on initial training (Davis, 1992). We are still in a situation where new teachers can be awkwardly unfamiliar with this technology ā possibly seeing it only as threatening (Bracey, 1988; Wellington, 1990). It remains true that many teachers will have had only superficial pre-service exposure to new technology; few will yet have enjoyed the experience of growing up themselves within an established culture of computer use. At the moment, the opportunities for teachers to gain confidence with new technology across the period of initial training are often limited. One survey in 1986 suggested that only 10 per cent of students would use IT on teaching practice (ITTE, 1987). The situation has improved recently, although most students report they are still encountering the technology as an isolated activity (Dunn and Ridgeway, 1991).
The urgency of this problem is hinted at by one extensive review of the effectiveness of microcomputer work (in primary classrooms). In a meta-analysis of recent evaluative research, Ryan (1991) documented the effects of forty variables on the impact of computer-based learning experiences. Only one external variable was found to exert any moderating effect of computer activity on pupil achievement: the extent of teacher pre-training on the activity under study. This draws attention to the fact that effective preparation involves more than instilling the confidence to motivate implementation. The success of computer-supported learning also depends upon teacher contact with pedagogic ideas concerning good practice with this technology: the enthusiastic teacher needs to be prepared in this sense also.
Of course, any present initiative for acting at the point of initial training is of little relevance to teachers already in post. Thus, attention to the format of Inset experiences remains important. At present, there is evidence that these experiences are not always ideal. The problems are not merely a limited cascade of expertise: the experience of course participants themselves is often one of disappointment. The problems are illustrated in an extensive study of in-service provision carried out by Rhodes and Cox (1990a, b). They were able to witness the management of training programs and to visit the schools of staff who had attended them, so they could observe classroom practice as well as interview participating teachers. A somewhat gloomy picture emerges from this comprehensive survey. The training regimes did not appear well matched to the experience or needs of these teachers. Consequently, they were not as effective as the tutors had expected or hoped. Much of what was achieved related to problems of using the technology itself, at the expense of tackling real educational issues. Half of the sample of teachers believed the computer resulted in an increase in their workloads and that it made no fundamental change to the way in which they worked ā merely reinforcing existing patterns of activity.
This snapshot of practitioner experience is sobering ā particularly to those of us researchers whose (possibly selective) contact with classroom practice may create a rosy picture of innovative possibilities. The situation is well summarised in one survey that revealed only 14 per cent of primary school teachers felt competent to use a range of IT applications without assistance (Davis, 1992). Yet how might the general picture be made more heartening? Rhodes and Cox identify the commitment of the head teacher as significant in determining attitudes within a school more generally. But they also urge more effective experiences for preparing and supporting teachers in their use of this relatively unfamiliar resource. Thus, at present, there may be few sites where a culture of computer use is comfortably established ā where the potential impact of particular computer-based activities can be evaluated in a convincing manner.
Nevertheless, my colleague Geoff Alred and I have recently had the opportunity to study one initiative where an effective context for innovation was carefully cultivated and thus where circumstances seemed more favourable to effective implementation of the kind that policy-makers hope for. A local education authority invited primary schools to volunteer staff for participation in a project to evaluate Turtle Logo. This is an activity that will be described in more detail in a later section. Suffice to say it is a challenging exercise in computer programming based upon controlling the movements of either a floor robot or a screen icon (āturtleā). New equipment was supplied; the project ran over a generous time period (at least four terms), and it incorporated specialised in-service support (with suitable teaching cover). In short, the conditions of the venture would seem to be very favourable: the focal activity (Logo) is widely endorsed in early education and the project was well supported by specialised training opportunities. Moreover, the participants were motivated (if not highly experienced) and could enjoy the advantage of being part of a community of innovators. In summary, this situation seemed to us to approximate what elsewhere has been described as the āidealā circumstance of an IT-related in-service provision (Owen, 1992, p. 130).
These teachers kept diaries summarising their experience and Aired and I were able to interview them at some length towards the end of the formal project. The emerging picture is a mixed one. Although it was not part of our purpose to observe the classroom activity directly, it was apparent that the children had enjoyed the Logo work: most of the teachers had been impressed by their engagement with it. Yet, on balance, the implementation project as a whole cannot be regarded as a great success. One measure of success would be how far the activity remained in use to become part of classroom routine for subsequent generations of pupils to enjoy. One year after the official end of the project, less than a quarter of the teachers were found to be still using Logo with their new classes. As it happens, our conversations with them had led us to expect this.
Although they recognised the innovative nature of the activity ā as well as the childrenās enthusiasm for it ā they were also keen to identify the practical difficulties associated with managing it on a routine basis. Mundane problems of unreliable turtles were a major source of disappointment. But as the activity can be supported on the screen alone (i.e. it does not depend on a working floor robot), this cannot explain the widespread failure to consolidate the experience beyond the life of the project. Other problems mentioned related to the time and effort involved in preparing the computer and ensuring its security. The teachers also remarked on the difficulty of monitoring and supporting the activity while classroom life continued as normal around it. Finally, taking such necessary effort into account, there was some doubt as to whether comparable academic achievements could not be reached in simpler ways.
To some extent the adequacy of the in-service provision arises again here: certainly, this was another source of some dissatisfaction among the participants. Many of the problems experienced seem as if they should have been tractable with experienced advice and encouragement; however, there is a danger in continually laying the blame at this door. At some point we may have to acknowledge that the creative deployment of this technology puts a lot more strain on the status quo of classroom life than has been recognised. More sensitive preparation and training might be some part of a solution to this problem but there is clearly an invitation to reflect more carefully on defining the optimal computer environment for supporting innovation as effortlessly as we can. That is an issue to which I shall return later in this book.