Evaluating in Practice

Ian Shaw

• Publisher: Routledge
• Year: 2018
• Print ISBN: 9780754678571
• eBook ISBN: 9781317138785
• Topic: Social Sciences
• Subtopic: Social Work
• Index: Social Sciences

Evaluation is not a self-contained phase of social work practice - one more dimension of the process - but a dimension of every phase. In this fully rewritten and updated second edition of his groundbreaking text Evaluating in Practice, Ian Shaw demonstrates how evaluation and inquiry are just as much practice tasks as planning, intervention and review. By demonstrating that good evaluating in practice helps sustain a commitment to evidence, understanding and justice, Shaw shows that for this to be achieved, evaluating in practice must permeate every aspect of social work. He: 1. Develops a framework for embedding evaluation and inquiry as a dimension of good practice in social work. 2. Demonstrates the central significance of a 'methodological practice' in social work that adapts, infuses, and translates social research methods as a dimension of the different aspects of social work, viz. assessment, planning, intervention, review and outcomes. 3. Facilitates good practice by exemplifying the argument through extensive worked examples and exercises. This book has much to say about the demanding skills that are necessary to achieve this shaping of practice and is a must-read for any social work student or practitioner.

1 Keeping Social Work Honest

Before I built a wall I’d ask to know
what I was walling in or walling out,
And to whom I was like to give offence.

Robert Frost, ‘Mending Wall’

I have three aspirations in writing this book. First, I am convinced social work will be greatly enriched if accomplished within a framework for embedding qualitative evaluation and inquiry as a dimension of good practice in social work. Second, I aim to persuade readers of the central significance of a ‘methodological practice’ in social work that adapts, infuses, inhabits and translates social research methods as a dimension of the different stages of social work – assessment, planning, intervention, review and outcomes. Finally, I hope to facilitate good practice by exemplifying the argument through extensive worked examples and exercises.

Several aspects of these aspirations need emphasising. First, I am writing about practice and not about research or evaluation. The case developed will be about evaluation and inquiry as practice tasks just as much as assessment, planning, intervention and review are practice tasks. But second, this book departs radically from almost all mainstream views of social work, in that evaluation is not seen as a self-contained phase of practice – as one more way of fracturing the social work process – but as a dimension of every phase.

Third, the book is not about the specific application of research or evaluation findings to practice but about the method of inquiry and evaluation. This distinction was helpfully made by the late William Reid:

Historically, the influence of science on direct social work practice has taken two forms. One is the use of the scientific method to shape practice activities, for example, gathering evidence and forming hypotheses about a client’s problem. The other form is the provision of scientific knowledge about human beings, their problems and ways of resolving them. (Reid, 1998: 3)

I would label this as the difference between research as ‘source’ for practice and research as ‘model’ for practice. It is the second of these that is central to my argument. As Støkken has expressed it, ‘Social services and other welfare services get increasing demands regarding their competence to document and evaluate their work. In this perspective, we must conclude that the research environment’s most important contributions are research methodology more than the actual research results’ (Støkken, 2009, emphasis in original). The book will have much to say about the demanding skills that I believe are necessary to achieve this shaping of practice – skills that I convey through the use of metaphors such as ‘translation’ and ‘inhabiting’.

Fourth, the book will primarily work from the rich literature and practice of qualitative inquiry and evaluation. This is partly intended as a counter-weight to the main alternative approach to using research methods and logic as practice methods. That alternative approach has been best developed in the United States and is associated with single system designs, evidence-based practice and the empirical practice movement, and is associated at its core with notions of control, outcomes and measurement. This book is not intended to be polemical – as with Frost in ‘Mending Wall’ I have no time for hunters who leave ‘not one stone on a stone ... to please the yelping dogs’. But I do aspire to provoke conversation and debate.

Fifth, I want to resist thinking about the book as advocating or describing a particular model of social work intervention. While it is reasonable to detect implications – on occasion critical ones – for existing ways of thinking about ways of doing social work, this is in no way a plea for replacing one or another model with a new one. I would not want to claim too much, and be guilty of the hubris of making one’s personal position stand for everything that matters about social work or anything else. Social work is not alone in being tempted by the allure of making this or that idea – be it globalisation, social capital, evidence-based practice or postmodernism – the prism through which everything is viewed and nothing is seen.¹ If at any point in the subsequent pages I seem to risk losing your or my way, it may be worth re-reading these opening paragraphs.

Is social work worth doing? And how do I know if I am doing it well, or even well enough? These questions – questions of evaluation – lie at the heart of what social work means. In this book I attempt to identify the knowledge, values and skills which are required for developing evaluation as a dimension of direct social work practice. The intention is to enable social workers, social work students and practice teachers to establish a critical, disciplined practice in partnership with service users, to evaluate their own practice, and to be rigorous in assessing the social work task. There are also secondary intentions – to speak to the social science community, and to those who deliver social work programmes in my own country and elsewhere. While I do not believe we need aspire to a circumstance in which social science and social work coexist in perfect amity, I would want to provoke a relationship between the two of intellectual reciprocity based on egalitarian respect – one that is less commonly encountered than might be expected. I also hope to unsettle the conventional practice competence models that undergird typical social work programmes.

There has been, at least until recently, animated, even passionate debate about the relative merits of positivist and humanist positions in social work. Yet the ways in which the debate has been conducted have been in large part unhelpful for our purposes. Not that there is nothing to debate, or that I stand as a neutral bystander (e.g. Shaw, 2008, 2010a; Shaw and Norton, 2008). But social workers have tended to adopt entrenched positions which make it difficult to get fully inside or outside the arguments. Hence positivism, for example, becomes ‘a swearword by which no-one is swearing’ (Williams, 1976), or we are sometimes left with the impression that if only we were courageous enough to ‘deconstruct’ a problem or take a ‘postmodern’ position, we would be more than half way to its solution. For both positivists and committed advocates of humanist alternatives Augustine’s comment is apposite – ‘total abstinence is easier than perfect moderation’. I believe social workers of any hue are mistaken when they adopt a debunking approach – when they give an account of positions with which they disagree, but only in the words of those who also disagree with the position.

Traditional arguments for humanist approaches, which until relatively recently proceeded roughly along the lines that qualitative methods provide a sounder portrayal of the real world than quantitative methods, have themselves come under sustained examination. There has been a blurring of the once firm boundary between social scientific ways of representing reality and literary forms of doing so. Critics who are pessimistic about the possibility of understanding the real world, and those who want evaluation and research to take an overtly political role, serve to re-open accepted beliefs about the relation between practice and evaluation.

Some social workers have explored the implications of different qualitative methodologies for direct practice (e.g. Atkinson, 2005; Riemann, 2005; Scott, 1989, 2002; Shaw and Ruckdeschel, 2002; White and Riemann, 2010). The thought is often expressed in a more general reflection as when Jim Drisko (forthcoming) recalls,

Each client I worked with represented a unique – sometimes even mysterious – person in a unique situation which had to be discovered as well as appreciated and understood. Isn’t that a kind of research? ... Yes, there are different methods for different purposes, but asking the right questions, staying attentive to both individual and cultural differences, and respectfully listening always aid learning and understanding.

Conversely, there are signs that some qualitative sociologists may have a reciprocal interest in the relevance of such work for practitioner audiences (Bloor, 2010; Bloor and McKeganey, 1987, 1989; Clandinin and Connelly, 2000; Janesick, 1998; Miller and Crabtree, 2005). For example, arising out of their comparative ethnography of eight therapeutic communities, Bloor and McKeganey argued that practitioners should be among the several different audiences for ethnography. They reflect on a number of practices observed in their research which appeared to promote therapy in the settings in which they were found. Without claiming any special privilege for their conclusions, they describe those practices for practitioner audiences. I hope that a side-effect of this book will be to increase this two-way traffic.

I have already implied that I do not wish to unthinkingly be ‘walling out’. But I am likely to think in terms of ‘meanings, norms, routines, rituals, interactions, deliberations, dilemmas, paradoxes, issues, and so forth – ideas that draw our attention to the immediacy and particularity of the experience’ (Schwandt and Burgon, 2006). This emphasis on ‘lived experience’ finds its roots in European philosophy. It includes a healthy scepticism of scientific evaluation of evidence-based practice kinds, although it does not preclude them. It requires more than a loose belief that everyone’s views are valuable, but requires ‘anthropological sensitivity’. Like the qualitative health researchers, Miller and Crabtree, I am prepared to ‘hold quantitative objectivisms in one hand and qualitative revelations in the other’. But I also concur with their comment that when reading most reports of randomised control trials, ‘the only voice you hear is the cold sound of the intervention and faint echoes of the investigator’s biases’, and, as they tellingly express it, ‘the sound of thin hush’ (Miller and Crabtree, 2005: 613).

This does not result in an individualistic understanding but includes language, rules, culture and institutions. It is the ‘practical’ rather than the inner world, whereby we grasp meaning in shared practices, in ways that are socially constructed. We will see in later chapters how this includes ways that people also ‘perform’ their constructed world. ‘Practice’ is a central word. It does not mean the same as the commonsense theory/practice distinction. ‘Practice’ is how we conduct life in society. It includes practical reasoning and the importance of practical knowledge. It calls for judgment and deliberation, and also connects doing and learning so that learning is not something we engage in when we are not busy doing practice. This brings in issues of inequity and power and appreciation of the role of narrative.

Illustration

A straightforward illustration may be useful at this stage.² Evaluating in practice can be assessed according to the extent to which it promotes evidence, understanding and just practice. These three principles can be spelt out as follows. Social workers should practice in the light of the best evidence about what works well for service users, so that service users and carers receive service that is fair, just and enabling. In turn they, service users, carers and the organisations in which they work should gain the capacity to learn about and understand both problems and ways in which they can be solved – it should lead to new learning capacity.

A social work student, working in a Social Services Department team in Wales, describes the work he undertook with a man and his partner. Gwynn Davies was in hospital at the time the work was referred.

Mr Davies is a 52-year-old man who is registered disabled. He has chronic mobility problems arising from bronchial problems, and needs annual hospitalisation to clear his chest of infection. His first marriage ended in divorce and he has been living with Mrs Watkins and her 14year-old daughter, Stephanie, in her house for the past three and a half years. They live in a South Wales valley town.

Mrs Watkins visited the office in a distressed state, to tell the duty social worker that she was unable to accept Mr Davies back into her home because he repeatedly exposed himself to Stephanie. Mr Davies was interviewed on the ward. The first interview was difficult. He appeared to deny or evade the issues raised by his partner, and found it difficult to accept that Mrs Watkins would not let him return. He made it clear he had no intention of searching for alternative accommodation. He seemed more interested in his weight-lifting prowess, and at one stage removed his shirt to show his body and asked the social worker what age he thought he was.

Subsequent progress over the following days was slow, and goal setting was difficult. Mrs Watkins continued to visit him on the ward and appeared to act as if relations between them were free from problems. The student discussed with Mrs Watkins the mixed messages she was giving to Mr Davies. He and the practice teacher spoke to the child protection team, conveying their serious concern about Stephanie’s welfare.

Hospital staff regarded the student’s concerns as misplaced and wanted Mr Davies home as soon as possible. This eventually led to communication difficulties between the hospital and the student.

Reluctantly accepting that the relationship had broken down, at least temporarily, Mr Davies was provided with a list of private landlords, and the local authority was notified of his threatened homelessness. A local housing association visited him in hospital. At this period Mr Davies required constant encouragement to undertake tasks for himself, e.g. using the telephone.

The local council offered him a flat, which he refused on the grounds that he could not manage the steps. Medical evidence was obtained, but the Council was not willing to accept that this provided sufficient grounds to make an alternative offer. He then accepted a bedsit letting, and said he intended to keep in touch with the housing association, but was not happy with the outcome of the social work that had been provided.

In asking whether this was good practice, the general question is, did the practitioner embed the three practice principles of evidence, justice and understanding in his work? But we should be more specific. If the concern is about evidence, examples of questions we may have in mind include:

1. Did the student have a clear proposition regarding the problem and its solution, which could be tested by the eventual outcome?
2. Was any intervention clearly specified and capable of being identified?
3. Do we know if the outcomes were the result of the student’s practice?
4. What would count as evidence of success or otherwise for each of the professionals interested in this case?

If justice issues are in view, questions would include:

1. Did the social worker take seriously the risk of sexual or physical harm to the women involved?
2. Did the intervention give proper recognition to the power differentials, e.g. between those involved in professional relationships, or on the basis of gender?
3. Were the participants able to give feedback on whether their experience of social work had been helpful?
4. Was the service user enabled to collaborate with the social worker in judging whether the intervention had been helpful?

Understanding and learning raise different questions:

1. Did the student explore his own personal knowledge of the problem gained through his own background and life experience?
2. Were there opportunities for those involved to reflect on the process of intervention as well as on the problems?
3. Were ‘lay’ explanations of the problem (i.e. those explanations put forward by Stephanie, Mrs Watkins and Gwynn Davies) treated with equal seriousness to the explanations put forward by professionals?
4. Was evaluation participatory?

How these questions are answered is of course a matter of judgement. But to suggest possible answers to just the third principle, we might conclude that Mr Davies appears to have reached a gradual, unspoken conclusion that it would not be possible for him to resume his relationship with Mrs Watkins and Stephanie where it was left off. This suggests that the work done by the student had an impact on what initially seemed an intractable position. Other aspects were not dealt with clearly. For example, there is no apparent reflection on the student’s personal, taken-for-granted knowledge. Neither was there any participatory evaluation. The recognition of lay explanations was difficult, and recognised to be so by the student. There were sharply divergent views between the partners, and the student was concerned that the views conveyed by Mrs Watkins may not represent the whole truth about her relationship with Mr Davies.

'Social Sadism'

Evaluation and research do not enjoy a good press among social workers. It was only a slight exaggeration to conclude that ‘At best practitioners experience research as irrelevant; at worst as the process of being ripped off’ (Everitt et al., 1992: 5). A staff member of a Home Office crime prevention project believed she was speaking for everyone when she said, ‘No-one likes evaluation – it makes us feel childlike.’ This deeply felt mistrust, allied to the disempowering effects of work done by outside ‘experts’, lends credence to a conviction among social...

Table of contents

• Cover
• Half Title
• Dedication
• Title
• Copyright
• Contents
• List of Figures
• Preface to the Second Edition
• Acknowledgements
• 1 Keeping Social Work Honest
• 2 Walking the Borders: Practice and Research
• 3 Practitioners Talk
• 4 A Frame for Evaluating in Practice
• 5 Interlude
• 6 New Agenda, New Methods: Evaluating Assessments and Plans
• 7 Social Work in Action: Evaluating the Process of Practice
• 8 The End Game: Evaluating Outcomes
• 9 Developing Evaluating in Practice
• Bibliography
• Index