Private Troubles or Public Issues?

Challenges for Social Work Research

Walter Lorenz, Ian Shaw, Walter Lorenz, Ian Shaw

• Publisher: Routledge
• Year: 2018
• Print ISBN:
  9780367143251
• eBook ISBN:
  9781351800914
• Topic: Social Sciences
• Subtopic: Sociology
• Index: Social Sciences

This book bears testimony to the value of a progressive form of academisation of social work education in most European countries, including former communist countries which had to re-establish social work education. It also manifests the confidence of contributors in belonging to a serious academic discipline, and the fruitfulness of bringing research 'home' from neighbouring disciplines such as sociology, psychology, social policy, or pedagogy into the mainstream of social work.

The contributions to this book converge on a small number of core issues for contemporary social work. These are methodologically the conceptualisation of different and interacting dimensions of diversity, and practically the defence of professionalism and discretion against encroachment by neo-liberal ideologies and cost-cutting regulations. In so doing, this underscores that theory matters in social work. Authentic social work research can demonstrate that social work practice has no reason to shy away from basing itself on evidence and being professionally accountable as long as its notion of evidence recognises and does justice to the complexity of social problems and acknowledges the value of inter-subjectivity in producing useable and ethically grounded evidence. This book was originally published as a special issue of the European Journal of Social Work.

Reconsidering the ‘idea’ of evidence in evidence-based policy and practice

Neuüberdenkung des “Konzepts” Evidenz in der evidenzbasierten Politik & Praxis

Edward J. Mullen

School of Social Work, Columbia University, New York, NY, USA

Evidence-based policy and practice (EBP) has become an important social work conceptual framework. Yet, the core EBP concept, the concept of evidence, remains ill-defined. I propose a modification of the concept of evidence as applied to EBP effectiveness questions. As a basis for this reformulation ideas about evidence are examined from cross-disciplinary and multidisciplinary perspectives including epistemology, philosophy of science, evidence-science, and law. I propose that for EBP effectiveness questions: (1) to be considered ‘relevant evidence’ an explanatory connection between an intervention and an outcome must be established rather than a mere association; (2) the EBP definition of ‘best available evidence’ should include total available evidence (rather than a subset) about effectiveness, causal roles (i.e., mechanisms), and support factors and be inclusive of high-quality experimental and observational studies as well as high-quality mechanistic reasoning; (3) the familiar five-step EBP process should be expanded to include formulation of warranted, evidence-based arguments and that evidence appraisal be guided by three high level criteria of relevance, credibility, and strength rather than rigid evidence hierarchies; (4) comparative effectiveness research strategies, especially pragmatic controlled studies, hold promise for providing relevant and actionable evidence needed for policy and practice decision-making and successful implementation.

Die evidenzbasierte Politik und Praxis (EBP) bildet in der Sozialarbeit einen wichtigen Begriffsrahmen. Der Kernbegriff der EBP, der Begriff Evidenz, ist jedoch immer noch ungenau definiert. Ich schlage eine Modifizierung des in EBP-Effektivitätsfragen benutzten Begriffs Evidenz vor. Als Grundlage dieser Neuformulierung dient eine Untersuchung der Evidenz-Konzepte aus bereichsübergreifender und interdisziplinärer Perspektive unter Einschluss der Epistemologie, Wissenschaftsphilosophie, Evidenzwissenschaft und der Gesetzgebung. Ich schlage folgendes für EBP-Effektivitätsfragen vor: (1) als ‘relevante Evidenz’ sollte gelten, wenn eine selbsterklärende Verbindung zwischen einer Intervention und einem Ergebnis existiert und nicht nur einfache Assoziation; (2) die EBP-Definition der ‘besten verfügbaren Evidenzen’ sollte die gesamten verfügbaren Evidenzen (keine Teilmenge) über Effektivität, kausale Rollen (d. h. Mechanismen) und Unterstützungsfaktoren beinhalten sowie erstklassige Experimentier- und Beobachtungsstudien und anspruchsvolle mechanistische Begründungen; (3) der bekannte fünfstufige EBP-Prozess sollte um die Formulierung einer begründeten, evidenzbasierten Argumentation erweitert werden, und die Beweiswürdigung sollte sich durch drei strenge Kriterien steuern lassen: Relevanz, Glaubwürdigkeit und Stärke anstatt feststehender Evidenzhierarchien; (4) Forschungs-strategien zur vergleichenden Bewertung der Effektivität, besonders pragmatische kontrollierte Studien, sind vielversprechend bei der Schaffung von relevanten und umsetzbaren Evidenzen, die für Entscheidungsfindung und erfolgreichen Einsatz in Politik und Praxis erforderlich sind.

For those with lingering doubts about the nature of evidence itself I remind them that while Gregor Mendel (1822–84) developed the monogenic theory of inheritance on the basis of experimentation, Charles Darwin (1809–82) conceived the theory of evolution as a result of close observation, and Albert Einstein’s (1879–1955) special theory of relativity was a mathematical description of certain aspects of the world around us. William Harvey’s discovery of the circulation of the blood—was based on an elegant synthesis of all three forms of evidence. (Rawlins, 2008, p. 2159)

Introduction

Evidence-based policy and practice (EBP) is an important conceptual framework across the human services. Yet the core EBP concept, the concept of evidence, remains ill-defined and controversial. I propose a reformulation of the concept of evidence as applied to EBP effectiveness questions. Effectiveness questions ask about the outcomes of interventions in contexts which are typical of where policies, programs, and services will actually be provided, in the complexity of service organizations, as distinct from efficacy questions which ask about outcomes in controlled, experimental contexts. As a basis for this reformulation ideas about evidence are examined from cross-disciplinary and multidisciplinary perspectives. I propose a modification of the EBP framework and draw out implications for researchers conducting effectiveness studies, for those conducting systematic reviews of such studies, and for those developing guidelines.

This topic is important and timely for two reasons: First, in a number of countries the research agenda and funding are being driven by EBP requirements. Regulatory and funding agencies are requiring policies and practices be evidence-informed. Second, human service professionals are being trained as evidence-based practitioners. Accordingly, research organizations and researchers are expected to produce and translate research findings into evidence that can support decisions about which policies, programs, and services to implement. Nevertheless, there is disagreement about what should be the meaning and role of evidence in the EBP context.

What is EBP?

EBP is a policy and practice decision-making process with two complementary components, namely (1) the process of evidence-base practice and (2) the use of evidence-based, research-tested effective practices.

The evidence-based practice process, shown on the left side of Figure 1, is used by practitioners to make decisions about which evidence-based, research-tested effective practices to use, shown on the right side.¹ The idea of evidence is central to both the process of evidence-based practice and to the identification of evidence-based, research-tested effective practices. The left oval, labeled ‘evidence-based practice’ is a process practitioners can use when providing services to individuals, communities, organizations, or, at the policy level, whole populations.

Figure 1. EBP as a policy and practice decision-making process (reproduced with permission from Institute for the Advancement of Social Work Research, 2007).

Source: http://www.socialworkpolicy.org/documents/EvidenceBasedPracticeFinal.pdf

The EBP process involves a critical assessment of available evidence about each of the domains shown in Figure 2.

With decision-making at the center, consideration is to be given to the best available evidence: (1) about benefits, harms, and costs of alternative interventions; (2) client system characteristics, needs, state, preferences, and values; (3) resources required and available including practitioner expertise and experience as well as team and organizational resources; and (4) relevant environmental and organizational variables. For informed decision-making then, evidence is required about variables in each of these information domains. In this paper I only address the idea of evidence pertaining to the top circle, namely best available research evidence and even more specifically only to evidence about effectiveness questions.²

In EBP the emphasis is on the use of best evidence wherein best evidence typically refers to highest quality evidence (i.e., evidential credibility and trustworthiness). For both efficacy and effectiveness questions in EBP it is traditional to define evidence quality using a hierarchical scheme. Filtered evidence coming from systematic evidence reviews and critically appraised topical reviews are taken as providing the highest quality of evidence, especially those whose inclusion criteria limit eligible studies to randomized controlled trials, preferably those using placebo controls and double-blinded protocols (RCTs). Below these sources unfiltered evidence comes in descending order of quality from individual RCTs, cohort studies, case-controlled studies, and lastly from background information (e.g., such as from mechanistic reasoning about causes) and, at the bottom of the evidence hierarchy is expert opinion (Straus, Richardson, Glasziou, & Haynes, 2005). There are evidence hierarchies for other types of EBP questions but my concern is only with effectiveness questions.

Figure 2. Transdisciplinary model of evidence-based policy and practice (reproduced with permission from Satterfield et al., 2009).

This particular hierarchical view of evidence quality is viewed by many as too restrictive, especially the privileging of RCTs over other sources of evidence for addressing effectiveness questions (as distinct from efficacy questions where RCTs are widely acknowledged as providing the highest quality of evidence; Upshur & Tracy, 2004).

It is generally accepted that evidence quality does not guarantee evidential relevance, that is the applicability of the evidence to outcomes of interventions in contexts which are typical of where policies, programs, and services will actually be provided in the complexity of service organizations (Cartwright & Munro, 2010). Recently, evidential relevance has become an important concern for those working in implementation science where much progress is being made (Brownson, Colditz, & Proctor, 2012; Palinkas & Soydan, 2012).

After examining ideas about evidence from a cross-disciplinary and multidisciplinary perspective I will conclude with a recommendation that for effectiveness questions the notion of best evidence should be broadened and leveled (i.e., nonhierarchical) and the criterion of relevance should become a primary consideration alongside the criterion of evidential quality. I will also conclude that as is the case in disciplines such as law and philosophy evidential relevance can only be established through carefully crafted rational, evidential arguments (Cartwright, 2013).

Evidence

Evidence-based practice can be considered a rational process for making decisions in the face of uncertainties, that is, in situations wherein certainty is not attainable. This process involves making uncertain inferences, usually using qualitative probabilistic reasoning about hypotheses based on available evidence. As documented by Franklin in his detailed historical survey of methods of evidence evaluation or of how mankind has used rational methods for dealing with uncertainty, what he calls the science of conjecture, such methods have been used since ancient times in law, science, commerce, philosophy, and logic (Franklin, 2001, p. xiii).

A cross-disciplinary science of evidence

Recently there has been much interest in exploring commonalities across the disciplines regarding concepts, principles, and methods of evidence and inference. That interest has been peaked by the emphasis across disciplines in the idea of EBP. One such program, Evidence, inference and enquiry: Towards an integrated science of evidence was established at University College London (UCL) with participants from education, economics, forensic science, health sciences, history, law, philosophy of science, statistics, and psychology (Dawid, Twining, & Vasilaki, 2012). Twining notes that although a wide range of diverse projects within an array of disciplines were represented in the UCL evidence project commonalities could be found as to how evidence, and inferences from evidence, could be viewed: ‘Notwithstanding differences … all of our projects involve, as part of the enterprise, drawing inferences from evidence to test hypotheses and justify conclusions and that the logic of this kind of inquiry is governed by the same principles’ (Twining, 2012, p. 94).

Twining identifies a number of frameworks which could guide development of a common view of evidence. Of particular relevance to EBP conceptualizations, because of its focus on relevance, credibility, and the synthesis or combination of recurrent evidence sources is the classification developed by David Schum (2012).

Schum proposes a substance blind science of evidence which has at its core a scheme for classifying evidence in terms of inferential properties (Schum, 2012; Wigmore, 1937). Schum proposes two classificatory dimensions that describe all such forms...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• Citation Information
• Notes on Contributors
• Introduction: Private troubles or public issues? Challenges for social work research
• 1 Reconsidering the ‘idea’ of evidence in evidence-based policy and practice
• 2 Science and social work: a sketch
• 3 Reaching the person—social work research as professional responsibility
• 4 Standing up to complexity: researching moral panics in social work
• 5 Social work education in a time of national crisis in Greece: educating the workforce to combat inequalities
• 6 Attitudes toward poverty among exit students of undergraduate social work programs in eight Latin American countries
• 7 The circle of social reform: the relationship social work—social policy in Addams and Richmond
• 8 The street-level delivery of activation policies: constraints and possibilities for a practice of citizenship
• 9 Active social policies revisited by social workers
• 10 Investigating the quality of social work. An experience of self-assessment with Italian social workers
• 11 Towards an interactional approach to reflective practice in social work
• 12 Critical factors of intensive family work connected with positive outcomes for child welfare clients
• 13 Migrant voices addressing social work: listening to Italian women in Germany
• 14 Culturally sensitive social work: promoting cultural competence
• 15 Education, ethnicity and gender. Educational biographies of ‘Roma and Sinti’ women in Germany
• 16 Social assistance trajectories in Switzerland: do they follow discernible patterns?
• 17 Standardisation—the end of professional discretion?
• Index