Principles of Assessment and Outcome Measurement for Occupational Therapists and Physiotherapists

Theory, Skills and Application

Alison Laver Fawcett

• Publisher: Wiley
• Year: 2013
• Print ISBN: 9781861564801
• eBook ISBN: 9781118709696
• Edition: 1
• Topic: Medicine
• Subtopic: Occupational Therapy

This textbook on assessment and outcome measurement is written for both occupational therapy and physiotherapy students and qualified therapists. It begins by defining what is meant by assessment, outcome, evaluation and measurement and discussing the complexity of therapy assessment and measurement, including the challenge of measuring human behaviour and the impact of factors such as task demand and context, including the environment. Methods of data collection (e.g. observation, interview, standardised testing) and sources (e.g. self-report, proxy) for collecting information about clients are then reviewed, and the main purposes of assessment (e.g. descriptive, evaluative, predictive, discriminative) presented. The book then addresses the topics of standardisation, levels of measurement, reliability, validity and clinical utility. There is a chapter describing and applying models for categorizing levels of function to aid assessment and measurement. The concept of clinical reasoning and reflective practice is then explored. Application of principles is supported through detailed case studies and worksheets and the criteria for test critique and guidelines for choosing a particular assessment approach are discussed.

1

The Importance of Accurate Assessment and Outcome Measurement

CHAPTER SUMMARY

This chapter focuses on the requirement of therapists to undertake thorough and accurate assessment and measurement. The chapter will describe some developments and policy directions in health and social care practice that have affected occupational therapy and physiotherapy assessment, including:

• a demand for evidence-based practice
• a shift towards the use of standardised assessments
• a requirement to measure outcomes and demonstrate effectiveness
• a focus on client-centred practice
• a demand for robust clinical governance and clinical audit activities
• the use of standards, care pathways, protocols and guidelines.

It also examines the impact of such developments on physiotherapy and occupational therapy assessment, for example the emphasis on demonstrating that intervention is effective leads to a need for reliable, valid and sensitive outcome measures that enable therapists to measure clinically relevant change. In light of a call for standardised measurement, the chapter will discuss some of the advantages and limitations of standardised versus non-standardised tests. This introductory chapter will also explore the complexity of assessment, including the challenges of measuring human behaviour and the impact of the environment, and reflect upon how such complexities influence what can be measured by therapists and the adequacy of these measurements. The chapter concludes by presenting a series of questions about assessment and measurement, which will then be addressed in detail in the following chapters.

ASSESSMENT AS A CORE PART OF THE THERAPY PROCESS

Assessment was defined in the Introduction as the overall process of selecting and using multiple data-collection tools and various sources of information to inform decisions required for guiding therapeutic intervention during the whole therapy process. Assessment involves interpreting information collected to make clinical decisions related to the needs of the person and the appropriateness and nature of their therapy. Assessment involves the evaluation of the outcomes of therapeutic interventions.

Assessment is a core component of health care and therapy processes. It is recognised by health care professionals that assessment is an essential part of a quality service, for example the Royal College of Physicians (RCP; 2002) states that ‘assessment is central to the management of any disability’. Assessment is embedded as an essential component of the health care process. The health care process can be simply described as the (Austin and Clark, 1993):

• needs analysis of the client
• identification of what service needs to be provided
• identification of the provider of the service
• provision of the service
• evaluation of the service provided.

Assessment is the first step in the health care process and provides the foundation for effective treatment. Assessment occurs again at the end of the health care process in the form of evaluation. It is also necessary to undertake a re-assessment at several stages during stage four of the process, service provision, because without thorough and accurate assessment the intervention selected may prove inappropriate and/or ineffective.

THE IMPACT OF HEALTH AND SOCIAL CARE POLICY ON ASSESSMENT PRACTICE

The organisational and policy context for health and social care has been under frequent change and reform, particularly over the last decade. In recent years, the provision of health and social care has been exposed to a more market-orientated approach in which government fund-holders and organisations who purchase therapy services have become more concerned about value for money and require assurances that the service provided is both clinically effective and cost-effective. The demand for cost-effective health care is forcing rehabilitation professionals to be able to prove the efficacy and efficiency of their interventions. In the current policy context that focuses on quality, national standards, best value and evidence-based practice (EBP), the ability to demonstrate service outcomes has become increasingly important; for example, the Department of Health (DoH; 1998a) states that the modernisation of care ‘moves the focus away from who provides the care, and places it firmly on the quality of services experienced by, and the outcomes achieved for, individuals and their carers and families’ (paragraph 1.7).

An emphasis on clinical governance means that therapists are more overtly responsible for the quality of their practice, and this is reflected in an increased interest in EBP. Sheelagh Richards, Secretary of the College of Occupational Therapists (COT), states:

Now critical appraisal, reflective practice, systematic audit, peer review, best value review, service evaluation, clinical governance and a host of other methodologies are accepted parts of the professional’s landscape. The need to deliver evidence-based practice is well understood and all professionals have to play their part in the ‘total quality management’ of service delivery. (Richards, 2002, p. xvii)

The Chartered Society of Physiotherapy (CSP; 2001a) also highlights these changes to its members and recognises that therapists, and all health care practitioners, are being put under increasing ‘pressure to demonstrate the added value of the service they provide’ (p. 2). The CSP appreciates that the clinical governance agenda has led to an increased demand for results and proven outcomes and that this helps to inform required service improvements. In order to meet these demands, physiotherapists are being encouraged to learn about measurement and to adopt appropriate outcome measures in their daily practice. This has been made explicit through the introduction of the use of outcome measures into the CSP’s revised standards of practice (Chartered Society of Physiotherapy, 2000), and this is helping to raise the profile of outcome measurement within the physiotherapy profession. For occupational therapists, the COT on its website states:

Every individual providing an occupational therapy service has a responsibility to maintain and improve effectiveness and efficiency through the use of outcomes measures and audit. Occupational therapists should employ a range of quality activities including: evidence-based practice, adherence to national and professional standards and guidelines, risk-management, continuing professional development and listening to the views of those who use the service. (http://www.cot.org.uk/members/profpractice/quality/intro.php, accessed 4.12.05)

In a paper on the use of standardised assessments by physiotherapists, Stokes and O’Neill (1999) state that ‘clinical effectiveness, evidence-based practice, outcome measures and clinical audit are the “buzz words” of today’s researcher and practitioner. They are the markers of an aspiration for accountability, productivity and objectivity within the provision of health care’ (p. 560). This continues to be true today.

Therapists need to be aware of the reasons that drive their practice. It is only reasonable to be influenced by financial and political drivers when the resultant change in practice yields true benefits for clients. Unsworth (2000) notes: ‘current pressures to document outcomes and demonstrate the efficacy of occupational therapy intervention arise from fiscal restraints as much as from the humanitarian desire to provide the best quality health care to consumers. However, measuring outcomes is important in facilitating mutual goal setting, increasing the focus of therapy on the client, monitoring client progress, as well as demonstrating that therapy is valuable’ (p. 147).

THE DEMAND FOR EVIDENCE-BASED PRACTICE

The World Confederation of Physical Therapy (WCPT), which was founded in 1951 to represent physical therapists internationally, ‘champions the principle that every individual is entitled to the highest possible standard of culturally-appropriate health care provided in an atmosphere of trust and respect for human dignity and underpinned by sound clinical reasoning and scientific evidence’ (World Confederation for Physical Therapy, 2006a). In its description of physical therapy the WCPT lists ‘principles’ supporting the description of physical therapy, and these include emphasising ‘the need for practice to be evidence based whenever possible’ (http://www.fisionline.org/WCPT.html#Iniziale2, accessed 27.10.05). The CSP, in the effective practice section of its website, begins by telling therapists that ‘whatever your occupational role – clinical physiotherapist, assistant, manager, researcher, educator or student – you need to use the best available evidence to inform your practice’ (http://www.csp.org.uk/director/effectivepractice.cfm, accessed 27.11.05). While the COT states that ‘occupational therapists should be delivering effective practice that is evidence-based where possible’ (College of Occupational Therapists, 2005c, p. 1).

SO WHAT IS EVIDENCE-BASED MEDICINE?

Therapists should explicitly be working towards achieving EBP in all areas of their practice. EBP has developed from work on evidence-based medicine (EBM), and expands the concept of EBM to apply across all health care professionals. EBM has been defined as:

the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research. (Sackett et al., 1996, p. 71)

Sackett and his colleagues further describe individual clinical expertise as the ‘proficiency and judgment that individual clinicians acquire through clinical experience and clinical practice’ (p. 71). They state that a clinician’s increasing expertise can be demonstrated in a number of ways ‘especially in more effective and efficient diagnosis and in the more thoughtful identification and compassionate use of individual patients’ predicaments, rights, and preferences in making clinical decisions about their care’ (p. 71). Belsey and Snell (2003) have written a useful fact sheet, What is evidence-based medicine?, which can be downloaded as a free pdf file from the EBM website at: www.evidence-based-medicine.co.uk/ebmfiles/whatisebm.pdf (accessed, 15.12.05). Belsey and Snell describe EBM as a ‘multifaceted process of assuring clinical effectiveness’ (p. 1) and describe four main elements:

• ‘Production of evidence through research and scientific review.
• Production and dissemination of evidence-based clinical guidelines.
• Implementation of evidence-based, cost-effective practice through education and management of change.
• Evaluation of compliance with agreed practice guidance and patient outcomes – this process includes clinical audit.’

SO WHAT IS EVIDENCE-BASED PRACTICE?

The College of Occupational Therapists Research and Development Group has defined EBP as the explicit use of the best evidence of clinical and cost-effectiveness when working with a particular client. It combines clinical reasoning, existing research and client choice (Research and Development Group, 1997). ‘Evidence-based practice encourages the integration of high quality quantitative and qualitative research, with the clinician’s clinical expertise and the client’s background, preferences and values. It involves the client in making informed decisions and should build on, not replace, clinical judgement and experience’ (OTseeker, 2005).

To identify the best available external clinical evidence, clinicians need to seek clinically relevant research, and therapists should particularly seek client-centred clinical research into the accuracy and precision of standardised tests and the efficacy of therapeutic interventions. When new evidence is acknowledged, it sometimes can invalidate previously accepted tests and treatments, and therapists are beholden to replace old unsubstantiated practices with evidence-based practices that are more effective, more accurate, more efficacious and safer (Sackett et al., 1996).

The WCPT provides a two-page overview on EBP for physiotherapists (World Confederation for Physical Therapy, 2003). This document equally applies to occupational therapists and is a good starting point. Like EBM, EBP is achieved through the integration of three factors, which are:

• the best available research
• clinical experience
• client’s beliefs and values.

This means that EBP ‘requires a combination of art and science’ (p. 2). The WCPT describes the rationale for EBP and asserts that undertaking EBP helps therapists to:

• ‘improve the care of patients, carers and communities
• reduce variations in practice
• use evidence from high quality research to inform practice, balancing known benefits and risks
• challenge views based on beliefs rather than evidence
• make decision making more transparent
• integrate patient preferences into decision-making
• ensure that knowledge continues to inform practice through life-long learning’ (World Confederation for Physical Therapy, 2003, p. 1).

IMPLEMENTATION OF EVIDENCE-BASED PRACTICE

In busy clinical settings, implementing EBP may be difficult. There are many potential barriers to the full implementation of EBP, including lack of time, lack of access to literature and lack of skills in finding and interpreting research. Some of the strategies that have been suggested (OTseeker, 2005) for supporting EBP in clinical environments include:

• fostering a supportive environment in the workplace for EBP
• providing continuing education to develop skills in literature searching, critical appraisal and research methods
• collaborating/participating in research evaluating therapy interventions
• participating in or establishing a journal club
• focusing on reading research articles that have a rigorous study design or reviews that have been critically appraised
• seeking out evidence-based clinical guidelines.

In order to use evidence, it is necessary to undertake a number of tasks.

• Search for and locate the evidence related to a specific clinical question.
• Appraise the evidence collected.
• Store and retrieve the evidence when required.
• Ensure the body of evidence used to inform clinical decisions is kept updated.
• Communicate the findings from the evidence and use these findings in clinical practice. (Belsey and Snell, 2003)

SO HOW DO YOU TRACK DOWN THE BEST EVIDENCE?

The COT has published a guide on finding and using evidence bases (Mountain and Lepley, 1998) that provides a useful starting point for therapists.

THE COCHRANE LIBRARY

In terms of databases, a good place to start is the Cochrane Library, which provides a collection of separate databases. Five of these provide coverage of EBM, and the other two provide information on research methodology. The databases are:

• the Cochrane Database of Systematic Reviews
• the Database of Abstracts of Reviews of Effectiveness (DARE)
• the Cochrane Controlled Trials Register
• the Cochrane Database of Methodology Reviews
• the Cochrane Methodology Register
• the Health Technology Assessment Database
• the NHS Economic Evaluation Database.

DARE includes structured abstracts of systematic reviews that have been critically appraised by reviewers at the NHS Centre for Reviews and Dissemination in York and by other people, for example from the American College of Physicians’ Journal Club and the journal Evidence-Based Medicine.

THE SCOTTISH INTERCOLLEGIATE GUIDELINES NETWORK (SIGN)

The SIGN has published over 80 clinical guidelines, some of which are of relevance to occupational therapists and/or physiotherapists. These can be accessed at: http://www.sign.ac.uk/ (accessed 10.12.05). For example, the guideline for the prevention and management of hip fracture on older people, ‘Section 9: rehabilitation and discharge’ states:

9.1 Early assessment: Early assessment by medical and nursing staff, physiotherapist and occupational therapist to formulate appropriate preliminary rehabilitation plans has been shown to facilitate rehabilitation and discharge. Evidence level 2+. (http://www.sign.ac.uk/guidelines/fulltext/56/section9.html, accessed 10.12.05)

The role of the physiotherapist and occupational therapist is also indicated in Section 9.22: ‘multidisciplinary rehabilitation’, which states:

Multidisciplinary team working is generally considered to be effective in the delivery of hip fracture rehabilitation. The professions, grades and interrelationships of members of the ‘multidisciplinary team’ vary between studies and, because these characteristics are rarely described in detail, the effectiveness of different approaches to team working is not yet well understood. Rehabilitation should be commenced early to promote independent mobility and function. The initial emphasis should be on walking and activities of daily living (ADL) e.g. transferring, washing, dressing, and toileting. Balance and gait are essential components of mobility and are useful predictors in the assessment of functional independence. Evidence level 2++. (http://www.sign.ac.uk/guidelines/fulltext/56/section9.html, accessed 10.12.05)

OCCUPATIONAL THERAPY SYSTEMATIC EVALUATION OF EVIDENCE (OTseeker)

OTseeker is a database that contains abstracts of systematic reviews and randomised controlled trials relevant to occupational therapy. It provides therapists with easy access to trials from a wide range of sources. The trials included have been critically appraised and rated to assist therapists to evaluate their validity and interpretability. These ratings will help therapists to judge the quality and usefulness of trials for informing their clinical interventions (http://www.otseeker.com/, accessed 26.10.05).

PHYSIOTHERAPY EVIDENCE DATABASE (PEDro)

PEDro is an initiative of the Centre for Evidence-Based Physiotherapy (CEBP). It has been developed to give rapid access to bibliographic details and abstracts of randomised controlled trials, systematic reviews and evidence-based clinical practice guidelines in physiotherapy. Most trials on the database have been rated for quality to help therapists quickly discriminate between trials that are likely to be valid and interpretable and those that are not. The PEDro site has been supported by a number of organisations, including the Australian Physiotherapy Association, the School of Physiotherapy at the University of Sydney, the Cochrane Collaboration and New South Wales’ Department of Health. The site can be found at: http://www.pedro.fhs.usyd.edu.au/index.html (accessed 26.10.05). It also contains two useful tutorials:

• Part I: Are the findings of this trial likely to be valid?
• Part II: Is the therapy clinically useful?

Do not forget that a significant amount of therapy research still remains unpublished but may be accessible, for example the COT’s library holds a significant number of occupational therapy PhD and Master’s theses and offers a loan service. Carr (1999) examines this collection in her publication Thesis Collection: The National Collection of Unpublished Occupational Therapy Research.

LEVELS OF EVIDENCE AND GRADES ...

Table of contents

• Cover
• Contents
• Title page
• Copyright page
• Dedication
• Contributors
• Foreword
• Preface
• Acknowledgements
• Introduction
• 1: The Importance of Accurate Assessment and Outcome Measurement
• 2: Methods of Assessment and Sources of Assessment Data
• 3: Purposes of Assessment and Measurement
• 4: Levels of Measurement
• 5: Standardisation and Test Development
• 6: Validity and Clinical Utility
• 7: Reliability
• 8: Test Administration, Reporting and Recording
• 9: Applying Models of Function to Therapy Assessment and Measurement
• 10: The Importance of Clinical Reasoning and Reflective Practice in Effective Assessment
• 11: Implementing the Optimum Assessment and Measurement Approach
• 12: The Final Case Study ‘Carol’: Experience of a Chronic Pain Service
• Brief Answers to Study Questions
• Worksheet
• Glossary
• List of Abbreviations
• Reference
• Index