Developing Norm-Referenced Standardized Tests

Lucy Jane Miller

• Publisher: Routledge
• Year: 2020
• ISBN:
  9781317940821
• Edition: 1
• Topic: Medicine
• Subtopic: Medical Theory, Practice & Reference

Transform your ideas and data into norm-referenced standardized tests with this "how-to" manual. Edited by the author of the Miller Assessment for Preschoolers (MAP), a nationally standardized, norm-referenced test, Developing Norm-Referenced Standardized Tests is designedspecifically for occupational and physical therapists who have an interest in conducting research, either with established scientists or independently in order to pursue questions of interest. This unique volume leads the reader through the process of test development step-by-step, including identification of a concept that should be subjected to testing, development of appropriate test items, and the procedures for standarizing a norm-referenced test. Not only will professionals learn to develop new tests, but they will also increase their understanding of the process of test development for instruments which are already available.

Chapter 1
Measurement in Developmental Therapy: Past, Present, and Future

Suzann K. Campbell

Suzann K. Campbell, PT, PhD, FAPTA, is Professor of Physical Therapy at the College of Associated Health Professions at the University of Illinois at Chicago, 1919 W. Taylor St., Chicago, IL 60612.

While writing this paper, the author was partially supported by Grant MCJ 9101 from the Bureau of Health Care Delivery and Assistance, USPHS.

People seldom learn from the mistakes of others—not because they deny the value of the past, but because they are faced with new problems.

—Ilya Ehrenburg
“What I Have Learned”
Saturday Review, September 30, 1967

INTRODUCTION

Physical and occupational therapists recently have generated a flurry of activity aimed at development of new standardized assessment tools for use in pediatric clinical and educational settings. For example, the Miller Assessment for Preschoolers,¹ the Movement Assessment of Infants,² and the Hughes Gross Motor Assessment³ are therapist-designed test that have become available in the past decade. Nevertheless, the subdisciplines of pediatric occupational and physical therapy have not yet embraced wholeheartedly the concept that therapists routinely should use and develop their own standardized assessments. Nor have professionals in these fields put those tests that are available to work in improving clinical diagnostic capabilities and accountability to clients and the public for the costly treatment provided.

The purpose of this introductory chapter is to describe the history of standardized testing, the current problem of lack of testing instruments, and the rationale for increased use of standardized tests in developmental therapy settings. The need for pediatric occupational and physical therapists to develop more scientifically validated tools is also addressed.

HISTORY OF STANDARDIZED TESTING

Critical to the future development of occupational and physical therapy as scientific and clinical disciplines is the generation of measurement tools specific to client needs. Many clinicians are depending upon tests developed by psychologists and other professionals with different types of clients or goals in mind. Worse, a trend of the 1970s in some areas of the subdisciplines was to avoid formal testing.

According to Nunnally, the movement toward standardized testing was generated in the last century by Darwin’s theory of evolution.⁴ The concept of “survival of the fittest” led to interest in measuring individual differences in abilities of various types. Galton, the founder of the eugenics movement aimed at improving the human race, studied the heritability of human traits using tests of sensory discrimination. Sensory acuity was believed to be the basis of intellect. Interest was so great that people were willing to pay for the opportunity to be measured with Galton’s techniques! His contributions were important for their explicit recognition of (1) the need for standardization in testing—the concept that each individual should be tested with the same items under the same conditions and with standard instructions, (2) for emphasizing the importance of individual differences in abilities, and (3) for the development of correlational methods, later refined by Pearson, with which to analyze the collected data.

Binet and Simon developed the first test of global intelligence in 1905 at the behest of the French government which had recognized the need for developing a testing tool to evaluate and classify children who were too mentally deficient to profit from schooling.⁴ This work led to the concept of norms for performance of children at different ages, calculation of mental ages, and eventually the search for factors in human intelligence. Psychologists, such as Spearman and Thurstone, theorized that intelligence included a general factor, g, and specific factors, such as verbal, numerical, spatial, memory, reasoning, analogy, and perceptual abilities. They developed factor analysis as a methodological approach to studying human cognitive abilities. Piaget further advanced the study of cognitive performance by carefully studying the processes, rather than the content, of mental development. This produced a revolutionary and continuing impact on developmental psychology.

The standardized testing movement spread widely to include assessment of most areas of human ability, and made significant contributions in practical application to personnel selection, school admissions, and psychiatric and other medical diagnostic tasks.⁴ Rare is the person who has never taken a standardized test of some kind before reaching adolescence. Although standardized tests are continually criticized for labeling and lack of cultural validity for some groups in the population, such as minorities, they remain the best known means of sorting, classifying, diagnosing, and measuring progress.

Researchers have also studied the factors contributing to motor development, to motor learning, and to skilled motor performance. Developmental therapists have been interested in studying the relationships between motor milestones, developmental reflexes and reactions, and “quality” of movement. Though interested, little advancement has been made in the measurement of motor dysfunction or progress during therapy. Undoubtedly, progress has been slow in part because of the tremendous complexity involved in sorting out the many factors, both sensory and motor, that contribute to skilled motor performance. “Quality” of movement is difficult to capture and describe because it does not consist of a single factor, but rather is a jargon term inclusive of coordination, postural control, and balance.⁵

Research on motor development has resulted in the identification of nine different aspects of gross and fine motor development in normal children from preschool age to adolescence.⁶ Specific factors implicated in gross motor abilities were speed, static and dynamic body balance, coordination, and strength. For fine motor performance, the identified factors were visual-motor tracking, response speed to a visual stimulus, visual-motor control of the hand, and upper extremity speed and precision in manipulation. Taken together, these factors were believed to include the elements of speed, precision, strength, balance, and coordination.

These factors unquestionably are important in motor performance of children with developmental disabilities as well. The abnormal sensorimotor system, however, presents a more complex measurement problem. For example, researchers have not yet been able to identify and isolate the specific factors operating on motor performance in the various types of central nervous system (CNS) dys-function. A review of the literature on motor control deficits in children and adults with cerebral palsy (CP), however, suggests that alterations in limb stiffness, reflex gain, sensory receptive fields, and movement synergies are candidates as factors in addition to those involved in normal motor performance.^5,7–¹³ Finally, research in motor learning suggests that likely factors important in both normal and pathologic movement include repetition, knowledge of results, motor memory, perceptual factors, specificity of exercise relative to movement goals, and the environmental context.¹⁴

Occupational and physical therapists historically have been slow to develop and standardize tests for use in studying motor development, control, and learning, and for diagnosing problems or measuring progress in clients. As an example, in the area of motor development in children with cerebral palsy, several tests appeared in the 1950s and 1960s. They were based primarily on the Gesell motor milestones, but added special tests of speed or ability to perform activities involving rhythmic reversal of direction.^15,16 These tests were aimed at quantifying motor performance in children with CP in a way that would produce developmental quotients or motor ages. Several important papers also were published which emphasized the importance of assessing and documenting motor development in quantitative ways that would reliably capture the progress of handicapped children.^17,18 Nonetheless, a statement made in 1951 that “We are unable to describe the patient with cerebral palsy in any way which lends itself to a statistical analysis”^15,p.698 remains true today.

The next decade produced almost no work on measurement of motor performance in CP other than gait analyses for assessment of outcome of orthopedic procedures^13,19–²⁰ and a few tests of specific functions. The 1980s saw the introduction of several new research tools for the study of pathokinesiology^7,21–²³ and introduction of the Movement Assessment of Infants,² a promising new tool still under development, but limited to neurologic assessment in infancy.

Professionals in occupational therapy have accomplished more than those in physical therapy in developing tools for assessing children with mild neurologic dysfunction, such as the Sensory Integration and Praxis Test²⁴ and the Miller Assessment for Preschoolers.¹ Yet much remains to be done.

LACK OF TESTING INSTRUMENTS

Important questions arise when examining the current lack of testing instruments. Why are there no universally agreed upon assessment batteries for each type of pediatric developmental disability? Why are formal tests necessary to supplement the clinician’s observational skills? What are the consequences of this lack of objective measurement tools?

No Universally Agreed Upon Assessment Batteries

Six main reasons surface in response to the issue of no universally agreed upon assessment batteries. First, cerebral palsy and other developmental disabilities are complex and those children afflicted are difficult to study scientifically. Second, the professions of occupational and physical therapy are service-oriented. Third, many academics are isolated from the clinical setting, and therefore, from joint research with clinicians. Fourth, historically faculty members have had high teaching loads, a lack of research training and experience, and a low level of psychometric knowledge. Fifth, funding for large scale research and development is difficult to obtain in fields which do not have a history of excellence in this area. And finally, much of the neurodevelopmental (NDT) and Rood clinical establishment embody anti-quantification, anti-scientific characteristics.

Clinical leaders in the area of neurodevelopment have contributed significantly to developing the theory and to teaching the art of clinical practice while advancing scientific clinical practice hardly at all. Although the sensory integration community has made significant contributions toward establishing valid and reliable measurement practices, much more remains to be accomplished. In their thirst for better tools, clinicians have either used ill-suited tests from other disciplines, such as psychology, or even more frequently uniformly rejected other available tests because of their weaknesses in accomplishing what they were never intended to do. Instead, many clinicians have created their own so-called “tests” of highly questionable reliability and validity.

Necessity of Formal Tests to Supplement Observational Skills

A long list of references can be found in the literature to document the notorious unreliability of the clinician’s diagnostic capabilities. Sackett and colleagues described a number of studies documenting the serious consequences of clinical misjudgment.²⁵ In one study of 93 adolescents who previously had been labeled as having organic heart disease or rheumatic fever, reevaluation revealed that 81 percent of the children studied actually had normal hearts.²⁶ Of special interest to occupational and physical therapists is that the misdiagnosed children experienced restriction of physical and social activity as much as the children with true heart disease; 30 (40 percent) of the 75 children with no current heart disease were restricted in some way...