Measurement, Statistics, and Research Design in Physical Education and Exercise Science: Current Issues and Trends

A Special Issue of Measurement in Physical Education and Exercise Science

Terry M. Wood

• Publisher: Psychology Press
• Year: 2014
• eBook ISBN: 9781317759010
• Topic: Psychology
• Subtopic: History & Theory in Psychology
• Index: Psychology

This inaugural issue is devoted to exploring measurement, research design, and statistics issues in six subdisciplines of exercise and sport science. Originally presented at the Eighth Measurement and Evaluation Symposium, all papers in this issue reflect the work of many renowned measurement specialists and content experts in their respective fields. The articles discuss the following topics:
* standards of assessment quality for physical educators and the problem of providing adequate assessment without adequate resources;
* the importance of properly conceptualizing and defining appropriate research questions as the "source and solution" for measurement and design issues in reference to motor learning/control and sport and exercise psychology;
* the study of individuals -- single-subject and other small-sample designs -- in contrast to the more traditional study of groups; and
* the importance of computing and reporting statistical power in research.

Dealing With the Practical Matter of Quality Performance Assessment

Richard J. Stiggins

Assessment Training Institute, Portland, Oregon

Recently, as I departed from an elementary school after presenting an after-school workshop on assessment, Jason—one of the participating teachers—called to me from across the parking lot to raise a follow-up issue. He apologized for delaying my return home but said that it was an issue he just couldn’t raise during the session. Jay faced a troubling dilemma and, frankly, didn’t want the others to know that he didn’t know how to handle it—especially his supervisor, who also participated in the workshop. Here is Jay’s description of the problem:

I am the physical education [PE] teacher for three different elementary schools. I spend part of each day in each school. My job is to take over for the regular classroom teacher for PE instruction. Each teacher delivers her or his charges to me for 40 minutes, I engage them in the activities I have planned for the period, and then the students return to their regular classroom. Every class period everyday, I face a new group of students. So every week, I see several hundred students—none for more than a few minutes.

Now here’s my dilemma: At report card time every 10 weeks, I am required to enter a grade for each student. The district provides me with a list of names of my students, and I am supposed to evaluate each of them. They expect me to differentiate between and among them and assign grades as appropriate. But as I read down page after page of names, I don’t even know three quarters of these kids by name! And even if I did, there is no way I could take time to assess their physical education achievement one by one—accurately or inaccurately. There are just too many of them and too little time. So, I complained to my supervisor, who advised me, just grade them on their effort. But if I don’t know who they are, how can I do even that? No one could get to know and assess this number of students. This is a joke—a game we play to try to make kids behave, and parents believe we have standards in PE. What should I do?

Jay and his physical education colleagues are not the only teachers facing this dilemma. Music, art, and other service teachers are forced to deal with this same set of circumstances regarding report-card grading. How should they address their assessment and reporting responsibilities?

As it turns out, this question has concrete answers, although some will find the answers unsatisfying. Possible solutions become more and more clear the higher one’s level of assessment literacy—that is, the more informed one is about the differences between sound and unsound assessment and grading practices. These circumstances stymie Jason because they present him with a series of immovable barriers to quality assessment and effective communication. Removal of those barriers will take specific action related to grading policy and practice. The path to resolution, however, will remain a mystery to those who are unschooled in the basic principles of sound classroom assessment.

This article spells out a set of commonsense standards of assessment quality that, if met in any assessment context, will maximize the accuracy of the information we generate about student achievement. Jay’s circumstances precluded him from meeting these standards. Let’s see why.

FIVE STANDARDS OF ASSESSMENT QUALITY

Here I briefly describe the keys to sound assessment. Then I explore how each plays out in Jason’s classroom.

Standard 1: Quality assessments arise from and accurately reflect clearly specified and appropriate achievement expectations for students. As educators, we must begin the assessment process with a sharply focused vision of the achievement we expect of our students, because different achievement targets require the application of different assessment methods. In any assessment context, we must begin assessment development with a clear vision of what it means for our students to succeed. Do we expect our students to:

1.Master (i.e., know and understand) a subject? (Does this mean that they must know it outright, or does it mean that they must know where and how to find it using references?)

2.Use knowledge to reason and solve problems?

3.Demonstrate mastery of specific performance skills—where it is the doing that is important?

4.Use their knowledge, reasoning, and performance skills to create products that meet standards of quality?

Because there is no single assessment method capable of assessing all of these various forms of achievement, one cannot select a proper method without a focused sense of which of these expectations are to be assessed. Jay would be prepared to assess for grading if he had determined in advance which of the four kinds of achievement just listed he expects his students to attain.

Standard 2: Sound assessments are designed to serve prespecified instructional purposes. We cannot design assessments without asking who will use the results and how they will use them. Because different users need different information in different forms at different times to fulfill their decisionmaking responsibilities, there is no single assessment that can meet everyone’s needs.

Table 1 lists the important users of assessment in schools. Each user needs different information to answer different questions. To provide quality information for teacher, student, and parent use at the classroom level, we need high-quality classroom assessments; to provide useful information for policy or instructional support, we need quality standardized tests. Because of the differences in information needs, we must begin each assessment event with a clear sense of whose needs we are meeting. Otherwise, our assessments are without purpose.

In Jay’s case, the publicly stated purpose for his assessments is to assign report-card grades. This means that he strives to communicate to students and parents how each student is doing in meeting the academic standards in physical education class. To make Jay’s report-card grades interpretable, then, both he (the message sender) and his students and their parents (both message receivers) need to know and understand what forms of achievement are to be factored into the grade symbols used (A, B, C, etc.).

Standard 3: High-quality assessments use methods that accurately reflect the intended target and serve the intended purpose. As teachers often have several different kinds of achievement to assess, and as no single assessment method can reflect them all, we must rely on a variety of methods. The options include selected-response tests (multiple choice, true–false, matching, fill-in), essay tests, performance assessments (based on observation and judgment), and assessments based on direct personal communication with the student (oral examinations, interviews, discussions, etc.). Our assessment challenge is to know how to match the assessment method with an intended achievement target, as depicted in Table 2. Our professional development challenge is to be sure that all concerned with quality assessment know and understand how the various pieces of this puzzle fit together.

TABLE...

Table of contents

• Cover
• Title Page
• Editor's Note
• Guest Editor's Note
• Dealing With the Practical Matter of Quality Performance Assessment
• Concerns and Issues in Studying and Assessing Motor Learning
• Measurement, Statistics, and Research Design Issues in Sport and Exercise Psychology
• Experimental and Statistical Design Issues in Human Movement Research
• Using Old Research Ideas to Study Contemporary Problems in Adapted Physical Activity
• Estimating Sample Size in Repeated-Measures Analysis of Variance
• Copyright Page
• Contributor Information