Communication Research Measures

A Sourcebook

Rebecca B. Rubin, Philip Palmgreen, Howard E. Sypher

• Publisher: Routledge
• Year: 2020
• ISBN:
  9781000149388
• Edition: 1
• Topic: Business
• Subtopic: Business Communication
• Index: Business

The development of communication as a discipline has resulted in an explosion of scales tapping various aspects of interpersonal, mass, organizational, and instructional communication.This sourcebook brings together scales that measure a variety of important communication constructs. The scales presented are drawn from areas of interpersonal, mass, organizational, and instructional communication--areas in which the use of formal, quantitative scales is particularly well developed.

Communication Research Measures reflects the recent important emphasis on developing and improving the measurement base of the communication discipline. It results in an equal amount of labor saved on the part of the scholars, students, and practitioners who find this book useful, and it contributes in a significant way to research efforts.
Originally published by Guilford Press in 1994, now available from Routledge.

Part I

_________________________

MEASUREMENT TRENDS AND ISSUES

Measures of Instructional Communication

Patricia Kearney
Michael J. Beatty

Even though the area of instructional communication is new to our discipline, in the last 10 to 15 years we have seen the development of numerous psychological scales, behavioral (skills) assessments, and qualitative coding schemes. Our tasks of identifying, describing, classifying, and evaluating the multitude of measures available have yielded a fairly exhaustive list of instructional communication measures commonly (and uncommonly) employed in this discipline. There are few measures reported in this chapter that we have not either used ourselves or evaluated based on others’ use.

Importantly, we made some criterial decisions up front in an effort to determine which instructional communication measures would receive intensive coverage (i.e., the profiles indexed in the latter part of this volume) and which would receive only cursory attention. Obviously, space limitations prevented us from showcasing them all. Our criteria were similar to those employed by all the authors in this book: sufficient reliability and validity, currency in the field, availability, and high or potentially high use. Moreover, some measures were redundant (in which case, we selected the “best” based on reliability and validity estimates). And, finally, some assessments did not lend themselves easily to our format: coding systems, teacher (or student) profiles, and some classification schemes.

The major criterion for including the remaining measures overviewed in this chapter was simple: They all had to be instructionally relevant. Thus, we began to sort through all the journals and edited texts of the past 15 years that regularly publish instructional communication research, including Communication Education, Communication Yearbook, and regional communication journals. Some of the measures discussed in this chapter overlap in emphasis with other areas of the discipline (e.g., most of the assessments of communication apprehension, nonverbal behavior, and decision rule use orientation). Nevertheless, we included them here as well in an effort to show students, teachers, and researchers both the constructs and the assessments that we normally use in instructional communication research. Moreover, we believe that most of those overlapping measures have instructional communication as their roots. What you will find in this chapter is a fairly extensive list of over 80 instructional communication measures frequently (or only occasionally) used in our discipline. We organized these measures according to the following major instructional themes: learning outcomes, teacher behaviors and characteristics, student behaviors and characteristics, communication skills assessment, and communication apprehension.

Note. Measures that are profiled in this volume (in Part II ) are typed in capitalletters when cited in the text or tables.

LEARNING OUTCOMES

Table 1 provides a variety of measures of students’ learning in the classroom. Instructional communication researchers have focused on both affective and cognitive learning outcomes. In the past, we argued that affect (or liking) should be a primary outcome of teacher communication variables, and, as expected, a number of studies substantiated that claim. Most of that research relied on J. F. Andersen’s Affective Learning measure, a scale we profile in this volume. Some practitioners have criticized the scale for problems associated with face validity; other researchers have attempted to strengthen the measure by adding or deleting particular items. Nevertheless, the original Andersen scale remains the most reliable, valid, and parsimonious measure of both lower- and higher-order affective learning. We encourage readers to examine our profile for a more intensive explanation of that conclusion.

TABLE 1. Learning Outcomes Measures

Affective learning Affective Learning Teacher Evaluation (affect toward teacher and class) (Norton & Nussbaum, 1980) Teacher Evaluation (affect toward teacher, course, and discipline) (McCroskey, Holdridge, & Toomb, 1974) Cognitive learning Free Recall (S. Booth-Butterfield, 1988a) Learning Loss (Richmond, McCroskey, Kearney, & Plax, 1987) Short-Term Information Acquisition (Andriate, 1982) Short-Term Recall (Beatty, Behnke, & Froelich, 1980; Beatty, Behnke, & Goodyear, 1979; Beatty & Payne, 1984; Kelley & Gorham, 1988) Self-Reported Cognitive Learning (Richmond, McCroskey, Kearney, & Plax, 1987) Grades Grade point average (Davis & Scott, 1978) Instructor-determined “judgment” grades and scores on objective exams (Harper & Hughey, 1986) Standardized course examinations/tests (Andersen, 1979; Elliot, 1979; Nussbaum, 1983; Nussbaum & Scott, 1980) Student self-reports (Richmond & McCroskey, 1984)

At the same time researchers were able to link relevant teacher communication variables with affective learning, they reasoned further that affect should be an important motivator of students’ cognitive learning. That is, when students like the subject matter (and often the teacher, too), they should also be more willing to learn, use, and generalize the information or skills beyond the traditional classroom. Unfortunately, efforts to test that relationship have been problematic, and no completely satisfactory solution has been obtained to date.

The fact remains that no completely valid means of measuring cognitive learning exists. Students themselves complain that examination scores, course grades, and grade point averages only partially reflect what they have learned. Moreover, we know that grades can be based on what students know before they enroll in a course, students’ prior history and attitudes toward the course content or teacher, arbitrary or irrelevant grading practices in the course (attendance), invalid and unreliable test items, subjective grading procedures, and so on. Even when standardized cognitive learning measures across multiple-section courses have been developed, we know that students are not all taught the same objectives, nor are they all taught the content in the same way. And, of course, by limiting ourselves to standardized courses within a specific area, we are unable to generalize the results across disparate content areas. (For a more in-depth explanation of these issues, see McCroskey & Richmond, 1992, pp.106–108.)

One of two alternatives remain in our attempts to assess cognitive learning reliably and validly. The first is commonly employed in the educational literature: short-term (and long-term) recall tests. The most recent example of this approach in instructional communication is Kelley and Gorham’s (1988) experimental study of teacher immediacy and student recall. Students were given four groups of six items (three unrelated words and three numbers) and then asked to recall each sequence. Whereas this procedure assessed short-term information acquisition and retrieval, we know that it is somewhat removed from what really occurs in the classroom. The second alternative is more subjective and depends on students’, as opposed to teachers’, assessments of learning. This second procedure relies on students’ reports of how much they have learned from a given teacher or course (Richmond, McCroskey, Kearney, & Plax, 1987).

To measure students’ reports of their own learning, students are asked two questions: “On a scale of 0–9, how much did you learn in this class, with 0 meaning you learned nothing and 9 meaning you learned more than in any other class you’ve had?” and “How much do you think you could have learned in the class had you had the ideal instructor?” (Richmond et al., 1987). By subtracting the response to the first question from the second, a learning loss score is obtained. This latter score is used to eliminate any possible bias resulting from students who might have been forced to take a class in a disliked content area. This procedure (a) allows us to look at learning beyond simple recall of infor...