Historical and Conceptual Foundations of Measurement in the Human Sciences

Credos and Controversies

Derek C. Briggs

• Publisher: Routledge
• Year: 2021
• ISBN:
  9781000465815
• Edition: 1
• Topic: Education
• Subtopic: Education General

Historical and Conceptual Foundations of Measurement in the Human Sciences explores the assessment and measurement of nonphysical attributes that define human beings: abilities, personalities, attitudes, dispositions, and values.

The proposition that human attributes are measurable remains controversial, as do the ideas and innovations of the six historical figures—Gustav Fechner, Francis Galton, Alfred Binet, Charles Spearman, Louis Thurstone, and S. S. Stevens—at the heart of this book. Across 10 rich, elaborative chapters, readers are introduced to the origins of educational and psychological scaling, mental testing, classical test theory, factor analysis, and diagnostic classification and to controversies spanning the quantity objection, the role of measurement in promoting eugenics, theories of intelligence, the measurement of attitudes, and beyond.

Graduate students, researchers, and professionals in educational measurement and psychometrics will emerge with a deeper appreciation for both the challenges and the affordances of measurement in quantitative research.

1

WHAT IS MEASUREMENT?

DOI: 10.1201/9780429275326-1

1.1 Keating’s War and Thorndike’s Credo

I have always loved the movie Dead Poets Society, but upon watching it again for the first time in many years, a particular scene stood out to me as providing a proper motivation for this book. The movie, which is set in 1959, tells the story of Mr. Keating (played by Robin Williams), a newly hired English teacher at a private preparatory school for boys and the events that are set in motion as he inspires his students to challenge conformity. An early and memorable scene comes when Mr. Keating asks one of his students to read an introduction to an anthology of 19th-century poetry. The passage is titled “Understanding Poetry” and has been written by (a fictitious) Dr. J. Edwards Prichard, PhD. The following passage is read aloud in its entirety:

To fully understand poetry, we must first be fluent with its meter, rhyme and figures of speech, then ask two questions: (1) How artfully has the objective of the poem been rendered? and (2) How important is that objective? Question 1 rates the poem’s perfection; question 2 rates its importance. And once these questions have been answered, determining the poem’s greatness becomes a relatively simple matter. If the poem’s score for perfection is plotted on the horizontal of a graph and its importance is plotted on the vertical, then calculating the total area of the poem yields the measure of its greatness. A sonnet by Byron might score high on the vertical but only average on the horizontal. A Shakespearean sonnet, on the other hand, would score high both horizontally and vertically, yielding a massive total area, thereby revealing the poem to be truly great. As you proceed through the poetry in this book, practice this rating method. As your ability to evaluate poems in this matter grows, so will your enjoyment and understanding of poetry.

While this is being read, Mr. Keating has plotted out two “greatness” areas for the Byron and Shakespeare sonnets on an illustrative graph, along with the equation G = A * I, where A represents a rating of the “artfulness” with which a poem’s objective has been rendered, I represents a rating of the “importance” of the poem’s objective, and the product of the two, G, represents the measure of a poem’s greatness. He turns back to the class with the equation and graph in place and, the passage reading complete, offers a concise evaluation: Excrement! He demands that the students rip out the entire Prichard introduction so that only the poetry itself is left, and as this is happening, he explains:

We’re not laying pipe, we’re talking about poetry. How can you describe poetry like [rating a song on the show] American Bandstand? … This is a battle, a war. And the casualties can be your heart and soul. Armies of academics going forward measuring poetry. No, we will not have that here. In my class you will learn to think for yourselves again, you will learn to savor words and language. No matter what anybody tells you, words and ideas can change the world.

It’s an inspiring scene. It concludes with Mr. Keating reading the Walt Whitman poem “Oh Me! Oh Life!” to the students, the final lines of which propose an answer to the meaning of life “that the powerful play goes on and you may contribute a verse.” With the students gathered around him and listening with rapt attention, he asks, “What will your verse be?”

While we can hardly take issue with Mr. Keating’s entreaty to seize the day, let’s turn our attention back to the implied villain of this scene, J. Edward Prichard, and the “army of academics” measuring poetry that he is intended to personify. Although the movie is a work of fiction, for the time period in which it was set, there was, in fact, an emerging army of academics, mostly from the emerging field of psychology, who were leading the charge to measure all sorts of educational products and psychological attributes that seem as equally resistant to quantification as poetry. One famous justification for this charge can be traced to the “Credo” articulated by the American educational psychologist Edward Lee Thorndike (1918):

Whatever exists at all exists in some amount. To know it thoroughly involves knowing its quantity as well as its quality. Education is concerned with changes in human beings; a change is a difference between two conditions; each of these conditions is known to us only by the products produced by it—things made, words spoken, acts performed, and the like. To measure any of these products means to define its amount in some way so that competent persons will know how large it is, better than they would without measurement. To measure a product well means so to define its amount that competent persons will know how large it is, with some precision, and that this knowledge may be conveniently recorded and used. This is the general Credo of those who, in the last decade, have been busy trying to extend and improve measurements of educational products.

(16)

Thorndike’s Credo does contain some pearls of wisdom. One purpose of education is, in fact, to bring about some change in what a person knows or can do. If Mr. Keating is teaching his students the craft of writing poetry over some period, there must be some collection of knowledge and skills related to the reading of poetry he is hoping that they will develop. Might there not be some value in identifying these as learning objectives so that he can track the extent to which his students are mastering them? And might not measurement play a role in all this? Indeed, perhaps it is not something about the poems we should be trying to measure but something about the students themselves!

But maybe we need to be posing a different question altogether. That is, in addition to arguing over whether we should measure the “greatness” of a poem (or the “ability” of a student to analyze poetry), we should also be asking why we are willing to believe that we can measure such things. Thorndike’s Credo emerged from a surrounding context of early 20th-century positivism that privileged quantity over quality, with the implication that if something exists, it can be measured, but if it cannot be measured, then it does not really exist. This belief, which Joel Michell has called the “quantitative imperative,” would indeed seem to place Dr. Prichard in direct opposition to Mr. Keating, who would surely point out that things like beauty, romance, and love all exist but that they are multifaceted. When we experience them, we experience them qualitatively, as matters of degree. These are things that might not be measurable. Even if we did decide to follow the procedure described in Prichard’s passage—to give a poem’s artfulness and importance numeric ratings and then multiply the two together—why and in what sense does it follow that this act of numeric assignment constitutes a legitimate instance of measurement? What is measurement, after all?

1.2 What Is (and What Is Not) Measurement?

1.2.1 Four Definitions of Measurement

If you are reading this book, you are probably interested in the answer to this question. You might also have some preexisting ideas about the topic. As it turns out, a consensus definition that goes beyond the circular understanding of measurement tantamount to “the act or process of measuring” can be elusive. Consider the following four single-sentence definitions, each of which differs from the other in subtle (and sometimes not-so-subtle) ways:

1. Measurement is the numerical quantification of the attributes of an object or event, which can be used to compare with other objects or events. [Conventional]
2. Measurement is the discovery or estimation of the ratio of a magnitude of a quantity to a unit of the same quantity. [Classical]
3. Measurement is the assignment of numerals to objects or events according to rules. [Psychological]
4. Measurement is the process of experimentally obtaining one or more quantity values that can reasonably be attributed to a quantity. [Metrological]

The first of these definitions, the “conventional” definition, is what can be found by conducting the laziest of internet searches (I typed the word measurement into my internet web browser). The top website that this returned was the internet encyclopedia Wikipedia, and this was the opening sentence of the definition to be found there in December 2020 (“Measurement,” n.d.). The citation associated with this sentence comes from a 1991 textbook, Measurement, Design, and Analysis: An Integrated Approach by Pedhazur, Schmelkin, and Pedhazur. Interestingly, the second sentence for the entry qualifies the opening one by noting that “[t]he scope and application of measurement are dependent on the context and discipline.”

If the first definition captures some sense of the conventional wisdom of the crowds as jointly curated by Google and Wikipedia, then the next two represent definitions from specific and influential scholarship. The second “classical” definition is taken from the book Measurement in Psychology: A Critical History of a Methodological Concept by Joel Michell. The third “psychological” definition comes from two very influential publications by the psychologist Stanley Smith Stevens in the mid-20th century (Stevens, 1946, 1951). The fourth “metrological” definition has been taken from the third edition of the International Vocabulary of Measurement (VIM) and represents a consensus definition from the field of metrology¹ (JCGM, 2012).

A commonality across these definitions is that measurement seems to have something to do with quantity (conventional, classical, and metrological definitions) and numbers or numerals (conventional and psychological definitions). But beyond this, if we spend some time scrutinizing each definition, they tend to raise as many questions about measurement as they answer. All but the psychological definition seems to place a focus on quantity or quantification. Magnitudes and units figure prominently in the classical definition; experimentation and attribution, in the metrological definition. The use of measurement for the purpose of comparison is a distinguishing feature of the conventional definition. The psychological definition stands out as the broadest conceptualization of measurement as an activity that need only involve the assignment of numerals according to rules (although perhaps it strikes some readers as unusual to refer to numerals instead of numbers). It also differs from the conventional definition in that measurement is specified as something that applies to objects and events, as opposed to an attribute of an object or event. The most important thing to appreciate at this point is the difficulty in finding any adequate single-sentence definition of measurement. In particular, although each of these definitions provides some hints about the sorts of activities that are ruled in as measurement, they do little to settle what gets ruled out.

1.2.2 Measurement Terminology

Attributes and Objects

At least some of this confusion can be addressed through clarity about terminology. In this book, I generally distinguish between objects²—things that exist and can be directly observed at some specific location and moment in space and time—and the attributes of objects. An attribute is a characteristic of an object. The distinction between object and attribute is important because the same object can have many different attributes. Minerals can be characterized by their size, color, hardness, and aesthetic appeal; poems by their use of metaphor, sentence complexity, and emotions elicited from readers; people by their intelligence, kindness, and curiosity. There are two other terms I could just as well have used in place of attribute that could be given an equivalent interpretation: property or quality. That is, we could just as easily refer to size, color, hardness, and aesthetic appeal as four distinct “properties” or “qualities” of minerals. I refer to an “attribute” instead of a “property” for the relatively trivial reason that property has a connotation that is often specific to inanimate objects in the physical sciences. I use attribute instead of quality because, as we will see, some attributes of objects either are, or can at least be conceptualized as if they were quantitative, and it seems more sensible to refer to a “quantitative attribute” than it does to refer to a “quantitative quality.”³

Classical Conception of Quantity, Magnitude, and Units

What defines a quantity? Aristotle (384–382 BCE) distinguished between two kinds of quantities, those that were discrete and those that were continuous. An example of a discrete quantity, which Aristotle referred to as a multitude, comes from the act of counting or aggregating individual objects that are of the same kind. Because numbers to the ancient Greeks were always whole numbers, a distinguishing feature of a multitude was that it was numerable and composed of discrete units. In contrast, a continuous quantity was known by its magnitude. Euclid (323–283 BCE) defined a magnitude as “a part of a magnitude, the lesser of the greater, when it measures the greater” and “the greater is a multiple of the less when it ...