Social Science Concepts and Measurement

New and Completely Revised Edition

Gary Goertz

• Publisher: Princeton University Press
• Year: 2020
• Print ISBN: 9780691205489, 9780691205465
• eBook ISBN: 9780691205472
• Topic: Social Sciences
• Subtopic: Politics
• Index: Social Sciences

A fully revised edition of the classic reference on concepts and their role in social science research

Social Science Concepts and Measurement offers an updated look at the theory and methodology of concepts for the social sciences. Emphasizing that most concepts are multilevel and multidimensional, this revised edition continues to bring the qualitative and quantitative closer together, with new chapters devoted to scaling, aggregation, and the methodological links between the semantics of concepts and numeric measures. In addition, it stresses that concepts are used for description and causal inference, and contain normative judgments.

Initial chapters focus on conceptualization, followed by chapters on issues of measurement. The textbook examines concepts in the international arena (such as the global performance indicators used by international organizations like the UN and the World Bank), as well as classic paired concepts such as poverty and wealth, democracy and authoritarianism, and war and peace. Additionally, it explores such topics as typologies, hybrid concepts, and how complex concepts constitute complex theories. The volume serves as a guide to the methodology of concepts in the classroom and is accompanied by more than two hundred exercises. Social Science Concepts and Measurement is an indispensable resource for graduate students and scholars.

1

Introduction

AN ONTOLOGICAL-SEMANTIC APPROACH TO CONCEPTUALIZATION AND MEASUREMENT

To define a thing, is to select from among the whole of its properties those which shall be understood to be designated and declared by its name; the properties must be very well known to us before we can be competent to determine which of them are fittest to be chosen for this purpose.

The essence of a thing … is that without which the thing could neither be, nor be conceived to be.

Every proposition consists of two names [concepts]: and every proposition affirms or denies one of these names, of the other…. Here, therefore, we find a new reason why the signification of names, and the relation generally, between names and the things signified by them, must occupy the preliminary stage of the inquiry we are engaged in.

J. S. MILL

JOHN STUART MILL began his System of logic with a “book” devoted to concepts. Starting with concepts was a logical choice since they are key building blocks for constructing theoretical propositions. Propositional logic involves the proper manipulation of symbols. For this to have usefulness in science, these symbols need substantive and empirical content.

Concepts are measured en route to statistical and empirical evaluations of theoretical propositions. So concepts must be quantified and measured in many social science applications. However, as Lazarsfeld and Barton state, one must conceptualize before moving to empirical analyses:

Before we can investigate the presence or absence of some attribute … or before we can rank objects or measure them in terms of some variable, we must form the concept of that variable. (Lazarsfeld and Barton 1951, 155; emphasis is mine)

Philosophers, lawyers, political and social theorists debate normative concepts such as democracy, justice, and human rights. Concepts are fundamental to description. For anthropology, ethnography, grounded theory, and similar methodologies, developing concepts is a core theoretical and empirical activity. Concepts are thus core in causal theories, normative philosophy, and empirical description.

This book provides a unified framework for working with, constructing, and evaluating concepts that applies in these different domains. These domains often overlap. To use an example dear to my heart, which will appear as an ongoing example, “peace” is normative, descriptive, and a domain of causal theories and empirical testing of those theories (Goertz, Diehl, and Balas 2016).

Concepts are fundamentally about meaning, semantics, and ontology. Thus a methodology of concepts must to a significant degree be about semantics and ontology (which is a theory of being):

Concepts are an answer to “what is” questions.

To ask what democracy is, or what poverty is, etc., is to inquire about ontology and definition. Concepts are about meaning and semantics.

As such, this volume uses terminology rarely seen in the literature on indicators, measurement, etc. Concepts are about definitions, semantics, ontology, meaning, and the like. Downstream we often want to have numeric, quantitative measures of these concepts—the so-called indicators. J. S. Mill started with “names” in his System of logic, not indicators of names.

It should be stressed that this ontology can vary and be contested on various grounds. For example, gender analyses of concepts contest the bias of their traditional ontologies, be it democracy, the welfare state, etc. This is closely related to normative issues that underlie many concepts. The concept and data for transitional justice are inherently normative. The ontology might vary depending on purpose and use. Finally, I’m personally working on the concept of civil or intrastate negative peace (Goertz et al. 2019), which has different versions depending on whether I’m focusing more on it as a dependent variable or as an independent and causal variable. This becomes quite evident in the guidelines regarding independent and dependent variables at the end of chapter 3.

The basic framework, described in detail in the next chapter, provides a methodology for analyzing, critiquing, and creating complex concepts (which might be given quantitative expression). Complex concepts are multidimensional and multilevel. Multidimensional appears semantically in definitions that include multiple attributes and features. They appear in data sets with multiple coding rules. Multidimensional is typically quite obvious; for example, in the next chapter see the analysis of the Multidimensional Poverty Index (MPI; Alkire et al. 2015).

The multilevel character of concepts in contrast has received almost no explicit attention. As discussed in the next chapter, the logic varies as one moves up and down the different levels. The logic of definitions is one of completeness and nonredundancy. Going back to Aristotle—and in philosophy in general—a good definition gives the set of necessary and jointly sufficient conditions for a concept. The logic of the data level is usually multiple indicators of one defining dimension. This is a logic of redundancy with no real need for completeness. One wants many—or enough—indicators of a one-dimensional concept: indicators are good when they are redundant.

This implies that the mathematics of the levels tends to differ significantly, which then means that final quantitative expression of the complex concept should typically incorporate different logics and the appropriate mathematics when generating the final numeric value.

In spite of the primordial importance of concepts, they have received relatively little methodological attention. At the same time there is a booming industry in complex indicator and index construction, such as the Multidimensional Poverty Index. International institutions such as the World Bank, the United Nations, OECD, and the EU—not to forget many prominent NGOs—generate hundreds of complex indicators. There has been a surge of books on indicators, such as Merry (2016) and Laurent (2018). Kelley and Simmons have an ongoing project examining dozens of “Global Performance Indicators” (2015); see also the Broome et al. project, e.g., 2018, for another long list of global indicators, as well as Weaver’s global indices project.¹ All of these indicators are complex multilevel and multidimensional concepts.

An equally long list, and perhaps an even bigger industry, involves measures of physical and mental health or, more accurately, measures of illness, sickness, and disability. For example, I will refer on occasion to the DSM manual, which is the bible for clinical psychology and psychiatry (American Psychiatric Association, various years). It consists of a set of concepts, aka mental illnesses, along with sets of symptoms or indicators of the illnesses. Governments generate and use for crucial policy decisions many health indicators. For children there is the US Department of Health and Human Services Children’s Bureau’s child well-being measure, UNICEF’s State of the World’s Children measure, and Stirling Children’s Wellbeing Scale, among others (see Alexandrova 2017 and Hausman 2015 for nice discussions of health concepts and scales). These are all very complex—multidimensional and multilevel—indices.

The terms “index” and “indicator” come from the idea of pointing, for example with one’s index finger. These are all indicators of something. This book focuses on that “something,” which is a concept. One core feature of the basic framework is that conceptually it works top down. It asks for definitions which then connect downward to empirical indicators and data. Quantitative measures work from the bottom up: one starts with the data–indicators and then moves up to the top level. There is no real separation between conceptualization and measurement: they are fused.

In the appendix to the next chapter I contrast this ontological-semantic approach to conceptualization and measurement with latent variable models. These present a radically different approach, focused much more on indicators and measurement and much less on ontology and semantics. Latent variables are cause indicators. The ontological–semantic approach does not have causal relationships within the basic framework. The core criteria of completeness and nonredundancy for concepts make no sense in the latent variable framework. In the basic framework the focus is first and above all on the definitions and concept structure and then secondarily on the empirical indicators; for latent variables the focus is on “measurement models.”

More generally, all data sets rest on concepts. It is hard to imagine the data being good if the underlying conceptualization is problematic, as is the case for example with terrorism. It is not uncommon for there to be disconnects between conceptualization and measurement, as we will see for the Polity democracy concept-measure. In short, this volume provides a methodology for the analysis of data sets of all sorts. The basic framework sees “coding rules” as conceptualizations with a certain aggregation structure. To understand data sets one needs a semantic and conceptual interpretation of the coding rules. “Coding rules”—always in the plural—implies multidimensional concepts.

Mill in the epigraph starts with “to define” and he continues with “its properties”: this is about semantics, meaning, and ontology. He follows with “a thing” that is something in the real world, identifying and locating those “things” in the world. He then moves to “propositions,” which are causal claims about the world. This volume focuses on the central role that concepts play in description and causal hypotheses as well as in normative analyses.

The Conceptual Juggling Act

My notion would be, that anything which possesses any sort of power to affect another, or to be affected by another, if only for a single moment, however trifling the cause and however slight the effect, has real existence; and I hold that the definition of being is simply power.

PLATO

A property carries its [causal] capacities with it, from situation to situation.

NANCY CARTWRIGHT

The failure to explain is caused by a failure to describe.

BENOÎT MANDELBROT

Developing valid concepts for social science involves the juggling of multiple conceptual balls. I use the juggling metaphor because typically the focus is on one or two of the balls and the others are left to fall to the ground. Some people focus on one or two balls in their research, while others focus on other balls. This volume argues that one needs to keep all of the balls in the air in one’s mind when thinking about concepts, and eventually downstream for measurement.

Figure 1.1 illustrates the nature of the juggling problem. All these aspects of concept development and analysis will appear prominently in the chapters to come. Given the emphasis on measurement in many contexts, it is worth noting that these balls typically do not appear in research design or measurement books. In contrast, the first three chapters of this volume focus in particular on these conceptual balls and it is only in later chapters that measurement appears. This does not mean that measurement will not be implicitly, or sometimes explicitly, present in the discussion, because one of the key issues is linking conceptualization with measurement.

In the middle of the figure lies the semantic ball. A core question is what does one mean by a concept? This means that conceptualization is about definitions. For example, in some areas, such as political and moral philosophy, the majority of the analysis is about the definition of some key political concept.

FIGURE 1.1. Juggling conceptual balls.

If one focuses exclusively on definitions, one risks getting a completely nominal view of concepts and their ontology. Hence it is absolutely critical to keep the three balls to the left of semantics in the air when discussing semantics. How does one determine what one should put into the semantics of concepts? As the arrows in figure 1.1 illustrate, there are three important factors or issues that should determine the semantic content of concepts in social science.

At the core of the methodology of concepts is the connection between the semantics of concepts and real-world phenomena. One dimension of this is the degree to which concepts and measures correctly capture and describe the world. For example, the degree to which Central American countries were democracies in the early 20th century is very debatable (Bowman et al. 2005). These debates combine the various possible conceptualizations of democracy with an application of those concepts to the reality of these countries in that time period. In general these are both conceptual and empirical questions.

Hence one conceptual ball is descriptive validity or what might be ...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Contents
• List of Tables
• List of Figures
• Preface
• 1. Introduction: An Ontological-Semantic Approach to Conceptualization and Measurement
• 2. The Basic Framework for Conceptualization and Measurement
• 3. Guidelines for Creating Concepts: Ontology, Semantics, and Definitions
• 4. Mapping Meaning: Linking Concepts with Data–Indicators
• 5. Scaling
• 6. Concept Structure: Aggregation and Substitutability
• 7. Gray Zone and Hybrid Concepts
• 8. Typologies
• 9. Intension–Extension: Concept Structure and Empirical Description
• 10. Two-Level Theories: Complex Concepts in Complex Causal Theories
• References
• Index