A Conceptual Introduction To Modeling

Qualitative and Quantitative Perspectives

David W. Britt

• Publisher: Psychology Press
• Year: 2014
• Print ISBN: 9780805819373
• eBook ISBN: 9781317779414
• Topic: Psychology
• Subtopic: Research & Methodology in Psychology
• Index: Psychology

When seeking to test specific hypotheses in large data sets, social and behavioral scientists often construct models. Although useful in such situations, many phenomena of interest do not occur in large samples and do not lend themselves to precise measurement. In addition, a focus on hypothesis testing can constrict the potential use of models as organizing devices for emerging patterns -- summaries of what we believe we know about the dynamics of situation.

This book bridges the gap between "quantitative" and "qualitative" modelers to reconcile the need to impose rigor and to understand the influence of context. Although there are many different uses for models, there is also the realistic possibility of doing credible research without their use. A critical reexamination of the assumptions used in quantitatively-oriented models, however, suggests ways to increase their effectiveness as organizers of both quantitative and qualitative data.

Students of methods in psychology, sociology, education, management, social work, and public health -- and their instructors -- are increasingly expected to become familiar with both quantitative and qualitative approaches. Unfortunately, they find few vehicles for communication regarding the implications of overlapping work between the two approaches. Using models as organizing devices for a better dialogue between assumptions and data might facilitate this communication process.

1

First Steps, Basic Dilemmas, Gulfs and Bridge Building

Every cobbler thinks leather is the only thing. (Trow, 1957)

… a slice of bread without caviar on top is still a slice of bread. (Lieberson, 1992)

Remember what it was like when we played with something like a logo set when we were children? Such sets have lots of names now, but they all came with simple blocks and sticks that could be put together to simulate buildings, cars, bridges, gargoyles and other imagined structures. We decided which blocks to use, either by ourselves or with the other kids with whom we were playing, and what they represented. When we decided which blocks should be attached, we connected them with sticks or piled them up in different combinations. When we decided which blocks should not be connected, we left the space between them empty or left the ones we did not want to use in the box … or scattered on the floor. There was constant reality testing, initially among ourselves and occasionally with others. If what we were building fell over, looked ugly, met with guffaws and chuckles from passers-by, or somehow did not match the scenario of our play activities, we changed blocks, sticks, spaces, and even where we played with the set.

Building models is a lot like playing with a logo set. There is a lot of simplification as we construct models to represent images of various realities. Instead of blocks, we have concepts, but the struggle to tentatively decide which blocks to use and what they mean is parallel. Instead of sticks, we have arrows. The struggle to tentatively decide how things relate to one another is similarly parallel. We decide what the nature of the variables is according to what we or the others with whom we are working think. We decide which variables should be related to each other and which ones should not, again in consultation. We wrestle with questions of context—and we do a lot of reality testing. You may think this is a lot of tentative decisions—you are right; the reason for this is that in the social sciences, there is no last word.

Causal models are collections of variables and assumptions about what the variables are and how they relate to one another. They simplify realities and focus our attention on certain aspects of situations as these situations change through time and place. Models may help us describe how aspects of situations are related to one another. In other cases, models may help us predict how events will unfold. If they capture the essence of the causal dynamics of situations, models may help us gain insight into how people and other social entities live through sequences of events. In still other cases, models may help us understand why things happen the way they do by providing a vehicle for pitting alternative explanations against one another. This book is an introduction to the process of modeling, with an emphasis on how to think about models, what to do with them, and what makes them credible. A continuing theme is the investigation of how the modeling process might be altered to create a bridge for the organization of knowledge from quantitative and qualitative sources. Creating such a bridge should facilitate a dialogue between assumptions and data from all sources or, to follow up on one of the quotes at the beginning, creating such a bridge will help us make shoes out of things other than leather.

There are a few biases that I should lay out before moving further. I see models primarily as organizing devices for a continuing, explicit dialogue between multiple sources of data and assumptions. In this light, models summarize what we believe we have learned about the dynamics of phenomena in patterns woven from different contexts, in different historical periods and with different individuals and social groups.

I believe we need to establish the legitimacy of modeling underlying realities that are approximately defined by the conjunction of context, history, and social entities (individuals and social groups). This objective of modeling is in opposition to simply modeling available data points to provide the best fit. By itself, this would be a leap of some magnitude beyond standard practice. In the context of considering models as organizing devices for both quantitative and qualitative data, moving toward underlying realities instead of only fitting models to quantitatively derived data points is a shorter leap, but a necessary one if we are to allow models to be more flexible and useful for researchers with different styles. Such an approach requires relentless rethinking of the meaning of context, and action no matter what the scale of the analysis or the sources of the data. For example, Isaac and Griffin (1989) reframed the nature of historical periods from arbitrary containers of numerical information to substantively meaningful and potentially different contexts. Only by doing so is it possible to appreciate how the relationship between union strength and U.S. labor militancy changes from one historical period to another.

On a smaller scale, the same issues apply. Modeling the effectiveness of different teaching strategies would, at a minimum, require detailed and rich knowledge of not only strategies and student behavior and attitudes but also an understanding of the school and community in which the classroom is located, as well as the interpenetrating histories of all of these factors and how they mutually define one another. This may all sound very abstract, but imagine how naive it would be to assume that schools do not set limits and create conditions that can either facilitate or retard what is happening in classrooms. Similarly, is it not much more reasonable to assume that the same strategies might work for some students in some communities, but be useless in other contexts? To simplify a bit, establishing the legitimacy of modeling underlying realities means starting with the assumption that social life is lumpy and that combinations of context, history and social entities must be examined.

A final bias concerns the process of making models credible and trustworthy. Standard practice in quantitative research judges the credibility of models by the confidence we may have in the estimates of coefficients. I believe that the credibility or trustworthiness of models and their relationships must rest on more than the preciseness of coefficients estimated from the statistical manipulation of large data sets. Coefficients have no inherent meaning. At the very least, they require an additional causal mechanism “that puts the substantive process in context and ‘makes sense’ out of it (Isaac, Carlson, & Mathis, 1994).” Isaac et al’s (1994) plea is for greater richness and interpretive understanding about how social entities are dealing with the dynamics in which they find themselves, and for which they may have been in part responsible.

Much of this book is about credibly making sense of what is going on in situations of interest. After laying out a basic dilemma faced by researchers and developing a framework for doing mixed-method modeling, the next two chapters discuss the building blocks of concepts and relationships while keeping in focus the importance of spaces—the things we do not believe belong in concepts or things that are unrelated to one another. The three chapters that follow discuss various forms of elaboration, strategies for adding concepts and relationships to models in ways that embrace the complexities of what we are trying to understand. Chapters 7 and 8 discuss criteria for evaluating models and moving from elaborated to working models. A final chapter briefly reviews important lessons and discusses the limits of modeling.

EXAMINING ALTERNATIVE DEFINITIONS

Having stated these biases up front, consider some examples of how models have been defined over the last 25 years so as to put my approach in perspective. Richardson and Pugh (1981) gave a standard definition of modeling from a system dynamics perspective:

The term ‘model’ stands for a representation … of some slice of reality … The purpose is to gain understanding … [by] exposing the model’s assumptions about a problem to criticism, experimentation and reformulation. (pp. 2–3)

There are three key points contained in this definition: simplification, explicitness, and reformulation. Models are collections of explicit, specified assumptions—not vague understandings regarding how the world works. They simplify reality to bring the critical dynamics that shape that reality into better focus. The Richardson and Pugh definition makes clear that assumptions in models are always open to criticism and being reformulated. This is the dialogue.

The nature of the assumptions that are being made explicit in models does not stand out particularly well in the Richardson and Pugh (1981) definition. Consider the following definition by Uslaner (1976), drawn from the introduction to a popular statistically based book on modeling by Asher(1976):

Causal modeling is a technique for selecting those variables that are the potential [cause] of the effects—and … [isolating] the separate contributions to the effects made by each [cause]. (p. 5)

Carving the world up into sets of variables is a key simplifying assumption that cannot be taken lightly (Abbott, 1988). Modeling may attempt to come to grips with causal dynamics, represented in this definition as distinguishing between causes and effects and assessing the relative importance of causes in generating effects. Such an approach hinges on the plausibility of assuming that these variables are free to assume values independent of one another rather than occurring in clusters. Modeling may also be used in situations where such an assumption is not seen as being plausible. It might be more realistic, for example, to think in terms of combinations of conditions that come together in particular configurations to produce certain outcomes (Ragin, 1987). Models may help graphically capture the explicit assumptions about configurations of conditions in ways that supplement either the equations or the prose common in comparative analysis.

A more general definition that brings home another typical characteristic of models is the following by Land (1969):

… the term path model is used to refer to the set of structural equations … representing the postulated causal and noncausal [i.e., merely asso-ciational] relationships among the variables under consideration, (p. 7)

As this definition makes clear—and the other definitions of modeling suggest—models have almost invariably been tied to sets of equations applied to quantitative data. The need to estimate coefficients with certain statistical properties appears to have played a critical role in the way in which the process of modeling is understood by most social scientists. Limiting modeling to those situations in which precise estimates of coefficients may be developed is, in the extreme handicapping. It diverts attention from the real business at hand. Tukey (1963) in an oft-quoted maxim, phrased the issue in normative terms:

The most important maxim for data analysts to heed, and one which many statisticians has shunned, is this: ‘Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.’ (p. 13)

The real business of modeling is not estimating coefficients. The real business of modeling should be helping to ask the right questions and to organize answers. In fact, because the language of modeling—in the social sciences of the last 25 years—started with sets of equations, there are not many examples of qualitative studies in which the language of causal modeling is used (but see Miles & Huberman, 1994). Yet, if you talk with qualitatively trained, applied, or preventive-intervention-oriented social scientists, there is general agreement on the role that modeling plays in their approach to social phenomena. They are likely to say something to the effect of, “First, you model the dynamics of the phenomenon in which you are interested—because you cannot design an intervention until you know what you are doing—and then you develop an intervention on the basis of the knowledge summarized in the model.”¹ I believe that by considering models as organizing devices that facilitate a continuing, explicit dialogue between multiple sources of data and assumptions, we can ask better questions and develop more valid answers in spite of having widely divergent methodological approaches.

A BASIC DILEMMA

The causal modeling process requires simplifying assumptions about the nature of the social world. Comparisons need to be made across cases, time, or different aspects of a phenomenon. Making simplifying assumptions immediately places researchers in a dilemma, however. If the assumptions we make do not reduce complexity and ambiguity to manageable bounds or make comparison possible across cases, times, or aspects of a phenomenon, causal analysis is not possible. On the other hand, if the assumptions we make are too unrealistically simple, if they presume to generalize all times and contexts or hinder our ability to get close to the phenomena of interest, the relationship of models to the complex realities being modeled becomes tenuous.

How this basic dilemma is resolved has implications for the kinds of analyses that are possible, for the range of phenomena that are analyz-able, and for the meaning of the results from either a practical or theoretical standpoint. Freedman’s work (1985, 1987, 1991), for example, reminded us of the fragility of the assumptions necessary in using multiple regression analysis, still the industry workhorse for quantitative work with large samples. We must make constraining assumptions about the direction of causation, measurement error in our variables, the linearity of the variables in the analysis, and the impact of variables left out of the analysis. Because these assumptions are often violated, the credibility of the analyses is often suspect.² Part of the overall problem is an attempt to emulate the precision of the natural sciences, with results that occasionally expose the differences between the two. In discussing the appropriateness of using regression analysis for certain tasks, Freedman (1985) drew the following inference:

… insofar as the problem of planetary motion now looks clean and simple, that is the result of centuries of hard work … The social sciences may be at the pre-Keplerian stage of investigation—the equivalent of figuring out which are the planets and which are the fixed stars. If so, using sophisticated analytical techniques like regression is bound to add to the confusion. The problem is to define the basic variables, to figure out ways of measuring them, to perceive the main empirical regularities. Estimating coefficients by least squares before the basic variables have been understood is like using a scalpel to clear a path through the jungle, (p. 352)

Precision may be a false goal for social science modeling. To justify the use of a scalpel, we have begun to consider only those larger-scale quantitative studies that have a chance of satisfying the assumptions of multiple regression analysis as credible. We have become enamored of large-sample studies that can compensate ...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• Preface
• Dedication
• 1 First Steps, Basic Dilemmas, Gulfs and Bridge Building
• 2 The Big Leap: Carving Situations up Into Concepts
• 3 Sticks and Spaces: Relationships Between Concepts
• 4 Elaboration Within the Constraints of an Additive Model
• 5 Closing Circles and Uncovering Dynamics: Feedback in Social Life
• 6 Conditional and Moderating Relationships: Elaborating the Contexts of Action
• 7 Criteria for Evaluating Models
• 8 Strategies for Moving From Elaborated to Working Models
• 9 Epilogue
• References
• Author Index
• Subject Index