Methods That Matter

Integrating Mixed Methods for More Effective Social Science Research

M. Cameron Hay

• Publisher: University of Chicago Press
• Year: 2016
• ISBN:
  9780226328836
• Topic: Social Sciences
• Subtopic: Sociology
• Index: Social Sciences

To do research that really makes a difference—the authors of this book argue—social scientists need questions and methods that reflect the complexity of the world. Bringing together a consortium of voices across a variety of fields, Methods that Matter offers compelling and successful examples of mixed methods research that do just that. In case after case, the researchers here break out of the traditional methodological silos that have long separated social science disciplines in order to better describe the intricacies of our personal and social worlds.Historically, the largest division between social science methods has been that between quantitative and qualitative measures. For people trained in psychology or sociology, the bias has been toward the former, using surveys and experiments that yield readily comparable numerical results. For people trained in anthropology, it has been toward the latter, using ethnographic observations and interviews that offer richer nuances of meaning but are difficult to compare across societies. Discussing their own endeavors to combine the quantitative with the qualitative, the authors invite readers into a conversation about the best designs and practices of mixed methodologies to stimulate creative ideas and find new pathways of insight. The result is an engaging exploration of a promising new approach to the social sciences.

Part One

Methods and Theory for More Holistic Human Sciences

One

Repairing the Fractured Social Sciences: An Introduction from a Historical Point of View

Robert A. LeVine

The social sciences today are divided into research communities that cherish their separation from each other. They resemble the Tower of Babel more than a mature science with complementary inquiries strengthening a common body of knowledge. The disciplines and sub-disciplinary research communities are not at war but are simply turned inward to such a degree that they do not recognize each other’s contributions as knowledge worth considering in their own research. Researchers do not cite the works of other research communities and often examine the same topics without reference to each other’s works. Their differences are largely methodological or epistemological, i.e., each research community is organized around a particular set of methods taken to generate knowledge; research findings from the use of other methods are ignored or dismissed as being irrelevant to knowledge generation. Thus the boundaries between disciplines, or in some cases between method-based research communities within disciplines, are wide and deep. It wasn’t always this way.

Far from the Omniscience Ideal

How different this is from the ideal formulated in 1969 by my old friend and collaborator Donald T. Campbell (1969) in his “fish-scale model of omniscience”! He envisioned the social sciences as scales on a fish in which each specialty was unique but overlapping its neighboring specialties, so that the mutual relatedness of topics and data was mediated through shared boundaries, guaranteeing that specialists could use knowledge generated by adjacent specialties, resulting in “omni-science” if not literal omniscience. Campbell was quite aware that this ideal was not being met when he formulated it, at a time when there were fewer journals policing the boundaries than there are now. He was in effect attacking the increasing specialization of his time, and the haughty pride—he called it the “ethnocentrism of disciplines”—that maintained it, which has only worsened since then.

This divisive tendency presents obstacles to interdisciplinary research, which is often marginalized in the academic world of the social sciences. Fields like psychological anthropology and human geography that attempt to bring together perspectives from several fields using varied methods are often regarded as lacking the core or coherence needed to conduct research and train students. The low regard in which these interdisciplinary fields are held has its origins before 1920 but has strengthened greatly since the middle decades of the twentieth century, when the social sciences as a whole were smaller and contained fewer research communities than they do now. And sixty years ago, there was more contact and communication between colleagues in adjoining disciplines like sociology and social anthropology. Campbell liked to tell the story of how around 1951, when he was teaching at the University of Chicago, he submitted a paper to American Anthropologist, then edited by Sol Tax at Chicago, and to his surprise it appeared in The American Journal of Sociology, edited by Everett C. Hughes down the hall. Tax had decided it wasn’t anthropological enough and sent it to his colleague, who promptly published it. That kind of collegiality across social science disciplines may be rare nowadays, particularly among journal editors, but Tax’s action showed how even then policing the boundaries was important to him as a midcentury journal editor.

When the social sciences emerged from philosophy in the late nineteenth and early twentieth centuries, many of the pioneering figures were interdisciplinarians, especially in connecting psychological and social thinking in their work. Prominent examples are the psychologists Wilhelm Wundt, James Mark Baldwin, F. C. Bartlett, and L. S. Vygotsky; the sociologists Max Weber and W. I. Thomas (of the Chicago School of Sociology); the anthropologists W. H. R. Rivers, Bronislaw Malinowski, and Edward Sapir; and (outside of academia) the psychoanalysts Sigmund Freud, Otto Rank, Abram Kardiner, and Harry Stack Sullivan.

Even Emile Durkheim, though sharply distinguishing social from psychological facts, was actually quite psychological in much of his theorizing (“social sentiments”). He also combined qualitative with quantitative methods, notably in Suicide (Durkheim, 1897, 1951), relating an ethnographic description with a numerical frequency, what Alex Inkeles (1959) would later call the correlation of a “state” and a “rate.”

“Social psychology” became a recognized field within sociology as well as in psychology during the first half of the twentieth century. There were interdisciplinary “movements” like the Frankfurt School in sociology and the culture and personality movement in American anthropology.

Many of these interdisciplinary trends dwindled as the academic disciplines expanded and were bureaucratized in universities and professional associations, especially during the great expansion of the American universities in the 1960s. Disciplinary orthodoxies hardened, not only in doctrine but also in the institutionalization of departments and journals with sharp boundaries and vested interests.

Methods That Came to Define Science

Yet in competing for scarce resources the social science disciplines and research communities were not simply an unordered collection of specialties; they made claims for recognition as science based largely on the degree to which their canonical methods seemed to resemble those of the natural sciences. Those using mathematics, controlled experiments, formal assessments, exact measurements, statistical analysis, and methods linked to physical growth, physiological reactions, or brain images—gained credibility as scientific, while others using more exploratory or observational approaches in uncontrolled settings came under pressure to justify themselves in scientific terms. The highest ranks were awarded to academic disciplines that had become highly mathematical, like economics, or that conducted experiments, like psychology; others survived in the wide and ramified world of US academia, but were marginalized to some degree.

The ranking of social science research specialties was often legitimized by taking physics as the model for all sciences. The logical positivist philosophy of science that rose to prominence in the 1930s and 1940s provided recipes for formalizing social research to make it resemble physics: first, derive testable propositions from a general theory; second, define concepts formally and operationally; third, verify or falsify hypotheses through quantitative (preferably experimental) research. Scientific knowledge would consist exclusively of propositions that survived this process. Later, historians of science would show this to be an idealized account of how physicists work, which was rarely so tidy (Kuhn 1971; Latour and Woolgar 1986), but in the 1950s and 1960s many social scientists were attracted to it, and some journals continue to follow its mandates.

American anthropology was not one of the disciplines most influenced by this conception, or misconception, of physics. Since early in the twentieth century, in the United States (unlike Britain, France, or Germany) anthropology has consisted of four distinct fields—physical or biological anthropology, prehistoric archeology, linguistic anthropology, and social or cultural anthropology—fields sharing a commitment to field research that expands knowledge through comparative and historical analysis of an increasing number of well-documented variants, rather than through controlled experiments in the laboratory. However it might reduce the scientific prestige of anthropology, field research for comparative and historical understanding is well established as a method of choice in the life sciences. Anthropology in this respect resembles Darwin’s field observations of plants and animals in all their diverse forms—a model appropriate to the building of knowledge about the social behavior of humans.

Biology once had a sharp division between experimentalists and field researchers, but this was resolved during the twentieth century, first by the “neo-Darwinian synthesis” incorporating genetics into evolutionary theory, and more recently with the integration of molecular findings from DNA research into biology at all levels. This success story could be read as demonstrating that even after the advent of molecular genetics there is still an important place for field studies in the new unified biology, but some take a different lesson from it, supporting the priority and urgency of grounding human behavior in the brain; hence the growing interest of psychologists and psychiatrists in “neuroscience,” reducing behavior to its biological underpinnings. Meanwhile, the need for integrating the social sciences—or at least reducing their dissonance and mutual ignorance—received less attention.

Glimmers of Interdisciplinarity in the Darkness of Disciplinary Divides

John Whiting and Beatrice Whiting at Harvard nonetheless conducted research and trained students, myself and others, some of whom are in this volume, in interdisciplinary research designing and conducting ambitious research projects (e.g., the Six Cultures Study and the East African studies) on child rearing and development in different cultures. Much has been published from and about these projects, but certain aspects deserve emphasis in the present context: the Whitings assumed that quantitative and qualitative methods were equally necessary to describe, analyze, and interpret childhood around the world (J. Whiting 1994; B. Whiting and Edwards 1988). At the time I began working with them in 1954, this assumption was not widespread among social scientists, especially psychologists, and was in fact going out of style. The strains that eventually led to the breakup of the Harvard Department of Social Relations in 1970 were already evident. But the Whitings’ continued pursuit of interdisciplinary research was based on their experience in the 1930s at Yale’s Institute of Human Relations (IHR) with teachers like Edward Sapir and John Dollard. Focused on psychocultural processes of child development, the Whitings kept the multiple-methods candle lit in a period of increasing interdisciplinary darkness.

For John Whiting, Dollard had probably been the key guide. The only important pioneer of the culture and personality movement who has not been the subject of a biography, Dollard had been trained as a quantitative sociologist at Chicago, where he met Sapir. On Sapir’s recommendation, he went to Berlin for training at the psychoanalytic institute there. Sapir then brought him to Yale, and he carried out field research in Mississippi for Caste and Class in a Southern Town (1937) and (with Allison Davis) for Children of Bondage (1940). Later Dollard radically changed his interests to the “learning theory” of Clark Hull (then the senior psychologist at the IHR), which led to a terminal break with Sapir and to his collaboration with the psychologist Neal Miller, with whom he wrote the influential books Social Learning and Imitation (1941) and Personality and Psychotherapy (1950). After John Whiting’s return from fieldwork in New Guinea, Dollard, along with Bronislaw Malinowski, supervised his postdoctoral work translating his dissertation into a book, Becoming a Kwoma (1941), a case study analyzing the child-rearing practices of a particular community in terms of Hullian behavior theory. From Whiting’s point of view, Dollard’s qualitative field study of a community and the Miller-Dollard combination of experimental child psychology and psychoanalysis constituted models for his own future research and that of his students. It is no exaggeration to say that in pursuing this combined model, Whiting ignored the decline of interdisciplinary work in American social science during much of his career. Beatrice Blyth Whiting added the influence of Sapir and other approaches from both anthropology and psychology. The Whitings’ collaboration was the most exciting example of psychocultural research extant. In my encomium for the Whitings in 1989, when they received the Society for Psychological Anthropology’s first Career Contributions Award, I likened their role to that of the monks who kept classical learning alive during Europe’s Dark Ages after the fall of the Roman Empire. (Not a perfect analogy, but it captures the importance of their place in our history.)

Mixed Methods and Interdisciplinary Training: Keys to Repairing Fractures

Psychocultural research survived its darkest age in the third quarter of the twentieth century (roughly 1950–1970). Its revival thereafter—particularly in Southern California but also in a few other places, and in our scholarly journal (Ethos, begun in 1973) and our association (Society for Psychological Anthropology, 1977)—gives hope that interdisciplinary research will rise again to lead the social sciences out of the fragmented condition I described in the opening of this chapter. This happened in the biological sciences when the traditional university departments proved a poor match for the boundary-crossing research activity in which biologists were actually engaged. In social science, this hope can be realized only if disciplines begin speaking the same language or at least recognizing as knowledge the evidence from varied sources as they go about their research. That is why the “mixed methods” movement—recognizing the mutual importance of both quantitative and qualitative findings in establishing validity—is not only a major step in that direction, it is also the essential basis for building substantive connections across disciplines.

Another indispensable condition for the integration of the social sciences is interdisciplinary training. This means that in attacking problems in border regions, a social scientist should have the knowledge and skills of the several disciplines involved and be able (and willing) to publish articles in their journals. As an anthropologist operating in the child development field, for example, Tom Weisner brings developmental psychology to bear on those problems. His devastating critique (in a psychological journal, Weisner, 2005) of the Bowlby-Ainsworth attachment theory for its exclusive focus on the mother-child dyad and its neglect of distributed care and social networks is that of an insider as well as outsider. Without this kind of skillful boundary-crossing, the study of child development—which was originally conceived as an interdisciplinary field but has often devolved into an inward-looking developmental psychology—will fail to progress in building a body of scientific knowledge.

Psychocultural researchers interested in child development were fortunate to have the Whitings as our role models, but that does not mean that the Whitings simply transmitted that distinctive blend of behaviorism and psychoanalysis that had been constructed by Miller and Dollard or that the Whitings’ students—including Ruth and Lee Munroe, Tom Weisner, Rick Shweder, Carolyn Edwards, Susan Seymour, Gerald Erchak, and me—became their disciples. In the child-development field of the twentieth century, there was a profusion of theories we now consider false starts, blind alleys, or just plain wrong—ranging from G. Stanley Hall’s recapitulationism through behaviorism in its several manifestations to Piaget’s cognitive stages, etc. Wedded to its scientific aspirations (and pretensions), child development research was nonetheless vulnerable to paradigm shifts and sectarian theoretical loyalties. The Whitings’ students attempted to shed empirical light from multiple perspectives on fundamental problems of development, as in the following examples.

Thomas Weisner trained local observers in western Kenya to make quantifiable home observations, enabling him to identify sibling care as a high-frequency aspect of infant environments, first in East Africa and then in many other parts of the world, raising basic questions about early relationships in child development (Weisner and Gallimore 1977). Also in western Kenya, Ruth and Lee Munroe (1971) devised the method of spot observations for assessing infant care, based on contextual evidence that daytime caregiving in East Africa occurred largely in outside settings that permitted a local fieldworker to observe unobtrusively (and thus more validly than interviewing the mother). The observations could be repeated over time and aggregated for quantitative analysis. In our Gusii Infant Study my colleagues and I used the Munroes’ spot observation method to assess continuitie...