Patterns of Discovery in the Social Sciences

Paul Diesing

• Publisher: Routledge
• Year: 2017
• Print ISBN: 9781138529625
• eBook ISBN: 9781351500463
• Topic: Philosophy
• Subtopic: Philosophy & Ethics in Science

Social scientists are often vexed because their work does not satisfy the criteria of "scientific" methodology developed by philosophers of science and logicians who use the natural sciences as their model. In this study, Paul Diesing defines science not by reference to these arbitrary norms delineated by those outside the field but in terms of norms implicit in what social scientists actually do in their everyday work.

1
Introduction

Books on social science methodology mostly fall into one of two classes. First, there are “methods” books, works which introduce the student to research techniques in some specialized area of the social sciences. Each field has its own methods: there are methods in social research, methods in cultural anthropology, research methods in human relations, and so on. These are “how-to-do-it” books. The student is taught how to write questionnaries, conduct interviews, calculate chi-squares, administer tests, write computer programs, and do whatever else is required in his special field of interest. Such books must be revised frequently, because new techniques are constantly appearing and old ones being modified. Then there are “method” or “scientific method” books. These are more abstract discussions of science in general, referring to specific fields only to illustrate what is true of all science at all times.

My approach is midway between these two. The methods I investigate are not the hundreds of particular scaling, testing, interviewing, and statistical techniques, nor the timeless logic of science in general, but rather the four or five different methods or modes of procedure, incorporating particular techniques as parts, that social scientists use today. By “method” or “mode of procedure” I mean the whole series of steps that a scientist or research team follows in the process of making a contribution to a field of knowledge. Not everything a scientist does is part of his method—teaching seminars in the subject, applying for research grants, politicking to get his theories accepted—but only those things that are an essential part of the achievement of knowledge. I call these methods “patterns of discovery,” using the terminology of the late Norwood Hanson (1958), because I am dealing with the whole process of inquiry, the whole process of “discovering” or creating or developing knowledge, and not just the verification aspect.

To discuss “methods” rather than method does not imply that there is no one basic method of science. However, a premature interest in this one method forces one’s attention to move to so abstract a level that much of what scientists do must be ignored as technical detail. Consequently, one’s account tends to become thin and abstract, and attention shifts to philosophical puzzles of little interest to scientists; or, in the attempt to achieve richness of detail, one tends mistakenly to identify a particular method, say, the method of nuclear physicists or experimental psychologists, with the general method of science. General scientific method is best discussed only after one has begun to appreciate the variety that exists in methods now in use.

Types of Methods

If one’s attention is directed to differences among methods, the most obvious difference is that between the clinical and the experimental method. This difference has often been noticed and has been accounted for in a great variety of ways. If one wishes to reduce differences to a minimum, one can say that there are only these two basic methods, the clinical and the experimental. However, with a bit of attention, one notices that survey methods are distinguishable from experimentation and that there are also variants of the clinical method, notably participant observation. One also finds that formal methods have characteristics that distinguish them from both clinical and experimental approaches, and that there are in turn several formal methods.

One could go on and make further distinctions, but let us provisionally stop here and say there are at present four main types of methods in use: experimentation, statistical survey research, participant-observer and clinical methods, and formal methods. For the time being, computer simulation can still be treated as a formal method, though perhaps in a few more years it may be more appropriately regarded as a fifth and distinct method.

Participant-observer and clinical methods can also be distinguished, but it is more convenient to group them together, to keep the list down to four. Such a list is not intended as a definitive classification of existing methods, but only as a set of initial distinctions useful for exploring the field. As one continues his investigations, it may become necessary to make further distinctions of varying degrees of sharpness and to notice continuities or overlapping between methods initially distinguished from each other.

Let us glance at each of these methods briefly to note their main characteristics.

The experimental method has been most fully developed among the social sciences in psychology and in social psychology. It has variables as its subject matter, that is, any natural occurrences that exhibit measurable variations in incidence, or rate of occurrence, or rate of change of occurrence. Its principal objectives are to discover variables that behave in a law like fashion and to discover the laws governing their variation. Presumably everything in nature changes somehow, but the experimentalist tries to find regular changes that can be described in relatively simple and precise terms. Originally variables were studied in pairs, but later Fisher’s statistical work (1935) enabled experimenters to deal with three or more variables simultaneously. When a pair of variables is being studied, one is ordinarily treated as an independent variable (“cause”) and the other as a dependent variable (“effect”). The correlations that may be found between the two serve as a first approximation or ingredient of some prospective law. More complex correlations and partial correlations among three or more variables point to more complex laws.

The experimental procedure, in outline, is to locate a potentially lawlike variable by examining previous experimental results and trying to find masking effects that disguised or covered over some hidden correlation. Theory is useful for suggesting possible masking effects and possible hidden correlations. It is also possible to examine a case study or even one’s own experience with the help of theory, to locate a possible lawlike variable, but this approach is more difficult and less likely to succeed because of the chaotic appearance of ordinary experience. The searching of ordinary experience is likely to be a haphazard, hit-or-miss affair, while the searching of experimental results can be more systematic because of the regularity of the data.

Next, one imagines an experimental situation in which the masking effects are removed or controlled so that the hidden correlation can be plainly observed. Control can be achieved in a variety of ways, including holding the masking factors constant, eliminating them entirely, limiting their range of variation, counteracting them, and subtracting their presumed effects statistically from the results. Once the controls are set up, the next steps are to introduce the independent variable and then measure the change in the dependent variable. The results are then compared with previous experimental results to see whether one has moved closer to the presumed hidden correlation. If one has moved closer, one continues the search in the same direction; if not, one starts looking in a different direction. Tests of significance are used to determine whether it is worthwhile to continue the search in the same direction or advisable to try something different. Significance criteria are set at a level such that not too many promising leads are discarded prematurely and not too many blind alleys are preserved; however, such tests are advisory only.

It is also possible to begin one’s work by examining plausible speculations on a subject, then operationalizing some of the key concepts and devising ground-breaking experiments. Such initial experiments cannot be expected to produce immediate success; they serve only to start the long search for hidden variables and correlations.

As the investigator gradually refines his variables and strengthens his correlations, he also tries to determine the limits of their validity. Do they hold only for college sophomores? For men only, or for women, too? For Japanese? Navahos? Frequently some speculation or theory can be used to suggest a class of subjects for whom the correlation might not hold, or for whom it holds very strongly. Such investigations not only uncover limits but also put one on the track of more general laws of which the original correlation was an instance.

Checking can occur throughout the search process. It is possible at any stage to repeat the experiment in a different place with a different experimenter and different instances of the variable to see whether the same results occur. However, most published instances of what are called “replication” are actually part of the search process, since small changes are made in the experimental setup in hopes of getting a slightly better correlation or of uncovering new limits on the original correlation. True replications, changing only experimenter, place, and specific subjects, are usually left to students, and their frequent failures to get the same results are explained as being due to inexperience.

Once the initial objective, a general law, is achieved and checked, attention shifts to the discovery of new laws. These may be supplementary, in that they limit the range of validity or applicability of the original law, or they may state the effects of the original dependent variable on other variables. The eventual result envisioned is a kind of network of linked variables, extending endlessly in all directions.

In the experimental method, definitions are always at least partly operational. Definitions of independent variables include a statement of the operations by which they are introduced and controlled, and definitions of dependent variables include a statement of the operations and measurements by which their presence can be determined. The reason is that the experimental discovery of laws depends on actual operations with the variables involved, which is possible only if the variables are reduced to operational terms. Similarly, replication is possible only if the original operations have been specified. It is not necessary to have a completely operational definition; in many cases it is thought that a single concept, for instance “group cohesion,” can have several different operational definitions, all sharing a vague common core of meaning. However, each new operational definition produces some shift of meaning, perhaps a large shift. Consequently, widespread use of the experimental method tends to produce a proliferation of variables and laws, many vaguely overlapping, rather than the single clear network of laws originally anticipated. When attempts are made to collect and systematize large numbers of empirical laws, as in March and Simon’s Organizations (1958), the results are suggestive rather than precise because of the shifting meanings of the central variables.

This difficulty in producing truly general laws is one of the chief problems in the experimental method in the social sciences, along with such problems of controlling variables as experimenter bias. Experimentation is effective in producing five-page reports in psychology journals, but these reports are consolidated only very gradually into a system of general laws. Consequently, scientists interested in developing general theory in a hurry sometimes shift to other methods, particularly the formal method, which are better adapted to the problems of general theory.

The survey method was devised to overcome another problem of the experimental method: the difficulty of dealing experimentally with large and complex subject matter. Experimentation always involves a considerable abstraction from natural complexity, and scientists who wanted to study complex sets of variables in their natural setting devised the survey method for this purpose. However, survey research has developed well beyond this original purpose and become a method in its own right, one that has been combined with and enriched other methods and has also produced its own kind of theory.

The experimental difficulty of dealing with large and complex subjects is met in the survey method by sampling and by substituting statistical controls for experimental ones. Similarly, correction and validation involve primarily the statistical manipulation of data. With the continuing development of statistical techniques it has become possible to devise quite complex research designs, involving many variables in a variety of relationships and yielding complex correlations. Thus the austere limits of the classical experimental method are transcended, and the complexity of actual societies can be more adequately handled.

Another advantage of the survey method is that it combines readily with all other methods. Experimentation has been enriched by statistical controls, for instance by using sampling techniques to select experimental subjects. Participant observers have used sample surveys to extend the range of their observations, while survey researchers have used a variety of clinical and quasi-clinical techniques, such as focused and unfocused interviews, various degrees of participant observation, and projective devices, to enrich their data. The variety of combinations in use is so great that survey research and participant observation can now be seen as two ends of a continuum rather than as two distinct kinds of methods. Formal methods have also used survey research data to provide interpretations and probable values of formal variables and to suggest new variables and relationships.

The participant-observer method was first developed by anthropologists, though it is also frequently used by sociologists, social psychologists, political scientists, and organization theorists. Its primary subject matter is a single, self-maintaining social system. The system may be a small community with its own culture, or a larger society with its culture, or a small and relatively isolated neighborhood, or a gang, clique, voluntary organization, or family, or a formal organization or institution, or a person (clinical method), or a historical period. In each case the emphasis is on the individuality or uniqueness of the system, its wholeness or boundedness, and the ways it maintains its individuality. The primary objective is to describe the individual in its individuality, as a system of rules, goals, values, techniques, defense or boundary-maintaining mechanisms, exchange or boundary-crossing mechanisms, socialization procedures, and decision procedures. In one important variant, the primary interest is in recurring processes within or around such individual systems.

The procedure is, first, to become socialized into the system, to learn a set of roles and normative elements, to form relationships, and thus to participate in the normal routines and occasional crises of the system. If the system is small, the researcher can gradually turn himself into an analogue of the system, so that he reacts as it reacts, feels as it feels, thinks and evaluates as it does. The next step is to make this implicit knowledge (Polanyi’s “personal knowledge,” verstehen in a sense) explicit. The researcher constructs hypotheses about parts of the system out of the recurrent themes that come to his attention and tests these hypotheses against a variety of data—what he sees, what others tell him, how he reacts, and how others react to his probing actions. Many detailed hypotheses are gradually combined into a model of the whole system, whose parts are tested by how well they fit together and how well they agree with the data.

The system model is continually checked against new data and revised. Since the researcher is part of the system he studies, new data are continually coming in and the model is never quite completed. Other researchers contribute further checks by providing their own models of the system, which are compared with one another for coherence as well as with the various sets of data.

All through this process the researcher is continually comparing his case with others familiar to him, looking for similarities and differences, and using one case to suggest things to look for in another. One eventual result of such a process of comparison is a typology, a classification of cases according to similarities and differences. Further study of a type should lead to hypotheses about which of its characteristics are particularly important in determining the rest and what are the dynamics of the type. Comparison of widely differing types enables one to search for still more general characteristics of many kinds of human systems—universal or nearly universal values, institutions, system problems, mechanisms, and the like. General theorizing of this sort tries to transcend the relativity inherent in the participant-observer method by looking for general characteristics of human systems, though it still recognizes that these characteristics vary considerably in detail.

At least three other methods similar to participant observation can be distinguished. First, the clinical method used in clinical psychology and psychiatry is basically the same in that it deals with a whole, unique, self-maintaining system—in this case a person—and aims at construction of a system model; it involves the intimate participation of the therapist in the functioning of his subject matter, so as to develop an intuitive understanding of it; it involves the development of specific hypotheses out of recurring themes and the testing of them against several kinds of data, including the clinician’s own reactions and the responses to his probing actions; and it involves the continuous reconstruction of the system model in terms of internal coherence and of agreement with the continuing supply of data. It falls short of participant observation at its best in that the clinician cannot, in principle, get as complete an inside understanding of his subject as can a group of field workers. If the personality is, in part, a system of roles and role expectations, the clinician participates in it by taking one or two roles that are offered him. He can then participate in and observe the activity of his subject in those roles. But the subject’s activity in other roles, as husband, father, employee, and the like, is not accessible to direct observation and must be reconstructed intuitively from the subject’s reports. This makes for an incompleteness of observation that is not necessarily the case for field studies. A partial solution to the clinician’s problem is to study a whole family, but this approach is likely to sacrifice some of the depth of knowledge that can be achieved by concentrating on a single subject or part of one.

Another similar method is used by some historians when they attempt to reconstruct a whole historical period out of available data and try to understand it, intuitively or “from the inside,” as a kind of integrated system with its own unique character or spirit. This method falls far short of the clinical method in that the historian cannot participate in his subject matter at all but must experience it vicariously and imaginatively. Nor is a historical period actually a self-maintaining system with actual boundaries; even if it were, it would be much too large to reconstruct in all its inner workings.

Still another similar method is occasionally proposed by some institutional economists. In it the self-maintaining system to be studied is the total set of institutions in which a particular economy functions, seen in historical perspective. I have not succeeded in understanding this method adequately since it seems to have remained a proposal rather than an actuality for over a half century. However, it would seem to involve all the difficulties of the historical method and more, owing to the size and complexity of its subject matter. Just as the participant-observer method has been most successful in studies of simple nonliterate societies or small formal organizations, so the most successful institutionalist studies have been of small primitive economies (such as Polanyi, 1957). Attempts to study the U.S. or world economy have necessarily involved great reliance on statistics and thus have moved toward the survey research method, which is much better suited to a large subject matter. My impression is that there is no one institutionalist method predominant at the present time; some people who call themselves institutionalists use statistical surveys, some use elaborate econometric models, some use participant observation supplemented by numerous statistics, and some use historical reconstruction. Conversely, if a unified institutionalist method is ever fully developed, it will probably be some amalgam of clinical-historical, survey research, and even formal methods.

Formal methods have long been in use in economics, and in recent years have become important in a number of fields, including psychology, sociology, international politics, and some newer interdisciplinary fields. These methods are in particularly rapid development right now, so it is difficult to give an adequate descrip...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Contents
• Acknowledgments
• 1. Introduction
• I. Formal Methods and Theories
• II. Participant–Observer and Clinical Methods
• III. Methods in the Philosophy of Science
• References
• Index