Social Science Information and Public Policy Making

Robert F. Rich

• Publisher: Routledge
• Year: 2018
• eBook ISBN: 9781351306300
• Topic: Social Sciences
• Subtopic: Social Science Research & Methodology
• Index: Social Sciences

A survey of federal officials reveals the belief that government should make the fullest possible use of social science information-and yet most of the information developed by social scientists winds up in specialized libraries or data banks, where it remains unused. Why don't public officials make greater use of the information social scientists develop? What can social scientists do to ensure that their findings are used? To answer these and related questions, Robert Rich reports the results of a unique experiment designed to facilitate the use of research data by public officials in federal agencies.

Rich interviewed both researchers and users of research data over the two-year life of a Continous National Survey (CNS) experiment to discover the extent to which the CNS mechanism was successful and to record the levels and types of use that officials made of the data provided. Rich reveals that factors such as the timeliness, cost, and relevance of data do not guarantee that information will be used. He examines patterns in the actual use of survey data by agency officials and explores key organizational factors, such as the compatibility of information with various bureaucratic interests. He discusses the preoccupation of public officials with bureaucratic issues regarding the ownership and control of information, identifies the incentives that prompt bureaucrats to pass along new information and the government officials' difficulties in developing policies and programs for meeting national needs.

Rich notes that studies of knowledge inquiry systems, found in the research literature of many social science disciplines have been dominated by a "rationalistic bias." This "bias" is expressed in terms of the belief that the act of acquiring information will automatically lead to its use, in turn, automatically leading to improved policy or decisions. He contends that empirical studies of how information is actually used do not support the assumptions of rational choice theory. The new chapter also discusses types of information, knowledge, and use; prospects for the development of learning organizations in government; and the politics of expertise. This book will be of interest to social scientists and public policy makers.

Robert F. Rich is professor of law and political science at the University of Illinois at Urbana-Champaign. He is also professor in the Institute of Government and Public Affairs, and was the director of that Institute from 1986-1997. He is the founding editor of Knowledge: Creation, Diffusion, Utilization (now Science Communication).

CHAPTER 1

Experiment in the Application of Survey Research

Few true administrative experiments funded and operated by a public agency at the federal level have been conducted in the United States’ bureaucracy.* Especially rare are experiments designed to alter the day-to-day procedures and practices adhered to by one or more agencies. Certainly programs or procedures have been labeled as experimental or innovative (especially many of the Great Society programs of the 1960s), but very few such “experiments” adhere to the principles of experimental or quasi-experimental design. Such experimental “interference” in the day-to-day operations of a busy agency is usually considered to be disruptive and unacceptable.

Yet in 1971 and 1972, the National Science Foundation (NSF), through its newly created Research Applied to National Needs Division (RANN), funded several administrative experiments to test procedures and mechanisms for transferring policy-relevant knowledge or information to public officials responsible for decision making. The experiments were to assess whether policy makers would accept particular knowledge-transfer mechanisms (that is, a set of organizational procedures) and whether they would use the information provided to them.

Officials in RANN who supported this program sought to provide federal agencies with new and unique sources of information, one of which was monthly and weekly public opinion or survey data. RANN funded an eighteen-month administrative experiment to test the feasibility of a multipurpose, multiagency national survey which would provide federal policy makers with public opinion surveys relevant to current issues. The objective of the survey was to improve the information base for the public policy and program decision-making processes. The success of this administrative experiment-the Continuous National Survey (CNS)-is the topic of the study reported in this volume.

Background

The adaptation of scientific knowledge to meet the needs of society is a recurring theme in western thought (Rich, 1979). Scientists, philosophers, and sociologists have argued that social change is directly related to changes in modes of knowing, changes in the way in which information is generated. Lynd drew attention to the importance of this phenomenon in Knowledge for What?, published in 1939. But, with the exception of studies carried out during World War II, the influence of the social sciences as a field and social scientists as experts was limited. The field developed as an autonomous social system and it continued as such with minimum regard for its social utility (see Caplan, 1980).

In the 1960s, the application of social science information concerned managers responsible for public policy programs and the allocation of research and development budgets. Blue-ribbon commissions (the Brim Commission, the President’s Commission on Federal Statistics, the Commission on the Organization of Government for the Conduct of Foreign Policy) and special, new divisions of agencies (for example, the Office of the Special Assistant for Information) were created to address the problem of translating research into action. Most recently, the Office of the Science Advisor in the White House established the Intergovernmental Science, Engineering, and Technology Advisory Panel in response to state governments’ desire to make use of the vast amount of research funded by federal dollars. The RANN program represented a commitment by the federal government to experiment with potentially feasible methods, knowledge-transfer mechanisms, and formal and informal administrative practices that would enhance the utility of social science knowledge for public policy makers.

The RANN program was, in part, a reaction to the role of research in the rise and demise of the Great Society programs. As Aaron (1978, p. 9) points out, during the Great Society years, “social scientists were actively engaged in the planning and later in the evaluation of programs of the Great Society.” In the early years, one cannot be sure about the role that social scientists played in crafting, implementing, and evaluating the major social programs of the time. It is not clear whether prevailing social science knowledge was consistent with the ideology of the time. People believed in the programs being put forward, and their commitment to these beliefs was more important than rigorous forms of analysis. Indeed, as Aaron (1978, p. 159) points out, “This need, this passion, commanded analysts and others to suggest policies best calculated, given the available information, to achieve desired ends.”

As it became clear that the Great Society and War on Poverty programs were not working, some officials blamed the social scientists who promised too much, too quickly; problems were not solved despite the promises that were made. Others reacted differently: “When the passion waned, partly because of external events and partly because of frustration at the apparently mixed results from national policies, the imperatives of the analytical process won out” (Aaron, 1978, p. 159). RANN was created in this atmosphere, in which it was uncertain as to what the status, prestige, and role of the applied social sciences should be. Moreover, it is important to remember that RANN was founded as part of the aftermath of the Special Commission on Social Sciences, which was established by the National Science Board in 1968. This commission, chaired by Orville Brim, “was charged with making recommendations for increasing the useful application of the social sciences in the solution of contemporary social problems” (“Knowledge into Action ...,” 1969, p. xi).

The growing importance that was attributed to the social sciences should be placed in the context of the expansion of the knowledge industry after World War II. Between 1945 and 1965, research and development expenditures in America increased fifteenfold. Between 1964 and 1969 alone, the percentage of the gross national product devoted to research and development (R&D) increased from 3.4 percent to 9 percent, or by a greater percentage than any other country in the western world (Bell, 1974, pp. 250-251). Indeed, as Bell (1974, p. 212) points out, the U.S. had become a knowledge-oriented society: “The postindustrial society, it is clear, is a knowledge society in a double sense: first, the sources of innovation are increasingly derivative from research and development (and more directly, there is a new relation between science and technology because of the centrality of theoretical knowledge); second, the weight of the society-measured by a larger proportion of the gross national product and by a larger share of employment-is increasingly in the knowledge field.”

In a recent study, the National Academy of Sciences (1977) reports that in fiscal year 1977, the federal government invested $2 billion to acquire and use knowledge of social problems. This sum probably reflects a major investment in the social sciences.

Moreover, regarding the translation of knowledge into action, figures compiled by the Office of Management and Budget (1976) indicate that many federal agencies are spending a large proportion of their R&D budgets on knowledge transfer and utilization. In particular, it is worth noting that the Department of Agriculture spent 46 percent of its R&D budget in fiscal year 1975 on knowledge transfer and utilization activities; the Law Enforcement Assistance Administration spent 13.6 percent; the National Institute of Education 10 percent. Clearly, agencies dealing with social science research are investing substantial resources to increase the utility of their R&D investments.

These budget figures indicate that government agencies are quite serious about translating research findings into action. The commitment is reflected in a requirement that contractors and grantees submit a plan outlining how their research results will be utilized with their request for research funds. It is also reflected in congressional appropriation hearings which, in the last ten to twelve years, have required cabinet and subcabinet members to document how publicly funded research results are being applied-What is the return on the taxpayers’ investments? Several blue-ribbon commissions have been created to investigate the utility of basic and applied research in public problem-solving activities; most recently, the National Academy of Sciences (1977) sponsored a study on the federal investment in social research and development.

Rationale for CNS

In response to this growth of information and information-directed resources as well as pressure to make social science relevant, public policy makers felt that they needed to allocate substantial resources to develop techniques for increasing the utility of scientific knowledge. On what basis was this objective to be accomplished? What, if anything, had history or experience taught about the formulation and implementation of knowledge-transfer procedures?

Two Cultures Perspective. Despite the fact that the social sciences have been viewed as only marginally relevant to the concerns of government officials, social scientists themselves have long believed the application of their work to policy making would improve the quality of public decision making and, hence, improve the quality of life for the public at large. However, there has been some discrepancy in the perceptions of relevance; decision makers have not seen the immediate relevance of social science knowledge for their work and, consequently, both formal and informal interactions between policy makers and social scientists have not always been successful.

Among social scientists, the prevailing belief is that empirically grounded knowledge is seriously underutilized in important policy decisions: “Social science still accumulates in libraries and in impractical retrieval systems rather than policy and governmental practices” (Caplan and Rich, 1976, p. 1). Policy makers, however, feel that they cannot understand the reports they receive, that the reports do not deal with the immediate problems on their agenda, and that the reports are not sensitive to political and bureaucratic pressures. Decision makers are concerned with the restrictions imposed by the need for timely decisions; immediate partial information is more useful than complete information, which cannot be used, later.

Researchers and scientists feel that decision makers do not clearly communicate their needs, do not have a sense of how long it takes to produce accurate information, and do not respect research for the function it serves. Moreover, researchers’ professional integrity requires that they not release information which is not of the highest quality.

These accounts of policy makers’ and researchers’ attitudes point to differences in communication, values, and problem-solving orientations among researchers and decision makers. These tensions are similar to those that Snow (1962) describes in discussing the humanities and the hard sciences as two separate cultures. Although most accounts of such tensions are based on the relationships between academic researchers and federal decision makers, they also characterize the relationships between researchers and decision makers inside government. (For a discussion of the significance of the “two cultures,” see Caplan, 1979, and Dunn, 1980.) This notion of two different cultures encompasses the set of problems that federal officials must address if they are to increase the overall utility of social R&D investments. Specifically, this set includes problems of in adequate communication, concerns over quality of information, the nature of the decision-making process, the constraints that process places on utilization, and bureaucratic reward and incentive systems. These factors taken together are presumed-by the policy-making and research communities alike-to account for the limited use of social science information during the last three decades.

Communication Barriers and Blockages. Policy makers and research personnel (both in and out of government) tend to have distinct vocabularies and languages, which hinders the coordination of their work. A congressional study on the uses of social science research in federal domestic programs highlights the communication problems: Scientific jargon interferes with the application of potentially relevant findings. “The scientists’ jargon acts as a deterrent to utilization. Complaints about the scientists’ ‘gobbledygook’ come from every corner. This would also seem to be a symptom, rather than a cause. The language of science is a highly perfected shorthand for communication between scientists. It is effective for its purpose, but not for the purpose of communicating with the public or with most practitioners [decision makers]....What is needed is a special group of writers who can aptly translate research findings into language meaningful to the public and to persons responsible for their practical application”(“Use of Social Research . . .,” 1967, p. 393).

In a similar vein, Moynihan (1973, p. 266) points to the potentially threatening quality of social science research from the perspective of public officials: “There is a . . . general set of circumstances that contributes to the emerging threatening quality of the social sciences, and that is that they are getting complicated. The methodology of most social science is now quite beyond the comprehension of non-social scientists. In particular, it is beyond the ken of the lawyer class that tends to wield the levers of power in American government. Thus a priestly role of interpreting the mysteries is gradually emerging. And with it the anticlericalism of priest ridden societies. The simple fact is that it is harder and harder to know what it is social science says about anything.”

The absence of a mutually understood set of terms (vocabulary) affects the ease with which policy makers are able to use research findings and research techniques; in addition, it adversely affects the esteem afforded researchers and research and tends to highlight the distinction between researchers and policy makers. Merton contends that levels of esteem and utilization are highly correlated: “Not only does utilization affect esteem, but esteem also affects utilization” (“Use of Social Research . . .,” 1967, p. 393).

The response reported by Moynihan, as a government advisor, and by the congressional study is one of some hostility and a certain feeling that the “two cultures” should remain separate, that some group should be responsible for mediation or translation. This problem is especially important when one realizes that policy makers want to make use of relevant information. Caplan and others (1975) report that 85 percent of the 204 federal executives studied believed that social science knowledge can contribute to the improvement of governmental policies; 87 percent agreed that government should make the fullest possible use of social science information. In other words, government officials feel they would and should make the best possible use of available data, but they need mechanisms and procedures to realize this desire.

Quality and Objectivity of Information. The quality, precision, and accuracy of information-its objectivity-also affect utilization. Caplan found that federal decision makers were concerned about the objectivity and accuracy of information; these individuals also believed that government officials did not have access to the highest-quality information. Caplan also demonstrates a direct connection between the quality of information and the use made of that information. In the cases in which policy makers consciously rejected social science information, he found that: “There appears to be no failure on the part of respondents to understand the meaning or relevance of their social science information to policy....Information is most often rejected on grounds of objectivity. Objectivity becomes an issue when the data base is viewed as weak, the study design is poor, or if there is a general belief that there is so much bad research in the social sciences that valid findings are indiscernible, especially when two studies on the same subjects produce opposite findings” (Caplan and others, 1975, pp. 29-30).

Weiss (1977), too, found that quality was relevant in policy makers’ determining whether to use policy-related information. Her study showed that policy makers are sophisticated consumers; they understand issues of quality control and make judgments concerning them.

Nature of the Policy Process. Lowering communication barriers and establishing objectivity are related to the problems involved in coming to an agreement concerning what policy goals should be and how policy is made. A clear understanding of policy is central to establishing and maintaining working relationships between researchers and policy makers. A State Department study concludes that the government has not received the kind of information it could use most efficiently because it has not been successful in communicating its needs: “the government only received abstract studies because of fundamental misperceptions which existed with respect to what U.S. policy was. Thus, no one felt that they understood what was required” (Einaudi, 1974, pp. 37-38).

Public officials are not in the practice of developing general principles that can be applied to many different problems; instead, their operations are focused on solving particular problems. A more general set of principles would be needed to dictate the development of long-term information gathering and processing capabilities.

The identification of specific information needs is dependent upon articulated goals, projections for future information needs, and projections identifying future problem areas. The nature of the policy process, which some scholars label a non-process, does not allow for this. A Presidential Commission on Federal Statistics highlighted this problem:

The basic difficulty lies in defining the goals of a program. In the words of an official responsible for planning and evaluation in a government agency “when researchers say, tell us what you want” it appears that they are not aware that they have asked the hardest-perhaps the impossible-question of gover...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Dedication Page
• Contents
• Introduction to the Transaction Edition
• Foreword
• Preface
• The Author
• 1. Experiment in the Application of Survey Research
• 2. Continuous National Survey: Structure and Analysis
• 3. Development and Funding of the Survey Experiment
• 4. Planning and Conducting the Project
• 5. Assessing the Survey Experiment
• 6. Utilization of the Survey Information
• 7. Future of Survey Research for Meeting National Needs
• Appendix A: Questionnaires
• Appendix B: Basic Coding Sheet and Summary Tables
• Appendix C: Agency Memos, I
• Appendix D: Agency Memos, II
• References
• Index