The Data Game

Controversies in Social Science Statistics

Mark Maier, Jennifer Imazeki

• Publisher: Routledge
• Year: 2014
• Print ISBN: 9780765629807
• eBook ISBN: 9781317457541
• Topic: Business
• Subtopic: Business General
• Index: Business

This book introduces students to the collection, uses, and interpretation of statistical data in the social sciences. It would suit all social science introductory statistics and research methods courses. Separate chapters are devoted to data in the fields of demography, housing, health, education, crime, the economy, wealth, income, poverty, labor, business statistics, and public opinion polling, with a concluding chapter devoted to the common problem of ambiguity. Each chapter includes multiple case studies illustrating the controversies, overview of data sources including web sites, chapter summary and a set of case study questions designed to stimulate further thought.

1

Introduction

□□□□

 

 

The Purpose of This Book

Interpreting social statistics can be frustrating. It seems as if there are numbers to prove anything—even entirely opposite points of view. For example, there are statistics to “prove” that the average U.S. family is becoming richer and that it is becoming poorer; that U.S. students are learning more and that they are learning less; that Americans vote less often or just as much as in the past; and that illegal drug use is rising and that it is decreasing.

A quite natural inclination is to reject all statistical results. After all, why trust any number if equally convincing numbers prove precisely the opposite conclusion? This cynical view is said to have been summed up by the nineteenth-century British politician and writer Benjamin Disraeli, who—according to American humorist Mark Twain—listed, in descending order of credibility, “lies, damn lies, and statistics.” Indeed, examples abound in which politicians, journalists, and policymakers fit statistics to their preconceived ideas. This book provides hints to alert readers to ways in which statistics can be misused.

Statistics are more than just sophisticated lies. In most cases the source of contradictory numbers is sincere disagreement between experts. If we can find out why the experts reach different conclusions, we will understand much more about the problem being analyzed. Consider the example of life expectancy, seemingly an easily measurable statistic and indeed one for which nearly complete data is available. However, U.S. life expectancy, about 78 years in 2009, turns out to be controversial when used to evaluate the nation’s health care system. Does the United States rank near the bottom of all developed countries? Or, if one omits early deaths primarily from homicides and accidents, does the United States have the longest life expectancy? Moreover, there are unexplained anomalies in the statistics, such as high longevity for Mexican immigrants, perhaps because some return to Mexico to die, or because the unhealthy are less likely to leave the land of their birth in the first place, or perhaps simply because this group is less likely to smoke than others.

Other statistical controversies presented in this book teach a similar lesson. For example, experts disagree about whether the death penalty deters murder, whether mortgage lending is affected by racial discrimination, and whether taxes are becoming more unfair. No book of any reasonable length could answer these or any of the many other policy questions raised in the following chapters. Rather, the intent here is to show why well-respected researchers are able to reach such contradictory conclusions. In some cases such understanding will help us decide which side is correct; often it is less important to decide which side is correct than to uncover the complex measurement problems that underlie a given issue.

Another purpose of this book is to help researchers, both students and more experienced practitioners, use social statistics correctly. Consider the following hapless case:

A social science researcher wanted to study the effect of military spending on jobs. Do communities with large military contractors benefit from increased employment, as advocates of military spending argue, or does military spending create relatively fewer jobs than other kinds of government spending, as critics of military spending have charged? To answer this question, the researcher obtained records of military contracts from the U.S. Department of Defense (DoD); the contracts were arranged by the city in which the contractor was located. To measure the number of jobs generated in a community by the presence of the military, the researcher consulted publications of the U.S. Bureau of Labor Statistics listing employment by location. Armed with a microcomputer statistical package and all the latest knowledge about statistical probability, the researcher was ready to punch in the numbers and find the answer to his question.

Then the project stalled. Just about every piece of data showed signs of being problematic. The DoD numbers were unusable because they listed contracts by the year in which they were awarded, which was not necessarily the year in which they were spent. To make matters worse, the location where the contract was awarded was not necessarily the location where people were hired; in fact, many contracts were subcontracted to other companies of unknown locale. There were problems with the employment data as well: When one employer dominated the industry, the data were not available on the grounds that the information would betray that company’s trade secrets. Finally, data from the U.S. Defense and Labor departments were incompatible because of different definitions of location—the “cities” and “metropolitan areas” under consideration in each survey were not necessarily the same.

One of us was the ill-informed researcher in this case. (Okay, actually it was Mark.) But we are not the first researchers to see good ideas flounder because of unusable data. It is a recurring complaint in the social sciences that researchers, from the student in training to the advanced scholar, do not know enough about the data they use. By examining the pitfalls encountered by previous researchers, this book will help today’s users of social statistics be more aware of which data sources are available and what limitations they possess. If—prior to undertaking the failed research on military spending—Mark had been aware of the Census Bureau confidentiality problems that frequently arise when researching corporations (see Chapter 10), he would not have expected to find employment data for large firms that dominate a single city’s industry. Similar examples will serve as cautionary tales for other researchers.

In summary, The Data Game is written for two groups: general readers and researchers. First, it will help everyone who is confused by statistics that seem to prove everything and anything. By sorting out the reasons behind seemingly contradictory statistics, we can better understand the issues under debate. Second, this book will assist researchers in assessing the problems of their underlying data. Without such knowledge, many social science projects will fail, as in the case of the military spending research; even worse, other projects will proceed without sufficient caution as to the data’s limitations.

How to Use This Book

Each of the chapters in this book is devoted to a single subject: demography; housing; health; education; crime; the national economy; wealth, income, and poverty; labor statistics; business statistics; government; and public opinion polling. Although students of a particular field will find the chapter in that area most useful, the book is intended to be read as a whole. Social scientists work within their own narrow specialty at considerable cost. Most projects use data from outside a narrow discipline, and the data may have limitations that are unknown to the researcher. For example, almost every area in social science measures variables on a per-person basis, a calculation that presupposes accurate population data, which is not necessarily a warranted assumption (as is discussed in Chapter 2 on demography). Similarly, geographic units such as Metropolitan Statistical Areas (Chapter 3) and corrections of price data for inflation (Chapter 11) are common throughout social science research. Thus it is useful for researchers to consult chapters beyond their specialty.

Each chapter in The Data Game opens with a brief overview of the “Data Sources” for a particular area of social science. These sections will acquaint readers with the names of the most important government and private-sector data sources, as well as the statistics they publish, and in many cases offer an illustrative “data sample.” The organizations, major publications, and websites for these data sources are listed in a table at the opening of each chapter.

Following the “Data Sources” section are “Controversies,” a series of debates about the use of statistics in each area. No attempt has been made to cover every debate in each field; instead, controversies were selected based on their relevancy to recent public policy disputes. These include controversies in the news, such as the U.S. Census population undercount, the disappearing middle class, and the number of homeless individuals in the United States. A second criterion for including a controversy was its usefulness as a revealing illustration of a statistical issue. For example, while the rating of individual cities as the best places to live or the lists of the nation’s largest corporations are not particularly critical policy questions, debates about these numbers teach important lessons on the use and misuse of ranking in social statistics.

All the controversies discussed by the authors obviously predate the publication of this fourth edition of The Data Game. But readers should resist the temptation to reject the examples from past years as out of date. Almost all the debates are ongoing, perhaps with different individuals or institutions, but still involving the same issues. As long as the underlying social and economic system remains the same, controversies based on fundamental measurement problems will stay with us.

Each chapter concludes with “Case Study Questions,” which instructors may assign to students for further investigation and the stimulation of thought about the issues raised in each chapter. In most cases there is no single correct answer to the questions; instead, they pose problems frequently encountered by researchers. In many instances citations related to the questions are provided as an aid to those readers interested in exploring underlying issues in greater depth.

Finally, readers should not overlook the “Notes” section at the end of each chapter. The works cited for each subject area are highly recommended guides to data sources, including both official government handbooks and privately published works. References tied to “Controversies” include popular presentations in magazines and newspapers, which are often the most accessible sources and are worth consulting to evaluate how the topic was generally understood—or misunderstood. In addition, there are references to summary reviews of each public policy debate; such reviews typically appear in academic journals. Citations for the key technical articles for each controversy are listed as well.

2

Demography

□□□□

Demography, the scientific study of population, provides researchers with some of the most fundamental social statistics. This chapter looks at demographic controversies surrounding the size of the population, the classification of individuals by race and ethnicity, household characteristics, immigration, and trends in cohabitation and divorce. These controversies have public policy implications for congressional representation, social security financing, affirmative action, and family law. In addition, because demographic data are used in so many areas of social science, the potential problems described here have implications for research outside the field of demography itself.

In the United States, the major source of demographic data is the U.S. Census, an attempt made every 10 years to count each individual, citizen or noncitizen, with or without legal documentation, who resides in the country. Less well known but similarly comprehensive are U.S. Vital Statistics that tabulate most births, deaths, marriages, and divorces. Researchers accustomed to surveys and the nagging problem of sampling error might wonder how there can be controversies about statistics based on complete data. This chapter identifies four major problems: (1) despite valiant efforts to be inclusive, not everyone is counted; (2) the categories used to classify race, ethnicity, and type of household are arbitrary and therefore subject to debate; (3) there may be multiple ways to define and measure variables such as the divorce rate, so care must be taken to match the definition to the question of interest; and (4) predictions about trends in demographic data are often based on assumptions that may not hold in the future.

---

Where the Numbers Come From

Organizations	Data sources	URL
Bureau of the Census, U.S. Department of Commerce	U.S. Census, American Community Survey	w­w­w­.c­e­n­s­u­s­.g­o­v­
National Center for Health Statistics, U.S. Department of Health and Human Services	U.S. Vital Statistics	w­w­w­.c­d­c­.g­o­v­/­n­c­h­s­/­
Office of Immigration Statistics, U.S. Department of Homeland Security	Records of admissions and naturalizations	w­w­w­.d­h­s­.g­o­v­

Data Sources

U.S. Census

Collected every 10 years since 1790, the U.S. Census is the longest-running consecutive data set in the world. It is also the world’s largest data set, compiling information about the sex, age, marital status, and race of nearly every individual residing in the United States. A number of surveys sponsored by the U.S. government use the Census as a statistical base, most notably the Current Population Survey (see Chapter 9).

Because of limited space, only a few questions can be asked on the short form filled out by all households. It was reduced in the 2000 Census to only eight questions and characterized as “the most efficient, cost-effective census in the nation’s history” by the then-acting director of the Census Bureau, James Holmes. In 2010, two additional administrative questions were added, but at 10 questions, it was still far shorter than any questionnaire sent prior to 2000. Nonetheless, every Census comes under fire for prying into individual privacy, a charge that often leads the Bureau to defend the importance of this data source.

Data Sample: In the 2010 U.S. Census, 77.3 percent of Hawaii residents reported a race other than non-Hispanic White, the highest percentage in the United States; the lowest minority percentage was 5.6 percent in Maine.

American Community Survey

For each decennial U.S. Census through 2000, about one in six households also received a “long form” asking 46 additional questions on such diverse matters as occupation and level of education. The long form was abandoned in 2010 and replaced with the American Community Survey (ACS). First administered in 2005, the ACS is a national survey that, on an annual basis, provides the same demographic, housing, social, and economic data that was previously collected only once per decade; three-and five-year average estimates are also provided for researchers. The ACS is a continuous survey, with questionnaires sent out every month, and the sample for annual estimates is much smaller than the decennial Census long form; note, however, that the ACS sample for the five-year estimates is not that different from the long form (one in eight for the ACS rather than one in six for the long form).

Data Sample: According to the American Community Survey, there were approximately 594,000 same-sex couple households in 2010; 42,000 of those were in the six states (including Washington, DC) that permit samesex marriages.

Vital Statistics

Most countries have a system for recording births, deaths, ma...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• List of Figures, Tables, and Boxes
• Preface to the Fourth Edition
• Acknowledgments
• 1. Introduction
• 2. Demography
• 3. Housing
• 4. Health
• 5. Education
• 6. Crime
• 7. The National Economy
• 8. Wealth, Income, and Poverty
• 9. Labor Statistics
• 10. Business Statistics
• 11. Government
• 12. Public Opinion Polling
• 13. Conclusions
• Index
• About the Authors