An Introduction to Survey Research, Volume I

The Basics of Survey Research

Ernest L. Cowles, Edward Nelson

• Publisher: Business Expert Press
• Year: 2019
• ISBN:
  9781948976046
• Edition: 2
• Topic: Betriebswirtschaft
• Subtopic: Entscheidungsfindung

Survey research is a powerful tool used in business, health care, government, and other fields that seek to understand how and why individuals behave the way they do. Properly conducted, surveys can provide accurate insights into areas such as attitudes, opinions, motivations, and values that serve as the drivers of individual behavior.

This two-volume set is intended to introduce fundamentals of good survey research to students and practitioners of the survey process as well as end users of survey information. It describes key survey components needed to design, understand, and use surveys effectively and avoid the pitfalls stemming from bad survey construction and inappropriate methods. In this first volume, the authors concentrate on the fundamentals of survey development and design as well as provide a review of key components in survey.

CHAPTER 1

Introduction

Research starts with a question. Sometimes these are why questions. Why do some people vote Democrat and others vote Republican? Why do some people purchase health insurance and others do not? Why do some people buy a particular product and others buy different products? Why do some people favor same-sex marriage and others oppose it? Why do some people go to college and others do not? Other times they are how questions. If you are a campaign manager, how can you get people to vote for your candidate? How could we get more people to purchase health insurance? How could you get customers to buy your product? How could we convince more people to go to college? But regardless, research starts with a question.

Have you thought about how we go about answering questions in everyday life? Sometimes we rely on what people in authority tell us. Other times we rely on tradition. Sometimes we use what we think is our common sense. And still other times we rely on what our gut tells us. But another way we try to answer questions is to use the scientific approach.

Duane Monette et al. suggest that one of the characteristics of the scientific approach is that science relies on systematic observations.¹ We often call these observations data and say that science is empirical. That means it is data based. However, the scientific approach doesn’t help you answer every question. For example, you might ask whether there is a God, or you might ask whether the death penalty is right or wrong. These types of questions can’t be answered empirically. But if you want to know why some people vote Democrat and others vote Republican, the scientific method is clearly the best approach. Relying on what people in authority tell you or what tradition tells you or your gut won’t work.

Research Design

Your research design is your plan of action. It’s how you plan to answer your research questions. Ben Jann and Thomas Hinz recognize the importance of questions when they say that “surveys can generally be used to study various types of research questions in the social sciences.”² The research design consists of four main parts—measurement, sampling, data collection, and data analysis. Measurement is how you will measure each of the variables in your study. Sampling refers to how you will select the cases for your study. Data collection is how you plan to collect the information that you will need to answer the research questions. And data analysis is how you plan to analyze the data. You need to be careful to decide on your research design before you collect your data.

In this book, we’re going to focus on data collection and specifically on surveys. The book is organized in two volumes. In the first volume, we’ll focus on the basics of doing surveys, and we’ll talk about sampling, survey error, factors to consider when planning a survey, and the different types of surveys you might use. In the second volume, we’ll focus on carrying out the survey, and we’ll talk about writing good questions, the actual carrying out of surveys, the impacts of current technology on survey research and survey reporting.

Observation and Questioning

Irwin Deutscher succinctly summarizes the different ways we collect data: “(1) we can observe it in process; (2) we can view the records men [and women] leave behind . . .; and (3) we can ask questions and listen to answers.”³ In this chapter, we’re interested in two of these approaches—observation and questioning.

Matilda White Riley makes the following comment about observation and questioning, noting that one method is not inherently superior to the other but that observation and questioning focus on different aspects of the social setting we are studying.⁴

Researchers sometimes feel—mistakenly, we believe—that they can obtain a true picture of a social phenomenon only if they observe it with their own eyes. To be sure observation and questioning often give different results; but this occurs, not because one method is more valid than the other, but because the two focus . . . on different sets of social system properties.

Observation and questioning give us different information about what is going on in the world. Observation gives us information about what people do. Questioning gives us information about what people say and the context to help interpret their observations.ⁱ This suggests that we often need both observation and questioning to give us a complete picture of what is happening and why it happens.

Elliot Liebow, in his book Tally’s Corner, provides a clear example of these two different approaches to data collection.⁵ Liebow studied a group of men who hung out on street corners in Washington, DC. He notes that “men and women talk of themselves and others as cynical, self-serving marauders, ceaselessly exploiting one another as use objects or objects of income.”⁶ The men in Liebow’s study “are eager to present themselves as exploiters to women as well as to men.”⁷ In other words, this is what they say. He goes on to say that “in practice, in their real relationships with real women, the men frequently gave the lie to their own words.”⁸ This is what the men do. So how does Liebow explain this apparent contradiction between what men say and what they do? He suggests that there are two opposing impulses at work. “The impulse to use women as objects of economic or sexual exploitation is deflected by countervailing impulses and goals, especially the desire to build personal, intimate relationships based on mutual liking and love.”⁹ The apparent contradiction between what the men say and what they do is explained by the “interplay of these opposing impulses.”¹⁰

Let’s consider another example. You’re doing a market research survey for a company that manufactures condoms. You want to know whether people purchase condoms and the particular brands they buy. It’s easy to imagine a discrepancy between what people say and what they do. Some people might be embarrassed to give you this information, and others might feel that it’s none of your business. What people say might not accurately reflect what they do.

Even though we see that observation and questioning give us different information about the world, we are still surprised when there is a lack of consistency between what we learn from observation and from questioning. Deutscher, in his book What We Say/What We Do, describes an early study by Richard LaPiere.¹¹ In the early 1930s, LaPiere traveled across the United States with a Chinese couple. They ate together in restaurants and stayed at hotels and autocamps and were refused service only once, and this was during a time in the United States when there was considerable prejudice toward Chinese. Six months later, LaPiere sent a questionnaire to these same hotels and restaurants asking the following question: “Will you accept members of the Chinese race as guests in your establishment?”¹² He describes the results of his survey as follows:

With persistence, completed replies were obtained from 128 of the establishments we had visited; 81 restaurants and cafes and 47 hotels, autocamps, and “Tourist Homes.” In response to the relevant question, 92 percent of the former and 91 percent of the latter replied “No.” The remainder replied “Uncertain, depends upon circumstances.”¹³

So what are we to make of this? Is this an example of the inconsistency between what people say and what they do? Or does it simply reflect that observation and questioning are telling us different things about the world? LaPiere’s classic study sparked a great deal of interest and follow-up studies. Howard Schuman, in his book Method and Meaning in Polls and Surveys, describes a study that he and Robert Brannon carried out in 1969.¹⁴ He refers to this as “an attitude–behavior field study.”¹⁵ In a survey, respondents were asked their opinion of open-housing laws. Here’s the question they were asked and the percentage of respondents giving each answer.¹⁶ (DK stands for don’t know and NA for no answer.)

	Suppose there is a community-wide vote on the general housing issue. There are two possible laws to vote on. Which law would you vote for?
1.	One law says that a homeowner can decide for himself whom to sell his house to or even if he prefers not to sell to blacks.	82%
2.	The second law says that a homeowner cannot refuse to sell to someone because of their race or color.	16%
DK, Neither, NA		2%
Total		100%
N		(640)

Three months after the survey, the same respondents were asked to sign a petition. One of the petitions supported the first law and the second petition supported the other law. Those who said they would sign the petition were then asked if they would be willing to have their name appear in the newspaper as a supporter of that petition. Schuman summarizes the overall consistency between what people said and what they were willing to do for those opposed to the open-housing law: “85 percent were consistent in signing the Owner’s Rights petition, and 78 percent were consistent in refusing to sign the Open Housing petition which gives an overall average of 82 percent consistency.”¹⁷ The same type of consistency was also found for those who supported open housing. Schuman concludes that in this study “attitudes can predict behavior to a reasonable extent, though of course not perfectly.”¹⁸

In a more recent study, Eleanor Singer et al. studied the “impact of privacy and confidentiality concerns on participation in the 2000 Census.”¹⁹ This is another example of what Schuman referred to as the attitude–behavior question. Their analysis found that attitudes toward confidentiality and privacy were significantly related to behavior (i.e., returning the census form). It’s interesting that they also report that other researchers found that “many more people . . . say they would not provide their SSN [Social Security number] to the Census Bureau than actually fail to provide it when it is asked for on their census form.”²⁰

There are many more examples of the attitude–behavior issue, but these are sufficient to show that sometimes people behave in a way that is consistent with what they say and that other times what they say is different from what they do. As Liebow pointed out, there are various factors affecting both what people say and what they do, and it is the interplay of these factors that eventually determines the outcome. For our purposes, it is important to keep in mind that observation...