Researching Social Problems

Amir Marvasti, A. Javier Treviño, Amir Marvasti, A. Javier Treviño

• Publisher: Routledge
• Year: 2019
• eBook ISBN: 9781351608404
• Edition: 1
• Topic: Social Sciences
• Subtopic: Sociology
• Index: Social Sciences

This book covers a wide range of contemporary methods for researching social problems and connects these approaches to the broader substance and theories of social problems. Expository and discursive in approach, chapters follow a uniform structure, with each offering research examples and a broad description of the related method and its theoretical context, together with a "how-to" guide for applying that method using substantive examples from the field of social problems. For every method explored, there is a research example that fully reviews and illustrates the application of the particular method, before giving a full assessment of the method's strengths and weaknesses and latest developments. With chapters exploring survey interviews, in-depth interviews, narrative inquiry, institutional ethnography, participatory action research, auto-ethnography, Actor-Network Theory, experimental research, visual research methods, and research ethics, Researching Social Problems will appeal to scholars and students of sociology and politics working in the fields of research methods and social problems.

1

SURVEY RESEARCH

Asking questions for assessing social problems

Karen G. Weiss

Introduction

Survey research is used for many purposes, from measuring and describing human behavior and attitudes to explaining motives and consequences. By posing a series of questions to a select group of people, surveys provide a vast array of information to describe the scope and severity of social phenomena generalizable to larger populations. For this reason, surveys are an effective method used by sociologists and other social scientists studying crime, victimization, and other large-scale social problems. Indeed, the modern survey shares much of its history with an emergent scholarly interest in the social problems of the 20th century, including war, political unrest, racial discrimination, and gendered abuses, coupled with improved methodological procedures that accurately measure these issues (Platt 2014). As such, survey data has long contributed to the “construction of social problems” (Best 2013; Loseke 2003; Reinarman 2006) by providing the empirical evidence necessary to generate public concern, as well as to justify social policies needed to address the problems.

This chapter reviews the procedures and challenges of survey research for assessing social problems. It does so by highlighting the interactional and cognitive processes of the survey method. Surveys collect data through an interactive exchange between researchers who ask questions and respondents who provide answers (Gubrium and Holstein 2002; Suchman and Jordan 1990; Sudman, Bradburn, and Schwarz 1996). Like all social interactions, successful surveys depend upon shared meanings between the parties. Misunderstandings, inaccurate recall, and dishonesty on either side can complicate the exchange of accurate and reliable information. Thus, this chapter focuses on both the utility of surveys for social research, and the challenges that the survey method presents for accurate assessments of social problems.

The chapter begins with an overview of some basic survey design and sampling procedures. It then examines techniques for writing “good” questions to improve shared understandings, accurate recall, and honest responses. Examples of survey questions from prior studies are used throughout the chapter to illustrate survey design decisions and question-writing procedures, specifically related to operational definitions, recall, and memory reconstruction. The chapter concludes with a detailed example of an original survey project on physical altercations between college students that describes the implementation of a survey questionnaire.

Survey design overview

All surveys ask questions. How questions are asked and to whom questions are asked vary considerably based on operational decisions related to format, type of questions posed, administration of survey, and sampling. Depending on such decisions, surveys can be highly structured or more conversational, self-administered, or completed in person. Also, surveys can involve vast numbers of participants or small select groups. This section provides an overview of these basic design decisions.

Format and type of questions asked

Three types of formats are typically used by survey researchers: structured, unstructured, and semi-structured. Structured surveys (referred to as questionnaires) use a standardized set of questions that are precisely ordered. The rigidity of these surveys allows for data reliability (i.e., consistency) (Dixon, Singleton, and Straits 2016; Fink 2013). Large national data sets, including the General Social Survey, the National Crime Victimization Survey (NCVS), and the U.S. Census, use structured surveys. Data collected from these surveys, which are redistributed each year (or in the case of the Census, every 10 years), can be reliably compared across subpopulations and time periods (Fink 2013).

Semi-structured and unstructured surveys are more flexible than structured questionnaires and are typically used for smaller studies and those conducted face-to-face (often referred to as interviews). Semi-structured surveys start with a series of scripted questions but allow researchers the flexibility to change course to pursue an interesting or unexpected response by asking follow-up “non-scripted” questions. Unstructured surveys are the most flexible type of survey and use a more conversational format. Based on a protocol of broad questions (e.g., “Tell me about your life living on the street?”; “How is your community addressing drug addiction among its residents?”), these surveys mimic everyday conversation, and are typically used for ethnographic studies that seek in-depth description about a place, practice, or group of people (Creswell 2014: 14). Both semi-structured and unstructured surveys allow researchers and respondents to ask for clarification and elaboration during the survey process, increasing shared meanings and more valid (i.e., conceptually accurate) responses (Creswell 2014: 201). The tradeoff, however, is that questions and responses from unstructured surveys are less consistent from one respondent to another, making results less reliable or generalizable to broader populations.

Corresponding with the survey format is type of questions asked. Structured and most semi-structured surveys typically use closed-ended questions that require respondents to select answers from a limited number of categories (Fink 2013: 32). The NCVS, for example, asks victims who do not report offenses to police to select their most important reason for not reporting from a series of predetermined options that include minor or unsuccessful crime, police wouldn’t do anything, and did not want to get offender in trouble (US Census, BJS 2000). The advantage of closed-ended responses is that they can be quickly coded or converted into numbers for statistical analysis. Still, the selection of appropriate response categories often requires more preparation on the front end to ensure that options are exhaustive (i.e., that they include all possible answers) and relevant, meaning that they include the most important concepts related to theory or research objectives (Creswell 2014).

Unstructured surveys, in contrast, tend to use open-ended questions that allow (or require) respondents to answer “openly” rather than choosing from a limited set of response options. Open-ended questions might simply ask, “How were you injured?” or “How did victimization impact you?” Such questions permit more diverse and unanticipated answers. They also tend to be less restrictive or leading than closed-ended questions. For example, when respondents in one survey were asked to indicate the most important concept that children should learn to help them prepare for life, 62 percent of respondents selected “to think for themselves” from a list of five pre-determined responses. But only 5 percent of respondents mentioned thinking for themselves in an open format (Schuman and Presser 1981, as cited in Clark and Schober 1992: 33).

Sometimes open-ended questions are used in structured questionnaires for purposes of clarifying closed-ended responses. In these instances, responses are generally left in their qualitative form to be integrated into analysis as quotes or anecdotes that illustrate how respondents think about an issue in their own words. Less often, responses are analyzed quantitatively by numerically “coding” phrases or patterns observed (Fink 2013). In my own research on sexual victimization (see Weiss 2006, 2010, 2011), I used this process to analyze open-ended responses from a final question in the NCVS that asked respondents to describe their victimization experiences in detail. For example, I coded the data for the presence or non-presence of predetermined indicators of self-blame and shame (e.g., it was my fault, I should have known better, I was too ashamed to talk about it). The coding process allowed me to calculate the frequencies of shame among victims of sexual assault, while at the same time, identifying exemplary quotes to use in my analysis to underscore the impact of shame on victimization.

Administration of survey

Surveys are typically administered in one of two ways: as self-administered questionnaires (paper or web-based) that respondents complete on their own, or as in-person interviews where researchers write down on paper, type into a computer, or transcribe from a tape recorder, respondents’ answers. Many large-scale studies use self-administered questionnaires that can be inexpensively disseminated (via mail, email, or social media websites such as Facebook or Twitter) to vast numbers of people from diverse geographic areas. A disadvantage of self-administered surveys, however, is that neither researcher nor respondent can obtain clarification of meanings, which can lead to misunderstandings or excessive guessing on the part of respondents (Kovar and Royston 1990).

By allowing for clarification, in-person surveys can reduce misunderstandings. Yet, in-person surveys have their own unique challenges related, ironically, to their resemblance to normal conversations. While ordinary conversation involves bidirectional participation, survey respondents often have little say in the direction of talk (Clark and Schober 1992). Lucy A. Suchman and Brigitte Jordan argue that the asymmetrical flow of survey exchanges violates expectations of normal and meaningful conversation (1990: 232–33). As such, respondents may react with boredom, withdrawal, or impatience, resulting in careless responses “to get it over with.”

In-person surveys are also more likely to be influenced by formed impressions based on visual cues, such as how someone is dressed, physical attributes, body language, or facial expressions (Babbie 2001: 259; Sandstrom, Martin, and Fine 2006: 39). Research conducted over the telephone may reduce visual bias while still providing the benefits of actual verbal exchanges. Phone surveys are also cost-efficient, as they can reach wide-spread populations from across the country without requiring researchers to travel. Researchers simply dial from a call center (or from their homes), saving time and money. Perhaps for this reason, phone surveys are commonly used for national surveys and polls that ask “average” Americans to answer questions about some specific behavior (e.g., commute time to work, exercise habits) or opinion (e.g., attitude towards legalizing marijuana, support for a tax bill).

Phone surveys, however, tend to have lower response rates than in-person surveys. A response rate is a guide to how well respondents represent the population being studied (Dixon et al. 2016; Fink 2013). A higher response rate, based on a numerical calculation of completed surveys divided by participants invited to take the survey, increases representativeness. Current response rates for phone surveys average at about 50 percent, with slightly higher rates (closer to 70 percent) for respected national surveys such as the General Social Survey or Gallop Poll (Babbie 2001; Cook, Heath, and Thompson 2000; Fowler 2009). While low response rates for phone surveys were once attributed to persons needing to be at home to answer calls and participate, today, with caller identification, fewer persons are willing to answer calls from unknown sources (Fink 2013: 13). Thus, many survey invitations, even from well-known organizations, are frequently dismissed as unwanted or unsolicited phone calls.

Online surveys, administered by email or though social media, have perhaps even lower response rates, with some studies suggesting numbers ranging from only 11 to 35 percent when no incentive is provided (see Cook, Heath, and Thompson 2000). Online surveys also have lower completion rates, meaning that people who start a survey tend to quit before answering all the questions. In-person surveys, in contrast, have the best response and completion rates. Respondents tend to be more willing to participate and complete surveys when asked to do so in person. This may be why election surveys or “exit polls” often combine in-person and self-administered techniques. For instance, voters are asked in-person to participate in a survey about their voting preferences to increase participation rates but are then given a questionnaire to complete in private (Babbie 2001).

The NCVS is another example of a survey administered by mixed techniques which, in this case, is in-person and by phone. Respondents who participate in the NCVS are interviewed every six months for three years, with initial interviews conducted in-person to establish rapport. Subsequent interviews are conducted by phone, utilizing computer-assisted telephone interviews (CATI) from a centralized facility where interviewers read questions to respondents and input responses into a computer (U.S. Department of Justice n.d). A particular benefit of CATI software is that it is designed with automated consistency checks that alert interviewers whenever there are sequencing problems or responses that need clarification. This method helps to reduce misunderstandings and achieve better accuracy.

Sampling

No matter how well designed a survey is, accurate data collection ultimately depends on finding an appropriate group of people—a sample—willing to participate in the survey. Ideally, a sample should accurately represent the larger population for whom researchers seek to study. Obtaining such a representative group typically means using probability sampling techniques that include simple random sampling (i.e., each person has an equal chance of being selected), stratified random sampling (i.e., populations are divided into subgroups and then a proportion of respondents are selected from each stratum), or systematic sampling (i.e., starting randomly, respondents are then selected by systematic counting, such as using every 10th name) (Fink 2013). Most large quantitative studies use probability sampling to increase representativeness and generalizability of findings to broader populations. Findings from probability samples also allow researchers to use advanced statistical analyses that assume randomization (Creswell 2014).

Most small qualitative studies use nonprobability (or convenience) samples that select participants from groups readily or conveniently available, such as at rallies, public parks, schools, or on social media websites. Two of the more common types of nonprobability samples are purposive samples (i.e., cases are selected to specifically represent a unique population to be studied) and snowball samples (i.e., cases are selected by referrals or word of mouth) (Dixon, Singleton, and Straits 2016). Purposive samples are especially helpful for studies of unique social phenomena, such as rare diseases, where a randomized sampling frame would be unwieldy and often ineffective for finding the small number of persons able to answer questions relevant to the survey’s topic.

Though convenience samples are not typically appropriate for quantitative studies, they are sometimes used for exploratory research that seeks to provide preliminary data for future and larger randomized studies. On occasion, convenience samples are also used for larger quantitative studies, despite cautions that findings may not be generalizable to larger populations. For instance, researchers studying caregivers for the elderly compared results from a survey that used both a convenience sample and a probability sample drawn from random-digit dialing methods (Pruchno et al. 2008). Results showed significant differences between the two samples, including higher levels of depression among caregivers from the convenience sample, suggesting that non-probability samples may overrepresent behaviors or attitudes being studied.

Concerns with overrepresentation are certainly nothing new, nor are the controversies that can ensue. In fact, such issues were made evident as far back as the late 1940s with the publication of Alfred C. Kinsey’s classic study on men’s sexuality. Based on a large convenience sample of more than 5,000 men, mostly white and many recruited from known gay groups, Kinsey concluded that one in ten American men identified as gay (with more than a third having had some overt homosexual experience) (Kinsey Institute n.d). Skepticism of the findings was immediate, and critics pointed to a potential bias from...

Table of contents

• Cover
• Half Title
• Series Page
• Title Page
• Copyright Page
• Dedication
• Table of Contents
• List of figures
• List of contributors
• Foreword: the splendid dangers and worthy methods of the social pathologist
• Introduction
• 1. Survey research: asking questions for assessing social problems
• 2. Research interviews: measuring, feeling, and constructing social problems1
• 3. Narrative inquiry: stories and storytellers of social problems
• 4. Institutional ethnography: a mode of inquiry and a strategy for change
• 5. Participatory action research: re-imagining the study and transformation of social problems
• 6. Autoethnography as a methodology in researching social problems
• 7. Considering materiality: the utility of Actor-Network Theory to study social problems
• 8. Systematic experimentation on social problems
• 9. Visual research methods: integrating images in the study of social problems
• 10. The ethics of the study of social problems
• Index