The Science of Self-report

Implications for Research and Practice

Arthur A. Stone, Christine A. Bachrach, Jared B. Jobe, Howard S. Kurtzman, Virginia S. Cain, Arthur A. Stone, Christine A. Bachrach, Jared B. Jobe, Howard S. Kurtzman, Virginia S. Cain

• Publisher: Psychology Press
• Year: 1999
• eBook ISBN: 9781135677404
• Topic: Psychology
• Subtopic: Cognitive Psychology & Cognition
• Index: Psychology

Rigorous methodological techniques have been developed in the last decade to improve the reliability and accuracy of self reports from research volunteers and patients about their pain, mood, substance abuse history, or dietary habits. This book presents cutting-edge research on optimal methods for obtaining self-reported information for use in the evaluation of scientific hypothesis, in therapeutic interventions, and in the development of prognostic indicators.

ALTERNATE BLURB:
Self-reports constitute critically important data for research and practice in many fields. As the chapters in this volume document, psychological and social processes influence the storage and recall of self-report information. There are conditions under which self-reports should be readily accepted by the clinician or researcher, and other conditions where healthy scepticism is required. The chapters demonstrate methods for improving the accuracy of self-reports, ranging from fine-tuning interviews and questionnaires to employing emerging technologies to collect data in ways that minimize bias and encourage accurate reporting.

Representing a diverse group of disciplines including sociology, law, psychology, and medicine, the distinguished authors offer crucial food for thought to all those whose work depends on the accurate self-reports of others.

PART I
GENERAL ISSUES IN SELF-REPORT

Jaylan S.Turkkan
The National Institute on Drug Abuse,
The National Institutes of Health

In this section, chapters by Wendy Baldwin and Donald Bersoff introduce broad issues in the use of self-report in research settings. Baldwin reminds us that even so straightforward a question as asking someone their address is fraught with potential for misunderstanding; in virtually all cases, however, we are willing to take the participant’s word for where they live. In many other protocols, we feel the need to gather additional data not based on selfreport. Baldwin makes the case that self-report data should not be thought of as mere proxies for other better verification methods. Indeed sometimes, as in past sexual behavior, there are no better measurement methods. In the case of past drug use also, until hair testing and other biologic verification techniques are completely validated, information on distant drug use will need to be primarily ascertained via self-report. Even where biologic or registry information is available, often it is too expensive, obtrusive, or time consuming to obtain. Consider also that biologic and other data can be as prone to false positives, false negatives, and other inaccuracies as self-report data.

Bersoff’s chapter reminds us that whenever we gather information from anyone, there are special ethical considerations that we must be more mindful of. When these data are collected from children and adolescents, or from adults who may not be fully capable of understanding the research protocols, additional sensitivity and care must be provided. Although most if not all major psychological and social science organizations have published clear ethical codes of conduct, subtle instances of coercion such as the slippery slope phenomenon abound. Moreover, well-meaning researchers are understandably confused when they uncover sensitive or criminal information from minors during research protocols. Should they reveal this information to parents? If they do, will they be violating the minor’s rights to privacy and confidentiality? These are questions for which there are no easy answers. Bersoff’s chapter expertly leads us through the maze of ethical dilemmas that can arise when asking research participants about their experiences and opinions.

CHAPTER 1
Information No One Else Knows: The Value of Self-Report

Wendy Baldwin
The National Institutes of Health

A volume on self-report is an opportunity to address methodological issues as well as fundamental issues of why and when we expect to use such data. I believe there is an inherent mistrust of self-report data because such data could be erroneous, not just because of measurement error with which we must always contend but because of the possibility of conscious bias in the person providing the data. Presumably, a desire to look good could distort data either intentionally or unintentionally.

However, self-report data are essential to behavioral and medical research. Also, such data are essential to medical practice, because self-report is the heart of the careful medical history that underlies so much of diagnosis and care. Therefore, it could be argued that it is an obligation of the behavioral community to help make the collection of these data the most robust that it can be.

Let me lay out two situations where we typically encounter self-report data: first, there are cases where other data exist, but they are too difficult to obtain or the costs of obtaining them are not worth the effort; then there are the cases where there is no other source for the information. The first case is one that should challenge us to clarify our underlying definitions, to consider how much error we can tolerate, to elaborate the possible incentives one might have to distort answers, and to raise our awareness about how we communicate with research participants. We routinely ask research participants or patients simple questions such as where they live. We do not follow them home and we do not usually get a verification, such as a piece of mail addressed to them. We take their word for it. Sometimes they lie. If we followed them home, we might get the wrong answer anyway; perhaps they are just staying with friends and that is not really home. Perhaps our definition of home is where they typically do live, not where their spouse still lives 100 miles away or vice versa. But, unless we are sending a bill there and fear they will not get it—and pay it—we simply ask the question and take the answer.

I am sure the readers are way ahead of me now. What about research looking at access to services? What if the hypothesis depends on measures of family and social supports? What if the environment is key to the problem of study? Then the error might be intolerable. In that case, I would suggest that it is still a case for self-report, but with better definitions and clearer questions to the participant. The correctness of the answer depends in large part on the clarity and precision of the question.

Let me take another case. There may be data that exist independent of the subject, and we would like to have them, but it is not clear that it is worth the effort to obtain them independently. This brings us to consider the distinction between what people know and may or may not tell you and what they actually do not know. Many studies of pregnancy outcome require information about birth weight of the infant. Most women, I contend, can tell you birth weight with pretty good accuracy. However, if you require great precision, this might not be enough, and if you require information about the baby’s length, it probably will not do at all. Why? One piece of information—weight—is more salient to most women than the other—length. This situation, and others like it, provide interesting research opportunities to understand self-reporting. When we know we can find out some pieces of information from independent sources, we are able to compare what we get from self-report and what those other sources provide, birth weight of babies being one example. The researchers still must decide how much precision they need.

This should then position us to identify the strategies that enhance the reporting of information people already know. What are the prompts that assist recall? For example, timelines of life events and historic events can help individuals to report on activities in their own lives. We can also calibrate the level of accurate reporting needed: a category for birthweight, such as under 5½ pounds, may be very accurately reported, whereas specific pounds-ounces might not be. One wonderful example of the use of self-report data to identify a problem is the paradox of Mexican-American immigrants. Their infants have higher birth weights generally than do infants born to those who have lived in the United States for several years, even Mexican Americans born in the United States. This finding was originally based solely on self-report data but has been shown to be quite reliable and robust, with many studies and surveys (and National Center for Health Statistics data) confirming that the phenomenon exists. Validating that certain information can be obtained through self-report is important because it opens up avenues of research where self-report data are the only data available.

We will need self-report data for items where we theoretically could get independent measures as well as where we cannot get independent measures; the former provide the research opportunities to develop our skills for the latter. What are the situations where we cannot reasonably obtain independent measures? Are they important and are they amenable to self-report? An obvious one is sexual behavior. There are numerous health issues that are confronting researchers with the need for information about their research participants, intimate behaviors that cannot be obtained in any way other than asking them: the number of partners, sexual practices, protective practices, interactions with potential partners, and more. This information may be key to understanding risk of pregnancy and sexually transmitted diseases, including those that may be lethal (such as HIV) as well as those that can have longer-term health consequences, such as human papilloma virus (HPV), which is linked to cervical cancer. These are significant pieces of relevant information. Information about what happens during sexual intercourse has no independent source; the partner is just another source of self-report. This information may be critical to understanding risk or changes in behavior or to finding opportunities for intervention.

Arguments abound over an individual’s desire to present a favorable image, so we can easily hypothesize why men and women might differentially misreport data about sexual partners. Again, the necessary level of candor should be considered. If one is doing case finding in a clinical setting, missing even one partner could be a significant problem. If one is categorizing research subjects into broad categories of sexual risk taking, missing one partner could be irrelevant. In either case, we are still left with self-report data but researcher anxiety might be significantly less.

How about incentives to misreport? In the area of sexual behavior, the criticism of self-report data has included the observation that individuals are not always candid with their sexual partners; yet this observation may be largely irrelevant to the research experience. People may recall personal situations where they gave less than accurate information and therefore doubt that others will be candid about such personal information with an interviewer. However, the interview situation is fundamentally different from most personal encounters. In the interview, there is anonymity, confidentiality, and a contribution to make to science, if not to one’s own welfare. The dialogue about sex in a singles’ bar probably has a motive to it that does not apply to an interview. The interviewer is not going home with the respondent regardless of what is reported. The personal experiences told to an interviewer are used anonymously and, in the aggregate, are a far cry from having shared an intimacy with a family member whom one sees year after year. Because nonresearchers may be slow to make these distinctions, the research community must do so.

Sexual behavior is perhaps the most obvious arena where no independent source of data exists, but it is by no means the only one. Any process that involves a behavior over which the individual has a high degree of control is likely to require self-reporting of that information. What kinds of data would that be?

Nutrition in food consumption is difficult to measure, but short of direct observations, we need self-report. The same is true of other behaviors, such as the use of automobile seat belts, bicycle helmets, the ingestion of herbal remedies. Even where external data theoretically exist, self-report may differ significantly from information in registries because people may tell an interviewer things they may be reluctant to share with legal authorities. Motivation for behaviors is another example where the individual is the only source of information, such as what may have prompted certain criminal behaviors or sexual activities, illicit drug use, even migration.

Another area where self-report is often the only or best source of information is health behaviors. Take exercise. The questions relate not just to the effect of exercise in the lab setting but how well individuals incorporate it into their daily lives, how they cope with disruptions in their schedules. And consider health behaviors during pregnancy such as smoking, consumption of alcohol or vitamins, or other specific nutritional practices. Another example is characteristics of the environment, such as exposure to physical and psychological threats. We are beginning to look at young children’s exposure to violence in their community—exposure only they can report. We know the behavioral components to many of the top diseases in the United States and we know that behavioral factors dominate the new morbidity, but the reach of health behavior issues is far wider.

Reports on global health indicate areas for research and health service attention. The concern here is not just something the United States has come to confront, but risks facing the world. What is the role for self-report there? Let me start with one high-priority area: maternal and child health. The risks here are tied not just to those associated with unsafe sexual practices (already discussed) but also the practices related to delivery (use of midwives or traditional birth attendants), use of herbal remedies or other folk practices, or specific pregnancy-related dietary practices found in some cultures. Even if there were high-tech ways to develop independent measures, health risks are greatest in geographic areas where the resources for health care and health research are the lowest. The high technology measures would be of little use and we can easily foresee the need for robust measures of self-report.

There are also risks from emerging and reemerging infections. Two noted examples are tuberculosis (TB) and malaria, each with a behavioral component. Adherence to medications for TB is a well-recognized concern. And compliance in the use of medicated mosquito nets for malaria prevention is another area of research and practical interest. As affordable interventions are developed, researchers will be dependent on self-reports of compliance as well as reports of additional risk behaviors. In addition, risks from noncommunicable diseases will be a growing threat to world health: heart disease, respiratory diseases, injury, and violence—all have some behavioral components. Injuries are a growing threat not only in the United States but in emerging economies. Our need for epidemiologic information will require self-report data about risk exposures, such as interpersonal, vehicular, and environmental. The risks from inequitable health services carry implications for research, and researchers need to understand the behavioral factors that influence people’s use of services, factors that are largely visible to us only through self-report.

There is concern that use of mortality alone—a measure relatively free of self-report or measurement errors—is inadequate to assess the burden of disease or priorities for research or distribution of health services. Alternative measures frequently require attention to conditions that disable, not just conditions that kill. The measurement of disability is rather difficult and often dependent on self-report. Even where independent, technologically based measures exist, it is unlikely that they will gain widespread use in countries with emerging economies. There is an enormous need for measures that are simple and robust and many of these will be self-report measures.

In summary, the problems with self-report data are similar to those we face with other forms of data collection: clarity in underlying premises, circumstances for which and under which the data are being used, uniformity of data collection procedures, the need to estimate likely bias, and there are others. But improvements can come and they will benefit almost every disease area, in both research and practice.

I cannot give an exhaustive list of the keys to better self-report data. That is the purpose of the other chapters. However, I have tried to share my observations of health problems within the United States and around the world and the role that self-report data can play, a role that is important to our learning as much as we can about these problems as we seek information that can lead to improved interventions and, ultimately, to some solutions.

CHAPTER 2
Ethical Issues in the Collection of Self-Report Data

David M.Bersoff
Diogenes Project

Donald N.Bersoff
Villanova University
Medical College of Pennsylvania—Hahnemann

There are very few publications that focus on the ethical issues confronting those who gather self-report data in a research context (see LaGreca, 1990). Perhaps one reason for this lack of attention is that, compared to naturalistic observation and deception-based research, self-report surveys seem rather benign. Self-report measures are usually fairly straightforward, and a willingness to fill out a survey or answer a researcher’s questions implies a consent to participate in the study being conducted. In contrast, unobtrusive observations and deception studies can not, by their nature, be fully consented to without compromising the data collected, and people in such studies may not even be fully aware that they are research participants. Despite these superficial differences, self-report data collection is fraught with just as many ethical concerns, albeit of a somewhat different nature, as experimental and observational research.

Beyond the issues of privacy, confidentiality, and informed consent that are present in almost all psychological research, there are ethical issues that are somewhat unique to self-report data collection. In particular, we focus on the potential self-report research has to glean clinically sensitive data, often putting that data in the hands of nonclinicians. In addition, self-report measures can be used to collect data about third parties who have not directly or even indirectly consented to be part of the research and who may not want such information revealed to others. Of course, there is a price to be paid for adhering to sound ethical practices, and we end with a consideration of the potential costs of placing more stringent controls on the collection of selfreport data.

PROTECTING BASIC RIGHTS

Informed Consent

We find many of the current practices used in the collection of self-report data to be generally inadequate in terms of protecting people’s basic rights. First of all, most survey research is exempted from Human Subjects Committee (IRB) review and is not required to have a formal consent procedure (see CFR §46.101, 1996). Currently, IRB review and formal consent are mandated for survey and interview procedures only if subjects can be uniquely identified from their data and/or if disclosure of their “responses outside the research could reasonably place subjects at risk of criminal or civil liability or be damaging to the subject’s financial standing, employability or reputation” (45 CFR §46.101 (b2)). There are two problems with these criteria.

The first is that risky or potentially damaging responses are not always foreseeable. Open-ended questions are quite common in self-report research and often a researcher simply cannot predict the type of information that may be gathered. A study that involves thought sampling (a technique in which subjects are beeped at random times during the day and asked to record their thoughts at that moment), for example, could just as easily yield someone’s shopping list as it could a suicide ideation or a passing violent fantasy. Asking a person to relate a life-altering experience could result in a story about being lost in a shopping mall as a child as well as a date rape.

Second, these federal guidelines are less stringent than the ethical requirements of the American Psychological Association (APA), and other major social science research organizations, that mandate that researchers “inform participants of the nature of the research…[and] of significant factors that may be expected to influence their willingness to participate …[including] limitations on confidentiality” (APA, 1992, p. 1608). This mandate to inform potential participants about the nature of one’s research still holds even if the information to be collected in a study would not necessarily put the person at risk of liability or be damaging to his or her financial standing, employability, or reputation. In addition, risk is not the only factor that can potentially and significantly influence an individual’s willingness to participate in a study (Singer & Frankel, 1982), and thus, risk should not be the primary or sole basis on which to determine the necessity of having a formal consent procedure.

For these reasons, we suggest that all survey research have some sort of informed consent. At a minimum, potential respondents should be informed, even if only orally, of (a) the topic and purpose of the research, (b) the nature of the questions to be asked, (c) the time required for participation, (d) their right to skip individual questions or to withdraw from the study at any time without penalty, (e) the limits on the confidentiality of the data collected, and (f) who they can contact if they have any concerns or questions regarding the study.

Participating in research represents an investment of time and energy on the part of respondents. They are entitled to know the size of the time commitment they are being asked to make as well as the significance of the issue being examined, both of which may be key factors influencing people’s willingness to take part in a given study (Singer & Frankel, 1982). If fully revealing the nature or purpose of a study, although not necessarily the particular hypotheses being tested, would compromise the data collected, then the study involves, if not a deception, at least a withholding of standard consent form information from the subject. As a consequence, assuming IRB approval of this withholding of information, respondents should at least be offered a debriefing as they would in a laboratory experiment that entailed some sort of deception (Holmes, 1976).

Some populations such as children or the mentally ill may not be considered legally competent to give informed consent. In such cases, consent will have to be obtained from a legal guardian. ...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Preface
• Part I: General Issues in Self-Report
• Part II: Cognitive Processes in Self-Report
• Part III: Self-Reporting Sensitive Events and Characteristics
• Part IV: Special Issues on Self-Report
• Part V: Self-Report of Distant Memories
• Part Vi: Self-Reporting of Health Behaviors and Psychiatric Symptoms
• Part VII: Self-Reporting Of Physical Symptoms