The focus of this book is on original reports of research found in academic journals. We say they are original because they are the first published accounts of particular sets of research findings. As such, they are considered primary sources of information, detailing the methodology used in the research and in-depth descriptions and discussions of the findings. In contrast, research summaries reported in textbooks, popular magazines, and newspapers, as well as on television and radio, are usually secondary sources, which typically provide only global descriptions of results with few details of the methodology used to obtain them. Furthermore, secondary sources are often incomplete, sometimes inaccurate, and their purpose tends to be more to garner casual readers’ interest than to engage scholars’ consideration and scrutiny. As scholars, you will want to emphasize primary sources when you review the literature on a particular topic. In fact, your instructor may require you to cite primary sources exclusively in your written reviews of literature.
Journals in the social and behavioral sciences abound with original reports of empirical research. The term empirical refers to observation, while the term empirical research refers to systematic observation. Research is systematic when researchers plan whom to observe, what characteristics to observe, how to observe, and so on. While empirical research is the foundation of any science, one could reasonably argue that all empirical research is inherently flawed. Hence, the results obtained through research should be interpreted with caution. For instance, the following is a list of three major issues that arise in almost all empirical studies and the problems they pose for reviewers of research.
■ Issue 1: Sampling. Most researchers study only a sample of individuals and infer that the results apply to some larger group (often called the population). Furthermore, most researchers use samples with some kind of bias that makes them unrepresentative of the population of interest. For instance, suppose a professor conducted research using only students in his or her introductory psychology class, or suppose a researcher mailed a questionnaire and obtained only a 40 percent return from recipients. Clearly, these samples may or may not be representative of the population of interest. In the first instance, the professor may be interested only in describing the behaviors of students in his class; but if his interest is in generalizing to a wider population the limitations of his population need to be noted.
Problem: A reviewer needs to consider the possibility of errors in sampling when interpreting the results of a study. Deciding how much trust to put in the results of a study based on a flawed sample is a highly subjective judgment.
■ Issue 2: Measurement. Almost all measures in empirical research should be presumed to be flawed to some extent. For instance, suppose a researcher uses a self-report questionnaire to measure the incidence of marijuana use on a campus. Even if respondents are assured that their responses are confidential and anonymous, some might not want to reveal their illegal behavior. On the other hand, others might be tempted to brag about doing something illegal even if they seldom or never do it. So what are the alternatives? One may conduct personal interviews, but this measurement technique also calls for revelation of an illegal activity. Another alternative is covert observation, but this technique might be unethical. On the other hand, if the observation is not covert, participants might change their behavior because they know they are being observed. As you can see, there is no perfect solution.
Problem: A reviewer needs to consider the possibility of measurement error. Ask yourself whether the method of measurement seems sound. Did the researcher use more than one method of measurement? If so, do the various methods yield consistent results?
■ Issue 3: Problem identification. Researchers usually examine only part of a problem—often just a very small part. Here is an example: Suppose a researcher wants to study the use of rewards in the classroom and their effect on creativity. This sounds manageable as a research problem until one considers that there are many kinds of rewards—many kinds and levels of praise, many types of prized objects that might be given, and so on. Another issue is that there are many different ways in which creativity can be expressed. For instance, creativity is expressed differently in the visual arts, in dance, and in music. Creativity can be expressed in the physical sciences, in oral expression, in written communication, and so on. No researcher has the resources to examine all of these forms. Instead, he or she will probably have to select only one or two types of rewards and only one or two manifestations of creativity and examine them in a limited number of classrooms.
Problem: A reviewer needs to synthesize the various research reports on narrowly defined problems in a given area, looking for consistencies and discrepancies from report to report while keeping in mind that each researcher defined his or her problem in a somewhat different way. Because empirical research provides only approximations and degrees of evidence on research problems that are necessarily limited in scope, creating a synthesis is like trying to put together a jigsaw puzzle for which most of the pieces are missing and with many of its available pieces not fully formed.
Considering the three issues presented, you might be tempted to conclude that reviewing original reports of empirical research is difficult. Undoubtedly, it sometimes is. However, if you pick a topic of interest to you and thoroughly read the research on that topic, you will soon become immersed in a fascinating project. On the vast majority of topics in the social and behavioral sciences, there are at least minor disagreements about the interpretation of the available research data, and often there are major disagreements. Hence, you may soon find yourself acting like a juror, deliberating about which researchers have the most cohesive and logical arguments, which have the strongest evidence, and so on. This can be a difficult, but interesting, activity.
You also might incorrectly conclude that only students who have intensively studied research methods and statistics can make sense of original research reports. While such a background is very helpful, this book was written with the assumption that any intelligent, careful reader can make sense of a body of empirical research if he or she reads extensively on the topic selected for review. Authors of reports of original research do not present statistics in isolation. Instead, they usually provide discussions of previous research on their topic, definitions of basic concepts, descriptions of relevant theories, their reasons for approaching their research in the way they did, and interpretations of the results that are moderated by acknowledgments of the limitations of their methodology. Thus, a skilled author of a report on original empirical research will guide you through the material and make it comprehensible to you even if you do not understand all the jargon and statistics included in the research report.
One final consideration: It is essential that you carefully and thoroughly read all the research articles that you cite in your literature review. Reading only the brief abstracts (summaries) at the beginning of research articles may mislead you because of their lack of detail and, therefore, cause you to mislead the readers of your literature review. Thus, it is your ethical responsibility to read each cited reference in its entirety.
Not every journal article is a report of original research. For instance, some articles are written for the explicit purpose of critiquing an existing theory or to propose a new one. Remember, a theory is a general explanation of why variables work together, how they are related to each other, and especially how they influence each other. As a unified set of constructs, a theory helps to explain how seemingly unrelated empirical observations tie together and make sense. Here is a brief example:
Consider the relational theory of loneliness
Among other things, this theory distinguishes between emotional loneliness
(utter loneliness created by the lack of a close emotional attachment to another person) and social loneliness
(feelings of isolation and loneliness created by the absence of a close social network). This theory has important implications for many areas of social and behavioral research. For instance, this theory predicts that someone who is in bereavement due to the death of a spouse with whom he or she had a close emotional
attachment will experience utter loneliness that cannot be moderated through social
Notice two things about the example given above. First, the prediction based on the theory runs counter to the commonsense notion that those who are lonely due to the loss of a significant other will feel less lonely with the social support of family and friends. The theory suggests that this notion is only partially true at best. Specifically, it suggests that family and friends will be able to lessen social loneliness but be ineffective in lessening the more deeply felt and potentially devastating emotional loneliness. Note that it is not uncommon for a theory to lead to predictions that run counter to common sense. In fact, this is a hallmark of theories that make important contributions to understanding human affairs and our physical world.
Second, the relational theory of loneliness can be tested with empirical research. A researcher can study those who have lost significant others, asking them about how lonely they feel and the types and strength of social support they receive. To be useful, a theory must be testable with empirical methods, which helps the scientific community to determine the extent of its validity.
Your job in reviewing literature will be made easier if you identify the major theories that apply to your topic of interest. Writers of empirical research reports often identify underlying theories and discuss whether their results are consistent with them. Following up on the leads they give you in...