Conducting Reaction Time Research in Second Language Studies

Nan Jiang

• Publisher: Routledge
• Year: 2013
• Print ISBN:
  9780415879330
• eBook ISBN:
  9781136508028
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

This book offers a general introduction to reaction time research as relevant to Second Language Studies and explores a collection of tasks and paradigms that are often used in such research. It provides a lucid explanation of the technical aspects of collecting reaction time data and outlines crucial research principles and concepts that will ensure accurate data. In addition, Conducting Reaction Time Research in Second Language Studies provides step-by-step instructions for using DMDX, a software program widely used for conducting reaction time research. From general guidelines to techniques to working with data, this complete "why and how" guide for conducting reaction time research is ideal for both students/beginners and more seasoned researchers.

1

INTRODUCING REACTION
TIME RESEARCH

Time is an integral dimension of any physical, social, or mental event. It takes time for events to occur, and it takes time for an individual to do anything and everything. Even though the meaning of time may remain to be a topic of philosophical debate, the measure of time has long been an essential part of human history; from the invention of the sundial by ancient Egyptians several thousand years ago, to the use of the sandglass for timekeeping in fourteenth-century Europe. The measure of time has also been a fundamental part of scientific research, from astronomy to physics, from chemistry to medical research.

The same is true in the study of the human mind and human behavior. Physiologists and experimental psychologists have long been interested in how the mind works, and they examined mental processes by measuring how fast people were able to respond to a stimulus or to perform a task. Such research is sometimes referred to as mental chronometry. This book deals with mental chronometry in language processing, or in second language studies (SLS), in particular. This chapter begins with a definition and characterization of research that involves the measure and analysis of the time individuals take to complete a task. It is then followed by a historical sketch of such research.

1.1 Understanding Reaction Time Research

1.1.1 Defining Reaction Time Research

Any empirical research involves the collection of some kind of data. Such data can vary a great deal across studies. They may be a child's first utterances, the volume of blood flow in an individual's brain, or the number of words correctly produced in a free recall task. In SLS, researchers have employed a variety of approaches to examine how a second language (L2) is learned and used. For example, a productive approach that dominated SLS in its early years was to examine the errors learners made. As rightly pointed out by Corder (1967), a learner's errors can inform us about how he or she goes about learning the new language. Researchers have also designed tasks, such as grammaticality judgment and elicited imitation, to assess the linguistic knowledge L2 learners have developed. In these studies, accuracy rates are computed as empirical data.

Reaction time research refers to any empirical study in which a research question is answered through the measurement and analysis of the amount of time individuals take in responding to a stimulus or performing a task. A task for this purpose is usually simple. In a lexical decision task, for example, individuals are asked to decide if a letter string is a word or not. They respond by pressing two buttons, one for a word and the other for a nonword. In a word or picture naming task, a participant is asked to read aloud or to name a word or picture as quickly as possible. An example of a more complicated task is sentence-picture matching (i.e., deciding if a sentence describes a picture correctly). Whatever task is used, a researcher's primary interest is in measuring how fast a person responds to a stimulus. The time a person takes to respond is referred to as reaction time (RT), response time, or response latency. It is usually measured in milliseconds (ms) from the onset of a stimulus to the point of time when a response is given. RT constitutes the primary data for exploring a research question in RT research, even though error rate (ER) can be informative, too.

The use of RT data is based on the premise that cognitive processes take time and by observing how long it takes individuals to respond to different stimuli or perform a task in different conditions, we can ask questions about how the mind works, and infer about the cognitive processes or mechanisms involved in language processing. For example, imagine that you are asked to read aloud the following two sets of words:

a	tree, task, home, make, good;
b	pint, slur, tusk, pate, puce.

Your intuition will probably tell you that you will name the first set of words faster than the second set. Such comparison of word naming times raises two questions immediately: Why do we need that extra time for naming the second set? What is going on in our minds during that extra time? A longer RT in one condition than in another may reflect the involvement of more mental operations, a higher level of complexity of operations, or a higher degree of difficulty encountered by the language processor. Such RT data, when collected under adequately designed experimental conditions, can shed light on what linguistic knowledge individuals have and how such knowledge is put to use. For another example, if you ask a group of native speakers (NSs) of English to read, for comprehension, a grammatical (with – s) and an ungrammatical (without – s) version of a sentence such as I did not read any of the book(s) on the shelf and measure their reading time for each word, you will probably find that they show a delay in reading the word book or the next word in the ungrammatical version (see Jiang, 2007 for an actual study that demonstrated this effect). We can infer from this delay that NSs of English are sensitive to plural errors even when they are not explicitly asked to pay attention to grammatical accuracy. This sensitivity allows us to further infer that NSs of English possess the integrated linguistic knowledge about plural marking that they put to use automatically in language processing.

In most RT studies, the focus is not to measure the absolute speed or rapidity a person shows in performing a task. People are different in how fast they act, think, or make decisions. Some people are faster than others in performing many tasks, including processing language and pressing buttons. Thus, RTs themselves alone are not very informative. Instead, the focus is on how fast an individual performs a task in different conditions. Two or more conditions are usually created in an RT experiment by varying the stimuli, tasks, or participants systematically. By examining how such variations affect participants’ RT, we can infer about the cognitive processes or operations involved in language processing.

1.1.2 Characterizing RT Research

A better appreciation of RT research can be achieved through an understanding of its most important characteristics. There are at least four: accurate timing, rigorous variable manipulation and control, time-sensitive assessment of behavior, and step-by-step progression.

1.1.2.1 Accurate Timing

Timing accuracy is the most basic and essential requirement of any RT research, without which RT data are of little use. Millisecond accuracy is required in RT research both in the display of stimuli and in the recording of response latencies.

Accurate timing in stimulus display has two components: duration timing and onset timing. The former refers to the length in time for a stimulus to remain available to a participant. A word may be displayed for 500 ms in one experiment and 50 ms in another. This duration is determined by many factors, and under some circumstances, accurate display duration is crucial for the interpretation of the results. For example, in a masked priming study, the prime has to be presented long enough to be processed but brief enough not to be noticed consciously by a participant (see Section 3.3 for more information on masked priming).

Another element of accurate stimulus display is onset timing, which refers to when a stimulus is presented. This applies to experiments in which a trial consists of two or more stimuli. For example, in a picture-word interference paradigm, a trial consists of the presentation of a picture and a word. The exact onset time of these two stimuli (i.e., the interval between their onset) can affect the outcome of a study. A strong interference effect occurs if a semantically related word (e.g., cat) is presented 150 ms before the onset of the picture of a dog, but not when it is presented 150 ms after (see Schriefers, Meyer, & Levelt, 1990 for the results).

Accurate timing is also required in the measure of participants’ RTs, as they are the primary data for answering research questions. A participant's RT is usually measured from the onset of a stimulus to the point of time when a response is provided. This is usually done through a computer clock which is turned on with the onset of a stimulus and stopped when a response is given. Accurate RT measurement depends on knowing when to start timing, and on the method used for recording RTs. According to the testing done by Forster and Forster (2003), input devices such as a game pad, a joystick, and a mouse offer better timing accuracy in recording responses than a keyboard.

1.1.2.2 Rigorous Variable Manipulation and Control

In RT research, great care is given to isolating the phenomenon under investigation from other related elements. This is important because RT can be affected by many factors. To link a set of RT data to a phenomenon requires careful consideration of all these factors.

For example, you have a feeling that the number of translations in the learners’ first language (L1) that an L2 word has would affect the time it takes to translate the L2 word. You believe that the more L1 translations an L2 word has, the longer it would take to translate this word because multiple L1 translations may compete for output. Thus, everything being equal, Chinese ESL speakers would translate the English word smart faster than serious because the former is usually translated into congming in Chinese but the latter has several likely translations such as yansu, yanzhong, and renzhen. In order to determine if this is true empirically, you need to identify a set of L2 words that each have one single L1 translation, and a set of L2 words that have multiple L1 translations. Careful steps need to be taken to ensure that the single-translation words indeed have only one translation in L1 and words in the multiple-translation condition have multiple L2 translations. A related issue to consider is how many L1 translations an L2 word should have in order to qualify as a multiple-translation word. Will two translations be enough? Or will three or more translations be needed to order to produce a robust effect?

When you have identified two sets of L2 words that differ in the number of L1 translations, you may also find that they differ in other aspects. For example, words in one set may occur more frequently or be more familiar to participants than the words in the other set. Or the words in one set may be shorter than those in the other set. Frequency, familiarity, and length can also affect how fast these words are translated. Thus, it is important to ensure that the two sets of words are matched on these properties. When two sets of words differ and only differ in the number of L1 translations they have, then the target phenomenon under investigation (i.e., the number of L1 translations) has been successfully isolated. When one set is responded to faster than the other, one can reasonably conclude that the number of L1 translations affects translation time.

In research design terms, the process of identifying L2 words with one or multiple L1 translations and dividing them into two sets is referred to as variable manipulation, and the process of making sure that the words are matched for other properties is one of variable control. Rigorous variable manipulation and control is the key to relating a set of RT data to the phenomenon such that the former can illuminate the latter (see Section 2.3.4 for more information on variable manipulation and control).

1.1.2.3 Time-Sensitive Assessment of Behavior

An important feature of RT research is that data are collected within a narrow window of time when language processing is still on-going or immediately after its completion. A standard part of the instructions for an RT experiment is to emphasize that participants have to respond as quickly as possible. Transient display of stimuli and feedback on speed often help reinforce the rapidity requirement. This emphasis on immediate and quick responses, the use of tasks that make such quick responses possible, and the collection of time-sensitive data (i.e., RT) work to maximize the chance for the observed data to reflect the moment-by-moment unfolding of cognitive processes under examination. As Tyler pointed out:

fast response tasks tap the listener's representation of the input at a specific moment in time. Given the input available to the listener when she or he makes the response, it is then possible to infer what types of analysis must have been performed upon this input to produce the effects reflected in the response.

(1983, p. 310)

Many RT tasks are referred to as on-line tasks or are said to provide on-line measures of cognitive processes because they help reveal what is going on in our minds while language processing is unfolding. The term on-line, though, is not always well defined or unanimously understood. The term is traditionally used to refer to the examination of sentence or discourse comprehension while the comprehension is on-going, or “during its operation” in Swinney's words (1979, p. 647), rather than after its completion. This usually means that a probe or data collection has to occur before the end of a sentence or discourse. However, the term on-line has also been used more liberally to refer to tasks that require fast or temporally constrained responses and measure RTs as data.¹

It is desirable to consider on-line as a relative term. All tasks that require fast responses and produce RT data can be considered on-line tasks with the understanding that different tasks differ in the degree to which the RT data reflect what analysis is being done, what linguistic knowledge is being involved, or what mental representation is being built at a particular point of time. Some tasks are more on-line than others, but these RT tasks share the common feature that responses are observed in close temporal proximity to the mental processes under examination.²

1.1.2.4 Step-by-Step Progression

Any scientific inquiry represents a continuation of effort to further one's understanding of a phenomenon one step at a time. This progressive nature of research is particularly pronounced in RT research, as compared to research involving other methods in SLS. This often results in multiple experiments in a study. It is the norm rather than exception that an RT study reports multiple experiments (e.g., 2 to 6 experiments) in a progressive manner. It is not rare for a study to include a larger number of experiments; for example, 10 experiments in Trabasso, Rollins, & Shaughnessy (1971), 11 experiments in Topolinski and Strack (2009), 17 Experiments in Eimas and Nygaard (1992), and 18 experiments in Xiong, Franks, and Logan (2003).

There are two major reasons for this characteristic. First, as discussed earlier, RT research aims to have vigorous variable control so that a target variable under investigation can be isolated and then linked to the observed RT data. This often means that only a single or a small number of variables are examined in any specific experiment. In other words, the scope of an RT experiment is usually very narrow and focused. In order to understand the generalizability of a finding (i.e., to understand whether a finding is restricted to the type of materials, tasks, or participants involved in an experiment, or whether the finding is affected by other variables), it becomes necessary under many circumstances to conduct follow-up experiments. These follow-up experiments usually incorporate variables not considered in the earlier experiments, or involve a different type of material, participant, or task. The second reason is also related to variable control. In studying complex phenomena such as human language and human behavior, perfect variable control is not realistic under many experimental conditions. It is also difficult to predict or anticipate all relevant variables that may affect a result. Consequently, a finding from an experiment may be unexpected, or subject to two or more alternative explanations. Follow-up experiments thus have to be done to explore an unexpected finding or to test alternative explanations.

A look at a study by Costa and Santesteban (2004) may help illustrate the progressive nature of RT research. The study was intended to explore a relatively new phenomenon often referred to as the asymmetry in switching costs, first discovered by Meuter and Allport (1999). In the 1999 study, bilingual speakers were asked to name pictures in two languages, with the background color of the pictures indicating which language to use, so they had...

Table of contents

• Front Cover
• CONDUCTING REACTION TIME RESEARCH IN SECOND LANGUAGE STUDIES
• second Language acquisition Research series: theoretical and methodological issues
• Title Page
• Copyright
• CONTENTS
• List of Figures
• List of Tables
• Preface
• 1 Introducing Reaction Time Research
• 2 Basic Concepts and General Guidelines
• 3 Lexical and Phonological Tasks
• 4 Semantic Tasks
• 5 Sentence-Based Tasks
• Appendix: A Tutorial for DMDX
• Notes
• References
• Index