Introduction
Functional neuroimaging techniques, such as functional magnetic resonance imaging (fMRI) and electro/magnetoencephalography (EEG/MEG), have had a major impact on the study of human memory over the last two decades. This impact includes not only new evidence about the parts of the brain that are important for memory (āfunctional localizationā or ābrain mappingā), which extends what was previously known from patients with brain damage, but also arguably informs our theoretical understanding of how memory works (e.g., Henson, 2005; Poldrack, 2006; though such claims have been questioned, e.g., Coltheart, 2006; Uttal, 2001). In this chapter, we illustrate ways in which functional neuroimaging has influenced our understanding of memory, going beyond research that was previously based primarily on behavioral techniques. We focus in particular on how memory processes might be implemented in the brain in terms of average levels of activity in certain brain areas, patterns of activity within areas, and connectivity between brain areas.
Theoretical Concepts That are Difficult to Measure Behaviorally, e.g., Retrieval States
Tulving (1983) theorized that we adopt a particular mind-set during episodic memory retrieval, a so-called āretrieval mode,ā which optimizes recovery of information from memory, and allows us to interpret that information as having come from the past (rather than from sensations in the present). Until recently, however, it has been difficult to evaluate theories like this owing to the difficulty of measuring such states behaviorally. Neuroimaging, on the other hand, is able to measure sustained brain activity directly associated with a state. This ability has reinvigorated such theories, leading to new hypothetical states that are assumed to be important for the encoding and retrieval of information, and even prompting new behavioral measures to investigate such theories further (see Chapter 5).
An early example of this use of neuroimaging is the study of DĆ¼zel and colleagues (1999), who recorded EEG during sequences of four words. Prior to each sequence, a cue instructed participants to decide whether or not each word was seen in a previous study phase (āepisodic taskā), or whether each word denoted a living or nonliving entity (āsemantic taskā). DĆ¼zel et al. found a sustained positive shift over right frontal electrodes for the episodic task relative to the semantic task. This positive shift emerged shortly after the instruction onset, but prior to the presentation of the first word (i.e., before any retrieval had taken place), and so was interpreted as evidence of a preparatory state for episodic retrieval, i.e., a retrieval mode.
This neuroimaging finding in turn prompted new theoretical proposals. Rugg and Wilding (2000) proposed that there may be different states even within a retrieval mode, in which people are oriented towards retrieving different types of episodic information. They called these āretrieval orientations.ā For example, Herron and Wilding (2004) reported a more positive-going left frontocentral EEG shift when participants prepared to retrieve the type of encoding task under which an item was studied, compared to when they prepared to retrieve the location in which an item was studied. Another example is the study of Ranganath and Paller (1999), which examined event-related potentials (ERPs) locked to the onset of correctly rejected, new (unstudied) items in a recognition memory test. Because such correct rejections are unlikely to elicit any episodic retrieval, any difference in their associated ERPs as a function of retrieval instructions is likely to be a consequence of a different retrieval orientation. In this case, Ranganath and Paller compared a retrieval task in which participants had to endorse objects that had appeared at study, regardless of their size on the screen (āgeneral taskā), with another task in which participants were only to endorse items as studied if they appeared in the same size as at study (āspecific taskā). A more positive-going ERP waveform to correct rejections was found post-stimulus onset over left frontal electrodes for the specific than for the general task.
It is also possible to measure such state-related brain activity with fMRI, though given its worse temporal resolution relative to EEG, special designs are needed that allow statistical modeling to separate state-related from item-related blood-oxygen-level-dependent (BOLD) responses. For example, Donaldson and colleagues (2001) showed state-related activity associated with blocks of a recognition memory task (relative to blocks of a fixation task) in bilateral frontal opercular areas. Moreover, the same brain areas also showed greater item-related activity for correct recognition (hits) than correct rejections, suggesting that frontal operculum supports both a sustained retrieval mode and transient processes associated with successful retrieval. A subsequent fMRI study by Otten, Henson, and Rugg (2002) provided analogous evidence for dissociable āencoding orientationsā. These authors found that the mean level of state-related activity during blocks of words varied as a function of the number of words later remembered within each block, independent of item-related activity associated with whether or not individual words were successfully remembered. Furthermore, this relationship between state-related activity and subsequent memory occurred in different brain areas as a function of the study task: occurring in left prefrontal cortex when participants performed a semantic (deep) task, and superior medial parietal cortex when participants performed a phonemic (shallow) task.
Importantly, the neuroimaging studies described above have not only led to new theoretical development (e.g., the concepts of retrieval and encoding orientations), but also prompted new behavioral experiments to further test these concepts. Building on the ERP studies such as that of Ranganath and Paller (1999) described above, Jacoby et al. (2005) conducted behavioral investigations of retrieval orientation. They used a second memory test to probe the fate of correctly rejected new items (foils) in a first recognition test, as a function of the retrieval orientation that was adopted during that first memory test. Participants studied one list of items under a semantic (deep) task, and another list of items under a phonemic (shallow) task. In the first recognition test, participants were expected to be oriented towards semantic information when distinguishing foils from deeply encoded targets, but oriented towards phonemic information when distinguishing foils from shallowly encoded targets. If so, the foils in the semantic condition should be processed more deeply than the foils in the phonemic condition, and hence themselves be remembered better on the final recognition test. This is exactly what the authors found. Thus, this (indirect) behavioral assay supported the theories of retrieval orientations that originated from neuroimaging research. Furthermore, this assay has been used to examine how retrieval orientations become less precise as people get older.
Supplementing Behavioral Dissociations with Neuroimaging Dissociations, e.g., Dual-Process Theories
Another situation in which neuroimaging data can complement behavioral data arises when seeking functional dissociations between hypothetical memory processes. For example, there has been a long-standing debate about whether behavioral data from recognition memory tasks are best explained by single- versus dual-process models. Single-process models claim that a single memory-strength variable is sufficient to explain recognition performance, normally couched in terms of signal detection theory (Donaldson, 1996; Dunn, 2004, 2008; Wixted, 2007; Wixted and Mickes, 2010). Dual-process models, however, assume that recognition involves at least two different processes, such as recollection, associated with retrieval of contextual information, and familiarity, providing a generic sense of a previous encounter, but without contextual retrieval (Aggleton and Brown, 1999; Diana et al., 2006; Rotello and Macmillan, 2006; Yonelinas, 2002; see also Chapter 9). It is not clear that behavioral data have yet resolved this debate (though the main protagonists may disagree!). One possible solution is to examine neuroimaging data from the same task: if conditions assumed to entail recollection produce qualitatively, rather than just quantitatively, different patterns of activity across the brain compared to conditions assumed to entail familiarity, then this would appear to support dual-process models (see Henson, 2005, 2006, for further elaboration and assumptions of this type of āforward inferenceā).
A methodological question then becomes how to define a āqualitativelyā different pattern of brain activity. With classical statistics, it is not sufficient, for example, to find a significant difference in one brain area for a contrast of a recollection-condition against a baseline condition, and in a different brain area for the contrast of a familiarity-condition with that baseline. This is simply because the failure to find significant activation for each condition in the other brain area could be a null result. However, even finding a significant interaction between two brain areas and two such contrasts is not sufficient, because we do not know the āneurometricā mapping between fMRI/EEG/MEG signal and the hypothetical processes of interest. This mapping may not be linear (i.e., a doubling in memory strength may not necessarily mean a doubling in BOLD signal or ERP amplitude). Moreover, the neurometric mapping may differ across different brain areas. Indeed, there may be a positive relationship between the neuroimaging signal and a memory process in one area (e.g., increasing BOLD signal associated with increasing memory strength in hippocampus), but a negative relationship between the neuroimaging signal and the same memory process in another area (e.g., decreasing BOLD signal associated with increasing memory strength in perirhinal cortex; Henson, 2006; Squire, Wixted, and Clark, 2007). These considerations mean that even a significant crossover interaction between two areas and two conditions does not refute single-process theories.
Fortunately, there is a method to solve this problem of unknown neurometric mappings, which assumes only that these mappings are monotonic (in other words, the neuroimaging signal must always increase, or always decrease, whenever engagement of the hypothetical process increases, even if it does not increase or decrease in equal steps). This method is called āstate-trace analysis,ā and it was developed in the psychological literature by Bamber (1979). The āreversed associationā pattern described by Dunn and Kirsner (1988), and by Henson (2005), is a special case of state-trace analysis. This method requires at least two dependent variables, e.g., neuroimaging signal in two brain areas, and at least three levels of the independent variable, e.g., three memory conditions. When plotting the data from each condition in a space whose axes are defined by the two independent variables, if the resulting āstate-traceā is neither monotonically increasing nor monotonically decreasing, then one can refute the hypothesis that there is a single underlying process (for further elaboration, see Newell and Dunn 2008).
This analysis has been recently applied to neuroimaging data, for the first time, by Staresina et al. (2013b). These authors examined the amplitude of the initial evoked component (peaking around 400 ms) in ERPs recorded directly from human hippocampus and perirhinal cortex during a recognition memory task. The task enabled definition of three trial types: (1) trials in which an unstudied item was correctly rejected, (2) trials in which a ...
