Difference due to memory

Difference due to memory (Dm) indexes differences in neural activity during the study phase of an experiment for items that subsequently are remembered compared to items that are later forgotten. It is mainly discussed as an event-related potential (ERP) effect that appears in studies employing a subsequent memory paradigm, in which ERPs are recorded when a participant is studying a list of materials and trials are sorted as a function of whether they go on to be remembered or not in the test phase. For meaningful study material, such as words or line drawings, items that are subsequently remembered typically elicit a more positive waveform during the study phase (see Main Paradigms for further information on subsequent memory). This difference typically occurs in the range of 400–800 milliseconds (ms) and is generally greatest over centro-parietal recording sites, although these characteristics are modulated by many factors.[1][2]

History

The first report of subsequently remembered items eliciting a more positive ERP waveform than subsequently forgotten items during the study phase was by Sanquist et al., in 1980.[3] This paper looked at a subset of the participants’ ERPs at the study phase and found those trials subsequently remembered had a more positive waveform in the time range of the late positive complex (LPC), approximately 450–750 ms after stimulus presentation. In the early and mid 1980s, several studies noted modulation of the P300 (P3b) component due to subsequent memory, with items that are remembered having a larger amplitude.[4][5][6] In 1987, Paller, Kutas and Mayes,[7] consistent with previous reports, observed that subsequently remembered items elicited more positivity in the later portions of the waveform compared to items later forgotten; they termed these observed differences at the study phase as "the Difference due to Memory" or Dm effect. Since this seminal paper by Paller, Kutas and Mayes, a wealth of research using ERPs has been conducted using the Dm effect and detailing the multitude of factors that influence the manifestation of the Dm and, by inference, encoding success. Additionally, the Dm has been studied using intracranial recordings [8] and in a variety of functional magnetic resonance imaging (fMRI) studies.[2]

Main paradigms

Overwhelmingly, the paradigm used to elicit a Dm effect in ERPs has been the "subsequent memory paradigm." An experiment employing a subsequent memory paradigm generally consists of two phases, a study phase (encoding phase) and a test phase (retrieval phase), with ERPs from scalp electrodes being recorded during each phase, time locked to stimulus onset. In the study phase, a series of items is displayed to the participant, usually one at a time; these items are most often words but pictures and abstract figures have also been used (though with less consistent Dm effects; see "Functional Sensitivity"). The test phase normally mixes together items that were shown during the study phase with others that are being shown for the first time, and the participant must classify each item as being "old" (if it was in the study phase) or "new" (if it is the first time it has been seen).

Critically for the Dm effect, the responses a participant makes to the old items in the test phase are used to backsort trials in the study phase as either "subsequently remembered" or "subsequently forgotten." If during the test phase a participant correctly classifies an old item as old, it falls into the "subsequently remembered" trial type for the study phase. On the other hand, if a person incorrectly calls an old item new at the test phase, or fails to respond "old" to an old item, this item becomes classified as "subsequently forgotten." The ERP waveforms, during the study phase, of all subsequently remembered trials are compared with those of all subsequently forgotten trials and a greater positivity is generally seen for the subsequently remembered trials.

For example, in the study phase of a subsequent memory paradigm, a participant may see the words "frog," "tree," and "car." Following the study phase the test phase occurs and the participant sees the words "shirt," "car," and "frog," and must say if each word is old or new. If the participant correctly classifies "car" as old, it becomes a subsequently remembered trial; however, if the subject incorrectly says "frog" is new, it is a subsequently forgotten trial. The neural activity elicited by the first presentation of "car" and "frog" at the study phase is then compared and the Dm effect is derived from this comparison.

A "continuous recognition paradigm" has also been known to elicit a Dm effect. In the continuous recognition paradigm, study and test phases are not separate entities, but rather, items are continuously presented and the participant is instructed to respond to an item as "old" if it has been seen before (generally presented a second time) in this continual stream of item presentation. Items that were correctly called "old" are the subsequently remembered trials, and items that were "missed" (not called old upon second presentation) make up the subsequently forgotten trials. The neural activity for subsequently remembered and forgotten trials is then compared for the first presentation of the items, and a Dm effect is computed.

Component characteristics

Broadly speaking, the Dm ERP effect is any difference in neural activity recorded during the study phase of an experiment that differentiates subsequently remembered items and subsequently forgotten items. Typically, this difference is seen in the form of subsequently remembered items eliciting waveforms that are more positive than subsequently forgotten items during encoding of the item. Most often, the difference between subsequently remembered and subsequently forgotten items emerges at approximately 400 ms post stimulus onset and is sustained until 800 or 900 ms, though this can vary depending on the stimuli used and experimental instructions.[1] The timing of this enhanced positivity suggests that the Dm may be a modulation of several ERP components, including the N400 component, with subsequently remembered items eliciting a less negative amplitude, as well as the P300 or an LPC, where items that are later remembered yield a more positive amplitude in this waveform.[1] In terms of scalp topography, the Dm effect is generally largest over centro-parietal recording sites. However, a Dm effect with a more anterior distribution can be observed by varying the instructions participants receive;[9][10] this is discussed further below.

Functional sensitivity

The canonical characteristics described above of the Dm effect give a general description of the component; however, the strength, timing, topographical distribution and even whether or not the effect is seen is sensitive to a variety of experimental manipulations.

Incidental versus intentional encoding

A large number of Dm ERP studies employ an incidental encoding approach to the subsequent memory paradigm. In this case the participant pays attention to the items presented during the study phase unaware that a memory test will follow. This was the approach used by Paller, Kutas and Mayes in the first Dm study,[7] and this technique reliably elicits a Dm effect. Experiments wherein the participant is explicitly told to remember the items presented during the study phase (intentional encoding) because a memory test will follow have yielded slightly differing results. Several studies have indeed recorded a Dm effect using intentional encoding instructions, but this effect sometimes differs from the Dm effect from incidental encoding. In a direct comparison of incidental vs. intentional encoding, Munte et al., (1988)[9] found a stronger Dm effect for the incidental encoding condition. Moreover, the Dm effect for the intentional encoding condition appeared later than the Dm for incidental encoding, and also showed a more frontal topography compared to the centro-parietal distribution observed in incidental encoding. This effect of a delayed and more frontal distribution for intentional encoding paradigms was also seen in two other reports.[10][11]

Levels of processing and rehearsal at encoding

Perhaps the most well known manipulation during the subsequent memory paradigm is how the participant is instructed to encode or process the material during the study phase. Generally speaking, participants may be instructed to observe the items at test and make a judgment regarding each item; crucially, this judgment may be of the "shallow" variety, such as deciding if the word presented contains more than two vowels, or it may be a "deeper" judgment (e.g. is this item edible?) These deeper judgments are more of the semantic variety and typically lead to a better representation of the item.[12] This is also reflected in the Dm effect. In the seminal paper by Paller, Kutas and Mayes (1987),[7] participants made shallow judgments based the physical properties of the word or deeper judgments reflective of more semantic information of the word. The Dm effect for words encoded in a semantic fashion was more positive than the Dm effect observed for words non-semantically encoded. It is important to note that a Dm effect can be seen for shallower processing as well, as was the case in one of the shallow processing tasks in the Paller, Kutas and Mayes (1987) paper,[7] as well as in Friedman, Ritter and Snodgrass (1996).[13]

In 1997, Weyerts et al.[14] found that both recognition memory as well as the Dm effect was larger for pairs of words that were relationally encoded (e.g. are these two words semantically related) versus non-relationally encoded (e.g. can the color white be associated with one of these words). This further suggests that the Dm effect may be enhanced when items are encoded on a semantic level.

Also, the Dm effect seems sensitive to the type of rehearsal strategies a participant performs. Specifically, Fabiani, Karis and Donchin [6][15] found that P300 modulation at encoding (particularly for "isolates,"stimuli presented in a deviant font relative to all other stimuli) correlated with later memory for subjects who engaged in rote rehearsal (such as simply repeating the word in one's head) but not for those who undertook elaborative rehearsal, which emphasizes linking the current word to other words presented and pre-existing knowledge. However, in the 1990 report[15] as well as a report by Karis, Fabiani and Donchin (1984),[5] a later positivity emerged in frontal electrodes corresponding to subsequent memory, and this was greater for those in the elaborative rehearsal condition.

Type of memory at retrieval

The Dm effect has been shown to be sensitive to how participants are asked to display their memory for previous items. In a 1988 paper by Paller, McCarthy and Wood,[16] a greater Dm effect was observed for items that were freely recalled with no external cues, compared to items that were presented and the subject was asked if he or she recognizes the item as old. This is suggestive of the Dm effect being larger for stronger representations, as recall is generally more difficult than recognition.

In a similar vein, Friedman & Trott (2000)[17] found that young adult participants displayed a robust Dm effect when they not only remembered seeing a word, but could also remember some details of the context of when it was presented. In comparison, a Dm effect for items that were subsequently judged as old, but only from a general sense of familiarity, did not emerge. A Dm effect was found in both conditions for older adults.

Stimuli

A host of studies have found a Dm effect when presenting words as stimuli.[1] However, experiments using pictures or abstract figures have found less consistent Dm effects. Experiments using a continuous recognition paradigm have found a Dm effect for pictures of everyday objects.[18][19] Van Petten and Senkfor (1996)[20] did not find a Dm effect when they presented participants with abstract drawings; however, a Dm effect was observed in the same group of participants when words were used as stimuli. A similar pattern of results is described by Fox, Michie and Coltheart, (1990).[21] Coupling the results of Dm effects for words and common pictures and the lack of Dm effects for abstract figures suggests the Dm effect may be contingent on using meaningful stimuli or some pre-existing knowledge of the stimuli.[1][20]

False memories

In an elegant report by Gonsalves and Paller (2000),[22] the Dm effect was found to be greater for false memories compared to correctly classified memories. In the study phase of this subsequent memory paradigm, participants saw a word which was followed either by a picture of that word or a blank box, in which case participants were asked to imagine a picture of the word they just saw. In the test phase, participants were shown a word and asked if it was presented with a picture during the study phase. 30% of the time participants erroneously said a picture accompanied a word when it had only been imagined by the participant. The waveform at the study phase of trials in which the participant falsely recalled studying the word with a picture elicited a more positive going amplitude compared to the trials where the participant correctly said only the word was presented. Gonsalves and Paller (2000)[22] interpreted this as indicating that better imagery at encoding led to greater source confusions at retrieval (“did I actually see this or just imagine it?”). More generally, this study demonstrates that backsorting procedures need not be limited to simply items remembered versus forgotten, but could include a wide range of more complex comparisons as long as test phase behaviors can be linked to specific study phase events.

Source

To the extent that greater positivity for subsequently remembered items spans several ERP components (P300, N400, and an LPC), coupled with differing topographical distributions depending on task, it is likely that the neural generators of the Dm effect are widespread in the brain. Pinning down the location in the brain that gives rise to any ERP component is very difficult if not impossible because of the inverse problem.

However, evidence from other cognitive neuroscience techniques can help to shed light on this question. Given that the Dm effect seems to be reflective of mnemonic processes at encoding, one brain area likely to play a role is the medial temporal lobe (MTL), as it is well known this brain area gives rise to the type of memory observed in Dm studies.[23]

Egler et al. (1997)[8] recorded electrical activity directly from the MTL in patients about to undergo surgery for temporal lobe epilepsy. While recording directly from the MTL, participants were shown novel stimuli and then later had a memory test for those stimuli; it was reported that the magnitude of the electrical activity from the MTL during the initial presentation of the stimuli correlated with subsequent memory performance.

Additionally, fMRI studies using subsequent memory paradigms have found evidence suggesting areas of the MTL are involved in the Dm effect, though the precise areas involved and their contributions are unclear.[24][25] Further, several fMRI studies have reported prefrontal cortex (PFC) activity during study predictive of subsequent memory, as well as activity in fusiform gyrus.[2]

Taken together, these findings from complimentary cognitive neuroscience methods suggest the neural events at encoding that lead to successful later memory are diffuse in the brain and unfold on multiple time scales. The Dm effect seen in ERPs likely represents a subset of these encoding processes.

Theory

Considering that the Dm is a comparison of neural activity during encoding, and that this activity is predictive of subsequent memory, it is likely the Dm indexes some difference between subsequently remembered vs. forgotten materials at encoding, presumably reflective of learning. The nature of this difference is not entirely clear though. Van Petten and Senkfor (1996)[20] suggest there may be a "family of Dm effects" that occur dependent on a variety of factors, and this seems quite plausible given the wide range of differences observed in the Dm as a function of stimuli used, encoding instructions, orienting tasks and types of retrieval decisions. Future research using different manipulations of the subsequent memory paradigm, as well as combining methods such as ERPs and fMRI or transcranial magnetic stimulation and fMRI have great potential to lead to further understanding of the Dm effect [1][2] and, more generally, of the neural and cognitive factors that promote later memory under different circumstances.

References

  1. ↑ 1.0 1.1 1.2 1.3 1.4 1.5 Wagner, AD., Koutstaal, W., & Schacter, D.L. (1999). When encoding yields remembering: insights from event related neuroimaging. Phil. Trans. Soc. Land. B, 354(1387), 1307–24.
  2. ↑ 2.0 2.1 2.2 2.3 Paller, KA. & Wagner, AD. (2002). Observing the transformation of experience into memory. Trends in Cognitive Sciences, 6 (2),93–102
  3. ↑ Sanquist, T.F., Rohrbaugh, J.W., Syndulko, K. and Lindsey, D.B. (1980). Electrophysiological signs of levels of processing: perceptual analysis and recognition memory. Psychophysiology, 17, 568–576.
  4. ↑ Karis, D., Bashore, T., Fabiani, M. and Donchin, E. (1982). P300 and memory. Psychophysiology, 19, 328.
  5. ↑ 5.0 5.1 Karis, D., Fabiani, M. and Donchin, E. (1984). "P300" and memory: individual differences in the Von Restorff effect. Cognitive Psychology, 16, 177–216.
  6. ↑ 6.0 6.1 Fabiani, M., Karis, D. and Donchin, E. (1985). Effects of strategy manipulation on P300 amplitude in a Von Restorff paradigm. Psychophysiology, 22, 588–589.
  7. ↑ 7.0 7.1 7.2 7.3 Paller, KA., Kutas, M. & Mayes, AR. (1987). Neural correlates of encoding in an incidental learning paradigm. Electroencephalography and Clinical Neuropsysiology, 67, 360–371.
  8. ↑ 8.0 8.1 Egler, C.E., Grunwald, T., Lehnertz, K., Kutas, M., Helmstaedter, C., Brockhause, A., Van Roost, D. and Heinze, H.J. (1997). Human temporal lobe potentials in verbal learning and memory processes. Neuropsychologia, 35, 657–667.
  9. ↑ 9.0 9.1 Munte, T.F., Heinze, H.J., Scholz, M. & Kunkel, H. (1988). Effects of a cholinergic nootropic (WEB 1881 FU) on event-related potentials recorded in incidental and intentional memory tasks. Neuropsychobiology, 19, 158–168.
  10. ↑ 10.0 10.1 Friedman, D. (1990a) ERPs during continuous recognition memory for words. Biological Psychology,30, 61–87.
  11. ↑ Paller, K.A. Recall and stem-completion priming have difference electrophysiological correlates and are modified differentially by directed forgetting. Journal of Experimental Psychology: Learning, Memory and Cognition, 16, 1021–1032.
  12. ↑ Craik, F.I.M. & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology, 104, 268–294.
  13. ↑ Friedman, D., Ritter, W. & Snodgrass, J.G. (1996). ERPs during study as a function of subsequent direct and indirect memory testing in young and old adults. Cognitive Brain Research, 4, 1–13.
  14. ↑ Weyerts, H., Tendolkar, I., Smid, H.G.O.M. & Heinze, H.J. (1997). ERPs to encoding and recognition in Two different inter-item association tasks. NeuroReport, 8, 1583–1588.
  15. ↑ 15.0 15.1 Fabiani, M., Karis, D. & Donchin, E. (1990). Effects of mnemonic strategy manipulation in a Von Restorff paradigm. Electroencephalography and Clinical Neurophysiology, 75, 22–35.
  16. ↑ Paller, KA., McCarthy, G. & Wood, CC. (1988). ERPs predictive of subsequent recall and recognition performance. Biological Psychology, 26, 269–276.
  17. ↑ Friedman, D. & Trott, C. (2000). An event-related potential study of encoding in young and older adults.Neuropsychologia, 38, 542–557.
  18. ↑ Friedman, D. & Sutton, S. (1987). Event-related potentials during continuous recognition memory. In Current trends in event-related potential research, EEG Suppl. 40 (ed. R. Johnson Jr., J.W. Rohrbaugh & R. Parasuraman), pp. 316–321. Amsterdam: Elsevier.
  19. ↑ Friedman, D. (1990b) Cognitive event-related potentials during continuous recognition memory for pictures. Psychophysiology, 27, 136–148.
  20. ↑ 20.0 20.1 20.2 Van Petten, C. & Senkfor, A.J. (1996). Memory for words and novel visual patterns: repetition, recognition, and encoding effects in the event-related brain potential. Psychophysiology, 33, 491–506.
  21. ↑ Fox, A.M., Michie, P.T. & Coltheart, M. (1990). ERP effects of temporal and spatial recall with verbal and visual stimuli. In Psychophysiological brain research (ed. C.H.M. Brunia, A.W. Gaillard & A Kok), pp 236–239. Tilburg, Germany: Tilburg University Press.
  22. ↑ 22.0 22.1 Gonsalves, B. & Paller, K.A. (2000). Neural events that underlie something that never happened. Nature Neuroscience, 3(12), 1316–1321.
  23. ↑ Eichenbaum, H., and Cohen, N.J. (2001). From conditioning to conscious recollection: Memory systems of the brain. New York: Oxford University Press.
  24. ↑ Brewer, J.B., Zhao, A., Desmond, J.E., Glover, G.H. & Gabrieli, J.D.E. (1998) Making memories: brain activity that predicts how well visual experience will be remembered. Science, 281, 1185–1187.
  25. ↑ Wagner, A.D., Schacter, D.L., Rotte, M., Koustaal, W., Maril, A., Dale, A.M., Rosen, B.R. and Buckner, R.L.(1998). Building memories: remembered and forgetting of verbal experiences as predicted by brain activity. Science, 281, 1188–1191.

See also