Transsaccadic memory

Transsaccadic memory is the neural process that allows humans to perceive their surroundings as a seamless, unified image despite rapid changes in fixation points. The human eyes move rapidly and repeatedly, focusing on a single point for only a short period of time before moving to the next point. These rapid eye movements are called saccades. If a video camera were to perform such high speed changes in focal points, the image on screen would be a blurry, nauseating mess. Despite this rapidly changing input to the visual system, the normal experience is of a stable visual world; an example of perceptual constancy. Transsaccadic memory is a system that contributes to this stability.

Transsaccadic memory is a relatively new topic of interest in the field of psychology. Conflicting views and theories have spurred several types of experiments intended to explain transsaccadic memory and the neural mechanisms involved.

Theories

Saccade target theory

McConkie’s and Currie's saccade target theory ^[1] is similar to research by Schneider who came up with a similar “reference object theory”.^[2] Both theories hypothesize that each saccade is preceded by processes in the visual system that chose an object as the target for the next fixation point. The object is usually located in peripheral vision.^[2] The object's features are stored as a mental representation in transsaccadic memory for identification of future fixations. These target features are searched for by the visual system when the eye lands on its fixation point, and the physical features are compared to the mental representation of the target object. The theory assumes that visual stability is attained when these processes are successful (when the visual stimuli and the mental representation of the target object match). This process occurs before each saccade.^[3] Experiments performed by McConkie to support the role of a saccadic target in transsaccadic memory show two things. There is a limited peripheral area where a saccadic target exists, and attention is vital in recollection of items in the target area. The experiments involved recalling changes to an image that occurred in the peripheral area. Irwin performed similar experiments in which participants recalled letters that occurred near the target area.^[2] Due to confounding factors of the controlled environment in the studies, the involvement of saccade target objects is inferred and not established.^[3]

Spatiotopic fusion hypothesis

Many radical views of transsaccadic memory exist. Some are favoured more than others because of the evidence that supports them. One less accepted theory, Breitmeyer's spatiotopic fusion hypothesis, suggested that successive images are fused based on environmental coordinates and not retinal ones. In other words, our memory stores a sequence of images from our surroundings and joins the images together to give us a full representation in our brain. The orientation of our retinas do not have an influence on this form of memory. This theoretical form of memory supposedly stores highly detailed representations of the world in a high capacity spatial buffer.^[2] Research and experiments by other psychologists provide empirical evidence against these claims.^[2]

Dennett

Daniel Dennett argued that the way we think we are seeing the world is for the most part, an illusion. Part of Dennett’s argument is the claim that each of us possess what he calls a “Cartesian Theater,” in which we believe there is a full representation of the visual world in our mind and that there is a place in the mind where it is observed.^[4] None of this exists according to Dennett. Instead, the only place where a full and rich representation exists is directly on the fovea, and every time a saccade occurs, the information is overwritten.^[4] Therefore, there is no such thing as transsaccadic memory. Information previously lost only appears to be retained in visual memory because we can look again. In this way, the outside world acts as a visual memory.^[4] Since our eyes are constantly moving, we are not aware that visual inputs are constantly being refreshed to give the illusion of the completed picture we think we are seeing. Dennett makes an important distinction between the presence of representation, and the representation of presence.^[4] The example he gives regarding this distinction is this: if you were to walk into a room covered in identical portraits of Marilyn Monroe, you would see that there are many of them, but you would not really be seeing them all at once. There would be no detailed representation of each individual portrait just the knowledge that they are present.^[4] Dennet’s theory raises two relevant questions: 1)How does the visual system detect change in the environment?^[4] 2) How much information is retained in each saccade? The proposed answer to these questions lies in several mechanisms that support a high sensitivity to change in each visual fixation. These mechanisms are: retinal adaptation, “pop out” systems, and motion detectors.^[4] The implication of this view is that little information is needed to be retained between each saccade.

Irwin

Irwin’s conclusion regarding transsaccadic memory is that there are no separate mechanisms, but rather, it is the same as short term visual memory.^[5] Irwin’s experiments showed that people cannot fuse pre-saccadic and post-saccadic images in successive fixations.^[2] These results are evidence against spatiotopic fusion. According to Irwin, there is no transsaccadic buffer ^[2] that holds visual information from one fixation point to the next. Also, transaccadic memory does not hold detailed spatial information, but a more abstract representation. Irwin describes transaccadic memory as an undetailed, limited capacity memory that is relatively long lasting.^[5]

Features

Visual short term memory

Main article: Visual short term memory

Many statements have been made concerning the relationship between transsaccadic memory and visual short term memory. Researchers have noted several similar characteristics between the two system, leading several to believe that transsaccadic memory is in fact visual short term memory^[6] or a part of visual short term memory.^[7] Transsaccadic memory has a limited capacity of three to four items, a slow decay rate and maskable characteristics. Basically, transsaccadic memory can hold three to four items for each saccade, and the retention of items decays or disappears from conscious slowly after the presentation of the stimulus. However, once a mask stimulus, such as a blank screen, is presented immediately after the stimulus, the items retained prior to the mask can be replaced and/or eliminated faster by the mask. All of these factors are typical characteristics of visual short term memory.^[7] The content stored in transsaccadic memory are less-image like, more abstract and are sparse representations of the objects, which is found to be similar to the type of representations in visual short term memory.^[7]^[8] Transsaccadic memory is different from visual short term memory in that it takes into account the changes of the target’s location due to the eye moving to new saccades.^[9] The information retained between saccades do not take into account positional information very well.^[7]^[10] Nevertheless, relational^[11] and identity information are well retained.^[10] Prime and colleagues (2006) hypothesize, transsaccadic memory utilizes egocentric mechanisms, like selective attention, to reduce the visual search of the target and allow for spatial information between saccades to be retained and updated by incorporation of information across saccades.

Attention

Main article: Attention

Transsaccadic memory is characterized by the integration of information gathered prior to the execution of a saccade (pre-saccadic) and information gathered after the saccade (post-saccadic stimuli.^[6] This involves attention, locating the targets necessary to perform a saccade and continuous updates of the environmental information. Attention occurs prior to the movement of the eyes,^[10] therefore it has a strong influence on the location of the saccade, what information is being encoded, remembered across saccades and stored into transsaccadic memory.^[7] In order to perform a successful and accurate saccade an individual is required to make an attentional shift to the saccadic target. This attentional shift causes the encoding of information to occur near the saccade target location.^[10] Attention to the target of the saccade results in better object identification and an increase in accuracy of identification. As well as, improvement in coding of locational information.^[10]^[12] Attention is restricted to one saccade target per area. It is impossible to keep attention on a target in one area, while performing another saccade simultaneously in another. This will cause prolonged saccadic latency periods (the time it takes to make a saccade from one area to another) and increased reaction times.^[2]^[13] However, attention can be distributed around the saccade target, rather than the precise location that will fall on the fovea,^[2] allowing for the detection of one or two objects situated around the saccadic target.^[12] Kowler (1995) proposed two models to explain the relationship between attention, saccades and transsaccadic memory. The spatial model, states that attention is distributed among the perceptual site (visual field) and the saccadic target during the latency period, which allows for the identification of the saccadic target and object surrounding that area. The temporal model states that attention determines where the saccade will occur and releases a “go” signal to initiate a saccade within that area.

Space constancy

Main article: Saccadic masking

Saccadic suppression is responsible for maintaining a continuous, stable, visual world by reducing visual sensitivity to events occurring before, during and after a saccade.^[2]^[14] The more complex the background on which the stimulus is presented, the larger the suppression.^[14] The increase in saccadic suppression can lead to a decrease in detection of change in the visual field.^[15] Saccadic suppression can be linked to the phenomenon of change blindness, in which individuals lack the ability to detect small or large changes within an environment without the aid of directed attention.^[16] There are two types of saccadic suppression, with the first concerning the detection of light spot flashes during saccades. The lower the spatial frequency, meaning less objects in the visual field, of the light flashes the stronger the saccadic suppression. With a less number of items in the visual field it is easier for saccadic suppression to occur. The higher the spatial frequency, which involves more complex objects within the visual field, the less likely there will be saccadic suppression.^[17] The second type concerns the detection of image displacement or a change in an image during eye movement. A displacement between saccades would not be detected because the movement of the eye eliminates the signal for the detection of the displacement.^[2] The location of the target forms an image feature that is used by the visual system to establish space constancy, which maintains the location of an object in space.^[2] Target blanking is used to study visual stability, space constancy and the content of information transferred across saccades. Blanking a target after a saccade eliminates an attempt to make a second saccade, resulting in the visual system failing to find the target information.^[2] Stability and constancy across the saccades is broken, as a result, image displacements become easier to detect when blanking occurs during or right after the saccade.^[2]

Neural Structures

Superior colliculus

A neural structure located behind the lateral geniculate nucleus that is understood to control eye movements. In particular, the deeper layers of the superior colliculus, known as lamina VI and VII, have been found to be involved in initiating and executing saccadic eye movements, which includes the desired speed and direction of the saccade.^[18] The cells in these layers are organized in a way that forms a map of the visual field. They are organized according to what direction each cell moves the eye.^[19] It has been found that activation of specific cells is directed by where objects in the environment are located in the visual field. Once a new object is detected, the cells that fire the strongest to stimuli within this specific area of the visual field will fire, causing the eyes to move and focus on this object. Although the superior colliculus may not be directly related to memory for objects across saccades, it is directly related to the control of saccades and selection of fixation targets.^[20]

V4

This is an area within the visual cortex that has been found to play an important role in the target selection of saccades. In other words, this area is important for determining which objects our eyes shift to when they move.^[21] Studies have shown that there is a large amount of activation within the visual area V4 before the saccade even takes place. This occurs in the form of shrinking receptive fields. The receptive fields of these brain cells tend to shift towards the object that the eye is about to move towards, generally more so if the object is close to the original fixation point.^[22] This dynamic change in receptive fields is thought to enhance the perception and recognition of objects in a visual scene. Because the receptive fields become smaller around the targeted objects, attention within the visual scene is very focused on these objects. Increased attention to target objects within a visual scene help direct eye movements from one object to another. Understanding of the visual scene becomes more efficient because these attention shifts guide the eyes towards relevant objects as opposed to objects that may not be as important.^[23]

Lateral intraparietal cortex

The lateral intraparietal cortex (LIP) is an area that is believed to be primarily responsible for keeping an image fluid and undistorted during a saccade (visual/spatial constancy).^[2] Past work has shown that the LIP stores information of objects before the saccade (presaccadic), which is then followed by a shift in the receptive fields of the brain cells of this area to compensate for the displacement of the retina during a saccade. This shift in the receptive fields occurs nearly 80 ms before the saccade. The LIP uses quantitative spatial information in order to achieve this shift, which allows for the perception of visual and spatial constancy. The receptive fields of the LIP are quite large and therefore are not well suited to hold details about the perceived objects in the scene during a saccade.^[24] This is why objects may seem somewhat blurry during eye movements.

Posterior parietal cortex

The posterior parietal cortex (PPC) is a cortical area that is located in front of the parieto-occipital sulcus and is known to play a role in spatial awareness for eye and arm movements. A study using transcranial magnetic stimulation (TMS) found that the PPC also plays an important role in the amount of information held across saccades.^[25] It is believed that transsaccadic memory has the ability to hold roughly three to four items across each saccade with a fully functional PPC. When the PPC is temporarily disrupted, especially the PPC of the right hemisphere, memory across saccades is significantly weakened. Disruption of the PPC leads to a reduction in the ability to store items from a span of three or four, to one item. Further research needs to be conducted in order to fully understand how the PPC is synthesized with object recognition within transsaccadic memory.^[25]

Experiments

Random Dot pattern

Irwin's early experiments tested participants ability to identify random dot patterns as the same, or as different across saccades. The control condition for this experiment presented dot patterns in the same spatial location without saccades, meaning participants had a single fixation point. A no-overlap control condition presented dot patterns in different spatial locations, while participants maintained fixation on one point. This tested the ability of visual short term memory in identifying two individual patterns.^[5] The experimental condition showed dot patterns in the same spatial location but were viewed at separate fixations, forcing the use of transsaccadic memory.^[5] For the experimental condition, participants underwent a calibration phase, where they were shown five points in separate location to fixate on individually, for less than two seconds. The next phase presented a single fixation point for less than two seconds, which was followed by a random dot pattern presented in a different location, acting as a saccade target. The dot pattern disappeared when the saccade was initiated. Another dot pattern then appeared in the same location. Participants had to identify whether the two patterns were the same or different. Results of the experiment showed performance of the experimental condition to be slightly less accurate than the no-overlap condition. Irwin attributed this decrease in accuracy to the extra neural processes involved in performing an eye movement, combined with processing the dot pattern. He concluded from this that transsaccadic memory does exist, but that it is very similar if not identical to short term visual memory and less similar to sensory memory.^[5]

Saccade target experiment

The next step for Irwin was to determine if transsaccadic memory is tied to spatial position. He performed experiments using dot patterns and later, performed the same type of experiment alongside McConkie in his effort to prove the saccade target hypothesis. In those experiments, the researchers used a picture instead of dot patterns. Participants in the experimental condition were given an initial eye position, followed by a saccade target position on the picture. Unlike Irwin’s first experiment, where the pattern did not change spatial location during the saccade, the picture underwent one of three shift conditions: All shift down, target object shift down, or background shift up.^[3] Detection of change ended up being much higher when only the target object moved compared to when the entire image shifted. This demonstrated the importance of the target object in detection of image displacements.^[3]

Drawing Attention to an Eccentric Target

Kowler and colleagues were among many to test the relationship between attention and saccades. In one of their first experiments they hypothesized that the initiation of saccades required a signal that would determine which point in space would be the target for the saccade. They questioned whether the signal was attributed to attention or if saccades held their own independent filter. If attention was what decided the location of the target then the saccade latency period would be longer for saccades made away from the target. Additional time would be required to shift attention away from the false target to the correct target. The apparatus was constructed of eight letters arranged around the perimeter of a circle with a cross fixated at the center. Three frames were presented at each trial, the first called the pre-mask held eight letters for 500 ms, the second, labeled critical frame contained seven characters and a numerical (0-9) saccade target and was held for 200 ms. The last frame, post-mask, contained all of the letters. Each trial the identity of the letters, numerical saccade target and the location of the numerical saccade target were chosen at random and blind to the participants. Participants focused on the cross fixation and pressed a button at the start of the trial. After 100 ms a sequence of the three frames were presented, participants were then instructed to make a saccade to either the numerical saccade target or to an opposite character. Once the saccade target was identified participants pressed a button. Results supported the hypothesis, showing that directed attention towards the saccade target made the target easier to identify, decreased latency period and improved on accuracy.^[12]