Takashi Okada, MD, PhD, Department of Neuropsychiatry, Kyoto University Graduate School of Medicine, 54 Shogoin-Kawaharacho, Sakyo-ku, Kyoto 606-8507, Japan. Email: firstname.lastname@example.org
Aims: The neural substrate for the processing of gaze remains unknown. The aim of the present study was to clarify which hemisphere dominantly processes and whether bilateral hemispheres cooperate with each other in gaze-triggered reflexive shift of attention.
Methods: Twenty-eight normal subjects were tested. The non-predictive gaze cues were presented either in unilateral or bilateral visual fields. The subjects localized the target as soon as possible.
Results: Reaction times (RT) were shorter when gaze-cues were congruent toward than away from targets, whichever visual field they were presented in. RT were shorter in left than right visual field presentations. RT in mono-directional bilateral presentations were shorter than both of those in left and right presentations. When bi-directional bilateral cues were presented, RT were faster when valid cues were presented in the left than right visual fields.
Conclusion: The right hemisphere appears to be dominant, and there is interhemispheric cooperation in gaze-triggered reflexive shift of attention.
EYE GAZE PROVIDES a vast amount of social information in everyday life.1 Deficits in eye gaze processing have been shown to severely impair social interactions in individuals with psychiatric disorders, such as pervasive developmental disorder and schizophrenia.2 Another person's gaze enables us to infer her or his emotional status, interests, and intentions.3 Sudden gaze shifts of another person could tell us the occurrence of crucial events in the environment.4 It is reasonable that we have a tendency to direct attention to where another person is looking. This tendency – joint attention – was thought to be a representational process due to the fact that the gaze direction provides a reliable cue to the presence of important events.5 Recently, however, it has been discovered that the shift of attention will occur reflexively even if the gaze direction does not predict any relevant events in the environment.4,6,7
Given the importance of reacting automatically to gaze directions, one might expect that we have a hard-wired circuit in the brain for perceiving gaze direction. Many researchers have identified neural systems specializing in gaze processing.1 Single-cell recording and brain lesion studies in monkeys, and a clinical report have suggested the involvement of the superior temporal sulcus (STS) in the perception of gaze direction.8–10 Another neurophysiological study showed the involvement of the temporoparietal projection from the STS to the intraparietal sulcus (IPS) in these processes.11 Neuroimaging in humans demonstrated the involvement of neural networks including the STS, IPS, and amygdala in gaze perception.12–22 Brain lesion studies in humans also reported the involvement of the STS and amygdala in the processing of gaze direction.23–27
Many human cognitive functions, such as language processing, are dominantly processed in a single hemisphere. Regarding the shift of attention in response to gaze direction, a recent study with two split-brain patients suggested that it is lateralized to a single hemisphere.5 Functional neuroimaging also suggested that the changes in gaze direction preferentially activated the right STS compared with the left STS, although the difference was not significant.12,13 Our recent report also suggested that gaze direction was dominantly processed in the right hemisphere.28 One of the possible explanations was, however, that the right hemispheric dominance in processing faces was suggested on the basis of these behavioral data in patients or normal human subjects. Laterality in face processing was suggested by the findings that bilateral as well as unilateral (often right-side) damage to inferior occipitotemporal region lead to prosopagnosia,29–31 and that functional neuroimaging found stronger activations in the right than left occipitotemporal regions while viewing faces in normal populations.32,33 Kingstone et al. reported that the attentional shift by gaze was weakened by inversion of faces, although eyes themselves also trigger the reflexive attention.5 These results suggested that gaze processing was influenced by the other facial components, being dominantly processed in the right hemisphere. Although Okada et al. suggested that the right hemispheric dominance in viewing gaze presents in isolation regardless of the hemispheric dominance in face processing,28 it is not convincing because there was no comparison of performance in perceiving gaze direction versus presence of other facial components. Moreover, none of the previous studies investigated interhemispheric cooperation. In the present study, we investigated whether (and if confirmed, in which hemisphere) gaze direction with or without other facial components was dominantly processed in a single hemisphere in a normal human population.
Subjects were recruited by announcement at university. Twenty-eight paid volunteers (17 male, 11 female) participated in the experiments. All subjects were right-handed, as assessed on the Edinburgh Handedness Inventory.34 The average age of the subjects was 23.4 ± 4.3 years (range, 19–34 years). All subjects were in good health, on no medication, and had no past history of psychiatric, neurological, or ophthalmologic disease. All subjects had normal or corrected-to-normal visual acuity. Subjects were unaware of the goal of the experiments and of the nature of the experimental conditions. Testing time was approximately 45 min. Informed consent was obtained after the procedure had been fully explained.
Stimuli were presented on a flat type 19-inch CRT monitor. The refresh rate of the monitor was set to 100 Hz. The resolution of the monitor was 1024 × 768 pixels. The presentation of stimuli was controlled using SuperLab Pro version 2.0 (Cedrus, San Pedro, CA, USA). This software allowed stimulus presentation within one screen refresh cycle (i.e. 10 ms) by setting up a new graphic page in the background of the screen. Reaction times (RT) and accuracy measures were based on responses through Cedrus RB-400 Response Box (Cedrus). Subjects were seated 57.3 cm from the monitor with a headrest to keep their heads fixed, and the experimenter ensured that the subjects were centered with respect to the monitor and the keys of the switch box.
In the experiments, gaze was presented in each visual field with either of the two kinds of stimuli: face or eye (Fig. 1). Schematic stimuli of the face and eye were adopted to minimize extraneous complexities associated with real faces (e.g. face asymmetry, hair, gender etc.), as in previous studies.4,5
The face display consisted of a white background with a black line drawing of two round faces subtending 3.6°, which were located 3.9° away from vertical axis of the screen. The eye display consisted of a white background with only two pairs of eyes. The eyes, pupils, fixation cross, and targets were located at the same position as the face display.
The experiments were performed individually. The subjects were seated in an armchair and instructed to look at the monitor situated in front of them. The start of a trial was signaled by a warning alarm and the two faces were presented on a CRT monitor. After 675 ms, the pupils of each face were randomly presented above or below. After 200 ms, two faces disappeared and two target circles were presented above or below until a response was made. The stimulus onset asynchrony (SOA) of 200 ms was adopted because the performances were predicted to be the most stable based on the results of previous studies that used different SOA4,6,7 and because this SOA is appropriate for the unilateral visual field presentation paradigm. The inter-trial interval was 675 ms. The procedure of the eye condition was almost identical to that of the face condition, except for the substitution of the eye stimulus for the face one.
Subjects were instructed to indicate whether targets appeared above or below the faces by pressing the upper or lower key on the switch box with the left or right index finger. The hand for response was counterbalanced in the subjects. RT were recorded in milliseconds and timed from target onset.
The presentations of the cues and the targets are approximately divided into two conditions: the mono-directional condition and the bi-directional condition (Fig. 1). The mono-directional condition was characterized by the valid (i.e. gaze direction toward the targets) or invalid (i.e. gaze direction away from the targets) cues presented in the unilateral or bilateral visual fields. When the cue was presented in the unilateral visual field, the opposite hemisphere was dominantly stimulated. When the same directional gaze cues were presented in both visual fields, both hemispheres were equally stimulated. The mono-directional condition was subdivided into three conditions according to the visual fields in which the cues were presented: left visual field (LVF), right visual field (RVF) and bilateral visual field (BVF) conditions. The bi-directional condition was defined as the condition in which the valid cue was presented in one visual field and the invalid cue was presented in the other visual field. The bi-directional condition was subdivided into two conditions: LVF (valid)–RVF (invalid), when the valid cue was presented in the LVF and the invalid cue was presented in the RVF; and LVF (invalid)–RVF (valid), when the invalid cue was presented in the LVF and the valid cue was presented in the RVF.
At the beginning of the experiments, subjects received 32 practice trials. After the practice trials, three blocks of 32 test trials were conducted twice for each stimulus condition (all together 384 test trials). The order of the test trials was randomized within each block. Short rests for approximately 15 s were interposed between blocks of test trials, and long rests for several minutes were interposed after three blocks of test trials were finished. The stimulus type conditions were tested in different blocks. The order of the stimulus type was randomized at first and then fixed for all subjects. After the practice trials, the first three blocks of test trials were carried out in the face condition, followed by the first three blocks of trials in the eye condition. Then, the second blocks of trials in the face and eye were performed. The side of the arm for pressing the switch box was suggested to have had no effect on the mean RT findings because all conditions consisted of the same number of trials where targets appeared above or below for each subject. Before the beginning of the test, the subjects were informed that it was important to fix their eyes on the central fixation cross while it was presented and that the gaze direction was not predictive of the location of the targets. They were also instructed to respond as quickly and as accurately as possible to the targets. Before they began the experiments, the subjects were offered an opportunity to ask questions about the procedure.
All data were analyzed with SPSS version 11.0J (SPSS Japan, Tokyo, Japan). Incorrect responses were excluded from the analysis of RT. The mean RT in each condition was calculated for each subject, rejecting any measurements that were not included within the mean ± 2 SD as artifacts.
The RT findings under mono-directional conditions were analyzed using a 2 × 3 × 2 repeated-measures anova performed on the mean RT with the stimulus type (face/eye), visual field (LVF/RVF/BVF), and cue validity (valid/invalid) as within-subject factors. The RT findings under bi-directional conditions were analyzed using a 2 × 2 repeated-measures anova performed on the mean RT with the stimulus type (face/eye) and cue validity (LVF (valid)–RVF (invalid)/LVF (invalid)–RVF (valid)) as within-subject factors. Post-hoc multiple comparisons were conducted using Ryan's method. Values were considered significant at P < 0.05.
The findings for male and female subjects were combined, because the initial analysis of the results indicated no gender effects (all P > 0.1).
The mean RT findings under mono-directional conditions are shown in Figure 2. The main effects were found for visual field (F(2,54) = 11.9, P < 0.001) and cue validity (F(1,27) = 123.6, P < 0.001). A significant interaction was found between the stimulus type and cue validity (F(1,27) = 6.8, P < 0.05). A significant interaction was also found between the visual field and cue validity (F(2,54) = 19.9, P < 0.001).
The interaction between the visual field and cue validity was further analyzed. First, simple main effects for cue validity were analyzed for each visual field condition. The results showed significant effects for cue validity in all of the LVF (F(1,81) = 64.3, P < 0.001), RVF (F(1,81) = 53.1, P < 0.001) and BVF (F(1,81) = 162.2, P < 0.001). This indicates that all of the right, left, and both hemispheric stimulations induced the cuing effect of gaze direction.
Simple main effects for the visual field were then tested for each validity condition. The effect for the visual field was significant only in the valid conditions (F(2,108) = 30.8, P < 0.001). Multiple comparisons were made for the significant simple main effect of the visual field in the valid conditions. Significant differences were found between the LVF and RVF (t = 2.0, P < 0.05); between the LVF and BVF (t = 5.6, P < 0.001); and between the RVF and BVF (t = 7.6, P < 0.001). This indicates that RT in the valid condition were faster when the cue was presented to the LVF than the RVF, and presented in both visual fields than when they were presented in either of them alone.
The interaction between the stimulus type and cue validity was also subjected to follow-up analysis. Significant effects were found for cue validity in both the face condition (F(1,54) = 128.5, P < 0.001) and the eye condition (F(1,54) = 91.2, P < 0.001). Significant effect was also seen for the stimulus type only in the invalid condition (F(1,54) = 6.8, P < 0.05), indicating shorter RT under invalid conditions for face stimuli than for eye stimuli.
The mean RT under bi-directional conditions are shown in Figure 3. A main effect was found for the visual field (F(1,27) = 4.6, P < 0.05).
The number of errors at each condition are shown in Figure 4. In the mono-directional conditions, the effects of the cue validity were analyzed for each condition, using the Wilcoxon signed-rank test (two-tailed, P < 0.05). Significantly less errors were made in the valid than in the invalid conditions when gaze direction was presented to the BVF in the eye condition (z = 3.2, P < 0.001); when the gaze direction was presented to the RVF in the face condition (z = 2.0, P < 0.05); and when the gaze direction was presented to the BVF in the face condition (z = 3.0, P < 0.01). In the bi-directional conditions, the effects of cue validity and visual field were analyzed, using the Wilcoxon signed-rank test. All results showed no significance (all P > 0.1). In sum, more errors were found in the invalid than in the valid conditions in both the face and eye conditions, and therefore the faster RT in the valid conditions, in comparison to the RT in the invalid conditions, cannot be explained by the speed–accuracy trade-off phenomenon.
In the current study, we presented normal subjects with gaze directions in each visual field separately and demonstrated that, in both the face and eye conditions, the RT were significantly faster when gaze was toward than away from the targets, whichever visual field the cue was presented in. These findings confirm those of previous studies in which gaze direction was presented in the center of the screen, and which showed that the shift of attention in response to the gaze direction occurs even when the direction did not predict the location of the target.4
In addition, the present study has demonstrated that (i) in valid conditions, RT were significantly faster when the cue was presented to the LVF than to the RVF in both the face and eye conditions; and (ii) in bi-directional conditions, the RT were significantly faster when the gaze in the LVF was toward the targets (i.e. the gaze in the RVF was away from the targets), when compared with the RT when the gaze in the RVF was toward the targets (i.e. the gaze in the LVF was away from the targets) in both the face and eye conditions. These findings strongly indicate that the gaze-triggered reflexive shift of attention was dominantly processed in the right hemisphere.
The present finding indicating the right hemispheric dominance of attentional shifts in response to gaze direction is in line with the literature on face-related processing. A previous neuropsychological study reported that a prosopagnosic patient with a right occipitotemporal lesion was deficient in determining the gaze direction, whereas a patient with a left occipitotemporal lesion could do it normally.35 Some neuroimaging studies showed that the changes in gaze direction activated the right STS more than the left.12–14,16,19 Behavioral neuropsychology studies in normal participants used the cuing paradigm with facial stimuli and reported that gaze direction was dominantly processed in the right hemisphere.28 The presents study of gaze direction versus the presence of facial components confirmed and extended the notion that gaze per se is processed dominantly in the right hemisphere.
Another important finding of the present study was the presence of bilateral coordination in gaze processing. Not only the right hemispheric dominance, but the current RVF presentations of stimuli (i.e. left hemispheric stimulations) also showed a significant cuing effect of gaze direction. Moreover, the RT were significantly faster when the cues were presented in both visual fields than when they were presented in either of them alone. These findings suggest the existence of left hemispheric processing and bilateral hemispheric coordination in response to gaze direction. In contrast, a previous neuropsychological study with two split-brain patients reported that the attentional shift in response to gaze direction was lateralized to a single hemisphere predominantly processing the face (i.e. one patient was right and the other was left hemisphere).5 Some neuropsychological studies, however, have reported that the occurrence of prosopagnosia requires bilateral damage in the occipitotemporal cortices in most cases, suggesting that the neural substrate of face processing may not be lateralized to a single hemisphere.29 Concomitant with this view, several neuroimaging studies have reported bilateral activation of the STS regions in response to gazing.15,17–19,21 Thus, it is suggested that gaze direction is processed not only in the right but also in the left hemisphere, and that there exists a functional coordination of both hemispheres in gaze processing.
In addition, there was no significant difference between the RT in the LVF, RVF and BVF in the invalid conditions. That is, in contrast to the aforementioned valid conditions, the RT were not delayed in the bilateral hemispheric stimulations, in comparison to the RT in the unilateral hemispheric stimulations. This is in line with the previously reported property of exogenous attention orienting, which has benefit at only the valid location without costs at the invalid location.4 Some studies, however, have suggested that the reflexive orienting in response to gaze direction has some specific merit such as the loss of inhibition of the return phenomenon.4 Thus, the current findings may also be a specific effect of gaze processing. Regarding the differences between face processing and eye processing, Campbell et al. demonstrated the dissociation between face recognition and gaze perception abilities in two prosopagnosic patients, suggesting that these two aspects of face processing proceed independently in humans.35 It was also demonstrated that the cuing effect of gaze direction in schematic faces was weakened when the faces were inverted,5 which suggests that face processing has a considerable effect on gaze processing, although both processes may proceed via different mechanisms. In the present study, in the invalid conditions, the RT in the face condition were significantly faster than the RT in the eye condition. Thus, the present findings support the idea that the process of face perception affects the process of gaze perception.
Promising directions for further investigation include comparison between gaze versus non-gaze (e.g. symbolic) stimuli for the hemispheric dominance and interhemispheric cooperation in reflexive attentional shift. Behavioral studies indicated that, as in the case of eye gaze, reflexive attentional shifts can be triggered by symbolic cues such as arrows.36–39 Some neuroimaging studies reported that different brain regions were active for the reflexive attentional shifts triggered by gaze and symbols.16,40 Other neuroimaging studies, however, found a commonality in the neural activation for the attentional shifts by gaze and by symbol.19,20 This issue remains unsettled, and behavioral investigations on hemispheric functional differences using the present paradigm would provide interesting insights regarding this issue.
In summary, the present findings demonstrate (i) that gaze-triggered reflexive shift of attention is dominantly processed in the right hemisphere; and (ii) that there exists a functional interhemispheric cooperation in processing of gaze direction.