SEARCH

SEARCH BY CITATION

Keywords:

  • affective stimuli;
  • emotional sounds;
  • exploratory eye movements;
  • schizophrenia;
  • visual recognition

Abstract

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. REFERENCES

Aims:  Emotion-associated sounds have been suggested to exert important effects upon human personal relationships. The present study was aimed to characterize the effects of the sounds of crying or laughing on visual cognitive function in schizophrenia patients.

Methods:  We recorded exploratory eye movements in 24 schizophrenia patients (mean age, 27.0 ± 6.1 years; 14 male, 10 female) and age-matched controls. The total eye scanning length (TESL) and total number of gaze points in the left (left TNGP) and right (right TNGP) visual fields of the screen and the number of researching areas (NRA) were determined using eye-mark recording in the presence/absence of emotionally charged sounds.

Results:  Controls' TESL for smiling pictures was longer than that for crying pictures irrespective of sounds. Patients' TESL for smiling pictures, however, was shorter than for crying pictures irrespective of the sounds. The left TNGP for smiling pictures was lower in patients than controls independent of sound. Importantly, the right TNGP was significantly larger with laughing sounds than in the absence of sound. In controls, the NRA for smiling pictures was significantly greater than for crying pictures irrespective of sound. Patient NRA did not significantly differ between smiling and crying pictures irrespective of sound.

Conclusion:  Eye movements in schizophrenia patients' left field for smiling pictures associated with laughing sounds particularly differed from those in controls, suggesting impaired visual cognitive function associated with positive emotion, also involving pleasure-related sounds, in schizophrenia.

HUMANS RECEIVE A wealth of sensory information in order to adapt to their environment. Specific information is selected from a wide variety of visual and auditory stimuli, and then processed for behavioral guidance. Affect and emotion are critical elements of human relationships: generally, negative emotions impair the quality of life, while positive emotions make life pleasurable.1,2 Emotion-associated sounds such as crying or laughing have been suggested to exhibit important effects upon human behavior over the course of interpersonal relationships. Laughing or pleasure-related sounds would be expected to strengthen relationships, while crying or angry sounds may inhibit relationships to some event.3,4 Even though emotionally laden sounds such as pleasurable music may enhance a variety of cognitive functions, such as attention, learning, communication, and memory both in healthy subjects and those with schizophrenia,5 little is known about the processing of affectively relevant environmental sounds. In particular, no studies have analyzed the effects of emotionally charged sounds on exploratory eye movements (EEM), which reflect human relationships.

Observing that schizophrenia patients have deficient positive emotion perception, Loughland et al. suggested that this deficit might reflect a failure to integrate visual information.6 Phillips and David reported that schizophrenia patients had dysfunction of attention to the left field of stimuli.7 Nishiura et al. recently reported that the positive affective content of viewed images reduced EEM in schizophrenia patients.8 Moreover, this impairment particularly involved scanning of the left field of the screen. These findings suggest the possibility of impaired visual information processing in schizophrenia patients, especially concerning positive emotion-associated visual stimuli presented in the left visual field.

Kurakake recently reported that in schizophrenia patients, sounds of laughing slowed responses to visual stimuli depicting positive emotion, while those sounds accelerated such responses in healthy subjects.9 We therefore suspected that, in schizophrenia patients, laughing sound might result in events concerning cognitive emotional processing that differed markedly from response in healthy controls.

The responsive search score (RSS) is a parameter indirectly indicating the degree of achieving an active preparatory state for exploratory behavior, such as reconfirmation of the figure presented to the eyes in response to a question such as ‘Is there any other difference?’10,11 In contrast to other examination designs, this examination involves verbal communication between the examiner and subject through an interview. Accordingly, the RSS is an objective parameter for assessing responses to personal communication. Therefore, in addition to the total eye scanning length (TESL) and total number of gaze points (TNGP) used so far,8,12–14 we also examined the number of researching areas (NRA), which was made on the model of RSS.

In the present study we investigated the effects of emotionally laden sounds such as laughing or crying on EEM concerning visual scanning of the screen, comparing healthy controls and schizophrenia patients using an eye-mark recorder. A second aim was to determine whether any particular eye movement response affected the NRA, reflecting human relationships.

METHODS

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. REFERENCES

Subjects

The schizophrenia patients consisted of 24 outpatients (mean age, 27.0 ± 6.1 years; range, 18–44 years; 14 male, 10 female) diagnosed according to ICD-10 by two psychiatrists. Schizophrenia was of the paranoid type in 18 patients, and non-paranoid type in six patients. Twenty-four healthy paid volunteers (mean age, 27.9 ± 6.7 years; range, 20–48 years; 10 male, 14 female) were enrolled, and matched with the schizophrenia group for age. All healthy subjects were interviewed in advance to ensure that they had no family history and no history of psychiatric or neurological disease or of drug addiction. The mean years of education for the schizophrenia group was 11.7 ± 2.6 and that for the controls was 14.4 ± 1.3. All subjects gave written informed consent for study participation. The Ethics Committee of Kurume University approved the present study. All patients were treated with neuroleptics. The clinical state of all patients was assessed using the Positive and Negative Symptom Scale (PANSS)15 by two psychiatrists within 1 week after eye movement recording. The positive symptom score was 23.3 ± 5.9 and the negative symptom score was 20.3 ± 4.6. When scores differed between the two psychiatrists, the final scores were determined by consensus before being analyzed. All patients were treated with neuroleptics (mean daily dose of chlorpromazine equivalent, 269.4 ± 107.5 mg/day). All healthy controls and patients were right-handed and had normal visual and auditory functions.

Apparatus and stimuli

Four kinds of pictures were used (Fig. 1a). We used two symmetrical babies' faces to expand the eye scanning and increase gaze points and also to evaluate the differences between left and right scanning of the screen. Babies' faces were used to recognize accurate affect for all subjects; adult faces cause strong and sustained affective stimuli for patients and sometimes do not yield an eye mark. Also, for babies's faces it is hard to distinguish sex differences. Picture 1 showed two symmetrically positioned smiling babies. Picture 2 depicted two symmetrically positioned crying babies. Picture 3 presented two symmetrically positioned smiling babies with dots flanking the mouth. In picture 4, there were two symmetrically positioned crying babies with dots flanking the mouth. The crying baby sounds had a range of 125–323 Hz and a sound power level from 65 to 75 dB. The laughing baby sounds had a range of 292–306 Hz and were from 68 to 75 dB. All subjects could hear this maximum sound level of 75 dB generated by two speakers positioned in front of them. The pictures were projected onto a screen to form images 120 cm wide and 90 cm high. The maximum angles of sight lines were 40° horizontally and 24° vertically. Each session consisted of a series of two pictures, each presented for 15 s. The picture and the sound were presented simultaneously.

image

Figure 1. (a) Four kinds of pictures. (b) Absence of sound and (c) presence of sound for a typical series of exploratory eye movements in a healthy subject (upward) and a schizophrenia patient (downward). Each dot indicates a gaze point and each line, a movement.

Download figure to PowerPoint

Eye movements were recorded using an eye-mark recorder (nac, EMR-8, Tokyo, Japan). Movement >1° with duration >100 ms was scored as an eye movement. Recorded data were assessed on computer-assisted analysis in terms of two parameters: TESL and TNGP, as reported previously.8,12–14 In the present study, we subdivided TNGP into the left and right halves of the screen (left and right TNGP).8 The NRA was calculated from the total scores for 20 subdivided areas during the first 5 s. Eight areas were included in regions of interest in the present study, essentially similar to previous studies.10,11

Procedure

In a darkened room, where visual and auditory sensory stimuli were attenuated, eye movement was recorded using the eye-mark recorder. Subjects were instructed to identify the emotional valence each picture exactly as it was presented. They were also instructed to fix their gaze at several corner points to check their eye movements in order to evaluate neurological deficits and low-level eye movement as a reflex organic saccade. No subjects showed deficits under the present procedure.

Recording was performed as follows using two kinds of babies' faces (crying or smiling). All subjects completed four conditions in the following procedure (crying babies' picture with or without crying sounds; smiling babies' picture with or without laughing sounds). The smiling and crying conditions were equally conducted in a random order.

In session 1 (memorization task), each subject was directed to view the picture carefully. The subject was then shown the original babies' faces. In session 2 (confidence task), the subject was instructed to compare the picture they saw previously with a subsequent one, and they were then shown the same picture. Immediately after viewing it, the subject was asked whether it differed from the picture they saw first. In session 3 (comparison task), the subject was instructed to compare the picture they saw previously with a subsequent one, and then they were shown a slightly different picture with a dot on the baby's face. Immediately after viewing it, the subject was asked whether it differed from the picture they saw first. In session 4 (repeat comparison task), after the subject had replied and while a slightly different picture with a dot on the baby's face was still being shown, they were asked, ‘Are there any other differences?’

Statistical analysis

The present study considered only data obtained from the confidence task (session 2) and repeat comparison task (session 4), because confidence and repeat comparison tasks have been reported to best reflect visual cognitive function.8 Three-way analysis of variance (anova; sound, group, and stimuli) was performed. In addition, using TNGP analysis, four-way anova (sound, group, stimuli, and visual field) was performed. Post-hoc analysis was conducted using Scheffe's test. Pearson's correlation coefficient was used to identify significant relationships between symptom scores and measures of eye movement. P < 0.05 was taken to indicate significance.

RESULTS

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. REFERENCES

Figure 1 shows representative sequences of EEM from a schizophrenia patient and healthy control obtained while viewing two pictures in session 2. The control subject's eye movements were coherent, focusing on the baby's eyes and mouth in both left and right visual fields (a double inverted triangle pattern). The patient's eye movements seemed to be random, and there appeared to be no organizing strategy, but movements were relatively limited in aggregate length.

TESL in the confidence task

Figure 2 shows the TESL obtained from controls and schizophrenia patients while viewing two facial expressions with and without sounds. Three-way anova (sound × group × stimuli) demonstrated that there were no significant sound- or stimuli-related effects, but did identify a significant difference between groups (F = 165.92, P < 0.0001). A significant interaction was observed between group and stimuli (F = 18.4, P < 0.0001). Post-hoc comparison indicated significant differences between patients and controls when viewing smiling babies, both with (P < 0.0001) and without (P < 0.0001) sounds. Furthermore, for crying babies, the difference was significant between patients and controls, both with sounds present (P < 0.0001) and absent (P < 0.0001). In controls, there was a significant difference between TESL for smiling and crying babies, both with (P < 0.05) and without (P < 0.05) sounds. In patients, there was a significant between TESL for smiling and crying babies, either with (P < 0.01) or without (P < 0.05) sounds.

image

Figure 2. Total eye scanning length (TESL) vs groups and stimuli for the smiling faces (□) and the crying faces (inline image). Data are given as mean ± SEM. (a) Absence of sound; (b) presence of sound. *P < 0.05, **P < 0.01, ***P < 0.001.

Download figure to PowerPoint

TNGP in left and right visual fields of the screen

Figure 3 shows the TNGP with and without sounds in two stimuli in controls and schizophrenia patients. Four-way anova (group × sound × stimuli × field) indicated a significant main effect of group (F = 74.9, P < 0.0001) and field (F = 15.1, P < 0.001). Interactions between group and stimuli (F = 9.7, P < 0.01), group and field (F = 7.6, P < 0.01), stimuli and field (F = 5.8, P < 0.05), group, stimuli, and field (F = 4.0, P < 0.05), and sounds, stimuli, and field (F = 4.6, P < 0.05) were observed. Post-hoc comparison demonstrated significant differences between controls and patients in left TNGP for both smiling (P < 0.0001) and crying (P < 0.01) babies without sounds, and only for smiling babies with sounds (P < 0.0001). In the right field, there was a significant difference between groups for smiling babies without sounds (P < 0.01). In patients, the differences were significant between stimuli in the left TNGP in the presence (P < 0.001) and absence (P < 0.05) of sounds, and in the right TNGP with sound (P < 0.05). In patients, the differences were significant between left and right TNGP for smiling babies with (P < 0.0001) and without (P < 0.05) sounds, and for crying babies without sounds (P < 0.05). In addition, there was a significant difference between presence and absence of sounds for right TNGP in patients viewing smiling babies.

image

Figure 3. Total number of gaze points (TNGP) vs sound and group for (a) left field of screen and (b) right field of the screen for the smiling faces (□) and the crying faces (inline image). Data are given as mean ± SEM. *P < 0.05, **P < 0.01, ***P < 0.001.

Download figure to PowerPoint

Number of researching areas

Figure 4 shows the representative sequences of EEM in a schizophrenia patient and healthy control in session 4 (repeat comparison task) while viewing two pictures, and the NRA with and without sounds for each stimuli in controls and schizophrenia patients. Three-way anova (group × sound × stimuli) indicated significant differences for group (F = 23.55, P < 0.0001) and stimuli (F = 9.72,P < 0.01). There was no difference observed for sounds. A significant interaction between group and stimuli was observed (F = 4.16, P < 0.05). Post-hoc comparison indicated significant differences for smiling babies between patients and controls, both with (P < 0.001) and without (P < 0.01) sounds. In controls, the difference was significant for smiling and for crying babies, both in the presence (P < 0.05) and absence (P < 0.05) of sound.

image

Figure 4. (a,b) Typical series of exploratory eye movements in a healthy subject (upward) and schizophrenia patients (downward) in session 4. (a) Absence of sound; (b) presence of sound. Each dot indicates a gaze point and each line, a movement. (c,d) No. researching areas (NRA) vs groups for the smiling faces (□) and the crying faces (inline image). (c) Absence of sound; (d) presence of sound. Data are given as mean ± SEM. *P < 0.05, **P < 0.01, ***P < 0.001.

Download figure to PowerPoint

Symptom scores and eye measures

Table 1 lists the relationship between EEM parameters and PANSS scores. A significant correlation was noted between TESL and negative symptom scores when viewing smiling babies with and without sound, and crying babies with sound. There was no correlation between positive symptom scores or drug doses and TESL.

Table 1.  EEM parameters vs PANSS scores in patients with schizophrenia
 TESL (cm)TNGP-Left (n)TNGP-Right (n)NRA
Sound (+)Sound (−)Sound (+)Sound (−)Sound (+)Sound (−)Sound (+)Sound (−)
  • *

    P < 0.05,

  • **

    P < 0.01,

  • ***

    P < 0.001.

  • EEM, exploratory eye movements; NRA, number of researching areas;PANSS, Positive and Negative Symptom Scale; TESL, total eye scanning length; TNGP, total gaze points.

NegativeSmiling−0.510***−0.521***−0.509*−0.450*−0.266−0.116−0.484***−0.535***
 Crying−0.462***−0.107−0.587**−0.496*−0.121−0.151−0.405***−0.603***
PositiveSmile−0.058−0.196−0.078−0.229−0.277−0.023−0.416***−0.359***
 Crying−0.104−0.055−0.413−0.277−0.329−0.235−0.137−0.663***

There were correlations between NRA and negative symptom scores for smiling and crying babies with and without sound. A significant negative correlation between NRA and positive symptom scores was noted for smiling babies either with or without sound, and for crying babies without sound. There was no correlation between the dose of antipsychotics and symptom scores. There was a correlation between TNGP in the left field and negative symptom scores for smiling babies with and without sound, and for crying babies with and without sound. There was no correlation between TNGP in the right field and negative symptom scores.

DISCUSSION

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. REFERENCES

The present study evaluated the effects of emotional sounds on visual cognitive function in schizophrenia patients by recording EEM. The most important finding of the present study was that emotionally charged sounds appeared to be a useful tool for investigating the characteristics of EEM in schizophrenia patients, most notably when viewing smiling pictures associated with laughing sounds. These observations highlight EEM as a biologic marker that is useful in evaluating clinical status in psychiatry. In the present study, however, some limitations should be considered. One is that we did not assess the intelligence of the subjects. It is possible that the intelligence level of the subjects might affect the results. In addition, in the present study, there were two subjects who had children in the healthy group. We used babies' faces and sounds as stimuli. Therefore, there is a possibility that whether the subjects had a child of their own affected the results. In consideration of the limitations described, each EEM parameter is discussed.

Total eye scanning length

Previous studies have reported that individuals with schizophrenia consistently show relatively more restricted scanpaths for faces, consisting of fewer fixations of a longer duration, and a shorter distance between fixations when viewing facial expressions.6–8,12 In the present study, TESL in patients was significantly shorter than that in controls for both smiling and crying babies irrespective of sound, consistent with previous evidence. In patients, TESL for crying was significantly longer than that for smiling babies, while the opposite was true for TESL in controls. Loughland et al. reported that schizophrenia patients had an impairment of scanning, and this was evident for happy and neutral faces rather than sad faces.6 Those authors concluded that impairment of scanning in schizophrenia patients might reflect a failure to integrate facial stimuli due to dysfunction of the synchronization between local and global processing, especially when viewing positive or neutral faces. Thus, the present findings of significant differences between responses to positive (smiling babies) and negative (crying babies) visual stimuli may be due to differences in the level of attention given to facial features; patients may pay less attention to smiling babies. An unexpected finding was the absence of a significant difference for sound. These findings indicated that the presentation of emotion-associated sounds might not greatly influence TESL of EEM. Recently, Nishiura et al. reported that TESL in patients hearing a laughing sound while viewing smiling babies was shorter than that in controls.8 These differences may reflect methodological and subject-related differences, but TESL of EEM is a prominent characteristic expected to be impaired in schizophrenia patients.

TNGP in left and right visual fields of the screen

We subdivided TNGP into the left and right halves of the screen (left and right TNGP), but there were no significant differences noted between these in control subjects for either smiling or crying babies irrespective of sound, as reported by Nishiura et al.8 In patients, TNGP in the left field was smaller than that in the right field for both crying and smiling babies without sound. This difference was greater for smiling babies with laughing sounds than for crying babies accompanied by crying sounds.

In the left field, TNGP in patients was significantly smaller than that in controls with or without sound, and also for crying or smiling babies. In patients, right TNGP for smiling babies without sound was smaller than that in controls. The right TNGP for smiling babies with sound was larger than that for crying babies. With sound, however, there were no significant differences in TNGP in the right field between patients and controls for either crying or smiling babies.

Schwartz et al. reported that for emotion-linked images, the eyes tended to move leftward, in contrast to rightward for other images: they suggested that this observation supported the notion of links between the right brain and emotional function.16 Furthermore, Schwartz et al. reported that right-handed subjects tended to look to the left when answering affective questions, similarly concluding that the right hemisphere plays a special role in emotion in the intact brain.16 The right brain has been considered to subserve emotional function and the left brain, symbolic functions.17 Phillips and David reported that schizophrenia patients had difficulty in redirecting the initial focus of attention to left-field stimuli, and proposed that left-specific scan paths are a marker of attention processes in schizophrenia patients.7 In the present study the left TNGP clearly differed between patients and controls with or without laughing sounds, in agreement with Nishiura et al.8 Thus, left-sided ocular scanning may be a particularly good marker of visual cognitive function in schizophrenia patients. Especially, this difference may be accentuated when laughing sounds accompany smiling pictures.

Most importantly, laughing sounds caused an increase in right TNGP in schizophrenia patients. Kurakake reported that emotionally charged eye-closing time in combination with happy sounds and a smiling baby picture was significantly longer than that for a picture without sound for schizophrenia patients.9 Furthermore, the amygdale shows increased activation for viewing of smiling babies with laughing sounds.18 That is, a smiling baby picture accompanied by laughing sounds may evoke more negative emotion for patients. Although it seems that the eyes tended to move leftward for emotional stimuli as indicated here, the right TNGP in patients increased. The present results indicate that schizophrenia patients may have a dysfunction of the right hemisphere, especially when receiving pleasure-related stimuli with sounds.

Number of researching areas

The NRA has been reported to be the best marker to evaluate visual cognitive function and a trait marker in schizophrenia patients.10 In the present study the NRA in patients shown smiling babies was significantly smaller than that in controls both with and without sounds. There were no significant differences in the NRA with or without sound. Furthermore, there was a stimuli difference only in controls: the NRA for smiling babies was significantly larger than that for crying babies. There was no significant stimuli-related difference in the NRA in patients, indicating, along with other results, that schizophrenia patients presented with smiling faces may gaze at many points within a small area, but do not look at a wider (overall) area. These findings also indicate that the NRA was barely influenced by emotion-related sounds, suggesting that the NRA is an optimal marker for evaluating visual cognitive function in schizophrenia patients, as reported by Kojima et al.10,11

Symptoms and eye movements

Negative psychiatric symptoms have been reported to correlate negatively with EEM.12,19 There was a significant negative correlation between TESL with or without laughing sounds and negative symptom scores in the present study. TESL for crying babies was negatively correlated with negative symptom scores only in the presence of crying sounds: crying sounds may cause a decrease in TESL in patients with severe negative symptoms. A significant negative correlation between TNGP in the left field and negative symptom scores was observed with or without sounds. There was significant correlation between TNGP on the right and symptom scores. These findings strongly indicate that the left TNGP is an optimal index and that whether or not emotion-related sounds are presented is an important factor when assessing dysfunction in schizophrenia patients. In the present study we evaluated the NRA. A significant negative correlation was observed between the NRA and negative symptom scores both with and without laughing or crying sounds. Furthermore, a significant negative correlation was observed between the NRA and positive symptom scores both with or without laughing sounds, and without crying sounds.

This finding strongly suggests that EEM are useful for clinical application. Correlations between scanning measures and symptoms showed that negative symptoms were related to scanning measures such as TNGP and TESL.8,20 The authors suggest that visual organization impairment may be related to cognitive inflexibility and frontal dysfunction.

Conclusions

Scanning of the left field of the screen when viewing smiling babies with laughing sounds led to the most events to characterize visual cognitive function in schizophrenia patients. This may result from three factors. One is due to dysfunction of the scanning property in which humans first scan the left field. This dysfunction is due to an impairment of major set to event processing. The second is an impairment of right brain function concerned with event processing, because the left inner position is innervated from the right brain. Koeda et al. suggested that schizophrenia patients have impairment of right hemispheric function for human voice perception.21 Third, the amygdale increase activation when viewing smiling babies with laughing sounds only in patients with schizophrenia.18 This suggests that schizophrenia patients have a different perception of smiling babies than healthy subjects.

Finally, EEM while viewing pictures identified certain differences between schizophrenia patients and healthy controls in the left field of the screen with or without emotional sounds. Thus, EEM appear to include clinical and hemispheric functional markers, being useful for exploring human visual cognition.

REFERENCES

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. REFERENCES