Psychometric properties of the Japanese version of the Social Phobia Inventory

Authors


Correspondence: Toshihiko Nagata, MD, PhD, Mental Health Clinic of Dr. Nagata at Nanba, 3-5-8 Nanba, Chuo-ku, Osaka 542-0076, Japan. Email: toshi@nanba-nagata.com

Abstract

Aim

The aim of the current study was to study the psychometric properties of the Japanese version of the Social Phobia Inventory (SPIN-J) among Japanese subjects with social anxiety disorder (SAD).

Method

The sample consisted of 86 subjects with SAD and 86 controls. Diagnosis was based on a modified version of the Structured Clinical Interview for the DSM-IV. In addition to the SPIN-J, clinician-administered and self-rating scales, including the Japanese versions of the Liebowitz Social Anxiety Scale, the Social Phobia Scale, and the Social Interaction Anxiety Scale, were used.

Results

The SPIN-J showed adequate internal consistency (0.82–0.96) for the total and subscales. Correlations between the SPIN-J and the Liebowitz Social Anxiety Scale, the Social Phobia Scale, and the Social Interaction Anxiety Scale ranged from 0.83 to 0.89 and indicated adequate concurrent validity. A cut-off point of 22 between subjects with SAD and controls showed a sensitivity of 96.5% and specificity of 87.2%, indicating robust discriminant validity.

Conclusion

The SPIN-J showed adequate reliability and validity for use as a screening tool for social anxiety disorder in Japanese clinical settings.

SOCIAL ANXIETY DISORDER (SAD) is a frequently occurring and disabling anxiety disorder. Recent epidemiologic studies have suggested that SAD is the most prevalent anxiety disorder.[1, 2] Moreover, SAD is associated with major impairment in multiple functional domains[3] even more so in help-seeking clinical populations.[4] The disorder is characterized by an excessive fear of exposure to situations that involve potential scrutiny by others and can be subdivided into a specific subtype, such as fear of public speaking or eating in public, and a generalized subtype, in which most social situations lead to fear or avoidance.[5]

Despite acceptance of SAD across cultures, there are a number of issues unique to appreciation and screening of it in Japan. First, until recently, taijin-kyofusho (a cultural variant of SAD distinguished by a pathological amplification of the social presentation of self and its impact on others)[6] rather than SAD, has been of clinical interest.[7] Second, even among clinical populations, SAD can go unrecognized without careful assessment.[8, 9] Shyness is respected as an advantage rather than as a deficit in terms of social competence and adjustment in East Asian culture.[10] As a result, SAD patients might not think their symptoms pathological. Third, and perhaps due in part to the preceding issues, estimates of 12-month prevalence are dramatically lower in Japan (0.8%)[2] versus the USA (6.8%).[1]

The Social Phobia Inventory (SPIN)[11] is a 17-item self-rating scale that was designed to assess three dimensions of SAD: fear, avoidance, and physiological arousal. The SPIN has already been translated into French,[12] Finnish,[13, 14] German,[15] Chinese (Taiwan),[16] Spanish,[17] and Brazilian Portuguese.[18] All the above studies supported excellent psychometric properties. Compared with other scales for SAD, such as the Social Phobia Scale (SPS)[19] and the Social Interaction Anxiety Scale (SIAS),[19] SPIN is unique in containing a physiological subscale. This subscale might be important in Japan because East Asian patients with anxiety disorders tend to somaticize their symptoms.[20] We have previously translated the SPIN into Japanese (SPIN-J) and used it to assess treatment response.[21, 22] However, to our knowledge its validity and reliability have not been formally and prospectively evaluated.

Thus, the aim of the current study was to investigate the psychometric properties of the SPIN-J.

Methods

Subjects

Ninety-six consecutive subjects with SAD who sought treatment in the outpatient psychiatric clinic at Osaka City University Hospital between 2004 and 2008 were recruited for participation in this study. Ten subjects were excluded because of incompleteness of self-report scales, resulting in a sample of 86 subjects with SAD. All subjects met the DSM-IV diagnostic criteria of SAD.[5] The diagnosis of SAD was confirmed by semi-structured interview using the anxiety section of the modified version of the Structured Clinical Interview for DSM Disorders (SCID-P).[23] SAD subjects with other concurrent anxiety or mood disorders were not excluded if the onset of SAD preceded the other disorder(s) and SAD was the disorder that primarily led to functional impairment. Eighty-eight controls were recruited among age- and sex-matched staff employed and students at non-university psychiatric clinics from September to December 2010. Exclusion criteria included any past psychiatric history and non-response to screening questionnaires. Controls were screened in terms of present and past histories of mood, anxiety, and substance use disorders employing the screening questionnaire of the SCID-I/P. Two controls were excluded because of incomplete and/or incorrect responses to the instrument, leaving a final sample of 86 controls (30 nursing staff, 32 medical clerks, and 24 students). All subjects provided written informed consent before entering the study. This study was approved by the institutional review board of the Osaka City University Graduate School of Medicine.

Measurements

The Japanese version of the SPIN (SPIN-J) was developed employing a translation and back-translation procedure with the original author's permission.[11] Specifically, a bilingual psychiatrist translated the SPIN into Japanese, another bilingual psychiatrist unaware of the original version back-translated it into English, the original SPIN author compared versions, and finally the Japanese version was revised accordingly.

Additional measurements for comparison included the Japanese versions of the Liebowitz Social Anxiety Scale (LSAS),[21, 24] the SPS,[19] the SIAS,[19] the Beck Depression Inventory (BDI),[25] and the State-Trait Anxiety Inventory (STAI).[26] For patients with SAD, these scales were administered on the first visit by T. N. or H. Y. For controls, these scales were administered by the second or fourth authors. SPS, SIAS, BDI and STAI were not administered to the first 24 controls, who were students.

Statistics

Demographic and background variables were analyzed using the Student's t-test and Fisher's exact test. Cronbach's alpha was used for evaluation of the internal consistency of the scales. The alpha values considered to be acceptable were those exceeding 0.70.[27] Pearson's correlation coefficient was used to assess concurrent validity between the total and subscale scores of the SPIN-J and other scales (LSAS, SPS, and SAIS). The level of the correlations detected was defined as follows: 0.51–0.70, strong; and above 0.71, very strong.

A confirmatory factor analysis using maximum likelihood estimation was conducted on the patient with SAD to evaluate the fit of the three-subscale model (fear, avoidance, and physiological subscales) and five-factor model yielded by the exploratory factorial analysis in the original developing study.[11] The following indices, with the following recommendations as cut-offs, were examined: the goodness of fit index (GFI); and the GFI adjusted for degrees of freedom (AGFI) (with values of ≥0.90, and ≥0.80, respectively, indicating reasonable fit to the data); the root mean square residual (RMR); and the root mean squared error of approximation (RMSEA) with values of ≤0.10, and ≤0.08, respectively, indicating reasonable fit.[14] Exploratory factorial analysis by principal component analysis with varimax rotation was carried out to assess the construct validity of the scales. The criteria used for factor composition were a Kaiser–Meyer–Olkin index above 0.60, a significant Bartlett sphericity test, percentage of variance of approximately 60% explained by the factors, and minimum factorial load of approximately 0.40.

A receiver–operator curve (ROC) was employed to determine the optimum cut-off point corresponding to a diagnosis of SAD.

The level of significance was set at P ≤ 0.05 in all analyses. All data were analyzed using spss 16.0 and amos 16.0 (spss, Chicago, IL, USA).

Results

There were no significant differences in the ratio of men or mean age between the SAD and control groups, as presented in Table 1. The SAD group showed significantly higher scores in all scales (SPIN-J, LSAS, SIAS, SPS, STAI, and BDI) than controls. The number of subjects with SAD and with comorbid disorders was 14 (16%) for generalized anxiety disorder, six (7%) for obsessive–compulsive disorder and two (2%) for panic disorder.

Table 1. Background demographics and clinical characteristics
 SADControlFisher's exact test or t (P)
n8686
  1. an = 62.
  2. BDI, Beck Depression Inventory; LSAS, Liebowitz Social Anxiety Scale; SAD, social anxiety disorder; SIAS, Social Interaction Anxiety Scale; SPIN-J, Japanese version of the Social Phobia Inventory; SPS, Social Phobia Scale; STAI, State–Trait Anxiety Inventory.
Male (%)35 (41)35 (41)(1.0)
Age (SD)26.9 (7.1)26.6 (5.4)0.2 (0.81)
Age of onset of SAD (SD)13.0 (4.6)  
Generalized SAD (%)80 (93)  
SPIN-J Total (SD)41.5 (11.4)11.2 (8.2)20 (<0.001)
SPIN-J Fear (SD)15.9 (4.1)4.4 (3.3)20 (<0.001)
SPIN-J Avoidance (SD)19.2 (4.6)5.4 (4.2)20 (<0.001)
SPIN-J Physiological (SD)6.5 (4.5)1.4 (1.6)9.9 (<0.001)
LSAS (SD)92.7 (21.6)29.0 (17.5)21 (<0.001)
SIAS (SD)56.3 (12.8)24.1 (13.1)a15 (<0.001)
SPS (SD)45.7 (17.2)10.5 (9.6)a15 (<0.001)
STAI State (SD)56.8 (12.7)38.4 (8.1)a13 (<0.001)
STAI Trait (SD)61.8 (12.8)39.5 (7.7)a13 (<0.001)
BDI (SD)22.8 (11.3)3.7 (4.0)a15 (<0.001)

Validity

SPIN total score and the three subscale scores (fear, avoidance, and physiological) were all significantly correlated with LSAS (n = 172), SIAS, SPS, STAI and BDI scales (n = 148) as shown in Table 2.

Table 2. Correlation between SPIN-J and other scales
SPIN-JLSASSIASSPSSTAI StateSTAI TraitBDI
  1. *P < 0.001 (Pearson's correlation coefficient).
  2. n = 172 (LSAS) and n = 148 (SIAS, SPS, STAI and BDI).
  3. BDI, Beck Depression Inventory; LSAS, Liebowitz Social Anxiety Scale; SIAS, Social Interaction Anxiety Scale; SPIN-J, Japanese version of the Social Phobia Inventory; SPS, Social Phobia Scale; STAI, State–Trait Anxiety Inventory.
Fear0.89*0.83*0.83*0.68*0.74*0.72*
Avoidance0.89*0.82*0.79*0.66*0.74*0.70*
Physiological0.68*0.62*0.75*0.60*0.63*0.60*
Total0.89*0.83*0.84*0.69*0.76*0.73*

Reliability

Cronbach's alpha coefficients as a measure of internal consistency of the total, fear, avoidance, and physiological subscales of SPIN-J were 0.96, 0.91, 0.92, and 0.82, respectively, in all subjects (n = 172).

Factor structure

A confirmatory factor analysis using maximum likelihood estimation was conducted on the patient with SAD (n = 86). As Table 3 shows, no indications of GFI, AGFI, RMR and RMSEA of both models (three-subscale and five-factor models in original developing study) reached the recommendation levels for reasonable fit. The Akaike's Information Criterion (AIC) indices for three-subscale and five-factor models were 331.82 and 248.31, respectively.

Table 3. Fit indices for three-subscale and original five-factor models of SPIN-J among the patients with social anxiety disorder (n = 86)
 GFIAGFIRMRRMSEA
  1. AGFI, goodness of fit index adjusted for degrees of freedom; GFI, the goodness of fit index; RMR, root mean square residual; RMSEA, root mean squared error of approximation.
Three-subscale model0.750.670.140.12
Five-factor model0.810.730.120.09

Exploratory factorial analysis was performed for the combined group (SAD patients and controls) and those with SAD only. The Kaiser–Meyer–Olkin index was 0.94 for all the subjects and 0.78 for only those with SAD. The Bartlett sphericity tests for the combined group and only those with SAD were significant (χ2 = 2439, d.f. = 136, P ≤ 0.001, and χ2 = 552, d.f. = 136, P ≤ 0.001, respectively). According to the Kaiser criteria, principal component analysis yielded one factor for the combined group (SAD patients and controls) that explained 59.8% of the variance and five factors for only SAD subjects, explaining 65.5% of data variance. The matrices after varimax rotation (among subjects with SAD) are presented in Table 4. The first factor was understood as ‘physiological reactions and parties.’ The second factor was explained as ‘fear of criticism and authority figures.’ The third factor included items of ‘fear of strangers and being watched.’ The fourth factor had two items of ‘avoidance of speech and attention.’ The fifth factor included ‘avoidance of criticism and embarrassment.’

Table 4. Factor loading of SPIN-J among only subjects with social anxiety disorder (n = 86)
 Current studyFactor of original study
Factors12345
Eigen value3.02.42.22.11.4
  1. SPIN-J, Japanese version of the Social Phobia Inventory.
17. Distressed by trembling or shaking (Physiology)0.760.250.13−0.060.033
13. Distressed by palpitations (Physiology)0.75−0.150.170.110.223
7. Distressed by sweating (Physiology)0.630.210.100.210.013
3. Fear of parties and social events (Fear)0.600.440.180.27−0.161
8. Avoids parties (Avoid)0.580.110.190.36−0.131
2. Bothered by blushing (Physiology)0.570.460.01−0.040.163
5. Fear of criticism (Fear)0.200.730.300.07−0.042
1. Fear of people in authority (Fear)0.090.670.040.390.004
6. Avoids embarrassment (Avoid)0.250.630.28−0.060.332
4. Avoids talking to strangers (Avoid)0.030.210.820.06−0.071
10. Fear of talking to strangers (Fear)0.350.160.79−0.01−0.021
16. Avoids talking to authority (Avoid)0.050.530.580.08−0.015
14. Fear of others watching (Fear)0.38−0.010.480.150.12
11. Avoids speeches (Avoid)0.180.050.070.890.015
9. Avoids being the center of attention (Avoid)0.150.180.040.860.115
12. Avoids criticism (Avoid)−0.04−0.090.080.340.812
15. Fear of embarrassment (Fear)0.210.30−0.19−0.270.682

Optimum cut-off score

The ROC curve comparing SAD subjects with controls is presented in Figure 1. Area under the curve (AUC) analysis based on the ROC was 0.979 (standard error = 0.009, 95% confidence interval = 0.963–0.996). A cut-off value of 22 (according to the ROC curve) distinguished between SAD subjects and controls with an appropriate balance of sensitivity (96.5%) and specificity (87.2%), as shown in Table 5.

Figure 1.

Receiver–operator curve of the accuracy of the Japanese version of the Social Phobia Inventory to distinguish social anxiety disorder and control subjects. Area under the curve = 0.979.

Table 5. Sensitivity and specificity for various cut-off scores
Cut-off scoreSensitivity (%)Specificity (%)Positive predictive value (%)Negative predictive value (%)
1997.783.785.797.3
2097.784.986.697.3
2196.584.986.596.1
2296.587.288.396.2
2394.288.489.093.8
2493.090.790.992.9
2591.993.092.992.0
2691.994.294.092.0
2791.995.395.292.1
2890.796.596.391.2

Discussion

This study provides evidence that the Japanese version of the SPIN has properties that make it appropriate for the screening of SAD among Japanese adults presenting to outpatient academic psychiatric clinics. Alpha coefficients of more than 0.7 are thought to be acceptable in terms of internal consistency, one kind of reliability. In our study population, alpha coefficients of the total and two subscales (fear and avoidance) of the SPIN-J exceeded 0.9 and the physiological subscale was 0.8, indicating high internal consistency. As for concurrent validity, correlation coefficients between the SPIN-J and related validated SAD instruments (LSAS, SIAS, and SPS) were adequate.

We found a one-factor solution in our combined group (SAD patients and controls) and a five-factor solution in patients with SAD. This is consistent with prior studies that have examined general and clinical populations. In studies using general adolescent populations, a one-factor model has been found in Finland (ages 12–16 years)[14] and Spain (ages 15–17 years).[17] Conversely, among subjects from clinical populations, a three- or five-factor structure is common.[11, 28] In the study of the development of the SPIN,[11] principal-component factor analysis among subjects with SAD also yielded five factors rather than the pre-designed three subscales.

Though the quantity of factors in the current study is similar to prior research, the quality of the factors among the patients with SAD differs somewhat from studies conducted in different parts of the world. In confirmatory factor analysis, both models (three-subscale and five-factor models in original developing study) did not reasonably fit. Alternatively, exploratory factor analysis showed that all four items of the physiological subscale and two items regarding ‘parties’ (item 3 and 8) were loaded on the first factor. On the other hand, other studies conducted in the West have found that four items of the physiological subscale made an independent third factor[11] and two items regarding ‘parties’ were loaded onto the first factor.[11, 28] These differences might be understood in a sociocultural context. Parties are uncommon social events in Japan and might evoke a physiological reaction just by anticipation. As expected, somatic symptoms (physiological subscale) were loaded onto the first factor, and somatic symptoms are important in understanding Japanese patients with SAD.

The current study closely mirrored the original study on the development of the SPIN in terms of distribution of scores between SAD and control subjects. In the study by Connor et al. on the SPIN,[11] the mean score (SD) for subjects with SAD and controls was 41.1 (10.2) and 12.1 (9.3), respectively, and in the current study the mean score (SD) for subjects with SAD and controls was 41.5 (11.4) and 11.2 (8.2), respectively. The mean SPIN-J total score was relatively higher than scores in Finnish,[13] Taiwanese,[16] Spanish,[17] and Brazilian adolescents[18] with SAD. These differences are probably because they recruited the adolescents with SAD from school populations, whereas the severity in academic medical centers tends to be higher.[29] This is evidenced by the high percentage of SAD subjects in this study with the generalized subtype. Along these lines, Connor et al. proposed 19 as a cut-off point between SAD and controls,[11] but other studies have proposed higher cut-off points: between 19 and 21,[18] 21,[17] 24,[13] and 25.[15, 16] Our suggested cut-off point of 22 for the Japanese population is therefore in the middle of cut-off points suggested by previous studies.

Important limitations of the current study bear mentioning. The study sample was modest and drawn from an academic center, and therefore the psychometric properties of the SPIN-J in the community or general population remains to be investigated. Healthy volunteers, who were health-care workers, were used as controls. Using health-care workers is perhaps akin to the routine use of undergraduate students as controls in studies of the general community, although there was the possibility that the level of functioning of controls was above average. Also, because controls were healthy, results are most applicable to practice settings – such as primary care – in which SAD is being assessed among a variety of patients with and without psychiatric histories. In the current study, SAD subjects with other comorbid anxiety and depressive disorders were not excluded. Future studies investigating the discriminate validity between SAD and other anxiety or depressive disorders, should only include subjects without other disorders.

Acknowledgments

None of the authors has any conflicts of interest to disclose.

Ancillary