We tested the applicability and reliability of the RIBS-J in a Japanese setting, also evaluating several aspects relating to the validity of the RIBS-J. This study adds new evidence for the psychometric properties of the RIBS, particularly regarding convergent validity, divergent validity, and goodness of model fit.
Applicability in Japan
Other than item 8 of the second subscale, the distribution of responses was not skewed for the other items of the RIBS-J. A ceiling effect was observed for item 8 (mean = 4.21, SD = 0.91); however, Evans-Lacko et al. also observed a ceiling effect for this item (mean = 4.04, SD = 1.02) in a study conducted in the UK. In other words, the responses given by our sample may have been somewhat similar to those provided by the community sample used in the UK study.
While the score on the RIBS-J showed no association with sex, major subjects, or academic grades (undergraduate/postgraduate), we found a weak correlation between age and scores on the first subscale of the RIBS-J, which addresses participants' past and present experiences. Not surprisingly, older participants had more opportunities to have encountered various individuals, including people with mental health problems, in their lives, as opposed to younger participants. Therefore, we cannot assume that the weak correlation between age and scores on the first subscale of the RIBS-J completely nullifies the validity and applicability of the RIBS-J in Japan.
The internal consistency of the second subscale of the RIBS-J was quite high, and satisfactory (α = 0.83) considering that, in general, the minimum threshold is 0.7. In addition, overall test–retest reliability for the RIBS-J (ρc = 0.71) was acceptable, when taking into account past studies aimed at developing scales measuring mental health-related stigma.[14, 18, 20] In addition, it has been noted that the Cronbach's alpha and Lin's concordance statistic obtained in this study were similar to those reported for a study conducted in the UK (α = 0.85 and ρc = 0.75).
This study illustrates the originality of the RIBS-J in its assessment of behavior and behavioral intentions through the divergent validity test. Low correlation coefficients between a measure and other conceptually different measures are given as evidence of divergent validity. The lack of correlation between the first subscale of the RIBS-J and either the MAKS or the SDSJ means that the first subscale of the RIBS-J does not measure participants' knowledge and attitudes, and thus indicates good divergent validity of the first subscale of the RIBS-J. The second subscale of the RIBS-J showed a weak correlation with the MAKS. In relation to this, past studies have also reported associations between participants' knowledge of mental health issues and their behavioral intentions or social distance towards people with mental health problems.[12, 24] Therefore, it seems that the weak correlation, rather than strong or moderate correlation, observed between the second subscale of the RIBS-J and the MAKS further indicated the divergent validity of the RIBS-J.
A significant correlation was found between the second subscale of the RIBS-J and the SDSJ. Strictly, the second subscale of the RIBS-J does not measure behavior, as such, but participants' future intentions to be in close proximity to individuals with mental health problems in four given situations. The SDSJ measures participants' attitudes or feelings, assuming participants' desire for social distance from individuals with schizophrenia in five particular situations. In other words, the second subscale of the RIBS-J partially resembles a social distance scale. It is assumed that a relatively high correlation between the second subscale of the RIBS-J and the SDSJ ensures good convergent validity of the RIBS-J.
On the other hand, the difference in the target conditions between RIBS-J (mental health problems) and SDSJ (schizophrenia) may affect the results. Generally, the general public tends to hold more stigmatizing attitudes towards people with schizophrenia than those with other psychiatric disorders, like depression, not only in Japan but also in Western countries.[11, 25, 26] It is presumed that higher correlation coefficient would be found, if we employed other measures for people's desire for social distance from people with mental health problems rather than schizophrenia.
Overall, this study obtained good model fit for the RIBS-J. Although the χ2 for goodness of fit proved significant, when sample size is enlarged, the power to detect statistically significant differences also increases.[27, 28] In short, the fact that the χ2 yielded statistically significant results does not always indicate that the model is poor. Indeed, other indices of model fit for the RIBS-J in this study seemed to yield good model fit values; the minimum threshold of the GFI, AGFI, and CFI were more than 0.9, while the RMSEA was less than 0.1.
The standardized path coefficients in seven of eight items were higher than 0.4 and all the standardized path coefficients were significant. The reason for the relatively low coefficient found for item 3 is unclear. However, it is assumed that this low coefficient may be due to the inherent differences in the constructs measured by item 3 and the other seven items. For instance, item 3 requires participants to provide answers about an unfamiliar person (a neighbor), whereas the other three items in the first subscale of the RIBS-J specifically relate to closer acquaintances, such as family members, friends, and colleagues. In Japanese culture, people often hesitate to mention distant acquaintances and their illnesses. Moreover, because communication with neighbors is relatively low in large cities, such as Tokyo, in Japan, people may not be well acquainted with their neighbors. As a result, the content of item 3 in relation to broader Japanese culture may have contributed to the relatively higher frequency (24.6%) of the ‘don't know’ response compared to other items; in turn, this may have influenced the low coefficient that was obtained for this item.
Some limitations of the study were identified. First, as opposed to a community sample, participants in this study comprised university students who attended a lecture on ‘stress management’; thus, the study's findings may not apply to the general public. Accordingly, the generalizability of the results could pose a challenge in the future.
Second, we used the MAKS to test the divergent validity of the RIBS-J, because the same research team in the UK developed these two scales.[14, 18] However, the psychometrics of the MAKS in a Japanese setting is unclear. Therefore, we may not properly evaluate the divergent validity between RIBS-J and MAKS.
The third limitation is the issue of social desirability. It cannot be denied that students may have reduced the extent of self-disclosure when completing the questionnaires due to social desirability distortion. To counter this, Evans-Lacko et al. conducted an online survey for their UK study to test the psychometric properties of the RIBS, as opposed to conducting interviews. This was because an online survey enhances participants' perception of anonymity as compared to interviews, leading to a reduced tendency to give socially desirable responses. Although this study was not conducted online, its findings were similar to those obtained in the UK study, particularly with regard to the distribution of responses and reliability values. These similarities indicate that the RIBS-J may not be strongly affected by participants' social desirability distortion. Despite this, we are not certain as to how social desirability influenced participants' scores on the RIBS-J in this sample.
We tested the psychometric properties of the RIBS-J, and found that most items did not have a floor or ceiling effect. Good internal consistency and reasonable test–retest reliability of the RIBS-J were obtained, similar to the UK study. Good construct validity, including convergent validity, divergent validity and good model fit, were also reported. Therefore, we consider that the RIBS-J is an appropriate and psychometrically robust scale for assessing behavior for mental health-related stigma in a Japanese university setting.