Reliability and validity of a Brief Self-rated Scale of Health Condition with Acute Schizophrenia

Authors

  • Hisanori Ohata OTR, MS,

    Corresponding author
    1. Division of Psychiatric Rehabilitation, Department of Rehabilitation Science, Kobe University Graduate School of Health Sciences, Kobe, Japan
    2. Kofu Hospital of Hyogo Prefecture, Kobe, Japan
    • Correspondence: Hisanori Ohata, OTR, MS, Division of Psychiatric Rehabilitation, Department of Rehabilitation Science, Kobe University Graduate School of Health Sciences, 7-10-2 Tomogaoka Suma-ku, Kobe 654-0142, Japan. Email: kofu_worktherapy_01@pref.hyogo.lg.jp

    Search for more papers by this author
  • Kayano Yotsumoto OTR, PhD,

    1. Division of Psychiatric Rehabilitation, Department of Rehabilitation Science, Kobe University Graduate School of Health Sciences, Kobe, Japan
    Search for more papers by this author
  • Masaru Taira MD, PhD,

    1. Kofu Hospital of Hyogo Prefecture, Kobe, Japan
    2. Department of Psychiatry, Kobe University Graduate School of Medicine, Kobe, Japan
    Search for more papers by this author
  • Yoshiro Kochi MD,

    1. Kofu Hospital of Hyogo Prefecture, Kobe, Japan
    Search for more papers by this author
  • Takeshi Hashimoto MD, PhD

    1. Division of Psychiatric Rehabilitation, Department of Rehabilitation Science, Kobe University Graduate School of Health Sciences, Kobe, Japan
    Search for more papers by this author

Abstract

Aim

The aim of this study was to develop a new Brief Scale of Self-rated Health Condition with Acute Schizophrenia (BsHAS) and to examine its reliability, validity and correlation to psychiatric symptoms.

Methods

We examined the reliability and validity of the BsHAS consisting of four items (physical health condition, mood, interpersonal fatigue and interest) for 199 inpatients with acute schizophrenia and compared the patients’ subjective health conditions as assessed by the BsHAS with their objective psychiatric symptoms.

Results

Cronbach's α coefficient was 0.79, indicating that the reliability of the scale was sufficient. These four items chosen from the previous studies were approved by an expert panel, which suggested that the scale has content validity. The BsHAS total score was significantly improved at discharge. However, the effect size was only −0.24. In particular, improvement was not recognized in the interpersonal fatigue item. These findings suggest that some patients weredischarged without realizing the improvement of their health condition. The patients answered all questions without subsequently showing deteriorating symptoms, suggesting that the scale can be applied to acute-phase patients with schizophrenia. No obvious relation was recognized between the patients’ subjective health condition as assessed by the BsHAS and their objective psychiatric symptoms. This result suggests that the BsHAS can provide additional information to the objective assessment of psychiatric symptoms.

Conclusion

These results show that the BsHAS can help psychiatric professionals to know patients’ subjective health conditions, and that the longitudinal use of this scale may be useful for evaluating the degree of recovery from schizophrenia.

Recently, in a policy shift from institutional to community care in Japan, Japanese psychiatric professionals have tried to improve the quality of care in acute psychiatric inpatient care units and introduce rehabilitation programs at an earlier phase of treatment to decrease hospital long-stays and readmission.[1, 2] Accordingly, psychiatric professionals need to more closely monitor the recovery of schizophrenia patients. In our acute psychiatric inpatient care unit, we have introduced rehabilitation programs and we monitor the patients’ symptoms by using objective scales to decrease hospital long-stays and readmission. But readmission due to treatment discontinuation is not uncommon in our involuntarily admitted patients with acute schizophrenia.

In addition to objective evaluation by psychiatric professionals, subjective evaluation by patients is also important.[3-7] Fleischhacker et al. reported that schizophrenia patients’ subjective evaluation in functioning and well-being is a different outcome dimension from objective evaluation of psychopathology.[8] Thus, we hypothesized that knowing a patient's subjective health condition, that is, evaluating the effect of treatment from the patient's point of view, would contribute to the recovery process from acute psychosis. We also believed that focusing on the patient's subjective health condition would lead to the patient's positive participation in the treatment. The psychiatric professionals can determine how patients with schizophrenia grasp their own health condition by subjective evaluation scales.[9-11] To assess the health of sub-acute-phase psychiatric patients, a 13-item self-administered scale was made in Japan.[12] When the authors applied these subjective evaluation scales to involuntarily admitted patients with acute schizophrenia in our acute psychiatric inpatient care unit, many patients did not complete these scales because they were too long and complicated. We thought that an executable scale was more than necessary for involuntarily admitted patients with acute schizophrenia, and that a briefer and simpler scale was needed.

Therefore, we developed the Brief Scale of Self-rated Health Condition with Acute Schizophrenia (BsHAS) that even involuntarily admitted patients could complete. We also examined its reliability and validity and its correlation to objective psychiatric symptoms.

Methods

Development of the BsHAS

The authors (O.H., Y.K., T.M. and H.T.), two psychiatrists and two occupational therapists with more than 10 years of experience in psychiatry selected four items for the BsHAS from previous studies[9-12] with which involuntarily admitted patients with acute schizophrenia can assess their health condition. Health condition has physical, psychological and social dimensions.[13] We selected a physical health item to assess the physical dimension, a mood item to assess the psychological dimension, and an interpersonal fatigue item and an interest item to assess the social dimension. We used simple questions for each item, such as ‘How would you describe your physical health?’ for the physical health condition item, ‘How are you feeling?’ for the mood item, ‘Do you feel exhausted when other people are present?’ for the interpersonal fatigue item and ‘Have you enjoyed something, recently?’ for the interest item. The responses were graded with a five-point Likert scale. The questions and answers of the BsHAS are shown in the Appendix. The answer for each item was scored 0–4 (with higher scores indicating better health condition), and the sum of all scores was the BsHAS total score.

A psychiatric professional handed the scale and a pencil to the patient and asked them to respond to the questions on the scale based on their experiences in the last few days.

Test–retest reliability of the BsHAS

We examined the test–retest reliability of the BsHAS in a preliminary study. The test–retest reliability of the scale was assessed by repeated administration of the scale to 64 stable schizophrenia inpatients with a 1-month interval between assessments (April 2008 to May 2008). The subjects consisted of 46 men and 18 women. The median (interquartile range) age of the patients was 51.5 (22) years, the median number of hospitalizations was five (6) times, the median duration of illness was 27 (23.8) years and the median antipsychotics dose (equivalent to chlorpromazine) was 801.5 (880) mg/day. The intraclass correlation coefficients via one-way anova[14] were calculated from the BsHAS score of test and retest.

Subjects

This survey was conducted from April 2008 to March 2011. Inclusion criteria were: (i) diagnosis of schizophrenia with the ICD-10;[15] (ii) involuntary hospitalization; and (iii) participation in rehabilitation programs in the acute psychiatric inpatients care unit. The patients in this study were involuntarily admitted to our hospital with the informed consent of their guardians/family members. These patients were not competent to consent for their admission because of their severe psychiatric symptoms. When the attending doctor judged that the patient was competent to join the rehabilitation programs and this study, the doctor obtained oral informed consent to participate in the programs and the first author obtained written, informed consent to participate in this study from the patient. We thought that the risk of this study to the patients was negligible and thus did not ask guardians/family members for consent for the patients to participate in the study. Patients received all the services as usual whether they chose to participate or not. The study was performed in accordance with the Helsinki Declaration. The present study was approved by the Medical Research Ethics Committee of Kofu Hospital of Hyogo Prefecture (Rd no. 20084).

Of a total of 713 admitted patients, 333 met inclusion criteria (i) and (ii) (Fig. 1). All were inpatients. Of these patients, 225 patients were invited to participate in the rehabilitation programs, while the remaining 108 patients were not because of discharge before program participation (96 patients) or high risk of harming others (12 patients). Of the 225 patients, 26 patients refused to participate in the study, leaving 199 consenting patients in the study. None of the 199 patients dropped out of the study.

Figure 1.

Study flowchart.

The subjects consisted of 81 men and 118 women (Table 1). The median (interquartile range) age was 40 (21) years, the median number of hospitalizations was two (3) times and the median duration of illness was 10 (19) years. All subjects were taking antipsychotics with a median chlorpromazine equivalent dose of 800 (675) mg/day. The median duration from admission to inclusion in this study was 27 (25) days and the median length of hospitalization was 88 (42) days.

Table 1. Baseline characteristics of patients
CharacteristicnMedian (interquartile range)
Sex  
Male81
Female118
Diagnosis  
Schizophrenia148
Persistent delusional disorders3
Acute and transient psychotic disorders21
Schizoaffective25
Unspecified non-organic psychosis2
Age (years)40 (21)
Number of hospitalizations (times)2 (3)
Duration of illness (years)10 (19)
Job experience  
Yes148
No49
No certificate2
Education  
Graduated junior high school41
Graduated high school92
Graduated junior college22
Graduated college42
No certificate2
Living situation on admission  
Alone34
With someone165
Medication  
Only atypical antipsychotics135
Atypical+typical59
Only typical antipsychotics4
No1
Dose antipsychotics  
Chlorpromazine equivalent (mg/day)800 (675)

Study design and statistical analysis

We examined the reliability and validity of the BsHAS and then compared the patients’ subjective health condition as assessed by the BsHAS and their objective psychiatric symptoms.

After examining the test–retest reliability of the BsHAS in preliminary studies, we examined its internal reliability. The Cronbach's α coefficients[16] were calculated from the BsHAS item score at the baseline.

The content validity of the BsHAS was examined by an expert panel (five psychiatrists and one psychiatric nurse) with more than 10 years of clinical experience and who were unfamiliar with the development process of the BsHAS. Responsiveness of the BsHAS was assessed with the Wilcoxon signed rank test and Cohen's d.[17] Cohen's d is a standardized measure of effect size and provides information on the amount of change in the measure relative to the variation within the measure. Cohen's d is computed as the difference between the baseline and discharge scores divided by the SD of baseline scores. Usability was expressed as the number of missing data and the number of subjects with deteriorated psychiatric symptoms after administration of the scale.

The subjects’ objective psychiatric symptoms were evaluated with the Brief Psychiatric Rating Scale (BPRS),[18, 19] which was rated by attending psychiatrists who were not involved in this study. The correlation between the BsHAS and BPRS scores was tested with Spearman's rank correlation coefficients. As a guideline, correlation coefficients from ±0.00 to ±0.20 indicate no correlation; from ±0.20 to ±0.40, a fair degree of correlation; from ±0.40 to ±0.70, a moderate correlation; and above 0.70, an excellent correlation.[16]

All statistical tests were two-tailed, and the significance level was set at P < 0.05. Statistics were analyzed with pasw Statistics 18 (spss, IBM, Chicago, IL, USA).

Results

Reliability

In a preliminary study, the test–retest reliability of the scale was assessed. The intraclass correlation coefficients between the score of test–retest of the BsHAS physical health condition item score was 0.61 (P < 0.001), mood item score was 0.47 (P < 0.001), interpersonal fatigue item score was 0.45 (P = 0.001) and interest item score was 0.61 (P < 0.001). Therefore, the test–retest reliability of the BsHAS was moderate.

The Cronbach's α coefficient calculated by the BsHAS item score at baseline was 0.79. A Cronbach's α coefficient greater than 0.7 indicates sufficient reliability.[16] Therefore the internal reliability of the BsHAS was assumed to be sufficient.

Validity

The expert panel's opinion was that the BsHAS had good content validity for acute schizophrenia patients’ subjective health condition.

The distribution of the BsHAS scores in this study is shown in Figure 2. The BsHAS total score was significantly improved at discharge (Table 2). In the items, the physical health condition, mood and interest items were also significantly improved at discharge, but the interpersonal fatigue item was not significantly improved (Table 2). The effect sizes in terms of changes in the BsHAS are shown in Table 2. Cohen defined the effect size of ±0.20 as small, ±0.50 as moderate and ±0.80 as large.[20] According to Samsa et al., an effect size of at least ±0.20 is recommended as the standard for supporting sensitivity to change.[20] The effect sizes in the total BsHAS and the physical health condition and interest items were small, and the effect sizes in the mood and interpersonal fatigue were less than the sensitivity standard.

Figure 2.

Distributions of scores of the Brief Scale of Self-rated Health Condition with Acute Schizophrenia (BsHAS) items in individuals with acute symptoms of schizophrenia at baseline and discharge. Rating categories: Physical health condition and mood (0) Poor; (1) Fair; (2) Neither poor nor good; (3) Good; (4) Very good. Interpersonal fatigue (0) Extremely; (1) Quite a bit; (2) Moderately; (3) Slightly; (4) Not at all. Interest (0) Not at all; (1) A few times; (2) Fairly often; (3) Usually; (4) Always. 0 (image); 1 (image); 2 (image); 3 (image); 4 (image).

Table 2. Changes in the BsHAS and the BPRS scores
VariablesBaselineDischargeP-valueEffect size d
MeanSDMeanSD
  1. P-value: Wilcoxon signed rank test.
  2. d: Cohen's effect size.
  3. *P < 0.05. **P < 0.01.
  4. BPRS, Brief Psychiatric Rating Scale; BsHAS, Brief Scale of Self-Rated Health Condition with Acute Schizophrenia.
BsHAS (n = 199)      
Total (0–16)9.273.8810.163.54<0.001−0.24**
Physical health condition (0–4)2.411.182.641.040.012−0.21*
Mood (0–4)2.551.152.751.050.027−0.18*
Interpersonal fatigue (0–4)2.051.282.151.180.225−0.08
Interest (0–4)2.271.322.611.18<0.001−0.27**
BPRS (n = 199)      
Total (18–126)40.6212.4832.849.76<0.0010.70**

All subjects answered the BsHAS within 3 min without any difficulty, and no missing data were observed. None of the subjects showed deteriorated psychiatric symptoms after administration of the BsHAS.

Correlation between subjective health condition and psychiatric symptoms

Changes in the BPRS scores are shown in Table 2. The psychiatric symptoms assessed with the BPRS were significantly improved at discharge. The correlations (Spearman's rank correlation coefficient) between the BsHAS and the BPRS are shown in Table 3. No clear correlation was found between the BsHAS and the BPRS at baseline, discharge and change from baseline.

Table 3. Correlations (Spearman's rank correlation coefficient) between the BsHAS and the BPRS at baseline, discharge and change from baseline (n = 199)
 BPRS (n = 199)
BaselineDischargeChange from baseline
  1. *P < 0.05. **P < 0.01. ***P < 0.001.
  2. BPRS, Brief Psychiatric Rating Scale; BsHAS, Brief Scale of Self-Rated Health Condition with Acute Schizophrenia.
BsHAS (n = 199)   
Baseline   
Total−0.22**
Physical health condition−0.23**
Mood−0.19**
Interpersonal fatigue−0.23**
Interest−0.05
Discharge   
Total−0.29***
Physical health condition−0.21**
Mood−0.28***
Interpersonal fatigue−0.28***
Interest−0.11
Change from baseline   
Total−0.18*
Physical health condition−0.15*
Mood−0.18*
Interpersonal fatigue−0.08
Interest−0.07

Discussion

All questions were answered by all subjects so there were no missing data. There were no subjects whose symptoms deteriorated after administration of the scale. These findings suggested that the BsHAS did not impose a load on patients with acute schizophrenia and therefore it can be used safely in clinical practice.

The BsHAS total score was significantly improved at discharge. However, the effect size showed a response of −0.24, indicating that the responsiveness of the scale was small. Slight improvements were observed in the physical health condition, mood and interest items, but no improvement was observed in the interpersonal fatigue item. This is consistent with a previous report, which found that interpersonal fatigue of schizophrenia showed the slowest improvement.[21] Most patients might not recover from their own interpersonal fatigue at discharge.

No clear correlation was observed between the subjective health condition assessed with the BsHAS and the psychiatric symptoms assessed with the BPRS. Our results resemble those of previous studies[8, 22] in which there was only a weak correlation between the subjective health condition and the objective psychiatric symptoms evaluated by the raters for patients with chronic schizophrenia. Patient-reported improvement in functioning and well-being was reported to be different from investigator-rated improvement of psychopathology,[8] which suggests the necessity for subjective assessment as an evaluation method for outcomes. The results of the study suggest that psychiatric inpatient care should target these subjective health conditions as well as objective psychiatric symptoms.

This study has three main limitations. First, the examination based on the data of stable inpatients does not fully prove the test–retest reliability of the BsHAS. Second, the subjects were involuntarily hospitalized, and were invited to participate in rehabilitation programs by the attending doctor. The representativeness of the subjects in this study might restrict the possibility of generalization of the findings. Third, there is no absolute scale for subjective health condition evaluation for patients with acute schizophrenia, so the concurrent validity of the scale cannot be determined absolutely. To clarify the clinical significance of the BsHAS, we are planning a follow-up study at multiple medical facilities to examine the correlation between the patients’ subjective health condition and their positive participation in the treatment.

Conclusions

Our results show that the BsHAS can be used even for involuntarily admitted patients with acute schizophrenia.

In the treatment of patients with acute schizophrenia, improvement of the health condition assessed by the BsHAS was small and no improvement was observed in the interpersonal fatigue item. Most patients might not recover from their own interpersonal fatigue at discharge. The results of the study suggest that psychiatric inpatient care should target these subjective health conditions as well as objective psychiatric symptoms.

Acknowledgments

The authors thank all the subjects, psychiatrists, nurses, occupational therapists, psychiatric social workers of Kofu Hospital of Hyogo Prefecture, people concerned with the facilities, and staff members and students of Kobe University Graduate School of Health Sciences for their support and suggestions for this study. The authors do not have any conflicts of interest.

Appendix: Appendix

Items and grades of rating in the Brief Self-rated Scale of Health Condition with Acute Schizophrenia (BsHAS)

  1. ‘How would you describe your physical health?’

    0 Poor, 1 Fair, 2 Neither poor nor good, 3 Good, 4 Very good

  2. ‘How are you feeling?’

    0 Poor, 1 Fair, 2 Neither poor nor good, 3 Good, 4 Very good

  3. ‘Do you feel exhausted when other people are present?’

    0 Extremely, 1 Quite a bit, 2 Moderately, 3 Slightly, 4 Not at all

  4. ‘Have you enjoyed something, recently?’

    0 Not at all, 1 A few times, 2 Fairly often, 3 Usually, 4 Always

Ancillary