Secluded/restrained patients' perceptions of their treatment: Validity and reliability of a new questionnaire

Authors


Abstract

Aim:  To develop a standardized self-reporting questionnaire to evaluate patients' perceptions of their overall treatment in specific relation to the use of seclusion and/or restraint (SR) measures as part of the treatment program.

Methods:  A 17-item self-rating questionnaire was given to 56 patients with experience of SR-related treatment to develop a new scale, the Secluded/Restrained Patients' Perceptions of their Treatment (SR-PPT). Concurrent validity was examined against the Client Satisfaction Questionnaire-8 Japanese Version (CSQ-8J). In addition, Patient burden induced by answering the SR-PPT was evaluated.

Results:  On factor analysis, two factors named as Cooperation with Staff (nine items) and Perceptions of SR (two items) were derived. Cronbach's coefficient alphas were 0.928 and 0.887, and correlation coefficients against the CSQ-8J were 0.838 and 0.609, respectively. Answering the SR-PPT was found to induce little burden on the patients.

Conclusion:  Adequate internal consistency and concurrent validity of the final version of the SR-PPT, which consists of 11 items, indicate that it is acceptable as a measurement scale. Use of this questionnaire will add the patient's view to the assessment of overall treatment involving SR.

IN PSYCHIATRIC INPATIENT care, seclusion and/or restraint (SR) is often used to secure the safety of a patient whose disruptive behaviors due to mental disorder pose a potential danger to the patient him/herself and to others in the immediate vicinity, such as patients and care staff.1 The aims of SR are to ensure a secure environment and to provide medication and care smoothly until SR is no longer considered necessary. It is also reported, however, that patients who have experienced SR felt fear, helplessness and distress. This suggests that they do not consider such intervention beneficial, but rather a form of punishment under the control of care staff.2–5

Through various discussions aimed at SR minimization and elimination,6,7 it has been clarified that the amount of SR in Japan is high compared to other countries. The minimization of SR is an urgent task in Japan.8 Finland, another country that recognizes itself as a heavy user of SR among European countries, has conducted substantial investigations and has been taking measures for SR minimization.9,10 From this common awareness, Japan and Finland launched a bilateral project called SAKURA in 2007 to investigate the quality of care involving SR. The project follows the structure, process and outcome proposed by Donabedian11 and as one of the outcomes, focuses on the evaluation of the patient's own perceptions of his/her treatment.

Recent studies have found that patient perception of coercive interventions and/or a weak alliance with care staff lead to poorer adherence to treatment,12 and that an involuntary admission without understanding the justification for treatment results in a higher rate of readmission.13 It has been shown that in community mental health care, where patients generally receive treatment at will, closer agreement between the patient's needs and the physician's justification of treatment is associated with a higher level of patient satisfaction and consequently better adherence to the treatment.14 In addition, the patient's involvement in making treatment decisions improves his/her quality of life (QOL) and satisfaction level.15,16 Such findings can possibly be extrapolated to patients who have experienced SR, because their perceptions of such treatment and its justification as well as their perceptions of therapeutic collaboration with the staff might influence their prognosis. It is, therefore, necessary for staff providing SR treatment to make efforts to build a therapeutic relationship with the patients, identify their therapeutic needs, and involve them in establishing their own treatment goals. Such tasks are accomplished not only through close communication with SR patients but also by various types of quality care provided to them, such as offering medication, supporting nutrition and hydration, assisting in personal hygiene, and observing the somatic condition. Thus, any evaluation of how these tasks are accomplished must examine the patients' own rigorously measured perceptions of both the SR itself and the overall treatment related to SR.

Among the existing questionnaires examining how SR is perceived, some focus on negative emotions such as fear, hopelessness and punishment, or about positive experiences such as a calming effect or feeling of safety. Other questionnaires directly ask about the efficacy of SR.2–5,17 The surveys of involuntarily admitted patients' perceptions of their treatment include questions referring to the involuntary admission itself such as perceived coercion, being respected and feeling safe, and those asking about the relationship with care staff, perceived improvement and satisfaction.18–21 Most of those surveys explain the results by item individually, but do not provide a discussion using a composite score of each item, to grasp the overall aspects of patient perceptions.

In contrast, several questionnaires addressing patient satisfaction and collaboration between the patient and care staff were designed as a measurement using the total score, but did not include items specific to SR.22–25 Moreover, some of them involve many questions, which imposes an excessive burden on a patient just after an SR event.

Accordingly, a questionnaire that measures all of the aforementioned aspects of patient perceptions in only a few items, to reduce patient burden, does not exist.

The aim of this study was to develop a self-reporting questionnaire as a tool for measuring patient perception in order to evaluate the quality of overall treatment related to SR – a questionnaire applicable even to emotionally labile patients right after an SR event.

METHODS

Scale development

To determine the items that would constitute the new questionnaire (hereafter referred to as the ‘Secluded/Restrained Patients' Perceptions of their Treatment’, SR-PPT), the items used in previous surveys and existing questionnaires were examined. These included surveys on perception of SR2–5,17 and involuntarily admitted patients' perceptions of their treatment,18–21 questionnaires on patient satisfaction,22,23 and the Working Alliance Inventory (WAI).24,25 The items identified from the existing questionnaires for development of the SR-PPT were reviewed by a professional group consisting of two psychiatrists, three psychiatric nurses and one psychiatric occupational therapist. In total, 17 items were selected and categorized into the following five domains: ‘working alliance for treatment’ (seven items) and ‘respect and autonomy’ (four items), which are considered to be the domains most influenced by the coercive manner of SR; and second, ‘how patients felt about their SR’ (three items), and then ‘satisfaction’ (two items) and ‘perceived improvement’ (one item) as general impressions. With regard to the number of items, careful consideration was given to minimize the survey-related burden on patients who might be distressed during or immediately after SR.

The SR-PPT consists of several existing items in English and new items originally drafted by the main author (T.N.) in Japanese. Both English and Japanese versions of the SR-PPT were prepared. Permission was obtained from all authors of the existing questionnaires in order to use the exact wording of the items. The existing items in English were translated into Japanese by the same author (T.N.) and back-translated into English by two independent native speakers. The back-translation was checked against the original English sentences by another native English-speaking psychiatric care worker. The original items in Japanese were translated into English by two independent native English speakers and then back-translated into Japanese. The back-translation was then checked by the same author (T.N.).

A 100-mm visual analogue scale (VAS) was chosen as the measurement scale, allowing responses ranging from ‘strongly disagree’ to ‘strongly agree’ (scored correspondingly from 0 to 100 mm). Respondents were requested to answer based on their perceptions at the time of filling in the questionnaire and not to recall retrospectively the feelings experienced during SR.

The study was conducted between May and August 2008.

Setting

Two emergency wards and one acute ward in two psychiatric hospitals (N Hospital and K Hospital) in Japan participated in the study. ‘Emergency ward’ and ‘acute ward’ are ward categories stipulated by the national reimbursement system in Japan. The emergency and acute wards are those with ≥40% of patients newly admitted and with ≥40% of the newly admitted patients discharged to their home within 3 months. Emergency wards must also accept a required minimum number of compulsory involuntary admissions under orders from the hospital's catchment area. Accordingly, the average registered nurse allocation for an emergency ward is 10 patients per nurse per day (vs 13 patients per nurse per day for an acute ward).

The characteristics of the participating wards (emergency ward in N hospital, emergency ward in K hospital and acute ward in K hospital) are, respectively, as follows: number of beds, 60, 26 and 44; mean hospital stay days, 56.7, 25.0 and 37.7 days (in 2007); mean seclusion days per 1000 patient-days 176, 487 and 154 (in February 2008); and mean restraint days per 1000 patient-days 24, 32 and 5 (in February 2008). All three wards were mainly responsible for patients with schizophrenia or schizophrenia-related disorders (F 20-F29 category of the ICD-10).

Participants

The inclusion criteria were: age 18–65 years, an SR episode during current hospitalization, and written informed consent from the patient and his/her family (mandatory in Japan). Patients were excluded if they were receiving i.v. infusion due to a somatic disease, if their psychiatrist in charge did not agree to cooperate with the researchers, or if their clinical condition prevented their participation as judged by their psychiatrist.

Eligible candidates were selected by checking the patient records. At the same time, baseline variables (sex, age, diagnosis, number of admissions), duration of current hospitalization, interval from last SR treatment event until the date of survey and total duration of all SR treatment events were obtained for each of the eligible candidates.

Assessment

Prior to filling out the SR-PPT, the investigator showed the patient how to fill in the VAS and the patient practiced answering the questionnaire using an example. The patient then filled in each VAS of the 17 items of the SR-PPT.

Following the SR-PPT, the patient filled in another newly developed VAS form, enquiring how much difficulty, fatigue and strain they felt when answering the SR-PPT.

To evaluate the criterion-related validity of the SR-PPT, the Japanese version of the Client Satisfaction Questionnaire-8 (CSQ-8J) was filled out on the same occasion. The CSQ-8J is a measurement tool to rate the patients' satisfaction of a care service and contains eight items, all 4-point Likert scales. The overall score ranges from 8 to 32, and higher score indicates higher satisfaction.22 It has been widely used with patients as part of the outcome assessments for health and welfare services.

There exists evidence of a correlation between the subjective outcome evaluation (completed by the patient him/herself) and the objective outcome evaluation (symptom assessment by a rater).13,26 To assess such a kind of correlation between additional external criteria and the SR-PPT, the following assessments were performed by the psychiatrist in charge on the same day as the SR-PPT: the Brief Psychiatric Rating Scale (BPRS; 18 items, score range 1–7),27 the Global Assessment of Functioning (GAF)28 and GAF improvement (change from the admission date).

Ethics

The study was approved by the Ethics Review Board of the National Center of Neurology and Psychiatry.

In accordance with the national ethics requirement to first obtain proxy consent for research participation of an involuntarily admitted patient with limited comprehension, consent from the patients' relatives was obtained. Before completing the survey, all eligible patients for whom the informed consent by proxy was obtained were given a comprehensive description of the study and informed that their participation or refusal would not affect their care. Patients were informed that the ward staff would not see their SR-PPT responses, that the completed questionnaire would be sealed in an envelope directly in front of them and that the data would be treated anonymously. Thereafter their own written consent was obtained.

Taking into consideration the fact that some of the patients were currently under treatment programs that included SR, the main author (T.N., a psychiatrist) carefully observed the patient's level of fatigue or irritability and discontinued the procedure when necessary. In addition, after completing all of the questionnaires, the ward head nurse monitored the patients for any deleterious symptoms that might have been induced by the study procedure.

Statistical analysis

For the 86 participant candidates who met the inclusion criteria, the differences in patient characteristics between those who completed the SR-PPT and those who did not were analyzed using Student's t-test for continuous variables of normal distribution (Shapiro–Wilk test, P ≥ 0.1%) and the Mann–Whitney U-test for variables of non-normal distribution (Shapiro–Wilk test, P < 0.1%). The χ2 test was applied for categorical variables. The reliability was estimated by identifying factors using factor analysis (main factor method) and by examining the internal consistency of the subscales using Cronbach's alpha coefficient. The concurrent validity was estimated using Pearson's correlation coefficient between the SR-PPT score and the CSQ-8J score. To estimate the correlation of SR-PPT score with the external criteria, Pearson's correlation coefficient (for GAF and BPRS) and the partial correlation coefficient (for GAF improvement) were used. The relationship between patient characteristics and patient burden induced by answering the SR-PPT was tested using Pearson's correlation coefficient for continuous variables of normal distribution, and Spearman's rank correlation coefficient for variables of non-normal distribution. For categorical variables, one-way anova was applied. The significance level was set according to two-tailed test. All statistical analyses were performed using SPSS version 15.0 (SPSS, Chicago, IL, USA).

RESULTS

Of 182 patients hospitalized on the study wards on the date of the survey, 110 patients were aged 18–65 years and had experienced SR. Of these, nine patients had been discharged prior to the survey date, five patients were treated by physicians who refused to cooperate in the study and 10 patients were, according to their attending psychiatrists, unable to tolerate the study procedure. Of the remaining 86 patients, two patients did not volunteer their consent. The families of 27 more patients could not be contacted by the staff and proxy consent was thus not obtained. One patient was excluded by the main author (T.N.) due to the patient's excessive fatigue while answering the questionnaire. Finally, the SR-PPT was completed fully by a total of 56 patients.

Patient characteristics are listed in Table 1 including the mean GAF and BPRS scores. There were no significant differences in the patient characteristics between the 56 participants and the 30 excluded patients.

Table 1. Patient characteristics (n = 56) and GAF/BPRS scores
  n, mean ± SD, or median (IQR 25%–75%)%
  1. International Classification of Disease Tenth revision (ICD-10); 20 patients experienced restraint.

  2. BPRS, Brief Psychiatric Rating scale (18 items, score range 1–7); GAF, Global Assessment of Functioning.

Sex  
 Male3155
Age (years)42.4 ± 13.0 
Diagnosis  
 F20–F293969
 F30–F391120
 F10–F1947
 Others24
No. admissions1.5 (1.0–4.0) 
Days between last seclusion/restraint event and investigation10.0 (3.5–38.5) 
Days between admission and investigation36.0 (16.0–64.0) 
Days of seclusion12.0 (6.0–21.0) 
Days of restraint5.0 (2.0–8.0) 
GAF at admission27.9 ± 11.4 
GAF at investigation49.8 ± 16.3 
BPRS at investigation40.1 ± 15.3 

Factor analysis

Principal factor analysis on the 17 items selected as candidates was performed, because none of the 17 items exhibited ceiling or floor effects. The eigenvalue shifts were 9.80, 1.48, 1.1 and 0.85, assuming that the two-factor structure was valid. In addition, one item having low commonality of 0.224 following factor extraction was removed. At this point, a two-factor hypothesis emerged and factor analysis was performed using the principal factor method and varimax rotation. Next, the five items with a loading of ≥0.35 on both the primary and secondary factors were removed. The factor analysis was then repeated using the principal factor method and varimax rotation on the remaining 11 items. Table 2 lists the final factor pattern following varimax rotation. Incidentally, the ratio explaining the total variance of the 11 items for the two factors prior to rotation was 64.5%. In the nine primary factor items, those items that involved communication with staff toward mutual understanding of the treatment process and goals had a high loading and were therefore named ‘Cooperation with Staff’. In the two secondary factors, those items involving perceptions of SR had a high loading and were thus named ‘Perceptions of SR’.

Table 2. Rotated factor matrix for 11 items of the SR-PPT
 Factor loading
12
  1. SR-PPT, Secluded/Restrained Patients' Perception of their Treatment.

Factor 1: Cooperation with staff  
 Do you and the staff agree about the things you will need to do in treatment to help improve your situation?0.8380.204
 Are you and the staff working towards mutually agreed upon goals?0.8320.323
 Do you feel that the staff members understand your concerns?0.8250.251
 Have you been respected on the ward as a person?0.8100.333
 Is your opinion taken into account with regards to your treatment?0.7460.184
 Are you being given enough time during your treatment or care?0.7370.216
 Do you collaborate with the staff on setting goals for your treatment?0.6850.066
 Can you voice your opinion?0.6670.130
 Do you feel that staff members have ignored you in any way?0.5570.176
Factor 2: Perception of seclusion/restraint  
 Was being restrained and/or secluded beneficial in treating your difficulties?0.2020.868
 Was it necessary for you to be restrained and/or secluded?0.2280.860
  Factor contribution5.961.13
  Contribution variance rate54.2%10.3%

Internal consistency of the SR-PPT

The subscale coefficient alpha was also calculated in order to evaluate internal consistency. Adequate alpha coefficients were obtained for Cooperation with Staff (0.928) and Perceptions of SR (0.887). The value for the 11 items of the SR-PPT was 0.916.

SR-PPT scores

The mean ± SD total score for all the final 11 items (ranging from 0 to 1100) was 658.7 ± 245.4, and the mean subscale scores for Cooperation with Staff (max. 900) and for Perceptions of SR (max. 200) were 559.3 ± 208.9 and 99.4 ± 65.9, respectively. Correlations between each subscale score and the total score were observed as shown (Table 3). No significant differences nor correlations between ST-PPT total scores and the patient characteristics (sex, age, diagnosis, number of admissions, days between last SR event or admission and investigation, and days of SR) existed.

Table 3. SR-PPT subscale correlations with total score
 SR-PPT scaleSR-PPT Cooperation with Staff subscaleSR-PPT Perception of SR subscale
  1. *P < 0.01, **P < 0.001.

  2. CSQ-8J, Client Satisfaction Questionnaire-8 Japanese version; SR-PPT, Secluded/Restrained Patients' Perception of their Treatment.

SR-PPT Cooperation with Staff subscale0.971**  
SR-PPT Perception of SR subscale0.648**0.445* 
CSQ-8J0.876**0.838**0.609**

Criterion-related validity

The mean ± SD CSQ-8J score was 21.7 ± 5.6. Significant correlations were observed between CSQ-8J score, SR-PPT scale score, SR-PPT Cooperation with Staff subscale score and SR-PPT Perceptions of SR subscale score (Table 3).

A significant negative correlation was found between SR-PPT total score and BPRS total score (r = −0.417, P < 0.01), and a significant positive correlation was seen between SR-PPT total score and both the GAF (r = 0.472, P < 0.001) and the GAF improvement (r = 0.406, P < 0.01) scores.

Burden of answering the SR-PPT

The mean ± SD scores for difficulty, fatigue and strain experienced by the patients when answering the SR-PPT were 23.5 ± 26.7, 24.8 ± 29.2 and 30.2 ± 30.0, respectively (max. 100). The rate of the lowest burden scores for patients (<20) with regard to difficulty, fatigue and strain was 41.9%, 40.7% and 34.9% and that of the highest burden scores for patients (>80) was 3.5%, 5.8% and 5.8%, respectively. No correlation was observed between length of the interval from the last SR event to day of the survey and the burden of answering the SR-PPT. The BPRS and (inversely) the GAF correlated with fatigue (r = 0.377, P < 0.01 and r = −0.296, P < 0.05) and strain (r = 0.519, P < 0.001 and r = −0.272, P < 0.05), respectively. No cases of worsening of symptoms due to participation in the survey were observed.

DISCUSSION

To our knowledge, the SR-PPT is the first measurement developed for assessments by patients of their overall treatment in specific relation to the use of SR measures as part of the treatment program. It assesses not only the patients' perceptions of experienced SR itself but aspects such as respect, autonomy, and working alliance, which are often hindered by coercive interventions. Of 17 candidate questions, 11 were found to be relevant and sufficient. These questions constituted two factors, namely, Cooperation with Staff (nine items) and Perceptions of SR (two items). Both had sufficient internal consistency and concurrent validity. Furthermore, the SR-PPT total score had a significant inverse correlation with BPRS score, and direct correlations with GAF and GAF improvement on the day of the survey used as external criteria. The rater's assessment using GAF (assess impairment in social functioning) and/or BPRS (assess anxiety, hostility, suspiciousness) reflected on some level the patient's negative perception of cooperation with staff. These results suggest the validity of the SR-PPT.

In cases when SR is applied to secure patients against imminent danger caused by their disruptive behavior due to mental disorder, the patient's own view of such intervention is often left behind, yet the objective and subjective views may also diverge.4 Indeed, although the correlations between the SR-PPT and, in contrast, the observer-rated assessment scales (GAF and BPRS) in the present study were statistically significant, the correlation coefficient of <0.7 was weak. This indicates that it is not sufficient to rely solely on the objective instruments, and that the staff assessment alone seems most likely to fail to identify adequately the dimension of patient perceptions. Because the patient's own perceptions of treatment considerably affect his/her prognosis, as mentioned in previous studies,12–16 it is crucial to make these perceptions overt and measurable. It is especially true for such elements of treatment as respect for patient dignity and empowerment in shared decision making – even if the overall treatment includes coercive measures. Against such need for a standardized self-rating subjective measure that is easy to complete immediately after or even during SR, the SR-PPT appears to be a feasible, as well as a valid and reliable tool.

The items derived from the five domains using factor analysis were assumed to have a two-factor structure. One of the domains, Perception of SR, loaded to the secondary factor, and the rest of the domains to the primary factor. Hansson et al. and McCabe et al. reported that subjectively important aspects of outcome interact via a common powerful mediator, namely, positive and negative feeling, and could be explained by a single factor.29,30 This is in accordance with the present factor analysis results. The high correlation of the primary factor with the CSQ-8J (r = 0.838, P < 0.001) means that both questionnaires have a similar powerful mediator. In contrast to the CSQ-8J, however, the SR-PPT consists of items specific to treatment involving SR. Moreover, the SR-PPT also allows examination by individual item.

It is preferable that the SR-PPT be used immediately after an SR treatment event, because if other treatment programs following SR are underway, they can affect the patient's response and thus influence the results. Because only 14 participants in this survey had experienced an SR event within the previous 3 days, whether earlier use of the SR-PPT is possible or not warrants future investigation. In addition, the median number of SR treatment days for the 56 participants in this study was longer than reported in the USA and Europe,10,31,32 and this raises the question of whether the SR-PPT is feasible for use in countries in which SR treatment events are routinely shorter. It is reported that in the USA, for instance, the mean duration of SR treatment events is a few hours.32 The recommendation of the Core Strategies7, however, suggests a debriefing be held between staff and the patient after an SR treatment event. Assuming that such debriefings are routinely performed even after short SR events, it seems feasible for patients to fill in the SR-PPT at that time.

Family informed consent, which is mandatory in Japan for research involving SR patients, was not available for 27 patients and they were therefore excluded. This exclusion criterion did not, however, bias the results, because it cannot be attributed to clinical or demographic patient characteristics.

The size and field of the present patient group were limited to ensure sufficient stability of loadings. Therefore further investigation with a larger sample is required including not only patients in the acute psychiatric setting but also those who are difficult to manage in chronic wards, and, furthermore, patients in other countries. A bilateral study using the SR-PPT, which is currently underway in both Finland and Japan, has a larger sample size and will enable a comparison of cross-national data.

ACKNOWLEDGMENTS

This study was funded by the Pfizer Health Research Foundation and was conducted as part of a research and development project on seclusion and restraint in psychiatric hospitals in Finland and Japan (the SAKURA project). The researchers would also like to thank the hospital staff who assisted with the survey. T.N. has received grants from the Pfizer Health Research Foundation. All other authors declare that they have no conflict of interest.

Ancillary