The quality in psychiatric care—Addiction outpatient instrument: Psychometric properties and patient views of the quality of care

Abstract Aim The aim of the study was to evaluate the psychometric properties and factor structure of the Quality in Psychiatric Care—Addiction Outpatient (QPC‐AOP) instrument and to describe the experiences with the quality of care among addiction outpatients. Design The study has a cross‐sectional design. Methods A sample of 244 patients with addiction and psychiatric disorders completed the QPC‐AOP. Results Confirmatory factor analysis showed adequate to excellent goodness‐of‐fit indices supporting the 9‐factor structure of the QPC‐AOP. The results thus demonstrate that the concept of quality of care to a large extent is equivalent among outpatients from general psychiatry and from outpatient addiction services. Internal consistency for the full QPC‐AOP was adequate, but poor for some of the separate factors. The patients’ ratings of quality of care were generally high; the highest rating was for Encounter and the lowest for Discharge.


| The outpatient addiction service context
People with substance use disorders are at greater risk for morbidity and mortality (Charlson et al., 2015) and have extensive needs of case management services (Penzenstadler et al., 2017) resulting in high healthcare costs (Darke et al., 2006). In order to meet these needs, and depending on the level of substance abuse, most addiction treatment is organized in inpatient programmes that are intensive, residential treatment programmes designed to treat serious addictions and outpatient programmes that are run part-time, allowing for less disruption to everyday life for the recovering user.
Although there is a discussion on the merits of these programmes (Rossegger et al., 2009;Spinelli & Thyer, 2017), outpatient addiction services can be vital to a person's ability to become drug free as he or she continues working and living his or her life while receiving treatment. However, few studies have evaluated outpatient addiction services from the patients' perspective.

| Measuring quality of care
The quality of the care given is an important factor affecting patient outcome of treatment (Buchanan et al., 2015). In order to ascertain quality, the perceived quality of care needs to be assessed in a reliable and valid way. Assessment of quality of care is usually made from the professionals' perspective (Baines et al., 2018). However, earlier studies have shown that patients ' and professionals' perceptions of what defines good quality of care do not always agree (Barbato et al., 2014). Therefore, it has been recognized as necessary to investigate patients' perceptions of quality of care (Beattie et al., 2015; | 1921 SCHRÖDER Et al. Taube & Berzina-Novikova, 2018) as well as including the patient's perspective in measurement development because it contributes to ensuring that the questions asked are relevant and important to the patients (Connell et al., 2018).
However, there are few instruments that have been developed to measure the quality of addiction care (Marsden et al., 2000;Miglietta et al., 2018;Pincus et al., 2016) from the patient's perspective in particular. General instruments have been used instead, which may not be sensitive to the key issues in addiction problems (Marsden et al., 2000). Important key issues identified are, for example, having enough time in care to deal with problems (Bacchus et al., 1999), the interpersonal patient-staff relationship, and planned for and offered aftercare (Lovejoy et al., 1995). A thorough search in databases resulted in only one addiction-specific instrument where patients have been included in the development of the instrument, namely "The treatment perceptions questionnaire" (TPQ). The TPQ has been developed from a review of existing instruments, research literature and interviews with eight patients and has been designed to measure the treatment satisfaction in addiction treatment programmes (Marsden et al., 2000). There is to the authors' knowledge no psychometrically tested instrument to measure quality of care in the addiction outpatient care from the patients' perspective. Rat et al., (2007) maintained that items based on interviews with patients are more valid than items based on, for example, literature reviews, patient focus groups or professionals. Connell et al., (2018) and Wiering et al., (2017) supported that patients should be involved in the design and wording of the instrument from the beginning as they have unique experience and important information. In addition, instruments measuring quality of care should have a clear definition of the concept (Sitzia & Wood, 1997).

| Assessing the quality of addiction centre services
In an endeavour to develop such instruments, the instrument Quality in Psychiatric Care (QPC) was developed. The QPC is a selfadministered instrument measuring different aspects of quality of care. It is based on a definition of quality of care (Schröder et al., 2007(Schröder et al., , 2010) that has been developed from a phenomenographic interview study with psychiatric in-and outpatients (Schröder et al., 2006).
The QPC differs substantially from the TPQ instrument in that it has been developed from the patient's perspective and measures different aspects of quality of care.
The aim of the present study was to evaluate the psychometric properties and factor structure of the Quality in Psychiatric Care-Addiction Outpatient (QPC-AOP) instrument and to describe the experiences with the quality of care among addiction outpatients.
The following research questions were addressed:

| Design
The study has a cross-sectional design.

| Participants and procedure
Data were collected during 8 weeks in April-May 2014. According to common rules of thumb (Hair et al., 1995), at least 5 respondents per item of the questionnaire are needed, resulting in a recommended sample size of 265. There were 610 patients at the clinics during the data collection period. Questionnaires were returned by 317 patients, but 73 were excluded because they had 30% missing items.
The final sample included 244 (144 male and 100 female) addiction patients with aged 18 to 77 (M = 39.0, SD = 14.1) at centres for outpatient addiction services in two Swedish municipalities, resulting in a response rate of 52%. The centres were staffed by multiprofessional teams and served both urban and rural populations. Patients with different substance abuse problems, such as alcohol, narcotics, anabolic androgenic steroids and pharmaceutical substance abuse problems in addition to one or more psychiatric diagnosis, could be referred to the centres from other caregivers or by self-referral. All patients who were able to understand and express themselves in Swedish were invited to participate by a contact person at the centres, who also ensured that the participants were cognitively able to answer the questionnaire. Patients were informed about the study both orally and in writing, including information that participation was voluntary and could be terminated at any time. Those who agreed to participate gave their oral consent and were requested to complete the QPC-AOP questionnaire anonymously and put it in a locked box at the centres. The study was approved by the regional research ethics committee at Uppsala, Sweden (reference number: dnr 2014/040).

| Development of the QPC-AOP instrument
The QPC-AOP was based on the Quality in Psychiatric Care-Outpatient (QPC-OP) instrument (Schröder et al., 2011). The

QPC-OP is based on the Quality in Psychiatric Care-Inpatient
(QPC-IP) instrument developed from the patients' perspective (Schröder et al., 2007(Schröder et al., , 2010 with certain modifications. The confirmatory factor analysis showed that the factor structure of the QPC-IP was to a large extent replicated in the QPC-OP and showed adequate psychometric properties (Schröder et al., 2011).
The QPC-OP consists of 30 items in eight dimensions, which were used verbatim in the QPC-AOP. Then, 24 items were added to form a tentative addiction-specific dimension (Table 1). These items were generated from a discussion group with six people who had personal experiences of being a patient in addiction outpatient care. The individuals were recruited from a patients' addiction association. The discussion group included those with alcohol, narcotics, anabolic androgenic steroids and pharmaceutical substance abuse problems, to form a heterogeneous group that covers patients with various substance abuse problems represented in the addiction outpatient care. They were asked to participate in the discussion group by a contact person. The discussion was led by two of the authors (AS and KS). Notes were taken and the discussion lasted about 2 hr.
Thereafter, the preliminary QPC-AOP (54 items in 9 dimensions) was tested for face validity according to Bowling (2005). Six other people with experience of addiction outpatient care recruited from the same patient association were asked to fill in the questionnaire at home and then evaluate it in writing. After one week, they attended a researcher-led discussion (face-to-face) about the questionnaire. The evaluation was carried out using a checklist where each of the 54 items was assessed as either being "Clear and easy to understand," "Acceptable" or "Unclear and hard to understand." Each item's importance for representing quality of care in the intended dimension was rated on a 5-point scale from "very important" to "of little importance." The questionnaire ratings were discussed by the group. Two of the authors (AS and KS) led the discussions, and notes were taken. The main questions of the discussions were "What's your general impression of the questionnaire?" and "How relevant and useful do you think the specific items are in relation the quality of outpatient addiction care?" The group discussion, which lasted 2 hr, resulted in a rewording of some items and the deletion of one of the 24 addiction-specific items as it was deemed less relevant to addiction care. All six persons in the second discussion thought the QPC-AOP was easy to complete within 10-15 min.
The preliminary version of the QPC-AOP thus included 53 (30 + 23) items concerning quality of addiction outpatient services (Table 1). Each item was related to the statement "I experienced that..." and scored on a 4-point Likert-type scale with a rating from 1 (totally disagree)-4 (totally agree). For each item, a "not applicable" alternative was provided. The QPC-AOP also included background questions about demography and general clinical characteristics.

| Data analysis
Descriptive statistical analyses were conducted using SPSS 22 (IBM Corp.). Prior to analysis, imputation was performed by replacing missing data points with the mean of the item in question.
The scales were assessed with Cronbach's alpha (Cronbach, 1951), using the 0.70 criterion for adequate homogeneity (Nunnally & Bernstein, 1994). Confirmatory factor analysis (CFA) was used to test the tenability of the a priori proposed factor structure model based on the QPC-OP. The CFA was performed using LISREL 8.8 (Jöreskog & Sörbom, 1996) with generally weighted least squares estimation on the asymptotic covariance matrices. Polychoric and polyserial correlation matrices were obtained by means of the PRELIS programme (Jöreskog & Sörbom, 1988). The Satorra-Bentler scaled chi-square (S-B χ 2 ; Satorra et al., 1988) was used to evaluate the adequacy of the model. The S-B chi-square permits correct goodnessof-fit indices and standard errors for data that are non-normally distributed. The S-B chi-square is, however, sensitive to sample size and even small misspecifications in the model can lead to a significant chi-square. Consequently, we also evaluated model fit with the comparative fit index (CFI), the standardized root mean square residual (SRMR) and the root mean square of approximation (RMSEA) and its 90% confidence interval (CI). CFI values ≥ 0.95, SRMR values ≤ 0.08 and RMSEA values ≤ 0.06 are viewed as evidence for a wellfitting model (Hu & Bentler, 1999).

| Psychometric evaluation of the QPC-AOP
The confirmatory factor analysis was first performed on the model  and a SRMR = 0.064, indicating an excellent goodness of fit.
Next, since there was no a priori tentative factor structure of the 23 addiction-specific items, all of them were entered into the model as a specific factor (i.e. a 9th factor). This model ( As shown in

| Descriptions of quality of addictive outpatient care
Mean and standard deviations of the nine QPC-AOP dimensions are given in Table 2. Ninety-two per cent of the participants rated the quality of care as positive (a mean score of 2.5 or higher, which is the centre of the scale). As seen in Table 2, and supported by Bonferroni-corrected t tests, the perceived quality of Encounter was significantly greater than the perceived quality of the second ranked dimension, Environment (t (243) = 6.69, p < .001). The perceived quality of Environment was significantly higher than the perceived quality of Support (t (243) = 3.826, p < .001). There were no significant differences between the perceived quality of Support, Addictionspecific, Participation-Empowerment, Next of kin, Accessibility and Participation-Information. However, Participation-Information was significantly higher than the lowest ranked dimension, Discharge (t (243) = 9.64 p < .001).

| D ISCUSS I ON
The QPC-AOP was designed to capture the quality of addiction outpatient services on the basis of the patients' own perspective. The results revealed that the factor structure of the QPC-AOP well fitted the factor structure of the QPC-OP (Schröder et al., 2011) that it was based upon. Furthermore, the four Addiction-specific items that were retained fitted the Addiction-specific factor adequately, thus capturing aspects of addiction services not covered by the other factors in the QPC-AOP.
Since the QPC-AOP fitted the a priori factor structure of the QPC-OP as well as other QPC instruments developed for other psychiatric contexts (Rask et al., 2017;Schröder et al., 2010Schröder et al., , 2013, the results indicate that the concept of quality of care among patients has a similar structure regardless of the psychiatric service context. The internal consistency of the total QPC-AOP and the dimensions were adequate except in the cases of Discharge, Environment and Next of kin. The low alphas of these three dimensions can probably, in part, be attributed to the low number of items (3, 3 and 2 items, respectively). Further studies are needed to investigate whether this is a replicable phenomenon. These dimensions are thus in need of modification, replacement or addition of further items.
Taking this into consideration, the 34 item QPC-AOP was deemed feasible.
A large number of participants perceived the quality of care as high (i.e. a mean score greater than 2.5, which is the centre of the scale). Earlier studies using other measures of quality of care have shown similarly high ratings of quality of care (Brunt et al., 2019;Jiang et al., 2019;Lally et al., 2013). This is a welldescribed issue. In a review study about patients' satisfaction with mental health treatment, Lebow (1983) found as early as 1983 that less than ten per cent of patients were dissatisfied. We can only speculate about the reasons here, but one explanation may be that patients in psychiatric care have difficulty in criticizing their care (Kondasani and Panda, 2015) in fear of impact on the mental health care they receive (Fernandes et al., 2019). Fernandes et al., (2019) maintain that low participation of patients in the development of items can influence the high ratings. This is, however, not the case in this study as the QPC instrument used was developed from the patients' perspective and validated among addiction patients as recommended by Kilbourne et al. (2010). In addition, Zieve et al., (2019) and Zimmerman et al., (2017) state that dissatisfaction or low ratings are often associated with dropping out of treatment.
The highest ratings of quality were found in the Encounter dimension, which contains items regarding interpersonal relationships that can be seen as an important aspect in psychiatric care and a core of psychiatric practice (Chambers, 1998) as well in addiction care (Hanpatchalyakul et al., 2012) and may influence the patient's willingness to come back to the care services (Larrabee et al., 2004). Previous studies on quality of psychiatric care show that both out-and inpatients put the greatest emphasis on the staff's empathy, on their being interested in, understanding, listening to and respecting patients (Lin et al., 2021;Moreno-Poyato et al., 2016;Taube and Berzina-Novikova, 2018) as well as and in other healthcare context (Fang et al., 2019). One qualitative study on how the patients perceived the concept of quality of care confirms the importance of encountering committed and competent staff who understood and confirmed the patients (Schröder et al., 2006). Hanpatchalyakul and colleagues (2012)

| Limitations
The goodness of fit of the QPC-AOP was excellent; however, there were below adequate levels of internal consistency of some individual dimensions. Given that these dimensions consisted of few items (two to three), lower internal consistency is to be expected.
The sample size was at the lower end of the sample size criterion and since this was the first time the addiction-specific items were evaluated, a new and larger sample is needed to confirm the adequacy of these items. It was impossible to analyse the dropouts because of incomplete "missing" patient registrations at the participating clinics.
Although the main objective of the study was psychometric evaluation, it should be noted that the cross-sectional design precludes conclusions about cause and effect.

| Implications for practice
The findings in this study imply several clinical recommendations that invites for reflection in the improvement of the addiction outpatient services and nursing education. Firstly, the knowledge about patients' ratings of low quality of care regarding the dimension Discharge can be used in the improvement of the quality of care and nursing education for the purpose to improve the information to the patients so that they know where to ask for help if they need care after the contact was completed. Secondly, QPC-AOP can be routinely used as a self-rating instrument for the purpose of improving psychiatric addiction outpatient care and can constitute a part in a quality system. Thirdly, QPC-AOP can be used as a tool to motivate care staff, especially nurses, to continually and systematically improve the care as the QPC-AOP is a simple, inexpensive and quick way to evaluate quality of care.

| CON CLUS IONS
The QPC-AOP is a psychometrically adequate instrument for evaluating patients' experiences of the quality of addiction services. Based on a patient perspective, the QPC-AOP will contribute to the improvement of addiction services and the development of theory in this area. Clinical recommendations for nursing are to use validated instruments, such QPC-AOP, for follow up the results and to increase the professional knowledge about patients' views of quality of care.

ACK N OWLED G M ENTS
We would like to thank the patients for participating in this study.

CO N FLI C T O F I NTE R E S T
The authors declare no conflicts of interest with respect to the research, authorship and/or publication of this article. This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

AUTH O R S' CO NTR I B UTI O N
All authors have contributed significantly and are in agreement with the content of the manuscript.

DATA AVA I L A B I L I T Y S TAT E M E N T
Data available on request from the authors. The data that support the findings of this study are available from the corresponding author upon reasonable request.