Reliability and validity of clinically useful depression outcome scale identifying mixed features in patients with manic episode

Abstract Objectives This study aims to explore the reliability, validity, and feasibility of Clinically Useful Depression Outcome Scale (CUDOS) in screening mixed features in patients diagnosed with mania. Methods A total of 109 patients with (hypo‐) manic episode were recruited. The reliability of Chinese version of CUDOS (CUDOS‐C) were analyzed with Cronbach's alpha and intraclass correlation coefficient (ICC). Spearman correlation coefficient was used to analyze the validity by comparing the correlation between CUDOS‐C and Patient Health Questionnaire‐9 (PHQ‐9), 32‐item Hypomania Checklist (HCL‐32). The score of MINI (hypo‐) manic episode with mixed features—DSM‐5 Module—Chinese version(MINI‐M‐C) ≥ 2 was considered as the gold standard of mixed features, and the receiver operating characteristic (ROC) curve analysis was used to calculate the optimal cut‐off values of CUDOS‐C score. Results The Cronbach's alpha value of CUDOS‐C was 0.898, and the ICC of CUDOS‐C test‐retest was 0.880 (95% CI: 0.812‐0.923, p < .05).The CUDOS‐C score was significantly correlated with PHQ‐9 score (r = 0.893, p = .000), but not with HCL‐32 score(r = 0.088, p = .364).The area under ROC curve was 0.909 (95% CI: 0.855 to 0.963, p < .001) for CUDOS‐C identifying mixed features in mania. The optimal cut‐off value was 11 with a sensitivity of 0.854 and a specificity of 0.868. The CUDOS‐C (score ≥ 12) identified 40.4% of the patients with mixed features, which was higher than those diagnosed by clinicians (18.3%) and screened using MINI‐M‐C (37.6%). Conclusions The results indicate the CUDOS‐C is a reliable and valid self‐administered questionnaire for assessing depressive symptoms and screening patients with mixed mania.


INTRODUCTION
Mixed state or mixed episode is a kind of mood disorder with a co-occurrence or overlapping state of (hypo-) manic and depressive episodes. At the end of the 19th century, Kraepeline first put forward the concept and definition of mixed state. Subsequently, mixed states have been almost completely neglected for decades under the influence of the idea that the combination of manic and depressive features should not be considered a mood disorder (Verdolini et al., 2015).The third and fourth editions of the United States diagnostic and statistical manual of mental disorders (DSM-III and DSM-IV) and the tenth edition of the international classification of diseases (ICD-10) classified mixed states as a subtype of bipolar I disorder. However, the criteria of mixed states in DSM or ICD is too strict to meet in clinical practice, which leads to misdiagnosis and inappropriate treatments, and con- The Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD) investigated the manic symptoms in 1380 patients with bipolar depressed episodes, and found that more than two-thirds of the subjects had concomitant manic symptoms. Still, only 14.8% of the patients met the DSM-IV criteria for mixed episodes (Goldberg et al., 2009). Based on the updated clinical practice and research data of diagnosis and treatment for bipolar disorders, the fifth edition of DSM (DSM-5) has changed the terms "mixed states and mixed episodes" into the "with mixed features" specifier (MFS) of (hypo-) manic and major depressive episodes, which could capture subthreshold and nonoverlapping symptoms of the opposite poles and form a continuous spectrum diagnosis from a manic episode to a depressive episode (Fagiolini et al., 2015;Perugi et al., 2014;Vieta & Valentí, 2013).
A recent meta-analysis showed that 35% participants had MFS in those patients diagnosed with bipolar (hypo-) mania or bipolar depression according to the DSM-5 diagnostic criteria, and the proportion of patients with MFS in unipolar depression was as high as 24% (Vázquez et al., 2018). However, there are still controversies about the validity and utility of MFS, and the worries about overdiagnosis and overtreatment of bipolar disorders are brought out (Koukopoulos et al., 2013;Verdolini et al., 2014;Vieta & Valentí, 2013). So far, most of the published data were retrospective, or used alternative definitions rather than DSM-5diagnostic criteria for MFS. More systematic and prospective studies are in pressing needs to fully assess the effects and implications associated with the use of MFS in clinical practice (McIntyre et al., 2013;Perlis et al., 2014;Verdolini et al., 2015).
Due to the diverse and complex clinical manifestations and lack of screening indicators, mixed features are usually insufficiently identified and diagnosed. We recently conducted a multicenter survey and found that the proportion of MFS among (hypo-) manic patients diagnosed by the clinicians using the DSM-5 criteria was only 18% in China, which was far lower than the reported data (Fei et al., 2020). It is necessary to adopt screening tools to help clinicians make an acute diagnosis for MFS (Fei et al., 2020). More than two scales are often used to evaluate the different symptoms of depression, mania and other dimensions in patients with mood disorders (especially bipolar disorders), which would cause prolonged scale assessments and noncompliance in patients.
The clinically useful depression outcome scale supplemented with questions for the DSM-5 MFS (CUDOS-M) includes three dimensions: depressive symptoms, manic symptoms, and functional impairment (Zimmerman et al., 2014). Recently, one study has explored the reliability and validity of the Chinese version of CUDOS-M (CUDOS-M-C) for the Chinese patients with depressive episodes, and the findings supported that CUDOS-M-C could effectively screen those patients with mixed depression (Du et al., 2021). Our study aims to explore the reliability, validity, and feasibility of the depressive dimensionality of CUDOS-M, that is, the Clinically Useful Depression Outcome Scale (CUDOS) (Zimmerman et al., 2008), to identify the mixed features in manic episodes and then find that CUDOS-M as a simple screening tool can simultaneously identify both mixed depression and mixed mania.

Participants
Convenience sampling method was adopted to recruit outpatients or were excluded from the survey. Besides, patients with manic episodes too severe to cooperate with the assessment of scales, and other conditions which were not suitable for participation in the study were also excluded.
This study was reviewed and approved by the Ethics Committee of Hongkou District Mental Health Center of Shanghai (approval number: 2018-B04). Before the implementation of any evaluation, all participants provided written informed consent.

Instruments
The original version of CUDOS was developed by Zimmerman et al.
and had good consistency with other self-rating scales of depressive symptoms (Jeon et al., 2017;Trujols et al., 2013;Zimmerman et al., 2008Zimmerman et al., , 2012. Afterwards, the items of manic symptoms were added to form a self-rating questionnaire composed of 31 items for screening (not present) to 4 (severe).

Clinical interview and measurement
The

Statistical analysis
All data were analyzed with SPSS version 25.0 statistical analysis software. The categorical data were presented as the number and frequency of observations, and the continuous data were presented as means ± standard deviation (SD) or median (25%, 75% quantile) if without normal distribution. The internal consistency of CUDOS-C was evaluated using Cronbach's α coefficient and item-total correlation. TA B L E 1 Internal consistency, corrected item-total correlation, and test-retest reliability of CUDOS-C items

Corrected item-total correlation
Test-retest reliability (n = 80) The intraclass correlation coefficient (ICC) between the scores at the baseline and at the first weekend was calculated to examine the testretest reliability of CUDOS-C. Spearman correlation coefficient was used to analyze the convergent and discriminant validity by comparing the correlation between CUDOS-C and PHQ-9, HCL-32. Setting the score of MINI-M-C ≥ 2as the gold standard, the diagnostic validity of CUDOS-C for screening mixed features was analyzed using the receiver operating characteristic (ROC) curve, and the sensitivity and specificity were evaluated to obtain the optimal cutoff score. All statistical tests were two-tailed, and a p value < .05 was considered to be statistically significant.

Internal consistency and test-retest reliability of CUDOS-C
The Cronbach's alpha value of internal consistency for the CUDOS-C was 0.898, and the Cronbach's alpha coefficients after deleting each item ranged from 0.877 to 0.906. The corrected item-total correlations were between0.108 and 0.752 at baseline. The lowest item-total correlations were for two atypical depressive symptoms [item 4 "increased appetite" (correlation coefficient = 0.108) and item 6 "hypersomnia" (correlation coefficient = 0.425)]. The ICC of CUDOS-C test-retest after 1 week in 80 patients was 0.880 (95% CI: 0.812-0.923,p < .05), and the ICC of each item were between 0.393 and 0.883 (p < .05). The results of reliability analysis are shown in Table 1.

Validity analysis of CUDOS-C
The scores for CUDOS-C, PHQ-9, and HCL-32 scales at baseline were 9.00(3.00, 19.50), 3.00(0, 10.00), and 14.00(8.00, 18.00), respectively. The spearman correlation analysis showed that CUDOS-C score   (Jeon et al., 2017;Trujols et al., 2013;Zimmerman et al., 2008). Though the minor depression symptoms at baseline and the therapeutic effect over 1 week could affect the correlation, the high ICC values (0.39-0.88) of total scale and each item demonstrated an acceptable test-retest reliability of the CUDOS-C. The internal consistency and test-retest reliability of the CUDOS-C was also comparable with that of the CUDOS-C-M (Du et al., 2021), and superior to the MINI-M-C in Chinese patients with mood disorders (Fei et al., 2020).
The Hamilton Rating Scale for Depression (HAMD) and Montgomery and Asberg Depression Rating Scale (MADRS) are the most frequently used scales evaluating depressive symptoms. However, they are clinician administered, requiring training and more time to administer reliably and validly, and only measure decrease in sleep and appetite (Furukawa, 2010;Huijbrechts et al., 1999). The CUDOS consists of 16 items for assessing the depressive symptoms, and measures increase and decrease in sleep or appetite using separate items, which is similar to Quick Inventory of Depressive Symptomatology (Rush et al., 2003;Zimmerman et al., 2012), and different with PHQ-9 using a single item (Levisz et al., 2019). The atypical depression symptoms (increase in sleep and appetite) had the lowest item-scale correlations consistent with the results of Korean validation study, that supported increase and decrease in sleep or appetite needs to be evaluated separately in clinical practice (Jeon et al., 2017).
The ROC curve demonstrated that CUDOS-C acted as a good screening tool for mixed mania at the optimal value as 11. The cutoff score was lower than those scores (optimal cut-off values as 19) for identifying depression in patients with type 2 diabetes mellitus and identifying remission in patients with depression (Zimmerman et al., 2012(Zimmerman et al., , 2004. The area under ROC curve (0.91), sensitivity (0.85), and specificity (0.87) of CUDOS-C in these participants were consistent with those of the original CUDOS in patients with major depressive