Establishing the sensitivity and specificity of the gynaecological cancer distress screen

Nuanced distress screening tools can help cancer care services manage specific cancer groups' concerns more efficiently. This study examines the sensitivity and specificity of a tool specifically for women with gynaecological cancers (called the Gynaecological Cancer Distress Screen or DT‐Gyn).


| INTRODUCTION
Psychological distress is a common experience for women with gynaecological cancer, with prevalence estimates ranging from 30% to 85%. 1,2For these women, distress is associated with poor treatment compliance, 3 increased health service utilisation but reduced adherence to surveillance screening, 4 impaired quality of life, 5 and mortality. 6Prompt identification and management of distress can reduce emotional suffering for patients and their families or caregivers, 7 reduce the potentially adverse effects of illness adjustment, and decrease healthcare expenditure and service utilisation. 8reening for distress can facilitate appropriate and timely supportive cancer care 9 and improve patient outcomes. 10Of course, there are trade-offs when choosing a screening tool, with a one-sizefits-all approach having limited utility given the number of cancerspecific sources of distress. 7Thus, understanding the contextual factors that increase susceptibility to distress will enable cancer service providers to manage women's concerns more efficiently.This is particularly important given that 24%-41% of cancer patients experience psychological symptoms, but most do not disclose them to their cancer care team, 11 and for women with gynaecological cancers specifically, holding back concerns is even more common. 12ssibly the limited disclosure by women with gynaecological cancer is because of the sensitive nature of many of their concerns (e.g., changes in sexual function and appearance of genitalia, 13 pain, decreased sexual desire and intimacy, 14 and alterations to perceived body image 15 ).Adapted screening measures are needed to facilitate robust communication between patients and their cancer care team, identify concerns not easily detected by generic tools, and enable the enactment of appropriate and timely referral pathways. 16 such, a nuanced distress screening tool specifically for women with gynaecological cancer was developed. 17However, while this tool, the Gynaecological Cancer Distress Screen or DT-Gyn, has undergone an extensive process to establish face and content validity, sensitivity and specificity have yet to be determined.The present study establishes the initial psychometric properties and has two primary aims to: (1) assess the sensitivity and specificity of the DT-Gyn in women with gynaecological cancer compared with four gold standard measures for anxiety, depression, and distress; and (2) establish the clinical cut-point scores appropriate for the intended population.

| Participants and procedure
Experienced cancer nurses at each site identified eligible participants at their outpatient appointments and determined if there were clinical, cognitive, or psychiatric conditions, or other reasons, preventing informed consent.A total of 321 individuals completed the written questionnaire (90% response rate).However, 5 participants did not provide data on key variables (DT-Gyn or Impact of Event Scale-Revised [IES-R]) and were not included in this analysis.
For the community sample, partner organisations and supportive advocacy services were asked to promote the study through flyers, posters, and online platforms (e.g., Facebook).Potential participants were then able to use a QR code or weblink to access the online survey via the REDCap virtual platform.The survey commenced with an extended information sheet and referral list; implied consent was assumed if individuals completed the survey.Overall, 75 women completed the survey, but almost one-quarter (n = 18) of the participants were excluded because of missing data.

| Measurement
A structured survey collected data on socio-demographic characteristics and cancer histories (e.g., age, ethnicity, marital status, education, employment status, cancer diagnosis, treatment received, and previous cancer diagnosis), as well as distress, post-traumatic stress, anxiety, and depressive symptoms.

| Distress thermometer
The distress thermometer (DT) is widely used to screen for global psychological distress in cancer patients, and while this instrument has been widely validated in a variety of cancer populations, one of its limitations is that it does not account for problems or concerns associated with specific cancer types. 18For this reason, the DT-Gyn was developed. 17This instrument was adapted from the National Comprehensive Cancer Network (NCCN) DT 19 and retained the single-item 11-point DT (higher scores denoting higher distress) and 11 of the original problem items.An additional 14 gynaecological cancer specific problem items and three open-ended follow-up questions (other sources of distress not currently listed, confirmation of the key sources of distress being experienced, and whether help for experienced distress is sought) were added (tool is provided in Appendix 1).Information about the process of DT-Gyn development have been discussed in detail previously. 17

| Anxiety scale
The GAD-7 is a 7-item instrument to assess generalised anxiety disorder (GAD).Raw scores are summed to form a scale between 0 and 21, with higher scores indicating greater anxiety levels. 20Initial scale development suggested scores ≥10 were consistent with potentially significant generalised anxiety (sensitivity 89%, specificity 82%, ICC = 0.83) 20 as well as other anxiety disorders (e.g., posttraumatic stress disorder [PTSD], panic disorders and social anxiety disorder 21 ) and its associated functional disability. 22

| Patient health questionnaire
The Patient Health Questionnaire (PHQ-9) is a 9-item instrument used to screen for the presence and severity of depressive symptoms. 23The instrument is scored by summing 8 of the 9 items, with higher scores denoting greater symptom severity and a cut-point ≥10 suggestive of depression.Validation of this instrument has shown sensitivity between 74% and 88% and a specificity of 88% and 91% in detecting major depression. 23

| Psychological distress measures
Two measures of distress were included in this study, the IES-R and Kessler's Psychological distress scale (K10).The IES-R is a 22-item measure that comprises three subscales (intrusion, avoidance and hyperarousal) which are summed to form a measure for current subjective distress (ranging from 0 to 88), with a cut-point of >33 denoting symptoms consistent with post-traumatic distress. 24The instrument has demonstrated excellent internal reliability for the total scale score (α = 0.94-0.95). 25The K10 includes 10-items related to psychological distress, 26 with lower scores (≤19) indicating low levels of distress, scores between 20 and 24 indicating moderate distress, and scores ≥25 indicating high levels of distress. 26

| Sample size
Sample size calculations were informed by DT validation studies. 27,28 assumed the ratio of those with clinically relevant distress (cases, DT ≥ 4) to those without (controls <4) in our sample would be 40:60, with an estimated tool sensitivity and specificity of 85% and 80%, respectively.To ensure a sensitivity ≥85% (95% confidence interval [CI] = 0.79-0.91)and a specificity ≥80% (95% CI = 0.74-0.81)with one-sided α = 0.025, we required 137 cases.Thus, a total sample of 343 women (137 cases/206 controls) was needed to achieve adequate power for this analysis.To account for missing data and non-response (10%), a final sample of 377 women was required.

| Statistical analysis
Statistical analyses were performed using SPSS (Statistical Package for the Social Sciences) version 23 29 and STATA 13. 30 Descriptive data are expressed as counts and percentages, mean and standard deviation (SD) or median and interquartile range (IQR), and bivariate statistics were performed using chi-square (χ 2 ) tests, ANOVA, Pearson correlation coefficients or their non-parametric equivalents.
Statistical significance was set at α = 0.05.Nonparametric estimation of the receiver operating characteristics (ROC) analysis with 95% Bamber confidence intervals SEIB ET AL.
(95% CI) 31 were used to determine the optimal cut-off for the DT-Gyn against the criterion measures for anxiety (GAD-7), depression (PHQ-9), and distress (IES-R and K10).ROC curves estimated the sensitivity (true-positive rate) and specificity (true-negative rate) for each DT value (0-10) relative to the established cut-off score of the GAD-7 (≥10), PHQ-9 (≥10), IES-R (≥33), and K10 (≥25).Optimal DT thresholds were determined using the area under the curve (AUC), which quantified the ability of the instrument to discriminate between participants with and without 'clinically relevant distress'.
For this study, diagnostic accuracy was defined as acceptable (AUC ≤ 0.70), good (AUC ≤ 0.80), and excellent (AUC ≤ 0.90). 32 addition to the ROC analyses, the percentage correctly classified, and the positive and negative likelihood ratios, were determined.To further evaluate performance at a given cut-point, the Youden index was calculated.The Youden index has a maximum value of 1 (a perfect test) and a minimum of 0 (denoting a test with no diagnostic value), with the optimum being when both the sensitivity and specificity are equal to 1. 33

| Participant characteristics
Table 1 presents the characteristics of participating individuals by study site.The average age was 57 years (SD = 14), with those recruited at the Mater and from the community being younger than Gold Coast University and John Hunter Hospital participants (F (3,374) = 8.886, p < 0.01).First Nations Australians comprised a small proportion of the sample and were primarily recruited from Mater and John Hunter Hospitals (χ 2 (3) = 8.524, p = 0.04).In contrast, the proportion of participants born outside Australia varied from 36% at the Mater Hospital, to 8% of the community sample (χ 2 (3) = 24.546,p < 0.01).Around three-quarters of participants reported a household income of $80,000AUD or less (29.6%, < $20,000; 34.9%, $20,000 to $59,999; 15.3%, $60,000 to $79,999), 58% were married or living with their partner, and one-third (34.5%) were retired.No differences were noted between these groups.

| Distress
Non-parametric descriptive statistics and correlations using Spearman's rho are outlined in Table 2.The median DT score for individuals in this study was 3 [IQR = 5.0], and using the recognised DT cut-off of 4, 46.6% of participants were classified as distressed (n = 174), and when using a cut-off of 5, 39.9% of participants were classified as distressed (n = 149).The median score for the IES-R was

| DISCUSSION
This study examined the sensitivity and specificity of the DT-Gyn, a nuanced tool designed to accurately identify and manage distress in women with gynaecological cancer.The DT-Gyn showed good sensitivity and specificity across all gold-standard measures of psychological distress (AUC = 0.85-0.89,sensitivity = 85-92, specificity = 73-78).Other studies in gynaecological cancer patients have recommended cut-points ranging from 2 to 7. 34,35 The heterogeneity of cancer populations (age, cancer type, symptom burden and its impact on interpersonal functioning 13,14 ) partially explains differences in defined 'clinically relevant distress', though the NCCN identifies DT scores ≥4 as significant. 19Moreover, while a lower DT cut point will increase sensitivity, the trade-off needs to be commensurate with the potential benefits.At a cut-point of three, our study provided little certainty about the probability of 'clinicallyrelevant distress' among those with a positive DT score (positive likelihood ratio ~1.8 36 ).In contrast, using a DT cut-point ≥5, almost Abbreviations: DT, distress thermometer; GAD-7, generalised anxiety disorder scale; IES-R, impact of event scale-revised; K10, Kessler's psychological wellbeing scale; PHQ-9, patient health questionnaire.
a Caseness was defined as ≥33 for Impact of Event Scale-Revised, ≥10 on the anxiety (GAD-7) and depression (PHQ-9) scales, and a score of ≥ 20 for the K10. b The optimal cut-off for the DT is given by the maximum value of the Youden index which measures the vertical distance from the line of equality to the ROC curve.
80% of women in this study were correctly classified and, given a positive result, had a 3.5-fold increased probability of having clinically relevant distress. 36sults from this study are consistent with other research in gynaecological cancer populations.Around 40% of women reported a DT score ≥5 compared with 35% of women in a US study. 37Using a cutpoint of 5 and a clinical pathway model for universal distress screening, the American study found a 15% increase in multidisciplinary service referrals and a 32% increase in social work referrals alone.Given the sources of distress reported by many women in this study and the request for informational and practical supports, it can be anticipated that implementing comprehensive distress screening programs would result in similar resource/infrastructure requirements.
Of course, the increased service requirements associated with routine distress screening must be balanced against the significant economic and psychosocial costs associated with unrecognised and untreated distress.According to the Australian Bureau of Statistics, around 15% of Australian adults report high or very high levels of psychological distress using the K10, which has a considerable economic impact on individuals, families and the broader community. 38 cancer populations, distress is associated with reduced treatment adherence, dissatisfaction with clinical care, and decreased survival, 39 and targeted and timely distress management is likely to reduce its potential negative consequences.Thus, while comprehensive screening programs can identify distress in women with cancer, understanding the contextual factors that increase susceptibility can help cancer care teams and service providers to manage women's concerns more efficiently.For example, the prevalence and severity of distress frequently differ by cancer type, socioeconomic status, and life stage 1,37,40 and during periods of uncertainty along the cancer journey. 40Clinicians and other health professionals in this arena need to be cognisant of the characteristics that increase vulnerability to distress to tailor supportive care pathways to meet the needs of specific sub-populations. 1,40

| Study limitations
Several limitations need to be highlighted.First, the study must be considered within the context in which it occurred, that is, the global F I G U R E 1 ROC curves for the Distress Thermometer criterion measures for anxiety (GAD-7), depression (PHQ-9), and distress (IES-R and K10).DT, distress thermometer; GAD-7, generalised anxiety disorder scale; IES-R, impact of event scale-Revised; K10, Kessler's psychological wellbeing scale; PHQ-9, patient health questionnaire; ROC, receiver operating characteristics.
SEIB ET AL.

| Clinical implications
Despite the brevity and simple structure of the DT-Gyn and other distress screening tools, the implementation of routine distress screening remains ad hoc. 41Implementing comprehensive distress screening is operationally complex and screening naïve services need to consider how to best integrate distress screening programs into their setting and ensure adequate systems of care to treat distressed patients. 42It is not enough to simply identify distress in these populations, and while not all distressed patients will accept referral, psycho-oncology infrastructure is needed.This is particularly important as identifying patients with clinically relevant distress is likely to increase the demand for supportive cancer care. 8 such, implementation of low intensity models is recommended, where access is the central guiding value and where intervention depth and focus can be guided by the level of distress and the nature of women's concerns. 43Using the tiered approach, low in-

| Conclusion
This study established the sensitivity and specificity of the DT-Gyn in women with gynaecological cancer.It suggests a DT cut point ≥5 is optimal in detecting 'clinically relevant' distress, anxiety, and depression in this population.Moreover, the information generated through the 'cancer-specific problem list' can enhance disclosure, thus providing a significant opportunity to improve the supportive care for women with gynaecological cancer.
paper presents data from 373 individuals previously treated for gynaecological cancer recruited through Australian gynaecological cancer services (Gold Coast University Hospital, Queensland, Hunter New England Centre for Gynaecological Cancer, New South Wales, Mater Hospital, Queensland), partner organisations (The Wesley Hospital Choices Cancer Support Centre [Choices], Icon Cancer Centre), and gynaecological cancer support/advocacy services (Cancer Council Queensland, Ovarian Cancer Australia, Australia New Zealand Gynaecological Oncology Group, sTEAL STRONG, Queensland Centre for Gynaecological Cancer).Inclusion criteria included being aged 18 and over, being diagnosed with gynaecological cancer in the past 12-month, and the ability to speak and read Grade 8 English, while those with medical contraindications preventing participation were excluded from this study.Women awaiting confirmation of results or who were considered suspected of having a gynaecological cancer (diagnosis pending) were deemed ineligible for participation.Ethical approval was obtained from relevant Human Research and Ethics Committees (HREC/2018/QGC/48488; GU Ref No: 2019/ 151) prior to participant recruitment.All participating women provided informed consent and participated in the study without being provided incentive to do so.
tensity interventions emphasising self-management, symptom control, and behaviour change can be delivered by a range of different health professionals involved in the woman's care (for example including specialist cancer nurses and allied health), while for those who report high distress and complex problems, more specialised or intensive interventions can be provided by trained specialist health professionals or multi-disciplinary team.43Reducing contextual barriers (poor health literacy, anticipated resistance in patients, lack of training for staff, and having sufficient time to devote to patients' identified concerns8 ) associated with screening non-adherence is also necessary.According to the NCCN, distress screening education needs to include the following components:(1) The prevalence of distress in women with gynaecological cancer and its potential impact on clinical care 40 ; (2) Using a systematic approach to identify patient distress; (3) Potential sources of distress reported by gynaecological cancer patients and the strategies for improving supportive cancer care,40 and; (4) Guidance in developing topic-elicitors that promote therapeutic communication with patients about their distress and the importance of managing their psychosocial concerns.8 Characteristics of the sample.a T A B L E 1

Measure a Sensitivity 95% CI Specificity 95% CI Youden index Distribution Optimum cut- point for the DT b
Dispersion, correlations and 95% confidence intervals a between the DT-Gyn and validated instruments.
a Estimation is based on Fisher's r-to-z transformation and Fieller, Hartley, and Pearson standard errors.**Spearman Rho for rank order correlation is significant at the 0.01 level (2-tailed).T A B L E 3 Summary validation measures for the DT-Gyn including the Youden index.
COVID-19 pandemic.COVID-19 has profoundly affected individuals, families, and the wider community, and the impact posed by lockdowns, physical distancing, and potential social isolation cannot be underestimated.While the effects of COVID-19 on health and wellbeing are well documented, sensitivity analysis in this study did not find significant differences in distress levels by study site over time (see FigureS1in Online Supplemental Materials for further details).
1trument.Finally, women in this study were treated as a homogenous group.While this was appropriate for statistical purposes, it is important to acknowledge the heterogeneity of women with cancer and the particular vulnerabilities affecting specific patient subgroups and further research exploring differences across stage (early vs. advanced), cancer type, and age are needed.1Despitethis, anecdotal evidence suggested that participants and research staff perceived the tool positively and welcomed enhanced cancer care opportunities.