Assessing decision regret in caregivers of deceased German people with cancer—A psychometric validation of the Decision Regret Scale for Caregivers

Abstract Background Decisional regret during or after medical treatments is linked to significant distress. Regret affects not only patients but also caregivers having an active or passive role during decision making. The Decision Regret Scale (DRS) is a self‐report measure for regret in patients after treatment decisions. However, practical and psychometrically robust instruments assessing regret in caregivers are lacking. Objective To develop and validate a caregiver version of the DRS (Decision Regret Scale for Caregivers [DRS‐C]). Design Psychometric validation based on a web survey. Setting and participants 361 caregivers of deceased German people/patients with cancer. Main variables studied Besides structural validity and test‐retest reliability, we evaluated measurement invariance accounting for gender, age and closeness of relationship, and tested hypotheses on convergent/discriminant validity. Results Forty‐five per cent of all caregivers demonstrated decision regret. Confirmatory factor analyses strongly supported the unidimensional structure of the DRS‐C and pointed to strict invariance. The DRS‐C demonstrated very good internal consistency (α = 0.83, 95% CI [0.81, 0.86]) and test‐retest reliability (ICC [A,1] = 0.73, 95% CI [0.59, 0.83]) along with sound convergent/discriminant validity. Concerning responsiveness, DRS‐C scores remained stable over a 12‐week period in 83.3% of all caregivers. Receiver operating characteristic analysis yielded a cut point of 43 for the identification of significant decision regret (AUC = 0.62, 95% CI [0.56, 0.68]). Discussion and conclusions The lack of a gold standard instrument prevented us from examining the criterion validity and determining a minimally important difference. Nevertheless, the DRS‐C provides valid and reliable information regarding caregiver regret following medical decisions. Above all, it captures a crucial aspect of the treatment experience in caregivers.


| INTRODUC TI ON
Many patients with serious illnesses (eg cancer diseases) and their caregivers are regularly confronted with difficult treatment decisions particularly when it comes to end of life. In the light of an unfavourable outcome, decision-related regret becomes more likely.
Connolly and Reb have defined regret as (a) aversive and avoided if possible, (b) an intimate interplay of thought and feeling, (c) distinct from other specific emotions, such as disappointment, and from general negative affect, and (d) based on a comparison of some event or process with another, better event or a process that 'might have been'. 1 Such comparisons occur regularly in the context of non-hypothetical treatment decisions, for example in cancer care. 2 For patients, the Decision Regret Scale (DRS) has been found to be a valid and reliable measure with a unidimensional structure and very good internal consistency. 3,4 Findings on associations between the DRS and comparator instruments point to sound construct validity.
The DRS has been translated into seven languages and adapted for the application in various cultural contexts. 2 The DRS has been applied in observational studies, for example, demonstrating that higher physician empathy predicts lower decision regret in people with cancer or patients with cancer. 5,6 The DRS has also been used in randomized controlled trials of decision aids which recently have been shown to reduce decisional conflict. [7][8][9] Indeed, decision regret often reaches high levels (increasing 6 months or more after a decision) due to very poor outcomes and definitive knowledge about the outcome. 10 Several risk factors have been identified as temporal predictors for decisional regret including decisional conflict, unmet information needs, serious adverse physical health outcomes and anxiety. 11 Notably, decision regret is commonly not associated with patients' sociodemographic characteristics but regularly linked to decreased mental well-being making. 11 Along with patients, caregivers are often affected by decision regret in the course or in the aftermath of participating in treatment decisions. 12,13 Indeed, one study demonstrated that caregivers experiencing decision regret have a lower health-related quality of life. 14 Furthermore, there is some recent evidence that decisional conflict in family members is negatively linked to their quality of end of life communication. 15 However, research on decision regret in caregivers is scarce since an equally practical and psychometrically robust instrument for its assessment is lacking. 4,11,16 Although one scale has been proposed some years ago, it was validated in a Japanese population only. 14 Given the cultural differences, the generalizability of this instrument to Western countries is questionable. Overall, the few available measures for the emerging field of decision regret research in caregivers seem not to have been evaluated sufficiently.
To fill this gap, this study aimed to derive a caregiver-adapted version of the widely used DRS originally conceptualized for patients. 2,3 Based on a comprehensive psychometric validation, we propose the robust and easy-to-administer Decision Regret Scale for Caregivers (DRS-C) as a new instrument for measuring decision regret in caregivers.

| Study design
In a cross-sectional web survey, we examined the psychometric properties of the DRS-C. Besides assessing the structural validity in a confirmatory factor analysis (applying the factorial structure of the original DRS) and examining test-retest reliability, we also conducted a cut point based on a receiver operating characteristic analysis. For evaluating construct validity and responsiveness over time, we predefined hypotheses specifying directions and magnitude with comparator instruments.

| Sample size estimation
Accounting for confirmatory factor analysis, we followed established conventional criteria and determined 300 individuals as required minimum sample size. 17 Assuming a type 1 error of 5% (two-tailed) with a power of 0.80 and a total sample size of 300 observations, we were able to detect an effect size of r = 0.16 for Pearson productmoment correlation coefficients.

| Sampling procedure
We collected data from 3 June 3 until 10 December 2015 and, for the retest, from 26 August 2015 until 3 March 2016. The median time interval between test and retest was 12 weeks. The sampling procedure followed a non-probability approach where individuals were recruited through volunteering in an online survey provided via the web-based interface SurveyMonkey ® . Specifically, we identified 21 German grief support websites and online groups in a systematic online search and then invited caregivers of deceased individuals through postings on these websites (Appendix S1). The web survey implementation followed the guidelines proposed by Dillman et al (2014). 18 To ensure unbiased answers and prevent measurement error, we did not offer any incentives to increase the attractiveness of participation.

| Reporting and ethical standards
We report the psychometric assessment in accordance with the Consensus-based Standards for the selection of health status K E Y W O R D S aggressiveness of care, cancer, caregiver, confirmatory factor analysis, decision regret, measurement invariance Measurement INstruments, COSMIN. 19 The study was approved by the Institutional Review Board of the Medical Faculty of Heidelberg University, Heidelberg, Germany. All participants had the opportunity to ask questions prior to enrolment and gave their informed consent prior to assessment. They could withdraw from the study at any time.

| Participants
The approached sample amounted to 559 eligible individuals with a subsequent completion rate of 65.3% (N = 365). After removal of four multivariate outliers, the data set for statistical analysis comprised 361 individuals. Participants were adults, aged 18-79 (M = 46.7, SD = 10.9), who were caregivers of deceased German people with cancer or patients with cancer aged 18 years or older at the time of diagnosis. We applied the caregiver definition of the American Cancer Society. 20 We did not enrol any living people with cancer or living patients with cancer. Exclusion criteria comprised a time period of <6 months between the patient's death and the enrolment of the bereaved.

| Decision Regret Scale for Caregivers (German version)
The DRS-C is the caregiver version of the Decision Regret Scale originally developed for patients by Brehaut and colleagues (the DRS-C is available in Appendix S2). 3 The DRS-C is a unidimensional, self-report instrument consisting of five items, which are answered on a 5-point bipolar intensity scale. Completers evaluate the item statements by circling a number from 1 (strongly agree) to 5 (strongly disagree). Items 2 and 4 are phrased in the negative direction to avoid acquiescence bias. After reversing the scores of these two items, the overall sum score is produced by taking the mean of the five items and converting it to a score ranging from 0 to 100 by sub-

| Satisfaction with decision scale (German version)
The Satisfaction with decision scale (SWD) measures satisfaction with health-care decisions independent from a good or bad decision outcome. 21 This unidimensional, self-report instrument includes six items, which are rated on a 5-point bipolar intensity scale. In the original validation study, internal consistency as measured in Cronbach's alpha was at α = 0.88 indicating good reliability. Construct validity has been assessed through correlation with the Confidence in Decision Scale (r = 0.64) and Decisional Conflict Scale (r = −0.54). In our study, we removed the fourth item on the expected success in carrying out decisions from the SWD since we inquired about decision making in the past.

| Inventory of complicated grief (German version)
The ICG-D is a self-report questionnaire that can be used to screen patients for complicated grief. 22

| Patient health questionnaire-9 (German version)
The PHQ-9 is a widely used brief depression severity measure with high validity and reliability which scores each of the nine DSM-IV criteria as '0' (not at all) to '3' (nearly every day). 24

| Generalized anxiety disorder-7 (German version)
The widely applied 7-item GAD-7 is a practical and valid self-report anxiety questionnaire with unidimensional structure and good internal consistency (α = 0.89). 25

| Data analysis
The data preparation included screening for normality and outliers.
First, we followed recommendations to inspect univariate distributions. 27 Specifically, we assumed multivariate normality if skewness and kurtosis for the DRS-C item values fell in the normal range (−2 to 2 and −7 to 7, respectively). 28 Additionally, we computed Mahalanobis D 2 and detected four multivariate outliers among the 365 respondents. These outliers were deleted prior to the subsequent analyses. Second, to account for potential missing data due to item non-response and gain efficiency relative to complete-subject analysis, we applied full information maximum-likelihood (FIML) estimation for incomplete data as part of the confirmatory factor analyses. FIML treats observations with random missing values. For descriptive statistics, we summarized results for discrete variables  34 We assessed DRS-C scores on an independent second administration for 60 participants who had agreed to be followed up by the study team. Test conditions were similar compared with the initial assessment (same type of administration and web environment with the same instructions).
To assess convergent validity, we tested the following a priori specified hypotheses in bivariate Pearson product-moment correlations. For magnitudes, we applied the thresholds for effect sizes introduced by Cohen. 35 We postulated that the DRS-C would correlate.
• Negatively with the SWD score. This association would amount to a large effect size as the SWD measures satisfaction with decisions.
• Positively with the ICG-D score. This association would amount to a small-to-moderate effect size, since complicated grief and regret are somewhat related constructs. However, we expected a moderate effect size at the most, since complicated grief has some overlap with depression and is highly variable over time. 36 • Positively both with PHQ-9 and with GAD-7. This association would only amount to a small effect size, since depression and anxiety exhibit a higher variability within-subject over time compared with regret and both constructs are related but not identical with regret.
Regarding differences between groups, we additionally hypothesized that DRS-C scores would be significantly higher in those bereaved caregivers who had witnessed aggressiveness of care (AOC) at the end of life of the deceased compared with those caregivers who had not.
For measuring AOC, we applied established claim-based indicators. 37 We assessed between-group differences in a univariate analysis of covariance (ANCOVA), adjusting for the effects of gender, age, tumour type and place of death of the deceased.
To assess responsiveness over time, we evaluated the hypothesis that DRS and SWD change scores from T1 to T2 would be significantly negatively correlated. To determine an optimal cut point, we used the pROC package for a receiver operating characteristic (ROC) analysis. 38 ROC curves graphically depict a test's ability to correctly identify 'truepositive' and 'true-negative' individuals for various test cut points. 39

| Content validity
The DRS-C aims to measure decisional regret in caregivers of patients with any type of disease. The concept of decision regret covers the 'negative emotion involving distress or remorse following a decision' that is linked to dissatisfaction with medical decision making, lower quality of life and poorer health. 2 Since we wanted to apply our instrument to a German population, we used a parallel translation approach to derive a German version of the DRS, which was then minimally adapted to derive the DRS-C. 41 Allowing for the evaluation of a whole treatment period, we adapted the item word- In our survey sample, we did not find any floor or ceiling effects indicating that no extreme items were missing in both ends of the scale. Descriptive statistics for the DRS-C items are given in Table 2 and Appendix S6.

| Internal consistency
Internal consistency of the DRS-C scale as measured by Cronbach's alpha amounted to α = 0.83, 95% CI (0.81, 0.86), which can be considered very good. 44

| Cross-cultural validity/measurement invariance
To evaluate across-group equivalence of the parameters, we tested for measurement invariance employing multi-group confirmatory F I G U R E 1 Final confirmatory factor analysis model for the Decision Regret Scale for Caregivers (DRS-C) with path coefficients presented in standardized form factor analyses across gender (male vs. female), age groups (under 50 years old vs. 50 years or older) and closeness of relationship (spouse/first-degree relative vs. other; Appendix S8). 45,46 In sum, we concluded that the respondents from different groups with same the latent level of decision regret responded similarly to a particular item of the DRS-C. Since we found strict invariance, we then tested for the equality of latent means and did not find any significant differences between the groups (two-sample t test for gender:

| Reliability
The median time interval between test and retest was 12 weeks (interquartile range: 6 weeks), which we considered long enough to prevent recall bias, yet short enough to ensure that no change in regret had set in. With respect to test-retest reliability, we identified robust measure-

| Measurement error
The smallest detectable change in the DRS-C score, defined as the change score beyond measurement error, was derived from the standard error of measurement of 10.74 (N = 60) and amounted to 29.76 for individuals and 5.29 for comparisons of mean scores between groups, respectively.

| Criterion validity: Optimal cut point and receiver operating characteristic analysis
We conducted a ROC analysis to identify an optimal cut point on the DRS-C that indicated clinically significant decision regret based on concomitant complicated grief (as indicated by an ICG-D score > 25).
The AUC for the DRS-C was 0.62, 95% CI 95% [0.56, 0.68], indicating that the DRS-C cut point was able to accurately discriminate individuals with clinically significant regret above random chance (see

| Hypotheses testing for construct validity
Bivariate associations between the DRS-C score and the comparator instruments can be found in Table 3. Since missing data per association did not exceed 3%, we performed pairwise-complete correlations at this point. The findings supported three of our four hypotheses. However, for AOC, we did not find any differences between DRS-C scores for those caregivers who had witnessed AOC compared with those who had not, F(4, 353)=0.91, P = .46 (adjusted for gender, age, tumour type and place of death of the deceased).

| Responsiveness
For the 60 participants for whom two measurements were available,

| D ISCUSS I ON
To address the need for a valid and reliable self-report measure of decision regret in caregivers, the current study assessed the psycho- with the one we observed. This difference might be due to the fact that this study recruited in palliative care where, in comparison with other medical specialties, caregiver involvement in decision making is much more common. 50 For the most part, decisions in palliative care are based on the needs of patients and caregivers so that regret might generally be lower in this particular context. 51 At any rate, Smith-Howell et al did not report whether they had validated their measure in a caregiver sample in advance. 15 This TA B L E 3 Means, standard deviations and correlations with confidence intervals study was published after we had completed our data collection, so that we could not account for it when pre-testing the DRS-C.
We found one study that explored decision regret in family members actively taking care of people with cancer or patients with cancer. 52 For this small sample of head and neck cancer patient caregivers, mean regret scores were significantly lower (M = 10.5, SD = 9.9) compared with our sample (N = 30). However, patients were treated with curative intent, data were collected by clinicians, and the authors did not report whether they had validated their DRS version for caregivers. In a study of surrogate decision makers for the chronically critically ill, the mean decision regret one week after making a tracheostomy or feeding tube decision also was significantly lower (M = 16.3, SD = 11.6) compared with our sample. 53 The content of the decision making was less comparable to the one under investigation in our study. Notably, the sample size was very small (N = 16), which is why we did not consider perceived effective decisions, which was identified as predictor for regret, for further refinement of the DRS-C.
All in all, previous findings on decision regret either were developed in caregivers actively looking after people with cancer or patients with cancer or concern less existential scenarios. The somewhat higher regret scores in our sample may be due to caregivers having not been able to make an informed choice for their loved ones, since shared decision making is not necessarily implemented on a regular basis in German hospitals. In this regard, there is some evidence that shared decision making may potentially modify the decisional regret experience of bereaved family caregivers. 54 Unfortunately, we did not assess whether an informed choice was made.
Some methodological limitations as sources of potential bias must be discussed. First, we did conduct only two cognitive interviews as part of pre-testing by which we may have missed aspects of relevance and/or comprehensiveness. Nonetheless, we considered this acceptable as our instrument is very similar to the original DRS for which content validity has been investigated in previous work. 3,4 Second, our sampling followed a non-probability approach as individuals volunteered in a web survey. Due to coverage error and potential self-selection, sampling error may have occurred as collected data may stem from only a subset of the entire population and potentially lack representativeness.
Specifically, non-whites, people aged older than 65 years, people with lower incomes and those with less education are known to have lower Internet access rates than their counterparts and, therefore, are more likely to be under-represented in web surveys. 55 Indeed, our sample was somewhat younger compared with the ones in the original DRS validation studies. 3 As in a comparable previous study, 14 we observed an oversampling of women who were on average relatively young and tended to be full-time working. However, DRS-C scores in our sample were sufficiently dispersed and to date, for web surveys, there is no simple procedure available for drawing samples (comparable to random-digit dialling) so that coverage/sampling error can be fully avoided. 18 For the German population at least, to our best knowledge, there is currently no Internet panel allowing for probability-based sampling. Furthermore, the population of bereaved caregivers itself cannot be well reached and drawing a sample from this population may be very difficult regardless of the sampling procedures. At any rate, our findings must be validated in a more comprehensive sample in future replications. Third, due to the nature of our sampling, we could not retrieve any information on non-responding participants which impeded sensitivity analysis between participants and non-respondents for the assessment of potential selection bias. Fourth, we relied on participants recollecting a remote exposure and hence cannot rule out information bias. Finally, the overall ability of the DRS-C to discriminate between individuals with and without clinically significant decision regret was rather poor. 56 At this point, we would therefore recommend the DRS-C for research and screening purposes that are followed by a definitive assessment of clinical distress with other patient-reported outcomes, only. The cut point we identified is somewhat higher than the ones that have been suggested previously. 2 However, this can be explained by the rather common adverse outcomes in cancer treatment that contribute to higher decision regret. In fact, it has been recently proposed to apply cut points of 30 or higher. 2

| CON CLUS ION
In sum, the findings of this study indicate that the DRS-C provides

ACK N OWLED G EM ENTS
We would like to thank Jamie C. Brehaut, PhD (for granting permission to use his original instrument and for checking the backward translation for language and interpretability), and Justus Tönnies, MSc (for proofreading the manuscript).

CO N FLI C T O F I NTE R E S T
The authors declare that there is no conflict of interest.