Aims: To develop subscales for clinical use of the Social Difficulties Inventory (SDI) with score interpretation guidance for use in routine oncology practice.
Background: Patient-reported outcome measures are used increasingly in cancer care but successful implementation is dependent on a combination of sound psychometrics, guidance on clinical meaning and good clinical judgement. The SDI, a 21-item instrument (SDI-21) developed for use in cancer care, demonstrated good psychometric properties. Rasch analysis of the SDI resulted in a 16-item interval scale of Social Distress (SD-16), which allowed for establishment of some clinical utility guidance but further work was required to optimise meaningful interpretation in clinical practice.
Data sources: Data were pooled from three studies investigating psychometrics and clinical utility of the SDI-21.
Statistical analyses: Common factor analysis was undertaken on SD-16 items. Subscales were derived from the resulting factors and calculated by summing the scores of associated items. Subscale reliability was evaluated using Cronbach's α.
Results: There were 652 participants. A three-factor model explaining 53.3% of the variance was extracted forming the basis of the subscales: Everyday living, Money matters and Self and others. Subscale reliability was good. In a clinical setting, a 2-point change in subscale score could be interpreted as a clinically meaningful difference.
Patient-reported outcomes (PROs) are being applied in clinical practice for screening and monitoring of problems, promotion of patient-centred care, as decision aids, facilitating team communication and for evaluation of care. However, evidence for efficacy is mixed possibly due to the complexity of the intervention which involves not only sound PROs but also training on interpretation of scores and management guidelines for staff 1. In the United States, guidelines and national initiatives have been published to promote assessment and management of supportive care needs of cancer patients including psychological, social and spiritual concerns 2. In 2007 the UK Cancer Reform Strategy 3 included plans for the introduction of Holistic Common Assessment 4. The assessment of psychological, social and spiritual issues was required for all cancer patients at a number of recommended time points. For assessment to become embedded within routine practice, easy methods must be found to allow busy staff to identify and spend time with patients who are experiencing problems. Successful implementation entails both psychometrically sound and clinically meaningful PROs with trained staff to interpret the scores using guidance provided and clinical judgement 5.
Although there is a considerable social impact following a diagnosis and treatment for cancer 6 there are few PROs which include sufficient items within the social domain. As part of a programme of work aimed at developing and evaluating a system of routine patient centred assessment, we developed a questionnaire to assess social difficulties experienced by cancer patients, the 21-item Social Difficulties Inventory (SDI-21) which could be used in everyday practice and administered either on a touchscreen computer or in paper format. The National Cancer Action Team adopted almost in its entirety the items from the SDI-21 as part of the assessment guidance for the social and occupational domain of Holistic Common Assessment of Supportive and Palliative Care Needs for Adults with Cancer 4.
Each item of the SDI-21 self-report covers a different area of potential difficulty in daily life: (1) Independence, (2) Domestic Chores, (3) Personal Care, (4) Caring for dependents, (5) Support available for those close to you, (6) Welfare benefits, (7) Finances, (8) Financial services, (9) Work, (10) Planning the future, (11) Communicating with those close, (12) Communicating with others, (13) Sexual matters, (14) Plans to have a family, (15) Body Image, (16) Isolation, (17) Getting around, (18) Where you live, (19) Recreation, (20) Holidays and (21) Other (not specified). The scoring ranges from no difficulty to very much difficulty (0–3). For details and examples see the Appendix for the full questionnaire. The first stages of development of the SDI-21 included reliability, validity and factor structure analyses 7. Factor analysis revealed a four-factor structure with factors labelled: physical ability, providing for the family, contact with others and an undefined group with no readily definable theme. The SDI-21 demonstrated good psychometric properties but as with other instruments this is not sufficient to justify use with confidence in clinical practice. In order to assist the development of a clinically meaningful measure of Social Distress based on the SDI-21, Rasch analysis was undertaken resulting in a 16-item uni-dimensional interval scale. Disordered categories were observed for four of the items, that is the direction of the estimate did not correspond to the incremental direction of the categories. The third and fourth categories were therefore collapsed for these items and were recoded to a 3-category response, i.e. 0,1,2,2. Using standard coding and recoding for the disordered categories allows summing of the 16 items to produce a scale of Social Distress (SD-16) 8. In a cross-sectional interview study matching expert evaluation with distress scores, a cut-off point of ≥10 on the SD-16 scale was found to be the optimum indicator of social distress 9. In a second study examination of minimally important differences (MIDs) in the SD-16 scale resulted in a recommendation of an estimate of 3 as a clinically meaningful difference for use in serial assessment 10. Table 1 provides an overview of the development of these measures of social difficulties and distress.
Table 1. Development of measures of social difficulties and distress
A tool for screening, monitoring or prioritising problems for use in routine practice must fulfil staff requirements for brevity, utility, relevance, ease of use and simplicity in interpretation 11 without compromising the scope of the tool and evaluation of individual patient-centred issues of concern 12. Accommodating staff demands may result in poor content validity. One resolution to this tension is to use a tool with a number of scoring options allowing for flexibility according to staff and patient priorities. Health care professionals now have two options for implementing the SDI-21 in routine patient-centred assessment: use of individual items for detailed assessment and, for more general assessment, use of the SD-16 scale total score with associated scoring guidance. However, when time is short or comparative assessment over time is required, the means to focus on certain aspects of SD-16 without having to examine each of the items individually may be beneficial to staff. The aim of this work is to develop subscales from the 16-item SD-16 scale with guidance on subscale score interpretation for use in everyday oncology practice.
Participants and procedures
Data for this analysis were collated from three separate studies: one examining the psychometric properties 7 and two examining the clinical meaning and utility of the SDI-21 9, 10. All were approved by local ethics research committees. In the psychometric study, 270 patients, at different stages of disease, completed the SDI-21 and were then randomised into either a test–retest reliability arm or a validation arm. The first study evaluating the clinical meaning and utility of the SDI-21 patients was cross-sectional with 189 patients, at different stages of disease, completing the SDI-21 followed by an interview within a week of the completion of the questionnaire. In the second clinical meaning and utility study, 193 newly diagnosed patients treated with curative intent completed the SDI-21 at baseline and then were followed up at 6, 12 and 24 months. Full details of these studies are reported in the original publications 7, 9, 10. Participants from the three studies were adult cancer patients recruited from haematology, oncology, chest medicine or surgical wards or outpatient clinics.
All participants completed the SDI-21, a number of other questionnaires and socio-demographic details. Clinical data were collected from the medical notes.
Common factor analysis using Principal Axis Factoring with the rotational method oblique (Oblimin with Kaiser normalisation) was undertaken on the 16 SD-16 items (using standard coding, 12 items, and recoding, 4 items). This was because factors were expected to be related to each other, following Rasch analysis 13. Missing data were excluded pair-wise. Factor loadings of less than 0.20 were disregarded.
Scale scores were calculated from SD-16 items by adding together the item scores for each subscale as derived from the factor analysis as well as for the total scale of social distress. Internal consistency of each summated scale, was evaluated using Cronbach's α with a lower acceptable limit of 0.7.
MIDs for subscale scores
In order to interpret the clinical meaning of the derived subscale scores, an analysis of MIDs was conducted, estimated by 1.0 standard error of measurement (SEM). As a high internal consistency could be expected from previous studies 5, a distribution-based method was employed to estimate MIDs for the SDI-21 subscale scores. Internal consistency was evaluated, and the SEM was calculated for each subscale as a function of the reliability and variance of that scale: . Although MIDs identified in this way will be specific to the socio-demographic and clinical profile of the sample population in this study, they will provide a point of reference for novel patient groups as well as a framework for developing values for populations of special interest, e.g. young men with germ cell cancer.
Again, internal consistency was considered acceptable above 0.7 using Cronbach's α. Both changes of 1.0 SEM and the more conservative threshold of 1.96 SEMs have been shown to be in agreement with MIDs in patient-reported quality of life measures, 1.0 SEM representing the best fit 14. The SEM values denote changes of one and two standard deviations, respectively, and both estimates will be included in this study and interpreted as minimally and moderately clinically meaningful differences for each derived subscale.
Participants were 313 men (median age=56 years and range=18–88 years) and 339 women (median age=56 years and range=21–88 years). Clinical characteristics are summarised in Table 2.
Table 2. Clinical characteristics of participants (N=652)
Head and neck
Disease-free survivors >2 years since diagnosis
Disease free <2 years since diagnosis
A three-factor model was extracted explaining 53.3% of the variance. The pattern matrix was reasonably simple with 13 of the 16 SD-16 items loading on to one of the three factors only and three items loading on to more than one factor (Figure 1). Item 10 ‘Planning the future’, loads almost equally on to the second factor ‘Money matters’ and third factor ‘Self and others’. Item 16 ‘Isolation’ loads on to two factors but reaches the 0.30 level for the third factor ‘Self and others’ only. Item 5 ‘Support available for those close to you’, loads across all three factors and not reaching the 0.30 level for any.
The pattern matrix was used as the basis for three proposed subscales. In Figure 1 reliability of the three subscales is given for the items grouped as shown. Where items loaded on more than one subscale (Item 5 ‘Support available for those close to you’ and Item 10 ‘Planning for the future’), the decision to keep items within a scale was based on maintaining the best reliability across the three subscales. Figure 2 shows the scoring of subscales based on the factor analysis.
MIDs for subscale scores
Each of the subscales showed an acceptable internal consistency ranging between 0.717 and 0.820. The calculated SEM values can therefore be considered valid distribution-based estimates of MIDs. Results are summarised in Table 3.
Table 3. Descriptive statistics for SD-16 subscales
Maximum possible score
Self and others
All three subscales displayed SEM values of similar magnitude. Using the more sensitive threshold of 1.0 SEM, which will be interpreted as a minimally clinically meaningful difference, a change of score between 1.0 and 1.5 is observed. For the more conservative estimate of 1.96 SEMs, which will be interpreted as a moderately clinically meaningful difference, the associated change of score is between 2.0 and 3.0. For practical purposes in a clinical setting, a change of about 2 points could be considered to represent a meaningful improvement or deterioration for an individual on each of the subscales.
Three subscales were identified following factor analysis of the SD-16 scale with a change in score of 2 or more indicating a clinically meaningful change in subscale score. Overall guidance on score interpretation now includes a variety of options dependent on the type of assessment and time available. The overall SD-16 score provides a quick evaluation of whether or not a patient is below or above the cut-off point for SD-16 and, if assessment is serial the value of change in SD-16 and subscale score over time. The subscale scores will provide information on the focus of SD-16 and flag when changes in score are of clinical significance, similar to the ‘nesting’ of the anxiety and depression subscales within the Hospital Anxiety and Depression scale 15, 16. Thus while the SD-16 flags social distress cases, the remaining individual SDI-21 items maintain the content validity of the Social Difficulties measure. These also allow for checking and follow-up of associated specific areas of concern to the patient. Recommendations are shown in Figure 3.
Graphical output of serial assessment displaying both SD-16 subscale scores and single-item responses may be of most help to staff. It is anticipated that in the future data will be collected via touch screen interfaces, for example, with automated reporting to clinicians. Clinical staff would therefore not have to be involved in scoring and summarising responses. In Figure 4 the change in SD-16 and subscale scores with the five single items absolute scores are shown for a 61-year-old woman with breast cancer who took part in the Longitudinal study 10. In this case the SD-16 score did not reach the cut-off point of 10 at any time point and therefore would have not triggered discussion. However, the change in SD-16 score between baseline and 6 months was a worsening of 5 (⩾3) which would have flagged the need for discussion. The subscale contributing most to this was Everyday living. Between 6 and 12 months, the SD-16 score did not change and was below the cut-off point of 10. If this had been the only score available, no discussion with staff would have been triggered. However, on examination of the change in subscale scores it can be seen that although Everyday living improved by a minimal clinically meaningful difference, the Money matters subscale worsened by a minimal clinically meaningful difference, which would have been a prompt for discussion by staff. Using the SD-16 guidance only would have masked this change. The single items Sexual matters and Holidays may have warranted some discussion at 6 and 12 months.
The Psychometric evaluation of the SDI-21 revealed a four-factor structure explaining 45.8% of the variance 7. In this analysis two factors of the three factors extracted include the same SDI-21 items as in the earlier analysis. Differences are found in the items included in the third factor where, in addition to three items in common, two additional items (body image, support available for those close to you) are included. The factor analysis of the 16 SD-16 items provides a framework on which to base the subscales resulting in good reliability and convincing face validity. Item 5 ‘Support available for those close to you’ loaded weakly onto all three factors. There may be an argument for discarding this item from any subscale as loading less than 0.3 may not be considered as meeting minimal criteria for inclusion; however, the Rasch analysis demonstrated clearly that this item ‘belongs’ with the other 15. This item reflects different interpretations of support including practical (Everyday living), financial (Money matters) and emotional (Self and others). The decision to include the item in the ‘Self and others’ subscale was taken to increase the reliability of this subscale. Item10 ‘Planning the future’ loads on to two subscales almost equally but has been included in the ‘Money matters’ subscale where loading is greatest. Discussion with patients should not exclude the possibility that ‘Planning the future’ may relate to issues concerning relationships and communication. Training for staff using the SDI-21 as part of the Holistic Common Assessment would include emphasis on the different interpretations of the items loading on to more than one subscale and encouragement of staff to explore with the patient the type of support or planning difficulties they are reporting.
There has been considerable debate over the use of PROs for use in everyday practice with Garssen and de Kok 17 suggesting that there should be less focus on psychometrics, more on clinical utility with the main benefit of screening to enhance communication and provide systematic care. A recent review identified that screening using PROs often leads to an increased detection rate of problems by clinicians but is rarely followed up by interventions or associated with an improvement of patient symptoms 18. This may be due to the fit of the intervention in clinical practice as well as the attitude of clinicians to the use of quality of life measures for decision making 1.
Advantages of using questionnaires to enhance communication have been demonstrated in two randomised studies with nurses 19 and doctors 20. Recklitis 5 champions the need for good psychometrics underpinning screening to avoid missing those patients whose condition is occult and not coming to clinical attention. Implementation of PROs into routine practice requires psychometrically sound and clinically useful assessment tools, trained staff with good communication skills and guidance for patient management. For assessment of the social domain in cancer care, the SDI-21 fulfils the psychometric and clinical utility requirements. The patients who took part in these studies represented a wide age range, a number of cancer sites and patients at different stages of disease. Older patients and less affluent patients may be under-represented, as is found in many research studies. However, in the original Rasch analysis no differential item functioning was observed across age groups, gender, stages of disease, disease sites and levels of deprivation for the SD-16, indicating the SD-16 works equally well for all groups 8. Therefore, although there is no certainty that guidance derived from this sample will apply to the whole cancer patient population, we believe that we have the basis for reasonably valid guidance. A recently completed study will provide information on a hierarchy of interventions available to staff for support of patients experiencing social difficulties from simple monitoring to referral for specialist help. Training for staff on score interpretation and guidance pathways for intervention will be developed from this program of work resulting in a guidance manual for general distribution. Evaluation of the SDI-21 for use in routine practice will start when the Holistic Common Assessment is introduced later in the year.
The three SDI-21 subscales with guidance on score interpretation and used in conjunction with the SD-16 cut-off and change score guidance provides an easy method for assessing social issues in oncology care. The development of three subscales and clinically significant difference scores for the SD-16, combined with the previously developed cut-off points, improves the clinical utility of the SDI-21 when assessing social issues in oncology care. This should allow health care professionals to focus on domains of concern using the scores, guidance provided and good clinical judgement.
The authors thank the patients who participated in the Social Difficulties Inventory project and research staff from the Psychosocial Oncology and Clinical Practice Research Group. The study was funded by Cancer Research UK grant number C7775/A7424.