The Italian Society of Psycho-Oncology (SIPO) Distress Thermometer Study Group is indebted to all the physicians, nurses, and patients who participated in the study.
Coauthors of this article, as part of the Italian Society of Psycho-Oncology Distress Thermometer Study Group, are: Federica Andreis (Brescia), Valentina Belbusti (Fano), Giorgia Bellini (Faenza), Alejandra Berardi (Forlì), Simona Brovelli (Novara), Rossana De Feudis (Bari), Monia Duca (Fabriano), Leonardo Fei (Firenze), Giovanna Ferrandes (Genova), Angelo Frigerio (Bergamo), Colette Galotti (Varese), Sabina Godi (Pesaro), Monica Guaraldi (Bologna), Giovanna Mantegna (Campobasso), Sabrina Marini (Fermo), Elena Meggiolaro (Forlì), Cinzia Molieri (Bergamo), Rosanna Montanaro (Bari), Maura Muccini (Riccione, Cattolica), Maria Luisa Nadalini (Verona), Luca Ostacoli (Torino), Sara Padovani (Ferrara), Alessandra Palma (Palermo), Daniela Palmisano (Torino), Valentina Panella (Prato), Laura Pedrinelli Carrara (Senigallia), Angela Piattelli (Cosenza), Patrizia Pugliese (Roma). Marco Romeo (Urbino), Francesca Romito (Bari), Giulia Rossetti (Ferrara), Sara Serrao (Ascoli Piceno), Ilaria Strada (Lugo), Anna Verrengia (Avellino), and Cristina Wuhrer (Brescia).
Routine screening for distress is internationally recommended as a necessary standard for good cancer care, given its high prevalence and negative consequences on quality of life. The objective of the current study was to contribute to the Italian validation of the Distress Thermometer (DT) to determine whether the single item DT compared favorably with referent criterion measures.
In total, 1108 outpatients with cancer were recruited from 38 representative oncology centers in Italy. Each participant completed the DT and a list of 34 possible cancer-related problems (the Problem List), the Hospital Anxiety and Depression Scale (HADS), the 18-item Brief Symptom Inventory (BSI-18), and a short visual analog scale to determine the understandability of the tools.
Receiver operating characteristic analysis revealed that DT cutoff scores ≥4 and ≥5 had optimal sensitivity and specificity relative to both HADS and BSI-18 cutoff scores for general caseness and more severe psychological distress, respectively. Patients with DT scores ≥4 (cases) were more likely to be women; to have had psychological problems in the past; to report more stressful events in the last year; and to currently have more family, emotional, and physical problems related to cancer or cancer treatment. Patients indicated that the DT was easier to fill out and to understand than the HADS, but not the BSI-18.
Several studies have demonstrated that 30% to 40% of patients with cancer report emotional distress symptoms as a consequence of the disease and treatment, many of whom meet the criteria for psychiatric diagnosis, such as adjustment, anxiety, and depressive disorders.1 Although this impairs the quality of life of patients and their family, leading to longer rehabilitation, <33% of patients with cancer who are diagnosed with distress are recognized in the oncology setting and, subsequently, are referred for proper clinical intervention.2
For these reasons, several guidelines for psychosocial screening have been developed, and distress has been endorsed as the sixth vital sign to be monitored constantly and routinely across the cancer disease trajectory to identify patients in need of clinical attention and intervention.3-5 In fact, it has been demonstrated that clinical judgment alone does not work as a screening tool. In a study of 401 patients with cancer by Mitchell et al,6 45% had distress according to the Distress Thermometer (DT), but nurses had difficulty identifying distress using their routine clinical judgment and made false-negative errors (sensitivity, 51%) and, to a lesser degree, false-positive errors (specificity, 80%). Even worse data were recently reported from a large sample of 2642 cancer outpatients in follow-up care by Werner et al,7 who observed a high prevalence of distress among patients but very low identification by physicians of those presenting clinically significant distress (1 of 10 patients; patient-physician concordance = 0.1).
In this background, the DT and the Problem List (PL) have been devised by the Distress Management Guidelines Panel within the National Comprehensive Cancer Network in the United States.8, 9 Over the last 10 years, the DT has become 1 of the most used tools and has been applied in several contexts (eg outpatient clinics, inpatient units, palliative care) and in different phases of cancer trajectories, and data have indicated its validity as a screening instrument. In a multicenter study of 380 cancer outpatients in the United States, Jacobsen et al10 demonstrated that, compared with psychological questionnaires used as “gold-standard” reference instruments (ie, the Hospital Anxiety and Depression Scale [HADS]11 and the Brief Symptom Inventory-18 [BSI-18]12), a DT cutoff score ≥4 optimized sensitivity and specificity for the detection of patients with emotional distress (caseness). More recently, the DT has been applied and validity confirmed in many countries by studies carried out in Japan,13 the Netherlands,14 France,15 Turkey,16 Korea,17 Taiwan,18 Australia,19 Denmark,20 Iceland,21 Israel,22 and Ireland23 (Table 1). Because the DT and PL have never been submitted to a nationwide validation study in Italy, the objective of the current study was to examine the validity and acceptance of the DT in a large sample of Italian patients with cancer.
Table 1. Summary of Cross-Cultural Studies Examining the Validity of the Distress Thermometer
Cutoff DT Score
Abbreviations: AD, adjustment disorders; AUC, area under the receiver operating characteristic curve; BSI-18, 18-item Brief Symptom Inventory questionnaire; DSM-IV, Diagnostic and Statistical Manual for Mental Disorders, 4th edition; DT, Distress Thermometer; GHQ-30, 30-item General Health Questionnaire; HADS, Hospital Anxiety and Depression Scale; MD, major depression; PHQ-9, Patient Health Questionnaire 9-item depression module; SCAN ICD-10, Schedules for Clinical Assessment in Neuropsychiatry with the International Classification of Diseases and Related Health Problems, 10th revision; WMH CIDI, World Mental Health Composite with the International Diagnostic Interview for the International Classification of Diseases and Related Problems, 10th Revision.
These were patients who were referred to consultation psychiatry.
The study involved 38 cancer centers in Italy that were representative of the different regions of the country (north, center, and south). The study was conducted in a 2-day period during an index week at all centers (November 20-27, 2010). Criteria for recruitment were between ages 18 and 75 years, a primary diagnosis of cancer, a Karnofsky performance status ≥80, a schedule for an outpatient appointment, and the ability to provide informed consent. The study was first approved by the ethical committee of the coordinating center (University of Ferrara) and then by the ethical committees of the other participating centers.
A short, individual, semistructured clinical interview was conducted by a research psychologist who had clinical experience to obtain information about the presence of life-time psychological disorders, the occurrence of stressful events within the last year (with the exclusion of events related to cancer), and current use of psychotropic drugs. All information was scored in a yes/no format. In addition, information on patients from the medical record was included in the analysis. Each patient was asked to complete a booklet containing the following psychological instruments: the DT, the PL, the HADS, and the BSI-18.
The Distress Thermometer
The DT8, 9 is a visual analog tool that asks the respondent to rate his/her level of distress in the past week on a scale from 0 (no distress) to 10 (extreme distress). The PL consists of a list of 34 problems grouped into 5 categories (practical problems, family problems, emotional problems, spiritual/religious concerns, and physical problems) and is rated in a yes/no format.
The Hospital Anxiety and Depression Scale
The HADS11 is a 14-item, self-report measure of psychological distress divided into 2 subscales: anxiety (7 items) and depression (7 items). For each item, respondents are asked to indicate which of 4 options (rated from 3 to 0; score range, 0-42) comes closest to describing how they have been feeling in the past week. For the purposes of the current study, only the total HADS score was used, with a score ≥15 considered indicating clinically significant distress (general caseness) and a more conservative score of 19 indicating more severe psychiatric conditions (severe caseness).
The Brief Symptom Inventory-18
The BSI-1812 is an 18-item questionnaire that also examines distress in the last week through 3 subscales: somatization (6 items), anxiety (6 items), and depression (6 items). Each item is rated on a Likert scale from 0 (“not at all”) to 4 (“extremely”), and a total distress score (General Stress Index [GSI]) is obtained by summing all the items. For the purposes of the current study, only the GSI was examined; and, according to Derogatis,12 a T-score ≥63 was used as indicative of caseness.
The grade of understandability of the tools also was evaluated by asking patients to answer a single question, “Was this tool easy to understand and to answer?” on a 10-point visual analog scale from 0 (“very easy”) to 10 (“not easy at all”).
Information about clinical data (eg, disease stage, type of therapy, medical comorbidity) was obtained from patients' medical records with the help of an oncologist who knew the patient. To understand the possible role of other medical comorbidities on emotional distress, the Charlson comorbidity index (CCI)24 was used in a modified version according to suggestions by Watkins et al.25 Cancer was excluded because it was the primary diagnosis rather than a comorbid condition. Likewise, dementia was excluded because of the inclusion criteria. For each of the other conditions (rheumatologic disease, chronic pulmonary disease, congestive heart failure, peripheral vascular disease, cerebrovascular disease, diabetes, moderate/severe renal disease, moderate/severe liver disease), a score was given according to the CCI. For the current analysis, the score (ie, the sum of the weights of conditions recorded as being present) was transformed into a 4-level ordinal scale on which the categories 0, 1, 2, and 3 corresponded to index scores of 0, 1 or 2, 3 or 4, and >5, respectively.2
Descriptive analyses, Pearson r correlation tests, chi-square tests, Student t tests, and analyses of variance were used as appropriate to examine correlations and differences between groups. Receiver operating characteristic (ROC) analysis was used to determine whether scores on the DT could validly distinguish “cases” and “noncases” as measured by both the HADS and the BSI-18. To do this, the sensitivity and specificity of each score in the range of the DT was calculated and used to determine how well the score distinguished patients who surpassed the HADS and BSI-18 cutoff scores from patients who did not. The ROC curve graphically represents the sensitivity and specificity coefficients that would be generated using each possible cutoff score in the range of DT scores, and the accuracy of the cutoff score is determined by calculating the area under the ROC curve (AUC) (from 1.0 [perfect accuracy] to 0.5 [accuracy no better than chance]).
Sociodemographic and Clinical Characteristics
Of 1410 patients who were eligible for inclusion, 142 (11%) declined participation, and 160 (11%) were excluded because their psychometric data were incomplete. The final sample consisted of 1108 patients (329 men [30%] and 779, 70% women; mean age, 53.4 ± 9.3 years). Sociodemographic and clinical data from the sample are provided in Table 2.
Table 2. Sociodemographic and Clinical Data of the Study Patients: Italy, 2010
No. of Patients (%)
Abbreviations: SD, standard deviation.
Age: Mean±SD [range], y
Stressful life events in the last year
Previous psychological disorders
Current psychotropic drugs
Charlson comorbidity index
Establishment of Distress Thermometer Cutoff Score
The frequency distribution of DT scores is reported in Table 3. According to Mitchell et al,6 distress was rated as mild (DT scores of 4 and 5), moderate (DT scores of 6 and 7), and severe (DT scores ≥8). The rates for mild, moderate, and severe distress were 22% (n = 229), 18% (n = 187), and 15% (n = 158), respectively. The mean (±standard deviation) score on the DT was 4.13 ± 2.92. In our ROC analysis, a DT cutoff score ≥4 yielded a sensitivity of 0.79 with moderate specificity (0.60) when a HADS cutoff score ≥15 was used as a criterion for general caseness, with an AUC of 0.76 (95% confidence interval [CI], 0.73-0.79) (Fig. 1). The same DT cutoff score ≥4 was associated with a sensitivity of 0.80 and a specificity of 0.61 in identifying cases according to a BSI-18 cutoff T-score ≥63 (AUC, 0.76; 95% CI, 0.73-0.79) (Fig. 2). In this way, 47% of our patients fulfilled criteria for caseness on the DT. When a conservative cutoff score of ≥19 was used on the total HADS as a reference criterion, more severe cases were identified by a DT cutoff score ≥5, with a sensitivity of 0.70 and a specificity of 0.77 (AUC, 0.81; 95% CI, 0.77-0.83). In this analysis, 33% of patients had distress identified according to the DT. Correspondence of the DT with HADS and BSI-18 scores is reported in Table 4.
Table 3. Frequency of Distress Thermometer Scores in the Sample
Distress Thermometer Score
No. of Patients (%)
Table 4. Correspondence of the Scores on the Distress Thermometer With Cutoff Scores on the Hospital Anxiety and Depression Scale and the Brief Symptom Inventory-18
Correlation of Distress Thermometer Cutoff Scores With Clinical and Sociodemographic Variables
Table 5 indicates that the DT score was related to sex, age, and a diagnosis of breast cancer. Patients who reported psychological problems in the past and the occurrence of life-events in the last year were more likely to have scores ≥4 on the DT.
Table 5. Correlation of a Distress Thermometer Cutoff Score of 4 With Demographic and Clinical Variables
No. of Patients (%)
DT Score ≤4
DT Score ≥4
Abbreviations: DT, Distress Thermometer.
Free from disease
Charlson comorbidity index
Previous psychological disorders
Life events in last year
Correlation of Distress Thermometer Caseness to Problem List Items
According to the study by Jacobsen et al,10 we examined the relation of DT scores with yes/no responses to the list of problems. With regard to practical problems, the DT cutoff score was not related to any of the 6 problems listed (0%). With regard to relations, the DT cutoff score was related significantly to 2 of 4 family problems (50%) (dealing with partner, P = .001; dealing with children, P = .004). In the area of emotional problems, the DT cutoff score was related significantly to all the problems listed (depression, nervousness, sadness, worry, and loss of interest in usual activities; P = .001). With regard to spirituality, the DT cutoff score was related significantly to spiritual problems and emotional concerns. The DT cutoff score was also associated with 10 of 21 physical problems listed (47%) (problems with appearance, constipation, eating, fatigue, feeling swollen, getting around, indigestion, memory and concentration, pain, and sleep; P = .001).
Understandability and Ease of Using the Tools
All of the instruments we evaluated were quite easy to understand and fill in, and >80% of patients endorsed this view. A slight but significant preference was demonstrated for the DT (t = 2.84; P < .01).
To our knowledge, this is the first nationwide study in Italy examining the validity of the DT as a screening tool for distress in a large sample of patients with cancer. Like other national evaluations, the DT was identified as a valid tool for detecting distress in Italian cancer patients compared with standard measures like the HADS and the BSI-18. We confirmed that a cutoff score of ≥4 on the DT maximized sensitivity and specificity for general psychosocial morbidity, with an AUC on both the HADS and BSI-18 that indicated acceptable accuracy of the DT. Among our patients, 47% had possible distress on the DT, and there was a tendency to overestimate caseness as measured by the HADS and BSI-18 (33% and 38%, respectively). When a more conservative cutoff score was used for more severe caseness (ie, a HADS score ≥19), a score ≥5 on the DT maximized sensitivity and specificity with an AUC of 0.80, and the results indicated significant distress in 33% of patients.
These values are in line with most of the literature on the DT as applied in English-speaking countries.10, 13, 14, 16-19, 20, 23 They also confirm the validation studies carried out in many other countries with different cultural backgrounds, with the exception of France,15 Iceland,21 and Israel,22 where the application of the DT has indicated a lower cutoff score (≥3) for caseness compared with scores on the HADS and the BSI-18. When distress on the DT was rated as mild, moderate, or severe, the proportion of patients with severe distress was greater than what was reported in the study carried out by Mitchell et al6 in the United Kingdom. The differences in DT casenesss and cutoff scores between studies are not easy to interpret. Because many variables can influence levels of distress (eg the study setting, possible clinical factors), more studies on more homogeneous populations are necessary.
In our investigation, distress was not related to age, education, marital status, stage of cancer, or type of intervention; it is also interesting to note that distress was not related to medical comorbidity. The latter is an important finding, because no other study tried to examine the possible role of other concomitant diseases in influencing scores on the DT in patients with cancer. In agreement with other authors,26, 27 distress was greater in women than in men and was associated with previous psychological disorders and the occurrence of stressful life events, other than cancer, in the year preceding the diagnosis of cancer. Also, cancer-related problems, including relational, emotional, spiritual, and physical problems, which emerged by administering the PL, were more markedly evident among patients who had distress identified by the DT than among patients without such distress. These findings are in line with data indicating that the presence of physical symptoms can increase the risk of distress.10 Likewise, the impact of cancer on interpersonal and individual-spiritual dimensions also was examined in several other studies, which confirmed the association between spiritual distress, emotional distress, and maladjustment to cancer.28-30 Conversely, it is understandable that, as a general condition, distress may overlap with problems in the domain of psychological symptoms on the PL, such as depression, nervousness, sadness, and worry about health, as indicated by our current results.
Regarding the intelligibility and clarity of the tools, our patients reported that the instruments were understandable and easy to complete, also supporting applicability of the DT as a screening instrument for distress in the clinic. It is likely that implementation and regular use of the DT can improve the recognition of distress and the proper referral rate of cancer patients in clinical follow-up.31 With respect to this finding, an educational program on routine use of the DT reportedly improved both the identification and the referral of patients with distress (from 7% to 23%) to psycho-oncology services.32 These results confirm what has been reported from Japan,33, 34 France,35 and Australia,36 indicating an increase in referrals for distress problems after introduction of the DT in clinical practice.
The strengths of the current study are that, with respect to other research into the validity of the DT, our patient sample was larger, population-based, and nationwide, representing several areas of the country. In addition, to our knowledge, this study is the first in the DT literature to examine the possible role of medical comorbidity and the acceptability of the tools by patients with cancer.
However, there also are limitations to the current study that should be considered. The sample consisted of cancer outpatients with a good performance status, although half of them were in a metastatic disease phase. The role of the DT in Italian cancer patients with poorer performance status and in the context of palliative care should be examined in further studies. Second, although we followed the existing literature in using psychometirc questionnaires as reference criterion for caseness, more information is needed regarding the accuracy of the DT with respect to a standard psychiatric interview. However, the only available Italian study relative to using the DT versus a psychiatric diagnosis based on the World Health Organization-Composite Interview for the International Classification of Diseases, 10th edition, confirmed the data presented here.37 Third, the cancer site was represented mostly by breast and gastrointestinal cancer, and this did not allow us to understand, in a more comprehensive way, the possible differences in DT scores according to cancer sites. With respect to this, Zabora et al26 conducted a large study involving 9000 patients with cancer and reported higher distress rates among patients with lung cancer (43%) and lower rates among patients with gynecologic cancer (30%). Again, because the percentages of women and patients in a metastatic stage of illness were high in our study, this may have had an impact on our results. More studies taking into account sex, patient age, and disease stage are necessary. Fourth, the list of problems used in the study, even if the methodology of other research was followed, could be improved, as recently suggested by Brennan et al.38 Fifth, more specific data on previous and current psychiatric history (eg specific diagnosis of substance abuse disorders, bipolar disorders, schizophrenia, and type of psychotropic medications) and categories of stressful events will be important to identify the true distress level in patients with cancer. Finally, we are aware that traditional criteria for a screening tool (eg, high specificity, high sensitivity, “do no harm” to patients, ease of use, cost effectiveness) need to be fulfilled before daily application in the clinical setting. These aspects also should be applied to the DT. In addition, both referral and treatment algorithms need to be developed.
In conclusion, in this study, we confirmed that a brief screening tool like the DT is a simple and effective screening instrument for detecting distress in Italian patients with cancer. The instrument is easy to understand and thereby promises high compliance among both patients and clinicians. Furthermore, the single-item DT compares favorably with longer measures that are used to screen for distress and that, when combined with the PL, favors the identification of cancer-related problems. There is a need for studies in Italy examining in depth the outcome of identifying distress in cancer patients in terms of both referral rates and, especially, treatments.
This study was supported by the University of Ferrara and the Fondazione Cassa di Risparmio di Ferrara (Dr. Grassi) and by the Danish Cancer Society (Dr. Johansen).