Screening and referral for psychosocial distress in oncologic practice

Use of the Distress Thermometer




The objectives of this study were to validate the Distress Thermometer (DT) in the Netherlands and to examine its correspondence with a 46-item Problem List, possible risk factors, and the wish for a referral.


A cross-sectional group of 277 cancer patients who were treated at 9 hospitals filled in the DT and the Hospital Anxiety and Depression Scale and rated the presence and severity of problems (response rate, 49%).


Receiver operating characteristic analyses identified an ideal cutoff score of 5 on the DT with a positive predictive value of 39% and a negative predictive value of 95%. The Problem List appeared to be a reliable measure. Five items on the Problem List correlated strongly with the DT, 13 items had a moderately strong correlation, 26 items were correlated weakly, and 2 items were not correlated significantly. Emotional control, nervousness, pain, and physical fitness appeared to contribute independently to the DT score. The percentage of patients scoring ≥5 (n = 118 patients; 43%) who wanted (14%) or maybe wanted (29%) a referral was significantly higher than the percentage of patients with DT scores <5 (5% and 13%, respectively) who wanted or maybe wanted a referral. Intensively treated patients reported more distress than those who only underwent surgery. No other clear risk factors for distress were identified.


The DT appeared to be a good instrument for routine screening and ruling out elevated distress. Emotional and physical problems contributed mainly to distress. Experiencing clinically elevated distress did not necessarily suggest that patients wanted a referral. Screening for distress and the wish for a referral can facilitate providing support for those patients who most need and want it. Cancer 2008. © 2008 American Cancer Society.

Diagnosis and treatment are highly stressful for cancer patients. Everyday life is disrupted for the vast majority of patients and their relatives, and the cancer experience often results in physical, psychologic, social, practical, and spiritual concerns.1, 2 The overall burden of cancer diagnosis and treatment often is referred to as distress.3, 4 Initial distress decreases when treatment is successful. However, an estimated 25% to 40% of cancer survivors continues to suffer from distress at a level that requires professional treatment additional to the care from the primary oncology team.1, 4, 5 Interventions could be offered by social workers, psychologists, psychiatrists, pastoral counselors, or physical therapists, depending on the area of life most disrupted.

The number of individuals who will develop cancer will grow because of aging. In addition, advances in the treatment of cancer will result in a growing group of survivors and patients living with their illness for extended periods of time. To address this future demand on healthcare, a National Cancer Control Program was developed and presented in the Netherlands in 2004.3 This program encompasses all aspects of cancer control and states objectives to be reached by 2010. Two objectives are 1) the availability and introduction of a validated screening instrument capable of indicating the need for specialized psychosocial assistance and 2) the inclusion of screening for psychosocial problems in the national guidelines for oncologic care. These 2 objectives are in line with the American National Comprehensive Cancer Network (NCCN) practice guideline of distress management.6 This guideline introduced the Distress Thermometer (DT), an easily understood, self-report measure of distress.7 Patients are asked to rate their overall distress on a visual analogue scale (a thermometer) from 0 (no distress) to 10 (extreme distress).

Several studies have indicated that the DT is highly valid and usable in various groups of cancer patients.7-14 Some studies indicated that a cutoff point of 4 yielded the best sensitivity and specificity,8, 11-13 whereas others indicated a cutoff of 5.7, 14 However, validity of the DT was not proven in other studies.15, 16

The DT is accompanied by a Problem List, which includes 35 items that address 5 life domains (practical, family/social, emotional, spiritual, and physical problems; version 1.2005). The domains in the Problem List were not designed to function as a scale; however, the 5 domains may represent a multidimensional scale of overall distress. A validation study on the DT in a fairly small sample of cancer patients (n = 68) indicated that the Problem List could function as a scale with good internal consistency (α = .81) and demonstrated that the DT score was correlated significantly with the physical, family/social, and spiritual domains (correlation coefficient [r] = 0.26-0.34), and was correlated strongly with the emotional domain (r = 0.66).10

Screening through the use of an instrument such as the DT can result in the identification and consequent treatment of patients in need of professional psychosocial care who may not otherwise have had access to it.17 Distress often goes unrecognized in the clinical setting, and only approximately 10% of patients eventually receive psychosocial support.18-23 It is possible that that 10% asks for help, whereas other distressed patients may not be capable of expressing their needs. It has been demonstrated that structuralized screening for distress results in a shift in the type of patients who receive treatment from socially competent and articulate patients toward less verbal patients who have lower social skills.4 However, a recent meta-analysis of screening instruments for mood disorders (including 19 studies on the DT) demonstrated that such instruments have high false-positive rates.24 The DT appeared to be better at ruling out clinical distress (84% negative predictive value [NPV]) than diagnosing patients with clinical distress (55.6% positive predictive value [PPV]). This finding implies that short screening methods are a first step toward identification of the truly distressed and are more fitted to ruling out those patients who do not need additional care. Immediate referral may not always be necessary when a patient scores high. High levels of distress are expected and normal during certain periods of illness and treatment, for example, immediately after diagnosis or after finding out about recurrence.25 In addition, patients may not want to be referred when they are capable of dealing with their distress themselves or with the help of informal support. Therefore, the Problem List should include a question that asks patients and survivors whether they wish to be referred to gain insight into correlations between the need for intervention and being identified as distressed.25 The current study had several objectives: 1) to validate the DT in the Netherlands and to examine its predictive value, 2) to examine the reliability of the Problem List and its separate domains, 3) to examine correlations between the DT and the Problem List, 4) to examine the wish for a referral, and 5) to examine sociodemographic and treatment-related variables related to distress and the wish for a referral.



This study was performed in the region of the Comprehensive Cancer Center North Netherlands (CCCN), which is a network organization that works together with professionals in hospitals and healthcare and palliative care institutions in its region to improve oncologic and palliative patient care. Nine hospitals participated in implementing psychosocial screening in standard care. Cancer patients visiting the departments of medical, surgical, or radiation oncology in these hospitals during a 4-week period were asked to complete a questionnaire. Patients were visiting the hospital at the beginning of their treatment, for a regular appointment during their treatment, or for follow-up. This resulted in a cross-sectional group of patients that varied in diagnosis, stage, and phase of disease (Table 1). To be eligible, patients had to be aged >18 years, had to be aware of their diagnosis and treatment plan, had to have sufficient command of the Dutch language, and had to be physically fit enough to fill in the questionnaire.

Table 1. Descriptives
CharacteristicMean±SD [Range]No. of Patients (%)
  1. SD indicates standard deviation; RT, radiotherapy, CT, chemotherapy.

Age, y60.5±10.9 [24-84] 
 Men 98 (35)
 Women 179 (65)
Marital status  
 Married/living together 234 (84)
 Single/widowed 43 (16)
 Yes 240 (87)
 No 37 (13)
 Children living at home  
  Yes 63 (23)
  No 177 (64)
 Data missing 34 (13)
Educational level, y3.6±1.9 [1-7] 
Daily activities  
 Employed for wages 87 (31)
 Not employed for wages 176 (63)
 Data missing 14 (6)
 Breast 111 (40)
 Colon 35 (13)
 Head/neck 26 (9)
 Lung 12 (4)
 Gynecologic 11 (4)
 Skin 11 (4)
 Other 58 (21)
 Missing 13 (4)
Time since diagnosis, y1.8±2.4 [0.06-7] 
Treatment status  
 Under Active treatment 82 (29)
 Follow-up 195 (71)
Treatment received  
 Surgery only 61 (22)
 Surgery and RT 44 (16)
 Surgery and CT 48 (17)
 Surgery, RT, and CT 53 (19)
 RT only 10 (4)
 CT only 29 (10)
 Data missing 32 (12)


This study was performed in line with the ethical guidelines of the participating hospitals. A predetermined number of 600 questionnaires were handed out by oncologists in their office in outpatient clinics and on nursing wards. Depending on the size of the hospital, 50 or 100 questionnaires were handed out. Oncologists checked whether patients fit the inclusion criteria and explained the goal of the study. Patients received a package that included the questionnaire, an informed consent form, an information leaflet, and a prepaid return envelope. Questionnaires were sent back to the CCCN for analysis.


Demographic characteristics (sex, age, education level, marital status, presence of children), employment status (‘working for wages’ [full-time job, part-time job, self-employed] or ‘not working for wages’ [housewife, student, unable to work, retired]), and illness-related characteristics (type of cancer, date of diagnosis, date treatment was started, treatment received, and current treatment status [‘under active treatment’, or ‘in follow-up’]) were obtained through the use of a self-report questionnaire. The DT is a single-item, self-report measure of distress with an 11-point range from 0 (no distress) to 10 (extreme distress). Patients were instructed to circle the number that best described the overall distress they experienced in the past week (including today).26 The Problem List, as specified in the 2005 NCCN guidelines, was presented to the following CCCN focus groups: medical specialists, oncology nurses, social workers, psychologists, pastoral counselors, physical therapists, dieticians, and representatives from patient associations. The focus groups advised on adaptation of the 35 items in the original version. This resulted in an adapted version of 46 items in 5 domains. Patients could indicate whether or not (yes or no) in the past week they had experienced practical problems (7 items: child care, housing, housekeeping, finances, insurance, transportation, work/school/study), family/social problems (3 items: dealing with children, dealing with partner, and dealing with family and friends), emotional problems (9 items: depression, fears, nervousness, emotional control, intrusions, self-esteem, loneliness, concentration, and guilt), religious/spiritual problems (2 items: meaning of life and trust in God/faith), or physical problems (25 items: appearance, changes in urination, constipation, diarrhea, eating, feeling bloated, fevers, mouth sores, nausea, nose dry/congested, pain, sexuality, skin dry/itchy, sleep, breathing, dizziness, speech, taste, weight change, tingling in hands/feet, bathing/dressing, getting around, fatigue, physical fitness, and muscle strength). The items above that are set in italics were added to the Problem List; the items finances and insurance were listed as 2 separate items; and the items sadness, worry, and loss of interest in usual activities were removed from the NCCN Problem List. To calculate correlations with the DT score, patients were asked to rate (from 0 to 10, with 10 the highest rating) the amount of burden they felt for each item on the Problem List that they indicated experiencing. In addition, patients could indicate (yes, maybe, or no) that they wanted to be referred (Would you like to talk with a professional about your problems?).

The Hospital Anxiety and Depression Scale (HADS) is a widely used and accepted measurement of anxiety and depression in the medical setting that also has been used with cancer patients.27, 28 The HADS is a self-rated, 14-item questionnaire with 7-item subscales for anxiety and depression. Respondents are asked to report symptoms over the previous week, which results in maximum scores of 21 on each subscale. The 2 subscales can be combined into a single scale, with scores ≥15 indicating clinically significant emotional distress.27, 29 The HADS has been validated in several subgroups in the Netherlands.30


Receiver operating characteristics (ROC) analysis was performed to examine the ability of the DT to detect distressed patients,31 and a HADS cutoff score ≥15 was set as the gold standard. The PPV and NPV were calculated for every DT score. Five domain summary scores were calculated by taking the total score of the items in each domain on the Problem List. Mean and standard deviation (SD) scores were calculated for the DT and domain scores and for the HADS total score and subscales. The Cronbach α was calculated for scores on the total Problem List and for the 5 domain scores to examine reliability. An α ≥ .70 was regarded as satisfactory, and α ≥ .80 was regarded as good.32 Pearson product-moment correlations were calculated to examine relations between the DT and items on the Problem List, the domain summary scores, and the HADS total score. A Pearson correlation coefficient <0.30 indicated a weak relation, a coefficient from 0.30 to 0.50 indicated a moderately strong correlation, and a coefficient >0.50 indicated a strong correlation.33 All items on the Problem List that had a correlation ≥0.30 with the DT were entered into a regression analysis. The type of treatment was dichotomized with the categories ‘nonintensive’ (surgery) and ‘intensive’ (radiotherapy, chemotherapy, or any combination of surgery, chemotherapy, or radiotherapy). Independent-sample t tests and chi-square tests (for sex, marital status, presence of children, daily activities, treatment status, and type of treatment) and correlational analyses (for age, education level, and time since diagnosis) were performed to examine correlations between sociodemographic and treatment-related variables and the DT and the wish for a referral. To investigate possible factors that contributed to the wish for a referral, we dichotomized responses into 2 groups: patients who did wish or maybe wished to be referred and patients who did not wish to be referred.


Patient Characteristics

Of the 600 questionnaires that were handed out, 296 were returned (response rate, 49%). Of these, 19 questionnaires had necessary data missing (either the DT or the HADS were partially completed). Therefore, data from 277 respondents were analyzed. Patient characteristics are summarized in Table 1. Nearly 66% of respondents were women, and the average patient age was 60 years. The majority of respondents had a partner; the majority of respondents had children, and 75% of children were not living with their parents. Patients were diagnosed an average of 1.8 years before study participation (range, from 3 weeks to 17 years). Forty percent of respondents had breast cancer; other respondents had colon cancer (13%), head and neck cancer (9%), lung cancer (4%), gynecologic cancer (4%), and skin cancer (4%). A majority of respondents (71%) had completed treatment. Sixty-six percent of respondents had received intensive treatment for their cancer.

Establishing Cutoff Points, Predictive Values, and Internal Consistency

The mean scores for the study measures are summarized in Table 2. The mean DT score (±SD) was 3.8 ± 2.6, and the median score was 4. The mean HADS score (±SD) was 8.9 ± 6.9. Twenty percent of respondents (n = 55) scored above the cutoff point for the HADS.

Table 2. Descriptives of Study Measures and Reliability of Problem List Domains
VariableMean Score±SDαScore Range
  • SD indicates standard deviation; DT, Distress Thermometer; HADS, Hospital Anxiety and Depression Scale.

  • *

    The value of α for all 46 items on the Problem List.

DT overall distress3.8±2.6 0-100-10
Problem List    
 Total Problem List score40.3±39.6.90*0-4600-213
 Practical problems3.4±6.2.600-700-45
 Social problems1.3±3.8.750-300-30
 Emotional problems11.2±15.4.890-900-68
 Spiritual problems1.4±3.5.640-200-18
 Physical problems28.3±27.2.740-2500-143
 Total HADS score8.9±6.9   
 No. scoring ≥15 (%)55 (20)   

The ROC curve predicting clinically elevated distress according to the HADS showed an area under the curve of 0.80 (standard error, 0.03; 95% confidence interval, 0.74-0.87; P < .001) (Fig. 1). Table 3 lists sensitivity and specificity values for all DT scores, including frequencies. Respondents reported DT scores across the entire range from 0 to 10. A cutoff score of 5 correctly identified 85% of HADS cases (sensitivity) and 67% of HADS noncases (specificity). Lowering the cutoff score would increase the number of HADS noncases being identified as anxious or distressed. Increasing the cutoff score to 6 would lower true-positive rates. By using the cutoff score of 5, 118 respondents (42.6%) reported clinically elevated distress. A DT score of 5 provided a PPV of 39% and an NPV of 95%.

Figure 1.

Receiver operating characteristic (ROC) curve of Distress Thermometer scores versus Hospital Anxiety and Depression Scale cutoff scores.

Table 3. Frequencies, Sensitivity, Specificity, Positive Predictive Values, and Negative Predictive Values of Distress Thermometer Scores
DT ScoreNo. (Cumulative %)SensitivitySpecificityPPVNPV
  1. DT indicates Distress Thermometer; PPV, positive predictive value; NPV, negative predictive value.

038 (14)
131 (25)0.980.160.230.97
222 (33)0.960.290.260.96
337 (46)0.920.380.280.95
429 (57)0.910.550.340.96
546 (73)0.850.670.390.95
632 (85)0.570.810.470.86
721 (92)0.390.920.500.86
812 (97)0.250.990.800.84
95 (99)0.070.990.700.81
104 (100)

The total Problem List appeared to have good internal consistency and reliability (α = .90). The domains for family/social problems (α = .75), emotional problems (α = .80), and physical problems (α = .74) also had good reliability. The domains for practical problems (α = .60) and spiritual problems (α = .64) were less reliable (Table 2).

Correlates of Distress

The DT was related strongly to the total Problem List (r = 0.68) and the emotional (r = 0.61) and physical (r = 0.64) domains (all P < .001). The DT was related moderately strongly to the practical (r = .39) and family/social (r = 0.31) domains and was related significantly weakly to the spiritual domain (r = 0.26; all P < .001). The DT also was related strongly to the HADS total score (r = 0.57; P < .001).

Housekeeping (r = 0.40), emotional control (r = 0.50), intrusions (r = 0.31), self-esteem (r = 0.44), fears (r = 0.48), depression (r = .49), nervousness (r = 0.58), loneliness (r = 0.38), concentration (r = 0.39), loss of control (r = 0.35), eating (r = 0.36), pain (r = 0.43), sleep (r = 0.34), tingling in hands/feet (r = 0.30), getting around (r = 0.50), fatigue (r = 0.54), physical fitness (r = 0.52), and muscle strength (r = 0.41) were correlated significantly (all P < .001) and (moderately) strongly with the DT. Twenty-six items were correlated significantly but weakly with the DT (r range, 0.18-0.29; P < .05). Only childcare (r = −0.11) and insurance (r = 0.10) were not related significantly to the DT. A linear regression analysis entering the 18 items that correlated ≥0.30 indicated that these items explained 62% of the variance (R2) in the DT score (F = 10.1; P < .001). Emotional control (β = .14; P < .05), nervousness (βa = .30; P < .001), pain (β = .21; P < .001), and physical fitness (β = .20; P < .01) contributed independently to the DT score.

Wish for Referral

Among all respondents, 25 (10%) wanted to be referred, 54 (19%) maybe wanted a referral, and 197 (71%) had no wish to be referred. Of the patients who scored ≥5 on the DT, 14% wanted to be referred, 29% maybe wanted a referral, and 57% did not want a referral. Of the patients who did not have clinically elevated distress, 5% wanted a referral, 13% maybe wanted to be referred, and 82% had no wish to be referred (chi-square = 21.4; P < .001).

Predictors of Distress and the Wish for Referral

Treatment received was the only factor identified that was related to the DT. Patients who received nonintensive treatment reported less distress (mean score [±SD], 2.8 ± 2.9) than patients who received intensive treatment (4.1 ± 2.3; t = −3.3; P < .001). Because of the skewed distribution of the DT score, we also analyzed relations with the DT cutoff score. Patients who scored above the cutoff received intensive treatment more often (chi-square = 8.5; P < .01) and had a lower education level (t = 2.4; P < .05).

Patients who wanted or maybe wanted a referral were somewhat younger (mean score [±SD], 57 ± 11.2) than patients who did not wish to be referred (61 ± 10.7; t = 3.1; P < .01). In addition, patients in the first group underwent intensive treatment more often than patients in the second group (chi-square = 4.9; P < .05). No other variables were associated significantly with the wish for a referral.


In the current study, we examined the validity of the 1-item DT in detecting distress in cancer patients in the Netherlands. A cutoff score of 5 resulted in optimal sensitivity and specificity relative to the HADS. Eighty-five percent of patients were identified correctly as distressed, and 67% were identified correctly as not distressed. The sensitivity of the DT in the current study was higher than was demonstrated in a recent meta-analysis (78.3%), whereas its specificity was comparable (66.5%).24 In addition, the NPV in the current study (95%) was higher than that reported in the meta-analysis (84%), confirming that the DT is an excellent tool for ruling out clinically elevated distress and is less of a diagnostic tool.

A useful screening tool must provide information not only on the level of distress experienced but also on the type of problems associated with that distress.34 Therefore, the DT is accompanied by a Problem List. Twelve items were added to the original NCCN list according to advice from focus groups. The total Problem List and the domains for family/social, emotional, and physical problems showed very good consistency. However, the domains practical and spiritual problems were somewhat less reliable. It is noteworthy that the α values observed in the current study all exceeded those reported in an earlier study, except for emotional problems, for which the values were comparable.10 It is possible that the adaptations made according to advice from focus groups resulted in improvement of the Problem List for cancer patients.

It appeared that the emotional and physical problem domains were related strongly to the DT score, the practical and social problems domains were related moderately strongly to the DT score, and the spiritual domain was related weakly to the DT score. The domains social and spiritual problems contain fewer items than the other domains, which may explain the weaker correlations. It must be noted that correlations between the DT and the practical and physical domains in the current study were stronger and more statistically significant than the correlations reported in an earlier study.10 The strengths of the other 3 correlations were comparable to those reported previously.

The importance of emotional and physical problems in explaining the DT score also appeared when the correlations between all 46 separate problems and the DT score were examined. Eighteen problems were strongly related to the DT score, of which all except 1 (housekeeping) were items from the emotional or physical domains. Of these 18 problems, emotional control, nervousness, pain, and physical fitness contributed uniquely to overall distress. These findings regarding which problems are related most to distress apply on a group level. For every individual patient, the reasons for their distress can be entirely different. The answers to the Problem List, therefore, should be a guide to further explore what is causing the patient's distress and what type of support is needed.

The types of problems that cause the most distress can indicate which kind of professional referral the patient should receive or which type of intervention may be most beneficial. Patients who experience problems in the emotional domain (such as little emotional control and nervousness) may benefit from psychosocial interventions, such as psychotherapy, social/group support, education, and promoting effective coping.35 The pastoral worker is the best caregiver for existential problems, and practical problems most likely are handled best by a social worker. A major advantage of the DT and the Problem List is that they also encompass physical problems, although they often are presented as psychosocial screening tools. Most physical problems do not require professional psychosocial support. Patients who have physical problems may benefit from extra attention from a medical specialist or oncology nurse who can provide more information related to the type of cancer and treatment that may cause such problems and can treat the patient for those issues. Patients with decreased physical fitness, decreased muscle strength, and increased fatigue may benefit from referral to a physical therapist or a rehabilitation program.36 A dietician can address problems such as weight change and diet.

Identifying individuals who may benefit from professional help does not suggest their wish for a referral to such support, which also was observed in the current study. We observed that only 10% of patients wished to be referred to a professional caregiver. This was lower than the rate reported in a study that screened patients with lung cancer using the DT, in which 22% indicated that they wanted help with their problems,37 and it was much lower than the rate reported in a group of Korean patients with cancer, in which 61% reported a need for psychosocial support.38 The percentage of patients with clinically elevated scores on the DT that (maybe) wanted a referral was significantly higher than that in patients with low distress (range, 0%-4%), but only a small percentage of patients with low distress (5%) indicated that they wished to be referred. This suggests that the DT score should not be the only indication to refer a patient to a psychosocial professional; patients also should be asked whether they want to receive additional care.

According to the current results, 19% of cancer patients said that they ‘maybe’ wanted a referral. It may not have been the right time for those patients to seek further support; they may have been too focused on getting better. A patient also may receive enough social support from family and friends to deal with their situation. This emphasizes the importance of repeated screening from the start of treatment to the end of follow-up. Patients' needs may change over time. It also is possible that patients who answer ‘maybe’ when they are asked about their wish for a referral are embarrassed or ashamed to visit a social worker or psychologist. This illustrates the importance of discussing patients' answers on the DT and Problem List. A physician or nurse can discuss possible barriers that a patient may experience to referral and to entering an intervention.

Previous research has demonstrated possible risk factors for clinically elevated cancer-related distress. Young age, being a woman, and lower education levels have been related to distress after a cancer diagnosis.25 Some studies have demonstrated no relation between high distress and demographic factors or illness-related characteristics, such as stage of disease, type of current treatment, and time since diagnosis.39 In the current study, we observed that patients who only underwent surgery reported less distress than patients who received more intensive treatment. In addition, more patients who received intensive treatment wanted to be referred for their problems. A possible explanation for this difference may be that patients who received chemotherapy or radiotherapy or received multiple treatment regimens were diagnosed with more advanced disease. This knowledge may keep distress elevated, even during follow-up. Furthermore, receiving chemotherapy or radiotherapy or receiving several types of treatments most likely is more invasive and disruptive to everyday life than only undergoing surgery, which may explain the elevated distress.

Here, we want to focus briefly on the following point, which may be relevant when considering cutoff scores in clinical decision making.40 Currently, the base rate of distress in a cancer population remains unclear. It may vary, depending on the study population, and it may be affected by whether the patient received a palliative or curative treatment, the type of cancer (prostate7), the type of treatment (bone marrow transplantation13), or cultural background (Japanese, Turkish,8, 12 or American7, 14). To our knowledge, determining whether a ‘standard’ cutoff score on the DT can be applied to the general population of cancer patients has not been explored to date and should be the focus of future studies.

The current study had some limitations that should be noted. First, a cross-sectional group of patients was included, which prevents insight into the course of distress over time. Previous longitudinal studies on the effect of screening (however, without use of the DT as an instrument for screening) indicated that structural screening can increase correct referrals and can decrease or even forestall distress.23 Second, an instrument that assesses emotional problems was used for validation of the multidimensional concept of distress (also including practical, physical, social, and spiritual aspects), a disadvantage that has been noted previously.9 Therefore, future studies on the DT and Problem List may focus on correlation with a multidimensional quality-of-life instrument to compensate for this shortcoming. A third limitation is the 49% response rate, especially because it was not possible to examine differences between participants and nonrespondents.

Although the current study had some shortcomings, it also had some clear advantages. We included a relatively large sample of cancer patients with various diagnoses, who were diagnosed between 3 weeks and 17 years before study inclusion, who were in various phases of treatment or follow-up, and who were recruited from several hospitals. In addition, we examined the relation between the DT and the Problem List in more detail and focused on the wish to be referred. Finally, according to the HADS cutoff score, only 20% of patients in the current study reported high distress: a lower percentage than that according to the DT cutoff score. The higher percentage established by using the DT may reflect the prevalence of problems reported by patients in quality-of-life domains other than the emotional domain.

The next most important goal is that of implementing and using the DT in daily clinical practice.23, 39 A recent study revealed that the DT has not yet been implemented as standard or according to guidelines in most institutions associated with the NCCN.41 The 9 hospitals that participated in our study are in the process of implementing psychosocial screening using the DT in daily practice. It is a challenge to find solutions for several difficult issues, such as the logistics of handing out the DT at several measurement points, how to recognize oncology patients in a mixed outpatient clinic, how to maintain continuity when patients transfer to other medical disciplines or out of the hospital, and promoting use of the DT as a communication instrument by oncology specialists and nurses. Hopefully, all countries that focus on implementing psychosocial screening with use of the DT will share their experiences. Implementation should be accompanied by research to determine whether such screening results in the referral of those cancer patients who need it and whether the process of screening improves quality of life and satisfaction with care.

In summary, the results from this study demonstrated the validity of the DT in the Netherlands. The adapted Problem List was useful for detecting the types of problems most relevant for patients and demonstrated good reliability. According to the cutoff score of 5, 42% of patients could benefit from additional professional (psychosocial) care. However, the NPV for the DT indicated that it is especially useful in ruling out distress (patients with scores from 0 to 4 were classified as not distressed with 95% accuracy). Emotional and physical problems were associated more strongly with overall distress than practical, family, and spiritual problems. Patients who received chemotherapy or radiotherapy or a combination of treatment modalities were more distressed than those who only underwent surgery. No other risk factors for distress were identified. Not all distressed patients wanted a referral; in total, 10% of patients wanted to be referred, and 19% said that they maybe wanted a referral. More patients with elevated distress wanted a referral than patients with low distress. Screening for distress and the wish for a referral can facilitate providing support for those patients who most need and want it.