Adjustment disorders and major depression are common psychiatric disorders in patients with cancer and have a serious impact on quality of life. The problem in clinical oncology settings is underrecognition of these disorders; as a result, screening is recommended to detect them. The goal of the current study was to develop a new, brief screening tool for adjustment disorders and major depression and to compare its performance with that of existing screening methods.
Patients with cancer completed the newly developed One-Question Interview (a 1-item, structured interview); the Distress Thermometer (a 1-item, self-report questionnaire), which previously was developed as a brief screening tool; and the Hospital Anxiety and Depression Scale (HADS; a 14-item, self-report questionnaire). Psychiatric diagnoses of adjustment disorders and major depression were made by psychiatrists and were based on criteria set forth by the Diagnostic and Statistical Manual of Mental Disorders, 4th edition.
Two hundred seventy-five patients participated in the study. Scores on both the One-Question Interview and the Distress Thermometer were significantly correlated with HADS score (One-Question Interview: r = −0.66, P < 0.01; Distress Thermometer: r = 0.71, P < 0.01). At the optimal cutoff points, the sensitivity and specificity for detection of adjustment disorders and major depression were 80% and 61%, respectively, for the One-Question Interview; 84% and 61%, respectively, for the Distress Thermometer; and 92% and 57%, respectively, for the HADS.
It is well known that approximately half of all patients with cancer suffer from psychiatric disorders, the most common of which are adjustment disorders and major depression.1 Adjustment disorders are relatively short maladaptive reactions to stress and are often associated with depressed mood and/or anxiety.2 Major depression is a psychiatric syndrome that is characterized by two core symptoms—depressed mood and loss of interest or pleasure—that persist for at least two weeks and interfere with normal functioning.3 The combined prevalence of adjustment disorders and major depression among patients with cancer is 9–42% (adjustment disorders, 4–35%; major depression, 3–23%).1, 4–7 These disorders have an unfavorable impact on patient quality of life,8 patient decision-making regarding cancer treatment,9 patient suicide,10 and caregiver distress.11 Nonetheless, both psychotherapy and pharmacotherapy are effective in treating these disorders.12, 13 Because of the impact and treatability of adjustment disorders and major depression in patients with cancer, early diagnosis and treatment are important.14
Psychologic distress often is underrecognized by medical staff members in clinical oncology settings.15, 16 The belief, held by both patients and clinicians, that depressive mood or anxiety is an appropriate reaction to cancer; the lack of physician training in the treatment of depression; and the stigma attached to the words psychiatric and psychological are possible causes of reluctance to discuss emotional issues.2, 14
A screening test is the recommended strategy for early diagnosis of psychiatric disorders, such as adjustment disorders and major depression, in patients with cancer. There is an advantage gained by the early treatment of these disorders, which are prevalent in the population but are not entirely evident in patients who suffer from them.17 Several screening tools for adjustment disorders and major depression have been developed.18–23 A number of important points must be kept in mind by those who perform screening for adjustment disorders and major depression in physically ill patients: screening items should not include somatic symptoms that are difficult to interpret,14 screening should be brief enough to be applicable to patients with cancer,2 the test should not be so stigmatizing that it is unacceptable to patients or difficult for the medical staff to use,24 and the test should be easy to score and evaluate.25
For the purposes of brevity and ease of use, one-question screening tests for adjustment disorders and major depression in patients with cancer have been developed. Single-Item Interview Screening, the Visual Analog Scale (VAS), and the Distress Thermometer have been used for patients with advanced stages of cancer18, 24; however, Single-Item Interview Screening requires some training for non–mental health professionals to use because it is part of a structured diagnostic interview, the VAS has poor screening performance (sensitivity, 0.72; specificity, 0.50), and there are no data regarding the screening performance of the Distress Thermometer in patients with cancer.
We developed a new screening interview for adjustment disorders and major depression that is brief and easy to use even for those who are not mental health specialists. In the current study, we compared the screening performance of this interview with that of the Hospital Anxiety and Depression Scale (HADS), which frequently is used to screen for psychologic distress in patients with cancer, and with that of the Distress Thermometer, a single-item, self-report questionnaire that was developed recently and was expected to be a useful screening method in patients with cancer.24
The objectives of the current study were as follows: to develop and assess the validity of a brief screening interview for adjustment disorders and major depression in patients with cancer and to compare its screening performance with that of the HADS and that of the Distress Thermometer.
MATERIALS AND METHODS
Patients were enrolled from two available groups. The first group consisted of patients with cancer who were referred to the Psychiatry Division of the National Cancer Center Hospital (Tokyo, Japan) or the National Cancer Center Hospital East (Kashiwa, Japan). We chose this group because the Distress Thermometer and HADS scores and other psychiatric diagnoses were available as a result of our usual clinical procedure. Because the study procedure required little deviation from standard clinical practice (i.e., the addition of one question about emotional distress), informed consent to participate in the study and institutional review board approval were not obtained for this group. We then chose a second group that was expected to contain more patients with no psychiatric diagnosis than did the referral group (which was expected to contain few patients who had no psychiatric diagnosis), because analysis of specificity required a certain number of patients with no psychiatric diagnosis. For hypothesized sensitivity and specificity values of 0.70 for the new screening test, the sample size required to make the 95% confidence intervals of the sensitivity and specificity < 0.1 was calculated to be > 81 patients with a psychiatric diagnosis and > 81 patients with no diagnosis. The second patient group consisted of patients with breast cancer who participated in another observational study on psychologic distress; data from the HADS and the Distress Thermometer scores and psychiatric examinations were used secondarily. The eligibility criteria were as follows: diagnosis of cancer, age > 18 years, and knowledge of the cancer diagnosis before psychiatric consultation. Exclusion criteria were as follows: excessive physical illness, cognitive impairment, being under psychiatric treatment during the current study, and being deemed likely to be psychologically harmed by participation. Patients with major psychiatric disorders other than adjustment disorders or major depression (e.g., schizophrenia or anxiety disorders) were excluded from the analysis, because the objective of screening was to detect adjustment disorders and major depression.
The One-Question Interview recently was developed as a screening tool for adjustment disorders and major depression in patients with cancer. It is a simple, structured interview that consists of a single inquiry: “Please grade your mood during the past week by assigning it a score from 0 to 100, with a score of 100 representing your usual relaxed mood. A score of 60 is considered the passing grade.” A lower score indicates an unfavorable status. The advantages of this screening tool are its brevity and the absence of any need for special training to evaluate its results, which are expressed as a simple score. The test is less stigmatizing than others, because it asks about state of mood, does not use the words psychological or psychiatric, and is easy to answer. In addition, the format is familiar to Japanese patients, because the perfect score on school examinations in Japan is 100 points, with 60 points being the minimum passing grade.
The Distress Thermometer, which was developed by Roth et al.,24 is a screening tool for psychologic distress in patients with cancer. It is a self-report questionnaire that consists of only one item. It reports distress on a thermometerlike 11-point Likert scale, with scores ranging from 0 to 10 and a higher score indicating a more unfavorable status. The advantages of this tool also are its brevity and its nonstigmatizing format. A score of 5 is defined as the cutoff point, but screening performance at the cutoff is not examined by diagnostic interview. For the current study, with written permission from the author, we prepared a Japanese version of the Distress Thermometer via a back-translation process. We first translated the questionnaire from English into Japanese, and then a bilingual Japanese individual who was blinded to the original questionnaire translated it back into English. A native English speaker compared the original and back-translated questionnaires, and after discussing the translation, we prepared the final Japanese version of the Distress Thermometer.
The HADS is widely used as a screening tool for anxiety and depression in physically ill patients.19 It is a 14-item, self-report questionnaire in which patients rate how they felt during the previous week on a 4-point Likert scale. Total scores range from 0 to 42, and a higher score indicates greater depression and anxiety. The Japanese version of HADS was validated in a population of patients with cancer. The optimal cutoff point for screening for adjustment disorders and major depression was found to be 10/11.22 Good screening performance was reported (sensitivity, 0.92; specificity, 0.65).22
From February 2000 to March 2001, eligible patients were asked to answer the One-Question Interview. The Distress Thermometer and the HADS were used routinely in our clinical practice and also in the group participating in the other protocol. After screening, psychiatrists interviewed patients and made psychiatric diagnoses based on criteria set forth by the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV).3 Data regarding sociodemographic and medical variables (age, gender, marital status, employment status, cancer site, clinical stage, performance status, and pain) were obtained from the consultation database and from medical charts.26 Performance status was assessed based on Eastern Cooperative Oncology Group criteria, which allow physical function to be measured objectively from 0 (no symptoms) to 4 (bedridden). Pain was rated on a four-tiered scale from none at all to intolerable.
To assess the criterion validity of the One-Question Interview and the Distress Thermometer, correlations between the scores on these tests and the HADS were assessed by Spearman rank correlations. Screening scores for the group of patients with no diagnosis and the group of patients with adjustment disorders and/or major depression were compared using the Mann–Whitney test. Construct validity was assessed by examining associations between screening scores (One-Question Interview and Distress Thermometer) and patient characteristics (adjustment disorders and/or major depression, age, gender, pain, performance status, education, and presence of metastasis or recurrence) using univariate analysis (Mann–Whitney test or Spearman correlation) and multiple regression analysis. The Mann–Whitney test was used for adjustment disorders and/or major depression, gender, and presence of metastasis or recurrence; and the Spearman correlation was used for age, pain, performance status, and education. Multiple regression analysis was performed only for variables that showed a significant correlation (P < 0.05) in the univariate analysis. Dummy variables were used when independent variables were categoric.
Sensitivity and specificity at each cutoff point were calculated using receiver operator characteristic (ROC) analysis.27 The likelihood ratio (LR), a measure of how much a given diagnostic test result will raise or lower the pretest probability of the target disorder, also was calculated.28 Positive predictive value (PPV) was calculated for the provisional prevalence rate of adjustment disorders and major depression. To calculate PPV in an ordinary group of patients with cancer (i.e., a group with a normal prevalence of adjustment disorders and major depression) and a high-risk group (i.e., a group with high prevalence), we postulated prevalence rates of 20% and 50%, respectively, based on previous reports.1, 4–7 There is no standard way to determine the optimal cutoff point;27 in the current study, we estimated that the point of highest sensitivity at which LR > 2 was the optimal cutoff point. This point was chosen because sensitivity is more important than specificity in detecting major depression and adjustment disorders and because screening that yields an LR value < 2 is not helpful.28 ROC curves were plotted to compare screening performance. All analyses were performed using SPSS software (SPSS Inc., Chicago, IL).
One thousand ten patients were referred during the study period, and 55 patients from the observational study of the psychologic status of patients with breast cancer participated. Of these patients, 290 (235 in the former group and 55 in the latter group) were eligible, but 15 did not complete at least 1 of the 3 self-report screening questionnaires. Eight did not complete the HADS (four had missing data for one item, three could not complete it because of loss of energy, and one rejected the questionnaire); six did not complete the Distress Thermometer (two could not complete it because of loss of energy, two circled the wrong point, one could not explain distress adequately with the thermometer, and one rejected the questionnaire); and six did not complete the One-Question Interview (four could not rate distress by a score and two did not complete it for unknown reasons). Ultimately, the data for 275 patients were analyzed. Table 1 shows patient characteristics. The mean patient age was 52.2 years (standard deviation, 12.2 years; range, 21–83 years; median, 52 years). The breast was the most common cancer site due to the participation of patients from the breast cancer study. About half of the patients had some pain or impairment of daily activity, and 41.9% had advanced disease (metastasis or recurrence). Adjustment disorders and major depression were diagnosed in 115 (41.8%) and 53 (19.3%) of 275 patients, respectively.
Table 1. Patient Characteristics (n = 275)
SD: standard deviation.
Performance status was defined by Eastern Cooperative Oncology Group criteria.
The Spearman correlation coefficient between HADS scores and One-Question Interview scores was −0.66 (P < 0.01), and the coefficient between HADS scores and Distress Thermometer scores was 0.71 (P < 0.01). The test scores for each diagnostic group (patients with no diagnosis and patients with adjustment disorders and/or major depression) were significantly different (P < 0.01) (Table 2).
Table 2. One-Question Interview, Distress Thermometer, and HADS Scores of Psychiatrically Diagnosed Patients (n = 275)
No diagnosis (n = 107)
AD and/or MD (n = 168)
Mean ± SD
Mean ± SD
HADS: Hospital Anxiety and Depression Scale, AD: adjustment disorders, MD: major depression; SD: standard deviation.
69.2 ± 19.5
46.0 ± 21.4
3.4 ± 2.6
6.6 ± 2.4
9.1 ± 5.7
20.2 ± 7.5
Univariate and multivariate regression analyses of associations of screening scores with patient characteristics are shown in Table 3. In addition to being associated with psychiatric disorders, Distress Thermometer score was associated with performance status and pain.
Table 3. Factors Correlated to One-Question Interview and Distress Thermometer Scores
The cutoff points, sensitivity, specificity, and LR of screening for adjustment disorders and major depression, as well as the PPV of screening in populations with 20% and 50% prevalence (PPV[20%] and PPV[50%], respectively), are shown in Table 4. Based on sensitivity and LR, the optimal cutoff points were 4/5 for the Distress Thermometer (sensitivity, 0.84; specificity, 0.61; LR, 2.13), 60/65 for the One-Question Interview (sensitivity, 0.80; specificity, 0.61; LR, 2.03), and 9/10 for the HADS (sensitivity, 0.92; specificity, 0.57; LR, 2.13). Figure 1 shows the ROC curves (sensitivity vs. 1-specificity at each cutoff point) for screening for adjustment disorders and major depression using the HADS, Distress Thermometer, and One-Question Interview. Based on the ROC curves, the screening performance of the HADS was the best of the three, and the screening performance of the Distress Thermometer was slightly better than that of the One-Question Interview.
Table 4. Test Performance at Various Cutoff Points for the One-Question Interview, Distress Thermometer, and HADS
HADS: Hospital Anxiety and Depression Scale; LR: likelihood ratio; PPV(20%): positive predictive value of screening in a population with 20% prevalence; PPV(50%): positive predictive value of screening in a population with 50% prevalence.
We developed the One-Question Interview, a brief screening interview for adjustment disorders and major depression; examined its criterion and content validity; and compared its screening performance with that of other tools for screening patients with cancer. We concluded that the One-Question Interview had acceptable criterion validity, based on the significant correlation of its scores with HADS scores and on the differences between scores for patients with no diagnosis and scores for patients with adjustment disorders and/or major depression. With regard to content validity, we suspected before the current study that brief screenings might be influenced by nonpsychologic factors (e.g., pain or performance status), because pain,29, 30 functional status,23, 29 advanced disease stage,23 age,31 and gender32, 33 had been found to be correlated with self-report questionnaire scores for patients with cancer in previous studies. Scores on the One-Question Interview, however, were unassociated with any patient characteristics except psychiatric diagnosis; this finding demonstrated that the content validity of the interview was acceptable. In contrast, the Distress Thermometer was influenced by performance status and pain independent of psychiatric diagnosis; consequently, these physical symptoms must be taken into account in assessing the results of the Distress Thermometer test.
The screening performance of the One-Question Interview was comparable to that of the Distress Thermometer and worse than that of the HADS. Both the One-Question Interview and the Distress Thermometer performed better than another 1-item screening tool, the VAS (sensitivity, 0.72; specificity, 0.50).18 If screening performance can be generalized to populations with different prevalence rates, PPV can be calculated for each prior probability (20% and 50% in the current study). The One-Question Interview, with a sensitivity of 0.80, a specificity of 0.61, a PPV(20%) of 0.34, and a PPV(50%) of 0.67, was acceptable. Aside from reflecting the acceptable criterion and content validity of the One-Question Interview, the data from the current study also suggested that the interview may be useful as a crude, rapid screening tool, because it does not require the use of paper or pencil and can be used in various situations (e.g., telephone interviews). The advantage of the One-Question Interview over the other screening tools is that it can be inserted into a clinical interview without the physician and patient engaging in formal psychiatric screening. This may be important for routine screening in clinical practice, because one of the major barriers to psychiatric screening is the reluctance of doctors and patients to deal with psychiatric issues.34
The Distress Thermometer also demonstrated acceptable levels of validity and performance (sensitivity, 0.84; specificity, 0.61; PPV[20%], 0.35; PPV[50%], 0.68) based on the diagnostic interview. The performances of Distress Thermometer and the One-Question Interview were similar, but their methods clearly differed. Accordingly, in the future, we can choose the test that is more appropriate in a given situation. For example, the Distress Thermometer can be used in outpatient waiting rooms, whereas the One-Question Interview can be used in telephone interviews or administered to physically impaired patients.2
A previous report on the performance of the HADS in screening for adjustment disorders and major depression in Japanese patients with cancer yielded an optimal cutoff point of 10/11, and its sensitivity and specificity were 0.92 and 0.65, respectively. At the optimal cutoff point, its sensitivity and specificity in the current study were comparable (sensitivity, 0.89; specificity, 0.65). The key measure of a screening tool is sensitivity, and the HADS yielded the highest sensitivity of the three tests that were evaluated. The difference in sensitivity between the One-Question Interview (sensitivity, 0.80) and the HADS (sensitivity, 0.92) means that the One-Question Interview would miss 2.4 cases in every 100 patients screened when the prevalence of psychologic problems was 20%. Nonetheless, this deficiency can be overcome by screening more patients with the One-Question Interview. For a population in which the prevalence was 20%, screening 115 patients with the One-Question Interview would identify the same number of cases as would screening 100 patients with the HADS; furthermore, because the One-Question Interview is brief and informal, administering it to more than 115 patients appears to be easier than administering the HADS to 100 patients. Because of its superior performance and inferior ease of administration, the HADS may be best suited to screening more susceptible populations—for example, patients who are clinically suspected of being depressed or patients who were found to be positive by a brief screening test (e.g., the One-Question Interview).
To avoid allowing adjustment disorders and/or major depression to go unrecognized or untreated, all patients with cancer should be screened routinely at appropriate intervals and not simply when they have clinical indications of depression.34 Therefore, a screening tool for adjustment disorders and major depression in patients with cancer must be brief in addition to having a high screening performance. In this respect, both brief screening tools demonstrated acceptable performance for crude screening, but their PPVs (0.34 for the One-Question Interview and 0.35 for the Distress Thermometer) are not indicative of precise results. Only one-third of patients who tested positive actually required psychologic intervention, whereas the remaining two-thirds had false positive results. Improvement of specificity (without sacrificing brevity) is needed for more useful screening. Combined use of the available crude and precise screening tools (e.g., using the One-Question Interview for all patients and the HADS only for patients who test positive on the One-Question Interview) may be more useful and should be easier than using the HADS for all patients.
The aim of brief screening is to identify undiagnosed patients who require treatment and to guide them to the appropriate treatment. Because the One-Question Interview is not a diagnostic test, further examination (e.g., an interview by a mental health specialist or a trained physician) is necessary to determine what treatment is adequate.21 The cutoff point can be adjusted according to the resources available at a given institution.
There were several limitations in the current study. The first limitation was the problem of the gold standard for psychiatric diagnosis. Psychiatrists who made diagnoses were not blinded to screening results; however, this probably did not have a significant effect on the results, because the optimal cutoff points of the brief screening tests were not known during the study period. The second limitation was that the interrater reliability of psychiatric diagnoses based on DSM-IV was not assessed, although patient diagnoses were checked regularly at weekly clinical conferences. The third limitation was that the test-retest reliability was not assessed. The fourth limitation resulted from spectrum bias due to selected sampling. The majority of the sample consisted of patients who had been referred to psychiatrists; this could have improved screening performance artificially, because referred patients already had undergone clinical screening by oncologists. When screening performance was calculated for only the 55 patients in the observational study of the psychologic status of patients with breast cancer, the sensitivity and specificity were 0.20 and 0.62, respectively, for the One-Question Interview; 0.60 and 0.60, respectively, for the Distress Thermometer; and 0.40 and 0.62, respectively, at the cutoff point with highest sensitivity at which < R > 2. However, these results were not reliable due to the inadequate size of the sample, which included only five patients who were diagnosed with adjustment disorders or major depression. A study that examines screening performance in a nonselected sample is needed to generalize the results. The fifth limitation of the study dealt with crosscultural applicability. In Japan, almost all patients consider 60 points to be the passing grade on examinations in school, but this may not be the case in other countries; consequently, interview results may not be generalizable across different cultures.
In conclusion, the newly developed, brief screening tool, the One-Question Interview, demonstrated acceptable validity and performance in detecting the need for psychiatric intervention. A brief method of screening may enable depressed patients to receive appropriate psychiatric intervention. We are planning a subsequent study to assess whether the One-Question Interview can lead depressed patients to seek psychiatric intervention and reduce the prevalence of adjustment disorders and major depression.
The authors thank Dr. Andrew J. Roth and Dr. Jimmie Holland of the Department of Psychiatry and Behavioral Science, Memorial Sloan-Kettering Cancer Center, New York, New York, for granting permission to use the Distress Thermometer. The authors also thank Ms. Ryoko Katayama, Ms. Masae Hitomi, and Ms. Yurie Sugihara of the National Cancer Center Research Institute East, Kashiwa, Japan, for their research assistance.