Fax: (503) 494-6197
Patient-physician disagreement regarding performance status is associated with worse survivorship in patients with advanced cancer†
Version of Record online: 8 SEP 2008
Copyright © 2008 American Cancer Society
Volume 113, Issue 8, pages 2205–2214, 15 October 2008
How to Cite
Schnadig, I. D., Fromme, E. K., Loprinzi, C. L., Sloan, J. A., Mori, M., Li, H. and Beer, T. M. (2008), Patient-physician disagreement regarding performance status is associated with worse survivorship in patients with advanced cancer. Cancer, 113: 2205–2214. doi: 10.1002/cncr.23856
Abstract presented as poster discussion at Proceedings of the American Society of Clinical Oncology, June 1–5, 2007
- Issue online: 3 OCT 2008
- Version of Record online: 8 SEP 2008
- Manuscript Accepted: 9 JUN 2008
- Manuscript Revised: 4 JUN 2008
- Manuscript Received: 4 MAR 2008
- performance status;
- patient-physician communication;
- advanced cancer;
- Eastern Cooperative Oncology Group
Physician-reported performance status (PS) is an important prognostic factor and frequently influences treatment decisions. To the authors' knowledge, the extent, prognostic importance, and predictors of disagreements in PS assessment between physicians and patients have not been adequately examined.
Using North Central Cancer Treatment Group (NCCTG) clinical trial data from 1987 through 1990, the authors compared PS (Eastern Cooperative Oncology Group [ECOG] and Karnofsky [KPS]) and nutrition scores reported by physicians and patients individually. Differences were analyzed using a Student t test for paired data and degree of disagreement by kappa statistic. The effect of disagreement on overall survival was determined by the Kaplan-Meier method and Cox regression analysis. Predictors of disagreement were identified by logistic regression.
In all, 1636 patients with advanced lung and colorectal cancer had a median survival of 9.8 months (95% confidence interval [95% CI], 9.4-10.4 months). Percent disagreement between patients and physicians regarding KPS, ECOG PS, and nutrition score were 67.1%, 56.6%, and 58.0%, respectively. Physicians were more likely to rate patients better than individual patients were to rate themselves: ECOG (mean 0.91 vs 1.30; P < .0001), KPS (mean 83.3 vs 81.7; P < .0001), and nutrition score (mean 1.6 vs 2.1; P < .0001). Disagreement between patients and their physicians was associated with increased risk of death: KPS (hazards ratio [HR] of 1.16; 95% CI, 1.04-1.30 [P = .008]) and nutrition scores (HR of 1.44; 95% CI, 1.29-1.61 [P < .0001]) after adjustment for covariates. Patient sociodemographic factors that predict disagreement were identified.
Physicians and patients frequently disagree regarding PS and nutritional status. Disagreement is associated with an increased risk of death in patients with advanced malignancies. These findings illustrate the limitations of physician-only assessed PS. Cancer 2008. © 2008 American Cancer Society.
Performance status (PS) is a commonly used prognostic factor in advanced malignancies. Several investigators have evaluated the validity and reliability of the Karnofsky performance status (KPS) and found that a low KPS (≤50) predicts death within a relatively short period of time and that there is a direct relation between the KPS and survival.1, 2 The KPS has been applied in cancer research to select and stratify patients for inclusion in treatment trials, to measure the efficacy of treatment, and to estimate prognosis.
Similarly, the Eastern Cooperative Oncology Group (ECOG) PS is widely used to assess the functional status of cancer patients. Both the KPS and the ECOG PS scales are highly correlated3, 4 and are used interchangeably; the ECOG PS is often preferred for its simplicity.
The KPS has varying degrees of interrater reliability, with differences noted depending on the setting (home vs clinic) of assessment.5 Importantly, PS is routinely assessed by the clinician and, as a rule, has not been reported by patients in past studies.
Similar to poor PS, the presence of anorexia has been validated as a marker of poor prognosis in patients in terminally ill cancer patients.6 Anorexia and cachexia affect up to 87% of patients with malignancies and has been implicated as a contributor to cancer-related mortality.7 Nutritional impairment correlates with the response to therapy and may be directly related to therapeutic results and severity of treatment-related toxicity.8 Despite its prognostic value, nutritional status is not routinely evaluated in most oncologic settings by either physicians or patients.
Loprinzi et al9 developed an instrument to measure patient-reported ECOG PS, KPS, and appetite/food intake. Using a Cox proportional hazards model, they demonstrated that patient and physician-reported ECOG PS, KPS, and nutritional assessments independently provide strong predictive information regarding survival. Importantly, patient-judged PS and nutrition was found to be complementary to, but not interchangeable with physician-judged PS and nutrition. Furthermore, this study raised important questions regarding the extent to which physician and patient scores agreed (and disagreed) about PS and nutrition; it also raised questions concerning the meaning of these disagreements.
A small, single-institution study by Blagden et al10 of consecutive patients with a suspected diagnosis of lung cancer assessed agreement between patients and their oncologists using ECOG PS. They reported a weighted kappa of 0.45 (95% confidence interval [95% CI], 0.33-0.59), indicating a moderate degree of agreement. Although the authors concluded that patients in that study were “marginally less optimistic” than their oncologists, the study was too small to draw meaningful statistical conclusions.
The goals of the current study were to use a large patient cohort within a multi-institutional research setting to evaluate the extent to which patients and physicians disagree regarding PS and nutritional assessments, to characterize the direction of such disagreement, to identify patient sociodemographic factors that predict disagreement, and to determine whether disagreement is associated with an increased risk of death.
Making use of the statistical power of large patient numbers, this study also evaluated the frequency of ratings over different PS and nutrition scores. It compared the measurement techniques themselves as well as the operating characteristics of patient versus physician reports of PS as predictors of patient survival, to gauge whether patient-reported adds precision to physician-reported PS.
Although previous studies have focused on comparing patient-reported and physician-reported PS, the focus on understanding the nature and implications of disagreement between patients and their physicians is the novel aim of this investigation.
MATERIALS AND METHODS
Patients and Performance Status
A total of 1636 patients with advanced colorectal and lung cancer from 6 North Central Cancer Treatment Group (NCCTG) cancer treatment trials for advanced incurable cancer conducted between the years of 1987 and 1990 were included in this study.
The original research involved the development of a 30-item questionnaire completed by both patients and physicians for purposes of refinement. The previously published results used variable selection and model building using data from 1560 patients and demonstrated that the following subset of 4 items was prognostic for survival: physician assessment of 1) PS and 2) likelihood of survival, 3) patient assessments of PS, and 4) patients' report of food intake/appetite.9, 11
Patients' PS (ECOG and Karnofsky) scores and nutritional status scales were completed by both physicians and patients themselves once at the time of study enrollment into each of the respective trials. Physician-reported assessments were recorded by the attending medical oncologist as part of routine pretrial screening. Information from patients was collected using a questionnaire developed by Loprinzi et al9 to ask patients to rate their own ECOG PS, KPS, and food intake/appetite. No formal training was provided to physicians or patients. Nurse investigators closely evaluated the patient-completed questionnaires and documented the questions that patients had difficulty understanding or answering. These questions were subsequently reworded or refined for better patient lay clarification. At the same time, patients were also provided a questionnaire in which they defined their own religious faith and activity, level of education, working status, and depression using check marks. For depression, possible answers were, “Half the time or more” or “No or seldom.” The checked answers were subsequently collected and tabulated.
Definitions of ECOG score (0, 1, 2, 3, and 4) and Karnofsky score (100, 90, 80, 70, 60, 50, 40, and 30) in physician and patient assessment forms were virtually identical.
The patient-reported food intake score, which had 6 response options (1, 2, 3, 4, 5, 6), was modified to match the physician-completed nutrition score, which had only 5 response options (0, 1, 2, 3, and 4). Briefly, a patient's food intake score of 1 or 2 (increased or the same) was recoded to match a physician's nutrition score of 0 (normal); a patient's score of 3 (slightly reduced) was recoded to match a physician's score of 1 (slightly impaired); a patient's score of 4 (moderately reduced) was recoded to match a physician's score of 2 (moderately impaired); a patient's score of 5 (markedly reduced) was recoded to match a physician's score 3 (considerable impaired); and a patient's score of 6 (severely reduced) was recoded to match a physician's score of 4 (severely impaired).
Identical scores between physician and patient regarding prognostic performance and nutritional status were defined as physician-patient agreement; score discrepancies between physician and patient were identified as any physician-patient disagreement.
Statistical analyses were performed with SAS statistical software (version 9.1; SAS Institute Inc, Cary, NC). Overall survival (OS) was calculated from the date of enrollment to the date of death or last follow-up. The distributions of scores from both the physician and the patient as well as the proportions of levels of discrepancy were examined using frequency analysis. The degree of score difference and agreement between the physician-completed and the patient-completed ECOG and KPS scores as well as between the physicians' nutritional scores (nutrition) and the modified patients' food intake scores (food intake) were examined by a Student t test for paired data and a kappa statistic. The impact of physician-patient disagreement and factors with significant influence on OS were determined by the Kaplan-Meier method and Cox regression. The potential association between patients' sociodemographic factors and physician-patient disagreement were assessed by logistic regression. All reported P values are 2-sided and a value of P < .05 was considered statistically significant.
Patient Sociodemographic Factors and Their Association With Overall Survival
A total of 1636 patients (55% male and 97% white) were entered into this study. The patients were concurrently participating in 1 of 6 clinical trials for advanced lung cancer or advanced colorectal cancer. Their median age was 63 years, with a range of 17 to 85 years.
The median survival of the enrolled patients was 9.8 months (95% CI, 9.4-10.4 months). The median follow-up time for patients alive (n = 55) at last follow-up was 4.45 years (range, 1.2-6.9 years). The median follow-up time for patients dead at last contact was 9.5 months (range, 0.03 months to 5.4 years). Physician data were available for all 1636 patients. Patient-reported data were available for >95% of patients. Of the patient sociodemographic factors listed in Table 1, male sex, older age, inability to work before retirement, and depression were significantly associated with shorter median overall survival.
|Distribution||Kaplan-Meier Survival Estimation||P*|
|Factors*||No. of Patients (%)||Median, Mo||(Log-rank)|
|Work full-time||295 (20)||12.2||<.0001|
|Work part-time||227 (15)||10.8|
|Unable to work||286 (19)||7.1|
|<High school||535 (34)||9.5||NS|
|High school||621 (40)||10.6|
|Very active||566 (36)||9.6||NS|
|Not active||416 (27)||9.8|
|Half the time or more||415 (27)||8.3||.01|
|No or seldom||1123 (73)||10.2|
Agreement and Disagreement Between Physicians and Patients
Greater than half of the patients disagreed with their physician regarding ECOG PS, KPS, or nutritional assessment (56.6%, 67.1%, and 58.0%, respectively). When there was disagreement, physicians were more likely to rate patients better than patients were to rate themselves better in all 3 scores: ECOG PS (physician better 75.1% vs patient better 24.9%), Karnofsky score (physician better 57.4% vs patient better 42.6%), and nutritional score (physician better 75.4% vs patient better 24.6%) (data not shown). Mean physicians' ratings of patients were better than mean patients' ratings of themselves in all 3 variables and the mean differences between physician and patient ratings were significant: ECOG PS (mean 0.91 vs 1.30; P <.0001), KPS (mean 83.3 vs 81.7; P < .0001), and food intake/nutritional status (mean 1.6 vs 2.1; P< .0001). Weighted kappa revealed only fair agreement between patients and physicians in KPS, ECOG PS, and food intake/nutrition score (kappa: .30, .35, and .26, respectively).12
The distribution, direction of disagreement, and levels of discrepancy are detailed in Table 2
|Level of Difference|
|Physician rated better by 1 level||497||31.9||.30|
|Physician rated better by 2 levels||149||9.6|
|Physician rated better by 3 levels||16||1.03|
|Patient rated better by 1 level||209||13.4|
|Patient rated better by 2 levels||10||0.6|
|Patient rated better by 3 levels||1||0.1|
|Mean score ± SD, physician/patient||0.9 ± 0.7/1.3 ± 1.0|
|Mean difference ± SD*||−0.4 ± 0.9|
|Level of Difference|
|Physician rated better by 1 level||367||23.6||.35|
|Physician rated better by 2 levels||192||12.3|
|Physician rated better by 3 levels||32||2.1|
|Physician rated better by 4 levels||9||0.6|
|Patient rated better by 1 level||315||20.2|
|Patient rated better by 2 levels||100||6.4|
|Patient rated better by 3 levels||28||1.8|
|Patient rated better by 4 levels||1||0.1|
|Patient rated better by 5 levels||1||0.1|
|Mean score ± SD, physician/patient||83.3 ± 11.9/81.7 ± 13.4|
|Mean difference ± SD*||1.6 ± 12.7|
|Nutrition/Food Intake Score|
|Level of Difference|
|Physician rated better by 1 level||417||26.6||.26|
|Physician rated better by 2 levels||198||12.6|
|Physician rated better by 3 levels||77||4.9|
|Physician rated better by 4 levels||18||1.2|
|Patient rated better by 1 level||181||11.6|
|Patient rated better by 2 levels||27||1.7|
|Mean score ± SD, physician/patient||0.7 ± 0.7/1.2 ± 1.2|
|Mean difference ± SD*||−0.5 ± 0.9|
Although the mean scores, proportion of physician-patient agreement, and levels of discrepancy presented in Table 2 revealed that, overall, physicians were more likely to rate patients better than individual patients were to rate themselves, in a minority of patients the opposite occurred and patients rated themselves better than their physicians rated them. The distribution of both physician-rated and patient-rated scores are graphically displayed in Figure 1 and revealed that physicians more frequently rated patients better than patients rated themselves when patients had relatively good PS and nutritional status. Conversely, patients whose PS and nutritional status were poor were more likely to rate themselves better than their physicians rated them. A similar pattern was found in all 3 scores.
Survivorship by Agreement Status
Figure 2 displays Kaplan-Meier estimated overall survival curves by agreement status for KPS and nutrition. Patients who disagreed with their physician had inferior survivorship when compared with those who agreed on KPS (median survival: 9.4 vs 11.8 months; P = .002) and food intake/nutritional status (median survival: 8.0 months vs 12.5 months; P < .0001).
Overall survivorships by levels of discordance are displayed in Figure 2. Patients who rated their own nutritional status at least 2 levels better (n = 27) than did their physicians had shorter survival than whose who rated themselves 1 level better (median OS: 8.7 months vs 9.6 months; P = .04). This subset of patients was found to have worse nutritional status, based on the frequency analysis displayed in Figure 1. A similar pattern was also observed in KPS. Agreement status for ECOG PS was not significantly associated with survivorship.
Risk of death remained significantly higher for patient and physician disagreement on KPS (hazards ratio [HR] of 1.16; 95% CI, 1.04-1.30 [P = .008]) and food intake/nutritional status (HR of 1.44; 95% CI, 1.29-1.61 [P < .0001]), after adjustment for age, sex, working status, and depression. Data for survivorship by agreement status are presented in Table 3.
|Kaplan-Meier Estimation||Cox Regression|
|Score||1-Year (%)||Median, Mo||P (Log-rank)||Unadjusted Hazards Ratio (95% CI)||P||Adjusted* Hazards Ratio (95% CI)||P|
|Disagree||39.8||9.6||NS||1.04 (0.94-1.15)||NS||1.02 (0.91-1.13)||NS|
|Disagree||37.5||9.4||.002||1.15 (1.03-1.27)||.01||1.16 (1.04-1.30)||.008|
|Food intake/nutritional status|
|Disagree||34.1||8.0||<.0001||1.41 (1.28-1.56)||<.0001||1.44 (1.29-1.61)||<.0001|
Patient Sociodemographic Factors as Predictors of Disagreement
The association of patient-physician disagreement with specific patient sociodemographic factors is displayed in Table 4. A univariate analysis revealed that patients not working full-time (working part-time, unable to work, or retired) were more likely to disagree than those who work full-time across all 3 scores. In addition, age, less than a high school education, and self-reported depression greater than half the time were also potentially associated with patient-physician disagreement.
|ECOG PS||Karnofsky PS||Food Intake/Nutrition|
|Factors||OR of Death||(95% CI)||P||OR||(95% CI)||P||OR||(95% CI)||P|
|Unable to work||5.40||(3.74-7.77)||<.0001||1.92||(1.36-2.73)||.0003||3.06||(2.17-4.32)||<.0001|
|Half the time or more||1.58||(1.24-1.99)||.0001||1.2||(0.94-1.53)||.15||1.90||(1.50-2.42)||<.0001|
|No or seldom||1||1||1|
A complete, appropriate multivariate analysis was not possible to assess independent predictors of disagreement because physician sociodemographic data (ie, physician age, years in practice, nature of practice [academic or community-based], etc) was not collected.
In PS assessment, clinician report has historically been considered the gold standard.1, 2, 13 This stands in contrast to quality of life, in which patient assessment has become the gold standard after it was shown that patients are more reliable and consistent at scoring their own quality of life than are their physicians.14, 15 Similarly, physician reporting of adverse treatment effects has been shown to lack both sensitivity and specificity and routine reliance on physicians alone to assess adverse effects of therapy has been questioned.16
Previous investigators have found that patient-judged ECOG PS, KPS, and appetite/food intake scores all provide independent prognostic survival information that is complementary to clinician-reported data.9 However, to our knowledge, there has been very little attention devoted to understanding the disagreement between patients and their physicians with respect to these assessments.
To our knowledge, this is the first study to demonstrate that the mean differences between physicians' ratings of patients and patients' ratings of themselves across KPS, ECOG PS, and nutrition scores were significant, and that the overall mean of physicians' ratings was better than the mean of patients' ratings in the setting of advanced cancer.
Furthermore, when patients disagree with their physicians regarding PS and nutritional assessments, the disagreement tends to be in the direction of physicians rating patients better, rather than vice versa.17 These findings are similar to those of symptom research studies, which consistently demonstrate that physicians and nurses underestimate symptom frequency and severity in comparison with patient ratings.18-21
Similarly, our results parallel the findings of Christakis and Lamont22 that physician are routinely overly optimistic in their estimates of survival of terminally ill patients by a factor of approximately 5.
One can hypothesize reasons why clinicians would rate patients better than patients rate themselves. Bias may be introduced if treatment is assigned to patients on the basis of the physician-assessed PS or nutrition score. Specifically, in the setting of a clinical trial, a worse PS or poorer nutritional status might affect candidacy for experimental therapies. Alternatively, a busy clinical practice setting might hamper precise assessment of PS and nutrition by providers.
Our finding that a subset of patients rate their PS and nutrition as better than their physicians is no less important. Several investigations evaluating physicians' prognostic estimates versus those of advanced cancer patients have shown a substantial discrepancy between the two,23 with patients tending to be comparatively overoptimistic. Such overoptimism can lead to misunderstanding the intent of treatment as curative versus palliative,24, 25 or the pursuit of medically futile therapies.26
This study was able to identify patient-specific factors (working status, level of education, and depression) that potentially predicted disagreement concerning PS and nutrition. This raises the question of whether physician PS assessment can be biased by sociodemographic patient factors. Such discordance may also be explained by patients with lesser education having difficulty interpreting the assessment questions themselves.
Patients with self-reported depression may have had psychologic comorbidity that, although undiagnosed by their oncologist, did impact their functional status. We are unaware of data examining the connection between depression and PS. However, self-reported depression is prevalent among cancer patients,27 and can negatively impact response to treatment.28
Sensitivity analysis of these data suggests that both patients and physicians have similar sensitivity in their ability to predict survival at 3 years (data not shown).
Furthermore, we are unaware of any evidence demonstrating superior accuracy of either patient or physician reporting with respect to their ability to predict survival. Prior investigations have found that both patient and physician KPS and nutritional status scores are highly significant, independent predictors of death (P < .0001),9 and that patients' report of PS and appetite supplement the predictive power of physician-reported ECOG PS for survival.9 In the absence of evidence demonstrating superior sensitivity or predictive ability of patient assessment, we cannot conclude that patient-reported PS or nutritional status should replace physician assessment as the gold standard.
To the best of our knowledge, this is the first study to demonstrate that disagreement in PS and nutritional status assessment is itself adversely prognostic. From this study, the reasons why disagreement was associated with poorer survival are not evident.
Considering the central role that PS plays in decisions to recommend for or against chemotherapy, inaccurate assessment of PS and nutritional status could have influenced therapeutic decisions in some patients. In the current trial, it is not clear to what extent this occurred and if patient outcomes were impacted.
There may be value in routinely collecting patient-reported PS and nutritional data at the point of care. Studies of computerized prompters suggest that clinical information presented selectively at the point of care is more likely to affect patient care decisions, particularly when it is patient-specific.29
Anorexia affects both the quality and quantity of life of many patients with advanced cancer.30, 31 Allowing patients to routinely report food intake data at the point of care may not only create an opportunity for the clinician to intervene therapeutically,32, 33 but also to identify an adverse prognostic factor with which to refine their clinical estimate of prognosis.6
This study has limitations that are important to note. As a secondary data analysis, this study is able to demonstrate significant associations but is not able to prove causality. These data are thus more useful for hypothesis generation than as a basis for a change of clinical practice.
Although significant discordance was noted with regard to clinician versus patient reporting of ECOG PS, the discordance did not adversely affect survival to a statistically significant degree, as it did with KPS and nutritional assessment. Although the ECOG assessment tool has been well-validated in a prospective fashion versus the Karnofsky instrument,3, 4 patients and physicians may have differing interpretations of the ECOG PS questions. For example, “up and about greater than 50% of the waking hours,” may be interpreted as “not in bed 50% of waking hours.” This could result in a frequency of disagreement that is comparable to the other scoring instruments, but result in different consequences of such disagreement.
We acknowledge that the role of working status as a predictor disagreement may be confounded by comorbidities. The absence of these data as well as that of physician-specific sociodemographic factors limited our ability to perform an appropriate multivariate analysis of the predictors of disagreement, because these data were not collected prospectively.
The study sample was 97% white and therefore these results may not be generalizable to other races or ethnic groups.
In patients with advanced cancer, accurate assessment of PS is critical for prediction of survival and clinical decision-making. Often, PS assessment drives physician estimates of the probability that palliative chemotherapy will be helpful and guides choices of which patients should receive chemotherapy and in which patients physicians should focus on supportive care alone. Nutritional status also has prognostic value in advanced cancer, but is not routinely evaluated in most clinical settings.
The finding that physicians and patients frequently disagree about PS and nutritional assessments raises important questions about the accuracy of physician-only assessment and the potential consequences of physicians rating patients better than patients rate themselves on treatment decisions for patients with advanced cancer.
The adverse prognostic impact of physician-patient disagreement is a novel finding. More effort is needed to better understand this phenomenon and approaches to reduce disagreement should be developed.
These results also bring into focus the challenge and confounders of effective physician-patient communication. Disagreement was more common when patients were working less than full time, had less education, and reported depression. It is not surprising that these demographic variables were associated with disagreement. These findings should serve as a reminder to physicians to recognize their own limitations in patient assessment when caring for all patients and especially patients at high risk for disagreement.
Although this secondary data analysis does not displace the historically proven clinician-reported PS, systematic collection of patient-reported PS and nutritional data at the point of care is feasible, and may add precision to clinician assessments.11 The effects of routinely presenting patient assessments of PS and nutrition to physicians should be evaluated prospectively in a larger study, with pertinent endpoints such as chemotherapy dose intensity/density, clinical outcomes, and therapeutic decision-making at the end of life.
- 7Physicians Guide to Cancer Care Complications, Prevention and Management. Fundamentals of Cancer Management. New York: Marcel Dekker; 1986..
- 13The clinical evaluation of chemotherapeutic agents in cancer. New York: Columbia University Press; 1949..
- 14The Measurement of Pain and Other Symptoms. In: DoyleD, HanksGWC, MacDonaldN, eds. Oxford Textbook of Palliative Medicine. 2nd ed. New York: Oxford University Press; 2001., .