Predictive validity of five comorbidity indices in prostate carcinoma patients treated with curative intent

Authors

  • David L. Boulos M.Sc.,

    1. Division of Cancer Care and Epidemiology, Queen's Cancer Research Institute, Queen's University, Kingston, Ontario, Canada
    Search for more papers by this author
  • Patti A. Groome Ph.D.,

    Corresponding author
    1. Division of Cancer Care and Epidemiology, Queen's Cancer Research Institute, Queen's University, Kingston, Ontario, Canada
    • Division of Cancer Care and Epidemiology, Queen's Cancer Research Institute, 10 Stuart St., 2nd Level, Kingston, Ontario K7L 3N6, Canada
    Search for more papers by this author
    • Fax: (613) 533-6794

    • Dr. Groome was an Ontario Ministry of Health and Long Term Care Career Scientist during the time that this work was conducted.

  • Michael D. Brundage M.D.,

    1. Division of Cancer Care and Epidemiology, Queen's Cancer Research Institute, Queen's University, Kingston, Ontario, Canada
    Search for more papers by this author
  • D. Robert Siemens M.D.,

    1. Department of Urology, Queen's University, Kingston, Ontario, Canada
    Search for more papers by this author
  • William J. Mackillop M.D.,

    1. Division of Cancer Care and Epidemiology, Queen's Cancer Research Institute, Queen's University, Kingston, Ontario, Canada
    Search for more papers by this author
  • Jeremy P.W. Heaton M.D.,

    1. Department of Urology, Queen's University, Kingston, Ontario, Canada
    Search for more papers by this author
  • Karleen M. Schulze M.Math.,

    1. Division of Cancer Care and Epidemiology, Queen's Cancer Research Institute, Queen's University, Kingston, Ontario, Canada
    Search for more papers by this author
  • Susan L. Rohland

    1. Division of Cancer Care and Epidemiology, Queen's Cancer Research Institute, Queen's University, Kingston, Ontario, Canada
    Search for more papers by this author

Abstract

BACKGROUND

Comorbidity is important to consider in clinical research on curative prostate carcinoma because of the role of competing risks. Five chart-based comorbidity indices were assessed for their ability to predict survival.

METHODS

This was a case-cohort study of prostate carcinoma patient cohort treated with curative intent in Toronto and Southeast Cancer Care Ontario regions between 1990 and 1996; the subcohort was drawn from these men, whereas cases were cohort members who died from causes other than prostate carcinoma. Comorbidity data were obtained from medical charts (269 subjects). Vital status, age, area of residence, and socioeconomic status information were available. Predictive validity was quantified by the percent variance explained (PVE) over and above age using proportional hazards modeling.

RESULTS

The Chronic Disease Score (CDS) (PVE = 11.3%; 95% confidence interval [95% CI], 3.5-22.8%), Index of Coexistent Disease (ICED) (PVE = 9.0%; 95% CI, 2.9-17.9%), Cumulative Illness Rating Scale (CIRS) (PVE = 7.2%; 95% CI, 1.4-17.1%), Kaplan-Feinstein Index (PVE = 4.9%; 95% CI, 0.6-12.8%), and Charlson Index (PVE = 3.8%; 95% CI, 0.3-10.9%) each explained some outcome variability beyond age. PVE differences among indices were not statistically significant. A comorbidity identified at the time of cancer diagnosis was the cause of death in 59.2% of cases (75% for cardiac or vascular causes).

CONCLUSIONS

The better-performing, more comprehensive indices (CDS, ICED, and CIRS) would be useful in measuring and controlling for comorbidity in this setting. The CDS was easiest to apply and explained the most outcome variability. Cancer 2006. © 2006 American Cancer Society.

Comorbidity is the presence of recognized conditions in a study participant other than the identified index condition.1 Comorbidity can confound our ability to study relations between treatment and outcomes through its role as a competing risk. This is especially true when the outcome of interest is overall survival. Comorbidity has been shown to be inversely associated with variables such as socioeconomic status (SES)2 and directly associated with variables such as age,3 survival,4 the recommended treatment,5 disability,6 and health service utilization.7 Comorbidity may also interact with treatment effectiveness, particularly when it is associated with incomplete or inadequate treatment delivery. Therefore, it is important to consider comorbidity when studying health outcomes, particularly in a chronic condition such as prostate carcinoma, in which long-term survival increases the chance that competing risks will play a role. A summary comorbidity index can be efficient for this purpose.

The literature describes a number of comorbidity indices that identify and summarize comorbid burden. We were particularly interested in those indices designed for use in retrospective chart review because this is a common setting for observational treatment effectiveness research. Adequate measurement of comorbid illness using this information source is crucial to proper consideration of this very important variable when trying to assess the success of one's treatment delivery. This work was preliminary to an Ontario-wide study designed to assess the role of case mix on the outcomes experienced by prostate carcinoma patients who were treated for cure. The challenge in the current study was to address the measurement of the comorbidity component of case mix in a group of patients who are generally healthy.5, 8–10 We assessed the predictive validity11 of 5 published comorbidity indices in this patient population with a view to identifying an index to use in the larger future study.

MATERIALS AND METHODS

The comorbidity indices we used included 1 general chronic disease index (the Chronic Disease Score12 [CDS]), 2 comprehensive comorbidity indices (the Index of Coexistent Disease13 [ICED] and the Cumulative Illness Rating Scale14 [CIRS]), and 2 indices that focus on those comorbidities most associated with survival (the Kaplan-Feinstein Index15 and the Charlson Index3). With the exception of the CDS, all of these indices have previously been used in studies of cancer survival.4, 10, 16–23 We included the CDS because it is based on drug-dispensing information only and therefore is quite easy to use. We hypothesized that the more comprehensive comorbidity indices (ICED and CIRS) would demonstrate higher predictive validity. We defined predictive validity as the ability of the scores to predict short-term, nonprostate carcinoma (i.e., other-cause) death.

Sampling and Follow-Up

The current study was a retrospective study that used a case-cohort design.24 We identified the study population using the Queen's Cancer Research Institute, Division of Cancer Care and Epidemiology (CCE) Cancer Database, which has been described elsewhere.25 Briefly, the CCE Cancer Database links a number of existing electronic databases that together contain information regarding the diagnoses, treatments, SES, and survival for all persons who were diagnosed with a cancer in Ontario, Canada. The data sources include the Ontario Cancer Registry's (OCR) records, the Canadian Institute for Health Information's (CIHI) hospital discharge information for individuals diagnosed with cancer, the Ontario cancer centers' radiotherapy records, and selected data from the Canadian census. Ethics approval for this research was granted by the Queen's University ethics review board.

Using the CCE database, we identified a cohort of 2384 men diagnosed from 1990-1995 who fit the following inclusion criteria: residence in either Metropolitan Toronto or the Southeast Cancer Care Ontario region (Southeast CCOR) at the time of diagnosis and curative treatment within 6 months of diagnosis using either surgery or radiotherapy. We targeted these regions because they represent the 2 types of cancer care delivery systems in Ontario. Toronto's system is very diffuse, whereas the Southeast CCOR has a more-centralized system, thereby providing information regarding the feasibility of data collection for the larger study. Cases were defined as cohort members who died from a cause other than prostate carcinoma by December 31, 1996. The death information in the database came directly from the cause of death listed on death certificates. Other-cause deaths included all International Classification of Disease (ICD-9) codes except 185, the code for prostate carcinoma. The total number of cases in the 2 regions was 120.

We identified a subcohort by taking a random sample from the cohort, stratified by region (Table 1). We also chose cases randomly from the pool of 120, stratifying again by region (i.e., Toronto region: n = 25 and Southeast CCOR region: n = 29). Using a 2-tailed significance level of .05, this sample size provided greater than 80% power to assess whether a given comorbidity index accounted for at least 5% of the variance in other-cause deaths.

Table 1. Case-Cohort Sampling: Characteristics of the Cases and Subcohort Study Subjects
Case and subcohort group characteristicsSoutheast CCORToronto
  • CCOR: Cancer Care Ontario region.

  • *

    To preserve the sampling regimen, the subjects who were not in the case sample yet experienced an other-cause death and were in the subcohort sample were censored at their time of death.

CasesDied of other causes2925
 Percentage sampled67.5%27.5%
SubcohortOverlap with the case sample51
 Censored other cause deaths*25
 Censored prostate carcinoma deaths41
 Alive at end of study period101113
 Total in the subcohort112120
 Percentage sampled17.8%6.5%

The time on study began when curative treatment was initiated and all covariates were measured at or before this zero time. As listed in Table 1, subjects were censored at their time of death when a prostate carcinoma death occurred (n = 5) or when a subcohort member, who was not in the case sample, experienced an other-cause death (n = 7). All remaining subjects were censored on December 31, 1996.

Study Variables

We used the CCE Cancer Database to obtain many of the data elements needed. Region of residence, defined as Southeast CCOR or Toronto, was assigned using the patients' county of residence at diagnosis. Age was captured directly from the CCE database. A derived SES variable in the CCE database that uses median household income from the census has been described elsewhere.25 This SES variable was categorized into 5 groups. A median household income of $20,000 or less characterized the first group, increments of $10,000 characterized the next 3 groups, and a median household income of $50,000 or more characterized the highest group. Vital status, date of death, and cause of death (i.e., prostate carcinoma or other) were also captured from the CCE database.

Two study abstractors augmented the data obtained from the CCE Cancer Database with information from medical charts. They collected data to confirm the cohort inclusion criteria and verify treatment information. They identified the presence and severity of morbidities relevant to scoring four of the comorbidity indices and they recorded the medications being taken at the time treatment was initiated for computation of the CDS index.

The 5 comorbidity indices were applied as indicated in earlier studies.3, 12, 13, 15 However, the Kaplan-Feinstein Index, which was developed for use in a diabetes population, did not contain a component to evaluate diabetes. We therefore added a diabetes component using the definition from the Charlson Index.3

Chart Management and Data Collection

We prioritized the multiple medical charts identified for each subject to help direct the efforts of the data abstractors. The cancer center chart was identified as the primary chart for patients who received radiotherapy because all radiotherapy in Ontario is provided by the province's 9 cancer centers. For patients who underwent prostatectomy, the treating hospital chart was identified as primary. At least 1 other medical chart was also consulted for each patient.

Two health care providers were trained as data abstractors and they collected information from medical charts over a 6-month period. A manual was provided that described the variables to be measured and they completed several practice sessions with debriefings on their performance. A reliability study was conducted during the data collection phase and interrater reliability estimated using the weighted kappa.26 Using the interpretation guide of Landis and Koch,27 the interrater reliability was substantial for the Charlson Index, the ICED, and the CIRS (categorized) and it was moderate for the Kaplan-Feinstein Index. Percent agreement for the CDS was 85.7% for identifying medications from the medical charts.

The abstractors collected comorbidity information from medical histories dated as far back as 5 years and up to the time treatment was given. For each medical condition, the chart information that indicated the greatest severity was used to determine a comorbidity score. When a clear indication of a comorbidity and its severity could not be assigned by the abstractors, 1 of the authors (D.B.) reviewed verbatim statements recorded from the charts to assign severity, defaulting to the less-severe level when the information was equivocal.

Analysis Methodology

The disease categories in the CIRS were used to describe the spectrum of comorbidity for the case and subcohort subjects. We compared the cause of death with the comorbid diseases identified for cases to assess how often comorbidities present at treatment were responsible for other-cause deaths.

Nonzero comorbidity proportions identified by CIRS categorizations for case and subcohort groups were compared using a Z-test for proportions. These tests were computed to give an indication of group differences and some low sample sizes in the case group were accepted (i.e., n < 30). Sampling fractions were applied when region data were combined.

Kaplan-Meier curves were generated for the covariates of age, SES, and region covariates, as well as for each comorbidity index. Because a case-cohort design was used, the probabilities were computed using sampling weights in order to generate Kaplan-Meier curves that were representative of the entire cohort. The sampling weights were defined as the inverse of the sampling fractions (i.e., 1.5 for cases and 5.6 for the subcohort in Southeast CCOR; 3.6 for cases and 15.4 for the subcohort in Toronto). Barlow28 described this method as a means for generating coefficient estimates in a Cox proportional hazards regression when subjects are selected using the case-cohort design, and it was adapted here for the generation of Kaplan-Meier curves.

Survival analysis techniques were used. Methods described by Self and Prentice29 were used to compute coefficient estimates, whereas the jackknife-based methods of Lin and Ying30, 31 were used to compute standard errors. The proportional hazards assumption was verified for the strata of each covariate using plots of the natural logarithm of the cumulative baseline hazard rates. When making a decision regarding the inclusion of covariates, we stated a priori that if the region and SES variables did not have statistically significant effect estimates (i.e., alpha = .05) and their sample-based proportion of variance explained (PVE) was < 5%, these variables would not be further assessed. However, we kept age in the assessment regardless of its significance because we were interested in the predictive ability of comorbidity over and above age. After a model was finalized for the covariates (i.e., age, SES, and region), each comorbidity index was added separately to this model to compute the change in the model's prognostic ability attributable to the index.

The proportion of variance in the other-cause death outcomes explained (PVE) by each of the 5 models was computed as: equation image which was derived from the likelihood ratio computed for a model (LR) and the total sample size, (n).32 Partial PVE estimates were computed for the comorbidity indices by first calculating the PVE for a model with only the comorbidity variable removed and then subtracting this from the PVE associated with a model containing the same variables plus the comorbidity variable.

Bootstrap techniques were used to estimate partial PVEs with associated standard errors.33 A standard Z-test was used to assess statistical significance of the transformed partial PVEs. The transformation was an arcsin of the square root of the partial PVE, which, as recommended by Schemper,33 created a distribution closer to normal. The bootstrap sampling strategy used was designed to reflect the case-cohort study design.34

To compare partial PVE estimates for the comorbidity indices, we used the bootstrap-based partial PVE estimates and their associated variances and covariances. A 2-sample Z-test was computed on the transformed partial PVE estimates.33 In addition, a correction for multiple comparisons was computed using the sequentially rejective Bonferroni test with an alpha of .05.35, 36

As a secondary analysis, when the relative risk effect estimates for sequential scores were similar they were grouped and the relative risks recalculated to provide a coding scheme for future use with the comorbidity indices in this population.

Survival analysis and bootstrap methods were implemented with SAS software (SAS Institute, Inc., Cary, NC)37 using core SAS coding for the case-cohort design discussed by Therneau and Li.38

RESULTS

Study Subjects

Sampling resulted in a subcohort consisting of 228 men and a case group of 59. Of these 287 individuals, 11 were excluded from the study; 3 did not meet the study inclusion criteria, we were unable to obtain primary charts for 7, and comorbidity information was not available for 1 subject. Table 2 presents the covariate distributions for the final study population by case and subcohort status by region. The Toronto cases were older, with a mean age of 69.5 years. Subjects' ages ranged from 48 years to 82 years. In Toronto, the subjects' area-level SES was weighted toward the higher categories, whereas subjects from the Southeast CCOR area exhibited an SES weighted toward the middle categories.

Table 2. Covariate Summary by Region and Case/Subcohort Status
Covariates Southeast CCORToronto
Case* (n = 27)Subcohort (n = 112)Case* (n = 22)Subcohort (n = 114)
  • CCOR: Cancer Care Ontario region; SD: standard deviation; SES: socioeconomic status.

  • *

    Members of the subcohort who also were in the case sample (see Table 1) are included in both groups.

Age, yMean (SD)66.5 (5.37)66.5 (5.27)69.5 (5.91)66.1 (6.52)
SES category≤ $20K1 (3.7%)7 (6.3%)3 (11.1%)5 (4.5%)
 > $20K to $30K8 (29.6%)19 (17.0%)2 (7.4%)4 (3.6%)
 > $30K to $40K13 (48.1%)44 (39.3%)1 (3.7%)23 (20.5%)
 > $40K to $50K4 (14.8%)24 (21.4%)6 (22.2%)36 (32.1%)
 $50K or more1 (3.7%)18 (16.1%)10 (37.0%)46 (41.1%)

Survival and Covariate Effects

Kaplan-Meier survival curves were generated for age tertiles (< 66 yrs, 66-70 yrs, and > 70 yrs), SES levels, and regions. Age and region demonstrated some separation between strata, but the SES strata exhibited much overlap. Statistical differences between the covariate strata were assessed using the Cox proportional hazards model. Each yearly increment in subject age was associated with a corresponding 7% increase in risk of experiencing an other-cause death but this effect was found to be only marginally significant (Wald chi-square = 2.91; degrees of freedom [df] = 1 [P = .09]). Age was also assessed in categoric form using tertiles, but both the level of significance and sampled-based PVE were lower compared with the Cox proportional hazards model that substituted age in continuous form.

Subjects from Toronto were 1.16 times more likely to experience an other-cause death, relative to the Southeast CCOR region, but this effect was not significant (Wald chi-square = 0.07; df = 1; [P = .80]). The relative risk of other-cause death was 1.3 for each decrease in SES, but this linear trend was found to be only marginally significant (Wald chi-square = 3.06; df = 1 [P = .08]). However, the first 2 SES levels and the 3 higher SES levels formed 2 distinct risk groupings, in which subjects in the lower grouping were 2.4 times more likely to die from an other-cause death (Wald chi-square = 5.41; df = 1 [P = .02]). The sample-based PVE estimate was 4.3% for a model that included the region, age, and the categorized SES variable. Because this estimate was below our criteria of 5%, SES and region were excluded from the Cox proportional hazards models that evaluated the comorbidity indices. However, as specified a priori, age was forced into the models so that the incremental contribution of comorbidity could be assessed.

Comorbidity of the Subjects

The subjects' comorbidity, classified using the CIRS disease categories, was summarized for the case and subcohort groups and Z-test results are provided as a guide to highlight large differences (Table 3). The cardiac category, which included myocardial infarctions, atherosclerotic heart disease, congestive heart failure, and arrhythmias, was overrepresented in the cases, with the difference in the Southeast CCOR reaching statistical significance. Vascular conditions, which included hypertension, peripheral atherosclerotic disease, and aortic aneurysms, were also overrepresented in the cases relative to the subcohort of both regions, but only reached statistical significance when both regions were combined. Lower gastrointestinal comorbidities, which included constipation, intestinal bleeding, diverticular disease, hemorrhoids, hernias, and bowel cancer, were higher in the cases residing in the Southeast CCOR area. Endocrine and metabolic comorbidities, which included diabetes mellitus, hormone replacement, and electrolyte disturbances as well as obesity, were overrepresented in the case group compared with the subcohort group, but only reached marginal statistical significance when regions were combined.

Table 3. Comorbidities Using the CIRS Categorizations Tabulated by Both Sampled Area and Case/Subcohort Status
CIRS categoryPercent of subjects with an illness in the CIRS category*
Southeast CCORTorontoBoth regions
Case (n = 27)Subcohort (n = 112)Case (n = 22)Subcohort (n = 114)Case (n = 49)Subcohort (n = 226)
  • CIRS: Cumulative Illness Rating Scale; CCOR: Cancer Care Ontario region; GI: gastrointestinal.

  • *

    Z-tests were not conducted when a zero proportion was observed.

  • Sampling fraction incorporated when combining region data.

  • Significant at the .01 level or lower, using a Z-test comparing proportions.

  • §

    Significant at the .05 level or lower, using a Z-test comparing proportions.

  • Significant at the .10 level, using a Z-test comparing proportions.

Cardiac problems63.023.240.925.448.324.8
Vascular48.131.340.934.243.333.4§
Respiratory59.361.636.432.544.240.1
Musculoskeletal29.626.818.220.221.721.9
Lower GI63.023.218.225.433.324.8
Upper GI11.115.24.58.86.710.4
Genitourinary0.00.90.02.60.02.2
Liver18.510.718.214.018.313.2
Renal0.06.34.54.43.34.9
Endocrine/metabolic29.621.418.212.321.714.7
Hemapoietic7.40.90.00.92.50.9
Neurological18.57.14.57.99.27.7
Psychiatric3.77.10.01.81.73.2
Eyes/ears/nose/throat11.19.84.55.36.76.5

There was 1 unusual item noted with renal comorbidities, which included kidney function problems. In the Southeast CCOR area, 0% of cases versus 6.3% of the subcohort had renal comorbidities, whereas there was little disparity noted with the Toronto area subjects.

Similar to the pattern for comorbidities, cardiac and vascular disease were common causes of death. Of the 49 other-cause deaths, 49% were attributed to cardiac or vascular causes; ischemic heart disease in 20 cases, and cerebrovascular disease in 4 cases. The remaining 25 cases died of malignancies of the pancreas, lung, colon, intestine, and heart or an unspecified malignancy or leukemia (n = 13); diabetes (n = 1); chronic airway obstruction (n = 2); alveolar capillary block (n = 1); disorder of iron metabolism (n = 1); pneumonia (n = 1); spondylitis (n = 1); urinary tract infection (n = 1); accidental fall (n = 1); unknown causes (n = 1); and unspecified disorder of the prostate (n = 2). These latter 2 subjects died shortly after their surgery.

We assessed the concordance of listed cause of death with comorbidity identified at diagnosis for each index. The CIRS captured 59.2% of the cases' listed cause of death as an identifiable comorbidity, ICED captured 53.1%, the Kaplan-Feinstein Index captured 44.9%, and both the Charlson Index and the CDS captured 36.7% each.

Comorbidity Indices: Summary Scores and Survival

The distribution of comorbidity scores varied between the case and subcohort groups (Fig. 1), with higher scores in the case group. This pattern was observed for all the indices to differing degrees. In both groups, the number of subjects with low and zero scores was quite high.

Figure 1.

Comorbidity score distributions between case and subcohort groups by index. ICED: Index of Coexistent Disease; CDS: Chronic Disease Score; CIRS: Cumulative Illness Rating Scale.

The CDS, ICED, and, to some degree, the Kaplan-Feinstein Index were able to identify groups with differing survival probabilities (Fig. 2), whereas the CIRS and Charlson Index performed less well (Fig. 2). Some clustering among the groupings was evident for each comorbidity index. Because of the case-cohort design we used the Cox proportional hazards model to test whether the index scores statistically significantly differentiated subject risk groups.

Figure 2.

Kaplan-Meier survival probabilities for the study subjects, using the scores for each comorbidity index to define strata. ICED: Index of Coexistent Disease; CDS: Chronic Disease Score; CIRS: Cumulative Illness Rating Scale.

Comorbidity Indices: Cox Proportional Hazards Model Results

The category-specific effects of each comorbidity index are presented in Table 4. A linear increase in relative risk was not evident for any of the indices and only the Kaplan-Feinstein Index demonstrated monotonic increases across the range of scores. Some of these inconsistencies are likely due to small numbers in some of the categories. Yearly increments in age were associated with a 3% to 6% increase in other-cause death when a model was fitted with age and each comorbidity index, but the effect was not found to be statistically significant with any of the models (Wald chi-square range, 0.56-2.8; df = 1 [P-value range, .09-.45] (results excluded from Table 4).

Table 4. Relative Risk Estimates* for Each Comorbidity Index
ModelVariableValueRelative Risk*Lower 95% CIUpper 95% CI
  • 95% CI: 95% confidence interval; ICED: Index of Coexistent Disease; CDS: Chronic Disease Score; CIRS: Cumulative Illness Rating Scale.

  • *

    Controlling for age.

  • Significant at the .05 level.

  • Significant at the .01 level or lower.

1Charlson index01.00
  11.410.643.08
  23.051.068.75
  31.350.1017.63
  42.130.4510.15
2ICED01.00
  10.950.392.36
  24.291.5311.98
  33.961.1813.27
3Kaplan-Feinstein index01.00
  11.030.492.14
  21.640.574.72
  35.541.5320.04
4CDS0 or 11.00
  21.130.343.73
  31.770.545.83
  42.350.5210.52
  59.143.2925.37
  6+3.481.0911.06
5CIRS01.00
  11.140.235.71
  20.840.164.34
  31.460.356.19
  40.820.164.32
  51.450.287.45
  62.050.3213.10
  73.070.5716.55
  81.570.259.66
  9+2.560.5112.82

For future use, the scores were regrouped to create groupings that might have distinct risks and the relative risk estimates were recalculated (Table 5).

Table 5. Relative Risk Estimates for the Grouped Comorbidity Scores
ModelVariableValueRelative Risk*Lower 95% CIUpper 95% CI
  • 95% CI: 95% confidence interval; ICED: Index of Coexistent Disease; CDS: Chronic Disease Score; CIRS: Cumulative Illness Rating Scale.

  • *

    Controlling for age.

  • Significant at the .01 level or lower.

6Charlson index01.00
  11.420.653.09
  2, 3, or 42.600.996.79
7ICED0 or 11.00
  2 or 34.312.069.01
8Kaplan-Feinstein index0 or 11.00
  21.620.624.25
  35.461.6218.39
9CDS0 or 11.00
  21.150.353.79
  3 or 42.000.765.30
  5+6.022.6213.86
10CIRS0, 1, 2, 3 or 41.00
  5 or 61.490.583.81
  7+2.220.994.98

Comorbidity Indices: PVE Estimates from the Proportional Hazards Model

Bootstrap-based PVE estimates were generated for age and each of the comorbidity index variables using the categories initially defined. By itself, age accounted for 2.0% of the variability in other-cause deaths (Table 6), which was significantly different from zero (Z = 2.04; P = .04).

Table 6. Bootstrap-Based Percent Variance Explained Estimates for Age and the Comorbidity Indices
Variable*Bootstrap NPVELower 95% CIUpper 95% CI
  • PVE: percent variance explained; 95% CI: 95% confidence interval; ICED: Index of Coexistent Disease; CDS: Chronic Disease Score; CIRS: Cumulative Illness Rating Scale.

  • *

    The PVE for age is a marginal PVE whereas the PVE for the Comorbidity Indices is a partial PVE (i.e., the PVE for the index, after the PVE for age has been removed).

Age1502.00.0037.58
Charlson index1503.80.3010.91
ICED1509.02.9417.85
Kaplan-Feinstein index1504.90.6212.78
CDS15011.33.5122.83
CIRS1507.21.3517.09

All the indices studied, the Charlson Index (Z = 2.72; P = .007), ICED (Z = 4.52; P < .001), Kaplan-Feinstein Index (Z = 3.04; P = .002), CDS (Z = 4.35; P <.001), and the CIRS (Z = 3.44; P <.001), had a bootstrap-based partial PVE estimate that was significantly different from zero (Table 6). The CDS resulted in the largest PVE estimate followed by the ICED, CIRS, Kaplan-Feinstein Index, and Charlson Index, respectively. When age was modeled alone, its PVE was found to be lower than the partial PVE estimates of any of the indices, and the differences between age and the partial PVE for CDS were close to reaching statistical significance (Sequentially Rejective Bonferroni test, Step 1: Z = 1.92; P = .05). The differences among the partial PVE estimates for the comorbidity indices were also statistically nonsignificant (Sequentially Rejective Bonferroni test, Step 1: Z = 1.41; P = .13).

DISCUSSION

The risk of a short-term other-cause carcinoma death was associated with higher comorbidity status because each comorbidity index accounted for a statistically significant proportion of the variance in this outcome. Generally, this relation was not linear, possibly due to small numbers in the higher index score groups. However, the post hoc grouping of relative risk estimates for each comorbidity index provides groupings that increase monotonically and are statistically significant. These groupings could be of use in future studies of this patient population.

We are aware of only 1 previous study that is closely related to the present work. Albertson et al.4 compared the ICED, Charlson Index, and Kaplan-Feinstein Index in a study of survival in prostate carcinoma patients with localized disease at presentation. These investigators found that all 3 comorbidity indices were predictive of both prostate carcinoma death and other-cause death.4 Our findings are from a more restrictive cohort whose members, through treatment selection processes, generally had less severe comorbidities. However, even with this more restrictive cohort we found that comorbidity indices were still predictive of other-cause death.

Clinicians use patient age to assess life expectancy when making a recommendation regarding treatment.10, 39 The low observed PVE estimate for the age variable (i.e., 2%) may be explained by such a consideration having already occurred because these patients were treated with a curative intent. The extra variability in the outcome explained by comorbidity illustrates the impact of comorbid illnesses over and above age, and underlines the importance of considering comorbidity in treatment decisions and when conducting studies of prostate carcinoma patient outcomes.

The Kaplan-Feinstein Index and Charlson Index were both created to be predictive of short-term, generally more severe outcomes, including death,3, 15 and they focus on more severe comorbid illnesses. The other indices we studied, the CDS, ICED, and CIRS, were meant to be predictive of a wider range of outcomes12–14, 40, 41 and, as such, they capture less severe as well as more severe comorbid conditions. The Kaplan-Feinstein Index and the Charlson Index had the lowest observed PVE estimates (both <5%) compared with the CDS, ICED, and CIRS, which were all >7%, indicating that a comprehensive consideration of comorbidity may be needed in this setting.

The CDS was the easiest to apply, requiring only the identification of the relevant medications and then matching these to the relevant disease categories. At the end of the study, the chart abstractors ranked the remaining 4 indices on ease of use and clarity of instructions. Both agreed that the CIRS was easiest to use and its instructions were the clearest. Their ranking placed the ICED second best, followed by the Kaplan-Feinstein Index and the Charlson Index, respectively. It is interesting to note that this ranking is close to the ranking of both the PVE estimates and the interrater reliability estimates.

Because we were interested in other-cause deaths, only parameters influencing this outcome were considered in our analyses.8 Specifically, we did not consider prostate carcinoma prognostic factors, such as prostate-specific antigen (PSA), tumor stage, and Gleason score. However, these variables would have been considered when deciding on treatment options; patients with disease that had likely extended beyond the prostate capsule would not have been recommended for curative treatment. Five prostate carcinoma deaths occurred in the study group. The risk of this outcome was low because of the case selection just described and the short study timeline.

Perhaps not surprisingly, cardiac comorbidities also were most often the cause of death and the distribution of death causes was consistent with what is reported for the general Canadian population.42 In general, comorbidity levels were low, with a large proportion of the cases having zero comorbidity scores. It is likely that some deaths occurred as a result of comorbid illnesses that were not recorded in the medical charts.

A study of deceased members of a California medical care program who previously had prostate carcinoma also found that cardiovascular disease was the most common cause of death.43 These investigators also assigned comorbidity status using discharge abstracts up to 5 years before the prostate carcinoma diagnosis. They found that after controlling for age, race, and prostate carcinoma stage, only the presence of a cardiovascular comorbidity in subjects was statistically significantly associated with a other-cause death.

It has been noted that discharge summaries underidentify comorbidities44 and are more likely to capture cardiovascular conditions relative to other conditions. This may also have been the issue in the medical histories used in the current study, but we did make every effort to use full medical histories, which are more complete than discharge diagnoses.

The results of the current study demonstrated that comorbidity indices designed for use in a medical chart review are capable of explaining a statistically significant proportion of the variance in short-term other-cause death outcomes in curatively treated prostate carcinoma patients. Although this study did not have the statistical power to differentiate between these results, the 3 more general comorbidity indices—the CDS, CIRS, and ICED—appear to have performed better than the Kaplan-Feinstein and Charlson indices. We conclude that any of these 3 are likely to be useful in measuring and controlling for comorbidity differences in this setting. The CDS was the easiest index to apply and its application explained the most variance in the outcome. Our findings emphasize the importance of considering comorbid illnesses in addition to age when making a decision regarding treatment for prostate carcinoma and when studying prostate carcinoma outcomes. Larger future studies comparing these indices will refine our understanding of their relative usefulness in considering comorbid illness in the analysis of prostate carcinoma outcomes.

Acknowledgements

The authors thank the Ontario Cancer Registry and Cancer Care Ontario for providing the Ontario registry data used in this study and Doreen Ulrichsen and Heidi Emery for diligent work in abstracting data from patient charts.

Ancillary