Predictive Ability of Pretransplant Comorbidities to Predict Long-Term Graft Loss and Death

Authors


* Corresponding author: Mark A. Schnitzler, schnitm@slu.edu

Abstract

Whether to include additional comorbidities beyond diabetes in future kidney allocation schemes is controversial. We investigated the predictive ability of multiple pretransplant comorbidities for graft and patient survival. We included first-kidney transplant deceased donor recipients if Medicare was the primary payer for at least one year pretransplant. We extracted pretransplant comorbidities from Medicare claims with the Clinical Classifications Software (CCS), Charlson and Elixhauser comorbidities and used Cox regressions for graft loss, death with function (DWF) and death. Four models were compared: (1) Organ Procurement Transplant Network (OPTN) recipient and donor factors, (2) OPTN + CCS, (3) OPTN + Charlson and (4) OPTN + Elixhauser. Patients were censored at 9 years or loss to follow-up. Predictive performance was evaluated with the c-statistic.

We examined 25 270 transplants between 1995 and 2002. For graft loss, the predictive value of all models was statistically and practically similar (Model 1: 0.61 [0.60 0.62], Model 2: 0.63 [0.62 0.64], Models 3 and 4: 0.62 [0.61 0.63]). For DWF and death, performance improved to 0.70 and was slightly better with the CCS.

Pretransplant comorbidities derived from administrative claims did not identify factors not collected on OPTN that had a significant impact on graft outcome predictions. This has important implications for the revisions to the kidney allocation scheme.

Introduction

The life expectancy gained with transplantation is comparable to the benefit derived from the elimination of perinatal conditions, congenital anomalies or other public health problems (1). The potential of transplantation to generate even more years of life for society is limited by an insufficient offer of deceased donor organs for which living donation does not compensate (2). Allocation policies determine the ability of transplantation to generate years of life from organs retrieved from deceased donors. Allocation policies strive to balance ‘equality’ of access (e.g. by offering equal opportunity to access transplantation to all candidates) and ‘utility’, that is, producing the maximum health benefit to society from the transplanted organs (3).

The current allocation scheme for kidneys favors equality over utility and emphasizes duration of waiting time, minimization of HLA-DR mismatches, allocation to those with a history of HLA sensitization but a negative HLA cross-match with the donor, pediatric recipients and those willing to accept extended criteria donors (ECD) (2). The Kidney Allocation Review Subcommittee (KARS) of the United Network for Organ Sharing (UNOS) is revising the current organ allocation schema to optimize the net survival benefit of transplantation, increasing utility subject to equity constraints (3). Lack of predictability of patient survival is one of the challenges recognized by the KARS. This may be in part related to the lack of inclusion of pretransplant comorbidities except for a history of diabetes in the recipient. The current proposed KARS allocation system relies heavily on age of the recipient and the presence of diabetes without consideration of other comorbid complications. The Scientific Registry of Transplant Recipients (SRTR) risk-adjustment models (4) explored the predictive ability of the Organ Procurement Transplant Network (OPTN) recorded comorbidites and did not found those to improve the predictive ability of graft loss, while previous malignancies was a useful addition in predictive models of patient death. Whether other comorbidities should be considered in a revised kidney allocation schema is unknown.

Medical comorbidities predict outcomes in many fields of medicine (5). In kidney transplantation, pretransplant comorbidities such as atherosclerosis (6,7), obesity (6,7), diabetes (6,7) and tobacco use (8,9) negatively influence transplant outcomes. In a Canadian registry, the Charlson Comorbidity Index was the best predictor of patient and graft survival (10,11). In a US study, a Charlson comorbidity index above 5 strongly correlated with poor patient survival (10,11). A report using the OPTN database showed that patient survival is associated with cardiovascular conditions, diabetes and history of malignancy (12).

Whether data collected by the OPTN is sufficient to comprehensively address organ allocation questions is not known. The United States Renal Data System (USRDS) relies on Medicare administrative claims to report primary and secondary diagnosis for hospitalizations and medical visits. The comorbidity information contained in the USRDS medical billing claims can be used as a comprehensive data source. The International Classification of Diseases, 9th revision, clinical modification (ICD-9-CM) system used in billing systems contains 14 000 codes that indicate diagnoses, medical procedures, functional status and other elements (5). Summary measures that reduce the complexity of the ICD-9-CM information to manageable and clinically meaningful categories are available. The objective of this study is to examine whether evidence provided by administrative claims might supplement data collected on pretransplant OPTN survey forms to provide information pertaining to comorbidities which have an impact on subsequent kidney allograft outcome. To achieve this aim, we explored the predictive ability of a wide array of pretransplant comorbidities regarding posttransplant graft loss and death using USRDS data and three diagnosis classification systems.

Patients and Methods

Study population

This was a retrospective cohort study of the USRDS registry. All deceased-donor, first renal transplant only, adult (>18 yrs.) recipients transplanted between 1995–2002 and reported to the OPTN with Medicare as the primary insurance carrier at time of transplant were included. Missing data for donor age were imputed with mean donor age, while recipient and donor body mass index (BMI) were imputed with the median BMI. Extreme values in BMI (smaller than 15 and greater than 45) were assigned to 15 and 45. No other data imputation was performed and patients with other missing data were excluded from the analysis. As a sensitivity analysis, models with a missing value category for donor age and donor and recipient BMI were calculated, rather than using mean or median imputation. To allow an appropriate time frame to ascertain pretransplant diagnostic claims, patients were included if they had at least one year of pretransplant follow up where Medicare was continuously the primary payor.

Comorbidity information from claims

Comorbidity information from claims was identified using the Medicare Part-A and Part-B claims files. Any primary or any secondary diagnoses ICD-9-CM codes for each hospitalization or physician visit during the last pretransplant year were considered.

Comorbidity summary measures

Three different systems were used to summarize the comorbidities captured in the ICD-9 codes. For the primary analysis, we used a system based upon ICD-9-CM codes developed by the Agency of Healthcare Research and Quality (AHRQ) called the Clinical Classifications Software (CCS) (13,14). AHRQ uses the CCS system to summarize their Health Care Utilization Project (HCUP) data and to provide descriptions of healthcare utilization in the US (15,16). The single-level CCS classifies ICD-9-CM diagnoses and conditions into 261 clinically meaningful categories. Two hundred and fifty-three single-level CCS categories were used. Eight CCS categories were excluded because they reflected process of care or very atypical codes, such as routine medical visit or accident (CCS 0, 254–260).

We also considered the comorbidities used in the Charlson (17) and Elixhauser (18) comorbidity scores. Both are widely used measures of comorbidity in predictive models (5). The Charlson index considers 19 conditions; while the Elixhauser index accounts for 31 comorbidities. Adaptations of both indices for claims databases were used (19).

Other study variables

Usual variables captured by the OPTN were included in the models, among them patient and donor age and race, number of ABDR HLA mismatches (categorized into none, AB mismatches, DR mismatches), cause of ESRD, OPTN recipient and donor comorbidities, CMV donor and recipient status, obesity, peak panel reactive antibodies (PRA) > = 50% and dialysis duration. Interactions between age and cause of ESRD and between categories of ABDR HLA mismatches and race were also included in all models. Delayed graft function (DGF), cold ischemia time and immunosuppressant therapy at discharge were excluded because the goal was to develop predictive models using factors occurring before the kidney was allocated.

Statistical analyses

Cox regression models were used to investigate 3 different outcomes: all cause graft loss, death with function (DWF) and death (including posttransplant cases). Patients were censored at loss of follow-up, graft loss or 9-years posttransplant, whichever occurred first. Nine years was chosen because it was the maximum length of follow-up.

Four models per each endpoint were developed for each endpoint: (a) Model 1—OPTN variables (OPTN only), (b) Model 2—CCS disease categories plus the OPTN variables (OPTN + CCS), (c) Model 3—Charlson comorbidities plus OPTN variables and (d) Model 4—Elixhauser comorbidities plus OPTN. In order to maximize their predictive power, continuous variables (recipient and donor age, recipient and donor BMI and time on dialysis) were not categorized. To account for potential non-linear effects in those continuous variables, restricted cubic splines were considered (20). Restricted cubic splines are functions that allow incorporating nonlinear effects in regression models, with changing slopes and improved model fit.

Predictive ability can be overestimated if the number of independent variables is too large, a problem known as overfitting (20). To estimate overfitting, we used a heuristic shrinkage indicator that relates the model chi-square fit to the number of variables (21). If the shrinkage index falls below 0.9, there is a danger of overfitting. The number of variables was adequate for the OPTN models. Models incorporating all CCS variables were too large and data reduction strategies were used (20). CCS variables with prevalence below 1 per 100 patients were eliminated. Variable clustering was used to summarize the remaining 130 CCS variables into a smaller number of variables. The CCS model for all-cause graft loss used 106 CCS clusters (Table S2). CCS interactions could not be fitted because of their large number. Interactions between Charlson and Elixhauser comorbidities and recipient age could be modeled. Models for DWF and death used all CCS variables with prevalence > = 1/1000 (192 variables) (Table 1).

Table 1.  Models degrees of freedom and expected shrinkage
ModelTotal dfChi-squareExpected shrinkage
  1. The shrinkage indicator shows how much of the model fit is due to modeled effects rather than fitting random noise in the model. A shrinkage value below 0.9 indicates that an excessive amount of predictive ability is specific to the dataset being analyzed and unlikely to be replicated in future observations. The shrinkage indicator is calculated as Model Chi-square/(Model Chi-square − Total df).

All cause graft loss
 All CCS with frequency >03121909.810.84
 All CCS with prevalence > = 1/1001971770.120.89
 All CCS with prevalence > = 1/100 grouped in 106 clusters1731737.250.90
 OPTN 671432.980.95
 Elixhauser1601694.2 0.91
 Charlson1181639.140.93
DWF
 All CCS with frequency >031221220.85
 All CCS with prevalence > = 1/1001972011.790.90
 OPTN 671670.180.96
 Elixhauser1601951.220.92
 Charlson1181892.410.94
Death
 All CCS with frequency >03122677.970.88
 All CCS with prevalence > = 1/10002592628.5 0.90
 OPTN 672103.460.97
 Elixhauser1602504.390.94
 Charlson1182415.870.95

Variable selection such as stepwise methods with p-values less than 0.05 when testing multiple variables can impair predictive ability and was not performed. Wald statistical tests were performed to test for blocks of interaction terms, comorbidities and linearity in the continuous variables. If the p-value for the test was <0.3, all the terms in the block were retained for further modeling, following recommended strategies (20). In sensitivity analyses, comorbidity variables in significant blocks and having p-values from the individual Wald test at or below 0.05 were retained.

The proportional hazards assumption was tested examining the time trend in the Schoenfeld residuals (22). A variable was considered to violate the proportional hazards assumption if the p-value of the proportional hazard test was below 0.05 divided by the number of proportional hazards test in the model. Cox models were stratified in those variables.

Predictive accuracy was calculated using calibration and discrimination measures. Calibration was determined by comparing the model-predicted survival with the Kaplan–Meier actual survival, grouping the predicted survival times by deciles. Discrimination examines whether model predictions correctly classify pairs of subjects based on actual outcome and was assessed with the Harrel's c-statistic (20,23). This statistic is a generalization of the area under the receiving operating characteristic (ROC) curve (used for logistic regression models) adapted to censored data. In logistic regression models, discrimination compares pairs of patients with and without occurrence of an event (such as death) and determines whether the patient that died had a higher probability of death as calculated by the logistic model. In Cox models, the Harrel's c-statistic indicates the proportion of patient-pairs where the model correctly predicted which patient would die (or who died first if both cases died). A c-statistic of 0.5 indicates that predictions are no better than random, while a value of 1 indicates perfect predictions and values close or above 0.7 may be considered clinically important (24). The c-statistic is routinely used in the evaluation of predictive models (24–26).

Model calibration or discrimination derived from a sample can be overly optimistic in the sense that it may overestimate model performance in future assessments. Methods to obtain more realistic (bias-corrected) calibration and discrimination statistics include cross-validation and bootstrapping. Bias-corrected calibration is more critical when models are developed in small datasets. Given the large sample size in these models, it was not expected that the discrimination and calibration statistics would be overly optimistic. In order to test this assumption, we used 100 bootstrap replications to obtain bias-corrected calibration and discrimination statistic for the CCS models for all cause graft failure, death with function and death. Bootstrapping was not performed in the rest of the models given it was computationally expensive with our large dataset and due to the secondary nature of this sensitivity analysis.

All analyses were performed in SAS 9.1 (Cary, NC).

Results

Patient population and comorbidities

The total population consisted of 27 177 patients who met initial study criteria. Of these, 3261 patients (12.9%) had donor age data that were imputed with the mean donor age. Missing data for recipient BMI (n = 849, 3.4%) and donor BMI (n = 894, 3.5%) were imputed with the median BMI. Thus, 25 270 patients remained after excluding patients with missing data in other variables. During the 9-year time frame, 7791 graft losses, 3284 DWF and 4689 deaths (including deaths after returning to dialysis) were observed. Most notable differences between patients with and without graft loss included recipient race, hypertension, donor age and cause of death, number of DR mismatches, use of induction and steroids (all p < 0.0001, Table 2). The mean number of comorbidities was similar between patients with and without graft failure (Table 2). The mean number of comorbidities increased according to the age groups. The 18–29 year old group had 8.37 CCS comorbidities, while the figures for the 30–44, 45–59 and 60+ groups were 9.24, 9.45 and 10.22, respectively. The mean number of Charlson comorbidities in each age group was 1.24, 1.52, 1.65 and 1.86, respectively; and the mean number of Elixhauser comorbidities was 2.78, 3.13, 3.21 and 3.51, respectively. The p-values for these differences varied. The most frequent CCS comorbidities included hypertension with complications and secondary hypertension (n = 11 738, 46.4%), iron deficiency and other anemia (n = 10 297, 40.7%) and essential hypertension (n = 9577, 37.9%) (Table S1).

Table 2.  Patient, donor and transplant characteristics of the study population
VariableNo graft loss (n = 17 479)Graft loss (n = 7791)p-ValueVariableNo graft loss (n = 17 479)Graft loss (n = 7791)p-Value
Recipient factors   Donor factors   
Age   Age  <0.0001
 18–291520 (8.7) 722 (9.3)<0.0001 <182736 (15.6)1085 (13.9) 
 30–444900 (28.0)1940 (24.9)  18–293572 (20.4)1279 (16.4) 
 45–596969 (39.6)2933 (37.6)  30–443732 (21.3)1688 (21.7) 
 60 +4090 (23.4)2196 (28.2)  45–593754 (21.5)2107 (27)  
 60 +1109 (6.3)  947 (12.2) 
 Unknown2576 (14.7)685 (8.8) 
Female gender  0.049Female gender  <0.0001
 No10 621 (60.8)  4836 (62.1)  No10 259 (60.8) 4196 (17.1) 
 Yes6858 (39.2)2955 (37.9)  Yes6619 (39.2)3455 (45.2) 
Racial background  <0.0001Racial background  <0.0001
 Black5659 (32.4)3194 (4)    Black1996 (11.4)1076 (13.8) 
 Other10 575 (60.5)  4203 (53.9)  Other14 373 (82.2) 6383 (81.9) 
 White1245 (7.1) 394 (5.1)  White1110 (6.3) 332 (4.3) 
Hispanic origin  <0.0001Hispanic origin  <0.0001
 No14 997 (85.8)  7003 (89.9)  No15 487 (88.6) 7039 (90.3) 
 Yes2482 (14.2) 788 (10.1)  Yes1992 (11.4)752 (9.6) 
Body mass index   Body mass index   
 Normal7614 (43.5)3213 (41.2)0.001 Normal9004 (51.5)3899 (50) <0.0001
 Overweight5512 (31.5)2487 (31.9)  Overweight4564 (26.1)2029 (26)  
 Obsese3792 (21.7)1803 (23.1)  Obsese3217 (18.4)1663 (21.3) 
 Unknown561 (3.2)288 (3.7)  Unknown694 (4) 200 (2.6) 
Limits  0.0003Donor cause of death  <0.0001
 No16 079 (92)   7059 (90.6)  Anoxia1645 (9.4) 678 (8.7) 
 Yes1400 (8)   732 (9.4)  Cerebrovascular/stroke6305 (36.1)3603 (46.2) 
Primary cause of ESRD  <0.0001 Head trauma8404 (48.1)3180 (40.8) 
 Type I diabetes2044 (11.7) 926 (11.9)  CNS tumor174 (1) 76 (1) 
 Type II diabetes2537 (14.5)1297 (16.6)  Other951 (5.4)254 (3.3) 
 Hypertension4227 (24.2)2173 (27.9)     
 PKD1421 (8.1) 426 (5.5)     
 Glomeruloneprhitis3414 (19.5)1402 (18)   Donor comorbidities   
 Other1633 (9.3) 654 (8.4) Hypertension  <0.0001
 Unknown2203 (12.6) 913 (11.7)  No14 549 (83.2) 5925 (76)  
Pretx dialysis duration    Yes2930 (16.8)1866 (23.9) 
 12–24 months3038 (17.4)1483 (19) 0.001Diabetes  <0.0001
 24–60 months10 229 (58.5)  4550 (58.4)  No16 934 (96.9) 7472 (95.9) 
 >60 months4212 (24.1)1758 (22.6)  Yes545 (3.1)319 (4.1) 
Alcohol abuse  0.0005
 No14 402 (82.4) 6277 (80.6) 
OPTN recorded    Yes3077 (17.6)1514 (19.4) 
 comorbidities
 Angina  0.0004Cigarette use  <0.0001
 No15 325 (87.7)  6705 (86.1)  No11 240 (64.3) 4585 (58.8) 
 Yes2154 (12.3)1086 (13.9)  Yes6239 (35.7)3206 (41.1) 
Arryhtmia  0.708Drug use  <0.0001
 No17 348 (99.2)  7736 (99.3)  No14 204 (81.3) 6617 (84.9) 
 Yes131 (0.7) 55 (0.7)  Yes3275 (18.7)1174 (15.1) 
Transplant characteristics   
Congestive heart failure  0.0056PRA> = 50%  0.0004
 No16 255 (93)   7319 (93.9)  No16 579 (94.8) 7304 (93.7) 
 Yes1224 (7)   472 (6.1)  Yes900 (5.1)487 (6.2) 
HLA mismatches  <0.0001
COPD  0.1777 01517 (8.7) 483 (6.2) 
 No17 191 (98.3)  7644 (98.1)  13984 (22.8)1686 (21.6) 
 Yes288 (1.6)147 (1.9)  211 978 (68.5) 5622 (72.2) 
Cerebrovascular disease  0.132Cold ischemia time  <0.0001
 No17 072 (97.7) 7585 (97.4)  0–12 h1173 (6.7) 406 (5.2) 
 Yes407 (2.3)206 (2.6)  13–24 h4487 (25.7)1953 (25.1) 
 23–30 h1663 (9.5)  874 (11.2) 
Hypertension  <0.0001 30 + hours1107 (6.3) 655 (8.4) 
 No3825 (21.9)2547 (32.7)  Undetermined9049 (51.8)3903 (50.1) 
 Yes13 654 (78.1) 5244 (67.3) Cytomegalovirus sero-pairing  <0.0001
Myocardial infarction  0.462 Donor−/recipient−2151 (12.3)778 (10)  
 No17 232 (98.6) 7690 (98.7)  Donor−/Recipient+3871 (22.1)1671 (21.4) 
 Yes247 (1.4)101 (1.3)  Donor+/Recipient−2817 (16.1)1393 (17.9) 
Diabetes  0.613 Donor+/recipient+6632 (37.9)3164 (40.6) 
 No12 177 (69.7) 5403 (69.3)  Undiagnosed2008 (11.5) 785 (10.1) 
 Yes5302 (30.3)2388 (30.6) Delayed graft function  <0.0001
 No11 178 (63.9) 4352 (58.9) 
Peripheral vascular  0.037 Yes6301 (36) 3439 (44.1) 
 disease
 No16 428 (94)   7269 (93.3) Induction immunosupression  <0.0001
 Yes1051 (6)   522 (6.7)  No8587 (49.1)4503 (57.8) 
 Yes8892 (50.9)3288 (42.2) 
Smoking history  0.0132Discharge immunosupression  <0.0001
 No16 955 (97)   7601 (97.6)  CsA + MMF6472 (37) 2856 (33.2) 
 Yes524 (3) 190 (2.4)  CsA + AZA1968 (11.3)1714 (22)  
Alcohol abuse history  0.1807 TAC + MMF5135 (29.4)1111 (14.3) 
 No17 372 (99.4) 7754 (99.5)  Other3904 (22.3)2380 (30.5) 
 Yes107 (0.6) 37 (0.5)     
CCS comorbidities,   Steroid use  <0.0001
 Mean (SD)9.27 (7.7) 10.50 (8.3) <0.0001 No870 (5) 760 (9.7) 
Charlson comorbidities,    Yes16 609 (95)7031 (90.2) 
 Mean (SD)1.58 (1.4) 1.82 (1.6) <0.0001    
Elixhauser comorbidities,       
 Mean (SD)3.13 (2.6) 3.57 (2.8) <0.0001    

Statistical models

All tests of comorbidities were statistically significant as well as interactions between cause of ESRD and recipient age. The interaction between ABDR HLA mismatches and recipient age was not significant. Linearity in recipient and donor age and BMI was rejected. Linearity in time on dialysis was rejected for models with death as endpoint (Table S3).

Models predicting graft loss were stratified in ESRD caused by polycystic kidney disease (PKD), while models predicting DWF and death did not require stratification. The need for stratification was due to an increasing hazard for all-cause graft loss in the first 6 months after transplantation for the PKD group, although the effect was small (correlations between residuals and time 0.045, p < 0.0001).

In all our models for all-cause graft failure, the variables recorded in the OPTN that are associated with increased risk were confirmed. The risk of graft loss increased with recipient and donor age, African American race and Hispanic ethnicity, donor and recipient BMI and dialysis duration. The hazard ratio for graft loss according to recipient age was U-shaped in the range 18–68 years, reached a minimum at 45 years and remarkably increased beyond age 70 (Figure 1). Both for recipient (Figure 2) and donor BMI, the hazard ratio was also U-shaped from 15 to 32. A BMI beyond 32 were associated with more marked increased risk. In the CCS model, the hazard ratio for graft loss increased with more time on dialysis (Figure 3). Later year of transplantation was associated with a decreased risk of graft failure, while increasing number of ABDR HLA mismatches increased the risk, as well as HLA-sensitization. CMV-sero pairings donor +/recipient −, donor+/recipient+ and unknown serostatus were associated with increased risk of graft loss and death. African American race was not a significant risk factor for the death endpoints (Tables S4–S9).

Figure 1.

Hazard ratio for recipient age (all-cause graft-failure model with CCS variables).

Figure 2.

Hazard ratio for recipient BMI (all-cause graft-failure model with CCS variables).

Figure 3.

Hazard ratio for time on dialysis (all-cause graft-failure model with CCS variables).

Comorbidities

Comorbidities recorded in the OPTN such as angina, congestive heart failure, smoking history and donor-hypertension were associated with increased risk of graft-loss in all models (Tables 3 and 4). Nine CCS comorbidities were significantly associated to graft-failure, among them acute cerebrovascular disease, congestive heart failure, chronic ulcer of the skin and gangrene, epilepsy, systemic lupus eryhtematosus and heart valve disorders (Table 3). In general, similar patterns of OPTN recorded comorbidities were observed for DWF and death, with the addition of diabetes and chronic obstructive pulmonary disease (COPD) (all models except CCS for death). The CCS comorbidities associated with DWF included some of the factors associated with graft loss and diabetes mellitus with complications, diseases of white blood cells, substance related mental disorders and others (Table 4). Generally, similar CCS factors were significant predictors of death with the addition of alcohol-related mental disorders, heart valve disorders, hypertension with complications, cardiac dysrhythmias and other lower respiratory disease (Table 4). The comorbidities associated with graft loss, DWF and death in the Charlson and Elixhauser models were related to those found in the CCS model. Additionally, moderate or severe liver disease was a strong risk factor for graft loss in the Charlson model (Tables S7–S8).

Table 3.  OPTN comorbidities and CCS clusters significantly associated with with all-cause graft failure
OPTN comorbiditiesHR (95% CI)p-Value
  1. Model controlled for recipient, donor, transplant characteristics and other comorbidities as discussed in the methods section. See Table S4 for a table with all predictors.

Angina1.14 (1.06–1.22)<0.01
Congestive heart failure1.15 (1.04–1.27)< 0.01
Smoking history1.30 (1.12–1.51)<0.01
CCS clusters
 20—Chronic ulcer of skin; gangrene1.28 (1.09–1.50)<0.01
 22—Other injuries and conditions due to external causes1.21 (1.08–1.35)<0.01
 26—Epilepsy; convulsions1.14 (1.02–1.28)0.02
 35—Congestive heart failure; nonhypertensive; other lower respiratory disease1.22 (1.13–1.31)<0.0001
 39—Respiratory failure; insufficiency; arrest (adult)1.19 (1.03–1.38)0.01
 41—Systemic lupus erythematosus and connective tissue disorders1.23 (1.07–1.42)<0.01
 59—Acute bronchitis1.19 (1.03–1.37)0.02
 93—Infective arthritis and osteomyelitis (except that caused by tuberculosis or sexually transmitted disease)1.29 (1.07–1.55)<0.01
 97—Acute cerebrovascular disease1.23 (1.06–1.42)<0.01
Table 4.  OPTN comorbidities and CCS clusters significantly associated with death with function and death
OPTN comorbiditiesDeath with functionDeath
HR (95% CI)p-ValueHR (95% CI)p-Value
  1. Models controlled for recipient, donor, transplant characteristics and other comorbidities as discussed in the methods section. See Tables S5 and S6 for tables with all predictors.

 Angina1.22 (1.10–1.35)<0.00011.20 (1.10–1.31)<0.0001
 Congestive heart failure1.15 (1.00–1.33)0.051.18 (1.04–1.34)<0.01
 COPD1.28 (1.02–1.60)0.03  
 Diabetes1.17 (1.04–1.33)<0.011.14 (1.03–1.27)0.01
 Hypertension0.88 (0.80–0.96)<0.01  
 Peripheral vascular disease  1.13 (1.01–1.26)0.03
 Smoking history  1.35 (1.11–1.65)<0.01
CCS comorbidity
 33—Cancer of kidney and renal pelvis  0.56 (0.37–0.86)<0.01
 50—Diabetes mellitus with complications1.16 (1.04–1.30)<0.011.17 (1.06–1.29)<0.01
 63—Diseases of white blood cells1.40 (1.07–1.83)0.01  
 66—Alcohol-related mental disorders  1.49 (1.08–2.05)0.01
 67—Substance-related mental disorders1.22 (1.03–1.45)0.021.16 (1.00–1.34)0.05
 95—Other nervous system disorders1.12 (1.01–1.24)0.03  
 96—Heart valve disorders  1.20 (1.08–1.32)<0.001
 98—Essential hypertension0.92 (0.85–1.00)0.040.92 (0.86–0.98)0.01
 99—Hypertension with complications and secondary hypertension  1.08 (1.01–1.16)0.03
 106—Cardiac dysrhythmias  1.15 (1.05–1.27)<0.01
 108—Congestive heart failure; nonhypertensive1.28 (1.16–1.42)<0.001  
 113—Late effects of cerebrovascular disease  1.30 (1.01–1.68)0.04
 114—Peripheral and visceral atherosclerosis1.16 (1.03–1.30)0.011.13 (1.03–1.25)0.01
 121—Other diseases of veins and lymphatics1.32 (1.07–1.63)0.011.23 (1.02–1.47)0.03
 126—Other upper respiratory infections1.21 (1.03–1.43)0.021.18 (1.02–1.36)0.02
 133—Other lower respiratory disease  1.09 (1.00–1.18)0.05
 156—Nephritis, nephrosis; renal sclerosis0.90 (0.81–0.99)0.040.91 (0.84–0.99)0.04
 163—Genitourinary symptoms and ill-defined conditions0.80 (0.66–0.97)0.020.83 (0.71–0.97)0.02
 166—Other male genital disorders1.42 (1.15–1.75)<0.011.37 (1.14–1.65)<0.001
 167—Nonmalignant breast conditions0.66(0.49–0.89)<0.010.79 (0.63–1.00)0.05
 201—Infective arthritis and osteomyelitis (except caused by tuberculosis or sexually transmitted disease)1.54 (1.21–1.96)<0.0011.38 (1.11–1.71)<0.01
 210—Systemic lupus erythematosus and connective tissue disorders1.51 (1.18–1.93)<0.011.40 (1.15–1.71)<0.001
 216—Nervous system congenital anomalies  2.16 (1.01–4.64)0.05
 232—Sprains and strains1.29 (1.03–1.60)0.021.22 (1.01–1.47)0.04
 244—Other injuries and conditions due to external causes1.26 (1.07–1.48)<0.011.29 (1.13–1.48)<0.001
 246—Fever of unknown origin  0.86 (0.75–0.98)0.03
 248—Gangrene1.29 (1.02–1.65)0.041.32 (1.07–1.62)0.01

Model calibration

The agreement between predicted and actual survival of patients grouped by deciles of observed survival was generally good. Maximum prediction errors were of about 10 absolute percentage points, but more frequent errors were in the order of 4 or 5 absolute percentage points. There was no clear advantage in calibration for any of the models (Table 5).

Table 5.  Models calibration: Observed and predicted survival by deciles
ModelCCSElixhauserCharlsonOPTN
ObservedPredictedErrorObservedPredictedErrorObservedPredictedErrorObservedPredictedError
All-cause graft loss
 First decile22.8319.36−3.4723.9220.10−3.8222.6020.33−2.2723.7221.74−1.98
 Second decile29.0132.083.0726.3232.586.2626.5732.696.1231.3033.432.13
 Third decile34.9839.124.1434.0539.425.3735.0739.504.4334.4439.945.50
 Fourth decile43.5344.500.9744.0544.610.5546.7244.68−2.0343.5144.681.17
 Fifth decile45.3149.033.7247.2248.921.7046.2348.912.6845.2848.733.44
 Sixth decile52.6753.060.452.2452.840.6053.0652.73−0.3255.0052.51−2.49
 Seventh decile57.0956.920.1759.5256.60−2.9259.0856.52−2.5657.6056.13−1.47
 Eigth decile65.7560.95−4.8063.1660.60−2.5562.7560.49−2.2662.5559.93−2.62
 Ninth decile66.5965.52−1.0667.9865.16−2.8367.6464.95−2.6967.5264.25−3.27
 Tenth decile76.1372.50−3.6375.5472.18−3.3575.5071.96−3.5474.3671.20−3.15
Death with function
 First decile37.1532.81−4.3439.4633.90−5.5636.1034.27−1.8336.1137.311.20
 Second decile42.8451.588.7444.3250.666.3448.3050.732.4350.9051.941.04
 Third decile65.2060.83−4.3855.2159.734.5254.8159.815.0061.8460.16−1.68
 Fourth decile58.8767.728.8564.2566.862.6165.7266.901.1857.0266.899.87
 Fifth decile69.9273.463.5473.8972.94−0.9572.6772.950.2773.1372.52−0.61
 Sixth decile83.7478.35−5.3978.5578.09−0.4678.7678.12−0.6381.7177.56−4.15
 Seventh decile83.7282.60−1.1387.1282.48−4.6483.8282.49−1.3385.5781.85−3.72
 Eigth decile89.5686.31−3.2688.9986.41−2.5890.5986.27−4.3288.0585.63−2.42
 Ninth decile91.9989.70−2.2991.4290.01−1.4191.9089.73−2.1889.7189.15−0.56
 Tenth decile94.3993.51−0.8795.0793.98−1.0994.2493.55−0.6993.4092.99−0.42
Death
 First decile25.0822.99−2.0929.0024.50−4.5026.5725.10−1.4728.0827.70−0.39
 Second decile41.0041.650.6536.6041.104.5036.1941.245. 0538.9742.713.74
 Third decile45.8051.675.8744.8050.705.9051.6750.84−0.8344.0851.427.33
 Fourth decile53.3959.065.6755.8058.302.5054.4758.343.8759.2058.45−0.74
 Fifth decile64.8365.430.6065.3364.87−0.4667.5764.80−2.7764.3864.34−0.04
 Sixth decile71.8570.86−0.9972.7870.56−2.2269.4070.501.1072.9869.80−3.18
 Seventh decile80.2575.81−4.4477.3575.70−1.6577.5075.64−1.8676.4574.81−1.65
 Eigth decile83.8080.49−3.3182.5080.50−2.0082.4380.30−2.1382.4979.47−3.03
 Ninth decile87.7485.00−2.7486.5085.20−1.3086.5184.86−1.6583.6284.030.41
 Tenth decile89.5990.060.4790.9190.67−0.2490.8090.20−0.6089.5689.05−0.51

Model discrimination

The c-statistics for OPTN graft loss model was 0.616 (0.609–0.623). The corresponding value for the CCS model was 0.629 (0.623–0.636), while it was 0.625 (0.618–0.632) for the Elixhauser model and 0.623 (0.616–0.630) for the Charlson model (Table 6). The three models including comorbidity information were practically and statistically equivalent to the OPTN model given the overlapping confidence intervals.

Table 6.  Models discrimination
Endpoint/ModelCCSElixhauserCharlsonOPTN
C95% CIC95% CIC95% CIC95% CI
All-cause graft loss0.6290.623–0.6360.6250.618–0.6320.6230.616–0.6300.6160.609–0.623
Death with function0.7140.704–0.7230.710.701–0.7190.7070.698–0.7160.6950.686–0.705
Death0.7090.701–0.7160.7020.694–0.710.6990.692–0.7070.6880.680–0.696

The c-statistic for the models predicting DWF and death were higher. For DWF, the OPTN model c-statistic was 0.695 (0.686–0.705), while it was 0.714(0.704–0.723) for the CCS model, 0.71 (0.701–0.719) for the Elixhauser model and 0.707 (0.698–0.716) for the Charlson model (Table 6). The models with comorbidiy information were statistically and almost practically equivalent to the OPTN model. For death, the OPTN c-statistic was 0.688 (0.680–0.696), the CCS value was 0.709 (0.701–0.716), the Elixhauser 0.702 (0.694–0.701) and the Charlson c-statistic was 0.699 (0.692–0.707). There was a small advantage in predictive ability using models containing comorbidity information, especially for the CCS model, which in this case was also statistically significant compared to the OPTN model.

We determined bias-corrected c-statistics in the three CCS models with different endpoints. We found only very minor variations in our results compared to our original models, with relative reductions in the predictive ability no bigger than 1.3%. Even smaller reductions in the predictive ability were observed for the CCS models with donor age and recipient and donor BMI missing value indicators. Given these results, the sensitivity analyses were not replicated for the other models. Finally, models with only CCS variables were also tested. The c-statistic for a model for graft-failure with the 130 CCS variables with prevalence equal or bigger than 1% was 0.565 (0.558–0.572), and the value for a model with all CCS variables with prevalence bigger than zero was 0.571 (0.564–0.578). This shows that the OPTN variables have better predictive ability.

Discussion

With shortages of available deceased donor organs, the appropriate allocation of kidneys remains critical to achieve optimal transplant results. The UNOS revision of the allocation policy will seek to optimize the net benefit of transplantation and will be based on predictive models that include an expanded number of patient factors (3). Comorbid diseases are factors that might be considered in organ allocation. We augmented the comorbidity information recorded in the OPTN with claims data to explore the usefulness of a wide array of comorbidites in the prediction of graft loss and patient survival. This is, to our knowledge, the first study that reports on the predictive ability of a wide array of comorbidities using the claims data from the USRDS database.

We found only slight improvements in predictions with the use of additional comorbidity models. However, the increase in predictive value was not of practical importance in models using comorbidity information. For graft loss, all models yielded a c-statistic around 0.61–0.63 (a 25.6% improvement over random predictions indicated by a c-statistic equal to 0.5). The CCS model had a 2.5% improvement over the OPTN model but this was not statistically significant. Models predicting death had better predictive performance (c-statistics around 0.7, a 40% improvement over random prediction) but this was practically the same for models with and without comorbidities. The CCS model improved predictive ability for death by 3% and was the only statistically significant improvement.

Although not strictly comparable, logistic regression models using posttransplant renal function reported similar ROC values for predictions of graft loss and death at 3 and 7 years (27). A recipient risk index that included some OPTN comorbidites using a Cox model in a patient survival model had a c-statistic comparable to ours (25). Our results were superior to those predictions of patient survival after liver transplantation using the MELD score and other factors for 3-month mortality, that had a c-statistic of 0.65 (26). Additionally, the improvement in predictive ability for the CCS model of death (3%) was superior to that of adding hyponatremia as a predictor of wait-listing mortality in liver transplant candidates (1% improvement) (24). However, we used a large array of comorbidities rather than a single variable. The SRTR model for 3-year graft-survival (4) had a c-statistic around 0.64, which compares favorably with our own 0.616 c-statistic for the same endpoint. Concurrently with the SRTR models, we found that patient death was better predicted than graft-failure and again, the SRTR model has higher predictive ability for that endpoint. The differences between the models could be explained by a number of reasons. Our inclusion criteria were different from the ones applied in the SRTR model (as we included older cohorts and also restricted the sample to patients with at least one year of pretransplant follow-up). Additionally, our time frame to assess the endpoints was different.

We performed a number of sensitivity analyses. A model with significant CCS clusters reduced the c-statistic for graft loss by 0.5% and by 1.1% in the model for death (compared to full CCS models) but in both cases was still higher than the OPTN model. These gains in precision did not seem justified given their small size and the fact that they were not statistically significant. Many comorbidities detected by the ICD-9 algorithms overlapped with causes of ESRD or with comorbidities collected in the OPTN. Moreover, it was difficult to estimate independent effects due to the large number of variables. Hence, these individual results must be interpreted with caution. Taking into account these caveats, some of the individual associations were unusual, such as the increased risk of graft-failure with stroke and systemic lupus erythematosus.

A recent study proposed a recipient risk score that uses OPTN information that included two OPTN recorded comorbidities (angina and diabetes mellitus) (25). Our results agreed with those findings because we also identified those two variables as predictive of mortality. Models without OPTN comorbidities for graft loss (c = 0.613) and death (c = 0.682) did not reduce the predictive value much and were statistically similar to the OPTN. However, given that these variables are already collected and that there are small increases in predictive ability, it seems that OPTN comorbididities should play a role in organ allocation. To further determine the value added by OPTN recorded comorbidities, we developed sequentially bigger models for all-cause graft loss using OPTN variables added in order starting with those that accounted for less variance in the linear predictor of the hazard ratio for the full OPTN Cox model. The increase in c-statistic was noticeable, starting at only 1% above chance (c = 0.505) and ending with a 20% improvement over chance (c = 0.616). The biggest addition of predictive ability in the model sequence came from the OPTN recorded donor comorbidities, while recipient comorbidities also contributed to the predictions.

Although previous work found strong associations between the Charlson score above 5 and patient survival in a single center study (28), our Charlson models did not yield noticeably better predictive power. The predictive value depends on the strength of the associations and the prevalence of the predictive factors. The single center study had 6.4% of patients with Charlson scores higher than 5 while only 1.4% were above that value in our sample. We found a positive association of 1.54 (1.27–1.84) for a Charlson score bigger than 5, that was smaller than the hazard ratio of the single center study.

Further research should be undertaken regarding the outcomes of patients with very high-risk comorbidities. Transplant centers differ in their evaluation and use of comorbidities for exclusion to transplantation. Extensive pretransplant evaluation and identification of coronary artery disease, peripheral vascular disease, COPD and other conditions which have a major impact on long-term outcome remain important. The available data do not allow us to analyze those centers with more comprehensive evaluation strategies and more strict listing criteria.

This study has some limitations. We relied on medical claims billing data to augment the comorbidity information recorded in OPTN (29). The completeness, reliability and validity of the ICD-9 information in the context of transplantation have been questioned by some researchers (5). In a recent work, the sensitivity of private insurance claims using OPTN comorbidities as gold standard for measuring comorbidities was low, while specificity was high (30). The authors concluded that it is likely that this reflects gaps in comorbidity assessment in the OPTN information rather than false positive claims. Additionally, our ascertainment of comorbidities was restricted to the last pretransplant year to examine all patients over a comparable period of time. This may have caused an underestimation of the disease prevalence. Both the nature of claims-data and the study design limitations allow a potential misclassification bias especially producing false negatives. This will bias downward the predictive ability of comorbidities. Claims-data are limited in their clinical depth as it is difficult to differentiate severity of disease, and laboratory and medical chart data are not available. Such information may improve predictions in future models. For example, adding laboratory data to claims data were found to improve predictions of hospital mortality (31,32). The results are limited to Medicare patients receiving transplants from deceased donors and having at least one year on dialysis. We used imputation with means and medians and missing value indicators to account for missing values. Further models should test multiple imputation methods. Finally, confounding by indication is a problem in observational research. However, we did not address the possibility that patients with higher comorbid load may be treated differently that patients with smaller or less severe comordibidites because our goal was to estimate models predictive performance using only pretransplant variables relevant to organ allocation. Such treatment options may influence the associations between certain comorbidities and outcomes. With future and improved treatments, it may be possible that the predictive ability of these models may change.

In conclusion, considering the slight improvement in predicting graft-survival and death and considering the relatively high cost of data collection (33), it does not seem appropriate to increase the burden of data collection in the OPTN by recording additional comorbid factors or to use comorbidities beyond those already collected in the OPTN in the kidney allocation schema development.

Summary

We examined the potential use of comorbidities not recorded in the OPTN together with OPTN variables to predict long-term graft loss and patient death. Although the models containing comorbidities have better discriminative ability, improvements were small compared to the OPTN models. Therefore, pretransplant comorbidities did not add a practical improvement in predictive ability to OPTN models.

The consideration of additional comorbidities would still be useful to monitor changes in the ESRD population and to reassess the allocation rules. Our results should be verified using private payor claims data, augmenting claims with laboratory data and with comorbidity measures not relying on claims. It remains possible that pretransplant comorbidities can predict posttransplant costs due to higher cost of patients with functioning grafts or for cost related to graft loss. An economic analysis incorporating those incremental costs may be useful for organ allocation even in the absence of clinical differences, given that economic resources are scarce. We intend to pursue economic analyses addressing those questions. With transplant centers increasingly facing global contracts, the identification of pretransplant comorbidities to predict cost is also valuable. Pretransplant comorbidities may help to predict other outcomes such as new onset comorbidities. Further study of predictive models using pretransplant comorbidities may prove relevant for organ allocation, and posttransplant patient management.

Acknowledgments

Data reported here have been supplied by the United States Renal Data System (USRDS). The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as an official policy or interpretation of the US Government. The authors thank Dr. Walter K. Kremers for his helpful comments on this article and for proving a SAS macro to determine the c-statistics.

The work was supported in part by a grant from the NIDDK K25-DK-02916-03, (MAS), NIDDK, K08-0730306 (KLL) and NIDDK, K24-DK-002886 (DCB).

Ancillary