SEARCH

SEARCH BY CITATION

Keywords:

  • Creatinine;
  • renal function;
  • transplant

Abstract

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References

Both medical care and pharmaceutical development have led to an increase in expected graft and patient survival for patients who undergo renal transplantation. From a research perspective, it has become increasingly difficult to study the efficacy of new therapies using traditional ‘hard’ endpoints. In reaction to this dilemma, the transplant community has sought a surrogate endpoint. A natural candidate for a surrogate marker for graft loss that has been proposed is renal function (serum creatinine or calculated GFR levels).

Using data from the USRDS, we conducted a retrospective evaluation of transplant data from 1988 to 1999 to quantify the predictive value of renal function for the outcomes of graft loss, death-censored graft loss, and patient death.

Renal function along with the change in renal function demonstrated a high relative risk for ultimate graft survival and graft loss (odds ratio = 2.2 for an increase of 1 mg/dL). However, the predictive value as measured by the area under the receiver operating characteristic curve (AUC) for this criteria was poor (0.627). These findings held true for the slope of creatinine and formulations of GFR.

While renal function is a strong risk factor and highly correlated with graft failure, the utility of renal function as a predictive tool for graft loss is limited.

The treatment and care for patients with kidney transplantation in the past decade has steadily improved. Both medical care and pharmaceutical development have led to a steady increase in expected graft and patient survival for patients who undergo this procedure (1,2). In addition, acute rejection rates continue to decrease annually (3–5). From a research and novel therapy development perspective this has led to a diminished ability to study the efficacy of treatments within a relatively short time period. If these traditional endpoints are the only acceptable assessment of novel therapies, ever increasing periods of study time and patient numbers will be needed to confirm the efficacy of these newer treatments or strategies. This may lead to prolonged evaluation of ineffective treatments, or, in a similar fashion, delayed acceptance of beneficial treatment modalities. Furthermore, pharmaceutical development costs may increase to the point that the possibility to study new drugs becomes increasingly limited. As a result, there has been a concerted effort among the transplant community to attain a diagnostic that may serve as a surrogate for eventual graft loss (6). A surrogate endpoint for graft loss could allow for more rapid evaluation of drug therapies, transplant techniques, and patient care protocols. A natural candidate for this surrogate endpoint that has been considered in recent literature is renal function. Creatinine levels (7) at 1-year post transplant and changes in creatinine levels (8) have been shown to demonstrate a strong association between poor renal function and declining renal function, respectively, with worse graft and patient survival (9).

While there is no disagreement concerning the strong correlation of renal function and renal transplant outcomes, caution is advised in extrapolating this correlation as predictive of the endpoint to which it serves as a surrogate. In establishing highly associated diagnostics as predictive entities, further steps of validation are required (10). Additionally from a statistical point of view, the interpretation of correlated values as possessing cause and effect relationships, particularly in retrospective analyses, is problematic. This point can be particularly important when counseling patients and in the decision of new clinical trials. While it may be natural to pursue assessing correlated variables as to their prediction potential, different statistical procedures must be conducted to evaluate the efficacy of variables to serve this purpose. In order to determine whether renal function can adequately serve as a predictor, and to add information regarding the acceptability of renal function as a surrogate (or substitute) endpoint for graft loss, we conducted an analysis attempting to quantify the predictive properties of renal function for graft loss from data available in the USRDS database.

Methods

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References

A retrospective study of all first transplant recipients age older than 17 years between 1988 and 1999 that had at least 2 years of follow-up information (data available through December 2001) was conducted to determine the predictive nature of certain patient diagnostics. The independent renal function parameters used included creatinine level (treated as continuous and categorical), the relative difference in 1/creatinine level from 6 months to a year, and calculated GFR levels. Outcomes of death-censored graft loss, patient death, and overall graft loss were measured as the response in the models. Observed patients were included that possessed follow-up information for a given time period (either 2 years or 7 years), and baseline levels at 6 months and 1 year were both tested as explanatory variables. Those patients with creatinine levels greater than four at these periods were excluded from the study, as their immediate health-related concerns were thought to confound their study outcome. For the purpose of calculating GFR estimates, values for height and weight were imputed, using the Markov Chain Monte Carlo methodology (11) with height (if nonmissing) and weight (if nonmissing) as predictors stratified by race and gender. The analyses that incorporated clearance levels were calculated as:

  • image

were repeated with and without imputation to verify the results (12). Values of weight and height that were thought to be miscoded were treated as missing (<20 kg and <80 cm, respectively), these observations represented less than 0.1% of all eligible records. The endpoints addressed were patient death, death-censored graft loss, and overall graft loss. Logistic regression was used to analyze the likelihood of particular outcomes for a given follow-up period. Output from the logistic regression was further utilized to generate predictive diagnostics and ROC plots. Univariate logistic regression was performed for creatinine levels (treated as both continuous and categorical), change in 1/creatinine level from 6 months to 1 year, and GFR:

  • image

Groupings for Scr (mg/dL), when treated as categorical, were (0−1.2, 1.2 ≥€ 1.4, 1.4 ≥€ 1.6, 1.6 ≥€ 1.8, 1.8 ≥€ 2.1, 2.1 ≥€ 2.5, 2.5 ≥€ 4.0). An alternative to the formula estimating filtration rate that was proposed by the MDRD Study was also tested (13):

  • image

Glomerular filtration rate levels were expressed per 1.73 m2 of body surface area; body surface areas were calculated utilizing the formula by Lam and Leung (14):

  • image

The area calculation for the ROC curve was made by the Trapezoidal Rule (15). In addition to the area under the ROC curve as a prediction diagnostic, we examined the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for specific renal function levels. The PPV was calculated as the conditional probability that the event of interest occurred given the at-risk diagnostic was present at the baseline time period, and in a similar fashion, NPV was calculated as the conditional probability that the outcome of interest did not occur given that the diagnostic baseline level was not designated as at-risk. Multivariate logistic regression was performed for the same outcomes; covariates used in the model were donor and recipient demographics, primary diagnosis, dialysis time before transplant, HLA-matching, cold ischemia time, year of transplant, PRA level, recipient CMV status, cadaveric/living donor, antiproliferative medication at transplant, and immunosuppressive medication at baseline. Medication regimens were not implemented in the multivariate models for the 7-year follow-up periods, as the majority of patients from that time period were on Azathioprine and Sandimmune in addition to steroids. Beyond the scope of the overall variable used in the model, we examined distinct levels of applicable diagnostics as to their predictive power for overall graft loss. In particular, we examined two potential surrogate endpoints for the creatinine level that have been suggested by the transplant community of 1.6 mg/dL and 1.8 mg/dL, and a –30% change in 1/creatinine that has also been proposed for this purpose. Statistical analysis was performed using SAS software v8.02 (Cary, NC).

Results

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References

The summarized output of the logistic regression models for overall graft loss can be viewed in Tables 1–4. Table 1 displays the output of the model for graft loss with creatinine as the explanatory variable with 2 and 7 years of follow up and using the 1-year creatinine level as the baseline measure. An increase in creatinine value (per 1 mg/dL) yielded a 2.2-fold odds ratio with 95% CI (2.1, 2.3) with an AUC of 0.627 (Figure 1). For this design, we additionally stratified the model to patients receiving a transplant before and after 1995. The AUC for the models were similar: before 1995 equal to 0.643, and after 1995 equal to 0.606. Models for different follow-up periods and baseline combinations yielded slightly lower prediction rates as measured by the AUC: 7-year follow up/1 year baseline (0.624), 2-year follow up/6-month baseline (0.616), and 7-year follow up/6-month baseline (0.605). Treating creatinine as a categorical variable yields very similar results with a maximum AUC (0.625) achieved for the 2-year follow-up outcome relative to a 1-year baseline level. For the increasing categorical levels of creatinine, we observe an increased odds ratio for graft loss. Relative to a reference grouping of [0 = Scr (mg/dL) = 1.2], subsequent grouping yielded increased odds ratios of: (1.2 < Scr = 1.4) OR = 1.118, (1.4 < Scr = 1.6) OR = 1.197, (1.6 < Scr = 1.8) OR = 1.492, (1.8 < Scr = 2.1) OR = 1.783, (2.1 < Scr = 2.5) OR = 2.545, and (2.5 < Scr = 4.0) OR = 4.716. Models for 7-year outcomes and utilizing 6-month baseline levels yielded similar results.

Table 1.  Univariate logistic models for 2- and 7-year overall graft outcome from 1-year creatinine (continuous) baseline levels
Variable of InterestFollow upBaselineObs.Fail percentageORCIAUC
  1. Obs: the number of observations that fit the criteria described and subsequently used for the model.

  2. Fail percentage: the actual fail percentage within the designated follow-up period.

  3. OR: odds ratio point estimate (for a unit change in SCr).

  4. CI: Wald confidence interval of the odds ratio.

  5. AUC: area under the receiver operating characteristic curve.

Creatinine (continuous)2 years1 year74 4807.22.22(2.13, 2.31)0.627
Creatinine (continuous)7 years1 year35 25545.42.40(2.31, 2.50)0.624
Variable of interestFollow upBaselineObs.Fail percentageCategoriesOR point estimateAUC
Creatinine (categorical)2 years1 year74 4807.2  0 ≥ 1.210.625
 1.2 ≤ 1.41.12 
 1.4 ≤ 1.61.20 
 1.6 ≤ 1.81.49 
 1.8 ≤ 2.11.78 
 2.1 ≤ 2.52.55 
 2.5 ≤ 4.04.72 
Creatinine (categorical)7 years1 year35 25545.4  0 ≥ 1.210.623
 1.2 ≤ 1.41.018 
 1.4 ≤ 1.61.224 
 1.6 ≤ 1.81.464 
 1.8 ≤ 2.11.766 
 2.1 ≤ 2.52.741 
 2.5 ≤ 4.05.613 
Table 2.  Univariate logistic models for 2- and 7-year graft outcome from 6-month change in 1/creatinine baseline levels
Variable of interestFollow upBaselineObs.Fail (%)ORCIAUC
  1. Obs: the number of observations that fit the criteria described and subsequently used for the model.

  2. Fail percentage: the actual fail percentage within the designated follow-up period.

  3. OR: odds ratio point estimate (per unit of change).

  4. CI: Wald confidence interval of the odds ratio.

  5. AUC: area under the receiver operating characteristic curve.

Change in 1/creatinine2 yearsChange from 6 months72 564 7.20.992(0.991, 0.993)0.556
   to 1 year     
Change in 1/creatinine7 yearsChange from 6 months34 40945.40.994(0.994, 0.995)0.548
   to 1 year     
Table 3.  Logistic model for 7-year graft outcome from change in 1/creatinine level from 6 months to 1 year
 Sensitivity = 8.3%
Output for 1/creatinine level of –30%Specificity = 96.9% PPV = 69.1% NPV = 55.9%Actual graft outcome
Non-failureFailureTotal
Predicted graft outcomeNon-failure18 19214 33732 529
 Failure   580  1300  1880
 Total18 77215 63734 409
Table 4.  Logistic model for 7-year graft outcome from the 1-year baseline creatinine level
 Sensitivity = 48.0%
Output for creatinine level of 1.8Specificity = 71.0% PPV = 57.9% NPV = 62.2%Actual graft outcome
Non-failureFailureTotal
Predicted graft outcomeNon-failure13 673  830921 982
 Failure  5587  768413 271
 Total19 26015 99335 253
image

Figure 1. ROC curve of creatinine for overall graft failure (2-year graft survival 1-year creatinine as a predictive quantity).

Download figure to PowerPoint

Table 2 displays the predictive model data obtained by using a relative percent change of 1/creatinine as indicator variable for eventual graft loss. This change was recorded from 6-month record as a baseline reference to the 1-year recorded 1/creatinine. An increase in function as measured by 1/creatinine incurred a reduced OR for graft loss, 95% CI (0.991, 0.993), for the 2-year follow-up model, but the AUC was 0.546 (Figure 2 displays the ROC for change in 1/creatinine rounded to the nearest 10%). Similarly, the 7-year model details a 95% CI for the OR of (0.994, 0.995) and an AUC of 0.548.

image

Figure 2. ROC plot for 7-year graft outcome with relative percent change in 1/creatinine levels from 6 months to 1 year as a predictor.

Download figure to PowerPoint

Table 3 displays specific information extracted from the 7-year model, using the percent relative change in 1/creatinine levels from 6 months to 1 year as baseline for a proposed value of –30%. At this particular level the model yields a sensitivity of 8.3% and a specificity of 96.9%. From the complement of 15 637 actual observations with graft failure, this specific level of change identifies 1300 of these. From an additional perspective, of the 1880 predicted failures, according to this specific level, 580 (30.9%) did not actually lose their graft for the 7-year follow-up period. The 2-year model for the –30% percent change level resulted in a sensitivity of 12.8% and a specificity of 95.2%. The positive predictive value, or probability of an actual event given an at-risk condition, for this specific cutoff of renal function was 69.1%, and the negative predictive value, or the conditional probability of a nonevent given a low-risk condition, was 55.9%. We additionally constructed a model for overall graft loss using the change in inverse creatinine measures from 6 months to 2 years. This model yielded a slightly higher AUC of 0.588 for outcomes at 7 years.

GFR was also found to be significantly associated with graft loss for the various models; for imputed models for both 2 and 7 years the AUC was equal to 0.627. For nonimputed models the 2-year follow-up period produced an AUC value of 0.636 and an AUC of 0.627 for a 7-year follow-up period. The MDRD estimate had similar results for the nonimputed models, but slightly less area explained than the calculated GFR estimation, the AUC for 2-year follow up equal to 0.613 and for the 7-year follow-up period of 0.610. The odds ratio estimates for the 2-year model without imputed values were 0.968 for the calculated GFR formula (per rate unit) and 0.980 for the MDRD estimation (per rate unit) as the variable of interest.

Multivariate models directly incorporating the creatinine level demonstrated the most predictive strength, according to the AUC, for 2-year model (0.683) and for the 7-year model (0.748). The 7-year models had slightly higher AUC values than the 2-year models for the GFR estimation, 0.704 compared with 0.670, and for the MDRD formula, 0.701 compared with 0.657.

For the 2-year model for graft outcome, we examined the specific rate of correctly predicted observations, false-positive rates, and false-negative rates for a proposed creatinine level of 1.6 mg/dL. For this level, the model output contained 74 480 total observations. Of these, 40 119 (53.9%) were predicted correctly, 36 670 were nonfailures and 3449 were failures. The sensitivity, the proportion of actual failures that were predicted correctly, and those of the model at this creatinine level was equal to (3449/5372) 64.2%, and the specificity or proportion of nonfailures accurately identified, was (36670/69108) 53.1%. Table 4 displays the sensitivity and specificity of a serum creatinine cutoff value of 1.8 mg/dL as a predictor for graft loss for a 7-year follow-up period from a 1-year baseline creatinine level. In this instance using 1.8 mg/dL as the test level, the specificity of the model was 48.0% and the sensitivity equal to 71.0%. For this model, 21 357/35 253 (60.6%) of the observations were correctly classified as to their outcome. This event actually occurred at a rate of approximately 45%. For this model, we see the true probability of the event occurring most closely approximated by a creatinine level of 1.7 mg/dL (predicted probability = 0.4526), and we can see in this case the ‘true’ overall false-positive rate equal to 45.3% and a false-negative rate of 36.3%.

Analyses were repeated for patient survival as an endpoint. The variables of interest were all significantly associated with patient survival for the univariate and multivariate models. The OR for creatinine in the univariate model was equal to 1.318 with a CI of 1.267, 1.372. The CI for other renal function estimates in the univariate models were (0.973, 0.977) for GFR, and (0.988, 0.991) for MDRD. The AUC value for the univariate model was highest for GFR as a predictor (0.608) followed by MDRD (0.566) and the creatinine level (0.542). For the multivariate models, we found the AUC value to be highest with creatinine as the variable of interest (0.743), followed by GFR (0.729), and MDRD (0.726). The variables of interest were similarly significantly associated with patient survival in these models.

Death-censored graft survival was also analyzed as an outcome for a 7-year follow up using 1-year baseline as the explanatory value. In the univariate model for creatinine, the confidence interval for the OR extends more than threefold, 95%CI (3.1, 3.4), for a unit increase of creatinine. The univariate AUC values for renal function measures were 0.672 (SCr), 0.635 (GFR), and 0.637 (MDRD). Multivariate models demonstrate similar significant associated relationships for creatinine: 95% CI (3.0, 3.4), GFR (0.963, 0.968), and MDRD (0.970, 0.974). In addition, the AUC values for the multivariate models were highest for those incorporating creatinine (0.764) as a measure of renal function, followed by MDRD (0.727), and finally GFR (0.726).

Discussion

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References

This analysis demonstrates the relatively limited predictive value of serum creatinine (renal function) in predicting both death-censored graft loss and patient mortality. Concordant with previous analyses, risk estimates associated with creatinine levels and slope of creatinine with graft and patient survival are clearly present. By examining the data of the USRDS describing renal function and patient outcome in multiple combinations, we have attempted to extend this research by exploring the predictive nature of these diagnostics. The analysis was repeated in a variety of permutations to reinforce the assertion that the results were robust to the particular conditions imposed in the model or the explanatory variable(s) used. As we have ventured to demonstrate, the variables that we examined were insufficient to be designated as reliable predictive quantities.

As an example, the model for 2-year overall graft outcome, utilizing the 1-year creatinine level as a baseline explanatory variable, generated a Wald 95% CI for the OR of (2.13, 2.31). This suggests a significant association between creatinine levels and graft failure. Specifically, there appears to be more than a twofold increase in the odds of graft failure for every unit increase in creatinine. This same model also generates an area under the ROC curve of 0.627. This essentially means that for only 62.7% of all possible observation pairs with different responses does the model assign a higher probability to a correct case than to an incorrect case. In other words, the ability of the model to discriminate observations between the actual failure cases and the cases that do not fail is a minimal gain from random allocation alone (analogous to an area of 0.500). Multivariate models naturally possess increased predictive power (AUC = 0.764 for the model with creatinine directly incorporated), however, it is obviously impractical to consider the full complement of factors as a potential surrogate marker. Another feature of the ROC curve that is notable (see Figure 1) is the particular shape of the curve. For some situations, the predictive quality is strong in a particular range, but then tends to lose its potency. This phenomenon would exhibit itself with a steep slope in the useful portion of the curve and a ‘flattening out’ in the remainder. However, for the variable creatinine (as with other diagnostics we tested), the shape of the curve does not lead one to conclude that any particular range of the variable has any predictive character or a discrete cutoff that could be utilized.

This study is in no way meant to convey the message that renal function is not a very strong independent risk for graft loss or patient death. In fact, in our analyses and others (16–18), level of renal function is perhaps the strongest risk factor for both of these endpoints. However, renal function as a predictor to correctly determine whether a study population will lose their graft or not (let alone for a single patient) appears limited. As a surrogate technically means a substitute for another, it is probably not reasonable to utilize this measure alone as a surrogate for graft loss. As it is a very strong risk factor for several important outcomes, it can serve as an independent endpoint. However, it does not appear that it can by itself replace graft loss.

This model was not tested on a prospective or novel population; rather it was tested on the population from which it was derived. This would tend to increase the predictive value we found, and likely means that for future prospective use the predictive value would in fact be even lower.

The lack of this very strong risk factor to have adequate predictive value likely lies in the high variance of the model. In the models utilized from national databases and even from single-center studies,r2 values are on the order of 5–15%. Thus, 85–95% of the variables that contribute to graft loss remain unexplained. This is not surprising given the lack of understanding of all the factors that can lead to chronic allograft nephropathy along with cardiovascular death (in this group of patients). It is also possible that events that merely change hemodynamics, but otherwise help other factors (ACE inhibitor therapy), may not necessarily lead to equivalent increases in risk. This analysis addresses the predictive value of creatinine. The utility of a predictive diagnostic in general is relative to the context in which it is used. In this circumstance, it would seem reasonable to expect a higher predictive measure of a particular diagnostic to be useful alone in discriminating patients who will lose their grafts over a 7-year period vs. those who do not. In addition, as predictive values are dependent on the incidence of the outcome examined, there may be increased predictive value when addressing subpopulations of this cohort that are more at-risk.

In conclusion, this study confirms earlier analyses that demonstrate the importance of renal function as a risk factor for death-censored graft loss and patient death. This association appears to be linear in effect. However, serum creatinine or other estimates of GFR on their own do not have a sufficient predictive value to serve as a reliable predictive test for graft loss or patient death, but clearly has utility as an intermediate endpoint of importance. The issue of whether renal function can be utilized as a surrogate endpoint for graft loss and patient death is problematic. This issue will require thoughtful and critical judgment by the transplant community.

References

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References
  • 1
    Hariharan S. Long-term kidney transplant survival. Am J Kidney Dis; 38: S4450.
  • 2
    Meier-Kriesche HU, Ojo AO, Port FK, Arndorfer JA, Cibrik DM, Kaplan B. Survival improvement among patients with end-stage renal disease: trends over time for transplant recipients and wait-listed patients. J Am Soc Nephrol 2001; 12: 12931296.
  • 3
    Meier-Kriesche HU, Ojo AO, Hanson JA et al. Increased impact of acute rejection on chronic allograft failure in recent era. Transplantation 2000; 70: 10981100.
  • 4
    Halloran P, Mathew T, Tomlanovich S, Groth C, Hooftman L, Barker C. Mycophenolate mofetil in renal allograft recipients: a pooled efficacy analysis of three randomized, double-blind, clinical studies in prevention of rejection. The International Mycophenolate Mofetil Renal Transplant Study Groups. Transplantation 1997; 63: 618.
  • 5
    Kahan BD. Two-year results of multicenter phase III trials on the effect of the addition of sirolimus to Cyclosporine-based immunosuppressive regimens in renal transplantation. Transplant Proc 2003; 35: S37S51.
  • 6
    Hariharan S, McBride MA, Cohen EP. Evolution of endpoints for renal transplant outcome. Am J Transplant 2003; 3: 933941.
  • 7
    Hariharan S, McBride MA, Cherikh WS, Tolleris CB, Bresnahan BA, Johnson CP. Post-transplant renal function in the first year predicts long-term kidney transplant survival. Kidney Int 2002; 62: 311318.
  • 8
    Kasiske BL, Andany MA, Danielson B. A thirty percent chronic decline in inverse serum creatinine is an excellent predictor of late renal allograft failure. Am J Kidney Dis 2002; 39: 762768.
  • 9
    Meier-Kriesche HU, Baliga R, Kaplan B. Decreased renal function is a strong risk factor for cardiovascular death after renal transplantation. Transplantation 2003; 75: 12911295.
  • 10
    Armitage P, Berry G, Matthews JNS. Statistical Methods in Med Research, 4th edn. Oxford : Blackwell Science Ltd, 2002: 692695.
  • 11
    Li KH, ‘Imputation Using Markov Chains’, J Stat Comput Simulation 1988; 30: 5779.
  • 12
    Cockcroft D, Gault MD. Prediction of creatinine clearance from serum creatinine. Nephron 1976; 16: 3141.
  • 13
    Levey AS, Bosch JP, Lewis JB, Greene T, Rogers N, Roth D. A more accurate method to estimate glomerular filtration rate from serum creatinine: a new prediction equation. Modification of diet in Renal Disease Study Group. Ann Intern Med 1999; 130: 461470.
  • 14
    Lam TK, Leung DT. More on simplified calculation of body-surface area. N Engl J Med 1988; 318: 1130.
  • 15
    SAS Institute Inc. SAS/STAT User's Guide, Version 8. Cary , NC : SAS Institute Inc. , 1999: 19621963.
  • 16
    Fried LF, Shlipack MG, Crump Cet al. Renal insufficiency as a predictor of CVD. JACC 2003; 41: 13641372.
  • 17
    Manjunath G, Tighiouart H, Coresh J et al. Level of kidney function as a risk factor for cardiovascular outcomes in the elderly. Kidney Int 2003; 63: 11211129.
  • 18
    Manjunath G, Tighiouart H, Ibrahim H et al. Level of kidney function as a risk factor for atherosclerotic cardiovascular outcomes in the community. J Am Coll Cardiol 2003; 41: 4755.