Renal failure persisting after renal transplant is known as delayed graft function (DGF). DGF predisposes the graft to acute rejection and increases the risk of graft loss. In 2010, Irish et al. developed a new model designed to predict DGF risk. This model was used to program a web-based DGF risk calculator, which can be accessed via http://www.transplantcalculator.com . The predictive performance of this score has not been tested in a different population. We analyzed 342 deceased-donor adult renal transplants performed in our hospital. Individual and population DGF risk was assessed using the web-based calculator. The area under the ROC curve to predict DGF was 0.710 (95% CI 0.653–0.767, p < 0.001). The “goodness-of-fit” test demonstrates that the DGF risk was well calibrated (p = 0.309). Graft survival was significantly better for patients with a lower DGF risk (5-year survival 71.1% vs. 60.1%, log rank p = 0.036). The model performed well with good discrimination ability and good calibration to predict DGF in a single transplant center. Using the web-based DGF calculator, we can predict the risk of developing DGF with a moderate to high degree of certainty only by using information available at the time of transplantation.
creatinine reduction ratio on posttransplant day 2
deceased donor score
delayed graft function
donor risk score
expanded criteria donor
intensive care unit
kidney donor risk index
panel reactive antibodies
receiver operating characteristic
united network for organ sharing
Renal failure persisting after renal transplant is known as delayed graft function (DGF). The frequency of DGF varies from 5% to 50% in deceased donor kidney transplants (1). DGF complicates immediate posttransplant management by prolonging patient hospitalization and increasing morbidity and health care costs (1–4). Moreover, DGF predisposes the graft to acute rejection and increases the risk of chronic allograft nephropathy and premature graft loss (1–3). Patients with DGF had a 41% increased risk of graft loss (1,5).
Although partly explained by the higher utilization of kidneys from expanded criteria donors (ECDs) and from donation after cardiac death, incidence and severity of DGF has not decreased over the last few years (6). The use of induction therapies with calcineurin inhibitor dose reduction or avoidance and calcium channel blocker therapy are common practices to prevent or to treat DGF, but there have also been no major therapeutic advances in this field in recent years (3,6,7). Future strategies or therapies, to prevent or treat DGF must be tested in patients who have already developed DGF or who are at a higher risk of developing it. In this sense it could be very interesting to have methods to diagnose DGF earlier than nowadays or to know which group of transplant recipients is more prone to develop DGF, respectively. On the one hand, several promising biomarkers allow earlier and accurate DGF detection after kidney transplantation (8). On the other hand, it is possible to use some models to predict patient and population DGF risk. These models can be used to optimize allocation strategies, to intervene, to prevent DGF, to compare the observed versus predicted DGF incidence in different cohorts of patients or between centers, to identify patients at higher DGF risk for being included in clinical trials or to stratify patients in such trials (9,10). Moreover, they can be used to improve the diagnostic ability of new biomarkers or to adjust their performance to detect DGF.
In addition to ECD classification, in the last few years several donor scoring systems have been developed to predict transplant outcome: deceased donor score (DDS), donor risk score (DRS) and kidney donor risk index (KDRI; Refs. 11–14). Whereas these scores were developed to predict long-term graft loss and first year renal function, Irish et al. and Jeldres et al. developed two DGF-specific scores (9,15). Irish's DGF-specific nomogram performed better than nonspecific scores (DDS and ECD) to predict DGF (16). Further analyses of North American and Australian populations have yielded conflicting results after applying Irish's nomogram (17,18).
In 2010, Irish et al. refined their model analyzing 24 337 deceased donor renal transplant recipients, considering additional risk factors obtained from the united network for organ sharing (UNOS) database. The new model was validated using a separate data set of transplanted patients, with a c index of 0.704, which indicates a good degree of discrimination. The new model was used to program a web-based DGF risk calculator, which can be accessed via http://www.transplantcalculator.com. Individual or population information can be entered into the web to obtain a DGF risk prediction (10). The predictive performance of this score has not been tested in a different population. In order to validate the score it is a key step to apply it to a single center population. Due to this, we aim to analyze the performance of this web-based calculator to predict DGF in our center.
Material and Methods
We analyzed 342 deceased-donor adult renal transplants performed in our hospital throughout 12 years. Patients were included if web obligatory data (recipient race and sex, donor final serum creatinine, donor weight and mismatches) were available. Patients were excluded if they received a preemptive transplant or an organ transplant other than the kidney and if they lost the kidney graft in the first 5 days because of vascular thrombosis. Data were collected from the prospectively maintained database of all renal transplant patients and the prospectively maintained database of all brain deceased donors of the intensive care unit (ICU) in our hospital. Both databases were matched anonymously by the hospital number of the donor. Warm ischemia time was collected from the operating room records of clinical charts. The study was conducted according to the guidelines of the Declaration of Helsinki and was approved by the ethical committee of our hospital.
The following recipient variables were recorded: peak panel reactive antibodies (PRA), dialysis duration, body mass index, race, sex, previous transplants, diabetes mellitus, transfusions, age lower than 16 years and prior or simultaneous extrarenal transplant. Recorded donor variables were terminal creatinine, age, weight, deceased or living donor, donation after cardiac death, history of hypertension and stroke or anoxia as causes of death. Transplant variables collected were the number of HLA mismatches and cold and warm ischemia times. All kidneys were preserved by simple ice preservation and none of them were machine perfused. Individual and population DGF risk was assessed using the web-based calculator (http://www.transplantcalculator.com) after entering recipient, donor and transplant information (Table 1).
Table 1. Recipient, donor and transplant characteristics at time of transplantation
N = 342
47.5 ± 13.0
Age < 16 years
Diabetes mellitus (%)
% Peak PRA (mean)
11 ± 20
Pretransplant transfusions (yes)
Body mass index (kg/m2)
25.7 ± 3.9
Duration of dialysis (days)
1484 ± 1916
44 ± 16
71 ± 12
Cardiac death (%)
Deceased donor (%)
Hypertension (% yes)
Terminal creatinine (mg/dL)
1.09 ± 0.4
Cause of death: Stroke (%)
Cause of death: Anoxia (%)
HLA mismatches (mean)
3.5 ± 1.1
Cold ischemia time (h)
18.8 ± 5.1
Warm ischemia tine (min)
42.7 ± 13.1
Kidney transplant outcome variables included were: DGF, creatinine on days 1, 2, 5 and at months 6 and 12, creatinine reduction ratio on day 2 (207 patients), urine volume on day 1, Doppler resistive index at day 2 (151 patients) and noncensored for death graft survival. Graft survival was calculated from the date of transplantation to the date of irreversible graft failure signified by return to long-term dialysis or retransplantation or the date of the last follow-up during the period when the transplant was still functioning or to the date of death. Creatinine reduction ratio on posttransplant day 2 (CRR2) was calculated by the equation: CRR2 = ([Cr Day1]–[Cr Day2]× 100)/Cr Day1 (5). DGF was defined as a dialysis requirement during the first week after transplantation.
Our standard immunosuppressive regimen consisted of triple maintenance immunosuppression with a calcineurin inhibitor, azathioprine (42%) or mycophenolate mofetil (47%) and steroids. Calcineurin inhibitors were started on the same transplant day. Target though levels in the first postoperative month were 250–350 ng/mL for cyclosporine and 8–15 ng/mL for tacrolimus. Some 5% of patients received induction with basiliximab and 2% with lymphocyte-depleting antibodies.
Continuous variables were analyzed using Student's t-test and categorical data by chi square test. Pearson's correlation was used to describe the relationship between calculated DGF risk and different outcome variables. Logistic regression analysis was used to determine the odds ratio for DGF according to the calculated DGF risk value. Model calibration was estimated by the Hosmer–Lemeshow “goodness-of-fit” test (19). DGF discrimination ability was measured by sensitivity, specificity, positive predictive value and ROC curve c statistic. Graft survival probability was estimated using the Kaplan–Meier method (20). Cox model was used to calculate the hazards ratio of graft failure associated with DGF risk. A p value of less than 5% was reported as statistically significant. Statistical analyses were performed with SPSS, version 15.0 (SPSS Inc, Chicago, IL, USA).
The overall rate of DGF in the study population was 30.9% (95% CI 26.0–35.9), whereas the web-calculated population risk was 26%. The average individual web-calculated risk was 25.3 ± 13.9%. Some 32.7% of patients showed a calculated DGF risk ≥30%. The mean calculated DGF risk was significantly different between patients with DGF (31.9 ± 13.4%) and those without (22.4 ± 13.1%, p < 0.001).
Logistic regression demonstrated that calculated DGF risk was related to DGF development when analyzed both as a continuous variable (OR 1.051 per unit increase in DGF risk, 95% CI 1.033–1.070, p < 0.001) and as a dichotomous variable with a risk above 30% (OR 3.387, 95% CI 2.088–5.494, p < 0.001). The area under the ROC curve to predict DGF was 0.710 (95% CI 0.653–0.767, p < 0.001), which indicates a good degree of discrimination. The Hosmer–Lemeshow “goodness-of-fit” test demonstrates that the DGF risk was well calibrated by the web method, because there was no significant difference (χ2= 9.409, degrees of freedom 8, p = 0.309). In Figure 1 we plotted observed DGF by quintile of calculated DGF risk.
Web-calculated risk was inversely related with creatinine reduction ratio on day 2 (r =−0.246, R2= 0.060, p < 0.001), with the diuresis on day 1 (r = -0.261, p < 0.001), and directly related with the second day resistance index available (r = 0.210, R2= 0.044, p = 0.012), both of them early markers of renal graft status. Similarly, it was related with the renal function throughout the first year: creatinine on day 2 (r = 0.440, R2= 0.193, p < 0.001) and day 5 (r = 0.288, p < 0.001), creatinine on month 6 (r = 0.315, p < 0.001) and month 12(r = 0.271, p < 0.001).
Figure 2 shows the Kaplan–Meier graft survival analysis of patients with a calculated DGF risk lower than 30% and ≥30%. Graft survival was significantly better for patients with a lower DGF risk (1-year survival 88.9% vs. 84.5%; 5-year survival 71.1% vs. 60.1%, log rank p = 0.036). According to the Cox regression model, a calculated DGF risk ≥30% has a negative impact on graft survival (HR 1.504, 95%CI 1.084–2.086, p = 0.015).
The sensitivity, specificity and positive predictive value of a calculated DGF risk ≥50% were 9.4%, 96.2% and 52.6%. The sensitivity, specificity and positive predictive value of a calculated DGF risk ≥30% were 51.8%, 75.8% and 49.1%.
In spite of trying to validate Irish's score in a totally different European population of a single center, the new score performed well to predict DGF. First, global calculated risk (26%) and the observed DGF rate were similar (30.9%). Second, there were significant differences in the mean values of the calculated DGF risk between patients with and without DGF. Single-center experiences with the Irish's 2003 nomogram showed conflicting results. While Grossberg et al. did not find significant differences in the predicted risk between DGF-positive and DGF-negative patients (0.45 ± 0.14 vs. 0.40 ± 0.14, p = 0.07) in a US population, Moore et al. detected significant differences in the nomogram score between both groups (141.3 ± 16.7 vs. 129.5 ± 15.5, p < 0.01) in a European population (16,18). Third, patients with an expected DGF risk over 30% showed a greater than three times risk of suffering DGF.
However, to assess the performance of a new statistical prediction model, we must determine discrimination and calibration (21). On the one hand, discrimination can be quantified with the area under the receiver operating characteristic curve. Similar to the 2010 Irish report (0.704), we found a c statistic of 0.710, indicating that more than 70% of patients were adequately classified (10). Validation studies of the Irish's 2003 nomogram showed similar values over 0.7 (16,17), whereas contradictory studies did not report a c index (18). An independent DGF score developed by Jeldres et al. showed a slightly higher c index (15). On the other hand, the Hosmer–Lemeshow “goodness-of-fit” test was used to quantify calibration, finding that there was no significant difference between calculated and observed DGF. In this way, Irish's new score performed well in a single center, with both good discrimination and calibration.
In the calibration plot (Figure 1) it was possible to see a near perfect prediction over the 45° line. For instance, in the fourth quintile the mean calculated DGF risk was 29.9% and the observed DGF rate was 32.5%, whereas in the first quintile the mean calculated DGF risk was 9.2% and the observed DGF rate was 9.1%. As expected, the score was better correlated with DGF at higher scores. The specificity and positive predictive value were higher in patients with a predicted risk over 50% than in those over 30%. Moore et al. previously reported that DGF prediction is more reliable in patients with extreme scores. Targeting therapeutic or diagnostic strategies to “high-DGF-risk” groups may be of particular benefit, thereby improving results (16).
The new score was associated with other aspects of early allograft outcome such as diuresis on day 1, serum creatinine on days 2 and 5, the creatinine reduction ratio on day 2 and resistance index at day 2 (5,22). The relationship of these markers with the calculated DGF risk had not been analyzed in the Irish et al. report (10). Although all these variables were significantly related with the estimated DGF risk, the new score accounted only for 19%, 6% and 4% of the variability of creatinine on day 2, creatinine reduction ratio on day 2 and resistance index at day 2, respectively. Similar findings were reported by Moore et al. They found that the previous Irish nomogram explained 18% and 15% of the variability of CRR2 and DGF, respectively (16). As pointed out by Kubal and Bhati, “postimplantation factors such as ischemia/reperfusion injury and immune activation may have an influence on DGF, which in turn may have an impact on the diagnostic potential of pretransplantation predictive models” (23).
One of the important findings of the Irish et al. study was that a twofold increase in calculated DGF risk is associated with a 30% increase in the risk of graft failure, irrespective of whether DGF occurs or not (10). In a similar way, we found that patients with a predicted probability of DGF over 30% are associated with a 50% increase in the risk of graft failure versus patients with a predicted probability of DGF less than 30%. Moreover, Irish's score was related with two surrogate markers of long-term allograft outcome, such as creatinine on months 6 and 12 (24).
However, is it possible to use this DGF risk calculator to take clinical decisions in individual patients? Although the mean values of calculated DGF risk were significantly different between patients with and without DGF, we found some overlap in DGF risk prediction, which limits the utility of the score for individual patients. Besides, according to the ROC curve, approximately 30% of patients can be misclassified in the DGF or non-DGF group. Specificity and positive predictive value for different cut-offs are not good enough to change clinical decisions, although the new score performed better at higher risk points. Targeting therapeutic strategies to reduce DGF in the group of patients with higher DGF risk may be of particular benefit, but this issue must be clarified in further studies. As pointed out by Irish et al., the model should not be used as a basis for clinical decisions but can be used to complement the decision making process (10).
There are several limitations to our study. First, inherent to all DGF studies, the need of dialysis within the first posttransplant week is not an objective definition. It is unlikely that any statistical model can predict more accurately the probability of such a subjective endpoint (10). Second, the sample size is relatively small when compared to large population transplant registries. Data are sourced from a single center over a period longer than reported by Irish et al. (10). The good discrimination and calibration of the web-based score in our single center population both emphasize that the new score can be used in different populations than it was originally reported, even in single centers. Because of the size of our sample we did not analyze the performance of Irish's new score in special subgroups of patients (e.g. expanded versus standard criteria donors). Last, Moore et al. showed that Irish's 2003 nomogram was also associated with DGF duration, highlighting the relationship between the score and DGF (16). Lack of information about DGF duration prevents us analyzing this relationship.
Nowadays, predicting graft outcome is not an exact science and the final decision to allocate and transplant a given kidney to a specific recipient still holds a significant degree of uncertainty (25). Irish et al. developed a web-based DGF risk calculator for predicting the likelihood of DGF. We validated this model by applying it to a smaller population of our single transplant center. We found that the model performed well with good discrimination ability and good calibration to predict DGF. Using the web-based DGF calculator, we can predict the risk of developing DGF with a moderate to high degree of certainty only by using information available at the time of transplantation. Moreover, the score was related with different markers of early and long-term renal function and with graft survival. Although it must not be used to take clinical decisions in individual patients, Irish's new score is the best tool that we have to predict DGF in a group of patients.
This work was supported by ISCIII (REDINREN 06/16). Fundación Marqués de Valdecilla-IFIMAV.
The authors of this manuscript have no conflicts of interest to disclose as described by the American Journal of Transplantation.