The kidney failure risk equation predicts kidney failure: Validation in an Australian cohort

Abstract Aims Predicting progression to kidney failure for patients with chronic kidney disease is essential for patient and clinicians' management decisions, patient prognosis, and service planning. The Tangri et al Kidney Failure Risk Equation (KFRE) was developed to predict the outcome of kidney failure. The KFRE has not been independently validated in an Australian Cohort. Methods Using data linkage of the Tasmanian Chronic Kidney Disease study (CKD.TASlink) and the Australia and New Zealand Dialysis and Transplant Registry (ANZDATA), we externally validated the KFRE. We validated the 4, 6, and 8‐variable KFRE at both 2 and 5 years. We assessed model fit (goodness of fit), discrimination (Harell's C statistic), and calibration (observed vs predicted survival). Results There were 18 170 in the cohort with 12 861 participants with 2 years and 8182 with 5 years outcomes. Of these 2607 people died and 285 progressed to kidney replacement therapy. The KFRE has excellent discrimination with C statistics of 0.96–0.98 at 2 years and 0.95–0.96 at 5 years. The calibration was adequate with well‐performing Brier scores (0.004–0.01 at 2 years, 0.01–0.03 at 5 years) however the calibration curves, whilst adequate, indicate that predicted outcomes are systematically worse than observed. Conclusion This external validation study demonstrates the KFRE performs well in an Australian population and can be used by clinicians and service planners for individualised risk prediction.


| INTRODUCTION
Chronic Kidney Disease (CKD) is common with an increasing incidence globally. 1 Patients with CKD, however, represent a highly heterogeneous cohort. There is great variation in rates of disease progression, with the prevalence of patients with CKD who reach kidney failure being 0.1% of all patients. 2 Being able to predict who will progress to kidney failure is important at a patient level, a clinician level, and a policy level. Understanding the risk of progression to kidney failure, allows patients and clinicians to make informed decisions about future care: ranging from the implementation of measures to slow kidney function decline, decisions about kidney replacement therapy (KRT) including when to start dialysis education, the timing of arteriovenous fistula creation and pre-emptive kidney transplantation. 3,4 It is also important on a population level for resource allocation and workforce planning. This need for prognostication has led to the need for more individualised prediction tools. The use of validated risk prediction tools is now recommended by the international Kidney Disease Improving Global Outcomes (KDIGO) guidelines. 5 Tangri et al developed a kidney failure risk prediction tool or equation within a Canadian cohort of CKD patients in 2011. 4 The Kidney Failure Risk Equation (KFRE) includes the use of routinely obtained demographic and pathology data to predict the progression of CKD to kidney failure (starting KRT). The KFRE was validated within the Canadian population and found to be more accurate at predicting progression to kidney failure than the previously used combination of estimated glomerular filtration rate (eGFR) and albuminuria alone. 4 The utility of the KFRE prediction model was further validated in a multinational assessment of 31 cohorts spanning 4 continents, which demonstrated high discrimination and adequate calibration. 6 This study determined that for some populations the equation needed a calibration adjustment factor before use. 6 The KFRE, however, has not been independently validated in an Australian population.
There are differences between the Australian 7 and Canadian cohorts with differences in clinical practice. 8 It is therefore unclear whether we can apply the score in our population without a calibration adjustment factor. The KFRE is available online 9 and its simplicity means it can be automated into electronic health records. 10 Several potential uses are identified including triaging for speciality nephrology care, identification of people who are at high risk of CKD progression and treatment planning. 11 The purpose of this study is to externally validate the KFRE in an Australian cohort, to evaluate its performance and therefore whether it can be used to guide management decisions for patients and clinicians in the Australian population.

| Study population
The Tasmanian Chronic Kidney Disease study (CKD.TASlink) is a retrospective cohort study of a dataset created through linkage of seven existing local health information datasets. Detailed methods and linked data obtained are available in separate methodology papers. 12,13 Linkage to the Australia and New Zealand Dialysis and Transplant Registry (ANZDATA), the Tasmanian public hospital admitted patient dataset, Tasmanian public hospital emergency presentation dataset, Tasmanian cancer registry and the Tasmanian death   registry was performed by the Tasmanian Data Linkage Unit (see sup-plementary methods). Diabetes was derived from ICD-10 diagnosis codes via Elixhauser comorbidity categories, or if a pathology result for Haemoglobin A1C > 6.5% or Fasting Glucose >7.0 mmol/L was recorded. The 6 variable equations rely on the past medical history of diabetes or hypertension. For this cohort we only used those who had linked data.

| Diagnosis of CKD
Chronic kidney disease in individuals 18 years and older was defined using KDIGO criteria, 5 requiring two eGFR <60 mL/min/1.73m 2 90 days apart. CKD severity was categorised by eGFR and urine ACR (uACR) by the KDIGO CKD staging. Tasmanian laboratories use enzymatic assays for measurement of creatinine, and immunoassay for urinary albumin, all results are isotope dilution mass-spectrometry (IDMS) aligned as previously reported. 14 eGFR was calculated using the 2009 CKD-EPI creatinine equation. 15 Those who had had a previous kidney transplant were excluded from the initial cohort.

| Risk scores
The three Tangri

| External validation
We externally validated the KFRE equations at 2 and 5-years using 3 measures 10 ; (i) Model fit (ii) Discrimination and (iii) Calibration.
Model fit: Model fit is how well the statistical model fits the outcome. We visually assessed this using goodness of fit and plotting the predicted risk compared to the observed risk.
Discrimination: Discrimination is how well a model can differentiate between individuals for an outcome. 10, 16 We assessed discrimination of the KFRE using the Harrell's C statistic. 17 This concordance statistic is a measure of discrimination; a value of 0.5 indicates no difference to chance alone and 1 indicates perfect discrimination.
Calibration: Calibration is how accurately the score predicts an outcome. To allow comparison of risk groups the KFRE scores were categorised into risk quintiles for both 2-year and 5-year risk scores with levels based on the original Tangri et al paper. 4 We assess calibration in two ways: 1. Plotting the observed 2-year and 5-year probability of KRT and comparing it to predicted risk derived from the risk equations.
2. Calculation of the Brier Scores, the mean squared difference between the predicted risk vs observed binary outcomes. A Brier score of 0 indicates perfect calibration and a score of 1 indicates no calibration. 18

| Recalibration
Equations used in different populations can require recalibration for different demographic and geographic setting. We recalibrated the equation, using a post-estimation prediction of the survival function from the data. 19

| Sensitivity analysis: Alternate outcome events
Typically, the KFRE is used to predict the risk of proceeding to kidney failure requiring KRT. Due to the low utilisation of renal replacement therapy in Tasmania compared to other populations, 7 two alternate endpoints (eGFR <10 and <7.5 mL/min/1.73m 2 ) were implemented to ensure the robustness of results; the methods for these are described in the supplementary appendix. The cut-off of 7.5 mL/min/1.73m 2 was chosen based on the median eGFR start of 7.5 mL/min/1.73m 2 for Australians starting Kidney Replacement Therapy 20 and the cutoff of 10 mL/min/1.73m 2 aligns with the secondary outcome used when originally developing the equation. 6

| Sensitivity analysis: Missing data
For the cohort without a uACR result within the 12 month window of an eGFR, a uACR was imputed using the multiple imputation techniques with 5 imputations with R package mice. 21 The arithmetic mean of the model risks produced was used to pool each person's imputations. The data were assumed to be missing at random.

| Statistical analysis and ethics
The statistical analyses were performed using R statistical software. 22 The C statistics were calculated using the pROC R package. 23 The CKD.Taslink protocol was reviewed and approved by the Tasmanian Human Research Ethics Committee (Approved study H0016499).

| RESULTS
During the 16-year study period, Figure 1

| Calibration
Calibration was assessed in two ways, visually and quantitatively using Brier scores. The calibration performed well on quantitively assessment but demonstrates some miscalibration on visual assessment.

F I G U R E 1 Consort diagram explaining study selection.
T A B L E 1 Baseline characteristics for the cohort. SD standard deviation, eGFR estimated glomerular filtration rate, uACR urinary albumin: creatinine ratio, KRT kidney replacement therapy, IQR Interquartile range

| Recalibration
The KFRE model may benefit from a recalibration whereby the baseline survival rate by updated to represent the local cohort.
The adjusted baseline survival for recalibration was calculated through post-estimation prediction of the survival function from the cohort's data.

| Sensitivity analysis: Alternative outcomes
The sensitivity analysis demonstrated similar outcomes.
Model fit: Figure s4 demonstrated the model fit for the different outcomes. The outcome of KRT or eGFR <7.5 mL/min/1.73m 2 had the best fit with the observed risk vs predicted risk overlying the 45-degree angle.
Discrimination: A number of other studies from Canada, the United Kingdom, Singapore, Korea and Europe have also validated the KFRE and they all showed similarly strong discrimination with C-statistics from 0.83-0.93. 19,[25][26][27][28] For calibration, the KFRE overestimates risk in our population.
The Brier score in our cohort was better than in the multinational validation study for non-north American results with 0.228-0.299. This may be due to the non-north American cohort being a more heterogenous population as it was made up cohorts from multiple different countries. 6 Recalibration, with use of a calibration factor to adjust for systematic differences, can account for the differences in clinical practice. Interestingly, all the independent studies that validated the KFRE showed some degree of miscalibration with overprediction of risk. 19,[25][26][27][28] Three of these studies 18,23,24 and the large validation study by Tangri et al 6, undertook recalibration for use in an external population. 19,25,26 Previous work has demonstrated differences in practice between  29 Our study has demonstrated that few people progress to requiring KRT, and thus this score may assist with alleviating patient anxiety. However, further work exploring consumers' understanding of the risk calculator and whether this could form part of a patient decision aid to improve understanding needs to be explored. 30 The clinical impact of incorporating the KFRE into routine care is still being studied 11 and further work is needed to assess the impact of the score on patientimportant outcomes and the cost-effectiveness of implementation.
This study has several strengths. It is a large validation study that has not previously been performed within an Australian population which may have system-level differences from the original Tangri  of people under 18 years. 12 This older age group represented in the study represents the age group with the greatest prevalence of CKD.
The use of data linkage of routine pathology results supports that the KFRE can be applied in primary care as well as in nephrology practice. This is particularly important as a great proportion of those with CKD are unlikely to see specialistic nephrology services. 32 We have undertaken an extensive validation study of 4,6, and 8 KFRE with multiple sensitivity analyses to ensure the robustness of our findings.
There are some weaknesses to the study. Despite the increased prevalence of CKD, Tasmania has the lowest incidence of kidney failure treated with dialysis or transplantation of any Australian state or territory. 20 In 2018 the incident rate of KRT was 87 per million population (pmp) in Tasmania, but 124pmp for Australia overall. 12 This may be due to practice level differences with older Tasmanian being less likely to have KRT compared to other parts of Australia and may limit the generalisability of these data to the entire Australian population. The limitations of using hospital admissions data to define comorbidities may mean missing comorbidity data, however we have restricted the 6 variable equation to only use those with linked hospital data to try and address this. We used the sensitivity analysis of multiple imputation to deal with the missingness of the uACR data which is an accepted method to account for missing data and increase model power with no change in the outcomes. 33 In conclusion, in this study, we have externally validated the KFRE and demonstrated exemplary discrimination and adequate calibration.
This calibration further improved with recalibration adjustment factor.
These findings support the use of the KFRE for risk prediction in an Australian population by patients and clinicians for clinical decisionmaking as well as health service and workforce planning. Further work is needed on incorporating the KFRE into clinical care, and the resultant patient and health system outcomes.

ACKNOWLEDGEMENTS
The authors would like to thank the following organisations; Diagnos-