Development and validation of a multivariable prediction model in pediatric liver transplant patients for predicting intensive care unit length of stay

Liver transplantation is the life‐saving treatment for many end‐stage pediatric liver diseases. The perioperative course, including surgical and anesthetic factors, have an important influence on the trajectory of this high‐risk population. Given the complexity and variability of the immediate postoperative course, there would be utility in identifying risk factors that allow prediction of adverse outcomes and intensive care unit trajectories.


| INTRODUC TI ON
In the pediatric population, liver transplantation is the life-saving treatment for end-stage liver failure, acute liver failure, several liver tumors, a spectrum of metabolic conditions, and other less frequent etiologies.2][3] However, early postoperative care of this patient population remains challenging due to the multisystem pathophysiologic changes that result from the underlying hepatic dysfunction as well as the physiologic stresses imposed by the surgical procedure itself. 4,5A review of an international database of over 2000 pediatric liver transplant recipients revealed that the most significant factors in predicting 6-month patient mortality and graft loss were surgical complications early in the perioperative period. 6Timely identification of patients who are at risk of perioperative complications would enable earlier interventions to mitigate adverse outcomes.To date, there are no clinical tools that accurately identify at-risk patients in the early postoperative period in the intensive care unit (ICU).The risk scores currently utilized [pediatric end-stage liver disease (PELD), Child-Turcotte-Pugh (CTP), and pediatric risk of mortality (PRISM-III)] were not designed for the purpose of predicting perioperative complications in the pediatric liver transplantation population. 7Several studies have attempted to determine risk factors for poor outcomes including postoperative mortality, graft loss, 8,9 acute kidney injury, 10,11 prolonged mechanical ventilation, 12 and graft vascular complications. 13Hospital length of stay (LOS) functions as a surrogate marker of early postoperative complications. 14LOS is an important indicator for a patient's postoperative clinical trajectory and a determinant of hospital resource utilization. 14[16] While several studies have attempted to incorporate risk factors from both the pretransplant and intraoperative phases of care when developing risk models, most studies fail to demonstrate a sequential contribution of risk factors to patient outcomes as they progress through the perioperative period (preoperative, intraoperative, and postoperative).Specifically, few studies have been able to demonstrate how preoperative risk changes over the trajectory of a pediatric liver transplant patient's acute perioperative course.
Furthermore, no validated risk scores exist for the prediction of early perioperative outcomes in the liver transplant population.
The objective of this study was to develop and validate risk prediction model of prolonged ICU LOS in the pediatric liver transplant population using factors that span the perioperative transplant clinical course (preoperative, intraoperative, and in-hospital postoperative).

| Study population
Research Ethics Board approval was attained from The Hospital for Sick Children (REB# 1000062862).We performed a retrospective analysis of all consecutive recipients of a pediatric isolated liver Preoperative, intraoperative, and early postoperative (perioperative transplantation period) recipient and donor characteristics were extracted from scanned paper charts and electronic medical records.All collected variables are listed in Table 1 and lab values in Table S1.We did not include calculated PELD or model for end-stage liver disease (MELD) scores in our analysis because the individual variables of the scores were included within the dataset.

| Clinical protocols
The institutional immunosuppression protocols during the study period were unchanged and followed by all study participants. 17Dual therapy consisting of tacrolimus and corticosteroids (with a steroid taper over 3 months to discontinuation) was the standard induction.
Patients with renal compromise received a kidney-sparing protocol until normalization of renal function. 18This consisted of corticosteroids, either antithymocyte globulin or two doses of basiliximab, and delayed initiation of tacrolimus at normalization of renal function.
For patients with hepatoblastoma as a primary diagnosis, sirolimus was used to replace tacrolimus starting at postoperative day 30. 19nclusions: We develop and validate a model to predict prolonged intensive care unit length of stay in pediatric liver transplant patients using risk factors from all phases of the perioperative period.

K E Y W O R D S
length of stay (LOS), pediatric liver transplant, perioperative risk score

| Outcome
The primary outcome in the study was ICU LOS greater than 7 days.
ICU LOS was chosen as the primary outcome in this study as it was considered representative of postoperative complications and outcomes. 20Furthermore, one-third of the patients in our derivation cohort had a LOS greater than 7 days and represents the longest one-third of ICU LOS; therefore, it was chosen as a marker of prolonged ICU LOS.  a binary indicator was developed from continuous and categorical variables that were dichotomized.Clinical experience as well as the ability to provide high sensitivity and specificity for discrimination of cases with and without prolonged ICU LOS >7 days were used for the dichotomization.Independent predictors were determined using a backward selection using p > .05for removal.The findings are presented as the coefficient regression and standard error (SE), OR and the associated 95% confidence interval (CI), as well as p values.Differences between those with/without ICU LOS >7 days was measured using the overall area under the receiver operating characteristic curve (AUC).

| Model derivation and statistical analysis
Cumulative AUCs were reported to assess the cumulative contribution of each variables included in our regression model.
A risk stratification score to predict ICU LOS >7 days was designed on the basis of predictors obtained from the multivariable logistic regression.The regression coefficient was multiplied and rounded to the nearest integer to attain risk scores of +1, +2, +3.
The area under the ROC curve was used to determine the discriminative ability of the risk score in the derivation cohort.For the risk stratification score, the predicted probability versus observed frequency of prolonged ICU LOS >7 days was represented using a smooth nonparametric calibration line that was created with the locally weighted scatterplot-smoothing algorithm.
Internal validation was performed to assess for possible op-

| RE SULTS
A total of 186 patients were included in the study with 130 comprising the derivation cohort and 56 included in the validation cohort.In our derivation cohort, 34% (44 out of 130) of patients experienced an ICU LOS >7 days.Demographic and comorbidity differences between the LOS cohorts are summarized in Table 1.The most common indication for liver transplant was cholestatic diseases, of which the most common etiology was biliary atresia (overall 38% patients).
Both groups had similar donor characteristics with no significant difference in donor type, graft type, and ischemic times.Deceased donor grafts were used in 37.7% of patients while 62.3% of patients received living donor grafts.There was a statistically significant difference between the ICU LOS cohorts in age, weight, height, days in hospital prior to transplant, and intraoperative transfusion requirements (Table 1).Furthermore, patients with a prolonged ICU LOS also demonstrated a statistically significant longer number of days of mechanical ventilation, hospital LOS, readmissions to the ICU, graft rejection episodes, and return to the operating room (Table 2).
Allograft rejection was diagnosed from histopathological interpretation following liver biopsy in seven patients.All patients that required return to the operating room during the hospital admission were included as it was felt that return to the operating room would be representative of complexity in the postoperative phase of care.
The exception to this were PICC line insertion, diagnostic imaging TA B L E 2 Outcomes in children undergoing liver transplantation in the derivation cohort.

thrombosis). An extended list of collected lab variables and univari-
ate results are presented in Table S1.
Using multivariable logistic regression, we found that We next used a temporal validation cohort of 56 children to evaluate the discriminative capacity of our risk stratification score.
The developed risk score demonstrates a good discrimination to predicting ICU LOS >7 days in the validation cohort with an AUC of 0.878 (95% CI, 0.784-0.972).Calibration assessment was performed graphically and shows good model calibration (Figure 2B).

| DISCUSS ION
In this single-institution retrospective study, we develop and validate a risk prediction score for prolonged postoperative ICU LOS in pediatric patients undergoing liver transplantation.We found that 34% of patients had an ICU LOS greater than 7 days and identified recipient age under 12 months at time of liver transplant surgery, metabolic or cholestatic disease, 30-day pretransplant hospital admission, intraoperative RBC transfusion >40 mL/kg, posttransplant return to the operating room, and incidence of major postoperative respiratory event as risk factors for prolonged ICU LOS.Furthermore, the model developed from these risk factors demonstrated a good TA B L E 3 Multivariable risk stratification score to predict prolonged intensive care stay in the derivation cohort.  of having an ICU LOS >7 days (Figure 1).This finding is important as currently no reliable method exists for predicting ICU LOS following pediatric liver transplantation.
We defined prolonged ICU LOS as an ICU stay longer than 7 days as this was considered to be a clinically significant marker of prolonged stay and 34% of the patients in our patient population required this duration of ICU care after surgery.Furthermore, prolonged ICU LOS serves as a marker of a complex postoperative course 20 and a predictive model can help identify patients who are more likely to require intensive resource utilization.Further, it may prove useful for evaluating impacts on hospital resource utilization and health-economic impacts. 14In our cohort, this outcome captured patients with an increased level of clinical complexity postoperatively.Specifically, we found that patients that were in the ICU for greater than 7 days had longer days of mechanical ventilation, longer hospital LOS, higher readmission to the ICU, increased rate of early graft rejection (acute cellular rejection diagnosed via biopsy), and increased likelihood of needing to return to the operating room.
Such adverse events occurring in the perioperative period are important clinical outcomes in the transplant population and highlight the utility of prolonged ICU LOS as an outcome to risk-stratify patients with a higher likelihood of morbidity in the perioperative period.Furthermore, the cohort of patients that had an ICU LOS of less than 7 days on average had clinical outcomes consistent with a less complicated ICU trajectory.For example, in this cohort, the median length of mechanical ventilation was less than 1 day and only 2% of patients required readmission to the ICU compared to the 7. Finally, return to the operating room and major respiratory event are consistent with the expectation of a prolonged ICU LOS given these factors are likely to require ICU therapies such as mechanical ventilation and have previously been associated with decreased patient and graft survival. 6Furthermore, when we looked at the preoperative disposition of patients, we note that 20 out of 80 (25%) patients that came from home required ICU LOS >7 days.This exemplifies the importance of a dynamic risk score as although these patients come from a less monitored preoperative disposition that may give a perception of better baseline health, the impact of the intraoperative and postoperative phase modifies the risk of prolonged ICU LOS.
The model developed in this study incorporates risk factors from the entirety of the perioperative period (preoperative, intraoperative, and in-hospital postoperative) as opposed to traditional risk scores based primarily on preoperative criteria.As these variables are sequentially included in the model, the predictive power for ICU LOS >7 days improves (Table 3).Furthermore, prior to incorporating transplant performed between April 1, 2013 and April 30, 2020 at the Hospital for Sick Children (SickKids) in Toronto, Canada.This study was approved by the local Research Ethics Board (REB).We included all recipients younger than 18 years of age at the time of liver transplant.Multiorgan transplant recipients were excluded.Patients with date of liver transplant between April 1, 2013 and June 1, 2018 were included in the derivation cohort and all patients transplanted from June 2, 2018 to April 30, 2020 were included in the validation cohort.
Median and interquartile range (IQR) are used to present continuous variables.Categorical variables are expressed as number and percentage.Univariable comparisons were performed using univariable logistic regression analysis.Variables of statistical significance obtained from the univariable analysis (p < .1)were included into the initial multivariable logistic regression model.Variables of clinical relevance were included into the model as well.For the multivariable logistic regression, TA B L E 1 Demographic and clinical characteristics and comorbidities in children undergoing liver transplantation in the derivation cohort.
timism and overfitting.The data were divided into 10 equal parts for a 10-fold cross-validation.We repeated all steps within each cross-validation fold.An average estimate of overfitting can be determined using this technique.The concerns of variance and bias in internal validation can be balanced using this technique (and k= 10)   as it provides an average estimate of overfitting.The mean AUC and associated 95% CI are reported for the results of the 10-fold crossvalidation for the risk stratification score.Temporal model validation was performed using the June 1, 2018-April 30, 2020, validation cohort of patients who underwent liver transplantation at our institution that were not included in our derivation cohort.The discriminative ability of the risk score in the validation cohort was determined using the area under the ROC curve.The predicted probability versus observed frequency of prolonged ICU LOS >7 days for the risk score in the validation cohort was represented using a smooth nonparametric calibration line that was created with the locally weighted scatterplot-smoothing algorithm.STATA version 16.1 for Mac OS was used for statistical analysis (Stata Corp).
age < 12 months (odds ratio [OR] 4.02, 95% confidence interval [CI] 1.20-13.51,p = .024),metabolic or cholestatic disease (OR 2.66, 95% CI 1.01-7.07,p = .049),30-day pretransplant hospital admission (OR 8.59, 95% CI 2.27-32.54,p = .002),intraoperative red blood cells (RBC) transfusion >40 mL/kg (OR 3.32, 95% CI 1.12-9.81,p = .030),posttransplant return to the operating room (OR 11.45, 95% CI 3.04-43.16,p = .004),and the incidence of major postoperative respiratory event (OR 32.14, 95% CI 3.00-343.90,p < .001)were associated with prolonged ICU LOS >7 days.Major postoperative respiratory event was defined as the occurrence of respiratory failure requiring additional support in intubated patients (HFO, VA ECMO), chylothorax/pleural effusion requiring medical/procedural intervention, apnea/hypopnea leading to intervention (positive pressure ventilation, BiPAP), and reintubation during the postoperative ICU admission.The cumulative AUC are reported in Table3and demonstrate improvement in the value as factors are incorporated in the entire patient perioperative course.Overall, the multivariable logistic regression model used to predict the incidence of prolonged ICU LOS >7 days showed a good discriminative ability with an AUC of 0.888 (95% CI, 0.824-0.951).We developed a risk stratification score ranging from 0 to ≥5 where incremental increases in the score demonstrate increased likelihood of ICU LOS >7 days.This was developed based on the results of the multivariable logistic regression.Figure1demonstrates the distribution of the risk stratification score and the associated observed incidence of prolonged ICU LOS >7 days in the derivation cohort.The calibration plot for predicted probability versus observed frequency of prolonged ICU LOS >7 days for the risk stratification score is demonstrated in Figure 2A.The risk stratification score 10fold cross-validation results show a mean AUC of 0.844 (95% CI of 0.729-0.916).

F I G U R E 1
Risk score to predict ICU LOS >7 days.The percentage of patients in each risk score stratum is represented by the bars on the graph with the percentage labeled on the left y-axis and the risk score labeled on the x-axis.The circle and dotted lines represent the percentage likelihood of ICU LOS >7 days for each risk score stratum and is labeled on the right y-axis.discriminative ability of predicting a prolonged ICU LOS [AUC of 0.888 (95% CI, 0.824-0.951)].We subsequently developed a risk score that demonstrated an increase in predicting likelihood of prolonged ICU LOS based on incremental increases in the score.For example, we demonstrate that a score of 4 leads to a > 80% likelihood the postoperative factors, there is already good discriminative ability of the model based on the preoperative and intraoperative factors, highlighting the ability for clinically useful risk stratification of patients on arrival to the ICU.This concept underscores the importance of our proposed dynamic risk model that can predict patient trajectories based on factors throughout the course of the perioperative phases of liver transplant surgery, rather than relying only on F I G U R E 2 Calibration plot for predicted probability versus observed frequency of prolonged ICU LOS >7 days.(A) This is the calibration plot for the risk stratification score in the derivation cohort.It demonstrates the predicted probability versus observed frequency of ICU LOS >7 days.(B) This is the assessment of graphically performed calibration in the temporal validation cohort for ICU LOS >7 days.
Data are presented a regression coefficient (Coeff.)and standard error (SE), odds ratio (OR) and 95% confidence interval (CI), and area under the receiver operative curve (AUC).