Frailty in the prediction of delirium in the intensive care unit: A secondary analysis of the Deli study

Delirium is an acute disorder of attention and cognition with an incidence of up to 70% in the adult intensive care setting. Due to the association with significantly increased morbidity and mortality, it is important to identify who is at the greatest risk of an acute episode of delirium while being cared for in the intensive care. The objective of this study was to determine the ability of the cumulative deficit frailty index and clinical frailty scale to predict an acute episode of delirium among adults admitted to the intensive care.


| INTRODUCTION
Delirium has been described as an acute change in behavior that is characterized by a fluctuating level of consciousness with impaired attention and cognition. 16][7] An acute episode of delirium is associated with a longer hospital stay ($10 extra days) and a prolonged duration of mechanical ventilation. 8,9The longer-term outcomes include ongoing cognitive impairment, dependency in activities of daily living and two to three times higher mortality rates, [8][9][10][11][12][13][14][15] high direct costs, and even higher indirect costs, due to loss of healthy life years. 16,17Importantly, patients who experience hypoactive delirium, have been shown at the greatest risk of long-term cognitive impairment. 8,18sed on evidence suggesting that at least 30% of episodes of delirium among older adults admitted to general hospital wards are preventable 19 we undertook a cluster randomized stepped-wedge trial to assess the effectiveness of a nurse-led non-pharmacological intervention in the adult ICU setting. 20Our study was unable to show a clear benefit in reducing the incidence of delirium, highlighting a strong association between being frailty at the time of admission to ICU and the risk of delirium. 7Routine screening for frailty in the ICU setting is not undertaken in all settings and when undertaken can be based on various forms of frailty screening tools.In the clinical setting the Clinical Frailty Scale 21 is common, however using routinely collected coded data an electronic frailty index, based on cumulative deficits, has been developed. 22Given we estimated frailty using both approaches in the Deli interventional study, we wanted to compare the ability of theses frailty screening tools to predict delirium.Therefore, this study was designed to determine the ability of the cumulative deficit frailty index and clinical frailty scale, and other characteristics at the time of admission, to predict an acute episode of delirium among adults admitted to intensive care.

| Subjects and setting
The Deli interventional study 20,23 was undertaken across the South-Western Sydney Local Health District that provides public hospital services for $1 million residents, with five acute care hospitals, with $230,000 separations each year.There are four adult intensive care units across the local health district, that have between 80 and 250 admissions each month.

| Recruitment and selection of participants
Study participants were part of the Deli intervention study, a hybrid stepped-wedge cluster randomized controlled trial to assess the effectiveness of the nurse-led intervention to reduce the incidence and duration of delirium among adults admitted to the four adult intensive care units in the south-west of Sydney, Australia. 23

| Inclusion and exclusion criteria
The initial trial registration (ACTRN12618000411246) exclusion criteria was extended to include a number of further specific conditions (traumatic brain injury, intra-cerebral hemorrhage, ischemic stroke, hypoxic brain injury, and dementia) that would prevent the assessment of delirium and were applied in the Deli interventional study. 20,23The Australian New Zealand Clinical Trials Registry has been updated to include the additional exclusion conditions.Final inclusion and exclusion criteria being: all patients admitted to the four adult ICUs during the study period were included in the study, except for patients among which delirium assessment was considered by the clinical staff as being impractical or inappropriate, for example: (1) patients at the end-of-life, and not expected to survive 24-h; (2) patients not expected to stay in the ICU for at least 24-h; (3) patients with acute or chronic neurological conditions that prevented assessment of delirium (traumatic brain injury, intracerebral hemorrhage, ischemic stroke, CNS infection, hypoxic brain injury, hepatic encephalopathy, severe mental disability, serious receptive aphasia, and dementia).

| Delirium screening
Delirium screening involved an overall assessment of consciousness, which involved two steps: firstly, assessment of the level of FROST ET AL. consciousness was undertaken using the Richmond Agitation-Sedation Scale (RASS). 24This scale is used as standard practice in all intensive care units that participated in the study.Patients who are in a coma or stupor (RASS À4 or À5) do not undergo further assessment.Among patients with a RAAS À3 or greater), further assessment of delirium assessment was undertaken using the CAM or CAM-ICU. 25,26In the Deli interventional study delirium status was assessed each shift using the CAM or CAM-ICU by nursing and medical staff, or when there was an acute change in the mental status of the patient.

| Primary outcome of interest
The primary outcome of interest was at least one episode of acute delirium in the intensive care.Patients were classified as having an incident case of delirium when they had at least one positive delirium screening episode using the CAM or CAM-ICU, or when they were actively treated for delirium with haloperidol, dexmedetomidine, olanzapine, or any other delirium treatment agent.

| Frailty assessment
Approaches to screening for frailty have generally followed the phenotype or cumulative deficit approach. 27The clinical frailty scale has been developed to screen patients in the clinical setting, especially in the emergency department.While the approach to screen for frailty using cumulative deficits has traditionally been in the context of a comprehensive geriatric assessment setting. 27,28In our adult ICU setting, frailty status is routinely assessed on admission to the ICU by the admitting medical officer, either directly from the patient, or their family and from review of previous medical notes, using Rockwood's Clinical Frailty Scale. 21Frailty status is based on the patient's level of physical function in the 2 months before their admission to the hospital for the index ICU stay.A Clinical Frailty Scale (CFS) of five or more were classified as frail. 21,29We also estimated frailty status using the cumulative deficit approach using the method developed by Clegg et al., 22 using clinical ICD-10-codes (Table A1).Cumulative deficit items were based on pre-existing conditions at the time of admission.
Based on the approach suggested by Clegg four cumulative frailty item groups are presented: 0-1 item; 2 items; 3 items; and 4-14 items.In this approach, the upper 99th population percentile of the indexed values (no. of deficits/36), in our case 4 items, then this range is divided into four equally distanced groups.

| Sample size
We limited our analysis to the first admission among study participants (n = 2566) during the 10-month study period, with 313 incident cases of acute delirium.Based on recommendations of at least 10-20 events per predictor, 30,31 our final models could potentially incorporate up to 12-15 predictor variables.

| Statistical analysis
Characteristics of adult patients included in our analysis are presented using descriptive statistics.The rates of delirium events and odds ratios (OR) are presented with associated 95% confidence intervals (95% CI).Potential predictors of delirium present at the time of admission to intensive care, were identified using a bootstrap method, 32 with each bootstrap sample incorporating a backward and forward deletion method with a generous p-value (.2) to retain potential predictors.This procedure was repeated 500 times to assess the frequency with which potential predictors were selected.Predictors retained in at least 70% of the bootstrap samples were included in our final models. 32The cumulative frailty deficit index and clinical frailty scale were initially planned to be forced into models to assess their given discriminative ability to predict delirium, however, both were retained in at least 70% of the bootstrap sample models.Furthermore, even though intervention status (post vs. pre) did not reach statistical significance in the Deli interventional study results, intervention status was forced into all models.
The ability of the final model to discriminate between individuals with an acute episode of delirium and those without was assessed by the area under the receiver operating characteristic (AUC) curve. 33An area of 1.0 reflects perfect discrimination, and an area of 0.5 reflects discrimination no better than random choice.Internal validation of the final predictive model included bootstrap methods, to assess how accurately the model would predict delirium in a similar population of patients.In this method, a sub-sample of 50 patients was used to create a training model that was then applied to the whole data set to estimate biases between the observed and predicted rates of the outcome.This procedure was repeated 200 times to create a distribution of bias between predicted and observed rates to estimate the mean absolute error. 32In addition to these approaches, due to some limitations of using AUC ROC only, 34 we have estimated the net reclassification of individuals based on prediction models with and without frailty indices. 34,35 approach to estimating the absolute risk of delirium at the time of admission to intensive care, using the final models are presented as nomograms to individualize the absolute risk. 32All analysis was undertaken using the R statistical language 36 and data management was undertaken using SAS (version 9.1).

| Ethical considerations
The study was conducted in accordance with the principles of the Declaration of Helsinki. 37A waiver for individual patient consent for inclusion in the study, being a cluster-randomized trial, hence, allocation to the intervention was at the "cluster" level rather than at the individual patient level, was granted.Ethical review and approval for the study was granted by the South Western Sydney Local Health District Human Research and Ethics Committee (HREC/18/ LPOOL/273).
The characteristics of the 2566 patients included in our analysis, based on delirium status, are presented in Table 1.For example, the average age was 65 years (1-SD 17.6), and the mean age among patients with an acute episode of delirium was higher compared with patients without delirium (72 vs. 64 years, p < .001).A higher frequency of males was observed among those with delirium, compared with those without delirium (63% vs. 55%, p < .001),and a similar increase in frequency with frailty (52% of those with delirium vs. 35% of those without, p < .001).The Charlson index and number of cumulative deficit items were both higher among patients with delirium compared with patients without delirium (both p-values <.001).
Patients with injuries associated with a fracture, admitted from the ward, and higher APACHE III (AP III) scores on admission, were all more common in the acute delirium group (all p-values <.05).Intensive care and hospital length of stay were higher among delirium patients, and mortality was also higher in this group compared with nondelirium patients (all p-values <.001).

| Final prediction models cumulative frailty deficit items or clinical frailty scale
Variables retained in the final models based on 500 bootstrap samples are presented in Table 2 (cumulative frailty deficit items) and Table 3 (CFS).Both final models retained age, sex, and APACHE III scores, however, the model with CFS also retained injury with fractures (Table 3).The AUC for these two models were 0.701 and 0.703  4. For example, the net reclassification of individuals with the addition of the CFS to a basic model including only age, sex, and APACHE III score was estimated to be 20.1% (95% CI 8.9-32.0),and the net reclassification for this approach using the cumulative frailty deficit index was 29.6% (17.7-41.4).We have been to show that both the cumulative deficits frailty index and CFS predict an acute episode of delirium among adults admitted to the intensive care unit.Importantly, these two approaches to screening for frailty on admission to intensive care are easy to collect, and in the instance that the CFS is not already routinely collected in an adult intensive care, the cumulative deficits approach can easily be collected from routinely collected hospital data.For this reason, either of these indices could be incorporated into predicting who is at the highest risk of an acute episode of delirium while in intensive care and support targeted prevention strategies, depending on local resources and preferences.

| Nomograms to estimate the absolute risk of an acute episode of delirium
Common approaches to the diagnosis of delirium in the intensive care setting, with assessment tools such as the CAM-ICU and the Intensive Care Delirium Screening Checklist (ICDSC), have been shown to be acceptably accurate (sensitivity >95%, and specificity >90%), 38 with their use being recommended in clinical practice guidelines. 39The use of prediction models has shown promise in predicting several types of delirium, including postoperative and subsyndromal delirium as well as delirium in the ICU. 40Frailty status has been identified as an important predictor in the surgical setting 41,42 and among trauma patients, 43 however, not in the general intensive care population.An important innovation in the intensive care setting appears to be the use of machine learning and the use of dynamic prediction models. 44These approaches appear to be able to create dynamic risk prediction estimates, that change during an individual's intensive care stay.
Outside of intensive care, it has been shown that multicomponent interventions are effective in preventing an acute episode of delirium. 45,468][49][50][51] For this reason, innovative approaches to identifying adults who are at the greatest risk of an episode of delirium while in intensive care, and targeting these patients with prevention strategies may improve the effectiveness of current delirium prevention models of care.
The results of our study need to be considered in the context of some potential weaknesses.For instance, generalizability of results as our rates of delirium were low compared with previous reports in the Australian ICU setting.On average 12%-14% among a population of adults ICU admissions with a rate of mechanical ventilation of 16%, previous estimates of the rate of delirium in the Australian setting have been as high as 20%, among patients with a 50% mechanical ventilation rate (Ankravs et al. 5 ).This low prevalence we feel may be due to our inclusion and exclusion criteria, that targeted patients that it was thought delirium screening would be feasible.Furthermore, we did not have details of pre-admission cognitive status.
In terms of prediction modeling, although the use of nomograms to predict individualized patient risk is widely reported in cancer research, [52][53][54][55][56][57][58] and sporadically in other clinical settings, [59][60][61][62]  error of the final models to predict the outcome in a similar patient population was to be <1.2%(the maximum mean difference between the predicted probabilities and observed frequencies), which is quite an acceptable error. 32The AUC from the final model used to develop the nomogram was 0.70, which is considered adequate for a model to predict the outcome in a similar external population. 32 the Australian ICU setting, the $175,000 admissions to our 218 adult intensive care units (ICU) each year represent a large population at risk of delirium. 64Given the approximate 30% prevalence of frailty among older admissions, and potentially 50% of these patients with a significant acute illness experiencing delirium, identifying those at the highest risk and focusing prevention strategies to prevent or reduce the severity of delirium will have an important impact on mortality, morbidity, mechanical ventilation days, ICU length of stay and hospital length of stay.We feel that explicitly estimating the absolute risk of delirium, at the time of admission, maybe a useful approach to target patients at the greatest risk.Our results indicate that the addition of frailty in the prediction of delirium risk will shift individuals from lower-risk to higher-risk categories and allow targeted interventions to prevent delirium.Such an approach would need to be tested in a randomized trial of absolute risk communication and could explore innovative approaches based on dynamic machine learning models. 44 conclusion, we have been able to show that both the cumulative deficits frailty index and clinical frailty scale predict an acute episode of delirium among adults admitted to intensive care.

(
moderate discriminatory ability), with estimated mean absolute errors of 0.006 and 0.011, respectively.A model with APACHE III, age and sex only has an associated AUC of 0.690.Study participants individual predicted absolute risk of an acute episode of delirium is based on age, CFS (top panel) or number of cumulative deficit items (bottom panel), and quartiles of AP III score are presented in Figure A1.The reclassifications of individuals based on models with and without frailty indices are presented in Table Nomograms to estimate an individual's absolute risk of an acute episode of delirium based on age, number of cumulative deficit items (top panel) or CFS (bottom panel) and injury with fracture, and APACHE III score, are presented in Figure A2.For example, using the bottom panel, a 70-year-old (4 points) male (1 point), with an injury associated with a fracture (2 points), CFS = 5 (2 points), and APACHE III score of 60 (3 points), giving total points = 12, has an estimated 20% risk of delirium while in the intensive care.The predicted absolute risk of an acute episode of delirium and associated 95% CI, based on age, sex, number of deficit items (top panel) or CFS (bottom panel), and quartiles of APACHE III score are presented in Figure A3.T A B L E 1 Characteristics of study participants based on delirium status.

F I G U R E A 1
Study participants individual predicted absolute risk of an acute episode of delirium is based on age, CFS (bottom panel) or no. of cumulative deficit items (top panel), and quartiles of III score (right vertical axes).F I G U R E A 2 Nomograms of absolute risk of an acute episode of delirium based on age, number of cumulative deficit items (top panel) or CFS (bottom panel) and injury with fracture, and AP III score.For example, using the bottom panel an aged 70 (4 points) male (1 point), with an injury associated with a fracture (2 points), CFS = 5 (2 points), and AP III score of 60 (3 points): total points = 12, has an estimated 20% risk of delirium while in the intensive care when compared with the Predicted Value axis.Using the top panel and replacing the CFS of 5 with a 4-14 number of deficit items results in an absolute risk of 15%.F I G U R E A 3 The absolute risk of an acute episode of delirium is based on age, number of deficit items (top panel) or CFS (bottom panel), and quartiles of AP III score (apiii.qrt).
Cumulative frailty deficit items: Observed rates, crude, and adjusted risk of delirium.
T A B L E 3 Clinical frailty scale, injury status: Observed rates, crude, and adjusted risk of delirium.
Reclassification of risk based on the addition of a frailty index.
63del would be well calibrated.Second, it depends on the discriminatory ability of the model, which in predicting a binary outcome is reduced to the proportion of all pairs of patients (one with delirium and one without), and the ability of the model to assign higher risk in individuals with the outcome: the area under the ROC curve (AUC).63Inthisstudy, which used bootstrap methods, the mean calibrationT A B L E 4