Predictive scoring model of mortality in Gram-negative bloodstream infection

Authors

  • M. N. Al-Hasan,

    Corresponding author
    1. Department of Medicine, Division of Infectious Diseases, University of Kentucky, Lexington, KY, USA
    2. Department of Medicine, Division of Infectious Diseases, College of Medicine, Mayo Clinic, Rochester, MN, USA
    • Corresponding author: M. N. Al-Hasan, MBBS, University of Kentucky Medical Center, 740 South Limestone, K512, Lexington, KY 40536, USA

      E-mail: majdi.alhasan@uky.edu

    Search for more papers by this author
  • B. D. Lahr,

    1. Department of Health Sciences Research, Division of Biomedical Statistics and Informatics, College of Medicine, Mayo Clinic, Rochester, MN, USA
    Search for more papers by this author
  • J. E. Eckel-Passow,

    1. Department of Health Sciences Research, Division of Biomedical Statistics and Informatics, College of Medicine, Mayo Clinic, Rochester, MN, USA
    Search for more papers by this author
  • L. M. Baddour

    1. Department of Medicine, Division of Infectious Diseases, College of Medicine, Mayo Clinic, Rochester, MN, USA
    Search for more papers by this author

Abstract

Mortality is a well-recognized complication of Gram-negative bloodstream infection (BSI). The aim of this study was to develop a model to predict mortality in patients with Gram-negative BSI by using the Pitt bacteraemia score (PBS) and other clinical and laboratory variables. A cohort of 683 unique adult patients who were followed for at least 28 days after admission to Mayo Clinic Hospitals with Gram-negative BSI from 1 January 2001 to 31 October 2006 and who received clinically predefined appropriate empirical antimicrobial therapy was retrospectively identified. Multivariable logistic regression was used to identify independent risk factors for 28-day all-cause mortality. Regression coefficients from a multivariable model were used to develop a risk score to predict mortality following Gram-negative BSI. Malignancy (OR 3.48, 95% CI 1.94–6.22), liver cirrhosis (OR 5.42, 95% CI 2.52–11.65), source of BSI other than urinary tract or central venous catheter infection (OR 5.54, 95% CI 2.42–12.69), and PBS (OR 1.98, 95% CI 0.92–4.25 for PBS of 2–3 and OR 6.42, 95% CI 3.11–13.24 for PBS ≥4) were identified as independent risk factors for 28-day mortality in patients with Gram-negative BSI. A risk-score model was created by adding points for each independent risk factor, and had a c-statistic of 0.84. Patients with risk scores of 0, 4, 8, 12 and 16 had estimated 28-day mortality rates of approximately 0%, 3%, 14%, 45%, and 81%, respectively. The Gram-negative BSI risk score described herein estimated mortality risk with high discrimination in patients with Gram-negative BSI who received clinically adequate empirical antimicrobial therapy.

Introduction

The outcome of patients with Gram-negative bloodstream infection (BSI) depends on multiple host-related and pathogen-related factors. Acute severity of illness scores, such as the Acute Physiology and Chronic Health Evaluation (APACHE) score, are mostly used in critically ill patients who are admitted to intensive-care units (ICUs) [1]. However, the majority of patients with Gram-negative BSI do not require ICU admission [2]. Moreover, many of the variables used in such complex scores are not pertinent to Gram-negative BSI. The Pitt bacteraemia score (PBS), in contrast, has been used to stratify patients with BSI according to acute severity of illness. It is a simple score that is calculated at the time of initial patient evaluation by using temperature (1 point for temperature of 35.1–36°C or 39.0–39.9°C and 2 points for temperature of ≤35°C or ≥40°C), blood pressure (2 points for hypotension), mental status (1 point for disorientation, 2 points for stupor, and 4 points for coma), and the presence or absence of mechanical ventilation (2 points) and cardiac arrest (4 points) [3, 4]. The PBS has been recently described as being superior to other acute severity of illness scores in predicting the outcome of patients with sepsis [5]. However, clinical variables other than acute severity of illness have been associated with mortality following Gram-negative BSI, such as primary source of infection and patients' underlying medical conditions [6-13].

In this retrospective cohort study, clinical predictors of 28-day all-cause mortality following Gram-negative BSI in adult hospitalized patients were identified. The aim of the study was to develop a scoring model with which to estimate the risk of mortality following Gram-negative BSI by using the PBS and other clinical and laboratory variables that were independently associated with mortality.

Methods

Setting

The study was conducted at two Mayo Clinic hospitals: Saint Mary's Hospital and Rochester Methodist Hospital, located in Rochester, Minnesota. Both are large tertiary-care hospitals that combine to provide over 1950 licensed beds and care for local residents as well as referral patients in a wide variety of medical and surgical subspecialties.

Case definition

Gram-negative BSI was defined as the growth of any aerobic Gram-negative bacillus in a blood culture. The primary source of BSI was defined according to the CDC criteria [14]. Immunocompromised hosts were defined as patients with any of the following conditions: neutropenia, recent chemotherapy, treatment with corticosteroids, human immunodeficiency virus infection, recipients of solid organs or bone marrow transplants, or recipients of other immunosuppressive medications. Patients with cancer were defined as those with a current diagnosis of malignant tumour, excluding skin basal and squamous cell carcinoma.

Case ascertainment

A cohort of 708 patients with first episodes of monomicrobial Gram-negative BSI from 1 January 2001 to 31 October 2006 was retrospectively identified from the Mayo Clinic microbiology laboratory database. The detailed case ascertainment methods, as well as inclusion and exclusion criteria for enrolment in this cohort, have been described previously [2]. Briefly, we included adult patients ≥18 years of age with first episodes of monomicrobial BSI caused by aerobic Gram-negative bacilli. Included patients were hospitalized at any medical or surgical floor unit or ICU. All patients included in this cohort received what was predefined clinically as appropriate empirical antimicrobial therapy for Gram-negative BSI within 24 h of initial presentation. This included β-lactam antibiotics with activity against aerobic Gram-negative bacilli, such as β-lactam/β-lactamase inhibitors, third-generation and fourth-generation cephalosporins, a monobactam, and carbapenems with or without fluoroquinolones. Aminoglycoside regimens were excluded to avoid potential interaction with serum creatinine.

Statistical analysis

The primary objective was to determine clinical predictors of 28-day all-cause mortality in patients with Gram-negative BSI. We included in the analysis only patients who were followed for at least 28 days from the onset of Gram-negative BSI (n = 683). Patients who were lost to follow-up within 28 days of BSI were excluded from the analysis (n = 25). Death was confirmed by reviewing medical records and the Minnesota death registry database.

Multivariable logistic regression was used to analyse 28-day mortality. The following variables were considered as candidate predictors of mortality: age, gender, diabetes mellitus, congestive heart failure, chronic pulmonary disease, dementia, end-stage renal disease, liver cirrhosis, malignancy, immunocompromised state, PBS, primary source of BSI, infection site of acquisition, serum creatinine, and peripheral white blood cell (WBC) count. The primary source of BSI was dichotomized into urinary or central venous catheter (CVC)-related vs. other sources of BSI. This was based on the results of previous studies demonstrating that Gram-negative BSI secondary to a urinary tract or CVC infection was associated with better outcomes than BSI resulting from other sources of infection [6-13].

The functional form of each of the continuous variables was assessed. A generalized additive plot of the log odds of 28-day mortality (logit of probability) against a smoothed version of each continuous variable was used to gain a sense of linearity of their relationship. A smoothing spline of four degrees of freedom to characterize the continuous variable in the generalized additive model producing the plot was used. The final functional form of the variable was then formally tested in a logistic model.

To determine a final multivariable model on which the risk score would be based, bootstrap resampling was used. A model was derived in each bootstrap sample by applying the same backward model selection criteria (entry p <0.15 and retention p <0.05) every time, thus accounting for uncertainty in the model selection technique itself. The frequency of selected variables was computed as a percentage across all 400 bootstrap samples. The final multivariable prediction model contained all variables that were individually retained in at least 70% of the bootstrap samples.

The probability of concordance, or c-statistic, was used to quantify the discriminative ability of the final multivariable logistic model. The c-statistic is equivalent to the area under a receiver operating characteristic curve, with a value of 0.5 denoting random predictions and a value of 1.0 denoting perfect predictions. To estimate the optimism bias from the model selection process and then compute a bias-corrected c-statistic, each bootstrap-selected model was examined on the original data, and its performance was measured [15]. Bias was estimated as the average difference in c-statistic values between the bootstrap model and the test model, which was then subtracted from the apparent concordance to obtain a bias-corrected c-statistic. To visually assess calibration, deciles of predicted risk were plotted from the model by the actual fraction of patients who died within 28 days.

From the final multivariable model, regression coefficients were used to derive a risk score for mortality. Points were assigned for the presence of each risk factor in the final multivariable logistic model, and weighted approximately by the corresponding regression coefficients. For each risk factor, the regression coefficient was divided by the minimum absolute value among all coefficients in the final multivariable model, and subsequently multiplied by three. Predicted probabilities obtained directly from the scoring model were plotted by risk score values to visualize the estimated risk of mortality. All analyses were carried out with the SAS statistical software package (version 8.2; SAS Institute, Cary, NC, USA).

Results

The mean age ± standard deviation of patients with Gram-negative BSI was 64 ± 17 years, and 60% (411/683) were males. The overall 28-day mortality rate for Gram-negative BSI in this cohort was 12% (85/683). The baseline clinical characteristics of 28-day survivors and non-survivors are shown in Table 1. Owing to skewed distributions, the natural log transformation of serum creatinine was used, and peripheral WBC count was categorized, for all analyses.

Table 1. Univariate logistic regression model results for risk factors for 28-day mortality in patients with Gram-negative bloodstream infection
VariableDied within 28 days (n = 85)Alive at 28 days (n = 598)OR (95% CI)p-value
  1. BSI, bloodstream infection; CVC, central venous catheter; IQR, interquartile range; SD, standard deviation; WBC, white blood cell.

  2. a

    Log-transformed values used to satisfy the regression assumption of normality.

Age (years), mean ± SD65.7 ± 15.864.2 ± 16.71.01 (0.99–1.02)0.43
Male gender, n (%)53 (62)358 (60)1.11 (0.70–1.77)0.66
Diabetes mellitus, n (%)18 (21)142 (24)0.86 (0.50–1.50)0.60
Congestive heart failure, n (%)17 (20)117 (20)1.03 (0.58–1.82)0.92
Chronic pulmonary disease, n (%)15 (18)108 (18)0.97 (0.54–1.76)0.93
Dementia, n (%)1 (1)13 (2)0.54 (0.07–4.15)0.55
End-stage renal disease, n (%)6 (7)18 (3)2.45 (0.94–6.35)0.07
Liver cirrhosis, n (%)18 (21)30 (5)5.09 (2.69–9.62)<0.001
Malignancy, n (%)59 (69)256 (43)3.03 (1.86–4.94)<0.001
Immunocompromised host, n (%)34 (40)196 (33)1.37 (0.86–2.18)0.19
Pitt bacteraemia score, n (%)   <0.001
012 (14)179 (30)1.0 (reference) 
15 (6)133 (22)0.56 (0.19–1.63)0.29
2–323 (27)159 (27)2.16 (1.04–4.48)0.04
≥444 (52)122 (20)5.38 (2.73–10.60)<0.001
Source of BSI other than urinary tract/CVC, n (%)78 (92)356 (60)7.57 (3.44–16.69)<0.001
Site of infection acquisition, n (%)   0.22
Healthcare-associated31 (36)210 (35)1.0 (reference) 
Community-acquired18 (21)178 (30)0.69 (0.37–1.27)0.23
Nosocomial36 (42)210 (35)1.16 (0.69–1.95)0.57
Serum creatininea (mg/dL), median (IQR)1.6 (1.2–2.4)1.3 (1.0–1.8)1.81 (1.17–2.80)0.007
Peripheral WBC counta (103/μL), median (IQR)10.6 (3.5–20.3)12.0 (5.7–16.7)1.06 (0.90–1.25)0.51
Peripheral WBC count categories (103/μL), n (%)   0.22
0–527 (32)132 (22)1.0 (reference) 
5–1526 (31)266 (44)0.48 (0.27–0.85)0.01
15–209 (11)105 (18)0.42 (0.19–0.93)0.03
>2023 (27)95 (16)1.18 (0.64–2.19)0.59
Time to start of antibiotics (days), mean ± SD0.3 ± 0.40.3 ± 0.40.97 (0.51–1.83)0.93
Inappropriate antibiotic therapy, n (%)2 (2)7 (1)2.06 (0.42–10.07)0.37

From univariate logistic regression modelling, liver cirrhosis, malignancy, a source of BSI other than the urinary tract or CVC infection, PBS, serum creatinine and peripheral WBC count were associated with a higher rate of 28-day mortality (Table 1). These variables were subsequently included in a multivariable regression analysis, and malignancy, liver cirrhosis, non-urinary/CVC source of infection, PBS and serum creatinine were all identified as independent risk factors for 28-day mortality. The following variables were retained in at least 70% of bootstrap samples, and thus deemed to be robust prognostic factors for 28-day mortality: malignancy, liver cirrhosis, non-urinary/CVC source of infection, and PBS. In fact, each of these variables was selected in >95% of bootstrap samples.

A multivariable logistic model that contained only variables that were retained in the final model after bootstrap sampling had an apparent c-statistic of 0.84 (Table 2). The corresponding bias-corrected c-statistic was 0.81. Model calibration looked satisfactory, as the observed outcomes appeared to be fairly close to the predictions (Fig. 1).

Table 2. Final multivariable logistic regression model results for predictors of 28-day mortality in patients with Gram-negative bloodstream infection and risk-score point allocation for each variable
VariableOR (95% CI)p-valueScoring points
  1. BSI, bloodstream infection; CVC, central venous catheter.

  2. Final multivariable model c-statistic = 0.84 (bias-corrected c-statistic = 0.81).

  3. Risk-score model c-statistic = 0.84.

Malignancy3.48 (1.94–6.22)<0.0013
Liver cirrhosis5.42 (2.52–11.65)<0.0014
Non-urinary/CVC source of BSI5.54 (2.42–12.69)<0.0014
Pitt bacteraemia score
01.0 (reference) 0
10.64 (0.21–1.90)0.420
2–31.98 (0.92–4.25)0.082
≥46.42 (3.11–13.24)<0.0015
Figure 1.

Calibration plot of final multivariable logistic model for predictors of 28-day mortality in Gram-negative bloodstream infection. Note: The observed frequency of 28-day mortality plotted by deciles of predicted probability from the original (grey filled circles, error bars, and dashed-line loess curve) and bootstrap/bias-corrected (black X markers, error bars and solid-line loess curve) modelling. Perfect calibration is represented by the Y = X line.

To derive a risk score for 28-day mortality, points were assigned for each variable in the final multivariable model, weighted approximately by the corresponding regression coefficients (Table 2). A subject's risk score is the cumulative number of points from their risk profile, and ranges in value from 0 to 16. Fig. 2 compares the receiver operating characteristic curves for the multivariable logistic regression and simplified risk-score models, and shows that the simplified risk score (c-statistic of 0.84) adequately summarizes the multivariable logistic regression model.

Figure 2.

Receiver operating characteristic plot of the final multivariable logistic model and risk-score model of 28-day mortality for Gram-negative bloodstream infection. Note: c-Statistic for both final multivariable logistic and risk-score models = 0.84

The risk score provides clinicians with a simple tool with which to estimate the risk of 28-day mortality in patients with Gram-negative BSI (Fig. 3). A higher score corresponds to an increased risk of mortality. For example, patients with risk scores of ≤4 have a relatively low estimated mortality rate of ≤3%. The estimated risk of mortality increases sharply for risk scores of >8 with wider 95% CIs, which is reflective of fewer numbers of subjects with increased risk.

Figure 3.

Predicted probability of 28-day mortality in patients with gram-negative bloodstream infection by risk score. Note: Vertical lines represent 95% CIs. The size of marker for point estimates is weighted approximately by the number of subjects with a corresponding risk score.

Discussion

The proposed Gram-negative BSI risk score estimates the risk of mortality in patients with Gram-negative BSI who receive timely empirical antimicrobial therapy that is considered to be clinically adequate for the treatment of Gram-negative BSI. The score is based on variables that are independently associated with mortality and include PBS, source of infection, and the presence or absence of malignancy and liver cirrhosis.

The PBS was a highly significant predictor of mortality in patients with Gram-negative BSI. This observation is consistent with the results of a recent investigation [5]. In this study, we used a more sensitive categorization of the PBS (0–1 vs. 2–3 vs. ≥4) rather than dichotomization of <4 and ≥4 as in previous studies [4, 16]. However, the PBS lacks some parameters that are related to outcomes in patients with Gram-negative BSI, such as primary source of infection. The current study demonstrated that a source of BSI other than urinary tract or CVC was independently associated with higher risk of mortality. Establishing a source of infection at the time of initial presentation may be challenging in some patients, especially those with a gastrointestinal or respiratory source of infection that may require imaging studies or further clinical samples to be submitted for culture. However, a urinary tract source of BSI may be established at the time of initial presentation on the basis of urinary symptoms, urinalysis, and urine Gram-stain results. Similarly, a CVC-related BSI can be supported by a differential ‘time to positivity’ of >2 h between simultaneous blood cultures obtained through a CVC and peripheral vein in the absence of other apparent foci of infection [17]. As the model dichotomizes the source of infection into urinary/CVC vs. others, the risk score may be applied prior to establishing the source of infection in some patients. For example, a patient without a CVC who has no urinary symptoms and unremarkable urinalysis findings probably has a source of infection other than the urinary tract or CVC infection. In other cases where the source of infection is not apparent at the time of initial presentation, a range of mortality risk may be estimated from the risk-score model with and without the allocated points for the source of infection.

The association between cancer and mortality in patients with Gram-negative BSI is congruous with findings from other studies [18, 19]. This is probably related to the immunocompromised status that typically occurs as a result of malignancy or its treatment and the effects of cancer on survival. In addition, the association between liver cirrhosis and mortality in patients with BSI has been recently described in three large studies [20-22]. It is conceivable that patients with liver cirrhosis have a poor prognosis with Gram-negative BSI. Liver cirrhosis is associated with a relatively low neutrophil count, neutrophil dysfunction, and complement deficiency resulting from both decreased production and increased consumption [23].

Delayed and inappropriate antimicrobial therapy are both associated with mortality in patients with BSI [7, 24]. As antimicrobial susceptibility results are not available at the time of initial presentation, our model was designed to eliminate upfront the majority of patients who received delayed or inappropriate antimicrobial therapy. All patients included in this cohort received antimicrobial regimens that were considered to be acceptable for the empirical treatment of Gram-negative BSI within 24 h of presentation [25]. As a result, the timing and choice of antimicrobial regimen did not appear to influence the model results, as most patients received early and appropriate antimicrobial therapy (Table 1).

As the derivation cohort for the Gram-negative BSI risk score was a referral rather than a population-based cohort, patients >80 years of age were under-represented, as previously demonstrated [26]. This is probably the explanation for the lack of an age effect on mortality in this study. Additionally, the prevalence rates of liver cirrhosis and cancer are probably higher in referral than in population-based cohorts. Therefore, the model should be externally validated in a population-based cohort with a fair representation of elderly patients and a lower prevalence of liver cirrhosis and malignancy.

The overall 28-day all-cause mortality rate of 12% in patients with Gram-negative BSI in this study was similar to the reported mortality rates in recently published population-based studies [11, 27]. However, the mortality rate in this study was markedly lower than that previously reported from studies of Gram-negative BSI performed during the prior two decades [10, 28]. Identifying independent risk factors for mortality in patients with Gram-negative BSI in the current era of advanced critical care and antimicrobial management provides a distinct advantage, and ensures that the model results are applicable to current clinical practice.

The study has some limitations. First, this was a retrospective cohort study, so clinical variables, including PBS, were collected retrospectively. Second, our cohort included patients hospitalized at tertiary-care facilities, rather than being a population-based cohort; therefore, the results may be affected by referral bias. Despite the possible selection bias resulting from excluding 25 subjects with insufficient follow-up, a post hoc analysis comparing excluded subjects with those included in the analysis showed no significant differences in demographic or clinical characteristics (results not shown). Finally, the study enrolled only adult patients; therefore, the results should not be extrapolated to children.

In summary, the risk of mortality in patients with Gram-negative BSI can be estimated by using a risk score derived from acute severity of illness as estimated by the PBS, clinical manifestations (primary source of BSI), and underlying medical conditions such as cancer and liver cirrhosis. The score is applicable to patients with Gram-negative BSI who are treated properly with clinically acceptable empirical antimicrobial regimens. This Gram-negative BSI risk score provides treating physicians with the tools to answer one of the most difficult questions asked by patients and their families at the time of initial presentation with Gram-negative BSI: what is the estimated risk of mortality? The Gram-negative BSI risk score may also have utility in future clinical research investigations.

Acknowledgements

The authors thank E. Vetter for providing us with vital data from the microbiology laboratory database at the Mayo Clinic in Rochester, MN, USA. The preliminary results of this study were presented, in part, at the 50th Interscience Conference on Antimicrobial Agents and Chemotherapy annual meeting on 13 September 2010 in Boston, MA, USA (Abstract L1-1016).

Transparency Declaration

M. N. Al-Hasan and B. D. Lahr have full access to all of the data in the study, and take responsibility for the integrity of the data and the accuracy of the data analysis. The study received funding from the Baddour Family funds and the Small Grants program at the Mayo Clinic, Rochester, MN, USA. All of the authors declare that there are no conflicts of interest.

Ancillary