See also pp. 132–139
‘Prediction is difficult, especially about the future.’
– Neils Bohr
In emergency medicine, we play a high-stakes game with a dealer who likes to shuffle the deadly disorders imperceptibly among the common benign disorders. We are the poker players of modern medicine; hold or fold, risks and probabilities are our game. We agonise over which of our patients might die, who can go home and who should stay in hospital.
Yet merely admitting patients to hospital does not keep them out of harm's way. Patients still need to be treated by the right people, in the right place, at the right time. The sickest of the sick need emergency physicians when they arrive and then intensivists during their admission. Despite our best efforts, medical emergency team (MET) calls, intensive care unit (ICU) admissions and deaths still occur soon after admission to the ED. Perhaps if high-risk patients could be reliably identified, then ICU review could occur earlier, end-of-life issues might be addressed urgently and timelier interventions instituted. We might even be able to avert these imminent adverse events.
Loekito and colleagues have published a retrospective observational study in this issue of Emergency Medicine Australasia that takes some initial steps towards facing this challenge. Their study is the first of its kind. There are studies looking at selected groups of patients, or ‘track and trigger’ early warning systems largely based on physiological parameters or ICU-derived scoring systems,[2-4] but none that focus solely on laboratory tests in undifferentiated ED patients.
Let us delve into the mechanics of this study. It assesses the ability of common laboratory measurements from undifferentiated ED patients to predict who imminently required a MET call, needed ICU admission or died. The common laboratory measurements used were those derived from the full blood count, urea, electrolytes and creatinine, liver function tests and blood gases. The authors scoured the database records of more than 70 000 patients at the Austin Hospital in Melbourne, Australia, who were admitted between 2000 and 2006. They determined which of these patients had MET calls (within 24 h), ICU admission or who died (on the same or next calendar day). They obtained over 2.5 million individual laboratory measurements taken from these patients. These were analysed in subgroups (called ‘batches’) of the 30 laboratory variables measured. Univariate logistic regression identified the nine laboratory variables that had the greatest individual predictive power of death (haemoglobin, haematocrit, bicarbonate, pH, bilirubin, albumin, urea, creatine and white cell count), which, for pragmatic reasons, were the ones chosen to be analysed further as predictors of MET calls and ICU admissions.
In combination, through multivariate logistic regression, these nine laboratory variables were found to be an excellent predictor of death (area under the receiver operating characteristic curve [AUC-ROC] = 0.90), a good predictor of ICU admission (AUC-ROC = 0.82) and a borderline to fair predictor for MET calls (AUC-ROC = 0.69). These findings were externally validated using data from 21 430 patients at The Alfred, another major teaching hospital in Melbourne. Almost identical results were obtained.
We can view this conglomerate of nine common laboratory measurements as a single diagnostic test for a truly imminent major adverse event. Like any diagnostic test, the threshold chosen for the test result to be deemed positive determines how it can be best used. Loekito and colleagues looked at numerous diagnostic cutpoint criteria to find the thresholds that produced the highest overall correct diagnosis rate. This resulted in impressive negative predictive values, such that if the combined laboratory test is negative, then the patient can be safely managed on the ward. In this study, 99.8% of such patients would not have had an imminent MET call or death, and none would have been admitted to the ICU.
However, there are some important caveats. As impressive as the negative predictive values are, even before any tests 99.8% of ED patients would not be admitted to the ICU anyway (there were 160 ICU admissions out of 71 453 patients), 99.5% would not have had MET calls and 98.8% would not die. Presumably, 1% of patients either died before getting to ICU, or was excluded for other reasons. If most of the excess deaths were patients deemed ‘not for ICU’, then there is little room for improvement. However, we do not know if this was the case.
Now that we have considered the implications of a negative combined laboratory test, what if the test was positive? Given that the pre-test probability of the outcomes of interest is low, and the positive likelihood ratios are at best moderate (positive likelihood ratio [+LR] 4.9 for death, +LR 4.44 for ICU admission and +LR 2.22 for MET calls), the positive predictive values are also low (4.0% for death, 0.8% for ICU admission and 0.7% for MET calls). This means that a positive combined laboratory test is hobbled by high rates of false positives and has limited predictive value.
This leaves us with an imperfect prediction tool. How might it be improved? Notably, this combined laboratory test does not include lactate. When the data were collected for this study, lactate was not available as part of the blood gas analysis results. Today, lactate is routinely available on many ED blood gas analysers in Australasia. Lactate is a useful predictor of mortality in sepsis[5, 6] as well as acutely ill patients in general, and might add to the predictive power of the combined laboratory test approach. Whether the predictive power of lactate is independent of bicarbonate and pH, laboratory parameters used in this study are not known.
Another way of improving the combined laboratory test might be to explore how it can be used together with physiological parameters. Vital signs predict mortality in ED patients, but unlike the combined laboratory test developed by Loekito and colleagues, vital signs have lower sensitivity and higher specificity. Vital sign derangement probably occurs relatively late in most critical illnesses, which is one purported reason why the MERIT study failed to find a benefit from MET teams on the wards. Finding a way to combine clinical parameters with laboratory markers of critical illness, which are more sensitive and occur earlier, seems to make sense. Previous attempts at this, with less commonly measured laboratory tests, showed some promise. Also, vital signs can be recorded electronically, which makes them more accessible, reliable and useful.
It is interesting that the combined laboratory test was better at predicting death than ICU admission and especially MET calls. Why was this? It makes intuitive sense that dying and critically ill patients will have some derangement of laboratory values. Yet anyone who has been part of a MET team knows that many MET calls do not arise from critically ill patients; the criteria are non-specific. In addition, it is fanciful to expect that laboratory tests could predict which patients will be made ‘not for MET calls’ or ‘not for ICU’ where this is deemed appropriate by the treating physicians.
Finally, we need to consider how this combined laboratory test approach could be used in practice. The laboratory measurements were obtained an average of 12 h before a MET call was activated, providing an ample window of opportunity to take action. The combined test might serve as a ‘d-dimer for death and disaster’ (though we all know the reputation of that much-maligned test). Negative tests confirm that the patient is free from immediate danger, but positive tests create a puzzle requiring further action. The test could trigger an automated alert that activates the treating doctor to consider interventions or appropriate referral. Unfortunately, even if you have a good test, meaningful improvement in outcomes is far from guaranteed, as has been found for other early warning systems. There is little point predicting the future unless we can change it. Future studies, in addition to improving the predictive value of the combined laboratory test, will need to look at how doctors then respond to these alerts to determine if patients actually benefit.
In the meantime we will continue to think carefully about the disposition of our admitted patients. The dealer has not put the pack of cards away just yet and we cannot rely solely on the Nostradamus in the blood tubes to tell our patients’ fortunes.