SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. AbstractResumen
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. References
  9. Supporting Information

Objectives

There are no validated guidelines to guide physicians with difficult disposition decisions for emergency department (ED) patients with heart failure (HF). The authors sought to develop a risk scoring system to identify HF patients at high risk for serious adverse events (SAEs).

Methods

This was a prospective cohort study at six large Canadian EDS that enrolled adult patients who presented with acute decompensated HF. Each patient was assessed for standardized clinical and laboratory variables as well as for SAEs defined as death, intubation, admission to a monitored unit, or relapse requiring admission. Adjusted odds ratios for predictors of SAEs were calculated by stepwise logistic regression.

Results

In 559 visits, 38.1% resulted in patient admission. Of 65 (11.6%) SAE cases, 31 (47.7%) occurred in patients not initially admitted. The multivariate model and resultant Ottawa Heart Failure Risk Scale consists of 10 elements, and the risk of SAEs varied from 2.8% to 89.0%, with good calibration between observed and expected probabilities. Internal validation showed the risk scores to be very accurate across 1,000 replications using the bootstrap method. A threshold of 1, 2, or 3 total scores for admission would be associated with sensitivities of 95.2, 80.6, or 64.5%, respectively, all better than current practice.

Conclusions

Many HF patients are discharged home from the ED and then suffer SAEs or death. The authors have developed an accurate risk scoring system that could ultimately be used to stratify the risk of poor outcomes and to enable rational and safe disposition decisions.

Resumen

Una Escala De Puntuación De Riesgo Para Identificar En El Servicio De Urgencias A Los Pacientes Con Insuficiencia Cardiaca Con Alto Riesgo De Presentar Eventos Adversos Graves

Objetivos

No existen guías clínicas validadas para orientar a los médicos en la dificíl decisión de la ubicación de los pacientes con insuficiencia cardiaca (IC) en el servicio de urgencias (SU). El objetivo fue desarrollar una escala de puntuación de riesgo para identificar a los pacientes con IC con alto riesgo de eventos adversos graves (EAG).

Método

Estudio de cohorte prospectivo que incluyó pacientes adultos que presentaron IC aguda descompensada en seis SU canadienses. Se recogieron variables clínicas y de laboratorio estandarizadas en cada paciente así como de EAG, definidos como muerte, intubación, ingreso en una unidad monitorizada o recaída con necesidad de ingreso. Se calcularon las OR ajustadas para predecir EAG mediante una regresión logística de pasos hacia atrás.

Resultados

De 559 consultas, el 38,1% finalizaron con el ingreso del paciente. De los 65 (11,6%) casos de EAG, 31 (47,7%) ocurrieron en los pacientes no ingresados inicialmente. El modelo multivariable y la escala Ottawa Heart Failure Risk Scale resultante constan de 10 elementos, y el riesgo de EAG varió de 2,8 a 89%, con buena correlación entre lo observado y las probabilidades esperadas. La validación interna mostró que las puntuaciones de riesgo eran muy certeras en las 1.000 réplicas usando el método bootstrap. Un umbral de 1, 2 ó 3 de puntuación total para ingreso estaría asociado con sensibilidades para detectar EAG del 95,2%, 80,6% y 64,5%, respectivamente, cualquiera de ellas mejor que la práctica actual.

Conclusiones

Muchos de los pacientes con IC son dados de alta a casa desde el SU y después sufren EAG o muerte. Se ha desarrollado un sistema de puntuación de riesgo certero que podría ser usado para estratificar el riesgo de malos resultados y permitir decisiones de ubicación racionales y seguras.

Acute decompensated heart failure (HF) is a common and serious condition that is the leading cause for hospital admissions for seniors in the United States and Canada.[1-5] An estimated one million HF patients seek emergency department (ED) care annually in the United States.[6] HF often coexists with other chronic diseases (such as diabetes mellitus, chronic obstructive pulmonary disease [COPD], and coronary artery disease), which further contribute to hospitalization and prolonged lengths of stay. Patients who are admitted to hospital for HF have high readmission rates in the months following initial discharge.[7] Moreover, patients discharged from the ED following treatment for acute exacerbation of HF have high rates of relapse and mortality.[8, 9] High-quality evidence to assist clinicians in ED decision-making is lacking, and only recently have clinical guidelines focused on the management of acute HF.[10, 11]

An important challenge facing emergency physicians when treating HF patients is deciding on disposition, whether to admit with or without monitoring, or whether to discharge, with or without early follow-up. It is not desirable to admit all patients, as many will respond to therapy in the ED and will not benefit from hospitalization. A small but important number of these patients have serious adverse events (SAEs), i.e., they die, require intensive care therapy, or suffer morbidity such as myocardial infarction (MI). Many other patients are discharged after prolonged ED management only to suffer a SAE or return to receive additional ED treatment and/or be admitted. The issue of disposition decisions is an important question because in many jurisdictions there is a shortage of hospital beds and many EDs are overcrowded. There are, however, no widely accepted or validated guidelines to aid with these difficult decisions.

The overall goal of this study was to develop a risk scoring system to guide the disposition decisions of physicians for ED patients with acute dyspnea secondary to HF. In particular, this risk scale is intended to be highly sensitive for predicting the potential for development of SAEs among such patients. Ultimately, this scale should improve and standardize disposition practices for these patients, diminishing both unnecessary admissions and unsafe discharge decisions.

Methods

  1. Top of page
  2. AbstractResumen
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. References
  9. Supporting Information

Study Design

We conducted a prospective observational cohort study. The study protocol was approved by the research ethics boards at each center; the boards at three hospitals determined that written informed consent was required, whereas those at the other three sites waived the need for written consent for this observational study.

Study Setting and Population

The study was conducted in six Canadian teaching hospital sites in Ottawa, Toronto, Kingston, Montreal, and Edmonton, with a combined annual ED volume of approximately 350,000 patient visits. We included a convenience sample of adults ≥ 50 years of age who presented with acute shortness of breath secondary to exacerbations of chronic HF or new-onset HF. We used pragmatic criteria for the diagnosis of HF as recommended by the working group on HF of the European Society of Cardiology.[12-14] Patients must have had appropriate symptoms (shortness of breath or fatigue) with clinical signs of fluid retention (pulmonary or peripheral) in the presence of an underlying abnormality of cardiac structure or function. If doubt remained, a beneficial response to treatment (for example, a brisk diuresis accompanied by substantial improvement in breathlessness) was also considered. While B-type natriuretic peptide (BNP) levels are not always used in these Canadian EDs for patients with dyspnea, when available, we reviewed BNP values to confirm values compatible with a diagnosis of HF.

We excluded patients who were obviously too ill to be considered for discharge within the 2- to 15-hour ED treatment study window or who were otherwise unsuitable for the study because of: 1) resting oxygen saturation < 85% on room air or after being on their usual home oxygen setting for 20 minutes on ED arrival; 2) heart rate greater than or equal to 120 beats/min on arrival; 3) systolic blood pressure < 85 mm Hg on arrival; 4) confusion, disorientation, or dementia; 5) ischemic chest pain requiring treatment with nitrates on arrival; 6) acute ST-segment elevation on electrocardiogram (ECG) on arrival; 7) terminal status—death expected within weeks from chronic illness; 8) from nursing home or chronic care facility; 9) enrolled into the study in previous 2 months; or 10) on chronic hemodialysis.

Study Protocol

Patient assessments were made by registered respiratory therapists or registered nurses who were on duty at variable times depending on the site. The target assessment period was 4 to 8 hours after initial ED treatment, but patients could be considered for enrollment at 2 to 15 hours after treatment. The research assistants were trained by means of lectures and practical demonstrations to assess all variables in a uniform manner. A standardized description of each assessment was provided and the research assistants recorded their findings on data collection sheets. There was ongoing evaluation of the quality of the patient assessments by a central study nurse coordinator who provided regular feedback to the sites.

The variables assessed in the study were chosen by the investigators based on their clinical experience and reports in the literature.[15-26] As shown in Tables 1-3, we collected data from history, general examination, laboratory tests, and the 3-minute walk test. For the 3-minute walk test, patients were asked to walk at their own pace in the ED for a fixed period of 3 minutes, regardless of the distance covered.[27] Patients could use their normal walking aids (e.g., cane or walker), but could not be physically supported by another person. Patients used no supplementary oxygen or their normal home oxygen flow level. The same model of recording pulse oximeter (Criticare 504DXP, Criticare Systems, Inc., Waukesha, WI) was used at all sites to record and continuously measure heart rate and oxygen saturation levels. Not all patients were well enough to undertake the walk test and some were unable to complete the entire 3 minutes. BNP samples were drawn from enrolled patients and then processed in batch by The Ottawa Hospital (NT-ProBNP with Roche Elecsys 2010 system, Hoffmann-La Roche, Ltd., Basel, Switzerland). At two sites, we used the N-terminal prohormone-BNP (NT-ProBNP) values provided by the hospital laboratories for clinical use, also analyzed by the Roche system.

Table 1. Patient Characteristics for 559 HF Patient Visits
CharacteristicN = 559
  1. a

    Canadian Triage Acuity Scale (CTAS) ranges from 1 (most urgent) to 5 (least urgent).

  2. ACE = angiotensin-converting enzyme; CABG = coronary artery bypass graft; COPD = chronic obstructive pulmonary disease; HF = heart failure; MI = myocardial infarction; PCI = percutaneous coronary intervention; TIA = transient ischemic attack.

Age (yr), mean (±SD)76.0 (±10.6)
Range50–101
Male, n (%)315 (56.4)
Hospital site, n (%)
Kingston General110 (19.7)
Ottawa Hospital Civic Campus167 (29.9)
Ottawa Hospital General Campus 97 (17.4)
University Alberta–Edmonton 80 (14.3)
Mount Sinai–Toronto 83 (14.9)
Jewish General–Montreal 22 (3.9)
Arrival status
Arrival by ambulance, n (%)204 (36.5)
Temperature (°C), mean (±SD)36.3 (±0.7)
Heart rate (beats/min), mean (±SD)82.4 (±20.2)
Respiratory rate (breaths/min), mean (±SD)21.8 (±5.3)
Systolic blood pressure (mm Hg), mean (±SD)141.8 (±27.9)
SaO2 by oximetry (%), mean (±SD)94.8 (±4.3)
Duration of respiratory distress (hours), mean (±SD)109.1 (±172.6)
Canadian Triage Acuity Scale, mean (±SD)a2.6 (±0.5)
Secondary diagnosis, n (%)56 (10.0)
COPD 12 (21.4)
Past medical history, n (%) 548 (98.0)
HF 371 (66.4)
COPD82 (14.7)
Admission for respiratory distress42 (7.5)
Intubation for respiratory distress8 (1.4)
MI/angina 287 (51.3)
CABG/PCI198 (35.4)
Pacemaker77 (13.8)
Atrial fibrillation208 (37.2)
Peripheral vascular disease (intervention)29 (5.2)
Cancer55 (9.8)
Hypertension352 (63.0)
Stroke or TIA73 (13.1)
Diabetes217 (38.8)
Chronic liver disease6 (1.1)
Dementia7 (1.3)
Chronic renal failure 62 (11.1)
Smoker, n (%)
Current35 (6.3)
Former231 (41.3)
Home oxygen, n (%)25 (4.5)
Current cardiac medications, n (%)541 (96.8)
ACE inhibitors256 (45.8)
Antiarrhythmics55 (9.8)
Anticoagulants237 (42.4)
Antiplatelet medications250 (44.7)
Beta blockers361 (64.6)
Calcium channel blockers152 (27.2)
Digoxin77 (13.8)
Diuretics413 (73.9)
Nitrates144 (25.8)
Statins271 (48.5)
Vasodilators32 (5.7)
Current respiratory medications, n (%)165 (29.5)
Antibiotics 47 (8.4)
Inhaled anticholinergics55 (9.8)
Inhaled beta-agonist88 (15.7)
Inhaled steroid71 (12.7)
Oral steroid26 (4.7)
Table 2. Patient Outcomes for 559 HF Patient Visits
Outcomen (%)
  1. HF = heart failure; ICU = intensive care unit; MI = myocardial infarction.

Admitted to hospital 213 (38.1)
Critical care or other monitored unit (n = 213)32 (15.0)
Noninvasive ventilation required after admission (n = 213)9 (4.2)
Intubation required after admission (n = 213)7 (3.3)
MI after admission (n = 213)10 (4.7)
Death after admission (n = 213)9 (4.2)
Discharged from ED346 (61.9)
Relapse back to ED and reasons (n = 346)52 (9.3)
Dyspnea 38 (73.1)
Fever 1 (1.9)
Chest pain 9 (17.3)
Other 8 (15.4)
Relapse and admitted to hospital (n = 346)27 (7.8)
Admitted to ICU5 (18.5)
Death within 30 days (n = 346)4 (1.2)
Serious adverse events 65 (11.6)
Admitted patients (n = 213)34 (16.0)
Discharged patients (n = 346)31 (9.0)
Table 3. Univariate Association with SAEs for Variables from History for 559 HF Patient Visits
CharacteristicSAE (N = 65)No SAE (N = 494)p-value
  1. ACE = angiotensin-converting enzyme; CABG = coronary artery bypass graft; COPD = chronic obstructive pulmonary disease; EMS = emergency medical services; HF = heart failure; MI = myocardial infarction; PCI = percutaneous coronary intervention; SAE = serious adverse event; TIA = transient ischemic attack.

Age (yr), mean ± SD76.8 (±10.0)75.9 (± 10.7)0.50
Male (%)56.956.30.92
Arrival by EMS (%)44.635.40.15
Past medical history (%) 98.598.60.94
COPD7.715.60.09
Admission for respiratory distress7.77.50.95
Intubation for respiratory distress4.61.00.02
Angina16.914.60.62
MI38.536.20.73
CABG32.324.70.19
PCI10.89.70.79
Pacemaker10.814.20.46
Atrial fibrillation41.536.60.44
Peripheral vascular disease 4.65.30.83
Cancer10.89.70.79
Hypertension63.163.00.98
Stroke or TIA21.511.90.03
Diabetes40.038.70.84
Dementia3.11.00.16
Chronic renal failure 12.310.90.74
Duration of respiratory distress (hours), mean (±SD) 104.1 (±118.9)109.6 (±177.2)0.84
Home oxygen (%)3.14.70.56
Smoker (pack-years), mean (±SD) (n = 20 and 187)36.4 (±25.8)35.3 (±25.2)0.86
Current cardiac medications (%)96.997.40.84
ACE inhibitors47.745.80.78
Antiarrhythmics10.89.80.80
Anticoagulants41.542.80.85
Antiplatelet medications46.244.80.84
Beta blockers50.866.40.01
Calcium channel blockers35.426.30.12
Digoxin13.913.91.00
Diuretics73.973.90.99
Nitrates33.924.90.12
Statins50.848.50.73
Vasodilators6.25.70.88
Current respiratory medications (%)24.630.40.34
Antibiotics 4.69.00.24
Inhaled anticholinergics4.610.60.13
Inhaled beta-agonist12.316.30.40
Inhaled steroid7.713.50.19
Oral steroid7.74.30.22
Caretaker at home (%) (n = 304)50.854.90.31
Able to drink fluids in ED (%) (n = 419)70.875.70.50

Outcome Measures

The primary outcome was SAE: death from any cause within 30 days of the ED visit, or any of the following within 14 days of the index ED visit, regardless of whether initially admitted: 1) Admission to a critical care or acute monitoring unit where the patient is too ill to ambulate; this excludes ambulatory telemetry units. 2) Endotracheal intubation or need for noninvasive ventilation after hospital admission, unless on noninvasive ventilation at home. 3) MI, as defined by international consensus standards.[28] Either one of the following criteria satisfies the diagnosis for an acute, evolving, or recent MI: i) Typical rise and gradual fall of troponin with at least one of the following: a) ischemic symptoms, b) development of pathologic Q waves on the ECG, c) ECG changes indicative of ischemia, or d) coronary artery intervention (e.g., coronary angioplasty). ii) Pathologic findings of an acute MI. 4) Major procedure defined as coronary artery bypass graft, percutaneous coronary intervention, other cardiac surgery, or new hemodialysis. 5) Relapse and hospital admission for patients who were discharged on the initial ED visit, defined as a return to the ED for any related medical problem within 14 days followed by admission to hospital; relapse to the ED without associated admission was not considered a SAE. Of note, Canadian EDs generally make disposition decisions within 12 hours of arrival and do not use observation units.

Assessment of the primary outcome measure was made by the investigators, blinded to the patient status for the predictor variables, from these source documents: 1) ED health records, 2) hospital health records, 3) computerized hospital patient tracking and record system, and 4) review of provincial death records. Patients were not contacted by telephone.

Data Analysis

The association between the primary outcome, SAE, and variables from the history, physical examination, and investigations was assessed by the appropriate univariate analyses. Continuous variables were also categorized using the most discriminative cut points. We imputed the values for NT-proBNP for 45% of cases using a multiple imputation procedure and assuming that missingness was random.[29-31] NT-proBNP was frequently missing because it was only gathered for research purposes at four of six sites, not as part of routine laboratory investigations, and required additional written patient consent. Logistic regression with stepwise selection was conducted for those variables found to be associated with SAE on univariate analysis (p < 0.05), as well as for clinically sensible interaction terms. Analyses were conducted by visit rather than individual patient. Using accepted approaches, a scale was created by rounding up the lowest likelihood ratio (LR) beta coefficient to one and then multiplying the other coefficients by the same factor and then rounding to the nearest whole number.[32] The classification performance of the final score categories was internally validated for sensitivity, specificity, positive LR, and negative LR across 1,000 replications using the bootstrap method.[33] We estimated that we would need between 60 and 100 cases positive for the primary outcome to conduct stable and valid multivariate analyses.

Results

  1. Top of page
  2. AbstractResumen
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. References
  9. Supporting Information

Study Patients

Table 1 shows the demographic and clinical characteristics of the 559 eligible patient visits from September 2007 to April 2010. Patients at the six participating sites had a mean (±SD) age of 76.0 (±10.6) years and an overall admission rate of 38.1%. Another 835 eligible patient visits at the study sites were not enrolled, primarily because patients presented when research staff were not available. The characteristics of these patients are very similar to those of the patients enrolled (Data Supplement S1, available as supporting information in the online version of this paper). Overall there were 65 (11.6%) SAEs and, of concern, 31 (47.7%) of these occurred in the 346 patients not admitted on the initial ED visit (Table 2). Also, 4 of the 13 study patient deaths (30.8%) occurred within 30 days among patients initially discharged home from the ED.

Univariate and Multivariate Data Analyses

Tables 3 and 4 show the association between the primary outcome, SAE, and variables from the history, physical examination, and investigations. Some continuous variables were further categorized using the most discriminative cut points (Table 5). Overall, 22 of these univariate associations were statistically significant.

Table 4. Univariate Correlation with SAEs for Variables from Physical Examination and Investigations for 559 HF Patient Visits
Characteristic SAE (N = 65)No SAE (N = 494)p-value
  1. A-V = atrioventricular; CTAS = Canadian Triage Acuity Scale; CXR = chest x-ray; ECG = electrocardiogram; HF = heart failure; NT-proBNP = N-terminal prohormone of brain natriuretic peptide; pCO2 = partial pressure of CO2; SAE = serious adverse event; SaO2 = oxygen saturation.

  2. a

    Numbers in parentheses indicate the actual n for which values were not missing.

Initial vital signs on arrival, means (±SD)
Temperature on arrival (°C) (n = 58 and 467)a36.1 (0.9)36.3 (0.7)0.21
Heart rate on arrival (beats/min)94.1 (23.6)80.8 (19.2)<0.0001
Respiratory rate on arrival (breaths/min) (n = 62 and 457)22.4 (5.6)21.8 (5.3)0.37
Systolic BP on arrival (mm Hg) (n = 65 and 488)136.7 (26.6)142.5 (28.0)0.12
Oxygen saturation on arrival (%) (n = 64 and 491)93.6 (6.2)94.9 (4.0)0.09
CTAS level on arrival (n = 63 and 486)2.4 (0.6)2.6 (0.5)0.02
Laboratory values, mean (±SD)
Urea (mmol/L)11.1 (5.3)10.0 (7.3)0.12
Creatinine (mmol/L)125.5 (43.4)118.8 (55.9)0.27
Serum CO2 (mmol/L)25.0 (4.4)26.2 (3.5)0.05
Glucose (mmol/L) (n = 65 and 482)8.8 (4.2)7.7 (3.2)0.04
pCO2 (mm Hg) (n = 11 and 46)42.7 (17.2)41.5 (11.2)0.78
pO2 (mm Hg) (n = 11 and 46)72.8 (12.8)78.2 (33.5)0.39
pH (n = 11 and 46)7.4 (0.1)7.4 (0.1)0.10
NT-proBNP level (ng/L) (n = 29 and 276)11,245.3 (10,852.0)6,782.7 (8,311.6)0.04
Hemoglobin (g/L)124.4 (21.0)123.7 (19.8)0.81
ECG findings (%)
Atrial fibrillation/flutter39.133.30.36
Acute ischemia17.23.7<0.0001
A-V conduction disturbance14.115.30.79
Intraventricular conduction disturbance32.830.60.72
Old infarction17.211.00.15
ECG QRS duration (mm), mean (±SD) (n = 64 and 473)111.2 (27.7)116.7 (34.9)0.22
CXR findings (%)100.099.00.41
Pulmonary congestion49.247.90.83
Pleural effusion63.145.0<0.01
Pneumonia10.85.90.14
Cardiomegaly53.949.70.53
Too ill to do walk test (%) 32.310.7<0.0001
3-minute walk test findings, mean (±SD) (n = 485)
Walk test
Baseline heart rate (n = 44 and 441)82.2 (16.5)77.4 (14.9)0.04
Baseline SaO2 (n = 44 and 441)95.0 (2.7)94.6 (3.0)0.44
Baseline Borg score (n = 42 and 426)2.0 (2.1)1.8 (1.7)0.46
Highest heart rate (n = 44 and 436)100.0 (19.0)93.2 (18.4)0.02
Lowest SaO2 (n = 44 and 437)90.6 (4.4)90.3 (4.5)0.61
Borg score at 3 minutes (n = 42 and 417)2.9 (2.1)3.3 (3.7)0.39
1 minute postwalk heart rate (n = 43 and 432)90.4 (18.5)82.3 (16.1)<0.01
1 minute postwalk SaO2 (n = 44 and 432)93.9 (5.1)94.0 (3.7)0.93
Change in HR from arrival (n = 44 and 441)6.4 (14.3)3.1 (14.6)0.14
Change in SaO2 from arrival (n = 44 and 438)-0.3 (4.9)0.7 (3.5)0.19
Walk test completed,% (n = 44 and 441)84.182.10.74
Table 5. Univariate Correlation with SAEs for Continuous Variables with Discriminative Cut Points
Characteristic (%)SAE (N = 65)No SAE (N = 494)p-value
  1. CTAS = Canadian Triage Acuity Scale; HR = heart rate; MI = myocardial infarction; NT-proBNP = N-terminal prohormone of brain natriuretic peptide; SAE = serious adverse event; SaO= oxygen saturation.

  2. a

    Numbers in parentheses indicate the actual n for which values were not missing.

  3. b

    Troponin I (TNI) or T (TNT) and MI levels by hospital: Kingston, Edmonton (TNI, >0.50 μg/L), Ottawa, Mount Sinai, Jewish General (TNT, >0.10 μg/L), and Ottawa after June 21, 2009 (TNI, >0.45 μg/L).

Age ≥ 85 yr24.623.90.90
HR on arrival ≥ 110 beats/min32.38.7<0.0001
Respiratory rate ≥ 30 breaths/min (n = 62 and 457)a16.19.00.08
SaO2 on arrival < 90% 21.99.8<0.01
SaO2 on room air < 88% (n = 36 and 318)8.34.40.30
SaO2 on room air < 90% (n = 36 and 318)13.99.10.36
CTAS level 1 or 2 54.042.60.09
Onset of respiratory distress < 4 hours (n = 44 and 434)11.45.30.10
Chest x-ray shows HF or cardiomegaly75.472.30.60
Hemoglobin < 100 9.211.50.59
Urea ≥ 12 mmol/L37.523.60.02
Creatinine ≥ 150 mmol/L27.718.00.06
Glucose ≥ 18 mmol/L (n = 65 and 482)6.21.90.03
CO2 ≥ 35 mmol/L6.21.80.03
pCO2 ≥ 70 mm Hg (n = 11 and 46) 9.12.20.26
pH < 7.3 (n = 11 and 46)36.46.5<0.01
pH < 7.35 (n = 11 and 46)36.413.00.07
pH ≥ 7.48 (n = 11 and 46)0.010.90.25
NT-proBNP ≥ 5,000 (n = 29 and 276)75.940.9<0.001
NT-proBNP ≥ 25,000 (n = 29 and 276)17.25.1<0.01
Troponin T or I ≥ 99th percentile (n = 65 and 473)12.32.1<0.0001
Troponin T or I  ≥  MI level (n = 65 and 473)b10.80.9<0.0001
Walk test highest HR ≥ 110 (n = 44 and 436)34.118.80.02
Walk test lowest SaO2 < 90% (n = 44 and 432)11.410.90.92
Walk test lowest SaO2 < 88% (n = 44 and 437)18.223.60.42
Borg score ≥ 5 (n = 42 and 417)19.124.00.47
Walk test duration ≤ 1 minute (n = 44 and 441)4.61.10.07
Walk test 1 minute postwalk highest HR ≥ 110 (n = 43 and 432)18.65.6<0.01
Walk test 1 minute postwalk SaO2 <90% (n = 44 and 432)11.410.90.92

Table 6 shows the multivariate logistic regression model that determined independent predictors for SAEs. This model was developed on a data set of 507 cases without missing values. This model has a nonsignificant Hosmer-Lemeshow chi-square goodness-of-fit statistic (p = 0.75) and an area under the receiver operating characteristic (ROC) curve of 0.77 (95% confidence interval [CI] = 0.71 to 0.84). Variables not significant in the final model included age, disposition status (admitted or discharged), and several interaction terms. We conducted a sensitivity analysis by performing multivariate analysis without NT-proBNP and found that the other nine variables remained in the model with very similar coefficients but a lower area under the ROC curve (0.75) (Data Supplement S2, available as supporting information in the online version of this paper).

Table 6. Independent Predictors of SAEs as Determined by Stepwise Logistic Regression Analysis for HF Patientsa
VariableBOdds Ratio95% CI
  1. Hosmer-Lemeshow Goodness-of-fit p-value = 0.753.

  2. Area under ROC curve = 0.774; 95% CI = 0.706 to 0.843.

  3. ECG = electrocardiogram; HF = heart failure; NT-proBNP = N-terminal prohormone of brain natriuretic peptide; ROC = receiver operating characteristic; SAE = serious adverse event; SaO2 = oxygen saturation.

  4. a

    Model developed for 507 patients without missing values; variables with missing values: ECG (12), SaO2 (4), troponin (21), urea (8), CO2 (6), heart rate during walk test (5).

  5. b

    BNP imputed for 254 cases.

NT-proBNP > 5,000 ng/Lb0.772.160.92–5.07
History of stroke or TIA0.952.581.24–5.40
Prior intubation1.544.670.91–23.90
ECG has acute ischemic changes1.193.301.25–8.73
Heart rate ≥ 110 beats/min on ED arrival0.992.691.23–5.88
SaO2 < 90% on arrival0.772.151.01–4.60
Troponin I or T elevated ≥ MI level 1.193.301.04–10.42
Urea ≥ 12 mmol/L0.661.931.01–3.69
Serum CO2 > 35 mmol/L1.594.921.14–21.24
Heart rate ≥ 110 beats/min during 3-minute walk test 0.641.890.97–3.69
Intercept -3.710.020.01–0.05

Ottawa Heart Failure Risk Scale

The Ottawa Heart Failure Risk Scale (Figure 1) consists of 10 elements from history, examination, or investigations and has a maximum score of 15. We found that the risk of SAE varied from 2.8%, for a score of 0, to 89.0%, for a score of 9. Figure 2 shows good calibration between observed and expected probability of SAE up to a score of 6, beyond which there is variability due to small numbers (Hosmer-Lemeshow goodness-of-fit p = 0.950).

image

Figure 1. Ottawa Heart Failure Risk Scale to identify ED patients with HF at high risk for SAEs. ECG = electrocardiogram; HF = heart failure; MI = myocardial infarction; NT-proBNP = N-terminal prohormone of brain natriuretic peptide; SAEs = serious adverse events; TIA = transient ischemic attack.

Download figure to PowerPoint

image

Figure 2. Observed versus expected probability of SAE score in HF patients. HF = heart failure; SAE = serious adverse event.

Download figure to PowerPoint

Table 7 shows the classification performance and expected admission proportions for the Ottawa Heart Failure Risk Scale compared to current practice at the six study hospitals. The scale could improve upon the sensitivity of current practice, which only sees 52.0% of patients with SAE being admitted on the first ED visit. For example, choosing total point score thresholds of 1, 2, or 3 as a decision for admission would be associated with sensitivities of 95.2, 80.6, or 64.5%, respectively. These theoretical admission threshold levels would result in admission rates of 80.9, 50.5, or 27.4%, respectively, compared to the current admission rate of 38.1% at the study sites.

Table 7. Classification Performance and Expected Admission Proportions for the Ottawa Heart Failure Risk Scale Compared to Current Practice at the Six Study Hospital Sites
 No. of PatientsSensitivitySpecificityLR+Estimated Probability of Having SAEEstimated Proportion Admitted,a%
  1. HF = heart failure; LR = likelihood ratio; SAE = serious adverse event.

  2. a

    For Heart Failure Risk Score, estimated hospital admission proportion if threshold were greater than or equal to the specific point total; e.g., admission 50.5% at point total threshold ≥2.

  3. b

    Model developed for 507 patients without missing values.

Current practice5590.5080.619 0.11638.1
HF risk score 507b     
0971.00001.00.028100
11540.9520.2111.20.05180.9
21170.8060.5371.70.09250.5
3600.6450.7782.90.15927.4
4380.4840.894.40.26115.6
5220.3550.9578.30.3988.1
690.2260.98920.50.5533.7
760.1450.99872.50.6982.0
830.0480.99824.00.8120.8
910.01610.8900.2

The classification performance of the final score categories was validated using the sensitivity, specificity, positive LR, and negative LR. This internal validation showed the risk scores to be very accurate across 1,000 replications using the bootstrap method (Data Supplement S3, available as supporting information in the online version of this paper).[33]

Discussion

  1. Top of page
  2. AbstractResumen
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. References
  9. Supporting Information

Interpretation of Results

We found a relatively high frequency of SAEs among patients who present to the ED with HF, a relatively low hospital admission rate, and a disturbingly high proportion of poor outcomes among patients discharged home from the ED. We were able to establish the strong association of some 22 clinical and laboratory predictors with the development of SAEs, and multivariate analyses resulted in a concise and accurate model of independent risk factors for SAEs. We then developed the 10-element Ottawa Heart Failure Risk Scale, which provides an estimate of the risk of poor outcomes. All criteria are sensible and some are unique, such as the 3-minute walk test and quantitative assessment of NT-proBNP. Physicians at sites that do not have immediate NT-proBNP values available can use the scale without this item. The scale does not require sophisticated imaging or expensive testing to assist decision-making. Use of this scale could assist in the identification of patients most at risk for adverse outcomes and who are most in need of admission or early follow-up. We expect that this risk scale, once fully validated, could be widely used and will improve both hospital admission practices and the safety of ED management decisions.

Previous Studies

In the past, virtually all patients with acute HF were hospitalized whereas today in Canada, many of these patients are seen in the ED and subsequently discharged home.[34] There is a lack of high-quality evidence, and the few studies to develop admission guidelines for HF have important limitations, particularly the use of large administrative databases rather than prospective and specific ED data collection.[15, 16, 35, 36] These studies did not evaluate patients after ED management and did not incorporate a functional measure, such as a walk test. None of these guidelines have been prospectively validated or implemented into practice. Lee et al.[18] developed a scoring model to predict mortality for HF patients already admitted to Canadian hospitals, but did not include patients discharged from the ED or their response to therapy in the ED. Recent studies have evaluated the value of troponin, ECG QRS duration, and precipitating factors in predicting outcomes of hospitalized HF patients.[37-39]

The serum marker BNP is being used more often for the diagnosis of patients with HF.[40-42] BNP is a cardiac neurohormone released by the ventricles, and higher levels are associated with increased mortality and morbidity for patients with HF.[43-45] Clinical trials have shown limited effect of routine BNP measurement on patient outcomes,[46-48] its value for improving care in the ED is unclear,[49] and its use is not widespread in many countries.[50, 51] Our study found that a very high NT-proBNP level is an independent predictor of SAE for HF patients in the ED.

Study Implications

We are concerned by the high rate of SAEs among HF patients discharged from the ED. The Ottawa Heart Failure Risk Scale has the potential to significantly improve patient safety by helping to ensure that physicians admit or arrange early follow-up for patients most at risk for poor outcomes. While Canadian hospitals would struggle with admitting 80% of HF patients as is the case for U.S. hospitals, we believe that even a modest increase in admission (e.g., to 50% from the current 38%) would very likely lead to safer management practices. More important than increasing the admission rate is ensuring admission of the correct patients, i.e., those at highest risk of a poor outcome. Alternately, the scale could be used to identify at-risk patients who should have guaranteed early follow-up if they are not admitted, perhaps in specialized HF clinics.[6]

Limitations

  1. Top of page
  2. AbstractResumen
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. References
  9. Supporting Information

We believe that the study has significant strengths, including multicenter and rigorous prospective collection of real-time clinical data, as well as the unique use of the 3-minute walk test and quantitative NT-proBNP values. One issue is the inclusion of both admitted and discharged patients, which we believe is the correct methodologic approach. Admission status may confound the likelihood of an SAE occurring, in that some admitted patients will not suffer SAEs because they receive more intensive therapy and that the same patients might have suffered SAEs if they had been discharged home. Our objectives are to develop a decision tool that ensures appropriate admission of high risk patients (i.e., sensitivity) as well as minimizing admission of low risk patients (i.e., specificity). We can only do this by evaluating admitted patients. We also note that admitted status was not significantly associated with SAE in the multivariate model.

We believe that use of the “monitored unit” (not an ambulatory telemetry unit) as a criterion for SAE is an important outcome that almost always reflects severity of illness. Such patients would likely suffer significant morbidity if they had been discharged, as they were bed-ridden and required constant vital sign monitoring. This new risk scale should be prospectively validated in a new set of ED patients with HF. In particular, future studies should attempt to obtain quantitative NT-proBNP values on all patients. We suggest that the scale can be used effectively with or without NT-proBNP values, depending on the availability of this test. We chose to impute missing values for NT-proBNP but not for other variables missing a small number of values. While we had relatively few outcomes relative to the number of predictors, internal validation with bootstrapping demonstrated that the estimate and classification performance for risk scores were stable.[52] We chose not to adjust the analysis despite the fact that some patients were enrolled more than once. We were unable to enroll a large number of eligible patients because they presented outside of normal business hours. We could detect neither selection bias nor threat to the validity of our findings. Finally, it is possible, although unlikely, that some SAEs were not identified in patients who returned to different hospitals. If such events did occur, we do not believe that this would have had a significant effect on the identification of variables strongly associated with the primary outcome.

Conclusions

  1. Top of page
  2. AbstractResumen
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. References
  9. Supporting Information

Many heart failure patients are discharged home from the ED and then suffer severe adverse events or death. We have developed an accurate and easy to use Ottawa Heart Failure Risk Scale that can be used to stratify the risk of poor outcomes for ED patients with heart failure and to enable rational and safe disposition decisions. With appropriate validation, this scale could ultimately benefit both patients and health care systems by ensuring appropriate admissions, targeting those who need early follow-up, and diminishing unnecessary hospitalizations.

The authors thank the invaluable assistance of the following individuals from the study sites: Jewish General Hospital (Montreal, Quebec)—Antoinette Colacone, RN, Chris Tselios, BSc, Yanka Mihaylova; Kingston General Hospital (Kingston, Ontario)—Katherine Bowes, RN, Jane Reid, RN; Mount Sinai Hospital (Toronto, Ontario)—Michelle Loftus, RN; Ottawa Hospital, Civic and General Campuses (Ottawa, Ontario)—Sara Dorrington, RRT, James Acheson, RRT, Michael Gatta, RRT, Michelle Saulnier, RRT, Krista Bangs, RRT, Christine Belisle, RRT, Vanessa Lawlor, RRT; and University of Alberta Hospital (Edmonton, Alberta)—Debbie Boyko, RN, Dion Pasichnyk. In addition, we thank the following individuals from the coordinating center in Ottawa, Ontario: Ottawa Hospital Research Institute—Tami Clavet, RN, Jennifer Brinkhurst, Connor Sheehan, Angela Marcantonio, Sheryl Domingo, My-Linh Tran; Department of Pathology and Laboratory Medicine, Ottawa Hospital—Carol Ann Jodouin; University of Ottawa Heart Institute—Li Chen. Finally we are grateful to the many physicians, nurses, and respiratory therapists at the study hospitals for their very helpful assistance with patient identification and data collection.

References

  1. Top of page
  2. AbstractResumen
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. References
  9. Supporting Information
  • 1
    Lloyd-Jones D, Adams R, Carnethon M, et al. Heart disease and stroke statistics–2009 update: a report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation. 2009; 119:e21181.
  • 2
    Schappert SM, Burt CW. Ambulatory care visits to physician offices, hospital outpatient departments, and emergency departments: United States, 2001–02. Vital Health Stat 13. 2006; (159):166.
  • 3
    Hugli O, Braun JE, Kim S, Pelletier AJ, Camargo CA Jr. United States emergency department visits for acute decompensated heart failure, 1992 to 2001. Am J Cardiol. 2005; 96:153742.
  • 4
    Canadian Government. Statistics Canada. Available at: http://www.statcan.gc.ca/start-debut-eng.html. Accessed Oct 20, 2012.
  • 5
    Tsuyuki RT, Shibata MC, Nilsson C, Hervas-Malo M. Contemporary burden of illness of congestive heart failure in Canada. Can J Cardiol. 2003; 19:4368.
  • 6
    Lee DS, Stukel TA, Austin PC, et al. Improved outcomes with early collaborative care of ambulatory heart failure patients discharged from the emergency department. Circulation. 2010; 122:180614.
  • 7
    McAlister FA, Lawson FM, Teo KK, Armstrong PW. A systematic review of randomized trials of disease management programs in heart failure. Am J Med. 2001; 110:37884.
  • 8
    Ezekowitz JA, Bakal JA, Kaul P, Westerhout CM, Armstrong PW. Acute heart failure in the emergency department: short and long-term outcomes of elderly patients with heart failure. Eur J Heart Fail. 2008; 10:30814.
  • 9
    Lee DS, Schull MJ, Alter DA, et al. Early deaths in patients with heart failure discharged from the emergency department: a population-based analysis. Circ Heart Fail. 2010; 3:22835.
  • 10
    Arnold JM, Howlett JG, Dorian P, et al. Canadian Cardiovascular Society Consensus Conference recommendations on heart failure update 2007: prevention, management during intercurrent illness or acute decompensation, and use of biomarkers. Can J Cardiol. 2007; 23:2145.
  • 11
    Dickstein K, Cohen-Solal A, Filippatos G, et al. ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure 2008: the Task Force for the Diagnosis and Treatment of Acute and Chronic Heart Failure 2008 of the European Society of Cardiology. Developed in collaboration with the Heart Failure Association of the ESC (HFA) and endorsed by the European Society of Intensive Care Medicine (ESICM). Eur Heart J. 2008; 29:2388442.
  • 12
    Cowie MR, Wood DA, Coats AJS, et al. Survival of patients with a new diagnosis of heart failure: a population based study. Heart. 2000; 83:50510.
  • 13
    The Task Force on Heart Failure of the European Society of Cardiology. Guidelines for the diagnosis of heart failure. Eur Heart J. 1995; 16:74151.
  • 14
    Fraticelli A, Gesuita R, Vespa A, Paciaroni E. Congestive heart failure in the elderly requiring hospital admission. Arch Gerontol Geriatr. 1996; 23:22538.
  • 15
    Auble TE, Hsieh M, Gardner W, Cooper GF, Stone RA, McCausland JB. A prediction rule to identify low-risk patients with heart failure. Acad Emerg Med. 2005; 12:51421.
  • 16
    Fonarow GC, Adams KF, Abraham WT, et al. Risk stratification for in-hospital mortality in acutely decompensated heart failure. JAMA. 2005; 293:57280.
  • 17
    Aronson D, Mittleman MA, Burger AJ. Elevated blood urea nitrogen level as a predictor of mortality in patients admitted for decompensated heart failure. Am J Med. 2004; 116:46673.
  • 18
    Lee DS, Austin PC, Rouleau JL, Liu PP, Naimark D, Tu JV. Predicting mortality among patients hospitalized for heart failure. JAMA. 2003; 290:25817.
  • 19
    Hsieh M, Auble TE, Yealy DM. Predicting the future: can this patient with acute congestive heart failure be safely discharged from the emergency department? Ann Emerg Med. 2002; 39:1819.
  • 20
    Enright PL, Sherrill DL. Reference equations for the six-minute walk in healthy adults. Am J Respir Crit Care Med. 1998; 158:13847.
  • 21
    Enright PL. The six-minute walk test. Respir Care. 2003; 48:7835.
  • 22
    Enright PL, McBurnie MA, Bittner V, et al. The 6-min walk test: a quick measure of functional status in elderly adults. Chest. 2003; 123:38798.
  • 23
    Curtis JP, Rathore SS, Wang Y, Krumholz HM. The association of 6-minute walk performance and outcomes in stable outpatients with heart failure. J Cardiac Failure. 2004; 10:914.
  • 24
    Brooks D, Parson J, Tran D, et al. The two-minute walk test as a measure of functional capacity in cardiac surgery patients. Arch Phys Med Rehab. 2004; 86:152530.
  • 25
    Solway S, Brooks D, Lacasse Y, Thomas S. A qualitative systematic overview of the measurement properties of functional walk tests used in the cardiorespiratory domain. Chest. 2001; 119:25670.
  • 26
    Sciurba F, Criner GJ, Lee SM, et al. Six-minute walk distance in chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2003; 167:15227.
  • 27
    Pan AM, Stiell IG, Clement CM, Acheson J, Aaron SD. Feasibility of a structured 3-minute walk test as a clinical decision tool for patients presenting to the emergency department with acute dyspnea. Emerg Med J. 2009; 26:27882.
  • 28
    Alpert JS, Thygesen K, Antman E, Bassan JP. Myocardial infarction redefined - a consensus document of The Joint European Society of Cardiology/American College of Cardiology Committee for the redefinition of myocardial infarction. J Am Coll Cardiol. 2000; 36:95969.
  • 29
    Donders AR, van der Heijden GJ, Stijnen T, Moons KG. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol. 2006; 59:108791.
  • 30
    Little RJ, Rubin DB. Statistical Analysis With Missing Data. 2nd ed. New York, NY: John Wiley and Sons, 2002.
  • 31
    Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. 2001; 54:77481.
  • 32
    Le Gal G, Righini M, Roy PM, et al. Prediction of pulmonary embolism in the emergency department: the revised Geneva score. Ann Intern Med. 2006; 144:16571.
  • 33
    Efron B, Tibshirani R. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Stat Sci. 1986; (1):5477.
  • 34
    Brophy JM, Deslauriers G, Boucher B, Rouleau JL. The hospital course and short term prognosis of patients presenting to the emergency department with decompensated congestive heart failure. Can J Cardiol. 1993; 9:21924.
  • 35
    Hsieh M, Auble TE, Yealy DM. Validation of the acute heart failure index. Ann Emerg Med. 2008; 51:3744.
  • 36
    Van Spall HG, Atzema C, Schull MJ, et al. Prediction of emergent heart failure death by semi-quantitative triage risk stratification. PLoS One. 2011; 6:e23065.
  • 37
    Peacock WF, De Marco T, Fonarow GC, et al. Cardiac troponin and outcome in acute heart failure. N Engl J Med. 2008; 358:211726.
  • 38
    Wang NC, Maggioni AP, Konstam MA, et al. Clinical implications of QRS duration in patients hospitalized with worsening heart failure and reduced left ventricular ejection fraction. JAMA. 2008; 299:265666.
  • 39
    Fonarow GC, Abraham WT, Albert NM, et al. Factors identified as precipitating hospital admissions for heart failure and clinical outcomes: findings from OPTIMIZE-HF. Arch Intern Med. 2008; 168:84754.
  • 40
    Mueller C, Scholer A, Laule-Kilian K, et al. Use of B-type natriuretic peptide in the evaluation and management of acute dyspnea. N Engl J Med. 2004; 350:64754.
  • 41
    Maisel AS, Krishnaswamy P, Nowak RM, et al. Rapid measurement of B-type natriuretic peptide in the emergency diagnosis of heart failure. N Engl J Med. 2002; 347:1617.
  • 42
    Harrison A, Morrison LK, Krishnaswamy P, et al. B-type natriuretic peptide predicts future cardiac events in patients presenting to the emergency department with dyspnea. Ann Emerg Med. 2002; 39:1318.
  • 43
    Maisel A, Hollander JE, Guss D, et al. Primary results of the Rapid Emergency Department Heart Failure Outpatient Trial (REDHOT). A multicenter study of B-type natriuretic peptide levels, emergency department decision making, and outcomes in patients presenting with shortness of breath. J Am Coll Cardiol. 2004; 44:132833.
  • 44
    Januzzi JL Jr, Sakhuja R, O'Donoghue M, et al. Utility of amino-terminal pro-brain natriuretic peptide testing for prediction of 1-year mortality in patients with dyspnea treated in the emergency department. Arch Intern Med. 2006; 166:31520.
  • 45
    Steinhart B, Thorpe KE, Bayoumi AM, Moe G, Januzzi JL Jr, Mazer CD. Improving the diagnosis of acute heart failure using a validated prediction model. J Am Coll Cardiol. 2009; 54:15151.
  • 46
    Schneider HG, Lam L, Lokuge A, et al. B-type natriuretic peptide testing, clinical outcomes, and health services use in emergency department patients with dyspnea: a randomized trial. Ann Intern Med. 2009; 150:36571.
  • 47
    Moe GW, Howlett J, Januzzi JL, Zowall H. N-terminal pro-B-type natriuretic peptide testing improves the management of patients with suspected acute heart failure: primary results of the Canadian prospective randomized multicenter IMPROVE-CHF study. Circulation. 2007; 115:310310.
  • 48
    Pfisterer M, Buser P, Rickli H, et al. BNP-guided vs. symptom-guided heart failure therapy: the Trial of Intensified vs Standard Medical Therapy in Elderly Patients With Congestive Heart Failure (TIME-CHF) randomized trial. JAMA. 2009; 301:38392.
  • 49
    Yealy DM, Hsieh M. BNP is not a value-added routine test in the emergency department. Ann Emerg Med. 2009; 53:3879.
  • 50
    Murray H, Cload B, Collier CP, Sivilotti ML. Potential impact of N-terminal pro-BNP testing on the emergency department evaluation of acute dyspnea. Can J Emerg Med. 2006; 8:2518.
  • 51
    Hohl CM. Should natriuretic peptide testing be incorporated into emergency medicine practice? (Commentary). Can J Emerg Med 2006; 8:25961.
  • 52
    Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol. 2007; 165:7108.

Supporting Information

  1. Top of page
  2. AbstractResumen
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. References
  9. Supporting Information
FilenameFormatSizeDescription
acem12056-sup-0001-DataSupplementS1.pdfapplication/PDF20KData Supplement S1. Characteristics of patients not enrolled in the study during 835 visits.
acem12056-sup-0002-DataSupplementS2.pdfapplication/PDF21KData Supplement S2. Independent predictors of serious adverse events as determined by stepwise logistic regression analysis excluding the variable NT-proBNP.
acem12056-sup-0003-DataSupplementS3.pdfapplication/PDF23KData Supplement S3. Validation of classification performance of the SAE score in HF using bootstrapping.

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.