Clinical Decision Rules to Improve the Detection of Adverse Drug Events in Emergency Department Patients

Authors

  • Corinne M. Hohl MD,

    1. From the Department of Emergency Medicine (CMH, GSH, JRB), the Department of Statistics (EY), the Department of Pharmaceutical Sciences (CPA), and the School of Population and Public Health (MOW, SBS, JS), University of British Columbia, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (CMH, JRB) and Pharmacy Services (WWYC), Vancouver General Hospital, Vancouver, British Columbia, Canada; the Centre for Clinical Epidemiology & Evaluation (CMH, EY, JRB), Vancouver Coastal Health Research Institute, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (GSH) and the Centre for Health Evaluation and Outcome Sciences (JS), St. Paul’s Hospital, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (FH), University of Washington Medical Center, Seattle, WA; and the Vernon Jubilee Hospital (CPA), Vernon, British Columbia, Canada.
    Search for more papers by this author
  • Eugenia Yu MSc,

    1. From the Department of Emergency Medicine (CMH, GSH, JRB), the Department of Statistics (EY), the Department of Pharmaceutical Sciences (CPA), and the School of Population and Public Health (MOW, SBS, JS), University of British Columbia, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (CMH, JRB) and Pharmacy Services (WWYC), Vancouver General Hospital, Vancouver, British Columbia, Canada; the Centre for Clinical Epidemiology & Evaluation (CMH, EY, JRB), Vancouver Coastal Health Research Institute, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (GSH) and the Centre for Health Evaluation and Outcome Sciences (JS), St. Paul’s Hospital, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (FH), University of Washington Medical Center, Seattle, WA; and the Vernon Jubilee Hospital (CPA), Vernon, British Columbia, Canada.
    Search for more papers by this author
  • Garth S. Hunte MD,

    1. From the Department of Emergency Medicine (CMH, GSH, JRB), the Department of Statistics (EY), the Department of Pharmaceutical Sciences (CPA), and the School of Population and Public Health (MOW, SBS, JS), University of British Columbia, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (CMH, JRB) and Pharmacy Services (WWYC), Vancouver General Hospital, Vancouver, British Columbia, Canada; the Centre for Clinical Epidemiology & Evaluation (CMH, EY, JRB), Vancouver Coastal Health Research Institute, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (GSH) and the Centre for Health Evaluation and Outcome Sciences (JS), St. Paul’s Hospital, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (FH), University of Washington Medical Center, Seattle, WA; and the Vernon Jubilee Hospital (CPA), Vernon, British Columbia, Canada.
    Search for more papers by this author
  • Jeffrey R. Brubacher MD,

    1. From the Department of Emergency Medicine (CMH, GSH, JRB), the Department of Statistics (EY), the Department of Pharmaceutical Sciences (CPA), and the School of Population and Public Health (MOW, SBS, JS), University of British Columbia, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (CMH, JRB) and Pharmacy Services (WWYC), Vancouver General Hospital, Vancouver, British Columbia, Canada; the Centre for Clinical Epidemiology & Evaluation (CMH, EY, JRB), Vancouver Coastal Health Research Institute, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (GSH) and the Centre for Health Evaluation and Outcome Sciences (JS), St. Paul’s Hospital, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (FH), University of Washington Medical Center, Seattle, WA; and the Vernon Jubilee Hospital (CPA), Vernon, British Columbia, Canada.
    Search for more papers by this author
  • Faegheh Hosseini PharmD,

    1. From the Department of Emergency Medicine (CMH, GSH, JRB), the Department of Statistics (EY), the Department of Pharmaceutical Sciences (CPA), and the School of Population and Public Health (MOW, SBS, JS), University of British Columbia, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (CMH, JRB) and Pharmacy Services (WWYC), Vancouver General Hospital, Vancouver, British Columbia, Canada; the Centre for Clinical Epidemiology & Evaluation (CMH, EY, JRB), Vancouver Coastal Health Research Institute, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (GSH) and the Centre for Health Evaluation and Outcome Sciences (JS), St. Paul’s Hospital, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (FH), University of Washington Medical Center, Seattle, WA; and the Vernon Jubilee Hospital (CPA), Vernon, British Columbia, Canada.
    Search for more papers by this author
  • Chelsea P. Argent BPharm,

    1. From the Department of Emergency Medicine (CMH, GSH, JRB), the Department of Statistics (EY), the Department of Pharmaceutical Sciences (CPA), and the School of Population and Public Health (MOW, SBS, JS), University of British Columbia, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (CMH, JRB) and Pharmacy Services (WWYC), Vancouver General Hospital, Vancouver, British Columbia, Canada; the Centre for Clinical Epidemiology & Evaluation (CMH, EY, JRB), Vancouver Coastal Health Research Institute, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (GSH) and the Centre for Health Evaluation and Outcome Sciences (JS), St. Paul’s Hospital, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (FH), University of Washington Medical Center, Seattle, WA; and the Vernon Jubilee Hospital (CPA), Vernon, British Columbia, Canada.
    Search for more papers by this author
  • Winnie W.Y. Chan BPharm,

    1. From the Department of Emergency Medicine (CMH, GSH, JRB), the Department of Statistics (EY), the Department of Pharmaceutical Sciences (CPA), and the School of Population and Public Health (MOW, SBS, JS), University of British Columbia, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (CMH, JRB) and Pharmacy Services (WWYC), Vancouver General Hospital, Vancouver, British Columbia, Canada; the Centre for Clinical Epidemiology & Evaluation (CMH, EY, JRB), Vancouver Coastal Health Research Institute, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (GSH) and the Centre for Health Evaluation and Outcome Sciences (JS), St. Paul’s Hospital, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (FH), University of Washington Medical Center, Seattle, WA; and the Vernon Jubilee Hospital (CPA), Vernon, British Columbia, Canada.
    Search for more papers by this author
  • Matthew O. Wiens PharmD,

    1. From the Department of Emergency Medicine (CMH, GSH, JRB), the Department of Statistics (EY), the Department of Pharmaceutical Sciences (CPA), and the School of Population and Public Health (MOW, SBS, JS), University of British Columbia, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (CMH, JRB) and Pharmacy Services (WWYC), Vancouver General Hospital, Vancouver, British Columbia, Canada; the Centre for Clinical Epidemiology & Evaluation (CMH, EY, JRB), Vancouver Coastal Health Research Institute, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (GSH) and the Centre for Health Evaluation and Outcome Sciences (JS), St. Paul’s Hospital, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (FH), University of Washington Medical Center, Seattle, WA; and the Vernon Jubilee Hospital (CPA), Vernon, British Columbia, Canada.
    Search for more papers by this author
  • Samuel B. Sheps MD,

    1. From the Department of Emergency Medicine (CMH, GSH, JRB), the Department of Statistics (EY), the Department of Pharmaceutical Sciences (CPA), and the School of Population and Public Health (MOW, SBS, JS), University of British Columbia, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (CMH, JRB) and Pharmacy Services (WWYC), Vancouver General Hospital, Vancouver, British Columbia, Canada; the Centre for Clinical Epidemiology & Evaluation (CMH, EY, JRB), Vancouver Coastal Health Research Institute, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (GSH) and the Centre for Health Evaluation and Outcome Sciences (JS), St. Paul’s Hospital, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (FH), University of Washington Medical Center, Seattle, WA; and the Vernon Jubilee Hospital (CPA), Vernon, British Columbia, Canada.
    Search for more papers by this author
  • Joel Singer PhD

    1. From the Department of Emergency Medicine (CMH, GSH, JRB), the Department of Statistics (EY), the Department of Pharmaceutical Sciences (CPA), and the School of Population and Public Health (MOW, SBS, JS), University of British Columbia, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (CMH, JRB) and Pharmacy Services (WWYC), Vancouver General Hospital, Vancouver, British Columbia, Canada; the Centre for Clinical Epidemiology & Evaluation (CMH, EY, JRB), Vancouver Coastal Health Research Institute, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (GSH) and the Centre for Health Evaluation and Outcome Sciences (JS), St. Paul’s Hospital, Vancouver, British Columbia, Canada; the Department of Emergency Medicine (FH), University of Washington Medical Center, Seattle, WA; and the Vernon Jubilee Hospital (CPA), Vernon, British Columbia, Canada.
    Search for more papers by this author

  • Best paper award, American College of Emergency Physicians Research Forum, San Francisco, CA, October 2011.

  • This study was supported by grants from the Canadian Patient Safety Institute, the Michael Smith Foundation for Health Research, and the Vancouver Coastal Health Authority. None of the sponsors had any role in study design, data collection or processing, or analysis or preparation of the manuscript. All authors had access to the study data and agreed to submit the manuscript in its current form. The authors have no potential conflicts of interest to disclose.

  • Supervising Editor: Mark B. Mycyk, MD.

Address for correspondence and reprints: Corinne M. Hohl, MD; e-mail: chohl@interchange.ubc.ca.

Abstract

ACADEMIC EMERGENCY MEDICINE 2012; 19:640–649 © 2012 by the Society for Academic Emergency Medicine

Abstract

Objectives:  Adverse drug events (ADEs) are unintended and harmful consequences of medication use. They are associated with high health resource use and cost. Yet, high levels of inaccuracy exist in their identification in clinical practice, with over one-third remaining unidentified in the emergency department (ED). The study objective was to derive clinical decision rules (CDRs) that are sensitive for the detection of ADEs, allowing their systematic identification early in a patient’s hospital course.

Methods:  This was a prospective observational cohort study carried out in two Canadian tertiary care hospitals. Participants were adults presenting to the ED having ingested at least one prescription or over-the-counter medication within 2 weeks. Nurses and physicians evaluated patients for standardized clinical findings. A second evaluator performed interobserver assessments of predictor variables in a subset of patients. Pharmacists, who were blinded to the predictor variables, evaluated all patients for ADEs. An independent committee reviewed and adjudicated cases where the ADE assessment was uncertain or the pharmacist’s diagnosis differed from the physician’s working diagnosis. The primary outcome was an ADE that required a change in medical therapy, diagnostic testing, consultation, or hospital admission. CDRs were derived using kappa coefficients, chi-square statistics, and recursive partitioning.

Results:  Among 1,591 patients, 131 (8.2%, 95% confidence interval [CI] = 7.0% to 9.7%) were diagnosed with the primary outcome. The following variables were associated with ADEs and were used to derive two CDRs: 1) presence of comorbid conditions, 2) antibiotic use within 7 days, 3) medication changes within 28 days, 4) age ≥80 years, 5) arrival by ambulance, 6) triage acuity, 7) recent hospital admission, 8) renal failure, and 9) use of three or more prescription medications. The more sensitive rule had a sensitivity of 96.7% (95% CI = 91.8% to 98.6%) and required 40.8% (95% CI = 37.7% to 42.9%) of patients to have medication review. The more specific rule had a sensitivity 90.8% (95% CI = 81.4% to 95.7%) and required 28.3% of patients to proceed to medication review.

Conclusions:  The authors derived CDRs that identified patients with ADEs with high sensitivity. These rules may improve the identification of ADEs early in a patient’s hospital course while limiting the number of patients requiring a detailed medication review.

It has been estimated that up to 23,750 Canadians die each year as a result of preventable adverse events related to medical care.1 This figure is eight times higher than the number of motor vehicle fatalities in Canada over the same time period.2 Of deaths related to adverse events, those related to medication use are the most common.3–6

Adverse drug events (ADEs) are unintended and harmful consequences of medication use. ADEs to outpatient medications account for up to one in nine emergency department (ED) visits in North America and for a large proportion of emergency hospital admissions.7–12 Recent data demonstrate that patients presenting to the ED with ADEs spend more days in the hospital, incur more outpatient health care encounters, and have higher health care costs after adjustment for known confounders, compared with other patients.13

Yet, despite the frequency of ADEs and their association with high health resource use, ADEs are commonly misdiagnosed in the ED.9,14 Estimates suggest that physicians misdiagnose 37% to 49% of presentations ultimately deemed medication-related.9,14,15 In one large prospective study, over half of these patients were discharged home without a medication review by a pharmacist.14 Lack of ADE identification, treatment, and documentation during the clinical encounter in the ED may contribute to the morbidity and health care cost associated with these events.13,15–19

Clinical pharmacists are a scarce and expensive resource, making routine screening of ED patients by pharmacists untenable.20–22 Even in centers that have access to ED pharmacists, there are no evidence-based criteria to assist pharmacists in identifying the highest-risk patients to optimize the use of their time, while ensuring that the majority of significant ADEs are identified. In centers with access to in-hospital pharmacists, long delays to inpatient services often exist, leading to inappropriate delays to the diagnosis of ADEs.23,24

The objective of this study was, therefore, to derive clinical decision rules (CDRs) that would identify patients at high risk for ADEs to improve early identification and treatment while limiting the number of patients requiring medication review by a pharmacist or another medication specialist.

Methods

Study Protocol

This was a prospective cohort study conducted in two Canadian tertiary care teaching hospitals (Vancouver General Hospital and St. Paul’s Hospital, Vancouver, British Columbia) with a combined annual ED census of 135,000 patients. The study protocol was approved by the University of British Columbia Clinical Research Ethics Board and registered with ClinicalTrials.gov (NCT00727610).

Selection of Participants

Patients presenting to the EDs of participating institutions between July 2008 and January 2009 were eligible for enrollment. Data collection shifts were stratified by time of day (00:00 to 07:59, 08:00 to 15:59, and 16:00 to 23:59) and day of the week and scheduled proportional to the volume of patients who had presented during the same time interval in the prior year. We applied a systematic selection algorithm to ensure enrollment of a representative sample during data collection shifts (Data Supplement S1, available as supporting information in the online version of this paper).12

Patients 19 years of age or older who reported using at least one prescription or over-the-counter medication in the 2 weeks prior to presentation, and who spoke English or had a translator available, were eligible for enrollment. Patients were excluded if they exhibited violent behavior, presented with an intentional self-poisoning, were previously enrolled, presented for a scheduled revisit, were transferred directly to an admitting service, or left against medical advice or prior to seeing the physician or pharmacist.

Study Design

After patients were enrolled, research assistants placed colored data collection forms containing potential predictor variables on top of the patients’ charts (Data Supplement S2, available as supporting information in the online version of this paper). Potential predictor variables were generated from a literature review, consultation with experts, and previous studies on ADEs.11 ED nurses and physicians filled out data collection forms at the beginning of each patient encounter. Forms were filled out by two nurses and two physicians on a convenience sample of patients to measure the interreliability of potential predictor variables. Data collection forms were then removed from patient charts.

Residency-trained clinical pharmacists (FH, CA, and WC) collected demographic and clinical information from patients and treating emergency physicians (EPs) and by chart review. They verified medication histories using PharmaNet, a province-wide prescription-filling database. All admitted patients were followed until discharge. Consenting patients were contacted after discharge by telephone when the pharmacists believed further follow-up was necessary.

Our case finding methods are described in detail in Data Supplement S3 (available as supporting information in the online version of this paper).12–14 Briefly, after completion of data collection, the pharmacists evaluated whether or not the patient’s ED visit was due to an ADE using three standardized causality algorithms (see Definitions, next paragraph).4,25,26 Inter-rater reliability of the pharmacists’ assessment algorithm was evaluated and reported. Research assistants then interviewed the patient’s treating physician using a standardized questionnaire to determine the physician’s working diagnosis (Data Supplement S4, available as supporting information in the online version of this paper). When the physician and pharmacist determinations of a patient’s diagnosis were concordant, this was considered the criterion standard. If there was disagreement, or if either of them were uncertain, an independent committee adjudicated the case using a previously developed algorithm.12

Outcome Measures and Definitions

Varying case definitions of ADEs are used in clinical practice.3,6,27,28 To derive CDRs that would meet the needs of clinicians, one rule was derived for each of the two commonly used definitions. The primary outcome, an ADE, was defined as an “untoward and unintended event arising from the use of prescription or over-the-counter medications.”3,6,27 To meet the primary outcome definition, the ADE had to be classified as either an adverse drug reaction or an adverse event due to nonadherence, a prescription error, drug withdrawal, or a drug interaction. The secondary outcome, an adverse drug reaction, was “a response to a drug that is noxious and unintended, and occurs at doses normally used in man for the prophylaxis, diagnosis, or therapy of disease.”3 All adverse drug reactions are ADEs; in other words, adverse drug reactions are a subset of ADEs.

The severities of all ADEs and ADRs were rated: 1) a severe event caused death or required admission; 2) a moderate event required a change in medical management (medical therapy, a diagnostic procedure or consultation); and 3) a mild event required no change in therapy.12 To meet either outcome definition, the event had to be at least moderate in severity.

Data Analysis

To derive CDRs with stable test performance characteristics, we compared the potential predictor variables collected by two nurses and two physicians and measured the inter-rater agreement using intraclass correlation and kappa coefficients. Weighted kappas were calculated for variables with three or more ordinal categories. We selected potential predictor variables with substantial inter-rater agreement (kappa score ≥ 0.6) and subsequently included variables with moderate inter-rater agreement (kappa score ≥ 0.5) in the recursive partitioning tree. We used univariate odds ratios (OR), chi-square tests, or Fisher’s exact tests for categorical variables and Student’s t-tests for continuous variables to determine the strength of association between the potential predictor variables and the outcomes to identify the best potential predictor variables. The reliability of the outcome measure was calculated using Fleiss’ extension for multiple raters and reported using kappa scores.29

We used recursive partitioning analysis to derive the prediction rules.30 Patients for whom data were missing were excluded from the analysis to avoid the use of surrogate variables or data imputation. Our goal was to derive parsimonious rules that made clinical sense and led to the greatest possible reduction in unnecessary medication review while maintaining an a priori defined minimal threshold for sensitivity of 95% for our primary outcome measure (determined by a national survey of EPs).20 A rule for the secondary outcome measure with high specificity was desired for EDs with limited pharmacist resources to enhance feasibility and uptake. To meet this goal, an a priori defined minimal threshold for sensitivity of 90% for the secondary outcome measure was set. More details on the parameters used to build the decision trees are listed in Data Supplement S5 (available as supporting information in the online version of this paper).

The sensitivity and specificity with 95% confidence intervals (CIs) were calculated for each of the derived rules. We also determined the proportion of patients expected to proceed from screening to medication review with each of the rules.

The sample size calculation was based on 1) the median desired sensitivity of the rule for the primary outcome definition (of ≥95%) derived from a national survey of EPs;20 2) the desired precision of the sensitivity estimate, based on 95% CIs of ±5%; and 3) the estimated prevalence of the primary outcome measure using an estimated prevalence of moderate and severe ADEs meeting the primary outcome definition for this study of 8.8%, derived from our previous work.12 Based on these estimates, we required a sample of 130 events, to derive a rule with a target sensitivity of 95.4% (95% CI = 90.3% to 97.8%). This yielded a sample size requirement of 1,480 patients.

Results

We approached 2,289 ED patients and enrolled 1,591 (Figure 1). The mean (±SD) age of enrolled patients was 51.4 (±20.3) years, and 50.9% were female (Table 1). The median number of prescribed medications was 2 (interquartile range [IQR] = 1 to 5).

Figure 1.

 Patient flow.

Table 1. 
Patient Characteristics
Patient CharacteristicAll Patients (N = 1,591)
  1. Values are n (%) unless otherwise noted.

  2. CTAS = Canadian Triage and Acuity Score; IQR = interquartile range; OTC = over-the-counter.

  3. *The denominator is less than 1,591 due to missing values.

  4. †Excludes the number of OTC medications.

Mean age, yr (±SD)     51.4 (±20.3)
Female810 (50.9)
ED treatment location*
 Acute care area of ED749 (47.2)
 Minor care area of ED839 (52.8)
Arrived from*
 Home1418 (89.2)
 Homeless/shelter67 (4.2)
 Nursing home65 (4.1)
 Other39 (2.5)
CTAS score
 CTAS 16 (0.4)
 CTAS 2227 (14.3)
 CTAS 3703 (44.2)
 CTAS 4583 (36.6)
 CTAS 572 (4.5)
Most common chief complaints
 Abdominal pain164 (10.3)
 Chest pain 117 (7.4)
 Shortness of breath101 (6.3)
Most common comorbid conditions
 Hypertension339 (21.3)
 Depression155 (9.7)
 Asthma123 (7.7)
Median no. of comorbid conditions (IQR)2 (1–3)
Most common prescription medications
 Acetaminophen and codeine197 (12.4)
 Ramipril138 (8.7)
 Salbutamol126 (7.9)
Median no. of prescription medications (IQR)†2 (1–5)
Complementary medication use*173 (11.0)
OTC medication use*1101 (69.8)
Smoker*361 (23.0)
Alcohol use ≥2 drinks/day*174 (11.1)
Illicit drug use*194 (12.4)
General practitioner*1358 (86.2)
Median no. of prescribing physicians (IQR)*1 (1–2)
Disposition from ED*
 Admitted287 (18.1)
 Home1292 (81.3)
 Deceased3 (0.2)
 Other7 (0.4)

The inter-rater reliability of the pharmacist assessment of ADE was 0.75 (95% CI = 0.52 to 0.98). Among enrolled patients, 131 (131/1,591; 8.2%, 95% CI = 7.0% to 9.7%) were diagnosed with 134 moderate or severe ADEs that met the primary outcome definition (Table 2 and Data Supplement S6, available as supporting information in the online version of this paper). Of these, 65 patients (4.1%, 95% CI = 3.2% to 5.2%) were diagnosed with one or more moderate or severe adverse drug reactions meeting the secondary outcome definition. Sixty-five patients were diagnosed with ADEs due to nonadherence (4.1%, 95% CI = 3.2% to 5.2%), and one patient (0.06%, 95% CI = 0.01% to 0.3%) had a drug interaction.

Table 2. 
Characteristics of 134 Moderate and Severe ADEs Identified in 131 ED Patients
Characteristics of ADEsNumber (%)
  1. ADE = adverse drug event.

  2. *When multiple culprit medications or multiple classes of culprit medications were listed as cause of the ADE, the pharmacist selected the medication and culprit medication that he or she felt was most significant in contributing to the event.

Relationship to chief complaint
 Chief complaint-related94 (70.1)
 Incidentally found40 (29.9)
Classification
 Adverse drug reactions68 (50.7)
 Noncompliance or drug withdrawal65 (48.5)
 Drug interactions1 (0.75)
Severity
Severe16 (11.9)
Moderate118 (88.1)
Preventability
Preventable99 (73.9)
Nonpreventable35 (26.1)
Most commonly implicated medications*
  1. Acetaminophen and codeine8 (6.0)
  2. Hydrochlorothiazide6 (4.8)
  3. Olanzapine6 (4.8)
  4. Phenytoin6 (4.8)
  5. Aspirin4 (3.0)
  6. Carbamazepine4 (3.0)
  7. Glyburide4 (3.0)
  8. Warfarin4 (3.0)
  9. Prednisone4 (3.0)
 10. Ramipril3 (2.2)
Most common medication classes*
  1. Antihypertensives/diuretics22 (16.4)
  2. Antiinfectives 
  3. Psychotropic agents 
  4. Narcotic analgesics 
  5. Endocrinologic agents 
  6. Antiepileptics 
  7. Antiplatelet 
  8. Nonnarcotic analgesics 
  9. Anticoagulants 
 10. Antihyperglycemic agents 

Numerous variables were associated with ADEs and adverse drug reactions and had inter-rater reliability that is considered acceptable (κ > 0.6) using a common classification system (Tables 3 and 4).29 These included age, the number and type of comorbid conditions reported, illicit drug use, some medication classes, the mode of arrival to the ED, and recent hospitalization, antibiotic use, or medication changes. The use of opioids or benzodiazepines and renal failure were fairly reliable (κ = 0.5 to 0.6) and considered secondary candidate variables for entry into the rules.

Table 3. 
Univariate Correlation of Potential Predictor Variables With ADEs
Potential Predictor VariablesPatients Without ADEs* (n = 1,460)Patients With ADEs* (n = 131)p-value†ORs (95% CIs)‡
  1. Values are n (%) unless otherwise noted.

  2. ADE = adverse drug event; ASA = acetylsalicylic acid; CTAS = Canadian Triage and Acuity Score; NSAID = non-steroidal anti-inflammatory drug; HIV = human immunodeficiency virus; OCT = over-the-counter.

  3. *ADE is defined as moderate or severe adverse drug reaction, adverse drug interaction, drug withdrawal reaction, prescription error, and nonadherence.

  4. †p-values for the associated with ADEs, computed using chi-square or Fisher’s exact tests for categorical variables, and Student’s t-tests for continuous variables.

  5. ‡ORs computed for continuous and binary variables only.

  6. §Denominator less than 1,460 for patients without ADEs, and less than 131 for patients with ADEs due to missing variables.

Demographic variables
 Mean age, yr (±SD)     51.1 (±20.2)   54.0 (±21.3)0.121.01 (1.00–1.02)
 Age cutoff ≥ 80 yr171 (11.71)27 (20.61)0.00311.96 (1.24–3.07)
 Female 753 (51.6)57 (43.5)0.0770.72 (0.50–1.04)
 CTAS score  0.012 
  CTAS 14 (0.3)2 (1.5)  
  CTAS 2210 (14.4)17 (13.0)  
  CTAS 3631 (43.2)72 (55.0)  
  CTAS 4547 (37.5)36 (27.5)  
  CTAS 568 (4.7)4 (3.1)  
 Creatinine ≥ 150 mmol/L (>1.7 mg/dl) 59 (7.7)12 (12.2)0.121.68 (0.87–3.25)
Potential predictor variables§
 Use of OTC supplements 549 (51.1)45 (45.5)0.280.80 (0.53–1.21)
 Use of herbal remedies 164 (15.3)11 (10.9)0.240.68 (0.35–1.30)
 Use of recreational drugs 120 (11.1)17 (17.2)0.0691.67 (0.96–2.91)
 Taking opioids or benzodiazepines353 (24.3)49 (37.4)0.0011.86 (1.28–2.71)
 Taking ASA or other salicylates 245 (16.8)27 (20.6)0.271.29 (0.82–2.01)
 Taking anticoagulant/antiplatelet agent 102 (7.0)14 (10.7)0.121.59 (0.88–2.87)
 Taking NSAID323 (22.1)23 (17.6)0.220.75 (0.47–1.20)
 Taking insulin/hypoglycemic agents74 (5.1)18 (13.7)<0.00012.98 (1.72–5.17)
 Taking antihypertensive/diuretics362 (24.8)57 (43.5)<0.00012.33 (1.62–3.36)
 Taking antiarrhythmics68 (4.7)9 (6.9)0.261.51 (0.74–3.10)
 Taking seizure medications59 (4.0)17 (13.0)<0.00013.54 (2.00–6.27)
 Taking chemotherapeutic agents11 (0.75)2 (1.5)0.292.04 (0.45–9.29)
 Number of prescribing physicians   <0.0001 
  At most one717 (68.7)43 (42.2) 0.33 (0.22–0.50)
  Two or more327 (31.3)59 (57.8)  
 History of renal failure54 (3.7)13 (9.9)0.00072.86 (1.52–5.40)
 History of heart failure54 (3.7)14 (10.7)0.00023.11 (1.68–5.77)
 History of atrial fibrillation66 (4.5)10 (7.8)0.0991.77 (0.89–3.54)
 History of diabetes98 (6.7)18 (13.7)0.00302.21 (1.29–3.79)
 History of cancer 78 (5.4)7 (5.3)11.00 (0.45–2.21)
 History of HIV30 (2.1)7 (5.3)0.0282.69 (1.16–6.25)
 History of psychiatric problems201 (13.8)39 (29.8)<0.00012.64 (1.77–3.96)
 Had a rash in the past 2 weeks 84 (7.7)15 (14.7)0.0152.06 (1.14–3.72)
 Had a seizure in the past 2 weeks19 (1.8)12 (11.8)<0.00017.50 (3.53–15.95)
 On ≥2 prescription medications 692 (47.4)93 (71.0)<0.00012.70 (1.85–4.00)
 On ≥3 prescription medications 555 (38.0)76 (58.0)<0.00012.27 (1.56–3.22)
 Ambulance arrival356 (27.4)57 (53.8)<0.00013.09 (2.07–4.61)
 Regular general practitioner839 (86.1)70 (81.4)0.230.70 (0.40–1.25)
 Last blood work  0.21 
  ≤7 days176 (18.9)22 (26.8)  
  1–4 weeks163 (17.5)9 (11.0)  
  1–3 months123 (13.2)12 (14.6)  
  ≥3 months468 (50.3)39 (47.6)  
 Hospitalized in the past 28 days129 (8.9)23 (17.6)0.00142.17 (1.34–3.53)
 Medication changes within 2 weeks377 (26.1)55 (42.0)<0.00012.05 (1.42–2.96)
 Compliant with prescription medication668 (91.1)61 (76.3)<0.00010.32 (0.18–0.56)
 On antibiotics in the last 7 days 191 (13.1)28 (21.4)0.00891.80 (1.15–2.80)
 Medications blister packed97 (10.4)17 (20.2)0.00622.19 (1.23–3.88)
 Assistance in taking medications193 (13.5)28 (21.9)0.00931.79 (1.15–2.80)
 Presence of mental health issues 139 (14.2)25 (28.7)0.00032.44 (1.48–4.01)
 Confusion104 (10.5)16 (18.0)0.0321.86 (1.05–3.32)
 ≥1 Comorbid conditions 1089 (74.6)125 (95.4)<0.00017.10 (3.10–16.24)
Table 4. 
Inter-rater Agreement and Measures of Association of Potential Predictor Variables with ADEs and Adverse Drug Reactions
Potential Predictor Variablesκ/ICC (95% CI)†Association With Outcomes (p-values)*
ADE‡Adverse Drug Reaction§
  1. ADE = adverse drug event; ASA = acetylsalicylic acid; HIV = human immunodeficiency virus; ICC = intraclass correlation coefficient; NSAID = nonsteroidal anti-inflammatory; OCT = over-the-counter.

  2. *p-values computed using chi-square statistics and Fisher’s exact tests for categorical variables and by logistic regression for continuous variables.

  3. †The sample of patients from which the inter-rater agreement statistics were calculated was n = 69.

  4. ‡Primary outcome: ADE (moderate or severe adverse drug reactions, adverse drug interactions, drug withdrawal reactions, prescription errors, and nonadherence).

  5. §Secondary outcome moderate or severe adverse drug reactions.

Age (continuous variable)1.00 (1 to 1)0.120.0016
Age cutoff ≥ 80 years1.00 (1 to 1)0.0031<0.0001
Use of OTC supplements0.35 (0.13 to 0.58)0.280.95
Use of herbal remedies0.55 (0.30 to 0.80)0.240.59
Use of recreational drugs0.80 (0.58 to 1)0.070.087
Taking opioids or benzodiazepines0.59 (0.36 to 0.82)0.0010.046
Taking ASA or other salicylates0.78 (0.58 to 0.99)0.270.048
Taking anticoagulants or antiplatelet agents0.83 (0.65 to 1)0.120.0059
Taking NSAID0.72 (0.51 to 0.92)0.220.97
Taking insulin or hypoglycemic agent1.00 (1 to 1)<0.00010.011
Taking antihypertensive or diuretic0.92 (0.81 to 1)<0.0001<0.0001
Taking antiarrhythmic0.38 (–0.18 to 0.93)0.260.13
Taking seizure medications0.20 (–0.21 to 0.61)<0.00010.12
Taking chemotherapeutic agents0.000.290.42
Number of prescribing physicians0.42 (0.18 to 0.66)<0.00010.018
History of renal failure0.51 (0.13 to 0.88)0.00070.0049
History of heart failure0.65 (0.28 to 1)0.00020.0003
History of atrial fibrillation0.55 (0.11 to 1)0.0990.12
History of diabetes1.00 (1 to 1)0.0030.049
History of cancer0.51 (0.13 to 0.88)10.78
History of psychiatric problem0.64 (0.42 to 0.85)<0.00010.44
Had a rash in the past 2 weeks0.29 (–0.04 to 0.62)0.0150.0022
Had a Seizure in the past 2 weeks0.49 (–0.11 to 1)<0.00010.38
On ≥2 medications 0.78 (0.62 to 0.93)<0.0001<0.0001
On ≥3 medications 0.84 (0.70 to 0.97)<0.0001<0.0001
Ambulance arrival0.87 (0.73 to 1)<0.00010.0037
Regular general practitioner0.94 (0.82 to 1)0.230.7
Last blood work0.90 (0.82 to 0.98)0.210.092
Last hospitalization in the past month0.76 (0.54 to 0.98)0.00140.0426
Medication changes in the past 28 days0.61 (0.38 to 0.84)<0.0001<0.0001
Compliant with prescription medications0.59 (0.41 to 0.76)<0.00010.42
On antibiotics in the last 7 days0.79 (0.56 to 1)0.00890.0002
Medications blister packed1.00 (1 to 1)0.00620.44
Requires assistance taking medications0.63 (0.30 to 0.96)0.00930.031
Presence of mental health issues0.70 (0.38 to 1)0.00030.27
Confusion0.73 (0.38 to 1)0.0320.47
No medical problems reported0.91 (0.86 to 0.95)<0.00010.0007

We derived one CDR for each outcome measure. The CDR for the primary outcome, the ADE Rule, initially screened patients for low-risk criteria: lack of comorbid conditions and lack of recent antibiotic use (Figure 2). If both low-risk criteria were met, no further workup for ADEs was recommended. In patients failing either or both low-risk criteria, the following five high-risk criteria were used to screen patients further: a recent medication change, arrival by ambulance and high triage acuity, recent admission to the hospital, renal failure, or taking three or more prescription medications. Application of these criteria identified 116 of 120 ADEs, yielding a sensitivity of 96.7% (95% CI = 91.8% to 98.6%), and would have required medication review in 40.8% of patients.

Figure 2.

 ADE rule. Sensitivity = 96.7% (95% CI = 91.8% to 98.6%); specificity = 40.3% (95% CI = 37.7% to 42.9%); proportion who screen CDR+ = 62.8% (95% CI = 60.2% to 65.2%); proportion of ED patients requiring medication review^ = 40.8%. *Derived by restricting to patients with complete data. ^Patients requiring pharmacist referral are patients who meet the inclusion criteria for application of the CDR (65%) and then screen CDR positive (62.8%). ADE = adverse drug event; CDR = clinical decision rule; CTAS = Canadian Triage Acuity Scale.

The CDR for our secondary outcome measure was designed for use in EDs with more limited pharmacist resources, and a higher specificity was targeted (Figure 3). The Adverse Drug Reaction Rule screened patients using the same low-risk criteria first. If both were met, the patient required no further workup. If neither or only one were met, the patient required further screening based on two high-risk criteria: age >80 years and a medication change within 28 days. This rule identified 59 of 65 adverse drug reactions, yielding a sensitivity of 90.8% (95% CI = 81.3% to 95.6%), and required 28.3% of ED patients to be referred for medication review. None of the ADEs or adverse drug reactions missed by either rule caused permanent disability or were fatal. All missed events were associated with good outcomes (Tables 5 and 6).

Figure 3.

 Adverse Drug Reaction Rule. Sensitivity = 90.8% (95% CI = 81.4% to 95.7%); specificity = 59.1% (95% CI = 58.7% to 59.3%); proportion who screen CDR+ = 42.9% (95% CI = 40.5% to 45.4%); proportion of ED patients requiring medication review^ = 27.9%. *Derived by restricting to patients with complete data only. ^Patients requiring pharmacist referral are patients who meet the inclusion criteria for application of the CDR (65% of all ED patients) and then screen CDR positive (42.9% of applicable patients). ADR = adverse drug reaction; CDR = clinical decision rule.

Table 5. 
Events Missed by the ADE Rule
IDSymptomCulprit MedicationCommentTreatment Required
  1. BP = blood pressure; F = female; HCTZ = hydrochlorothiazide; M = male.

2583  Shortness of breathFluticasone27M had shortness of breath after being noncompliant with his inhaled steroids. This patient was treated in the ED and discharged. No alternate cause for the asthma exacerbation was identified.Restarted on prior medications.
2603High BPHCTZ82M with hypertension who stopped HCTZ 3 weeks ago. No alternate cause for his elevated blood pressure was identified.Restarted on previous medications.
Table 6. 
Events Missed by the Adverse Drug Reaction Rule
IDSymptomCulprit MedicationCommentTreatment Required
  1. F = female; HCTZ = hydrochlorothiazide; LOC = level of consciousness; M = male.

336Eye painMoxifloxacin (ophthalmic)24M given moxifloxacin eye drops for viral conjunctivitis. Immediate increase in local irritation, swelling, and redness after using the drops. Diagnosed with a chemical keratitis.Discontinuation of drug.
888PalpitationsYohimbe23M presented with palpitations (resolved on arrival) and anxiety shortly after ingesting yohimbe, an over-the-counter stimulant and aphrodisiac. No alternate cause was identified.Discontinuation of drug.
464RashHydrocortisone dibucaine72F with extensive local contact dermatitis after application of the cream for hemorrhoids.Discontinuation of drug.
649Decreased LOCGlyburide62F with type II diabetes mellitus with decreased appetite from community-acquired pneumonia, continued to take her antihyperglycemic medication without dose adjustment despite decreasing in oral intake. Presented to the ED with glucose of 30 mg/dL.Treated for hypoglycemia in the ED, and her medications adjusted.
970ConstipationOxycodone70M with 3-day history of constipation while on oxycodone for lung cancer. On no bowel protocol.Addition of laxatives to existing regimen.
2424Feeling unwellHCTZ79M presented with difficulty urinating. Incidentally found to be hyponatremic (Na 124) secondary to HCTZ. No alternate cause identified.Discontinuation of drug.

Discussion

To our knowledge, this is the first prospective study to develop CDRs to identify patients at high risk of ADEs and adverse drug reactions in the ED setting. We created decision rules that are clinically sensible, parsimonious, and based on standardized variables, many of which are collected as part of patient care. Either decision rule could easily be integrated into existing ED triage or bedside nursing assessments. Both are sensitive for the detection of clinically significant ADEs and adverse drug reactions and missed no fatal events in the derivation study. They both limited the number of patients requiring medication review, with the Adverse Drug Reaction Rule being less sensitive to facilitate implementation in centers with less pharmacist manpower.

After validation, use of either rule would allow ED nurses or physicians to effectively identify high-risk patients who require further scrutiny of their medication regimen and increase the proportion of patients with ADEs that are identified and treated early in their hospital course and prior to being sent home if they are discharged. The present inaccuracy in ADE identification, along with the political enthusiasm for widespread application of costly and time-consuming medication review interventions such as medication reconciliation, suggests an identified need for this type of risk stratification.9,14,31 Future prospective implementation studies are required to quantify any benefit that might be obtained from this strategy.

To implement evidence-based best care practices to reduce drug-related morbidity, better information about the real-world safety of medications outside of the controlled environment of clinical trials is needed.32–34 Postmarket surveillance systems in North America rely largely on passive reporting of ADEs by health care workers or on documentation of ADEs in medical charts for manual or electronic review.33–39 Given the high rate of missed ADEs in ED patients, it is unlikely that a high proportion of ADEs are captured using these methods.9,14,15,40 This limits the utility of ED and hospital data for surveillance purposes and limits the effectiveness of ADE data capture using trigger tools.41 Implementation of either of the proposed CDRs could enhance the documentation of ADEs in ED and hospital data and have the potential to enhance surveillance and provide more complete data for drug safety and effectiveness evaluation. In addition, this methodology may assist researchers in identifying ADEs prospectively, by providing a more efficient means of identifying the majority of clinically significant events (≥90% based on our data) without the need to have pharmacists and physicians assess all patients for ADEs.

Strengths of this study include adherence to rigorous methodologic standards for the development of CDRs.30 The primary and secondary outcome measures were clearly defined and independently assessed by two raters: one a pharmacist and the other a physician. If any disagreement or uncertainty occurred, an independent committee reviewed and adjudicated the case. Our outcome measures took into account the varying case definitions of ADEs and reactions prevalent in the research literature and in clinical practice, enhancing the utility of our rules. The clinical variables used as predictors were standardized and collected without knowledge of the patient’s outcome. The reproducibility of the variables was assessed by having two physicians and two nurses independently collect data on a subset of patients. The study subjects were selected using a standardized enrollment algorithm to ensure a representative and unbiased sample. In addition, a more sensitive rule was derived based on performance characteristics determined by practicing EPs, and a second rule was derived taking into account the resource constraints of smaller centers with less pharmacist and medical consultant availability.

Limitations

Our study is not without limitations. The sensitivity of our rules was deliberately not set to 100% to enhance specificity and facilitate their uptake into clinical practice. While some clinicians may view this as a disadvantage, no life-threatening ADEs or reactions were missed, and recognition of even 90% of events represents a substantial improvement compared to current practice in which only 51% to 62% of events are attributed to medication use.9,14

The range of severity of the identified ADEs varied from life-threatening consequences of medication use (e.g., saddle pulmonary embolus on a patient on the oral contraceptive pill) to less impressive presentations (e.g., constipation due to narcotics). While all presentations were bothersome to patients and significant from a health resource use perspective because they were associated with ED visits, only a proportion were immediately life-threatening. Because the manifestations of ADEs are heterogeneous, we were unable to define a physiologic cut-point for determining clinical significance and therefore relied on the decision of the treating physician to alter medical management as a consequence of the ADE. Clearly, the physicians felt that to prevent future health resource use and/or deterioration of the patient, treatment of less impressive manifestations of ADEs was warranted. We felt that using this threshold for treatment of the outcome was appropriate, because CDRs are intended to assist with medical decision-making.

Finally, our rules were derived in two Canadian tertiary care centers. Therefore, our rules’ generalizability is limited to similar institutions. In a prospective validation and refinement study, adaptation of the rules to community hospitals and non–tertiary care EDs can be conducted. The rules can also be used as a starting point for deriving and validating rules for other practice settings. Finally, the effectiveness of our rules at improving the identification of ADEs, and their effect on clinical practice and patient outcomes, needs to be assessed prospectively in a new sample of patients.

Conclusions

There is currently substantial variability and inaccuracy in the identification of adverse drug events and adverse drug reactions in ED patients. We have developed clinical decision rules that may enable resource-efficient, systematic screening in this patient population by flagging high-risk patients for medication review by a medication specialist. If prospectively validated, our rules have the potential to standardize and improve the efficiency with which drug-related morbidity is detected and reported in EDs and could streamline the referral process for time-consuming medication review.

Acknowledgments

The authors acknowledge the physicians and nurses working at Vancouver General Hospital and St. Paul’s Hospital emergency departments during the study period. This study would not have been possible without their dedication and support. Particular thanks go out to Jan Buchanan for her diligence and support, as well as Drs. Jim Christenson, Rob Stenstrom, and Riyad B. Abu-Laban for their critical review of the manuscript.

Ancillary