Presented, in part, at the Air Medical Transport Conference, Tampa, FL, September 18, 2007; and the Halifax 7: The Canadian Healthcare Safety Symposium, Ottawa, Ontario, Canada, October 11, 2007. Awarded Best Scientific Abstract at the Halifax 7: The Canadian Healthcare Safety Symposium, Ottawa, Ontario, Canada, October 13, 2007.
Epidemiology of Adverse Events in Air Medical Transport
Version of Record online: 10 SEP 2008
© 2008 by the Society for Academic Emergency Medicine
Academic Emergency Medicine
Volume 15, Issue 10, pages 923–931, October 2008
How to Cite
MacDonald, R. D., Banks, B. A. and Morrison, M. (2008), Epidemiology of Adverse Events in Air Medical Transport. Academic Emergency Medicine, 15: 923–931. doi: 10.1111/j.1553-2712.2008.00241.x
Dr. MacDonald is the recipient of a grant from the Foundation for Air Medical Research and Education and its “Vision Zero” initiative in support of this research.
- Issue online: 1 OCT 2008
- Version of Record online: 10 SEP 2008
- Received May 15, 2008; revision received June 30, 2008; accepted July 7, 2008.
- patient transfer;
- air ambulance;
- medical error;
Objectives: This observational study determined frequency and describes all-cause adverse event epidemiology in a large air medical transport system.
Methods: Records of a mandatory reporting system were reviewed and a data set containing all of the patient care records was searched to identify aviation- and non–aviation-related adverse events. Two reviewers independently identified adverse events and categorized them using an established taxonomy. Descriptive statistics were used to report adverse events, with frequency calculated per 1,000 flights and 1,000 hours flown.
Results: Between January 1, 2002, and June 30, 2005, there were 1,447 reports, of which 598 included an adverse event. Case-finding identified an additional 125. A complete report was available in 680 of 723 (94.1%) events. There were 58,956 flights and 103,632 hours flown during the study period, for a rate of 11.53 adverse events per 1,000 flights (95% CI = 10.7 to 12.4 adverse events) or 6.56 per 1,000 hours flown (95% CI = 6.1 to 7.1 adverse events). The frequencies of events by category were as follows: communication (229; 33.7%), transport vehicle (143; 21.0%), medical equipment (88; 12.9%), patient management (77; 11.4%), clinical performance (68; 10.0%), weather (30; 4.4%), unclassified (24; 3.5%), and patient factors causing death (21; 3.1%). There was possible patient harm in 117 events.
Conclusions: Air medical transport is associated with a low incidence of adverse events and possible patient harm. Communication problems were the most common cause of an event. Determining event epidemiology is necessary to identify modifiable factors, propose solutions to decrease the adverse events, and direct future efforts to improve safety.
The 1999 Institute of Medicine (IOM) report “To Err is Human”1 emphasizes the need for increased attention to patient safety in health care. The report identified the emergency department (ED) as the most error-prone environment within the hospital. The report, however, does not discuss the emergency medical services (EMS) or air medical transport. In many cases, air medical transport systems provide the same advanced care to patients as an ED, but do so in a less well-controlled environment with no immediately available medical backup.
A key step to reduce errors in health care is to obtain better information about the numbers and types of errors that occur.2 Hospitals analyze patterns of error to improve common practices or develop better ones, but similar effort to analyze errors in the EMS or air medical transport are not widespread. This knowledge gap was identified in 2001, when the National Highway Traffic Safety Administration (NHTSA) sponsored a roundtable to discuss EMS error reduction.3 Participants highlighted the lack of information on errors in EMS and the need for research on errors in this setting.
When compared to the inpatient setting, safety in air medical transport is more complex due to the concurrent domains of aviation safety and patient safety. Accidents involving EMS aircraft may be related to a variety of factors, including pilot error, weather, mechanical failure, and obstacle strikes. Loss of EMS aircraft are often high profile,4–8 and their investigation falls under the jurisdiction of federal agencies9 using a standardized reporting system. Adverse events in air medical transport, however, can be related to either delivery of patient care or aircraft operation or the interaction between these two factors. There is no comparable, unified, or comprehensive investigation or reporting system for adverse events occurring in air medical transport.
To mitigate risk, delineate root causes, and improve patient safety in the air medical transport setting, it is necessary to first identify all aviation and nonaviation causes of adverse events. Adverse events represent a broad group of unfavorable and potentially harmful occurrences, including errors. Identifying and analyzing all adverse events is necessary before implementing a systems-based approach to modify common practices or develop new ones that improve safety. By examining all adverse events, one can identify and modify potentially harmful practices and improve patient safety.
This study has two objectives. The primary objective is to determine the frequency of all causes of adverse events in a large air medical transport program. The secondary objective is to describe the epidemiology of adverse events in this setting using established categorization methods.
We conducted a retrospective observational analysis of routinely collected data to determine the frequency of all causes of adverse events and describe the epidemiology of adverse events in this setting using established categorization methods. The study was approved by the Research Ethics Board of Sunnybrook Health Sciences Centre and was conducted in accordance with the confidentiality policies and procedures established by Ornge Transport Medicine and Ontario’s Ministry of Health and Long-Term Care.
Study Setting and Population
Ontario is a large Canadian province (approximately 1.1 million km2 or 424,600 miles2) with a mix of urban, suburban, rural, and remote areas. The health care system is publicly funded and serves a population of approximately 12.5 million. Ornge Transport Medicine is the publicly funded air medical transport system providing all air medical patient transfers in Ontario. Ornge is North America’s single largest air medical transport provider, with 33 rotor- and fixed-wing aircraft at 26 bases, carrying out approximately 17,000 patient transports annually.
Ornge operates its own dedicated communication and dispatch center and utilizes a dedicated medical decision algorithm to determine patient acuity and call priority. The center also has a medical control physician dedicated to providing full-time medical control without any other competing clinical duties. Medical control physicians are board certified in emergency medicine (EM) and have an extensive EMS or transport medicine background. Physicians also provide indirect medical control, providing medical oversight to the service’s quality care, investigation, pharmacy, therapeutics, and education committees.
Ornge aircraft operate as either advanced or basic life support aircraft, with flight paramedics as the sole care providers during transport. The scope of practice for the highest crew designation, critical care, includes management of patients who require intubation, mechanical ventilation, use of inotropes or vasopressors, administration of fibrinolytic agents, and maintenance of intraaortic balloon-pump therapy10 without additional medical personnel. The scopes of practice are consistent with the National Occupational Competency Profile (NOCP).11 Flight paramedics provide care using standing orders and medical directives developed by the Ornge Medical Advisory Committee and contact medical control physicians when the care exceeds these orders or directives.
Since 2002, the Ornge Quality Management Department designed, implemented, and has maintained a publicly accessible Web-based portal for stakeholders to submit inquiries, compliments and complaints, and report adverse events. The application, referred to as the Decision Support Application (DSA),12 collects records in a standardized manner, in keeping with a recommendation from the IOM.13 Paramedics are required to report any adverse event resulting from medical care or aircraft operation using the DSA. A dedicated investigations officer, operations supervisor, and medical director review all reports. This study includes all reports and records of adverse events from January 1, 2002, to June 30, 2005.
We obtained all DSA records received during the study period and included in this study any case where either the DSA report or review of patient record data identified an adverse event as defined in Table 1. If more than one possible adverse event took place during a single flight or patient encounter, we considered each event as a discrete, separate event. If multiple reports from different stakeholders or sources described the same event on the same flight or with the same patient, we considered it a single event. Reports were excluded if they did not contain an adverse event.
|An adverse event is an unfavorable and potentially harmful occurrence during or after the course of patient care. Adverse events are due to circumstances that may or may not be preventable. An error is a preventable adverse event|
|A significant adverse event is an occurrence that results in serious, undesirable, or unexpected patient outcome that has the potential to negatively impact the patient’s health and quality of life. Examples include death, loss of function, or change in patient condition due to equipment malfunction or administration of an inappropriate drug or drug dose. Administration of any inappropriate drug or drug dose is considered a significant adverse event|
|A nonsignificant adverse event is an occurrence that does not impact patient outcome, now or in the foreseeable future. Examples include a fall or equipment malfunction in which no injury or clinical effect occurred|
To ensure complete case-finding, we also queried the patient care report database that contains administrative, aviation, and patient care information manually abstracted from all air ambulance patient care reports (“run sheets”). Report completion is mandatory and occurs immediately after patient transfer, and data are abstracted and entered by quality management personnel as part of the program’s administrative process. The query identified any flight or patient encounter where paramedics included a report of any unusual occurrence, including patient death, aircraft mechanical problem, or weather event that took place during transport (Table 2).
|Delay and special occurrence codes|
|Ambulance delay to scene|
|Ambulance delay at scene|
|Ambulance delay to destination|
|Ambulance delay at destination|
|Delay returning to aircraft|
|Delay to patient contact|
|Vehicle delay—land en route to destination|
|Vehicle delay—land en route to aircraft|
|Delay—due to paramedic|
|ED not ready after being notified|
|Pronouncement of death|
|Communication and patching|
|Failure—obtaining patch (medical control) due to phone/radio|
|Delay—obtaining patch (medical control) due to phone/radio|
Patient demographics, sending and receiving hospital names, aviation details, chronological information, special event codes (Table 2), and the narrative text description of each potential case were exported to a database (Microsoft Access, Version 2002, Microsoft Corp., Redmond, WA) for review. We also retrieved the original patient care reports for each potential case. Two authors (RDM, BAB) independently reviewed all retrieved materials to identify adverse events.
We categorized each identified adverse event in a blinded manner using a taxonomy of event impact and event type (Figure 1) adopted from the patient safety event taxonomy developed by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO).14 We resolved disagreements regarding presence of an adverse event in categorization or in patient impact by consensus, with arbitration by an additional author (MM) as necessary. Events were categorized as “possible or actual harm” when the report or patient record documented the presence of an event that would injure the patient or actual documentation of an injury due to an event. Figure 2 outlines the identification and classification process.
The primary outcome is the frequency of adverse events. The secondary outcome is the categorization of adverse events using established methods.
We categorized adverse events as discrete, unordered variables and reported using descriptive statistics. We computed the interreviewer measure of agreement for the presence and categorization of adverse events and presence of possible patient harm using the kappa statistic (κ) for each JCAHO major classification level and calculated the frequency of adverse events per 1,000 flights and 1,000 hours flown. This study includes all reports and records of adverse event during the study period.
Between January 1, 2002, and June 30, 2005, there were 1,447 Web-based entries, of which 598 included report of an adverse event. Case-finding in the patient care report database identified an additional 125 cases, for a total of 723 adverse events. The final study cohort included 680 of 723 (94.1%) for which a complete report was available for review. There were 58,956 flights and 103,632 hours flown during the study period. Table 3 summarizes the patient demographic and flight characteristics for this study period.
|Patient gender||Male: 57.5%|
|Age (years)||Mean: 43.9|
|IQR: 22, 64|
|Diagnostic category||Trauma: 18.3%|
|Scheduled appointment/ repatriation: 7.7%|
|Vehicle type||Fixed wing: 366 (53.8%)|
|Rotor wing: 271 (39.9%)|
|Land: 4 (0.6%)|
|No vehicle: 39 (5.7%)|
|Mean time per flight||Fixed wing: 3.06 hours|
|Rotor wing: 1.12 hours|
|Flights by time of day*||Day (0700–1900 hours): 82.3%|
|Night (1901–0659 hours: 17.7%|
|Event occurred after patient contact†||Yes: 553 (81.3%)|
|No: 127 (18.7%)|
The rate of adverse events was 11.53 per 1,000 flights (95% confidence interval [CI] = 10.7 to 12.4), or 6.56 per 1,000 hours flown (95% CI = 6.1 to 7.1). This represents an event rate for aviation, nonaviation, and unclassified categories of 2.93, 8.19, and 0.41 per 1,000 flights, respectively. Table 4 summarizes the frequency and categorization of adverse events, including presence of possible patient harm, by aviation- and nonaviation factors. Figure 3 breaks down event type within each category. The reviewers agreed on the presence of an adverse event in 635 of 680 cases (k = 0.84; 95% CI = 0.82 to 0.87) and on the major JCAHO classification level for each adverse event in 472 of 680 cases (κ = 0.66; 95% CI = 0.62 to 0.70).
|Number (% of Total Events)||Number with Possible Harm (% of Category)|
|Transport vehicle and associated equipment||143||12|
|Medical equipment (malfunction/availability)||88||7|
|Patient factors causing death||21||21|
The adverse event resulted in possible patient harm in 117 of 680 cases (1.99 per 1,000 flights). This represents a rate of possible harm due to aviation, nonaviation, and unclassified categories in 0.31, 1.58, and 0.10 per 1,000 flights, respectively. The reviewers agreed on presence of possible patient harm due to the event in 479 of 680 cases (κ = 0.49; 95% CI = 0.44 to 0.55). Figure 4 breaks down event type within each category.
There were 21 patient deaths during the study period, all of which were attributed to the underlying patient condition. There was one aircraft accident during the study period. The fixed-wing aircraft accident, occurring on takeoff, resulted in no harm to the patient or crew.
To our knowledge, this observational study is the first to describe all-cause adverse events in a large population-based transport medicine setting. There were 11.53 adverse events per 1,000 flights, or 6.56 per 1,000 hours flown. While there are no comparable figures in the air medical transport setting, 1.15% (11.53 per 1,000) is comparable to the range of 1.24% to 5.3% for patients in the ED15–18 and less than 3.7% to 11.2% for hospital admissions19,20 and 20.2% for patients in intensive care.21 Absolute numbers of adverse events during air medical transport may be lower than in-hospital settings due to the short patient contact time. However, rates of patient-specific adverse events per unit-time during transport22 are comparable to those in anesthesia and other acute-care settings.23,24
We also identified potential harm to the patient in 1.99 cases per 1,000 flights, or 17.3% of all adverse events. This is similar to 14.7% of all adverse events resulting in permanent disability or death in the ED.20 Mortality in this study (0.04%) was also comparable to an ED-specific mortality rate of 0.06%,20 but less than a range of 0.5% to 0.79% in the hospital inpatient setting.16,17,19
The interobserver agreement for event categorization of 0.66 is substantial25 and comparable to agreements of event type categorization (κ = 0.51–0.64)26,27 in classification of events by trained reviewers in the health care setting. The interobserver agreement for presence of possible patient harm of 0.49 is moderate and comparable to agreements of seriousness of event (κ = 0.50–0.62)26,28 in other studies. Our data are also consistent with the literature in that judgment on event categorization appears to be more reliable than determining degree of patient harm due to the event. Event categorization and determination of harm remain subjective and will constrain interpretation of this type of data.
We found issues in communication between stakeholders, such as the sending facility, communication center, flight paramedics, pilots, medical control physician, and receiving facility, are associated with one-third of all identified adverse events. Communication problems are consistently identified as a leading cause of system breakdown in patient care1,29,30 and a leading root cause of preventable morbidity and mortality.31 Communication in air medical transport is comparable to or possibly more complex than hospital settings because communication takes place between multiple stakeholders at different locations, many of whom do not communicate directly with one another. Our data suggest a lack of tight communication in transport medicine. Our organization is utilizing this data to scrutinize communication transactions as potentially correctable causes of events and is examining use of technology to exchange patient information between communication center and transport crews to improve reliability.
Other domains that depend on tightly coupled, highly reliable communication, such as the aviation industry, utilize crew resource management (CRM) strategies to improve reliability. CRM recognizes that human factors, not weather or equipment, are responsible for the majority of accidents and incidents in aviation.32 Specifically structured team training was developed to improve performance and reliability, with effective teams engaged in advance planning and better performance.33 This resulted in highly reliable communications. The application of CRM principles to the health care setting appears promising,34,35 but data on its impact on patient morbidity and mortality are not yet available.36
In our study, 145 (21.3%) adverse events were due to problems with either clinical performance or patient management decision-making. Of these, 36 of 145 (24.8%) had possible patient harm, the highest proportion of potential harm. People are prone to err37 and will make errors when working long hours and performing complex tasks.38 Cognitive and psychomotor skills do not guarantee excellent, event-free patient care.39 High-reliability organizations (HROs), such as nuclear power plants and the aviation industry, are defined by their capacity to perform high-hazard, high-risk operations in a nearly error-free manner.40 Redundancy, training, and an integrated safety culture have successfully improved the margin of safety and prevented errors in HROs. Although health care organizations and medical practice differ from these organizations,41 these approaches may help prevent errors and improve patient safety.42
We identified that malfunction of an aircraft or its associated aviation equipment occurred in 143 flights. Events due to malfunction of medical equipment occur in patient transfer,43–45 and data from this study are comparable. Our organization is collaborating with human factors experts to determine what role equipment and aircraft interior design may have on this event category. Events due to aviation factors leading to a crash or death, however, are uncommon.46,47 While the overall figure is very low in proportion to the overall number of flights (0.24%), this represents a potential risk where failure could be catastrophic.
The IOM’s goal of reducing errors requires several strategies to mitigate risk and reduce harm. The first step is to provide detailed information on the number and types of events in a particular setting. This study is the first to provide this information in the air medical transport setting. The results in this setting are comparable to other health care settings.
Key barriers in making progress in patient safety include the identification of problems and ability to measure progress and demonstrate improvements in adverse events.48 A number of setting- and industry-specific event taxonomies exist to systematically collect, report, and compare event data. Developing a taxonomy specific to air medical transport could permit more detailed analysis of all types of events in this setting and provide a tool to measure progress and improvement in mitigating adverse events.
This study categorized events retrospectively using categorization methods designed for hospital-based care. A transport-specific taxonomy and categorization algorithm may improve the categorization validity, but one specific to transport medicine does not exist. In addition, routine case-finding for events in all patient encounters in real time may yield a greater number than were obtained with self-reporting methods. If this were the case, the figures presented in this study would be an underestimate of such events.
Paramedic personnel typically underrecognize and underreport medical errors,49,50 and there is a possibility that adverse events were not reported. Patient care reports are routinely audited for adverse event, specific occurrence, and procedure codes and cross-referenced to ensure paramedic compliance with reporting requirements. This study identified 125 additional events using case-finding methods, of which 109 (87.2%) were aviation- or weather-related events and not directly related to patient care. It is difficult to determine the true extent of underreporting and the influence this may have on the results of our study.
An undetected adverse event may occur while the patient is undergoing transport, but the effect of this event manifests itself only after transfer of care to staff at the receiving facility. Staff may not attribute this event to care rendered by air medical crews and therefore would not report the event. This could result in a bias a lower adverse event rate due to underreporting. There is no method to identify such events with the current reporting system. A prospective reporting system that routinely obtains records of adverse events occurring in hospital shortly after air medical transfer may help identification of posttransfer adverse events attributable to the transfer itself.
A land ambulance is necessary to transport patients from fixed wing aircraft to and from hospitals. While the air medical crews accompany and provide care to patients during transport between hospitals and airports, the time spent during this land portion of air medical transport is not reflected in the total hours flown. While this is a limitation, the additional time spent providing care en route to and from airports, and not in flight, results in an overestimation of the event rate in this study.
Weather is considered as a category of adverse event, but available data were limited to weather-related events that took place after a flight commenced. Examples of this include a missed runway or helipad approach, diversion to an alternate landing site, or unserviceable aircraft due to adverse weather conditions. The available data did not include weather-related events that prevented a flight from commencing, where a flight record would not be generated. A prospective, unified reporting system of flights declined or delayed due to weather would provide all-inclusive information regarding weather as an adverse event.
Finally, the aviation safety record of this transport medicine agency is excellent. Ornge’s rotor wing aircraft fleet has over 100,000 accident-free flying hours in EMS operations,51 and the accident rate in the fixed-wing fleet is exceedingly low. Study results may not be translated to other agencies that do not mirror this aviation safety record.
We found a low incidence of adverse events and potential patient harm associated with air medical transport in our system. The epidemiology is similar, but the rate of adverse event is less than inpatient settings. Determining event epidemiology is necessary to direct future efforts to improve patient safety in this setting. Future analysis of events requires an industry-specific taxonomy to identify modifiable factors, propose solutions to decrease the frequency of adverse events, and direct future efforts to improve safety.
The authors thank flight paramedics at Ornge Transport Medicine and Ornge’s external stakeholder for reporting adverse events and enabling this investigation to occur. The authors also thank Dr. Chris Mazza, President and Chief Executive Officer, and the Board of Directors of Ornge Transport Medicine for their support of this and other ongoing research initiatives pertinent to safety in transport medicine.
- 1To Err Is Human: Building a Safer Healthcare System. Washington, DC: National Academies Press, 1999., , .
- 3National Highway Traffic Safety Administration. Patient Safety in Emergency Medical Services. Washington, DC: NHTSA, 2002.
- 4Air ambulance crashes spur safety reviews. USA Today. 2005 Jan 14; Sect A:3(col.3)., .
- 5Surge in crashes scars air ambulance industry. USA Today. 2005 Jul 17. Available at: http://www.usatoday.com/news/nation/2005-07-17-air-ambulance-crashes_x.htm. Accessed Jul 17, 2005., .
- 9National Transportation Safety Board. Safety Study – Commercial Emergency Medical Services Helicopter Operations. January 1988; PB88-017001: NTSB/SS-88/01.
- 11Paramedic Association of Canada. National Occupational Competency Profiles (NOCP). Available at: http://www.paramedic.ca/Content.aspx?ContentID=4&ContentTypeID=2. Accessed August 26, 2008.
- 12Ornge Transport Medicine. Decision Support Application. Available at: http://www.ornge.ca/newinquiries.html. Accessed Mar 18, 2007.
- 13Quality Interagency Coordination Task Force. Doing What Counts for Patient Safety: Federal Actions to Reduce Medical Errors and Their Impact. Report to the President, February 2000. Available at: http://www.quic.gov/report. Accessed Jun 25, 2005.
- 20Adverse events in New Zealand public hospitals I: occurrence and impact. N Z Med J. 2002; 115:U271., , , , , .
- 30Errors in general practice: development of an error classification and pilot study of a method for detecting errors. Qual Saf Health Care. 2003; 12:443–7.
- 31Joint Commission on Accreditation of Healthcare Organizations. Preventing infant death and injury during delivery. Sentinel Event Alert. 2004; 30:1–3.
- 36Will airline safety models work in medicine. in: RosenthalMM, SutcliffeKM (eds). Medical Errors: What Do We Know? What Do We Do? San Francisco, CA: Jossey-Bass, 2002, pp 217–34., .
- 37Absent-minded? The psychology of mental lapses and everyday errors. Englewood Cliffs, NJ: Prentice Hall, 1982., .
- 39Joint Commission on Accreditation of Healthcare Organizations. In sentinel events: evaluating cause and planning improvement. Oakbook, IL: JCAHO, 1988.
- 42The power of health care teams: strategies for success. Oakbrook, IL: JCAHO, 1997..
- 51Canadian Helicopters. EMS Flight Safety. April 2003. Available at: http://www.canadianhelicopters.ca/frame/ems_2.html. Accessed Dec 17, 2006.