Medical errors are a common cause of morbidity and mortality in a variety of health care settings.1–3 Their importance has been increasingly recognized, as reflected in a recent report by the Institute of Medicine that drew widespread public attention.4,5 It is estimated that in the United States alone, medical errors result in 44,000 to 98,000 unnecessary deaths and approximately 1 million excess injuries each year.4,6 Extremes of age, complex and/or urgent care, and prolonged hospital stay are associated with more errors, but significant errors may also occur in the outpatient setting and for patients of all ages.4,6 Despite the increasing recognition of the importance of this phenomenon, there is limited knowledge of which interventions may be used to effectively reduce the incidence and impact of errors on medical care. There is optimism that both simple behavioral interventions, system approaches, and information technology-based approaches may alter the incidence and consequences of medical errors.5,7 However, much of the data pertaining to medical error have been generated using epidemiologic observational studies or intervention studies with before-after comparisons. Randomized evidence has not been carefully scrutinized.
In the present systematic review, we undertook to retrieve and critically evaluate the available randomized evidence on interventions specifically aimed at reducing medical errors. Given the intricacy of defining what constitutes a medical error and the multifaceted dimensions of this phenomenon, we anticipated that this field of research would pose important study design issues. Therefore, in addition to summarizing the available evidence, the aim of this overview was to evaluate study designs and limitations in order to make recommendations for improving future research in this field. Pertinent nonrandomized studies were also evaluated briefly for comparison and to obtain complementary information.
- Top of page
For the identification of randomized trials, 52 papers were retrieved and reviewed at full length. Of those, 39 were excluded as ineligible (nonrandomized design, improper randomization, or no control arm [n = 14], interrater variability outcome only and/or no medical error outcomes [n = 5], trainees only being involved and fictitious/simulated cases only being considered [n = 1], intervention consisting of new technology for drug delivery or mode of reading a diagnostic test [n = 6], focus only on redundant testing [n = 5], focus only on preventive interventions [n = 8]).
Characteristics of the 13 eligible randomized studies11–23 are summarized in Table 1. These studies addressed very diverse aspects of errors in health care delivery, ranging from medication, prescription, and dosage errors to misdiagnosis/mismanagement and lack of recognition of the seriousness of illness. Errors could be made by the physicians, nurses, pharmacists, patients themselves, or (in the case of sick children) their parents. The interventions were equally diverse, including leaflets, automated systems and computerized reminders, instructive protocols, multidisciplinary approaches, algorithms, pharmacist interventions, ergonomic changes (illumination in the workplace), and performance of aspects of health care by health professionals other than physicians. Outcomes were measured either on the basis of patients or opportunities for errors. Three studies involved more than 1,000 patients and 2 more had more than 1,000 opportunities for error. The settings were extremely diverse, offering a sample of the multifarious nature of current medical care and the mean age of the patient populations varied from 2 weeks to 83 years, when reported.
Table 1. Characteristics Of Randomized Studies
|Study (Year)||Error definition||Intervention||Sample Size||Setting||Mean Age||Quality Components|
| || || || || || ||Mask||Rand||AC||WD|
|Klassen16 (1993)||Missed fracture, dislocation or effusion after trauma||Brand protocol by triage nurse||991 patients||Pediatric emergency room||11 y||+||+(P)||+||+|
|McCarthy18 (1990)||Parent not recognizing illness as being serious||Teaching acute illness observation scales to mothers||369 mothers||Pediatric outpatient||2 w||−||+(P)||−||+|
|Attard17 (1992)||Wrong management (operate or not)||Pain relief with papaveretum for acute abdominal pain||100 patients||Surgical||45 y||+||+(P)||−||−|
|Sakr23 (1999)||Significant clinical management error||Nurse practitioner vs. junior doctor providing care||1,453 patients||Adult emergency room||36 y||−||+(P)||−||+|
|Overhage12 (1997)||Error of omission*||Computerized reminders of corollary orders||2,181 patients||Inpatient medical||54 y||−||+(S)||−||+|
|Owens11 (1990)||Inappropriate drug choice†||Multidisciplinary approach†||436 patients||Acute hospital||83 y||−||+(P)||−||+|
|Ley13 (1976)||Medication error‡||Leaflets (easy, moderate, difficult-to-read, none)||160 patients||Psychiatric outpatient||ND||−||+(P)||−||+|
|Barker14 (1984)||Medication error§||Automated bedside dispensing||1,775 OE||Surgical unit||ND||−||+(W)||−||+|
|McMahon15 (1997)||Dose error from parents¶||Syringe (marked or not) and demonstrated dose||90 patients||Pediatric outpatient||<4 y||−||+(P)||−||+|
|Bates19 (1998)||Serious medication error (nonintercepted)||Team intervention**||4,220 patients||Acute hospital||53 y||−||+(W)||−||−|
|Pereles20 (1996)||Medication error||Self-medication program||107 patients||Inpatient geriatric units||80 y||−||+(P)||−||+|
|Buchanan21 (1991)||Prescription error (content or labeling error)||Illumination in the workplace (three levels)||10,888 OE||Army outpatient||ND||−||+(T)||−||+|
|Leape22 (1999)||Prescription error leading to preventable ADE||Pharmacist participation in rounds||150 patients||Acute hospital||ND||−||+(W)||−||−|
Only 1 randomized study scored positively on all 4 quality items. Masking was implemented in only 2 trials, and only 1 of the 2 provided detailed information on how allocation concealment was ensured. Masking and allocation concealment were usually impractical or even impossible. All studies offered appropriate details on their randomization procedures. The unit of randomization could be the patient (or parent of a sick child), the health care provider, the ward, or even a time unit (in one study, 3 different levels of illumination were randomized to be applied in the workplace for 7 random days each in a sample of 21 days). Ten of the 13 studies offered adequate data on withdrawals and exclusions.
As shown in Table 2, the definition of errors used as outcomes was adequately clear, with the exception of one study which simply stated that a serious error is one which has a potential health risk, without further details. Four studies targeted diagnostic errors, 9 studies targeted medication errors, and 3 also targeted other management errors. In 10 of the 13 studies, the targeted errors were not necessarily clinically serious, and in most cases they were probably not serious at all. Only 2 studies17,19 clearly reported clinically significant harm to the patients; no study recorded any information on deaths. It is unclear whether any deaths due to errors occurred at all in any of the 13 studies, although 17 errors were coded as life-threatening in one study.
Table 2. Definitions of Errors and Methods of Data Collection
|Study (Year)||Error Definition|| ||Clinically Serious Errors||Recorded Clinical Harm to Patient||Deaths Recorded||Data Collection|
| ||Clarity||Type of error|
|Klassen16 (1993)||Yes||D||2/16||No||No||Active data entry per protocol|
|McCarthy18 (1990)||Yes||D||Yes||No||No||Active data entry per protocol|
|Attard17 (1992)||Yes||D||Yes||Yes||No||Chart review?|
|Sakr23 (1999)||Yes||D, Ma||Yes||No||No||Active data entry per protocol|
|Overhage12 (1997)||Yes||M, Ma||Not necessarily, most probably not||No||No||Electronic medical records|
|Owens11 (1990)||Yes||M, Ma||Not necessarily, most probably not||No||No||Chart review|
|Ley13 (1976)||Yes||M||Not necessarily, most probably not||No||No||Pill count|
|Barker14 (1984)||Yes||M||Not necessarily, most probably not||No||No||Direct observation|
|McMahon15 (1997)||Yes||M||Not necessarily, most probably not||No||No||Direct observation|
|Bates19 (1998)||Yes||M||17/134 were life-threatening another 65/134 were serious||Yes||No||Chart review + active data entry|
|Pereles20 (1996)||No||M||Not necessarily, most probably not||No||No||Home visit 40 d postdischarge|
|Buchanan21 (1991)||Yes||M||Not necessarily, most probably not||No||No||Direct observation|
|Leape22 (1999)||Yes||M||Mentions severity grading in the protocol, but no data are provided||No||No||Chart review|
The data collection methods varied across protocols. Several studies used chart reviews or benefited from the electronic medical records they had set up as part of the intervention. Direct observation of health care delivery by an independent investigator was also commonly used.
The main results of the 13 randomized trials are summarized in Table 3. In 9 of the 13 studies, the interventions were found to be effective in reducing error rates. However, even in the 4 cases where the randomized intervention did not reduce error rates significantly, the conclusion of the study authors was (at least in part) favorable for the tested intervention. In one study,16 the intervention (a structured protocol for assessing the need for obtaining a radiograph after lower extremity injury) actually increased the error rate; however, it significantly decreased the emergency department waiting time. In another study, investigators demonstrated that nurse practitioners did not make more errors than junior doctors in evaluating minor injuries in the emergency department; this was an equivalence rather than superiority design.23 In a third study without statistical significance,18 errors were not the primary endpoint of the study — the intervention was superior to control in terms of interrater concordance of the parent with the pediatrician in recognizing the child's severity of illness. Finally, one study19 found no additional benefit from adding a team approach to a computerized physician order entry for reducing medication errors, but in a before-after comparison (in the same report), the computerized physician order entry system already seemed to provide a large benefit.
Table 3. Summarized Results of Randomized Studies
|Study (year)||Error Definition||Bases of Error Count||Errors Made (%)||Comparison Statistics|
| || || ||Intervention||Control||RR||RD||NNT||P Value|
|Klassen16 (1993)||Missed serious injury||Patients||16/493 (3)*||0/498 (0)||—||−3||harm||.001|
|McCarthy18 (1990)||Parent not recognizing illness as being serious||Patients||2/10 (20)||1/10 (10)||2.00||−10||harm||NS|
|Attard17 (1992)||Wrong management (operate or not)||Patients||2/50 (4)||9/50 (18)||0.22||14||7||.051|
|Sakr23 (1999)||Significant clinical error||Patients||65/704 (9)||80/749 (11)||0.86||1||69||<.2|
|Overhage12 (1997)||Error of omission (immediate)||OE||ND (22)||ND (46)||0.47||24||4||<.001|
| ||Error of omission (24 h)||OE||ND (29)||ND (50)||0.58||21||5||<.001|
| ||Error of omission (hospital-stay)||OE||ND (37)||ND (56)||0.66||19||5||<.001|
|Owens11 (1990)||Inappropriate drug choice||Patients||40/221 (18)||73/215 (34)||0.53||16||6||<.001|
|Ley13 (1976)||Medication error (antidepressants)||OE||ND (3) [easy leaflet]||ND (16)||0.17||13||8||<.001|
| || || ||ND (8) [moderate leaflet]|| ||0.50||8||12|| |
| || || ||ND (15)[difficult leaflet]|| ||0.90||2||61|| |
| ||Medication error (tranquillizers)||OE||ND (6) [easy leaflet]||ND (15)||0.39||9||11||<.001|
| || || ||ND (8) [moderate leaflet]|| ||0.53||7||14|| |
| || || ||ND (15) [difficult leaflet]|| ||1.01||0||harm|| |
|Barker14 (1984)||Medication error||OE||96/902 (11)||139/873 (16)||0.67||5||19||<.001|
|McMahon15 (1997)||Dose error from parents||Patients||5/30 (17) [syringe]||19/30 (63)||0.26||47||2||<.001|
| || || ||0/30 (0) [marked syringe]|| ||0.00||63||2|| |
|Bates19 (1998)||Serious medication error||Patients†||80/2173 (4)||54/2047 (3)||1.40||−1||harm||NS|
|Pereles20 (1996)||Medication error||Patients†||14/51 (27)||25/56 (45)||0.61||17||harm||<.001|
|Buchanan21 (1991)||Prescription error||OE||106/4038 (3) [high illumination]||137/3602 (4)||0.69||1||85||<.001|
| || || ||126/3249 (4) [mid illumination]|| ||1.01||−0||harm||NS|
|Leape22 (1999)||Prescription error||Patients||3/75 (4)||8/75 (11)||0.38||7||15||<.001|
In 9 of the 13 randomized studies, the error rates in the control arms of the studies were remarkably high (10% to 63.3%). Given the high control rate and significant treatment effects, sizable risk differences and attractive estimates of numbers needed to treat were calculated in most of the studies (Table 3).
A total of 24 nonrandomized studies were also retrieved and evaluated.14, 19, 24–45 Their characteristics are summarized in Table 4. They included 18 before-after comparisons (comparisons of different time periods), and 6 studies with concurrent controls, of which 2 were pseudo-randomized (patients were rotated in the different arms,38 or a random sample selected for evaluation from the experimental arm with allocation that had been performed sequentially).42 Thirteen of the 24 studies had been published before 1990 (vs only 2 of the 13 randomized trials). The studies dealt usually with medication errors, but some had also management errors as well. Interventions and sample sizes varied widely and are summarized in Table 4. Five studies did not even specify their sample size (typically error rates were reported). Five studies had unclear definitions of what counted as an error. Only 4 studies gave some data on clinically serious errors, but even these did not have serious errors as the main study outcome. Finally, the intervention was almost ubiquitously found to be effective. There were only 3 exceptions: in one study,34 the authors described 14 different interventions, but claimed that only 3 were actually implemented — the failure of implementation was felt responsible for the lack of effect; one other exception occurred in a pseudorandomized study42 which did not demonstrate any improvement in errors reflecting serious drug interactions by using a computerized medication profile; the third exception occurred in a study evaluating an automated medication cart-filing system.43
Table 4. Summary of Nonrandomized Controlled Studies
|Study (Year-Design)||Error Definition||Intervention||Sample Size||Clear Definition||Data on Serious Errors||Intervention Effective|
|Barker28 (1969-BA)||Medication errors||Unit dose distribution system||14,058 OE||Yes||No||Yes|
|Barker14 (1984-BA)||Medication errors||14 interventions||ND||Yes||No||No|
|Bates19 (1998-BA)||Serious medication errors||Computer physician order entry||4,538 patients||Yes||Yes||Yes|
|Bates24 (1999-BA)||Medication errors (excluding missed dose)||Computerized physician order entry with decision support||1,817 patients||Yes||Yes||Yes|
|Borel29 (1995-BA)||Drug-dispensing system errors||Automated drug dispensing||1802 OE||Yes||No||Yes|
|Crooks45 (1967-BA)||Wrong prescription or administration||Single prescription sheet||14,949 doses||No||No||Yes|
|Evans37 (1994-BA)||Allergic/idiosyncratic reactions and severe ADEs||Computerized alerts, standard antibiotic administration, prompt physician notification of ADEs||79,719 patients||Yes||Yes||Yes|
|Evans25 (1998-BA)||Wrong orders for antibiotics||Computerized antibiotic management program||1,681 patients||No||No||Yes|
|Gard44 (1995-BA)||Dosing errors (antibiotics)||Computer-generated protocol||ND||No||No||Yes|
|Hayman32 (1989-BA)||Medication dispensing errors||Computer monitoring + disciplinary actions||ND||No||No||Yes|
|Hill26 (1967-BA)||Wrong prescription or administration||Redesigned drug sheet||1,738 doses||Yes||No||Yes|
|Hynniman40 (1970-CC)||Medication errors||Unit dose distribution system||11,481 doses||Yes||No||Yes|
|Klein43 (1994-BA)||Medication errors||Automated dispensing system for cart filling||7,842 doses||Yes||No||No|
|Koepsell42 (1983-CC)||Drug interaction prescription errors||Computerized medication profile||6,186 patients||Yes||No||No|
|Leape22 (1999-BA)||Ordering errors resulting in preventable ADEs||Pharmacist participation in rounds||150 patients||Yes||Yes||Yes|
|McDonald38 (1977-CC)||Medication errors||Counselling elderly before discharge/memory aids||165 patients||Yes||No||Yes|
|Means41 (1975-CC)||Medication errors (commission)||Computerized unit dose system||2,671 OE||Yes||No||Yes|
|Milliken30 (1990-BA)||Medication discrepancies||Computerized unit dose system||7,802 doses||Yes||No||Yes|
|Morris35 (1989-BA)||Dispensing errors||Participation in clinical trials||50 patients||Yes||No||Yes|
|Raehtz33 (1988-BA)||Calculations and labeling errors||Computer-assisted calculations||509 OE||Yes||No||Yes|
|Ray36 (1995-CC)||Dispensing errors||Automated point-of-use unit drug distribution system||ND||No||No||Yes|
|Schnell27 (1976-BA)||Medication errors||Unit dose distribution system||21,856 OE||Yes||No||Yes|
|Simborg39 (1975-CC)||Medication errors||Automated medication system||2,662 doses||Yes||No||Yes|
|Wilson31 (1997-BA)||Medication occurrences||Computerized records shared by pharmacy and nursing||ND||Yes||No||Yes|
In the comparison of randomized versus nonrandomized studies, nonrandomized reports were more likely to have been published before 1990 (OR, 6.5; 95% confidence intervals [95% CI], 1.2 to 36; P = .035). As compared to nonrandomized reports, those of randomized design were more likely to report data on clinically serious errors (OR, 3.1; 95% CI, 0.7 to 14.7; P = .229), to find that the intervention was not effective in reducing errors (OR, 3.1; 95% CI, 0.7 to 15; P = .213), and to specify the sample size (OR, undefined; P = .140), but these associations were far from being significant.
- Top of page
Medical errors recently have drawn large publicity and the identification of simple measures to reduce their impact has been considered a public priority.4 This overview shows the complexity involved in studying interventions aimed to minimize errors in health care delivery. The definition of medical errors is difficult to set within clear boundaries. Medical errors are multifaceted and may involve the diagnosis, management, ordering, carrying out of orders, prescriptions and other medical care by physicians, nurses, pharmacists, other staff, the patients themselves, or the parents and custodians responsible for their care. The heterogeneity of the studies retrieved in this overview shows these diverse components in the fabric of health care where errors may arise.
In this overview, we used a very strict definition of errors with many exclusion criteria in an effort to bring the topic into sharp focus. We acknowledge that we may have missed some studies which are sometimes difficult to even code for database purposes. Other studies such as those assessing interventions to reduce the omission of preventive measures or even those assessing interventions to reduce redundant test ordering or improve the behavior and reaction time of physicians may be perceived as interventions aiming to reduce errors in a broader sense. There is an even more extensive literature on these topics, some of which has been already reviewed previously.8
Given the observed high frequency of errors, there is a dearth of randomized trial data about ways to reduce medical errors. Even with a less comprehensive search, we were able to identify a larger amount of nonrandomized evidence. Compared with the randomized trials, the observational studies were older, although several recent ones were also identified; and they had an equally high, if not even higher, rate of positive findings, raising concerns about publication bias and selection biases in targeting study settings and patient populations. In many cases, it is conceivable that when large improvements are seen in nonrandomized studies, randomized trials may never be performed, such as in the case of unit dose distribution systems or, more recently, several computerized interventions. Randomized trials may not always be applicable to all settings and questions46 and observational studies may not always square with the results of randomized trials.47 Observational studies have the advantage of studying effectiveness in real practice settings, while randomized trials may be prone to study efficacy under more controlled circumstances with less generalizability. Observational evidence may yield useful insights and should not be discarded.
However, there are several cases where randomization would have been easy to implement without affecting the generalizability or the difficulty of conducting a study. Studies using concurrent controls usually are easy to transform into randomized designs. Finally, before–after comparisons are easier to implement than randomized trials, but one should caution that the use of historical controls may sometimes inflate the magnitude of the treatment effect. Before-after comparisons may be particularly biased, if they extend over a long period of time during which several other changes and interventions that may affect the error rate are implemented (either systematically or sporadically) in the health system where the study is conducted. Such serendipitous changes may result in large reductions of the error rate and they may often be unknown or difficult to model and adjust appropriately. For example, a Hawthorne effect may be observed where physicians make fewer errors over time, especially after a computerized system has been installed, regardless of the merits of the system itself.
Many other important studies in the field were excluded, because they had no control comparison at all. For example, one otherwise well-designed study implemented a computer-alert system to prevent injury from adverse drug events.48 The outcome was the number of true-positive alerts. Measuring such parameters in uncontrolled studies is useful for assessing the performance of a system. However, comparative outcomes (vs no intervention or a standard intervention) on hard endpoints, e.g., deaths or serious morbidity due to iatrogenic errors, would prove indispensable. Such outcomes need to be studied with the gold standard of randomized trials, especially if the proposed interventions are not devoid of cost.
The heterogeneity of the retrieved studies precludes any attempt at their quantitative synthesis. However, it is interesting that the majority of both randomized and nonrandomized studies give positive results for reducing medical errors, and practically almost all of them show that some or all of the interventions under study are worthwhile adopting, based on the results for one or more outcomes. The magnitude of the reported treatment effects is often very large. This probably means that there is large room for improvements in controlling medical errors, often with relatively simple interventions. Regardless of the magnitude of the treatment effects, the magnitude of the error rates in the control groups is also remarkably high, ranging from 10% to 63% in 9 of the 13 studies. These high rates are probably strong evidence for the high incidence of medical errors.
Alternately, it is also possible that biases may be operating in this field of research. Such biases could stem either from publication lag and publication bias.49,50 or from the difficulty of assessing outcomes objectively in studies where masking and allocation concealment are difficult to achieve.9 Theoretically, this is a field of research where a study with negative results (e.g., when a behavioral or computerized intervention does not affect the error rate) may be difficult to publish. However, given that the large majority of studies, both randomized and nonrandomized, yield positive results, the evidence published to date gives limited insight on why error prevention interventions may sometimes fail rather than succeed. Studies explicitly designed to study this aspect should be encouraged. The available data suggest that even with the large error reductions achieved, there are still large rates of errors even in the interventional groups. Reasons for these failures require better study.
Some other methodologic limitations of the available randomized evidence on medical errors need to be discussed. First, many of the tested interventions may be difficult to generalize and/or their effectiveness may depend on familiarity of and training on their principles. Studies in acute hospitals were sometimes conducted in renowned tertiary care, university-affiliated medical centers in the United States; it is unknown whether the same favorable results would be obtained in other hospitals or in other parts of the world. Furthermore, many of the prior trials have dealt with specialized populations such as children and their parents, the elderly, acute care, or specific outpatient settings. Their findings may not be generalizable to different populations. Furthermore, many involved cognitive interventions (reminders, educational tools, etc.) and their effectiveness may vary depending on the rigor and intensity of their application.
Some studies used less than optimal randomization methods, in ways that the allocation of the patients to each of the compared arms could have been manipulated. Almost all of the included studies were unmasked, although this was usually fully justified given the nature of the studied observations. The unit of randomization in these studies must be carefully selected. In some cases, using the patient as the unit of randomization may be inappropriate: skills acquired while managing intervention patients may be applied to the care of control patients.
These limitations being acknowledged, clearly all the identified pertinent studies required substantial ingenuity for their design, since studying medical errors requires innovation both in the way interventions are applied and outcomes are assessed. Several of the studies included more than 1,000 patients, and the results are probably valid. However, before the proposed interventions are widely adopted, the cost-effectiveness and effects on quality of life should also be considered alongside with their effects on error counts. Variability across the trials may make generalization difficult. While it is easy to agree that preventing errors is desirable, specific interventions must be evaluated according to their own merits with respect to effectiveness and cost-effectiveness in specific health care settings.
Finally, all errors are not equal, and their impact may vary tremendously. For example, although medication errors are strongly linked to adverse drug events,51 most medication or prescription errors will not result in any harm. Many studies on medical errors have given limited information of the seriousness of the captured errors. Even when outcomes were defined to include only “serious” errors, the seriousness may have varied substantially from study to study and even within the same study different “serious” errors may have had different consequences for patients. We found a dearth of data pertaining to actual clinical harm to the patient and no study focused on patient deaths. Many studies assessed parameters that almost certainly would not have resulted in death. Although less serious errors that affect many patients and that may expend extensive resources are definitely worth studying, we believe that studies incorporating hard patient outcomes, such as iatrogenic mortality and serious iatrogenic illness, should be encouraged.
In order to perform such studies, large numbers of patients would have to be included, since the event rates are likely to be low. Given the fact that errors occur at various steps in the sequence of medical care and could involve various care providers (e.g., physicians, pharmacists, nurses) or even the patients themselves, observational studies may be used to provide insight on which steps are more error-prone. These steps could then be the target for probing the efficacy of interventions by randomized trials. Still, in order to design such studies, we have to first recognize that medical errors, including serious ones, should be subjected to study from the perspective of a systematic approach and become the target of large experimental investigation, rather than be silenced for legal purposes.