SEARCH

SEARCH BY CITATION

Keywords:

  • CPOE;
  • computers in medicine;
  • patient safety;
  • errors in medicine

Abstract

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. CONCLUSIONS
  7. Acknowledgements
  8. REFERENCES

BACKGROUND

Computerized physician order entry (CPOE) is a widely advocated patient safety intervention, yet little is known about its adoption by attending physicians or community hospitals.

METHODS

We calculated the order entry rates of attending physicians at 2 hospitals by measuring the number of orders entered directly and dividing this by the sum of orders entered directly and those written by hand. These findings were paired with the results of a survey that assessed attitudes concerning the impact of CPOE on personal efficiency, quality of care, and patient safety.

RESULTS

Three hundred and fifty-six (71%) of the 502 surveys were returned by physicians, whose median order entry rate was 66%. Forty-two percent of respondents placed at least 80% of their orders electronically (high use), 26% placed 21%-79% of their orders electronically (intermediate use), and 32% placed 20% or less of their orders electronically (low use). Sex, years since medical school graduation, years in practice at the study institution, and use of computers in the outpatient arena were not meaningfully different among the 3 groups. However, use of the system to place orders varied by specialty, and those with intermediate or high use of the system were more likely than low users to have used CPOE during training and to be regular users of computers for personal activities. These physicians were more likely to believe that CPOE enabled orders to be placed efficiently, that directly entered orders were carried out more rapidly, and that such orders were associated with fewer errors.

CONCLUSIONS

The adoption of CPOE by attending physicians at community hospitals varies widely. In addition to purchasing systems that support physician work flow, hospitals intent on successfully implementing CPOE should emphasize the benefits in safety and quality of this new technology. Journal of Hospital Medicine 2006;1:221–230. © 2006 Society of Hospital Medicine.

It is widely acknowledged that the U.S. health care system is plagued by error and inefficiency and that these factors contribute to as many as 44,000-98,000 deaths each year in U.S. hospitals. In To Err Is Human: Building a Safer Health System, the Institute of Medicine1 outlined the critical role that information technology can play in improving patient safety and highlighted computerized physician order entry (CPOE) systems for their potential to reduce the frequency of medication errors and to improve the quality of medical care.

Computerized physician order entry systems are specialized software applications that allow physicians to place orders directly into a computer. This process has a number of potential advantages over traditional handwritten ordering, including the ability to structure the ordering process to ensure the completeness of individual orders, to provide clinical decision support through diagnosis-based order sets, and to automatically check orders for potential drug–allergy, drug–drug, and drug–food interactions.2 Finally, entering orders directly into a computer eliminates the problem of transcription-related errors that stem from the difficulty of interpreting handwriting. In clinical trials, the introduction of CPOE has been shown to reduce the frequency of medication errors, to improve the use of preventive services, and to reduce costs.3–6 Recognition of the benefits of these systems has not been confined to the medical community. The Leapfrog Organization, a coalition of large businesses in the United States, has chosen CPOE as one of its 3 initial safety “leaps” and has established a threshold that 70% of medication orders should be entered directly by physicians.7

Although the benefits of CPOE systems are widely recognized, few hospitals have implemented these systems successfully.8, 9 Those that have, have often developed the applications internally, and many have relied on house staff to do most or all of the actual ordering.10 However, most hospitals do not have the expertise for internal development and instead rely on commercially available products. Moreover, most patients hospitalized in the United States are cared for by attending physicians working without the assistance of house staff.11 In light of the importance of successfully implementing CPOE systems in such settings, we assessed the adoption of CPOE by attending physicians at 2 community hospitals where its use was voluntary and examined the characteristics and attitudes associated with use of the system to place orders.

METHODS

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. CONCLUSIONS
  7. Acknowledgements
  8. REFERENCES

Setting and Participants

Baystate Medical Center is a 600-bed teaching hospital in Springfield, Massachusetts, where approximately 50% of patients are cared for with the assistance of house staff. Franklin Medical Center is a 125-bed community hospital in rural Greenfield, Massachusetts, and is not a house staff training site. Medical staff membership at the 2 hospitals is largely voluntary. Both institutions share a vendor-supplied computerized order entry system that was implemented in the early 1990s (E7000, Eclipsys Corporation, Boca Raton, FL). The system provides a structured format for the creation of medication, laboratory, and radiology orders and contains thousands of preconstructed medication order sentences and hundreds of order sets designed to standardize ordering for common diagnoses and procedures. Pharmacists are alerted of potential drug–allergy and drug–drug interactions and use clinical judgment about whether to communicate this information to the physician. Although the house staff at Baystate Medical Center is mandated to place orders in the system, attending physicians have no such requirement at either institution. Access to the system is provided though the many fixed workstations located on nursing units, in operating rooms, and in the health sciences library. On a typical medical-surgical patient care unit most computers are behind the nurses' station, though some are distributed along hallways and in physician charting rooms. No computers are in patient rooms. Although the number varies slightly across units, the average ratio of computers to patient beds is roughly 1 to 1.

Survey

In June 2003 we mailed a 20-item survey to attending physicians who had been responsible for a minimum of 25 orders during the preceding month at either Baystate or Franklin Medical Center. Orders counted toward this minimum if they had been written, given verbally in person or by phone, or entered directly into the computer by the physician. The survey consisted of 20 questions focused on the topic of computerized order entry. In addition to collecting information about sex and specialty, we asked respondents to describe their use of CPOE during training, their use of computers at home, and, where applicable, their use of computers in their outpatient practices. The survey included questions about how often respondents used the order entry system when caring for hospitalized patients and which features of the system they used. To assess physician attitudes about the order entry process, we asked respondents to consider whether it was faster to place orders directly into the system than it was by handwriting them, whether orders placed in the system were carried out more rapidly, whether placing orders in the system led to fewer medication and other errors, whether order sets were important for the efficient use of the system, whether order sets helped to ensure that important aspects of care did not “slip through the cracks,” whether the system's user interface supported their work flow, and whether the encouragement of nurses was an important factor in their use of the system. Questions that assessed physician attitudes were presented on a 5-point Likert scale. Nonrespondents were sent reminder letters along with duplicate surveys twice, approximately 1 and 2 months after the initial mailing. No financial incentive was offered for participation. The study protocol was approved by the Institutional Review Board of Baystate Health System.

Order Entry Rates

Regardless of whether an order is placed directly by a physician into a computer, given verbally, or handwritten, all orders are ultimately entered into the CPOE system. Working with our hospitals' Departments of Information Services, we developed a report that provided physician-specific information about order entry patterns. For each physician in the study, we determined the total number of orders generated during the month preceding the initial June mailing, as well as the absolute number and percentage of orders of each of the following categories: directly entered, telephone, verbal, and written. Because verbal and telephone orders are required during urgent situations and when physicians give orders from outside the hospital, we calculated and report an adjusted order entry rate as the total number of orders placed directly into the system divided by the sum of the orders entered directly and the number of written orders.

Analysis

Summary statistics for the overall sample were constructed using simple frequencies and proportions for categorical variables and medians and interquartile ranges for continuous variables. We compared characteristics of respondents from the 2 hospitals using chi-square tests of association for categorical factors and Wilcoxon rank-sum tests for continuous scale data. We compared the total number of orders placed during the study month and the order entry rates of responders and nonresponders using the Wilcoxon rank-sum test. We categorized physicians as low (≤20%), intermediate (21%-79%), and high (≥80%) users of the system based on their calculated order entry rate. Responses to each of the attitude questions in the survey were tabulated, and the responses “strongly agree” and “agree” were combined for analyses comparing responses. Demographic variables and physician attitudes were tested for associations with order entry rate categories via the Pearson chi-square for categorical factors, the Mantel-Haenszel chi-square for ordered factors, and Kruskal-Wallis analysis of variance for continuous variables. Initial analyses were stratified by hospital; where no differences in association were found across strata, the data were combined. Statistical tests were performed using SAS version 9.1 (SAS Institute, Cary, NC)

RESULTS

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. CONCLUSIONS
  7. Acknowledgements
  8. REFERENCES

During the study period the target group of physicians placed a total of 135,692 orders, of which 69,654 (51%) were placed directly into the CPOE system, 38,878 (29%) were made using pen and paper, 7,208 (5%) were made verbally, and 19,952 (15%) were placed by telephone. Three hundred and fifty-six (71%) of the 502 surveys sent out to physicians at the 2 hospitals were returned. Thirteen surveys were excluded from analysis because the respondent was not a physician, and 2 because we were unable to match the survey to system usage data, leaving a total of 341 surveys for analysis. Order entry rates were not computed for an additional 3 physicians who only placed verbal and telephone orders during the study period. Response rates did not differ by clinician specialty (P = .53); compared to those of nonresponders, respondents had a similar median total number of orders (111 vs. 101, P = .67) and a higher median order entry rate (66% vs. 48%, P = .03).

Characteristics of Respondents

Seventy-two percent of physicians who completed the survey were men; half had graduated from medical school at least 20 years ago, and the median duration of practice at the study institution was 11 years (Table 1). Forty percent practiced internal medicine, 18% were surgeons, and 16% were pediatricians. Thirty-five percent completed training at an institution that had computerized physician order entry, and 86% cared for patients primarily at Baystate Medical Center. More than half reported they used the system many times each day for patient care, and the features they used most commonly were retrieval of results (95%), placing of orders (78%), and viewing and printing of patient lists (75%). Among those with outpatient practices, 81% used computers in their outpatient practice, and more than half used computers for personal activities at home at least once a day. On average, respondents from Franklin Medical Center had graduated from medical school farther in the past and reported less reliance on the system to carry out all activities other than viewing results.

Table 1. Characteristics of Physicians Participating in the Study (N = 341)
 Overall n (%)Baystate n (%) 293 (85.9)Franklin n (%) 48 (14.1)Chi square P value
  • a

    From survey responses.

  • b

    CPOE rate = (orders directly entered)/(orders directly entered + orders placed in writing).

  • c

    Data of actual use during month preceding survey mailing.

Sex   .64
 Male244 (71.6)211 (72.0)33 (68.8) 
Specialty   .24
 Anesthesia23 (6.7)23 (7.9)0 (0.0) 
 Internal medicine135 (39.6)112 (38.2)23 (47.9) 
 Medicine/pediatrics13 (3.8)6 (2.0)7 (14.6) 
 OB/GYN36 (10.6)30 (10.2)6 (12.5) 
 Pediatrics54 (15.8)51 (17.4)3 (6.3) 
 Surgery61 (17.9)55 (18.8)6 (12.5) 
 Other19 (5.6)16 (5.5)3 (6.3) 
Use of CPOE systema   .09
 Many times a day176 (52.2)160 (55.0)16 (34.8) 
 At least once a day77 (22.9)61 (21.0)16 (34.8) 
 A few times a week55 (16.3)45 (15.5)10 (21.7) 
 Once a week or less29 (8.6)25 (8.6)4 (8.7) 
Features useda    
 Viewing and printing patient lists254 (75.2)212 (72.6)42 (91.3).01
 Looking up results320 (94.7)277 (94.9)43 (93.5).70
 Viewing current medications218 (64.5)204 (69.9)14 (30.4)< .01
 Placing orders263 (77.8)244 (83.6)19 (41.3)< .01
 Entering discharge summaries72 (21.3)70 (24.0)2 (4.4)< .01
Use of order setsa    
 Rarely or never98 (29.0)74 (25.3)24 (52.2)< .01
 Minority of patients92 (27.2)78 (26.7)14 (30.4) 
 Majority of patients104 (30.8)97 (33.2)7 (15.2) 
 For all or nearly all patients44 (13.0)43 (14.7)1 (2.2) 
Percentage of orders placed using order setsa   < .01
 None46 (13.7)26 (9.0)20 (44.4) 
 1%-25%62 (18.5)50 (17.2)12 (26.7) 
 26%-50%29 (8.7)23 (7.9)6 (13.3) 
 51%-75%45 (13.4)43 (14.9)2 (4.4) 
 76%-99%103 (30.8)98 (33.8)5 (11.1) 
 All50 (14.9)50 (17.2)0 (0.0) 
Use of computer in outpatient practiceab243 (81.3)206 (80.8)37 (84.1).60
Personal computer usea   .47
 At least once a day209 (61.7)185 (63.4)24 (51.1) 
 Several times a week84 (24.8)67 (23.0)17 (36.2) 
 A few times a month21 (6.2)18 (6.2)3 (6.4) 
 Rarely25 (7.4)22 (7.5)3 (6.4) 
Training at an institution that had CPOE117 (34.7)105 (36.1)12 (26.1)0.19
Use of system to enter orders should be mandatorya    
 Yes113 (35.2)106 (38.4)7 (15.6)<.01
 Median (IQR)Median (IQR)Median (IQR)WilcoxonPvalue
Years since medical school graduationa20 (13, 26)20 (13, 26)24 (17, 28).02
Years in practice at study institutiona11 (5, 18)11 (5, 18)13 (7, 19).39
Orders directly enteredc23 (2, 99)27 (5, 108)1 (0, 27)< .01
Orders placed by telephonec14 (5, 49)12 (3, 38)49.5 (16, 123.5)< .01
Orders placed verballyc2 (0, 11)3 (0, 13)1 (0,3)< .01
Orders placed in writingc21 (4, 73)14 (3, 45)220 (106.5, 391)< .01
CPOE ratebc66% (3%, 94%)76% (19%, 96%)0.25% (0%, 17%)< .01

Attitudes Toward Computerized Physician Order Entry

Physicians who completed the survey offered diverse opinions about the impact of computerized order entry on work flow, patient safety, and quality of care. Only 22% believed the system's user interface supported their work flow (Q7), 34% believed it was faster to enter orders directly into the system than to handwrite them (Q1), and 41% believed orders placed into the system were carried out more rapidly (Q2) (Table 2). On the other hand, 63% of respondents believed that placing orders directly into the system led to fewer medication errors (Q3), and 51% stated the system generally reduced medical errors (Q4). Sixty-nine percent stated order sets were important for efficient use of the system (Q5), and 71% believed order sets served an important decision support role (Q6). Twenty-six percent stated that the encouragement of nurses was an important factor in their use of the system (Q8). Finally, 35% of attending physicians believed use of the system to place orders should be mandatory.

Table 2. Attitudes of Respondents (N = 341) to Computerized Physician Order Entry
inline image

Characteristics and Attitudes of High, Intermediate, and Low Users

The median order entry rate of respondents was 66%. One hundred and forty-one (42%) placed at least 80% of their orders directly into the system, whereas 109 (32%) placed no more than 20% of their orders directly in the system (Fig. 1). There was not a significant difference between the low, intermediate, and high use groups in the total number of orders that each physician placed during the study period (Table 3). Sex, years since graduation from medical school, years in practice at the study institution, and use of computers in the outpatient setting were not meaningfully different between the 3 categories of users (Table 3). On the other hand, medical specialty was strongly associated with use of the system, with anesthesiologists, pediatricians, and surgeons the specialties with the largest proportion of high users. Furthermore, physicians who were trained in a CPOE environment and those who reported daily use of computers for personal activities showed the highest levels of adoption. Physicians at Franklin Medical Center showed lower levels of order entry than their counterparts at Baystate.

thumbnail image

Figure 1. Distribution of direct order entry rate among clinicians responding to survey.

Download figure to PowerPoint

Table 3. Characteristics of Survey Respondents (n=338) with Written and/or Direct Entry Orders in Month Preceding Survey according to Low, Intermediate, and High Usage of a CPOE System
 Low (≤20%) n (row %)Intermediate (20%-79%) n (row %)High (≥80%) n (row %)P value
  • a

    Among n = 299 with outpatient practice.

  • b

    Because of missing survey responses, category values may not add up to total.

  • c

    Pearson chi-square P value.

  • d

    Mantel-Haenszel chi-square P value.

  • e

    Kruskal-Wallis P value

 n = 109n = 88n = 141 
Hospital   < .01c
 Baystate73 (25)79 (27)138 (48) 
 Franklin36 (75)9 (19)3 (6) 
Sex   .69c
 Female28 (29)24 (25)43 (45) 
 Male81 (33)64 (26)98 (40) 
Specialty   .0001c
 Anesthesia8 (35)3 (13)12 (52) 
 Internal medicine45 (33)37 (27)53 (39) 
 Medicine/pediatrics6 (46)5 (38)2 (15) 
 OB/GYN20 (56)12 (33)4 (11) 
 Pediatrics13 (24)9 (17)32 (59) 
 Surgery14 (23)21 (34)26 (43) 
 Other3 (19)1 (6)12 (75) 
Do you use a computer in your outpatient practice?a
 Yes75 (31)61 (25)105 (44).22c
 No20 (36)18 (33)17 (31) 
Level of personal computer useb   .045d
 Rarely11 (44)8 (32)6 (24) 
 A few times a month7 (33)4 (19)10 (48) 
 Several times a week28 (35)25 (31)28 (35) 
 At least once a day62 (30)50 (24)97 (46) 
Training at an institution that had CPOE   .037c
 Yes30 (26)40 (34)46 (40) 
 No76 (35)48 (22)94 (43) 
 Median (IQR)Median (IQR)Median (IQR) 
Years since graduation from medical school21 (16, 28)18 (14, 25)19 (12, 25).06e
Years in practice at study institution12 (5, 19)12 (6, 19)12 (6, 17).84e
Total number of orders placed112 (45, 306)105 (56, 254)113 (44, 382).92e

Use of the system was highly associated with physician attitudes toward CPOE, with the views of intermediate and high users consistently different than those of low users (Fig. 2). The associations found held true regardless of hospital: low, intermediate, and high users from Franklin had similar responses to those from Baystate (P > .05 for all questions), and the data from the 2 hospitals therefore were combined for presentation. Although few physicians believed that the user interface of the system supported their work flow, high and intermediate users were 3 times as likely to share this view than were low users (Q7; Fig. 2). Similarly, 19% of low users, 31% of intermediate users, and 45% of high users believed that entering orders into the system was faster than writing orders (Q1). High and intermediate users of the system were more likely than low users to believe that orders entered into the system were carried out more rapidly (Q2) and led to fewer medication (Q3) and nonmedication (Q4) errors. Regardless of their utilization pattern, most physicians believed that order sets played an important role in promoting efficiency and quality.

thumbnail image

Figure 2. Attitudes of low-, intermediate-, and high-use users of a computerized physician order entry system.

Download figure to PowerPoint

DISCUSSION

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. CONCLUSIONS
  7. Acknowledgements
  8. REFERENCES

In this study of the clinical computing practices of physicians at 2 community hospitals, we observed wide variation in the adoption of CPOE by individual attendings. Although roughly one-third rarely placed orders directly into the system, 42% had an order entry rate of at least 80%. Contrary to our initial expectation, we found little association between a physician's order entry rate with years in practice, duration of exposure to CPOE, or use of computers in the outpatient setting. On the other hand, we observed marked differences in use of the CPOE system across specialty lines and found that physicians who were exposed to CPOE during training and those who were regular users of computers for personal activities were more likely to embrace this technology. Further, we observed important differences between physicians who used the system to place some or most of their orders and those who did so only rarely in their beliefs and attitudes about the impact and benefits of CPOE. Physicians with higher order entry rates were more likely than their colleagues to believe that placing orders electronically was faster than handwriting and that use of the system led to fewer medical errors. These findings should be encouraging to hospitals hoping to implement CPOE because they suggest that successful adoption of CPOE is not limited to physicians who have just completed their residencies or to hospitals with the capability of designing and building their own systems. On the contrary, we documented that women, older physicians, and those with limited CPOE experience were as likely to be frequent users, especially if they perceived CPOE to be safer than handwriting and if they believed the user interface supported the efficient entering of orders.

On the basis of these results we recommend that in addition to purchasing systems that meet physician work-flow needs and support the efficient entry of orders, hospital leaders should emphasize the quality and safety benefits of CPOE as part of a comprehensive change management strategy. The differences we observed in order entry rates across specialties may have resulted from several factors, including inherent differences in personality type associated with choice of specialty and in the level of customization of a system reflected in which and how many order sets are included. Such findings suggest that when it comes to CPOE, one size does not fit all, and implementation planning should be carried out at the specialty level. Finally, our observation that physicians who had exposure to CPOE during training were more likely to use the system to place orders suggests that the nation's training institutions will play an important role in fostering universal adoption of this technology.

Several earlier studies have reported on physician experiences with CPOE systems. Murff and Kannry12 surveyed 94 internal medicine house staff to compare experiences with 2 CPOE systems: the Department of Veterans Affairs Computerized Patient Record System (CPRS) and a commercially available product. They found striking differences in user satisfaction with numerous aspects of the systems, however they did not address attitudes toward safety or quality, and because house staff were required to place orders electronically they were unable to correlate responses with actual usage patterns. Weiner et al.13 compared the opinions of internal medicine house staff, attendings, fellows, and nurses about the benefits and challenges of using a computerized provider order entry system. In contrast to the findings from our study, Weiner et al. reported that more than half of physicians believed that provider order entry led to a greater number of errors, and only a minority believed the system increased quality of care overall. Finally, Lee et al.14 surveyed medical and surgical house officers and nurses at a large academic medical center about their satisfaction with a locally developed order entry system. They found that attitudes about the impact of the system on productivity and ease of use were more strongly associated with overall satisfaction than having undergone training or experience with personal computers. These findings are congruous with our own observation that beliefs about the speed with which orders are placed are closely associated with actual use of the system. They reported, as have we, that physicians placed a high value on order sets.

Our study had a number of strengths. First, we were able to offer insight into the attitudes and behaviors of a previously neglected, but critically important group—attending physicians who care for patients at community hospitals without the assistance of house staff. Second, whereas previous studies primarily assessed physician satisfaction with CPOE, we explored how physician attitudes about the impact of CPOE on work flow and on safety were associated with actual ordering habits. Information about ordering was obtained directly from the order entry system and not through self-report. We conducted the study at 2 hospitals, a large urban community teaching hospital and a smaller rural hospital, and focused on a CPOE system that is in use at many institutions throughout the country, thereby increasing the generalizability of our findings. Although adoption of the system by physicians at the 2 hospitals differed, factors that associated with the use of CPOE to place orders were similar. Finally, we surveyed a large number of physicians, had a high response rate, and found only small differences in the utilization patterns of responders and nonresponders, suggesting that our portrayal of the attitudes of physicians was representative of the views of physicians practicing in our community.

The study had a number of weaknesses. First, we cannot be sure whether preexisting beliefs about the benefits of CPOE directly influenced physicians' use of the system or, conversely, if these attitudes developed in response to experience as users. Nevertheless, it seems practical to suggest that hospitals focus on purchasing systems that support the efficient entering of orders while simultaneously adopting a communication and change management strategy that emphasizes the safety and quality benefits of CPOE more broadly. Second, we did not attempt to validate the opinions expressed by physicians about the usability or safety benefits of the system. That said, the purpose of the study was to determine whether physician attitudes toward these issues was associated with the use of the system to place orders. Whether or not this particular CPOE system actually prevented medication errors, most physicians believed it did, a belief strongly associated with the observed order entry rates. Third, we studied a single CPOE system implemented approximately 10 years ago that does not reflect state-of-the-art user interface design or functionality. Nevertheless, our observation about the importance of the user experience is probably no less relevant today. Fourth, we were unable to ascertain every order given by physicians, as some so-called “MD to RN orders” may never have made it into the system. Finally, there is a small risk that some written, telephone, and verbal orders may have been randomly or systematically assigned to incorrect physicians, which would have led us to calculate inaccurate utilization rates.

CONCLUSIONS

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. CONCLUSIONS
  7. Acknowledgements
  8. REFERENCES

In a voluntary community hospital environment the adoption of CPOE by attending physicians varies widely. While placing a premium on the purchase of systems that meet the work-flow needs of physicians and support the efficient entry of orders, hospital leaders can enhance physician adoption of this technology by communicating the role of CPOE in improving quality and safety.

Acknowledgements

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. CONCLUSIONS
  7. Acknowledgements
  8. REFERENCES

The authors thank Gilad Kuperman, MD, PhD, for his thoughtful comments on an earlier version of the manuscript.

REFERENCES

  1. Top of page
  2. Abstract
  3. METHODS
  4. RESULTS
  5. DISCUSSION
  6. CONCLUSIONS
  7. Acknowledgements
  8. REFERENCES