Effects of Computerized Provider Order Entry and Nursing Documentation on Workflow

Authors


Address for correspondence and reprints: Phillip Asaro, MD; e-mail: asarop@wustl.edu.

Abstract

Objectives:  The objective was to measure the effects of the implementation of computerized provider order entry (CPOE) and electronic nursing documentation on provider workflow in the emergency department (ED).

Methods:  The authors performed a before-and-after time-motion study of the activities of physicians and nurses. The percentages of time spent in task categories were calculated by provider session and averaged across provider sessions.

Results:  There was a shift in physician time from working with paper alone, 13.1% to 9.6% (p = 0.05), to working with paper while using a computer, 1.6% to 4.3% (p = 0.02), and an increase in time spent working on computer and/or paper from 30.0% to 38.9% (p = 0.02). For nurses, the increase in time spent on computer from 9.5% to 25.7% (p < 0.01) was offset by a decrease in time spent working with paper from 16.5% to 1.8% (p < 0.01). Direct care decreased minimally for nurses from 56.9% to 55.3% (p = 0.69), but from 36.8% to 29.1% (p = 0.07) for physicians, approaching statistical significance. Care planning decreased for nurses from 9.4% to 6.4% (p = 0.04) and from 21.7% to 19.5% (p = 0.60) for physicians.

Conclusions:  The net effects of an implementation on provider workflow depend not only on changes in tasks directly related to the provider–computer interface, but also on changes in underlying patient care processes and information flows. The authors observed an unanticipated shift in physician time from interacting with nurses and patients toward retrieving information from the electronic patient record. Implementers should carefully consider how implementations will affect information flow and then expect the unexpected.

Electronic documentation and computerized provider order entry (CPOE) may in themselves take longer than their paper counterparts,1,2 but provide an infrastructure that supports improved patient safety, improved information flows, and smoothed patient care processes.3–9 Although there are recognized barriers to implementation of CPOE in the emergency department (ED),10 there are significant potential benefits to doing so.11–13 Published studies of CPOE implementations in the ED have demonstrated improved compliance with guidelines14 and improved time to administering time-sensitive care.15 A study of CPOE implementation in the ED by Piasecki et al.16 found cost savings in terms of nonphysician staff time, but not in physician time. It is also known that implementation of CPOE can lead to unexpected results including effects on workflow and communication.17 A reciprocal impact of CPOE on communication has been previously reported.18

The Barnes-Jewish Hospital ED (affiliated with Washington University in St. Louis, MO) added CPOE and fully electronic nursing documentation to its implementation of a commercial ED information system (EDIS). The EDIS (Healthmatics ED, Allscripts, Chicago, IL) had previously been implemented with functionality including electronic tracking board, triage nursing documentation, and physician discharge processing. Along with CPOE and full nursing documentation, bidirectional interfaces with the laboratory, radiology, and registration systems created efficiencies in order processing and result retrieval. Physician documentation remained on paper problem-based templates through this implementation. The objective of this study was to measure the effects of the implementation of CPOE and electronic nursing documentation on provider workflow.

Methods

Study Design

This was a before-and-after observational study of physician and nurse activities using continuous work sampling (time-motion) methodology. The study was approved by the Human Studies Committee of Washington University. Verbal consent was obtained from patients and providers.

Study Setting and Population

The implementation of CPOE and electronic nursing documentation took place in May 2003. At the time of CPOE implementation, all nursing documentation and order writing in the entire ED shifted from paper to electronic entry. Although there is a provision for verbal orders entered by nurses in the electronic system, the vast majority of orders have been entered directly by ordering providers since the CPOE implementation.

Study Protocol

Observation sessions were performed 3 to 6 weeks before and 5½ to 6½ months after the EDIS implementation. Observation sessions were 4 hours in length and took place in the main part of the ED (not in the trauma–critical care unit or the fast-track area of the ED). Observation sessions consisted of simultaneous observation of all emergency physicians and nurses making up a functional team. The activities of each physician and nurse provider were recorded by a single observer assigned to that provider for the entire 4-hour data collection session. Thus, all of the physician and nurse activities performed in a functional unit were captured during each session and are represented in our data. Functionally, the main part of the ED is divided into three areas. During the busy evening hours in which this study was conducted, there are three attending physicians with zero, one, or two residents assigned to their team. A functional team consists of the attending physician, the resident physicians reporting to that attending physician, and the nurses caring for the patients in the rooms covered by those physicians. For physician providers, 11 physicians were observed before implementation and 10 after implementation. Of these 21 physician observations, 10 were with resident physicians, 5 before implementation and 5 after implementation. Although rotating resident physicians from other services sometimes work in the ED, all of the residents involved in this study were emergency medicine residents in Postgraduate Years 1 through 4. For nurse providers, we observed 13 before and 13 after implementation. All observation sessions took place on weekday evenings when ED resources are typically saturated with patients waiting to be seen. The timing of the 4-hour sessions was such that the entirety of the observation sessions fell within typical physician and nurse shifts. Thus, we did not observe physician or nurse turnover, excepting a few unusual circumstances. In these cases, the observer continued observing the initial provider until they left duty and then immediately switched to observing their replacement. Sessions were scheduled around the availability of observers without regard for provider schedules. Provider selection was thus “random.” The observers were students who had been trained in the use of the data collection tool.

Time-motion data were collected using a personal digital assistant (PDA) application created by the primary investigator (PVA) and previously described.19 The observer selects a broad category such as “with patient” or “using computer,” starting a task timer. A list of specific task categories in the selected broad category appears on the screen from which the observer then selects the most appropriate task. The end of a task coincides with the start of the next task. The observer invokes a “multi” mode when description of the provider’s activity requires selection of two task categories. For example, if a physician is discussing a patient’s care by phone with a consulting physician, the task would be “discussing with consultant” under the broad category of “on phone.” If the physician then begins to read from the electronic patient record while still speaking on the telephone, this portion of the activity is also recorded as “chart review” within the broader category of “using computer.” The PDA application uses two separate timers to track the overlapping periods of activity descriptions. Either timer may be stopped, allowing the other to continue.

Measures

Task descriptions were assigned to mutually exclusive categories for analysis. Provider activities involving interaction with a patient are categorized as direct care. Direct care–related activities included acquisition and preparation of supplies or medications, search for equipment or supplies, transit to and from patient rooms, hand washing, and specimen handling. Care-planning activities include discussions between providers about patient care, giving or obtaining verbal orders, and telephone calls with other providers or ancillary departments. Knowledge acquisition relates to accessing general medical references. Result review includes reading electrocardiograms, reviewing radiology films or results, and obtaining lab results other than by viewing them on computer (e.g., obtaining lab results by telephone). Time spent in activities requiring two task descriptors was assigned to “combination” categories (e.g., computer + paper). We calculated the percentage of time spent in each category by provider session.

Service time data were obtained from time stamps in the EDIS for the period February 2002 through June 2004. Wait time is the time from arrival until the patient is placed in a treatment bed. Treatment time is the time from patient placement in a treatment bed until a disposition order for that patient is written. As a measure of ED volume, we calculated daily ED arrivals (number of patients presenting to the ED for care). As a measure of ED outflow resistance, we calculated daily total boarding time (hours per day of ED bed occupancy by patients awaiting placement in an inpatient bed).

Data Analysis

The pre- and postimplementation means of percentage of time per category were calculated for both physicians and nurses. The significance of the difference between means was determined using the independent sample t-test without an assumption of equal variances. Statistical calculations were performed with SPSS 14.0 (SPSS, Inc., Chicago, IL).

RESULTS

The mean percentage of time spent in each category for nurses and physicians is reported in Tables 1 and 2, along with the pre- to postimplementation change in percentage. Several summary categories (preceded by “All”) are also shown. Figures 1 and 2 graphically display provider time by category as a percentage of total time. For both physicians and nurses, there were significant shifts in time spent working with paper and/or computer. For physicians, the change in total time spent working with paper from 15.9% to 16.2% (all paper) was not significant. However, there was a shift from working with paper alone to working with paper while using a computer (from 13.1% to 9.6% for paper and from 1.6% to 4.3% for computer + paper). The increase in time physicians spent working on computer and/or paper was from 30.0% to 38.9% (all computer and/or paper). For nurses, a significant increase in time spent on computer from 9.5% to 25.7% (all computer) was offset by a similar decrease in time spent working with paper from 16.5% to 1.8% (all paper).

Table 1.   Time Percentages for Physicians (Mean Percentage of Time)
CategoryBeforeAfterChangep-Value
Care planning18.714.9−3.80.27
Computer + care planning2.22.40.20.87
Computer11.919.98.0<0.01
Computer + paper1.64.32.70.02
Paper13.19.6−3.50.05
Paper + care planning0.82.21.40.10
Paper + direct care0.40.2−0.20.35
Direct care30.124.1−6.00.14
Direct care–related6.34.8−1.50.15
Result retrieval2.12.50.40.67
Knowledge acquisition0.40.3−0.10.83
Search for chart0.40.1−0.30.17
Idle/waiting6.06.30.30.91
Other2.82.2−0.60.52
Personal time3.36.33.00.26
Total100.0100.0  
All computer15.727.011.3<0.01
All paper15.916.20.40.87
All computer and/or paper30.038.98.90.02
All care planning21.719.5−2.20.60
All direct care30.524.2−6.30.13
All direct and related care36.829.1−7.70.07
Table 2.   Time Percentages for Nurses (Mean Percentage of Time)
CategoryBeforeAfterChangep-Value
Care planning8.14.8−3.3<0.01
Computer + care planning0.81.60.80.17
Computer7.921.914.0<0.01
Computer + paper0.60.2−0.40.21
Paper14.91.5−13.4<0.01
Paper + care planning0.50.0−0.50.02
Paper + direct care0.40.0−0.40.05
Computer + direct care0.11.91.80.12
Direct care43.640.3−3.30.40
Direct care–related12.713.00.30.85
Search for chart0.40.1−0.20.10
Idle/waiting0.84.53.70.04
Other4.83.4−1.40.16
Personal time4.36.52.20.36
Total100.0100.0  
All computer9.525.716.2<0.01
All paper16.51.8−14.7<0.01
All computer and/or paper25.327.31.90.51
All care planning9.46.4−2.90.04
All direct care44.142.3−1.90.65
All direct and related care56.955.3−1.60.69
Figure 1.

 Physicians’ time percentages.

Figure 2.

 Nurses’ time percentages.

Observers were usually able to differentiate paper and computer tasks at a more granular level. The breakdown appears in Table 3. When observers were uncertain of the specific task, the task was classified as paper-unknown or computer-unknown. Observers recorded order writing on paper approximately 3.4% of the time before implementation and electronic order writing 4.2% of the time after implementation. It is possible that these measures are somewhat underestimated, with upper bounds of the underestimate indicated by addition of the unknown categories. Most of the paper plus computer time (70% before implementation and 91% after implementation) was observed as visit documentation on paper while reviewing information in the electronic patient chart.

Table 3.   Breakdown of Observed Physician Computer and Paper Tasks (Mean Percentage of Time)
 PaperComputer
BeforeAfterBeforeAfter
Visit documentation8.412.5NANA
Orders3.4NANA4.2
Chart/result review0.12.17.313.8
DispositionNANA3.42.3
Other0.90.31.62.2
Unknown3.01.33.44.4
Total15.916.215.727.0

The overall effect of the implementation on time spent directly caring for patients differed between physicians and nurses. For nurses, all direct and related care changed minimally from 56.9% to 55.3%, whereas for physicians the decrease was from 36.8% to 29.1%, approaching statistical significance at the 95% confidence level. Care planning time decreased significantly for nurses, from 9.4% to 6.4% (all care planning), whereas the decrease for physicians from 21.7% to 19.5% was not statistically significant.

Monthly means of service times are depicted in Figure 3. Monthly mean wait times and treatment times are graphed along with the monthly means of daily ED arrivals and daily total boarding times.

Figure 3.

 Process times before and after implementation (implementation at vertical line).

Discussion

In this implementation, both physicians and nurses experienced significant increases in time spent interacting with computers. Whereas the decrease in time spent working on paper offset the increase in time on computers for nurses, this did not hold true for physicians, superficially consistent with previous studies.2,16 However, the time differences that we measured are a result of the interaction of electronic nursing documentation and CPOE implementation. For nurses, several order processing steps were completely eliminated. The lack of overall improvement in computer and/or paper time suggests that electronic nursing documentation itself may have required additional time.

Order writing was observed 3.4% of physician time before implementation and CPOE 4.2% of the time after implementation. Although these measures may be somewhat underestimated, it appears very likely that much of the 8.9% increase in overall physician time on computers and/or paper does not relate directly to ordering tasks. Much of the physician time spent working on paper (both before and after implementation) was directly related to documentation of the patient visit. While the medium of documentation remained unchanged, we saw a shift toward collecting information from the computerized patient chart while creating the visit documentation.

Our data indicate that the physician time taken up by additional indirect care tasks was taken from time spent in direct patient care and closely related activities (direct care–related). The observed decrease in all direct and related care time by physicians approached significance at the 95% confidence level. This pattern of increased indirect patient care task load at the expense of physician–patient interaction might raise concerns regarding physician efficiency.

On the other hand, there is reason to believe that improvement in patient care processes implemented via these information system changes should have improved service times. Transcription of laboratory and radiology orders by nurses was eliminated, and there is automatic printing of specimen labels based on physician-entered orders. With the new process, medication administration records are automatically created based on physician-entered orders. For medications not held in the ED, the prior process required a nurse to fax a copy of the order to the pharmacy, whereas the new process immediately prints the order in pharmacy. The postimplementation system provides much better status information for ordered tests and treatments, all within the patient’s computerized chart. Rather than speaking with a nurse or checking in on a patient to discover this information, the physician can review the patient’s electronic chart. The observed use of computers while performing visit documentation, along with the portion of increase in computer time not explained by computer order writing, suggest that physicians find the integration of information in the electronic record useful.

A significant proportion of care planning time spent by nurses occurs in conversations with physicians and includes obtaining verbal orders and clarification of written orders. The time-motion data reported in this study, along with a related nurse survey previously published,20 suggest that these direct physician–nurse communications were noticeably reduced by this implementation. While most of the care planning time of physicians is spent in interaction with other physicians, a component of physician care planning is interaction with nurses. While the relative change in physician care planning was small and did not reach statistical significance, the decrease is similar in absolute magnitude to the decrease in nurse care planning time, further supporting the hypothesis of a shift in information flow between nurses and physicians from direct verbal communication to communication via the information system. A majority of CPOE orders are generated from order sets and are thus “standardized.” Before implementation, laboratory results were obtained by physicians from a separate legacy computer system. After implementation, results flowed directly into the patient record in the EDIS, readily visible to nurses as well as physicians. Standardized orders and improved information flows appear to have reduced the need for direct physician–nurse interactions.

The increase in time physicians spent reviewing charts and results from 7.4% before implementation to 15.9% after implementation (Table 3, summing paper + computer) appears paradoxical, given the efficiency with which results can be reviewed in the information system after implementation. This suggests that physicians are spending more time reviewing electronic documentation and time stamps as a means to track the progress of work-up and care (via the electronic medication administration record, documentation of nursing procedures, status of laboratory and radiology testing, etc.). This may explain at least some of the decrease in direct care time by physicians in that physicians may be finding it more efficient to monitor the progress of work-up and therapeutics from the information system in lieu of checking in with the patient directly.

Service times, as shown in Figure 3, do not appear to have been affected by the implementation (vertical line). Because changes in overall ED patient volume and ED boarding time would be expected to influence wait time,21 we have superimposed graphical representation of these variables. The marked variations in observed ED boarding time relate primarily to inpatient hospital factors, including various admission process changes implemented during the period shown. There were some experimental periods of an additional attending physician working with nurses in the triage area of the ED for part of some days during the latter part of 2003, but this was not continued. There were otherwise no physician staffing changes made through the period shown on this graph. We might expect to find that improvements in workflow and patient care processes would result in decreased treatment time and that decreased treatment time would lead to decreased wait time, but there were no apparent changes in treatment time (solid line) or wait time (dotted line) attributable to the implementation, although small changes may be overshadowed by variability due to other causes. Potentially time-saving process improvements may have been offset by additional time spent by physicians in data entry and retrieval. This study did not evaluate the computer–user interface. It is possible that usability issues have prevented gains in workflow efficiency and service times that might otherwise have been achieved. On the other hand, it is possible that other factors in this complex academic environment (such as awaiting specialty consultations and/or inpatient beds) overshadowed process improvements related to this implementation.

Summarizing, we can say that information flows have changed significantly, resulting in a much greater dependence on the computerized patient record, with indications that physicians monitor the processes of evaluation and treatment from the computerized patient chart in lieu of direct communication with nurses and possibly in lieu of direct interaction with patients. While the possibility of an increase in ordering time was known from the outset of this implementation, the observed shift in physician time from interactions with nurses and patients to interaction with computers was not anticipated. In spite of apparent process and workflow improvements, we see no apparent change in overall service times attributable to the implementation. Unfortunately, assessment of the quality of information is much more difficult than quantifying time spent in various tasks. While we can use our data to make inferences about changes in the pathways of information flow and provider time spent in relation to the information flows, we do not know how this has affected the quality of patient care or of documentation.

Limitations

This study was performed in a single busy urban academic ED with one particular commercial EDIS product. Our results may have limited direct generalizability to other institutions; however, they clearly demonstrate expected and unexpected changes in provider workflow and highlight the importance of unanticipated effects of changes in information flows by clinical application implementations.

The data collection process used in this study has limitations. We have not attempted validation or interrater reliability testing of the data collection process itself. Measurement and interpretation of interrater reliability would be highly dependent on the level of aggregation of observations. At higher levels of aggregation, one would expect a very high degree of agreement. For example, all of the activities that are aggregated into direct care require selection of the “with patient” button in the data collection tool and observers are taught to always select “with patient” to record face-to-face activities with patients. Similarly, documentation of any activity involving computers requires selection of the “InfoSys” button. On the other hand, the ability of observers to discriminate between “chart review” and “result review” within the electronic medical record will be much more limited, and interrater agreement at this level of aggregation may be poor. We have given careful consideration to these issues in our reporting and discussion of findings, tempering our assertions regarding specific findings based largely on the confidence we have that the aggregations are at face value likely to be consistent across observers.

The division of work across an attending-resident physician team presents challenges in interpretation of our results. As described under Methods, the activities of each physician were recorded by a single observer assigned to that physician for the entire 4-hour data collection session. An attending physician was present during every session, but the number of resident physicians varied. Because of the variation in how the physician group “divided” patient care tasks, some physicians could easily spend more time on certain tasks than other physicians. The activities that would be performed by each physician if they independently performed all of the tasks necessary for patient care is estimated by the arithmetic mean of the entire physician group. However, the sample variance for physicians operating with a team approach would be expected to be greater than in the case of physicians independently performing patient care. An alternate approach to analysis would be to consider the physician team as the sampling unit, calculating the proportion of physician activities in each activity category at the team level. Although we recognize that less experienced physicians are not as efficient as more experienced physicians, and that there is some duplication of effort in teaching situations, teams with more residents do indeed cover more patient beds and do see more patients. Therefore, with the team-as-analysis-unit approach, in pre- to postimplementation comparisons, we would have weighted the activity proportions for each observation session according to the number of physicians in the observed team. In comparison to the individual-physician-analysis approach we took, this team-analysis approach would have yielded the same overall pre- and posttime percentages, but would incorporate smaller degrees of session-to-session variance. The net effect of this difference in approach to analysis would be to indicate greater statistical significance for the same pre- to postimplementation time percentage differences. In this sense, the individual physician approach to analysis is the more conservative approach. More importantly, we feel that the approach we took yields more understandable and generalizable results as an estimation of the impact of the information system implementation on a “typical physician.”

Sample size limits the significance of some of our activity measures more than others, depending in large part on session-to-session variability for the activity. For example, personal time for physicians appears to have doubled. Personal time consisted of brief non–patient care–related activities, such as a 2-minute conversation with a colleague involving recent sporting event scores or a trip to the restroom, as well as longer breaks to grab a meal. If a physician happened to take a 15-minute break during a 4-hour observation session, this would amount to about 6% of total time. The lack of statistical significance for the doubling of personal time arises from the marked session to session variability in this category.

Conclusions

The net effects of a clinical information system implementation on provider workflow depend not only on changes in tasks directly related to the provider–computer interface, but also on changes in underlying patient care processes and information flows. Effects on information flows may be subtle, although significant, and may not be anticipated. Implementers must carefully consider how changes to clinical applications will affect information flow and then expect the unexpected.

Ancillary