This paper was presented at the “Women Deliver: A Global Conference” in London, England, on October 18, 2007.
Using Emergency Obstetric Drills in Maternity Units as a Performance Improvement Tool
Article first published online: 3 MAR 2009
© 2009, Copyright the Authors Journal compilation © 2009, Wiley Periodicals, Inc
Volume 36, Issue 1, pages 43–50, March 2009
How to Cite
Osman, H., Campbell, O. M. R. and Nassar, A. H. (2009), Using Emergency Obstetric Drills in Maternity Units as a Performance Improvement Tool. Birth, 36: 43–50. doi: 10.1111/j.1523-536X.2008.00294.x
This study was funded by the Centre for Research on Population and Health at the American University of Beirut, Lebanon, with generous support from the Wellcome Trust, London, England.
- Issue published online: 3 MAR 2009
- Article first published online: 3 MAR 2009
- Accepted August 14, 2008
- obstetric emergencies;
- team response
ABSTRACT: Background: Obstetric drills are being used increasingly to test, improve, and maintain knowledge and skills related to obstetric emergencies as a means to improve proficiency and efficiency of practitioners. The purpose of this study was to assess the feasibility and usefulness of conducting drills to evaluate the response to obstetric emergencies using a holistic approach that tested the hospital system.Methods: A prospective trial was conducted at three hospitals (two tertiary referral centers and one small community hospital) in Beirut, Lebanon. Two different emergency obstetric drills at two points in time were conducted between April and May 2006 either in the emergency room or on the labor floor. The drills included medical and paramedical staff, a female actor (simulating a pregnant woman), a research assistant (acting as her companion), and a physician trained in obstetrics (the drill leader). Responses were recorded and critically analyzed.Results: Although overall quality of care was within standards of care, problems were identified related to hospital policies, supplies and equipment, communication, and clinical management. Some technical problems related to administration of the drills were identified. Most drill participants appreciated the exercise and found it beneficial.Conclusions: Obstetric drills provide a useful tool to identify and address deficiencies in the hospital system. This finding could have implications on improving quality of care provided to obstetric patients. (BIRTH 36:1 March 2009)
Although some obstetric complications can be anticipated based on risk factors noted by clinicians, many are not predictable and present as obstetric emergencies. Global initiatives to reduce maternal mortality recognize the importance of ensuring that women have access to emergency obstetric care. However, the relative rarity and unpredictability of life-threatening emergencies make it difficult to train staff to deal with them appropriately and efficiently. Even well-trained and experienced clinicians may have difficulty maintaining their clinical competence in conditions that they do not commonly encounter (1). In addition, some evidence exists that obstetric emergencies are frequently mismanaged (2).
Simulated training in obstetric emergencies has been proposed and studied as a means to improve practitioner proficiency and efficiency. A review of the literature on acute obstetric emergency training identified several approaches to simulation training (1–11), most of which comprised a training session run over a period of 1 to 3 days and targeted maternity care practitioners. Programs usually involved computer-based case simulations using mannequins and were given in a classroom setting rather than a maternity ward. Few programs were performed on the labor floor in the presence of a local team involved in obstetric care (8,9,11). Most programs were evaluated by a questionnaire administered to the course participants to assess their satisfaction, by group discussion, or by checklists designed to identify appropriate and inappropriate actions.
Obstetric drills are being used increasingly to test, improve, and maintain knowledge and skills related to obstetric emergencies (8,9,11,12). A study assessing the practice of conducting obstetric emergency drills in England and Wales (12) showed that one-half of the centers surveyed conducted drills and 14 percent were developing a drill program. More recently, the Simulation and Fire-drill Evaluation (SaFE) trial in the United Kingdom demonstrated that simulation training using mannequins improved the management of shoulder dystocia, and most participants retained their skills at 12 months (8,9). These studies were conducted in a classroom setting rather than the actual labor ward and primarily targeted obstetric care practitioners. Several units have published the effects of training in their practice settings. Examples from the Liverpool Women’s National Health Service Foundation Trust illustrate these steps, including how they were developed and the obstacles that were encountered (13).
Our study aimed to assess the feasibility and usefulness of conducting drills to evaluate the response to obstetric emergencies using a holistic approach that tested the hospital system. We wished to develop an approach that was relevant to Lebanon, a middle-income country where the vast majority of births are facility based, and where the health system is largely private, fragmented, and poorly regulated (14). In most hospitals in Lebanon, maternity care is provided primarily by obstetricians with the support of midwives or residents.
We developed and conducted two different emergency obstetric drills at two points in time between January 2006 and May 2006 (a total of 12 drills) in three Beirut hospitals. Two are large teaching hospitals where residents are the first point of contact with obstetric patients and the third is a smaller community hospital where midwives attend obstetric patients under the supervision of the obstetrician. The number of deliveries in each hospital ranges between 1,100 and 1,500 annually, which constitutes approximately 37 percent of total deliveries in Beirut.
The American University of Beirut provided Institutional Review Board approval. Approval was also obtained from the administrations of the three hospitals involved. The heads of departments we anticipated involving (obstetrics, anesthesia, emergency medicine, laboratory medicine, blood bank, and nursing) were contacted and asked to inform their staff of the drills. They were advised not to reveal the content of the drills and to ask their staff to proceed as with a healthy parturient unless instructed otherwise. Department heads were reassured that the drills would not compromise patient care. In case of a true emergency or an unusually high patient volume, the study protocol ensured the drill would be aborted. Drills could occur at any time of the day and neither department heads nor staff were aware of the timing.
The two emergency obstetric drills developed were based on the American Academy of Family Physicians Advance Life Support in Obstetrics course handbook (15), guidelines from the American College of Obstetricians and Gynecologists, and guidelines from the Royal College of Obstetricians and Gynaecologists. Clinical scenarios that required specific interventions were developed—one involving medical management of eclampsia in the emergency room and the other surgical management of third trimester bleeding on the labor floor. Each scenario specified the woman’s age, gravidity, parity, and gestational age, and detailed the chief complaint. The responses of the teams were measured against agreed-on standards within the facilities that were developed in accordance with the American College of Obstetricians and Gynecologists’ recommendations (16,17).
Table 1 summarizes the main features of each drill. The drill started when a female actor simulating a pregnant woman arrived and stated her chief complaint. In the eclampsia drill, the complaint was severe headache and nausea. A few minutes after being in the emergency unit, the actor would fake a generalized seizure. In the second drill, the woman would present with excessive vaginal bleeding close to term. A research assistant (acting as her companion) and a physician trained in obstetrics (the drill leader) accompanied her. The companion timed and recorded all critical events using a stopwatch. The drill leader guided the drill according to the written scenarios, provided results of vital signs and laboratory results as specified in the protocol, and gave drill participants hints on management when needed. All interventions proceeded as for a real patient unless the drill leader told the drill participant to stop. For example, intravenous fluids, line, and needle were prepared, a tourniquet was applied, intravenous medications were prepared for administration, and the skin was cleaned in preparation for intravenous line insertion. However, the drill leader stopped the person placing the intravenous line before they broke the skin. The simulated patient was not undressed, no pelvic exams were performed, and no invasive procedures were conducted. If the drill participant was about to draw blood, he or she was asked what studies they intended to request. They were then handed previously drawn tubes of blood (carried by the drill leader) to send to the laboratory for processing. Laboratory staff were not aware that specimens were part of a drill until all results were reported.
|Procedure||Surgical Drill||Medical Drill|
|Start of drill||Patient walks into maternity unit||Patient walks into emergency room|
|Chief complaint||Heavy vaginal bleeding||Complains of headache|
|Medical history/physical findings that would require emergency delivery||Immediate (fake) generalized tonic-clonic seizure|
|End of drill||Patient at door of operating room||Patient stabilized with appropriate medications and transferred to maternity ward|
|Surgical set open; scrub technician ready|
|Anesthetist/surgeon ready for surgery|
|Laboratory results reported; units of blood ready for transfusion|
A data collection sheet was developed specifically for each drill. Critical interventions such as measurement of vital signs, placement of intravenous lines, time of arrival of physician, administration of medications, and readiness for surgery were prerecorded, and the times these interventions were conducted were noted by the research assistant (Figs. 1 and 2). The drill leader also assessed the functioning of the team qualitatively and recorded observations with points for potential improvement. In addition, using an interview guide, the research assistant conducted informal interviews with drill participants immediately after each drill to assess their response to the exercise, its perceived benefits, and the problems encountered. No scoring system was used by the research assistants in assessing responses from drill participants.
In each hospital, the findings of the first set of drills were presented to the Department of Obstetrics in the form of a seminar. These sessions were conducted by the research team and attended by the medical and paramedical staff involved in the drills. Feedback was discussed and suggestions for improvement were made. Written feedback was also sent to the heads of the departments involved in the drills with comments on performance and suggestions for improvement.
The second set of drills was conducted after departments had been given time to make changes based on the feedback. These drills aimed to identify new problems and to assess whether feedback from the earlier round led to any changes being implemented. After completion of all four drills, final feedback letters were sent to all departments and to the hospital administration.
Results were obtained on the quality of clinical care provided during the drills, on the feasibility and practical aspects of conducting drills, and on the views of drill participants.
Quality of Care Provided in Drills
Overall, the performance of physicians, nurses, and laboratory and other support staff was in accordance with the facility standards in all drills in the three hospitals. The simulated patient was attended to and appropriate care within the developed standards was initiated. The mean time to readiness for surgery in the six surgical drills was 16.4 ± 7.8 minutes with a range of 6.3 to 25.5 minutes. The mean time to initiating a loading dose of magnesium sulfate in the eclampsia drills was 16.3 ± 3.8 minutes (range 9.5 to 28.3 min; one drill in which it was never started). Nursing and clerical staff were efficient and organized.
The average number of participants, medical and paramedical, in each drill was 8. The times to critical interventions did not significantly differ between the first and second drill. The mean time to intravenous line insertion was 4.6 ± 1.2 minutes (range 3.2–5.4 min) versus 3.8 ± 0.6 minutes (range 3.1–4.2 min) in the first and second surgical drills, respectively (p= 0.360). The mean time to prepare blood for transfusion was 55.4 ± 29.4 minutes (range 34.23–89.0 min) in the first drill and 37.3 ± 9.7 minutes (range 27.5–47.0 min) in the second (p= 0.369). Staff reported more confidence in responding to these drills even though, on average, 50 percent of the people involved in the various drills were not the same.
Problems affecting the quality of clinical care noted during the drills are summarized in Table 2. The drills identified a range of problems relating to a lack of correct drugs and/or equipment in the correct places, problems in communication, deficiencies in policies for emergencies, and inappropriate clinical management. The drills also identified some examples of good practice. For example, the delivery suite in one hospital had premixed magnesium sulfate bags ready for emergency use, whereas the other hospitals did not. Premixed magnesium sulfate allows staff to treat an eclamptic woman quickly without wasting time mixing the magnesium sulfate and potentially minimizes dosing errors.
|Lack of correct equipment/drugs in correct place|
|Delay in checking fetal cardiac activity; lack of fetal Doppler probe in emergency room; fetal Doppler machine was brought from delivery suite||Provide emergency room with a Doppler machine||Fetal Doppler probe was obtained for emergency room|
|Ultrasound machine not available in delivery suite||Provide delivery suite with ultrasound to assess patients properly and save time making management decisions||Ultrasound machine now based in delivery suite|
|Lack of medications (e.g., hydralazine) or inadequate quantities of medications (e.g., magnesium sulfate) in emergency room||Need for such medications to be readily available in emergency room||Lack of hydralazine was not rectified despite addressing this problem in feedback after the first drill; adequate amounts of magnesium sulfate were available by second drill|
|Problems in communication|
|Problem reaching resident on call to emergency room when he/she is in operating room; it took several paging attempts before receiving answer||Need for a system to alert emergency room who is the resident to contact if the on call resident is not available||An “on call” pager was provided to be handed to the resident who was covering the emergency room to avoid similar problems in future|
|In one hospital, the laboratory did not call back results to delivery suite or emergency room; staff caring for the critical patient had to keep calling to check for results||In emergencies, it would be more efficient if the laboratory called with the results as soon as they were out||This policy had not been implemented by the time the second drill was conducted|
|Two hospitals had problems with execution of telephone orders in emergency room; for example, labetalol was ordered but was not given in emergency room. It was only on arrival to delivery suite, that it was noticed that the medication had never been given. In another hospital, the resident asked to prepare magnesium sulfate while he/she made his/her way to emergency room, but the infusion had not been prepared by the time he/she arrived||Since this was a recurring problem, it was suggested to stress the importance of responding to telephone orders provided they are followed by written orders when the physician is physically present in emergency room||This problem was formally addressed after the drill, and these problems did not recur during the second set of drills|
|Delay in reaching attending physicians because of busy operators and no direct access to telephone lines||Need for a special operator for emergencies or direct access lines to call attending physicians||No changes were made by the end of the study|
|Problems with policies for emergencies|
|A common problem was in the processing of laboratory specimens; the blood bank and laboratory refused to process specimens for patients not fully registered and on the computer systems||Need for policies for managing samples in emergency situations that allowed laboratory staff to proceed without having to ensure payment or patient registration; apparently, such policies existed but were unclear and staff members were not aware of them||Policies were disseminated to current staff, and it was suggested to include this information as part of orientation for new staff|
|Inappropriate clinical management|
|In one hospital, magnesium sulfate was not initiated at any stage of the eclampsia drill||Advised development of eclampsia protocol. Such protocol existed in that hospital||The eclampsia protocol was reviewed and reinforced with staff|
|For emergency room drills, nonobstetric staff were sometimes involved in care before arrival of the obstetrics resident/midwife and did not always follow obstetric best practice, for example, diazepam was ordered by a surgery resident for seizure control||Suggested training all emergency room staff on evidence-based practices for obstetric emergencies||No changes were observed by the end of our study|
Problems Related to Administering Drills
Problems occurred in some instances with our administration of the drills. In all three hospitals, some delay occurred during the first set of drills because the staff was unclear about whether they should prepare medications, send blood to the laboratory, process specimens, or call other team members to participate. In one hospital, staff had not been informed that drills were going to take place, were not familiar with the concept of a drill, and did not know what was expected of them. Time was lost explaining this plan to them before the scenario could proceed. During the second set of drills, staff had learned the drill endpoints and focused on reaching them without taking the time to assess the simulated “patient” appropriately as they would have done in a real emergency. Participants suggested that the simulated patient should be changed, since they could easily deduce, on seeing the same patient, that a drill was underway.
We estimated that each drill took an average of 2.75 person hours to complete. Feedback sessions lasted approximately 30 minutes and were attended by approximately 20 people each.
Response of the Participants
Overall, hospital staff appreciated the drills and thought that they benefited from the feedback given. They found them informative and “very close to reality.” Most did not find them stressful or threatening and reported that the process gave them more confidence in their skills. The emergency room staff in two of the three hospitals reported that the drills were beneficial, whereas emergency room staff in the third hospital thought that they should not be involved in training related to obstetric patients.
A few participants reported that the drills were stressful, either because they were busy with other patients or because they felt their individual performance was being evaluated. Some residents were concerned that the drills could compromise patient care. Others were unwilling participants and as a result were passive-aggressive or angry. In two drills, residents from surgery and internal medicine initiated care of the simulated patient in the emergency room while awaiting the arrival of the obstetric resident. Both were from departments that had not been informed about the drills and both expressed anger and frustration at being involved in the drills.
When asked whether running drills on a regular basis would be useful, most answered positively stating that the drills enhanced their confidence in their ability to handle a true emergency and the feedback helped them improve their performance. Many believed that they learned new things during the drills. One resident answered that they were confident enough and did not need drills to practice. Some drill participants thought it would be more useful to receive immediate feedback after the drills. Others stated that allowing for actual insertion of intravenous lines and drawing blood would be a good way to test how the participants would function under stressful conditions. Overall, participants were highly receptive to the idea of instituting drills and welcomed the process as a means to improve performance.
We conducted a set of two emergency obstetric drills twice in three hospitals. They were feasible to conduct and helpful in identifying problems related to supplies and equipment, poor communication, unclear or deficient policies for emergencies, and inappropriate clinical management. Most deficiencies were addressed immediately; however, some were not corrected during the course of this study. Hospitals may have needed more time to implement certain changes. More drills conducted at a later point in time may have identified additional improvements.
Although many of the deficiencies identified by the study were minor, some significant deficiencies that could have an important influence on quality of care were also identified. For example, the lack of awareness of laboratory staff of the protocol for preparing blood in emergency life-saving cases was a significant deficiency in all three hospitals. The inability of the emergency room to contact the resident on call in a timely manner in one hospital was another significant deficiency. Both of these problems were identified and rectified through the administration of the drills. We believe that such improvements in the system can potentially save lives in the case of a true emergency.
Although improvements were noted in response times between the first and second drills, these findings must be interpreted carefully. None of the changes reached statistical significance, probably due to the limited number of drills. These improvements may reflect better performance of the medical team but could alternatively stem from familiarity with the drill procedure. Additional drills would need to be conducted to identify statistically significant improvements in mean and maximum response times.
The mechanics of carrying out the drills were fairly straightforward, but adequate preparation of staff seems essential. In addition to written letters to department heads asking that the procedure be explained to staff, follow-up telephone calls and meetings to clarify the drills process would have been useful. These actions would help ensure that all drill participants were familiar with the concept of a drill and what was expected from them during its course. We noted that staff members who were adequately prepared were more engaged in the process, whereas people who were not adequately briefed were not engaged or even hostile. The attitudes of participants in general reflected attitudes of their departments. When they knew that the administration took the drills seriously, they put more effort and were more interested in improving their performance. Participants generally found the drills to be beneficial, as reported by others (3,10). They valued the authenticity of the experience and suggested that the element of surprise was important.
Training programs have focused on clinical staff and have largely ignored the roles of support staff and management in ensuring that obstetric emergencies are managed efficiently. Team performance and communication have been noted to play an important role in the quality of obstetric care (4). Studies show that life-saving practices in life-threatening situations require a collective effort and a multidisciplinary approach (18). Drills in general have been found to be useful in testing team functioning, and in rating practical skills, and performance during management of emergencies (10,19). To the best of our knowledge, only one other study has evaluated the functioning of the medical and paramedical teams in their actual practice setting (11). Other studies had at least 50 percent of the participants train in their own units, on their own labor wards; however, all of these courses trained part of the units and not the whole hospital system (8,9).
Obstetric emergency drills were useful at highlighting problems at the individual and system level that may otherwise have not been noted. This qualitative study demonstrated the feasibility of such an approach. Identifying deficiencies and taking appropriate action before the occurrence of a true emergency could have a significant impact on maternal and infant health. Increased practitioner confidence and improvement in team functioning may be additional benefits of this approach. Further research to confirm our findings will require the administration of a larger number of drills to allow better quantitative analysis with adequate power. It will help caregivers to better understand the obstacles that have to be overcome when administering such drills, ongoing challenges, and whether good risk management would translate into better, safer health care.
The authors thank Rana Zarwi, Rawan Shayboub, Hazar Kobaya, and Hana Itani for their assistance in the administration of the drills; Roland Tannous for assistance with developing the drills; and Iman Charara and Saadeddine Itani for facilitating the implementation of the drills.
- 11The development of an eclampsia box and ‘fire drill’. Br J Midwifery 2000;9:492–496., , .
- 15American Academy of Family Physicians. Advanced Life Support in Obstetrics (ALSO). Leawood, Kansas: Author, 2000.
- 16American Academy of Pediatrics and American College of Obstetricians and Gynecologists. Diagnosis and management of preeclampsia and eclampsia. ACOG Practice Bulletin No. 33, 2002. Available at: http://www.acog.org/publications/guidelinesForPerinatalCare/.
- 17American College of Obstetricians and Gynecologists. Guidelines for Perinatal Care. 6th ed. Washington, DC: Author, 2007.