- Top of page
Please cite this paper as: Fransen A, van de Ven J, Merién A, de Wit-Zuurendonk L, Houterman S, Mol B, Oei S. Effect of obstetric team training on team performance and medical technical skills: a randomised controlled trial. BJOG 2012;119:1387–1393.
Objective To determine whether obstetric team training in a medical simulation centre improves the team performance and utilisation of appropriate medical technical skills of healthcare professionals.
Design Cluster randomised controlled trial.
Sample The obstetric departments of 24 Dutch hospitals.
Methods The obstetric departments were randomly assigned to a 1-day session of multiprofessional team training in a medical simulation centre or to no such training. Team training was given with high-fidelity mannequins by an obstetrician and a communication expert. More than 6 months following training, two unannounced simulated scenarios were carried out in the delivery rooms of all 24 obstetric departments. The scenarios, comprising a case of shoulder dystocia and a case of amniotic fluid embolism, were videotaped. The team performance and utilisation of appropriate medical skills were evaluated by two independent experts.
Main outcome measures Team performance evaluated with the validated Clinical Teamwork Scale (CTS) and the employment of two specific obstetric procedures for the two clinical scenarios in the simulation (delivery of the baby with shoulder dystocia in the maternal all-fours position and conducting a perimortem caesarean section within 5 minutes for the scenario of amniotic fluid embolism).
Results Seventy-four obstetric teams from 12 hospitals in the intervention group underwent teamwork training between November 2009 and July 2010. The teamwork performance in the training group was significantly better in comparison to the nontraining group (median CTS score: 7.5 versus 6.0, respectively; P = 0.014). The use of the predefined obstetric procedures for the two clinical scenarios was also significantly more frequent in the training group compared with the nontraining group (83 versus 46%, respectively; P = 0.009).
Conclusions Team performance and medical technical skills may be significantly improved after multiprofessional obstetric team training in a medical simulation centre.
- Top of page
In the UK, the Confidential Enquiry into Maternal and Child Health (CEMACH) identified substandard care in a large proportion of preventable maternal and perinatal deaths, ranging from 45 to 73%.1–3 The reports of both CEMACH and the Institute of Medicine (USA) have advocated the introduction of interdisciplinary team training.2,4 The underlying message from these two reports is that communication and coordination of the care team need to be improved.
Local training programmes on an individual basis already occur frequently. However, if the care team, not the individual, is responsible for most clinical errors, then team training might be more effective than training at an individual level. Such multiprofessional team training can easily be applied to obstetric emergency situations, because different healthcare professionals and disciplines work together in the labour ward. Grogan et al.5 already demonstrated that team training in crew resource management leads to a positive attitude of trainees towards dealing with fatigue, teambuilding, communication, recognising dangerous situations and decision making.
To evaluate these medical team training courses, Kirk-patrick’s theoretical model for the evaluation of training can be applied (Figure 1).6 It consists of four levels of training achievement with the bottom two levels referring to the reaction (satisfaction after training) and the improvement in knowledge of the trainee. Level three measures the implementation of learned skills and behaviour into clinical practice. The highest level relates to the effect of training on measurable clinical outcome(s).
Few articles evaluating the effect of teamwork training with simulation models have been published, and even fewer have examined objective measurements of improvement in the management of acute obstetric emergencies.7 There is only one retrospective study at the fourth level of Kirkpatrick, describing a significant effect of team training on perinatal outcome, using Apgar scores.8 At the third level of Kirkpatrick, Crofts et al.9 identified a significant effect of training individuals on clinical skills during an in situ simulation on shoulder dystocia. However, there are no randomised controlled trials evaluating the effectiveness of team training in obstetric emergencies with simulation methods at the third or fourth level of Kirkpatrick’s model.
As a consequence of the paucity of evidence, we performed a multicentre randomised controlled trial to investigate the effectiveness of team training in a medical simulation centre.10 The primary outcomes for this randomised controlled trial are based on maternal and perinatal outcomes, i.e. the fourth level of Kirkpatrick’s model. For this report we evaluated the clinical behaviour of the medical team more than 6 months after team-based training, i.e. level three of Kirkpatrick’s model. We investigated the hypotheses that obstetric team training in a medical simulation centre will improve the team clinical performance and increase the employment of essential clinical skills assessed using an unannounced clinical simulation.
- Top of page
The current study is part of a multicentre cluster design randomised controlled trial to assess the effectiveness of team training in a medical simulation centre.10 Eligible units were hospital-based obstetric departments in all teaching and nonteaching hospitals in the Netherlands with at least 1000 deliveries per annum. Teaching hospitals provide training for residents in obstetrics and gynaecology, whereas nonteaching hospitals do not provide such training. Obstetric units with an existing local multidisciplinary team training course could not participate. To prevent data contamination, the included units were not allowed to do any medical team training until the study period was ended. In addition, the results of the present study will not be published before the follow-up of the initial randomised controlled trial is completed. As this was a cluster randomised clinical trial allocating interventions at a group level, the institutional review board of the Máxima Medical Centre judged that consent from participating patients was not needed. The primary outcome of this randomised controlled trial was the number of obstetric complications during the first year after team training. In the current study we only report on team performance and medical technical skills more than 6 months after the training intervention.
The units were randomly allocated to the intervention and control groups using a computer-generated list that was stratified for teaching and nonteaching hospitals. Obstetric units allocated to the intervention group received a 1-day team training course in a medical simulation centre in Eindhoven. Eighty percent of the training time is given to crew resource management and 20% to medical technical skills. Each obstetric unit consisted of a number of multiple multiprofessional teams consisting of a gynaecologist, a midwife, a resident and two or three nurses. Each team was formed around a single gynaecologist. In total 74 teams, corresponding to the number of employed gynaecologists in the 12 units allocated to the intervention group, were included for training. Team training was delivered using high-fidelity mannequins by a gynaecologist and a communication expert. All the training instructors underwent an instructor training course for running simulation-based trainings, with emphasis on crew resource management. The birthing simulator Noelle™ (Gaumard, Miami, FL, USA) or the Emergency Care Simulator ECS™ (METI, Sarasota, FL, USA) were used in six obstetric emergency scenarios: fetal distress including cardiotocographic analysis, shoulder dystocia, postpartum haemorrhage, umbilical cord prolapse, eclampsia and resuscitation of a pregnant woman. These scenarios were based on national and international guidelines from The Dutch Society of Obstetrics and Gynaecology, Royal College of Obstetricians and Gynaecologists and (Managing Obstetric Emergencies & Trauma [MOET]) in the UK.11 Every scenario started with a briefing by an introductory video for approximately 5 minutes, where the clinical situation is performed by actors on a wide screen. After this introduction the team moved to the simulation delivery room where they were required to manage the simulated patient. All the scenarios, lasting approximately 15 minutes, were videotaped. After finishing each scenario, the team returned to the briefing room for a 30-minute debriefing session. Feedback on teamwork and the application of medical technical skills was provided by reviewing the relevant video recordings. Feedback on teamwork concentrated on components of crew resource management, i.e. communication, leadership, decision making and situational awareness. All individual teams from each hospital were trained within a time period of 4 weeks.
More than 6 months after completing these training courses, an unannounced in situ clinical simulation with two scenarios, shoulder dystocia and amniotic fluid embolism, was conducted at all obstetric units (in both the intervention and control group) recruited into the study. At the time of their training the participants were aware that an unannounced in situ simulation would take place more than 6 months later. Only a single person at each of the obstetric units was contacted in advance to organise the use of one of the delivery rooms in the unit for the purpose of the clinical simulation. All the on-duty individuals who were allocated to staff the delivery room participated in the simulation. For both scenarios a standardised script was used. The scenario started with an actress conducting a delivery. Because of difficulties related to the delivery, the (simulated) case was handed over to a clinical midwife or resident obstetrician. The intention was that participants were unaware of the simulated setting until they entered the delivery room. In the first scenario on shoulder dystocia, a hybrid high-fidelity simulation with the PROMPT™ birthing simulator (Limbs & Things, Bristol, UK) was used. The simulation continued until the baby was delivered in the all-fours position. If the all-fours manoeuvre was not applied, the baby was delivered anyway. Between the first and second scenario the participants left the delivery room, without knowing whether the in situ simulation was still ongoing. The next scenario on amniotic fluid embolism required the resuscitation of the pregnant woman. For this scenario we used a mannequin called Resusci Anne™ (Laerdal, Stavanger, Norway), which was modified into a pregnant woman. The simulation was terminated after delivery of the baby by a perimortem caesarean section or when the medical team failed to make any progress in the management of the scenario. If help was requested, one of the investigators could attend, but would not perform any clinical tasks unless requested to do so. The participants were encouraged to ask for help from colleagues as well. Both scenarios lasted a maximum of 10 minutes and were videotaped. To prevent any learning effect of the in situ simulation in the control group, no feedback on teamwork and medical skills was provided to participants in both groups after the simulation.
The Clinical Teamwork Scale (CTS) was used for assessment of team performance in a simulated obstetric emergency context.12 At the time of designing this study, it was the only validated rating scale designed specially for obstetric team performance. The CTS has been reported to have substantial agreement (Kappa 0.78) and score concordance (Kendall coefficient 0.95) between raters, and a high inter-rater reliability (interclass correlation coefficient 0.98). It has been developed based on the important topics in crew resource management. The CTS consists of 15 items in five different teamwork domains: communication, decision making, role responsibility, situational awareness/resource management and patient-friendliness. For all 15 items, a rating scale ranging from 0 to 10 was adopted except for the ninth item. The ninth item is about the presence of target fixation (yes or no). Target fixation exists when team members exhibit tunnel vision that prevents progress from being made in the management of the scenario.12
We also investigated whether medical team training can lead to the acquisition of medical technical skills. To assess this, we ascertained whether a prespecified, recommended but unfamiliar obstetric procedure was employed in each of these obstetric emergencies. For the scenario on shoulder dystocia, we have chosen the manoeuvre in which the baby was delivered in an all-fours maternal limbs position (all-fours manoeuvre). Although this manoeuvre is recommended as a useful manoeuvre in several national guidelines on shoulder dystocia (e.g. Dutch Society of Obstetrics and Gynaecology, Royal College of Obstetricians and Gynaecologist) and international courses (MOET; Managing Obstetric Emergencies & Trauma) many obstetric teams are not familiar with it.11,13–15 In the second scenario a perimortem caesarean section had to be carried out within 5 minutes. Many obstetric teams have little experience with this procedure, because of the low incidence of resuscitations of pregnant women.
The ascertainment of whether both outcome measures were present in the scenarios was conducted by an expert panel of two trained assessors. They were not involved in the training nor in performing the in situ simulations. Both were obstetricians who were familiar with the principles of crew resource management and the use of CTS. Although the study participants were aware of the arm of the study to which they were allocated, the individuals in the expert panel were kept blinded to the study allocation.
The calculation on the power of the study was based on the primary outcome measures of this study, i.e. perinatal and maternal outcomes throughout the first year after intervention, and not on the secondary outcomes for the purpose of this report. The perinatal and maternal complications are defined as the number of neonates with perinatal asphyxia (5-minute Apgar score < 7 or arterial umbilical pH < 7.05), hypoxic ischaemic encephalopathy, number of neonates with damage caused by shoulder dystocia (e.g. lesion of brachial nerve plexus, clavicle fracture), number of women with eclampsia, number of women with severe postpartum haemorrhage (more than four blood transfusions of packed cells, embolisation, hysterectomy).10 We expected the incidence of adverse perinatal and maternal outcomes in the control group to be 15%. Based on this assumption, we calculated that a sample size of 24 hospitals with at least 200 deliveries per year was needed to give a power of 80% to detect a reduction in adverse perinatal and maternal outcomes from 15 to 5% (using a two-sided type 1 error of 5%) in the intervention group.
Data management and analysis were performed using SPSS (Statistics 18.0; SPSS Inc., Chicago, IL, USA). The CTS scores were described using the median (range). To compare the differences in team performance between the intervention and control groups, the total median and median scores of each of the items of the CTS were calculated and compared using the Mann–Whitney U test. The frequency of application of the relevant obstetric procedures in the two scenarios was compared by using a chi-square test. When indicated, a Fisher’s exact test was used instead of a chi-square test. P-values ≤ 0.05 were considered to indicate statistical significance.
- Top of page
The obstetric departments in 36 hospitals were invited to participate in this multicentre cluster randomised controlled trial. Four hospitals declined participation and eight hospitals did not meet the inclusion criteria for this study. An overview of enrolment of the various obstetric departments is presented in the CONSORT flowchart (Figure 2). Eventually, 24 obstetric departments were included and randomly assigned to either team training in a medical simulation centre (n = 12) or to a control group (n = 12). In the training group, a total of 74 clinical teams were identified for training. The training and nontraining groups were comparable in relation to the number of teaching hospitals in each group, the total and mean number of deliveries per year and the staffing level (Table 1). All obstetric departments that were randomly assigned to the intervention group received the team training course. There were no hospitals lost to follow up in either the training or nontraining group.
Table 1. Baseline characteristics of training and nontraining groups
| ||Training group (hospitals; n = 12)||Nontraining group (hospitals; n = 12)|
|Teaching hospitals (n)||5||5|
|Total deliveries per year (n)||14 726||14 951|
|Mean deliveries per hospital||1230||1299|
The 24 hospitals were recruited into the study from January to November 2009. Participating clinical teams in the intervention group visited the simulation centre for training from November 2009 until July 2010. In situ simulations were performed 6–10 months after the study randomisation, with a mean interval of 8.27 months (8.43 ± 1.62 months for the intervention group and 8.12 ± 1.36 months for the control group; P = 0.6).
In each obstetric unit, in both the intervention group and the control group, the multiprofessional on-call team was targeted on the day the in situ clinical simulation was conducted. Each team participated in both the clinical scenarios. Consequently, the in situ simulations yielded a total of 48 video recordings for the assessment of team performance and medical technical skills.
Team training was associated with a higher overall median CTS score. The total median score of all items was significantly higher in the training group (median 7.5, range 2.0–8.5) than in the nontraining group (median 6.0, range 2.0–8.0; P = 0.01) (Table 2). Comparison of the median scores on the five separate teamwork domains of the CTS showed a statistically significantly difference in communication (P = 0.008) and decision making (P = 0.01) between the training and nontraining groups (Table 2). The differences between the remaining three domains of the CTS were not statistically significant.
Table 2. Comparison of team performance between training and nontraining groups, assessed by the CTS
| ||Training group Median (min–max)||Nontraining group Median (min–max)|| P-value|
| Total median score on the CTS ||7.5 (2.0–8.5)||6.0 (2.0–8.0)||0.014|
| Per item |
|1. Overall score||7.0 (2.0–8.0)||6.0 (1.0–8.0)||0.022|
|2. Overall communication||7.0 (3.0–8.0)||4.5 (2.0–8.0)||0.008|
|3. SBAR||7.0 (0.0–8.0)||4.0 (1.0–8.0)||0.007|
|4. Transparent thinking||7.0 (2.0–8.0)||5.0 (2.0–8.0)||0.015|
|5. Directed communication||7.0 (4.0–8.0)||5.0 (1.0–8.0)||0.002|
|6. Closed loop||7.0 (3.0–8.0)||4.5 (0.0–8.0)||0.002|
|7. Overall situational awareness||7.5 (2.0–9.0)||7.0 (1.0–8.0)||0.078|
|8. Resource allocation||7.5 (1.0–9.0)||6.5 (2.0–8.0)||0.220|
|9. Target fixation|| (See Table 3) || (See Table 3) ||0.666|
|10. Overall decision making||8.0 (2.0–9.0)||7.0 (2.0–8.0)||0.012|
|11. Prioritise||7.5 (2.0–9.0)||7.0 (1.0–8.0)||0.025|
|12. Overall role responsibility||8.0 (1.0–9.0)||7.0 (3.0–8.0)||0.118|
|13. Role clarity||8.0 (0.0–9.0)||7.0 (3.0–8.0)||0.082|
|14. Perform as a leader/helper||8.0 (1.0–9.0)||7.0 (3.0–8.0)||0.170|
|15. Patient friendly||7.0 (2.0–8.0)||7.0 (3.0–8.0)||0.448|
Table 3. Comparison of the presence of target fixation (item 9 of the CTS) between training and nontraining groups
|Item 9||Training group (n)||Nontraining group (n)||Total|
|No target fixation||22||20||42|
For the analysis of the quality of medical technical skills employed in the two clinical scenarios, only 45 of the 48 video recordings could be used because there was a technical problem within the simulated scenario in three cases. All these technical problems were encountered in the first scenario on shoulder dystocia with the PROMPT™ birthing simulation. Eventually, 23 video recordings from the 12 training units and 22 from the 12 nontraining units were available for analysis.
The required obstetric procedures (delivery of the baby in the all-fours maternal position for the scenario on fetal shoulder dystocia and the performance of a perimortem caesarean section within 5 minutes in the amniotic fluid embolism scenario) were performed in 19 of the 23 in situ simulations recordings (83%) in the trained units compared with 10 of the 22 recordings (46%) in the untrained units (P = 0.009). Nine trained teams used the all-fours manoeuvre compared with four nontrained teams (P = 0.08). Ten trained teams performed a perimortem caesarean section in comparison with six nontrained teams (P = 0.193).
- Top of page
In this multicentre randomised clinical trial we found a significant improvement in team performance and a significant increase in the use of new medical technical skills 8 months after obstetric, multiprofessional team training in a medical simulation centre. The training courses focused on both crew resource management (for 80%) and medical technical skills (for 20%).
Three previously reported studies have shown no benefit from isolated teamwork theory training on team performance.16–18 Nevertheless, the Institute of Medicine suggested that the training of medical teams might be an important contributing factor in the improvement of health care.4 Besides, in two of three studies only obstetricians and midwives were involved and it is also unclear whether the improvement of the teamwork was only applicable to specifically constructed teams or whether this effect would still be present when the composition of such teams was more random.17,18 For these reasons, we have deliberately designed our study using teamwork training, based on crew resource management, in combination with training of medical skills.
The training group had a significantly higher median and overall score on the CTS. However, it is clear that in particular ‘communication’ and ‘decision making’ are significantly better in the training group. This remarkable improvement could be explained by the well-defined nature of the sub-items concerning communication. These sub-items, i.e. SBAR (situation, background, assessment, recommendation), closed loop and directed communication, are well described and easy to understand. Therefore these items might be more easily implemented. In contrast, role responsibility and situational awareness are more dependent on the already existing behaviour of the care team. Changing this behaviour will comprise more than teaching new communication tools in a 1-day course and will remain a challenge.
The effect of team training on team performance and the appropriate employment of medical technical skills reported in this study is applicable to level three of Kirkpatrick’s model on the evaluation of training, i.e. the implementation of behaviour and skills in practice. Previous research conducted at a single centre has also reported a similar improvement in such skills.9,16 However, this earlier research included obstetricians and midwives, but allocated them to different roles from their own normal professional responsibilities. Furthermore, the trainees in one of these studies were tested only 3 weeks following training, which may result in the assessment of a temporary learning effect rather than a permanent change in clinical behaviour.9
It could be argued that the relatively small number of clinical teams tested during the in situ simulation is a limitation of our study. The composition of these teams (which is dependent on the on-call rota) could also have been different from those that underwent the team training. However, we assumed that simultaneous training of different healthcare professionals would lead to a change in work ethics, which is not dependent on the composition of a specific team. Our aim was to perform an unannounced clinical assessment, which would represent the real-life team performance and skills of the medical on-call team. Despite this potential limitation, we still demonstrated a significant improvement in the performance and skills of the trained units compared with the nontrained units. We believe that the assessment of the on-call team is a perfect representation of the real-life effect of the training on the multiprofessional staff and therefore should not be interpreted as a limitation.
Another limitation of our study might be the employment of the methods of evaluation of the acquisition of medical technical skills. Because of the paucity of validated rating scales to assess medical team skills available for obstetric emergencies, we decided to use the presence of predefined obstetric procedures in the emergency situations. It indicates the ability to learn recommended but unfamiliar medical procedures by team training. However, the development of a validated rating tool for the assessment of medical technical skills is a subject for future research.
Team training has been implemented in a wide spectrum of disciplines in medicine. Disciplines like anaesthesia, surgery, obstetrics and paediatrics are frequently involved in simulation-based (team) training research. Nevertheless, there was still a paucity of good-quality evidence for the effect of team training on Kirkpatrick’s level three and level four outcomes.19 Our study adds further evidence for level three outcomes of Kirkpatrick’s model. We expect that the results of our study could easily be applied to different disciplines. Further research should focus on the evaluation of the effect of training on measurable clinical outcomes.
Disclosure of interests
All authors declare that they have no financial, personal, political, intellectual or religious competing interests.
Contribution to authorship
SGO, BWM and SH were involved in conception and design of the study. AF and JV performed the in situ simulations. AERM and LDW-Z performed the assessment of the video recordings. AF and SH performed the statistical analyses. AF, SGO and BWM drafted the manuscript. JV, AERM, LDW-Z and SH reviewed the manuscript.
Details of ethics approval
As this was a cluster randomised clinical trial allocating interventions at group level and not at patient level, the institutional review board of the Máxima Medical Centre judged that consent on the level of the patient was not needed. We confirm that the institutional review board decided that an ethical approval was not needed for this trial design in the Netherlands.
This study is funded by Zon-MW, the Dutch, governmental organisation for health research and development, grant (170992303). All researchers are independent from this funder.