Impact of introducing Practical Obstetric Multi-Professional Training (PROMPT) into maternity units in Victoria, Australia

Authors


Abstract

Objective

To assess the introduction of Practical Obstetric Multi-professional Training (PROMPT) into maternity units and evaluate effects on organisational culture and perinatal outcomes.

Design

A retrospective cohort study.

Setting

Maternity units in eight public hospitals in metropolitan and regional Victoria, Australia.

Population

Staff in eight maternity units and a total of 43 408 babies born between July 2008 and December 2011.

Methods

Representatives from eight Victorian hospitals underwent a single day of training (Train the Trainer), to conduct PROMPT. Organisational culture was compared before and after PROMPT. Clinical outcomes were evaluated before, during and after PROMPT.

Main outcome measures

The number of courses run and the proportion of staff trained were determined. Organisational culture was measured using the Safety Attitude Questionnaire. Clinical measures included Apgar scores at 1 and 5 minutes (Apgar 1 and Apgar 5), cord lactate, blood loss and length of baby's stay in hospital.

Results

Seven of the eight hospitals conducted PROMPT. Overall about 50% of staff were trained in each year of the study. Significant increases were found in Safety Attitude Questionnaire scores representing domains of teamwork (Hedges’ g 0.27, 95% confidence interval [95% CI] 0.13–0.41), safety (Hedges’ g 0.28, 95% CI 0.15–0.42) and perception of management (Hedges’ g 0.17, 95% CI 0.04–0.31). There were significant improvements in Apgar 1 (OR 0.84, 95% CI 0.77–0.91), cord lactates (odds ratio 0.92, 95% CI 0.85–0.99) and average length of baby's stay in hospital (Hedges’ g 0.03, 95% CI 0.01–0.05) during or after training, but no change in Apgar 5 scores or proportion of cases with high blood loss.

Conclusion

PROMPT can be introduced using the Train the Trainer model. Improvements in organisational culture and some clinical measures were observed following PROMPT.

Introduction

Practical Obstetric Multi-professional Training (PROMPT) is one of a number of training programmes developed to reduce adverse neonatal and perinatal outcomes through multi-professional training aimed at improving communication and teamwork.[1] Implementation of the programme in the UK has resulted in significant improvements across a range of outcomes.[1-3] Typically the programme involves a full day of hospital-based training comprising short lectures and scenario-based simulation training in obstetric emergencies, by trainers drawn from the hospital's staff. The programme has been introduced in many hospitals in the UK and in other countries (such as the USA), in most cases using a ‘Train the Trainer’ model.[4] Under this model a small group of trainers from each hospital is given 1 day of training and teaching materials and then expected to conduct training in their own hospitals.

PROMPT was selected to address a gap in the training available in maternity units in Victoria. It was selected as there was sufficient evidence, at least in South-West England, to show its cost effectiveness and sustainability by delivering training in the maternity unit by staff from within a maternity unit.

A number of reviews have highlighted the need for objective evaluation of obstetric emergency training courses.[5, 6] The introduction of such courses and the training of maternity staff comes at a significant cost, and not all training has been associated with improvements in outcomes.[1] Despite the widespread introduction of PROMPT internationally, to the best of our knowledge there is no published evidence regarding its effectiveness outside the UK, including both clinical outcomes and improvements in organisational culture.

The aim of this paper was to investigate overall changes in organisational and clinical outcomes across all sites before and following PROMPT training in Victoria.

Methods

Ethics approval for the project was obtained through the Eastern Health Ethics Committee in February 2010.

Eight hospitals were chosen to participate in a pilot project to introduce PROMPT into Victorian hospitals. Hospitals are referred to as Site 1 to Site 8. The sites were specifically selected to ensure representation of rural, regional and metropolitan hospitals. A 1-day training course (‘Train the Trainer’) was conducted by an experienced faculty in early 2010. It was attended by representatives from each of the eight participating hospitals. Each hospital was invited to send four clinical teachers, an anaesthetist, an obstetrician and two midwives. These participants were trained to run the PROMPT course. They were then provided with teaching materials with which to conduct courses within their own hospitals. The teaching materials supplied included prepared presentations and material for conducting scenario-based simulation training. Each site was expected to train all their maternity unit staff once in each of the 2 years of the project (until the end of 2011). The CEO at each health service signed a Memorandum of Understanding agreeing to establish and run PROMPT courses for the 2 years of the study. The project manager and a management committee provided support and encouragement to the hospitals and staff from the co-sponsor of the project, Victorian Managed Insurance Authority, the hospital insurer, also provided encouragement to individual hospitals.

PROMPT lectures and scenario-based drills covered teamwork, maternal collapse including basic life support, maternal cardiac arrest and advanced life support, breech presentation, eclampsia, postpartum haemorrhage and shoulder dystocia. As hospitals taking part in this project varied in size, location (regional versus metropolitan) and maternity tasks handled, training modules were slightly modified at each hospital to cover specific requirements.

Rollout of the project was monitored by recording the time taken from the initial Train the Trainer course to conducting the first PROMPT course at each site, number of courses conducted and percentage of staff trained. Retention of staff trained at Train the Trainer courses was also monitored.

The Kirkpatrick model[7] was used to assess the effectiveness of PROMPT training. The four levels of Kirkpatrick and Kirkpatrick's evaluation model essentially measure—Level 1: Reaction (to what degree participants react favourably to the training), Level 2: Learning (to what degree participants acquire the intended knowledge, skills, attitudes, confidence and commitment based on their participation in a training event), Level 3: Behaviour (to what degree participants apply what they learned during training when they are back on the job), and Level 4: Results (to what degree targeted outcomes occur as a result of the training event and subsequent reinforcement).

To assess the first level of the Kirkpatrick model (Reaction), staff satisfaction with local training was assessed using a Training Evaluation Questionnaire. Questionnaires were distributed to participants at the end of each training session, structured around the obstetric emergency topics covered. The questionnaire included statements about each of the topics (e.g. ‘the lecture was relevant to me’ or ‘the content was about right’). A five-point Likert scale was used to ask participants about the extent to which they agreed or disagreed with the statements (strongly agree, agree, no opinion, disagree and strongly agree).

To address the second level of the Kirkpatrick model, a Safety Attitude Questionnaire (SAQ) was implemented, designed to assess whether training enhanced organisational culture through a positive effect on patient safety attitudes and staff morale.[8] The SAQ assessed staff attitudes through six climate scales (referred to here as ‘scales’): teamwork; safety; job satisfaction; perceptions of management; working conditions; and stress recognition.[8] The SAQ also includes questions on the respondents’ demographics.

The SAQ has demonstrated good psychometric properties and has been recommended as a tool to measure caregiver attitudes, to promote interventions to improve safety attitudes and to measure the effectiveness of these interventions.[8] Staff at hospitals participating in this project completed the Labour and Delivery version of the SAQ before and following training (in February 2010 and then 2011). Items in the SAQ use a five-point Likert scale (ranging from disagree strongly to agree strongly). For each of the six scales, a 100-point scale score is calculated.[8]

Levels three and four of the Kirkpatrick evaluation model focus on behaviour and results, and in this study were assessed by improvements in clinical outcomes.

Patient outcomes were evaluated by the following clinical measures:

  1. One-minute Apgar Scores (Apgar 1): Percentage of Apgar 1 scores <7.
  2. Five-minute Apgar Scores (Apgar 5): Percentage of Apgar 5 scores <7.
  3. Postpartum haemorrhage: Blood loss values > 1500 ml.
  4. Baby's length of stay in hospital following birth
  5. Cord lactate: Abnormal cord lactate defined as the percentage of cord lactate levels above 5.27 mmol/l, corresponding to the 75th percentile of the distribution of the aggregated data. Assessment of lactate at time of birth may help in determining the risk of brain damage.[9]

The clinical measures were compared in the following time periods:

  1. Pretraining: July 2008 to December 2009
  2. Training period: January 2010 to December 2010
  3. Post-training: January 2011 to December 2011

The clinical records were obtained from the electronic maternity data-base used in each maternity unit. The following exclusions were applied:[1] still births, babies born before arrival at the hospital, babies delivered by elective caesarean section, multiple pregnancies and preterm babies (gestation <37 weeks).

Data analysis

To assess responses from the training evaluation questionnaires, distribution of responses (percentage of participants who agreed, disagreed, etc.) were calculated. For the SAQ, differences in scale scores were assessed using one-tailed t tests to assess whether there were increases in scores following training. Further comparisons of SAQ scale scores before and following training were performed using ‘percentage agreement’ scores.[10] The ‘percentage of respondents’ with a scale score of 75 or higher constitute the ‘percentage agreement’ (i.e. agree slightly or strongly agree). Respondent demographics were compared before and after PROMPT using Wilcoxon rank-sum and chi-square tests.

Ordinary least squares and logistic regression were used to test differences in clinical measures at the different time periods. Changes in clinical measures over time were also examined. To do this the data were aggregated to monthly summaries. These were modelled using standard ordinary least square regression, adjusting for birth months. To account for any within-hospital correlation all models were adjusted using robust standard errors (clustering on hospitals). Results were considered significant at < 0.05. Measures of effect size (Hedges’ g or odds ratios) and 95% confidence intervals were calculated for comparisons. Statistical analyses were performed using Stata version 12.1 (StataCorp, College Station, TX, USA) and Matlab (Release 2012b; The MathWorks, Inc., Natick, MA, USA).

Results

Results presented indicate outcomes from data aggregated across all participating hospitals. One participating site (Site 5) completed the SAQ according to the study protocol at two time points but did not complete any training during the period specified. The clinical data from this site were therefore excluded from analysis and only SAQ data are presented.

Introduction of PROMPT

Time taken to conduct first PROMPT course

There was considerable variation in the time it took each hospital to run its first course following the Train the Trainer. As shown in Figure 1, the first course at Site 4 was conducted the day after the Train the Trainer course, making use of assistance from the experienced staff who had conducted the Train the Trainer. Other sites took up to 186 days before conducting their first course. One hospital (Site 5) did not conduct a course at all.

Figure 1.

Days from training (Train the Trainer) until first in house PROMPT training day.

Courses conducted and staff trained

The number of courses conducted at each site is shown in Table 1. PROMPT was conducted as a full-day course at all sites, apart from Site 7 (a regional hospital), where it was conducted as two half days, to accommodate the staff from a very small maternity unit.

Table 1. Trained staff at Train the Trainer from each site at the beginning and end of 2010 and courses conducted and staff trained in 2010
SiteTrain the TrainersCourses and staff trained
Number of staff in initial Train the TrainersNumber of staff involved in PROMPT at the end of 2010Number of staff initially trained but no longer involved in PROMPT at the end of 2010Training days plannedActual training daysPercentage of staff trained
18914461
24706685
3513155100
48915575
5451200
64515210
71102 × 1/22 × 1/275
871306421

Overall, 51% of all possible staff were trained in the first year of the programme. As seen in Table 1, two hospitals trained over 85% of their staff, and a further three trained between 60% and 75% of their staff. Three hospitals trained <20% of their staff.

Retention of staff trained at the Train the Trainer course

At the start of the project, it was anticipated that training would be conducted by staff who attended a Train the Trainer course and others who did not. This was the case throughout the project, with the latter group comprised of clinicians who became involved with the project after the initial Train the Trainer course and were provided with the trainer material but did not attend a course. This group comprised 44% of the trainers with 56% having attended a Train the Trainer day.

Table 1 shows the number of staff from each site that were trained in the original Train the Trainer day, the number of staff actually involved in facilitation of PROMPT training days at the end of 2010, and the number of staff who had been trained and were no longer involved with PROMPT training days. Overall the retention of staff trained at the Train the Trainer course was satisfactory and an increase in total number of trainers was seen throughout 2010.

The training evaluation questionnaire

Results from the training evaluations demonstrated very high participant satisfaction with the training programme. For each training topic (e.g. maternal cardiac arrest or advanced life support), similar response distributions for each of the statements were found. Results for the last item ‘session should remain’ are therefore presented to reflect overall feedback to that topic. Over 33 training days conducted across all sites, over 95% of participants strongly agreed or agreed that each session should remain.

The questionnaire contained an ‘overall’ section, which included a statement on recommending the training to their colleagues (‘I would recommend it to my colleagues’). At one site <1% of respondents disagreed that they would recommend the training. All other participants strongly agreed or agreed with the statement.

The staff attitude questionnaire

Respondent demographics

In total, 933 staff completed the SAQ at the two time-points. Before training, a total of 432 surveys were handed out across all sites with an average response rate of 47.6%. After training, 501 surveys were returned, resulting in a response rate of 45.9%. Of the total returned surveys ten pretraining and 18 post-training were excluded because respondents worked at multiple sites and it was not clear to which site their responses referred. It should be noted that respondents completing the survey at the two time points were not necessarily the same people.

Comparison of respondent demographics before and after training showed no significant difference in respondents’ distribution of age, gender, usual shifts worked, job status or first language. Staff who completed the survey before training had more experience in their specialty than those who completed the survey after training was rolled out—mean (SD): 20.82 (13.03) versus 13.03 (10.88) years; P < 0.001. They had also worked at their current hospital for longer than those who completed the SAQ after PROMPT rollout—mean (SD): 22.42(12.79) versus 8.69 (9.14) years; P < 0.001.

SAQ scale scores

Results from comparison of mean SAQ scale scores from seven sites are shown in Table 2. Data from Site 5 are presented separately because it did not perform PROMPT training as planned and can be viewed as a control site. There was a significant increase in teamwork, safety and perception of management scores following training at the sites where PROMPT was performed but not at Site 5.

Table 2. Mean (SD) and effect size (Hedges’ g) of SAQ scales before and following PROMPT for seven participating sites and the site where no training was performed
Scale7 Sites (PROMPT conducted)Site 5 (no PROMPT conducted)
Scores pretraining mean (SD)Scores Post-training mean (SD)Hedges’ g Effect size (95% CI)Scores pretraining mean (SD)Scores post-training mean (SD)Hedges’ g Effect size (95% CI)
  1. Differences assessed using one-tailed t-tests.

  2. a

    < 0.05.

  3. b

    < 0.01.

  4. c

    < 0.001.

Teamwork67.1 (13.4)71.1 (16.1)b0.27 (0.13–0.41)71.8 (12.5)70 (15.9)−0.12 (−0.54 to 0.31)
Safety66.2 (13.3)70.3 (14.6)c0.28 (0.15–0.42)69.4 (14.4)69.9 (13.7)0.03 (−0.40 to 0.45)
Stress recognition71.4 (20.2)70.8 (19.4)−0.03 (−0.17 to 0.10)71.6 (18.7)66.6 (25.3)−0.21 (−0.64 to 0.22)
Working conditions60.2 (20.5)61.1 (21.8)0.04 (−0.09 to 0.18)69.2 (14.6)63.9 (15.1)−0.35 (−0.77 to 0.08)
Job satisfaction69.4 (17.5)69.7 (18.7)0.01 (−0.11 to 0.15)77.8 (13.6)71.5 (20.5)−0.34 (−0.77 to 0.09)
Perception of management51.8 (19.4)55.3 (21.1)a0.17 (0.04 to −0.31)64.8 (17.8)61.8 (16.5)−0.17 (−0.60 to 0.25)

Further comparisons of SAQ scale scores before and after training were performed using ‘percentage agreement’ scores because these have been used in previous studies.[8] The percentages of respondents with a scale score of 75 or higher constitute the ‘percentage agreement’ (i.e. agree slightly or strongly agree). Similar to the results found above, post-training respondents were more likely to score above 75 in the teamwork (χ² = 12.481, P < 0.001), safety (χ² = 15.998, P < 0.001) and perception of management scales, than those who completed the survey before training (χ² = 7.962, P = 0.005).

Clinical outcomes

Results from clinical data aggregated across seven sites are presented. Data from Site 5 were not included because no training was performed during the period specified. A total of 50 905 births were identified. Following the exclusions applied, a total of 18 364 births in the pretraining period, 12 586 during training and 12 458 in the post-training period were identified.

Table 3 summarises the average measures during each evaluation period. The average monthly changes assessed using fitted regression lines, are shown in Table 4.

Table 3. Comparison of average clinical measures before, during and after training
 Pre vs duringPre vs postDuring vs post
Odds ratio (95% CI) P Odds ratio (95% CI) P Odds ratio (95% CI) P
% Apgar 1 <70.91 (0.83–0.98)0.017a0.84 (0.77–0.91)<0.001b0.92 (0.84–1.01)0.087
% Apgar 5 <70.96 (0.8–1.16)0.6941.05 (0.88–1.26)0.5731.09 (0.89–1.34)0.380
% Cord lactate > 5.270.99 (0.92–1.06)0.7740.92 (0.85–0.99)0.028a0.93 (0.86–1.01)0.07
% Blood loss > 1500 ml0.98 (0.79–1.2)0.8411.1 (0.89–1.35)0.3701.12 (0.9–1.4)0.316
 Hedges’ d effect size (95% CI) P Hedges’ d effect size (95% CI) P Hedges’ d effect size (95% CI) P
  1. Logistic regression was used to test Apgar 1, Apgar 5, cord lactate and blood loss. Ordinary least squares regression was used to test babies’ length of stay.

  2. a

    < 0.05.

  3. b

    < 0.01.

Baby length of stay (days), mean (SD)0.03 (0.01–0.05)0.006b0.02 (−0.01 to 0.04)0.156−0.01 (−0.04 to 0.01)0.250
Table 4.  Difference between coefficients of fitted regression lines (showing average monthly change in clinical measures) between evaluation periods. 95% CI of difference is also shown
 Pre vs duringPre vs postDuring vs post
Difference between mean coefficients (95% CI) P Difference between mean coefficients (95% CI) P Difference between mean coefficients (95% CI) P
  1. Logistic regression was used to test Apgar 1, Apgar 5, cord lactate and blood loss. Ordinary least squares regression was used to test babies’ length of stay.

  2. a

    < 0.05.

  3. **< 0.01.

Apgar 1 <7 (percentage/month)0.173 (−0.02 to 0.37)0.0860.16 (−0.01 to 0.33)0.058−0.01 (−0.22 to 0.21)0.93
Apgar 5 <7 (percentage/month)0.04 (−0.12 to 0.10)0.129−0.00 (−0.06 to 0.07)0.957−0.04 (−0.11 to 0.03)0.272
Cord lactate >5.27 (percentage/month)−0.38 (−0.76 to −0.01)0.044a−0.17 (−0.55 to 0.21)0.3580.21 (−0.18 to 0.61)0.134
Blood loss> 1500 ml (percentage/month)0.00 (−0.04 to 0.05)0.857−0.04 (−0.11 to 0.03)0.273−0.04 (0.11 to 0.03)0.481
Baby length of stay (hr/month)−0.13 (−042 to 0.16)0.361−0.31 (−0.79 to 0.17)0.186−0.18 (−0.71 to 0.35)0.324

Apgar 1

Before PROMPT 9.1% of cases had Apgar 1 scores <7. This was significantly reduced to 8.3% during the training period. Following training, a further reduction to 7.7% was seen; however, this did not reach significance.

Before PROMPT the rate of reduction in low Apgar 1 scores was greater than the other two intervention periods (Figure 2A). Overall, differences between the slopes for the three evaluation periods were not significant.

Figure 2.

Average monthly values of clinical measures: (A) percentage Apgar 1 scores <7; (B) percentage Apgar 5 scores <7; (C) percentage cord lactate values > 5.27 mmol/l (i.e. 75th centile, which is used as threshold); (D) percentage blood loss > 1500 ml; (E) babies’ length of stay (days) in hospital.

Apgar 5

Before PROMPT 1.5% of cases had Apgar 5 scores <7. This percentage remained unchanged during training but increased to 1.6% following training. Differences between evaluation periods were not significant. The rate of decrease in low Apgar 5 scores was similar in the three periods (Figure 2B).

Cord lactate

Before PROMPT, 25% of cases had cord lactate values > 5.27 mmol/l. This was significantly reduced to 24.7% during and 23.4% after training.

Before training, there was no discernible variation in the average monthly percentage of high cord lactate values (Figure 2C). This was significantly reduced during the training period.

Postpartum haemorrhage

As seen in Figure 2D, there was no significant change in percentage of cases with high blood loss in the three evaluation periods (1.2% before and during training and 1.3% after training).

In the periods before and during training there was a similar increase in the monthly changes (Figure 2D). Post-training, however, this was reduced. Differences between the slopes in the three evaluation periods were not significant.

Length of baby's stay in hospital

A significant reduction in length of stay was seen during training (mean [SD]: 2.79 [1.55] days) compared with pretraining (2.85 [1.55] days). Differences with the post-training period were not significant (2.82 [1.55] days) (Figure 2E). Post training there was a steeper reduction in average monthly length of stay compared with the pretraining period as well as during training; however, the difference did not reach significance.

Discussion

Main findings

The multi-professional training programme PROMPT has previously shown improvement in neonatal clinical outcomes.[1] This study supports previous findings and in addition shows improvements in staff culture as assessed by the SAQ. The incidence of infants born with Apgar 1 scores <7 significantly decreased during the year in which PROMPT was rolled out. This was then further reduced in the year following training. The proportions of babies with high cord lactates were also significantly reduced in the post-training period. The lengths of stay of babies in hospital were significantly reduced during the training period. Results from the SAQ showed significant improvement in staff attitudes toward safety, teamwork and perception of management following PROMPT.

Strengths and limitations

It was anticipated that all staff in the participating hospitals would be trained in each of the 2 years of the project. However, in practice about 50% of staff in the participating hospitals were trained. Nevertheless, this study demonstrates significant improvements in safety culture and in some clinical outcomes. It appears that improvements can be seen even when a relatively small proportion of staff are trained.

It has previously been demonstrated that PROMPT training results in increased knowledge in managing maternity emergencies among participants.[11] Assessment of knowledge acquisition was excluded from this study because it was felt that this would assist in defining PROMPT as a learning experience rather than an assessment process, and would in turn encourage participants’ acceptance of the program. However, this could potentially be included to complement evaluation of the course in a future study.

The effects of PROMPT on staff attitudes had not previously been investigated. A limitation, however, was that not all participants who completed the questionnaire in the post-training period had attended PROMPT and they were a younger group with less experience compared with staff in the pretraining period. We chose to include data from these participants in the analyses because our aim was to investigate overall organisational outcomes following PROMPT training.

Limitations of the evaluation of clinical results are that comparisons were made between the time period before training and then during and after training with training occurring over the 2 years of the project and not at a particular single time point. Also, changes may have been due to factors other than the training programme, of which we were unaware. Background secular trends may also have been present in the data, although we believe the inclusion of data from all seven sites that conducted PROMPT would reduce the likelihood of such an effect on the results. Data over longer periods before the training and interrupted times series methods would be required to investigate these affects more accurately.

Interpretation

Our results show significant improvements in staff attitudes toward safety, teamwork and perception of management. Although the effect sizes reported were small, for teamwork and safety these were interpreted as educationally significant.[12] We believe this is an indication of the effectiveness of PROMPT because these are two areas of focus within the course.

The safety score in the SAQ is an indication of perceptions of a strong and proactive organisational commitment to safety and has been associated with lower risk-adjusted patient mortality (J. B. Sexton, unpbl obs.) and fewer medication errors.[13] The teamwork score in the SAQ represents the perceived quality of collaboration between staff members. Deficiencies in teamwork have been reported as a factor in the mismanagement of obstetric emergencies[14] with increased knowledge shown to improve management of these cases.[11] The increase in teamwork scores observed here is therefore likely to result in improved clinical outcomes.

Perception of management represents overall staff attitudes toward managerial action. Although perception of management is influenced by a wide range of factors, rollout of a training course with which the vast majority of staff were satisfied and that they felt they needed (as indicated by the evaluation questionnaires) could strongly influence their perception of managerial action.

Scores for job satisfaction, stress recognition and working conditions did not show significant improvements when measured after training. As PROMPT by nature did not specifically address these topics, we did not expect significant changes in staff attitudes on these scales.

The SAQ scores from the hospital at which no training was organised showed no significant changes in the second set of questionnaires completed. The similarity of the SAQ results before and after the rollout of PROMPT appeared to further emphasise the effectiveness of training on improvements in scale scores at the other sites.

The SAQ has previously been used to identify attitudes in a maternity team in the UK where PROMPT had been in place for a number of years.[15] Although scores in teamwork and training in this study were lower (5% and 3.7%, respectively), they were within 0.4 standard deviations of those reported by Siassakos et al.[15] In contrast, the score in perception of management after training was 8.3% higher and within 0.5 standard deviations of their score.

Clinical outcomes were also assessed as part of the evaluation model used.[7] Improvements in Apgar 1 scores, cord lactate values and length of stay of babies in hospital were found. In contrast to the reduction in Apgar 1 scores, the proportion of babies with low Apgar 5 scores remained constant throughout the evaluation periods. Apgar scores at 1 minute and 5 minutes after the birth of a baby are believed to be predictors of subsequent neurological disability.[16, 17] This association is generally agreed to be stronger with Apgar 5 scores than with Apgar 1 scores. However, it has been demonstrated that Apgar scores at 1 minute, but not at 5 minutes, were related to neurological evaluation at 24 hours.[18, 19] This indicates that infants with low Apgar 1 scores may recover at 5 minutes but this does not improve neurological evaluation at 24 hours after birth. We would therefore contend that the improvement in Apgar 1 scores is clinically important.

Our results in relation to low Apgar 5 scores were in contrast to results previously reported by Draycott et al.,[1] which showed a reduction in this proportion. One reason for this could be the initially smaller population of babies with low Apgar 5 scores (1.5%) compared with those with low Apgar 1 scores (8%), which already show improvements. We did observe that changes in clinical measures occurred at different times relative to the rollout of training (i.e. during or after training), so it is possible that with more time, changes would be observed in Apgar 5. A review of the active components of effective obstetric emergency training programmes has shown high participation rates with regular training to be important factors in improving outcomes.[20] Training a higher number of staff than the 50% achieved as well as more regular training days may therefore be needed to achieve a reduction in low Apgar 5 scores comparable to those previously reported.[1] Modified training modules may also have affected outcomes. The content used in this course was not identical to that used in the UK in that fetal surveillance education was not included, because this was delivered separately.

The slight increase in the percentage of cases with high blood loss in the post-training period in the year of the introduction of the programme could be explained by an increased awareness of blood loss and a greater acceptance of the value of accurate measurement of blood loss following training. It is well known that blood loss estimation following childbirth is inaccurate and the amount is often under-estimated.[21] These facts are taught in the training programme and it is possible that the increased reported blood loss is the result of better reporting rather than an actual increase. This is consistent with the observation that the rate of increase appeared to slow in the period after training, with the trend in the fitted regression line reversed from a positive slope to a slight negative slope indicating a slow reduction in average monthly blood loss values.

Although positive, improvements in clinical outcomes are relatively modest compared with those previously published[1] and effect sizes for changes in staff attitudes are small.[22] This may be explained by the fact that only a proportion of staff were trained and it is possible that a stronger effect would be seen with ongoing training of a larger number of staff. We believe that this association, as well as previous work showing the effectiveness of PROMPT,[1] provides some evidence of a likely causal relationship between training and improvements in safety culture and clinical outcomes according to Hill's criteria for causation.[23] We acknowledge that further work is required to better establish this causal association and its ongoing effectiveness. We believe for the programme to be most efficacious, methods need to be identified that improve the uptake of training. Training is a significant expense and it is incumbent on health services to continue to evaluate its effectiveness.

Conclusion

The results presented here suggest that PROMPT training can be introduced using the ‘Train the Trainer’ model and, in addition to improving clinical outcomes, can also improve staff attitudes.

Disclosure of interests

All authors declare that they have no conflict of interest related to this study.

Contribution to authorship

MS was involved in the execution of the research project as well as the design and execution of the statistical analyses. She wrote the first draft of the manuscript and was involved in its review. MB was involved in the conception, organisation and execution of this project. He was involved in the writing and review of the manuscript. FM was involved in the organisation and execution of the research project. She also reviewed the manuscript. JF was involved in the design, and execution of the statistical analyses and review of the manuscript.

Details of ethics approval

Ethics approval for the project was obtained through the Eastern Health Ethics Committee in February 2010 (reference LR48/0910). This approval was also processed through the ethics committees of each of the participating hospitals.

Funding

This project was funded by the Victorian Managed Insurance Agency (VMIA) and Eastern Health. VMIA is a Victorian government statutory authority, which provides insurance products and risk management services to the State Government of Victoria, including Public Hospitals.

Acknowledgements

We acknowledge the encouragement and support of Tim Draycott and the PROMPT foundation as well as the staff at all participating hospitals.

Simulation training needs to be adequate to improve clinical outcomes

  • P Øian,a,b G Acharyaa,b

  • aWomen's Health and Perinatology Research Group, Department of Clinical Medicine, University of Tromsø, Tromsø, Norway bDepartment of Obstetrics and Gynaecology, University Hospital of Northern Norway, Tromsø, Norway

Mini commentary on ‘Impact of introducing Practical Obstetric Multi-Professional Training (PROMPT) into Maternity Units in Victoria, Australia’

Modern obstetric practice in developed countries is generally safe with good results. Although serious complications are rare, they can lead to death or permanent injury to the mother and/or child. Several studies have shown that many of these complications and their sequelae could have been avoided (Andreasen et al. Acta Obstet Gynecol Scand 2012;91:1191–5; Berglund et al. BJOG 2010;117:968–78). Medico-legal claims related to negligent intrapartum and postpartum care make up the majority of the National Health Service annual litigation bill in the UK, and the same is true for many other countries. Audits of maternal and perinatal deaths demonstrate negligence or substandard care in up to 50–70% of cases (Cantwell et al. BJOG 2011;118 Suppl. 1:1–203). This is often associated with lack of knowledge, clinical skills or proper communication among the healthcare personnel.

Intuitively one would think that multi-professional simulation-based training must be the way to improve patient safety and clinical outcomes. Several studies have shown better knowledge and skills after training in the simulation centres and course venues, but when it comes to improved downstream patient care practices and clinical outcomes the evidence is limited because there are few studies that have reported on clinical outcomes (Reeves et al. J Interprof Care 2010;24:230–41).

Shoushtarian et al. have studied the effect of PROMPT after it has been introduced in eight hospitals in Victoria, Australia. A total of 43 408 deliveries have been studied during the pretraining, training and post-training periods. Improvements were noted in staff attitudes towards safety, teamwork and perception of management (Safety Attitude Questionnaire scores), but the effect sizes were small. Regarding clinical outcomes, a reduction in 1-minute Apgar score of <7 at and cord lactate >5.27 mmol/l was observed but no difference in Apgar score of <7 at 5 minutes and blood loss >1500 ml was found.

Observational studies like this have inherent weakness in demonstrating the cause-and-effect phenomena because of the issue of confounding. However, there are other problems too. Those who would conduct the PROMPT courses received a single day ‘Train-the-Trainers’ course, but it is unclear how their knowledge and skills were validated. Nelissen et al. (Acta Obstet Gynecol Scand 2014;93:287–95) reported a pass rate of 3% for basic delivery and management of postpartum haemorrhage following a half-day simulation-based training. Furthermore, in the study by Shoushtarian et al. one of the hospitals did not conduct any simulation trainings, three hospitals trained <25% of the staff in the first year and in total only 2–6 days of training was provided in each hospital. The training scenarios included maternal cardiac arrest and advanced life support, breech delivery, eclampsia, postpartum haemorrhage and shoulder dystocia. In our opinion, the training time provided for the trainers was too short, the numbers of clinical scenarios too many, the numbers of training days in the hospitals too few, and only about 50% of the staff completed the training.

Are the modest results with regards to improvement in clinical outcome surprising or disappointing? We think not. We have to remember that the clinical outcomes depended on population characteristics, quality of antenatal care, intrapartum surveillance, theoretical and practical obstetric skills, communication skills, neonatal care facilities, and organisation of maternity and paediatric services. Hence, multi-faceted interventions are probably needed and multi-professional simulation training is one important element (Draycott BJOG 2006;113:177–82). The present study is a good start for the hospitals involved, but such training has to be a continuous process, refined and involving all the staff for a considerable time before improvement in important clinical outcomes can be expected. Further studies that report clinical outcomes should be encouraged.

Disclosure of interests

The authors have no conflict of interest to declare.

Ancillary