More to teamwork than knowledge, skill and attitude

Authors


Dr D Siassakos, Women’s Health, Chilterns, Southmead Hospital, Westbury on Trym, Bristol BS10 5NB, UK.
Email jsiasakos@googlemail.com

Abstract

Please cite this paper as: Siassakos D, Draycott T, Crofts J, Hunt L, Winter C, Fox R. More to teamwork than knowledge, skill and attitude. BJOG 2010;117:1262–1269.

Objective  To assess whether team performance in simulated eclampsia is related to the knowledge, skills and attitudes of individual team members.

Design  Cross-sectional analysis of data from the Simulation and Fire Drill Evaluation randomised controlled trial.

Setting  Six secondary and tertiary maternity units in south-west England.

Participants  One hundred and fourteen maternity professionals in 19 teams of six members; one senior and one junior obstetrician; two senior and two junior midwives.

Methods  We validated a team performance ranking scheme with respect to magnesium administration (Magnesium Administration Rank, MAR) by expert consensus (face validity) and correlation with clinical measures (construct validity). We tested for correlation between MAR and measures of knowledge, skills and attitudes.

Main outcome measures  Correlation between team performance (MAR) and scores in validated multiple-choice questionnaires (MCQs) (knowledge), a measure of individual manual skill to manage an obstetric emergency (skill) and scores in a widely used teamwork/safety attitude questionnaire (attitude).

Results  There was no relationship between team performance and cumulative individual MCQs, skill or teamwork/safety attitude scores.

Conclusions  The knowledge, manual skills and attitudes of the individuals comprising each team, measured by established methods, did not correlate in this study with the team’s clinical efficiency in the management of simulated eclampsia. The inference is that unidentified characteristic(s) play a crucial part in the efficiency of teams managing emergencies. Any emphasis of training programmes to promote individual knowledge, skills and attitudes alone may have to be re-examined. This highlights a need to understand what makes a team efficient in dealing with clinical emergencies.

Introduction

Obstetric emergencies are unpredictable and sudden. Successful management requires a rapid coordinated response by ad hoc multiprofessional teams. The need to provide training in team coordination and communication for clinicians has been repeatedly identified as a safety priority for developed countries.1–3

Eclampsia is an emergency that can result in severe complications for both mother and baby.4,5 The members of clinical teams that manage eclampsia might not have worked together in this context previously. The teams need to perform a number of key tasks, in particular the administration of magnesium sulphate, in accordance with national guidelines derived from confidential enquiries and systematic reviews of evidence.6–8. Magnesium administration for women with eclampsia is associated with a decrease in the recurrence of convulsions, a trend for lower maternal mortality and a significant reduction in serious maternal morbidity.8 For the baby, its administration is associated with a significant decrease in perinatal mortality and fewer admissions to special care baby units.8 Some teams are efficient in administering magnesium in this context; other teams are less efficient or never consider giving it.4,9–12

It appears that some clinical teams possess characteristics that make them more efficient than others, and so are better able to achieve good outcomes by performing key actions in a timely manner. This association between efficiency and overall team performance, as well as outcome, has been shown not only for simulated eclampsia,12 but also for simulated postpartum haemorrhage13 and real-life umbilical cord prolapse.14 Moreover, it has been shown that an inefficient response to other medical emergencies, in association with poor teamwork, is the root cause of many adverse outcomes, often leading to medical litigation.15

If the characteristics of efficient teams could be identified, the information could be used to inform training programmes. The aim of this study was to explore the relationship between team efficiency and team members’ knowledge, skills and attitudes (KSA).

Methods

Design

This is a cross-sectional analysis of data from a large randomised controlled trial of training for obstetric emergencies [Simulation and Fire Drill Evaluation (SaFE) study].13‘SaFE’ was a portfolio of studies commissioned by the Department of Health for England and Wales, the primary aim of which was to compare training at a simulation centre with training in local units, and clinical training alone versus clinical training with the addition of extra teamwork training. The methodology has been described in detail elsewhere.12,16–19 In this report, we focus on the results prior to the training process.

Participants

Participants were recruited to the study in 2004–5 from six large maternity units in the south-west of England.12 Participants were randomly selected from staff lists and allocated to 24 simulation teams (four teams from each unit). The individual teams were made up of staff from one unit, and each team comprised one senior doctor, one junior doctor, two senior midwives and two junior midwives. For the SaFE study, there were no data of training effectiveness on which to base power calculations.

Measurements

Knowledge, skills and attitudes (KSA)  All participants were evaluated individually for knowledge [using validated multiple-choice questionnaires (MCQs)],13,16 skill at managing an obstetric emergency (using simulated shoulder dystocia with a standardised scenario)19 and attitudes to safety and teamwork (using a validated questionnaire).13,20

The MCQ questions, written by expert midwives, obstetricians and obstetric anaesthetists, were based on evidence-based guidance and published literature. The maximum number of marks for the MCQs was 185, which included 30 marks specific to eclampsia (Appendix S1, see Supporting information).

For skills, one of the main aims of training was to improve both the accoucheur’s efficiency (to limit fetal hypoxia) and care (to reduce trauma). The multiprofessional research team looked at the SaFE study records and devised an ordinal score (rank) based on the skills and effectiveness of individual team members exhibited during shoulder dystocia drills. The ordinal score reflected recommendations from national confidential enquiries21 and guidelines,22 and correlated significantly with the maximum amount of force used by the participants (Kendall’s taub = 0.19, P = 0.016; Appendix S2, see Supporting information).

For attitudes, each team member completed a validated safety and teamwork attitudes questionnaire.20 It comprised 57 items and participants answered with a five-point Likert scale. The maximum score was 100, and higher scores indicated more positive attitudes.

Team performance  Teams were evaluated for their ability to manage simulated eclampsia with a standardised clinical scenario. The team evaluations were undertaken before the teams entered a training programme. The team members were not aware of the nature of the simulation before it started. The video recordings were evaluated with checklists derived from national recommendations.6–8 All evaluations were undertaken by two trained external assessors (doctor and midwife) working independently, who viewed the digital recordings in different sequences randomly generated by computer.

We regarded success in obtaining, preparing and administering magnesium sulphate as the most important observable and documented team action, and formulated a clinical efficiency ranking based on the administration of magnesium sulphate (Magnesium Administration Rank, MAR). To reach consensus and establish face validity of the ranking scheme, we used a ‘Delphi’ technique, modified for combined face-to-face meetings and email communication. Consensus was reached over four formal meetings by four clinical obstetricians, a research midwife and a statistician/methodologist. The teams were grouped, blind to their KSA scores to avoid bias, according to their performance as follows: (i) did not obtain magnesium; (ii) obtained but did not prepare magnesium; (iii) prepared but did not administer magnesium; (iv) administered magnesium but ≥6 minutes from the start of the drill; (v) administered magnesium <6 minutes from the start of the drill. The 6-minute cut-off was based arbitrarily on the frequency distribution of the teams’ recorded timings to obtain roughly equal group sizes.

Validation  The team MARs correlated with the ability to perform other key clinical actions for the management of eclampsia (Figure 1), confirming the construct validity of the scores. The team MARs also increased after training (Table 1, P = 0.035, two-tailed sign test), providing further supportive evidence of construct validity.

Figure 1.

 Correlation between Magnesium Administration Rank (MAR) and other measures of clinical team efficiency. (A) Lowering of head rest: higher magnesium scores (MAR) tended to be associated with shorter time intervals: Kendall’s rank correlation coefficient taub = −0.32, P = 0.09. (B) Administration of oxygen: higher magnesium scores (MAR) were generally associated with shorter time intervals: taub = −0.29, P = 0.12.

Table 1.   Validation of Magnesium (Mg) Administration Rank (MAR) as a team performance measure: comparison of team MARs before and after training. Statistically significant improvement after training confirmed the validity of the scheme (P = 0.035, two-tailed sign test)
MAR pre-trainingMAR post-training
12345Total
  1. *Higher rank: n = 12.

  2. **Lower rank: n = 3.

  3. ***No change: n = 4.

1 Did not obtain Mg0***1*0*0*0*1
2 Obtained but did not prepare Mg00***0*3*1*4
3 Prepared but did not administer Mg000***1*1*2
4 Administered Mg but ≥6 minutes from start of drill0001***5*6
5 Administered Mg <6 minutes from start of drill001**2**3***6
Total01171019

Analysis

This study tested for correlation between the team MARs and various aspects of individual team members’ KSA as follows.

For ‘knowledge’, we used the following: (i) the team’s minimum individual score (the weakest link in the team); (ii) the team’s maximum individual score (regardless of profession or seniority); (iii) the senior doctors’ MCQ scores alone; (iv) the average of all the team members; (v) the first principal component of the team members’ scores, a weighted average that ‘best’ represented the individual staff member’s scores and maximised the variability between them. As the pairs of junior doctors and junior midwives were interchangeable, the averages of their pairs of scores were used in the principal component analysis. We repeated the analysis for both total and eclampsia-specific MCQs.

For ‘skills’, we used the team members’ shoulder dystocia ranking score, based on their ability to deliver a baby with simulated shoulder dystocia. Again, we tested for correlation between MAR and team average, team maximum and senior doctor’s skill score.

For ‘attitudes’, we looked at the team average scores in the six domains (‘teamwork climate’, ‘safety climate’, ‘job satisfaction’, ‘stress recognition’, ‘perceptions of management’, ‘work conditions’) of the validated teamwork/safety attitudes questionnaire, calculated in accordance with the instructions of the authors.20

We were not interested in the effect of training and report analyses based on the baseline (pre-training) data only. To study the relationships between MAR and KSA, we calculated nonparametric correlations (Kendall’s taub) because of the ordinal nature of MAR (a part qualitative measure). The Sas software package (SAS v 9.1, 2002–3, SAS Institute Inc., Cary, NC, USA) was used to obtain approximate 95% confidence intervals.

Results

Although 24 teams underwent pre-training evaluation, one simulation was not recorded because of a fault in the recording equipment and four teams were incomplete (five members only) because of failure of individual attendance. To ensure comparability, we report here on the remaining 19 complete teams whose 114 participants had complete audiovisual, MCQ and shoulder dystocia records. Attitude questionnaires were complete for all 114 individuals and all 57 questions, apart from eight participants who had not answered isolated questions; domain scores for these were the averages of the relevant completed questions.

There was a wide spread of values for all components of the KSA categories (Table 2). There was no correlation between team leader (senior doctor), team maximum or team minimum MCQ scores and team MAR (Table 3). Similarly, there was no correlation between the average of the team member MCQ scores and team rank (MAR), regardless of whether a simple or weighted average (first principal component) was used (Table 3). For eclampsia-specific MCQ scores, Kendall’s taub was consistently negative, albeit low (<0.3) and statistically nonsignificant (P > 0.05) (Table 3b).

Table 2.   Descriptive data
 Mean (SD)Range
  1. MCQ, multiple-choice questionnaire; SD, standard deviation.

  2. *PC, the first principal component of the team members’ scores is a weighted average that ‘best’ represents the individual results and maximises the variability between them.

Knowledge
Total MCQ score
Team minimum54 (13)32–82
Team maximum111 (9)96–128
Senior doctor109 (13)72–128
Team average80 (7)68–93
Team first PC* (unstandardised)44 (18)15–92
Eclampsia component of MCQ score
Team minimum6 (3)−2 to 10
Team maximum20 (3)14–24
Senior doctor19 (3)14–24
Team average13 (2)7–17
Team first PC (unstandardised)18 (5)3–24
Manual skills
Team maximum2.8 (1.3)0–4
Senior doctors2.2 (1.3)0–4
Team average1.1 (0.7)0–2.2
Teamwork/safety attitude components (team average)
Teamwork climate72.0 (5.3)57.6–81.3
Safety climate68.4 (5.6)56.5–79.2
Job satisfaction64.8 (8.6)50.8–75.6
Stress recognition70.8 (8.0)59.4–87.5
Perceptions of management46.0 (7.5)33.3–57.3
Work conditions59.0 (6.4)46.9–69.8
Table 3.   Team knowledge (multiple-choice questionnaire, MCQ) scores and team rank (Magnesium Administration Rank, MAR): no correlation of MAR with either total (all obstetric emergencies) MCQs (a) or specific (eclampsia-only) MCQs (b)
 Team rank (MAR)
1 (n = 1)2 (n = 4)3 (n = 2)4 (n = 6)5 (n = 6)Kendall’s taub [approximate 95% CI]
  1. PC, principal component analysis.

  2. *Group means are shown together with standard deviations (SD) in parentheses.

  3. **In 15 of 19 cases, the team maximum total score was that of the senior doctor.

(a) Total MCQ score
Team minimum58*48 (SD 5)54 (SD 1)50 (SD 16)62 (SD 15)0.17 [−0.16 to 0.49]P = 0.36
Team maximum**116115 (SD 10)119 (SD 6)104 (SD 4)112 (SD 9)−0.15 [−0.53 to 0.23]P = 0.42
Senior doctor116115 (SD 11)119 (SD 6)96 (SD 14)112 (SD 10)−0.16 [−0.50 to 0.19]P = 0.40
Team average8378 (SD 5)82 (SD 2)78 (SD 8)84 (SD 9)0.12 [−0.24 to 0.48]P = 0.51
Team first PC (unstandardised)4234 (SD 12)37 (SD 1)50 (SD 24)47 (SD 21)0.24 [−0.10 to 0.58]P = 0.19
(b) MCQ eclampsia score
Team minimum8*6 (SD 3)10 (SD 0)6 (SD 2)5 (SD 4)−0.26 [−0.58 to 0.07]P = 0.17
Team maximum**2320 (SD 2)22 (SD 0)19 (SD 3)20 (SD 3)−0.15 [−0.52 to 0.21]P = 0.42
Senior doctor2320 (SD 2)21 (SD 2)17 (SD 3)19 (SD 3)−0.19 [−0.55 to 0.18]P = 0.32
Team average1512 (SD 2)16 (SD 2)13 (SD 2)12 (SD 3)−0.14 [−0.52 to 0.24]P = 0.44
Team first PC (unstandardised)2118 (SD 4)20 (SD 6)18 (SD 4)17 (SD 7)−0.03 [−0.37 to 0.32]P = 0.88

There was also no correlation between team MAR and team average, team maximum or senior doctors’ manual skill scores (Table 4). Finally, there was no correlation between team average teamwork/safety attitude scores in any of the six domains and MAR scores (Table 5).

Table 4.   Tests for correlation between team rank (Magnesium Administration Rank, MAR) and team members’ manual skills (shoulder dystocia practical management scores): no correlation with team maximum, team average or senior doctor’s scores
 Team rank (MAR)
1 (n = 1)2 (n = 4)3 (n = 2)4 (n = 6)5 (n = 6)Kendall’s taub [approximate 95% CI]
Team maximum manual skill score
0 (did not deliver)00011 
1 (delivered. within 4–5 minutes)010000.21
2 (delivered within 3–4 minutes)10001[−0.18 to 0.60]
3 (delivered within 2–3 minutes)03131P = 0.30
4 (delivered within 1–2 minutes)00123 
Senior doctor’s manual skill score
0 (did not deliver)00021 
1 (delivered within 4–5 minutes)110000.16
2 (delivered within 3–4 minutes)01112[−0.23 to 0.54]
3 (delivered within 2–3 minutes)02131P = 0.42
4 (delivered within 1–2 minutes)00002 
Team average of manual skill scores0.71.3 (SD 0.8)1.3 (SD 0.9)1.1 (SD 0.8)1.1 (SD 0.6)−0.01 [−0.36 to 0.34] P = 0.94
Table 5.   No correlation between team rank (Magnesium Administration Rank, MAR) and team average teamwork/safety attitudes scores in any of the six domains of the validated questionnaire
Team average (team attitudes questionnaire domains)Team rank (MAR)
1 (n = 1)2 (n = 4)3 (n = 2)4 (n = 6)5 (n = 6)Kendall’s taub [approximate 95% CI]
  1. *Group means are shown together with standard deviations (SD) in parentheses.

  2. **We calculated the component team attitude questionnaire scores as the average of the relevant questions that were not missing.

  3. ***For stress recognition and work conditions, there were no missing values.

Teamwork climate**77.1*69.7 (SD 8.2)75.0 (SD 2.0)72.6 (SD 4.2)71.3 (SD 5.2)−0.26 [−0.66 to 0.14]P = 0.15
Safety climate**75.064.5 (SD 6.7)69.0 (SD 0.8)72.0 (SD 5.3)66.1 (SD 3.7)−0.12 [−0.50 to 0.26]P = 0.51
Job satisfaction**75.660.1 (SD 9.6)67.1 (SD 4.1)69.9 (SD 5.1)60.3 (SD 9.0)−0.18 [−0.60 to 0.25]P = 0.34
Stress recognition***63.575.5 (SD 11.7)63.5 (SD 5.9)67.7 (SD 6.6)74.5 (SD 4.8)0.19 [−0.18, 0.57]P = 0.29
Perceptions of management**52.142.7 (SD 10.2)50.5 (SD 3.7)49.0 (SD 5.8)42.7 (SD 7.3)−0.18 [−0.54 to 0.19]P = 0.34
Work conditions***65.655.5 (SD 8.3)59.9 (SD 5.2)62.8 (SD 5.7)56.3 (SD 4.7)−0.13 [−0.51 to 0.25]= 0.46

Discussion

This study assessed the ability of a large cohort of multiprofessional teams from across a large health region in England to manage a simulated complex obstetric emergency. We examined the relationship between team performance and measures of KSA for individual clinicians, using data from pre-training evaluations to represent the prevailing level of knowledge and skills within the region. The results are compatible with an absence of a relationship between the KSA of individuals and team performance.

We devised a method for ranking team performance (MAR) using the ability to administer magnesium sulphate, an internationally recommended treatment for eclampsia. This does not incorporate all aspects of care important for successful outcome, but is a measure of team efficiency. It has not been validated against true clinical outcomes, but face, content and construct validity were established, including an assessment against the performance of other key actions for eclampsia.

Some might argue that MCQs are a superficial measure of knowledge. The MCQ questions in this study focused on the factual knowledge required to manage successfully emergencies, including eclampsia, and were linked to the objectives of the training intervention of the SaFE study to ensure content validity.16 After a pilot study, only items that were sufficiently discriminatory were retained to establish construct validity for that study,13 further supported by evidence that MCQ scores improved after clinical training.16 Further to this, one component of the skill assessment tested the ability to apply theoretical knowledge in a coordinated manner during a simulated emergency (shoulder dystocia) that relied solely on the individual rather than the team. Measuring attitudes is a difficult task. The attitudes scores in this study were derived from a validated questionnaire, specifically designed for maternity care, which has been shown to correlate with patient outcomes.23 It is suitable for surveys to evaluate views on team and patient safety, and as a quality improvement tool to identify residual or new challenges in units with established safety programmes.24,25

The data showed that there was a wide range of outcomes for the management of simulated eclampsia; some teams administered magnesium in a timely manner, but other teams did not even consider its use. To our surprise, there was no relationship between team performance in eclampsia and the measures of KSA that were used in this study. These are conventional means used to assess individual doctors and midwives,26–30 but clearly, used alone, these measures can neither predict the effectiveness of clinical teams, nor help to understand the problems of ineffective teams. It follows that the focusing of postgraduate training on these conventional domains of learning for individuals alone might not maximise team performance, nor optimise the outcome of emergencies that rely on complex rapid multiprofessional working.

A criticism might be that the sample size was not sufficient. However, this study was based on a pragmatic sample from a very large study of training that will be very difficult to replicate. Not only were P-values nonsignificant in our study, but correlation coefficients were also very low, indicating a true absence of correlation rather than a lack of power to detect a correlation. This lack of correlation was confirmed by visual inspection of scatter plot charts of all variables against MAR (DS and LPH).

Reports and enquiries into poor outcome and patient safety2,5,31 have repeatedly demonstrated that failures in teamwork are more frequent than failures of individuals. As a result, training in healthcare has evolved to include practical team training. Regular practical clinical training (skills drills) has been associated with improvements in real-life perinatal outcomes in observational studies,14,32–34 but even after training there remains a wide range in team performance.12,13 The inference of our study is that there is some other characteristic of clinical teams or team members that governs team efficiency, and which is not directly related to conventional clinical measures of the individual members.

It is unclear why teams differ in their abilities, and why teams do not improve equally after training. It seems likely that there are other characteristics of individuals, perhaps involving attributes such as composure, confidence and communication. One systematic review of team performance concluded that the ability to retrieve and use knowledge and skills is impaired during acute events.35 It might be that variation in team performance is influenced by the different abilities of team members to deal with the effects of anxiety. Further insight into how the behavioural characteristics of individuals influence the way teams work during acute events might provide a means of coaching individual clinicians and/or clinical teams to become more effective. By further studying clinical teams in simulation and real life, we can hope to develop specific tools for assessment and training, rather than use methods that were developed for airplane pilots.12,36

Conclusions

The data obtained in this study show that team efficiency amounts to more than the conventional clinical characteristics of individual team members. It is often said that a team is more than the sum of its parts, but, in truth, it seems likely that we simply do not yet know how to measure the full contribution of individual clinicians. There is an important need to understand these individual factors and how they make a team more efficient in dealing with emergencies. This improved understanding could inform the development of specific evidence-based methods for the analysis and improvement of teamwork in maternity care and other clinical specialities.

Disclosure of interest

TJD and CW are members of the steering committee of PROMPT, a UK-based charity running training courses, but derive no financial interest from this association.

Contribution to authorship

DS conceived the idea, coordinated the analysis, and wrote and edited the manuscript. TJD conceived the idea, coordinated the analysis, and wrote and edited the manuscript. JFC coordinated the SaFE study, helped with conceptual analysis and edited the manuscript. LPH reviewed the methodological considerations, conducted the statistical analysis, wrote parts of the manuscript related to methods and edited the manuscript. CW coordinated the SaFE study, helped with the conceptual analysis and edited the manuscript. RF conceived the idea, coordinated the analysis, and wrote and edited the manuscript. All authors participated in several multiprofessional meetings to develop the study tools and methods.

Details of ethics approval

Ethical approval was granted by a Regional Research Ethics Committee (SOUTHWEST DEVON MREC 04/Q2103/68). This study was conducted in accordance with the Research Governance Framework for Health and Social Care and Good Clinical Practice. Data storage and protection were in accordance with the Research Governance Framework and the Data Protection Act.

Funding

This study is an extension to the SaFE (‘Simulation and Fire Drill Evaluation’) study. The extension is funded by The North Bristol Research and Development Small Grant Scheme. The SaFE study was funded by the Department of Health Patient Safety Programme. None of the sponsors were involved in the study design, collection, analysis and interpretation of the data, writing of the manuscript or the decision to submit the article for publication.

Acknowledgements

The authors thank the following individuals for their contributions to the SaFE Study: Christine Bartlett, Karen Cloud, Maureen Harris, Sarah Fitzpatrick, Derek Tuffnell, Bryony Strachan, Stephanie Withers, Valentine Akande, Fiona Donald, Imogen Montague, Penny Watson, Anne McCrum, Sarah Read, Heather Smart, Melanie Robson, Katie Harrison, Neil Liversedge, Joanne Crocker and Simon Grant.

Ancillary