Correspondence: Dr S. Beckley, Perinatal Research Group, Postgraduate Medical School, Department of Obstetrics, Derriford Hospital, Plymouth PL6 8DH, UK
Objective The development and evaluation of a computer-assisted teaching programme of cardiotocog-raphy and acid-base balance.
Design Randomised controlled trial.
Participants One hundred and seventeen midwifery and obstetric staff at Derriford Hospital, Plymouth.
Methods The obstetricians and midwives were randomly allocated to use the teaching programme, either early or late. The late group (control) used the teaching programme three months after the early group. To assess the effect of the teaching programme, participants were tested on four occasions over eight months by a multiple choice questionnaire. Two questionnaires on ease of use were also completed.
Main outcome measures Multiple choice questionnaire scores and opinion questionnaire results.
Results The mean score in the early group improved from 50.8% (test 1, pre-teaching programme) to 70.2% (test 2, post-teaching programme). The mean score in the control group was 50.3% (test 1) and 54.8% (test 2). Knowledge was retained up to seven months.
Conclusions The teaching programme was effective in improving knowledge of acid-base balance and cardiotocography and can be used by all staff whilst on duty on the labour ward.
Labour is a short but critical period in the life of an infant. The provision of high quality care in labour with fetal monitoring is important and should be a basic provision by a health service. Intermittent auscultation may be appropriate in low risk women, but in high risk women a policy of continuous electronic fetal monitoring and fetal blood sampling is indicated1.
However, as one authority2 states: “for monitoring to be effective, it must be performed correctly, its results must then be interpreted satisfactorily; and this interpretation must provoke an appropriate response”. In practice, this is not easy to achieve. Interpretation of cardiotocograms is difficult requiring considerable training and experience, skills which are difficult to acquire and maintain, especially where there is a rapid turnover of obstetricians and midwives. It is usually the most junior member of staff who first has to recognise an abnormal fetal heart rate pattern and know that a more experienced further opinion is required. The problem is large, for at least 50% of women in the United Kingdom undergo electronic fetal monitoring in labour.
When electronic monitoring is performed without appropriate training the results are well known: an increased rate of caesarean sections3 and increased litigation from avoidable intrapartum asphyxia. The Confidential Enquiry into Stillbirths and Deaths in Infancy4 found that the incidence of intrapartum-related deaths has remained constant at about 1 in 1500. In just over half of these better care ‘would reasonably be expected to’ make a difference to the outcome. Most of the adverse comments on care relate to labour, the commonest avoidable factor contributing to these deaths being failure to recognise or react to an abnormal fetal heart rate trace. The report of the Confidential Enquiry concluded that better interpretation of and response to abnormal cardiotocography was the principal method to decrease the risk of stillbirth in labour, and recommended regular teaching sessions for obstetricians and midwives.
Teaching sessions are difficult to organise such that attendance may be poor, and so the sessions may not prove to be useful. We therefore developed training in cardiotocography using a computer programme which would allow universal access by being available at all times on the labour ward.
A few programmes already exist5–7. However, we were keen to produce a programme that would lay a foundation in physiology and the normal fetal response interpretation of the cardiotocogram and appropriate management of labour. There would also be emphasis on the interpretation of a cardiotocogram in context— not isolating the cardiotocogram from the overall clinical situation.
Development of the teaching programme
The computer programme was developed over three years within our group (largely by S.B.). It consists of two chapters (acid-base balance and cardiotocogra-phy), each chapter being divided into lessons (nine in the acid-base chapter, 15 in the cardiotocography chapter). The content is described in Table 1. The programme is interactive, with pictures, diagrams and animated sequences.
Table 1. Content of the teaching programme. CTG = cardiotocograph, cardiotocogram; FHR = fetal heart rate.
Acid-base balance (2–4 hours)
Introduction (20–30 min)
Rationale for cord blood sampling; its usefulness for neonatal assessment, clinical audit and medicolegal purposes discussed.
Physiology (1–2 hours)
Normal physiology: Placental gaseous exchange, aerobic metabolism.
Disturbed acid– base balance (1): Causes of impaired gaseous exchange, fetal adaptations to hypoxaemia hypoxia.
Disturbed acid–base balance (2): Anaerobic metabolism, the influence of maternal acid-base status upon the fetus.
Clinical background: Importance of considering the clinical background in interpreting the trace and determining management.
Fetal blood sampling
The second stage
The acid-base chapter underwent a preliminary evaluation by a group of thirteen midwives and obstetricians at the Royal Devon and Exeter Hospital in September 1994. This study showed that use of the acid-base package increased the knowledge of participants and was enjoyed by all. It also identified some which required modification. Following this preliminary evaluation, the cardiotocograph chapter was written. Each of the 15 lessons began with patho-physiology and was followed by descriptions of actual cases demonstrating the abnormality, the whole presentation being interactive. The cardiotocograms were displayed, long sections of trace being scrolled across the screen to simulate the unfolding of clinical events. The questions vary from the simple (‘what is the baseline fetal heart rate?’) to the complex, in which the user must decide on the appropriate management (e.g.‘Would you: (a) observe; (b)do a fetal blood sample; or (c) do a caesarean section?’). Feedback to each response, whether correct or incorrect, is given to reinforce knowledge. The outcome of the case is also given, including the condition of the baby at birth, so that the user learns the likely consequences of his actions. A detailed discussion of the decision is also given. The lessons are subsequently summarised to emphasise important practical principles of interpretation and management of the cardiotocogram. The emphasis of the programme is on the need to interpret make correct decisions concerning labour and delivery. The programme is constructed so that the later topics depend upon the earlier topics, so that the programme should be read from the beginning, like a novel. Once familiar with the programme, the user can profitably select any lesson for study.
Access to the programme is gained only by typing a recognised personal identification number, which will also create a unique record file for that user, recording the time taken to complete each lesson, and the answers to all the questions. If a session is interrupted the user can start again where he left off. A course supervisor can readily monitor the progress of each participant, and ensure completion of the programme.
The teaching programme was evaluated in a randomised trial at Derriford Maternity Unit. The purpose of the trial was to test whether the programme resulted in an improved knowledge of acid-base balance and cardiotocography. This paper reports the result of the knowledge test; the ability to interpret cardiotocograms was tested by a set of simulated cases presented on a computer, and will be reported in full in a separate paper.
The randomised trial was carried out in Derriford Maternity Unit from February to November 1996. There were 122 midwives and obstetricians with regular responsibility for the labour ward. Three were excluded as they were involved in running the trial and two declined, so that 117 staff took part. They were randomised (from random number tables) into two groups; early and late. The early group was given early access to the teaching programme, while the late group who had access to existing methods of training (weekly cardiotocography meetings, weekly meetings to discuss emergency caesarean sections, books freely available on the delivery suite) was given later access to the teaching programme. Midwives and obstetricians were randomised separately to ensure their equal distribution in the two groups. The design of the study is shown dia-grammatically in Fig. 1.
The teaching programme was entered into five computers in the ‘Resource Room’ situated on the delivery suite. Participants were given free access to the programme at all times by using their unique identification number. Access was granted only during the 30-day period allocated to the group and was denied at all other times. The late group could not use the programme early in the study, nor could the early group use the programme late in the study (Fig. 1).
Each participant was asked to complete two question naires on their opinion of the teaching programme—one on the acid-base chapter, one on the cardiotocograph chapter. Twelve statements were given (Table 2). Participants were required to indicate their level of agreement with each statement by assigning a score between 1 and 5; 1 = disagree, 2 = disagree slightly, 3 = don't know, 4 = slightly agree, 5 = agree. Free comments were also invited.
Table 2. Results of opinion questionnaires: median scores for each question in early teaching package and late groups. Scores: 1 = disagree; 2 = disagree slightly; 3 = don't know; 4 = slightly agree; 5 = agree.
Median score (10th centile)
I enjoyed the package
I found the package boring
I found that using the teaching package was very tiring
I felt very much ‘at ease’ using the computer
The lessons were relevant to my work
Ease of use
Overall, I found the package easy to use
Getting from topic to topic was straightforward
I could find my way around the package without getting lost
It was difficult to see how to use the buttons
Presentation of material
The overall appearance (e.g. colour schemes, layout etc.) was attractive
The diagrams helped to explain the text
It was difficult to concentrate on the lessons
Participants were assessed on four occasions by a multiple choice questionnaire and by ten real cases. Test 1 assessed baseline knowledge. Both groups were tested immediately after the early group had completed the teaching programme (test 2) and again six weeks later (test 3). The late group used the teaching programme only after test 3. Test 4 measured retention of knowledge, at seven months for the early group and at four months for the late group.
The multiple choice questionnaire consisted of 80 questions, each answered ‘True’, ‘False’ or ‘Don't know’. All the questions were derived from the teaching programme. For example, one set of five questions consisted of the following:
Babies who have:
a. A cord artery pH of less than 7.05 are likely to develop cerebral palsy.
b. A cord artery pH of less than 7.00 are more likely to suffer early neonatal convulsions than babies with a higher arterial pH.
c. A metabolic acidaemia will always have a low Apgar score.
d. A respiratory acidaemia must have suffered an episode of hypoxia during labour.
e. A cord vein pH of more than 7.40 are at high risk of developing respiratory problems in the first 24 hours of life.
All 80 questions were given at each test. The tests were marked manually and separately by two of the investigators. One mark was given for a correct answer, one mark subtracted for an incorrect answer, and no marks given for a ‘Don't know’.
A full description of the assessment of the interpretation of the fetal heart rate traces will be given in a separate paper. Briefly, ten real cases were presented to each participant at each test. These were from the same database of 50 cases used for our evaluation of the consistency of experts in the interpretation of the fetal heart rate trace8. Participants were required to state their opinion of the significance of the fetal heart rate trace every 15 minutes, and their intended management. A few of the fetal heart rate traces were easy to interpret, and several were difficult.
Each test was held over three days. Participants unable to attend during this time sat the test on a separate occasion, when possible within a week of the main test. Most participants took between 60 and 90 minutes to complete the test; a time limit was not set.
Almost universal participation in the study was achieved with the help of strong support from senior midwifery and medical staff. Participants were paid for some of the overtime they put in to complete the study, although most worked for at least twice the time they were paid. The results were analysed using the Data Analysis Tools of Microsoft Excel.
All but two eligible members of staff took part in the study (95 midwives and 22 doctors). All completed Test 1, 115 completed the teaching programme and Tests 2 and 3, and 111 completed Test 4. The majority completed each test during the three-day main sitting of the test. A mean of 18 participants per test attended later, usually because of annual leave; of these, 16 sat the test within two weeks and 2 within three weeks.
Multiple choice questionnaires
The results of the multiple choice questionnaires are shown in Fig. 2. The mean improvement from test 1 to test 2 was 19.4% (95% CI 16.5, 22.3%) (P < 0.0001, paired t test) for the early group, and 4.3% (95% CI 1.6, 7.0%) (P= 0.003, paired t test) for the late group. While there is a significant improvement in both groups, the difference between the early and late groups is highly significant (P < 0.0001, two-sample t test). The scores at seven months were very well maintained in the early group; there was no significant difference in score between Test 2 (70.2%) and Test 4 (68.6%) (P= 0.2, two-sample t test).
In the late group the mean scores at test 4 were significantly different compared with the mean scores at tests 1, 2 and 3. The improvement from test 1 to test 4 was smaller in the late group (mean 13.3%; 95% CI 10.3, 16.2%) than for the early group (mean 17.8%; 95% CI 15.0, 20.7%), despite more recent use of the teaching programme; this difference between the groups was statistically significant (P= 0.03, two-sample t test).
Nine participants (three from the early group, six from the late group) achieved a lower score at test 4 than at test 1. The number of these in the late group is not statistically greater than in the early group (p= 0.2, χ2 test, Yates' correction).
We examined the results of the multiple choice questionnaire at tests 1 and 4 in relation to profession and seniority (Fig. 3). The differences in mean scores between the categories were significant (p < 0.0001, analysis of variance).
The results of the opinion questionnaires for each group are shown in Table 2. The programme was well received, although participants found it quite tiring, and concentration was sometimes a problem.
The results of this study show that the computerised teaching programme led to improved knowledge of acid-base balance and cardiotocography. Importantly, this knowledge was retained almost intact for seven months. Since some decay of knowledge is certain repeated study and testing may be necessary. If the initial learning has been good, a very brief re-exposure to learning material9 or even repeat testing alone10,11 is effective in restoring knowledge. Testing is helpful to prompt recall of information and to provide a degree of stress and challenge. An appropriate amount of stress may enhance learning, whilst too much stress may be counterproductive12.
Testing is important for it measures actual, not perceived, knowledge. One study of training in cardiopul-monary resuscitation found that although 88% of the participants felt confident of their ability in cardiopul-monary resuscitation, only 1% were competent13. and the results of our study agree with this recommendation. There should also be a more frequent opportunity to review abnormal fetal heart rate traces, perhaps at regular departmental meetings at which difficult cases are discussed.
The sustained improvement in score in the early group may have been due in part to revision prior to the tests. We know from structured interviews carried out on a sample of participants after the study was completed that some had revised from their own notes while others had done some extra reading. This suggests that the discipline of repeat testing itself was important in maintaining knowledge; in addition, the obstetricians and midwives regularly interpreted fetal heart rate traces and acid-base balance in the course of their everyday work, and this constant use of their knowledge would promote retention.
It was difficult to ensure that the difficulty of the questions in the four multiple choice questionnaires was the same, and so we administered the same test four times. While this gave the participants opportunity to learn from other sources in preparation for the next test, the late group enabled us to quantify this. There was a very small but statistically significant improvement in score prior to the introduction of the teaching package.
Our results therefore suggest that any teaching programme should include regular testing, as the tests themselves reinforce retention of knowledge. It is possible that conventional teaching with repeated testing may be more successful, but to test this hypothesis would teaching with the computerised teaching programme.
It is of interest that the late group scored less at test 4 than the early group, despite the fact that it had used the programme more recently. Several participants in the late group were dissatisfied at being tested three times before having a chance to use the teaching programme. This may have diminished their motivation.
The midwives tended to learn more than the obstetricians (as judged by the degree of improvement from test 1 to test 4; see Fig. 3). This was probably in part related to the higher scores achieved by the doctors before the teaching programme, such that they had less room for improvement. The mean score for the midwives at test 4 was 62.4%, compared with 82.0% for the obstetricians. This suggests that the midwives could increase their scores further, perhaps by using the teaching programme again.
Nine participants (8%) appeared to have gained very little from the teaching programme since their scores decreased after use of the programme. Four achieved a score of > 75% at test 1, so there was relatively little room for improvement. It is not clear why the other participants fared worse in the fourth test, but it does raise issues of formal accreditation.
The improved knowledge demonstrated in this study does not necessarily result in improved clinical practice, but we believe the acquisition of knowledge is important to satisfactory obstetric and midwifery performance. The effect of the teaching programme upon clinical skills is to be reported separately.
The teaching programme was received very favourably by midwives and obstetricians at every level of seniority. Ten of the 12 questions in the opinion questionnaire which covered overall enjoyment, interest and relevance received very good scores, as did those questions relating to ease of use and standard of presentation. However, the programme was thought to be tiring, and concentration was sometimes difficult to maintain. External factors were important since the midwives often sat down to the programme after a shift, already fatigued. The comments also reflect the very large amount of information in the package, and the fact that the acid-base physiology which we had felt very important was new to many of the midwives.
It is possible, since the trial was carried out in the institution in which the package was developed, that the participants wanted to comment positively to reward the researchers. However, it was used in two other units where it was also well received and it is now in use in 20 hospitals in the United Kingdom.
Computerised educational programmes have been produced in many medical specialities. There are difficulties in comparing computerised programmes with conventional learning due to a large number of con-founding factors16. However, many programmes have been evaluated and have been shown to improve knowledge17–20. This has also been our experience.
However, improving knowledge of cardiotocography and acid-base balance is not enough; the real effectiveness of our computer-assisted teaching programme can only be measured by improvements in clinical practice. We are currently analysing a retrospective audit of the intrapartum care before and after use of the teaching programme.
We are grateful to our colleagues in the Perinatal Research Group, who provided invaluable technical and clinical support: Dr R. Keith, Mr J. Skinner and Mr J. Garibaldi (engineers) and Miss J. Westgate, Mr G. Hughes and Mrs M. Harris (clinicians). We would especially like to thank all the midwifery and medical staff at Derriford Hospital who took part in and supported the study. The development of the programme was funded by a South and West Research and Development Project Grant and the randomised trial by a WellBeing Project Grant.