Impacts of participating in confidential enquiry panels: a qualitative study
Dr J Rankin, School of Population and Health Sciences, Framlington Place, University of Newcastle, Newcastle Upon Tyne NE2 4HH, UK. Email firstname.lastname@example.org
Objective To describe the impacts of participating in confidential enquiry panels for the Confidential Enquiry into Stillbirths and Deaths in Infancy.
Design Qualitative interview study.
Setting The former northern health region of England.
Sample Eighteen health professionals who had participated in at least one confidential enquiry panel.
Methods Semistructured one-to-one interviews using purposive sampling; transcripts were analysed by identifying recurring themes. Data were organised and coded using NUD*IST.
Main outcome measures Views on the impacts of participation on clinical practice and views on the strengths and limitations of confidential enquiries.
Results Participants valued attendance at panels as a learning experience that provoked reflection on their own clinical practice. Participants felt that taking part had a positive impact on their clinical thinking and practice by increasing their awareness of standards of care. These impacts occurred through both the detailed examination of cases and the interaction with colleagues from different disciplines and hospitals. Learning impacts were cascaded to colleagues through informal discussion and teaching. Concrete examples of changes in practice at the organisational level, stimulated by panel attendance, were reported.
Conclusions The confidential enquiry approach was supported not only as an effective way of assessing care but also as a valuable learning experience that motivated change in clinical practice. Local benefits of nationally coordinated confidential enquiries should be valued and supported in their future development. Wide multidisciplinary participation in enquiry panels coordinated through regional clinical networks should be promoted.
Since the first Confidential Enquiry into Maternal Deaths (CEMD) in the UK more than 50 years ago,1 the confidential enquiry method has been used extensively within obstetrics2–5 and other disciplines6,7 to identify avoidable factors contributing to death and other adverse events. In the UK, four national confidential enquiries investigate factors surrounding maternal deaths (CEMD), stillbirths and deaths in infancy (Confidential Enquiry into Stillbirths and Deaths in Infancy [CESDI]), perioperative deaths (NCEPOD) and suicides and homicides by people with mental illness.8 The essential feature of these enquires is the independent, multidisciplinary, anonymous review of cases to identify avoidable factors.
CESDI was established by the Department of Health in 1992 to improve understanding of how the risks of death in late fetal life and infancy might be reduced by identifying factors that could be attributed to suboptimal clinical care.9 CESDI has since been reorganised into the Confidential Enquiries into Maternal and Child Health (CEMACH)10 and is managed with the other national enquiries by the National Patient Safety Agency.
Uniquely among the UK enquiries, CESDI enquiries are organised at a regional level and involve review of case notes, with identifying data removed by a multidisciplinary panel of local clinicians. The focus of the enquiries is set at a national level, with a rolling programme of enquiries into subsets of deaths. Each panel meeting considers four or five cases, and notes with no identifying data are sent in advance to the panel members. At the meeting, the panel completes a standard form highlighting preventable or avoidable factors and outlining areas of suboptimal care. These decisions are reached by consensus. The findings are aggregated nationally and form the basis of the national annual reports, which make recommendations to improve the quality of care.11 As the process is anonymous, no direct feedback is given to the individual maternity units about cases from those units.
Panel membership is variable depending on the enquiry topic but includes at least four experienced clinicians from obstetrics, midwifery, neonatal nursing, neonatology, paediatrics and pathology. Diabetologists and diabetes specialist nurses were added for enquiries into the quality of care in diabetic pregnancies.12
While the explicit purpose of CESDI is to make recommendations based on enquiry findings to improve the quality and outcome of care, it has been recognised that regional panels may also fulfil an important educational role.9 A report on an enquiry into perinatal deaths in pregnancies complicated by diabetes in the northern region of England stated that panel participants ‘found the discussion in a multidisciplinary panel a stimulating learning environment’.12 However, the potential impact on clinical practice of participating in confidential enquiry panels has not been previously studied.
This study reports the views of participants on the impacts of attending confidential enquiry panels around deaths in late fetal life and infancy. It focuses on the perceived value of participation in panels as a learning experience; further findings from the study are reported elsewhere.13,14
Semistructured, in-depth interviews based on a topic guide were conducted with eighteen health professionals between September and November 2002. The topic guide was piloted with two participants and refined. Topics included impacts of participation on personal and organisational practice and views on the strengths and limitations of the confidential enquiry panel process.
Health professionals were purposively sampled on the basis of specialty, location, type of panel and number of panels attended. The aim was to include professionals with a range of characteristics that might influence their responses in relation to the research aims. Professionals were eligible to take part in the study if they had participated in at least one confidential enquiry panel either for the national 27/28 week project15 or for the regional diabetic pregnancy enquiries.12
The CESDI coordinator (M.R.) provided the researcher (J.B.) with a coded list of participants by specialty, location, type of panel and number of panels attended. J.B. selected potential participants from this anonymous listing. M.R. sent a letter and an information sheet to each potential participant, informing them of the existence of the study, giving a brief background to it and asking them to let M.R. know whether or not they agreed to their contact information being given to J.B. Those agreeing were contacted by J.B. by telephone within 2 weeks to discuss the study further and to arrange a visit. Following a written consent, an in-depth interview was conducted. Seventeen of the 18 interviews were tape recorded with participant permission; one participant did not give permission for the interview to be recorded, so notes were taken.
Ethical approval was received from the Trent Multi-centre Regional Ethics Committee.
Transcripts were analysed by identifying recurring, emergent themes using constant comparison of the interview transcripts.16,17 All the interview transcripts were read by J.B. and J.R. and a coding framework was developed. Transcribed material was entered into the NUD*IST 4 qualitative analysis package18 to organise, access and code the data and to assist with the analysis. Interviewees are referenced by their specialty and the number of panels they had attended.
Eighteen health professionals were interviewed: six obstetricians, four from midwifery and nursing, six paediatricians and two diabetologists. Seven had taken part in up to five panels and 11 in six or more. The length of time participants had been involved in confidential enquiry panels ranged from 1 to 10 years.
Participation in panels as a learning experience
Participants recognised, and highly valued, a learning element associated with attending such panels in addition to their function in assessing care. Participation was compared favourably with other educational activities:
It's a learning experience and it's also a very supportive environment. And at the end of it, I don’t think there's any that I have been involved with where I haven’t come away thinking oh I will remember that, or do that differently or we need to look at it, which is why I am really keen to do them. … personally that's one of the ways I learn. I’d far rather do that than go and sit in a meeting. (consultant obstetrician, more than five panels)
At the time I remember saying to some of the midwives here this is more productive in terms of updating you than any study day you can do about aromatherapy or baby massage … because it made you read up on the references otherwise you’d look a right fool. (nurse manager, more than five panels)
I think this has been some of the most useful personal postgraduate, if you want to call it that, activity I have done over the last seven or eight years. I’ve never been to a CESDI panel and come away thinking that was a waste of time. (consultant obstetrician, more than five panels)
At the time of the study, the main outcome of the CESDI enquires was an annual report summarising the main findings of the enquires at a national level and making recommendations for clinical practice;9,11 this report was disseminated widely to clinicians.
Among the participants, there was a dominant view that actually attending a confidential enquiry panel had much greater personal learning value than reading the annual report:
I mean it is very interesting to read the report but I was just amazed at how much I learned from the panels. (specialist nurse, more than five panels)
Even if it's the best report in the world if they’re not motivated to change, nothing will make them change. By sitting them on a panel you shift that motivation right up and so they’re much more likely then to change …. People would be much more likely to change their practice as a result of sitting on an enquiry panel than they ever would from no matter how much bumph they sent out. (consultant paediatrician, more than five panels)
Impacts on personal clinical practice
Most participants felt that attending panels had impacted positively on their clinical thinking and practice. Participants felt that panel participation stimulated a ‘reflective process’ of ‘self-examination’. The multidisciplinary and multihospital nature of panels meant that panellists were able to reflect upon their practice and compare procedures and standards in their own department, both with those of the cases being discussed in the meeting and with other panellists:
It's a fantastic opportunity to be reflective. What would I have done? Can I understand the pressures this person was under? Why did they make that decision at the time? I can see with the benefit of a relaxing environment and a bit of hindsight … well actually that was wrong. On the other hand, if they were faced with an impossible problem nobody could have done it. To me that's a very useful exercise to go through …. You’re questioning your own practice every time. You are being asked to look at existing guidelines and all different standards which is always a good thing and compare them, particularly you’re being asked to look at the resources as well which I think is always useful. (consultant obstetrician, more than five panels)
… major time of reflection and thinking but for the grace of God there goes our unit and we must tighten up on this. (consultant obstetrician, more than five panels)
One participant was more cautious of attributing personal impacts to participating in confidential enquiry panels because there was an absence of objective standards against which impacts could be measured:
I mean it just generally makes me reflect on what I do for a few weeks. Whether it's changed my practice or not, although I’d like to think it has, but again there's no objective measure of it. (consultant paediatrician, more than five panels)
Many participants mentioned a heightened awareness of the importance of good documentation and the need for clinical notes to be written in a clear and accessible way so that other people can follow the train of thought:
It's made me a lot more conscious of the quality of data. Even simple things like being able to identify the grade of the person who is dealing with the patient …. It has made me think that it is necessary and desirable to put ones thought train down on paper rather than assume that anybody reading it would obviously understand why I did such and such with hindsight. It's obvious what the thought processes were to the persons who made or not made the entry into the records and I think it has made me realise that it is important to actually put down on paper what you think. (consultant obstetrician, two panels)
I thought it was a very good learning experience because when you’re actually looking after someone yourself, you document as much as you can, you don’t realise some of the obvious things that maybe you know but you don’t actually write down. So it was quite good to know that you were looking at somebody else’s, it wasn’t yours but you were thinking well somebody else is looking at yours and have I left out information like that? So it made me very much aware of the importance of the notes. (diabetes nurse specialist, one panel)
… just by reading through the notes you can get information from the layout. Wow that's a good way of doing it or that's a very bad way of doing it. So you learn from just the way the records are kept. (consultant paediatrician, more than five panels)
Impacts beyond individual participants
Participants reported a range of actions arising from their participation in enquiry panels, which impacted on their colleagues and/or departmental practice. Some participants were motivated to initiate specific changes to improve communication within their own units:
Well it certainly heightened record keeping. And that had sort of been on my mind for quite a while, but what it did do, it pushed me into saying, right, we’ve actually now made a form that matches the pregnancy but it's for pre-conception …. So we’ve managed to do that. I actually got down to doing it, not just by myself, I did it with the consultant and another nurse but we actually managed to do it. (diabetic nurse specialist, one panel)
One thing we decided to do was to construct a short letter which we give to new patients coming to the antenatal Diabetes Clinic. (consultant diabetologist, one panel)
We revamped a form that we use here after I’d seen one in somebody else's pile (of notes). I think it was a fluid balance sheet. (nurse specialist, more than five panels)
Around two-thirds of the participants had discussed their participation in enquiry panels with their colleagues. Most had performed this informally via discussions and conversations. A small number of participants had fed back formally via presentations and meetings. All the participants stated that they would encourage a colleague to take part in a regional confidential enquiry panel as the educative value of attending was considered to be high.
Around one-third of the participants stated that their department had used a confidential enquiry approach internally both for assessing care and also in the context of teaching:
We’ve done two sorts of different CESDI panels for trainees. It's a very good, disciplined, structured way for reviewing the proper [procedure]. We also ran our own critical incident reviews using the CESDI structure and forms just because it was by then a format that we were all familiar and comfortable with. It was a very useful structure, a non judgemental structure designed to produce a productive assessment at the end. (consultant obstetrician, more than five panels)
Factors enhancing the value of panel participation
Participants highlighted group interaction as a valuable aspect of panel participation:
… because we’re working together as a group things come out that you don’t actually pick up when you’re reading through your cases alone. I think the main strengths are that you get multi-disciplinary input, sharing of opinion and networking as well between different people. (consultant obstetrician, more than five panels)
I think face to face stuff's very important because you can get to a level of detail and level of perception and understanding that you can’t in … audit is not concrete and specific enough often. (consultant diabetologist, one panel)
Time to devote to detailed consideration of individual cases was also valued:
… there is dedicated time to focus on the issues of clinical care, the quality of that care is measured against pre-defined standards and there is an attempt, more or less successfully, to relate outcomes to what is delivered. (consultant diabetologist, two panels)
Limitations of the enquiry approach
One of the major problems highlighted with participation in panels was the time needed for preparation and attendance, although despite this, most felt that the exercise was ‘productive’ and ‘worthwhile’:
One of the reasons why I have attended only a couple is because of time constraints. I see no way around it though. I do actually believe in the concept of these confidential enquiries and if it's at all possible I like to cooperate with them but time is the major constraint.(consultant obstetrician, two panels)
… in spite of the fact that your heart always sinks if you’re asked to do one because you know it involves quite a lot of work, its quite refreshing to have a day out of hospital in a sort of intellectual environment rather than, you know, the daily grind. (consultant paediatrician, more than five panels)
One frustration expressed by participants was that the findings from regional CESDI panels were not adequately fed back to individual units about cases from those units, therefore preventing those units from learning from their own cases and restricting opportunities to improve standards of care:
It's such a useless, it's a waste, it's a complete waste not to feed back the findings, it's an utter waste. I mean there will be some legal process that will stop it, but in terms of it being a learning experience it's too valuable to be missed. (nurse manager, more than five panels)
And then there were times when you felt that the information you had gathered was so poor that something needed to be done about it. In other words, a practice that was unacceptable…and they needed to understand that their practice needed to change. (consultant paediatrician, more than five panels)
Another participant highlighted the dangers of focussing solely on negative aspects of care:
… it's all criticism rather than reward or encouragement or acknowledgement of good practice as well. (consultant obstetrician, more than five panels)
Suggestions for change included broadening the disciplines involved in panels, encouraging junior medical staff to attend panels as part of their training and introducing a confidential enquiry approach to examine cases at the unit level.
This study documents the perceived value and impact on clinical practice of participating in confidential enquiry panels. Participants highly valued the experience, in particular the face-to-face interaction with their colleagues from a variety of disciplines and hospitals. They reported positive impacts on their own learning, their clinical practice and those of their colleagues. There were examples of concrete change occurring as a result of participation in panels, as well as effects on motivation and awareness of good practice.
There are a number of factors that may have influenced the views expressed. The panel participants were a self-selected group and may value the confidential enquiry process more than those clinicians who choose not to participate. The number of interviews undertaken was small as they were drawn from a relatively small group of health professionals, and the extent to which the views expressed are representative of members of confidential enquiry panels may be questioned. However, the recruitment strategy ensured that there was a broad spread in terms of specialty, location and number and type of panel attended.
Some factors that facilitated the learning impacts of the enquiry panel process—friendliness, group dynamics, active involvement and high motivation—may be specific to our region. However, the key elements of independent, anonymous review by a local multidisciplinary panel of clinicians, which appeared to underpin the positive impacts of participation in enquiry panels, are readily transferable to other settings. A tangible benefit of participating in the panels was that it enabled lessons learned to be taken back to hospital departments by individual participants. This was undertaken both formally via presentations and informally through discussions. One-third of the participants had used the confidential enquiry format within their own department for teaching purposes.
Many of the reported impacts on personal practice involved an increased awareness of the need for clear documentation and good communication. This is particularly pertinent, as deficiencies in communication have been repeatedly highlighted in confidential enquiry reports.11,19
The multidisciplinary nature of the panels meant that participants learnt from the expertise of other panellists. Indeed, some reported that participating in a panel was a more effective learning exercise than, for example, study days. Further, the personal impact of attending panels was felt to far exceed that of the intended output of the enquiry process—national reports and recommendations for practice.
There have been few formal evaluations of the impact of national confidential enquiries on clinical practice. A telephone survey of obstetrical and midwifery staff found that awareness of a recent CEMD report was high and that 65% had read some of the report, but a median of only 3 of 18 key recommendations were recalled. There was no evaluation of the impact on practice.20 Audit data show that in the UK, maternity services have improved in areas highlighted by the CEMD, but the extent to which the enquiry has directly contributed to this improvement is unclear.21 A before-and-after study of clinical practice in relation to one recent recommendation from the CEMD showed dramatic self-reported change in practice, with 84% of respondents stating that they had changed their practice, 95% giving the CEMD report as the main reason for change.22 In a survey of anaesthetists, 74% reported that personal clinical practice had been influenced by recommendations from the NCEPOD and 80% said that local guidelines or protocols had been influenced.23 No previous report, however, has investigated the potential impact of individual participation in the enquiry process.
Our study did not seek to quantify the impact of participation in confidential enquiries on specific changes in clinical practice. However, participants clearly felt that the educational value was high, and further evaluation of this type of activity as a tool to stimulate change in clinical practice is warranted. Research suggests that many other commonly employed methods for promoting improvements in clinical practice, such as the dissemination of guidelines, produce only modest or inconsistent changes in behaviour.24
Our findings have implications for the development of confidential enquiries. While the credibility of national enquiries among clinicians in the UK is well recognised, the educational value or indeed potential for beneficial change in clinical practice that may arise as a result of actual participation in enquiry panels has been overlooked.8 Our study suggests that this potential may be greatest where enquiry methodology encourages wide clinical involvement, as with the regional CESDI panels in the UK. We suggest that not only the formal outputs of enquiries but also the process of enquiry itself has the potential to drive local change in clinical practice.
One of the disadvantages of commonly used confidential enquiry approaches is that they focus on deficient practice in relation to adverse outcomes and may consequently be perceived to be demotivating and morale lowering. However, it is possible to develop enquiry approaches that also recognise and reinforce elements of good practice.25 Our study suggests that participation in panels promotes the sharing of good practice at an individual level. Further development of national enquiries could promote wider exchange of examples of good practice.
The dominant perceived deficiency of confidential enquiry panels highlighted in this study was that the findings from individual panels were not adequately fed back to individual units about cases from those units and that this limited the opportunity to improve standards of care. There was also concern that there was no possibility of feedback to units where there appeared to be systematic failures or major concerns about an individual's practice. The clinicians interviewed in this study found it difficult to suggest appropriate ways of achieving this while still maintaining the anonymity and ‘blame-free’ approach, which underpins confidential enquiries and may be crucial to wide participation.
Our findings suggest that participation in multidisciplinary confidential enquiry panels motivates panel members to reflect on their own clinical practice and can stimulate specific initiatives to improve care. Thus, the enquiry process itself can be a powerful tool to educate and to drive change through the experience of the participants. To date, the value of such participation as a means of maintaining and improving standards of care has been overlooked. Enquiry processes that promote wide, multidisciplinary participation within regional clinical networks and that facilitate open exchange of views in a supportive environment are likely to maximise the potential learning value of confidential enquiries.
We are very grateful to all the staff for taking the time to be interviewed. We thank Dr Chris Wright for advice, and Terry Lisle, Emma Hutchinson and Carole Frazer for secretarial support. Dr J.B. was supported by funds from the Regional Maternity Survey Office.
J.R. contributed to the conception, design, analysis and interpretation and wrote the paper. J.B. contributed to the design, execution, analysis, interpretation and paper writing. R.B. contributed to the interpretation of the results and paper writing. P.C. contributed to the conception, design and interpretation and commented on the paper. M.R. contributed to the execution and interpretation and commented on the paper. All authors are the study guarantors.
M.R. is the regional CEMACH coordinator, P.C. is the director of the Regional Maternity Survey Office. J.R. and R.B. are involved with research using datasets partly funded through CEMACH support.