The difficulty of questioning clinical practice: experience of facility-based case reviews in Ouagadougou, Burkina Faso
F Richard, Quality and Human Resources Unit, Department of Public Health, Institute of Tropical Medicine, Nationalestraat 155, B-2000 Antwerp, Belgium. Email email@example.com
Objective To describe the implementation of facility-based case reviews (medical audits) in a maternity unit and their effect on the staff involved.
Design Cross-sectional descriptive study.
Setting A 26-bed obstetric unit in a district hospital in Ouagadougou, Burkina Faso.
Sample Sixteen audit sessions conducted between February 2004 and June 2005. Thirty-five staff members were interviewed.
Methods An analysis of all the tools used in the management of the audit was performed: attendance lists, case summary cards and register of recommendations. The perceptions of the staff about the audits were collected through a questionnaire administrated by an external investigator from 10 June 2005 to 16 June 2005.
Main outcome measures Session participation, types of problems identified, recommendations proposed and implemented and staff reaction to the audits.
Results Only 7 midwives from a total of 15 regularly attended the sessions. Eighty-two percent of the recommendations made during the audits have been implemented, but sometimes after a delay of several months. Interviewed personnel had a good understanding of the audit goals and viewed audit as a factor in changing their practice. However, midwives highlighted problems of bad interpersonal communication and lack of anonymity during the audit sessions, and pointed out the difficulty of practising self-criticism.
Conclusions A lack of staff commitment and the resistance of maternity personnel to being evaluated by their peers or service users are reducing acceptance of routine audits. The World Health Organization must take all these factors into account when promoting the institutionalisation of medical audits in obstetrics.
Offering quality hospital care is a common objective for all healthcare systems, be it in industrialised or in developing countries. The medical audit is considered an efficient method among the different approaches to quality improvement.1 It consists of reviewing the different stages of care with the aim of identifying any problems and finding solutions.2 Medical audits were developed primarily in the UK. Their institutional development culminated in the publication of the White Paper Working For Patients in 1989, which stated that all doctors should become involved in audit. They were first introduced among physicians, and the approach was then broadened to include nursing staff and all clinicians.2,3 In developing countries, although the experience is limited, audits have increased in the past 10 years, principally in the field of obstetrics. They were presented as a method of improving obstetric emergency management4–9 although their impact on maternal mortality has not been clearly demonstrated.10 The World Health Organization (WHO) has promoted medical audits in its publication Beyond the numbers, which was launched during training workshops in around 20 developing countries to familiarise all health personnel with the different methods proposed in the reference document.11,12 Regional workshops were also organised by WHO in 2005 to help countries to develop and institutionalise audits.13 Although hospital managers are invited to integrate the audit into the routine activities of obstetric departments, little is said about their practical implementation. Several authors have already described the implementation difficulties in the UK,3,14–16 but little has been published on the feasibility of these audits in developing countries.4 This article aims to provide further information about the reality of medical audits in a context of limited resources based on a study conducted in an urban maternity unit in Ouagadougou, Burkina Faso, where the audits started in February 2004. We present in this study the way in which they were introduced, perceived by healthcare personnel, and what changes they brought about.
We conducted a cross-sectional descriptive study of the implementation of facility-based case reviews (medical audits) in the district hospital of ‘Secteur 30’ in Ouagadougou with the aim of evaluating the integration of audit into the activity of the maternity department (participation in audits and perception of audits by the staff) and the changes that resulted from them.
Case reviews were implemented from February 2004 in the district hospital of ‘Secteur 30’ in Ouagadougou. ‘Secteur 30’ is one of the four districts in the health region of the Centre and serves about 470 000 inhabitants. The district comprises 41 first-level facilities (public and private), which refer their complicated cases to the district hospital. At the hospital, one gynaecologist, two midwives, one auxiliary midwife and one helper are on permanent duty. The maternity department had 26 beds for 4182 admissions and 3509 deliveries in 2005.17 In case of technical problems in the operating theatre or complications beyond the scope of the district level, women were referred to the university hospital Yalgado Ouédraogo 8 km away.
Case reviews11 were introduced with the technical assistance of teams from the Institute of Tropical Medicine, Antwerp, and the Institut de Recherche pour le Développement (IRD) within the Programme d’Amélioration de la Qualité et de l’Accès aux Soins Obstétricaux d’Urgence (AQUASOU). A situation analysis performed at the beginning of 2003 had shown substandard quality of care in the maternity ward with poor follow up of women after childbirth, use of oxytocics and neonatal resuscitation. Faced with this situation, several measures were taken to improve the quality of care (morning staff meetings for handover and case reviews, purchase of medical supplies, training in obstetric and neonatal emergency care and medical audits). This multilevel approach did not enable us to identify the impact of the audit alone on the quality of care. The effect of the total package will be reported elsewhere.
Description of the intervention
Before starting case reviews, a number of conditions had to be met. First, the WHO clinical standards for management of obstetric emergencies were given to the midwives in the delivery room, in the form of a large folder containing all the management protocols,18 and several training workshops in obstetric emergencies were given. The quality of the medical file has clearly been improved by the introduction of a Ministry of Health standardised obstetric file. Until 2004, there was only a temperature chart. Medical equipments (blood pressure metres, delivery sets, vacuum extractor, etc.) were acquired so that the delivery room was equipped for comprehensive obstetric emergency care.19
Two midwives volunteered to organise the sessions in the first year. The sessions were usually held once a month and lasted 1–2 hours. Attendance was not mandatory and was not linked to any financial reward. By choice, the first eight sessions were restricted to the midwives and auxiliary midwives of the maternity ward, excluding the gynaecologist and the rest of the health staff. The intention of the midwives was to re-establish ownership of the care and responsibility of their work by doing their own evaluation in their own specific field and also familiarising themselves with evidence-based medicine. Once they were used to presenting a case and discussing the observed deficiencies among themselves, the audit was widened (from the eighth session onwards) to include other hospital personnel, as had been initially intended: medical doctors, laboratory technicians, pharmacist and administrators. During the first session, an audit chart was adopted by the participants, one of the essential elements of which was respect of confidentiality (Box 1).
Table Box 1. . Audit charter for the maternity unit of the district hospital ‘Secteur 30’
|The medical audit consists in a systematic and critical analysis of the quality of care by comparison to defined standards (norms and care protocols). It enables the members of a team to discuss and question or improve certain practices. The audit must never be used to sanction a member of staff. Its purpose is to propose recommendations and actions aimed at avoiding in future the deficiencies or errors observed.|
|We, staff of the maternity of the hospital ‘Secteur 30’, promise to respect the rules of good practice that follow:|
|1 To arrive on time for audit sessions.|
|2 To respect the statements and ideas of everyone.|
|3 To respect the confidentiality of the team discussions. Information and problems raised during the audit must not be communicated outside the team (friends, relatives, colleagues in other health departments, etc.).|
|4 To participate actively in the discussions.|
|5 To accept discussion and debate among participants without verbal violence.|
|6 To refrain from hiding or falsifying information that could be useful in understanding the case being audited.|
|7 To try as much as possible (because it is not easy) to accept questioning one’s own actions.|
|Ouagadougou, 25 February 2004 Staff of the maternity department|
Women for audit were usually selected by the midwives and doctors during staff meetings. They consisted of women transferred from peripheral health centres to the district hospital, serious maternal complications and cases of fresh stillbirths. Initially, the sessions only dealt with clinical facts. However, the presentations were later supplemented by comments by the woman and her family, which were elements collected during home visits conducted by the IRD team. The interview included the management of previous pregnancies and deliveries, the outcome and follow up of the present delivery, the cost of transport and hospitalisation, the preceding knowledge about this hospital, the perception of the care and caregivers, the care received and suggestions for improvements.
In audit sessions, a midwife first read out a clinical summary of the selected case (while respecting the anonymity of the patient and staff) together with any information gathered during the home interviews. Then, in an approach called ‘gate to gate’, all stages of the case (referral, admission, diagnosis, treatment and discharge) were reviewed and compared with the standard of care.18 A midwife moderator helped to conduct the process. Any elements of substandard care were recorded on a large board as the discussion progressed. The group then analysed the causes of these deficiencies and grouped them into categories of personnel, equipment and organisation of care.4 The last stage consisted of selecting the most urgent of the identified incidents and proposing solutions. Each audit resulted in three or four recommendations, and staff members were selected to follow these up.
For this study, all the data collected from audits conducted between February 2004 and June 2005 were analysed. This consisted of attendance lists, case summary cards and a register of recommendations. A questionnaire was also administrated to the maternity personnel in June 2005 by an external midwife who was writing a thesis for her qualification as health officer in nursing and obstetric care (V.Z.). Permission to perform the study was obtained from the regional health authority. Ethical regulations regarding consent of the participants and the treatment of data (anonymity) were followed. The study included 35 members of staff from the maternity and surgical departments of the district hospital who were interviewed individually: 2 gynaecologists/obstetricians, 15 midwives, 7 auxiliary midwives, 2 assistant surgeons and 9 assistant anaesthetists. All personnel in the maternity unit were interviewed, but six staff from the surgical unit did not respond to the questionnaire because they had never attended an audit session or because they were absent during the interview period. Pretesting of the questionnaire was conducted in a peripheral maternity clinic where the audits had also been introduced. The structured questionnaire encompassed closed- and open-ended questions and was administrated face-to-face by a single interviewer (V.Z.). Questions related to the goals of audit, staff participation, conduct of the audit, incentives, invitation to the sessions, feedback given to the staff, implementation and follow up of recommendations, impact on quality, behaviour changes and suggestions for improvement. The data were analysed using a qualitative and quantitative approach. Results from closed-ended questions were summarised with frequencies, and answers to open-ended questions were coded. Answers were then grouped according to themes to build tables. The analysis was conducted by V.Z. and F.R.
In the 16-month period, 16 sessions were held with an average participation of 17 people, 2 to 3 of whom were involved in the management of the case. A wide range of health personnel attended including midwives, doctors, operating theatre staff, postoperative staff, laboratory and administrative personnel and trainees. All members of staff attended at least once, but few came regularly: for example, only 7 midwives of 15 were regularly present during the study period.
Types of problems identified
Problems were identified in six areas: staff (‘negligence’ or ‘lack of rigour’), equipment (lack of or poor maintenance), medicine and consumables (availability of emergency supplies), protocols (absence or disregard of protocol), organisation (inadequate transmission of information between teams and irregularity of physician’s rounds) and communication (either within the team or between staff and patient) to the patient and her social circle (difficult attitudes of visitors and poverty). The addition of interviews with the woman and her family to the case analysis brought elements to light that were not found in the clinical file: individual experience of the pregnancy, information on preceding deliveries, perceptions of care and caregivers by the woman and her relatives and the real cost of the episode (transport and prescriptions).
Recommendations proposed and implemented
The 49 recommendations resulting from the audit sessions touched on the following subjects: staff attitude (10), equipment (3), medication and consumables (5), protocols (2), care (5), organisation of the service (11), registration of data (6), relationship with the family (3) and communication within the team (5). The recommendations therefore centred mainly on organisation of the service and the staff. The last two categories of recommendations ‘relationship with the family’ and ‘communication within the team’ were only added after the inclusion of the interviews with the women was introduced. Eighty-two percent of the proposed recommendations were implemented, albeit sometimes after several months’ delay.
At the end of year 1, after ten audit sessions, a summary of the recommendations falling under the responsibility of the district management structure was officially handed over to the chief medical officer of the district so that he could take these into account during the setting up of the action plan for 2005.
Perception of the audits by the maternity, medical and auxiliary staff
Midwives represented 43% of the interviewed staff, and 86% of the interviewees had more than 5-year experience. Interviewed personnel had a good knowledge of audit: 86% were able to give a definition of an audit and all the interviewees acknowledged that the goal of audit was to improve quality of care to reduce maternal and neonatal mortality.
Lack of full participation by the staff was explained by various factors. In order of frequency, these were interpersonal communication during the sessions principally between department heads and staff (45% of interviewees), difficulty in practising self-criticism (40%, e.g. ‘women are lying and socio-anthropologists are listening to them’) and a poor attitude towards the audit, which was seen as a way of controlling staff (20%). The most regular attendees at the sessions thought that it was not necessary to receive financial compensation for attending, whereas those who participated less often wanted a fee or at least reimbursement of their travel costs to motivate them to come to the hospital outside their regular working hours. Half the interviewees were dissatisfied with the way the audits were held. Examples of this were, feeling that the attitude of those in charge was too repressive (50%—‘why do they speak loudly to us if they want to improve quality?’), the audit highlighted only the negative aspects of case management (35%), the anonymity of staff involved in the audited case was not respected (20%) and that the case notes with deficiencies from medical doctors were not audited (15%—‘it’s not fair, only cases of midwives are audited, they have never chosen cases of the bosses. They do errors too’).
The staff was also critical of some of the recommendations issued from the audits: problems reported included the lack of follow up of recommendations to evaluate the improvements achieved, the absence of feedback for the staff who did not participate and a weakness at the district level when implementing the recommendations under their responsibility. Despite these identified weaknesses, 77% of interviewees recognised that the audit had a positive influence on their professional practice. Improvements included the systematic notification of all information in the patient’s file, the speed of case management, improved patient reception, improved practice in the delivery room and better communication with the women. They rated it as the principal promoter of change in their working practice after morning staff meetings, training sessions and clinical guidelines. It is not always clear if they changed their practice with the conscience that it will improve management of the woman or only by fear of blame during the audit session (‘I write down every thing I do in the case-notes, I wouldn’t like that the day my case is audited, they accused me of not filling in my case-notes’).
Some particular sessions are still in the minds of the staff: ‘I think always of a sentence of this woman who lost her baby. She said that she lost her baby because of the midwife. The woman described the way she was treated, I was not proud of my profession. The midwife in charge didn’t recognize her error. We are all human beings and we can do effort to change ourselves, she should have recognized her error’. ‘One day, we listened on a tape to a husband’s interview. I can always remember his words, “we came with a baby alive in the womb of my wife and we left with our dead baby in a carton box”. There was a big silence in the room’. At the end of the questionnaire, the interviewee was invited to give some proposals for improving the audits. Suggestions included ‘When the complication occurs, the team is often overloaded. Workload should be acknowledged during the analysis of the case’, ‘to avoid humiliation, it will be better to make the case anonymous or to take older cases’, ‘if they want the activity to continue, doctors should respect more the midwives. To say to one of us “ an auxiliary midwife is better than you” doesn’t stimulate the person to come back’. The proposals are summarised in Table 1.
Table 1. Suggestions given by the interviewed staff to improve audit practice
|To acknowledge the workload in the delivery room during the analysis of the case||10|
|To avoid humiliation during the session and promote mutual respect of the people||9|
|To institute a consultation framework for the staff (regular staff meetings)||8|
|To encourage self-criticism||7|
|To set up a monitoring and evaluation team to follow the implementation of recommendations||6|
|To preserve anonymity of the persons implicated in the management of the woman||5|
|To improve objectivity of the unit chief during the session||2|
|To extend the audits to other units of the district hospital||2|
To persuade a healthcare team to meet once a month to discuss problems encountered in the management of maternity patients, knowing that some of the problems will be blamed on themselves, is an achievement in itself. However, this was not an easy process.
Anonymity and confidentiality
Although the prerequisites were respected (adoption of an audit charter and anonymous presentation of women), we were nevertheless confronted with the consequences of lack of true anonymity, as has been described by other authors,20–22 which limits this type of case review audit. It is clear that even if the name of the woman does not appear in the case presentation, it is easy for the carers involved to recognise the woman, especially if it was recent or unusual. During the discussion, the care team involved was often asked for additional details or made itself known spontaneously to correct certain statements by other participants. Maintaining total anonymity proved difficult. In spite of the charter, and although the moderator repeated at every session that ‘the audit is not a tribunal’, the carer involved was not inclined to admit his/her mistakes in public,23 and it was difficult for the people in charge to retain a neutral attitude when negligence or major errors came to light that had caused physical or psychological complications to the woman or even the death of the neonate. A possible consequence of the loss of anonymity is the loss of confidentiality for the information discussed during the sessions.6 This type of incident occurred at least once in 16 sessions. A midwife who recognised herself during an audit session complained to the parents of a woman. In her interview, the woman had mentioned harsh treatment received from this midwife. There were serious repercussions on the social life of this woman. The interviewers should also be vigilant about the risk of breaking professional confidentiality when writing down statements during individual interviews. How can anonymity and professional confidentiality be guaranteed in an environment of interconnected networks?
Context of limited resources
Hospital teams that have held obstetric audits in the past have all highlighted the necessity for a minimum level of human and material resources before starting out.4,24–26 Insufficient resources impair quality, either directly by the lack of equipment and medicine or indirectly by its detrimental effect on health staff motivation. A second condition to guarantee a successful audit is sufficient income for the health workers. Where wages are low (a Burkina Faso midwife earns officially 130–150 € per month) and inflation is high, any nonpaid activity outside working hours means a serious loss of income if private, paid activities could take place during those hours (private health care, commercial activities, etc.).27 Interviews of the staff revealed that 66% would like financial compensation for their participation in the audits. Facilitating factors for a good integration of audit are summarised in Box 2.
Table Box 2. . Factors that facilitate good audit
| • A minimum level of resources (human and material) to be able to implement the recommendations discussed in audit session|
| • Commitment and support of the administrative authority to solve the problems under their responsibilities (organisation, logistics, and affectation of personnel) and to facilitate the venue of the audit session (comfortable room and compensation for travel costs)|
| • Will and commitment of the maternity staff to improving their practices|
| • Support of hospital team in ‘audit methodology’ (help from an external expert)|
| • Training in communication skills and team management|
Tools for quality versus culture of quality
There are many ways of improving the quality of care, but audit combined with retrospective information seems effective, at least in the short term.1,28 Methods combining audits, training and supervision appear to produce more results, but it is then difficult to ascertain the weight of each activity in the improvement of care. There have not been enough studies to prove the efficacy of methods such as individual clinical training in situ (by a fully trained colleague in the workplace of the caregiver) or the establishment of clinical pathways, but these approaches are promising.28,29 The interactive approach towards the promotion of the WHO Reproductive Health Library in Mexico and Thailand has not shown clear benefits.30 Lomas et al. have demonstrated the importance of leadership in a team for the introduction of new clinical guidelines as opposed to audits only.31 The role model has always played an important part in the training of health professionals: 23distribution of clinical guides and recommendations decided during audit sessions will not change anything if the person in charge does not himself give the example in his clinical practice.
More than any one method, it is the desire of the health staff to change and the context of its implementation that count.32 The context that often prevails in public health institutions, with its rigid procedures, centralised human resources management and decision making, inhibits initiatives by the clinicians.33 If initiatives towards more quality, such as audits, are effective and tolerated by the system within the framework of a particular project, they still do not truly transform the system itself and rarely survive beyond the duration of the project.34 It is therefore important to prepare clinicians for self-evaluation of their practices and quality assurance, starting from their basic training. Professional associations of midwives and gynaecologists should be the driving force for change towards a culture of quality by encouraging their members to respect the principles of medical deontology,35 but often they suffer from the same rigidity and bureaucracy as public administrative bodies.
In the past few years, medical audit has become a tool of choice for quality improvement in health service by managers. Audit methods have been developed, but there is little literature on the practical difficulties of conducting them. In particular, case review audits bring up the problem of lack of confidentiality that can inhibit the climate of confidence and freedom of speech that is necessary within the team for the analysis of the audited case and examination of individual and work environment-related practices. In a context of limited resources, the necessary pre-existing conditions (technical level, qualified personnel and regular and sufficient salaries covering the staff’s basic needs) are generally not met, and this inhibits the integration of audits into the package of activities of the maternity units. Are case reviews a priority when one is faced with the minimum for the management of a complication?21,36 Case reviews are just one method among many to improve the quality of care, and it is the adequacy to the environment that counts more than the tool. The audit method that encourages examination of practices, collegiality, documentation of decisions and evaluation of the results clashes with the bureaucratic organisation culture of the health system. Only a major reorganisation of the care system with increased professionalism and with caregivers taking responsibility for their actions will bring about this culture of quality. The present movement towards institutionalising obstetric audits in developing countries should be accepted favourably while bearing in mind that the risk of falling into a culture of command and control instead of initiative and innovation is ever present.
Contribution to authorship
F.R.: Midwife, MSc, Department of Public Health, Institute of Tropical Medicine, Antwerp, Belgium. F.R. participated in the introduction of audits in the maternity unit of the hospital ‘Secteur 30’, in the conception of the research, collection and analysis of the data. F.R. was the principal author of the article.
C.O.: Gynaecologist–Obstetrician, Department Head of the Maternity Unit of the hospital ‘Secteur 30’, Ouagadougou, Burkina Faso. C.O. participated in the introduction of audits, in the research conception and in the collection and analysis of data in his department. C.O. contributed in writing the article and approved the final version.
V.Z.: Midwife, Lecturer in Obstetric and Nursing Care, Ecole Nationale de Santé Publique, Ouagadougou, Burkina Faso. V.Z. has conceived, managed and analysed the study on the perception of audits by the staff in June 2005. She contributed in writing the article and approved the final version.
F.O.: Anthropologist, UMR 912 (INSERM-IRD-U2) ‘Sciences Economiques et Sociales, systèmes de santé, sociétés’, Marseille, France. F.O. participated in the introduction of audits in the maternity unit of the ‘Secteur 30’ hospital, in the conception of the research, in gathering anthropologic data and in the analysis. F.O. contributed in writing the article and approved the final version.
S.Z.: Sociology Master Degree, UMR 912 (INSERM-IRD-U2) ‘Sciences Economiques et Sociales, systèmes de santé, sociétés’, Marseille, France. S.Z. conducted and analysed the interviews with the women and their families and reported the results during the audit sessions. S.Z. contributed in writing the article and approved the final version.
M.E.G.: Anthropologist, UMR 912 (INSERM-IRD-U2) ‘Sciences Economiques et Sociales, systèmes de santé, sociétés’, Marseille, France. M.E.G. participated in the introduction of audits in the maternity unit of the hospital ‘Secteur 30’, in the conception of the research, in gathering anthropologic data and in the analysis. M.E.G. contributed in writing the article and approved the final version.
V.D.B.: Professor of Public Health, Institute of Tropical Medicine, Antwerp, Belgium. V.D.B. participated in the development of the research, in the analysis of the results and in writing the different versions of the article in collaboration with F.R. V.D.B. reviewed and approved the final version.
Details of ethics approval
Permission to perform the study was obtained from the regional health authority (date of approval 19 April 2005, reference no. 5/00487/MS/RCEN/DRSC). Ethical regulations regarding consent of the participants and the treatment of data were followed according to the World Medical Association Declaration of Helsinki.
This work is based on data gathered within the programme ‘Amélioration de la Qualité et de l’Accès aux Soins Obstétricaux d’Urgence (AQUASOU)’ funded by the French Ministry of Foreign Affairs (Projet FSP 2001-149). We thank the district management team, the maternity and surgery staff for their engagement in the implementation of the audits, the women and their families for the confidence and the team of the Health and Family Directory of the Ministry of Health for its constant support.