Using verbal autopsy to ascertain perinatal cause of death: are trained non-physicians adequate?

Authors


Corresponding Author C. Engmann, Division of Neonatal-Perinatal Medicine, Department of Pediatrics, University of North Carolina School of Medicine, CB # 7596, 4th Floor, UNC Hospitals, UNC-Chapel Hill, Chapel Hill, NC 27599-7596, USA. E-mail: cengmann@med.unc.edu

Summary

Objectives  To develop a standardized verbal autopsy (VA) training program and evaluate whether its implementation resulted in comparable knowledge required to classify perinatal cause of death (COD) by physicians and non-physicians.

Methods  Training materials, case studies, and written and mock scenarios for this VA program were developed using conventional VA and ICD-10 guidelines. This program was used to instruct physicians and non-physicians in VA methodology using a train-the-trainer model. Written tests of cognitive and applied knowledge required to classify perinatal COD were administered before and after training to evaluate the effect of the VA training program.

Results  Fifty-three physicians and non-physicians (nurse-midwives/nurses and Community Health Workers [CHW]) from Pakistan, Zambia, the Democratic Republic of Congo, and Guatemala were trained. Cognitive and applied knowledge mean scores among all trainees improved significantly (12.8 and 28.8% respectively, P < 0.001). Cognitive and applied knowledge post-training test scores of nurse-midwives/nurses were comparable to those of physicians. CHW (high-school graduates with 15 months or less formal health/nursing training) had the largest improvements in post-training applied knowledge with scores comparable to those of physicians and nurse-midwives/nurses. However, CHW cognitive knowledge post-training scores were significantly lower than those of physicians and nurses.

Conclusions  With appropriate training in VA, cognitive and applied knowledge required to determine perinatal COD is similar for physicians and nurses-midwives/nurses. This suggests that midwives and nurses may play a useful role in determining COD at the community level, which may be a practical way to improve the accuracy of COD data in rural, remote, geographic areas.

Abstract

Utilisation de l’autopsie verbale pour déterminer la cause du décès périnatal: Les agents formés non-médecins sont-ils adéquats?

Objectifs:  Développer un programme de formation à l’autopsie verbale (AV) standardisée et évaluer si son application résulte en des connaissances requises comparables pour classer les causes des décès périnataux par des médecins et des non-médecins.

Méthodes:  Le matériel de formation, les cas étudiés, les scénarios écrits et de simulation pour ce programme d’AV ont été développés au moyen de directives d’AV conventionnelle et de l’ICD-10.

Ce programme a été utilisé pour former des médecins et des non-médecins à la méthodologie de l’AV en utilisant un modèle basé sur la formation des formateurs. Des épreuves de connaissances écrites et de compétences pratiques requises pour classer les causes des décès périnataux ont été administrées avant et après la formation afin d’évaluer l’effet du programme de formation sur l’AV.

Résultats:  53 médecins et non-médecins (sages-femmes infirmières/infirmiers et agents de santé communautaire [ASC]) du Pakistan, de la Zambie, de la République démocratique du Congo et du Guatemala ont été formés. Les scores moyens des connaissances cognitives et pratiques chez tous les stagiaires se sont améliorés de façon significative (12,8 et 28,8% respectivement, p < 0,001). Les scores de connaissances cognitives et pratiques après la formation des sages-femmes, infirmières/infirmiers étaient comparables à ceux des médecins. Les ASC (diplômés d’écoles secondaires avec 15 mois ou moins de formation officielle en soins infirmiers/de santé) ont connu les plus grandes améliorations après la formation pratique avec des résultats comparables à ceux des médecins et des sages-femmes infirmières/infirmiers. Toutefois, les scores des connaissances cognitives des ASC post-formation étaient significativement inférieurs à ceux des médecins et des infirmiers.

Conclusions:  Avec une formation appropriée à l’AV, les connaissances cognitives et pratiques requises pour déterminer les causes des décès périnataux sont similaires pour les médecins et les sages-femmes infirmières/infirmiers. Cela suggère que les sages-femmes et infirmiers pourraient jouer un rôle utile dans la détermination des causes des décès au niveau de la communauté, ce qui pourrait être un moyen pratique pour améliorer la précision des données sur les causes de décès dans les zones géographiques rurales, éloignées.

Abstract

Utilizando la autopsia verbal para establecer la causa perinatal de muerte: es adecuado para ello el personal no-médico entrenado?

Objetivos:  Desarrollar un programa de entrenamiento para autopsia verbal estandarizada (AVE) y evaluar si su implementación resulta en los conocimientos requeridos para clasificar la causa de muerte (CDM) perinatal, comparables entre médicos y no médicos.

Métodos:  Se desarrollaron materiales de entrenamiento, casos de estudio, y escenarios por escrito y simulados para este programa de AV, utilizando las guías convencionales de AV y las ICD-10. Este programa se utilizó para instruir médicos y no-médicos en la metodología de AV utilizando un modelo de “entrenar al entrenador”. Se utilizaron pruebas escritas de conocimientos cognitivos y aplicados para clasificar las CDM perinatales, las cuales fueron administradas antes y después del entrenamiento, con el fin de evaluar el efecto del programa de entrenamiento en AV.

Resultados:  Se entrenaron 53 médicos y no-médicos (enfermeras/comadronas/trabajadores sanitarios comunitarios (TSC)) de Pakistan, Zambia, la República Democrática del Congo y Guatemala. Los puntajes medios en conocimientos aplicados y cognitivos de todos los que recibieron el entrenamiento aumentaron significativamente (12.8 y 28.8% respectivamente, p < 0.001). Los resultados de las pruebas cognitiva y de conocimientos aplicados tras el entrenamiento de las enfermeras/comadronas fueron comparables con aquellos de los médicos. Los TSC (graduados escolares con 15 meses o menos de entrenamiento formal en salud/enfermería) tenían la mejora más grande en el conocimiento aplicado post-entrenamiento, con puntajes comparables con aquellos de los médicos y las enfermeras/comadronas. Sin embargo, los puntajes de conocimiento cognitivo post-entrenamiento para los TSC eran significativamente menores que aquellos de médicos y enfermeras.

Conclusiones:  Con el entrenamiento apropiado en AV, el conocimiento cognitivo y aplicado requerido para determinar las CDM perinatales es similar entre médicos y enfermeras/comadronas. Esto sugiere que las comadronas y enfermeras podrían jugar un papel muy útil en la determinación de las CDM a nivel comunitario, lo cual puede ser una manera práctica de mejorar la exactitud de los datos de CDM en áreas rurales y geográficamente remotas.

Introduction

Early neonatal deaths (death of a newborn <7 days of age) and stillbirths, together known as perinatal deaths, account for 7 m annual deaths worldwide (Lawn et al. 2005, 2006; Stanton et al. 2006; WHO 2007a. Perinatal deaths contribute four percent of the global burden of disease and are the leading cause of the burden of disease measured in disability adjusted life years (DALYs) in low and middle income countries (LMIC) (Lopez et al. 2006). Perinatal deaths are the leading cause of death (COD) among children 0–14 years of age worldwide, and in this age group, cause twice as many deaths as malaria and HIV/AIDS combined (Lopez & Mathers 2006). 98% of these perinatal deaths occur in LMIC, with more than 70% occurring in community settings, often in the home (Bang et al. 2005; Baqui et al. 2006; Sule & Onayade 2006). Furthermore, a high percentage of births and deaths are not recorded in vital registration systems (Carlough & McCall 2005; Lawn et al. 2005; McClure et al. 2007).

Understanding causes of perinatal death, which vary between and within geographic regions, is essential when developing strategies to reduce perinatal mortality. Decisions about health care spending have competing considerations. Paramount among these must be a thorough and accurate understanding of the leading causes of perinatal death. Unfortunately, a major barrier to improving perinatal mortality outcomes and data collection in LMIC is that vital registration systems that include COD are available for only 3% of all perinatal deaths worldwide. Thus, current data on perinatal COD may be inaccurate, and perinatal mortality rates may be underestimated (Jehan et al. 2007).

Several studies have examined causes of community-based perinatal deaths using verbal autopsy (VA) (Hinderaker et al. 2003; Baqui et al. 2006; Setel et al. 2006). VA is a technique used to assign probable COD, based on an interview with a primary caregiver (usually the mother) where health registration systems are weak or unavailable. During the interview, a systematic description of the signs, symptoms, and circumstances preceding the death is recorded. Subsequently, a physician panel removed from the site of death independently assigns COD from the interview data (Marsh et al. 1993; Chandramohan et al. 2005; Baiden et al. 2007). VA provides useful data describing causes of perinatal deaths in developing countries and is an important tool to guide health care programmes and policy in low-resource settings (Whiting et al. 2006).

The utility of VA may be limited by the lack of a standard, widely applicable methodology to determine COD, such as validated, training techniques for interviewers and physician panels (Soleman et al. 2006). To address this deficiency, we designed a study to develop and evaluate a VA training program. Our objectives were twofold: (1) to develop a standardized VA tool and training program and (2) to evaluate whether implementation of this tool and training program, using a train-the-trainer model, would result in comparable knowledge regarding uniform classification of perinatal COD by non-physicians compared to physicians.

Methods

Design and development of the VA tool and training program

The VA tool, training manual, and guidelines for conducting the VA interview were adapted from the previously validated SAmple Vital registration with Verbal AutopsY (SAVVY 2006) instrument (Mswia et al. 2006). The training program consisted of four modules: (1) Causes of stillbirth and early neonatal deaths worldwide; (2) Principles, guidelines, and rules for assigning COD developed from the International Statistical Classification of Diseases and Related Health Problems, tenth revision (ICD-10) (WHO 2005b); (3) Case definitions of maternal and neonatal causes of death with commonly associated signs, symptoms, and complications, and representative case studies to illustrate common perinatal problems in verbal autopsy; and (4) Written tests of cognitive and applied knowledge.

Consistent with ICD-10 classification, the underlying cause of death was defined as the single most important disease or condition which initiated the train of morbid events leading directly to fetal or neonatal death or the circumstances of the accident or violence which produced the fatal injury. The final cause of death was defined as the one factor/disease that was the final event which resulted in the neonatal/fetal death. Contributing factors were all conditions/diseases contributing to neonatal/fetal mortality. Case definitions for the most common underlying, final, or contributing causes of fetal and neonatal death were grouped into maternal and fetal/neonatal causes (Figure 1). These classifications were made to the third character of the ICD-10. To develop these case definitions, all available electronic health and social science reference libraries (including indexed and non-indexed journals) published from 1950-2006 were searched. These included Pub Med/MEDLINE, the Cochrane Reference Library, POPLINE, LILACS, PAHO, African Index Medicus, and EMRO. All papers with an abstract in English were reviewed. Manual reviews of documents and recommendations from the World Health Organization (WHO) and other experts, the Cochrane Reference Library, and monographs on stillbirth and neonatal death from African Ministries of Health and non-governmental organizations such as Saving Newborn Lives and UNICEF were conducted. The training materials were developed and reviewed by an international panel of experts and adapted according to their suggestions. Pilot testing was done using a convenience sample of investigators from the National Institute of Child Health and Human Development Global Network for Women’s and Children’s Health Research (GN).

Figure 1.

 Case definitions for maternal and neonatal/fetal underlying, final, and contributing causes of death.

Study setting

This study was conducted within the GN, a multi-country research network committed to improving the health of mothers and infants, building local capacity for performing research, and strengthening scientific and community partnerships. Participating sites included Argentina, Guatemala, Democratic Republic of Congo (DRC), Zambia, Kenya, India, and Pakistan. This VA study involved sites in Guatemala, DRC, Zambia and Pakistan and was conducted between May 2007 and June 2008.

The VA training methods

A train-the-trainer model was used. Initial training was provided by an expert in ICD-10 classification, rules and guidelines for assignment of COD, and VA methodology. Subsequently, nine trainees (eight physicians and one nurse-midwife) from Pakistan, Zambia, the DRC, and Guatemala (designated national coordinators) were trained by this individual over two days. Physician backgrounds included one pediatrician, two internists and five generalists. Upon returning to their respective countries, these trainees trained an additional 53 trainees including 13 physicians and 40 non-physicians. Physician backgrounds included two pediatricians and 11 generalists. The non-physicians consisted of 21 nurse-midwives and 19 Community Health Workers (CHW). CHW included Lady Health Workers (LHW) from Pakistan with 8–12 years of secondary education and 15 months of government health training and community nurses from Guatemala with less than one year of formal nursing education. Study oversight during local training was provided by VA national coordinators. Additionally, a VA core group (CE, WC, LW, EM and BG) conducted telephone conferences with the VA national coordinators to clarify issues that arose during training.

Evaluation

Pre- and post-training tests were administered immediately before and after the completion of a two-day course to evaluate changes in cognitive and applied knowledge. The tests had two modules: (1) Module A (cognitive knowledge module) consisted of 37 true/false questions to evaluate the theoretical knowledge domain and recall of trainees (Figure 2). These questions included assignment of COD based on internationally accepted ICD-10 rules, case definitions used to determine neonatal deaths, and implementation of VA. (2) Module B (applied knowledge module) consisted of case-scenarios which utilized problem-based learning to assess the application of knowledge and decision-making in a given scenario. These case-scenarios included descriptions of circumstances surrounding a stillbirth or early neonatal death from which assignment of COD had to be determined. This module had a maximum of 36 correct answers.

Figure 2.

Figure 2.

 Module A (Cognitive knowledge): Questions used to assess cognitive knowledge.

Figure 2.

Figure 2.

 Module A (Cognitive knowledge): Questions used to assess cognitive knowledge.

Statistical analysis

Average pre- and post-training test scores were calculated for all participants and by professional background (physician, nurse midwife/nurse and CHW). Differences between pre- and post-training test scores (overall and for each professional group) were analyzed using a paired t-test. To compare scores of both cognitive and applied knowledge modules across professional groups, regression models were calculated using physicians as the reference standard.

Results

All trainees demonstrated significant improvement in cognitive and applied knowledge following training (13 ± 10% improvement in cognitive knowledge and 29 ± 23% improvement in applied knowledge, P < 0.01, Table 1). Nurse-midwives/nurses and CHWs scored significantly lower on cognitive knowledge pre-training tests compared to physicians (64% ± 15% and 50% ± 17%, vs. 75% ± 11%), while post-training, the scores of nurse-midwives/nurses were comparable to those of physicians (77% ± 18%vs. 86% ± 9%, respectively). All groups had similar percentage changes in cognitive knowledge from pre-training test scores.

Table 1.   Scores on pre- and post-tests by professional group
 n Pre-training test score Mean (SD) Post-training test score Mean (SD)Percentage change from pre-training test scoreP-value (change from pre-training test) b
  1. aCommunity health workers include community nurses from Guatemala who received less than one year of formal nursing education and Lady Health Workers (LHW) from Pakistan. LHWs are Pakistani government employees with 8–12 years of education and 15 months basic government health training.

  2. bDifferences between pre- and post-training test mean scores were tested using a paired t-test.

Cognitive knowledge
 All5362 (18)75 (18)13 (10)<0.001
 Physician1375 (11)86 (9)11 (6)<0.001
 Nurse midwife/nurse2164 (15)77 (18)13 (12)<0.001
 Community health worker a1950 (17)64 (19)14 (10)<0.001
APPLIED KNOWLEDGE
 All5244 (23)72 (20)29 (23)<0.001
 Physician1262 (15)77 (10)15 (16)0.009
 Nurse midwife /Nurse2153 (20)75 (19)22 (18)<0.001
 Community health worker a1924 (12)67 (25)44 (22)<0.001

Nurse-midwives/nurses had a 50% greater percentage change from pre-test and attained similar scores to physicians when applied knowledge was assessed. CHWs post-training applied knowledge scores (67 ± 25%) were nearly triple that of their pre-training scores (24 ± 12%). Of the three professional groups, CHWs had the largest percentage change from pre-training test scores, which was approximately double that of nurse midwives and nurses and triple that of physicians. Nevertheless, their post-training test scores remained significantly lower than physicians and nurse midwives/nurses.

By contrast to nurse-midwives/nurses who scored significantly lower on cognitive knowledge pre-training tests compared to physicians, yet post-training had comparable scores to those of physicians (Table 2), CHWs post-training cognitive knowledge test scores remained significantly different compared to those of physicians.

Table 2.   Regression models comparing cognitive knowledge scores across professional groups
ScoreBSEP
  1. B, unstandardized regression coefficient; REF, reference category.

  2. Comparisons between CHW and nurse/midwives on the following: pre-training test (B(SE) = −14.67 (4.65); P = 0.003); post-training test (B(SE) = −12.83 (5.24); P = 0.018); difference score (B(SE) = 1.84(3.16); P = 0.563).

Pre-training test
 Community health worker−25.65.3<0.001
 Nurse/Midwife−10.95.20.041
 PhysicianREF  
Post- training test
 Community health worker−22.36.0<0.001
 Nurse/Midwife−9.55.80.110
 PhysicianREF  
Difference (Post – Pre)
 Community health worker3.23.60.374
 Nurse/Midwife1.383.50.696
 DoctorREF  

Nurse-midwives/nurses applied knowledge pre and post-training test results were comparable to physicians, in contrast to pre-training test scores of CHW which were significantly different to physicians (Table 3). Post-training applied knowledge test scores of all professional groups were comparable.

Table 3.   Regression models comparing applied knowledge scores across professional groups
ScoreBSEP
  1. B, unstandardized regression coefficient; REF, reference category.

  2. Comparisons between CHWs and nurse/midwives on the following: pre-training test (B(SE) = –29.14 (5.15); P < 0.001); post-training test (B(SE) = −7.01 (6.38); P = .277); difference scores (B(SE) = 22.14 (6.16); P < 0.001).

Pre- training test
 Community health worker−38.26.0<.001
 Nurse/Midwife−9.05.9.132
 PhysicianREF  
Post-test
 Community health worker−9.47.4.212
 Nurse/Midwife−2.47.3.746
 PhysicianREF  
Difference (Post – Pre)
 Community health worker28.87.2< .001
 Nurse/Midwife6.67.0.350
 DoctorREF  

Discussion

Using a train-the-trainer model, our standardized educational program resulted in improvement in post-training test scores compared to pre-training test scores among all health workers in all sites. The train-the-trainer model was chosen for several reasons: (1) it utilizes active-mode learning and multimodal techniques involving dialogue between participants and instructors; (2) there is widespread experience with this mode of learning within the Global Network as it has been employed within previous studies (McClure et al. 2007); and (3) it has demonstrated utility as a method of diffusing knowledge (Trabeau et al. 2008). Train-the-trainer models have many advantages including increasing trainers’ knowledge, promotion of ownership, and increasing local capacity. Additionally, the relatively low-cost of this train-the-trainer methodology may make this an attractive consideration for VA scale-up in resource-constrained countries.

One of the major limitations of the widespread use of VA for describing perinatal COD has been the lack of harmonization of perinatal VA tools, field procedures and training, and assignment of COD methods (Soleman et al. 2006). In November 2006 (after the development of our tool used for this study), the WHO tasked an expert panel to harmonize the numerous verbal autopsy tools then in use (Baiden et al. 2006). This effort resulted in the publication of a manual on ascertaining and attributing COD that is currently being used in at least six countries (WHO 2007c). How widely this document will be utilized is uncertain; however its development appears to be an important step in standardizing tools and procedures for VA. The development and evaluation of a standardized training methodology has been a missing link in the widespread use of VA, and our study attempted to address this.

Studies of non-physician or mid-level provider-led clinical services both in LMIC and high income countries suggest that with adequate training, mid-level providers are able to conduct specified clinical tasks previously thought to be solely within the domain of physicians (Orfaly et al. 2005; Burr et al. 2006; Murphy et al. 2008). The paucity of physicians in LMIC has prompted many countries to actively explore and institute service provision by mid-level providers (Braveman & Roemer 1985; Kyriacos et al. 2005; Malawi MOH 2007). The most common method used to determine COD in VA has been by physician panels which generally consist of two to three physicians. These physicians independently determine COD for each death, together discuss differences in their responses, and complete the process by providing a single underlying COD (Fantahun et al. 2006; Garenne & Fauveau 2006). The overall goal of our study was to determine if non-physician providers could be trained to use VA to accurately assign perinatal COD. Our results demonstrate that nurse-midwives/nurses can achieve a level of both cognitive and applied knowledge comparable to that of physicians, to assign COD using VA. This was most evident for the module that tested applied knowledge using case scenarios. These case scenarios were intended to represent the realities of determining community-based COD and require high-order problem-solving skills. Given the critical lack of perinatal mortality data, particularly in rural areas, our results suggest that with uniform training, nurses and midwives may be a valuable resource in determining COD at the community level and may thus contribute to strengthening data systems in rural, remote, geographic areas.

Despite having significantly lower pre-training test scores on both cognitive and applied knowledge modules, CHWs who have considerably less health training than nurses, midwives, and physicians demonstrated a remarkable increase in cognitive and applied knowledge. This increase was most striking in the applied knowledge module where their post-training test scores were nearly triple that of their pre-training test scores and their percentage change from pre-training test scores double that of nurse-midwives/nurses and triple that of physicians. Perhaps surprisingly, given that they had attained comparable applied knowledge scores to other health professionals, CHWs cognitive knowledge post-training test scores were significantly lower than those of physicians and nurse midwives/nurses. This may be because the concepts tested in this module, such as ICD-10 nomenclature and rules, are less familiar to CHW compared to concepts of application of known illnesses tested by case scenarios. It is unclear whether the level of CHW cognitive knowledge may be adequate to qualify them to reliably assign an underlying COD given the timeframe of training provided.

In addition to educating participants on detailed clinical aspects of COD ascertainment, a primary goal of this study was to incorporate in the training program a module which addressed the derivations of various indicators of childhood mortality. These included case definitions, methods, formulae and data sources for the numerators and denominators utilized. We believe that adequate comprehension of these aspects is essential for VA reviewers to understand the relevance and significance of their tasks in COD ascertainment and to understand the impact of their decisions in such cases on the mortality measures which form the overall objective of their activity.

It bears emphasizing that the accuracy in COD ascertainment is largely dependent on the quality of VA data collection, which has its own determinants in terms of interviewer training, respondent characteristics, recall periods and language utilized (Soleman et al. 2006). Therefore, good quality data collection would facilitate accuracy in COD ascertainment, particularly by nurse-midwives, who would apply condition-specific diagnostic algorithms with limited clinical experience in cases where there is ambiguity in the information in the completed VA questionnaire.

The major strengths of this study were the extent of data standardization, the study oversight provided by trainers, and the community-based focus and diversity of the clinical settings. Additionally, each trainee served as his/her own control, minimizing confounders in results. There are a number of limitations to this report. The relatively small sample size for the subject groups, particularly physicians, confers reduced statistical power and generalizability of this study. Although this study evaluated the change in specific short-term cognitive level following the development of a VA training program, the ultimate test of this program is whether in the field, non-physicians can determine COD comparably to physicians. A larger adequately powered study comparing responses by non-physicians to physician panels using this VA tool and educational program may demonstrate broad applicability and is currently underway.

Conclusion

We developed a VA educational package, implemented it using a train-the-trainer methodology, and assessed the short-term change of trainees’ cognitive and applied knowledge following its introduction. All types of health providers showed improvement in post-training test scores compared to pre-training test scores. Mid-level providers (nurses and midwives) demonstrated comparability to physicians in post-training cognitive and applied knowledge test scores. CHWs demonstrated a large increase in post-training test scores when compared to pre-training test scores, although it is unclear whether CHW are adequately equipped to reliably assign perinatal COD using VA. With appropriate training in VA, nurses and midwives may be able to determine perinatal COD with accuracy comparable to that of physicians, and therefore may play a useful role in determining and improving accuracy of COD data in rural, remote, geographic areas.

Acknowledgement

This work was funded by grants from the National Institutes of Child Health and Human Development (U01 HD 40636) and the Bill and Melinda Gates Foundation.

Ancillary