Validating a verbal autopsy tool to assess pre-hospital trauma mortality burden in a resource-poor setting

Authors


Abstract

Objectives

To present the validation of a verbal autopsy (VA) tool using inpatient deaths in order to ultimately assess the burden of adult pre-hospital trauma mortality in Lilongwe, Malawi.

Methods

A WHO VA tool was administered at the Kamuzu Central Hospital (KCH) morgue in Lilongwe to family members of inpatient deceased. Two physicians assigned cause of death as ‘trauma’ or ‘non-trauma’ as well as a standard VA cause of death based on the VA tool. These assignments were compared to the ‘gold standard’ of physician review of hospital records using a kappa statistic.

Results

The VA method had near-perfect agreement with the hospital record in determining ‘trauma’ vs. ‘non-trauma’. There was moderate agreement when comparing types of death, for example cardiovascular vs. infectious disease, and limited agreement when comparing specific causes of death.

Conclusion

This VA tool can accurately ascertain trauma-related mortality with almost perfect agreement. The next step is to assess pre-hospital trauma mortality burden using the VA tool to determine whether hospital records underestimate the burden of trauma in the community.

Abstract

Objectifs

Présenter la validation d'un outil d'autopsie verbale (AV) basé sur les décès de patients hospitalisés pour évaluer ultimement la charge de trauma-mortalité pré-hospitalière chez les adultes à Lilongwe, au Malawi.

Méthodes

Un outil d’AV de l’OMS a été appliqué à la morgue de l'Hôpital Central de Kamuzu (KCH) à Lilongwe aux membres de la famille des patients hospitalisés décédés. Deux médecins ont classifié comme «traumatique» ou «non traumatique» la cause de décès, ainsi que selon la cause standard de décès de l’AV sur base de l'outil d’AV. Ces classifications ont été comparées à la référence étant l'analyse des dossiers de l'hôpital par le médecin à l'aide d'une statistique kappa.

Résultats

La méthode de l’AV avait une concordance presque parfaite avec l'enregistrement de l'hôpital pour la détermination de cause «traumatique» et «non -traumatique”. Il y avait une concordance modérée dans la comparaison des types de décès, par exemple: maladies cardiovasculaires versus infectieuses et une concordance limitée dans la comparaison des causes précises de décès.

Conclusion

Cet outil d’AV peut déterminer avec précision la mortalité associée à un traumatisme avec une concordance presque parfaite. L’étape suivante consistera à évaluer la charge de trauma-mortalité pré-hospitaliers en utilisant l'outil d’AV pour déterminer si les dossiers des hôpitaux sous-estiment la charge du traumatisme dans la communauté.

Abstract

Objetivos

Presentar la validación de una herramienta para Autopsias Verbales (AV) utilizando las muertes de pacientes ingresados con el fin de evaluar la carga de mortalidad por trauma prehospitalario en adultos en Lilongüe, Malawi.

Métodos

Se utilizó una herramienta de la OMS para AV en la morgue del Hospital Central de Kamuzu (HCK) en Lilongüe con familiares de pacientes ingresados fallecidos. Basándose en la herramienta, dos médicos asignaron la causa de la muerte como ‘trauma’ o ‘no-trauma’, al igual que la causa estándar de muerte por AV. Utilizando la estadística kappa, estas asignaciones se compararon con el ‘patrón oro’: la revisión de la historia clínica por parte de un médico.

Resultados

El método de AV tenía un acuerdo casi perfecto con la historia clínica a la hora de determinar “trauma” versus “no-trauma”. Existía un acuerdo moderado al comparar los tipos de muerte, por ej. cardiovascular versus enfermedad infecciosa, y un acuerdo limitado al comparar causas específicas de muerte.

Conclusión

Esta herramienta de AV puede determinar de forma precisa, con una concordancia casi perfecta, la mortalidad relacionada con un trauma. El siguiente paso será evaluar la carga de la mortalidad por trauma pre-hospitalario utilizando la herramienta de AV para determinar si las historias clínicas subestiman la carga del trauma en la comunidad.

Introduction

Injuries are a leading cause of global death and disability, with road traffic injuries (RTIs) being the greatest contributor (Peden 2005). Road traffic injuries are estimated to be the second leading cause of lost disability-adjusted life years (DALYs) in developing countries by 2020 (Phillips et al. 1993; Murray & Lopez 1997). Mortality due to RTI in Africa is among the highest in the world, at 28.3 deaths per 100 000 population, while North America has a rate of 14.6 deaths per 100 000 population and Europe at 11.0 deaths per 100 000 population (Ameratunga et al. 2006). Most developed countries measure the burden from trauma-related mortality using vital registration systems, a nationwide civil record of births and deaths plus antecedent causes. However, only 7% of African deaths are recorded via vital registration systems (Mathers et al. 2009). Malawi, like many African nations, does not have a vital registration system, therefore, information on injury or trauma mortality is estimated using police reports and hospital-based trauma registries (Samuel et al. 2012).

In 2008, the University of North Carolina (UNC) in collaboration with Kamuzu Central Hospital (KCH) Department of Surgery established a trauma registry at the KCH casualty department. This provided information on the type and burden of injury presenting to KCH, which mirrored many patterns of injury in the region: primarily young males involved in RTIs and assaults (Samuel et al. 2009). However, while our initial 2008 data set recorded only 3% of patients as brought in dead (BID; also known as Dead on Arrival), we recognised that this was a gross underestimate.

Verbal autopsy (VA) is an indirect method of ascertaining biomedical causes of death from information on symptoms, and circumstances preceding death, obtained from the deceased's caretakers. The VA technique has been used to ascertain neonatal causes of death where the majority of deaths occur without medical supervision (Kumar & Datta 1986; Datta et al. 1988; Bang & Bang 1992; Martinez et al. 1993; Phillips et al. 1993; Chandramohan et al. 1994). We hypothesised that verbal autopsy can be used to determine how many pre-hospital mortalities are due to trauma, and this information would augment our hospital-based data on trauma mortality in a resource-poor setting. We report here the validation of the UNC/KCH verbal autopsy questionnaire.

Materials and methods

Study area and population

Malawi is a landlocked nation in the southeast region of sub-Saharan Africa with a population of 16 million (WHO 2009). This study was conducted at Kamuzu Central Hospital (KCH), an 800-bed tertiary care centre situated in Lilongwe, the capital city of Malawi, with a catchment population of 5 million people in the central region. There are a 4-bed intensive care unit (ICU), a 4-bed surgical high dependency unit (SHDU), and three surgical adult wards, one orthopaedic adult ward and three paediatric surgical wards. There is no pre-hospital care system in Malawi, and minimal basic life support measures are available prior to arrival at hospital.

The KCH mortuary is located on the main hospital grounds. All inpatient deaths are brought to the mortuary by hospital staff, while community deaths or brought-in-dead patients are brought to the mortuary by family members, bystanders or police and kept at the mortuary until families collect the bodies for burial.

Interview process

The UNC/KCH verbal autopsy (VA) study was designed so that interviews of family members of the deceased would occur at the time of body collection. We adapted the World Health Organization (WHO) verbal autopsy tool (WHO 2001) for adult deaths to the local situation; while the WHO VA tool had been previously validated, we validated the modified WHO VA tool in a mortuary setting to assess the burden of trauma mortality. The questionnaire included both open narrative and closed questions. The narrative questions recorded explanations of the circumstances of death, while the closed questions dealt with specific symptoms and conditions. Non-physician interviewers administered the questionnaire at the time of body collection from the KCH mortuary to those who had witnessed the deaths and/or took care of the deceased prior to death. All interviewers had completed secondary school education and were fluent in the local language, Chichewa. They had no medical training to eliminate the possibility of attribution bias and were trained in how to approach grieving respondents. A supervisor coordinated the interviewers' activities, oversaw data collection process and checked questionnaires for completeness, as well as randomly witnessing about 5% of interviews to assure quality control. In most cases, data were collected within 1 week of death. Those respondents who were too despondent to respond immediately were offered a telephone interview at a later date.

Validation of UNC/KCH verbal autopsy questionnaire

We used physician-certified verbal autopsy (PCVA) (Bauni et al. 2011), also called Physician Review (PR) in other studies (Oti & Kyobutungi 2010), to assign cause of death based on the UNC/KCH verbal autopsy questionnaire. Historically, the gold standard for cause of death assignment in a vital registration system is physician or hospital-assigned cause of death (HCOD). To validate PCVA, it was compared against HCOD for the same patient subset.

Physician certification of VA questionnaires

Physicians with experience working in the Malawian environment served as physician coders; however, most were not present in Malawi at the time of coding, thus limiting the burden on their clinical duties in Malawi. These four physicians were trained in the use of the WHO 2007 Verbal Autopsy Standards which code cause of death based on the 10th revision of the International Classification of Diseases (ICD-10) list (WHO 2001). Physicians first assigned whether the death was ‘trauma’ or ‘non-trauma’ and then assigned a specific immediate and underlying cause of death based on verbal autopsy questionnaire data. At least two physicians coded each questionnaire, and when they disagreed, a third physician provided the tiebreak. The causes of death and whether it was trauma or non-trauma related assigned by the physicians were coded and entered into a Microsoft Excel database.

Hospital cause of death (HCOD): the ‘gold standard’

The KCH in-hospital cause of death was collected by a physician member of the VA study team. This physician reviewed the charts of all in-hospital cause of death assigned by hospital personnel, which included both physicians and clinical officers (non-physician providers) in this resource-poor setting. Hospital cause of death was based on standard guidelines (full medical history) and reflected the best judgment of the provider, substantiated by relevant radiological or laboratory investigations. In cases where cause of death was not clearly stated by the provider, the VA study physician reviewed patient's presentation, history, examination and all available laboratory and imaging data. The final diagnosis was coded based on the VA list of causes of death by WHO, which the WHO has correlated with ICD-10 diagnoses (WHO 2001). The physicians collecting HCOD data were separate from the physicians assigning PCVA and were blinded to those assignments.

Assessment of trauma burden

We estimated that the incidence of in-hospital trauma-related mortality was approximately 10% based on review of prior inpatient deaths recorded in the mortuary register. Paired t-test power analysis indicated that we needed at least 40 trauma-related deaths to compare PCVA and HCOD. This meant we required over 400 interviews to validate the UNC/KCH VA questionnaire to assess trauma mortality burden. These interviews were conducted between 12 April 2010 and 20 December 2010.

Ethical approval

The UNC/KCH verbal autopsy study was reviewed and approved by the Malawian National Health Science Research Committee and by the University of North Carolina Ethics Review Board.

Data management and statistical analysis

Hospital Cause of death was used as the gold standard for validating PCVA based on the primary cause of death. Cause-specific mortality fractions (CSMF) were used to measure agreement at the group level, and case-by-case agreement between the methods was measured by Cohen's kappa (Cohen 1960). CSMFs were determined as the proportion of all deaths that were attributed to a specific cause across the HCOD and the PCVA.

Cohen's kappa statistic

We used Cohen's kappa statistic to measure the level of agreement between the PCVA and the HCOD (the gold standard) for the underlying causes of death.

The kappa measure of agreement is stated as:

display math

where P(A) is the proportion of times the raters agreed, and P(E) is the proportion of times the raters were expected to agree by chance alone. Complete agreement corresponds to a kappa of 1, and complete disagreement corresponds to a kappa value of 0. A negative value of kappa would mean negative agreement. We used the following kappa scale to rate the strength of agreement as described previously (Roberts 1998): a κ < 0.21 is considered poor, a κ between 0.21 and 0.40, fair; a κ between 0.41 and 0.60, moderate; a κ between 0.61 and 0.80, good; and a κ > 0.80, very good.

Statistical analysis was performed using Stata version 12 (College Station, TX, USA).

Results

From April to December 2010, 2378 bodies were brought to the KCH morgue. Of these, 1957 were adult deaths, and 1361 were inpatient adult deaths. Family members or caregivers of 461 deceased acquiesced to interviews, and of these, 454 had hospital records and were included in the analysis (Figure 1). The main reasons for refusing the interview were excessive grief and not having time for it. The mean age at death for the inpatient population was 43 years (± 18 years), and 43% were male. On the basis of PCVA, 423 (93%) cases were coded with a cause of death, and 31 (7%) cases were coded as indeterminate.

Figure 1.

Patient selection for validation of verbal autopsy tool.

The majority of deaths were attributed to five categories by both HCOD and PCVA; these were infectious disease, cardiovascular disease, injury, cancer and gastrointestinal disease. The mortality fractions obtained from PCVA were compared to those cited by HCOD (Figure 2). The PCVA mortality fractions were within 1% of the gold standard HCOD for four of the five categories and within two percentage points for the fifth category (gastrointestinal disease). Both PCVA and HCOD designated infectious disease as the highest cause of death with PCVA attributing 238 deaths (52.5%) to infectious disease and HCOD attributing 236 deaths (52.1%). With regard to injury, PCVA attributed 46 (10.2%) deaths to injury, while HCOD designated 47 (10.4%) deaths to injury (Figure 2).

Figure 2.

Cause-specific mortality fractions for 454 adult inpatient deaths.

The Cohen's Kappa (κ) for trauma vs. non-trauma designation between PCVA and HCOD had near-perfect agreement, κ = 0.81 (standard error 0.05) and 96.3% agreement (Table 1). However, for cause-specific code (for example, HIV vs. tuberculosis), κ = 0.21 (Standard Error 0.01) with 26.2% agreement, which is considered fair; for disease categories (for example, infectious disease vs. cardiovascular disease), κ = 0.65 (Standard Error 0.04) with 79.3% agreement, which is considered moderate agreement.

Table 1. Cohen's kappa for physician-certified verbal autopsy vs. hospital cause of death
 Number of recordsAgreement (%)Cohen's KappaStrength
Trauma vs. non-trauma45496.30.81Almost perfect
Verbal autopsy (VA) cause-specific code45426.20.21Fair
VA category code45479.30.65Moderate

Discussion

This mortuary-based study at a tertiary hospital in sub -Saharan Africa validated verbal autopsy to measure pre-hospital deaths and augment the findings of a hospital-based trauma registry. Many previous studies have validated verbal autopsy for mortality estimation in adults (Chandramohan et al. 1994; Kahn et al. 2000; Lulu & Berhane 2005; Kumar et al. 2006), but our study shows that physician-certified verbal autopsy (PCVA) can accurately quantify burden from trauma-related mortality in poor settings and provide an adjunct to hospital-based trauma registries to quantify the community burden of trauma mortality in LMIC. While many people dying outside hospital will not be brought to the mortuary, in an urban setting such as Lilongwe, the mortuary houses the dead while families gather resources to transport the deceased to his or her home village for burial. In Malawian culture, even the poorest families want their dead buried at their home village, so they will use the mortuary to gain time to arrange for transport.

From a public health standpoint, assessing mortality burden due to a certain disease category vs. a specific cause of death is most important. The physician-certified verbal autopsy (PCVA) method reported category-specific mortality within 2% of hospital-certified cause of death (HCOD) for the top five categories of mortality. Therefore, PCVA had broad agreement with HCOD and provides useful information for healthcare allocation of resources.

With regard to burden of trauma mortality, PCVA had near-perfect agreement with HCOD and only varied substantially with regard to specific cause of non-traumatic deaths. This is understandable as the majority of deaths in the validation phase were due to infectious diseases, and in our hospital, many patients present with HIV and other co-infections, which makes the exact cause of death difficult to ascertain. It should be noted that PCVA can be limited by recall bias; if the families of deceased do not accurately remember the events surrounding the deceased's death, it can change the cause of death assigned. For example, the one trauma death where PCVA and HCOD varied in our study was when the deceased's family did not mention that the deceased fell but detailed his history of alcoholism. Whereas his HCOD was a cervical spine injury from the fall, PCAVA attributed his death to alcoholism. For our purposes, PCVA overall provided an accurate assessment of trauma mortality burden based on the validation phase.

Using verbal autopsy in a resource-poor setting presents some key obstacles. First, the language and cultural context are different so that the interviewer has to translate the caregiver's viewpoint to answer the specific questions of the questionnaire. In addition, the basis of verbal autopsy validation uses hospital cause of death as the ‘gold standard’. However, at KCH, as in many other public hospitals in sub-Saharan Africa, healthcare workers are overwhelmed, and in multiple instances, minimal diagnostic methods were used to justify the cause of death assigned in the patient chart. For example, adults with fever during the rainy season were often written to have died of ‘malaria’ without any documentation of a malaria smear or rapid test having been performed. To overcome this limitation, one of the authors reviewed all patient charts for the HCOD to ascertain as best as possible that the hospital-assigned cause of death was reasonable. Taking this into account, the term HCOD as the ‘gold standard’ should be qualified as the ‘best possible’ or ‘most reasonable’ cause of death in a resource-limited setting.

Future directions

Historically, physician review of the verbal autopsy questionnaire (PCVA) is used to assign cause of death. This process is time-consuming, particularly in a setting with a dearth of healthcare providers. More recently, probabilistic modelling has been put forward as a viable methodology to infer cause of death from a pattern of symptoms. We plan on using InterVA, one probabilistic modelling method that has previously been validated (Byass et al. 2003; Oti & Kyobutungi 2010), to assess whether this is a better method than PCVA to assess trauma burden as it is less time-consuming.

Conclusion

Verbal autopsy is an important adjunct to hospital-based trauma registries to assess the burden of trauma mortality in environments without vital registration systems. Our study shows that it can be used to accurately assess trauma mortality burden.

Acknowledgements

This study is part of a long-standing collaboration between the Departments of Surgery at the University of North Carolina, Chapel Hill, USA, and Kamuzu Central Hospital, Lilongwe, Malawi. We are grateful to the leadership and support of Dr. Anthony Meyer and Dr. Arturo Muyco for this continued partnership. Grants from the North Carolina Jaycee Burn Center and NC TRACS supported this work. This work was supported by the Fogarty International Center of the National Institutes of Health under Award Number K01TW009486. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Ancillary