• Open Access

Accuracy of national mortality codes in identifying adjudicated cardiovascular deaths

Authors


Correspondence to:
Dr Linton Harriss, Department of Epidemiology and Preventive Medicine, Monash University, Victoria; e-mail: linton.harriss@monash.edu

Abstract

Objective: This study investigated the sensitivity and specificity of the national mortality codes in identifying cardiovascular disease (CVD) deaths and documents methods of verification.

Methods: A 12-year retrospective case ascertainment of all ICD-coded CVD deaths was performed for deaths between 1990 and 2002 in the Melbourne Collaborative Cohort Study, comprising 41,528 subjects. Categories of non-CVD codes were also examined. Stratified samples of 750 deaths were adjudicated from a total of 2,230 deaths. Expert panels of cardiologists and neurologists adjudicated deaths.

Results: Of the 750 deaths adjudicated, 582 were verified as CVD [392 coronary heart disease (CHD) and 92 stroke] and 168 non-CVD. Estimated sensitivity and specificity of national mortality codes for identifying specific causes of death were: CHD 74.2% (95% CI: 69.8–78.5%) and 97.6% (96.0–99.2%), respectively; myocardial infarction 59.9% (50.9–69.0%) and 94.2% (92.4–96.0%), respectively; haemorrhagic stroke 58.9% (46.0–71.7%) and 99.8% (99.4–100.0%), respectively and; ischaemic stroke 38.7% (20.5–56.9%) and 99.9% (99.6–100.0%), respectively. Misclassification was most common for deaths with primary ICD codes for endocrine-metabolic and genito-urinary diseases.

Conclusions: National mortality coding under-estimated the true proportion of CHD and stroke deaths in the cohort by 13.6% and 50.8%, respectively.

Implications: Misclassification of cause of death may have implications for conclusions drawn from epidemiological research.

The International Classification of Diseases (ICD) is used to classify causes of death recorded on death certificates. These national mortality codes are used widely in epidemiological research to assess population health and inform policy in regards to funding for activities such as health care and research. It is relatively easy and inexpensive to use primary ICD codes in such research compared with collection of clinical data, particularly in large cohorts. However, ICD coding has limited accuracy for identifying specific diagnoses.1–8 Misclassification of cause of death may have major implications for the conclusions drawn from epidemiological research.9,10 Observational studies now often verify outcomes related to cardiovascular disease (CVD) to improve the validity of their results.7,11–14 These studies use varying methods of surveillance to identify possible events. Clinical notes are retrieved from relevant institutions and the event adjudicated according to strict definitions. This system seems to be most easily applied when the cohort has been purposely resourced at an early stage to conduct event verification or when the cohort is relatively small.

In this study, we aimed to examine the performance of national mortality codes in a 12-year retrospective case ascertainment of CVD deaths in the Melbourne Collaborative Cohort Study (MCCS). The purpose was not only to document the methods of verification but also investigate the sensitivity and specificity of the national mortality codes in identifying CVD deaths.

Materials and methods

Study population

The MCCS is a prospective cohort study of 41,528 participants (24,479 women) aged 40–69 years at baseline (1990–1994). It was established to investigate the role of diet and other lifestyle factors in the development of chronic disease. Details of the study design and recruitment strategies have been published previously.15 Deaths were identified through the Victorian Registry of Births, Deaths and Marriages, the Australian National Death Index and the Victorian Cancer Registry. The Cancer Council Victoria's Human Research Ethics Committee approved the study and subjects gave written informed consent to participate and have medical records accessed by investigators. Ethics approval relevant to the follow-up of deaths was also gained from 18 Melbourne public hospitals, 11 Melbourne metropolitan private hospitals, 19 Victorian regional hospitals, Monash University, the Royal Australian College of General Practitioners and the Victorian Institute of Forensic Medicine.

National mortality codes

Australian national mortality codes are assigned by the Australian Bureau of Statistics (ABS). Underlying (primary) and associated (secondary) causes of death are assigned by the relevant medical practitioner/coroner who completes a ‘Standard Medical Certificate of Cause of Death’ (death certificate). From this information, the ABS uses an automated process to code the death according to regulations set by the current ICD edition.16 If the ABS staff do not have sufficient information to be able to allocate codes, a query letter is sent to the certifying doctor requesting further or more specific information.17

Stratified sampling of deaths for verification

Deaths were stratified into three groups on the basis of CVD codes using ICD-9 and ICD-10 (390–459 and I00-I99), respectively (Figure 1).18,19 The ‘Primary CVD’ group consisted of all deaths with a CVD code listed as the primary cause of death. The ‘Secondary CVD’ group consisted of all deaths with a primary cause that was not coded as CVD, and a CVD code listed as the secondary cause of death. The ‘Non-CVD’ group consisted of all deaths without a CVD code listed as the primary or secondary cause.

Figure 1.

Selection process of three groups of deaths for verification using ICD-9 and ICD-10 CVD codes (390–459), respectively, in the Melbourne Collaborative Cohort Study to 31 December 2002.

Cases in the Secondary CVD and Non-CVD groups were classified according to primary cause of death using adapted burden of disease codes (Figure 2).20 All deaths in the Primary CVD group were adjudicated. It was not practical to verify all other deaths in the cohort, therefore a sampling method was followed for the two remaining groups. For the Secondary CVD group, a random sample of 15% of all cases in each disease category was adjudicated (except for ‘malignant and other neoplasms’ where a random sample of 10% of deaths was adjudicated). If ≥20% of deaths in a category were adjudicated as CVD and there were at least two deaths adjudicated as CVD, then all remaining deaths in the disease category were adjudicated. For the Non-CVD group, we adjudicated a random sample of 15% of deaths in each disease category that had been completely adjudicated in the Secondary CVD group. This was based on the assumption that these disease categories were most likely to contain misclassified CVD deaths. If ≥20% of deaths in a category were adjudicated as CVD and there were at least two deaths adjudicated as CVD, then all remaining deaths were adjudicated.

Figure 2.

ICD-9 and ICD-10 primary cause of death codes used to classify death in the Melbourne Collaborative Cohort Study.

Accessing medical information

A standard operating procedure manual was developed to train a team of research nurses in reviewing past medical records and collection of the standardised clinical data required by the adjudicators to ascertain cause of death. Clinical data included hospital admission and discharge summaries, results of pathology (biochemistry, haematology, microbiology, histology, etc) and imaging (CT, MRI, X-ray, US, echocardiogram, angiogram, etc), ECGs and procedure/operation reports. Death certificates were obtained for all cases and where possible, autopsy and police reports. Relevant clinical details were de-identified, coded with a unique subject identification number and scanned into PDF format. A Word document was also generated for each case detailing past medical history in chronological order.

Ascertainment of deaths

De-identified, electronic files of each death were adjudicated by panels consisting of three cardiologists or three neurologists. Deaths with primary ICD codes for cerebrovascular disease (430–438 and I60-I69) or that were considered likely from clinical notes read by research nurses to be stroke-related, were initially sent to the neurologists. All remaining cases were sent to the cardiologists. Neurologists were instructed to refer deaths to the panel of cardiologists (and vice versa) if they believed the presumed cause of death warranted their expertise. Files were sent to the first and second adjudicators from the relevant panel, who independently determined the cause of death or outcome code. If the outcome differed, a third adjudicator was then sent the files. Agreement was required by two adjudicators for each outcome. If this could not be achieved, a decision was reached by consensus. A sequence was followed to ensure that adjudicators were allocated as first, second or third on an equivalent number of occasions.

Mortality definitions used for adjudication of deaths

Mortality definitions were modified from those used in the ‘Second Australian National Blood Pressure (ANBP2)’ study,21,22 the ‘Long-term Intervention with Pravastatin in Ischaemic Disease (LIPID)’ study23,24 and the ‘Multinational Monitoring of Trends and Determinants in Cardiovascular Disease (MONICA)’ project.25,26 Feedback from adjudicators was also incorporated into the final outcome definitions used to classify events (Figure 3).

Figure 3.

Mortality definitions used for adjudication of 750 deaths in the Melbourne Collaborative Cohort Study (1990–2002).

Statistical analysis

Analyses were performed with STATA 9.2 (Stata Corp, College Station, Texus, USA). National mortality codes, with CVD as the primary cause, were used for sensitivity and specificity analyses, with final adjudication by cardiologists and neurologists as the gold standard. Binomial exact 95% confidence intervals (CIs) were calculated for sensitivity and specificity. National mortality codes included ICD-9 and ICD-10 codes, respectively for: all CVD (390–459 and I00-I99); coronary heart disease (CHD) (410–414 and I20-I25); myocardial infarction (MI) (410 and I21-I22); heart failure (428 and I50); all cerebrovascular disease (430–438 and I60-I69); haemorrhagic stroke (430–431 and I60-I61, I690-I691); ischaemic stroke (433–434 and I63, I693); all stroke (all codes for haemorrhagic and ischaemic stroke); and non-cardiovascular disease (all codes excluding 390–459 and I00-I99).

Sampling weights27 were used in calculations of sensitivity and specificity to reflect the stratified sampling by CVD group and, within the Secondary CVD and Non-CVD groups, by disease category (Table 1). For two disease categories in the Secondary CVD group (‘malignant and other neoplasms’, and ‘acute and chronic respiratory diseases’), no adjudicated CVD deaths were observed, and therefore the corresponding disease categories in the Non-CVD group were not sampled. For these disease categories we assumed that there would have been no cases of adjudicated CVD outcomes even if all deaths in these categories had been adjudicated, and sampling weights for the Secondary CVD sample included the number of corresponding cases in the Non-CVD group in the weights numerator. For disease categories ‘nervous system and sense organ disorders’ and ‘other diseases’, adjudicated CVD deaths were observed in the sample from the Secondary CVD group, but too few to trigger adjudication of the corresponding categories in the Non-CVD group. For these disease categories, the calculation of sampling weights was further stratified by whether or not they were adjudicated CVD deaths and, assuming that 10% of the Non-CVD group would have been adjudicated as CVD deaths, included Non-CVD group numbers in the numerator and denominator of the sampling weights.

Table 1.  Nationally coded primary disease categories using 2,230 deaths in the Melbourne Collaborative Cohort Study to 31/12/2002, according to factors used in determining sampling weights for analyses of sensitivity and specificity.aThumbnail image of

Several sensitivity analyses were performed. First, we explored whether our conclusions depended on the assumption that 10% of the Non-CVD group would have been adjudicated as CVD deaths. This was done by calculating new weights (and hence sensitivity and specificity values) based on alternative assumptions of: (i) 0%, and (ii) the proportion of adjudicated CVD deaths observed in the Secondary CVD group for the corresponding disease category. In addition, all analyses were repeated limited to (i) only those cases having hospital/family practitioner notes within 28 days of death, (ii) only those cases requiring two adjudicators, (iii) only those cases having ICD-10 codes and (iv) only those cases with an autopsy report. We also re-analysed sensitivity and specificity by age at death in years (65+, 70+ and 75+) for adjudicated heart failure cases.

Results

A total of 2,339 deaths were identified from baseline to 31 December 2002. After excluding deaths occurring outside Victoria (63) and those coded by the Victorian Cancer Registry (46), 2,230 deaths were considered for verification. There were 589 deaths in the Primary CVD group, 290 in the Secondary CVD group and 1,351 in the Non-CVD group. Coronary heart disease accounted for 345 of all Primary CVD deaths, with 202 coded as MI. Cerebrovascular disease accounted for 119 of all Primary CVD deaths, with 46 coded as haemorrhagic stroke and 15 as ischaemic stroke.

Considerable effort was required in tracking medical information due to the retrospective nature of case ascertainment. Information was collected from more than 140 different hospitals, family practitioner offices and nursing homes over a 30-month period (January 2004 – June 2006). Men accounted for 68% of all deaths (Table 2). Fifty-three per cent of deaths occurred in hospitals, 23% at home, 6% in nursing homes, 12% in an unknown location (suburb details only) and 6% in other locations (e.g. shopping centre, footpath, family practitioner office). Death certificate information was available for all cases. In addition to this, 91% of adjudicated deaths had hospital or family practitioner notes available to assist in the final diagnosis, and 21% had an autopsy report. Five per cent of cases relied on a police report as the only additional source of information, 4% relied on death certificate information only and 10% relied on documents created more than 28 days prior to death. The majority of deaths (59.6%) required only two adjudicators to reach agreement on the outcome.

Table 2.  Characteristics of 750 verified deaths, in subjects participating in the Melbourne Collaborative Cohort Study, according to national mortality code groups using ICD codes for cardiovascular disease.a
CharacteristicsNational mortality code groups
Primary CVDbSecondary CVDcNon-CVDd
 n=589n=119n=42
  1. Notes:

  2. a. National mortality codes determined primarily from death certificate information. National mortality codes for cardiovascular disease (CVD) using ICD-9 and ICD-10 (390–459 and I00-I99), respectively.

  3. b. Primary CVD group consisted of all deaths with a CVD code listed as the primary cause of death.

  4. c. Secondary CVD group consisted of all deaths with a CVD code listed as a secondary cause of death only.

  5. d. Non-CVD group consisted of all deaths without a CVD code listed as the primary or secondary cause.

Country of birth
 Australia3848026
 United Kingdom4031
 Italy929212
 Greece73733
Gender (men/women)407/18275/4427/15
Place of death
 Home148205
 Hospital2947329
 Nursing Home34111
 Other (e.g. shop)3453
 Unknown (suburb details only)79104
Source of information
Hospital or family practitioner notes available53611037
Death certificate only2922
Police report + death cert only2473
Autopsy report available131199
Info >28 days prior to death58143
No. of adjudicators for outcome agreement
Two3417630
Three182338
Round table5294
Reviewed by both cardiologists & neurologists141

In the Primary CVD group, 85.9% of deaths were adjudicated as due to CVD (Table 3). After random samples of deaths were adjudicated from each disease category in the Secondary CVD group, four categories were identified as contributing to the majority of misclassification. All deaths in these categories were adjudicated and included deaths with primary codes for: ‘diseases of the digestive system’ (3 CHD and 1 Other Fatal CVD), ‘endocrine and metabolic diseases’ (30 CHD, 2 Other Fatal CVD and 5 Stroke), ‘genito-urinary diseases’ (5 CHD and 3 Other Fatal CVD), and ‘musculo-skeletal diseases’ (3 CHD and 2 Other Fatal CVD). The proportion of deaths adjudicated as CVD in these categories was 20%, 78.7%, 57.1% and 50%, respectively. When random samples of deaths from these four disease categories were adjudicated in the Non-CVD group, one category was identified to require complete adjudication: ‘endocrine and metabolic diseases’ (15 CHD, 3 Other Fatal CVD and 1 Stroke). The proportion of deaths adjudicated as CVD in this category was 59.4%.

Table 3.  National mortality codes according to adjudicated CVD outcomes in the Melbourne Collaborative Cohort Study.
 Adjudicated outcomesa
 All CVDNot CVD
 CHDOther fatal CVDStroke 
National mortality codesbMIPMISCDOCICNCCOVHFISSAHICHINDSNCVD
  1. Notes:

  2. a. CVD, cardiovascular disease. CHD, coronary heart disease. MI, myocardial infarction. PMI, possible MI. SCD, sudden cardiac death. OC, other coronary death. IC, indeterminate coronary death. NCC, non-coronary cardiac death. OV, other vascular death. HF, heart failure. IS, ischaemic stroke. SAH, subarachnoid haemorrhage. ICH, intracerebral haemorrhage. INDS, indeterminate stroke. NCVD, non CVD.

  3. b. National mortality codes determined primarily from death certificate information.

  4. c. Primary CVD group consisted of all deaths with a CVD code listed as the primary cause of death. One of these deaths was coded as both MI and SAH death, and one death was coded as both OC and IS.

  5. d. Secondary CVD group consisted of all deaths with a CVD code listed as a secondary cause of death only.

  6. e. Total number adjudicated as CVD / Total number adjudicated.

  7. f. Non-CVD group consisted of all deaths without a CVD code listed as the primary or secondary cause.

Primary CVD group (n=589)c78232268394630291441183
Secondary CVD group (n=119)d
Malignant and other neoplasms (0/15)e
Diseases of digestive system (4/20)e
K9221            
K729  1          
K810  1          
K559      1      
Endocrine and metabolic diseases (37/47)e
25001 6 1   1    
2502  1          
2503  1          
25061 1          
2720  1          
27241 1          
2873      1      
E109  2          
E119  5       1  
E149  3     1 1  
E780  4          
E785  2      1   
Genito-urinary diseases (8/16)e
5849  1          
5859   1         
5959  1          
N189     1 1     
N191 1    1     
Musculo-skeletal diseases (6/12)e
7239      1      
M069  1          
M3131            
M348      1      
M889  1          
Nervous system / sense organ disorders (1/3)e
G932         1   
Acute and chronic respiratory diseases (0/8)e
Other diseases (1/2)e
R991            
Non-CVD group (n=42)f
Diseases of digestive system (0/6)e
Endocrine and metabolic diseases (19/32)e
2500613 2     1  
2720  1          
27241 2          
2773  1          
2780      1      
Genito-urinary diseases (0/2)e
Musculo-skeletal diseases (1/2)e
7140      1      

‘Fatal CHD’ accounted for 56.9% of all adjudicated outcomes from the Primary CVD group (Table 4). ‘Other Fatal CVD’, ‘Fatal Stroke’ and ‘Fatal non-CVD’ accounted for a further 14.9%, 14.4% and 14.1%, respectively. The majority of deaths adjudicated as ‘Fatal CHD’ and ‘Other Fatal CVD’ had national mortality codes related to CHD (89.6% and 29.5%, respectively). Similarly, the majority of deaths adjudicated as ‘Haemorrhagic Stroke’ and ‘Ischaemic Stroke’ came from corresponding deaths with national mortality codes for haemorrhagic stroke (74.5%) and ischaemic stroke (41.4%), respectively. National mortality coding identified 345 deaths as CHD and 61 as stroke (haemorrhagic and ischaemic). The adjudication process identified a total of 392 CHD deaths and 92 as stroke.

Table 4.  Adjudicated outcomes of 750 deaths occurring in subjects who participated in the Melbourne Collaborative Cohort Study, according to groups created to select deaths for verification, using national mortality codes for cardiovascular disease.a
 National mortality code groups
 Primary CVDb n=589Secondary CVDc n=119Non-CVDd n=42Total deaths n=750
Adjudicated outcomeAll CVDCHDMIHaem strokeIsch strokeAll strokeAll CeVD   
 n=589n=345n=202n=46n=15n=61n=119   
  1. Notes:

  2. a. National mortality codes determined primarily from death certificate information. All CVD, all cardiovascular disease codes using ICD-9 and ICD-10 (390–459 and I00-I99), respectively. CHD, coronary heart disease. MI, myocardial infarction. SCD, sudden cardiac death. Haem stroke includes subarachnoid and intracerebral haemorrhage. Isch stroke, ischaemic stroke. All Stroke includes Haem stroke, Isch stroke and Indeterminate stroke death. All CeVD, all cerebrovascular diseases.

  3. b. Primary CVD group consisted of all deaths with a CVD code listed as the primary cause of death.

  4. c. Secondary CVD group consisted of all deaths with a CVD code listed as a secondary cause of death only.

  5. d. Non-CVD group consisted of all deaths without a CVD code listed as the primary or secondary cause.

  6. e. Includes one national mortality coded death as haemorrhagic stroke that was adjudicated as both haemorrhagic stroke and MI death.

  7. f. Includes one national mortality coded death as ischaemic stroke that was adjudicated as both ischaemic stroke and other coronary death.

Total fatal CHD events33530019311244215392
MIe7872621137792
Possible MI2320150124
SCD226203116347267
Other coronary deathf85111109
Total other fatal CVD events88265184100
Indeterminate coronary death321126
Non-coronary cardiac death921010
Other vascular death465314252
Heart failure301712032
Total fatal stroke events853421254726192
Ischaemic629211213212031
Haemorrhagic (total)5514141514160
Subarachnoide141212132016
Intracerebral4112929382144
Indeterminate stroke death1001
Other fatal non-CVD events83         
Not a cardiovascular death391645113103
Not a stroke death444374412965

Of the 392 deaths adjudicated as CHD, 300 had primary ICD codes for CHD (76.5%), 35 had primary ICD codes for other types of CVD (9.0%) and 57 had primary ICD codes for non-CVD (14.5%). This suggests that misclassification of non-CVD deaths may be a greater concern than the misclassification of deaths within subclasses of CVD. The majority of additional CHD cases were identified from ICD codes representing ‘diseases of the digestive system’, ‘endocrine and metabolic diseases’, ‘genito-urinary diseases’, ‘musculo-skeletal diseases’ and ‘other diseases’. Deaths from ‘diseases of the digestive system’ included gastro-intestinal haemorrhage (1), hepatic failure (1) and cholecystitis (1). Deaths from ‘endocrine and metabolic diseases’ included diabetes mellitus (29), disorders of lipoprotein metabolism and other lipidaemias (13) and amyloidosis (1). Deaths from ‘genito-urinary diseases’ included renal failure (4) and cystitis (1). Deaths from ‘musculo-skeletal diseases’ included rheumatoid arthritis (1), Wegener's granulomatosis (1) and Paget's disease of bone (1). Deaths from ‘other diseases’ included ill-defined and unknown causes of mortality (1).

Of the 92 deaths adjudicated as stroke, 54 had primary ICD codes representing haemorrhagic or ischaemic stroke (58.7%), 31 had primary ICD codes representing other types of CVD (33.7%) and seven had ICD codes for non-CVD (7.6%). The majority of additional stroke deaths came from deaths previously coded with other forms of cerebrovascular disease (18) and other CVD (13). This suggests that misclassification of deaths within subclasses of CVD is the major issue rather than the misclassification of non-CVD deaths.

Specificity (95% CI) was high in all categories of ICD-coded deaths, ranging from 93.6% (91.7–95.4) for definite MI to 99.9% (99.6–100) for ischaemic stroke (Table 5). Conversely, weighted sensitivities of ICD-coded deaths were relatively low. Sensitivity (95% CI) ranged from 12.5% (0.4–24.6) for heart failure to 83.5% (80.4–86.5) for all CVD. Sensitivity for MI ranged from 47.7% (42.7–52.7) to 59.9% (50.9–69.0) depending on the combination of adjudicated outcome definitions used (definite ± possible ± SCD ± other coronary).

Table 5.  Adjusted and unadjusted sensitivity and specificity of national mortality coded cardiovascular deaths, in subjects who participated in the Melbourne Collaborative Cohort Study. Adjudication by cardiologists and neurologists is gold standard.a
 Un-weightedWeighted
 (750 adjudicated deaths)(2,230 deaths)
Cardiovascular disease deathsnSensitivitySpecificitySensitivitySpecificity
  1. Notes:

  2. a. National mortality codes determined primarily from death certificate information. All CVD, all cardiovascular diseases. CHD, coronary heart disease. MI, myocardial infarction. SCD, sudden cardiac death. All CeVD, all cerebrovascular disease. All Stroke includes Haemorrhagic stroke, Ischaemic stroke and Indeterminate Stroke death. Haemorrhagic stroke includes subarachnoid and intracerebral haemorrhage.

  3. b. Upper confidence interval limit truncated at 100%.

All CVD vs non-CVD (95% CI)58286.950.683.595.0
  (83.9–89.6)(42.8–58.4)(80.4–86.5)(91.7–98.4)
CHD vs non-CHD (95% CI)39276.587.474.297.6
  (72.0–80.6)(83.5–90.7)(69.8–78.5)(96.0–99.2)
MI (definite) vs non-MI (95% CI)9267.478.759.393.6
  (56.8–76.8)(75.4–81.8)(49.1–69.6)(91.7–95.4)
MI (definite/poss) vs non-MI (95% CI)11666.480.359.994.2
  (57.0–74.9)(77.0–83.3)(50.9–69.0)(92.4–96.0)
MI (definite/poss/SCD) vs non-MI (95% CI)38350.497.548.899.5
  (45.3–55.5)(95.4–98.9)(43.8–53.8)(98.8–100)b
MI (definite/poss/SCD/other coronary) vs non-MI (95% CI)39249.297.547.799.5
  (44.2–54.3)(95.3–98.8)(42.7–52.7)(98.8–100)b
Heart failure vs non-heart failure (95% CI)3212.598.212.599.4
  (3.5–29.0)(96.9–99.0)(0.4–24.6)(98.9–100)
All CeVD vs non-stroke (95% CI)9278.392.970.897.8
  (68.4–86.2)(90.6–94.7)(61.4–80.3)(96.7–99.0)
All Stroke vs non-stroke (95% CI)9258.798.953.199.7
  (47.9–68.9)(97.8–99.6)(42.7, 63.5)(99.2–100)b
Haemorrhagic stroke vs non-haemorrhagic (95% CI)6068.399.358.999.8
  (55.0–79.7)(98.3–99.8)(46.0–71.7)(99.4–100)b
Ischaemic stroke vs non-ischaemic stroke (95% CI)3138.799.638.799.9
  (21.8–57.8)(98.8–99.9)(20.5–56.9)(99.6–100)b

Results were similar to those in Table 5 if analysed with weights that assumed there would be no adjudicated CVD in the Non-CVD group for disease categories ‘nervous system and sense organ disorders’ and ‘other diseases’. If we assumed the unlikely scenario that the same proportion of adjudicated CVD observed in the Secondary CVD group was observed in the Non-CVD group for the corresponding disease category, in general, sensitivity was similar or slightly lower than in Table 5. Finally, analyses were re-performed using (i) only those deaths having hospital/family practitioner notes available within 28 days of death, (ii) only those cases requiring two adjudicators, (iii) only those cases having ICD-10 codes and (iv) only those cases with an autopsy report. This reduced the number of cases from 750 to 611, 447, 427 and 159, respectively, however there were no appreciable differences in sensitivity and specificity analyses compared with Table 5.

There were only 32 adjudicated heart failure deaths in the cohort and 17 nationally coded heart failure deaths. When we re-analysed sensitivity and specificity by age at death in years (65+, 70+ and 75+) this reduced the number of adjudicated cases from 32 to 29, 24 and 10, respectively. Weighted sensitivity decreased from 12.5% (all adjudicated heart failure cases) with each increment in age and specificity remained high.

Discussion

Follow-up of all CVD deaths occurring in the MCCS was successfully completed to 31 December 2002. Samples of non-CVD deaths were also reviewed to identify deaths with possible misclassification. For nationally coded deaths with a primary CVD code, 85.9% were adjudicated as CVD. For deaths with secondary CVD codes only, misclassification was most evident for disease categories with primary codes for ‘diseases of the digestive system’, ‘endocrine and metabolic diseases’, ‘genito-urinary diseases’ and ‘musculo-skeletal diseases’. For deaths with no CVD codes, misclassification was most evident in the disease category with primary codes for ‘endocrine and metabolic diseases’. National coding identified those who did not die of CHD or stroke (testing specificity), but was relatively poor in identifying those who did die from these diseases (sensitivity). Compared with deaths adjudicated by panels of cardiologists and neurologists, national coding under-estimated the true proportion of CHD and stroke deaths in the cohort by 13.6% and 50.8%, respectively.

The underestimation of CHD resulted primarily from misclassification of non-CVD deaths rather than misclassification of deaths within CVD subclasses. Conversely, the underestimation of stroke resulted primarily from misclassification within subclasses of CVD rather than the misclassification of non-CVD deaths.

This study had several strengths. Ninety four percent of all deaths occurring in the first 12 years of the study were available for verification. The verification process was rigorous with deaths adjudicated by panels of expert clinicians using internationally accepted definitions. Agreement from at least two adjudicators was required and adjudicators were blinded to outcomes assigned by their colleagues. In the event of disagreement a round table discussion required unanimous agreement. Finally, we attempted to identify misclassification among non-CVD disease categories.

The World Health Organization (WHO) Global Burden of Disease report28 identifies certain mortality codes as ill-defined. These codes are not helpful in providing information on cause of death. Ill-defined CVD codes include those for heart failure, ventricular dysrhythmias, generalised atherosclerosis, and descriptions / complications of heart disease.28 Research involving mortality codes from 105 WHO member states suggests the median proportion of deaths coded to ill-defined CVD causes was 5.3 (2.7–7.7) per cent.29 It has been estimated that 50–95% of ill-defined CVD codes should be reassigned to CHD in countries with high rates of ill-defined CVD codes such as France, Japan, Portugal and Spain.30 Conversely, reassignment was not suggested for countries with low rates of ill-defined CVD codes such as Australia, Canada, Finland, Ireland, New Zealand, Norway, Northern Ireland and Scotland. Of the 589 cases with a primary CVD code in our cohort, 25 (4.2%) had an ill-defined CVD code. Of these, 12 were adjudicated as CHD (48%). This clearly illustrates that ill-defined CVD codes contributed to the under-estimation of CHD in our cohort.

While we attempted to control for biases inherent in case note reviews by using a strict verification process, we were unable to control for the likelihood that a panel of cardiologists would be more likely to code deaths as cardiac and a panel of neurologists would be more likely to code deaths as neurological. This type of bias has been reported in other studies.31 It is likely that the decisions to label some of the cases as ‘misclassified’ by the two expert panels occurred as a result of these biases. Any such biases would affect the sensitivity and specificity values reported.

We excluded 63 interstate deaths identified through the National Death Index (NDI) because these deaths only had codes for the primary cause of death and we required codes for secondary cause of death for stratified sampling. The NDI sources primary cause of death codes from the Australian Bureau of Statistics, and therefore the exclusion of interstate deaths in our study was unlikely to have substantially influenced our findings.

There were no nationally coded ‘sudden cardiac deaths’ (SCD) in our cohort. Although ICD codes for CHD do not encompass SCD, we considered these deaths as a sub-group of CHD in our study because it is believed that most of these deaths are related to coronary disease. Of the 267 deaths considered by the adjudicators to be SCD, the majority (57%) had ICD codes for MI. To test if this had substantial impact, analyses were performed progressively by adding subgroups of CHD (definite MI, possible MI, SCD and other coronary death). This resulted in only small changes in sensitivity and specificity.

There were only 32 adjudicated heart failure deaths in the cohort and 17 nationally coded heart failure deaths (weighted sensitivity 12.5%). When we re-analysed sensitivity by increasing age at death, sensitivity decreased even further. This supports the belief that heart failure can be a convenient diagnostic cause of death especially among the elderly with co-morbid conditions.

Comparison of these results with other studies is difficult due to the heterogeneous nature of methods which have been employed by various research groups, the different populations studied, the numerous combinations of ICD codes used for disease categories, the gold standard employed, and definitions used for adjudication. The degree to which coding has been shown to misclassify events is also a function of the factors that limit comparison between studies. The Cardiovascular Health Study (CHS) is possibly most similar to the MCCS.7 Investigators compared medical record discharge diagnosis with the adjudicated findings of an Events Subcommittee for fatal and non-fatal events. For MI (ICD-9 codes 410, 427.4, 427.5), sensitivity and specificity was 81% and 96%, respectively, with discharge diagnosis underestimating the true number of MI cases (89 vs. 93). For stroke (ICD-9 codes 430, 431, 432, 434, and 436), sensitivity and specificity was 82% and 86%, respectively, with discharge diagnosis again underestimating the true number of stroke cases (79 vs. 87). In our study, MI and stroke cases were also underestimated by ICD coding. Our sensitivity estimates were considerably lower than those in the CHS. A contributing factor to this (and a major strength of our study) was our intention to identify misclassification by sampling non-CVD disease categories.

Norwegian and US studies6,8 compared community-based stroke registries with ICD hospital discharge diagnoses and found that discharge coding for all cerebrovascular diseases (430 to 438) significantly over-estimated the number of stroke cases. This is similar to our findings where the same diagnostic codes (ICD-9 and ICD-10) yielded 119 deaths compared with 92 strokes determined by adjudication. However, this should be expected because codes for all cerebrovascular diseases encompass conditions other than haemorrhagic and ischaemic stroke. When we restricted our analyses to include only these types of stroke, we found that ICD coding underestimated the number of strokes (61 nationally-coded deaths compared with 92 adjudicated deaths).

The retrospective nature of case ascertainment may have impacted on our results. In general, the greater the time lapse between death and follow-up, the more difficult it was to find relevant clinical data pertaining to the death. There were a number of reasons for this including destruction of medical histories, family practitioner relocation or retirement, and change of address by subjects or their practitioner after the baseline visit. To minimise bias from data collection, nurses were trained using standard operating procedures and regular meetings were held to discuss strategies for difficult to retrieve cases. As a result, hospital/family practitioner notes within 28 days of death were available for adjudication for 611 of the 750 cases. Estimated sensitivity and specificity were virtually unchanged when using only these cases.

There were 79 deaths in our study with a primary code for endocrine and metabolic diseases. The majority of these were for diabetes (54) of which 57% were adjudicated as CHD (31). In Part One of the death certificate,17 medical practitioners are instructed to name the ‘Direct cause of death’ on Line 1a (e.g. myocardial infarction), then antecedent causes on Line 1b (the cause of Line 1a, e.g. coronary artery disease), Line 1c (the cause of Line 1b, e.g. hypertension) and Line 1d (the cause of Line 1c, e.g. diabetes mellitus), etc. In this way diabetes may be selected as the underlying (primary) cause of death using the coding rules of the ICD. Diabetes is one of the major risk factors for CVD along with age, smoking, inactivity, hypertension, high blood cholesterol and overweight. As such we feel that in cases where a discrete CVD event precedes the death and diabetes is a pre-existing condition, then the CVD event should be coded as the underlying cause of death and diabetes as the associated cause of death.

The deaths in our cohort did not receive any preferential treatment in the completion of death certificates or mortality coding. Therefore we would expect similar misclassification of CVD codes in the wider community. In general, insufficient and incorrect information on death certificates remains a problem for accurate cause of death coding.17 Death certificates remain the primary source of information for mortality coding and limited physician training and experience is likely to contribute significantly to the misclassification of cause of death.31,32 It is therefore essential that physicians receive adequate training in completion of death certificates and that further studies are performed which can help inform the quality of ABS mortality coding.

In summary, the true proportions of CHD and stroke deaths were underestimated by 13.6% and 50.8%, respectively in our cohort. This misclassification may impact on conclusions drawn from epidemiological research, and for estimations of burden of disease. In addition, the implications for funding of health care resources and research could be substantial. While verification of events is costly and time-consuming, these results reinforce the probable need for this. This is of particular importance given the decline in autopsy rates in many countries.33

Funding

This study was supported by grants from the National Health and Medical Research Council (284476, 209057, 124317 and 251533), a VicHealth Supplement Grant (2003–0759), and a VicHealth Research Fellowship (2002–0191). Cohort recruitment was funded by VicHealth and The Cancer Council Victoria. Further infrastructure support was provided by The Cancer Council Victoria and Monash University.

Acknowledgements

This study was made possible by the contribution of many people and organisations including the original investigators, those who recruited participants, the team who tracked and collected event information, the cardiologists and neurologists who adjudicated deaths, the medical record departments of 53 Victorian hospitals, the Royal Australian College of General Practitioners, the Victorian Institute of Forensic Medicine, and the many Victorian General Practitioners and Medical Specialists who graciously allowed access to their medical histories. We would also like to express our gratitude to the many thousands of Melbourne residents who have, and who continue to participate in the study.

Ancillary