• Open Access

Agreement between ethnicity recorded in two New Zealand health databases: effects of discordance on cardiovascular outcome measures (PREDICT CVD3)

Authors


Correspondence to: Dr Roger J. Marshall, School of Population Health, University of Auckland, Private Bag 92019, Auckland, New Zealand. Fax: +64 9 373 7503; e-mail: rj.marshall@auckland.ac.nz

Abstract

Objectives: To assess agreement between ethnicity as recorded by two independent databases in New Zealand, PREDICT and the National Health Index (NHI), and to assess sensitivity of ethnic-specific measures of health outcomes to either ethnicity record.

Method: Patients assessed using PREDICT form the study cohort. Ethnicity was recorded for PREDICT and an associated NHI ethnicity code was identified by merge-match linking on an encrypted NHI number. Agreement between ethnicity measures was assessed by kappa scores and scaled rectangle diagrams.

Results: A cohort of 18,239 individuals was linked in both PREDICT and NHI databases. The agreement between ethnicity classifications was reasonably good, with overall kappa coefficient of 0.82. There was better agreement for women than men and agreement improved with age and with time since the PREDICT system has been operational. Ethnic-specific cardiovascular (CVD) hospital admission rates were sensitive to ethnicity coding by NHI or PREDICT; rate ratios for ethnic groups, relative to European, based on PREDICT were attenuated towards the null relative to the NHI classification. Conclusions: Agreement between ethnicity was moderately good. Discordances that do exist do not have a substantial effect on prevalence-based measures of effect; however, they do on measurement of the admission of CVD.

Implications: Different categorisations of ethnicity data from routine (and other) databases can lead to different ethnic-specific estimates of epidemiological effects. There is an imperative to record ethnicity in a rational, systematic and consistent way.

Ethnicity is often a key risk factor or confounding variable in epidemiological and public health research and, although it is widely thought that there is no gold standard as ethnicity is not a biological construct,1 in New Zealand and elsewhere self-identified ethnicity has become the accepted way to record the concept.2 In practice, however, ethnicity may be otherwise recorded, for example, as a judgement of an administrative clerk. Misclassification is therefore a possibility. Since misclassification of exposures, confounders and outcomes in epidemio-logical research can lead to bias,3–5 ethnic-specific analyses that adjust for ethnicity are potentially unreliable.

As ethnicity is recorded, either self-identified or otherwise, at different times and on different databases and in different ways, it is important to determine the degree of disagreement and the impact on results of epidemiological analysis. In the United States, studies1,6–11 have shown that agree-ment may not always be good, particularly for non-European ethnicity groups. In New Zealand, there are also substantial differences between ethnicity recorded in hospital records and in surveys12,13 and general practice records.14

A key issue is whether disagreement really makes a great deal of difference to the results of an analysis and especially when ethnic-specific health outcomes are of interest. For mortality rates in New Zealand, at least among Maori and Pacific Island people, it does; undercount in mortality records under-estimates mortality rates.15–17 Breast cancer mortality and admission rates have also been shown to vary depending on how ethnicity is classified.18

New Zealand has an ethnically diverse population predominately of European extraction, but a large indigenous Maori population and immigrant people from the Pacific, the Indian subcontinent and South-East Asia. The issue of how to record ethnicity in research is an ongoing concern.2,16,17 The Ministry of Health19 follows the categorisations of Statistics New Zealand (SNZ), the Government statistical agency,20 by using self-identified ethnicity. There is some criticism of these categor-isations, especially of the use of the broad-brush term ‘Asian’.21

In this paper we assess the agreement between ethnicity as measured in two different New Zealand databases: the National Health Index (NHI) database, which is maintained by the New Zealand Health Information Service (NZHIS), and a general practice cardiovascular decision support system called PREDICT, a web-based system used by more than 400 general practitioners throughout the Auckland region.22 It provides cardiovascular disease (CVD) risk assessments and evidence-based management recommendations according to national guidelines at the point of care and a database for research.

Further, we investigate how sensitive the ethnic-specific summary health measures are to which of these two classification systems, PREDICT or NHI, is adopted. The summary health measures that are considered include prevalence of raised serum cholesterol and blood pressure, high body mass, diabetes, smoking cigarettes, as well as follow-up rates of cardiovascular hospital admissions.

Methods

The analysis is of 18,260 patient records that were entered into the PREDICT database between August 2002 and August 2005. Eligibility for entry in PREDICT follows national CVD risk assessment guidelines:23 men over 45 years old, women over 55 years old and 10 years earlier for Maori, Pacific and Indian people as well as individuals with known elevated risk factors. All individuals with diabetes or prior history of CVD were also eligible.

The NHI database assigns a unique identifier – the NHI number – to each healthcare user. A patient's recorded NHI ethnicity is meant to be by self-identified ethnic category. This categorisation is supposed to be rechecked each time a patient is admitted to hospital or in general practice when patients are required to enrol in a specific primary healthcare organisation. The NHI consists of basic demographic information – date of birth, sex, area of residence and ethnicity – of all individuals in New Zealand who have an NHI number, now considered to be about 98% of the population. The PREDICT database was linked to the NHI through matching the encrypted NHI number, enabling information on all hospital admissions of PREDICT enrollees to be gathered. There were 18,239 individuals who matched and these form the study cohort.

Ethnicity classification in both the NHI and PREDICT systems is meant to follow standardised protocols for ethnicity collection coding and reporting as laid down by the NZ Ministry of Health.19 Ethnicity is self-identified and multiple ethnicities are allowed. The detailed ethnic categorisations were reduced to five summary categories following recommendations of the NZ guidelines group23 and as adopted by PREDICT. These categories are: European, Maori, Pacific, Indian and Other Asian. People were Indian if they were of ethnicities relating to the Indian subcontinent (which included Fijian Indian) while Other Asian comprised people from China and South-East Asia. In the few circumstances where multiple ethnicities were recorded we used a standard system of prioritisation to group codes.19 Where a person was neither Maori, Pacific, Indian or Other Asian, an assignment to European was made. Therefore, although predominately self-identified Europeans, this group may have included some others. Observations with unrecorded ethnicity in either the NHI or PREDICT records were ignored.

Clinical information from the PREDICT database was used to give prevalence of some common CVD risk factors, in particular the prevalence of raised blood pressure (over 160/90 mmHg), type 2 diabetes, high body mass (BMI>30 kg/m2), smoking cigarettes, and raised serum cholesterol (over 7 mmol/l). If a patient had more than one assessment, data from the first were used.

To obtain rates of hospital CVD admissions, the NZHIS hospital database was searched for CVD admissions up to August 2005. We did not exclude individuals with a previous history of CVD. Where multiple CVD hospital admissions occurred, the first was selected. A CVD event was defined as including ischaemic heart disease, ischaemic stroke, cerebral atherosclerosis, and atherosclerotic peripheral vascular disease. The hospital admission analysis is the basis of a masters dissertation24 of one of the authors (ZZ).

Cohen's unweighted kappa coefficient was used to measure agreement between ethnicity coding.25 Displaying agreement was done with scaled rectangle diagrams using SPAN software.26 All other analyses were done using Stata 8 software. Admission rates were calculated as incidence rates, that is, the number of cases divided by person-time. To test for differences between ethnic-specific admission and prevalence, from the two separate measures of ethnicity, required developing our own formulae. These are given in reference 24 or can be supplied on request. Formulae for statistically testing corresponding differences between relative risk measures were not available.

Results

Table 1 shows a cross-tabulation of ethnicity as coded by the NHI and PREDICT systems. The reduction of the cohort of 18,239 to 15,902 arises because individuals with missing ethnicity were ignored. Most (90.9%) of the 2,337 missing values are in the NHI record. From Table 1 there are 1,169 discordances, or 7.4% of the total.

Table 1.  The agreement of ethnicity codes between PREDICT and NHI datasets.
PREDICT ethnicityEuropean/otherMaoriNHI ethnicity PacificIndianOther AsianTotal
European/Other11,5978440125011,783
Maori28882238151,154
Pacific237271,5914031,898
Indian6725333323478
Other Asian12712249390589
Total12,3169361,74443547115,902

Figure 1 demonstrates the extent of agreement using scaled rectangle diagrams for the Maori, Pacific, Indian and Other Asian ethnic groups. They show, by area representing frequency, the extent of overlap of the PREDICT and NHI measures. Considering the relative sizes of the two rectangles, rather more individuals are classified as Maori, Asian, Indian or Pacific in PREDICT than in NHI. This is otherwise clear in Table 1 by noting the marginal totals: there were 218 more Maori, 118 more Other Asian, 43 more Indian and 154 more Pacific people in the PREDICT data than the NHI data. Conversely, there were 533 fewer Europeans. It is also easy to see in Figure 1 that there are substantially more Maori identified by PREDICT who are not recorded as Maori in NHI, than there are individuals who in NHI are recorded as Maori and not recorded as Maori in PREDICT.

Figure 1.

Agreement between NHI and PREDICT ethnicity classifications represented by scaled rectangle diagrams 26(darker rectangle shows NHI ethnicity classification, lighter is PREDICT).

The unweighted kappa coefficient for overall agreement of the data in Table 1 was 0.819 (95% confidence interval 0.809 to 0.829), indicating that the agreement was, on the usual scale for interpreting kappa, “good”.25 Kappa scores were marginally better for women: 0.805 (0.798-0.819) for men and 0.834 (0.819-0.851) for women. Further, agreement improved with age from a kappa of 0.800 (0.783-0.817) in 16-40 year-olds to 0.851 (0.819-0.883) in the over 70 year-olds. There was also a steady improvement in agreement over time, from 0.772 (0.710-0.833) in 2002 to 0.826 (0.808-0.844) in 2005.

Table 2 shows ethnic-specific measures of health calculated using both the PREDICT and NHI ethnicity codes. The CVD admission rates are based on 259 hospital admissions in the PREDICT record with non-missing ethnicity and 266 in the NHI record, over an average 1.13 person-years of followup. The differences between the NHI and PREDICT measures for European and others are, with the exception of elevated blood pressure, all statistically significant. However, in absolute terms the differences are not substantial and statistical significance reflects the large European sample. For Maori, Pacific, Indian and Other Asian categories, there are no statistically significant PREDICT versus NHI differences for any health measures except CVD admission and prevalence of diabetes in Maori. In both these cases the estimates are lower by PREDICT.

Table 2.  Estimates of health measures. CVD admission rates per 1,000 per year (with number of cases) and prevalence percentage of risk factors by using the PREDICT and NHI classifications of ethnicity.
 PREDICTNHIDifference (95% CI)p value
European/Other    
 CVD admission/1000/year12.0 (185)12.7 (191)-0.7 (-1.3 – -0.1)<0.0001
 % obesity (BMI>30)24.725.71.0 (0.7-1.4)<0.0001
 % elevated blood pressure28.228.40.2 (-0.1 – 0.3)0.28
 % smoking cigarettes11.411.80.4 (0.2-0.6)<0.0001
 % diabetes type 29.710.10.4 (0.2-0.6)<0.0001
 % raised cholesterol (>7mmol/l)14.714.4-0.3 (-0.6 – -0.1)0.021
Maori    
 CVD admission rate20.0 (24)28.9 (26)8.9 (2.5-15.3)0.005
 % obesity (BMI>30)60.262.22.0 (0.7-4.6)0.14
 % elevated blood pressure38.540.01.5 (-0.4 – 3.5)0.12
 % smoking cigarettes32.732.1-0.6 (-2.6 – 1.2)0.52
 % diabetes type 224.528.13.6 (1.8-5.3)<0.0001
 % raised cholesterol (>7mmol/l)11.410.0-1.4 (-3.2 – 0.4)0.11
Pacific people    
 CVD admission rate18.3 (34)22.0 (35)3.7 (-3.1 – 10.7)0.23
 % obesity (BMI>30)67.065.6-1.4 (-2.9 – 0.1)0.06
 % elevated blood pressure32.332.0-0.3 (-1.3 – 0.8)0.58
 % smoking cigarettes16.717.50.8 (-0.1 – 1.7)0.08
 % diabetes type 237.538.20.7 (-0.4 – 1.8)0.24
 % raised cholesterol (>7mmol/l)5.96.20.3 (-0.4 –1.1)0.39
Indian    
 CVD admission rate20.9 (11)27.6 (11)-6.7 (-16.8 – 1.2)0.38
 % obesity (BMI>30)17.317.90.6 (-2.8 – 4.1)0.71
 % elevated blood pressure24.326.21.9 (-1.0 – 4.9)0.19
 % smoking cigarettes9.07.6-1.4 (-3.2 – 0.4)0.14
 % diabetes type 234.334.0-0.3 (-3.5 – 2.9)0.86
 % raised cholesterol (>7mmol/l)8.511.22.7 (0.0-5.4)0.05
Other Asian    
 CVD admission rate8.3 (5)12.5 (3)-4.2 (-11.2 – 2.8)0.23
 % obesity (BMI>30)10.07.5-2.5 (-5.3 – 0.4)0.09
 % elevated blood pressure29.428.50.9 (-3.8 – 1.9)0.53
 % smoking cigarettes6.85.7-1.1 (-2.6 – 0.5)0.18
 % diabetes type 222.423.61.2 (-1.5 – 3.8)0.39
 % raised cholesterol (>7mmol/l)9.39.40.1 (-2.6 – 2.8)0.92

Relative risks, taking European as the reference category, are shown in Table 3. Generally, relative risks by both PREDICT and NHI classifications are similar. However, there are some differences. In particular, CVD admission rate ratios for Maori by the PREDICT and NHI systems are quite different: 1.66 (1.04-2.55) and 2.28 (1.45-3.45) respectively. As mentioned in Methods, confidence intervals on relative risk differences cannot be calculated. However, given that the absolute NHI versus PREDICT differences for Maori and European admission are statistically significant, and in the opposite directions, it is reasonable to conclude that rate ratios are statistically different. The NHI admission rate ratios of CVD for Pacific and Indian ethnic groups are also greater than those based on PREDICT and, further, the Other Asian protective effect on CVD risk is greater by the NHI estimate. In summary, the PREDICT CVD risk ratios are generally attenuated towards the null value.

Table 3.  Estimates of relative risk (rate ratios for CVD admission), relative to European/Other, by the PREDICT and NHI classifications of ethnicity.
 PREDICT RRNHI RR
Maori  
CVD admission rate1.66 (1.04-2.55)2.28 (1.45-3.45)
Obesity (BMI>30)2.44 (2.27-2.66)2.42 (2.23-2.63)
Elevated blood pressure1.36 (1.26-1.47)1.41 (1.30-1.53)
Smoking cigarettes2.86 (2.60-3.15)2.71 (2.44-3.00)
Diabetes type 22.54 (2.26-2.85)2.79 (2.48-3.13)
Raised cholesterol (>7mmol/l)0.78 (0.63-0.99)0.70 (0.53-0.92)
Pacific people  
CVD admission rate1.52 (1.02-2.21)1.74 (1.17-2.50)
Obesity (BMI>30)2.71 (2.54-2.88)2.55 (2.39-2.72)
Elevated blood pressure1.14 (1.06-1.22)1.13 (1.05-1.22)
Smoking cigarettes1.46 (1.31-1.64)1.48 (1.32-1.65)
Diabetes type 23.88 (3.58-4.20)3.79 (3.50-4.10)
Raised cholesterol (>7mmol/l)0.40 (0.31-0.52)0.43 (0.34-0.56)
Indian  
CVD admission rate1.74 (0.85-3.18)2.17 (1.07-3.98)
Obesity (BMI>30)0.70 (0.53-0.92)0.70 (0.53-0.92)
Elevated blood pressure0.86 (0.73-1.01)0.92 (0.78-1.08)
Smoking cigarettes0.78 (0.59-1.06)0.64 (0.46-0.89)
Diabetes type 23.55 (3.09-4.07)3.37 (2.92-3.88)
Raised cholesterol (>7mmol/l)0.58 (0.38-0.87)0.78 (0.54-1.23)
Asian  
CVD admission rate0.69 (0.22-1.64)0.55 (0.11-1.64)
Obesity (BMI>30)0.41 (0.28-0.58)0.29 (0.18-0.47)
Elevated blood pressure1.04 (0.92-1.18)1.00 (0.87-1.16)
Smoking cigarettes0.59 (0.43-0.81)0.48 (0.33-0.70)
Diabetes type 22.32 (1.97-2.72)2.34 (1.97-2.77)
Raised cholesterol (>7mmol/l)0.63 (0.43-0.91)0.65 (0.43-1.00)

Discussion

We have demonstrated that, although there is moderately good agreement between ethnicity recording in two NZ health databases, the NHI and the general practice-based PREDICT systems, the disagreement that does exist has implications for the measurement of ethnic-specific risks. Routine general practice recording of ethnicity (not from PREDICT) also diverges from the NHI,14 as does self-reported ethnicity from direct survey questions.12,13 In general, these studies find that the NHI may undercount non-European ethnic groups, in particular Maori, a view supported by an audit of the NHI processes of ethnicity collection.27 Although we do not know the ‘correct’ ethnicity, that there are fewer individuals classified as Maori and Pacific by the NHI system than by PREDICT also suggests that the NHI may undercount. Our discordances are sufficiently large to lead to different measures of health outcomes, at least of CVD admission, depending on which ethnicity coding is used. Differences in ethnicity coding have also been reported internationally to a similar degree1,6,7,28 and have the potential to bias risk estimates.28,29

In New Zealand, the NHI system is a central register that can be used (provided an NHI number is known) to classify a person's ethnicity and, although it appears to undercount Maori and other minority groups, it does provide a consistent yardstick. Undercount of Maori is generally regarded as leading to under-estimation of risks, at least in mortality assessments.15–17 Here we have found that using the apparently undercounted NHI classification of ethnicity leads, for assessment of CVD, to greater admission rate ratios for Maori and other minority groups than by the PREDICT ethnicity classification, a result that is at first sight surprising. However, under-estimated ethnic-specific mortality rates arise because of numerator-denominator bias. That the NHI classification here gives greater admission rate ratios than the PREDICT can be understood by considering the pattern of discordant classifications in Table 1 and Figure 1.

For example, if we presume (in Table 1) that those 822 designated Maori in both datasets are genuinely Maori, then there is an 822/936 (87.8%) probability that an NHI-designated Maori is indeed Maori, while only a 822/1,154 (71.2%) probability for a PREDICT-designated Maori. The discordant classifications may include non-Maori at lower risk of CVD. Measurement of Maori CVD admission in the PREDICT database is attenuated by more non-Maori with lower admission rates. This may also explain the attenuation towards the null of PREDICT-based CVD admission rate ratios for Pacific, Indian and Other Asian (see Table 3).

A closer examination of the discrepant NHI versus PREDICT Maori admissions (20.0 vs. 28.9, Table 2) supports the argument. The NHI Maori admission of 28.9 per 1,000 per year is from 26 cases in 899.2 person years and the value 20.0 for PREDICT is from 24 cases in 1,199.0 person years. For those defined by Maori by both NHI and PREDICT the rate is 28.5 or 22 cases in 772.0 person years. Therefore, the admission for the NHI, but not PREDICT, group is 4/(899.2-772.0)=31.4, similar to the NHI figure. However, for the PREDICT, but not NHI, group the admission rate is low, 2/(1199-772.0)= 4.7, which dilutes the PREDICT admission rate.

The absolute differences between the NHI and PREDICT ethnic-specific prevalence measures were less than for CVD admission. None were statistically significant, with the exception of diabetes prevalence in Maori which is significantly greater by the NHI classification. It may be type 1 error, or possibly also a dilution effect.

Where participants were recorded with more than one ethnicity we have adopted a prioritised system for coding. In fact, there were very few such participants. In the PREDICT system only 36 recorded a valid second or third ethnicity and in the NHI record only 105. An alternative analysis would be to allow these individuals to contribute more than once. However, owing to the statistical problems such an analysis imposes (at least for calculating confidence intervals), we prefer an analysis with unique prioritised ethnicity. The analysis would not, anyway, alter the Maori statistics that we have presented, although relative risk may alter somewhat.

The study is based on a relatively large cohort but it has limitations. The paper concerns measurement error of ethnicity, but the effects of measurement of the health outcomes is not considered. Prevalence of risk factors is probably well measured in the PREDICT database,13 but the admission rates may be problematic due to short follow-up and other potential errors in linking to the central NZHIS hospital database. We have not adjusted for confounders in establishing relative risks but, on the other hand, the point of the paper is less about the actual magnitude of prevalence, admission, or relative risks, and is more concerned with sensitivity to ethnicity coding. Confounding, for instance, by age in the comparisons is anyway not an issue because the NHI and PREDICT comparisons are ostensibly of the same individuals.

Measuring ethnicity appropriately presents many challenges and, in general, reporting how ethnicity is measured in research journals remains poor.30 One aspect of the challenge is reproducibility; for, although there may be contention about what a person's ethnicity actually is, research comparisons are meaningful only if measurements of ethnicity are made consistently. Our research shows that despite the development of ethnicity protocols to standardise data collection and reporting in New Zealand, inconsistencies remain and have potential to mislead epidemiologic analyses.

Acknowledgements

We acknowledge ProCare Health Ltd, a primary health care organisation, and the general practitioners who belong to it. Funding was partly from Project Grant 03/183 of the Health Research Council of New Zealand. Useful comments were made by Tim Kenealy and Maori co-investigator Tania Riddell.

Ancillary