SEARCH

SEARCH BY CITATION

Keywords:

  • morbidity;
  • pregnancy;
  • sensitivity;
  • specificity;
  • validation

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Population health datasets are a valuable resource for studying maternal and obstetric health outcomes. However, their validity has not been thoroughly examined. We compared medical records from a random selection of New South Wales (NSW) women who gave birth in a NSW hospital in 2002 with coded hospital discharge records. We estimated the population prevalence of maternal medical conditions during pregnancy and found a tendency towards underreporting although specificities were high, indicating that false positives were uncommon.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Routinely collected population health datasets such as hospital discharge records are a potentially significant source of morbidity data. Such data may aid the evaluation of health-care outcomes and health service utilisation and be of value for research. Previously, there has been some reluctance to use such records because of concerns about their accuracy and reliability. However, the use of linked datasets may help to overcome some of these issues in addition to information from carefully designed validation studies.1 Here, we present the first study to assess the accuracy of hospital discharge data in recording maternal medical conditions in Australia. We have also estimated the prevalence of selected medical conditions, such as diabetes and heart conditions, during pregnancy.

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Data sources

Medical record data were obtained from a statewide validation study of 1200 randomly selected women who gave birth in a New South Wales (NSW) hospital in 2002. To ensure representation of hospitals that provide care for low-risk women, we oversampled from small rural maternity hospitals. A detailed description of the methods used is provided elsewhere.2 Briefly, selected medical records were reviewed and data were recorded on a standardised, detailed, but non-identifying data abstraction form by three clinicians experienced in chart review. The collected medical record data included information from the patients’ notes, including antenatal admissions prior to the birth admission, all medical and nursing notes, laboratory reports and medication charts, regarding: chronic hypertension, pregnancy hypertension (onset > 20 weeks gestation), pre-existing diabetes, gestational diabetes, thyroid diseases, parathyroid diseases, renal disorders, cardiac conditions, lung disease, asthma, nervous system disorders, gallbladder conditions, nutritional and haemolytic anaemias, coagulation disorders, connective tissue disorders, psychiatric disorders, psychotic episodes and deep-vein thrombosis during pregnancy. We made note as to whether these conditions affected the current admission (ie the birth admission) or not. The abstracted medical record data were then merged with the corresponding hospital discharge data from the NSW Admitted Patient Data Collection (referred to as ‘hospital data’). In 2002, hospital data were coded according to the International Statistical Classification of Diseases and Related Health Problems 10th revision – Australian Modification (ICD-10-AM) in a maximum of 21 separate fields for the principal diagnosis and any comorbidities. The ICD-10-AM codes for each condition are available from the authors.

Data analysis

The prevalence of each medical condition during pregnancy (referred to as ‘pregnancy prevalence’) was ascertained from the medical record data. Coders of hospital data are only required to record conditions that affect the current admission, therefore only those conditions affecting the birth admission were used to determine the accuracy of reporting and held as the ‘gold standard’ for comparison to the hospital data.

To determine accuracy of reporting we calculated sensitivity, specificity, positive and negative predictive values and Cohen's kappa statistic for each condition, using the medical record data as the ‘gold standard’. The kappa statistic indicates the agreement beyond chance between the information abstracted from the medical records and what is recorded in the hospital data. A Kappa value of > 0.75 indicates excellent agreement and between 0.40 and 0.75 good agreement beyond that expected by chance alone.3 The positive predictive value (PPV) gives the proportion of patients whose diagnosis is recorded correctly in the hospital data and as such gives an indication of accuracy. All estimates and exact binomial confidence intervals4 were weighted by the inverse of the sampling probabilities to provide unbiased estimates that are representative of the population.

The study was approved by the NSW Department of Health Ethics Committee.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Of the 1200 records selected, 1184 were available for review. The pregnancy prevalence for individual medical conditions ranged from 0.2 per 1000 for acute psychosis to 123 per 1000 for asthma (Table 1). Sensitivity of reporting for each condition in the hospital data, compared to the ‘gold standard’ of review of the medical record, was variable (Table 1). Conditions where the sensitivity was over 80% included: chronic hypertension, pre-existing diabetes, thyroid disease, haemolytic anaemia, coagulation disorders and connective tissue disorders. For the remaining conditions the sensitivity was poor. In contrast, the specificities were uniformly high (98.9–100%) for all conditions investigated, indicating that there were few false positive reports in the hospital data.

Table 1.  Prevalence and accuracy of reporting of maternal medical conditions in the birth admission
ConditionNo. of ‘true’ cases nPregnancy prevalence per 1000 Sensitivity % (95% CI)Specificity % (95% CI)Positive predictive value % (95% CI)Negative predictive value % (95% CI)Kappa coefficient (κ)
  1. All values are weighted according to the probability of sampling.†The number of ‘true’ cases includes any record of the condition in the medical record, not just those affecting the current admission.‡Estimated prevalence for each condition is based on any record or history in the notes.§Incalculable. CI, confidence interval.

Pregnancy hypertension16583.468.2 (58.1–77.2)99.6 (99.1–99.9)94.4 (86.2–98.4)97.2 (96.0–98.1)0.78
Asthma135122.812.3 (0–88.6)98.9 (98.2–99.4)2.0 (0–28.1)99.8 (99.4–100)0.03
Gestational diabetes6947.668.6 (54.8–80.3)100.0 (99.6–100.0)99.7 (90.3–100.0)98.5 (97.6–99.1)0.80
Mental health disorders6343.94.4 (0.0–55.3)100.0 (99.7–100.0)60.3 (0.0–100.0)99.6 (99.0–99.9)0.08
Nutritional anaemias4434.85.7 (0.8–18.6)99.9 (99.5–100.0)73.1 (11.6–99.7)97.0 (95.9–97.9)0.10
Thyroid diseases3425.496.6 (35.3–100.0)99.7 (99.2–99.9)50.0 (14.8–85.2)100.0 (99.7–100.0)0.66
Cardiac diseases3216.722.9 (5.1–53.3)100.0 (99.7–100.0)95.9 (27.9–100.0)99.1 (98.4–99.6)0.37
Chronic hypertension2512.845.6 (16.4–77.2)99.8 (99.4–99.9)71.6 (28.5–96.5)99.5 (98.9–99.8)0.55
Renal diseases1713.147.0 (1.3–97.9)100.0 (99.7–100.0)100.0 (0.0–100.0)99.9 (99.5–100.0)0.64
Haemolytic anaemias1311.493.6 (13.2–100.0)99.4 (98.8–99.8)22.5 (2.6–61.6)100.0 (99.7–100.0)0.36
Nervous system disorders127.86.6 (0.0–87.6)100.0 (99.7–100.0)40.0 (0.0–100.0)99.8 (99.4–100.0)0.11
Pre-existing diabetes113.0100.0 (35.7–100.0)100.0 (99.7–100.0)100.0 (35.7–100.0)100.0 (99.7–100.0)1.00
Gallbladder conditions107.549.3 (6.5–93.0)100.0 (99.7–100.0)90.8 (13.5–100.0)99.8 (99.4–100.0)0.64
Coagulation disorders105.288.9 (0.0–100.0)99.6 (99.1–99.9)8.8 (0.0–64.0)100.0 (99.7–100.0)0.16
Connective tissue disorders 75.8100.0 (0.0–100.0)99.9 (99.4–100.0)6.8 (0.0–88.6)100.0 (99.7–100.0)0.13
Chronic lung disease 41.67.2 (1.0–22.4)99.1 (98.3–99.5)17.3 (2.6–47.7)97.5 (96.5–98.3)0.09
Deep-vein thrombosis 3  0.2025.2 (0.0–100.0)100.0 (99.7–100.0)22.9 (0.0–100.0)100.0 (99.7–100.0)0.24
Parathyroid diseases 2  2.420.0 (0.0–97.3)100.0 (99.7–100.0)§99.9 (99.5–100.0)§
Psychotic episodes 2  0.1528.1 (0.0–100.0)100.0 (99.7–100.0)100.0 (0.0–100.0)100.0 (99.7–100.0)0.44

The PPVs ranged from 2.0 (for asthma) to 100 for diabetes. The medical conditions with a PPV greater than 90% were: pre-existing diabetes, gestational diabetes, pregnancy hypertension, renal disease, cardiac conditions, gallbladder conditions and psychosis. However, the Kappa statistics for these five conditions ranged from 0.37 for cardiac conditions to 1.00 for pre-existing diabetes.

The sensitivity of reporting of medical conditions in NSW is compared with those from comparable studies (Table 2).

Table 2.  Sensitivity of reporting in hospital discharge data compared to the gold standard of medical record review: comparison of our results to other studies of maternal medical conditions
Condition or complicationNSW data (Present study) %Washington State data (Lydon-Rochelle et al. 2005)7%Californian data (Yasmeen et al. 2006)8%
Chronic hypertension464958
Pre-existing diabetes1009575
Gestational diabetes698164
Pregnancy hypertension6871
Renal diseases4712
Cardiac diseases2353
Lung diseases  716
Thyroid diseases9710
Asthma1242
Nutritional anaemias  612

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

This is the first Australian study to report the population prevalence of maternal medical conditions during pregnancy. Our findings on the prevalence of conditions compared favourably to population prevalence data. For example, the prevalence of asthma was 12.3% which is similar to the population prevalence of asthma in Australia of 10.6% in women aged 35–44 years, 15.2% in those aged 25–34 years and 17.0% in those aged 15–24 years.5 The prevalence of chronic hypertension in our study was 1.28%, compared to the Australian population prevalence estimates of 3.9% for women aged 25–34 years and 7.7% for women aged 35–44 years.6 Finally, the prevalence of pre-existing diabetes was 0.3% in the validated data compared to a population prevalence of ‘known diabetes’ of 0.3% for women aged 25–34 years and 0.9% for women aged 35–44 years.6

Although administrative data are used widely for surveillance and research, very few validation studies on the reporting of maternal medical conditions have been conducted. A study of maternal medical conditions in discharge data from Washington State hospitals demonstrated wide variation in reporting but a uniformly small proportion of false positives (0–0.4%).7 Another validation study of Californian hospital discharge data from 1992 showed positive predictive values ranging from 23% to 99%.8 Clearly, it is also vital for Australian discharge data to be validated as there may be local idiosyncrasies in data recording and coding. For instance, since 1998 NSW hospital data have been coded using the ICD-10 system, whereas USA data are still coded using an earlier version (ICD-9). Mapping between the two classification systems is available and if comparisons of validation data between countries proved similar, research findings may be more readily extrapolated internationally. Indeed, most comparisons between the sensitivity of reporting in NSW and USA data were similar (Table 2), indicating that the factors leading to a failure to record morbidities in the hospital records may be similar in both countries.

Of note, prepregnancy diabetes had complete concordance between discharge data and medical record review. The sensitivity and specificity of reporting of chronic hypertension were 45.6% and 99.5%, respectively; these data were comparable to findings in another Australian study of perinatal conditions9 which found 85.7% and 99.8%, respectively, for chronic hypertension but was based on only seven cases. The sensitivity of reporting for cardiac conditions was poor (22.9%); however, the positive predictive value was 95.9%.

Validation studies are expensive and time-consuming to conduct yet they are imperative for valid interpretation of results from studies using population health datasets. Such data are being used increasingly for health outcomes research and this type of validation data will be invaluable. Indeed, validation of hospital discharge data by chart-database comparison studies has been identified by an international consortium as one of the key areas for future research.10 De Coster et al. have also recommended training standards for health record coders that are consistent from country to country to enable international collaboration and comparisons.10 Hospital discharge data are an important resource and it is important to continue to improve the quality of such data. Accurate recording in the medical notes by hospital staff would help to ensure that the sensitivity and PPV of population health data are optimal.

Here we have presented the first validation study of the reporting of maternal medical conditions in NSW hospital discharge data. It is particularly important to validate such data as they are often used to inform and modify health-care policy. We have shown that some conditions, such as diabetes and thyroid disorders, are reported reliably and may be used to answer research questions with some confidence. For other conditions, such as asthma, mental health and disorders of the nervous system, caution in their use should be exercised. The reliability of the reporting of these conditions may, however, be improved by record linkage either over time and/or to other datasets.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

We gratefully acknowledge M. Pym and the medical records departments of NSW hospitals for their assistance.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  • 1
    Holman CD, Bass AJ, Rouse IL, Hobbs MS. Population-based linkage of health records in Western Australia: Development of a health services research linked database. Aust N Z J Public Health 1999; 23: 453459.
  • 2
    Roberts C, Bell J, Ford J, Hadfield R, Algert C, Morris J. The accuracy of reporting of the hypertensive disorders of pregnancy in population health data. Hypertens Preg 2008; in press.
  • 3
    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159174.
  • 4
    Cha S. Calculate sensitivity and specificity. Rochester, Minnesota: Mayo Clinic College of Medicine. 2005.
  • 5
    The Australian Institute of Health and Welfare – Asthma in Australia. http://www.aihw.gov.au/publications/index.cfm/title/10158 2005.
  • 6
    Commonwealth Department of Health and Aged Care. The Australian Diabetes, Obesity and Lifestyle Study (AusDiab). http://www.diabetes.com.au/pdf/AusDiab_Report.pdf 2001; Canberra, ACT.
  • 7
    Lydon-Rochelle MT, Holt VL, Cardenas V et al . The reporting of pre-existing maternal medical conditions and complications of pregnancy on birth certificates and in hospital discharge data. Am J Obstet Gynecol 2005; 193: 125134.
  • 8
    Yasmeen S, Romano PS, Schembri ME, Keyzer JM, Gilbert WM. Accuracy of obstetric diagnoses and procedures in hospital discharge data. Am J Obstet Gynecol 2006; 194: 9921001.
  • 9
    Taylor LK, Travis S, Pym M, Olive E, Henderson-Smart DJ. How useful are hospital morbidity data for monitoring conditions occurring in the perinatal period? Aust N Z J Obstet Gynaecol 2005; 45: 3641.
  • 10
    De Coster C, Quan H, Finlayson A et al . Identifying priorities in methodological research using ICD-9-CM and ICD-10 administrative data: Report from an international consortium. BMC Health Serv Res 2006; 6: 77.