Objective: To compare the National Health Survey (NHS) derived estimates of hospital admissions with the number of hospital separations registered in the National Hospital Morbidity Dataset (NHMD).
Methods: Using the person weights in the NHS, the Expanded Confidential Unit Record File of the 2004–05 NHS was used to derive a population estimate of the number of hospital admissions in the 12 months preceding the conduct of the survey. These estimates, by age and sex categories and whether or not the admission involved an overnight stay, were compared with the number of hospital separations registered in the NHMD.
Results: The number of hospital admissions estimated from the NHS was approximately two thirds the number of hospital separations registered in the NHMD. The discrepancy between the two data sources was greater when hospital episodes did not involve an overnight stay in hospital.
Conclusion: There are systematic differences between the number of admissions/separations derived by the NHS and the NHMD for reasons including the technical difference between a hospital admission and a separation, and the sampling frame and scope of the NHS. Researchers looking for individual level data on hospital utilisation must take note of the differences between NHS and the NHMD, and recognise that there are methods to simulate a representative population by enhancing an existing dataset with information from multiple data sources, thus providing researchers a cost-effective data resource.
Population health surveys collect at repeated intervals, an array of information on demographic and socioeconomic characteristics and health related indices from a representative sample of the population of interest. Such surveys provide a cost-effective resource for epidemiological research, as has been demonstrated by the effective use of the National and State Health Surveys in Australia to examine and monitor risk and disease prevalence and health related activity of Australians and to identify priorities at a population level.1–6
There are wider applications for population based surveys that make available individual level (micro) data to researchers. The availability of microdata permits the impact of policy interventions to be modelled and assessed at the level at which the policy interventions are intended to operate (such as the individual person, household or income unit level, or a hospital, etc).7 In such applications, exact or statistical matching methods are employed to adjust the survey data for misrepresentation (resulting from sampling bias or low response rates) and missing information, thus creating a simulated population representative of the population of interest, to which the policy options are applied.7 The availability of data to inform such adjustments and enhancement ultimately determines the accuracy of the estimates derived from the model.
With the exception of the Western Australian Linkage Study, researchers in Australia generally have no access to data that link at an individual level, information on demographic and socioeconomic characteristics, health status and health service utilisation. The cross-sectional nature of the National Health Survey (NHS) limits its use in examining causal relationships and testing hypotheses, however, the wealth of health-related information collected in the NHS at an individual level, together with its size and scope, makes the NHS an attractive and cost-effective resource for undertaking policy research.4
Adjusting a database and enhancing it using other data sources to generate a simulated base population needs objective and systematic assessment of the representativeness of the base survey data in variables other than the demographic characteristics that are used to allocate survey weights. In this study, we have chosen to focus our attention on hospital admissions, and examine the representativeness of the hospital admissions reported in the NHS, against the number of hospital separations registered in the National Hospital Morbidity Database (NHMD) that registers all hospital separations in Australia (with some exceptions).8 The findings presented in this paper will help researchers decide on the representativeness of the NHS data on hospital utilisation and its appropriateness as a data source for specific research projects examining hospital utilisation.
This study draws on data from the 2004–05 NHS released as expanded Confidential Unit Record Files (CURFs) and accessed via the Remote Access Data Laboratory of the Australian Bureau of Statistics (ABS); and the NHMD 2004–05 accessed via the interactive data cubes available from the Australian Institute of Health and Welfare website.9,10
The NHS series in Australia commenced in 1989–90 and was conducted every five years until the new series was introduced in 2001 and administered triennially.11 The 2004–05 NHS surveyed approximately 0.1% of the Australian population (25,906 people of all ages) living in 19,501 private dwellings.12 The survey was conducted during the period August 2004 to June 2005. Each record in the NHS includes a person weight that is an estimate of the number of persons in the Australian population with the same age and sex characteristics such that the sum of person weights in the NHS equals the total Australian population. In relation to health related action taken, NHS participants were asked whether they had been admitted to a hospital in the last 12 months, and if so the number of admissions during the 12 month period. The term ‘admission’ is not defined further, and the number of admissions up to and including four admissions is entered as a continuous variable with the top-code set at five or more admissions. Excluding records with missing information, the Australian population estimate of the number of hospital admissions in the preceding 12 month period for each sex and age group was calculated as follows:
•where X̂ is the total number of admissions to hospital
•whi is the weight of the ith individual in age-sex group h
•xhi is the number of hospital admissions for the ith individual in age-sex group h
A measure of the sampling error (that is, “the difference between the survey estimate and the value that would have been produced had all dwellings in scope of the survey been included”12) for each age and sex specific estimate is given by the standard error (SE), calculated using the 60 replicate weights attached to each record of the NHS CURF and the jackknife non-parametric method for estimating standard error in Stata.4,13 The 95% confidence intervals (95% CI), which indicate the range within which there is a 95% probability the population value is captured, was calculated as follows:
The NHS derived population estimates of hospital admissions were then compared to the number of separations registered in the NHMD in 2004–05, as obtained via the AIHW online data cubes.10 The NHMD registers all separations or episodes of care in public and private hospitals in Australia, except for those hospitals operated by the Department of Defence, Correctional Services, some private sector hospitals, and all off-shore hospitals. A hospital separation is defined as: “An episode of care for an admitted patient, which can be a total hospital stay (from admission to discharge, transfer or death), or a portion of a hospital stay beginning or ending in a change of type of care (for example, from acute to rehabilitation). Separation also means the process by which an admitted patient completes an episode of care either by being discharged, dying, transferring to another hospital or changing type of care.”14 The NHMD does not include individual level information on income, employment, family relationships, or health status (that is available in the NHS), nor does it include a linking variable that would enable multiple separations to be attributed to the one person; thereby restricting its use per se for purposes such as policy research.
The two data sources were deemed to be different if the number of separations in the NHMD were not within the 95% CI range for the NHS estimate.
Hospital admissions/separations were examined separately by same day/non same day status. A same day admissions does not involve an overnight stay in hospital, whereas a non-same day admission involves staying at least one night in hospital.
Of the total 25,906 persons participating in the 2004–05 NHS, 5,501 persons reported being admitted at least once during the preceding 12-month period. Presented in Table 1 are the number of hospital admissions estimated from the NHS by applying the weights as described above, and the number of hospital separations registered in the NHMD. Of the 6,804,839 separations registered in the NHMD in 2004–05, 84 records were excluded from this analysis due to missing information on sex or age category, leaving a total of 6,804,755; the 4,030,967 hospital admissions determined by the NHS in 2004–05 was just two-thirds of this number.
Table 1. Hospital admissions/separation as determined by the NHS (weighted) and the NHMD, 2004-05.
95% Confdence interval
Age group in years
(97,387 - 173,481)
(151,137 - 243,149)
(165,434 - 232,027)
(180,328 - 271,142)
(175,675 - 272,992)
(234,049 - 341,249)
(185,620 - 284,191)
(154,799 - 238,314)
(19,324 - 64,225)
(1,711,585 - 2,013,484)
(65,689 - 111,928)
(80,851 - 152,774)
(196,194 - 266,149)
(382,268 - 494,752)
(265,640 - 347,951)
(254,909 - 339,996)
(195,017 - 283,423)
(172,443 - 249,869)
(163,099 - 226,967)
(26,335 - 60,612)
(2,048,366 - 2,288,500)
(3,834,628 - 4,227,307)
Stratifying by whether or not the admission involved overnight stay in hospital (Figures 1 and 2) showed that the discrepancy between the two datasets was less when the admission/separation involved an overnight stay in hospital (that is, was non-same day) compared to same day hospital admissions/separations. The non-same day admissions/separations in Figure 1 shows that the NHS estimates are significantly lower than the NHMD estimates in the 0–4 year age group (both sexes), males 65 years and older, and females 55 years and older. The NHS derived estimates in males aged 35–44 and females aged 25–34 years were higher than the NHMD numbers, but not significantly so.
In the case of same day admissions, overall, the NHS numbers are less than half the numbers registered in the NHMD (Figure 2). The two datasets are more compatible in numbers in the younger age groups, including the 0–4 year age group. The differences begin to emerge earlier in the case of females from 15–24 years onwards compared to 35–44 years onwards in males. With both sexes, the difference becomes greater as age increases.
This study clearly identifies that the number of hospital admissions estimated from the NHS are approximately two thirds the number of separations registered in the NHMD. This represents a prima facie discrepancy of more than 2.7 million separations. We examine several possible explanations for this discrepancy in numbers.
First and foremost, there is a technical difference between a hospital admission and a separation. By definition, a hospital separation is a portion of a hospital stay that begins or ends in a change in type of care.14 This means that an admission to hospital as considered by the patient may constitute one or more separations; for example, an admission that included a change in status from acute to rehabilitation will constitute two separations. In 2004–05 in Australia, 303,162 separations from public and private hospitals were a discharge or a transfer to another hospital or psychiatric hospital, with another 72,556 separations being identified as change in type of care and therefore constituted a statistical discharge as opposed to being discharged from hospital.14 These 375,718 separations are likely to contribute to some of the difference between the two data sources.
Also, contributing to the difference is the fact that the NHMD is a register of all inpatient episodes of care and includes episodes that result in death while in hospital, whereas the NHS, by being a survey of persons, will not include hospital admissions that result in death. In 2004–05, there were a total of 70,799 deaths that occurred in public and private hospitals in Australia14 explaining a component of the difference between the NHS and the NHMD. If, as suggested by evidence from Western Australia, persons in their last year of life have a higher number of short stay hospital admissions,15 missing the 70,799 people who died while in hospital may, in fact, mean that we have missed the high number of hospital episodes that they are likely to have had in their last year of life. Exclusion of the admissions that results in death also has implications when undertaking costing studies. Evidence from overseas suggests that there is a concentration of health care expenditure in the last year of a person's life, with decedents comprising of 1% of the population accounting for 28.9% of hospital expenditure.16 If so, the costs associated with the 70,799 deaths that occurred while admitted to hospital are not going to be factored in when using NHS-derived estimates of hospital admissions to determine hospital expenditure.
The sampling strategy determined by the scope of the NHS also contributes to the discrepancy between the two data sources because of the exclusion of non-private dwellings and therefore the elderly living in residential aged care accommodation and nursing homes. This group are particularly susceptible to developing acute medical problems that require hospitalisations.17 Exclusion of the elderly and those living in residential aged care accommodation from the NHS is therefore likely to result in an underestimation of the hospital admissions in the older age group. There is some suggestion of this underestimation by the fact that in Table 1, those aged 75 years or older account for 19.7% and 17.8 % of all hospital separations (males and females, respectively) in NHMD, but only for 12.8% and 11.0% of hospital admissions (males and females, respectively) based on NHS estimates. Given the under-representation of the elderly group (particularly those aged 85 years and older) in the NHS compared to the Australian population (Table 2) and the fact that the elderly are different in their patterns of hospital utilisation, the application of greater weights to the elderly may in fact increase the resulting selection bias.21 The exclusion of the susceptible elderly from the NHS limits the use of the NHS as the single source of data for examining outcomes that are of particular relevant to the elderly; a key consideration for a nation with an ageing population.
Table 2. Proportion of elderly participating in the NHS compared to the Australian population.
Estimated Resident Population 30th June 200419
NHS weighted 2004-0520
The inability to link multiple admissions in an administrative database to one person limits the ability to determine the extent of the error caused by the top code of five or more admissions in the NHS. The Western Australian Data Linkage study reported that between 1994 and 1998, 9.5% of Perth residents had more than five separations per person over the five-year period.22 While this does not inform us of the extent of the error we could expect with the NHS, in the absence of an estimate in the public domain for the proportion of persons having more than five admissions per year, we can speculate from the Western Australian study that the proportion would perhaps be no more than 1–2 % of all admissions.
Other factors contributing to the NHS population estimates being lower than the number of hospital separations registered in the NHMD include the inclusion of hospital in the home episodes (where people are managed in their place of residence) in the NHMD database that may not be recognised by survey participants as a hospital admission, and the potential for recall bias in reporting multiple admissions in a 12-month period particularly when the 12-month period crosses over a calendar year as does the NHS. Recalling the number of admissions since a significant event such as Christmas or the New Year may be easier than remembering the number of admissions during a 12-month period beginning and ending mid year.
In order to understand better the differences between the two datasets, Figures 1 and 2 examine separately the estimates depending on whether the admission/separation involved an overnight stay in hospital. In doing so, it was clear that the discrepancy was greater with same day admissions/separation (not involving an overnight stay), with NHS derived estimates being less than half the number of same-day separations registered in the NHMD. This suggests that there could be reporting bias resulting from the survey participants not considering a same day stay in hospital as an admission. As identified above, and in common with the non-same day admissions (although not to the same extent), there is also a lower number of hospital admissions reported by the elderly in the NHS compared to NHMD registered separations. It is possible to speculate that the effect of under-represented elderly is confounded by older persons being more likely to misclassify a same day admission and also being subject to recall bias. It is also possible to speculate that a component of the higher NHMD separations, particularly in the older age groups, is due to chronic disease resulting in multiple hospital episodes in a given year that are not picked up due to the top coding of this information in the NHS to five or more admissions during the year.
Examining the non-same day admissions separately revealed that the 0–4 year olds show significant differences between the NHS and the NHMD estimates; the NHMD had 80,938 more separations compared to NHS derived estimates of hospital admissions. A significant majority of these extra separations in the NHMD register are likely to be births in hospital resulting in a newborn separation. All newborns in hospital are issued with their own medical record and a Unit Record Number, and therefore results in a hospital separation being registered in the NHMD. In responding to a health survey, it is unlikely that a mother would report the newborn as being admitted to hospital. In 2004, there were 254,200 births registered in Australia.23 This is a much larger number than the 80,938 additional separations in the NHMD for the 0–4 year age group. The 2004–05 NHMD registered 451,139 separations in the principle diagnostic group XV relating to pregnancy, childbirth and the puerperium. Without more information, it is not possible to explain in full the discrepancy in the 0–4 year age group, suffice to note the discrepancy.
This analysis helps identify a systematic difference in the number of admissions/separations derived by the NHS and the NHMD. Key issues identified with the NHS are the under representation of the aged and institutionalised population; the underreporting of same day hospital episodes; the underestimation of non-same day hospital episodes in the 0–4 year age group (possibly explained by hospital births); and the bias resulting from the NHS missing out on the hospital admissions that have death as the mode of separation. The findings that the NHS estimate was higher (but not significantly so) than the number of hospital separations in the NHMD in five of the categories examined (males 35–44 and females 25–34 non same day; and males 5–14, males 15–24, and females 0–4 same day) are possibly chance findings.
The NHS is designed for a specific purpose in terms of providing national benchmark information on a range of health issues including health status and use of health services and facilities, and also allows for trends in health status, health behaviour, and health service utilisation to be examined. When using this rich source of information for other purposes including informing policy, it is important to recognise the representation and the generalisability of the NHS. The survey weights in the NHS adjust the sample to represent the Australian population living in private dwellings in non-very remote areas.24 Unless there is evidence to suggest that hospital utilisation of the Australian population as a whole is correlated with the characteristics used to weight the NHS (i.e. age and sex distribution of those living in private dwellings in non-very remote areas), it is unlikely that the estimates of hospital admissions from the NHS are representative of the population as a whole. Published studies have utilised NHS data in keeping with the intended objective of the NHS, that is, to obtain national benchmarks on particular health issues and to monitor changes in health status and behaviour over time. However, with increasing emphasis on the ageing population and as a consequence of there being no formal attempts to examine particular attributes in relation to national data collections (such as the NHMD), there is the potential for researchers to overlook the limitations imposed by the nature and scope of the NHS.
Despite the limitations in representing the elderly and generalisability beyond persons living in private dwellings, the value of the NHS should not be disregarded. Facilitated by advances in computing technology and power, information collected in the NHS has been complemented and enhanced by other data collections to create databases that can be used for epidemiological and economic analysis and for examining the impact of various policy initiatives.25–28 Until such time privacy and other issues have been overcome, as has been done in Western Australia permitting researchers access to linked data,29 the NHS provides a cost-effective data resource that can be enhanced to simulate the Australian population; which in turn can be used to model the impact of policy interventions and to inform policy decisions.
This work was supported by an Australian Government National Health and Medical Research Council Health Service Research Grant (Grant ID 334114) ‘Modelling the Economics of the Australian Health Care System for Policy Analysis’. Access to the NHS 2004–05 Confidential Unit Record Files was obtained under the Australian Vice Chancellors’ Committee (AVCC) agreement. The 2004–05 hospital separation data were extracted from the interactive NHMD data cubes available on the AIHW website.