Development and Initial Validation of a Risk Score for Predicting In-Hospital and 1-Year Mortality in Patients With Hip Fractures

Authors


  • The authors have no conflict of interest.

Abstract

Our objectives were to better define the rates and determinants of in-hospital and 1-year mortality after hip fracture. We studied a population-based cohort of 3981 hip fracture patients. Using multivariable regression methods, we identified risk factors for mortality (older age, male sex, long-term care residence, 10 prefracture co-morbidities) and calculated a hip fracture-specific score that could accurately predict or risk-adjust in-hospital and 1-year mortality. Our methods, after further validation, may be useful for comparing outcomes across hospitals or regions.

Introduction: Hip fractures in the elderly are common and associated with significant mortality and variations in outcome. The rates and determinants of mortality after hip fracture are not well defined. Our objectives were (1) to define the rate of in-hospital and 1-year mortality in hip fracture patients, (2) to describe co-morbidities at the time of fracture, and (3) to develop and validate a multivariable risk-adjustment model for mortality.

Materials and Methods: We studied a population-based cohort of 3981 hip fracture patients ≥60 years of age admitted to hospitals in a large Canadian health region from 1994 to 2000. We collected sociodemographic and prefracture co-morbidity data. Main outcomes were in-hospital and 1-year mortality. We used multivariable regression methods to first derive a risk-adjustment model for mortality in 2187 patients treated at one hospital and then validated it in 1794 patients treated at another hospital. These models were used to calculate a score that could predict or risk-adjust in-hospital and 1-year mortality after hip fracture.

Results and Conclusions: The median age of the cohort was 82 years, 71% were female, and 26% had more than four prefracture co-morbidities. In-hospital mortality was 6.3%; 10.2% for men and 4.7% for women (adjusted odds ratio, 1.8; 95% CI, 1.3-2.4). Mortality at 1 year was 30.8%; 37.5% for men and 28.2% for women (adjusted p < 0.001). Older age, male sex, long-term care residence, and 10 different co-morbidities were independently associated with mortality. Risk-adjustment models based on these variables had excellent accuracy for predicting mortality in-hospital (c-statistic = 0.82) and at 1 year (c-statistic = 0.74). We conclude that 1 in 15 elderly patients with hip fracture will die during hospitalization, and almost one-third of those who survive to discharge will die within the year. The determinants of mortality were primarily older age, male sex, and prefracture co-morbidities. Our hip fracture-specific risk-adjustment tool is pragmatic and reliable, and after further validation, may be useful for comparing outcomes across different hospitals or regions.

INTRODUCTION

HIP FRACTURES IN the elderly, most of which are related to osteoporosis, are associated with significant morbidity and mortality and are a substantial burden for both caregivers and health systems.(1–3) The direct costs of hip fracture have been estimated at $650 million dollars a year in Canada(3) and almost $9 billion dollars a year in the United States.(2) There were 340,000 hip fractures in the United States in 2000,(4) and the number of hip fractures is steadily increasing and projected to double within the next 20 years.(2,3) Previous studies have estimated that in-hospital mortality for a patient with a hip fracture is between 4% and 12%,(5–11) whereas 1-year mortality is 12–37%.(5,7–17)

Some of these studies have looked at possible determinants or predictors of hip fracture mortality and considered patients' age, sex, and pre-existing or baseline co-morbidities.(1–3,4–8,11–17) Although most studies agree that older age is independently associated with increased mortality,(5,7,8,11–16) the literature is much less clear about the roles and relative contributions to outcome of variables such as male sex or co-morbidity.(2,5–8,12,13,16,17) Part of the reason for this lack of clarity may have to do with inherent study limitations, such as small non-population-based samples, patient selection bias, the inability to distinguish baseline co-morbidity from in-hospital complications,(18) or less than optimal statistical methods. Given how common and costly hip fractures are, it is important to understand the independent correlates of outcomes in this frail elderly population. This is particularly true for quality improvement initiatives or health systems undertaking profiling or “report cards” of hospitals, because the outcomes of hospital-specific care must be carefully controlled for differences in case-mix or severity of illness of patients.(6,18,19) Otherwise, high-quality evidence-based care delivered to older and sicker populations may be misinterpreted as poor quality of care because of increased rates of unadjusted or inadequately adjusted mortality.

Although there have been a number of published reports describing the various factors associated with increased risk of mortality after a hip fracture,(5–17) there have been very few rigorously developed and executed multivariable risk models capable of predicting either short-term or long-term mortality,(5–7,10,11,13,15,16) and only one previous study has attempted any form of external validation.(15) Moreover, all prior studies have included preoperative as well as intraoperative and postoperative variables to predict mortality,(5–7,10,11,13,15,16) making them unsuitable for the purposes of risk-adjusting or comparing mortality rates across different hospitals or health regions.(18,19)

Accordingly, we undertook a large population-based cohort study with three main objectives: (1) to define the rate of in-hospital and 1-year mortality in patients ≥60 years of age who suffered a hip fracture, (2) to describe the prevalence and burden of co-morbidities that this population had at the time of hip fracture, and (3) to develop and validate a multivariable risk score that would allow for valid and reliable risk adjustment of mortality outcomes across different hospitals and healthcare settings.

MATERIALS AND METHODS

Setting and subjects

Capital Health Region (Edmonton, Alberta, Canada) is the largest integrated health delivery system in Canada, with a population of about 1,000,000 people cared for by ∼1000 primary care physicians and with an annual healthcare budget of almost $2 billion dollars.(20) In addition, it serves a cachement area of suburban and rural patients throughout Northern and Central Alberta; this referral base includes another 600,000 people. All patients within Capital Health and its outlying cachement areas who suffer a hip fracture are admitted to one of two comparably large tertiary care hospitals, the University of Alberta Hospital (UAH) and the Royal Alexandra Hospital (RAH). By virtue of the single-payor Canadian health care system, all hip fracture patients are fully insured and guaranteed universal access for their surgery and all other required hospital and medical services; the two study hospitals were the only sites of hip fracture repair within Capital Health and its outlying cachement areas during the study. Therefore, our cohort is a population-based representation of all patients with a hip fracture in northern and central Alberta. This study was approved by the Health Research Ethics Board of the University of Alberta.

During the study period, from March 1994 to February 2000, 4440 consecutive hip fracture patients were admitted and treated at one of these two regional hospitals. From this population-based cohort, we included 3981 patients ≥60 years of age in this analysis. We excluded 459 patients (10%) from consideration because they were <60 years of age, had multiple traumatic fractures that also included a hip fracture, had pathologic hip fractures, or had bilateral hip fractures.

Data sources and measurements

The primary sources of data for this study were the electronic administrative discharge records from each hospital. Hip (that is, femoral neck, intertrochanteric, subtrochanteric, or subcapital) fracture patients were identified according to International Classification of Disease (ICD), ninth revision, categories 820.0-820.9.(21) These records also contained some sociodemographic information (e.g., age, sex, long-term care residence) and clinical information that consisted of physician-assigned co-morbidity data. A particular advantage of Canadian hospital discharge data is the presence of a “diagnosis-type” indicator that specifically distinguishes between diagnoses present at the time of hospital admission (i.e., a prefracture or baseline co-morbidity) and diagnoses arising after hospitalization (i.e., a complication).(18,21)

For this study, we were only interested in baseline co-morbidity, because our purpose was to develop a risk-adjustment model that would adjust outcomes for patient case-mix or preoperative severity of illness. All baseline co-morbidities with at least 0.5% or greater prevalence were classified according to their respective ICD-9 Major Diagnostic Codes (see Appendix for a detailed list). It should be emphasized that the prefracture co-morbidities recorded were based on each treating physicians' documentation of what was considered to be a clinically important co-morbidity at the time of presentation; this is true of all administrative databases. Therefore, to validate the quality of these electronic administrative databases and the co-morbidities recorded, we conducted an independent medical chart review of the key variables listed in the Appendix in a random sample of ∼5% of included hip fracture patients (n = 222). The overall agreement rate between the administrative records and independent medical chart review exceeded 90% for each and every variable included in the Appendix.

Outcomes

The primary outcome of interest, inpatient mortality, was obtained from the aforementioned electronic discharge records. We linked Alberta Health Vital Statistics databases to the unique patient identifiers from our hospital discharge records to verify inpatient mortality and to determine the main secondary outcome, 1-year mortality. The Vital Statistics databases that we accessed are considered among the most reliable and accurate in Canada and have been used in a number of previous studies with both inpatient and 1-year (or longer) mortality as outcomes after discharge from hospital.(22,23)

Analysis

We present descriptive statistics (frequencies and medians, as appropriate) stratified by sex and by site of treatment. All baseline co-morbidities were dichotomous variables, that is, they were either present or absent. Because our primary purpose was to develop a risk-adjustment model for in-hospital mortality that would be useful for cross-hospital comparisons, we derived our model using data from one hospital (RAH, the training set) and validated this model with data from the second hospital (UAH, the validation set). We calculated univariable associations between potential independent correlates and the outcome of inpatient mortality using logistic regression. Nevertheless, we considered all variables as candidates for potential inclusion in our risk-adjustment model, irrespective of conventional statistical significance. We forced age (in decades) and sex into these models, entered all of the other variables using a multivariable backward selection procedure for logistic regression, and used p < 0.05 for retention in the model. We tested for interaction terms between age and sex, and each of these terms with the included co-morbidities, and none achieved statistical significance (p values all >0.10); thus, no interaction terms were included in the final models. Model goodness-of-fit was evaluated using the Hosmer-Lemeshow test statistic,(24) and overall predictive accuracy of the model was assessed using the c-statistic, which is equivalent to the area under the receiver operating characteristic (ROC) curve.(25) As a general rule of thumb, c-statistics between 0.70 and 0.79 are considered acceptable and between 0.80 and 0.89 are considered excellent.(24) We then converted the final multivariable logistic regression model into an easy-to-use risk score by taking the regression coefficient of each variable, multiplying by 10, and rounding off to the nearest integer.(26) We summed the risk scores of all variables in our final model to obtain the total risk score for each patient and present mortality according to quartiles of this calculated score. Last, we plotted predicted versus observed mortality to evaluate model calibration.(24) The same models and risk-scoring strategy were used to examine the association between baseline co-morbidity and 1-year mortality. To offset issues around the potential for over-fitting with respect to the training set (RAH), we present our final multivariable results with data from the validation set only, that is, UAH patients. All analyses were performed using SPSS version 11 (SPSS, Chicago, IL, USA).

RESULTS

Over the 6 years of our study, 3981 eligible patients ≥60 years of age were admitted and treated for a hip fracture at our two study hospitals. The median age was 82 years (interquartile range, 75–87 years), 58% were ≥80 years of age, and 71% were female. Although we could not directly collect race/ethnicity data from our hospital records, previous studies of osteoporosis-related fractures at these two hospitals have documented that about 80% of this population is white.(20,27) These elderly patients had a great deal of pre-existing baseline co-morbidity, with 26% having four or more of the co-morbidities listed in Tables 1 and 2. Fewer than 0.5% of patients had osteoporosis documented as a co-morbid diagnosis. Except for hypertension, urinary tract infection, and hypothyroidism, men had a greater prevalence of every single co-morbidity at the time of hospital admission (Table 1). Of note, approximately one-third of patients were admitted to hospital from a long-term care facility. Among those who survived to discharge, the median length of stay in hospital was 8 days (interquartile range, 6–13 days). Table 2 presents patient-level characteristics stratified according to the site of treatment; in general, hip fracture patients treated at the RAH (training set) were comparable with patients treated at the UAH (validation set).

Table Table 1. Prefracture Characteristics of 3981 Elderly Patients With Hip Fracture, Stratified by Sex
original image
Table Table 2. Prefracture Characteristics of 3981 Elderly Patients With Hip Fracture, Stratified by Siteof Treatment
original image

In-hospital mortality

Overall in-hospital mortality was 6.3%; it was 4.7% for women and 10.2% for men (p < 0.001). There were no between-site differences for in-hospital mortality (p > 0.5), and there were no temporal changes in mortality rates over the 6 years of the study (p for trend = 0.44). Figure 1 presents in-hospital mortality stratified according to age in decades and sex. It can be seen that, for our youngest patients, 60–69 years of age, mortality was similar among men and women, but as age increased, a gender disparity became evident, with men having higher rates of mortality than women in each successive decade. Among patients ≥90 years of age, we found that men had fully twice the in-hospital mortality rate of women (17.5% versus 8.7%, p = 0.01). Table 3 presents the unadjusted and adjusted odds ratios for the associations between age (in decades), sex, and baseline co-morbidities with in-hospital mortality. Whereas there were many potential correlates of in-hospital mortality, other than older age, male sex, and long-term care resident at the time of fracture, only 10 variables remained in our final adjusted multivariable model. The five baseline co-morbidities associated with the greatest independent risk of mortality (in order of decreasing risk) included malnutrition, renal failure, pneumonia, pre-existing malignancy, and previous myocardial infarction (Table 3). In the training set (RAH patients, n = 2187), the Hosmer-Lemeshow goodness of fit test statistic had a p > 0.50 (indicating good fit), and the c-statistic was 0.83 (indicating excellent predictive ability). In the validation set (UAH patients, n = 1794), the c-statistic was 0.82, indicating that the model performed just as well and just as accurately in an independent population.

Table Table 3. Unadjusted and Adjusted Odd Ratios and Risk Scores for Variables Associated With In-Hospital Mortality After a Hip Fracture
original image
Figure Fig. 1..

In-hospital mortality among a cohort of 3981 hip fracture patients according to their sex and age (combined training and validation sites).

Table 3 also presents a “weighted” point score for each variable included in the final multivariable model. We calculated a risk score for each patient by summing his or her individual scores according to baseline prefracture characteristics. Patients' scores could vary from <5 (e.g., a woman [0], 60–69 years of age [0], with chronic obstructive pulmonary disease [COPD] [4]) to ≥50 (e.g., a man [6], ≥90 years of age [13], with renal failure [19], and a previous myocardial infarction [13]). Figure 2 presents in-hospital mortality according to quartiles of calculated risk score in the validation set (UAH), and indeed, shows that a higher score predicts a greater risk of mortality. The risk of in-hospital mortality varied from <1% for patients in the lowest quartile of risk to >15% for those in the highest quartile (Fig. 2). In essence, Fig. 2 allows one to convert a simple calculated score derived from age, sex, and baseline co-morbidity data into a predicted probability of in-hospital mortality.

Figure Fig. 2..

In-hospital mortality among 1794 hip fracture patients according to quartiles of their calculated risk score (validation site).

One-year mortality

Mortality at 1 year was also substantial for the 3730 patients who survived hospitalization and were discharged. Overall, it was 30.8% at 1 year, with a mortality rate of 37.5% for men and 28.2% for women (p < 0.001). One-year mortality did not differ according to site of treatment (31.7% in the RAH training set versus 29.6% in the UAH validation set, p = 0.18). Figure 3 presents 1-year mortality according to quartiles of the calculated risk score in the UAH validation set and again shows that a higher score predicted a greater risk of mortality. The c-statistic for the final model was 0.75 in the training set and 0.74 in the validation set, indicating acceptable discrimination and predictive accuracy. The variables and their adjusted odds ratios for predicting 1-year mortality were virtually identical to those predicting in-hospital mortality (data not shown), suggesting that essentially the same factors (age, sex, long-term care residence, and prefracture co-morbidities) are associated with both short-term and long-term mortality.

Figure Fig. 3..

One-year mortality among 1687 hip fracture patients that were discharged from hospital according to quartiles of their calculated risk score (validation site).

DISCUSSION

We found that mortality after a hip fracture in elderly patients is common. The rate of in-hospital mortality was 6.3%, and among those who survived to discharge, the 1-year mortality after hip fracture was 30.8%. Older age and male sex were independently associated with a greater risk of mortality; otherwise, it seemed that the primary determinants of mortality risk were related to the burden and mix of co-morbidities present at the time of hip fracture. Indeed, risk-adjustment models that we developed and validated that were based on age, sex, long-term care residence, and co-morbidities were robust predictors of mortality during hospitalization as well as at 1 year. Risk scores based on these parameters were easy to calculate and would be suitable for cross-hospital comparisons of risk-adjusted mortality outcomes for elderly hip fracture patients.

Our population-based estimates of in-hospital and 1-year mortality are very consistent with estimates from studies in different populations and different settings conducted with different methods that have been published over the last 10–20 years.(1–17) For in-hospital mortality, our estimate of 6.3% is similar in magnitude to estimates that have ranged from 4% to 12%(5–11); for 1-year mortality, our estimate of 30.8% is also consistent with previous estimates that have ranged from 12% to 37%.(5,7–17) Many previous studies have documented the independent association of older age with increased mortality(1–3,5,7,8,11–16); conversely previous data regarding the independent association of male sex and increased mortality has been somewhat conflicting.(2,5–9,12,16,17) In our fully adjusted multivariable models, male sex was associated with an almost 2-fold increased risk of in-hospital mortality (adjusted odds ratio, 1.8; 95% CI, 1.3-2.4). Given the size and representativeness of our population, we believe our data provide some of the strongest evidence to date that male sex is an important and independent risk factor for mortality in the setting of hip fracture. Whereas our study cannot answer the question why males might be at higher risk for mortality after hip fracture, we can reasonably assert that it is not because of their age or burden of co-morbidity at the time of fracture.

Perhaps the most noteworthy finding of this study is just how important the type, mix, and number of baseline co-morbidities are as determinants of mortality in the hip fracture population, both in-hospital and at 1 year. Whereas previous studies have also documented that co-morbidities(1–7,14,16,17) (such as malnutrition, renal failure, pneumonia, pre-existing malignancy, and previous myocardial infarction), or simply the number of co-morbidities,(5,7) are independently associated with mortality, there have been few attempts to develop robust and easy to use predictive models that used multivariate logistic regression methods as we did. Furthermore, to our knowledge, there has been only one previously published hip fracture-specific mortality prediction model that attempted independent validation with a “split-sample” technique(15); while a rigorous study, those investigators also examined the effect of perioperative (e.g., timing of surgery) and intraoperative (e.g. American Society of Anesthesiology class at the time of surgery) factors on 1-year mortality. The robustness of our approach is attested to by the fact that the c-statistic in the derivation set (0.83) and the validation set (0.82) was virtually identical for models of in-hospital mortality and by the fact that the same risk score accurately predicted both in-hospital and 1-year mortality outcomes. Thus, we believe the operating characteristics of our hip fracture-specific risk score are such that it would be useful for comparing appropriately risk-adjusted outcomes across different hospitals or health care settings, for the purposes of quality improvement, performance measurement, or even the production of hospital profiles or report cards.(18,19) Our results also suggest that any attempts to undertake the development of a population-based hip fracture “registry” at the least should consider including the variables that we have described herein.

This large population-based cohort study has several limitations that need to be considered. First, we did not have access to certain information that is not routinely documented in the discharge record and that may be associated with mortality, such as prefracture functional status, cognitive status, or the severity of the hip fracture itself. Second, co-morbidities were based on the clinical diagnoses of the treating physicians (orthopaedic surgeons and internists who provided routine preoperative assessments on all patients) as documented in the electronic administrative databases, rather than strictly predefined by our study. Nevertheless, this would mean that we most likely underascertained the presence or absence of co-morbidities while routinely capturing the more severe end of the disease spectrum. Third, we did not consider timing of surgery, intraoperative factors such as type of anesthesia or need for transfusions, or postoperative complications. However, these factors are related to practice style and processes of care and would be expected to (perhaps) lead to variations in outcomes.(3) Our stated purpose was to develop a risk adjustment model to control the case-mix of patients that arrived at the hospital with a hip fracture and “level” the playing field to allow for valid cross-hospital or cross-setting comparisons of achieved outcomes. Fourth, we did not examine important nonmortality outcomes such as adequate ambulation, return to independent function, health-related quality of life, satisfaction with care, caregiver burden, or costs.

Last, our most important limitation is likely the extent to which we have validated our models.(28) Whereas we derived our models in one hospital's population and validated it another hospital's population, we drew our sample from only one large health region in Canada, and some may be concerned that our results are not necessarily generalizable. We can allay these concerns to some degree, given how large our population base was (1.6 million people) and how broadly consistent our findings are with previous literature from elsewhere in Canada, the United States, Europe, Australia, and New Zealand.(4–17) Nonetheless, even though we have validated our results to a much greater degree than most previous studies,(5,6,7,10,11,13,16) it could be argued that our risk models are not yet ready for widespread adoption. Justice et al.(28) have described a hierarchy of validation for assessing the generalizability of prognostic models. According to their framework, we have adequately shown internal validity, reproducibility, and transportability, but only begun the process of external validation; these authors would suggest that our models be tested on data from different time periods, in patients at lesser risk of short-term and long-term mortality, with data from different geographical locations, and from multiple sites by investigators independent of our group.(28)

With these limitations in mind, we conclude that 1 in 15 elderly people with hip fracture will die during hospitalization, and almost one-third of those who survive to discharge will die within the year. The primary determinants of mortality in this population seem to be older age, male sex, and the number, type, and mix of pre-existing baseline co-morbidities. Given that in-hospital mortality rates could vary from <1% to >15% according to prefracture characteristics and co-morbidities, any comparison of outcomes across different hospitals demands adequate, robust, and validated risk-adjustment tools. Our risk scoring method (although requiring some further validation) may be sufficient to meet these demands, and we would encourage its further study by those who might embark on the task of comparing hospital-specific outcomes of hip fracture or who might be considering the development of population-based registries for this condition.

Acknowledgements

We thank Dr Wei-Ching Chang (Biostatistician, Department of Medicine, University of Alberta) for statistical assistance. This study was supported by grants from the Alberta Heritage Foundation for Medical Research (AHFMR). SRM is a Population Health Investigator of AHFMR and a New Investigator of the Canadian Institutes of Health Research.

Appendix

Classification of CO-Morbidities According to Their ICD-9 Codes

 

Table  .  
original image

Ancillary