The views expressed in this manuscript are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
Comparison of three comorbidity measures for predicting health service use in patients with osteoarthritis†
Article first published online: 5 OCT 2005
Copyright © 2005 by the American College of Rheumatology
Arthritis Care & Research
Volume 53, Issue 5, pages 666–672, 15 October 2005
How to Cite
Dominick, K. L., Dudley, T. K., Coffman, C. J. and Bosworth, H. B. (2005), Comparison of three comorbidity measures for predicting health service use in patients with osteoarthritis. Arthritis & Rheumatism, 53: 666–672. doi: 10.1002/art.21440
- Issue published online: 5 OCT 2005
- Article first published online: 5 OCT 2005
- Manuscript Accepted: 30 APR 2005
- Manuscript Received: 14 JAN 2005
- Department of Veterans Affairs
- Veterans Health Administration
- HSR&D Service: Program 824 Funds
- HSR&D Service Merit Review Entry Program award
- HSR&D investigator initiated award (02–034)
- Risk adjustment;
- Computerized medical record systems
To compare the ability of 3 database-derived comorbidity scores, the Charlson Score, Elixhauser method, and RxRisk-V, in predicting health service use among individuals with osteoarthritis (OA).
The study population comprised 306 patients who were under care for OA in the Veterans Affairs (VA) health care system. Comorbidity scores were calculated using 1 year of data from VA inpatient and outpatient databases (Charlson Score, Elixhauser method), as well as pharmacy data (RxRisk-V). Model selection was used to identify the best comorbidity index for predicting 3 health service use variables: number of physician visits, number of prescriptions used, and hospitalization probability. Specifically, Akaike's Information Criterion (AIC) was used to determine the best model for each health service outcome variable. Model fit was also evaluated.
All 3 comorbidity indices were significant predictors of each health service outcome (P < 0.01). However, based on AIC values, models using the RxRisk-V and Elixhauser indices as predictor variables were better than models using the Charlson Score. The model using the RxRisk-V index as a predictor was the best for the outcome of prescription medication use, and the model with the Elixhauser index was the best for the outcome of physician visits.
The Rx-Risk-V and Elixhauser are suitable comorbidity measures for examining health services use among patients with OA. Both indices are derived from administrative databases and can efficiently capture comorbidity among large patient populations.
Measurement of comorbidity is important in the context of health services research, epidemiologic studies, and clinical trials. Comorbidity assessment can also play an important role in health care planning and policy making. Because osteoarthritis (OA) and other rheumatic diseases are more prevalent among older adults (1, 2), additional chronic illnesses (i.e., comorbidities) are common in these patient groups. Adequate assessment of comorbidities is therefore essential for characterizing the overall health status in this patient population.
There are many methods and scales available for assessing comorbidity, including patients' self reports, medical record abstraction, and administrative databases. There has been growing interest in the use of administrative databases for comorbidity assessment, since large databases are increasingly available in many health care systems. The main advantages to this approach are ease of data acquisition, cost and time efficiency, and lack of reliance on accurate reporting by patients. There are also limitations, such as diagnosis coding errors and omissions (3, 4). However, administrative databases are an important source of data that can efficiently capture comorbidity among large patient populations.
Three validated, database-derived comorbidity indices are the Charlson Score, Elixhauser method, and RxRisk-V. The overall goal of these measures is to identify patients' significant chronic illnesses as comprehensively as possible, through the use of electronic medical records, and to create a valid method of coding and scoring that captures the burden of illness. However, there are some significant differences in these measures, including the method of development, type of data used, and the number and type of diseases included. The Charlson Score has been the most widely used comorbidity index. Originally developed for use with medical record abstraction (5), this score has also been adapted for administrative databases that use the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) (6). The Charlson Score has been shown to predict mortality and hospitalization outcomes in a variety of patient populations (6–8). The Elixhauser method also uses ICD-9-CM codes and has been shown to predict mortality and hospitalization outcomes (9, 10). In contrast to the Charlson Score, the original Elixhauser method involves retaining individual binary indicators for each disease category (rather than creating a summary score by adding indicators for all diseases). Two recent studies suggest that the Elixhauser method is a superior predictor of mortality compared with the Charlson Score (10, 11). The RxRisk-V is a pharmacy-based comorbidity index that is a revision and expansion of the Chronic Disease Score (12). Studies have shown that the RxRisk-V is a significant predictor of total health care costs (13, 14).
To date, these comorbidity measures have not been validated specifically among groups of patients with arthritis or other rheumatic conditions, and studies have not compared these 3 measures in the same sample. Two of these indices (the Charlson Score and the Elixhauser method) were developed on the basis of predicting mortality and/or hospitalization-related outcomes (5, 6, 9, 15), but their ability to predict other health service outcomes has not been well documented. This is a limitation for research in the rheumatic diseases, where health service outcomes such as physician visits and medication use may be of greater relevance than mortality. The purpose of this study was to compare the ability of the Charlson Score, Elixhauser method, and RxRisk-V to predict overall health service use among individuals with OA. Specifically, we examined the ability of these comorbidity indices to predict physician visits, hospitalization, and prescription drug use.
PATIENTS AND METHODS
Participants were veterans who were under care for OA at the Veterans Affairs (VA) Medical Center in Durham, NC. We selected a random 10% sample of patients (n = 306) who were identified by an ICD-9-CM code for OA (715) between October 1998 and September 1999, using national VA inpatient and outpatient databases.
All comorbidity scores were calculated using 1 fiscal year of data (October 1998–September 1999). National VA inpatient and outpatient clinic files were used to create the 2 ICD-9-CM–based comorbidity indices (Charlson and Elixhauser). The Charlson Score includes 19 conditions. Each condition is assigned a weight from 1 to 6, which was originally derived from relative risk estimates of a proportional hazards regression model (5). The Charlson Score is the sum of the weights for all conditions. The Elixhauser method involves creating separate indicator variables for 30 different diagnoses (9). As an alternative to the original Elixhauser method, we also summed all of the indicators to create a total Elixhauser score. The RxRisk-V was calculated using pharmacy data obtained from local VA electronic medical records. This score involves the creation of 45 indicators for general drug categories (i.e., human immunodeficiency virus drugs, diabetic drugs). These indicator variables are summed to create the total RxRisk-V score. Within the VA health care system, medication copayments are low ($3 at the time of this study) and equal for all drugs. In addition, the VA covers the costs of some over-the-counter medications such as ibuprofen, acetaminophen, and aspirin. Therefore, VA patients have a high incentive to obtain their medications within the VA health care system, and the VA pharmacy database provides an excellent reflection of patients' medication use.
Health service use outcomes.
We examined 3 health service use variables: number of physician visits, number of prescription drugs, and whether patients were hospitalized. We included all prescription drugs and visits to any type of physician, because we were interested in examining the ability of the comorbidity scores to predict overall health care use. All outcome variables were calculated using data from October 1999 to September 2000 (the fiscal year following creation of the comorbidity indices). Using the national VA outpatient clinic file, we calculated the total number of physician visits during the year. Local (Durham VA Medical Center) electronic pharmacy records were used to calculate the total number of prescriptions filled during the year. The national VA inpatient treatment file was used to examine hospitalizations. Because very few participants had more than 1 hospitalization during the year, we simply examined whether patients were hospitalized (rather than the number of hospitalizations).
In the first step of the analysis, the proportion of participants with each drug class/disease indicator was examined for each of the comorbidity indices. Correlations between comorbidity indices were also calculated. In the next step, 4 separate models, each using a different comorbidity index (Charlson, Elixhauser total summed score, Elixhauser with individual disease indicators, and RxRisk-V) as a predictor variable, were fit to each of the 3 health service outcome variables. Negative binomial regression models were used for models of the number of physician visits and number of prescriptions. Initially, we applied Poisson regression models, but these models did not fit the data well. For the binary hospitalization outcome, logistic regression models were used. Model selection was used to identify the best comorbidity index to use for predicting each of the health service outcomes. A model selection criterion, the Akaike Information Criterion (AIC) (16), was applied to the 4 models for each outcome, and the model with the lowest AIC was selected as the best model. AIC is computed from the log likelihood of a model and the number of parameters in a model. In general, the rule of thumb for identifying a model or set of models as “better” is a difference in AIC values >4 between models (17). We also examined the fit of our models using deviance values for negative binomial regression models and c statistics for logistic regression models. All analyses were conducted using SAS for microcomputers (SAS Institute, Cary, NC).
Among the 306 participants in this study, 96% were men, the mean ± SD age was 59.8 ± 14.2 years, 49% were white, and 32% were African American (with the remainder of participants having no race indicator in the electronic medical record). The RxRisk-V yielded the highest mean number of drug/disease indicator categories (4.3), followed by the Elixhauser method (2.2) and the Charlson Score (1.0). (Note that the RxRisk-V also involved the greatest possible number of indicators , followed by Elixhauser  and Charlson .) The proportions of participants with each drug or disease indicator for all comorbidity measures are presented in Table 1. These results show that the RxRisk-V captures information on a larger number of disease categories than the other 2 measures; however, 15 of these disease indicators were present for ≤1% of the sample. All comorbidity scores were significantly correlated (Pearson's r = 0.42–0.60, P < 0.001).
|Solid tumor without metastasis||—||9.5||0.0|
|Peripheral vascular disease||—||5.6||4.3|
|Pulmonary circulation disorders||—||1.0||—|
|Gastric acid disorder||35.6||—||—|
|Inflammatory bowel syndrome||0.3||—||—|
|Liver disease (mild)||—||1.3||1.0|
|Liver disease (severe) or failure||1.3||—||0.0|
|Peptic ulcer disease||—||3.3||3.6|
|Rheumatoid arthritis/collagen vascular diseases||—||18.4||5.6|
|Paraplegia or hemiplegia||—||—||0.0|
|Other neurologic disorders||—||3.0||—|
|Blood loss anemia||—||0.3||—|
|Fluid and electrolyte disorders||—||2.3||—|
|Anxiety and tension||13.4||—||—|
|Benign prostatic hypertrophy||17.3||—||—|
|Chronic pulmonary disease||—||12.8||12.8|
|Reactive airway disease||13.4||—||—|
|Mean ± SD score (range)||4.3 ± 2.7 (0–16)||2.2 ± 1.8 (0–10)||1.0 ± 1.3 (0–6)|
There were notable differences among the comorbidity indices in the measurement of musculoskeletal and pain-related conditions (Table 1). The Elixhauser method and the Charlson Score include broad indicators for rheumatic diseases, but not for OA or other pain-related conditions specifically. The Elixhauser method identified a larger number of participants as having a rheumatic disease than the Charlson Score (18.4% compared with 5.6%) because the Elixhauser method includes a broader spectrum of diseases. However, neither score includes an ICD-9-CM code for OA in the rheumatic disease category. The RxRisk-V does not contain specific indicators for any rheumatic conditions except gout. However, the RxRisk-V includes indicators for pain medications (opioid and nonopioid analgesics) and pain/inflammation medications (nonsteroidal antiinflammatory drugs [NSAIDs]), which were present for a large proportion of this sample (38.9% and 63.4%, respectively). Although the RxRisk-V comprehensively assesses analgesics and NSAIDs, it does not include indicators for some rheumatoid arthritis (RA) medications, including disease-modifying antirheumatic drugs and biologic response modifiers. Furthermore, methotrexate and cyclosporine (both disease-modifying antirheumatic drugs) are included as indicators of other disease categories (malignancies and transplants, respectively). The RxRisk-V does include an indicator for steroids (i.e., steroid responsive conditions category; 8% of this sample), which are used to treat RA and a variety of other conditions.
There are also notable differences among the indices in the measurement of some other conditions of interest among individuals with arthritis. Both the RxRisk-V and the Elixhauser method include indicators for depression, which is a common comorbidity among individuals with arthritis (18). However, the proportion of participants identified as having this condition was higher using the RxRisk-V (24.5% compared with 13.8%). Only the Elixhauser method includes an indicator for obesity (a risk factor for OA ), which was present in 16.1% of this sample. The Elixhauser method and Charlson Score include indicators for peptic ulcer disease, which can be a side effect of NSAID use (20). Peptic ulcer disease was present among ∼3% of this sample, according to both the Elixhauser and Charlson measures. The RxRisk-V contains a broader, more general indicator for gastric acid disorders, which was present in a much larger proportion of the sample (35.6%).
The mean ± SD number of physician visits during the followup year was 26.4 ± 28.3 and the mean number of prescription drugs was 10.4 ± 10.2; 14.8% of the sample were hospitalized. Relationships of the comorbidity indices to health service use variables are presented in Table 2. In the negative binomial regression models, all summary comorbidity indices were significantly associated with the number of physician visits (P < 0.001) and the number of prescription drugs (P < 0.001). Individual indicators for all disease categories in the original Elixhauser method are not presented, but 8 of the 30 diseases were significantly associated with the number of physician visits (P < 0.05) and 7 of the 30 diseases were significantly associated with the number of prescription drugs (P < 0.05). In the logistic regression model, all summary comorbidity indices were significantly associated with probability of hospitalization (P < 0.01). However, the logistic regression model for hospitalization using individual disease indicators defined by the Elixhauser method did not converge, due to small sample sizes in some of the categories.
|Physician visits||Prescription drugs||Hospitalization|
|RxRisk-V||0.08||< 0.001||−10105||0.29||< 0.001||−43386||1.21||< 0.001||247.6|
|Charlson||0.15||< 0.001||−10101||0.25||< 0.001||−43278||1.41||0.002||250.0|
|Elixhauser—total score||0.17||< 0.001||−10121||0.22||< 0.001||−43287||1.31||0.001||248.7|
For the number of physician visits outcome, the models with the Elixhauser total score and individual indicators were the best models based on AIC values and were clearly better than models with either the RxRisk-V or Charlson Score as a predictor variable. Both of these models also had acceptable model fit according to deviance values (deviance/degrees of freedom [df] = 1.14–1.26) (21). For the number of prescription drugs outcome, the model with the RxRisk-V score was clearly the best model based on AIC values. This model also had acceptable model fit according to the deviance value (deviance/df = 1.21). For the dichotomous hospitalization outcome, no best model was identified based on AIC values. The c statistic for these logistic regression models ranged from 0.65 to 0.67.
This study compared 3 comorbidity measures, the Charlson Score, RxRisk-V, and Elixhauser method, among a group of patients with ICD-9-CM codes for OA. Although these measures have been validated previously, it is important to examine the performance of comorbidity measures in the context of specific settings and health outcomes. In this case, we were interested in comparing these measures among patients with OA and examining their ability to predict health service use variables of particular relevance to this patient group (physician visits, prescription drug use, and hospitalization).
Results of this study showed that all 3 comorbidity measures were significant predictors of health service use among individuals with OA. However, there were some differences in their ability to predict specific health service variables. Among the 3 measures, the Elixhauser method (both the total score and the individual indicators) provided the best fit for the model predicting physician visits. The RxRisk-V was associated with the next best fit, followed by the Charlson Score. The RxRisk-V provided a substantially better fit than the other measures among the models predicting prescription medications. The Elixhauser total score provided the next best fit, followed by the Charlson Score. The Elixhauser total score was the strongest predictor of hospitalization, but the differences in predictive ability among the models were small, indicating no clear advantage for any of the indices.
In general, these results suggest that the RxRisk-V and Elixhauser method may be preferable to the Charlson Score when predicting overall health service use (especially physician visits and prescription medication use). Although the Charlson Score has been widely validated and commonly used, it was originally developed to predict mortality. In this study, the Charlson Score was a significant predictor of all health service use outcomes. However, it was not as strong a predictor of overall physician visits and prescription medication use, which are often of interest in the context of chronic, long-term diseases such as arthritis. This finding is important because it indicates that the Charlson Score may not provide as comprehensive of a comorbidity adjustment in models predicting these 2 health service use outcomes. Results also showed that the RxRisk-V was a somewhat stronger predictor of prescription medication use than the Elixhauser method, and the converse was true in models predicting physician visits. This is not surprising because the RxRisk-V is a pharmacy-based measure, and the Elixhauser measure is based on diagnosis codes from physician visits. However, results indicate that the RxRisk-V and Elixhauser method are each suitable for examining both physician visits and prescription medications.
There are strengths and limitations to both pharmacy-based and diagnosis-based comorbidity indices that should be considered when selecting and using a measure. One strength of diagnosis-based measures is that in some cases they can identify specific diseases more accurately than pharmacy-based measures. For example, some medications are used to treat both hypertension and congestive heart failure, 2 conditions with very different levels of severity. In addition, physicians may prescribe medications for “off label” uses, leading to misclassification of a disease. Whereas pharmacy-based measures do not detect the indication for the prescription, diagnosis-based measures identify the specific disease being treated. A diagnosis-based measure could also identify comorbid rheumatic diseases (i.e., OA and RA). However, diagnosis-based measures (including the Elixhauser method and Charlson Score) typically treat rheumatic diseases broadly and do not identify separate conditions. Therefore the overall burden of rheumatic conditions is typically not reflected in diagnosis-based measures.
The main limitations of diagnosis code-based indices are coding errors, omissions, and biases (4). Coding errors can be made by clinicians or at the point of data entry. Coding biases occur when some ICD-9-CM codes are used preferentially because of insurance reimbursement rules. Pharmacy-based comorbidity measures are not as subject to coding errors as are diagnosis-based measures. However, pharmacy-based comorbidity measures may fail to comprehensively assess diseases if patients choose not to fill prescriptions, if prescriptions are obtained from multiple health care sources, or if medications are available without a prescription. We restricted our study to a 10% random sample because of the time burden of extracting pharmacy data from local electronic medical records at the time of this study. However, large, automated national pharmacy databases are increasingly available and accessible, which improves the feasibility and efficiency of pharmacy-based comorbidity measures.
There are also several specific characteristics of the RxRisk-V and Elixhauser methods that should be considered, particularly when using these measures among patients with arthritis or other rheumatic diseases. The RxRisk-V includes indicators for analgesic and antiinflammatory medications. This may be an advantage, because medications are a significant component of arthritis-related health care costs (22). However, analgesic and antiinflammatory medications are used for other conditions, and the RxRisk-V does not include specific indicators for either OA or RA. The Elixhauser method includes a relatively comprehensive measure of rheumatic diseases. However, it does not include the diagnosis code for OA. This may be a particular disadvantage when examining outcomes that are significantly affected by OA, including prescription medication use, overall health care costs, and health-related quality of life (22, 23).
In addition to their differences in identifying arthritis and other rheumatic conditions, the RxRisk-V and Elixhauser method vary in their treatment of other conditions of importance among this patient population (i.e., depression, obesity, gastric acid disorders) (18–20, 24). The RxRisk-V includes a more comprehensive assessment of gastric acid disorders than the Elixhauser method (which identifies only peptic ulcers and not milder conditions or symptoms). The RxRisk-V also identified a larger number of individuals with depression than the Elixhauser method. However, only the Elixhauser method includes an indicator for obesity.
There are some limitations to this study. The data were collected from one VA medical center, and the sample was predominantly male. Additional studies should compare and validate these measures for health service use outcomes among other patient populations. The size of our sample did not allow us to estimate the predictive ability of the original Elixhauser method (with individual disease indicators) for our binary hospitalization outcome. However, this is likely to be a general limitation of the original Elixhauser method. Because of the large number of binary disease indicators in the Elixhauser method, sample sizes must be very large to avoid overfitting models. Studies should continue to examine the validity and utility of a summed Elixhauser score, as was tested in this study.
In summary, 3 comorbidity measures, the Charlson Score, RxRisk-V, and Elixhauser method, were found to be significant predictors of health service use among a group of patients with OA. However, the RxRisk-V and Elixhauser measures were somewhat better predictors than the Charlson Score. The RxRisk-V is based on prescription medication use and the Elixhauser method is based on ICD-9-CM codes. Therefore these measures provide different options that may be more feasible for different health care institutions, depending on the type of electronic medical record data available. The RxRisk-V and Elixhauser measures each have strengths, limitations, and other features (i.e., inclusion/omission of some diseases or disease categories) that should be carefully considered in the context of specific studies. However, these results support the validity of both the RxRisk-V and Elixhauser scores for predicting overall health service use among patients with chronic disease. These measures can be valuable tools for comorbidity measurement in the context of research, health care planning, or policy development.
- 2Centers for Disease Control and Prevention. Public health and aging: projected prevalence of self-reported arthritis or chronic joint symptoms among persons aged >65 years: United States, 2005–2030. MMWR Morb Mortal Wkly Rep 2003; 52: 489–91.
- 16Information theory and an extension of the maximum likelihood principle. In: PetrovBN, CaskiF, editors. Proceedings of the second international symposium on information theory. Budapest: Akademiai Kiado; 1973. p. 267–81..
- 17Model selection and multimodel inference: a practical information-theoretic approach. New York: Springer-Verlag; 2002., .
- 21Regression analysis of count data. Cambridge: University Press; 1998., .