Validation of an International Classification of Disease, 10th revision coding adaptation for the Charlson Comorbidity Index in United States healthcare claims data

Abstract Purpose An International Classification of Disease (ICD‐10) Charlson Comorbidity Index (CCI) adaptation had not been previously developed and validated for United States (US) healthcare claims data. Many researchers use the Canadian adaption by Quan et al (2005), not validated in US data. We sought to evaluate the predictive validity of a US ICD‐10 CCI adaptation in US claims and compare it with the Canadian standard. Methods Diverse patient cohorts (rheumatoid arthritis, hip/knee replacement, lumbar spine surgery, acute myocardial infarction [AMI], stroke, pneumonia) in the IBM® MarketScan® Research Databases were linked with the IBM MarketScan Mortality file. Predictive performance was measured using c‐statistics for binary outcomes (1‐year and postoperative mortality, in‐hospital complications) and root mean square prediction error (RMSE) for continuous outcomes (1‐year all‐cause medical costs, index hospitalization costs, length of stay [LOS]), after adjusting for age and sex. C‐statistics were compared by the method of DeLong and colleagues (1988); RMSEs, by resampling. Results C‐statistics were generally high (≥ ~ 0.8) for mortality but lower for in‐hospital complications (~0.6–0.7). RMSEs for costs and hospitalization LOS were relatively large and comparable to standard deviations. Results were similar overall between the US and Canadian adaptations, with relative differences typically <1%. Conclusions This US‐based coding adaptation and a previously published Canadian adaptation resulted in similar predictive ability for all outcomes evaluated but may have different construct validity (not evaluated in our study). We recommend using adaptations specific to the country of data origin based on good research practice.


| INTRODUCTION
Comorbid illnesses are important indicators of health, and their presence increases the risk of mortality and other health outcomes. The Charlson Comorbidity Index (CCI) was developed to measure the burden of disease from comorbidities and predict 1-year mortality risk. 1 The CCI is a commonly used algorithm to measure comorbidities and adjust for risk in health outcomes and similar studies; a Google Scholar search for the published CCI validation 1 resulted in more than 30 000 citations. The CCI score is determined by 19 comorbid conditions, and a weight is assigned to each based on its severity. The weights for an individual are summed to derive his or her CCI score. With the widespread use of administrative databases for clinical outcomes research, adaptations of the CCI to the International Classification of Disease, 9th Revision, Clinical Modification (CM) were created and validated. [2][3][4][5][6][7][8][9][10] In 1992, the tenth revision to the International Classification of Disease (ICD-10) was published by the World Health Organization, 11 and efforts were made outside the United States (US) to adapt the CCI to ICD-10. [12][13][14] Sundararajan et al (2007) 15  can influence the performance of algorithms. 16 We developed an ICD-10 coding adaptation for the US 17 ; the full current code set is available in machine-readable files at https://doi.org/10.5281/zenodo.3604394.
The primary objective of this study was to validate our ICD-10 CCI adaptation in a US administrative claims database that included mortality data. We evaluated the predictive performance of our ICD-10 CCI coding adaptation and compared its performance with that of the ICD-10 CCI coding adaptation developed in Canada, 14 which is currently being used by many for US healthcare research since there is presently no published validated algorithm for US healthcare claims. [18][19][20] Rather than focusing on a single condition or outcome, we sought to validate the coding adaptation for diverse cohorts (i.e., rheumatoid arthritis, hip or knee replacement, lumbar spine surgery, acute myocardial infarction [AMI], stroke, and pneumonia) and outcomes (i.e., 1-year and postoperative mortality, in-hospital complications, 1-year all-cause medical costs, index hospitalization costs, and length of stay [LOS]).

| Coding adaptations
The same comorbidities and weights from the original study by Mary Charlson and colleagues (1987) 1 Table S1. Coding trends were evaluated in a US claims data source, the IBM ® MarketScan ® Research Databases, during the fourth quarters of 2014 (ICD-9) and 2015 (ICD-10). 17  • C-statistics for mortality outcomes using either the US or Canadian adaptation were close to or exceeded 0.8 for patient cohorts (range 0.670-0.875), indicating high predictive power for 1-year and postoperative mortality.

| Data sources
• Predictive power was equally poor for both CCI coding adaptations for in-hospital complications, 1-year medical costs, index hospitalization costs, and index hospitalization length of stay.
• Using the Canadian adaptation had minimal impact on predictive ability but could result in different construct validity (i.e., erroneous assignment of individual comorbidities) given different ICD-10 coding adaptations and real world reimbursement environments in Canada and the US; construct validity was not evaluated in our study.
We recommend using adaptations specific to the country of origin of the data, consistent with good research practice guidelines.

| Study populations
Adults (ages 18 years or older) with evidence of rheumatoid arthritis (inpatient or outpatient), hip or knee replacement (inpatient), lumbar spine surgery (inpatient), AMI (inpatient), stroke (inpatient), or pneumonia (inpatient or outpatient) between October 01, 2016 and April 01, 2017 were included in the study. The patient selection criteria used to identify these conditions or procedures relied on validated algorithms [21][22][23][24][25][26] and are shown in Appendix A. These six medical conditions were selected to understand the performance of the CCI in diverse populations. The index date was the date of the patient's first diagnosis (any) code for RA; first principal diagnosis code for stroke, AMI, and pneumonia; or first hospital admission date for lumbar spine surgery and hip/knee replacement during the index period. For all outcome analyses except for mortality outcomes, patients were required to have at least 12 months of continuous enrollment with medical and pharmacy benefits before the index date (i.e., pre-index period) and following the index date (i.e., post-index period). For mortality outcomes analyses, patients who died within 12 months after the index date were permitted and were followed until date of death. Patients with capitated claims were flagged and a gross pay amount was assigned to capitated services with a pay proxy, which used noncapitated claims and was specific to US region, year, and current procedural terminology (CPT) code (where applicable).

| Outcome measures
Predictive performance was measured for 1-year and postoperative mortality, in-hospital complication, 1-year all-cause medical costs, and index hospitalizations costs and LOS. The CCI score and outcomes were developed using all claims during a 6-month pre-index period.
One-year mortality (the proportion of patients who died within 1 year of index date) was measured using the IBM MarketScan Mortality File. All costs were assessed using fully adjudicated and paid  For all outcome analyses except for mortality outcomes, patients were required to have at least 12 months of continuous enrollment with medical and pharmacy benefits following the index date as well as prior to the index date (12-month pre-index period). For mortality outcomes analyses, patients who died within 12 months following the index date were permitted and were followed until date of death.

| Statistical analyses
The predictive abilities of the US and Canadian adaptations were measured using methods appropriate for each type of outcome. For outcomes of 1-year mortality, postoperative mortality, and in-hospital complications, the area under the receiver operating characteristic curve (c-statistic) was calculated. For outcomes of hospitalization LOS, hospitalization costs, and 1-year all-cause medical costs, the root mean square prediction error (RMSE) between actual and predicted values was calculated. The rationale for selecting these performance statistics was relevance and ease of interpretation for the target audience for this research (i.e., researchers using the CCI in US healthcare claims data).
Generalized linear regression models were fit to patient-level data to calculate predicted outcomes for comparison with actual outcomes. Each model included the numeric CCI score, age, and sex as explanatory items.
For binary outcomes, the model included a binomial error with logistic link; for LOS, the model included a negative binomial error with log link; and for cost outcomes, the model included a gamma error with log link.

| RESULTS
A total of 123 626 patients met the study inclusion criteria across the six disease cohorts ( Table 1). The characteristics of studied individuals T A B L E 2 (Continued)  C-statistics across cohorts for 1-year mortality models ranged from 0.67 to 0.87 (Figure 1(A)) and for postoperative mortality models ranged from 0.80 to 0.93 (Figure 1(B)). Mortality c-statistics were similar between the US and Canadian coding adaptations with relative percentage differences in the range of −0.17% to 0.79% (Table 4). Cstatistics for in-hospital complications models ranged from 0.57 to 0.69 (Figure 1(C)), with relative percentage differences ranging from −0.32% to 0.76% (Table 4).
For both coding adaptations, the RMSEs for 1-year medical costs, index hospitalization costs, and LOS were relatively large, similar to the mean and standard deviations (Figure 2(A-C)). The relative The 1-year mortality analyses include patients with evidence of death during the 12-month follow-up period (Social Security Administration death master file) or at least 12 months of follow-up time. b One-year mortality was defined as dying within 12-month follow-up. c In-hospital mortality was defined as dying during the index inpatient admission or within a 6-week period following hospital discharge. d Hospital complications analyses include patients with 12-month follow-up. Patients in pneumonia and rheumatoid arthritis cohorts without an inpatient admission during the 12-month follow-up period were excluded from this analysis. e Hospital complications were defined as having an ICD-10-CM diagnosis for infection, ICD-10 procedure for blood transfusion, or diagnosis-related group for complications of treatment during the index hospitalization (see Appendix B for full list of concepts and codes).  Table 4). The relative percentage differences for hospitalization costs and LOS for the index hospitalization ranged from −0.26% to 0.45% (Table 4). Statistically significant differences were observed between coding adaptations for the pneumonia cohort for 1-year mortality and in-hospital complications and for the lumbar spine surgery cohort for in-hospital complications.

F I G U R E 1 Performance (c-statistics) of models predicting 1-year mortality (A), in-hospital mortality (B), and in-hospital complications (C).
Patients included in the 1-year mortality model are those with evidence of death during the 12-month follow-up period (Social Security Administration death master file) or at least 12 months of follow-up time. One-year mortality was defined as death within the 12-month followup period. (A). Patients included in the in-hospital mortality model are those with evidence of death during the 12-month follow-up period (Social Security Administration death master file) or at least 12 months of follow-up time. Postoperative mortality was defined as death during the index inpatient admission or within a 6-week period following hospital discharge (B). In-hospital complication models include patients with 12 months of follow-up time. Patients in pneumonia and rheumatoid arthritis cohorts without an inpatient admission during the 12-month follow-up period were excluded from the in-hospital complication model. In-hospital complication was defined as having an ICD-10-CM diagnosis for infection, ICD-10 procedure for blood transfusion, or diagnosis-related group for complications of treatment (see Appendix B for full list of concepts and codes) (C). All models include as explanatory items the numeric CCI score, numeric age, and indicator of female sex. The CCI score was calculated based on both inpatient and outpatient claims during the 6-month period prior to Index.  Because of different codes or categorization of codes in some cases, our US CCI coding adaptation produced slightly higher index scores than the Canadian coding adaptation (Table 2). While there were only two codes in the publication by Quan et al 14 that did not appear in our coding adaptation, there were over 400 codes in our coding adaptation that either do not appear in the Quan et al publication or are included in a different category (e.g., differences in interpretation between "diabetes with chronic complications" and "diabetes without chronic complications"). Some differences in coding were due to different interpretations of the code range implied in Quan's publication, Table 1 (e.g., E10.6 is included but not E10.6*, which we interpreted to mean a single E10.6 code only and not including the full range of codes under E10.6). To promote clarity in the interpretation of code lists, we provide a machine-readable list of our individual ICD-10 codes along with the codes from Quan's publication used in this study (https://doi.org/10.5281/zenodo.3604394) as well as the full code list in Table S1. However, some differences in coding are due to different ICD-10 coding adaptations between Canada and the US. Some codes in the US ICD-10 system represent different concepts or do not exist in the Canadian ICD-10 system. For example, US billable ICD-10 codes that specify "with coma" or "with-    has been shown to appreciably decrease the observed rate of comorbidities in US healthcare claims. 17 Our US adaptation represents a point in time early in the adoption and evolution of ICD-10, but future changes in ICD will be important for researchers to review and consider when deciding how to use this coding adaptation.
Our intent was not to evaluate whether the original Charlson concepts or weights can be improved upon; others have considered these issues. [29][30][31][32][33] Our aim was to evaluate the predictive validity of an ICD-10 coding adaptation for Charlson comorbidities in US healthcare data and compare it with a standard that many researchers in the US were likely using in the absence of a previously validated US coding adaptation. In general, both the US and Canadian adaptations performed well and similarly in this study.

Rheumatoid arthritis (RA)
Patients will be identified as having RA based on an adaptation of published claims-based algorithms with high positive predictive values (PPVs). 23 Patients will be required to have an ICD-10-CM diagnosis code for RA, an encounter with a rheumatologist, and evidence of disease-modifying antirheumatic drug (DMARD) use during the study period in either an inpatient or outpatient setting. In the outpatient setting, to determine whether to use all outpatient claims or restrict to non-diagnostic claims on professional medical encounters, separate sample counts will be calculated and compared. Depending on sample size, patients may not be required to have evidence of a rheumatologist encounter to be included in the RA cohort.

Hip or knee replacement
Patients will be identified as having undergone hip or knee replacement based on a validated algorithm using administrative data (ICD-9-CM) with a PPV of 95% for knee replacements and 98% for hip replacements 22 and adapted to ICD-10-CM. Patients will be required to have an inpatient claim with an ICD-10-CM diagnosis or procedure (Current Procedural Terminology (CPT), ICD-10-PCS) code for knee or hip replacement during the study period.

Lumbar spine surgery
Patients will be identified as having lumbar spine surgery based on a validated claims-based algorithm (using ICD-9-CM) 25 and adapted to ICD-10-CM. Patients will be required to have an inpatient procedure (CPT, ICD-10-PCS) code indicating lumbar fusion, with or without decompression, for back pain, herniated disc, stenosis, spondylolisthesis or scoliosis during the study period. The specificity of this algorithm ranges from 85.6% to 97.3%, depending on the surgery indication. 25

Acute myocardial infarction
Patients will be identified as having AMI based on a validated algorithm using administrative data with a PPV of 82.2%. 24 Patients will be required to have an inpatient claim with an ICD-10-CM diagnosis for AMI (I21-I22) in the primary diagnosis position during the study period.

Stroke
Patients will be identified as having a stroke based on a systematic review analyzing the validity of stroke codes in administrative databases. 26 Patients will be required to have an inpatient claim with a diagnosis of acute stroke (ICD-10-CM I60, I61, I63) during the study period, as these codes are highly predictive of true cases of acute stroke. 26

Pneumonia
To the best of our knowledge, there is a lack of validated U.S. claimsbased algorithms to identify pneumonia with a high PPV. Aronsky