Assessing the utility and accuracy of ICD10‐CM non‐traumatic subarachnoid hemorrhage codes for intracranial aneurysm research

Abstract Background The 10th revision of International Classification of Disease, Clinical Modification (ICD10‐CM) increased the number of codes to identify non‐traumatic subarachnoid hemorrhage from 1 to 22. ICD10‐CM codes are able to specify the location of aneurysms causing subarachnoid hemorrhage (aSAH); however, it is not clear how frequently or accurately these codes are being used in practice. Objective To systematically evaluate the usage and accuracy of location‐specific ICD10‐CM codes for aSAH. Methods We extracted all uses of ICD10‐CM codes for non‐traumatic subarachnoid hemorrhage (I60.x) during the first 3 years following the implementation of ICD10‐CM from the billing module of the electronic health record (EHR) for UCHealth. For those codes that specified aSAH location (I60.0‐I60.6), EHR documentation was reviewed to determine whether there was an active aSAH, any patient history of aSAH, or unruptured intracranial aneurysm/s and the locations of those outcomes. Results Between 1 October 2015 and 30 September 2018, there were 3119 instances of non‐traumatic subarachnoid hemorrhage ICD10‐CM codes (I60.00‐I60.9), of which 297 (9.5%) code instances identified aSAH location (I60.0‐I60.6). The usage of location‐specific codes increased from 5.7% in 2015 to 11.2% in 2018. These codes accurately identified current aSAH (64%), any patient history of aSAH (84%), and any patient history of intracranial aneurysm (87%). The accuracy of identified outcome location was 53% in current aSAH, 72% for any history of aSAH, and 76% for any history of an intracranial aneurysm. Conclusions Researchers should use ICD10‐CM codes with caution when attempting to detect active aSAH and/or aneurysm location.


| INTRODUCTION
The lure of "big data" for research into neurosurgical diseases is strong. Utilization of claims-based data represents a leap forward in our ability to study less prevalent diseases like aneurysmal subarachnoid hemorrhage (aSAH). Although roughly 6.5 to 9.7 million people in the United States harbor an unruptured intracranial aneurysm, only about 30 000 of those aneurysms rupture in a given year. [1][2][3] The rupture of an intracranial aneurysm causes an aSAH which has a high (50%) mortality rate and often leaves survivors with permanent disability. 4 Conducting traditional randomized controlled trials (RCTs) is challenging for aSAH due to the low prevalence of the condition. For example, the International Subarachnoid Aneurysm Trial (ISAT), one of the largest RCTs in neurosurgery, required 7 years to recruit a cohort of 2143 patients across 43 centers in Europe and North America. 5 While RCTs are the gold standard for clinical evidence, the large sample size of administrative claims data is alluring for real-world evidence generation.
According to a systematic review of claims-based research in neurosurgery, 50 studies of aneurysms and aSAH had been published in the top three neurosurgery journals between 2000 and 2016. One of the major limitations identified across these studies was the lack of specific clinical data required for appropriate risk adjustment. 6 Claims databases often use International Classification of Disease, 9th Revision, Clinical Modification (ICD9-CM) codes to identify clinical diagnosis. Unfortunately, ICD9-CM only has a single code for non-traumatic subarachnoid hemorrhage (430), which includes both aSAH and spontaneous hemorrhages that do not harbor aneurysms. Even though aSAH represents the majority (85%) of non-traumatic subarachnoid hemorrhage, 7 aneurysm location (which is an important risk factor for aneurysm rupture and correlates with aSAH outcome [8][9][10][11] ), is not captured by this code. These data are not typically available in large claim databases, reducing the impact and value of these databases for aSAH research.
However, on 1 October 2015, the United States began using the 10th revision of ICD codes (ICD10-CM) which dramatically expanded the number of codes for non-traumatic subarachnoid hemorrhage from 1 to 22. These new codes give a specific location for common sites of ruptured intracranial aneurysms, which represents a significant improvement in the utility of administrative claims databases for aSAH research. However, the accuracy of these codes to identify aSAH or aSAH location has not yet been determined.
In this study, we sought to evaluate the overall usage and accuracy of location-specific ICD10-CM codes for aSAH. We examined all billing uses of ICD10-CM codes for non-traumatic hemorrhage in the first 3 years post-implementation at a large multi-hospital healthcare system located across the front range of Colorado.
Detailed chart review was performed to verify aSAH status and aneurysm rupture location. These data should provide insight into the utility of large ICD10-CM-based claims databases for aSAH research.

| Assess utility of codes for intracranial aneurysm research
We calculated the frequency of each ICD10-CM code used for billing (eg, every "code instance"), "unique visit" (code/s billed on the same day for the same patient), and "unique patient" (code/s ever billed for the same patient).

| Assess accuracy of codes for intracranial aneurysm research
To determine the accuracy of the code used, CR, SK, and/or MW reviewed progress notes, imaging, procedure reports and patient history from the EHR for each visit billed with an ICD10-CM code that specified an artery location (I60.0-I60.6). Reviewers determined whether there was active management of an aSAH at that code instance date. "Active aSAH" was defined as the hospitalization when an aneurysm rupture was diagnosed/treated, an immediate hospitalization or inpatient rehabilitation following the diagnosis/treatment (at the same or different facility), or acute follow-up visits within 6 weeks post-discharge (accounting for complications and wound care issues). Reviewers then also considered the patient's entire chart to determine whether they had ever experienced an aSAH ("History of aSAH") or were diagnosed with an aneurysm ("History of Aneurysm").
The location/s of all identified active aSAH and historical aSAH/aneurysms were also recorded. During this review, we considered radiology reports and images to be the gold standard definition of aneurysm presence and location, though we accepted provider documentation when imaging was unavailable. Conflicts during review were adjudicated by CR, a board-certified neurosurgeon trained in both open and endovascular treatment of aSAH.
Using the results from the chart review, we first calculated the percent accuracy of each code instance identifying an active aSAH, history of aSAH, or history of aneurysm outcome regardless of specified location. Any code instance that had no occurrence of the specified outcome was classified as a "false positive." For each outcome (aSAH, history of aSAH, and history of aneurysm), we then assessed the accuracy of aneurysm location identified by the ICD10-CM code instance. Code instances where an aneurysm location could not be identified from the medical record were removed from this analysis.
Code instances for I60.6-"Other Artery" were classified as incorrect (wrong artery) if there was an available ICD10-CM code for the actual artery location of the outcome (eg, carotid siphon and bifurcation, middle cerebral artery, anterior communicating artery, posterior communicating artery, basilar artery, or vertebral artery). For patients with multiple aneurysms, accuracy was assigned manually for each F I G U R E 1 Frequency of ICD10-CM codes that specify location of non-traumatic subarachnoid hemorrhage. The usage of each ICD10-CM code that specifies aSAH location (ie, I60.0-I60.6) is provided as a percent of the total code instances (light blue) or unique patients (dark blue)  outcome. Active aSAH and history of aSAH outcomes were only considered correct for the ruptured artery/bleeding source. For patients with multiple historical sites of aSAH and/or aneurysms, we assigned status conservatively giving the most credit possible (ie, accurate > incorrect-wrong side > incorrect-wrong artery).
We also analyzed whether ICD10-CM code occurrences could be used to identify patients with any history of multiple aneurysms (ruptured or unruptured). First, we tested whether patients who had two or more distinct ICD10-CM codes across their record had a history of multiple aneurysms. Then, we considered the more specific condition: whether patients who explicitly had two or more arteries identified (ie, two or more I60.0-I60.6 codes) had a history of multiple aneurysms.
Chi-square or Fisher's exact tests were performed to evaluate for group differences, using a significance level of P < .05. No adjustment was made for multiple testing, as outcomes were considered complementary. We also calculated the positive predictive value for both definitions.

| Characterize usage of codes over time
To understand whether ICD10-CM code usage changed over time, we compared the frequency of codes that specified an artery location (I60.0-I60.6) to those that did not (I60.7-I60.9) each month over the study period. Similarly, we compared the accuracy of codes that specified an artery location (I60.0-I60.6) monthly over the study period.
Study data were collected and managed using REDCap electronic data capture tools hosted at University of Colorado Anschutz Medical Campus. 12 All analyses were performed using R version 3.6.0 13 and a variety of packages for data processing, graphing, and reporting. [14][15][16][17][18] This study was reviewed and approved by the Colorado Multiple Institutional Review Board. instances did not specify laterality. Table 1 contains usage frequencies for all non-traumatic SAH codes over the study period.

| RESULTS
Chart review revealed that out of all location-specific code instances, no evidence of an aSAH (current or historical) could be found in 48 (16.3%), and 35 (11.9%) had no evidence of any intracranial aneurysm. In the cases where no aneurysm was documented, 18 had a current non-aneurysmal SAH (non-traumatic), and 17 had neither an SAH (of any origin) nor an aneurysm. In two code instances, a distant history of multiple aSAHs was documented, but location could not be determined; these instances were dropped from the remaining analyses. A summary of the accuracy rate for all three tested outcomes (current aSAH, history of aSAH, and history of aneurysm) is presented in Figure 2.

| Ability to detect current aneurysmal subarachnoid hemorrhage
Of the codes that specified artery location, 190 (64.4%) were entered during an "active/current" aSAH episode. Over half of the code instances identified the correct aSAH location with 154 (52.2%) with an exact match and 4 (1.4%) that indicated the correct artery location but did not specify laterality. Of the incorrect instances, 1 (0.3%) identified the correct artery but incorrect laterality, while 31 (10.5%) identified the wrong artery. Accuracy of individual ICD10-CM codes to detect active aSAH is available in Table 2.
F I G U R E 2 Accuracy of ICD10-CM that specify location of nontraumatic subarachnoid hemorrhage to identify current aneurysmal subarachnoid hemorrhage, any history of subarachnoid hemorrhage, or any history of intracranial aneurysm. The total number of code instances that correctly identified both the outcome (current aSAHleft, history of aSAH-middle, history of intracranial aneurysm-right) and artery is displayed in green. The total number of code instances that correctly identified the outcome but not the correct artery is displayed in orange. The total number of code instances that did not correctly identify the outcome is displayed in red 3.2 | Ability to detect any history of aneurysmal subarachnoid hemorrhage Of the codes that specified artery location, 247 (83.7%) were used for patients who had an aSAH at any point in their history.
The majority (n = 205, 69.5%) correctly identified the location of the current or previous aSAH with 7 (2.4%) indicating the correct artery location without specifying artery laterality. Of those that indicated the incorrect location, 1 (0.3%) identified the correct artery, but indicated the wrong side, and 34 (11.5%) had the wrong artery. In 13 code instances (4.4%), the patient had one or more aneurysms, but no history of an aSAH. Accuracy of individual ICD10-CM codes to detect any history of aSAH is available in Table 3.

| Ability to detect any history of intracranial aneurysm
The majority of location-specific aSAH code instances (n = 260, 88.1%) accurately identified patients who had any history of an intracranial aneurysm at the time of entry. The location accuracy rate was highest among the three outcomes with 216 (73.2%) having an exact location match and an additional nine (3%) that identified the correct artery but did not specify that aneurysm laterality. The majority of the incorrect locations indicated the wrong artery (n = 34, 11.5%), while one (0.3%) identified the correct artery, but indicated the wrong laterality. Accuracy of individual ICD10-CM codes to detect any history of one or more intracranial aneurysm is available in Table 4.

| Ability to detect multiple aneurysms
Of those individuals with a location specified aSAH code, 35 (18%) had a history of multiple aneurysms, representing 76 total code instances. The association between having two or more distinct SAH ICD10-CM codes across their record and chart review revealing a history of multiple aneurysm was not significant (Χ 2 = 1.12, P = .29) and had a positive predictive value of 20.3%. Comparing the more strict   year 3). Figure 3 shows the frequency of monthly code use over the study period. Among those code instances that specified an aneurysm location, the accuracy of the location specified was 65.2% in the first

| Coding usage and accuracy over time
year and improved to 72.3% and 76.6% in the second and third years. ture. 19,20 These data suggest that there are not obvious differences between the patient population seen at UCHealth and those represented in the scientific literature.
Although we specifically examined billing uses of the ICD10-CM codes (ie, the diagnosis used to justify payment for healthcare services rendered), only 64% were used during an active aSAH episode. This performance is lower than international studies that validated ICD10 SAH detection rates across three hospitals in Calgary, Canada (91% correct; CI: 76%-98%). 21 This difference may be due to the different F I G U R E 4 Accuracy of ICD10-CM codes that specify location of nontraumatic subarachnoid hemorrhage to detect any history of intracranial aneurysm across 1 October 2015 to 30 September 2018. The monthly total number of ICD10-CM code instances that specify location of non-traumatic subarachnoid hemorrhage (I60.0-I60.6) across the study period. The total number of code instances that correctly identified both the history of an intracranial aneurysm (ruptured or unruptured) and the artery location is displayed in green. The total number of code instances that correctly identified a history of intracranial aneurysm but not the correct artery is displayed in orange. The total number of code instances that did not correctly identify a history of intracranial aneurysm is displayed in red healthcare reimbursement landscapes in Canada (single payer) vs the United States. Based on our data, researchers using US claims data should be wary of interpreting ICD10-CM non-traumatic subarachnoid hemorrhage code use as indicative of an active aSAH. However, the accuracy of these codes to detect any patient history of an intracranial aneurysm was high (88.1%) suggesting that cautious use of these codes for aneurysm research may be appropriate.
When detecting any patient history of an aneurysm, the location accuracy was moderate (73.2%) and increased to 76.6% in the third year of the study. The majority of errors indicated the wrong artery location (14.9%), and these errors were not evenly distributed across arteries. Among the three most common arteries identified (middle cerebral, anterior communicating, and posterior communicating), the At our institution, the coding team meets annually with the neurovascular provider team but do not have specialty informatics tools to support more specific coding. The number of very similar abbreviations used across neurosurgical notes makes these types of errors more likely and should be considered by researchers when using codes for these sites.
Although the focus of this project was on determining the accuracy of ICD10-CM codes for identifying aSAH occurrence and aSAH location, we also investigated whether these codes could be used to detect patients who harbor multiple aneurysms. Patients who have multiple aneurysms are at higher risk for aSAH. [8][9][10][11] Unfortunately, neither definition of multiple aneurysms used had sufficient resolution (PPV < 35%) for use in research.
This study is limited by the fact that it represents a single healthcare system, with the majority of reviewed records coming from a single hospital. Additionally, we did not characterize reasons for less specific code usage (ie, to determine whether less-specific codes were used because the aneurysm location was not specified in the record) or examine potential false negatives (ie, aSAH records that never received an ICD10-CM code at all). Finally, we only examined the first 3 years of data after implementation of ICD10-CM. In our data, both the usage and accuracy of these codes increased over the study period. It is not clear whether this trend will continue linearly or eventually plateau at a stable usage rate. More studies utilizing other centers as well as longitudinal studies to assess for increasing trends of specific and correct usage will help clinicians and data scientists better assess the utility of ICD10-CM codes for long-term intracranial aneurysm research.
It is logical to assume that the newly available level of specificity of ICD10-CM will lead many researchers to incorporate aneurysm location into future clinical data science studies. Aneurysm location is an important predictor of rupture for unruptured aneurysms, 22-24 and the location is often a surrogate for the difficulty in repair. However, our data suggest that these codes are used infrequently and have variable accuracy to detect intracranial aneurysm outcomes and location.
Therefore, depending on their specific research question, researchers should use these data with caution when attempting to detect active aSAH and/or aneurysm location.