Validation of ICD‐10 hospital discharge diagnosis codes to identify incident and recurrent ischemic stroke from a US integrated healthcare system

This study validated incident and recurrent ischemic stroke identified by International Classification of Diseases, Tenth Revision, Clinical Modification (ICD‐10) hospital discharge diagnosis codes.


Key Points
• Little is known of the validity of ICD-10 codes for recurrent ischemic stroke.Using ICD-10 codes, we identified incident and recurrent ischemic stroke cases of 75 patients through electronic health records and neurologists adjudicated these stroke cases through chart reviews.
• Our results show that ICD-10 diagnosis codes were correctly identified for most incident ischemic stroke cases but did not perform as well for recurrent ischemic stroke cases.
Removing stroke-related procedures improved the validity of ICD-10 codes for recurrent ischemic stroke.
• Algorithms to further improve the accuracy of ICD-10 codes for recurrent ischemic stroke may be necessary.

| INTRODUCTION
Stroke is a leading cause of morbidity and mortality in the US and worldwide. 1[7][8][9] Furthermore, to date, no studies have validated ICD-10 codes for recurrent ischemic stroke.This study validated ICD-10 hospital discharge diagnosis codes of incident and recurrent ischemic stroke.

| Study population
The study population included adult members with an ischemic stroke diagnosis within the Kaiser Permanente Southern California (KPSC) integrated healthcare delivery system.KPSC provides services to over 4.8 million members at 233 medical offices throughout Southern California.
Using electronic health records (EHR) data, we limited our sample size to members with ≥1 principal hospital discharge ICD-10 diagnosis codes of ischemic stroke, I63.x (cerebral infarction), G46.3 (brain stem stroke syndrome), or G46.4 (cerebellar stroke syndrome) from October 1, 2015 to September 30, 2020.We further limited the study population to members with ≥2 principal hospital discharge ICD-10 codes of ischemic stroke that were at least 2 days apart.We identified 1347 patients with 1422 hospital admissions for potential stroke events.Among the 1347 patients, 75 patients had ≥2 hospital admissions for potential stroke events (all had I63.x codes).We conducted chart review of first (incident) and second (recurrent) potential stroke events of these 75 patients (total 150 cases).

| Case adjudication
Two neurologists (PC, EW) independently reviewed EHR data associated with the 150 incident and recurrent ischemic stroke cases using the 2013 American Heart Association/American Stroke Association 1 case definition of ischemic stroke.Ischemic stroke is defined as an episode of neurological dysfunction caused by focal cerebral, spinal, or retinal infarction based on (1) pathological, imaging, or other objective evidence of cerebral, spinal cord, or retinal focal ischemic injury in a defined vascular distribution or (2) clinical evidence of cerebral, spinal cord, or retinal focal ischemic injury based on symptoms persisting ≥24 h or until death, and other etiologies excluded. 10All associated admission records were examined: progress notes, discharge summary, magnetic resonance imaging, computerized tomography scan, medications, and laboratory values.A third neurologist (NS) reviewed discrepant adjudications to finalize the result.Adjudicators provided rationale for incorrect stroke diagnoses.

| Statistical analysis
We evaluated inter-rater reliability by calculating Gwet's agreement coefficient (AC1) by dividing the total number of cases by the number of cases in agreement between two reviewers. 11Positive predictive value (PPV) for incident and recurrent ischemic stroke was calculated by dividing the number of confirmed cases by the total number of cases reviewed, respectively.PPVs were stratified by each ICD-10 code and by days between the first and second stroke events (20-, 30-, 45-, 60-, 90-, 180-or 365-day gap).Statistical significance of differences in PPVs was determined using Fisher's exact tests.As an exploratory analysis, we calculated PPVs after removing hospital admissions related to stroke-related procedures (CPT codes 35301 [thromboendarterectomy], 37215 and 37217 [transcatheter placement of intravascular stent]).

| Case adjudication
Through case adjudication of 75 incident stroke cases, 72 cases (96%) were confirmed as any stroke and 70 cases (93%) were confirmed as ischemic stroke (Table A1).Most (85%, 64 cases) met both objective and clinical ischemic stroke adjudication criteria.Through chart review of the 75 recurrent stroke cases, 55 (73%) were confirmed cases as ischemic stroke, and 54 (72%) were confirmed as recurrent ischemic stroke (Tables A2 and A3).The median number of days between the first and second stroke events was 48 (IQR: 16, 249).Most recurrent stroke cases (70%, 52 cases) occurred after 21 days from the first event.effect of previous stroke (5 cases), and recrudescence of stroke (4 cases).

| DISCUSSION
This study validated ICD-10 hospital discharge diagnosis codes to identify incident and recurrent ischemic stroke from a US integrated healthcare system.It benefitted from access to robust EHR data, and the expertise of trained neurologists performing double adjudication.
The neurologist reviewers had high agreement on incident stroke cases identified via ICD-10 codes.[7][8][9] Identifying recurrent ischemic stroke via ICD-10 diagnosis codes seems more challenging, particularly when the recurrent stroke occurs immediately after the incident stroke.For recurrent ischemic stroke, compared to incident stroke, adjudicators encountered more false positives, with the PPV of 72%.Epidemiological literature suggests applying a gap of 21-28 days between the incident and recurrent stroke. 12,13en applying a 20-day gap between two stroke events, the PPV increased from 65% for events occurring within 20 days to 75% for events occurring more than 20-days apart, although this increase did not reach statistical significance.When exploring alternative gaps between two stroke events and stroke-related procedures, increasing the gap between two events did not significantly improve the PPVs compared with not considering any gap.Additionally, when excluding hospitalizations with stroke-related procedures from recurrent stroke cases, the PPV for recurrent stroke increased to 78%; this increase was not statistically significant.While the PPV of 78% is a lower range of previously reported PPVs for incident stroke, it is higher than reported PPVs (12.9%-72.5%)for transient ischemic attack. 14ough the PPV of ICD-10 codes for ischemic incident stroke has been examined, 6 to date, no studies have examined the PPV of ICD-10 codes for recurrent ischemic stroke.This study has some limitations.
Since the results focus only on one healthcare system in a specific region, results may not be generalizable to other settings.We did not include claims data from external hospitals.While findings for incident ischemic stroke are reassuring, future studies investigating a larger array of healthcare networks would be useful in validating ICD-10 codes across settings.
Additionally, we were not able to differentiate recrudescence (the return of ischemic stroke symptoms previously resolved) from recurrent stroke using structured data.Though we were able to exclude hospitalizations with stroke-related procedures using codes, recrudescence events were only identified through chart reviews and often incorrectly coded as a new stroke event.Finally, we limited our review to first and second stroke cases, though a few patients had three or more stroke events.
In conclusion, we found a high PPV for incident ischemic stroke using ICD-10 hospital discharge diagnosis codes, but a less optimal PPV for recurrent ischemic stroke.Algorithms to improve ICD-10 code accuracy, such as removing stroke-related procedures to identify recurrent ischemic stroke, may be necessary.