Validation of International Classification of Diseases criteria to identify severe influenza hospitalizations

Abstract In this cohort study of hospitalized patients with linked medical record data, we developed International Classification of Diseases (ICD) criteria that accurately identified laboratory‐confirmed, severe influenza hospitalizations (positive predictive value [PPV] 80%, 95% confidence interval [CI] 71–87%), which we validated through medical record documentation. These criteria identify patients with clinically important influenza illness outcomes to inform evaluation of preventive and therapeutic interventions and public health policy recommendations.


| INTRODUCTION
Influenza control is a global public health priority. However, as of 2014, only 59% of countries had an influenza vaccination policy. 1 In recommendations for influenza vaccine research and development, the World Health Organization (WHO) stated that "well-designed studies demonstrating influenza vaccine impact on important public health outcomes," such as pneumonia and severe illness, "would strengthen the case for their use globally." 2 A major limitation to such studies is that the incidence of severe influenza illness is relatively rare, making administrative claims databases across large populations the most efficient data sources for these studies. 3,4 We aimed to determine the accuracy of International Classification of Diseases (ICD) codes from claims-based data in identifying severe influenza hospitalizations, which we validated through medical record documentation. Our goal was to address the WHO call for more relevant data on clinically important influenza illness outcomes to inform research, evaluation of preventive and therapeutic interventions, and public health policy recommendations.

| METHODS
We conducted a retrospective cohort study of Tennessee Medicaid (TennCare) 5 (Table S1): • Influenza pneumonia: One or more ICD-9/10 codes for influenza pneumonia • Influenza with respiratory insufficiency: One or more ICD-9/10 codes for influenza AND one or more ICD-9/10 codes for acute respiratory distress/failure, respiratory and circulatory disorders, or continuous mechanical ventilation • Influenza with other non-respiratory illness or organ system involvement: One or more ICD-9/10 codes for influenza AND one or more ICD-9/10 codes for central nervous system disorders, diseases of the digestive or genitourinary system, shock, sepsis, or in-hospital death To access and manually extract medical record information, we restricted our study population to hospital encounters at VUMC.
We developed a case report form for medical record extraction with a team of influenza, pulmonary, and critical care experts (Material S1). Two independent physicians extracted medical record information from a random subset of VUMC encounters from unique patients, including laboratory confirmation of influenza virus infection by polymerase chain reaction (PCR), viral culture, and/or rapid antigen test. Pneumonia diagnosis was identified through medical record documentation of pneumonia and/or radiographic findings.
Respiratory insufficiency was identified through medical record documentation of apnea, asthma/chronic obstructive pulmonary disease exacerbation, cystic fibrosis exacerbation, mechanical ventilation, oxygen requirement or increased oxygen requirement over baseline, or documented pulmonary function decline. Other nonrespiratory illness or organ system involvement was identified through medical record documentation of acute renal, cardiac, or neurologic deterioration; secondary bacterial infection; sepsis/bacteremia; sickle cell pain crisis; or diabetic ketoacidosis. Hospitalized patients must have had both laboratory confirmation of influenza and medical record documentation of pneumonia, respiratory insufficiency, or other non-respiratory illness or organ system involvement to be defined as having laboratory-confirmed, severe influenza based on medical record extraction. We also broadly defined laboratory-confirmed influenza hospitalizations as encounters with laboratory confirmation of influenza with or without medical record documentation of pneumonia, respiratory insufficiency, or other non-respiratory illness or organ system involvement based on medical record extraction.
We calculated the positive predictive value (PPV) for (1) laboratory-confirmed influenza hospitalizations and (2) laboratoryconfirmed, severe influenza hospitalizations by dividing the number of patients with medical record documentation of each of these conditions by the total number of patients identified using our severe influenza hospitalization ICD criteria. We calculated 95% confidence intervals (CIs) for the PPVs using Wilson's formula. 7 We additionally performed sensitivity analyses to assess the validity of our criteria in identifying severe influenza hospitalizations among children and among individuals with and without underlying respiratory comorbidities. We performed all analyses using R software version 4.0.4 (R foundation for statistical computing, Vienna, Austria).
Additional information on methodology can be found in Material S1.

| RESULTS
We identified 25,521 hospitalizations among TennCare enrollees that met the ICD criteria for severe influenza from 1995 through 2017 ( Figure S1). Approximately 93% of these hospitalizations occurred during the influenza season. Among these encounters, 1% were hospitalizations at the study hospital. We extracted medical record information from a random subset of 100 hospitalizations from unique patients. More than 50% of these patients were non-Hispanic, White, female, and non-smokers (Table S2) (n = 12) had influenza with other non-respiratory illness or organ system involvement, and 55% (n = 44) had more than one of these severe influenza events ( Figure 2). All five patients with laboratoryconfirmed influenza during hospitalization but no medical record documentation of pneumonia, respiratory insufficiency, or other non-respiratory illness or organ system involvement had chest X-rays without indication of pneumonia (Figure 1).
We performed sensitivity analyses to assess the validity of our criteria in identifying severe influenza hospitalizations among children and individuals with and without underlying respiratory comorbidities.

| DISCUSSION
In this cohort study of hospitalized patients with linked medical record data, we developed ICD criteria that accurately identified patients with laboratory-confirmed, severe influenza hospitalizations, which we validated through medical record documentation. Our criteria also had high PPVs when assessed among children and individuals with and without underlying respiratory comorbidities. Severe influenza is a rare but important outcome, particularly for studies estimating influenza morbidity and vaccine effectiveness. 2,8,9 Therefore, these criteria could be used to identify patients with clinically important influenza illness outcomes to inform research, evaluation of preventive and therapeutic interventions, and public health policy recommendations.
The accuracy of diagnostic codes in identifying individuals with influenza illness has been previously studied with varying results. In a recent study utilizing a population-based Canadian cohort, ICD-10 criteria identified hospitalized patients with laboratory-confirmed influenza with moderate sensitivity (73%) and high PPV (94%). 10 Influenza-specific ICD codes have previously been shown to have sensitivities ranging from 65-86% among children specifically. 3,11,12 ICD-9 codes have also been found to accurately estimate the prevalence of influenza pneumonia in hospitalized adults. 13 To our knowledge, this is the first study to develop criteria to identify severe influenza illness.
Our study was strengthened by our use of a large administrative database to identify severe influenza hospitalizations and linkage with medical records for confirmation. Our detailed case report form was developed and reviewed by a team of experts, and medical record extraction was performed by two independent physicians, which increased the validity of our findings.
We must also recognize some limitations. We restricted our study population to those who had severe influenza during the influenza season, which would have missed cases occurring outside the typical influenza season for the United States. However, as long as the coding pattern is consistent during the influenza season and non-season, our algorithm should be equally capable of identifying severe influenza events occurring outside the influenza season. As medical record extraction is a tedious endeavor, we were only able to extract information for a subset of patient records, which resulted in small sample sizes for sensitivity analyses and prevented us from having sufficient cases to assess the validity of our ICD criteria among additional subgroups at high risk for influenza hospitalization. Medical record extraction was also only performed at a single academic medical center, which may not reflect documentation and coding at other hospitals.

| CONCLUSIONS
We developed ICD criteria that had high PPV for identifying hospitalized patients with laboratory-confirmed, severe influenza. As we created these criteria to specifically identify severe influenza-related hospitalizations for research, they could be applied to identify patients with important influenza public health outcomes, thus addressing the WHO call for more relevant data to inform influenza policy and the impact of preventive and therapeutic interventions.