A systematic review of validated methods for identifying heart failure using administrative data

ABSTRACT

Purpose

To identify and describe the validity of algorithms used to detect heart failure (HF) using administrative and claims data sources.

Methods

A systematic review of PubMed and Iowa Drug Information Service searches of the English language was performed to identify studies published between 1990 and 2010 that evaluated the validity of algorithms for the identification of patients with HF using and claims data. Abstracts and articles were reviewed by two study investigators to determine their relevance on the basis of predetermined criteria.

Results

The initial search strategy identified 887 abstracts. Of these, 499 full articles were reviewed and 35 studies included data to evaluate the validity of identifying patients with HF. Positive predictive values (PPVs) were in the acceptable to high range, with most being very high (>90%). Studies that included patients with a primary hospital discharge diagnosis of International Classification of Diseases, Ninth Revision, code 428.X had the highest PPV and specificity for HF. PPVs for this algorithm ranged from 84% to 100%. This algorithm, however, may compromise sensitivity because many HF patients are managed on an outpatient basis. The most common ‘gold standard’ for the validation of HF was the Framingham Heart Study criteria.

Conclusions

The algorithms and definitions used to identify HF using administrative and claims data perform well, particularly when using a primary hospital discharge diagnosis. Attention should be paid to whether patients who are managed on an outpatient basis are included in the study sample. Including outpatient codes in the described algorithms would increase the sensitivity for identifying new cases of HF. Copyright © 2012 John Wiley & Sons, Ltd.

INTRODUCTION

Large administrative and claims databases can identify individuals and hospitalizations for use in population-based research and surveillance. Although the use of administrative and claims data may efficiently identify patients for inclusion in a study cohort, the validity of published algorithms for identifying patients with heart failure (HF) has not been well described.

HF is a major public health problem and an emerging epidemic.[1] It is estimated that more than 700,000 new cases of HF occur annually[2, 3] and that one in every five middle-aged man and woman will develop HF during their lifetime.[4] Administrative and claims databases have been used in the active surveillance of HF and to compare the effectiveness of different treatments for patients with this clinical syndrome.[5-8] To perform such studies or evaluations, it is necessary to develop and understand the validity of algorithms used to identify patients with HF in administrative and claims data. The goals of this project were to identify algorithms used to detect HF using administrative and claims data sources and to describe the performance characteristics of these algorithms as reported by the studies in which they were used.

This project was conducted as part of the U.S. Food and Drug Administration's Mini-Sentinel pilot program. As part of this program, systematic reviews to identify validation studies of algorithms to identify health outcomes of interest in administrative and claims data were commissioned. This manuscript provides an overview of the HF algorithm. The full report can be found at http://mini-sentinel.org/foundational_activities/related_projects/default.aspx.

METHODS

The methods and details for Mini-Sentinel systematic reviews have been described elsewhere (see “Mini-Sentinel's systematic reviews of validated methods for identifying health outcomes using administrative and claims data: methods and lessons learned” by Carnahan and Moores on page 82). The specific search strategy for the HF review can be found in the full report, which can be found at http://mini-sentinel.org/foundational_activities/related_projects/default.aspx. Briefly, PubMed and Iowa Drug Information Service (IDIS) searches were performed to identify studies published between 1990 and June 2010 that evaluated the validity of algorithms for identifying HF in administrative and claims data. Certain search terms related to administrative and claims data are described in detail by Carnahan and Moores on page 82 and were included in all Mini-Sentinel systematic review searches. In addition to these key words, the following PubMed search terms were used for the HF report: ‘Heart Failure’ [Mesh]. In addition, the IDIS search included specification of the following terms: 428. (Note: 428 includes failure, heart NEC; failure, heart, congestive; failure, heart, left; failure, heart, systolic; and failure, heart, diastolic.) Mini-Sentinel collaborators were requested to identify any published or unpublished work that validated an algorithm to identify HF in administrative and claims data.

Two Mini-Sentinel investigators reviewed all abstracts identified through the initial PubMed and IDIS searches, identifying potentially relevant articles on the basis of predefined criteria. Articles were excluded from full-text review if they did not study HF, were not based on an administrative or claims data set, or included a data source outside of the USA or Canada. Articles identified for full review by either investigator were retrieved and reviewed by two investigators. In the event of disagreement between reviewers, the full article was reviewed.

Selected articles were reviewed with the goal of identifying validated algorithms for identifying HF in administrative and claims data. Investigators also identified citations from the article's reference sections if they were cited as studies validating an algorithm for HF or were otherwise deemed to be potentially relevant. Articles identified through reference sections were reviewed in a similar manner. A single investigator abstracted information for each study, which included the following: database, coding system (e.g. International Classification of Diseases, Ninth Revision [ICD-9], codes), study population (including information on inpatient and outpatient composition of the sample), time period, incident or prevalent case, specific algorithm used to identify cases of HF, adjudication criteria (e.g. Framingham criteria), validation process (e.g. medical record review), and validation statistics. The second reviewer confirmed the accuracy of abstracted information.

Cohen's kappa for agreement was calculated between reviewers for the inclusion versus exclusion of abstracts and full-text text articles.

RESULTS

Identification and selection of articles

The initial PubMed and IDIS searches identified 887 unique abstracts for review. Of these, 499 were selected for full-text review. Cohen's kappa for agreement between reviewers on whether or not to include the study in the full-text review was 0.65. One of the articles selected for full-text review could not be located. Of the 498 articles included in the full-text review, 25 were included in the final report and evidence table (Table 1). Reviewers identified 40 additional citations for review from full-text article references. Of these, 9 were included in the final report. Cohen's kappa for agreement between reviewers on the inclusion versus exclusion of full-text articles in the final report was 0.83. Lastly, Mini-Sentinel collaborators identified one additional report that had not been identified through other searches and was included in the final report. Thus, a total of 35 studies reporting the validation of algorithms to identify cases of HF through administrative and claims data were included in the final report and evidence table (Table 1).

Table 1. PPVs by algorithm to identify HF

Citation	Study population and time period	Description of outcome studied	Algorithm	Validation/adjudication procedure, operational definition, and statistics
Ahmed et al.[30]	Secondary analysis of data from study by DeLong et al.[34]; Medicare beneficiaries 65 years or older identified using the Alabama Quality Assurance Foundation database, 1994	Hospitalizations (prevalent and incident)	Principal discharge diagnosis of HF identified with ICD-9-CM codes 428 and 402.91	Medical record review was conducted (n = 1091); outcome was confirmed based on history of HF symptoms, signs (or radiographic evidence of HF), or treatment with both digoxin and diuretic
				Two or more criteria: PPV = 99%
				Three or more criteria: PPV = 86%
Alqaisi et al.[31]	Members 18 years and older of a large HMO in southeast Michigan receiving care from a large, multispecialty medical group, 2004 to 2005	Prevalent and incident	At least one encounter code for HF (excluding all emergency department encounters); various algorithms evaluated that included ICD-9 codes: 428.xx, 398.91, 402.01, 402.11, or 402.91 plus laboratory data	Medical record review; outcome was confirmed if Framingham criteria for HF met: two major criteria or one major and two minor criteria; PPV = 86%
Ansari et al.[24]	Members of northern California Kaiser Permanente, 1996 to 1997	Incident	Outpatient encounter form with ICD-9 codes 428.0, 425.0, 402.1, 402.11, 402.91, 404.01, 404.3, 404.11–404.15 (excluding patients with a prior outpatient visit or primary or secondary diagnosis of an HF-related diagnosis on a prior hospital discharge and patients admitted within 24 h of their diagnosis)	Medical record review; outcome was confirmed using Framingham criteria
				PPV = 97% for confirmation of HF
				PPV = 78% for confirmation of ‘incident’ HF
Austin et al.[9]	Patients 20 years and older included from Fastrak II acute coronary syndromes registry and matched with Canadian Institute of Health Information hospital discharge data, before March 2000	Hospitalizations (incident and prevalent)	Primary discharge diagnosis ICD-9 code 428; primary or secondary diagnosis ICD-9 code 428	Linkage to Fastrak II registry was performed; 14% of patients with discharge diagnosis could be linked to the Fastrak II CCU registry; outcome was confirmed if HF diagnosis present in Fastrak II registry
				Primary diagnosis: specificity = 96.8%, sensitivity = 58.5%, PPV = 65.1%
				Primary or secondary diagnosis: specificity = 84.3%, sensitivity = 85.4%, PPV = 35.8%
Baker et al.[32]	Patients older than 18 years seen two or more times in the general internal medicine clinic of Northwestern Faculty Foundation, 2003 to 2004	Incident and prevalent	Diagnosis of HF on problem list or medical history but no encounter diagnoses and patients who had only a single-encounter diagnosis of HF (ICD-9-CM codes: 398.91, 402.01, 402.11, 402.91, 404.91, 404.01, 404.03, 404.11, 404.13, 404.91, 404.93, 428.x)	Medical record review was conducted; reviewed 28 charts for all patients who had a diagnosis of HF on problem list or medical history but not encounter diagnoses and reviewed charts for 66 patients who had only a single-encounter diagnosis of HF; outcome was confirmed if there was documentation of HF in physician notes
				PPV = 57%
Birman-Deych et al.[29]	Medicare beneficiaries who were hospitalized with atrial fibrillation identified using the National Registry of Atrial Fibrillation II data set	Hospitalizations (prevalent and incident)	Inpatient ICD-9-CM codes 428.x, 398.91, 402.01, 402.11, 402.91, 404.01, 404.11, 404.91, 404.03, 404.13, 404.93	Medical record review
				Outcome was confirmed if there was documentation of a history of HF and/or current HF
				Current or past HF: sensitivity = 76%, specificity = 97%
				Primary diagnosis for baseline hospitalization: sensitivity = 33%, specificity = 99%
				Any position for baseline hospitalization: sensitivity = 83%, specificity = 86%
Borzecki et al.[33]	Veterans Affairs patients with at least one hypertension diagnosis (ICD-9-CM code 401, 402, or 405) and additional sample without a hypertension diagnosis identified using the outpatient clinic and patient treatment file, Department of Veterans Affairs (VA) databases, 1998 to 1999	Incident or prevalent	Inpatient or outpatient ICD-9-CM codes: 398.91, 402.01, 402.11, 402.91, 404.01, 404.03, 404.11, 404.13, 404.91, 404.93, 414.8, 428.xx	Medical record review (981 patients with a hypertension diagnosis and 195 without a hypertension diagnosis); outcome was confirmed based on documentation of HF in medical notes.
				Sensitivity = 77%; specificity = 99%
Brar et al.[25]	Female members of Kaiser Permanente Southern California hospitalized with HF 6 months before or 9 months after delivery, 1996 to 2005	Hospitalizations/incident peripartum cardiomyopathy	Hospitalization with HF identified through ICD-9-CM codes 428.0, 428.1, 428.4, 428.9, 425.4, 425.9	Medical record review (n = 240); peripartum cardiomyopathy was confirmed if all following criteria were met: ejection fraction <0.50, met Framingham criteria for HF, new symptoms of HF or initial diagnosis of left ventricular dysfunction occurred in the month before or in the 5 months after delivery, and no other cause of HF could be identified
				PPV = 25%
Brophy et al.[10]	Patients diagnosed with atrial fibrillation identified using the Veterans Affairs Boston Healthcare System database, 1998 to 2001	Incident and prevalent	Inpatient or outpatient ICD-9-CM code 428.x	Medical record review; criteria for confirmation of cases were unspecified
				Sensitivity = 98%, specificity = 83%, PPV = 80%
Curtis et al.[26]	Members of a large geographically diverse US healthcare organization 50 years and older with at least two ICD-9-CM diagnosis codes for rheumatoid arthritis or Crohn's disease plus tumor necrosis factor-α antagonist or immunosuppressive drug use, 1998 to 2002	Incident	Inpatient or outpatient ICD-9-CM codes: 428.xx, 402.01, 402.11, 402.91, 404.01, 404.03, 404.11, 404.13, 404.91, 404.93, 425.4, 425.5, 425.7, 425.8, 425.9	Medical record review (n = 29); confirmed cases satisfied at least one major and two minor modified Framingham criteria and clinical judgment of physician reviewers
			Excluded patients with a diagnosis of HF before the index date	PPV = 31%
Dauterman et al.[11]	Medicare patients 65 years and older identified using data from the Medicare Professional Review Organization project, California state hospital discharges, 1993 to 1994, 1996	Hospitalizations (prevalent and incident)	Primary discharge diagnosis of ICD-9 428.0, 428.1, 428.9	Medical record review (n = 1720); outcome was confirmed based on history and physical examination and either an LVEF <40% or a chest radiograph with pulmonary edema or cardiomegaly
				PPV = 96%
DeLong et al.[34]	Medicare beneficiaries 65 years or older identified using the Alabama Quality Assurance Foundation database, 1994 (baseline) and 1995 to 1997 (follow-up)	Hospitalizations (prevalent and incident)	Hospitalization with DRG 127	Medical record review (n = 1251 at baseline and n = 743 at follow-up); outcome was confirmed if at least three of the following were documented: shortness of breath, dyspnea on exertion, orthopnea, paroxysmal nocturnal dyspnea, fatigue, tiredness, exhaustion, or lower extremity edema
				PPV = 79.1% for patients identified at baseline
				PPV = 83.6% for patients identified at follow-up
Ezekowitz et al.[19]	Patients older than 18 years, Alberta, Canada, 2002 to 2003 (from Richter et al., 2009)	Incident and prevalent	Emergency department most responsible diagnosis ICD-10 I50.X code	Medical record review (n = 483).
				Outcome was confirmed based on Framingham criteria or physician's final diagnosis
				PPV = 93%
Go et al.[35]	Kaiser Permanente of Northern California members 20 years and older, 1996 to 2004	Hospitalizations (prevalent and incident)	One or more hospitalization with a principal diagnosis of HF (ICD-9 codes: 398.91, 402.01, 402.11, 402.91, 428.0, 428.1, 428.9); two hospitalizations with a secondary diagnosis of HF with the principal diagnosis related to the disease (e.g. coronary heart disease); three or more hospitalizations with secondary diagnosis of HF; two or more outpatient diagnoses; three or more emergency department visit diagnoses; two or more inpatient secondary diagnoses plus one outpatient diagnosis	Medical record review (n = 9533); outcome was confirmed if a physician-assigned HF diagnosis was documented
				PPV = 97%
Goff et al.[12]	Patient admitted to special care units at seven hospitals in Nueces County, Texas, with diagnoses possibly indicative of coronary heart disease and those who underwent bypass surgery or revascularization, aged 25 through 74 years, 1998 to 1994	Hospitalizations (incident and prevalent)	Discharge diagnosis ICD-9 codes: 398.91, 402.x1, 404.x, 415.0, 416.9, 425.4, 428.x, 429.4, 514, 518.4, 786.0; three algorithms assessed: (i) presence of ICD-9 428; (ii) presence of either ICD-9 code 428 or 402; (iii) presence of any of ICD codes previously listed	Medical record review (n = 5083); outcome was confirmed if documentation in a progress note or in the discharge summary that the patient experienced an episode of acute HF or notation of pulmonary edema in a report of a chest radiograph
				ICD-9 428: Sensitivity = 62.8%, Specificity = 95.4%, PPV = 83.5%, NPV = 87.4%
				ICD-9 code 428 or 402: sensitivity = 66.2%, specificity = 93.3%, PPV = 78.5%, NPV = 88.2%
				Any of ICD-9 codes listed: sensitivity = 67.1%, specificity = 92.6%, PPV = 77.1%, NPV = 88.3%
Grijalva et al.[13]	TennCare enrollees 18 years and older diagnosed with rheumatoid arthritis, 1995 to 2004	Hospitalizations (new onset or exacerbation of HF)	Principal discharge diagnosis of ICD-9-CM code 428.X	PPV = 100%
Havranek et al.[36]	Medicare patients throughout the USA (National Heart Failure project), 1988 to 1999	Hospitalizations (incident and prevalent)	Primary discharge diagnosis ICD-9 codes: 402.01, 402.11, 402.91, 404.01, 404.11, 404.91, 428.x	Medical record review (n = 100); outcome was confirmed based on cardiologist review and judgment
				PPV = 99%
Iribarren et al.[27]	Kaiser Permanente Northern California members 19 years and older with diabetes who were responders to a survey and who had no previous hospitalization with a primary or secondary diagnosis of HF during the 5 years before, 1995 to 1997	Hospitalizations (incident)	Primary discharge diagnosis of ICD-9 428.x, 402.01, 402.11, 402.91	Medical record review was conducted for a random sample of 200 patients; outcome was confirmed based on Framingham criteria
				PPV = 97%
Jollis et al.[22]	Discharges containing a procedure code for coronary arteriography identified using administrative or insurance claims of Duke University Medical Center, 1985 to 1990	Hospitalizations (incident and prevalent)	Discharges with an ICD-9-CM code of 428.0, 428.1, 428.9, 398.91, 402.01, 402.11, 402.91	Clinic database was compared with coding by medical record technicians (n = 12937); outcome was confirmed based on documentation in the clinical data
				Sensitivity = 36%, specificity = 96%
Jong et al.[14]	Patients 20 years and older, hospitalized in Ontario (14 acute care hospitals; Canadian Institute for Health Information), 1997 to 1999	Hospitalizations (incident)	Primary diagnosis of ICD-9 code 428; excluded those cases in which it was not the first admission for HF and patients who had a diagnosis of HF coded during any hospital admission in the 5 years before this study	Medical record review (n = 1346); outcome was confirmed if two major or one major and two minor Framingham criteria were concurrently present, or if the Carlson HF score exceeded 4 points
				Framingham criteria: PPV = 96%
				Carlson criteria: PPV = 90%
Klatsky et al.[15]	Kaiser Permanente members, San Francisco and Oakland, 1978 to 1985 (baseline) through 2000	Hospitalizations (incident and prevalent)	Primary discharge diagnosis code 428 (and no separate primary discharge diagnosis of CAD-codes 411 to 414)	Medical record review (n = 1907); outcome was confirmed based on Framingham criteria
				PPV = 95%
Lee et al.[16]	Patients 105 years and younger admitted to 14 hospitals in Ontario, 1997 to 1999	Hospitalizations (incident and prevalent)	Primary most responsible diagnosis of HF ICD-9-CM code 428.x	Medical record review 836 women and 805 men); outcome was confirmed based on Framingham criteria and Carlson criteria
				Framingham criteria: PPV = 94.3% (PPV = 94.6% in women and 93.9% in men)
				Carlson criteria: PPV = 88.6% (PPV = 89.4% in women and 87.8% in men)
Lee et al.[37]	Kaiser Permanente of Northern California members 18 years and older, 1999 to 2000	Hospitalizations (incident and prevalent)	Primary diagnosis ICD-9 codes: 402.01, 402.11, 402.91, 425.0 to 425.5, 425.7, 428.0, 428.1, 428.9	Medical record review (n = 1700); outcome was confirmed based on Framingham clinical criteria
				PPV = 93.6%
Lentine et al.[23]	Kidney transplant patients at Washington University 18 years and older with Medicare as primary insurer, 1991 to 2002	Incident or prevalent	ICD-9-CM codes: 398.91, 422, 425, 428, 402.x1, 404.x1, 404.x3, V42.1; identified with Medicare Part A (institutional) claims and/or Medicare Part B (physician/suppliers) claims	Transplant center's clinical database was used to confirm HF, including physician-reported diagnosis plus objective evidence of cardiac dysfunction: echocardiography or other forms of ventriculography, chest radiograph, and/or B-natriuretic peptide
				Claims within 30 days from event date recorded in the database:
				Medicare Part A sensitivity = 75.0% (95%CI = 63.7–86.3%)
				Part B sensitivity = 85% (95%CI = 75.7%–94.3%)
				Part A or B sensitivity = 92.5% (95%CI = 85.6%–99.4%)
				One Part A claim or two Part B claims submitted at least 1 day but no more than 365 days apart: sensitivity = 92.5%
McCullough et al.[38]	Henry Ford Health System members, 1989 to 1999	Incident or prevalent	Two or more outpatient or one hospitalization ICD-9 CM codes: 428.x, 398.91, 402.01, 402.11, 402.91, 404.00, 404.01, 404.03, 404.10, 404.11, 404.13, 404.90, 404.91, 404.93. Hospitalizations required DRG 127 OR one of the ICD-9-CM codes in the principal position OR DRG 124 and one of the abovementioned ICD-9 codes in the principal diagnosis position	Medical record review (1% sample; n = 271); outcome was confirmed based on Framingham criteria, NHANES definition of HF, and confirmation by an internist and cardiologist by chart notes
				Framingham criteria: PPV = 63.5%
				NHANES definition (score ≥ 3): PPV = 55.7%
				Physician assessment: PPV = 82.9%
Owan et al.[39]	Patients admitted to Mayo Clinic hospitals, 1987 to 2001	Hospitalizations (incident and prevalent)	Inpatient ICD-9-CM code 428 plus DRG code 127	Medical record review (n = 135); outcome was confirmed based on modified Framingham criteria or the clinical criterion (diagnosis of HF recorded on the chart by the attending physician)
				Framingham criteria: PPV = 95%
				Clinical or Framingham criteria: PPV = 99%
Park et al.[40]	Medicare beneficiaries 65 years and older, 1983 to 1984	Hospitalizations (incident and prevalent)	Primary diagnosis ICD-9 codes 398.91, 402.11, 402.91, 428.0, 428.1, 428.9, 785.51	Medical record review (n = 1600); outcome was confirmed based on review and determination by physician principal investigator that the primary diagnosis was accurately coded
				PPV = 84%
Philbin et al.[41]	New York state hospital discharges (Statewide Planning and Research Cooperative System (SPARCS) database—New York state), 1995	Hospitalizations (incident and prevalent)	Primary diagnosis ICD-9-CM codes 428.0, 402.91, 404.93, 428.1, 402.11, 398.91, 404.91, 404.13, 402.01, 404.03, 404.11, 404.01, 428.9	Medical record review (3% sample); outcome was confirmed based on documentation of typical symptoms, physical findings, laboratory results, and response to appropriate therapy
				PPV = 96%
Philbin et al.[42],	Patients from 10 acute care hospitals collaborating in a study of quality of care in HF, 1995	Hospitalizations (incident and prevalent)	DRG codes 127 and DRG code 124 with principal diagnosis was one of the ICD-9 codes required for DRG 127	Medical record review; outcome was confirmed based on presence of appropriate medical history, physical findings, laboratory results and response to appropriate therapy
				PPV = 96%
Quan et al.[43]	Hospitalizations identified using Calgary Regional Health Authority data, 1996 to 1997	Hospitalizations (incident and prevalent)	ICD-9-CM codes 428, 428.9	Medical record review n = 1200); outcome was confirmed based on definitions described by Charlson et al. 1987
				Sensitivity = 77.3%, specificity = 98.7%, PPV = 87.6%, NPV = 97.3%
Rathore et al.[44]	Medicare beneficiaries from CMS National Heart Failure Project, 1998 to 1999, 2000 to 2001	Hospitalizations (incident and prevalent)	ICD-9 codes 402.01, 402.11, 402.91, 404.01, 404.91, 428	Medical record review w (n = 66178); outcome was confirmed based on clinical evidence
				PPV = 92.4%
Rodeheffer et al.[45]	Olmstead County, MN residents ages 0 to 74 years (Rochester Epidemiology Project), 1981 to 1982	Incident and prevalent	ICD-8 code 427	Medical record review (n = 366); outcome was confirmed based on Framingham criteria
				PPV = 69.6%
Roger et al.[18]	Olmstead County, MN residents (Rochester Epidemiology Project), 1997 to 2000	Incident	First diagnosis of HF based on ICD-9-CM codes: 428, 402.01, 402.11, 425, 429.3, 514	Medical record review; outcome was confirmed based on Framingham criteria
				ICD-9-CM 428: PPV = 82%
				Other codes used in isolation without a code 428: PPVs range from 14% to 30%
Schellenbaum et al.[28]	Cardiovascular health Study: Medicare eligible residents (≥65 years) in Sacramento County, CA; Washington County, MD; Forsyth County, NC; Allegheny County, PA, 1989, 1990, 1992, 1993	Hospitalizations (incident)	Discharge diagnosis ICD-9 428, 997.1, 425, 402.01, 402.11, 402.91, 398.91	Medical record review (n = 1209); outcome was confirmed based on decision by an events committee consisting of five physicians after review of documentation on medical history, physical examination, chest X-ray reports, and medication use
				PPV = 54%
So et al.[17]	Patients 20 years and older hospitalized with acute myocardial infarction at four teaching hospitals in Alberta, Canada, 2003	Hospitalizations (incident and prevalent)	Inpatient ICD-9-CM codes: 428.x; ICD-10 codes: I09.9, I11.0, I13.0, I13.2, I25.5, I42.0, I42.5-I42.9, I43.x, I50.x, P29.0	Medical record review (n = 193); outcome was confirmed based on evidence of HF in chart
				ICD-9-CM: sensitivity = 81.8% (95%CI = 69.1–92.0); specificity = 96.4% (91.8–98.8); PPV = 90.0% (78.2–96.7); NPV = 93.0% (87.5–96.6)
				ICD-10 codes: sensitivity = 80.0% (67.0–90.0); specificity = 97.8% (93.8–99.6); PPV = 93.6% (82.5–98.7); NPV = 92.5% (86.9–96.2)

Algorithms and validation

Validation algorithms

All 35 publications listed in Table 1 used ICD-8, ICD-9, ICD-10, or diagnosis related group (DRG) codes to identify patients with HF. Most of included studies used ICD-9 codes to identify patients with outpatient encounters or hospitalizations for HF. All of the studies that used ICD-9 codes included code 428.x alone or in combination with other ICD-9 codes. Other common ICD-9 codes used in the algorithms of the included studies were 398.91, 402.01, 402.11, 402.91, 404.01, 404.03, 404.11, 404.13, 404.91, 404.93, 425, and 429.3 (for definitions of these HF-related codes, see Appendix A). In general, differences in the positive predictive values (PPVs) for the use of ICD-9 code 428.x alone as compared with its combined use with other ICD-9 codes were negligible. Nine studies reported validation statistics for ICD-9 code 428.x alone.[9-17] With one exception,[9] these studies reported high PPVs (range = 84%–100%). Two studies compared the validation of ICD-9 code 428.x with other ICD-9 codes. One study[12] directly compared the validation of ICD-9 code 428.x with other algorithms using ICD-9 428 in combination with other ICD-9 codes and reported the highest PPV for ICD-9 code 428 alone (PPV = 84%); the combination of ICD-9 428 or 402 yielded a PPV of 79% and the combination of ICD-9 428, or the presence of several ICD-9 codes for HF yielded a PPV of 77%. A second large community-based study[18] reported a PPV of 84% for ICD-9 code 428.x compared with PPVs of 14% to 30% for other algorithms (including ICD-9 codes 402.01, 402.11, 425, 429.3, and 514) that did not include code 428.

Only two studies validated a diagnosis of HF against ICD-10 codes;[17, 19] ICD-10 code I50 was always included in these algorithms. One study[17] also included the following additional ICD-10 codes: I09.9, I11.0, I13.0, I13.2, I25.5, I42.0, I42.5-I42.9, I43.x, and P29.0. Because few studies reported the validation of ICD-10 codes, it is difficult to comment on the validation statistics between ICD-9 and ICD-10 coding algorithms. However, one study[17] directly compared the validation of ICD-9 and ICD-10 codes for evidence of HF in medical charts. This study found the PPV of the ICD-10 codes (I09.9, I11.0, I13.0, I13.2, I25.5, I42.0, I42.5-I42.9, I43.x, I50.x, and P29.0) to be slightly higher than the PPV of the ICD-9 code 428.x (94% vs 90%).

Several studies used DRG code 127; one study used DRG code 124 in combination with an ICD-9 code for HF. The PPVs for DRG codes ranged widely from 55% to 96% with the majority being less than 70%.

Validation criteria and method

Nearly all studies included in the report validated administrative and claims coding data through the abstraction of medical charts. Documentation of HF in the medical records was generally based on physician notes. Several studies specified criteria from medical records (e.g. signs, symptoms, or radiographic evidence of HF; treatment with both digoxin and a diuretic; ejection fraction <40%). The most commonly used ‘standardized criteria’ to validate algorithms were the Framingham criteria for HF (presence of two major or one major and two minor criteria),[20] which were applied in 14 studies. The Carlson criteria (≥4)[21] were also applied in two studies, but these studies also applied the Framingham criteria. When both the Framingham and Carlson criteria were applied, the PPVs for validation using the Framingham criteria were higher (PPVs = 94-96%) than that of the Carlson criteria (PPVs = 88-90%). The National Health and Nutrition Examination Survey (NHANES) criteria (≥3) were applied in one study as a comparison with the Framingham criteria. Similar to the Carlson criteria, the NHANES criteria had a lower PPV (56%) compared with the Framingham criteria.

One study[9] used registry data to validate administrative claims data. Jollis et al.[22] and Lentine et al.[23] used clinical databases developed for specific patient populations (patients undergoing cardiac catheterization and kidney transplantation, respectively) to validate administrative codes. The reported estimates for sensitivity varied greatly with the different algorithms assessed in these studies. Although the reported estimates for specificity in the studies by Jollis et al.[22] and Austin et al.[9] were generally quite high (96%), the reported PPVs were quite low in the study by Austin et al. (65% or lower). However, in this study only 14% of patients with a discharge diagnosis for HF could be linked to the registry.[9]

Age and sex of study population

Most studies included only adult populations, with many studies including patients who were older (≥65 years). Studies that included younger patients were generally those that included entire member populations of health plans, who were over a certain age (most commonly 18 years and older). Because the prevalence of HF increases with advancing age, in studies that included wide age ranges, it is likely that a large proportion of the patients were 65 years and older. No information was provided on the proportion of validated cases of HF by age group. In general, PPVs did not vary significantly according to whether the study populations included all ages or were restricted to older patients. Only one study[16] reported the validity of ICD-9-code 428.x according to patient's sex and found similar PPVs in men and women (94% and 95%, respectively, based on Framingham criteria for HF).

Time period of data collection

This report includes publications between 1990 and 2010; most of the studies included study populations identified between 1990 and 2005. Several studies examined earlier periods (e.g. Jollis et al.[22] reported on data from 1985 to 1990 and Klatsky et al.[15] reported on baseline data from 1978 to 1985 with follow-up through 2000). The resulting validation statistics did not vary significantly in earlier study periods (i.e. before 2000) compared with later study periods (e.g. 2000 and later).

Incident versus prevalent outcome validation

Most of the studies validated both incident and prevalent cases of HF. Seven studies reported only on incident outcomes.[14, 18, 24-28] In general, the validation statistics for studies of incident cases of HF were adequate, ranging from 54% to 97%. With the exception of one study,[28] all studies validating incident cases of HF used the Framingham criteria as the validation criteria. Schellenbaum et al.[28] reported a significantly lower PPV for the validation of incident HF events in the Cardiovascular Health Study (PPV = 54%). In this study, events were confirmed by an events committee rather than by standardized clinical criteria (e.g. Framingham criteria),[20] which may be a potential explanation for the lower percentage of cases validated.

Two studies examining incident episodes of HF systematically excluded patients with a prior diagnosis of HF (based on the ICD-9 codes used to identify the incident cases) in the 5 years before the years under study.[14, 27] Both studies reported high PPVs (96% and 97%), suggesting that the systematic exclusion of prevalent cases using the algorithm for identifying incident cases may be an important consideration in studies ascertaining newly diagnosed cases of HF. Only one study reported data to calculate validation statistics for both incident and prevalent outcomes,[24] thus allowing for a comparison of the two. This study reported a high PPV (97%) for all cases (prevalent and incident) of HF and a significantly lower, although adequate, PPV (78%) for incident cases of HF. A PPV for prevalent cases alone was not reported.

Primary versus secondary diagnosis

The outcome for most of the studies included in this report was hospitalization for HF. Approximately half of the studies reporting on hospitalization for HF specified that HF was the primary or most responsible discharge diagnosis in their algorithm for the identification of cases of HF. In general, the validation of HF in the studies that used the primary discharge diagnosis was high, with all but one study[9] reporting PPVs >90%. Studies that identified cases of HF according to discharge diagnoses in any position had slightly lower validation statistics compared with the studies that used only the first or primary diagnosis, with PPVs ranging from 79% to 96%, with more than half <90%.

Two studies separately validated both a primary discharge diagnosis of HF and a discharge diagnosis of HF in other positions.[9, 29] Birman-Deych et al.[29] compared an algorithm using the primary discharge diagnosis with another algorithm using the discharge diagnosis in any other position. The sensitivities and specificities for the primary discharge diagnosis of HF were 33% and 99%, respectively, compared with 83% and 86% for a diagnosis in any position. Using data from a registry of patients with an acute coronary syndrome as validation criteria, Austin et al.[9] reported a PPV of 65% for patients with a primary discharge diagnosis of HF and a PPV of 36% for patients with a primary or secondary discharge diagnosis.

Hospital discharge diagnosis versus outpatient encounter

Several studies used outpatient encounters alone or in combination with hospital discharge diagnoses to identify patients with HF. In general, these studies had lower PPVs than studies using hospital discharge diagnoses only, with PPVs ranging from 63% to 97%, most of which were <90%. Several studies used algorithms that included both inpatient and outpatient diagnoses; however, they did not directly compare algorithms using one versus the other. For instance, the algorithm reported by Go et al.[35] included one or more hospitalization with a primary diagnosis of HF, two or more outpatient diagnoses of HF, or ≥3 emergency department visit diagnoses for HF. This algorithm yielded a PPV of 97% for medical record review of physician assigned diagnosis of HF.

DISCUSSION

The use of administrative and claims databases for efficiently identifying patients with specific conditions from large population-based samples is extremely valuable. Understanding the validity of coding systems and corresponding algorithms is paramount for outcomes research and surveillance using these populations. We examined the validity of algorithms used to detect HF using administrative and claims data sources.

Although several algorithms resulted in high PPVs, the use of ICD-9 code 428.x seems most appropriate to yield cases of HF on the basis of this review, with the caveat that it will have a high PPV and specificity but may have a low sensitivity.

Several factors related to validation statistics are important to consider when choosing a diagnostic algorithm. The distinction between incident and prevalent cases of possible HF is important in deciding on which algorithm to use. It is important to specify at the outset of a study which cases are of interest; a study examining the association between a medical product exposure and the new occurrence of HF will require an algorithm that identifies incident cases with a high PPV. However, prevalent cases would be of interest when examining the prescribing of medications or medical products that are contraindicated in patients with HF, or in identifying a cohort of patients with an existing diagnosis who are followed prospectively for exacerbations leading to hospitalization. The mixing of incident with prevalent cases will affect then interpretation of validation statistics.

Validation statistics were higher, in general, for studies that reported on hospitalized patients only (most PPVs >95%). Those that included both hospitalizations and outpatient encounters tended to have lower PPVs (most <90%). Because most care for patients with HF presently occurs in the outpatient setting, identifying algorithms that perform well and incorporate both inpatient and outpatient encounters deserves further study. Administrative and claims data sets may lack the clinical accuracy needed for surveillance of certain disease states, particularly chronic diseases managed largely on an outpatient basis.

Several clinical methods have been routinely used to confirm the presence of HF including the Framingham criteria and the criteria of Carlson et al. However, the inclusion of contemporary and widely used diagnostic tests, including BNP (brain natriuretic peptide) levels into HF diagnostic criteria, may improve sensitivity and specificity.

Gaps in the current literature include specific comparisons of algorithms for hospitalized versus outpatient study populations with possible HF. Comparison of the validation of inpatient and outpatient algorithms against the Framingham Heart Study criteria for HF would be most useful to compare the findings from other studies, as would the validation of incident versus prior events. Algorithms also have not been validated in different age strata, particularly in elderly and very old patient populations with additional comorbidities, who comprise most patients presenting with HF. In addition, very few validation studies have been conducted on ICD-10 codes or in patients of different race/ethnicities in whom the criteria published to date may have varying sensitivities and specificities. Lastly, current coding systems do not allow for algorithms that distinguish systolic and diastolic HF or that detail a patient's disease severity.

CONCLUSION

HF can be validly identified using administrative and claims databases. Studies that included a primary hospital discharge diagnosis of ICD-9 code 428.X had the highest PPV and specificity. This algorithm, however, may compromise sensitivity because many patients with HF are managed on an outpatient basis. Characteristics of the sample population and details related to the diagnosis of HF, including whether cases are incident or prevalent, should be considered when choosing a diagnostic algorithm.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

KEY POINTS

• Specific algorithms to identify patients with HF had high PPVs (>90%).
• PPVs differed according to whether cases were prevalent or incident and whether hospital or outpatient encounters were used.
• Administrative data are appropriate to use to identify patients with HF.

ACKNOWLEDGEMENT

This study was supported by the US Food and Drug Administration through the Department of Health and Human Services (HHS) Contract Number HHSF22320091006I. The views expressed in this document do not necessarily reflect the official policies of the Department of Health and Human Serves, nor does the mention of trade names, commercial practices, or organizations imply endorsement by the US Government. Dr. Saczynski was supported in part by funding from the National Institute on Aging (K01 AG33643) and the National Heart Lung and Blood Institute (U01 HL105268). Dr. Cutrona was supported in part by Award Number KL2RR031981 from the National Center for Research Resources (NCRR). Dr. Harrold was funded by K23AR053856 from the National Institute of Arthritis and Musculoskeletal and Skin Diseases.

APPENDIX: List and definitions of ICD or procedural codes included in algorithms

Type of Code	Code	Description
DRG	124	Circulatory disorders other than acute myocardial infarction with cardiac catheterization and complex diagnosis
DRG	127	Heart failure and shock
ICD-8	427	Symptomatic heart disease
ICD-9	398.91	Rheumatic heart failure
ICD-9	402.01	Malignant hypertensive heart disease with heart failure
ICD-9	402.1	Benign hypertensive heart disease
ICD-9	402.11	Benign hypertensive heart disease with heart failure
ICD-9	402.91	Hypertensive heart disease unspecified with heart failure
ICD-9	404.00	Malignant heart and chronic kidney disease without heart failure and with chronic kidney disease stage I–stage IV or unspecified
ICD-9	404.01	Malignant hypertensive heart and chronic kidney disease with heart failure
ICD-9	404.03	Malignant hypertensive heart and chronic kidney disease with heart failure and chronic kidney disease stage V or end-stage renal disease
ICD-9	404.10	Benign hypertensive heart and chronic kidney disease without heart failure and with chronic kidney disease stage I–stage IV or unspecified
ICD-9	404.11	Benign hypertensive heart disease and chronic kidney disease with heart failure and with chronic kidney disease stage I–stage IV or unspecified
ICD-9	404.12	Benign hypertensive heart disease and chronic kidney disease without heart failure and with chronic kidney disease stage V or end-stage renal disease
ICD-9	404.13	Benign hypertensive heart disease and chronic kidney disease with heart failure and chronic kidney disease stage V or end-stage renal disease
ICD-9	404.90	Hypertensive heart disease and chronic kidney disease unspecified without heart failure and with chronic kidney disease stage I–stage IV or unspecified
ICD-9	404.91	Hypertensive heart disease and chronic kidney disease unspecified with heart failure and with chronic kidney disease stage I–stage IV or unspecified
ICD-9	404.93	Hypertensive heart disease and chronic kidney disease unspecified with heart failure and with chronic kidney disease stage V or end-stage renal disease
ICD-9	414.8	Other chronic ischemic heart disease
ICD-9	415.0	Acute cor pulmonale
ICD-9	416.9	Chronic pulmonary heart disease unspecified
ICD-9	422	Acute myocarditis*
ICD-9	425	Cardiomyopathy*
ICD-9	425.0	Endomyocardial fibrosis
ICD-9	425.1	Hypertrophic obstructive cardiomyopathy
ICD-9	425.2	Obscure cardiomyopathy of Africa
ICD-9	425.3	Endocardial fibroelastosis
ICD-9	425.4	Other primary cardiomyopathies
ICD-9	425.5	Alcoholic cardiomyopathy
ICD-9	425.7	Metabolic cardiomyopathy
ICD-9	425.8	Cardiomyopathy in other diseases classified elsewhere
ICD-9	425.9	Secondary cardiomyopathy unspecified
ICD-9	428	Heart failure*
ICD-9	428.0	Congestive heart failure unspecified
ICD-9	428.1	Left heart failure
ICD-9	428.20	Systolic heart failure unspecified
ICD-9	428.21	Acute systolic heart failure
ICD-9	428.22	Chronic systolic heart failure
ICD-9	428.23	Acute on chronic systolic heart failure
ICD-9	428.30	Diastolic heart failure unspecified
ICD-9	428.31	Acute diastolic heart failure
ICD-9	428.32	Chronic diastolic heart failure
ICD-9	428.33	Acute on chronic diastolic heart failure
ICD-9	428.40	Combined systolic and diastolic heart failure
ICD-9	428.40	Combined systolic and diastolic heart failure unspecified
ICD-9	428.41	Acute combined systolic and diastolic heart failure
ICD-9	428.42	Chronic combined systolic and diastolic heart failure
ICD-9	428.43	Acute on chronic combined systolic and diastolic heart failure
ICD-9	428.9	Heart failure unspecified
ICD-9	429.3	Cardiomegaly
ICD-9	429.4	Heart disease following cardiac surgery
ICD-9	514	Pulmonary congestion and hypostasis
ICD-9	518.4	Acute lung edema unspecified
ICD-9	785.51	Cardiogenic shock
ICD-9	786.0	Dyspnea/respiratory abnormality*
ICD-9	997.1	Cardiac complications during or resulting from a procedure
ICD-9	V42.1	Heart transplant status
ICD-10	I09.9	Rheumatic heart disease unspecified
ICD-10	I11.0	Hypertensive heart disease with heart failure
ICD-10	I13.0	Hypertensive heart and chronic kidney disease with heart failure and chronic kidney disease stage I–stage IV or unspecified
ICD-10	I13.2	Hypertensive heart and chronic kidney disease with heart failure and chronic kidney disease stage V or end-stage renal disease
ICD-10	I25.5	Ischemic cardiomyopathy
ICD-10	I42.0	Dilated cardiomyopathy
ICD-10	I42.5	Other restrictive cardiomyopathy
ICD-10	I42.6	Alcoholic cardiomyopathy
ICD-10	I42.7	Cardiomyopathy due to drugs and external causes
ICD-10	I42.8	Other cardiomyopathies
ICD-10	I42.9	Cardiomyopathy, unspecified
ICD-10	I43	Cardiomyopathy in diseases classified elsewhere
ICD-10	I43.0	Cardiomyopathy in infectious and parasitic diseases classified elsewhere
ICD-10	I43.1	Cardiomyopathy in metabolic diseases
ICD-10	I43.2	Cardiomyopathy in nutritional diseases
ICD-10	I43.8	Cardiomyopathy in other diseases classified elsewhere
ICD-10	I50	Heart failure
ICD-10	I50.0	Congestive heart failure
ICD-10	I50.1	Left ventricular failure
ICD-10	I50.9	Heart failure unspecified
ICD-10	P29.0	Neonatal cardiac failure