A systematic review of validated methods for identifying acute respiratory failure using administrative and claims data


G. Schneider, Epidemiology and Database Analytics, United BioSource Corporation, 430 Bedford St., Suite 300, Lexington, MA 02420, USA. E-mail: gary.schneider@unitedbiosource.com



The Food and Drug Administration's (FDA) Mini-Sentinel pilot program initially aims to conduct active surveillance to refine safety signals that emerge for marketed medical products. A key facet of this surveillance is to develop and understand the validity of algorithms for identifying health outcomes of interest (HOIs) from administrative and claims data. This paper summarizes the process and findings of the algorithm review of acute respiratory failure (ARF).


PubMed and Iowa Drug Information Service searches were conducted to identify citations applicable to the anaphylaxis HOI. Level 1 abstract reviews and Level 2 full-text reviews were conducted to find articles using administrative and claims data to identify ARF, including validation estimates of the coding algorithms.


Our search revealed a deficiency of literature focusing on ARF algorithms and validation estimates. Only two studies provided codes for ARF, each using related yet different ICD-9 codes (i.e., ICD-9 codes 518.8, “other diseases of lung,” and 518.81, “acute respiratory failure”). Neither study provided validation estimates.


Research needs to be conducted on designing validation studies to test ARF algorithms and estimating their predictive power, sensitivity, and specificity. Copyright © 2012 John Wiley & Sons, Ltd.


Mini-Sentinel is the Food and Drug Administration's (FDA) pilot program that aims to conduct active surveillance of automated health care data. The initial goal is to refine safety signals that emerge for marketed medical products. Essential components of this exercise are (i) to identify administrative and claims data—friendly algorithms used to detect various health outcomes of interest (HOIs)—and (ii) to identify the performance characteristics of these algorithms as measured within the studies in which they were used. In this article, we describe the algorithm review process and findings for 1 of the 20 HOIs selected for review by the FDA: acute respiratory failure (ARF).

The respiratory system can be divided into two parts: the lung, which is responsible for gas exchange, and the “pump” (the chest wall, central nervous system [CNS] respiratory controllers, and the nerves that connect the two), which ventilates the lung.[1, 2] ARF is a condition in which one or both of these parts fail. Hypoxemia (type I respiratory failure) occurs when the lung fails and gas exchange becomes dysfunctional; failure of the pump causes ventilator failure, which manifests as hypercapnia (type II respiratory failure).[2] Type I respiratory failure is generally caused by lung disease, including pneumonia, pulmonary embolism, and acute respiratory distress syndrome (ARDS). Type II respiratory failure is caused by anatomical and functional defects of the CNS, impairment of neuromuscular transmission, and mechanical defects of the ribcage, as well as conditions leading to fatigue of the respiratory muscles.[2] Acute hypercapnia can also occur during acute exacerbations of chronic pulmonary conditions such as chronic obstructive pulmonary disease (COPD), although it is important to differentiate it from chronic-onset or insidious-onset hypercapnia, which is most frequently seen in COPD and is associated with a particularly poor prognosis.[2]

Patients with ARF vary in terms of demographics and disease etiology, severity, and prognosis. Treatment can occur at a range of settings, including emergency departments, hospital wards, or the intensive care unit (ICU).[3] These characteristics, as well as a lack of a consensus definition for the condition, make ARF difficult to study in the community. Therefore, most studies of the epidemiology of ARF focus on its most severe forms. A US study defined ARF as the condition for which inpatients have discharge records of acute respiratory distress or failure, mechanical ventilation, and at least 24 h of hospitalization; using that definition, the incidence of ARF was estimated to be 137.1 hospitalizations per 100 000 US residents aged ≥5 years.[4] Studies from Berlin[5] and Scandinavia[6] defined ARF as the condition wherein patients are intubated and mechanically ventilated for ≥24 h in ICUs, and these studies reported ARF incidences of 88.6 and 77.6 cases per 100 000 population per year, respectively. The incidence of ARF increases substantially with age and is especially high among persons 65 years of age and older.[1, 4] ARF requiring mechanical ventilation is associated with poor survival: in-hospital mortality ranges from 35.9%[4] to 42.7%,[5] with lower survival influenced by older age and the degree of comorbid multi-organ failure.[3]


The general search strategy originated from prior work by the Observational Medical Outcomes Partnership and its contractors and was modified slightly for the 20 HOIs selected for review.

Details of the methods for these systematic reviews can be found in the accompanying manuscript by Carnahan and Moores.[7] In brief, the base PubMed search was combined with the following terms to represent the HOI: “respiratory insufficiency,” “respiratory” AND “insufficiency,” “respiratory failure” AND “respiratory” AND “failure.”

To identify other relevant articles that were not found in the PubMed search, the Iowa Drug Information Service Web (IDIS/Web) was searched using a similar search strategy. Both the PubMed and IDIS searches were conducted on 10 May 2010. An additional PubMed search was conducted on 6 July 2010 to amend the original search strategy with additional databases. All searches were restricted to articles published in 1990 or later. The details of these searches can be found in the full report on the Mini-Sentinel website: http://mini-sentinel.org/foundational_activities/related_projects/default.aspx.

The search results were compiled, and duplicate results were eliminated. The results were then output and provided to organizations contracted to conduct the literature reviews. Mini-Sentinel collaborators were also asked to help identify relevant validation studies.

The abstract of each citation identified was reviewed by two investigators. When either investigator selected an article for full-text review, the full text was reviewed by both investigators. Agreement on whether to review the full text or include the article in the evidence table was calculated via Cohen's kappa statistic. A single investigator abstracted each study for the final evidence table; data included in the table were confirmed by a second investigator for accuracy. A clinician or topic expert was consulted to review the results of the evidence table and discuss how they compared with diagnostic methods currently used in clinical practice. This included whether certain diagnostic codes used in clinical practice were missing from the algorithms, and the appropriateness of the validation definitions compared with diagnostic criteria currently used in clinical practice.


The total number of citations identified from the combined searches was 242 (PubMed: 170, IDIS: 69, additional PubMed: 3); with the exclusion of overlaps, the number of unique citations was 207. Mini-Sentinel collaborators provided no additional reports of validation studies.

Of the 207 abstracts reviewed, we accepted six for full-text review. The straightforward inclusion criteria, consisting of (i) examination of the HOI of interest, (ii) use of administrative and claims databases, and (iii) study conducted in the USA or Canada, enabled perfect agreement between the two reviewers on acceptance/rejection status, although there was substantial variation in the reasons for rejection.

Of the six full-text articles reviewed, two were excluded for not being administrative and claims database studies and two for not focusing on the HOI. The two remaining studies did not report validation of the ARF coding algorithm directly in the article, nor within a reference cited in the article. Perfect agreement between reviewers on inclusion versus exclusion of full-text articles was achieved.

Summary of algorithms

We identified only two studies containing ARF algorithms, neither of which had corresponding validation estimates. Wu et al.[8] used ICD-9 code 518.8 (other diseases of lung) to identify respiratory failure cases among Medicare beneficiaries. ICD-9 code 518.81 (acute respiratory failure) was used by Dransfield et al.[9] to identify hospitalized patients with a primary diagnosis of ARF. Information on the study populations, outcomes, and algorithms used in each of these studies is presented in Table 1.

Table 1. Acute respiratory failure coding algorithms
CitationStudy population and study periodDescription of outcome studiedAlgorithm
Dransfield et al. 2008[9]Patients admitted to University of Alabama Hospital whose discharge or death summaries indicated a primary diagnosis of acute exacerbation of chronic obstructive lung disease (International Classification of Diseases, Ninth Edition [ICD-9] code 491.21) or a primary diagnosis of acute respiratory failure (518.81) and a secondary diagnosis of acute exacerbation were identified. Patients with a diagnosis of asthma (493) were excluded; 825 patients met the inclusion criteria for the study, of which 410 were males and 415 were females. The mean age of patients was 66.5 years. The study period was 1 October 1999–30 September 2006.In-hospital mortality after use of β-blockers.Acute respiratory failure: 518.81.
Wu et al. 2003[8]Patients over the age of 65 years who underwent total hip arthroplasty in the Medicare database were identified from the part B data using CPT codes (n = 23 136; 8180 males and 14 956 females). Patients were eligible to be included if the procedure was performed by an orthopedic surgeon (part B) with an accompanying inpatient record for the same procedure (part A). Major morbidity counts at 7 and 30 days after the procedure were obtained from part B based on ICD-9 diagnosis codes, one of which corresponded to respiratory failure (ICD-9 code 518.8). The study period was 1994–1999.Morbidity and death at 7 and 30 days after hip replacement surgery.Respiratory failure: 518.8.


In practice, one would expect that ICD-9 code 518.81 would be commonly used to identify ARF cases, as it explicitly defines ARF. One would also expect this code to have high positive predictive value and high specificity.

The sensitivity of ICD-9 code 518.81 may be more questionable. It is possible that patients with ARF may be coded with other conditions in administrative and claims data. Likely examples are chronic respiratory failure (CRF; ICD-9 code 518.83) and ARDS (ICD-9 code 518.82). CRF differs from ARF in terms of presentation, treatment, and interventions[3]; therefore, examination of procedure codes may prove beneficial in discerning these two distinct types of respiratory failure. By contrast, treatments and procedures for ARDS are similar to ARF; in fact, ARDS is a possible cause of ARF.[10] Thus, examination of procedural codes may be inadequate to differentiate between these conditions. Administrative and claims data with corresponding laboratory results, however, may prove viable in distinguishing between ARF and ARDS. There is also an ICD-9 code specifying both conditions (i.e., ICD-9-CM code 518.84, Acute and chronic respiratory failure); examination of procedural codes may prove useful here as well.

Much of what is known about ARF in the USA is from a study using data from the 1994 Nationwide Inpatient Sample.[4] As these are not administrative and claims data, this article did not meet the inclusion criteria of the present study. Nevertheless, it provides an example of an ARF algorithm that implements both diagnostic and procedure codes. Diagnostic codes for acute respiratory distress or failure (ICD-9-CM 518.5 [Pulmonary insufficiency following trauma and surgery], 518.81, or 518.82) combined with a procedure code for continuous mechanical ventilation (ICD-9-CM 96.7) were used, thereby identifying severe ARF cases. Via this definition, the oft-cited US ARF incidence of 137.1 hospitalizations per 100 000 US residents aged ≥5 years[4] was estimated.

It should be noted that in both the medical literature and clinical practice, ARF is usually described as secondary to other diseases/abnormalities or trauma.[3, 4] If interest lies in ARF as a result of a specific condition, we suggest that algorithm development incorporate disease/condition-specific codes combined with ICD-9 code 518.81 (or an alternative ARF algorithm).


The most recent estimates are that approximately 330 000 ARF hospitalizations occur annually in the USA, with 31-day hospital mortality at 31.4%.[4] Despite this epidemiology, our current search highlights a scarcity of literature providing validated or non-validated algorithms for ARF that can be applied to administrative and health care data. Research needs to be conducted on designing validation studies to test ARF algorithms and estimating their predictive power, sensitivity, and specificity.


The authors declare no conflict of interest. This is not product-specific or privately funded research. The views expressed in this document do not necessarily reflect the official policies of the Department of Health and Human Services, nor does mention of trade names, commercial practices, or organizations imply endorsement by the US government.


  • There is limited literature focusing on acute respiratory failure that provides administrative and claims data-based coding algorithms and validation estimates.
  • Additional research is needed regarding the use of administrative and claims data-based coding algorithms to identify acute respiratory failure.


This work was supported by the Food and Drug Administration (FDA) through Department of Health and Human Services (HHS) contract number HHSF223200910006I.