SEARCH

SEARCH BY CITATION

Keywords:

  • erythema multiforme;
  • administrative and claims data;
  • Mini-Sentinel;
  • coding algorithm

ABSTRACT

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. SUMMARY OF ALGORITHMS
  7. DISCUSSION
  8. CONCLUSIONS
  9. CONFLICT OF INTEREST
  10. ACKNOWLEDGEMENTS
  11. REFERENCES

Purpose

The Food and Drug Administration's (FDA) Mini-Sentinel pilot program aims to conduct active surveillance to refine safety signals that emerge for marketed medical products. A key facet of this surveillance is to develop and understand the validity of algorithms for identifying health outcomes of interest (HOIs) from administrative and claims data. This paper summarizes the process and findings of the algorithm review of erythema multiforme and related conditions.

Methods

PubMed and Iowa Drug Information Service searches were conducted to identify citations applicable to the erythema multiforme HOI. Level 1 abstract reviews and Level 2 full-text reviews were conducted to find articles that used administrative and claims data to identify erythema multiforme, Stevens–Johnson syndrome, or toxic epidermal necrolysis and that included validation estimates of the coding algorithms.

Results

Our search revealed limited literature focusing on erythema multiforme and related conditions that provided administrative and claims data-based algorithms and validation estimates. Only four studies provided validated algorithms and all studies used the same International Classification of Diseases code, 695.1. Approximately half of cases subjected to expert review were consistent with erythema multiforme and related conditions.

Conclusions

Updated research needs to be conducted on designing validation studies that test algorithms for erythema multiforme and related conditions and that take into account recent changes in the diagnostic coding of these diseases. Copyright © 2012 John Wiley & Sons, Ltd.


INTRODUCTION

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. SUMMARY OF ALGORITHMS
  7. DISCUSSION
  8. CONCLUSIONS
  9. CONFLICT OF INTEREST
  10. ACKNOWLEDGEMENTS
  11. REFERENCES

Mini-Sentinel is the Food and Drug Administration's (FDA) pilot program that aims to conduct active surveillance using automated health care data. The initial goal is to refine safety signals that emerge for marketed medical products. Essential components of this exercise are (i) to identify administrative and claims data-friendly algorithms used to detect various health outcomes of interest (HOIs) and (ii) to identify the performance characteristics of these algorithms as measured within the studies in which they were used. In this article, we describe the algorithm review process and findings for 1 of the 20 HOIs selected for review by the FDA: erythema multiforme (EM) major/minor/not otherwise specified, Stevens–Johnson syndrome (SJS), or toxic epidermal necrolysis (TEN).

Erythema multiforme is an acute inflammatory disorder of the skin and/or mucous membranes that is considered a hypersensitivity reaction to certain medications and infections.[1, 2] EM was previously considered to be a spectrum of clinical conditions including (in order of severity) the following: EM minor, EM major, SJS, and TEN (also known as Lyell's disease).[1] It now is recognized as a distinct condition with clinical and epidemiological characteristics separate from those of SJS and TEN.[2-4] SJS and TEN are considered to be the same disease entity with different severities.[3, 5]

Erythema multiforme is characterized by skin lesions that are generally palpable “typical targets” or “raised atypical targets,” with epidermal detachment for less than 10% of the body's surface area (BSA) and minimal mucous membrane involvement.[6] Infections are the primary etiology, with herpes simplex virus (HCV) accounting for more than 50% of cases.[2] Other common etiologies include mycoplasma pneumonia, fungal infections, and medications such as barbiturates, hydantoins, nonsteroidal anti-inflammatory drugs, penicillins, phenothiazines, and sulfonamides.[2]

Stevens–Johnson syndrome and TEN are both characterized by severe mucosal erosions; diffuse, non-palpable, flat atypical targets; and, commonly, a prodrome of fever and flu-like symptoms. The two conditions differ in the extent of epidermal detachment, with SJS limited to less than 10% of BSA and TEN involving 30%–100%.[1, 3] SJS and TEN are most often caused by a hypersensitivity reaction to a medication; however, certain infectious diseases have also been associated with SJS/TEN.[4]

The incidence of EM, SJS, and TEN has not been well studied. One study estimated the overall incidence of US hospitalization for these combined conditions at 7.4 per million person-years.[7] More recent reports estimate TEN incidence at 1.9 per million annually.[4] Both SJS and TEN can be fatal, with reported mortality rates of 1%–5% and 25%–35%, respectively.[4]

METHODS

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. SUMMARY OF ALGORITHMS
  7. DISCUSSION
  8. CONCLUSIONS
  9. CONFLICT OF INTEREST
  10. ACKNOWLEDGEMENTS
  11. REFERENCES

The general search strategy originated from prior work by the Observational Medical Outcomes Partnership and its contractors and was modified slightly for the 20 HOIs selected for review.

Details of the methods for these systematic reviews can be found in the accompanying manuscript by Carnahan and Moores.[8] In brief, the base PubMed search was combined with the following terms to represent the HOI: “Erythema Multiforme,” “Toxic Epidermolysis,” “Toxic Epidermal Necrolysis,” “Stevens–Johnson Syndrome,” “Fiessinger–Rendu Syndrome,” “Lyell's Syndrome,” “Lyell Syndrome,” “Badder's Syndrome,” “Badder Syndrome,” “Rowell's Syndrome,” “Rowell Syndrome,” and “Staphylococcal Scalded-Skin Syndrome.”

For the identification of other relevant articles that were not found in the PubMed search, the Iowa Drug Information Service Web (IDIS/Web) was also searched using a similar search strategy. Both the PubMed and IDIS searches were conducted on 10 May 2010. A similar EMBASE search was conducted on 23 June 2010. An additional PubMed search was conducted on 6 July 2010 to amend the original search strategy with additional databases. All searches were restricted to articles published in 1990 or after. The details of these searches can be found in the full report on the Mini-Sentinel website at http://mini-sentinel.org/foundational_activities/related_projects/default.aspx.

The search results were compiled, and duplicate results were eliminated. The results were then output and provided to organizations contracted to conduct the literature reviews. Mini-Sentinel collaborators were also requested to help identify relevant validation studies.

The abstract of each citation identified was reviewed by two investigators. When either investigator selected an article for full-text review, the full text was reviewed by both investigators. Agreement on whether to review the full text or include the article in the evidence table was calculated using the Cohen's kappa statistic. A single investigator abstracted each study for the final evidence table; the data included in the table were confirmed by a second investigator for accuracy. A clinician or topic expert was consulted to review the results of the evidence table and discuss how they compared with diagnostic methods currently used in clinical practice. This included whether certain diagnostic codes used in clinical practice were missing from the algorithms, and the appropriateness of the validation definitions compared with diagnostic criteria currently used in clinical practice.

RESULTS

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. SUMMARY OF ALGORITHMS
  7. DISCUSSION
  8. CONCLUSIONS
  9. CONFLICT OF INTEREST
  10. ACKNOWLEDGEMENTS
  11. REFERENCES

The total number of citations identified from the combined searches was 745 (PubMed: 34, IDIS: 19, EMBASE: 691, additional PubMed: 1); with the exclusion of overlaps, the number of unique citations was 715. Mini-Sentinel collaborators provided no additional reports of validation studies.

Of the 715 abstracts reviewed, we accepted eight for full-text review. Agreement between the two reviewers in regard to acceptance/rejection status of an abstract for full-text review was high (Cohen's kappa = 0.77).

Of the eight full-text articles reviewed, four were determined to fulfill all inclusion criteria;[7, 9-11] perfect agreement between reviewers on inclusion versus exclusion of full-text articles reviewed was achieved. Two of the included articles reported results from the same study,[10, 11] leaving the number of unique studies fulfilling inclusion criteria at three. Note that two papers reporting results from the same study [10, 11] used a subset of the data used in a later study.[9] None of the excluded studies contained validation estimates,[12-15] although two contained algorithms with no validation.[12, 14]

SUMMARY OF ALGORITHMS

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. SUMMARY OF ALGORITHMS
  7. DISCUSSION
  8. CONCLUSIONS
  9. CONFLICT OF INTEREST
  10. ACKNOWLEDGEMENTS
  11. REFERENCES

We identified four reports (albeit three unique studies) fulfilling all inclusion criteria,[7, 9-11] and another two with algorithms but with no validation.[12, 14] All studies showed consistency in defining EM and related conditions using International Classification of Diseases-Adapted, Eighth Modification (ICDA-8) code 695.1[7] or International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code 695.1.[9-12, 14] Details of the four studies with algorithm validation are presented in Table 1.

Table 1. Erythema multiforme coding algorithms and positive predictive values (PPV) of citations with validation
CitationStudy population and study periodDescription of outcome studiedAlgorithmValidation/adjudication procedure and operational definitionValidation statistics
  1. EM, erythema multiforme; HMO, Health Maintenance Organization; SJS, Stevens–Johnson syndrome; SSSS, staphylococcus scalded-skin syndrome; TEN, toxic epidermal necrolysis; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification.

Chan et al. 1990[7]HMO administrative and claims data from Group Health Cooperative (GHC) of Puget Sound, Seattle, WA, with hospital discharge diagnosis of EM (n = 61), 1972–1986.Incidence of EM, SJS, and TEN requiring hospitalization.Diagnosis of ICDA-8: 695.1 (erythema multiforme). ICDA-8 695.1 should, in principle, include all hospitalized cases of EM, SJS, and TEN.Review of discharge summaries, hospital records, and outpatient charts via criteria for diagnosis as described in paper.PPV (without SSSS) = 59.6%.
Strom 2001[9]Computerized On-Line Medicaid Pharmaceutical Analysis and Surveillance System (COMPASS), a computerized database consisting of Medicaid administrative and claims patients from the states of Michigan, Minnesota, Florida, Missouri, and Nebraska with an inpatient diagnosis of EM and medical charts available for review (N = 167), 1980–1985.Assessed the validity of the ICD-9-CM 695.1 code to ascertain SJS.ICD-9-CM 695.1 for SJS and EM.Medical record review.PPV (without SSSS) = 54.0%.
Strom et al. 1991[10, 11]COMPASS, with an inpatient diagnosis of EM and medical charts available for review (N = 128), 1980–1984.Determined the incidence of SJS.ICD-9-CM 695.1 for SJS and EM.Medical record review.PPV (without SSSS) = 53.7%.
Of the 273 potential cases from which the medical records were obtained, majority of the cases were from Michigan (67%) and aged 0–19 years (60.4%); males comprised 39%, and 55% resided in urban areas.[11]

Through September 2008, ICD-9-CM code 695.1 incorporated the EM conditions of erythema iris and herpes iris, SJS, TEN/Lyell's syndrome, and staphylococcus scalded-skin syndrome (SSSS).[16] Because this code is multi-diagnostic, reporting a positive predictive value (PPV) statistic for each unique disease under its umbrella would be deceptive. We therefore, in Table 1, report the PPV of the combination of diseases of study interest: EM, SJS, and TEN. SSSS, which was responsible for 15%–16% of this code, was excluded from these PPV calculations because it is no longer incorporated into ICD-9-CM code 695.1. After excluding SSSS, between 53% and 60% of ICD-9-CM code 695.1 reports were validated cases of EM, SJS, or TEN.

DISCUSSION

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. SUMMARY OF ALGORITHMS
  7. DISCUSSION
  8. CONCLUSIONS
  9. CONFLICT OF INTEREST
  10. ACKNOWLEDGEMENTS
  11. REFERENCES

The literature review identified only ICD-9-CM code 695.1 as a coding algorithm used for EM major/minor/not otherwise specified, SJS, or TEN. Chan et al.[7] used ICDA-8 code 695.1, and Strom[9] and Strom et al.[10, 11] used ICD-9-CM code 695.1. These studies consistently found that slightly more than half of patients with this diagnostic code had EM, SJS, or TEN. Other skin diseases and truly misclassified diagnoses contributed approximately 35%–40% and 6% to the total, respectively, illustrating that clinical experts frequently disagree with the discharge diagnosis of EM/SJS/TEN (ICD-9-CM 695.1). Such disagreement may also be the result of the inherent difficulties in validating a diagnostic code based on a review of source documents.

Increased understanding of the diseases covered by ICD-9-CM code 695.1 has occurred since the publication of the articles included in this review. For example, TEN and SJS differ in etiology from SSSS: medications are a common cause of TEN/SJS, whereas staphylococcal infection is typically the causative agent for SSSS.[17] This increased knowledge led to SSSS being recoded to a separate ICD-9-CM code as well as the inclusion of a fifth digit to ICD-9-CM code 695.1. This fifth digit provides specific codes for each disease, excluding SSSS, previously contained in the four-digit, multi-diagnostic code (ICD-9-CM code 695.1). These updates were effective as of October 2008.

In an attempt to mirror the current diagnostic coding of EM, SJS, and TEN, we excluded SSSS from the PPV calculations. Despite this decision, it is likely that we are underestimating the true PPV because we assumed that all other common misdiagnoses would remain within ICD-9-CM code 695.1; in reality, a proportion of these would likely be transferred to the new code representing SSSS.

CONCLUSIONS

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. SUMMARY OF ALGORITHMS
  7. DISCUSSION
  8. CONCLUSIONS
  9. CONFLICT OF INTEREST
  10. ACKNOWLEDGEMENTS
  11. REFERENCES

The studies identified in our search showed consistency in defining EM and related conditions. We found that clinical experts frequently disagreed with the discharge diagnoses of EM, SJS, and TEN. Our search, however, revealed limited literature that provided validated algorithms and validation estimates. Furthermore, the most recent data identified in our search were 25 years old and therefore did not incorporate the October 2008 diagnostic coding changes. Because the identified literature supplying both algorithms and validation estimates is dated, the Mini-Sentinel goal of refining safety signals focused on these specific diseases is compromised. Updated research should be conducted on development and validation of coding algorithms for EM, SJS, and TEN.

KEY POINTS

  • There is limited literature focusing on erythema multiforme and related conditions that provides administrative and claims data-based coding algorithms and validation estimates.
  • The available literature is dated and does not incorporate recent changes to the diagnostic coding of erythema multiforme and related conditions that became effective on 1 October 2008.
  • Additional research is needed regarding the use of administrative and claims data-based coding algorithms to identify erythema multiforme and related conditions.

CONFLICT OF INTEREST

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. SUMMARY OF ALGORITHMS
  7. DISCUSSION
  8. CONCLUSIONS
  9. CONFLICT OF INTEREST
  10. ACKNOWLEDGEMENTS
  11. REFERENCES

The authors declare no conflict of interest. This is not product-specific or privately funded research. The views expressed in this document do not necessarily reflect the official policies of the Department of Health and Human Services, nor does mention of trade names, commercial practices, or organizations imply endorsement by the US government.

ACKNOWLEDGEMENTS

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. SUMMARY OF ALGORITHMS
  7. DISCUSSION
  8. CONCLUSIONS
  9. CONFLICT OF INTEREST
  10. ACKNOWLEDGEMENTS
  11. REFERENCES

This work was supported by the Food and Drug Administration (FDA) through Department of Health and Human Services (HHS) Contract Number HHSF223200910006I.

REFERENCES

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. SUMMARY OF ALGORITHMS
  7. DISCUSSION
  8. CONCLUSIONS
  9. CONFLICT OF INTEREST
  10. ACKNOWLEDGEMENTS
  11. REFERENCES