A New Diagnosis Grouping System for Child Emergency Department Visits

Authors

  • Evaline A. Alessandrini MD, MSCE,

    1. From the Department of Pediatrics (EAA, ERA) and the Department of Emergency Medicine (EAA), University of Pennsylvania School of Medicine and Emergency Medicine (EAA, ERA), The Children’s Hospital of Philadelphia, PA; the Department of Pediatrics, George Washington University School of Medicine (JMC), and the Division of Emergency Medicine, Children’s National Medical Center (JMC), Washington, DC; the Department of Medicine, Division of General Internal Medicine (JAS), University of Pennsylvania School of Medicine, Philadelphia, PA; and the Department of Pediatrics, Medical College of Wisconsin and Section of Emergency Medicine, Children’s Hospital of Wisconsin, Milwaukee, WI; in collaboration with the Pediatric Emergency Care Applied Research Network. Dr. Alessandrini is currently with the Department of Pediatrics, University of Cincinnati College of Medicine, Emergency Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH.
    Search for more papers by this author
  • Elizabeth R. Alpern MD, MSCE,

    1. From the Department of Pediatrics (EAA, ERA) and the Department of Emergency Medicine (EAA), University of Pennsylvania School of Medicine and Emergency Medicine (EAA, ERA), The Children’s Hospital of Philadelphia, PA; the Department of Pediatrics, George Washington University School of Medicine (JMC), and the Division of Emergency Medicine, Children’s National Medical Center (JMC), Washington, DC; the Department of Medicine, Division of General Internal Medicine (JAS), University of Pennsylvania School of Medicine, Philadelphia, PA; and the Department of Pediatrics, Medical College of Wisconsin and Section of Emergency Medicine, Children’s Hospital of Wisconsin, Milwaukee, WI; in collaboration with the Pediatric Emergency Care Applied Research Network. Dr. Alessandrini is currently with the Department of Pediatrics, University of Cincinnati College of Medicine, Emergency Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH.
    Search for more papers by this author
  • James M. Chamberlain MD,

    1. From the Department of Pediatrics (EAA, ERA) and the Department of Emergency Medicine (EAA), University of Pennsylvania School of Medicine and Emergency Medicine (EAA, ERA), The Children’s Hospital of Philadelphia, PA; the Department of Pediatrics, George Washington University School of Medicine (JMC), and the Division of Emergency Medicine, Children’s National Medical Center (JMC), Washington, DC; the Department of Medicine, Division of General Internal Medicine (JAS), University of Pennsylvania School of Medicine, Philadelphia, PA; and the Department of Pediatrics, Medical College of Wisconsin and Section of Emergency Medicine, Children’s Hospital of Wisconsin, Milwaukee, WI; in collaboration with the Pediatric Emergency Care Applied Research Network. Dr. Alessandrini is currently with the Department of Pediatrics, University of Cincinnati College of Medicine, Emergency Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH.
    Search for more papers by this author
  • Judy A. Shea PhD,

    1. From the Department of Pediatrics (EAA, ERA) and the Department of Emergency Medicine (EAA), University of Pennsylvania School of Medicine and Emergency Medicine (EAA, ERA), The Children’s Hospital of Philadelphia, PA; the Department of Pediatrics, George Washington University School of Medicine (JMC), and the Division of Emergency Medicine, Children’s National Medical Center (JMC), Washington, DC; the Department of Medicine, Division of General Internal Medicine (JAS), University of Pennsylvania School of Medicine, Philadelphia, PA; and the Department of Pediatrics, Medical College of Wisconsin and Section of Emergency Medicine, Children’s Hospital of Wisconsin, Milwaukee, WI; in collaboration with the Pediatric Emergency Care Applied Research Network. Dr. Alessandrini is currently with the Department of Pediatrics, University of Cincinnati College of Medicine, Emergency Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH.
    Search for more papers by this author
  • Marc H. Gorelick MD, MSCE

    1. From the Department of Pediatrics (EAA, ERA) and the Department of Emergency Medicine (EAA), University of Pennsylvania School of Medicine and Emergency Medicine (EAA, ERA), The Children’s Hospital of Philadelphia, PA; the Department of Pediatrics, George Washington University School of Medicine (JMC), and the Division of Emergency Medicine, Children’s National Medical Center (JMC), Washington, DC; the Department of Medicine, Division of General Internal Medicine (JAS), University of Pennsylvania School of Medicine, Philadelphia, PA; and the Department of Pediatrics, Medical College of Wisconsin and Section of Emergency Medicine, Children’s Hospital of Wisconsin, Milwaukee, WI; in collaboration with the Pediatric Emergency Care Applied Research Network. Dr. Alessandrini is currently with the Department of Pediatrics, University of Cincinnati College of Medicine, Emergency Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH.
    Search for more papers by this author

  • Presented at the Pediatric Academic Societies meetings, Washington, DC, May 2005, and San Francisco, CA, May 2006.

  • Funding was obtained through a HRSA Emergency Medical Services for Children Targeted Issues Grant (Grant H34MC02547).

Address for correspondence: Evaline A. Alessandrini, MD, MSCE; e-mail: evaline.alessandrini@cchmc.org. Reprints will not be available.

Abstract

Objectives:  A clinically sensible system of grouping diseases is needed for describing pediatric emergency diagnoses for research and reporting. This project aimed to create an International Classification of Diseases (ICD)-based diagnosis grouping system (DGS) for child emergency department (ED) visits that is 1) clinically sensible with regard to how diagnoses are grouped and 2) comprehensive in accounting for nearly all diagnoses (>95%). The second objective was to assess the construct validity of the DGS by examining variation in the frequency of targeted groups of diagnoses within the concepts of season, age, sex, and hospital type.

Methods:  A panel of general and pediatric emergency physicians used the nominal group technique and Delphi surveys to create the DGS. The primary data source used to develop the DGS was the Pediatric Emergency Care Applied Research Network (PECARN) Core Data Project (PCDP).

Results:  A total of 3,041 ICD-9 codes, accounting for 98.9% of all diagnoses in the PCDP, served as the basis for creation of the DGS. The expert panel developed a DGS framework representing a clinical approach to the diagnosis and treatment of pediatric emergency patients. The resulting DGS has 21 major groups and 77 subgroups and accounts for 96.5% to 99% of diagnoses when applied to three external data sets. Variations in the frequency of targeted groups of diagnoses related to seasonality, age, sex, and site of care confirm construct validity.

Conclusions:  The DGS offers a clinically sensible method for describing pediatric ED visits by grouping ICD-9 codes in a consensus-derived classification scheme. This system may be used for research, reporting, needs assessment, and resource planning.

ACADEMIC EMERGENCY MEDICINE 2010; 17:204–213 © 2010 by the Society for Academic Emergency Medicine

Emergency department (ED) visit volumes continue to increase each year in the United States, with nearly a quarter of these visits by children.1 To accurately describe the scope and epidemiology of ED visits,1–3 it is important to have a widely used and informative method of classifying diagnoses. The International Classification of Diseases, Ninth Revision (ICD-9), is a readily available, familiar, and widely utilized system.4 However, the ICD-9 system is cumbersome in that it includes more than 12,000 diagnoses, and its diagnosis categories are not uniformly clinically sensible for pediatric diseases. For example, “diseases of the respiratory system” includes such disparate diagnoses as pharyngitis and asthma, while “diseases of the nervous system and sense organs” contain both seizures and otitis media. These groupings fail to describe diagnoses in a clinically meaningful way for pediatric emergency care, yet they serve as the basis for reporting diagnoses given to children in nationally representative surveys such as the National Hospital Ambulatory Medical Care Survey (NHAMCS) and the Medical Expenditure Panel Survey.1,5 Other tools have been developed specifically for classifying diagnoses. Most of these are based on the ICD-9 system, but have limited value in pediatric emergency care because they were developed principally for adult diagnoses6 or they were developed for pediatric primary care.5,7

Developing a method of classifying and grouping diseases for emergency medical services for children (EMSC) was noted as an area warranting targeted research efforts by the Institute of Medicine and EMSC experts.8,9 A clinically sensible system of grouping diseases is needed for describing diagnoses for both research and reporting and may enable better pattern recognition for disease surveillance. For example, various signs and symptoms such as fever or abdominal pain are often assigned as diagnoses in emergency medicine and are combined within the same ICD-9 diagnosis category (“symptoms, signs, and ill-defined conditions”). A newly developed grouping system might better assign these common diagnoses within separate but more clinically relevant groups. Ideally, an improved classification system could improve discrimination, for example, between pharyngitis and asthma as different conditions of the respiratory system and at the same time act to minimize unimportant variation in diagnosis coding (e.g., grouping acute tonsillitis with acute pharyngitis). In a prior study, investigators demonstrated limited agreement between ED diagnoses abstracted from the medical record and those obtained from administrative billing sources.10 However, ad hoc clustering of clinically similar conditions improved agreement between administrative and abstracted data sources for ED diagnoses. Improved agreement, and more meaningful information, may be possible with a diagnosis grouping system (DGS) developed specifically for pediatric ED diagnoses.

The primary objective of this study was to create a DGS for child ED visits using ICD-9 codes that is 1) clinically sensible with regard to how diagnoses are grouped and 2) comprehensive in accounting for nearly all diagnoses (>95%) found in several ED data sets. Our second objective was to assess the construct validity of the DGS by examining variation in the frequency of targeted groups of diagnoses within the concepts of season, child age, sex, and hospital type.

Methods

Study Design

A panel of emergency physicians used consensus methods, including the nominal group technique and Delphi surveys, to create the categories of the DGS. These consensus methods provide a means of synthesizing information where unanimity of opinion does not exist owing to insufficient information or, conversely, an overload of information.11–13 They assess the extent of agreement (consensus measurement) and resolve disagreement (consensus development). Strengths of these methods include 1) anonymity—dominance is avoided by using private ratings in nominal group and questionnaires in Delphi; 2) iteration—processes occur in “rounds,” allowing individuals to change their opinions; 3) controlled feedback—the distributions of the group’s responses are shown; and 4) statistical group response—judgment is expressed using summary measures of the full group response, giving more information than just a consensus statement.11–13

Diagnoses (ICD-9 codes) from existing emergency medicine data sets were assigned to DGS categories to assess comprehensiveness and construct validity of this new system. The study was reviewed and determined to be exempt or approved by the institutional review boards of the participating investigators’ hospitals and by the university of the central data management and coordinating center of the research network in which this study was conducted.

Study Setting and Population

The study was performed within the Pediatric Emergency Care Applied Research Network (PECARN), a federally funded national network with diverse hospital types, patient populations, and providers.14–16 We convened an expert panel of fourteen general and pediatric emergency physicians. Because panel members were recruited from the four nodes of the PECARN, our panel members included physicians from across the United States, practicing in a variety of hospital settings. Specifically, we had representatives from both academic and nonacademic hospitals; small, medium, and large volume EDs; and urban, suburban, and rural hospitals. There were several ED physician-managers with in-depth experience using the ICD-9 system as it relates to diagnosis coding for emergency medicine.

Study Protocol

Principal Data Source.  The data source used to develop the DGS was the PECARN Core Data Project (PCDP).15 These data were obtained from extant electronic data sets from the 20 EDs within the PECARN with reliable ED discharge diagnosis coding, including more than 750,000 visits from 2002. We included all patients treated in the year 2002 aged birth to the 19th birthday. Data were collected, cleaned, and stored as previously described by the central data management and coordinating center at the University of Utah.15 Because up to 15 ICD-9 codes could be recorded per visit (although 50% of visits had a single diagnosis), more than 1 million diagnoses were included in the 2002 PCDP. After reviewing the distribution of diagnoses, we chose any code that occurred at least 10 times in the PCDP data set, resulting in 3,041 individual ICD-9 codes (diagnosis or V-code) for inclusion in the DGS. E-codes were not included in this system because they modify injury codes (should only accompany an injury diagnosis code and not be listed alone) and other systems exist for describing injury mechanism. Given that these 3,041 codes accounted for 98.9% of all diagnoses given to children in PECARN hospital EDs, we elected not to include the nearly 6,000 additional ICD codes due to a large work load for diagnoses affecting a very limited number of ED visits.

Creating the DGS.  The expert panel met at the first consensus meeting in September 2004 to develop the framework for the DGS. This meeting utilized the nominal group technique and was guided by an expert facilitator. Various schemata were proposed, recorded, and discussed.

After the framework was determined, all participants were asked to record, individually and without discussion, their own lists of potential candidate diagnosis groups (e.g., trauma, child abuse) within the chosen framework. Participants were reminded of their charge to create a system that was driven by clinical sensibility and comprehensiveness. Each individual shared his or her list of important candidate diagnosis groups with the panel, and ideas were recorded so that everyone could see the composite list. This process was repeated until no new potential diagnosis groups were listed by panel members.

Next, participants evaluated each candidate diagnosis group separately and, when necessary, clarified their ideas through discussion. Groups were eliminated, merged, and added as a result of this discussion. The panel voted on the inclusion or elimination of diagnosis groups and on the final grouping system that was created after all groups were finalized.

Assigning ICD-9 Codes Into the DGS.  Two study investigators (EAA and ERA) initially assigned the 3,041 ICD-9 codes from the PCDP into the resultant DGS, placing each code into one and only one diagnosis group (choice of diagnosis group was mutually exclusive). Panel members then used electronic Delphi surveys to rank the appropriateness of the initial assignment of each ICD-9 code within its given diagnosis group from 1 (strongly disagree) to 6 (strongly agree). Each panel member was asked to rate a computer-generated random sample of approximately 600 codes. Codes that the two study investigators felt sensibly fit within a particular diagnosis group (e.g., facial laceration within laceration group) were rated by at least two panel members. Other codes (e.g., syncope, which potentially could have been assigned to a cardiac, neurologic, or another group) were rated by four to six members. Consensus on the placement of any particular code was defined as a code receiving all “5” and “6” ratings.

Diagnosis codes without consensus grouping were sent for a second Delphi survey round to the same reviewers. In this round, reviewers were asked to consider altering their original response after reviewing the distribution of all scores from the first round. For codes that were reviewed by two expert panel members, and on which consensus was not reached in Round 1, an additional two reviewers were added for the second round. Again, consensus on the placement of any particular code was defined as a code receiving all “5” and “6” ratings in this second round.

Finalizing the DGS.  The purpose of the second consensus meeting, held in January 2005, was to assign ICD-9 codes for which consensus was not achieved during the two Delphi survey rounds into appropriate diagnosis groups. Discussion of the remaining ICD-9 codes was organized around several “disagreement themes” that described particular difficulties that the panel had in determining the most appropriate category for certain codes or groups of codes. Solutions proposed by investigators or panel members were adopted if agreed upon by two-thirds of panel members. Recognizing that all standards are arbitrary,17,18 and balancing the need to move the project forward with knowledge that these codes had already been through two rounds of review, the investigative team chose two-thirds rather than a more stringent cutoff.

DGS Comprehensiveness.  After development of the DGS, its comprehensiveness was evaluated by applying it to the following 2002 databases: the NHAMCS ED data set1 and the Connecticut and Wisconsin state ED data sets. The Connecticut Hospital Association (CHA) ChimeData program maintains a proprietary healthcare information system that incorporates statewide clinical, financial, and patient demographic data and is available on a fee-for-service basis (Chime, Inc., http://www.cthosp.org/). Wisconsin ED data are available for public use from the Bureau of Health Information, Division of Health Care Financing, Department of Health and Family Services, State of Wisconsin. Our goal was that the 3,041 codes chosen from the PCDP would account for more than 95% of diagnoses given in these three additional data sets.

Construct Validity.  Construct validity seeks agreement between a theoretical concept and other specific measured concepts or variables.19 In general, the goal is to see if observed scores for a theoretical construct “behave in expected ways.” To understand whether a set of scores or other performance values has construct validity, three steps should be followed. First, the expected theoretical relationships must be specified. In this case, the construct of interest is disease grouping, which is represented by the observed prevalence in diagnosis group categories, “scores” that should be reproducible and behave in expected ways. Second, expected empirical relationships between the measures and representations of the construct and other concepts must be examined. After the development of the DGS, but before validity testing, four study investigators independently rated all diagnosis groups (representations of the theoretical construct) on whether their prevalence would be expected to vary based on the following four concepts for which data were available: season, child age, child sex, and site of care (tertiary care children’s hospitals versus other). Any diagnosis group or construct pair selected by three of four raters was included. Specifically, we assessed convergent construct validity to evaluate the general agreement among investigator opinion that the four hypothesized variables would covary with the diagnosis groups, gathered independently. Third, the empirical evidence must be interpreted in terms of how it clarifies the construct validity of the particular measure being tested (see Data Analysis).19,20 For construct validity testing, all diagnoses from each of the four databases included in the study were combined.

Data Analysis

Results of all Delphi surveys were summarized using simple counts and frequencies. Comprehensiveness of the DGS, when applied to the NHAMCS, Connecticut, and Wisconsin ED data sets, was assessed using proportions of diagnoses captured with 95% confidence intervals (CIs). For the construct validity analysis, we hypothesized for each concept where the prevalence of a given diagnosis group would be highest, for example, in summer or for males. For each diagnosis group for which we hypothesized such an association, we calculated the proportion of observations that were found within the hypothesized highest frequency component (e.g., summer, male), and the proportion observed for the remaining components (e.g., winter-spring-autumn, or female), dividing them to calculate the relative risk. For example, we hypothesized that lacerations would comprise a larger proportion of total diagnoses in the summer months component of the season concept, so we calculated the proportion of visits with a laceration group diagnosis during the summer and divided that by the proportion of visits with laceration diagnoses during the rest of the year. A priori, we considered construct validity to be exhibited if the lower bound of the 95% CI on the relative risk exceeded 1.2 (i.e., at least a 20% relative difference). In this initial exploratory set of analyses, we were interested in examining a broad set of hypothesized relationships, and this 20% increased risk is consistent with the definition of a minimal effect size.21

Results

DGS Framework and Composition

Figure 1 summarizes the work of the expert panel. At the first consensus meeting, the expert panel established a DGS framework representing a clinical approach to the diagnosis and treatment of pediatric emergency patients. This preliminary DGS included 20 major groups. Fifty-two subgroups within the original 20 major groups were also defined to offer further detail.

Figure 1.

 Summary of the process used to develop the DGS. DGS = diagnosis grouping system.

After investigators assigned each of the 3,041 ICD-9 codes to one of the 52 DGS subgroups and panel members rated their agreement with the assignments, consensus was achieved for 82% of codes after two rounds of electronic Delphi surveys. We addressed the remaining 18% of codes organized into 12 disagreement themes. For example, the panel came to consensus that all substance abuse diagnoses would be assigned to the psychiatric diseases group in lieu of the toxicologic emergencies group. Discussion and consensus resulted in refinement of the DGS to 21 major groups and 77 subgroups with concomitant assignment of these diagnosis codes.

The first column of Table 1 depicts the major groups and subgroups of the DGS displayed in alphabetical order by major group. Major groups with the largest number of subgroups include trauma (14 subgroups), gastrointestinal diseases (seven subgroups), and neurologic diseases (six subgroups). Some major groups, such as child abuse and neoplastic diseases, stand alone and do not have subgroups. Because so much of acute care pediatrics involves infectious diseases, infection was incorporated as a subgroup of most DGS major groups. Many major groups also have a subgroup for medical devices. For example, a devices and complications of the nervous system subgroup of the neurologic diseases major group includes diagnoses for devices such as ventriculoperitoneal shunts. Many subgroups represent single important disease states. For example, diabetes is a subgroup of the endocrine/metabolic/nutritional disease major group, and sickle cell disease is a subgroup of hematologic diseases.

Table 1.   The DGS Distribution of Diagnosis Codes Within the DGS in the PCDP, the NHAMCS, Wisconsin ED Data Set, and Connecticut ED Data Set
Major Group and SubgroupNumber ICD-9 CodesPCDPNHAMCSWisconsinConnecticut
  1. All data sets are from calendar year 2002. Numbers represented in bold are totals for the major groups and numbers represented in italics are totals for the subgroups within each major group.

  2. DGS = diagnosis grouping system; ENT = ear, nose, and throat; NHAMCS = National Hospital Ambulatory Medical Care Survey; PCDP = Pediatric Emergency Care Applied Research Network (PECARN) Core Data Project.

Allergic, immunologic, and rheumatologic diseases439,635 (0.9)389,740 (0.9)5,502 (0.8)4,356 (1.0)
Child abuse144,860 (0.4)89,791 (0.2)1,373 (0.2)891 (0.2)
Circulatory and cardiovascular diseases9211,314 (1.0)157,329 (0.4)3,528 (0.5)2,110 (0.5)
 Congenital circulatory and cardiovascular   diseases242,849 (0.3)2,378 (0.0)569 (0.1)335 (0.1)
 Devices and complications of the circulatory   system92,699 (0.2)47,159 (0.1)1,166 (0.2)374 (0.1)
 Dysrhythmias181,727 (0.2)44,044 (0.1)770 (0.1)605 (0.1)
 Other circulatory and cardiovascular diseases414,039 (0.4)63,748 (0.2)1,023 (0.2)796 (0.2)
Eye diseases8017,941 (1.6)629,365 (1.5)11,264 (1.7)5,906 (1.3)
 Infectious diseases of the eye2512,402 (1.1)458,933 (1.1)8,633 (1.3)4,272 (0.9)
 Noninfectious diseases of the eye555,539 (0.5)170,432 (0.4)2,631 (0.4)1,634 (0.4)
Endocrine, metabolic, and nutritional diseases9211,384 (1.0)140,870 (0.3)3,094 (0.5)2,429 (0.5)
 Diabetes mellitus163,407 (0.3)58,167 (0.1)1,205 (0.2)935 (0.2)
 Other endocrine, metabolic, and nutritional   diseases767,977 (0.7)82,703 (0.2)1,889 (0.3)1,494 (0.3)
ENT, dental, and mouth diseases170184,970 (16.5)9,537,861 (22.8)144,167 (22.0)76,598 (16.8)
 Infectious ear disorders2754,828 (4.9)3,245,478 (7.7)54,669 (8.4)27,883 (6.1)
 Infectious dental disorders134,118 (0.4)87,655 (0.2)1,474 (0.2)996 (0.2)
 Infectious mouth and throat disorders2135,090 (3.1)2,347,434 (5.6)30,391 (4.6)16,949 (3.7)
 Infectious nose and sinus disorders,   including URI1865,388 (5.8)3,150,908 (7.5)43,896 (6.7)22,575 (5.0)
 Noninfectious ENT, dental, and mouth diseases9125,546 (2.3)706,386 (1.7)13,737 (2.1)8,195 (1.8)
Fluid and electrolyte disorders1619,859 (1.8)444,756 (1.1)6,684 (1.0)5,017 (1.1)
 Dehydration117,543 (1.6)436,188 (1.0)6,246 (1.0)4,590 (1.0)
 Other fluid and electrolyte disorders152,316 (0.2)8,568 (0.0)438 (0.1)427 (0.1)
Gastrointestinal diseases253131,392 (11.8)3,704,430 (8.8)61,549 (9.4)39,179 (8.6)
 Abdominal pain2227,968 (2.5)825,469 (2.0)17,389 (2.7)11,081 (2.4)
 Appendicitis82,330 (0.2)54,379 (0.1)88 (0.0)985 (0.2)
 Devices and complications of the   gastrointestinal system173,177 (0.3)33,859 (0.1)942 (0.1)423 (0.1)
 Gastroenteritis2540,376 (3.6)1,281,756 (3.1)17,223 (2.6)10,864 (2.4)
 Infectious gastrointestinal diseases19846 (0.1)17,029 (0.0)169 (0.0)327 (0.1)
 Vomiting524,011 (2.1)726,253 (1.7)13,890 (2.1)8,313 (1.8)
 Other gastrointestinal diseases15732,684 (2.9)765,685 (1.8)11,848 (1.8)7,186 (1.6)
Genital and reproductive diseases13014,164 (1.3)638,622 (1.5)11,746 (1.8)6,098 (1.3)
 Infectious genital and reproductive diseases335,066 (0.5)208,111 (0.5)3,897 (0.6)1,327 (0.3)
 Pregnancy402,571 (0.2)221,027 (0.5)3,904 (0.6)2,106 (0.5)
 Other genital and reproductive diseases576,527 (0.6)209,484 (0.5)3,945 (0.6)2,665 (0.6)
Hematologic diseases5912,470 (1.1)164,887 (0.4)2,256 (0.3)1,844 (0.4)
 Sickle cell anemia65,411 (0.5)46,814 (0.1)631 (0.1)565 (0.1)
 Other hematologic diseases537,059 (0.6)118,073 (0.3)1,625 (0.3)1,279 (0.3)
Musculoskeletal and connective tissue diseases14331,178 (2.8)1,077,042 (2.6)15,474 (2.4)10,849 (2.4)
 Chest pain57,293 (0.7)201,701 (0.5)3,143 (0.5)2,229 (0.5)
 Devices and complications of the   musculoskeletal system5256 (0.0)38,653 (0.1)86 (0.0)70 (0.0)
 Infectious musculoskeletal and connective   tissue diseases14388 (0.0)4,274 (0.0)18 (0.0)69 (0.0)
 Musculoskeletal pain2015,571 (1.4)629,067 (1.5)9,287 (1.4)6,366 (1.4)
 Noninfectious musculoskeletal and   connective tissue diseases997,670 (0.7)203,347 (0.5)2,940 (0.5)2,115 (0.5)
Neoplastic diseases (cancer, not benign neoplasms)592,796 (0.3)28,822 (0.1)333 (0.1)394 (0.1)
Neurologic diseases19047,546 (4.3)882,258 (2.1)17,629 (2.7)12,182 (2.7)
 Developmental disorders364,858 (0.4)31,242 (0.1)1,192 (0.2)962 (0.2)
 Devices and complications of the nervous   system62,855 (0.3)2,244 (0.0)477 (0.1)214 (0.0)
 Headache1111,769 (1.1)237,560 (0.6)6,988 (1.1)3,897 (0.9)
 Infectious neurologic diseases191,173 (0.1)30,160 (0.1)62 (0.0)230 (0.1)
 Seizures1814,708 (1.3)355,400 (0.8)4,751 (0.7)3,984 (0.9)
 Other neurologic diseases11012,183 (1.1)225,652 (0.5)4,159 (0.6)2,895 (0.6)
Psychiatric, behavior, and substance abuse15629,863 (2.7)581,066 (1.4)15,232 (2.3)18,616 (4.1)
Respiratory diseases134136,983 (12.3)4,098,560 (9.8)63,659 (9.7)43,413 (9.5)
 Asthma1454,454 (4.9)1,124,888 (2.7)20,449 (3.1)18,161 (4.0)
 Bronchospasm and wheezing27,019 (0.6)150,485 (0.4)4,514 (0.7)1,765 (0.4)
 Devices and complications of the   respiratory system7654 (0.1)7,883 (0.0)168 (0.0)75 (0.0)
 Infectious respiratory diseases3536,964 (3.3)2,052,977 (4.9)24,943 (3.8)14,006 (3.1)
 Other respiratory diseases7637,892 (3.4)762,327 (1.8)13,585 (2.1)9,406 (2.1)
Skin, dermatologic, and soft tissue diseases19855,854 (5.0)1,703,187 (4.1)25,077 (3.8)18,674 (4.1)
 Infectious skin, dermatologic, and   soft tissue diseases7619,815 (1.8)731,169 (1.7)8,619 (1.3)6,261 (1.4)
 Noninfectious skin, dermatologic, and   soft tissue diseases12236,039 (3.2)972,018 (2.3)16,458 (2.5)12,413 (2.7)
Systemic states102135,482 (12.1)4,049,512 (9.7)48,829 (7.5)42,592 (9.4)
 Acute systemic states2121,411 (1.9)245,404 (0.6)4,978 (0.8)3,123 (0.7)
 Bacterial and fungal illnesses214,726 (0.4)137,689 (0.3)706 (0.1)1,121 (0.2)
 Chronic systemic states252,289 (0.2)31,430 (0.1)690 (0.1)366 (0.1)
 Fever155,175 (4.9)1,632,594 (3.9)21,832 (3.3)20,461 (4.5)
 Viral illnesses3451,881 (4.6)2,002,395 (4.8)20,623 (3.2)17,521 (3.9)
Toxicologic emergencies (including environmental)1067,501 (0.7)344,929 (0.8)5,697 (0.9)3,748 (0.8)
Trauma713195,908 (17.5)10,110,593 (24.1)179,451 (27.4)136,318 (30.0)
 Abdominal trauma 24755 (0.1)10,393 (0.0)126 (0.0)220 (0.0)
 Brain and skull trauma6217,602 (1.6)613,556 (1.5)10,352 (1.6)8,055 (1.8)
 Burns (external, of any body part)986,523 (0.6)182,794 (0.4)4,089 (0.6)2,603 (0.6)
 Chest trauma272,391 (0.2)163,915 (0.4)2,370 (0.4)1,654 (0.4)
 Complications of trauma12429 (0.0)5,715 (0.0)155 (0.0)172 (0.0)
 Contusions and abrasions   (external, of any body part)7140,375 (3.6)2,607,491 (6.2)49,433 (7.6)34,070 (7.5)
 Face, dental, mouth and eye trauma599,998 (0.9)400,871 (1.0)7,827 (1.2)4,874 (1.1)
 Fractures and dislocations (extremities)14926,808 (2.4)1,248,754 (3.0)19,379 (3.0)16,601 (3.6)
 Lacerations, amputations, and uninfected   foreign bodies (external)11651,576 (4.6)2,707,882 (6.5)50,294 (7.7)39,791 (8.7)
 Pelvis and external genitalia trauma161,038 (0.1)20,382 (0.0)594 (0.1)339 (0.1)
 Spinal trauma (incl. spinal cord and   vertebrae trauma)11403 (0.0)3,637 (0.0)150 (0.0)141 (0.0)
 Strains and sprains (extremities)4417,454 (1.6)1,626,929 (3.9)27,220 (4.2)21,737 (4.8)
 Other extremity trauma128,389 (0.8)188,295 (0.4)3,648 (0.6)2,364 (0.5)
 Other trauma1212,167 (1.1)329,979 (0.8)3,814 (0.6)3,697 (0.8)
Urinary tract diseases10017,402 (1.6)874,965 (2.1)11,238 (1.7)6,279 (1.4)
 Devices and complications of the   urinary system12294 (0.0)1,761 (0.0)83 (0.0)42 (0.0)
 Infectious urinary tract diseases138,989 (0.8)679,423 (1.6)7,604 (1.2)4,284 (0.9)
 Other noninfectious urinary tract   diseases758,119 (0.7)193,781 (0.5)3,551 (0.5)1,953 (0.4)
Other19126,614 (2.4)824,249 (2.0)14,291 (2.2)12,020 (2.6)
 Screening exams, labs, and   administrative issues7612,545 (1.1)636,792 (1.5)9,043 (1.4)9,119 (2.0)
 Other devices and complications263,230 (0.3)70,057 (0.2)985 (0.2)755 (0.2)
 Other infectious diseases404,824 (0.4)45,302 (0.1)2,238 (0.3)1,118 (0.2)
 Other neonatal disorders363,287 (0.3)25,924 (0.1)1,077 (0.2)428 (0.1)
 Other noninfectious diseases132,728 (0.2)46,174 (0.1)948 (0.1)600 (0.1)
Diagnoses not categorized 13,189 (1.2)1,460,546 (3.5)6,325 (1.0)5,443 (1.2)
TOTAL30411,118,305 (100)41,933,380 (100)654,398 (100)454,956 (100)

The major group systemic states was created to account for diagnoses that are generally multisystem or of varying etiologies. The subgroup acute systemic states includes systemic signs and symptoms (e.g., unspecified hypotension, tachycardia, and syncope). The bacterial and fungal illnesses subgroup includes systemic or multisystem infections such as sepsis, Lyme disease, and malaria. Chronic systemic states includes many chromosomal disorders, as well as diseases such as tuberous sclerosis and congenital anomalies that are not organ-specific. The viral illnesses subgroup includes influenza and varicella, as well as viremia and unspecified viral illness.

Within the “other” major group, the other infectious diseases subgroup includes bacterial, viral, and fungal infections of unspecified site. Other neonatal disorders include diagnoses primarily related to prematurity and low birth weight. The other noninfectious diseases subgroup diagnoses relate to personal and family history of other diseases (V-codes).

Table 1 depicts the frequency distribution of diagnoses within the major groups and subgroups of the DGS within the PCDP data and the three external data sets. For each data set, ear, nose and throat (ENT), dental and mouth diseases, and trauma are the largest major groups, and the five largest major groups account for approximately 70% of all visit diagnoses. The distribution of the 20 most common subgroups is depicted in Figure 2.

Figure 2.

 Percentage of diagnoses in the 20 most common DGS subgroups for PCDP, the NHAMCS, the Wisconsin state data set, and the Connecticut state data set. DGS = diagnosis grouping system; ENT = ear, nose, and throat; NHAMCS = National Hospital Ambulatory Medical Care Survey; PCDP = Pediatric Emergency Care Applied Research Network (PECARN) Core Data Project; URI = upper respiratory infection.

Table 1 also shows the number of ICD-9 codes within each major group and subgroup. Some subgroups include only a small number of ICD-9 codes, yet these codes account for a large proportion of diagnoses within a given data set. For example, the fever subgroup includes only one code, but accounts for nearly 5% of diagnoses in the PCDP and Connecticut data sets. Some subgroups, such as asthma, contain a moderate number of ICD-9 codes (14), and still account for a large proportion of diagnoses. On the other hand, some subgroups such as infectious eye diseases contain a large number of ICD-9 codes (76), but account for a very small proportion of diagnoses overall.

After each of the 3,041 ICD codes was manually assigned to its final DGS group by the investigators and expert panel, we developed programming routines in both Microsoft Excel (Microsoft Corp., Redmond, WA) and SAS (SAS Institute, Cary, NC) to perform this grouping electronically. ICD-9 code diagnosis fields from any electronic source can be run through one of these programs for grouping results. These programs may be accessed at http://www.pecarn.org/tools.

DGS Comprehensiveness

The DGS proved to be comprehensive when applied to external data sets. We used diagnosis codes from the PCDP that represented 98.9% of all diagnoses in the data set to derive the DGS. After applying the DGS to other data sets, we found that the DGS remained comprehensive. Only 3.48% (95% CI = 3.48% to 3.49%) of diagnoses in NHAMCS were not accounted for by the DGS. In the Connecticut and Wisconsin state ED databases, only 1.20% (95% CI = 1.16% to 1.23%) and 0.97% (95% CI = 0.94% to 0.99%), respectively, of diagnoses were not categorized by the DGS (see Table 1, “Diagnoses not categorized”).

DGS Construct Analysis

For construct validity analyses, more than 4 million diagnoses from the four 2002 data sources were combined and mapped to DGS subgroups. Table 2 depicts the results of the construct validity analyses for the 19 groups and subgroups hypothesized to have an association across our four concepts. Increases in subgroup diagnosis frequency (constructs) were demonstrated as hypothesized for all concepts except season. In the seasonal analysis, asthma and wheezing/bronchospasm subgroups occurred less commonly in the fall, contrary to what was hypothesized. Additionally, although subgroups including fracture/dislocation and laceration occurred more commonly in summer months, the strain/sprain subgroup did not demonstrate the hypothesized association with the summer season.

Table 2.   Construct Validity
Concept ComponentHypothesized Highest Frequency ComponentPercentage of Highest Frequency ComponentPercentage of Other ComponentsRelative Risk95% CI
  1. *Did not meet construct validity criterion (relative risk of at least 1.2).

Age
 Dehydration<1 yr and 1–4 yr1.220.911.33(1.33–1.34)
 Infectious genital/reproductive diseases>12 yr1.810.1810.2(10.1–10.3)
 Headache>12 yr1.460.295.05(5.01–5.09)
 Psychiatric/behavior/substance abuse>12 yr2.500.733.41(3.40–3.43)
 Toxicologic emergencies1–4 yr and >12 yr1.480.602.47(2.46–2.49)
Site of care
 Congenital circulatory/cardiovascular diseasesTertiary care0.310.074.20(3.86–4.58)
 Chronic systemic statesTertiary care0.230.082.79(2.56–3.04)
 Sickle cell anemiaTertiary care0.560.115.06(4.72–5.42)
 Neoplastic diseasesTertiary care0.290.055.50(4.98–6.08)
 Devices and complicationsTertiary care1.420.463.08(2.97–3.19)
Season
 DehydrationJan–March1.440.851.69(1.68–1.70)
 GastroenteritisJan–March3.862.631.47(1.46–1.47)
 AsthmaOct–Dec2.672.770.96(0.96–0.97)*
 Bronchospasm/wheezingOct–Dec0.310.390.79(0.78–0.79)*
 Fracture/dislocationJuly–Sept3.482.751.27(1.26–1.27)
 LacerationJuly–Sept8.825.831.51(1.51–1.52)
 Strain/sprainJuly–Sept3.483.191.09(1.08–1.09)*
Sex
 Infectious genital/reproductive diseasesFemale0.990.224.51(4.47–4.56)
 Infectious urinary tract diseasesFemale2.800.456.24(6.20–6.28)

Discussion

We successfully created a clinically sensible DGS using expert consensus that is based upon ICD-9 codes commonly used in pediatric emergency care. The DGS is comprehensive, allowing for classification of greater than 96% of diagnoses found in external data sets within 21 major groups and 77 subgroups. Finally, we demonstrated construct validity of 19 diagnosis groups/subgroups using the four concepts of age, sex, seasonality, and site of care.

A system for grouping child ED diagnoses has important implications for EMSC. Using the system, conditions can be grouped easily and efficiently to better understand patterns of disease. Planning research, reporting disease epidemiology, and measuring health system utilization all rely on a system that allows for clinically sensible descriptions of ED visits. The DGS may allow individuals to better evaluate the burden of a common group of diseases within or between sites, by season, by time of day, or by provider. The DGS may allow for planning for staffing and resources as well as meeting educational needs of trainees by understanding exposure to different diagnoses.

Ideally, our grouping system has improved discrimination between disparate diseases, while minimizing unimportant differences compared to groupings in the ICD system. For example, pharyngitis and asthma are not grouped together in the DGS as they are in the ICD respiratory category. We have minimized unimportant variation in diagnosis coding by grouping acute tonsillitis with acute pharyngitis, for example, diagnoses with different ICD codes. Compared with prior work showing limited agreement in ED diagnoses derived from existing data sources,10 improved agreement and better discrimination may now be possible with a DGS developed specifically for child diagnoses commonly treated in the ED.

Our expert panel developed a DGS framework to capture the critical diagnostic thinking of physicians and reflect consultation and resource utilization in the ED. When compared to the ICD-9 organ-based grouping system, we propose that our system provides improved clinical sensibility in grouping pediatric ED diagnoses. For example, in the ICD-9 system, diseases of the circulatory system include supraventricular tachycardia (SVT), nontraumatic subdural hemorrhage, and hemorrhoids. In the DGS, SVT is in the dysrhythmia subgroup, nontraumatic subdural hemorrhage is in the neurologic diseases group, and hemorrhoids is in the gastrointestinal diseases group.

The clinical practice of emergency medicine is driven by chief complaint, and the short therapeutic window inherent in emergency care may not allow for a specific, final determination of diagnosis by the time of ED disposition. Therefore, nearly one-quarter of ED ICD-9 diagnosis codes are for nonspecific signs and symptoms such as fever or abdominal pain.1 The DGS assigns these diagnoses to pertinent major groups or subgroups instead of a single, less useful, category as is done in the ICD system.4

A major challenge to an organ-based classification system is multisystem diseases such as influenza, sepsis, and trauma. Therefore, the DGS includes a systemic states major group with subgroups that allow for differentiation by etiology and chronicity. Similarly, the DGS trauma major group contains subgroups to allow differentiation by body area (e.g., abdominal trauma subgroup), injury pattern (e.g., contusion and abrasion), or complication.

Infectious etiologies of disease are very common in pediatrics; therefore, the DGS allows for discrimination of these processes. Eleven of the 21 major groupings have one or more subgroups accounting for infectious diseases. This allows for segregation of infectious diseases from the rest of the diagnoses within that group, and also for concentration of overall infectious etiologies, by combining infectious subgroups across major groups. In addition to infectious subgroups, the DGS has seven devices and complications subgroups within differing major groups. The ability to identify and group these diagnoses within and across major groups has become increasingly important as we seek to improve our care for ever-increasing numbers of children with special health care needs.

The DGS also exhibited strong construct validity using the concepts of age, sex, and site of care. The DGS demonstrated less robust construct validity with respect to the concept of season for diagnosis groups hypothesized to occur most commonly in the summer months (certain trauma diagnosis groups) and fall months (asthma diagnosis group). We believe that this is likely due to the fact that season was defined by quarter (summer = quarter 3 or July, August, September), in lieu of calendar months (summer = June, July, and August). To maintain patient confidentiality, many of our data sets did not report the date of the ED visit, but rather the quarter in which it occurred, limiting our ability to fully apply the concept of seasonality in these analyses.

To our knowledge, there are no other DGSs for pediatric emergency care. Other systems have been developed, but are not well-suited for pediatric emergency diagnoses. For example, diagnosis groupers for pediatric primary care focus on well-child visits and screening and preventive services;5,7 these are not relevant to pediatric emergency care. The Clinical Classifications Software (CCS) was developed as part of the Healthcare Cost and Utilization Project (HCUP), sponsored by the Agency for Healthcare Research and Quality.6 Preliminary comparisons of the DGS and CCS were performed by applying both to the 2002 NHAMCS data set (O'Reilly et al., unpublished data). Whereas the DGS has 77 different subgroups, the CCS has 259 groups and uses all ICD codes. Using the DGS, a total of only 2.1% of all NHAMCS diagnoses were not classified. Using the CCS, 1.5% of all diagnoses fall into a not classified group even though the CCS includes four times the number of ICD-9 codes than the DGS. Furthermore, 74 different CCS groups are completely unpopulated with NHAMCS pediatric emergency diagnoses, and 169 groups contain <0.1% of all NHAMCS diagnoses. Whereas pregnancy and malignancy each have one subgroup in the DGS, in the CCS there are 37 and 21 different groups, respectively. These comparisons demonstrate how the DGS accounts for the vast majority of pediatric emergency medicine diagnoses with relative parsimony and clinical sensibility.

Limitations

The reliability of ICD-9 codes, particularly in ambulatory settings, is unknown.10,22–24 We believe that our system has minimized unimportant coding variation and may be a first step in overcoming issues with coding reliability, but further study is needed to confirm this. Second, while the DGS is comprehensive and accounts for almost all diagnoses in our data sets, we did not include every ICD-9 code. A future goal is to include each ICD code in the DGS, but given the infrequent occurrence of these codes in our data sets (less than 0.001% of all diagnoses), we do not believe that the inclusion of further codes would result in significant changes in distribution of diagnoses within our major and subgroups. In addition, the DGS will need to be updated with changes to the ICD-9 system. Finally, the DGS may be considered too general for every scenario in which diagnosis grouping is needed. For example, while having a single diagnosis group for neoplastic diseases fits a clinical approach to the diagnosis and treatment of emergency patients (indwelling catheters, neutropenia, sepsis, need for blood products), this approach may be overly simplistic for other needs. However, disease-specific diagnosis groups are available for many conditions, including asthma and appendicitis. Over time and with increasing use, modifications to the system may be made to fit more users’ needs.

Conclusions

The diagnosis grouping system offers a clinically sensible method for describing pediatric ED visits by grouping ICD-9 codes in a consensus-derived classification scheme. The DGS is comprehensive and demonstrates construct validity. This system is available in the public domain (accessed at http://www.pecarn.org/tools) and may be used for research, reporting, needs assessment, and resource planning.

Ancillary