General Features of NC DETECT
This study demonstrates that a statewide near-real-time ED database (NC DETECT), which uses secondary data, has face and content validity and summary descriptive measures that are consistent with a trusted, mature national sample survey (NHAMCS) that estimates similar summary descriptive measures about U.S. EDs. Despite the differences in data sources, collection, abstraction, and validation, the summary measures are consistent. This is to be expected from well-designed systems, even with data obtained from a mixture of clinical and administrative sources.9
While a head-to-head comparison of both systems is not possible, the fact that NC DETECT has face and content validity with NHAMCS, and that the systems’ summary measures are consistent with each other, provides evidence that a system such as NC DETECT works and provides data of reasonable accuracy and precision.
North Carolina DETECT data represent nearly all ED visits in North Carolina in 2006, and data for 2008, with 98% of EDs reporting, will include over 4,000,000 ED visits.10 NC DETECT therefore provides the closest approximation of population-based information available about the burden of acute illness and injury for the state of NC.
While NC DETECT and NHAMCS are consistent with each other, they are by no means identical. Both NC DETECT and NHAMCS provide information about ED visits, but with different goals. NHAMCS uses a sampling scheme to produce a national snapshot of patient, hospital, and visit characteristics and provides retrospective trends over time of national ED utilization. The NHAMCS survey is confidential; individual hospitals are not identifiable and data cannot be obtained for a specific state or region.11 Statewide and regional rates and proportions may vary from national estimates, and the burden of specific types of diseases and injury, and the vulnerable populations, would be expected to vary from state to state.
A system like NC DETECT, based on near-real-time data, provides immediately available and actionable results in a more timely manner than survey-based estimates. Aggregations need to be done periodically to be useful and, depending on the measures assessed, NC DETECT can, and does, provide information on a daily, monthly, quarterly, and yearly basis.12
Because NC DETECT receives data from both hospital administrative and clinical systems and provides standardized mapping of data elements from all institutions, its method is generalizable to other state or multihospital systems. A system like NC DETECT receives data that are generated electronically as part of normal hospital operations and is scalable and interoperable.
We compared 2006 NC DETECT data with 2005 NHAMCS data because the latter data set was first publicly available in June 2007. It would not be possible to do a timely comparison between NC DETECT and NHAMCS because of the time lag of publication of NHAMCS. We compared 2006 NC DETECT data because an average of 79% of North Carolina ED visits were captured data in 2006, and 2006 was the first year that NC DETECT was able to achieve reasonably comprehensive data reporting.
NC DETECT rate estimates were scaled upward to approximate data from nonreporting hospitals (see Methods). Assumptions for determining the rate denominator, i.e., the population estimate, differ for NC DETECT and NHAMCS. For the NC population, NC DETECT uses North Carolina demographic data,6 which are the midyear estimates of population including institutionalized and military populations. To calculate visit rates by age and gender, NHAMCS uses an estimate of the civilian noninstitutionalized population. Prisoners, the active military, and nursing home residents are excluded.13,14 For the numerator of rate calculations, NC DETECT uses all reported ED visits without exclusions, scaled upward to account for records from nonreporting hospitals. NHAMCS excludes records when more than half of data elements are empty.15
In Table 4, “Distribution of Injury-related Visits, NC DETECT vs. NHAMCS,” the injury definition varies between the two data sets (Table 1). For NC DETECT, 86.7% of visits with an injury diagnosis code 800–999 contain an E-code; for NHAMCS, 82.2%. Regardless of definition, the proportion of injury visits identified through NC DETECT in North Carolina is lower than NHAMCS. It should be noted that NHAMCS data collection identifies a much larger proportion of ED visits as injuries (25% in NC DETECT, 36.4% in NHAMCS) when the definition of injury-related visit is expanded16,17 (Tables 1 and 4). Reasons for the differences in injury-related ED visits are unknown, but could be due to coding or classification error, empty data fields, or fewer injuries resulting in ED visits in North Carolina than in the United States as a whole. This requires further investigation.
Some data element comparisons cannot be made between NC DETECT and NHAMCS because of specific data elements not collected in one of the systems, inherent data definition or collection differences, or lack of standardization of data elements between systems. The most important of these are “reason for visit” or chief complaint (CC), triage acuity, CC and diagnosis clusters, and race/ethnicity. The definition of CC has not been standardized, and CC terminology has not been adopted by standards organizations; thus, individual EDs develop their own methods for recording CC. CC is most often recorded as free text or with local terminology.18 NHAMCS calls this element “reason for visit” (Data Supplement S7, available as supporting information in the online version of this paper). CC data are then classified according to the “reason for visit classification”19 and organized into clusters that are unique to NHAMCS. While some processing and standardizing of terms is done for CC within NC DETECT for purposes of assigning syndrome classifications, no aggregation of CC data into clusters is currently performed. “Immediacy with which patient should be seen” is categorized by NHAMCS as “immediate,”“emergent” (1–14 minutes), “urgent” (15–60 minutes), “semiurgent” (61–120 minutes), or “nonurgent” (121 minutes–24 hours). The immediate category was first used in the 2005 Emergency Department Summary.2 NC DETECT collects but does not use data on triage acuity because there is no standardized triage system used across all EDs in North Carolina nor is there a valid method for aggregating triage acuity measures from self-defined three-, four-, or five-level categorizations. NHAMCS also determines rates of ED utilization by race/ethnicity. Race and ethnicity are not required data elements for the legislated collection of ED data in NC DETECT. Previous attempts to collect and use race and ethnicity data in North Carolina revealed that most EDs categorize race or ethnicity by self-report or administrative assignment. In addition, race and ethnicity data are often not captured and, when captured, are coupled at the hospital level and cannot be separated, making DEEDS compliance impossible. The accuracy of race and ethnicity data extracted directly from hospital information systems in North Carolina cannot be validated, and it is also impossible to determine if federal standards are applied to the reporting of these data elements in administrative databases. As a result, no effort was made to include them in the legislated mandate for the collection of ED data in North Carolina. See Data Supplement S8 (available as supporting information in the online version of this paper) for a complete list of data elements collected in NC DETECT ED data.