Using Computer-Extracted Data from Electronic Health Records to Measure the Quality of Adolescent Well-Care
Article first published online: 29 JAN 2014
© Health Research and Educational Trust
Health Services Research
Volume 49, Issue 4, pages 1226–1248, August 2014
How to Cite
Gardner, W., Morton, S., Byron, S. C., Tinoco, A., Canan, B. D., Leonhart, K., Kong, V. and Scholle, S. H. (2014), Using Computer-Extracted Data from Electronic Health Records to Measure the Quality of Adolescent Well-Care. Health Services Research, 49: 1226–1248. doi: 10.1111/1475-6773.12159
- Issue published online: 23 JUL 2014
- Article first published online: 29 JAN 2014
- Agency for Healthcare Research and Quality. Grant Number: U18HS020503
- Quality measurement;
- pediatric well-care;
- electronic health records
To determine whether quality measures based on computer-extracted EHR data can reproduce findings based on data manually extracted by reviewers.
We studied 12 measures of care indicated for adolescent well-care visits for 597 patients in three pediatric health systems.
Data Collection/Extraction Methods
Manual reviewers collected quality data from the EHR. Site personnel programmed their EHR systems to extract the same data from structured fields in the EHR according to national health IT standards.
Overall performance measured via computer-extracted data was 21.9 percent, compared with 53.2 percent for manual data. Agreement measures were high for immunizations. Otherwise, agreement between computer extraction and manual review was modest (Kappa = 0.36) because computer-extracted data frequently missed care events (sensitivity = 39.5 percent). Measure validity varied by health care domain and setting. A limitation of our findings is that we studied only three domains and three sites.
The accuracy of computer-extracted EHR quality reporting depends on the use of structured data fields, with the highest agreement found for measures and in the setting that had the greatest concentration of structured fields. We need to improve documentation of care, data extraction, and adaptation of EHR systems to practice workflow.