Address correspondence to Alan N. West, Ph.D., VA Outcomes Group REAP, VA Medical Center, White River Junction, VT 05009. William B. Weeks, M.D., MBA, is with the VA Outcomes Group REAP, VA National Quality Scholars Fellowship Program, & Field Office of VA National Center for Patient Safety, VA Medical Center, White River Junction, VT, and the Departments of Psychiatry and Community & Family Medicine, Dartmouth Medical School, Hanover, NH. James P. Bagian, M.D., PE, is with the VA National Center for Patient Safety, Ann Arbor, MI.
Rare Adverse Medical Events in VA Inpatient Care: Reliability Limits to Using Patient Safety Indicators as Performance Measures
Article first published online: 17 JUL 2007
Health Services Research
Volume 43, Issue 1p1, pages 249–266, February 2008
How to Cite
West, A. N., Weeks, W. B. and Bagian, J. P. (2008), Rare Adverse Medical Events in VA Inpatient Care: Reliability Limits to Using Patient Safety Indicators as Performance Measures. Health Services Research, 43: 249–266. doi: 10.1111/j.1475-6773.2007.00760.x
- Issue published online: 17 JUL 2007
- Article first published online: 17 JUL 2007
- Patient Safety;
- administrative data;
Objective. To assess Agency for Healthcare Research and Quality's Patient Safety Indicators (PSIs) as performance measures using Veterans Administration hospitalization data.
Data Sources Study Setting. Nine years (1997–2005) of all Veterans Health Administration (VA) administrative hospital discharge data.
Study Design. Retrospective analysis using diagnoses and procedures to derive annual rates and standard errors for 13 PSIs.
Data Collection/Extraction Methods. For either hospitals or hospital networks (Veterans Integrated Service Networks [VISNs]), we calculated the percentages whose PSI rates were consistently high or low across years, as well as 1-year lagged correlations, for each PSI. We related our findings to the average annual number of adverse events that each PSI represents. We also assessed time trends for the entire VA, by VISN, and by hospital.
Principal Findings. PSI rates are more stable for VISNs than for individual hospitals, but only for those PSIs that reflect the most frequent adverse events. Only the most frequent PSIs yield significant time trends, and only for larger systems.
Conclusions. Because they are so rare, PSIs are not reliable performance measures to compare individual hospitals. The most frequent PSIs are more stable when applied to hospital networks, but needing large patient samples nullifies their potential value to managers seeking to improve quality locally or to patients seeking optimal care.