Address correspondence to Justin B. Dimick, M.D., M.P.H., Assistant Professor of Surgery, Department of Surgery, University of Michigan, M-SCORE offices, 211 N. Fourth Avenue, Suite 301, Ann Arbor, MI 48104; e-mail: email@example.com. Douglas O. Staiger, Ph.D., Professor of Economics, is with the Department of Economics and the Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth College, Hanover, NH. John D. Birkmeyer, M.D., Professor of Surgery and Director of Michigan Surgical Collaborative for Outcomes Research and Evaluation (M-SCORE), is with the Department of Surgery, University of Michigan, Ann Arbor, MI 48104.
Ranking Hospitals on Surgical Mortality: The Importance of Reliability Adjustment
Article first published online: 16 AUG 2010
© Health Research and Educational Trust
Health Services Research
Volume 45, Issue 6p1, pages 1614–1629, December 2010
How to Cite
Dimick, J. B., Staiger, D. O. and Birkmeyer, J. D. (2010), Ranking Hospitals on Surgical Mortality: The Importance of Reliability Adjustment. Health Services Research, 45: 1614–1629. doi: 10.1111/j.1475-6773.2010.01158.x
- Issue published online: 8 NOV 2010
- Article first published online: 16 AUG 2010
Objective. We examined the implications of reliability adjustment on hospital mortality with surgery.
Data Source. We used national Medicare data (2003–2006) for three surgical procedures: coronary artery bypass grafting (CABG), abdominal aortic aneurysm (AAA) repair, and pancreatic resection.
Study Design. We conducted an observational study to evaluate the impact of reliability adjustment on hospital mortality rankings. Using hierarchical modeling, we adjusted hospital mortality for reliability using empirical Bayes techniques. We assessed the implication of this adjustment on the apparent variation across hospitals and the ability of historical hospital mortality rates (2003–2004) to forecast future mortality (2005–2006).
Principal Findings. The net effect of reliability adjustment was to greatly diminish apparent variation for all three operations. Reliability adjustment was also particularly important for identifying hospitals with the lowest future mortality. Without reliability adjustment, hospitals in the “best” quintile (2003–2004) with pancreatic resection had a mortality of 7.6 percent in 2005–2006; with reliability adjustment, the “best” hospital quintile had a mortality of 2.7 percent in 2005–2006. For AAA repair, reliability adjustment also improved the ability to identify hospitals with lower future mortality. For CABG, the benefits of reliability adjustment were limited to the lowest volume hospitals.
Conclusion. Reliability adjustment results in more stable estimates of mortality that better forecast future performance. This statistical technique is crucial for helping patients select the best hospitals for specific procedures, particularly uncommon ones, and should be used for public reporting of hospital mortality.