Address correspondence to Yue Li, Ph.D., Department of Medicine, University of California, Irvine, CA 92697; e-mail: firstname.lastname@example.org. Xueya Cai, M.A., is with the Division of Biostatistics, Indiana University School of Medicine, Indiana University Purdue University Indianapolis, Indianapolis, IN. Laurent G. Glance, M.D., is with the Department of Anesthesiology, The University of Rochester School of Medicine and Dentistry, Rochester, NY. William D. Spector, Ph.D., is with the Center for Delivery, Organization, and Markets, Agency for Healthcare Research and Quality, Rockville, M.D. Dana B. Mukamel, Ph.D., is with the Center for Health Policy Research, University of California, Irvine, CA.
National Release of the Nursing Home Quality Report Cards: Implications of Statistical Methodology for Risk Adjustment
Article first published online: 6 OCT 2008
© Health Research and Educational Trust
Health Services Research
Volume 44, Issue 1, pages 79–102, February 2009
How to Cite
Li, Y., Cai, X., Glance, L. G., Spector, W. D. and Mukamel, D. B. (2009), National Release of the Nursing Home Quality Report Cards: Implications of Statistical Methodology for Risk Adjustment. Health Services Research, 44: 79–102. doi: 10.1111/j.1475-6773.2008.00910.x
- Issue published online: 15 JAN 2009
- Article first published online: 6 OCT 2008
- Nursing home;
- quality report cards;
- activities of daily living;
- risk adjustment;
Objective. To determine how alternative statistical risk-adjustment methods may affect the quality measures (QMs) in nursing home (NH) report cards.
Data Sources/Study Settings. Secondary data from the national Minimum Data Set files of 2004 and 2005 that include 605,433 long-term residents in 9,336 facilities.
Study Design. We estimated risk-adjusted QMs of decline in activities of daily living (ADL) functioning using classical, fixed-effects, and random-effects logistic models. Risk-adjusted QMs were compared with each other, and with the published QM (unadjusted) in identifying high- and low-quality facilities by either the rankings or 95 percent confidence intervals of QMs.
Principal Findings. Risk-adjusted QMs showed better overall agreement (or convergent validity) with each other than did the unadjusted versus each adjusted QM; the disagreement rate between unadjusted and adjusted QM can be as high as 48 percent. The risk-adjusted QM derived from the random-effects shrinkage estimator deviated nonrandomly from other risk-adjusted estimates in identifying the best 10 percent facilities using rankings.
Conclusions. The extensively risk-adjusted QMs of ADL decline, even when estimated by alternative statistical methods, show higher convergent validity and provide more robust NH comparisons than the unadjusted QM. Outcome rankings based on ADL decline tend to show lower convergent validity when estimated by the shrinkage estimator rather than other statistical methods.