National Release of the Nursing Home Quality Report Cards: Implications of Statistical Methodology for Risk Adjustment

Authors

  • Yue Li,

    1. Department of Medicine, University of California, Irvine, CA 92697,
    Search for more papers by this author
    • Address correspondence to Yue Li, Ph.D., Department of Medicine, University of California, Irvine, CA 92697; e-mail: ylill@uci.edu. Xueya Cai, M.A., is with the Division of Biostatistics, Indiana University School of Medicine, Indiana University Purdue University Indianapolis, Indianapolis, IN. Laurent G. Glance, M.D., is with the Department of Anesthesiology, The University of Rochester School of Medicine and Dentistry, Rochester, NY. William D. Spector, Ph.D., is with the Center for Delivery, Organization, and Markets, Agency for Healthcare Research and Quality, Rockville, M.D. Dana B. Mukamel, Ph.D., is with the Center for Health Policy Research, University of California, Irvine, CA.

  • Xueya Cai,

    1. Division of Biostatistics, Indiana University School of Medicine, Indiana University Purdue University Indianapolis, Indianapolis, IN,
    Search for more papers by this author
  • Laurent G. Glance,

    1. Department of Anesthesiology, The University of Rochester School of Medicine and Dentistry, Rochester, NY,
    Search for more papers by this author
  • William D. Spector,

    1. Center for Delivery, Organization, and Markets, Agency for Healthcare Research and Quality, Rockville, M.D., and
    Search for more papers by this author
  • Dana B. Mukamel

    1. Center for Health Policy Research, University of California, Irvine, CA
    Search for more papers by this author

Abstract

Objective. To determine how alternative statistical risk-adjustment methods may affect the quality measures (QMs) in nursing home (NH) report cards.

Data Sources/Study Settings. Secondary data from the national Minimum Data Set files of 2004 and 2005 that include 605,433 long-term residents in 9,336 facilities.

Study Design. We estimated risk-adjusted QMs of decline in activities of daily living (ADL) functioning using classical, fixed-effects, and random-effects logistic models. Risk-adjusted QMs were compared with each other, and with the published QM (unadjusted) in identifying high- and low-quality facilities by either the rankings or 95 percent confidence intervals of QMs.

Principal Findings. Risk-adjusted QMs showed better overall agreement (or convergent validity) with each other than did the unadjusted versus each adjusted QM; the disagreement rate between unadjusted and adjusted QM can be as high as 48 percent. The risk-adjusted QM derived from the random-effects shrinkage estimator deviated nonrandomly from other risk-adjusted estimates in identifying the best 10 percent facilities using rankings.

Conclusions. The extensively risk-adjusted QMs of ADL decline, even when estimated by alternative statistical methods, show higher convergent validity and provide more robust NH comparisons than the unadjusted QM. Outcome rankings based on ADL decline tend to show lower convergent validity when estimated by the shrinkage estimator rather than other statistical methods.

Ancillary