Improving Quality Assessment through Multilevel Modeling: The Case of Nursing Home Compare

Authors

  • Greg Arling,

    1. Center for Health Policy, School of Public and Environmental Affairs, IUPUI, 324 N Senate Ave., Indianapolis, IN 46204,
    2. Indiana University Center for Health Policy, School of Public and Environmental Affairs, Indiana University–Purdue University Indianapolis,
    3. Indiana University Center for Aging Research,
    Search for more papers by this author
    • Address correspondence to Greg Arling, Ph.D., IU Center for Health Policy, School of Public and Environmental Affairs, IUPUI, 324 N Senate Ave., Indianapolis, IN 46204. Dr. Arling is with the Indiana University Center for Health Policy, School of Public and Environmental Affairs, Indiana University–Purdue University Indianapolis and with the Indiana University Center for Aging Research. Teresa Lewis, B.A., is with the Minnesota Department of Human Services, Nursing Facility Rates and Policy Division, St. Paul, MN and also with the Division of Health Policy and Management, School of Public Health, University of Minnesota, Minneapolis, MN. Robert L. Kane, M.D., and Shannon Flood, B.A., are with the Division of Health Policy and Management, School of Public Health, University of Minnesota, Minneapolis, MN. Christine Mueller, R.N., Ph.D., is with the School of Nursing, University of Minnesota, Minneapolis, MN.

  • Teresa Lewis,

    1. Minnesota Department of Human Services, Nursing Facility Rates and Policy Division, St. Paul, MN,
    2. Division of Health Policy and Management, School of Public Health, University of Minnesota, Minneapolis, MN,
    Search for more papers by this author
  • Robert L. Kane,

    1. Division of Health Policy and Management, School of Public Health, University of Minnesota, Minneapolis, MN,
    Search for more papers by this author
  • Christine Mueller,

    1. School of Nursing, University of Minnesota, Minneapolis, MN
    Search for more papers by this author
  • Shannon Flood

    1. Division of Health Policy and Management, School of Public Health, University of Minnesota, Minneapolis, MN,
    Search for more papers by this author

Abstract

Objective. To demonstrate how multilevel modeling and empirical Bayes (EB) estimates can improve Medicare's Nursing Home Compare quality measures (QMs).

Data Sources/Study Setting. Secondary data from July 1 to September 30, 2004. Facility-level QMs were estimated from minimum data set (MDS) assessments for approximately 31,000 Minnesota nursing home residents in 393 facilities.

Study Design. Prevalence and incidence rates for 12 nursing facility QMs (e.g., use of physical restraints, pressure sores, and weight loss) were estimated with EB methods and risk adjustment using a hierarchical general linear model. Three sets of rates were developed: Nursing Home Compare's current method, unadjusted EB rates, and risk-adjusted EB rates. Bayesian 90 percent credibility intervals (CIs) were constructed around EB rates, and these were used to flag facilities for potential quality of care problems.

Data Collection/Extraction Methods. MDS assessments were performed by nursing facility staff, transmitted electronically to the Minnesota Department of Health, and provided to the investigators.

Principal Findings. Facility rates and rankings for the 12 QMs differed substantially using the multilevel models compared with current methods. The EB estimated rates shrank considerably toward the population mean. Risk adjustment had a large impact on some QM rates and a more modest impact on others. When EB CIs were used to flag problem facilities, there was wide variation across QMs in the percentage of facilities flagged.

Conclusions. Multilevel modeling should be applied to Nursing Home Compare and more widely in other health care quality assessment systems.

Ancillary