Get access

Validity of Selected AHRQ Patient Safety Indicators Based on VA National Surgical Quality Improvement Program Data

Authors

  • Patrick S. Romano,

    1. UC Davis Division of General Medicine and Center for Healthcare Policy and Research, 4150 V Street, PSSB Suite 2400, Sacramento, CA 95817,
    Search for more papers by this author
    • Address correspondence to Patrick S. Romano, M.D., M.P.H. – UC Davis Division of General Medicine and Center for Healthcare Policy and Research, 4150 V Street, PSSB Suite 2400, Sacramento, CA 95817; e-mail address: psromano@ucdavis.edu. Amy K. Rosen, Ph.D., Hillary J. Mull, M.P.P., Shibei Zhao, M.P.H., Susan Loveland, M.A.T. are with the Center for Health Quality, Outcomes and Economic Research, Bedford VAMC (152), Bedford, MA. Peter E. Rivard, Ph.D., is with the Center for Organization, Leadership, and Management Research, Boston VA Medical Center, Boston, MA., William G. Henderson, Ph.D., M.P.H., is with the Colorado Health Outcomes Program, University of Colorado Health Sciences CenterAurora, CO. Dennis Tsilimingras, M.D., M.P.H., is with the 1108 Blairmoor CT, Grosse Pointe Woods, MI. Cindy L. Christiansen, Ph.D., is with the Health Policy and Management Department, Boston University School of Public Health, Boston, MA.

  • Hillary J. Mull,

    1. Center for Health Quality, Outcomes and Economic Research, Bedford VAMC (152), Bedford, MA,
    Search for more papers by this author
  • Peter E. Rivard,

    1. Center for Organization, Leadership, and Management Research, Boston VA Medical Center, Boston, MA,
    Search for more papers by this author
  • Shibei Zhao,

    1. Center for Health Quality, Outcomes and Economic Research, Bedford VAMC (152), Bedford, MA,
    Search for more papers by this author
  • William G. Henderson,

    1. Colorado Health Outcomes Program, University of Colorado Health Sciences CenterAurora, CO,
    Search for more papers by this author
  • Susan Loveland,

    1. Center for Health Quality, Outcomes and Economic Research, Bedford VAMC (152), Bedford, MA,
    Search for more papers by this author
  • Dennis Tsilimingras,

    1. 1108 Blairmoor CT, Grosse Pointe Woods, MI
    Search for more papers by this author
  • Cindy L. Christiansen,

    1. Health Policy and Management Department, Boston University School of Public Health, Boston, MA
    Search for more papers by this author
  • Amy K. Rosen

    1. Center for Health Quality, Outcomes and Economic Research, Bedford VAMC (152), Bedford, MA,
    Search for more papers by this author

Abstract

Objectives. To examine the criterion validity of the Agency for Health Care Research and Quality (AHRQ) Patient Safety Indicators (PSIs) using clinical data from the Veterans Health Administration (VA) National Surgical Quality Improvement Program (NSQIP).

Data Sources. Fifty five thousand seven hundred and fifty two matched hospitalizations from 2001 VA inpatient surgical discharge data and NSQIP chart-abstracted data.

Study Design. We examined the sensitivities, specificities, positive predictive values (PPVs), and positive likelihood ratios of five surgical PSIs that corresponded to NSQIP adverse events. We created and tested alternative definitions of each PSI.

Data Collection. FY01 inpatient discharge data were merged with 2001 NSQIP data abstracted from medical records for major noncardiac surgeries.

Principal Findings. Sensitivities were 19–56 percent for original PSI definitions; and 37–63 percent using alternative PSI definitions. PPVs were 22–74 percent and did not improve with modifications. Positive likelihood ratios were 65–524 using original definitions, and 64–744 using alternative definitions. “Postoperative respiratory failure” and “postoperative wound dehiscence” exhibited significant increases in sensitivity after modifications.

Conclusions. PSI sensitivities and PPVs were moderate. For three of the five PSIs, AHRQ has incorporated our alternative, higher sensitivity definitions into current PSI algorithms. Further validation should be considered before most of the PSIs evaluated herein are used to publicly compare or reward hospital performance.

Ancillary