Measuring Patients' Experiences with Individual Primary Care Physicians

Results of a Statewide Demonstration Project

Authors

  • Dana Gelb Safran ScD,

    1. The Health Institute, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Boston, Mass, USA
    2. Department of Medicine, Tufts University, Boston, Mass, USA
    Search for more papers by this author
  • Melinda Karp MBA,

    1. Massachusetts Health Quality Partners Inc., Boston, Mass, USA.
    Search for more papers by this author
  • Kathryn Coltin MPH,

    1. Massachusetts Health Quality Partners Inc., Boston, Mass, USA.
    Search for more papers by this author
  • Hong Chang PhD,

    1. The Health Institute, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Boston, Mass, USA
    2. Department of Medicine, Tufts University, Boston, Mass, USA
    Search for more papers by this author
  • Angela Li BS,

    1. The Health Institute, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Boston, Mass, USA
    Search for more papers by this author
  • John Ogren MD, ScM,

    1. The Health Institute, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Boston, Mass, USA
    Search for more papers by this author
  • William H. Rogers PhD

    1. The Health Institute, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Boston, Mass, USA
    2. Department of Medicine, Tufts University, Boston, Mass, USA
    Search for more papers by this author

The authors have no conflicts of interest to report.Address correspondence and requests for reprints to Dr. Safran: The Health Institute, Tufts-New England Medical Center, 750 Washington Street, Box 345, Boston, MA 02111 (e-mail: dsafran@tufts-nemc.org).

Abstract

BACKGROUND: Measuring and reporting patients' experiences with health plans has been routine for several years. There is now substantial interest in measuring patients' experiences with individual physicians, but numerous concerns remain.

OBJECTIVE: The Massachusetts Ambulatory Care Experiences Survey Project was a statewide demonstration project designed to test the feasibility and value of measuring patients' experiences with individual primary care physicians and their practices.

DESIGN: Cross-sectional survey administered to a statewide sample by mail and telephone (May–August 2002).

PATIENTS: Adult patients from 5 commerical health plans and Medicaid sampled from the panels of 215 generalist physicians at 67 practice sites (n=9,625).

MEASUREMENTS: Ambulatory Care Experiences Survey produces 11 summary measures of patients' experiences across 2 domains: quality of physician-patient interactions and organizational features of care. Physician-level reliability was computed for all measures, and variance components analysis was used to determine the influence of each level of the system (physician, site, network organization, plan) on each measure. Risk of misclassifying individual physicians was evaluated under varying reporting frameworks.

RESULTS: All measures except 2 achieved physician-level reliability of at least 0.70 with samples of 45 patients per physician, and several exceeded 0.80. Physicians and sites accounted for the majority of system-related variance on all measures, with physicians accounting for the majority on all “interaction quality” measures (range: 61.7% to 83.9%) and sites accounting for the largest share on “organizational” measures (range: 44.8% to 81.1%). Health plans accounted for neglible variance (<3%) on all measures. Reporting frameworks and principles for assuring misclassification risk ≤2.5% were identified.

CONCLUSIONS: With considerable national attention on the importance of patient-centered care, this project demonstrates the feasibility of obtaining highly reliable measures of patients' experiences with individual physicians and practices. The analytic findings underscore the validity and importance of looking beyond health plans to individual physicians and sites as we seek to improve health care quality.

Ancillary