Clinical Effectiveness Research in Managed-care Systems: Lessons from the Pediatric Asthma Care PORT


  • Jonathan A. Finkelstein,

  • Paula Lozano,

  • Kachen A. Streiff,

  • Kelly E. Arduino,

  • Cynthia A. Sisk,

  • Edward H. Wagner,

  • Kevin B. Weiss,

  • Thomas S. Inui

The authors would like to acknowledge the contributions of Kathleen Loane, the PAC PORT Educational Coordinator, and the many other project staff. We also thank the health plan administrators who have supported this work in their organizations, and the providers who gave their time and energy to this project. The Pediatric Asthma Care PORT is funded by the Agency for Healthcare Research and Quality and the National Heart, Lung, and Blood Institute, grant # HS08368-01.

Address correspondence to Jonathan A. Finkelstein, M.D., M.P.H.; Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, 133 Brookline Ave., 6th Floor, Boston, MA 02215. Paula Lozano, M.D., M.P.H., Cynthia A. Sisk, B.A., Edward H. Wagner, M.D., M.P.H., are with the University of Washington, and the Center for Health Studies Group Health Cooperative Seattle, WA. Dr. Wagner is also with the MacColl Institute for Healthcare Innovation, Center For Health Studies, Group Health Cooperative, Seattle, WA. Kachen A. Streiff, M.S., and Thomas S. Inui, Sc.M., M.D., are with the Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, Boston, MA. Kelly E. Arduino, M.A., and Kevin B. Weiss, M.D., M.P.H., are with the Midwest Center for Health Services and Policy Research, Hines V. A. Hospital, Chicago, IL. Dr. Weiss is also with the Center for Healthcare Studies and Division of General Medicine, Department of Medicine, Northwestern University, Feinberg School of Medicine, Chicago, IL.


Objective. To highlight the unique challenges of evaluative research on practice behavior change in the “real world'' settings of contemporary managed-care organizations, using the experience of the Pediatric Asthma Care PORT (Patient Outcomes Research Team).

Study Setting. The Pediatric Asthma Care PORT is a five-year initiative funded by the Agency for Healthcare Research and Quality to study strategies for asthma care improvement in three managed-care plans in Chicago, Seattle, and Boston. At its core is a randomized trial of two care improvement strategies compared with usual care: (1) a targeted physician education program using practice based Peer Leaders (PL) as change agents, (2) adding to the PL intervention a “Planned Asthma Care Intervention'' incorporating joint “asthma check-ups'' by nurse-physician teams. During the trial, each of the participating organizations viewed asthma care improvement as an immediate priority and had their own corporate improvement programs underway.

Data Collection. Investigators at each health plan described the organizational and implementation challenges in conducting the PAC PORT randomized trial. These experiences were reviewed for common themes and “lessons'' that might be useful to investigators planning interventional research in similar care-delivery settings.

Conclusions. Randomized trials in “real world'' settings represent the most robust design available to test care improvement strategies. In complex, rapidly changing managed-care organizations, blinding is not feasible, corporate initiatives may complicate implementation, and the assumption that a “usual care'' arm will be static is highly likely to be mistaken. Investigators must be prepared to use innovative strategies to maintain the integrity of the study design, including: continuous improvement within the intervention arms, comanagement by researchers and health plan managers of condition-related quality improvement initiatives, procedures for avoiding respondent burden in health plan enrollees, and anticipation and minimization of risks from experimental arm contamination and major organizational change. With attention to these delivery system issues, as well as the usual design features of randomized trials, we believe managed-care organizations can serve as important laboratories to test care improvement strategies.