Address correspondence to Eric S. Holmboe, M.D., F.A.C.P., F.R.C.P., Chief Medical Officer and Senior Vice President, American Board of Internal Medicine, 510 Walnut Street, Suite 1700, Philadelphia, PA 19106; e-mail: eholmboe@abim.org. Weifeng Weng, Ph.D., Health Services Researcher, Gerald K. Arnold, Ph.D., Health Services Researcher, Sarah Hood, M.S., Manager, Academic Affairs, Rebecca S. Lipner, Ph.D., Vice President, Psychometrics and Research Analysis, are with the American Board of Internal Medicine, Philadelphia, PA. Sherrie H. Kaplan, Ph.D., M.P.H., Assistant Vice Chancellor, is with the Healthcare Evaluation and Measurement, University of California, Irvine, CA. Sherrie H. Kaplan, Ph.D., M.P.H., Executive Co-Director, is also with the Health Policy Research Institute, University of California, Irvine, CA. Sharon-Lise Normand, Ph.D., Professor of Health Care Policy (Biostatistics), is with the Department of Health Care Policy, Harvard Medical School Department of Biostatistics, Harvard School of Public Health, Boston, MA. Sheldon Greenfield, M.D., Donald Bren Professor of Medicine, Executive Co-Director, is with the Health Policy Research Institute, University of California, Irvine, CA.
1475-6773/asset/HESR_left.gif?v=1&s=e1aefc6479f5e5bfb0939ee5cf47664a10a5284c)
1475-6773/asset/HESR_centre.gif?v=1&s=214ade28bc6ce575c2996d5bbf437466f695ada9)
1475-6773/asset/HESR_right.gif?v=1&s=b41919a083fd565ed8cb6b14c656f8af2160073b)
The Comprehensive Care Project: Measuring Physician Performance in Ambulatory Practice
Article first published online: 1 SEP 2010
DOI: 10.1111/j.1475-6773.2010.01160.x
© Health Research and Educational Trust
Additional Information
How to Cite
Holmboe, E. S., Weng, W., Arnold, G. K., Kaplan, S. H., Normand, S.-L., Greenfield, S., Hood, S. and Lipner, R. S. (2010), The Comprehensive Care Project: Measuring Physician Performance in Ambulatory Practice. Health Services Research, 45: 1912–1933. doi: 10.1111/j.1475-6773.2010.01160.x
- †
Address correspondence to Eric S. Holmboe, M.D., F.A.C.P., F.R.C.P., Chief Medical Officer and Senior Vice President, American Board of Internal Medicine, 510 Walnut Street, Suite 1700, Philadelphia, PA 19106; e-mail: eholmboe@abim.org. Weifeng Weng, Ph.D., Health Services Researcher, Gerald K. Arnold, Ph.D., Health Services Researcher, Sarah Hood, M.S., Manager, Academic Affairs, Rebecca S. Lipner, Ph.D., Vice President, Psychometrics and Research Analysis, are with the American Board of Internal Medicine, Philadelphia, PA. Sherrie H. Kaplan, Ph.D., M.P.H., Assistant Vice Chancellor, is with the Healthcare Evaluation and Measurement, University of California, Irvine, CA. Sherrie H. Kaplan, Ph.D., M.P.H., Executive Co-Director, is also with the Health Policy Research Institute, University of California, Irvine, CA. Sharon-Lise Normand, Ph.D., Professor of Health Care Policy (Biostatistics), is with the Department of Health Care Policy, Harvard Medical School Department of Biostatistics, Harvard School of Public Health, Boston, MA. Sheldon Greenfield, M.D., Donald Bren Professor of Medicine, Executive Co-Director, is with the Health Policy Research Institute, University of California, Irvine, CA.
Publication History
- Issue published online: 8 NOV 2010
- Article first published online: 1 SEP 2010
Keywords:
- Comprehensive assessment;
- quality of care;
- primary care;
- composite measures
Objective. To investigate the feasibility, reliability, and validity of comprehensively assessing physician-level performance in ambulatory practice.
Data Sources/Study Setting. Ambulatory-based general internists in 13 states participated in the assessment.
Study Design. We assessed physician-level performance, adjusted for patient factors, on 46 individual measures, an overall composite measure, and composite measures for chronic, acute, and preventive care. Between- versus within-physician variation was quantified by intraclass correlation coefficients (ICC). External validity was assessed by correlating performance on a certification exam.
Data Collection/Extraction Methods. Medical records for 236 physicians were audited for seven chronic and four acute care conditions, and six age- and gender-appropriate preventive services.
Principal Findings. Performance on the individual and composite measures varied substantially within (range 5–86 percent compliance on 46 measures) and between physicians (ICC range 0.12–0.88). Reliabilities for the composite measures were robust: 0.88 for chronic care and 0.87 for preventive services. Higher certification exam scores were associated with better performance on the overall (r=0.19; p<.01), chronic care (r=0.14, p=.04), and preventive services composites (r=0.17, p=.01).
Conclusions. Our results suggest that reliable and valid comprehensive assessment of the quality of chronic and preventive care can be achieved by creating composite measures and by sampling feasible numbers of patients for each condition.