Hierarchical models for ROC curve summary measures: Design and analysis of multi-reader, multi-modality studies of medical tests
Article first published online: 5 MAR 2007
Copyright © 2007 John Wiley & Sons, Ltd.
Statistics in Medicine
Volume 27, Issue 2, pages 243–256, 30 January 2008
How to Cite
Wang, F. and Gatsonis, C. A. (2008), Hierarchical models for ROC curve summary measures: Design and analysis of multi-reader, multi-modality studies of medical tests. Statist. Med., 27: 243–256. doi: 10.1002/sim.2828
- Issue published online: 20 DEC 2007
- Article first published online: 5 MAR 2007
- Manuscript Accepted: 12 DEC 2006
- Manuscript Received: 8 MAR 2005
- NCI. Grant Number: U01 CA079778
- hierarchical modelling;
- sample size determination;
- multi-reader multi-modality ROC analysis;
Comparative studies of the accuracy of diagnostic tests often involve designs according to which each study participant is examined by two or more of the tests and the diagnostic examinations are interpreted by several readers. Tests are then compared on the basis of a summary index, such as the (full or partial) area under the receiver operating characteristic (ROC) curve, averaged over the population of readers. The design and analysis of such studies naturally need to take into account the correlated nature of the diagnostic test results and interpretations.
In this paper, we describe the use of hierarchical modelling for ROC summary measures derived from multi-reader, multi-modality studies. The models allow the variance of the estimates to depend on the actual value of the index and account for the correlation in the data both explicitly via parameters and implicitly via the hierarchical structure. After showing how the hierarchical models can be employed in the analysis of data from multi-reader, multi-modality studies, we discuss the design of such studies using the simulation-based, Bayesian design approach of Wang and Gelfand (Stat. Sci. 2002; 17(2):193–208). The methodology is illustrated via the analysis of data from a study conducted to evaluate a computer-aided diagnosis tool for screen film mammography and via the development of design considerations for a multi-reader study comparing display modes for digital mammography. The hierarchical model methodology described in this paper is also applicable to the meta-analysis of ROC studies. Copyright © 2007 John Wiley & Sons, Ltd.