Assessing the incremental value of diagnostic and prognostic markers: a review and illustration
Article first published online: 4 JUL 2011
© 2011 The Authors. European Journal of Clinical Investigation © 2011 Stichting European Society for Clinical Investigation Journal Foundation
European Journal of Clinical Investigation
Volume 42, Issue 2, pages 216–228, February 2012
How to Cite
Steyerberg, E. W., Pencina, M. J., Lingsma, H. F., Kattan, M. W., Vickers, A. J. and Van Calster, B. (2012), Assessing the incremental value of diagnostic and prognostic markers: a review and illustration. European Journal of Clinical Investigation, 42: 216–228. doi: 10.1111/j.1365-2362.2011.02562.x
- Issue published online: 6 JAN 2012
- Article first published online: 4 JUL 2011
- Accepted manuscript online: 31 MAY 2011 07:47AM EST
- Received 19 December 2010; accepted 26 May 2011
- Incremental value;
- logistic regression model;
- performance measures;
Eur J Clin Invest 2011
Background New markers may improve prediction of diagnostic and prognostic outcomes. We review various measures to quantify the incremental value of markers over standard, readily available characteristics.
Methods Widely used traditional measures include the improvement in model fit or in the area under the receiver operating characteristic (ROC) curve (AUC). New measures include the net reclassification index (NRI) and decision-analytic measures, such as the fraction of true-positive classifications penalized for false-positive classifications [net benefit (NB)]. For illustration, we discuss a case study on the presence of residual tumour vs. benign tissue in 544 patients with testicular cancer. We assessed three tumour markers [Alpha-fetoprotein (AFP), Human chorionic gonadotropin (HCG) and Lactate dehydrogenase (LDH)] for their incremental value over currently standard clinical predictors.
Results AUC and R2 values suggested adding continuous LDH and AFP whereas NB only favoured HCG as a potentially promising marker at a clinically defendable decision threshold of 20% risk. The NRI suggested reclassification potential of all three markers.
Conclusions The improvement in standard discrimination measures, which focus on finding variables that might be promising across all decision thresholds, may not detect the most informative markers at a specific threshold of particular clinical relevance. When a marker is intended to support decision-making, calculation of the improvement in a decision-analytic measure, such as NB, is preferable over an overall judgment as obtained from the AUC in ROC analysis.