Equivalence of improvement in area under ROC curve and linear discriminant analysis coefficient under assumption of normality

Authors

  • Olga V. Demler,

    Corresponding author
    1. Department of Biostatistics, Boston University, 801 Massachusetts Avenue, Boston, MA 02118, U.S.A.
    • Department of Biostatistics, Boston University, 801 Massachusetts Avenue, Boston, MA 02118, U.S.A.
    Search for more papers by this author
  • Michael J. Pencina,

    1. Department of Biostatistics, Boston University, Harvard Clinical Research Institute, 801 Massachusetts Avenue, Boston, MA 02118, U.S.A.
    Search for more papers by this author
  • Ralph B. D'Agostino Sr.

    1. Department of Mathematics and Statistics, Boston University, 111 Cummington Street, Boston, MA 02215, U.S.A.
    Search for more papers by this author

Abstract

In this paper we investigate the addition of new variables to an existing risk prediction model and the subsequent impact on discrimination quantified by the area under the receiver operating characteristics curve (AUC of ROC). Based on practical experience, concerns have emerged that the significance of association of the variable under study with the outcome in the risk model does not correspond to the significance of the change in AUC: that is, often the variable is significant, but the change in AUC is not. This paper demonstrates that under the assumption of multivariate normality and employing linear discriminant analysis (LDA) to construct the risk prediction tool, statistical significance of the new predictor(s) is equivalent to the statistical significance of the increase in AUC. Under these assumptions the result extends asymptotically to logistic regression. We further show that equality of variance–covariance matrices of predictors within cases and non-cases is not necessary when LDA is used. However, our practical example from the Framingham Heart Study data suggests that the finding might be sensitive to the assumption of normality. Copyright © 2011 John Wiley & Sons, Ltd.

Ancillary