Volume 32, Issue 30
Special Issue Paper

Estimating and comparing time‐dependent areas under receiver operating characteristic curves for censored event times with competing risks

Paul Blanche

Corresponding Author

University Bordeaux, ISPED, Centre INSERM U897‐Epidémiologie‐Biostatistique, F‐33000 Bordeaux, France

INSERM, ISPED, Centre INSERM U897‐Epidémiologie‐Biostatistique, F‐33000 Bordeaux, France

Correspondence to: Paul Blanche, INSERM U897 ‐ Equipe de biostatistique, ISPED, Université Bordeaux Segalen, 146 rue Leo Saignat, 33076 Bordeaux cedex, France.

E‐mail: Paul.Blanche@isped.u‐bordeaux2.fr

Search for more papers by this author
Jean‐François Dartigues

University Bordeaux, ISPED, Centre INSERM U897‐Epidémiologie‐Biostatistique, F‐33000 Bordeaux, France

INSERM, ISPED, Centre INSERM U897‐Epidémiologie‐Biostatistique, F‐33000 Bordeaux, France

Search for more papers by this author
Hélène Jacqmin‐Gadda

University Bordeaux, ISPED, Centre INSERM U897‐Epidémiologie‐Biostatistique, F‐33000 Bordeaux, France

INSERM, ISPED, Centre INSERM U897‐Epidémiologie‐Biostatistique, F‐33000 Bordeaux, France

Search for more papers by this author
First published: 12 September 2013
Citations: 239

Abstract

The area under the time‐dependent ROC curve (AUC) may be used to quantify the ability of a marker to predict the onset of a clinical outcome in the future. For survival analysis with competing risks, two alternative definitions of the specificity may be proposed depending of the way to deal with subjects who undergo the competing events. In this work, we propose nonparametric inverse probability of censoring weighting estimators of the AUC corresponding to these two definitions, and we study their asymptotic properties. We derive confidence intervals and test statistics for the equality of the AUCs obtained with two markers measured on the same subjects. A simulation study is performed to investigate the finite sample behaviour of the test and the confidence intervals. The method is applied to the French cohort PAQUID to compare the abilities of two psychometric tests to predict dementia onset in the elderly accounting for death without dementia competing risk. The ‘timeROC’ R package is provided to make the methodology easily usable. Copyright © 2013 John Wiley & Sons, Ltd.

Number of times cited according to CrossRef: 239

  • Tumor Infiltration by OX40+ Cells Enhances the Prognostic Significance of CD16+ Cell Infiltration in Colorectal Cancer, Cancer Control, 10.1177/1073274820903383, 27, 1, (107327482090338), (2020).
  • Risk factors and a 3-month risk score for predicting pacemaker implantation in patients with atrial fibrillation, Open Heart, 10.1136/openhrt-2019-001125, 7, 1, (e001125), (2020).
  • Predicting per-lesion local recurrence in locally advanced non-small cell lung cancer following definitive radiation therapy using pre- and mid-treatment metabolic tumor volume, Radiation Oncology, 10.1186/s13014-020-01546-y, 15, 1, (2020).
  • Tumor-associated macrophages in classical Hodgkin lymphoma: hormetic relationship to outcome, Scientific Reports, 10.1038/s41598-020-66010-z, 10, 1, (2020).
  • Establishment and validation of an immune-based prognostic score model in glioblastoma, International Immunopharmacology, 10.1016/j.intimp.2020.106636, 85, (106636), (2020).
  • Bioinformatics analysis of microRNA linked to ubiquitin proteasome system in traumatic osteonecrosis of the femoral head, Medicine, 10.1097/MD.0000000000021706, 99, 33, (e21706), (2020).
  • Identification of Biomarkers to Construct a Competing Endogenous RNA Network and Establishment of a Genomic-Clinicopathologic Nomogram to Predict Survival for Children with Rhabdoid Tumors of the Kidney, BioMed Research International, 10.1155/2020/5843874, 2020, (1-27), (2020).
  • Development of Autophagy Signature-Based Prognostic Nomogram for Refined Glioma Survival Prognostication, BioMed Research International, 10.1155/2020/1872962, 2020, (1), (2020).
  • Circulating tumor DNA methylation profiles enable early diagnosis, prognosis prediction, and screening for colorectal cancer, Science Translational Medicine, 10.1126/scitranslmed.aax7533, 12, 524, (eaax7533), (2020).
  • Predictive Medicine in Multiple Sclerosis: a Systematic Review, Multiple Sclerosis and Related Disorders, 10.1016/j.msard.2020.101928, (101928), (2020).
  • Combining the Fibrinogen/Albumin Ratio and Systemic Inflammation Response Index Predicts Survival in Resectable Gastric Cancer, Gastroenterology Research and Practice, 10.1155/2020/3207345, 2020, (1-9), (2020).
  • Non-lethal diagnostic methods for koi herpesvirus in koi Cyprinus carpio, Diseases of Aquatic Organisms, 10.3354/dao03456, 138, (195-205), (2020).
  • Testing longitudinal data for prognostication in ambulatory heart failure patients with reduced ejection fraction. A proof of principle from the GISSI-HF database, International Journal of Cardiology, 10.1016/j.ijcard.2020.03.064, (2020).
  • Genetic variants in PDSS1 and SLC16A6 of the ketone body metabolic pathway predict cutaneous melanoma‐specific survival, Molecular Carcinogenesis, 10.1002/mc.23191, 59, 6, (640-650), (2020).
  • Predicting worse survival for newly diagnosed T cell lymphoma based on the decreased baseline CD16−/CD16 + monocyte ratio, Scientific Reports, 10.1038/s41598-020-64579-z, 10, 1, (2020).
  • Co-expression network analysis identifies a gene signature as a predictive biomarker for energy metabolism in osteosarcoma, Cancer Cell International, 10.1186/s12935-020-01352-2, 20, 1, (2020).
  • An Enhanced Prognostic Score for Overall Survival of Patients with Cancer Derived from a large Real World Cohort, Annals of Oncology, 10.1016/j.annonc.2020.07.013, (2020).
  • Developing an Improved Statistical Approach for Survival Estimation in Bone Metastases Management: The Bone Metastases Ensemble Trees for Survival (BMETS) Model, International Journal of Radiation Oncology*Biology*Physics, 10.1016/j.ijrobp.2020.05.023, (2020).
  • Development and validation of a prognostic classifier based on HIF-1 signaling for hepatocellular carcinoma, Aging, 10.18632/aging.102820, 12, 4, (3431-3450), (2020).
  • Development of an immune-related prognostic index associated with hepatocellular carcinoma, Aging, 10.18632/aging.102926, 12, 6, (5010-5030), (2020).
  • EFEMP2 indicates assembly of M0 macrophage and more malignant phenotypes of glioma, Aging, 10.18632/aging.103147, 12, 9, (8397-8412), (2020).
  • A novel molecular-clinicopathologic nomogram to improve prognosis prediction of hepatocellular carcinoma, Aging, 10.18632/aging.103350, 12, 13, (12896-12920), (2020).
  • Prognostic ability of the distance‐saturation product in the 6‐minute walk test in patients with chronic obstructive pulmonary disease, The Clinical Respiratory Journal, 10.1111/crj.13141, 14, 4, (364-369), (2020).
  • Assessment of immunodeficiency scoring index performance in enterovirus/rhinovirus respiratory infection after allogeneic hematopoietic stem cell transplantation, Transplant Infectious Disease, 10.1111/tid.13301, 22, 4, (2020).
  • Accounting for grouped predictor variables or pathways in high-dimensional penalized Cox regression models, BMC Bioinformatics, 10.1186/s12859-020-03618-y, 21, 1, (2020).
  • Flexible Humidity and Pressure Sensors Realized by Molding and Inkjet Printing Processes with Sandwich Structure, Macromolecular Materials and Engineering, 10.1002/mame.202000287, 305, 8, (2020).
  • Targeted multiplex proteomics for prediction of all-cause mortality during long-term follow-up in outpatients with peripheral arterial disease, Atherosclerosis, 10.1016/j.atherosclerosis.2020.06.015, (2020).
  • Development of a novel prognostic score combining clinicopathologic variables, gene expression, and mutation profiles for lung adenocarcinoma, World Journal of Surgical Oncology, 10.1186/s12957-020-02025-0, 18, 1, (2020).
  • One-Hour Oral Glucose Tolerance Tests for the Prediction and Diagnostic Surveillance of Type 1 Diabetes, The Journal of Clinical Endocrinology & Metabolism, 10.1210/clinem/dgaa592, 105, 11, (2020).
  • An approach to classifying subjective cognitive decline in community‐dwelling elders, Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring, 10.1002/dad2.12103, 12, 1, (2020).
  • Development and validation of a robust immune-related prognostic signature in early-stage lung adenocarcinoma, Journal of Translational Medicine, 10.1186/s12967-020-02545-z, 18, 1, (2020).
  • Global Right Heart Assessment with Speckle-Tracking Imaging Improves the Risk Prediction of a Validated Scoring System in Pulmonary Arterial Hypertension, Journal of the American Society of Echocardiography, 10.1016/j.echo.2020.05.020, (2020).
  • Multi-omics Data Analyses Construct TME and Identify the Immune-Related Prognosis Signatures in Human LUAD, Molecular Therapy - Nucleic Acids, 10.1016/j.omtn.2020.07.024, 21, (860-873), (2020).
  • Predicting progression to advanced age-related macular degeneration from clinical, genetic and lifestyle factors using machine learning, Ophthalmology, 10.1016/j.ophtha.2020.08.031, (2020).
  • A Viral Exposure Signature Defines Early Onset of Hepatocellular Carcinoma, Cell, 10.1016/j.cell.2020.05.038, (2020).
  • Predictors of medium- and long-term mortality in elderly patients with acute pulmonary embolism, Heliyon, 10.1016/j.heliyon.2020.e04857, 6, 9, (e04857), (2020).
  • High-Sensitivity Cardiac Troponin I Is a Strong Predictor of Cardiovascular Events and Mortality in the AGES-Reykjavik Community-Based Cohort of Older Individuals, Clinical Chemistry, 10.1373/clinchem.2015.250811, 62, 4, (623-630), (2020).
  • Identification of prognostic splicing factors and exploration of their potential regulatory mechanisms in pancreatic adenocarcinoma, PeerJ, 10.7717/peerj.8380, 8, (e8380), (2020).
  • Muscle Loss after Chemoradiotherapy as a Biomarker of Distant Failures in Locally Advanced Cervical Cancer, Cancers, 10.3390/cancers12030595, 12, 3, (595), (2020).
  • Construction and evaluation of a prognosis lncRNA model for hepatocellular carcinoma, Journal of Cellular Biochemistry, 10.1002/jcb.29608, 0, 0, (2020).
  • Immunoscore Predicts Survival in Early-Stage Lung Adenocarcinoma Patients, Frontiers in Oncology, 10.3389/fonc.2020.00691, 10, (2020).
  • A Preoperative Clinical Risk Score Including C-Reactive Protein Predicts Histological Tumor Characteristics and Patient Survival after Surgery for Sporadic Non-Functional Pancreatic Neuroendocrine Neoplasms: An International Multicenter Cohort Study, Cancers, 10.3390/cancers12051235, 12, 5, (1235), (2020).
  • Identification and validation of a combined hypoxia and immune index for triple‐negative breast cancer, Molecular Oncology, 10.1002/1878-0261.12747, 0, 0, (2020).
  • A Prognostic Prediction Model Developed Based on Four CpG Sites and Weighted Correlation Network Analysis Identified DNAJB1 as a Novel Biomarker for Pancreatic Cancer, Frontiers in Oncology, 10.3389/fonc.2020.01716, 10, (2020).
  • Classification and Prognosis Prediction from Histopathological Images of Hepatocellular Carcinoma by a Fully Automated Pipeline Based on Machine Learning, Annals of Surgical Oncology, 10.1245/s10434-019-08190-1, (2020).
  • smoothROCtime: an R package for time-dependent ROC curve estimation, Computational Statistics, 10.1007/s00180-020-00955-7, (2020).
  • A risk prediction model of DNA methylation improves prognosis evaluation and indicates gene targets in prostate cancer, Epigenomics, 10.2217/epi-2019-0349, (2020).
  • Quantification of 18F-fluorodeoxyglucose uptake to detect residual nodal disease in locally advanced head and neck squamous cell carcinoma after chemoradiotherapy: results from the ECLYPS study, European Journal of Nuclear Medicine and Molecular Imaging, 10.1007/s00259-020-04710-4, (2020).
  • Handgrip Strength and Timed Up-And-Go (TUG) Test are Predictors of Short-Term Mortality Among Elderly in a Population-Based Cohort in Singapore, The journal of nutrition, health & aging, 10.1007/s12603-020-1337-0, (2020).
  • Validation and refinement of the revised 2017 European LeukemiaNet genetic risk stratification of acute myeloid leukemia, Leukemia, 10.1038/s41375-020-0806-0, (2020).
  • Prediction of contralateral breast cancer: external validation of risk calculators in 20 international cohorts, Breast Cancer Research and Treatment, 10.1007/s10549-020-05611-8, (2020).
  • Indexation of left ventricular mass to predict adverse clinical outcomes in pre-dialysis patients with chronic kidney disease: KoreaN cohort study of the outcome in patients with chronic kidney disease, PLOS ONE, 10.1371/journal.pone.0233310, 15, 5, (e0233310), (2020).
  • Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology, BMJ, 10.1136/bmj.m1501, (m1501), (2020).
  • Prognostic model and nomogram construction based on autophagy signatures in lower grade glioma, Journal of Cellular Physiology, 10.1002/jcp.29837, 0, 0, (2020).
  • N6-Methylandenosine-Related lncRNAs Are Potential Biomarkers for Predicting the Overall Survival of Lower-Grade Glioma Patients, Frontiers in Cell and Developmental Biology, 10.3389/fcell.2020.00642, 8, (2020).
  • Extraction of radiographic findings from unstructured thoracoabdominal computed tomography reports using convolutional neural network based natural language processing, PLOS ONE, 10.1371/journal.pone.0236827, 15, 7, (e0236827), (2020).
  • A combined risk score enhances prediction of type 1 diabetes among susceptible children, Nature Medicine, 10.1038/s41591-020-0930-4, (2020).
  • PathwayPCA: an R/Bioconductor Package for Pathway Based Integrative Analysis of Multi‐Omics Data, PROTEOMICS, 10.1002/pmic.201900409, 0, 0, (2020).
  • Early M‐Protein Dynamics Predicts Progression‐Free Survival in Patients With Relapsed/Refractory Multiple Myeloma, Clinical and Translational Science, 10.1111/cts.12836, 0, 0, (2020).
  • Long-term follow-up of JAK2 exon 12 polycythemia vera: a French Intergroup of Myeloproliferative Neoplasms (FIM) study, Leukemia, 10.1038/s41375-020-0991-x, (2020).
  • A novel microRNA-based signature predicts prognosis among nasopharyngeal cancer patients, Experimental Biology and Medicine, 10.1177/1535370220958680, (153537022095868), (2020).
  • Radiomics may increase the prognostic value for survival in glioblastoma patients when combined with conventional clinical and genetic prognostic models, European Radiology, 10.1007/s00330-020-07335-1, (2020).
  • A prognostic five long–noncoding RNA signature for patients with rectal cancer, Journal of Cellular Biochemistry, 10.1002/jcb.29549, 121, 8-9, (3854-3860), (2019).
  • The ternary complex factor protein ELK1 is an independent prognosticator of disease recurrence in prostate cancer, The Prostate, 10.1002/pros.23932, 80, 2, (198-208), (2019).
  • Identification of novel prognostic alternative splicing signature in papillary renal cell carcinoma, Journal of Cellular Biochemistry, 10.1002/jcb.29314, 121, 1, (672-689), (2019).
  • Validation of discrete time‐to‐event prediction models in the presence of competing risks, Biometrical Journal, 10.1002/bimj.201800293, 62, 3, (643-657), (2019).
  • Comparison of pre‐ and post‐bronchodilator lung function as predictors of mortality: The HUNT Study, Respirology, 10.1111/resp.13648, 25, 4, (401-409), (2019).
  • Genome‐wide screening diagnostic biomarkers and the construction of prognostic model of hepatocellular carcinoma, Journal of Cellular Biochemistry, 10.1002/jcb.29480, 121, 3, (2582-2594), (2019).
  • A deep learning model for early prediction of Alzheimer's disease dementia based on hippocampal magnetic resonance imaging data, Alzheimer's & Dementia, 10.1016/j.jalz.2019.02.007, 15, 8, (1059-1070), (2019).
  • Liver stiffness and fibrosis-4 alone better predict liver events compared with aspartate aminotransferase to platelet ratio index in a cohort of human immunodeficiency virus and hepatitis C virus co-infected patients from ANRS CO13 HEPAVIH cohort, European Journal of Gastroenterology & Hepatology, 10.1097/MEG.0000000000001408, 31, 11, (1387-1396), (2019).
  • Assessment of the 2017 AHA/ACC/HRS Guideline Recommendations for Implantable Cardioverter-Defibrillator Implantation in Cardiac Sarcoidosis, Circulation: Arrhythmia and Electrophysiology, 10.1161/CIRCEP.119.007488, 12, 9, (2019).
  • A predictor of early disease recurrence in breast cancer patients using a cell-free RNA and protein liquid biopsy, Clinical Breast Cancer, 10.1016/j.clbc.2019.07.003, (2019).
  • Predicting the Risk of Huntington’s Disease with Multiple Longitudinal Biomarkers, Journal of Huntington's Disease, 10.3233/JHD-190345, (1-10), (2019).
  • Prognostic assessment of breast carcinoma submitted to neoadjuvant chemotherapy with pathological non-complete response, BMC Cancer, 10.1186/s12885-019-5812-0, 19, 1, (2019).
  • Identification of practitioners at high risk of complaints to health profession regulators, BMC Health Services Research, 10.1186/s12913-019-4214-y, 19, 1, (2019).
  • Prognostic value of muscle measurement using the standardized phase of computed tomography in patients with advanced ovarian cancer, Nutrition, 10.1016/j.nut.2019.110642, (110642), (2019).
  • Predicting Islet Cell Autoimmunity and Type 1 Diabetes: An 8-Year TEDDY Study Progress Report, Diabetes Care, 10.2337/dc18-2282, 42, 6, (1051-1060), (2019).
  • Development and Validation of Risk Prediction Models, Principles and Practice of Clinical Trials, 10.1007/978-3-319-52677-5, (1-22), (2019).
  • Comparison of the 8th UICC staging system for esophageal and gastric cancers in Siewert type II junctional adenocarcinomas, European Journal of Surgical Oncology, 10.1016/j.ejso.2019.12.011, (2019).
  • Prediction and clinical utility of a contralateral breast cancer risk model, Breast Cancer Research, 10.1186/s13058-019-1221-1, 21, 1, (2019).
  • Discovery and validation of DNA methylation markers for overall survival prognosis in patients with thymic epithelial tumors, Clinical Epigenetics, 10.1186/s13148-019-0619-z, 11, 1, (2019).
  • Cardiovascular outcome of breast cancer patients with concomitant radiotherapy and chemotherapy: A 10-year multicenter cohort study, Journal of Cardiology, 10.1016/j.jjcc.2019.02.001, (2019).
  • Comparison of three lymph node staging schemes for predicting the outcome in patients with small bowel adenocarcinoma: A population-based cohort and international multicentre cohort study, EBioMedicine, 10.1016/j.ebiom.2019.02.043, (2019).
  • Are tumor size changes predictive of survival for checkpoint blockade based immunotherapy in metastatic melanoma?, Journal for ImmunoTherapy of Cancer, 10.1186/s40425-019-0513-4, 7, 1, (2019).
  • A lymph node metastasis‐related protein‐coding genes combining with long noncoding RNA signature for breast cancer survival prediction, Journal of Cellular Physiology, 10.1002/jcp.28600, 234, 11, (20036-20045), (2019).
  • Prediction of incident heart failure by serum amino‐terminal pro‐B‐type natriuretic peptide level in a community‐based cohort, European Journal of Heart Failure, 10.1002/ejhf.1381, 21, 4, (449-459), (2019).
  • Age‐specific diastolic dysfunction improves prediction of symptomatic heart failure by Stage B heart failure, ESC Heart Failure, 10.1002/ehf2.12449, 6, 4, (747-757), (2019).
  • Highly sensitive chimerism detection in blood is associated with increased risk of relapse after allogeneic hematopoietic cell transplantation in childhood leukemia, Pediatric Transplantation, 10.1111/petr.13549, 23, 7, (2019).
  • Joint Shock/Death Risk Prediction Model for Patients Considering Implantable Cardioverter-Defibrillators, Circulation: Cardiovascular Quality and Outcomes, 10.1161/CIRCOUTCOMES.119.005675, 12, 8, (2019).
  • Prediction of lung cancer risk at follow-up screening with low-dose CT: a training and validation study of a deep learning method, The Lancet Digital Health, 10.1016/S2589-7500(19)30159-1, (2019).
  • Identification of a RNA-seq-based signature to improve prognostics for uterine sarcoma, Gynecologic Oncology, 10.1016/j.ygyno.2019.08.033, (2019).
  • Plasma Level of Interferon-γ Predicts the Prognosis in Patients With New-Onset Atrial Fibrillation, Heart, Lung and Circulation, 10.1016/j.hlc.2019.11.004, (2019).
  • Growth differentiation factor-15 and all-cause mortality in patients with suspected myocardial infarction, International Journal of Cardiology, 10.1016/j.ijcard.2019.04.088, (2019).
  • One- to 10-year Status Epilepticus Mortality (SEM) score after 30 days of hospital discharge: development and validation using competing risks analysis, BMC Neurology, 10.1186/s12883-019-1540-y, 19, 1, (2019).
  • Prognostic model for multiple myeloma progression integrating gene expression and clinical features, GigaScience, 10.1093/gigascience/giz153, 8, 12, (2019).
  • Predictors of clinical or cerebral lesion progression in adult moyamoya angiopathy, Neurology, 10.1212/WNL.0000000000007819, 93, 4, (e388-e397), (2019).
  • Development of a prognostic score for recommended TACE candidates with hepatocellular carcinoma: a multicentre observational study, Journal of Hepatology, 10.1016/j.jhep.2019.01.013, (2019).
  • Association Between Tumor Size Kinetics and Survival in Patients With Urothelial Carcinoma Treated With Atezolizumab: Implication for Patient Follow‐Up, Clinical Pharmacology & Therapeutics, 10.1002/cpt.1450, 106, 4, (810-820), (2019).
  • A predictive model of overall survival in patients with metastatic castration-resistant prostate cancer, F1000Research, 10.12688/f1000research.8192.2, 5, (2674), (2019).
  • Threshold body mass index and sex-specific waist circumference for increased risk of heart failure with preserved ejection fraction, European Journal of Preventive Cardiology, 10.1177/2047487319851298, (204748731985129), (2019).
  • See more

The full text of this article hosted at iucr.org is unavailable due to technical difficulties.