We agree with the comments of Stephan et al. that the modelling type matters little, while the ability to predict with accuracy matters the most. This said, it is not surprising that various modelling techniques that are based on data-driven relationships might outperform one another. For example, a nomogram based on cubic splines (data-driven) can be as powerful in predicting the outcome of interest as are ANNs. Although it might be of interest to some to debate which model is most accurate, once the maximum accuracy has been reached and it is not 100% perfect, some clinicians might reject the option of model-derived predictions. For now, no model in urological oncology can provide 100% accurate predictions. Residual sources of error account for a misclassification rate of 15–25%, when for example biochemical recurrence (BCR) or other endpoints are examined. Should this error rate discourage clinicians from relying on multivariable predictor and prognostic tools? The answer is definitely not. Although predictive and prognostic tools are not perfect, their ability to foretell the outcome of interest exceeds that of expert clinicians substantially and statistically significantly. For example, clinician experts at Memorial Sloan-Kettering Cancer Center were 54% accurate in predicting lymph node metastases, vs 72% for nomogram-based predictions . Similarly, clinicians were, on average, 68% accurate in predicting the life-expectancy of patients with prostate cancer treated with either RP or radiotherapy, vs 84.3% for a nomogram predicting the same outcome [2,3]. Of various predictive models, nomograms recently emerged as the preferred format when the opinions of North American urologists were polled . Despite the established benefit of various multivariable models in predicting BCR and other prostate cancer endpoints, scepticism might be encountered when their implementation into routine clinical practice is suggested. Lack of a prospectively confirmed benefit in patient outcomes is commonly used as an argument against the use of predictive and prognostic models. At first glance, clinicians who are accustomed to randomized prospective trials that quantify the benefits and detriments of a standard of care to that of a novel molecule tend to agree with absence of data supporting the usefulness of nomograms in clinical practice. However, a randomized prospective evaluation of nomogram-based decision-making vs standard-of-care (clinician-based) decision-making is neither practically nor ethically possible. First, ethical considerations would not allow the randomization of patients to management that is purely based on information technology. Second, nomograms and other predictive or prognostic tools are merely meant to assist the clinician in decision-making. In that regard, they provide the clinician with a probability of a given endpoint, say BCR. When the nomogram states that the patient is 80% likely to have BCR after RP, the clinician might, for example, decide to (i) do nothing, (ii) increase the frequency of follow-up, (iii) initiate adjuvant radiotherapy, or (iv) start androgen deprivation. Although the nomogram-derived prediction of elevated risk of BCR prompts the clinician to select one of the four treatment options, it entirely depends on the clinician’s preference which of the choices is selected. It is therefore impossible to objectively test the effect of nomograms on health outcomes, as clinicians’ choices remain the decisive factors in diagnostic and/or therapeutic decision making.
Pierre I. Karakiewicz*, Umberto Capitanio♯, Hendrik Isbarn*, Claudio Jeldres* and Shahrokh S. Shariat†
*Cancer Prognostics and Health Outcomes Unit, University of Montreal, Montreal, Quebec, Canada;♯Department of Urology, Vita-Salute San Raffaele, Milan, Italy;†Department of Urology, Memorial Sloan-Kettering Cancer Center, New York, NY, USA