Question: Predictive models constitute an important tool in multiple ecological applications. In this paper, we examine and compare the performance of six state-of-the-art methods commonly used in ecological modelling: Multiple Logistic Regression (MLR), Artificial Neural Networks (ANN), Support Vector Machines (SVM), Classification and Regression Trees (CART), Maximum Entropy (MAXENT) and Multivariate Adaptive Regression Splines (MARS).
Location: Northern Spain.
Methods: We used presence/absence data of 15 plant species of an alpine rangeland in northern Spain and 14 topographical and geomorphological descriptors to build the models. We used leave-one-out cross validation on each model and computed the area under the receiver operating characteristic (ROC) curve (AUC) and the resolution and calibration diagrams of the resulting probabilistic predictions. We also analysed the binary presence/absence deterministic predictions and computed the corresponding confusion matrices to calculate sensitivity, specificity, Cohen's kappa and the True Skill Statistic (TSS).
Results: In general, CART and MAXENT showed poor performance and MLR was competitive with the more sophisticated ANN, MARS and SVM methods. The best predictive resolution was obtained, in most cases, by ANN followed by SVM and CART models; however, MLR and MARS were generally the best calibrated. The MAXENT predictions attained, in general, poor resolution and moderate to good calibration.
Conclusion: Assessment of model calibration and resolution, in addition to ROC and confusion matrix-derived indices, is an important step for model choice depending on the final aim. Most of the target species were accurately predicted showing that the variables used are suitable descriptors at the scale of analysis.