Our decision to use a continuous scale rather than classes for the different models resulted from the recognition of the increasing variety and overlap between different model types and philosophies. Maybe it is helpful to know that the original idea was to write two papers, one discussing ‘process-based’ models and the other discussing ‘correlative’ species distribution models (SDMs). Discussion among the authors revealed that current SDMs increasingly include components of both approaches. Thus, the decision to discuss SDMs on a continuum between ‘forward process-based’ and ‘correlative’ should not be viewed as a neglect of differences among models, as implied by Kriticos et al., but rather as an acknowledgement of their variety.
The reason we looked at the correlation–process continuum in particular is that the lack of process representation in SDMs has created controversy, for example regarding the interpretation of niche information derived from SDMs, aspects of which Kriticos et al. (2013) touch upon in their introduction as well. A key message of our paper is that no method is intrinsically superior to another and that some problems and assumptions are shared by both ends of the continuum as well as by the stages in between (e.g. hybrid models). All in all, we agree with Kriticos et al. on many critical points they mention, and we are grateful to them for reinforcing some of our points by their correspondence.
Kriticos et al. (2013) express four main points of criticism, which we would like to respond to in more detail.
1. ‘It is misleading to refer to the range of methods as a continuum when there are marked and important distinctions between the available methods and models…’
As an alternative, Kriticos et al. (2013) propose consideration of multiple axes and a division into three clusters: (1) process-based, (2) correlative, and (3) fitted process-based. We maintain that a continuum can accommodate more nuances and better represent novel modelling approaches, while a three-category system is likely to yield borderline cases. In addition, an initial organization of models along a continuous axis lends itself to subsequent classification by drawing lines across this axis if one so desires. We do, however, agree with Kriticos et al. that it may be helpful to consider a continuum along more than one axis. In our paper we concentrate on the axis of explicit process representation, but we also consider the different extents to which modelling approaches rely on species distribution data as model input. It might be helpful to explicitly separate these axes and look at possible correlations between them.
In this context, we would like to clarify that by referring to ‘fitted process-based models’ we mean process-based models in which some parameters are estimated inversely from at least a subset of the data they aim to predict, i.e. species distributions (Hartig et al., 2012). By ‘hybrid models’, on the other hand, we mean correlative models (describing the environmental niche) that additionally incorporate various processes such as dispersal, demography or competition to better describe the realized species distribution. Hybrid models, so defined, would not settle easily into the three categories proposed by Kriticos et al. This underlines the fundamental shortcoming of any categorical classification, namely that borderline cases emerge as a consequence of predefined classification boundaries. Regarding the illustration in Figure 1 of Dormann et al. (2012), we wish to stress that the partial overlap between the text ‘Hybrid models’ and the text ‘Fitted process-based models’ is not to be interpreted in the sense that these models share the same methodologies. It just represents the flexibility of both approaches regarding the extent to which the outcome is determined by independent ecological knowledge or by fitting to species distribution data.
2. ‘Dormann et al. (2012) present all correlative models as a single group. By failing to draw attention to the significant differences between correlative modelling methods, they miss the opportunity to show modellers that such differences may be critical to the success of their study.’
In our paper, we contrasted models that directly relate environmental variables to the presence or absence of species (‘correlative’) to those where the distributions emerge based on implemented processes (‘process-based’). The various statistical approaches used to fit the parameters of correlative models are irrelevant for our comparison. Of course, there are different methodologies, giving rise to associated advantages and pitfalls, but this is true for process-based approaches as well. We discuss purely correlative models and purely forward process-based models as the end members of a ‘continuum’, suggesting that there are no easily defined distinct groups and that most actual models fall somewhere in between groups and end members. We maintain that, given the focus of our paper, a detailed analysis of differences within a particular group of models is beyond our scope and in fact can be found in existing literature, as confirmed by the references in Kriticos et al. (2013).
3. ‘The Dormann et al. (2012) review misrepresents the methodology and conceptual underpinnings of so-called “fitted process-based models”.’
Kriticos et al. (2013) allege that we wrongly treated fitted process-based models ‘as having the same set of traits as correlative methods’ by placing them ‘between the extremes of correlative methods on the one hand, and so-called “forward process-based” methods on the other’. They claim that fitted process-based models have the unique advantage of being able to draw on the strengths of both correlative and process-based approaches. Of course, combining the strengths of the two approaches is the commonly expressed desire; however, we also wish to emphasize the possible consequence of combining their weaknesses or creating new problems, which is much less commonly discussed in species distribution modelling.
A process-based model whose parameters are fitted to species distribution data (or that is supplemented by range-limiting functions that are derived from such data) does share some of the same assumptions, data requirements, validation issues and sources of uncertainty as a correlative model. Hence it inherits some of the associated weaknesses. This was discussed at length in the respective sections of Dormann et al. (2012).
To give another example: if we impose a mechanistic dispersal kernel onto a correlative habitat suitability map in a hybrid model, we are faced with the problem that the effects of dispersal limitation could, to some extent, already be included in the habitat suitability map, which, after all, was fitted to reproduce the observed distribution. By adding a dispersal model on top, we may include the same effect twice and additionally incur scale- and resolution-dependent errors arising from the dispersal kernel (Bocedi et al., 2012). Thus, the combination of correlative and process-based models may not only propagate previously existing problems, but also create new problems that were not present in the individual approaches.
With regard to the alleged advantages of fitted process-based models, we would like to reiterate our point that the fitting of process-based models is often done not by strategic design, but by necessity. Process-based models generally contain a large number of parameters, some of which are difficult to measure or estimate a priori (see e.g. Hartig et al., 2012). Therefore, in practice, most process-based modelling studies work with a mix of parameters that are measured directly and parameters that are calibrated to reproduce observations. Process-based models, in particular those that ‘evolved’ over a number of years, have a legacy of tinkering with parameters. Randall & Wielicki (1997, p. 404) put it bluntly: ‘Tuning is bad empiricism. Calibration is bad empiricism with a bag over its head. The problem with tuning is that it artificially prevents a model from producing a bad result. As discussed above, the most scientifically valuable thing that can come out of a comparison of measurements with model results is to show that the model has failed.’
Tuning is inevitable in many cases, but it is important to take into account its consequences for model validation. Fitted process-based models may create an illusion of predictive power by reference to their mechanistic underpinning, but if the process-based model structure and independent ecological knowledge do not sufficiently constrain potential outcomes, fitting the model parameters to observed species distributions may produce drawbacks in terms of transferability and extrapolation that are similar to those in purely correlative models. Therefore, we maintain that fitted process-based models lie somewhere in between completely correlative and completely forward process-based models.
4. ‘The literature presented in the Dormann et al. (2012) review and their Appendix S1 is unfortunately incomplete and biased…’
In an attempt to avoid bias, we invited a diverse range of co-authors to contribute, who themselves have published on purely correlative, fitted process-based and forward process-based SDMs. Nevertheless, the combination of space limitations and the range of viewpoints means that it will be impossible to satisfy everyone regarding the completeness of our literature review. As for the claim of Kriticos et al. (2013) that the CLIMEX manual lists 160 papers involving transferability, we wish to clarify that by ‘transferability’ we mean ‘usefully applicable in other settings’, not ‘applied in other settings’. The fact that someone uses a hammer to drive a screw into a wall does not mean that a hammer is applicable for screwing. We checked the first 20 papers cited in the CLIMEX manual that we could easily access and that did not have any authors in common, and found that all those studies were applications of CLIMEX without systematic validation. We are not claiming that there are no studies that transfer SDMs in time, space or to other species, but that the vast majority of these studies only present predictions without validation and hence no proof of their applicability. The point of the appendix was not to cite every paper that uses a particular model, but to cite an illustrative range of papers developing different kinds of SDMs and to give an overview of existing approaches to the reader. We are grateful for the addition of the CLIMEX papers by Kriticos et al., but we do not believe that our omission of these papers made our review biased.
In closing, we would like to thank Kriticos et al. for their interest in our paper and for fuelling a fruitful discussion. After all, we all agree that ‘the best way to improve modelling methods is an open, honest discussion of their different strengths and weaknesses’. In our paper we attempted to encourage such a discussion and we hope that it will continue from here.