1Evaluating the distribution of species richness where biodiversity is high but has been insufficiently sampled is not an easy task. Species distribution modelling has become a useful approach for predicting their ranges, based on the relationships between species records and environmental variables. Overlapping predictions of individual distributions could be a useful strategy for obtaining estimates of species richness and composition in a region, but these estimates should be evaluated using a proper validation process, which compares the predicted richness values and composition with accurate data from independent sources.
2In this study, we propose a simple approach to estimate model performance for several distributional predictions generated simultaneously. This approach is particularly suitable when species distribution modelling techniques that require only presence data are used.
3The individual distributions for the 370 known amphibian species of Mexico were predicted using maxent to model data on their known presence (66 113 presence-only records). Distributions were subsequently overlapped to obtain a prediction of species richness. Accuracy was assessed by comparing the overall species richness values predicted for the region with observed and predicted values from 118 well-surveyed sites, each with an area of c. 100 km2, which were identified using species accumulation curves and nonparametric estimators.
4The derived models revealed a remarkable heterogeneity of species richness across the country, provided information about species composition per site and allowed us to obtain a measure of the spatial distribution of prediction errors. Examining the magnitude and location of model inaccuracies, as well as separately assessing errors of both commission and omission, highlights the inaccuracy of the predictions of species distribution models and the need to provide measures of uncertainty along with the model results.
5The combination of a species distribution modelling method like maxent and species richness estimators offers a useful tool for identifying when the overall pattern provided by all model predictions might be representing the geographical patterns of species richness and composition, regardless of the particular quality or accuracy of the predictions for each individual species.
The estimation of the geographical distribution of species richness distribution is necessary to understand the ecological and evolutionary determinants of spatial patterns of biodiversity (Rosenzweig 1995; Ricklefs 2004). Additionally, the identification of distribution patterns for species facing conservation problems is increasingly urgent. To detect species-rich areas and those where geographically restricted species converge can help us to develop effective conservation strategies, and to make better use of the human and economic resources that are available (Bojórquez-Tapia et al. 1995). However, it is not easy to evaluate species richness distributions across large, heterogeneous regions where biodiversity is high but collecting efforts have not been sufficient or adequately planned (Hortal, Lobo & Jiménez-Valverde 2007).
Recently, the species distribution modelling has emerged as a novel approach for generating probable distributions based on the relationship between species’ presence or abundance records and environmental variables (Araújo & Guisan 2006). A sizeable body of species-occurrence data and environmental data bases are available on the web, and several modelling techniques are also available as free software. Thus, modelling species distribution could become a valuable tool for basic studies as well as for conservation purposes, particularly for poorly sampled regions where biodiversity is high (Soberón 1999; Anderson, Gómez-Laverde & Peterson 2002).
Modelling methods attempt to predict the probability of occurrence or environmental suitability for the species as a function of a set of selected environmental variables. Geographical areas that, in theory, meet the environmental conditions required by the species represent its potential distribution, while the areas actually inhabited by the species represent its realized distribution (Soberón & Peterson 2005; Phillips, Anderson & Schapire 2006; Soberón 2007; Jiménez-Valverde, Lobo & Hortal 2008). To model the realized distribution of species, reliable absence data as well as variables representing the role of historical, biotic or dispersal limitation factors are required (see Chefaoui & Lobo 2008; Lobo 2008a). Given that such factors are difficult to include in the modelling process and good information for species absence is often lacking, predictions mostly delimit a broad potential range of distribution (the places where a species could be present according to environmental conditions). Thus, obtaining a prediction of the distribution of a species that is close to its realized distribution requires excluding the zones from which the species is known or can be inferred to be absent (according to expert opinion or by directly eliminating unsuitable habitats; see Ortega-Huerta & Peterson 2004; Soberón & Peterson 2005). Unfortunately, this removal process is risky and subjective, especially when the information on the biological requirements of the species is poor or unavailable.
The bias and overestimation of each prediction of individual distribution could be magnified by overlapping many individual models to obtain a prediction of species richness distribution over a region (see Hortal & Lobo 2006). Some studies overlap predictions of individual distribution and obtain estimates of species richness (Cumming, 2000; Ortega-Huerta & Peterson 2004; García 2006), but they do not carry out a reliable evaluation and calibration procedure to compare resulting species richness values with those from other, independent sources. Our aim in this study is to propose a methodological approach to evaluate how well the overlaying of individual model predictions represents the geographical patterns of species richness and composition. To do this, we modelled the distribution of all the amphibian species present in Mexico and made several species richness predictions. Subsequently, with the aim of selecting the most accurate prediction, we compared the resulted values with the observed and predicted species richness values obtained from accumulation curves and nonparametric estimators (Colwell & Coddington 1994) of well-surveyed sites. Finally, we obtained a measure of spatial distribution of prediction errors and analysed the similarity between observed and predicted compositions of the well-surveyed areas. Such an assessment allows us to both validate the results of multiple individual geographical predictions, and reduce the frequent overestimation of such models in a more objective manner. In this way, we will be able to determine when the results of accumulating individual model predictions are accurate enough and how to make these models valuable to predict descriptors of biodiversity.
Maps predicting habitat suitability across Mexico for 348 of the 370 species were generated using maxent Species Distribution Modelling software version 2·3 (Phillips, Dudík & Schapire 2006). The remaining 22 species were not modelled because they occurred in just one locality with one or two records. In these cases, we assumed that these species are present only in the cells where they were recorded. maxent method was selected because it is considered one of the most accurate when only presence information is available, and it outperforms other, more conventional methods that use presence–absence information (Elith et al. 2006). In brief, maxent uses a deterministic algorithm able to find an optimal probability distribution based on a set of environmental constraints (see Phillips et al. 2006; Phillips & Dudík 2008 for a detailed explanation on the method). Individual maps were made by relating observed presence data to the 19 bioclimatic variables from the worldclim data base version 1·3 (Hijmans, Cameron & Parra 2005): annual mean temperature, annual precipitation, isothermality, maximum temperature of warmest month, mean diurnal range, mean temperature of coldest quarter, mean temperature of warmest quarter, mean temperature of wettest quarter, minimum temperature of coldest month, annual temperature range, temperature seasonality, precipitation of coldest quarter, precipitation of driest month, precipitation of driest quarter, precipitation of warmest quarter, precipitation of wettest month, precipitation of wettest quarter and precipitation seasonality. The resolution of the climate layers used in the analyses was 5 min (i.e. 0·083°, or squares of c. 10 × 10 km). The maps were subsequently exported to obtain a matrix of suitability values for each species in every one of the 24 997 cells or sites of c. 10 × 10 km that cover the whole of Mexico. As suitability values calculated by maxent range from 0 to 100, it is necessary to set a threshold for converting these continuous values to binary ones (presence/absence) for each species, and subsequently overlapping all individual models to derive a possible species richness. Initially, we set 21 thresholds selected from 1 to 100 at intervals of five to obtain the respective presence–absence maps for each species; we assumed presence in the cells with suitability scores equal to or higher than 1, 5, 10 ... and 100 (hereafter T1, T5, T10 ... and T100). Thus, after overlapping all individual models according to the threshold used, we obtained 21 possible scenarios of species richness distribution, whose values would be subsequently compared with observed values of well-surveyed sites to select the threshold that represents better the patterns in species richness.
selecting the most appropriate representation of species richness
Well-surveyed sites can be recognized by parametric and/or nonparametric species richness estimators (Colwell & Coddington 1994). We identified well-surveyed cells using three different and complementary methods: nonparametric estimators based in the number of rare species (Chao 2 and Jackknife 1; see Hortal, Borges & Gaspar 2006), the final slope of the accumulation function describing the cumulative rise in the number of species as the sampling effort increases (Hortal & Lobo 2005), and the number of species predicted at the 95% upper confidence interval of the accumulation curves produced with the Mao Tau analytical function (Chang, Colwell & Chang 2005). Mao Tau, Chao 2 and Jackknife 1 species richness estimates were obtained with estimates 7·5 (Colwell 2005). In every case, the number of data base records was used as a surrogate of sampling effort (Hortal & Lobo 2005; Hortal et al. 2006; Lobo 2008b). Well-surveyed cells are those which have completeness values higher than 75% and for which the final slope of the accumulation curve is lower than 0·1 (see Hortal & Lobo 2005). Completeness was calculated by relating the maximum species richness value predicted by any of the three formerly mentioned estimators (Mao Tau, Chao 2 and Jackknife 1) and the observed richness (observed/predicted × 100). These calculations revealed that 118 cells were well surveyed (see Fig. 1b).
The observed (SOBS) and estimated (SMAX) richness values, as well as the similarity between the observed and predicted species composition in the well-surveyed sites were used to assess the accuracy of the predictions derived from overlaying individual distribution models. SMAX was considered as the maximum species richness estimated by any of the three previously mentioned estimators. To select the option that represents better the patterns in species richness and composition among the different scenarios produced by applying the 21 thresholds, we carried out comparisons of the SOBS or SMAX values against the predicted species richness values from the overlay of individual predictions (ST1, ST5, ST10,ST15 ... ST100). The Spearman rank correlation coefficient was used for these comparisons, and the species richness representation with the highest correlation result was selected as that which best represented species richness. In addition, both the rate of prediction errors [(predicted-observed)/observed × 100] and the number omission and commission errors were detected for each of the 118 well-surveyed cells. Setting a threshold on predicted suitability values as was carried out in this study can reduce overestimation, but it will also inevitably generate errors of omission because certain species will be predicted as absent when they are, in fact, present. When the objective is obtaining a reliable estimation of species richness variation for conservation purposes, the relative cost of underestimation is higher than that of overestimation (Fielding & Bell 1997; Rondinini et al. 2006). Due to this, the resulting representation of species richness was adjusted to obtain lower levels of underestimation only for those species erroneously predicted as absent in the cells with a low number of species. Thus, we identified all species that had been observed in areas where their species richness values were predicted as zero. After identifying these species, the lowest non-zero predicted value associated with an observed presence record in these cells was used as their new threshold. Since omission and commission errors are inversely related, we also limited the influence of commission errors by changing the threshold of those species that were erroneously included in three or more well-surveyed cells. For these species, we selected the next threshold value above that at which they had been previously found. Subsequently, this new representation of species richness obtained from the overlay of individual predictions was again compared to SOBS or SMAX species richness values for the well-surveyed cells, examining the general rate of prediction errors, omission and commission errors as well as prediction errors per species. In order to determine whether these errors are spatially structured, their values were regressed against the nine terms of a third-degree polynomial where latitude and longitude are the explanatory variables (trend surface analysis; see Legendre & Legendre 1998). Latitude and longitude were standardized and a backward stepwise regression carried out, removing nonsignificant spatial terms. If commission or omission errors could be accounted for by spatial variables, then the accuracy of the model results would vary according to geographical location (Hortal et al. 2007). Finally, to analyse the similarity between the observed and predicted species composition in the 118 well-surveyed cells, we calculated the inverse of the beta-Simpson's index, which focuses on compositional differences independently of species richness gradients (Koleff, Gaston & Lennon 2003). The values of this dissimilarity index range from 0 when two species groups have identical composition, to 1 when the composition of the two groups is completely different.
sampling effort distribution
The available distribution information for the 370 Mexican amphibian species is highly fragmentary and biased (see Fig. 1a). From the 24 997 cells that make up the country, only 3736 cells (15%) had one or more data base records. The mean (± SD) number of data base records per cell is 2·6 ± 25·3 and the mean number of recorded species is 0·5 ± 1·9. The cells with the greatest number of data base records have 2029, 1238 and 1032 records, respectively, while 1028 cells have only one data base record (see Fig. 2). In total, 23 911 cells (95·6%) have fewer than 10 data base records, and a similar number of cells (23 841; 95·4%) have three or fewer observed species.
Data base records and observed species richness values are positively and significantly correlated (Spearman rank correlation coefficient, rs = 0·824; P < 0·001), so the observed distribution of amphibian species richness seems to be highly conditioned by historical sampling effort. Moreover, both variables show a neat geographical distributional pattern (Fig. 1a) in which the greatest survey effort is distributed along the central and eastern mountain ranges, as well as in some mountainous areas of the southern states of Oaxaca and Chiapas. In contrast, most of the central part of the northern region has been poorly surveyed and there are some smaller gaps on both peninsulas: Baja California (in the northwest) and the Yucatán (in the southeast), as well as on the southern mountainous region of the Pacific slope. These gaps indicate areas that have been poorly surveyed or that are completely unsurveyed. Finally, there has been a moderate surveying effort in other regions of the country, particularly the central and northern areas of the Pacific slope and the north of the Yucatán. The spatial structure of the survey effort across the country, as well as the wide-ranging number of data base records among cells show the colossal degree of survey effort still necessary to obtain a relatively reliable estimate of the amphibian species richness patterns in Mexico.
The 118 cells considered as relatively well surveyed were distributed throughout the country (Fig. 1b), mainly in the central and eastern regions, as well as in the north-western region and the state of Chiapas in the south. The observed species richness values in these well-surveyed cells (SOBS) ranged from 3 to 39 species, while the maximum estimated species richness (SMAX) was from 3 to 51 species. However, there are regions that lack well-surveyed sites in the central part of the northern region, on the Baja California Peninsula and most of the Pacific slope.
select the best representation of species richness
The correlation between SOBS and SMAX for species richness values from well-surveyed cells and species richness values derived from the overlay of individual models follows a similar pattern depending on the threshold selected (Fig. 3). Correlation values are highest when predicted species numbers occur at thresholds between 50 (rs = 0·821) and 75 (rs = 0·819), where representation at the threshold of 70 had the highest correlation value (rs = 0·852) with a mean prediction error of 28·1% ± 24·5% (SD). The mean number of species erroneously included in each cell was 4·4 ± 4·2, while the mean number of species omitted was 3·7 ± 2·9 (relative commission error = 31% ± 29%; relative omission error = 26% ± 16%). This representation (ST70) was the one we pre-selected, in which species richness ranged from 0 to 47 species. In 14 701 cells (59% of the country), the occurrence of one or more species was predicted, and there were notable gaps in the northern region and some parts of the Baja California and Yucatán Peninsulas (Fig. 4a).
When this pre-selected representation of species richness was examined more closely, 778 data base records belonging to 78 species (21% of the total) were detected inside the area for which predicted species richness values were zero (see Fig. 4b), while 98 data base records belonging to 21 species (6% of the total) were detected as commission errors in three or more well-surveyed cells. New thresholds were assigned for these 99 species (78 + 21) and the individual geographical representations obtained were then replaced to get new correlations between observed and predicted species richness values. The prediction derived from a threshold of 70 again had the highest correlation both with SOBS observed values (rs = 0·873) and with SMAX (rs = 0·874). The mean proportion of errors (± SD) varied from 33·2% ± 32·3% to 24·2% ± 19·9% for SOBS and SMAX, respectively. Species richness ranged from 0 to 45 showing a remarkable heterogeneity in the distribution of this variable across the country (see Fig. 4c) and the occurrence of one or more species was predicted in 24 298 cells (97% of the country). As expected, both the number of species erroneously included and the relative percentage of commission errors increased (5·9 ± 4·1 and 46% ± 33%, respectively), but the number of omitted species and the relative percentage of omission errors decreased (2·6 ± 2·5 and 17% ± 13%, respectively).
In the backward stepwise selection of the nine terms of the third-degree polynomial for latitude and longitude against the percentage of commission errors, only the linear term of longitude [F(1,116) = 7·11, P = 0·009] was statistically significant, accounting for 6% of the total variability. The two terms of the quadratic function of longitude were the only significant ones with respect to errors of omission [F(2,115) = 5·01, P = 0·008], explaining about 8% of total variability. Thus, there is an increase in commission errors towards the east while omission errors tend to occur more frequently towards the centre of the country.
The prediction errors per species (Fig. 5a) show that more than half of the species were erroneously predicted as absent only in less than 10% of the well-surveyed cells. However, the rates of omission error were yet high: around 12% of the species were predicted to be absent when they were present in more than 50% of the well-sampled cells. With regard to commission errors, 30% of the species are erroneously predicted as present in more than half of the well-sampled cells (Fig 5a). The mean (± SD) dissimilarity between observed and predicted compositions measured by the inverse of the beta-Simpson index is 0·155 ± 0·124 for the well-surveyed cells, ranging from 0 (in 23 cells) to 0·57 (one cell). Two-thirds of the cells have values lower than 0·2 (Fig. 5b), suggesting a high similarity between observed and predicted composition, while only a 12% of the cells have dissimilarity values higher than 0·3.
The complementary use of a distribution modelling method like maxent and species richness estimators allows us to produce a geographical representation of the amphibian species richness distribution and its species composition per locality over a territory like Mexico and, most importantly, a measure of its accuracy. However, a representation obtained this way has biases and limitations. The methodological approach we propose in this study, which is related to one of the strategies to predict biodiversity at the level of assemblage, defined by Ferrier & Guisan (2006) as ‘predict first, assemble later’, allow us to identify when the overall pattern provided by all individual model predictions might be representing the geographical patterns of species richness and composition, regardless of the particular accuracy of the predictions for each individual species.
The accurate description of a dynamic phenomenon such as species richness distribution is very difficult. As shown, the results of species distribution models are an unreliable description of the species ranges. Rather, they provide a provisional description of these ranges, that should be continuously updated when new data are available or environmental conditions change. Species distributions predicted by relating biological data to environmental variables generally tend to overestimate the actual extent of the ranges due, in part, to using only the environmental conditions of the sites where the species has been recorded as model predictors. If the absences generated by historical, dispersal limitation or biotic factors (Ricklefs & Schluter 1993; Hanski 1998; Pulliam 2000) are not taken into account, model predictions inevitably tend to approach the potential distribution of species (i.e. the environmentally suitable sites where a species could occur according to a group of environmental variables; see Soberón & Peterson 2005; Soberón 2007; Chefaoui & Lobo 2008; Lobo 2008a; Jiménez-Valverde et al. 2008). Consequently, when predicted distribution maps are overlaid to build a representation of species richness across a territory, a set of errors and biases will accumulate, resulting in an unreliable representation of reality (Hortal & Lobo 2006). In our study, we overlaid 348 individual amphibian species predictions plus 22 directly created individual distributions and suggest a method for correcting overestimates that also allows us to obtain both a measure of the uncertainty of the results produced by the model, and a spatial representation of the distribution of model errors.
Selecting a uniform threshold value for the results of all individual models is an initial step to develop a hypothesis of species richness. In this study, an initial cut-off at a suitability value of 70 appeared to be the most appropriate for predicting both species richness and species composition (see also Guisan & Theurillat 2000). Other unpublished modelling exercises following the same procedure provide different suitability threshold values depending of the taxonomic group being studied and the area and characteristics of the target region. Thus, our method provides reliable criteria for decision-making, and selecting potential species richness distribution simulations without resorting to subjective choices (Gioia & Pigott 2000). However, obtaining an appropriate representation of species richness requires a deeper analysis of the results obtained to detect areas with a notable level of omission errors, and also those species whose records were located in areas where zero species richness was predicted. Comparing the initial representation of species richness distribution obtained with model predictions with the observed one and the analysis of errors are the next steps for adjusting the predicted distributions of a subset of species, thereby improving the picture of species richness. Reducing errors of omission or commission can be achieved by prioritizing either sensitivity or specificity (Fielding & Bell 1997). Model accuracy must be always interpreted according to the intended purpose of the study (Araújo & Guisan 2006), with the differential weighting of false-positives and false-negatives. In this study, the cost of leaving out observed species was assumed to be higher, so we have tried to minimize omission errors. However, from a conservation perspective, both commission and omission errors are important (Rondinini et al. 2006); ignoring the existence of a species when it is actually present leads to the proposal of networks of conservation areas that are smaller than required, while the erroneous inclusion of species in some localities can result in the selection of redundant or ineffective reserves. We opted for reducing the occurrence of omission errors but only for those cells in which the predicted species richness were zero, in order to provide a better description of the distribution of species richness. A different strategy could be followed depending on whether it is desirable to reduce errors of commission or omission.
After including the individual distributions derived from the modified thresholds, our new representation seemed to be more reliable and to reveal a more reasonable distribution of amphibian species richness in Mexico. However, the next step is to analyse the biases and inaccuracies arising from the model, not only for predicting species richness across the country, but also for predicting species composition: the species richness of a cell can be accurately predicted, but predicted species composition might be quite different from reality. This way it is possible to detect the error and uncertainty originating from the model and its spatial prediction (Guisan et al. 2006). Our results show that, on average, there is still a 33% prediction error in our model, mainly due to the addition of about six species per cell. Thus, despite manipulating the model to minimize omission errors, more than 10% of the species were erroneously predicted as absent in half of the cells. Furthermore, the prediction errors seem to show a spatial trend and in c. 12% of the well-surveyed cells, the predicted composition is highly dissimilar from the observed composition. We propose that this type of analysis should be always applied because model assessment must include not only a measure of the magnitude of the error, but also of its location within the territory studied (Fielding & Bell 1997; Fielding 2002). Thus, model representations such as those provided in this study can be considered helpful approaches for associating a specific measure of uncertainty with the representation of species richness patterns derived from this kind of distribution modelling technique. The errors detected may result from many different factors; the most likely being those related to the geographical and environmental coverage of the biological data (Hortal et al. 2007), along with the lack of reliable absence data that would represent environmentally favourable but inhabited sites (Jiménez-Valverde et al. 2008). To decrease the level of error in the future, new distributional and taxonomical data obtained by traditional survey methods are required.
Julián Bueno helped collect and georeference database records and Pablo Sastre helped with GIS management. Bianca Delfosse revised the English text of the manuscript and offered valuable suggestions. CONACYT-SEMARNAT Fund (project 23588) and CONABIO (project EE005/DE026/06) provided financial support for this research. E. Pineda thanks the Agencia Española de Cooperación Internacional for a postdoctoral grant. Two anonymous reviewers provided helpful suggestions that improved this manuscript.