Ground validation of presence-only modelling with rare species: a case study on barbastelles Barbastella barbastellus (Chiroptera: Vespertilionidae)

Authors

  • Hugo Rebelo,

    Corresponding author
    1. School of Biological Sciences, University of Bristol, Woodland Road, Bristol BS8 1UG, UK
    2. CIBIO, Centro de Investigação em Biodiversidade e Recursos Genéticos, Instituto de Ciências Agrárias de Vairão, R. Padre Armando Quintas, 4485-661 Vairão, Portugal
      *Correspondence author. E-mail: hugo_reb@sapo.pt
    Search for more papers by this author
  • Gareth Jones

    1. School of Biological Sciences, University of Bristol, Woodland Road, Bristol BS8 1UG, UK
    Search for more papers by this author

*Correspondence author. E-mail: hugo_reb@sapo.pt

Summary

1. We evaluated the accuracy of presence-only modelling for predicting the distribution of rare species, when data are scarce and usually collected with sampling biases. We modelled the potential distribution in Portugal of one of the rarest European bats Barbastella barbastellus and subsequently ground-validated predictions by using acoustic transects.

2. We used ecological niche factor analysis (ENFA) and maximum entropy (Maxent) modelling to build distribution models of B. barbastellus, and determined which ecological factors were more relevant for each model. As ENFA only accepts continuous variables, we built one Maxent model using the same variables as ENFA and another using land cover as a categorical variable.

3. Ecological niche factor analysis and both Maxent models predicted similar areas of occurrence in central and northern regions of Portugal, although ENFA predicted suitable habitat over a wider range. Conversely, there was substantial disagreement on the location of high-suitability areas in the south. This could be a consequence of a different choice of important variables made by each model. Native woodland and average temperature were the most relevant variables for Maxent, while in ENFA B. barbastellus was linked to higher altitudes although avoiding production forests and infrastructures.

4. Threshold-independent and -dependent statistics showed that Maxent models outperformed ENFA, probably as a consequence of divergent predictions in the new areas of occurrence. Overall, 15 new B. barbastellus sites were discovered and known distribution was extended c. 100 km to the south.

5.Synthesis and applications. Our results support the use of presence-only modelling as an indispensible tool for survey design as shown by the discovery of B. barbastellus populations outside of the previously known range. ENFA seems to be more suited to determining a species’ potential distribution, although failing to extrapolate it. In contrast, Maxent is better suited to determining a species’ realized distribution. It was successful in predicting occurrence in previously unsurveyed areas and can be recommended as a technique for determination of a conservative distribution for a species. Maxent modelling would greatly aid biodiversity conservation, especially when it is necessary to develop survey plans or first assessments of a species’ distribution.

Introduction

Accurate knowledge of a species’ distribution is of prime interest for conservation management. Over the last two decades, several multivariate techniques have been developed to predict species’ distribution (e.g. Hirzel et al. 2002; Phillips, Anderson & Schapire 2006). For these methods, the data for the focal species are usually described in the form of presence/absence in a set of sampled locations. However, absences could include ‘false absences’, hence biasing the model’s predictions. False absences arise in situations where the species could not be detected even though it was present (Hirzel & Le Lay 2008). This situation is especially relevant when studying bat species, because intensive population surveys are difficult to conduct. Additionally, bats have a nocturnal and elusive behaviour with large home ranges and problems exist in identifying species in flight (Walsh & Harris 1996).

Presence-only modelling appears to be an alternative technique to address these constraints because it does not require the existence of absences in the data, despite the majority of these methods using background data as pseudo-absences for the calculations (e.g. Phillips et al. 2006).

In recent years several presence-only techniques have been developed, and their reliability has been tested for different sample sizes and spatial scales (Elith et al. 2006; Hernandez et al. 2006; Pearson et al. 2007; Tsoar et al. 2007; Wisz et al. 2008). These techniques are assuming a greater relevance for species conservation, especially for rare or poorly documented species, with some showing strong performance even with scarce presence data (Hernandez et al. 2006). Moreover, some of these techniques seem to be robust to non-systematic data (although a model’s best performance is achieved from systematic surveys) which is especially relevant when working with presence data collected by different researchers and methodologies (Elith et al. 2006), e.g. museum or herbarium collections (Hirzel et al. 2002). In fact, datasets are typically characterized by sampling biases where available data usually do not cover the whole range of ecological conditions in which the study species may occur (Wisz et al. 2008). The effect of sampling bias can have a severe impact on model quality; nevertheless it has received little attention in the predictive modelling literature (Hortal et al. 2008; Phillips et al. 2009). Indeed, it has been stressed that more research should focus on the effect of sampling biases on model predictions (Pearson et al. 2007; Elith & Graham 2009). In this study, we compared two presence-only modelling techniques, ecological niche factor analysis (ENFA) and maximum entropy (Maxent), regarding their efficacy in predicting new areas of occurrence for a rare species. These techniques differ in their modelling approaches with Maxent being a complex technique, establishing a flexible relationship between the dependent and independent variables, hence theoretically more suited to predict the realized distribution of a species (the locations and range of environmental conditions in which a species actually lives). ENFA is also a presence-only method, although it limits the shape of a species’ response to an ecogeographical variable (EGV) to a normal distribution, thus generally providing predictions closer to the potential distribution or the distribution a species could inhabit if it occupied all areas with favourable conditions (for a detailed discussion see, Jiménez-Valverde, Lobo & Hortal 2008a). To our knowledge, these techniques have never been compared despite becoming widely used.

There are several statistical methods to validate habitat-suitability models including Receiver Operating Characteristics (ROC) plots, the Boyce Index, Kappa statistics and null models (Fielding & Bell 1997; Liu et al. 2005; Hirzel et al. 2006). However, the majority of the model validation methods involve comparing subsets of the training data with the predictions generated by the models. Very often the models give new predicted areas of occurrence for the studied species, and these are clearly among the most relevant outputs of ecological modelling (Hernandez et al. 2006). However, new areas of predicted occurrence are rarely verified in the field (Greaves, Mathieu & Seddon 2006; Jiménez-Valverde et al. 2008b) which is of concern, especially for areas outside the known geographical range of the species (Elith et al. 2006; Randin et al. 2006). Such a lack of validation is highly relevant for rare species, which are usually a conservation priority, and for which the knowledge gap is generally wider than for more common species. We chose to study one of the rarest European bat species, the barbastelle Barbastella barbastellus Schreber, 1774. This tree-dwelling bat is widespread in Europe although its distribution is highly fragmented (Mitchell-Jones et al. 1999). It seems to depend upon native mature woodland, so deforestation and habitat fragmentation are probably associated with its suspected population decline (Russo et al. 2004). In this study, we modelled and tested B. barbastellus distribution in Portugal, where the species was first discovered in 1980 (Palmeirim 1990), and for which only a few locations were known by 2004. Two major bioclimatic zones exist in the country, the Atlantic and the Mediterranean (Sillero et al. 2009). Almost all known B. barbastellus locations were situated in the Atlantic bioclimatic area in the centre and north of the country. By modelling the distribution of this bat species in Portugal, it would be also possible to evaluate the accuracy of model projections in different bioclimatic zones and hence to test their behaviour under a sampling bias in the presence data. We believe that our study mirrors the limitations of working with rare species in general: data are likely to have geographical sampling biases, while predicting whether a species is likely to occur in areas outside the known range is one of the most important and desirable outcomes expected from distributional modelling (Randin et al. 2006). Hence, the main objectives of this study were: (i) to investigate both modelling techniques regarding their predictions and discrepancies for the distribution of B. barbastellus in Portugal; (ii) to determine which ecological factors are relevant for each model; and (iii) to validate these modelling techniques by comparing predicted distributions with ground-truthing results obtained by undertaking acoustic transects.

Materials and methods

The study was carried out in mainland Portugal (approximately between coordinates 36°N–41°N and 7°W–9°W). Two major climate zones divide the country. In the north and in the majority of the central parts of the country the Atlantic temperate climate dominates with mild summers and cold, rainy winters. The landscape is mountainous with native forests mainly composed of oaks (Quercus robur, Q. pyrenaica, Q. faginea), and other flora that constitute a typical European Atlantic forest. The southern regions are characterized by a Mediterranean climate with mild winters and hot, dry summers. As a consequence, the landscape is dominated by an oak savannah-like woodland (Quercus suber and Q. ilex) spreading over vast plains (Sillero et al. 2009).

Presence data and environmental variables

For model calculation, we used all known locations of B. barbastellus in Portugal since its discovery in 1980 up to 2004 (source Instituto da Conservação da Natureza e Biodiversidade; Fig. 1) as the dependent variable. Only records with a finer resolution than 30 arc s were considered for modelling (n = 17). Moreover, a set of independent EGV was selected as environmental predictors: annual average temperature (°C), annual average precipitation (mm), annual average temperature range (°C) and altitude (source WORLDCLIM; http://www.worldclim.org) and land cover (source Global Land Cover 2000; http://www-gvm.jrc.it/glc2000/ and Instituto Geográfico Português). This latter EGV was reclassified into six classes namely, infrastructures (towns and villages, industrial areas, roads, etc.), agriculture fields including steppes, production forests (mainly conifer and Eucalyptus spp.), scrubs and regenerating forest, native woodland and water bodies. Although, land cover data refer to recent years while presence data comes from as early as 1980, this was unlikely to affect modelling because areas where B. barbastellus was found remained fairly constant over time, i.e. those areas of mature woodland still remain intact. Regarding chosen climatic variables, we were limited by the low number of presences in the training data. Thus, we chose variables that are acknowledged to have a great relevance on bat physiology and survival: temperature and water availability (Baken & Kunz 1988), here expressed as two temperature variables and by precipitation. All digital information had a resolution of 30 arc s (∼900 m), thus the study area included 133 291 cells corresponding to an area of c. 90 100 km2.

Figure 1.

 Maps with the location of the training (left) and test (right) data. Squares on the training data map indicate the location of the occurrence records used to build the models. Each square may contain more than one location. The test map shows the location of the acoustic transects where empty circles represent absences (or unconfirmed presences) while circles with crosses within represent confirmed presences. Each symbol may contain more than one transect.

Modelling procedure

We used two recently developed presence-only modelling techniques to build habitat-suitability maps for B. barbastellus in Portugal, which involve different theoretical approaches (see Introduction) and which have not been tested through ground validation: ENFA and Maxent.

For computing ENFA models, Biomapper v4·0 was used (http://www2.unil.ch/biomapper/; Hirzel, Hausser & Perrin 2007). Because all land cover data were in a Boolean format, we derived quantitative variables from the original ones as this is a requirement for ENFA. Therefore, considering the home range (ca. 5 km) reported for this bat (Greenaway 2001; Hillen, Kiefer & Veith 2009; Kerth & Melber 2009) the frequency within a 2·5 km radius (proportion of cells with the attribute within that radius) was calculated for the following EGV: agriculture, production forest, scrubs and regenerating forest, native woodland and water bodies. We also calculated the distance between the focal cell and the cells where infrastructures were present. The modules CircAn and DistAn in the Biomapper software package were used to perform these operations (Hirzel et al. 2007). Altitude, precipitation and temperature variables were obtained directly as quantitative data. Prior to the development of the models, all EGV were normalized using the Box-Cox transformation (Sokal & Rohlf 1995) as available in the Biomapper software.

The ENFA then summarizes all ecological variables into a few uncorrelated factors in a similar way to principal component analysis. The number of factors to be included in the model was obtained by comparing each factor’s Eigenvalue with the MacArthur’s broken stick distribution (Hirzel et al. 2002). The correlation coefficients between the EGV and the extracted factors allowed the determination of which variables were relevant to explain species distribution. Subsequently, ENFA compared the cell values where a species occurred with cell values from all employed EGV. For computing the habitat the harmonic-mean was used because this algorithm seems to be more adequate for data from rare species (Hirzel et al. 2007).

Maxent is a machine-learning process employing a statistical mechanics approach that also uses presence-only data. Maxent estimates the range of a species by finding the Maxent distribution (i.e. closest to the uniform) given the constraint that the expected value for each EGV closely matches the empirical average of the occurrence data (Phillips et al. 2006). It differs from ENFA by this approach which makes it more suited to describe a species’ realized niche. To check which variables were the most important for model building, a Jackknife analysis of the gain (a statistics that measures how well an EGV distinguishes localities where species occurs from the total area under study) was made with the presence data. To test if the change of a categorical EGV to continuous had impact on Maxent’s performance we calculated two models. One used exactly the same continuous variables as the ones employed in ENFA (hereafter this model will be called Max_cont). Because categorical variables can be used in this algorithm, another model was calculated using land cover in a binomial format (hereafter this model is called Max_cat). All calculations were performed in Maximum Entropy Species Distribution Modelling v3·2·19 (http://www.cs.princeton.edu/~schapire/maxent/).

Acoustic transects and sound analysis

To ground validate the habitat-suitability models, acoustic surveys were conducted consisting of a 30 min transect walked at a regular pace using an ultra-sound detector (D-240×; Pettersson Elektronik AB, Uppsala, Sweden), so that around 1 km was covered. Transect surveys started at civil twilight after sunset and lasted for the following 2 h and 30 min to match the peak activity period of B. barbastellus (Russo, Cistrone & Jones 2007). To avoid auto-correlation between transects on the same night, surveys were performed at least 5 km from each other. The average B. barbastellus home range has been reported to be up to 5 km (Greenaway 2001; Hillen et al. 2009; Kerth & Melber 2009), hence with this distance between transects it was highly unlikely that the same individual would be recorded in different areas. Whenever weather conditions were adverse for bat activity, such as strong winds or rain, transects were cancelled (Russo & Jones 2003).

Surveys were performed with the heterodyne output of the detector tuned between 32 and 34 kHz to match the frequencies of maximum energy emitted by B. barbastellus (Russo & Jones 2002). All detected bat calls were then recorded in time-expansion mode onto a digital recorder (Edirol R-09 (Roland Corp., Shizuoka, Japan); files saved in WAV format; sampling rate 44·1 kHz and 16 bits/sample) and analysed with sound analysis software (BatSound Pro v3.31b; Pettersson Elektronik AB) using a 512-pt FFT with Hamming window for spectrogram analysis. Call parameters were measured using crosshair screen cursors; temporal variables were measured from oscillograms, while frequencies were taken from power spectra. Barbastella barbastellus has distinctive echolocation calls that allow reliable acoustic identification (Denzinger et al. 2001; Russo & Jones 2002). Transect results were used later as test data to evaluate the performance of the distribution models. This is a crucial aspect of our work because test data were in a presence/absence format obtained independently of the training data, rather than consisting of pseudo-absences or a subset of the initial data set as is frequently used (Guisan & Thuiller 2005). Ground-validation data were obtained with an equal resolution to model’s training data (30 arc s).

Transect locations were defined to cover the range of model scores proportional to the amount each cell values occupied area in the study area (Fig. 2). Additionally, transects were planned to cover the extent of the whole study area (Fig. 1). Whenever possible, we repeated transects (up to two times) where B. barbastellus presence was not confirmed, taking care never to replicate them in the same season of the year. This bat can be detected up to a maximum distance of c. 20 m with the employed detector. Barbastella barbastellus is often found in mature woodland with water sources nearby (Sierro 1999; Greenaway 2001). Thus, transects were located in habitats as similar as possible to those containing these features. Between the months of April and September of 2005–2007, 72 transects were walked in 52 different locations and 121 B. barbastellus passes were recorded (average of 1·7 ± 3·3 passes/transect). We acknowledge that a higher number of transects would increase statistical power; however, the number of acoustic sampling was limited by the extensive logistics involved.

Figure 2.

 Proportion of each model cell values in the study area (global availability) and in the location of the acoustic transects. See model definitions in Materials and methods.

Model evaluation

We started by calculating a two-tailed Wilcoxon-signed rank test using each model’s cell values to test if they generated different predictions (Randin et al. 2006; Phillips et al. 2009). Additionally, we calculated pairwise linear regressions of the models using cell values where B. barbastellus presence was confirmed, which also allowed investigation of whether the transformation of categorical to continuous variables had any influence on model predictions.

It is important to use more than one statistic to evaluate the models because each quantifies different aspects of model performance (Elith & Graham 2009). Therefore, we tested model accuracy between the test data and the habitat-suitability models using a set of threshold-dependent and -independent statistics. ROC plots with respective area under curve (AUC) and Pearson’s correlation coefficient (COR) were the selected threshold-independent indices. The AUC provides a single measure of model performance (Fielding & Bell 1997) by giving the probability that the model correctly ranks a random presence site vs. a random absent site (Phillips et al. 2009). It ranges from 0·5 (randomness) to 1 (perfect discrimination). The correlation, COR, between the test data and model predictions is known as the point biserial correlation and can be calculated as a Pearson correlation coefficient (Elith et al. 2006). It differs from AUC by measuring the degree to which the predictions vary with the acoustic transects data, thus it is likely to be more sensitive to the relative sampling intensity (Phillips et al. 2009).

We also tested model performance after model reclassification into binary maps (or presence/absence), because this is one of the most important and widely used outcomes for applications such as biodiversity assessment, reserve selection and climate change predictions (Lobo, Jiménez-Valverde & Real 2008). The definition of the threshold should not be chosen arbitrarily but should be based on the objectives of the modelling (Hernandez et al. 2006) taking into account the quality and precision of the data employed. However, there has been a lack of agreement on the criteria that can be used to define a threshold in presence-only modelling. The 10th percentile presence value (above which it is considered that the species is present) assumes that 10% of presence data may suffer from errors or lack of resolution (Raes et al. 2009). This is especially relevant when dealing with datasets gathered over large time-spans where reliability and precision has probably varied. Thus, this threshold value was used to reclassify our models and subsequently for calculation of the threshold-dependent statistics, namely Cohen’s Kappa, correct classification rate, sensitivity and specificity. The Kappa statistics (Cohen 1960) define the accuracy of the prediction and ranges from −1 to +1, where +1 indicates a perfect agreement between test data and predictions, while 0 indicates an agreement no better than random. The correct classification rate indicates the proportion of correctly classified presences and absences of the test data according to the defined threshold. Additionally, by calculating sensitivity and specificity we evaluate how well models identify presences and absences respectively (Jiménez-Valverde et al. 2008a; Lobo et al. 2008). All statistical analyses were performed using spss v15.0 (SPSS Inc., Chicago, IL, USA).

Results

Analysis of the model predictions

The three models predicted the majority of the high-suitability areas in the central and northern regions of Portugal (Fig. 3), although they differed in the predicted extent of that area. The area of high suitability under Max_cat was the most fragmented of all models, whereas in ENFA suitable areas showed a higher potential for population connectivity over a wide region (binary maps available in Appendix S1, Supporting information).

Figure 3.

 Habitat-suitability maps for Barbastella barbastellus in Portugal, as computed by the three models: (a) Maxent including categorical variables, (b) Maxent with continuous variables only, and (c) ecological niche factor analysis.

On the other hand, predictions of suitable area in the south clearly differed between both Maxent models and ENFA. The highest Maxent scores were located in mountainous areas or close to the coastline, while in ENFA equivalent areas were mainly located in the steppe and agricultural plains.

Determination of most relevant ecogeographical variables for each model

Eecological niche factor analysis selected the first four factors as significant (Table 1), explaining 100% of the marginality (in ENFA the first axis always accounts for 100% of marginality) and 93% of specialization. The marginality factor alone explained 24% of the specialization, showing that this species occupies a very restricted range of conditions that differ from average conditions found in Portugal.

Table 1.   Correlations for the first four factors of ecological niche factor analysis (ENFA) for Barbastella barbastellus records in Portugal
EGVMarginalitySpecialization
Factor 1 (24%)Factor 2 (34%)Factor 3 (22%)Factor 4 (13%)
  1. Per cent indicate the amount of variance explained by each factor. Marginality factor: values higher than 0 indicate a higher correlation between the bat’s presence and the ecogeographical variable (EGV); positive and negative signs indicate selection or avoidance of the EGV respectively. Specialization factor: higher values indicate a higher specialization for the EGV; positive and negative signs have no meaning for this factor. Values >0·4 are in bold for an easier visualization of the table.

Agriculture−0·217−0·3090·5000·404
Altitude0·4390·4970·210−0·331
Native woodland0·0850·0510·5660·056
Human infrastructures0·423−0·1890·109−0·109
Precipitation0·3650·5480·2210·085
Production forest−0·2010·4360·5640·766
Scrubs and regenerating forest0·1940·2550·1760·262
Annual temperature−0·035−0·035−0·0040·202
Water bodies0·2020·012−0·0030·047
Temperature range0·5640·2510·402−0·082

The ENFA results indicated that the presence of B. barbastellus was linked to lower temperature ranges than those found on average in Portugal and to higher altitudes, and also indicated that the bat avoided areas of infrastructure, agricultural fields and production forests, preferring dense areas of native woodland.

Maxent jackknife analysis (Fig. 4) showed that the most important variables for Max_cat were native woodland and annual average temperature with Max_cont also selecting those as the most important variables although in the reverse order. Furthermore, in both models the overall gain decreased the most when native woodland was withdrawn from calculations, meaning that this EGV is the most uncorrelated among all employed EGV.

Figure 4.

 Representation of each variable’s importance for the Maxent models (see Materials and methods for abbreviations). The per cent of contribution of each variable to the model is represented by the black bar and corresponding values may be found on the left axis. The other two bars represent the jackknife results for the model with only one variable (var. alone) or with all variables but the analysed one (without var.). Values for the jackknife results are represented on the right axis.

Model evaluation

Model predictions were significantly different among all models after Bonferroni corrections for multiple comparisons (< 0·001, two-tailed Wilcoxon-signed rank test, paired by model). This is especially interesting given that mean habitat-suitability values for Max_cat and Max_cont were very similar in the training data and very different in the presence test data (Table 2). Also of note is that ENFA produced a decrease in value from training to presence test data, and as a probable consequence there is an overlap on the 95% confidence interval values of presence and absence in the test data. This denotes ENFA’s lack of power to distinguish between presence and absence. By comparing the cell values where B. barbastellus was present in each model (Fig. 5), we see that the influence of variable transformation (continuous vs. categorical) had more impact in the high-suitability scores, especially in the Max_cat vs. Max_cont comparison. Comparing ENFA with both Maxent models, there is a similar pattern with a high variability of ENFA scores where Maxent models only achieved high-suitability results.

Table 2.   Mean habitat-suitability values (±95% confidence interval) of training (presence-only) and test data (presence and absence) respective to each model
ModelTraining presenceTest data
PresenceAbsence
  1. For model abbreviations see Materials and methods.

  2. ENFA, ecological niche factor analysis.

Max_cat0·642 (±0·132)0·592 (±0·096)0·135 (±0·073)
Max_cont0·650 (±0·1)0·328 (±0·107)0·108 (±0·05)
ENFA53·2 (±16·2)36·9 (±11·2)27·8 (±8)
Figure 5.

 Comparison between each model cell value where Barbastella barbastellus was confirmed present. Diagonal dashed line shows the linear regression between the two models. Model abbreviations may be found in Materials and methods.

The models were highly successful in locating previously undiscovered populations of this rare species. Barbastella barbastellus was discovered in 15 new areas and the species’ known distribution in Portugal was extended by c. 100 km to the south (Fig. 1). Of these 15 new locations, 13 were predicted by Max_cat, 11 by Max_cont and seven by ENFA. Two detected populations occurred in areas predicted to be of low suitability by all the models. The discrepancies in each model’s success occurred in the southern part of Portugal, where ENFA predictions were not in agreement with the new findings. Conversely, both Maxent models achieved a similar performance: Max_cat predicted the location of six new southern locations while Max_cont predicted five.

Threshold independent evaluation

The AUC scores were high for Max_cat and Max_cont models with ENFA registering a poor score (Table 3). Additionally, B. barbastellus presence/absence was also significantly correlated with all models although only Max_cat achieved a high score. For the other two models the correlation value was almost half of the Max_cat reflecting the high dispersion and variability in the cell values where B. barbastellus presence was confirmed (Fig. 6).

Table 3.   Comparison of model results with presence/absence test data
ModelThreshold independentThreshold dependent
AUCCORKappaCorrect classificationSensitivitySpecificity
  1. For all statistics, the value of one is the best achievable result. For model abbreviations see Materials and methods.

  2. *< 0·05.

  3. AUC, area under curve; COR, correlation coefficient; ENFA, ecological niche factor analysis.

Max_cat0·930*0·727*0·8530·8590·9630·767
Mac_cont0·827*0·472*0·7420·7510·5190·967
ENFA0·614*0·312*0·6120·6140·4440·767
Figure 6.

Barbastella barbastellus confirmed presence or suspected absence from the acoustic transects with respective model score (see model abbreviations in Materials and methods). Each symbol may correspond to more than one transect. Horizontal lines represent threshold values for each model above which the species is assumed to be present.

Threshold-dependent evaluation

Max_cat was the only model where almost all presences were recorded in cells with values higher than the defined threshold for presence (Fig. 6). As stated before, for the other two models the presence values were rather dispersed. On the other hand, for models using only continuous variables almost all of the absences (or unconfirmed presences) were recorded in values below the aforementioned threshold. Nevertheless, even for Max_cat all absences were recorded in cells with values <0·4.

Considering evaluation indices dependent on the defined threshold, again the performance rank is maintained with Max_cat being the best and ENFA registering the lowest scores in Kappa statistics, correct classification rate and sensitivity. Regarding model performance to correctly detect absences (expressed by specificity), all models performed well, with Max_cont achieving a very high score while Max_cat and ENFA behaved identically.

Discussion

Our Maxent models proved successful in the discovery of new barbastelle populations outside the known range, even when there was a sampling bias in the presence data. Nevertheless, predictive modelling should be used with care when projecting results to new areas. In fact, model accuracy could be compromised when projected to values outside the range that it originated from (Peterson, Papeş & Eaton 2007). Sampled data should cover the range of a species’ ecological conditions so that the modelled niche can approximate to the realized niche (Wisz et al. 2008). Yet, one of the most unique applications provided by habitat-suitability modelling is to give insights into new areas in which a species could occur (Pearson et al. 2007) or to forecast how distributions might be affected by new combinations of biotic or abiotic factors (Guisan & Thuiller 2005). However, it is important to question whether modelling does necessarily predict distribution accurately (Hirzel & Le Lay 2008). The geographical distribution of a species could be limited by several factors that usually lay outside the scope of species distribution modelling, for example, existence of geographical barriers, limited dispersal abilities and biotic interactions (Hirzel & Le Lay 2008). For species with a high potential for dispersal, such as bats, most of these factors would probably have little relevance to the ability to colonize potential habitats. Hence, the difference between realized and potential niches may be much smaller when dispersal limitations are accounted for (Hirzel & Le Lay 2008). The quality of our results could have been different at a different spatial scale, geographical area or simply with a different modelled species. However, for species with similar ecological characteristics (rare, high dispersal potential) we could expect that model quality would be similar. In fact, models generated for specialist species tend to have greater accuracy and predictive power than for generalists even in cases where the data quality for specialists is poor (Elith et al. 2006). A specialist species usually occurs in a more restricted range of values within an ecological factor than a generalist species. Hence, it is more likely that fewer data can cover much of the species’ realized niche (Brotons et al. 2004).

The most recently developed modelling techniques (e.g. Maxent) outperformed more established methods like Genetíc Algorithm for Rule-set Production, glm or Bioclim (Elith et al. 2006). Maxent especially has achieved a robust performance showing a good accuracy with low sample sizes and an excellent predictive ability (Hernandez et al. 2006; Pearson et al. 2007; Wisz et al. 2008). Our results are in agreement with those studies, because Maxent performed with great accuracy with a small and biased data set. Max_cat did not predict broad areas of distribution for the B. barbastellus. However, bats were found in all high-suitability areas, even in the south of Portugal that was clearly outside the range of the training data. As such, Maxent proved to be robust to sampling bias at this scale which allowed the discovery of new populations and the extension of known distribution. In fact, Maxent and ENFA had different levels of success in predicting the occurrence of new populations, with ENFA failing to identify the location of new populations in southern Portugal.

Zaniewski, Lehmann & Overton (2002) showed that ENFA produced accurate results although recognizing a tendency to overestimate the spatial extent of distributions, especially on the periphery of ranges (Brotons et al. 2004). Our results showed a similar pattern. A broad and accurate area of high-suitability was identified in the central and northern regions of Portugal but ENFA was very inaccurate in its predictions outside the geographical range of the training data.

It is important to understand, why these two techniques yielded such different results in the discovery of new populations. Tsoar et al. (2007) concluded that more complex techniques (in our case Maxent) are better predictors that the simple ones as they establish more flexible relationships between the dependent and independent variables. In fact, models that have no pre-defined shape of response curves can build models closer to the training data such as those based on smoothing techniques, like Maxent (Randin et al. 2006). On the other hand, parametric methods like ENFA are limited by the normal distribution making them more sensitive to bias or extrapolations (Elith et al. 2006; Randin et al. 2006). Overall, Maxent seems prone to overfitting presence data (Peterson et al. 2007) hence more likely to develop omission errors or false absences (the species exists in low suitability areas) while ENFA seemed to have greatest problems in reducing the commission error rate or false positives (predicts occurrence where the species does not exist) in areas outside the range of the training data.

It should also be stated that it is likely that our models developed in Portugal might not work in other parts of Europe. Portuguese populations are in the edge of the species’ range, and consequently ecological conditions where barbastelles exist in Portugal may be not representative of the species’ ecological optimum. Nevertheless, several studies support the theory that niche positions are more than regional phenomena, because a species can occupy similar niche positions in different regions (Randin et al. 2006). In fact, the distribution of B. barbastellus predicted by Maxent was mainly limited by the availability of native woodland within a range of temperature values. It has been shown that B. barbastellus is dependent upon mature woodland for roosting (Russo et al. 2004; Russo, Cistrone & Jones 2005) and foraging (Hillen et al. 2009; Kerth & Melber 2009), where their main prey, moths, are more abundant (Sierro 1999). Our results reinforce the need to protect these mature woodland habitats for effective conservation of this threatened species. On the other hand, the more important EGV determined in ENFA were not directly related to this habitat. Regarding land cover variables, ENFA predictions were mainly influenced by habitats actively avoided by B. barbastellus (like production forests and infrastructures). Higher altitudes also seem to be highly relevant in the model, but could be correlated with the presence of native Atlantic forest typically confined to isolated areas in high mountains where the majority of the training data were located. Altitude by itself is probably not a limiting factor for B. barbastellus distribution (as our fieldwork results have shown) but its selection could be a product of sampling bias. This selection of variables could explain why ENFA failed in predicting B. barbastellus occurrence in the south while being accurate in the remainder of the country. The combination of abiotic factors in the south clearly differs from the rest of the country although the area of native woodland (in this case oak savanna-like woodland) is vast. By choosing variables different from the ones chosen by Maxent, the location of ENFA high-suitability areas in the south was totally different.

Furthermore, we acknowledge that our absent data may contain some false absences (unconfirmed presences) that could have a potential impact on the evaluation statistics. Nevertheless, we believe that our results are consistent over the sampling area hence the impact of that error on model predictions does not justify the differences in their performance.

In conclusion, our models proved to be very useful tools for defining suitable areas and for the subsequent discovery of new populations of a rare species over a relatively large geographical scale. A similar approach to other rare species could also be highly productive when addressing issues such as planning field surveys, defining reintroduction areas, identifying population fragmentation, planning rapid biodiversity assessments in poorly documented regions and supporting a wide range of conservation strategies or policies (Sattler et al. 2007). Indeed, some studies have successfully used predictive modelling to determine species’ conservation status (Sattler et al. 2007) or even to find new populations of cryptic species (Raxworthy et al. 2003). Nevertheless, we should stress that models are only an approximation to reality hence they can never replace a detailed, ongoing collection of field data (Hirzel & Le Lay 2008). With enhanced computer processing and the increase of available geographical data, surely new applications will widen the scope of species distribution modelling.

Acknowledgements

This work was only possible because of the help from numerous volunteers. We are especially grateful to Bruno Simões, Raquel Barata, João Tiago Marques, Ana Augusto, Pedro Alves, Sofia Lourenço, João Paulo Lopes and Filipe Moniz. José Carlos Brito helped on the modelling while Paul Craze assisted in the statistical analysis. Likewise, several institutions provided invaluable support: Parque Natural do Douro Internacional, Parque Nacional Peneda-Gerês, Área de Paisagem Protegida da Serra do Açor, Parque Natural do Vale do Guadiana and Instituto da Conservação da Natureza e Biodiversidade. We would like also to thank Danilo Russo and two anonymous referees for their valuable suggestions and comments. Financial support was provided by Fundação para Ciência e Tecnologia (SFRH/BD/17755/2004).

Ancillary