NichePy: modular tools for estimating the similarity of ecological niche and species distribution models



1. Ecological niche and species distribution models (ENMs and SDMs) are widely used in macroecological studies to investigate the potential geographic distributions of invasive species, the effects of climate change on past, present and future species distributions as well as the evolution of ecological niches. For many of these studies, estimating the similarity of different ENMs and SDMs is critical to evaluate the robustness of models.

2. We implement previously described randomization tests for evaluating the similarity of ENMs and SDMs in a package of three programs/scripts using the Python Programming Language. Our implementation is modular and flexible. It can be used with a variety of existing ENM and SDM algorithms and can be integrated into high-throughput analyses using a command-line shell (e.g. the Bash shell that is found on many Unix-like operating systems).

3. We demonstrate the application of NichePy using distributional data of two Anolis species from Hispaniola. An analysis integrating NichePy and openModeller, including a simple and straight-forward approach to ensemble modelling, is outlined.


Correlative ecological niche models and species distribution models (ENMs and SDMs; niche models hereafter) are being used in many areas of biological research (e.g. Peterson et al. 2011). In general, these correlative models are generated by overlaying occurrence records for an organism with raster environmental data layers in a geographic information system (GIS). Part of the species’ ecological niche is estimated using machine learning algorithms, regression analysis, principal component analysis or higher dimension bounding boxes that reconstruct the species’ limits in environmental dimensions based on the relation between observed species occurrences and (in some cases) pseudo-absences or absences and the observed environment (Soberón 2007; Soberón & Nakamura 2009; Godsoe 2010; Peterson et al. 2011). Niche models that are generated in such a way can be used, for example, to address questions on the evolution of ecological niches, the invasive potential of a species or the potential impact of climate change on species distributions.

Many of the questions that can be addressed using niche models require a quantification of the overlap of the estimated niche. Warren, Glor & Turelli (2008) proposed two statistical tests to address this issue. The first test tries to answer the question of whether or not two niche models are equivalent (identity test) based on pseudo-replicate occurrence data sets drawn from the combined occurrence data of the two species. The second test tries to answer the question of whether or not two niche models are more or less similar than can be expected by chance. This test is based on pseudo-replicate occurrence data-points drawn from the geographic region in which each species is known to occur (background test). Both tests are randomization tests that rely on calculating the overlap between two niche models. Null distributions of niche overlap are calculated from models generated from pseudo-replicate data sets. The niche overlap calculated for the models generated using the original data sets is then compared to the null distribution. Quantitative metrics for measuring niche overlap were recently evaluated by Rödder & Engler (2011). We implement both identity and background tests using three niche overlap metrics: the corrected modified Hellinger distance I, Schoener’s D and Bray–Curtis distance BC (see Rödder & Engler 2011).

Implementation and features of NichePy


NichePy was written in the Python Programming Language (, an interpreted high-level computer programming language. Python is open source software and is either installed by default on many computer operating systems or can be obtained free of charge from NichePy does not feature a graphical user interface (GUI) and consists of three independent programs or scripts. We deliberately chose to develop NichePy modular and for command-line interface (CLI) execution to facilitate its use as part of customized and automated analysis pipelines that may run extended analyses in batches without or with limited user input (Fig. 1 outlines the work flow for using NichePy). NichePy, its documentation and example data can be obtained from

Figure 1.

 Flowchart outlining the work-flow for using NichePy.

Resampling of Occurrence Data: The nicheIdentity Script

The script nicheIdentity resamples species occurrence data as outlined in Warren, Glor & Turelli’s (2008) test of niche identity. Given occurrence data sets for species X and Y containing a total of nX and nY occurrence points, nicheIdentity creates a pooled set of occurrence points with size nX + nY. Pseudo-replicate data sets of sizes nX and nY are then generated from the pooled data set. A comma-delimited text file, containing occurrence points for the original and pseudo-replicate data sets, that can serve as input for niche modelling programs is the output of this script.

Resampling of Occurrence Data: The nicheBackground Script

The nicheBackground script allows resampling of species occurrence data to perform Warren, Glor & Turelli’s (2008) background test. Given species X and Y, let oX and oY denote the cells in which each species is known to occur in study areas NX and NY. This script resamples oX cells from NX and oY cells from NY. The user must supply ARC/INFO ASCII grids representing the geographic background of the study areas for their species of interest. For example, one may create two grids in a GIS using a euclidean distance buffer around oX and oY. The original and pseudo-replicate data sets are written out into a comma-delimited text file that can be used to create niche models.

Quantifying Niche Overlap: The getMetric Script

The getMetric script calculates three niche overlap indices (a modified version of Hellinger’s distance I, Schoener’s D and Bray–Curtis distance BC; Rödder & Engler 2011) from niche models that were generated using occurrence data resampled by nicheIdentity or nicheBackground. For the niche identity test, getMetric calculates an overlap metric for the niche models of the original occurrence data sets and all pairwise comparisons of niche models generated from pseudo-replicate occurrence data sets (Fig. 1). For the niche background test, getMetric calculates the overlap of niche models for the original data set as well as the niche models of each species and each pseudo-replicate niche model generated from the background of the other species (Fig. 1). Currently, one-tailed tests are implemented in getMetric to evaluate whether the overlap calculated between niche models generated using the original occurrence data sets is statistically significant.

Example analysis: Anolis coelestinus and Anolis chlorocyanus on Hispaniola

Data and Modelling Strategy

We used NichePy to perform the identity and background tests for a data set of two Anolis species, Anolis coelestinus and Anolis chlorocyanus, from the island of Hispaniola (Fig. 2). These data were previously analysed by Glor & Warren (2011). A. coelestinus and A. chlorocyanus are distributed on either side of a biogeographical boundary (Glor & Warren 2011). We obtained Glor & Warren’s (2011) data sets containing 261 unique localities for A. chlorocyanus and 160 unique localities for A. coelestinus. The environmental data consisted of the first four principal components of 19 bioclimatically significant variables extracted from the WorldClim database (; Hijmans et al. 2005). We generated 100 pseudo-replicate data sets for each species using the nicheIdentity script. We used the nicheBackground script to generate 100 pseudo-replicate occurrence data sets from each species’ geographic background, resampling the geographic area contained within a radius of 1 km around each occurrence point of each species (Fig. 2). Niche models were generated for all data sets, originals and pseudo-replicates, using Ecological Niche Factor Analysis (ENFA; Hirzel et al. 2002) and unsupervised Random Forests (RF; Breiman 2001) with 100 starting trees as implemented in openModeller version 1.1 (Muñoz et al. 2009). Ensemble models were created by averaging the values in each grid cell for all corresponding pairs of ENFA and RF models using shell scripts. I, D and BC as well as P values were calculated using getMetric for ENFA, RF and ensemble models. Each step of this analysis can be chained together using the built-in functions of a CLI (e.g. Bash which is the default shell on many Unix-like operating systems including Mac OS and many popular flavours of GNU/Linux).

Figure 2.

 Location of occurrence data points (solid circles) on the island of Hispaniola, Greater Antilles. The area used for background sampling in the niche background tests are indicated as hollow circles around each occurrence point. Niche models using Ecological Niche Factor Analysis (ENFA) and unsupervised Random Forests (RF) are shown for each species, as well as ensemble models that represent the average of both ENFA and RF. Scales indicate suitability scores.

Results and evaluation

The biogeographical patterns and possible ecological and historical explanations of the geographic distributions of A. chlorocyanus and A. coelestinus were discussed in detail in Glor & Warren (2011). Our results largely agree with Glor & Warren (2011). As discussed by Rödder & Engler (2011), I generally leads to estimates of niche models being more similar than D and BC. In our analysis, we found that the metrics D and BC were identical in all comparisons. Overlap measured among niche models generated using different modelling algorithms leads to different values of I and D/BC, but none of these differences lead to different conclusions in hypotheses tests (Figs 3 and 4). As expected, the ensemble models calculated by averaging cell values from both ENFA and RF models resulted in values of I and D/BC that were intermediate to those obtained from ENFA and RF models (Figs 3 and 4). ENFA appears to produce models that are fitted more closely to the observed occurrence data, whereas unsupervised RF fit models much more loosely to the observed data (Fig. 2). In our example, ENFA and RF represent opposite extremes, and model averaging used in our ensemble models dampens these extremes.

Figure 3.

 Distribution of overlap indices calculated from pseudo-replicate datasets (I*, D*, and BC*) for the niche identity test. I, D, and BC represent overlap metrics for the original occurrence datasets. P was calculated by dividing the number of I*, D*, or BC* values that are smaller than I, D, or BC by the total number of I*, D*, or BC* values. We test the null-hypothesis that the two niche models for Anolis chlorocyanus and Anolis coelestinus are identical.

Figure 4.

 Distribution of overlap indices calculated from pseudo-replicate datasets (I*, D*, and BC*) for the niche background test. I, D, and BC represent overlap metrics for models generated using the original occurrence datasets. The null hypothesis that two niche models are more similar than expected by chance is being addressed using a one-tailed test where P is calculated as the fraction of I*, D*, or BC* values that are larger than I, D, or BC.

In all significance tests, we found that the niche models of A. choelestinus and A. chlorocyanus are not identical regardless of modelling method or overlap metric used (Fig. 3). The strictness of the identity test makes it extremely robust to differences among modelling algorithms and overlap metrics. Effects of differences among modelling techniques are more obvious in the background test. We performed one-tailed significance tests for the background test, as we are concerned with evaluating whether or not the niche models for A. coelestinus and A. chlorocyanus are more similar than expected by chance. Similar to Glor & Warren (2011), we cannot reject the null hypothesis that the niche models for A. coelestinus and A. chlorocyanus are no more similar to one another than can be expected by chance (see Fig. 4 background tests for both A. coelestinus vs. the background of A. chlorocyanus and A. chlorocyanus vs. the background of A. coelestinus). When comparing A. chlorocyanus to the background of A. coelestinus, we see a large difference in P values, using D/BC, between ENFA models (= 0·06) and both RF (P = 0·24) and ensemble models (P = 0·23). It is possible that different modelling algorithms and overlap metrics may lead to different conclusions in certain cases, which deserves further investigation. It is beyond the scope of this contribution to discuss this topic in detail or discuss the advantages or disadvantages that one modelling algorithm may have over the other.


Analysing data sets using different approaches may be advisable because differences in niche models may affect the outcome of niche identity or niche background tests. Ensemble modelling has been suggested as an approach to account for variability among modelling strategies (e.g. Araújo & New 2007). This approach may be of special interest when informed choices of which modelling algorithm is most appropriate for the question addressed cannot be made. In addition, different niche overlap metrics may result in different and possibly conflicting answers. Of the three metrics implemented in NichePy, I results in the most lenient measure of niche overlap, whereas D and BC represent stricter measures (cf. Rödder & Engler 2011). Similar to Rödder & Engler (2011), we observed that D and BC provide identical estimates of niche overlap. Considering this, one may choose to calculate one or the other rather than both. In terms of computational speed, it is faster to compute D than BC in getMetric.

Warren, Glor & Turelli (2010) implemented both niche identity and niche background tests using the metrics I and D in ENMTools, while Heibl (2011) included these tests as part of the phyloclim R package (R Development Core Team 2009). Our aim is not to compete with these tools, but to provide modular scripts to take advantage of quantitative approaches for evaluating niche models while maintaining a maximum of flexibility. Our scripts can be assembled into a custom pipeline that analyses batches of data without or with limited user intervention using the built-in functionality of a CLI. In addition, the NichePy scripts can be used with ease in conjunction with any modelling algorithm, while other software tools generally default to using the modelling software package MaxEnt (Phillips, Anderson & Schapire 2006).

How to quantify differences among ecological niches using niche models remains an area of active debate. Current studies, including the work presented here, generally rely on estimating the overlap of niche models by calculating distance metrics in geographic space (i.e. pairwise distances of suitability values on a pixel-by-pixel basis on geographic grids). If the focus is evaluating differences of spatial distributions of organisms, this approach seems well justified (Peterson et al. 2011). Depending on the application, however, it may be more desirable to estimate differences in environmental rather than geographic space (cf. Peterson et al. 2011; pp. 176–178). In this context, Broennimann et al. (2011) present an interesting initial framework for estimating niche overlap in environmental dimensions.


This work was funded through US NSF Doctoral Dissertation Improvement grant DEB 0910237 and US NSF Assembling the Cnidarian Tree of Life grant EF-053179. DL Warren generously provided the Anolis and WorldClim data sets. We thank two anonymous reviewers and the associate editor for their valuable comments and suggestions that greatly improved this manuscript.