Measuring nestedness: A comparative study of the performance of different metrics

Abstract Nestedness is a property of interaction networks widely observed in natural mutualistic communities, among other systems. A perfectly nested network is characterized by the peculiarity that the interactions of any node form a subset of the interactions of all nodes with higher degree. Despite a widespread interest on this pattern, no general consensus exists on how to measure it. Instead, several nestedness metrics, based on different but not necessarily independent properties of the networks, coexist in the literature, blurring the comparison between ecosystems. In this work, we present a detailed critical study of the behavior of six nestedness metrics and the variants of two of them. In order to evaluate their performance, we compare the obtained values of the nestedness of a large set of real networks among them and against a maximum‐entropy and maximum‐likelihood null model. We also analyze the dependencies of each metrics on different network parameters, as size, fill, and eccentricity. Our results point out, first, that the metrics do not rank networks universally in terms of their degree of nestedness. Furthermore, several metrics show significant dependencies on the network properties considered. The study of these dependencies allows us to understand some of the observed systematic shifts against the null model. Altogether, this paper intends to provide readers with a critical guide on how to measure nestedness patterns, by explaining the functioning of several metrics and disclosing their qualities and flaws. Besides, we also aim to extend the application of null models based on maximum entropy to the scarcely explored area of ecological networks. Finally, we provide a fully documented repository that allows constructing the null model and calculating the studied nestedness indexes. In addition, it provides the probability matrices to build the null model for a large dataset of more than 200 bipartite networks.

• 43 Host-parasite networks. Here, the links depict a parasitic relationship, where one of the species obtains benets in detriment of the other. Explicitly, these networks are formed by dierent ea species which feed on diverse mammal species. Although this is not a mutualistic interaction, the system may still be represented by a bipartite network where the two guilds correspond to ea and mammals species.
• 4 Plant-herbivore networks. Here, the links represent a consumer-resource interaction between insect species (one guild) and plant species (the other guild). In detail, the networks depict dierent communities where macrolepidopteran species feed on several Prunus species. The economic networks, which are publicly available in [2], consist of: • 8 economic networks representing buyers-sellers interactions in the Boulogne-sur-Mer Fish Market in France [3]. These are mutualistic networks taken from a very dierent context. Each network describes the transactions observed in dierent days in the bilateral or in the auction Fish Market. These daily networks are typically much denser than ecological ones. SI2: Computation of the nestedness index for each of the studied metrics Temperature We calculated the temperature metrics using the R software [4] and, specically, the bipartite package [5,6] version 1.13.0. In particular, we used the nested function and we set as method the binmatnest2 option. This calculates the temperature metrics by using an implementation by Jari Oksanen [7] of the binmatnest program by Miguel Rodriguez-Girones [8]. We have redened the resulting temperature in order to uniform the interpretation of the values yielded by all the metrics such that the higher the value of the corresponding index, the higher the nestedness.

NMD
We calculated the nestedness metrics based in the Manhattan distance (NMD) using the R software [4]. We used the nestedness.corso function (currently deprecated) from the bipartite package [5,6] version 0.90. For each measure (both for the real networks and the sampled networks), we set to 500 the number of null networks that eventually permits evaluating the signicance. Again, we redened the index to simplify the interpretation of results.

NODF
We wrote a program in FORTRAN90 that computes the NODF and stable-NODF metrics for the real networks, as well as for the corresponding set of null networks.
Importantly, when performing the calculations over the sample of null networks, we kept the same normalization for all sampled networks. That is, we divided the number of overlapping connections, calculated for each null network, by the original number of rows and columns, independently of whether some of the nodes came to have zero degree in the null network.

Discrepancy
We computed the discrepancy metrics using the R software [4] and the bipartite package [5,6] version 1.13.0. In particular, we performed the calculation using the method discrepancy from the nested function.
The nal nestedness value measured is directly proportional to the density of links and size of the network. With the aim of removing such dependencies, we divided the resulting value of the metrics by the total numbers of links. This results in a relative discrepancy. Once again, we nally rescaled the index in order to obtain the same monotonic variation for all the indices.

NIR
We implemented a program in FORTRAN90 that calculates the NIR value of the real network and of the corresponding set of null networks. In each case, the resulting value of nestedness is multiplied by 100 in order to preserve the same scale for all the metrics.
In order to account for the possible eects of the degeneracy in the ordering, that is, the fact that multiple congurations are possible when we order rows and columns by their degree, we computed the resulting NIR as the average over a large number of equivalently ordered congurations. These congurations were produced by randomly swapping the matrix position of nodes with the same degree. In more detail, to generate a new ordering we run over all the nodes with degenerate degree and, for each node, we accept a position swap with probability 1 2 .
For each real network, we calculated the degeneracy, ideg, the number of repeated degrees.
Then, we produced a total of 10 · ideg congurations with the same degree order but diverse row and column positions. This procedure was carried out both for the real network and for each null network in the sampling with the exception of the Robertson's network [9] for which, due to its very large size (1500 species), only 10 degenerate congurations have been computed.

Spectral radius
We computed the largest eigenvalue using the R software [4], in particular the eigs_sym function from the rARPACK package [10].
In order to calculate the normalized version of this metrics, we need an estimation of the largest spectral radius of a perfectly nested network of the same size and ll. To estimate each of these values, for each real network in our dataset we produced 100 new networks, characterized by being perfectly nested. These networks were generated using the SNM algorithm [11], which preserves the number of connected nodes (network size) and links, but modies the pattern of connections and the degree sequences. This algorithm is divided into two procedures. First, the real network is randomized preserving only the ll and the size (that this, ensuring that every node has at least one connection). Second, the SNM algorithm is performed, which consist of iterating the following rules: (i) We attempt to modify a link by proposing a new partner, randomly selected but dierent to the original node. The rewiring is susceptible of being accepted only if the new partner has a larger degree than the previous one. This step performs a static version of preferentialattachment.
(ii) If the proposed reconnection leaves one of the nodes with zero degree, the move is discarded.
This ensures that the number of connected nodes does not change, thus preserving the network size.
By iterating over these steps i and ii, one generates a new matrix which is more nested as well as more heterogeneous in its degree sequences than the original one (see [12]). The iteration stops when no more moves are allowed. However, given condition ii, this process is not unique and might end up in multiple perfectly nested congurations. To handle this, we generated several optimal congurations per each real network. Specically, we generated 100 new networks for each empirical network, and exceptionally, for computational reasons, 50 networks for the very large Robertson network. This means that the normalized spectral radius is actually calculated as: where ρ is the spectral radius of the real network and ρ perfect,i represents an optimal conguration with the same size and ll of the real network, produced by the SNM algorithm. N perf corresponds to the number of perfectly nested generated, that in general we set to 100.
When sampling the ensemble, we generated 10 perfectly nested networks per each null network (N perf = 10), and in order to keep the calculations computationally feasible we reduced the sampling size to 500 null networks (N samp = 500). Accordingly, the average normalized spectral radius is calculated as: where ρ null,j represents a null network sampled from the statistical ensemble and ρ perfect,i,j represents a perfect conguration produced with the SNM algorithm, having the same size and ll as the corresponding null network.

SI3: Statistical calculations
The statistical correlations were numerically calculated using Python. The Spearman rank correlation coecient r s and its p-value were calculated using the Scipy package [13], in particular the scipy.stats.spearmanr function.
We performed the linear ts using the Statsmodels package [14], which carries out a multilinear least-square regression and provides multiple information, including the adjusted R 2 , the partial regression coecients, their standard deviation and their associated p-value. The t-ratio i,j corresponding to each partial regression coecient, β i,j , is calculated as follows: where σ i,j is the standard deviation associated to that coecient. This index provides, hence, information on how signicantly dierent from zero is a certain regression coecient.