Ecological networks are often composed of different subcommunities (often referred to as modules). Identifying such modules has the potential to develop a better understanding of the assembly of ecological communities and to investigate functional overlap or specialization.
The most informative form of networks are quantitative or weighted networks. Here, we introduce an algorithm to identify modules in quantitative bipartite (or two-mode) networks. It is based on the hierarchical random graphs concept of Clauset et al. (2008 Nature 453: 98–101) and is extended to include quantitative information and adapted to work with bipartite graphs. We define the algorithm, which we call QuanBiMo, sketch its performance on simulated data and illustrate its potential usefulness with a case study.
Modules are detected with a higher accuracy in simulated quantitative networks than in their binary counterparts. Even at high levels of noise, QuanBiMo still classifies 70% of links correctly as within- or between-modules. Recursively applying the algorithm results in additional information of within-module organization of the network.
The algorithm introduced here must be seen as a considerable improvement over the current standard of algorithms for binary networks. Due to its higher sensitivity, it is likely to lead to be useful for detecting modules in the typically noisy data of ecological networks.
The ecological literature is replete with references to interacting groups of species within systems, variously termed compartments (May, 1973; Pimm 1982; Prado & Lewinsohn 2004), modules (Olesen et al. 2007; Garcia-Domingo & Saldañ a, 2008; Dupont & Olesen 2009), cohesive groups (Bascompte et al. 2003; Danieli-Silva et al. 2011; Guimarães, Jordano & Thompson 2011) or simply communities (Fortunato 2010). Their attraction, for ecologists, is that they promise a way to simplify the description and understanding of an ecological system, by representing not each and every species, but aggregating their interactions and energy fluxes into a more manageable set of modules (e.g. Allesina, 2009). In the following, we will refer to such aggregated sets of interacting species as ‘modules’. Their characteristic hallmark is that within-module interactions are more prevalent than between-module interactions (Newman 2003; Newman & Girvan, 2006; Fortunato 2010).
In the extreme case, modules are completely separated from each other and are then typically called compartments (Pimm 1982). This strict definition has seen some relaxation (Dicks, Corbet & Pywell 2002), but most recent studies converge on the term ‘module’ for any identifiable substructure in interaction networks (Prado & Lewinsohn 2004; Lewinsohn et al. 2006; Olesen et al. 2007; Ings et al. 2009; Joppa et al. 2009; Cagnolo, Salvo & Valladares 2011).
The identification of modules, and the membership of species to modules, has received considerable interest in the physical sciences (as reviewed in extenso by Fortunato 2010). Particularly the work of Newman and co-workers (e.g. Newman 2003; Newman & Girvan 2004; Newman 2004a, 2004bb, 2006) has practically defined the current paradigm of module definition and identification. Algorithms to identify modules are ‘greedy’, that is, highly computationally intensive, relying on some way of rearranging module memberships and then quantifying ‘modularity’ until a maximal degree of sorting has been achieved (Clauset, Newman & Moore 2004; Newman 2004b; Newman 2006; Pons & Latapy 2006; Schuetz & Caflisch 2008). The focus of virtually all these algorithms was on unweighted and one-mode networks (see, e.g., Clauset, Moore & Newman 2008; Kovács et al. 2010; Lancichinetti & Fortunato 2011; Jacobi et al. 2012, for a recent example). Unweighted (or binary or qualitative) refers to the fact that only the presence of a link between species is known, but not its strength (Levins 1975; Pimm 1982). One-mode refers to the structure of the community, in which all species are potentially interacting with each other. The typical ecological example is a species × food web matrix, in which entries of 1 depict an existing interaction.
In recent years, weighted and bipartite interaction networks have become more intensively studied. In a weighted network, the link between two species is actually quantified (e.g. by the number of interactions observed or the strength of the interaction inferred from the data: Newman 2004b). In a bipartite network, the species fall into two different groups, which interact with members of the other group, but not within their group. A typical example are pollinator–visitation networks (Vázquez et al. 2009), where pollinators interact with flowers, but flowers do not interact among themselves (see Fig. 1). Another well-studied examples is host–parasitoid network (e.g. Morris, Lewis & Godfray 2004; Tylianakis, Tscharntke & Lewis 2007).
While popular among ecologists (Blüthgen 2010; Pocock, Evans & Memmott 2012; Poisot et al. 2012; Schleuning et al. 2012), weighted bipartite graphs are not amenable to any of the existing module detection algorithms for one-mode networks or for unweighted bipartite networks. Existing software uses one-mode networks or, more precisely, one-mode projections of bipartite networks (Guimerà, Sales-Pardo & Amaral 2007; Martín Gonzàlez et al., 2012; Thébault 2013), or unweighted (binary) bipartite networks (Guillaume & Latapy 2004; Marquitti et al. 2013), while other approaches focus on the identification of crucial leaves through quantifications of their position in the network (e.g. centrality or degree: Ravasz et al. 2002; Borgatti 2006; Martín González et al., 2010). This lack of an algorithm to identify modules in quantitative, bipartite networks is particularly problematic, as such networks find their way into conservation ecological considerations (Tylianakis et al., 2010) and are the focus of a vibrant field of macroecological research (Ings et al. 2009). Furthermore, from a statistical point of view, weighted networks offer much more information and are less likely to lead to erroneous conclusions about the system (Li & Chen 2006; Scotti, Podani & Jordan 2007; Blüthgen 2010; Fortunato 2010).
Here, we present an algorithm to identify modules (and modules within modules) in weighted bipartite networks. We build on an algorithm provided by Clauset, Moore & Newman (2008) for unweighted, one-mode networks, the weighted modularity criterion developed by Newman & Girvan (2004), and the bipartite modularity proposed by Barber (2007).
Modules can be interpreted as link-rich clusters of species in a community. An alternative to finding and delimiting such modules is to group species by ordination (Borgatti & Everett 1997; Lewinsohn et al. 2006). Correspondence analysis (CA) of the adjacency matrix is a simple and fast way to organize species. Typically, however, correspondence analysis will not be able to identify modules sufficiently well, even if modules are actually compartments (i.e. perfectly separated: Fig. 2 left, centre). The QuanBiMo algorithm we present here can do so, at least in principle (Fig. 2 right). If modules are perfectly separated, with no species interacting with species in another module, they are called compartments and will be visible as clearly separated groups of species. It is relatively straightforward to implement a recursive compartment detection function, but compartments are much coarser than modules and not the topic of this publication.
One algorithm proposed and available for detecting modules in bipartite networks is due to Guimerà, Sales-Pardo & Amaral (2007) called ‘bipartmod_w’), which is derived from a one-mode algorithm (Guimerà et al. 2005). Their approach differs substantially from single-run bipartite algorithm in that they employ a one-mode modularity algorithm on each level of a bipartite network separately, although they discuss the approach later developed by Barber (2007). The Guimerà et al. approach was used in several ecological applications of modularity (Olesen et al. 2007; Dupont & Olesen 2009; Fortuna et al. 2010; Guimerà et al. 2010; Carstensen et al. 2011; Trøjelsgaard & Olesen 2013), although the algorithm does not explicitly identify combined modules (as stated in Barber 2007; Fortuna et al. 2010). Most recently, Thébault (2013) investigated, through simulations, the ability of three modularity measures (those of Newman & Girvan 2004; Barber 2007; Guimerà, Sales-Pardo & Amaral 2007) to identify modules in binary bipartite networks and comes out in support of that of Guimerà et al. (2007).
Finally, Allesina & Pascual (2009) have proposed an approach for one-mode networks. It identifies ‘groups’, rather than modules, which reveal more about the structure of a food web than modules do, since also their relation towards each other emerges from the analysis. Their approach is based on a binary one-mode matrix, however, even when applied to bipartite networks (as was done by Martín González et al., 2012).
QuanBiMo: a quantitative bipartite modularity algorithm
The new algorithm (Dormann & Strauß, 2013 builds on the hierarchical random graph approach of Clauset, Moore & Newman (2008), which builds a graph (i.e. a dendrogram) of interacting species so that nearby species are more likely to interact. It then randomly swaps branches at any level and evaluates whether the new graph is more likely than the previous one, recording and updating the best graph. Fit is computed as modularity of the current graph (detailed in section 'Goal function'). The swapping is a Simulated Annealing-Monte Carlo approach, that is, sometimes a worse graph is chosen as the starting point for the next swap, thereby avoiding being trapped in a local maximum. Each node of the graph contains the information of whether it is part of a module, so that the graph can be transgressed top-down to identify modules.
Our modifications consist of (i) allowing branches between species to be weighted by the number of interactions observed between them, thereby making the algorithm quantitative; and (ii) taking into account that species in one group can only interact with species in the other group, rather than the one-mode network the algorithm was initially developed for. Taken together, our algorithm computes modules in weighted, bipartite networks, based on a hierarchical representation of species link weights and optimal allocation to modules.
A graph G = (V,E) denotes a set of vertices υ ∈ V connected by edges e ∈ E. An edge e connects two nodes, thus e = c(υi,υj), where . G is a weighted (= quantitative) graph if each edge e has a weight w ∈ W associated with it (w ⊆ R+). We normalize edge weights so that ∑w ∈ Ww = 1. (For binary graphs w = 1/|E| for all existing edges, where |.| symbolizes the number of elements.)
For bipartite graphs, the vertices V are of two non-overlapping subsets, VH and VL (higher and lower level), such that VH ∩ VL = ∅ and for all edges, the connected vertices are in different subsets: υi ∈ VH ⇔ υj ∈ VL (⇔ symbolises equivalence, i.e. if we know υi is in VH, υjmust be in VL, and vice versa).
A graph can be represented as a dendrogram D, that is, a binary tree with the vertices of the graph G being the tips (or leaves) of the dendrogram D (Fig. 3a). Thus, any internal split (or vertex) of D defines a subset of G. The idea of the algorithm is now to find internal vertices of D so that the subset it defines is a module.
The algorithm has to divide G into a set of modules C such that
each module c ∈ C is a connected subgraph of G. (This means each species has to have a partner).
each vertex υ belongs to exactly one module c. (The uniqueness requirement).
edge weights within a module are higher than edge weights outside modules. (The modularity definition).
To specify point 3 above, Barber (2007) has defined modularity for bipartite networks as
where m is half the total number of observed links in the network and Aij is in fact a binary version of our edge matrix E, with values of 1 if a link between i and j exists. The expected value for each link, based on an appropriate null model, is given in the matrix P. The module to which a species i or j is assigned is ci, cj. The indicator function δ(ci,cj) = 1 if ci = cj and 0 if ci ≠ cj. Q ranges from 0, which means the community has no more links within modules than expected by chance, to a maximum value of 1. The higher Q, the more do the data support the division of a network into modules.
Following Newman (2004b), we can expand eqn. (1) by allowing Aij and Pij to be weighted, rather than binary. To avoid confusion, we then replace Pij by Kij, which is not a normalized probability matrix, but really the matrix of expected weights, based on the null model. Furthermore, for the weighted case, we compute m = ∑i,jAij. We then get:
where , with ki = Ai. and kj = A.j being the marginal totals for rows and columns of A, respectively (Newman 2004b).
One crucial point of our modifications of the original hierarchical random graph algorithm of Clauset, Moore & Newman (2008) was to assign an indicator value to each dendrogram vertex to label it as being within a module, or not. To do so, we have to compute the expected value for each value of Aij in order to be able to evaluate whether the observed value is lower or higher (the term over which eqn. (2) sums). This step is not required if edges are unweighted, since then the expectation will always be the same. For weighted edges, and under random interactions among individuals, however, we would expect the edge eij connecting two nodes i and j representing abundant species to have a high value of wij. Similarly, nodes representing rare species could be expected to have low edge weights.
Thus, at every internal vertex of the tree, the subtree is interpreted as module and the algorithm computes the expectation matrix K based on the cross-product of marginal totals of all species in the module, divided by the sum of the number of observed interactions in that module . (Since we normalized all edge weight to sum to 1, K is actually a probability matrix). In other words: the number of interactions we expect in a cell depends on the number of observations we have for both interacting partners; frequently observed species contribute more to the expected number of interactions than rarely observed species. Ecologically speaking, we interpret the marginal totals as being proportional to the probability of observing this species in interactions. This need not reflect abundance, but rather activity in a given community context. In fact, we would expect modules to change when further species are added, as this ecologically affects not only their relative abundance but also their preferences with different resources becoming available. If the vertex gives rise to a module, that is, if ∑ij ∈ c(Aij−Kij)>0, this vertex is labelled as a module. We can now sum the contributions of all vertices and modules according to eqn. (2) to compute to total modularity of graph G. For a formal description of this part of the algorithm, please see Appendix S1.
The algorithm starts with a random dendrogram, where modularity Q is likely to be very low. Through random swapping of branches and their optimization, Q increases during a simulated annealing procedure. The algorithm stops when a pre-defined number of swaps did not further increase the value of Q.
Random swaps are implemented as exchange of two randomly selected vertices in the dendrogram, subject to the following constraint (Fig. 3b). The vertex to be swapped cannot be a leaf. Since terminal vertices always connect leaves from the two bipartitions Vi and Vj, thus representing an interaction, they can be swapped, while their leaves cannot.
After each swap, the modularity of the entire dendrogram is recomputed (for computational efficiency, only those parts affected by the swap). If the new configuration has a higher value of Q, it is stored and becomes the new best dendrogram, otherwise the previous configuration will be used as the starting point for the next swap. A worse configuration is accepted with the probability , where δQ is the change in modularity from the last configuration to the new one and T is the current temperature of the simulated annealing algorithm. We observed that the algorithm converges notably faster if the temperature is not decreased monotonously, but rather set back to the average temperature at which an increase in Q occurs. This is also a better approach in our case, since we do not know, a priori, how many steps the algorithm will take or which value of Q can be obtained.
Since the hierarchical dendrogram is computed through iterative proposing, evaluation and rejecting dendrogram structure in a Markov Chain Monte Carlo approach, Clauset, Moore & Newman (2008)'s, and hence our, algorithm cannot guarantee finding the optimal module configuration. Since the algorithm is coded in C++, even billions of MCMC swaps are feasible in a few minutes, yielding reasonable results for typically sized ecological networks (see below) at acceptable handling time. For large networks, this algorithm can run for hours to days. See appendix for an example session on how to employ the algorithm through R (R Development Core Team 2012).
Output & nested modules
The algorithm returns an object identifying modules and sequence vectors for species, as well as a re-order network ready for visualization of modules and the modularity Q. It can be plotted as an interaction matrix (Fig. 4, top).
QuanBiMo can be invoked recursively, searching for modules within modules (Fig. 4, bottom). When doing so, QuanBiMo will ignore links outside the inner module and treat the ‘parent’ module as all there is. While such nested modules become ever smaller and are thus ever faster to detect, there are plenty of them, and hence, nesting will typically dramatically prolong the search for patterns.
Evaluation of QuanBiMo
We analysed over 2000 simulated networks with known module structure, as a factorial combination of two sizes, seven levels of noise, two number of modules and two sampling intensity. The voluminous output is presented in the supplementary material; here, we only briefly summarize our conclusions from these analyses.
Our simulations show that the analysis of quantitative networks is clearly superior to that of binary data of the same network (Fig. 5).
Modularity Q and accuracy are substantially higher in this case. This seems to be the logical consequence of implicitly down-weighting rare species and singletons, whose correct allocation to modules is given a lower importance than that of species with more information (i.e. a higher number of interactions). In a sense, the outcome of such a comparison is trivial: a binary network does not allow any weighting of species by information content. Also, the null model implicit in the modularity criterion of Newman (2004b, eqn. (2)) can be computed with more nuances and hence has a higher chance of correctly identifying modules.
The QuanBiMo algorithm is by no means perfect: at high levels of noise, such as one would expect in ecological network data, only two-thirds to three-quarter of all links are correctly classified (Appendix Fig. 9 right). The levelling-off at low noise levels gives us some confidence that a misclassification of 25% in small networks seems realistic, which seems acceptable. For large networks, however, the misclassification of over 40% seems too high for comfort, though. Adapting the settings (specifically the number of steps of no improvement before the algorithm stops) seems a logical option. However, runtimes increase substantially with every order of magnitude of number of steps. The price may be worth paying, as the example of the Memmott network shows, where increasing the values for steps from 1e5 to 1e10 boosted Q from 0·1 to 0·3 (see Fig. 4 for the resulting modules).
Modularity Q as a network index
Modularity Q is likely to be correlated with other network metric, as specialization of module members is the prime reason for the existence of modules. Across the 22 quantitative pollination networks of the NCEAS ‘interaction webs’ database 1, Q was evidently highly positively correlated with complementary specialization H2’ (Fig. 6).
Ecologically, the correlation with specialization makes good sense. Modules only exist because some species do not interact with some others, that is, because they are specialized. An overall low degree of specialization is equivalent to random interactions, which will yield no modules.
Furthermore, the absolute value of Q is, like virtually all network indices (Dormann et al. 2009), dependent on network size (i.e. the number of species) as well as the number of links and the total number of interactions observed (see also Thébault 2013). We would thus recommend a null model comparison (e.g. Vázquez & Aizen 2003; Blüthgen et al. 2008; Dormann et al. 2009), to correct the observed value of Q by null model expectation (e.g. by standardizing them to z-scores:; see appendix for an example). The choice of the null model is and will be contentious, since it is not clear whether observed interactions are independent from each other and among species. They may thus differ among network types (e.g. pollination networks may be usefully compared to a null model based on marginal totals, while host–parasitoid networks, with a stronger feedback between the two levels, are likely to be overcorrected by such an approach). Modularity Q is in itself not an index of an ecological feature. It is merely a measure of how well links and interactions can be separated into different modules. Large networks, with many species and links, allow for more combinations of species in modules, leading to higher values of Q, as Allesina (2009) pointed out for any grouping algorithm.
Using modularity to identifying species with important roles in the network
Guimerà et al. (2005) and Olesen et al. (2007) propose to compute standardized connection and participation values, called c and z, for each species to describe their role in networks, where c refers to the even distribution of links across modules (called ‘participation coefficient’ P by Guimerà et al. 2005) and z refers to within-module degrees. Originally, both are computed based on the number of links, but a weighted version based on species strength (sensu Bascompte et al. 2006) is implemented, too. Guimerà et al. (2005) suggest critical values for a binary and one-mode networks of c and z of 0·625 and 2·5, respectively. Species exceeding both of these values are called ‘hubs’ because they link different modules, combining high between-module with high within-module connectivity.
In the case of the pollination network of Fig. 1, c-values range between 0 and 0·78 (with 23 of 79 pollinators and 13 of 25 plant species exceeding the threshold of 0·625); z-values range between −1·21 and 5·00 (with two pollinators but no plant species exceeding the value of 2·5: Fig. 7). Put together, only the syrphid Syritta pipiens (and hawkbit Leontodon hispidus almost) exceeded both thresholds and would thus be called a ‘hub species’. As can be seen in Fig. 4 (top), this syrphid is relatively rare but clearly not randomly distributed over the six modules, thus linking modules three, five and six (from the left). In contrast, Leontodon hispidus is a common plant species, visited by many different pollinators, and it actually links all modules with the exception of module two.
To objectively define this threshold one could run null models of the original network and employ 95% quantiles as critical c- and z-values. For the pollinators in the network of Fig. 1 these would be 0·67 (±0·039) and 1·45 (±0·220), respectively, based on 100 null models (for plants: ccritical = 0·72 ± 0·036 and zcritical = 1·78 ± 0·297; Fig. 7 left). While for plant species this has little effect (except for moving Leontodon hispidus across the threshold), three more pollinators would become hub species (the common hoverfly Episyrphus balteatus, the tachinid fly Eriothrix rufomaculata and undetermined fly ‘Diptera spec.22’).
We here presented an algorithm to compute modularity Q and detect modules in weighted, bipartite networks. Because it uses the strength of links as quantitative information, this approach is more sensitive and also more specific, than current binary algorithms. By making the algorithm easily available, we hope that network ecology will benefit from new insights into the structure of interaction networks.
We like to thank Aaron Clauset for inventing the hierarchical random graph idea and for making his code freely available. CFD acknowledges funding by the Helmholtz Association (VH-NG 247). Many thanks to Lili Ingmann and Matthias Schleuning for extensively testing the robustness and ecological meaningfulness of this algorithm. Additional thanks to Peter Csermely and Pedro Jordano for drawing our attention to some work in progress in this field, to Pedro Peres-Neto and an anonymous reviewer for improving the quality of this publication.