By continuing to browse this site you agree to us using cookies as described in About Cookies
Notice: Please be advised that we experienced an unexpected issue that occurred on Saturday and Sunday January 20th and 21st that caused the site to be down for an extended period of time and affected the ability of users to access content on Wiley Online Library. This issue has now been fully resolved. We apologize for any inconvenience this may have caused and are working to ensure that we can alert you immediately of any unplanned periods of downtime or disruption in the future.
Francisco C. Santos, IRIDIA CP 194/6 – Université Libre de Bruxelles, Avenue Franklin Roosevelt 50, 1050 Bruxelles, Belgium. Tel.: +32 02 6502712; fax: +32 02 6292715; e-mail: email@example.com
The Prisoner's Dilemma (PD) constitutes a widely used metaphor to investigate problems related to the evolution of cooperation. Whenever evolution takes place in well-mixed populations engaged in single rounds of the PD, cooperators cannot resist invasion by defectors, a feature, which is somewhat alleviated whenever populations are spatially distributed. In both cases the populations are characterized by a homogeneous pattern of connectivity, in which every individual is equivalent, sharing the same number of neighbours. Recently, compelling evidence has been accumulated on the strong heterogeneous nature of the network of contacts between individuals in populations. Here we describe the networks of contacts in terms of graphs and show that heterogeneity provides a new mechanism for cooperation to survive. Specifically, we show that cooperators are capable of exploring the heterogeneity of the population structure to become evolutionary competitive. As a result, cooperation becomes the dominating trait in scale-free networks of contacts in which the few highly connected individuals are directly inter-connected, in this way contributing to self-sustain cooperation.
A community is efficiently modelled as a set of n agents, each one endowed with one of two strategies – cooperator or defector. In the one-shot PD, each pair of agents interacts only once. A defector exploiting a cooperator gets an amount T and the exploited cooperator receives S, both agents receiving R upon mutual cooperation and P upon mutual defection, such that T > R > P > S. Therefore, it is best to defect regardless of the opponent's decision. In well-mixed populations, in which each agent interacts with all other agents, evolution under replicator dynamics (Hofbauer & Sigmund, 1998; Gintis, 2000) shows that cooperators are unable to resist invasion by defectors. The fate of cooperators is somewhat alleviated whenever the PD evolves on a spatially structured population (Nowak & May, 1992), such that agents are constrained to play with their nearest–neighbours. Indeed, in spatially structured populations cooperators are now able to resist invasion by defectors, but only for a very limited range of game parameters (see discussion in connection with Fig. 3). Nevertheless, the impact of topological constraints is known to induce profound evolutionary effects, as beautifully demonstrated experimentally in the study of the evolution of different strains of Escherichia coli (Kerr et al., 2002).
The two scenarios described above may be naturally associated with regular graphs (Fig. 1a), in which agents occupy the vertices and the Network Of Contacts (NOCs) is defined by the edges linking the vertices. They correspond to homogeneous NOCs, in the sense that all agents are topologically equivalent, having the same number of neighbours, such that the associated graphs exhibit a characteristic single-peak shape for the degree distribution d(k), defined and illustrated in Fig. 1.
Recently, compelling evidence has been accumulated that a plethora of natural, social and technological real-world NOCs are heterogeneous (Barabási & Albert, 1999; Amaral et al., 2000; Albert & Barabási, 2002; Dorogotsev & Mendes, 2003), exhibiting multi-peaked degree distributions (Fig. 1), reflecting the diversity of connections of different agents. Moreover, they often portray the coexistence of local connections (spatial structure) with nonlocal connections (or shortcuts). The celebrated small-world graphs (Watts, 1998) and scale-free graphs (Barabási & Albert, 1999) illustrated in Fig. 1 provide examples of such NOCs, which have been recently associated with realistic NOCs.
The heterogeneity of NOCs is known to have a strong impact in different fields, notably epidemiology, the case of AIDS constituting a paradigmatic example (May et al., 2001). At the heart of the theory, heterogeneity leads to an ubiquitous modification of the basic reproductive number R0, which ultimately defines the threshold for epidemic outbreaks (Anderson & May, 1991; May & Lloyd, 2001; May et al., 2001). Compared to its homogeneous counterpart, the modified R0 grows proportionally to the second moment of the degree distribution, so that on extreme heterogeneous NOCs, such as scale-free NOCs, the likelihood of epidemic outbreaks dramatically increases. This, in turn, is associated with the fact that, since now different individuals undergo different numbers of contacts, highly connected individuals will naturally acquire higher chance of infecting others (becoming infected) whenever they are infected (susceptible).
How does heterogeneity affect the co-evolution of defectors and cooperators under natural selection?
In heterogeneous populations, different agents will typically undergo a different number of interactions, which in turn will proceed along different patterns of connectivity. Since the payoff accumulated by each agent throughout one generation (see below) will be used as its fitness, dictating the success with which the agent's strategy will be replicated in the next generation, heterogeneity will certainly have an impact on the co-evolution of defectors and cooperators.
Figure 2 illustrates a detail of a typical heterogeneous community. We concentrate on two focal agents, a cooperator with N1 neighbours (N1 = 7), and a defector with N2 neighbours (N2 = 5), and let us imagine that both have the same number of defectors, say (ND = 2). After interacting with all their neighbours, the cooperator accumulates a payoff of PC = (N1 − ND)R + NDS whereas the defector ends up with PD =(N2 − ND)T + NDP. If the NOCs were homogeneous, all one needs to do is to make N1 = N2 = z, the average connectivity of the population, in the previous expressions. This, however, leads to very different outcomes in what concerns the accumulated payoffs in homogeneous and heterogeneous NOCs. Indeed, whereas in homogenous NOCs the answer to whether PD > PC relies exclusively on the relative ordering of (T, R, P, S), in heterogeneous NOCs the answer depends now on the fact that N1 ≠ N2. In other words, the pattern of connectivity also contributes to define the accumulated payoff of each individual, a feature, which, being natural, is absent in homogeneous NOCs. Indeed, heterogeneity embeds the intuition that agents pertaining to different neighbourhoods are involved in different patterns of connectivity, opening a new route to the evolution of cooperation: Cooperators will increase their fitness to the extent they succeed in maximizing their amount of cooperative interactions per generation. However, defectors will also increase their fitness by exploiting more cooperators per generation. Therefore, it remains an open problem who – defectors or cooperators – will be able to profit from heterogeneity, and to which extent.
Because no analytic solutions exist for this problem, agent-based simulations (Nowak & May, 1992; Hauert & Doebeli, 2004) provide a viable alternative to study the evolution of cooperation in realistic populations, a framework we shall adopt here.
Networks of contacts
The histogram degree distribution d(k) illustrated in Fig. 1 is defined, for each particular graph with n vertices, as d(k) = nk/n, where nk gives the number of vertices with k edges. In terms of the degree distribution, the average connectivity can be written as z = Σk d(k). We shall consider two basic types of heterogeneous NOCs: Watts–Strogatz NOCs (Watts, 1998), for which heterogeneity is moderate, and scale-free NOCs, for which heteregeneity is strong. Starting from a regular ring (Fig. 1a) with a fixed number N of vertices with z connections per vertex, we generate a Watts–Strogatz graph by rewiring, with probability pSW, each edge of the graph (Fig. 1b). Rewiring means here replacing the original edge maintaining its origin and choosing randomly the ending vertex, such that self-connections – loops – and double connections are excluded. For pSW =0, we have a regular graph, whereas for pSW = 1 we obtain a graph very similar to a random graph, except that there are no vertices with connectivity smaller than z/2. The larger the probability pSW, the more heterogeneous is the associated graph, and we shall carry out simulations on Watts–Strogatz NOCs with maximized heterogeneity (pSW = 1, Fig. 1c).
In order to study scale-free, strongly heterogeneous NOCs, characterized by many vertices of small connectivity and a few highly connected vertices (so-called hubs) we shall make use of the simple scale-free model of Barabási & Albert (1999), illustrated in Fig. 1d. The construction of such a scale-free graph involves two processes: 1 – Growth: starting with a small number (m0) of vertices, at every time step we add a new vertex with m ≤ m0 edges that link the new vertex to m different vertices already present in the system; 2 – Preferential attachment: When choosing the vertices to which the new vertex connects, we assume that the probability pi that a new vertex will be connected to vertex i depends on the degree ki of vertex i: pi = ki/Σki. Preferential attachment corresponds to well-known mechanisms of overwhelming importance in other areas of science, namely the ‘rich get richer’ effect in economics (Simon, 1955), as well as the ‘Matthew effect’ in sociology (Merton, 1968). After t time steps this algorithm produces a graph with n =t + m0 vertices and mt edges. Because vertices appear at different moments in graph-generation time, so-called age-correlations arise (Albert & Barabási, 2002; Dorogotsev & Mendes, 2003). In order to single out the role of heterogeneity in evolution, we subsequently exchange randomly and repeatedly the ends of pairs of edges of the original graphs (Maslov & Sneppen, 2002; Molloy & Reed, 1995), in this way eliminating any type of correlations between the vertices, without changing the scale-free degree-distribution. In our study, scale-free graphs with n individuals have been generated, starting with m = m0 =z/2 vertices and iterating the appropriate number of time-steps until n vertices are created. It is on top of these graphs that we initiate the evolutionary studies with the PD.
In all our simulations, the NOCs will be associated with static graphs (that is, topology remains unchanged throughout evolution) with as many vertices as agents in the population, each agent adopting a well-defined strategy – cooperator or defector. Since we fix the size n and the average connectivity z, in all cases the number of edges of all graphs will be the same.
The prisoner's dilemma game
Following Nowak & May (1992), we shall adopt a simple one-parameter formulation of the PD such that 2 ≥ T = b > R = 1 > P = S = 0, b representing the temptation to defect.
Given a population of n agents (the n vertices of the graph), inter-connected following the edges of the graph, during each generation (which constitutes our unit of discrete evolutionary time), all pairs of individuals x and y, directly connected, engage in a single round of the game, their accumulated payoff being stored as Px and Py, respectively. This means that during one generation, there will be as many rounds of the game as edges in the NOCs. At the end of a generation, after all individuals have played once with all their partners, all strategies are updated synchronously, that is, one selects which strategy will occupy each vertex in the next generation simultaneously for all vertices.
At the end of a generation agents will have an accumulated payoff resulting from their one-round interactions with each of their immediate neighbours. This accumulated payoff is associated with the individual fitness, which in turn dictates whether the strategy located in a given vertex will be replicated to the next generation or, instead, will be replaced by the strategy of a better fit neighbour.
In their seminal work Foster & Young (1990) introduce stochastic effects in order to account, among other features, for the ‘…variability in the number of individuals of a given type who meet individuals of another type in any given time period…’. Clearly, in biological terms not only quality but also quantity actually matters (Hammerstein, 2003). As such, the accumulated payoff provides a natural choice for the individual fitness of these agents at the end of their life-cycle (one generation) and, due to the heterogeneity of the NOCs, accounts for the fact that different individuals interact at different rates throughout their life span. Such a choice for fitness goes in line with recent studies on cooperation (Abramson & Kuperman, 2001) economics (Jackson & Wolinsky, 1996) social learning (Young, 1998) and the recently developed evolutionary graph theory (Lieberman et al., 2005). In this way agents have the opportunity to explore heterogeneity to increase the chance of survival of their strategy to the next generation. We make no discrimination whatsoever with respect to who may benefit from that: both cooperators and defectors may profit from such a possibility.
A more realistic model, on the other hand, should not overlook that individuals who interact more may expend more energy in doing so, which would likely act to effectively diminish their fitness. The work of Jackson & Wolinsky (1996) provides a feasible pathway towards inclusion of such refinements. A simpler (and cruder) model would be to impose a cutoff in the maximum number of interactions that each individual may engage on per generation, in this way redefining the underlying NOCs.
Other studies, on the other hand, have adopted the criterium of dividing the accumulated payoff by the number of interactions that each agent engages on. The rationale behind such a procedure may be generally related to the assumption that agents interact at a similar rate, a feature which would favour such averaging.
Although we believe this reasoning is not in accord with what happens in realistic settings, both biological and social, we shall also study the evolution of cooperation making use of such artificial procedure. Clearly, the previous discussion in connection with Fig. 2 may be easily extended to the case in which averaging of accumulated payoffs is performed, showing that even in this case heterogeneity plays a role, in spite of its natural attenuation due to the partial cancellation resulting from averaging.
At the end of a generation, the strategy associated with a given vertex x will be replicated or not to the next generation according to the following transition probabilities, which constitute the finite population analogue of replicator dynamics (Gintis, 2000; Hauert & Doebeli, 2004), to which the results of our simulations converge in the limit of well-mixed populations (a complete, regular graph with z = n − 1).
We first select a neighbour y at random among all kx neighbours of x. If Py ≤ Px the vertex x will maintain the same strategy in the next generation. Otherwise, the strategy occupying vertex x will be replaced by that associated with vertex y with a probability p = (Py − Px)/[k>(T − S)], where k> is the largest between kx and ky, ensuring that 0 ≤ p ≤ 1.
The present results are very robust with respect to changes in the detailed form used for strategy update: no qualitative changes occur if we adopt an asynchronous updating of strategies (Hauert & Doebeli, 2004) as well as if we replace the denominator k> (T − S) in p by, for example, kmax (T − S), with kmax the maximum degree of connectivity of the network. However, we believe that the update rule we adopted is more suitable to describe the (necessarily) simple biological entities, which may be modelled as engaging in rounds of such a simple game. Indeed, the present update rule does not require sophisticated cognitive capabilities at an individual level, since information processed by each agent is restricted to his immediate neighbourhood. On the contrary, replacing k> by kmax implies that individuals must be aware of the entire network topology, which requires extraordinary cognitive capabilities in communities with 10 000 individuals.
Individuals are confined to vertices on rings with n =104 vertices, such that each graphs has as many vertices as agents; we used the values z = 4, and m = m0 =2 (z ≤ d(k) ≤ 2m). Following Hauert & Doebeli (2004), equilibrium frequencies of cooperators and defectors are obtained by averaging over 1000 generations after a transient time of 10 000 generations. We confirmed that averaging over larger periods or using different transient times did not change the results, although the transient period depends on z and n. Simulations start with an equal percentage of strategies – cooperators and defectors – randomly distributed among the elements of the population. For different values of the parameters, the evolution of the fraction of cooperators who survive evolution as a function of b for the PD has been computed. To this end, each data point depicted in the figures corresponds to an average over 100 simulations: 10 realizations of the same type of NOCs specified by the appropriate parameters (n, z, m = m0) and 10 runs for each of the realizations.
Results and discussion
The results of our agent-based simulations for Watts–Strogatz NOCs, whenever heterogeneity is maximized (pSW = 1, Fig. 1c), are shown in Fig. 3 with solid circles. When compared with the corresponding results obtained on homogeneous NOCs for the same values n = 10 000 and z = 4 (Fig. 1a), shown with a solid line, it is clear that heterogeneity, though moderate, strongly affects the evolution of cooperation. Moreover, the results indicate that cooperators profit more from heterogeneity than defectors, with a resulting sizeable enhancement of cooperation, when compared to the results associated with homogeneous NOCs.
Since the heterogeneity of Watts–Strogatz NOCs is moderate when compared to real-world NOCs, it is worth investigating the impact of more realistic scale-free NOCs in the evolution of cooperation. To this end let us start with scale-free model of Barabási & Albert (1999), illustrated in Fig. 1d. However, and in order to single out the scale-free features of the NOCs, without introducing any additional age-correlations, let us randomize the connections without changing the degree-distribution, as detailed in the previous section.
The results obtained on such random scale-free NOCs are shown in Fig. 3 with solid squares, and unravel an additional enhancement of cooperation as a result of such strong heterogeneous features. Indeed, now cooperators are able to resist invasion by defectors for all values of the temptation to defect b. This remarkable result shows clearly that cooperators benefit most from the heterogeneity of the NOCs.
Further insight into the mechanism responsible for the strong enhancements depicted in Fig. 3 is provided by the results shown in Fig. 4, in which we plot the relative distribution of strategies per degree, which allows one to assess directly which degree is preferred by which strategy throughout evolution: in the beginning (upper panel), strategies are equally distributed. Evolution leads to a stationary regime (lower panel) in which the distribution becomes strongly asymmetric. Indeed, cooperators occupy all (few) vertices of high degree, whereas defectors manage to survive only on vertices of moderate degree.
How can this happen? From an individual agent perspective, and given two individuals x and y, if x is a cooperator (defector), y's strategy which maximizes x's payoff in a one-shot PD will be given by max(R,S) [max(T,P)] corresponding, in both cases to a cooperator. In other words, it is better both for defectors and cooperators to be surrounded by cooperators. As such, highly connected defectors surrounded by a significant amount of cooperators will be very successful. The more successful they become, the easier it will be to invade most of their cooperator neighbours. However, in doing so their fitness will tend to decrease, since defector–defector interactions contribute less to fitness than defector–cooperator interactions. As a result, highly connected defectors see their fitness decrease down to values comparable to those attained by the remaining cooperator neighbours, which will then get a chance to invade such a highly connected vertex (hub).
Once cooperators manage to invade such (few) hubs, the situation changes profoundly. Indeed, as stated before, cooperators benefit most by interacting with like cooperators. Therefore, as their fitness increases, so will increase the fraction of their neighbours, which become cooperators, a feature which acts to self-sustain the cooperators sitting on such hubs. In other words, once a cooperator invades a hub, his fitness will rapidly increase, making it very difficult to become invaded by a defector. This, in turn, can only happen to the extent that the NOCs are heterogeneous, since in this way cooperators will maximize their fitness by maximizing the number of cooperative interactions. On the other hand, the previous discussion exemplifies how heterogeneity may be evolutionary disadvantageous to defectors. Indeed, via this mechanism cooperators manage to outperform defectors and endure evolution under natural selection.
This line of reasoning is entirely corroborated by the results illustrated in Fig. 5, in which we show the evolutionary dynamics of a single defector in an otherwise cooperative population in scale-free NOCs.
In the homogeneous limit we know that such an initial condition will lead to the rapid extinction of all cooperators for the values of b employed in Fig. 5: b = 1.5 and b = 1.9. For scale-free NOCs, it is easy to convince oneself that the most advantageous location for placing a single defector is on the vertex with largest connectivity. Starting from this scenario, the evolutionary dynamics is startling, as shown in Fig. 5. The initial defector quickly invades roughly 80% of the nearest (cooperative) neighbours. This will reduce his fitness in such a way that a cooperator nearby suceeds in invading. Cooperator invasion leads to the steep jump observed in the percentage of cooperators surrounding the cooperator hub, providing an impressive account of the self-sustaining efficiency induced by a cooperator-hub. Finally, Fig. 5 shows that, in what concerns the largest hub in the NOCS, the actual value of b acts only to delay the number of generations required for invasion to take place. This result, however is not general and relies on the special initial conditions chosen here.
Understanding how cooperators manage to outperform defectors enables us to design NOCs in which cooperators may acquire an overall evolutionary advantage. In fact, the original NOCs of Barabási–Albert provide an excellent means to this end, since growth and preferential attachment lead to NOCs, which exhibit age-correlations between vertices, very convenient for this purpose. Indeed, during graph generation the older vertices are not only those which exhibit largest connectivity, but also they become naturally interconnected with each other. Such direct inter-connections provide an ideal scenario for highly connected cooperators to self-sustain each other. Indeed, once hubs are occupied by cooperators, the fact that they are directly connected means that it will be even harder for defectors to invade cooperator-hubs. In other words, cooperator-hubs protect each other from defector invasion, which will naturally result in an enhancement of cooperation. This reasoning is fully supported by the corresponding results shown in Fig. 3 with solid triangles, where cooperators now dominate over the entire range of values of b, whenever populations exhibit such age-correlation patterns of connectivity between individuals. Taking into account that the degree distributions of the NOCs used to generate both age-correlated and random scale-free results coincide, the results of Fig. 3 put in evidence the power of this mechanism. As expected, cooperation will be suppressed whenever the edges, which directly connect the ‘strong cooperators’ are artificially clipped by hand. In retrospect, the random scale-free NOCs considered before provide an automatic means to destroy such links.
Finally, we would like to point out that the results for the evolution of cooperation are affected if, instead of using the accumulated payoff for each agent, its fitness is associated with the accumulated payoff divided by the number of interactions each agent engages during his life-cycle. The corresponding results are shown in Fig. 6, which shows that heterogeneity is still capable of promoting cooperation, in particular ensuring the survival of cooperators up to larger values of the temptation to defect, when compared to the results in homogeneous populations. Nonetheless, it is clear that by averaging the payoffs, the mechanisms discussed before will be severely hampered, as such precluding the clear cut dominance of cooperators over defectors obtained.
To sum up, the diversity of connectivity patterns in a population, which translates into heterogeneous NOCs, is efficiently explored by cooperators to outperform defectors, leading to evolutionary outcomes in which cooperators easily survive and may even dominate. NOCs exhibiting strong heterogeneities and tight connections between the few most connected agents favour the dominance of cooperation.
Throughout evolution, the topology of the NOCs itself must have evolved, a feature which is not included here. Nonetheless, it is clear that whatever NOCs one may envisage or have emerged throughout evolution, they are certainly heterogeneous, and as such provide a better ground for cooperators to resist defectors and endure selection and evolution. In this context, it is worth mentioning that the impact of heterogeneity in what concerns cooperation between individuals may possibly extend beyond the framework considered here. In what concerns kin-selection, for instance, local dispersal and population structure play a sizeable role (Rousset & Billiard, 2000). To the best of our knowledge, heterogeneity effects in the population have not been addressed yet in such a context. We hope our work will stimulate the study of heterogeneity effects in other areas of evolutionary biology.
The authors thank Tom Lenaerts, Martin Nowak and Fabio Chalub for stimulating discussions. F.C.S. also acknowledges the support of COMP2 SYS, a Marie Curie Early Stage Training Site, funded by the EC through the HRM activity.