The problem of collective action
When interactions occur among more than two players, the problem of cooperation is usually described as follows. Cooperators pay a cost for making a contribution that defectors do not pay; the contributions are gathered, transformed in some way, and then redistributed to everybody (including to any of the defectors). A collective action problem arises: defectors can free ride on the efforts of the cooperators, so how is cooperation stable (Olson 1965)? Social dilemmas of this kind occur at all levels of biological organisation, from self-promoting genetical elements (e.g. Burt & Trivers 2006), to microbes (e.g. Crespi 2001), to vertebrates (e.g. Creel 1997; Bednarz 1988), to human societies (e.g. Hardin 1968; Kollock 1998).
The solution to the problem of cooperation is usually believed to be some form of positive assortment among cooperators. Assortment can be due to kin selection (Hamilton 1964; reviewed by Frank 1998; Grafen 2009) brought about, for example, by kin discrimination or spatial structure (Grafen & Archetti 2008; Lehmann et al. 2007) or by repeated interactions, which allow reciprocation, reputation effects, or punishment (Axelrod & Hamilton 1981). This section reviews recent articles showing that cooperation in N-player social dilemmas can be maintained even in the absence of all assortment mechanisms.
To understand cooperation in N-player interactions, we must introduce the concept of the public good (Samuelson 1954), which is any good that is non-excludable (no one can be prevented from using it) and non-rivalrous (consumption does not reduce availability to other users). In practise, public goods are often partially excludable (club goods) or rivalrous (common goods), but club and common goods can be treated approximately as public goods.
Linear public goods
The simplest model of the production of a public good in N-player interactions is the N-player version of the prisoner’s dilemma (NPD; Hamburger 1973): individuals can be cooperators or defectors; cooperators pay a contribution c that defectors do not pay. Contributions are summed, the sum is multiplied by a reward factor r (> 1), and then redistributed to all individuals (both cooperators and defectors). If groups of N individuals are formed at random, i.e. without assortment, the probability of having j cooperators among the other N-1 individuals is fj (x). In an infinitely large population, fj (x) would be the probability mass function of a binomially distributed random variable with parameters (N−1, x), where x is the frequency of cooperators in the population. The fitness of a defector and of a cooperator are, respectively:
If r/N < 1, which is reasonable in any sizeable group, at equilibrium, everybody defects. Hence, in the NPD, cooperation requires additional forces like spatial structure, kin discrimination, or repeated interactions to allow cooperators to interact preferentially with cooperators (if r/N > 1, there is no dilemma, and everybody cooperates). The NPD, however, is only a special case of the public goods game. It relies on the critical assumption that the public good is a linear function of the individual contributions. This apparently innocuous assumption has fundamental consequences, which have been largely ignored until recently.
Is linearity at least justified from an empirical point of view? The answer is a firm no: nonlinear public goods are the rule in biology. For instance, in microbes producing extracellular enzymes (Crespi 2001; Lee et al. 2010; Gore et al. 2009; Chuang et al. 2010), the benefit of enzyme production is generally a saturating function (Hemker & Hemker 1969), a sigmoid function (Ricard & Noat 1986) or a step function (Mendes 1997; Eungdamrong & Iyengar 2004) of its concentration. Similarly, in vertebrate social behaviour, benefits from alarm calls, group defence and cooperative breeding are also saturating functions of the number of cooperators (Rabenold 1984; Bednarz 1988; Packer et al. 1990; Stander 1991; Creel 1997; Yip et al. 2008). As far as we know, no evidence of linear public goods exists except for human systems, and the NPD is therefore not a biologically realistic model of N-player interactions except in human experiments. Nonetheless, it has become so customary to use the NPD in the study of cooperation, that the two terms (NPD and public goods game) have become virtually synonymous in evolutionary biology. This is unfortunate, as nonlinear public goods games have completely different results. To see why, let us keep the assumption of random group formation (no assortment) and relax, instead, the assumption that the public good is a linear function of the individual contributions.
Nonlinear public goods
We define an N-player public goods game by a benefit β( j ) monotonically increasing in j (the number of cooperators) and a cost γ( j ) monotonically decreasing in j. Note that the benefits need not be equal for cooperators and defectors, since any difference in benefit can be incorporated in the cost γ, without loss of generality (thus, we can include scenarios in which cooperators benefit more from their own contributions than do defectors). The average fitness of a defector and of a cooperator are, respectively:
As we are assuming no assortment, in an infinitely large population
The simplest nonlinear benefit function is the Heaviside step function:
and we can assume that . In economics, this game is sometimes called the Volunteer’s Dilemma if k = 1 (Diekmann 1985) or the Teamwork Dilemma if k > 1 (Myatt & Wallace 2009; see also Palfrey & Rosenthal 1984). For brevity, we use Volunteers’ Dilemma (VD) to cover any N-player game in which the public good is produced only if at least k (≤ N) individuals pay a cost c. The dilemma is that each individual would rather avoid the cost c of volunteering and exploit the public good produced by others, but if the public good is not produced, everybody pays a cost higher than that of volunteering. The equilibrium frequency of cooperators xeq can be found exactly (numerically) from (Archetti 2009b,c)
At equilibrium, therefore, cooperators and defectors coexist (Fig. 2), and, in large groups, the frequency of cooperators is approximately the frequency of volunteers (k/N) necessary to produce the public good. This mixed equilibrium (or polymorphic equilibrium of defectors and cooperators), however, exists only if the c/b ratio is below a critical threshold (Fig. 2) (Archetti & Scheuring 2011). The equilibrium is still Pareto-inefficient (an improvement is possible by which at least one player could receive a higher payoff without reducing any other player’s payoff: hence the inefficiency; no one, however, has an incentive to change their behaviour: hence the equilibrium), like in the NPD. But unlike in the NPD, there is no dominant strategy. Cooperation is maintained at intermediate levels (cooperators and defectors coexist) in the absence of any additional force.
Figure 2. The mixed equilibrium in the volunteers’ dilemma. The grey lines show the fitness for a cooperator (WC, dotted line), for a defector (WD, dashed line), and for the mixed strategy (WM = xWC + (1 − x) WD, black line), as a function of the frequency of cooperators (x). N = 10, b = 1, k = 5. Circles show the equilibria, stable (filled) or unstable (empty); arrows show the change in frequency of cooperators. The grey square shows the value of x for which the mixed strategy has the highest fitness (Wmax). (a) c = 0.5; there is no mixed equilibrium. (b) c = 0.1; at the mixed equilibrium, WM is close to Wmax, and the frequency of cooperators is close to the frequency required for Wmax.
Download figure to PowerPoint
The frequency of cooperators at the stable equilibrium xeq+ can be very close to the frequency xM required to produce maximum fitness, and fitness at xeq+ can be close to fitness at xM (Fig. 2). Although the existence of the mixed equilibrium depends on the c/b ratio, as long as this equilibrium exists, the amount of public good produced is affected only slightly by the value of c, especially in large groups (Archetti & Scheuring 2011). Variations of the game that allow the cost of cooperation to be inversely related to the number of cooperators (Souza et al. 2009), or in which the public good increases linearly only above a threshold (Pacheco et al. 2009) behave similarly. So far, however, we have replaced a biologically unlikely assumption on the shape of the public good (linearity: the NPD) with only a slightly more likely assumption (a step function). What happens if we use a more realistic benefit function?
Figure 3. A general public goods game. The public good is a nonlinear function of the individual contributions: the benefit due to the public good increases monotonically with the number of cooperators; too few or too many contributions add little or nothing to the public good; at intermediate levels of contribution the increase is first synergistic (accelerating) and then discounting (decelerating). This type of public goods is very common in biology; in microbes that secrete extracellular enzymes, for example, the effect of enzyme production, and thus the level of benefit, is generally a saturating function of its concentration. The public-goods function has two characteristics, the inflection point k (black dot), where it shifts from accelerating to discounting, and the steepness s at the inflection point (dashed line). Group size is N. The public-goods function is , where and i is the number of cooperators. In this figure, k = 10, s = 1, and N = 20. When s∞, the public-goods function becomes a step function of the individual contributions, with a threshold k (volunteers’ dilemma). When s0, the public good becomes a linear function of the individual contributions (N-player prisoner’s dilemma) (see Fig. 4).
Download figure to PowerPoint
It can be shown (Fig. 4; Archetti & Scheuring 2011) that changing the shape of the public good (s) changes the position of the mixed equilibrium only slightly, unless the function is almost linear (s0), returning us to an NPD. Therefore, many social dilemmas can be approximated by the VD, and the results we have discussed for the VD hold for most public goods games. Seen in this general formulation, the NPD is just a degenerate case in which the mixed equilibrium does not exist and for which relatedness or other forms of assortment are needed to maintain cooperation. The existence of a mixed equilibrium in which cooperators and defectors coexist is the main result that emerges from the analysis of nonlinear public goods.
Figure 4. Cooperation in public goods games, from the N-player prisoner’s dilemma to the volunteers’ dilemma. For different values of s, the 2-dimensional plots show the amount of public good produced as a function of the number of cooperators (the continuous lines are only for guidance; the public good exists only for integer values of N – the grey dots) and the fitness of cooperators (WC, grey) and defectors (WD, black) as a function of the frequency of cooperators (x). N = 20, c/b = 1/10, k = 10. In the second column, the circles indicate equilibria, stable (black) or unstable (white), and the arrows show the change in the frequency of cooperators. The N-player prisoner’s dilemma corresponds to s0: every contribution increases the public good by a fixed amount proportional to 1/N. The Volunteers’ Dilemma corresponds to s∞: the public good is produced only if at least k individuals volunteer to pay a contribution; an individual contribution adds nothing to the public good unless the number of other cooperators is k-1. In the intermediate cases, too few or too many contributions add little or nothing to the public good, and the increase is first synergistic and then discounting (see Fig. 3). If s0 (N-player prisoner’s dilemma), no mixed equilibrium exists. As s increases, two internal equilibria can exist (one stable, one unstable) because it is beneficial to be a defector only if there are too many or too few volunteers. Note that the position of the equilibria is almost the same for s = 1 and s∞, so nonlinear public goods can be approximated by the volunteers’ dilemma. In contrast, for s0, the results are very different, and therefore the N-player prisoner’s dilemma is not a good representation of public goods games in general. The 3-dimensional plots show the frequency of cooperators (x) at equilibrium and the amount of public good (PG) produced at equilibrium, as a function of the cost-benefit ratio of cooperation c/b and the parameter k (the position of the inflection point in the public goods function; the number of volunteers required for the production of the public good in the volunteers’ dilemma). Note that, when the c/b ratio is such that the mixed equilibrium exists, the public good produced is substantial and almost independent of c/b (Modified from Archetti & Scheuring 2011).
Download figure to PowerPoint
The reason why cooperators and defectors coexist in nonlinear public goods games can be explained intuitively. Consider first the simplest nonlinear public goods game: a public good is produced if and only if at least one individual cooperates. Why do cooperators and defectors coexist? Because if nobody else cooperates, the best strategy is to cooperate (not getting one’s share of the public good is assumed to be worse than paying the cost of volunteering); but if someone else cooperates, the best strategy is to defect (the public good is produced anyway, and the cost of volunteering is spared). The result is that the best strategy is to cooperate with some probability (which depends on the cost/benefit ratio and group size). Now, imagine instead that the public good is produced if and only if at least k (>1) individuals volunteer: if k-1 other individuals volunteer, the best strategy is to volunteer, but if k or more other individuals already volunteer, the cost of volunteering is wasted. Again, the best strategy is to volunteer with a probability that is a function of the cost/benefit ratio, group size, and k. The next step is to allow the public good to be a smooth sigmoid function rather than a step function. When the steepness (s) of this function is high, the public good is almost a step function, as in the previous case (Fig. 4) and the result is similar; volunteering with a probability. The coexistence of cooperators and defectors is possible as long as the cost of volunteering c is not too high. As the public good approaches a linear function, this critical cost c decreases, and when the public good is linear (NPD), no coexistence is possible for c > 0.
In summary, cooperation in N-player games should be modelled using the theory of general public goods. The NPD predicts that public goods can be produced only in the presence of some kind of assortment (due to kin selection or iterations). However, unlike the NPD, most public goods are nonlinear functions of the individual contributions. The VD or more general public goods games are the appropriate analytical tools in these cases, and these games have a mixed equilibrium in which cooperators and non-cooperators coexist in the absence of any form of assortment.
This does not mean that assortment is irrelevant. One reason is that, because the mixed equilibrium is generally not efficient, some assortment (but not too much – see Fig. 2) can improve fitness. Another reason is that assortment (for example genetic relatedness) can reduce the critical value of the cost/benefit ratio for which the mixed equilibrium exists; if a mixed equilibrium exists, however, increasing relatedness does not much improve the production of the public good (Archetti 2009a,b). Finally, assortment can help spread cooperation in a population of all defectors. It must be clear, however, that assortment is not necessary for the stability of public goods and the coexistence of cooperators and volunteers.
Implications for the evolution of cooperation
The theory of public goods is useful for understanding some recent developments in the study of cooperation in microbial systems.
While there is no doubt that assortment, and relatedness in particular, is an important determinant of cooperative behaviour, there is a tendency to attribute a priori any instance of cooperation to assortment. For example, Lee et al. (2010) recently reported that general antibiotic resistance in a non-structured population of Escherichia coli is due to a tiny minority of antibiotic-resistant cells that, at personal cost, export the chemical indole, which induces drug efflux and oxidative stress protection in the non-resistant cells. Lee et al. (2010) tentatively attributed their results to kin selection. However, their observations of the stable coexistence of cooperators and defectors and of higher group benefit (colony growth rate) occurring at intermediate frequencies of cooperators could be easily explained by the theory of nonlinear public goods without resorting to relatedness. Similar reasoning could be productively applied to virtually all articles that discuss cooperation in microbes. A review of these studies is not appropriate here, but it is worth relating one example in detail.
Consider invertase production by yeast, one of the most studied examples of public-goods cooperation in microbes; cells that secrete invertase are cooperators. This system was initially described as a prisoner’s dilemma (Greig & Travisano 2004), but the prisoner’s dilemma framework could not explain the coexistence of cooperators and defectors. The system was later described as a snowdrift game (Gore et al. 2009), which does explain coexistence. However, describing invertase production as a 2-player game like the snowdrift game is clearly mistaken. MacLean et al. (2010) recently elaborated on the invertase system and found that, empirically, maximum group benefit occurs at an intermediate frequency of defectors, which, as they correctly point out, is not predicted by the snowdrift game. While MacLean et al. (2010) attribute the mismatch between theory and data to the failure of game theory to deal with complex systems, it is clear that the mismatch is simply due to the fact that invertase production is not a 2-player game but an N-player public goods game, for which, as we have shown, maximum group benefit is indeed expected at an intermediate frequency of cooperation, consistent with observation. Game theory can easily incorporate nonlinearities, differential costs and benefits and other elaborations.