#### The problem of collective action

When interactions occur among more than two players, the problem of cooperation is usually described as follows. Cooperators pay a cost for making a contribution that defectors do not pay; the contributions are gathered, transformed in some way, and then redistributed to everybody (including to any of the defectors). A *collective action problem* arises: defectors can free ride on the efforts of the cooperators, so how is cooperation stable (Olson 1965)? Social dilemmas of this kind occur at all levels of biological organisation, from self-promoting genetical elements (e.g. Burt & Trivers 2006), to microbes (e.g. Crespi 2001), to vertebrates (e.g. Creel 1997; Bednarz 1988), to human societies (e.g. Hardin 1968; Kollock 1998).

The solution to the problem of cooperation is usually believed to be some form of positive assortment among cooperators. Assortment can be due to kin selection (Hamilton 1964; reviewed by Frank 1998; Grafen 2009) brought about, for example, by kin discrimination or spatial structure (Grafen & Archetti 2008; Lehmann *et al.* 2007) or by repeated interactions, which allow reciprocation, reputation effects, or punishment (Axelrod & Hamilton 1981). This section reviews recent articles showing that cooperation in *N-*player social dilemmas can be maintained even in the absence of all assortment mechanisms.

To understand cooperation in *N-*player interactions, we must introduce the concept of the *public good* (Samuelson 1954), which is any good that is non-excludable (no one can be prevented from using it) and non-rivalrous (consumption does not reduce availability to other users). In practise, public goods are often partially excludable (*club* goods) or rivalrous (*common* goods), but club and common goods can be treated approximately as public goods.

#### Linear public goods

The simplest model of the production of a public good in *N*-player interactions is the *N-*player version of the prisoner’s dilemma (NPD; Hamburger 1973): individuals can be cooperators or defectors; cooperators pay a contribution *c* that defectors do not pay. Contributions are summed, the sum is multiplied by a reward factor *r* (> 1), and then redistributed to all individuals (both cooperators and defectors). If groups of *N* individuals are formed at random, i.e. without assortment, the probability of having *j* cooperators among the other *N*-1 individuals is *f*_{j} (*x*). In an infinitely large population, *f*_{j} (*x*) would be the probability mass function of a binomially distributed random variable with parameters (*N−1*, *x*), where *x* is the frequency of cooperators in the population. The fitness of a defector and of a cooperator are, respectively:

If *r*/*N *<* *1, which is reasonable in any sizeable group, at equilibrium, everybody defects. Hence, in the NPD, cooperation requires additional forces like spatial structure, kin discrimination, or repeated interactions to allow cooperators to interact preferentially with cooperators (if *r/N *>* *1, there is no dilemma, and everybody cooperates). The NPD, however, is only a special case of the public goods game. It relies on the critical assumption that the public good is a *linear* function of the individual contributions. This apparently innocuous assumption has fundamental consequences, which have been largely ignored until recently.

Is linearity at least justified from an empirical point of view? The answer is a firm no: nonlinear public goods are the rule in biology. For instance, in microbes producing extracellular enzymes (Crespi 2001; Lee *et al.* 2010; Gore *et al.* 2009; Chuang *et al.* 2010), the benefit of enzyme production is generally a saturating function (Hemker & Hemker 1969), a sigmoid function (Ricard & Noat 1986) or a step function (Mendes 1997; Eungdamrong & Iyengar 2004) of its concentration. Similarly, in vertebrate social behaviour, benefits from alarm calls, group defence and cooperative breeding are also saturating functions of the number of cooperators (Rabenold 1984; Bednarz 1988; Packer *et al.* 1990; Stander 1991; Creel 1997; Yip *et al.* 2008). As far as we know, no evidence of linear public goods exists except for human systems, and the NPD is therefore not a biologically realistic model of *N-*player interactions except in human experiments. Nonetheless, it has become so customary to use the NPD in the study of cooperation, that the two terms (NPD and public goods game) have become virtually synonymous in evolutionary biology. This is unfortunate, as nonlinear public goods games have completely different results. To see why, let us keep the assumption of random group formation (no assortment) and relax, instead, the assumption that the public good is a linear function of the individual contributions.

#### Nonlinear public goods

We define an *N-*player public goods game by a benefit *β*( *j* ) monotonically increasing in *j* (the number of cooperators) and a cost *γ*( *j* ) monotonically decreasing in *j*. Note that the benefits need not be equal for cooperators and defectors, since any difference in benefit can be incorporated in the cost *γ*, without loss of generality (thus, we can include scenarios in which cooperators benefit more from their own contributions than do defectors). The average fitness of a defector and of a cooperator are, respectively:

As we are assuming no assortment, in an infinitely large population

The simplest nonlinear benefit function is the Heaviside step function:

and we can assume that . In economics, this game is sometimes called the Volunteer’s Dilemma if *k *=* *1 (Diekmann 1985) or the Teamwork Dilemma if *k *>* *1 (Myatt & Wallace 2009; see also Palfrey & Rosenthal 1984). For brevity, we use Volunteers’ Dilemma (VD) to cover any *N-*player game in which the public good is produced only if at least *k* (≤ *N*) individuals pay a cost *c*. The dilemma is that each individual would rather avoid the cost *c* of volunteering and exploit the public good produced by others, but if the public good is not produced, everybody pays a cost higher than that of volunteering. The equilibrium frequency of cooperators *x*_{eq} can be found exactly (numerically) from (Archetti 2009b,c)

At equilibrium, therefore, cooperators and defectors coexist (Fig. 2), and, in large groups, the frequency of cooperators is approximately the frequency of volunteers (*k/N*) necessary to produce the public good. This mixed equilibrium (or polymorphic equilibrium of defectors and cooperators), however, exists only if the *c*/*b* ratio is below a critical threshold (Fig. 2) (Archetti & Scheuring 2011). The equilibrium is still Pareto-inefficient (an improvement is possible by which at least one player could receive a higher payoff without reducing any other player’s payoff: hence the inefficiency; no one, however, has an incentive to change their behaviour: hence the equilibrium), like in the NPD. But unlike in the NPD, there is no dominant strategy. Cooperation is maintained at intermediate levels (cooperators and defectors coexist) in the absence of any additional force.

The frequency of cooperators at the stable equilibrium *x*_{eq}^{+} can be very close to the frequency *x*_{M} required to produce maximum fitness, and fitness at *x*_{eq}^{+} can be close to fitness at *x*_{M} (Fig. 2). Although the existence of the mixed equilibrium depends on the *c*/*b* ratio, as long as this equilibrium exists, the amount of public good produced is affected only slightly by the value of *c*, especially in large groups (Archetti & Scheuring 2011). Variations of the game that allow the cost of cooperation to be inversely related to the number of cooperators (Souza *et al.* 2009), or in which the public good increases linearly only above a threshold (Pacheco *et al.* 2009) behave similarly. So far, however, we have replaced a biologically unlikely assumption on the shape of the public good (linearity: the NPD) with only a slightly more likely assumption (a step function). What happens if we use a more realistic benefit function?

It can be shown (Fig. 4; Archetti & Scheuring 2011) that changing the shape of the public good (*s*) changes the position of the mixed equilibrium only slightly, unless the function is almost linear (*s*0), returning us to an NPD. Therefore, many social dilemmas can be approximated by the VD, and the results we have discussed for the VD hold for most public goods games. Seen in this general formulation, the NPD is just a degenerate case in which the mixed equilibrium does not exist and for which relatedness or other forms of assortment are needed to maintain cooperation. The existence of a mixed equilibrium in which cooperators and defectors coexist is the main result that emerges from the analysis of nonlinear public goods.

The reason why cooperators and defectors coexist in nonlinear public goods games can be explained intuitively. Consider first the simplest nonlinear public goods game: a public good is produced if and only if at least one individual cooperates. Why do cooperators and defectors coexist? Because if nobody else cooperates, the best strategy is to cooperate (not getting one’s share of the public good is assumed to be worse than paying the cost of volunteering); but if someone else cooperates, the best strategy is to defect (the public good is produced anyway, and the cost of volunteering is spared). The result is that the best strategy is to cooperate with some probability (which depends on the cost/benefit ratio and group size). Now, imagine instead that the public good is produced if and only if at least *k* (>1) individuals volunteer: if *k*-1 other individuals volunteer, the best strategy is to volunteer, but if *k* or more other individuals already volunteer, the cost of volunteering is wasted. Again, the best strategy is to volunteer with a probability that is a function of the cost/benefit ratio, group size, and *k*. The next step is to allow the public good to be a smooth sigmoid function rather than a step function. When the steepness (*s*) of this function is high, the public good is almost a step function, as in the previous case (Fig. 4) and the result is similar; volunteering with a probability. The coexistence of cooperators and defectors is possible as long as the cost of volunteering *c* is not too high. As the public good approaches a linear function, this critical cost *c* decreases, and when the public good is linear (NPD), no coexistence is possible for *c *>* *0.

In summary, cooperation in *N-*player games should be modelled using the theory of general public goods. The NPD predicts that public goods can be produced only in the presence of some kind of assortment (due to kin selection or iterations). However, unlike the NPD, most public goods are nonlinear functions of the individual contributions. The VD or more general public goods games are the appropriate analytical tools in these cases, and these games have a mixed equilibrium in which cooperators and non-cooperators coexist in the absence of any form of assortment.

This does not mean that assortment is irrelevant. One reason is that, because the mixed equilibrium is generally not efficient, some assortment (but not too much – see Fig. 2) can improve fitness. Another reason is that assortment (for example genetic relatedness) can reduce the critical value of the cost/benefit ratio for which the mixed equilibrium exists; if a mixed equilibrium exists, however, increasing relatedness does not much improve the production of the public good (Archetti 2009a,b). Finally, assortment can help spread cooperation in a population of all defectors. It must be clear, however, that assortment is not necessary for the stability of public goods and the coexistence of cooperators and volunteers.

#### Implications for the evolution of cooperation

The theory of public goods is useful for understanding some recent developments in the study of cooperation in microbial systems.

While there is no doubt that assortment, and relatedness in particular, is an important determinant of cooperative behaviour, there is a tendency to attribute *a priori* any instance of cooperation to assortment. For example, Lee *et al.* (2010) recently reported that general antibiotic resistance in a non-structured population of *Escherichia coli* is due to a tiny minority of antibiotic-resistant cells that, at personal cost, export the chemical indole, which induces drug efflux and oxidative stress protection in the non-resistant cells. Lee *et al.* (2010) tentatively attributed their results to kin selection. However, their observations of the stable coexistence of cooperators and defectors and of higher group benefit (colony growth rate) occurring at intermediate frequencies of cooperators could be easily explained by the theory of nonlinear public goods without resorting to relatedness. Similar reasoning could be productively applied to virtually all articles that discuss cooperation in microbes. A review of these studies is not appropriate here, but it is worth relating one example in detail.

Consider invertase production by yeast, one of the most studied examples of public-goods cooperation in microbes; cells that secrete invertase are cooperators. This system was initially described as a prisoner’s dilemma (Greig & Travisano 2004), but the prisoner’s dilemma framework could not explain the coexistence of cooperators and defectors. The system was later described as a snowdrift game (Gore *et al.* 2009), which does explain coexistence. However, describing invertase production as a 2-player game like the snowdrift game is clearly mistaken. MacLean *et al.* (2010) recently elaborated on the invertase system and found that, empirically, maximum group benefit occurs at an intermediate frequency of defectors, which, as they correctly point out, is not predicted by the snowdrift game. While MacLean *et al.* (2010) attribute the mismatch between theory and data to the failure of game theory to deal with complex systems, it is clear that the mismatch is simply due to the fact that invertase production is not a 2-player game but an *N*-player public goods game, for which, as we have shown, maximum group benefit is indeed expected at an intermediate frequency of cooperation, consistent with observation. Game theory can easily incorporate nonlinearities, differential costs and benefits and other elaborations.