## Introduction

Cooperation is ubiquitous in biological systems, and so is its exploitation. Cooperation is a conundrum, whereas its exploitation is not, at least not at first sight. Cooperative entities make a sacrifice: they help others at a cost to themselves. Exploiters, or cheaters, reap the benefits and forego costs. Based on utilitarian principles – be it in the form of evolution by natural selection of the ‘fittest’ type, or in the form of ‘rational’ behaviour generating the highest payoff – exploitation should prevail, and cooperation should be rare.

Yet the history of life on Earth could not have unfolded without the repeated cooperative integration of lower level entities into higher level units. Thus, major evolutionary transitions (Maynard Smith & Szathmáry 1995), such as the evolution of chromosomes out of replicating DNA molecules, the transition from uni-cellular to multi-cellular organisms, or the origin of complex ecosystems, could not have occurred in the absence of cooperative interactions. Similarly, the emergence of complex animal and human societies requires cooperation (Maynard Smith & Szathmáry 1995; Crespi & Choe 1997; Dugatkin 1997).

Since its invention by von Neumann & Morgenstern (1944), the mathematical framework of game theory has been a central tool for understanding how cooperative entities can overcome the obvious fitness and payoff disadvantages and persist in the face of cheating and exploitation. Game theory embodies the concept of frequency dependent selection, which is at the heart of the problem of cooperation, because the actual costs of cooperation ultimately depend on the type of individuals a cooperator interacts with. Maynard Smith & Price (1973) ingeniously related the economic concept of payoff functions with evolutionary fitness, thus marking the advent of an entirely new approach to behavioural ecology that inspired numerous theoretical and empirical investigations. In particular, evolutionary game theory has been used extensively to study the problem of cooperation (Nowak & Sigmund 2004).

These attempts go back to a seminal paper by Trivers (1971), in which he introduced the notion of reciprocal altruism. This notion embodies the idea that cooperation may evolve in a context in which future behaviour may be determined by current payoffs. Reciprocal altruism was famously embedded into evolutionary game theory by Axelrod & Hamilton (1981). Their models are based on the Prisoner's Dilemma game (PD), perhaps the single most famous metaphor for the problem of cooperation (Box 1). In this game, natural selection favours defection and thereby creates a social dilemma (Dawes 1980), because when everybody defects, the mean population payoff is lower than if everybody had cooperated. In the past two decades, it has been a major goal of theoretical biology to elucidate the mechanisms by which this dilemma can be resolved.

The social dilemma of the PD can be relaxed by assuming that cooperation yields a benefit that is accessible to both interacting players, and that costs are shared between cooperators. This results in the so-called Snowdrift game (SD), which is also known as the Hawk-Dove game, or the Chicken game (Maynard Smith 1982; Sugden 1986, Box 1). Its essential ingredient is that in contrast to the PD, cooperation has an advantage when rare, which implies that the replicator dynamics (Taylor & Jonker 1978; Hofbauer & Sigmund 1998) of the SD converges to a mixed stable equilibrium at which both C and D strategies are present. Starting with Maynard Smith & Price (1973), the SD (or Hawk-Dove game) has been well studied in the context of competition and escalation in animal conflicts, but its role as a simple metaphor in the broader context of the evolution of cooperation has been much less emphasized. In spite of this, we think that the SD may actually be widely applicable in natural systems.

Here we review models of cooperation that are based on the PD and SD games. Since the dynamics of these models is easily understood (Box 1), studying suitable extensions can reveal mechanisms by which cooperation can either be enhanced or reduced as compared with the baseline models. In particular, since the PD does not allow for cooperation, any extensions that do can be viewed as representing mechanisms that promote cooperation. The essential feature of any mechanism to promote cooperation is that cooperative acts must occur more often between cooperators than expected based on population averages. Thus, there must be positive assortment between cooperative types (Queller 1985). In the PD, positive assortment can for example arise because of direct reciprocity in iterated interactions, due to spatially structured interactions, or because of indirect reciprocity with punishment and reward. We first review insights gained from such extensions about the conditions under which cooperation can thrive in models based on the PD. We then demonstrate that the same mechanisms may not always give rise to positive assortment, and hence increased cooperation, in the SD. This is surprising, because both games represent social dilemmas, and if anything, the relaxed conditions in the SD would appear to be in favour of cooperators. Nevertheless, extensions of the SD game can reveal general principles for the evolutionary dynamics of cooperation. The PD and the SD are simple mathematical models, but much has been learned from analysing these games and their extensions about one of the core problems in evolutionary biology. We conclude by outlining promising directions for further explorations of the evolution and maintenance of cooperation based on these games and their applications in empirical model systems.

### Box 1: The Prisoner's Dilemma and Snowdrift games

In the PD, players can adopt either one of two strategies: cooperate (C) or defect (D). Cooperation results in a benefit *b* to the opposing player, but incurs a cost *c* to the cooperator (where *b* > *c* > 0); defection has no costs or benefits. This results in the following payoffs (Table 1a): if the opponent plays C, a player gets the reward *R* = *b* − *c* if it also plays C, but it can do even better and get *T* = *b* if it plays D. On the other hand, if the opponent plays D, a player gets the lowest payoff *S* = −*c* if it plays C, and it gets *P* = 0 if it also defects. In either case, i.e. independent of whether the opponent plays C or D, it is, therefore, better to play D. In evolutionary settings, payoffs determine reproductive fitness, and it follows that D is the evolutionarily stable strategy (ESS) (Maynard Smith 1982). This can be formalized using replicator dynamics (Taylor & Jonker 1978; Hofbauer & Sigmund 1998), which admits pure defection as the only stable equilibrium.

In the SD, cooperation yields a benefit *b* to the cooperator as well as to the opposing player, and incurs a cost *c* if the opponent defects, but only a cost *c*/2 if the opponent cooperates. This results in the following payoffs (Table 1b): *R* = *b* − *c*/2 for mutual cooperation, *T* = *b* for D playing against C, *S* = *b* − *c* for C playing against D, and *P* = 0 for mutual defection. If *b* > *c* > 0 as before, then C is a better strategy than D if the opponent plays D. On the other hand, if the opponent plays C, then D is still the best response. Thus, both strategies can invade when rare, resulting in a mixed evolutionarily stable state at which the proportion of cooperators is 1−*c*/(2*b* − *c*). It is important to note that in this state the population payoff is smaller than it would be if everybody played C, hence the SD still represents a social dilemma (Hauert & Doebeli 2004).