Worker reproduction and policing in insect societies: an ESS analysis

Authors


Tom Wenseleers, Laboratory of Apiculture and Social Insects, Department of Animal and Plant Sciences, University of Sheffield, Sheffield S10 2TN, UK.
Tel.: (+44) (0)114 222 0144; fax: (+44) (0)114 222 0002;
e-mail: t.wenseleers@sheffield.ac.uk

Abstract

Insect societies are vulnerable to exploitation by workers who reproduce selfishly rather than help to rear the queen's offspring. In most species, however, only a small proportion of the workers reproduce. Here, we develop an evolutionarily stable strategy (ESS) model to investigate factors that could explain these observed low levels of reproductive exploitation. Two key factors are identified: relatedness and policing. Relatedness affects the ESS proportion of reproductive workers because laying workers generally work less, leading to greater inclusive fitness costs when within-colony relatedness is higher. The second key factor is policing. In many species, worker-laid eggs are selectively removed or ‘policed’ by other workers or the queen. We show that policing not only prevents the rearing of worker-laid eggs but can also make it unprofitable for workers to lay eggs in the first place. This can explain why almost no workers reproduce in species with efficient policing, such as honeybees, Apis, and the common wasp, Vespula vulgaris, despite relatively low relatedness caused by multiple mating of the mother queen. Although our analyses focus on social insects, the conclusion that both relatedness and policing can reduce the incentive for cheating applies to other biological systems as well.

Introduction

Group living can be rewarding. Proof of this are several ‘major transitions in evolution’ in which independent lower-level units have joined into larger co-operative groups (Maynard Smith & Szathmáry, 1995). Genes combined to form genomes; genomes co-operated in the formation of cells; cells teamed up into multicellular organisms, and some organisms went on to live in single-species societies or multi-species mutualisms (Maynard Smith & Szathmáry, 1995; Keller, 1999; Queller, 2000). The down side is that any social system can also be threatened by conflicts among its members (Ratnieks & Reeve, 1992; Keller, 1999; Queller, 2000). Given this potential for conflict, what protects the interests of the whole group?

Insect societies, such as ants, bees, wasps and termites, are excellent models to study this basic question, as many aspects of their reproduction can be subject to conflict (Ratnieks & Reeve, 1992; Bourke & Franks, 1995). For example, there can be conflict over queen rearing (nepotism, Ratnieks & Reeve, 1992), swarming (Visscher, 1993), queen replacement (Monnin et al., 2002), sex allocation (Bourke & Franks, 1995; Crozier & Pamilo, 1996), the caste fate of developing individuals (Bourke & Ratnieks, 1999; Wenseleers et al., 2003; Wenseleers & Ratnieks, 2004) and the production of males (Trivers & Hare, 1976; Bourke, 1988). Conflict over male production is of particular importance in eusocial Hymenoptera. In most species, workers have lost the ability to mate, and cannot, therefore, lay fertilized female-destined eggs, but they do retain functional ovaries and so can lay unfertilized, male eggs (Wilson, 1971). This potentially allows workers to rear their sons and nephews rather than the queen's offspring (Fletcher & Ross, 1985; Bourke, 1988; Choe, 1988). Worker reproduction can have significant costs because reproductive workers usually perform little or no useful work in the colony (Landolt et al., 1977; Cole, 1984, 1986; Moritz & Hillesheim, 1985; Ross, 1985; Hillesheim et al., 1989; Martin et al., 2002; Hartmann et al., 2003). Hence, worker reproduction can cause a ‘tragedy of the commons’ (TOC, Hardin, 1968; Wenseleers et al., 2003; Wenseleers & Ratnieks, 2004), with individual exploitation (worker reproduction) potentially causing a cost to the whole group (reduced foraging and lowered colony productivity).

In many species few workers attempt to reproduce and only a small proportion have active ovaries (e.g. Ratnieks, 1993; Foster et al., 2000; Foster & Ratnieks, 2001a). Hence, the TOC does not always occur or is greatly reduced. For example, in the honeybee, Apis mellifera and the common wasp, Vespula vulgaris only approximately 0.01–0.1% and 1%, respectively, of the workers have active ovaries and lay eggs (Ratnieks, 1993; Foster & Ratnieks, 2001a). Why are these societies so co-operative that the majority of workers opt to be sterile and work to rear siblings with just a small minority attempting to reproduce?

The aim of this paper is to develop a theoretical model to provide insight into this long-standing question (Hamilton, 1964; West-Eberhard, 1975; Oster & Wilson, 1978). Two key factors will be considered: relatedness and policing. Relatedness is expected to be important because workers in insect societies are usually related (Hamilton, 1964; Wilson, 1971) and if worker reproduction reduces the rearing capacity of the colony, inclusive fitness costs should favour reduced worker reproduction (Hamilton, 1964; West-Eberhard, 1975; Oster & Wilson, 1978; Cole, 1986; Bourke, 1988). Policing is also expected to affect observed levels of worker reproduction. In many species, worker-laid eggs are selectively removed or ‘policed’ by either the queen or fellow workers (Ratnieks, 1988; Ratnieks & Visscher, 1989; Barron et al., 2001; Foster & Ratnieks, 2001c; Monnin & Ratnieks, 2001). Although egg policing does not directly prevent workers from laying eggs, it should greatly reduce the incentive for them to do so (Ratnieks, 1988; Visscher, 1989; Ratnieks & Reeve, 1992; Bourke, 1999). Hence, Ratnieks (1988) suggested that social policing could potentially select for workers to ‘self-police’, i.e. to choose not to reproduce even when they possess functional ovaries.

The role of relatedness and policing in the evolution of worker reproduction has previously been analysed verbally (Bourke, 1988, 1999; Ratnieks, 1988; Visscher, 1989; Ratnieks & Reeve, 1992). This approach is limited and conclusions have been contradictory. For example, Bourke (1988) suggested that low relatedness should lead to more selfishness and higher levels of worker reproduction. Ratnieks (1988), on the other hand, showed that lower relatedness, caused by multiple mating of queens, should select for worker policing (see below) thereby leading to less worker reproduction. In the current model, we resolve this apparent contradiction by analysing the joint effect of relatedness and policing on the expected level of worker reproduction. Our results show that both relatedness and policing can reduce worker reproduction, but only policing can reduce worker reproduction to very low levels when relatedness is <1. The implications for the evolution of conflict and conflict resolution in social groups in general are discussed.

Background information: policing in social insects and factors promoting it

Before modelling how policing affects worker reproduction, we provide relevant background material, particularly the taxa in which policing is known to occur and the conditions that select for it.

Both queen and worker policing are documented in many species. Queen policing occurs in Ponerinae ants (Diacamma sp.: Kikuta & Tsuji, 1999 and Dinoponera quadriceps: Monnin & Peeters, 1997; Monnin et al., 2002), bumblebees (Free et al., 1969; Pomeroy, 1979; Cnaani et al., 2002), Dolichovespula wasps (Greene, 1979; Foster & Ratnieks, 2001c), Polistes paper wasps (Fletcher & Ross, 1985; Reeve, 1991) and the halictid bee Lasioglossum zephyrum (Michener & Brothers, 1974). All these taxa are characterized by small colonies, which is suggested to be a requirement for effective queen control of worker reproduction (Ratnieks, 1988; Ratnieks & Reeve, 1992; Bourke, 1999).

Worker policing by egg-eating has been documented in three species of Apis honeybees (A. cerana, A. florea and A. mellifera, e.g. Ratnieks & Visscher, 1989; Ratnieks, 1993; Halling et al., 2001; Oldroyd et al., 2001; Martin et al., 2002, reviewed in Ratnieks, 2000; Barron et al., 2001), four species of Vespidae wasps (V. vulgaris: Foster & Ratnieks, 2001a; D. saxonica: Foster & Ratnieks, 2000; D. maculata: Greene, 1979; V. crabro: Foster et al., 2002) and three ant species, Camponotus floridanus (Endler et al., 2004), Diacamma sp. (Kikuta & Tsuji, 1999) and Pachycondyla inversa (D'Ettorre et al., 2004). Worker policing can also occur via aggression or killing of laying workers, and this occurs in the yellowjacket, V. atropilosa (Landolt et al., 1977) and in four species of ants (Aphaenogaster smythiesi: Iwanishi et al., 2003; Gnamptogenys menadensis: Gobin et al., 1999; Harpegnathos saltator: Liebig et al., 1999; Platythyrea punctata: Hartmann et al., 2003). There is also evidence that reproductive workers are aggressed in the honeybee, A. mellifera (Sakagami, 1954; Visscher & Dukas, 1995) but the extent to which this reduces their reproduction is unknown.

The factors selecting for queen and worker policing include favouring the rearing of closer kin and enhancing colony function (Ratnieks, 1988). Queen policing, provided it has little cost, is always selected for because a queen is always more related to her own offspring (0.5) than to her daughter workers’ offspring (0.25). Similarly, worker policing in single-queen Hymenoptera is selectively favoured when the queen is mated to more than two males, because workers are then more related to the queen's sons (0.25) than to other workers’ sons (<0.25) (Ratnieks, 1988). If policing also enhances colony productivity (Ratnieks, 1988) or if it enables the workers to cause a more female-biased sex allocation ratio in their colony (Foster & Ratnieks, 2001b), then worker policing can also be selected for under single mating. These additional benefits must be important as worker policing occurs in singly mated species (e.g. in ants, A. smythiesi: Iwanishi et al., 2003, C. floridanus: Gadau et al., 1996; Endler et al., 2004, Diacamma sp.: Kikuta & Tsuji, 1999; in wasps, D. maculata: Greene, 1979; Foster et al., 2001, V. crabro: Foster et al., 2002) and even in species or subspecies where workers reproduce thelytokously (P. punctata: Hartmann et al., 2003; A. mellifera capensis: Pirk et al., 2003).

Methods

Theory: methods and assumptions

To determine how colony-level costs, relatedness and social policing affect the incentive for workers to produce sons, we calculate the ESS (evolutionarily stable strategy, Maynard Smith, 1982) probability that a worker should reproduce. This ESS is calculated using a modified version of Hamilton's (1964) rule in which the costs and benefits of altruism (worker sterility) or selfishness (worker reproduction) are allowed to be dependent on the behaviour of other group members, as in the kin selection methods developed by Frank (1998). However, unlike Frank (1998) we will take an inclusive fitness rather than a direct or ‘neighbour-modulated’ fitness perspective. In a Mathematica notebook, however, available as an electronic Appendix to this paper (see Supplementary material), we show that all our results can also be derived from a direct fitness and even a group selection perspective (cf. Wenseleers et al., 2003). In addition, the electronic Appendix shows that all inclusive fitness results can be derived using exact population genetic models, and that these give numerically identical results in all cases. The inclusive fitness approach, however, is more general and can be applied for any level of relatedness, whereas separate genetic models have to be constructed for each specific colony kin structure (e.g. single vs. double mating, see Ratnieks, 2001; Wenseleers et al., 2003).

Specific models are developed to determine the ESS in both queenless and queenright colonies. The most important assumption of both models is that worker reproduction has a cost and that the total productivity of the colony, in terms of young queens and males reared, will diminish if it contains more reproductive workers [note that we do not assume that worker reproduction has a fixed colony-level cost, as in the model of Cole (1986), but instead that the cost increases as worker reproduction in the colony increases].

A cost of worker reproduction is supported by many studies. For example, in queenless colonies of the ant Leptothorax allardycei, worker dominance activity reduces the time spent on brood care by 15% (Cole, 1986), and in the wasps P. fuscatus, P. chinensis and V. atropilosa, the productivity of queenless colonies with worker reproduction is 50, 75 and 74% that of equal-sized queenright colonies (Landolt et al., 1977; Reeve, 1991). A cost of worker reproduction occurs because laying workers often aggressively compete for reproduction and generally do not work, forage or engage in colony defence, as carrying out such risky tasks could jeopardize their reproductive futures (Franks & Scovell, 1983). Empirically, a reduced working ability of reproductive workers has been shown in Bombus and Lasioglossum bees (Michener, 1974; van Honk et al., 1981), the Cape honeybee (Moritz & Hillesheim, 1985; Hillesheim et al., 1989; Martin et al., 2002), Dinoponera, Rhytidoponera and Platythyrea ants (Ward, 1983; Monnin & Peeters, 1999; Hartmann & Heinze, 2003; Hartmann et al., 2003) and the jellowjackets, V. vulgaris and V. atropilosa (Potter, 1965; Landolt et al., 1977; Ross, 1985). As one example, Landolt et al. (1977) mention that laying workers in queenless nests of the prairie yellowjacket V. atropilosa‘were never observed foraging or constructing envelope and only rarely fed larvae, chewed prey, or added paper to cells’. and that ‘When active, they were either ovipositing or harassing other workers’.

Another model assumption is that, in queenless colonies, even a low level of worker reproduction can provide sufficient male eggs for rearing (in queenright colonies we assume that the queen can lay sufficient male eggs). This is reasonable, given that the main investment in male production is not in producing the eggs but in collecting food in order to rear the larvae and that workers are typically quite fecund. For example in the honeybee, a single worker can lay approximately 10 eggs per day (Visscher, 1996), and if a colony were to rear 50 males per day (Seeley, 2002), only approximately five workers or 0.01–0.05% of the worker population would be required to provide sufficient eggs. Similarly, in D. sylvestris, a single worker can lay approximately 10 eggs per day (T. Wenseleers and A. Tofilski, unpublished data), which would be enough to provide all the males reared in the colony.

In addition, and unlike a recent model (Hammond et al., 2003), we assume that worker reproduction in queenright colonies does not make the colony's sex-ratio more male biased, and hence does not affect the relative value of the sexes. For the honeybee, this is appropriate as males are only reared in special large-diameter ‘drone’ cells (Winston, 1987). Consequently, male worker-laid eggs do not substitute queen-laid female eggs and worker reproduction does not affect the colony sex-ratio. For other species, an effect on the sex-ratio is probably also unlikely, given that the sex-ratio can usually be regulated at low cost at an early larval stage (e.g. Sundström, 1994; Sundström et al., 1996; Queller & Strassmann, 1998; Hammond et al., 2002).

In the model we only consider policing via egg-eating as a regulatory mechanism and do not consider aggression of laying workers which has been suggested to control the number of laying workers in some species (e.g. Visscher & Dukas, 1995; Dampney et al., 2002; Iwanishi et al., 2003). Aggression of laying workers could directly inhibit the egg layer, potentially leading to less worker reproduction than predicted by our model. Nevertheless, policing by egg eating appears to be a more common way of preventing successful worker reproduction, so that this limitation is probably not very severe. Finally, we do not analyse how policing should co-evolve with the level of worker reproduction but instead consider it as a fixed parameter in the analysis. This is appropriate given that we just want to analyse how policing per se affects the incentive for workers to reproduce.

Results

Below we determine the ESS proportion of reproductive workers in queenless and queenright colonies. We start with the model for queenless colonies as this is analytically simpler and prepares the ground for the queenright model.

Worker reproduction in queenless colonies

In queenless colonies, many workers activate their ovaries and lay male eggs which are reared to produce a final crop of males (Bourke, 1988; Choe, 1988; Bourke & Franks, 1995). Nevertheless, not all workers in queenless colonies reproduce (e.g. Marchal, 1896; Ross, 1985; Bourke, 1988; van Walsum et al., 1998; Miller & Ratnieks, 2001). To understand why, consider the following model.

Assume that a focal worker in a colony activates her ovaries and lays eggs with probability y, and that each of the (n − 1) nestmates activate their ovaries with probability Y, so that the colony contains nz egg-laying workers where z is the average probability with which workers activate their ovaries, z = (1/n)y + [(n − 1)/n]Y (Table 1 gives a summary of all terminology used). We can then write the total number of males produced by this focal worker and by each nestmate as

image((1.1))
Table 1.  Notation and definition of parameters.
SymbolDefinition
nColony size (number of workers)
WfTotal number of males produced by a focal worker
WnTotal number of males produced by each of the (n − 1) nestmates
WMqTotal number of males produced by the queen
WQTotal number of queens or swarms produced by the colony
yProbability that a focal worker in a colony activates her ovaries and lays eggs
YProbability that a nestmate worker in a colony activates her ovaries and lays eggs
zColony average proportion of laying workers = (1/n)y + [(n − 1)/n]Y
ZPopulation average proportion of laying workers
z*Evolutionarily stable strategy (ESS) proportion of laying workers
G(z)Colony productivity as a function of the proportion of laying workers in the colony
kParameter describing how worker reproduction affects total colony productivity (k < 1: convex, k = 1: linear, k > 1: concave, etc.)
qFecundity of the queen in terms of male production relative to one laying worker
PwProbability with which a worker-laid egg is policed before it can hatch
PqProbability with which a queen-laid egg is policed before it can hatch
inline imageRelative survival of worker-laid eggs
rRegression relatedness
νmThe sex-specific reproductive value of males to females
RLife-for-life relatedness, the product of reproductive value and regression relatedness

and

image((1.2))

where G(z) is the colony productivity (total number of males reared) as a function of how many laying workers there are in the colony, and the ratios y/nz and Y/nz are the fraction of all males that are the focal and nestmate worker's sons. If we use a linear function for G(z), i.e. G(z) = 1 − z, we arrive at the TOC model of Frank (1995). The only extension required to make the model more realistic for worker reproduction is to allow nonlinear colony-level cost functions. A versatile and appropriate function is G(z) = (1 − z)k, which allows for linear (k = 1), concave (k > 1) or convex (k < 1) reductions in colony productivity (Fig. 1a). This function states that more males are reared when fewer workers reproduce (z ≅ 0) (i.e. most work and forage) and that if all workers laid eggs, then no males would be reared because of a dearth of foragers (Fig. 1a).

Figure 1.

Evolutionarily stable strategy (ESS) proportions of laying workers in queenless hymenopteran colonies. (a) Example curves for the colony productivity cost function G(z) = (1 − z)k, illustrating cases where worker reproduction has a linear (k = 1), less than linear (k < 1) or greater than linear cost (k > 1) on colony productivity. (b) The ESS proportions of laying workers in queenless colonies [eqn (1.8)] as a function of sister–sister relatedness, using the same cost functions.

To calculate the optimum proportion of laying workers we need to calculate the costs and benefits of worker reproduction. If the focal worker slightly increases its probability of reproducing, this will give her an increase in direct fitness of

image((1.3))

where the derivative measures how a small change in the probability of breeding affects the focal worker's fitness. This derivative is evaluated for the case where the focal worker's behaviour approaches that of its nestmates (yY), because we would like to calculate the invasion condition for a mutant that behaves only slightly different from the wild-type (Frank, 1998). As worker reproduction reduces colony productivity, an increase in the focal worker's reproductive tendency will also have inclusive fitness costs. The direct fitness cost to each of the (n − 1) nestmates can be calculated as

image((1.4))

Therefore, the total indirect fitness cost of increased reproduction by the focal worker to all (n − 1) nestmates is given by

image((1.5))

where, Rsister is the relatedness to sister workers. Application of Hamilton's (1964) rule then shows that increased worker reproduction has inclusive fitness benefits when

image((1.6))

that is when the direct fitness benefit exceeds the total indirect fitness cost. Rather than contrasting a benefit to self with a cost to sisters, one can equivalently consider the benefit and cost as occurring to the offspring, sons and nephews (Bourke, 1997), as in

image((1.7))

An ESS or ‘evolutionarily stable strategy’ (Maynard Smith, 1982) occurs when the left and right hand sides of eqns (1.6) or (1.7) become equal. That is, when the direct fitness benefit of greater personal reproduction equals the inclusive fitness cost of worker reproduction reducing total male production by the colony. Calculating the partial derivatives (see Supplementary material) and solving for Y shows that this occurs when the proportion of laying workers is

image((1.8))

where r is the average relatedness to nestmate workers (including self), (1/n)×1 + [(n − 1)/nRsister (Pepper, 2000), which approaches the relatedness to sisters (Rsister) for large colony sizes (n large).

This result can be expressed equivalently in terms of the relatedness to the males that are reared and in ratio form, as meaning that a queenless social insect colony should have

image((1.9))

where Rmales is the average relatedness to males reared in the colony, (1/nRson + [(n − 1)/nRnephew (Pepper, 2000). For large colonies (n large), the relatedness to own sons can be neglected and Rmales ≅ Rnephew. Intuitively, at this equilibrium workers become reproductive or sterile in exact proportion to the inclusive fitness payoffs that each strategy confers. The payoff of reproduction is that it allows nephews (value Rnephew) to be replaced by sons (value Rson) whereas the benefit of sterility, that is being a working worker, is that it allows the rearing of k more males (value Rnephew).

Some numerical examples can illustrate the model predictions: with a linear cost function (k = 1), and with queen-mating frequencies of 1 (r = 0.75), 2 (r = 0.5) and 10 (r = 0.3), the ESS is for (1 − 0.75) = 25%, (1 − 0.5) = 50% or (1 − 0.3) = 70% of all workers to lay eggs. This shows that relatedness is the key factor that keeps worker reproduction in queenless colonies in check. High relatedness favours reduced reproduction, as it decreases the benefit of replacing nephews by sons, and increases the inclusive fitness costs. A more severe colony-level cost (k > 1) further reduces the ESS level of worker reproduction. Figure 1b plots the expected equilibrium for various values of k and r.

Worker reproduction in queenright colonies

Within queenright colonies policing joins kinship as a powerful factor than can reduce actual conflict over reproduction. (In queenless colonies nonreproductive workers could police other workers’ eggs, but this would not affect the relatedness to the males that are reared, and hence would not affect the ESS.) To see how policing affects the evolution of worker reproduction consider the following model.

Let n be the number of workers in the colony, Pw and Pq the probabilities with which worker-laid and queen-laid eggs are policed, whether by the queen or by the workers, and q the relative fecundity of the queen in terms of male production relative to a single reproductive worker. Assume, as before, that a focal worker in a colony activates her ovaries and lays eggs with probability y, and that each of the (n − 1) nestmates activate their ovaries with probability Y, so that the colony contains nz egg-laying workers where z is the average probability with which workers activate their ovaries, z = (1/n)y + ((n − 1)/n)Y. We can then write the total number of males produced by this focal worker and by each nestmate as

image((2.1))

and

image((2.2))

where G(z) is the colony productivity (total number of males reared) as a function of how many laying workers there are in the colony. The term following this represents the proportion of all males that are the focal or nestmate worker's sons, which is given by the total number of males produced by the focal or nestmate worker that survive policing (y(1 − Pw) or Y(1 − Pw)), divided by the total number of surviving worker's sons (nz(1 − P)) plus the number of surviving queen's sons (1 − Pq), produced at relative fecundity q (or at a relative rate qt if t is time). As before, we will use a power function, G = (1 − z)k for colony productivity G.

By a similar argument, the total number of males produced by the queen can be written as

image((2.3))

Eqns (2.1–2.3) can be simplified by dividing the numerator and denominator by (1 − Pq) and substituting Sw for (1 − Pw)/(1 − Pq), the relative survival of worker eggs (the relative chance to survive until hatching). We then get

image((2.4))

and

image((2.5))

and

image((2.6))

Finally, the total amount of female reproduction by the colony (winged queens or swarms for swarm-founding species such as honeybees) produced is also a decreasing function of z. For simplicity, we assume that worker reproduction reduces queen and male production equally. Hence, the total number of queens produced is

image((2.7))

As before, the ESS proportion of laying workers z* in the colony can be determined by standard application of inclusive fitness methods (Frank, 1998; Appendix 1) and is given by the standard solution for the roots of a quadratic equation

image((2.8))
image

If policing is perfectly selective, i.e. if no queen-laid eggs are mistakenly destroyed (Pq = 0), Sw is simply 1 − Pw, the proportion of surviving worker's sons. Figure 2 plots this optimum under the assumption that no queen-laid eggs are policed (Pq = 0). Two main effects on the ESS can be seen. First, the ESS is higher (more workers reproduce) the lower the relatedness among the workers (rsister). This is because low sister–sister relatedness increases the relatedness gain of replacing nephews by sons and decreases the potential inclusive fitness cost, which will be borne by more distant relatives. Secondly, the ESS is lower when policing is more effective (high Pw or low Sw), because the benefit to a worker of laying eggs [Appendix 1, eqn (A.1)] reduces as the proportion of these that are reared diminishes. These two predictions confirm earlier verbal argument (Bourke, 1988; Ratnieks, 1988). Additional predictions are that the lower the ESS level of worker reproduction the more severely worker reproduction affects colony productivity (high k), and the higher the ESS level the larger the colony, because competition over male production will then primarily be with other workers rather than with the queen. By the same argument, the ESS will also be higher as the fecundity of the queen increases.

Figure 2.

Evolutionarily stable strategy (ESS) proportions of laying workers in queenright colonies [eqn (2.8)] as a function of the effectiveness of policing Pw– the proportion of worker-laid eggs that are eaten. The ESS is shown for convex (k = 0.5), linear (k = 1) and concave (k = 5) colony-level costs of worker reproduction and for different colony sizes (n). Each individual graph shows the ESS for three relatedness values (r = 0.25: full lines, r = 0.5: dashed lines, r = 0.75: dotted lines) and for a relative queen fecundity q of 1 (top), 2, 10 and 30 (bottom). In all cases, the ESS is lower when policing is more effective. This is because it becomes less profitable for workers to lay eggs as more of these are later policed. In addition, fewer workers are selected to lay eggs when (1) workers are more related (high r), (2) the colony-level cost of worker reproduction is more severe (high k), (3) the colony is smaller (small n, because laying workers then compete primarily with the queen rather than with each other) or (4) the queen has a very high fecundity relative to one laying worker (high q, the same argument as in 3). The point where each curve crosses the x-axis is the critical effectiveness of policing that must be exceeded for complete worker sterility in queenright colonies to be favoured. That is, for zero workers to activate their ovaries.

If there is no policing (Pw and Pq = 0) and if colony size is very large (n→∞), the ESS proportion of laying workers simplifies to

image((2.9))

which simplifies further to

image((2.10))

This ESS is independent of the relatedness to brothers because in very large colonies, workers can dominate male production even if each has very low fecundity. In other words, in large colonies the ESS is set by competition among fellow workers, not by competition between queen and workers. The ESS can also be written in equivalent ratio form, with the optimum being for any colony to have

image((2.11))

As in queenless colonies, workers at this ESS should be reproductive or sterile in exact proportion to the inclusive fitness payoffs that each strategy confers. The payoff from becoming a laying worker is that it allows nephews (value Rnephew) to be replaced by sons (value Rson); the benefit of becoming a sterile ‘working worker’ is that it allows the rearing of k more males (value Rnephew) and queens (value Rsister).

Comparing eqns (2.11) with (1.9) shows that the ESS proportion of laying workers is lower in queenright than in queenless colonies. This is because worker reproduction in queenright colonies has a greater cost, as it causes a proportionate reduction in male and queen production [eqn (2.11)], as opposed to just male production in queenless colonies [eqn (1.9)]. In the absence of policing, and with large colony size and a linear cost function (k = 1), the ESSs are for (1 − 0.75)/(1 + 0.75) = 14%, (1 − 0.5)/(1 + 0.5) = 33% or (1 − 0.3)/(1 + 0.3) = 54% of the workers to reproduce [eqn (2.10)] when the colony is headed by a single mother queen mated to 1, 2 or 10 unrelated males (note that we can use regression relatedness because, as mentioned before, all males are workers’ sons). By contrast, the corresponding optima for queenless colonies are 25, 50 and 70%. (Note that in this comparison, we assume that the cost function of worker reproduction in queenright and queenless colonies is approximately the same. However, as the cost in queenright colonies would be felt as a reduction in the production of new workers as well as sexuals, this could increase the cost for queenright colonies relative to queenless colonies. If so, this would further increase the expected difference between queenright and queenless colonies.)

Despite the ESS worker reproduction being lower than in queenless colonies, very many workers are still selected to reproduce and this would cause a significant reduction in colony productivity. If an effective police system is in place, however, the payoffs of worker reproduction are greatly reduced and the ESS is much lower (Fig. 2). Indeed, complete worker sterility in queenright colonies can actually be favoured if the effectiveness of policing is very high (high Pw). Worker sterility is favoured when worker laying cannot invade from a situation in which it is initially absent. Substitution of Y = 0 in eqn (A.5), shows that this occurs when

image((2.12))

(we again assume that mortality of queen eggs is zero, Pq = 0).

This equation has the intuitive interpretation that a single laying worker in a colony would replace a fraction (1 − Pw)/q of the queen's male eggs by sons, but that having a single reproductive worker who would not work would also reduce the productivity of the colony (male and queen production) by a fraction k/n. From eqn (2.12) it is clear that workers are selected to completely refrain from egg-laying when the effectiveness of policing (Pw) is greater than

image((2.13))

The critical effectiveness of policing required to favour complete worker sterility, for different relatedness values and n to q ratios, is plotted in Fig. 3. Zero worker reproduction is hard to be selected for in large colonies, such as honeybees, because the productivity effect of a single worker becomes vanishingly small.

Figure 3.

The critical effectiveness of policing required to favour complete worker sterility in queenright hymenopteran colonies [eqn (2.13)] as a function of relatedness r and for different n/q ratios (colony size/relative queen fecundity) and colony-level cost functions. The full, dashed and dotted lines are n/q ratios of 10, 100 and 1000, and each case is plotted for convex (k = 0.5, top), linear (k = 1, middle) and concave (k = 2, bottom) colony-level cost functions. Complete sterility is favoured above and to the right of each line.

Discussion

Our results show that both kinship and policing can prevent the high cost of unchecked worker reproduction and thereby reduce the possibility of a TOC (Hardin, 1968; Frank, 1995; Wenseleers & Ratnieks, 2004). Higher relatedness favours reduced worker reproduction because unrestrained worker reproduction causes greater inclusive fitness costs (cf. Hamilton, 1964; West-Eberhard, 1975; Oster & Wilson, 1978; Cole, 1986; Bourke, 1988). This prediction is similar to that made by reproductive restraint and concession-based models of reproductive skew, which show that reproduction within social groups should be divided more unequally when intragroup relatedness is high (Vehrencamp, 1983a,b; Reeve & Ratnieks, 1993; Cant, 1998; Reeve & Keller, 2001). But in addition to relatedness, policing also favours less worker reproduction. This is because the incentive for workers to lay eggs is reduced as the probability that their eggs will be reared diminishes. In other words, social policing should select for ‘self-policing’, resulting in most or even all the workers deliberately refraining from reproduction, even if they retain ovaries (Ratnieks, 1988; Visscher, 1989; Ratnieks & Reeve, 1992; Bourke, 1999).

Although the aims of our paper are primarily theoretical, the model makes a number of clear predictions which can be empirically tested:

In queenless colonies an excess of workers should reproduce

We predict that the proportion of reproductive workers in queenless colonies should be so great that more eggs are produced than can be reared (Fig. 1), leading to a TOC (Hardin, 1968; Frank, 1995; Wenseleers & Ratnieks, 2004). Support for this prediction comes from the fact that in queenless colonies of both wasps and honeybees, so many workers activate their ovaries that cells contain multiple eggs (Marchal, 1896; Ross, 1985; Seeley, 1985; Miller & Ratnieks, 2001). This clearly shows that an excess of workers reproduce, as only one male can be reared per cell. Additional support could come from taxa, such as ants, which do not rear brood in individual cells, if it were shown that the collective worker-egg production in queenless colonies exceeds colony-rearing capacity.

In queenless colonies worker reproduction should be negatively correlated with relatedness

The ESS proportion of reproductive workers decreases from 75 to 25% as relatedness increases from 0.25 (all workers half-sisters) to 0.75 (all workers full-sisters) (Fig. 1). This prediction awaits detailed testing. Data on the proportion of laying workers in queenless colonies are available only from a scattered range of taxa (e.g. ants: van Walsum et al., 1998, wasps: Marchal, 1896; Ross, 1985 and honeybees: Miller & Ratnieks, 2001; Oldroyd et al., 2001), and the colony-level cost of worker reproduction probably varies across these groups. To ensure a consistent test, it would be better to compare colonies within species or closely related species or genera, which differ in relatedness but are otherwise similar. The Vespinae wasps might be a good group for such a test as they display a wide range in relatedness (Foster & Ratnieks, 2001c), and yet are similar in their overall biology.

In queenright colonies effective policing should favour self-restraint

Data from the honeybee, A. mellifera and the common wasp, V. vulgaris provide good support for this prediction. In queenright A. mellifera honeybee colonies, only approximately 0.01–0.1% of the workers have active ovaries (Ratnieks, 1993), and policing is effective, as only approximately 2% of all worker-laid eggs survive until hatching (Ratnieks, 1993). Similarly, the common wasp, V. vulgaris has efficient policing and little worker reproduction (1% of the workers have activated ovaries, Foster & Ratnieks, 2001a). Based on our model, it is clear that such low levels of worker reproduction can only be explained as a response to policing, not to kinship alone. Both the honeybee and the common wasp are polyandrous (Palmer & Oldroyd, 2000; Foster & Ratnieks, 2001a), which leads to low worker relatedness (Apis: r = 0.3, V. vulgaris: r = 0.5, Palmer & Oldroyd, 2000; Foster & Ratnieks, 2001a). The model predicts that with such relatedness structures, and in the absence of policing, a very high proportion of the workers should reproduce, 56 and 33% in Apis and Vespula. The self-restraint favoured by policing can prevent the high cost that such a high level of worker reproduction would bring about. Further support for policing favouring self-restraint could be provided if it were shown that the effectiveness of policing across species correlates negatively with the proportion of workers with active ovaries in the colony.

Queen loss should induce a greater amount of worker reproduction

This is because the suppressive effect of policing disappears when the queen is lost (Miller & Ratnieks, 2001) and because worker reproduction has a lesser cost in queenless colonies, as it causes a proportionate reduction in just male production [eqn (1.9)] as opposed to male and queen production in queenright colonies [eqn (2.11)]. A sudden increase in the fraction of laying workers following orphanage has been reported many times (reviewed in Fletcher & Ross, 1985; Bourke, 1988; Choe, 1988). For example, in the honeybee, A. mellifera and the wasp, V. vulgaris, only 0.01–0.1 and 1% of the workers have active ovaries in queenright colonies, but this rises to 36 and 25% following orphanage (Marchal, 1896; Miller & Ratnieks, 2001). Similarly, in the paper wasp, P. chinensis, the percentage of laying workers increases from 14 to 25% when the colony becomes queenless (Suzuki, 1998) [interestingly, as this species is singly mated (Tsuchida et al., 2003), these are precisely the predicted ESS values if policing is rare or absent]. Note that in all cases, the amount of worker reproduction in queenright and queenless colonies closely follows the workers’ interests. This adds support to the idea that the low amount of worker reproduction in queenright colonies is not due to the action of suppressive queen pheromones (e.g. Alexander, 1974; Oster & Wilson, 1978; Seeley, 1985; Bourke, 1988; Keller & Nonacs, 1993; Foster et al., 2000), but that workers merely use the queen pheromone to monitor queen presence and respond to it in their own best interests (Seeley, 1985; Keller & Nonacs, 1993; Reeve, 1997; for reviews on the honeybee queen pheromone, see Hepburn et al., 1991; Winston & Slessor, 1998)

Aside from these main predictions, several other more detailed predictions were made, such as how colony size and queen fecundity should affect the expected number of laying workers. For example, the model showed that a relatively greater percentage of the workers should reproduce in large colonies because in large colonies competition over male parentage is primarily among the workers rather than with the queen. This result apparently contradicts Bourke (1999), who predicted that fewer workers should reproduce in large colonies. Our model shows that this is a valid prediction only if large colony species have a more effective policing system. This, however, appears to be the case as both the honeybee and Vespula have large colonies of approximately 30 000 (Winston, 1987) and 2500 workers (Spradbery, 1973), and both have efficient policing (Ratnieks & Visscher, 1989; Ratnieks, 1993; Foster & Ratnieks, 2001a).

To what extent do the results of our model shed light on the evolution of conflict and co-operation in general? The most important result, that social policing should make exploitation less profitable, also applies to other types of societies and has parallels in other contexts. For example, if parents were to police their young by not responding to excessive levels of begging, this would favour greater co-operation in the same way as our model predicts (Mock & Parker, 1998). In co-operative breeders, the risk for individuals to be expelled when they claim too much of a group's reproduction (‘the threat of eviction’) is a similar case where a form of policing indirectly favours greater co-operation (Johnstone & Cant, 1999). In all these cases, the main effect of policing is to make attempted cheating unprofitable (e.g. excessive begging or unrestrained breeding). This strengthens the view that policing does not always have to act through direct coercion (Boyd & Richerson, 1992; Frank, 2003).

Is policing a universal mechanism that can promote co-operation in biological systems? Possibly it is, although there are two caveats. First, individuals could evolve resistance to policing, which could lead to transitional ‘episodes of revolution’. For example, in some lines of anarchistic honeybees, the workers cheat by laying eggs that are not policed (Barron et al., 2001). Secondly, there are examples of conflicts where policing is simply not possible (Beekman et al., 2003). For example, in social Hymenoptera, developing females cannot be prevented from selfishly turning into queens when queens and workers are the same size and develop away from social interference in closed cells (Bourke & Ratnieks, 1999; Wenseleers et al., 2003; Wenseleers & Ratnieks, 2004). This occurs in Melipona stingless bees, where as a result of such ‘caste fate conflict’, queens are continually overproduced (Wenseleers & Ratnieks, 2004). Nevertheless, our analysis clearly demonstrates that when policing is possible, it can be a much more efficient mechanism than pure kinship for promoting cooperation within social groups (cf. Frank, 1995; Wenseleers et al., 2003; Wenseleers & Ratnieks, 2004).

Acknowledgments

This work was supported by the ‘INSECTS’ TMR network, an E.C. Marie Curie Fellowship to T.W., the Academy of Finland (project no. 42725), an Evolutionary Ecology Graduate School grant to H.H. and an NERC grant to FLWR and AGH. Authors are very grateful to Michael Cant, Steve Frank, Andy Gardner, Stephen Martin and Stuart West for valuable comments and discussion.

Supplementary material

The following material is available from http://www.blackwellpublishing.com/products/journals/suppmat/jeb/jeb751/jeb751sm.htm

Appendix A1. Electronic appendix – Worker reproduction and policing in social insects: an ESS analysis.

Appendix

Appendix 1. Derivation of the ESS fraction of laying workers

The personal benefit to a worker from increased direct reproduction is given by

image( (A.1))

Increased reproduction by the focal worker has a direct fitness cost to each of the (n − 1) nestmates of

image((A.2))

Finally, increased reproduction by the focal worker will reduce brother and sister queen production by

image((A.3))

and

image((A.4))

For a linear colony-level cost function, k = 1, eqn (A.4) simplifies to 1/n, implying that a single laying worker would reduce colony productivity by a fraction of 1/n.

To determine the net inclusive fitness benefit of worker laying, these costs and benefits need to be weighed by life-for-life relatedness and by the number of individuals of each class that are affected. Thus, the inclusive fitness benefit of worker laying is given by

image((A.5))

This is a Hamilton's (1964) rule condition with four affected parties, and the R values measure the life-for-life relatednesses to each of them. Life-for-life relatedness is the product of regression relatedness (a measure of the proportion of genes the recipient has in common with the actor relative to a random individual) and sex-specific reproductive value, which measures the genetic contribution of a given sex to the future gene pool (Pamilo, 1991). Hence, the R values can be simplified to Rson = rson × νm = 1 × νm, Rnephew = rsister × νm and Rbrother = rbrother × νm = (1/2) × νm where, νm is the relative reproductive value of males, which is given by 1/(2 − ψ) if ψ is the population-wide proportion of males that are worker produced (Pamilo, 1991). In our case, it can be seen that ψ = nZSw/(nZSw + q), where Z is the average proportion of laying workers in the population.

To determine the equilibrium condition, we can solve for the value of Y for which eqn (A.5) becomes zero (see electronic Appendix). This ESS is given in eqn (2.8).

Ancillary