SEARCH

SEARCH BY CITATION

Keywords:

  • dynamic programming;
  • kin selection;
  • optimal control;
  • social evolution

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Direct fitness
  5. Direct fitness for dynamic resource allocation
  6. Box 1 – Illustrated summary of the approach
  7. Example
  8. Discussion
  9. Acknowledgments
  10. References
  11. Appendix

Abstract The direct-fitness approach to modelling the evolution of social traits is an alternative to the classical inclusive-fitness-based approach. Despite both its utility and popularity, the direct-fitness approach has not yet been extended to include the analysis of dynamic traits, i.e. traits whose level of expression may vary over time. In this article, I apply the direct-fitness approach to cope with the evolution of a dynamic resource-allocation behaviour when this behaviour influences the fitness of relatives. I am able to implement the direct-fitness approach using components (reproductive value, fitness changes and measures of relatedness) found in standard, social-evolutionary models. I illustrate the modified direct-fitness model with an example studied by previous authors, and I show how the direct-fitness perspective can aid the validation of analytical results by means of a genetic algorithm.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Direct fitness
  5. Direct fitness for dynamic resource allocation
  6. Box 1 – Illustrated summary of the approach
  7. Example
  8. Discussion
  9. Acknowledgments
  10. References
  11. Appendix

The inclusive-fitness method of social-evolutionary analysis (Hamilton, 1964) can be used to explain the adaptive significance of a variety of traits. An inclusive-fitness analysis centres around a tally of the ways in which one individual displaying that trait (the actor) affects the fitness of one or more relatives (the recipients). This tally is sometimes called the inclusive fitness effect of the trait.

The direct-fitness approach to social-evolutionary problems (Taylor & Frank, 1996) provides researchers an alternative to inclusive-fitness methods. In contrast to inclusive fitness, direct fitness centres around a tally of the effect one or more actors have on the fitness of one related recipient (the focal individual).

Direct fitness has become popular for a variety of reasons. Unlike its cousin, the direct-fitness approach can be applied in a systematic way (e.g. Rousset & Billiard, 2000, p. 821), a feature that those new to the field (especially students) find quite helpful. Direct fitness can also, in some cases, allow researchers to draw conclusions that are beyond the scope of the inclusive-fitness method (Frank, 1997; Day & Taylor, 1998), making direct fitness the more versatile tool.

Despite its popularity, the direct-fitness method has not yet been extended to cope with dynamic traits (i.e. traits whose expression vary with time), even though an inclusive-fitness framework for their analysis has been available for more than 15 years (McNamara et al., 1994; Day & Taylor, 1997, 2000), and even though population genetic models suggest that it is possible (Day & Taylor, 2000). My purpose here, then, is to show how the direct-fitness method can be applied to study dynamic traits, in order to provide an alternative to the existing inclusive-fitness approaches. I show how the standard building blocks of social-evolutionary theory (fitness changes, relatedness, reproductive value) can be assembled in a non standard manner to study resource-allocation problems usually tackled with optimal-control theory (e.g. Perrin & Sibly, 1993; Day & Taylor, 1997; Iwasa, 2000; Wakano, 2005).

In the following section, I briefly review the direct-fitness method. I pay particular attention to the application of direct fitness to the study of certain class-structured populations, i.e. populations in which different classes of individual contribute differentially to future generations (Taylor, 1990). In Direct fitness for dynamic resource allocation, I show how the typical class-structured version of the direct-fitness approach can be modified to study dynamic traits expressed at discrete time points. The main modification that arises, here, involves the computation of small changes in non equilibrium reproductive values (typical direct-fitness calculations rely on equilibrium reproductive values and do not consider marginal changes in these values). In Example, I explore an example previously studied by Day & Taylor (2000). With the help of the direct-fitness perspective, I am not only able to recover the discrete-time version of their result but also able to confirm its stability (as a best strategic response to itself) with simulation. In Discussion, I briefly discuss the results and suggest how they may drive future work.

Direct fitness

  1. Top of page
  2. Abstract
  3. Introduction
  4. Direct fitness
  5. Direct fitness for dynamic resource allocation
  6. Box 1 – Illustrated summary of the approach
  7. Example
  8. Discussion
  9. Acknowledgments
  10. References
  11. Appendix

The direct-fitness perspective popularized by Taylor & Frank (1996) helps one derive the approximate change in the frequency of a mutant allele when mutants (individuals that carry the mutant allele) have the opportunity to interact with one another.

To better understand the direct-fitness perspective, consider a haploid asexual mutant (the focal individual) that only interacts with others in its social group (the focal group). Assume that all social groups (the focal group included) are composed of exactly N members and that the outcome of any social interaction depends on the level at which some phenotype is expressed. Let inline image (a real number) denote the level at which non mutants express the phenotype of interest. Similarly, let ui denote the level at which the ith member of the focal group expresses the phenotype of interest (henceforth simply the phenotype of the ith member of the focal group).

I store the phenotypes of all members of the focal group (including the focal individual) in the vector inline image. If ɛ is the phenotypic effect of the mutant allele, then

  • image(1)

where inline image is a vector whose ith element gives the probability that the ith member of the focal group is, itself, a mutant, and where 1 is a vector of N ones. Note that Ri(ɛ) can also be interpreted as the relatedness between the focal individual and the ith member of the focal group, and it may depend on inline image.

If the fitness of the focal individual, denoted by W, is determined by its social interactions, then it is reasonable to treat W as a function of u(ɛ). When ɛ is small, I ignore terms in the order of ɛ2 (they are even smaller than ɛ) and approximate the fitness change associated with mutants’ deviant behaviour as

  • image(2)

where ∇u is the gradient operator, (∂/∂u1(t),  … , ∂/∂uN(t)) that, when applied to a scalar function like W, returns a vector that stores all first partial derivatives of that function. In this case, ∇uW = (∂W/∂u1(t),  … , ∂W/∂uN(t)) and ∇uW(u(0)) reminds us that the partial derivatives are to be evaluated when ɛ = 0.

The mutant allele increases (respectively decreases) in frequency when the sign of the change in line (2) is positive (respectively negative) (Taylor, 1989). When the change is zero, the population is at evolutionary equilibrium, and in this case, the corresponding inline image is what some call an evolutionarily singular strategy (e.g. Geritz et al., 1998). Note that in eqn (2), the vector R is calculated for a neutral population (ɛ = 0). Although one can determine R under the assumption that the mutant is rare (Wild, 2010), rareness is not always required (Rousset & Billiard, 2000, and references therein); rareness will not be relied upon here.

Equation (2) expresses how neighbours’ phenotypes and the phenotype of the focal individual influence the fitness of the focal individual, itself. This stands in contrast to the inclusive-fitness perspective (e.g. Hamilton, 1964), where one considers how the focal individual's phenotype influences the fitness of its neighbours. The direct-fitness perspective is often preferred, because, as illustrated previously, it results in straightforward model development and analysis. That said, inclusive fitness has its own advantages (Grafen, 2006a; Gardner, 2009), and both perspectives yield equivalent descriptions of kin selection (Taylor et al., 2007).

When the direct-fitness perspective is applied to study a class-structured population, it is almost always the case that the class structure under consideration is discrete (Grafen, 2006b, is one notable exception). For example, an individual in the population might be classed as a juvenile/adult, small/medium/large, or male/female. If the focal individual belongs to one particular class, but has components of fitness spread across several different classes, then the overall fitness change associated with its deviant behaviour is a kind of weighted average. Specifically, the overall change is a weighted average of the changes observed in each of the different fitness components, where weights are given by reproductive value (i.e. the contribution one unit of a given component fitness makes to the gene pool in the distant future) (Taylor, 1990; Taylor & Frank, 1996). If Wc denotes the focal individual's class-c component of fitness and vc denotes the reproductive value of one unit of class-c fitness, then the overall change in fitness that is because of slightly deviant mutant behaviour is

  • image(3)

where, again, the approximation neglects terms involving ɛ2.

The reproductive values used in the summation above convert class-specific fitness changes to a ‘common currency’ (namely long-term genetic contribution to the population). I should emphasize that reproductive values are calculated under the assumption that the mutant allele has no effect on mutant behaviour (Taylor, 1990; Taylor & Frank, 1996). I should also emphasize that, like eqns (2) and (3) represents a count (here, a weighted count) of the fitness effects group members impose on the focal individual.

Of course, the direct-fitness perspective can be applied to study more general class-structured populations (Taylor & Frank, 1996; Rousset & Ronce, 2004). However, these more general applications go beyond what I will need in order to make a connection between the direct-fitness perspective and standard dynamic resource-allocation problems.

Direct fitness for dynamic resource allocation

  1. Top of page
  2. Abstract
  3. Introduction
  4. Direct fitness
  5. Direct fitness for dynamic resource allocation
  6. Box 1 – Illustrated summary of the approach
  7. Example
  8. Discussion
  9. Acknowledgments
  10. References
  11. Appendix

Dynamic resource-allocation problems typically focus on identifying a sequence of resource-allocation phenotypes (or decisions), u(0),u(1),u(2),  … , u(T − 1) that maximizes total reproductive output during an extended, but fixed, period of time (here, T discrete time steps). At the heart of any such problem is a life-history trade-off; for example growth versus reproduction (e.g. Perrin & Sibly, 1993; Iwasa, 2000) or production of sons versus production of daughters (e.g. Wakano, 2005). In general, the resource-allocation decision made at a particular point in time, called u(t), reflects how a particular trade-off is settled. For example, u(t) determines what fraction of resources go to growth, or what fraction of resources go to production of sons.

The assumption that current resource-allocation decisions influence future reproductive output through changes to individual condition (e.g. size and physiological state) makes finding the optimum sequence of decisions a challenging task (see e.g. Perrin & Sibly, 1993). Social versions of these kinds of problems complicate matters further by allowing an individual's reproductive output and condition to both depend on the allocation decisions made by the members of that individual's social group (e.g. Mirmirani & Oster, 1978; Day & Taylor, 1997, 2000).

To apply the direct-fitness perspective to a social version of the dynamic resource-allocation problem, I treat each allocation decision in the sequence of decisions as an independent, continuous phenotype and devise a version of (3) specific to each. In this way, my problem is similar to many other published models that consider the joint evolution of multiple social traits (Leturque & Rousset, 2003; Wild & Taylor, 2004; Brown & Taylor, 2010). The key differences with this model, though, come in the way classes are structured and in the way reproductive values are handled (Box 1).

Before I deal with class structure and reproductive value in this model, it makes sense to explore some more basic assumptions. I assume that all social groups persist for exactly T time steps, and following Day & Taylor (1997), I assume that social groups consist of exactly N individuals at every point in time. To be clear, I also assume that the dilemma faced by each individual involves how to best divide ones resources between reproduction and improvements in one's own condition.

My focus is on the evolution of only the fraction of resources allocated to reproduction during time step t, i.e. the allocation phenotype u(t). Phenotypes expressed at all other points in time τ ≠ t are assumed to be found at the non mutant level inline image. As before, I store the phenotype of the focal individual and each member of the focal group in an N-dimensional vector, now denoted u(t) with

  • image(4)

for ɛ small.

As I have said, in social situations, the allocation decisions made by each member of the social group can possibly affect the condition of every other member. Let xi(t) (a real number) denote the condition of the ith member of the focal group at time t. I refer to the vector of conditions inline image as the state of the focal group at time t, and I assert that

  • image(5)

with inline image. (At this point I drop the time argument from x and u in most cases for convenience.)

The condition of each member of the focal group helps to form the basis of the class structure I impose here. Specifically, an individual in condition x that also inhabits a group in state x at time t will be placed in class c = (t,x,x). For later use, I note that the reproductive value of each individual in the focal group can be stored unambiguously in the vector-valued function v(t,x).

Because each individual in the focal group is in the same initial condition inline image, and because all previous allocation decisions are assumed to have been made in the non mutant fashion, I can be sure that each member of any group is in the same condition, inline image, during time step t. It follows that the focal individual is always found in one particular class, namely class inline image. The change in its fitness, then, can be derived in much the same way that the change in (3) was derived.

Let inline image express the reproductive success (or the class-inline image component of fitness) achieved by the focal individual during time step t. [More generally, let W(c′)(c;u) denote the number of class-c′ offspring produced by the focal individual in class c.] By neglecting terms in the order of ɛ2, as I did previously, I approximate the change in the reproductive success of the focal individual as

  • image(6)

This is, of course, only one way in which the fitness of the focal individual is changed.

In addition to affecting reproductive success, the mutant allele changes the state of the focal patch at time t: from its would-be state, inline image, to a new state, x′. This change effectively creates one class-(t + 1,x,x′) ‘offspring’ for the focal individual (the focal individual, itself, one time step into the future) while effectively destroying one class-inline image‘offspring.’ These fitness changes can, therefore, be expressed exactly as

  • image(7)

The expressions in (7) are not typical of the direct-fitness method. Typically, the weak action of selection (infinitesimal ɛ) results in small changes to numbers of offspring (compare with Taylor & Frank, 1996; Rousset & Billiard, 2000). In line (7), though, one can see that even the smallest of epsilons result in substantially larger changes in fitness. It is with these rather abrupt changes that our direct-fitness analysis begins to deviate from its usual path.

Although classes in this model contain continuous variables, the fact that there are only a finite number of fitness components affected (eqns 6–7, respectively) means I can use eqn (3) as a template to express the overall effect of the mutant allele as

  • image(8)

where [·] extracts the element of any N-dimensional vector indexed by the value of i that corresponds to the focal individual. Note that in the case of inline image, it does not matter which element I extract, because inline image is a vector of constants.

Line (8) can be simplified by using eqns (6–7) and by setting (without loss of generality) inline image, thus

  • image(9)

The reproductive values that remain in (9) are associated with fitness changes that are – as I mentioned earlier – unusually abrupt. Thus, unusual care must be taken when computing reproductive values.

Using eqn (5), I find that

  • image(10)

where Dxv = [∂vi/∂xj] and Duf = [∂fi/∂uj] represent N × N matrices of first partial derivatives. Substituting (10) into (9) and adopting the notational convention inline image, I arrive at an approximation for the overall fitness change:

  • image(11)

To be clear, when line (11) is positive (respectively, negative) selection favours (respectively, disfavours) deviant allocation of resources to reproduction during time step t.

If investigation of (11) for times t = 0,  … , T − 1 reveals an evolutionary singular sequence of phenotypes, then the evolutionary stability of the sequence can be addressed using various simulation approaches. I explore one such approach with an example in the following section. Before proceeding to that example, though, I must consider more carefully the vector R(0) and the matrix Λ(t + 1), above.

It is typical to identify the elements of the vector R(0) as equilibrium solutions of recursions that, themselves, assume that the mutant's distribution has achieved some stationary or quasi-stationary state (e.g. Taylor, 1992; Rousset, 2004; Wild, 2010). Alternatively, if the social group is composed of individuals whose familial relationships with one another are known, it can be possible to treat R(0) as simply a vector of constants. It should be noted, however, that treating R(0) always as some parameter that is completely independent of other model assumptions can obscure important biological features like kin competition (Griffin & West, 2002).

That said, one always states Λ(t + 1) by means of a recursive expression. To do so, we remind ourselves that the i,jth entry of Λ(t), called it λij(t), describes the marginal effect that the improved condition of individual j during time step t has on the reproductive value of individual i, measured during the same time step (see eqn 11). As this effect must be determined for a group that lacks deviant-allocation behaviours (again, see eqn 11), I make my calculations by first setting ɛ = 0, so that inline image and inline image for all t before proceeding. Next, I note that a small change in an individual's condition will affect the immediate reproductive success of every individual in the group. In a selectively neutral population (ɛ = 0) the reproductive success of each individual in the group can be stored in the vector

  • image(12)

The instantaneous rate at which the reproductive value of individual i is affected by changes in j’s condition – specifically because i’s reproductive output has been altered – could be calculated by taking the partial derivative of the ith element of (12) with respect to the jth element of the state vector inline image (recall that the reproductive value of reproductive success was set equal to one). Equivalently, the instantaneous rate in question could simply be found as the i,jth entry of the matrix of first partial derivatives, which I write as

  • image(13)

A small change in an individual's condition will also affect the reproductive value of its neighbours through changes to the future condition of each one. In a selectively neutral population, the condition of each member of the group in the immediate future can be expressed using the vector,

  • image(14)

As was the case in line (13), the instantaneous rate at which individual k’s condition during the next time step is affected by changes in the condition of individual j can be found as the k,jth element of the matrix of partial derivatives

  • image(15)

Treating Λ(t + 1) as a matrix of ‘exchange rates’ whose i,kth entry converts the change in condition of individual k at time t + 1 into changes to the reproductive value of individual i, we use matrix multiplication to arrive at new matrix

  • image(16)

The ijth entry of this matrix expresses the rate at which the reproductive value of individual i is affected by changes in j’s condition specifically because j’s altered condition, in turn, alters the future state of i’s social group. Summing (13) and (16), we find

  • image(17)

As there can be no reproduction beyond time step t = T − 1, we attach the boundary condition, Λ(T) = 0 to eqn (17).

The recursion in eqn (17) looks more daunting than it actually is. Because I assume in eqn (17) that ɛ = 0, every individual i in the social group is identical to every other individual. Thus, any row of Λ(t) will be a simple permutation of another row, and we need to only consider one of these (i.e. one particular i). Furthermore, any row of Λ(t) will consist of two basic kinds of entries: λii(t) describing the marginal effect that improvement in one's condition has on one's own reproductive value, and λij(t) for j ≠ i describing the marginal effect that improvement in another's condition has on one's reproductive value. To solve (17), then, I need only solve a system of two recursions. To be clear, I begin with the expression in (17), and I use the observations just made to determine that

  • image(18)

where fi and fk are components of f. From the perspective of the individual indexed with i, each non self group member (indexed by k) is the same as every other non self group member in a neutral population. It follows that changes in the condition of any of these group members must affect the reproductive value of individual i at the same rate. In other words, it must be true that λik(t + 1) is independent of k. After λik(t + 1) is factored out of the summation in (18), we see that the summation itself can take one of two values: one value if i = j and another if i ≠ j. The different forms of the summation term appear in equations for λii(t) and λij(t) (i ≠ j), respectively, and in both cases, I still have λii(T) = λij(T) = 0 for all i and j.

Box 1 – Illustrated summary of the approach

  1. Top of page
  2. Abstract
  3. Introduction
  4. Direct fitness
  5. Direct fitness for dynamic resource allocation
  6. Box 1 – Illustrated summary of the approach
  7. Example
  8. Discussion
  9. Acknowledgments
  10. References
  11. Appendix

The direct-fitness approach to modelling kin selection counts the change in numbers of offspring produced by an individual (the focal individual) owing to the deviant behaviour of that individual and, possibly, the altered behaviour of its neighbours. If genetic relatedness between the focal individual and its neighbours is high, then so too is the possibility that both focal individual and neighbours act in the same deviant manner.

In direct-fitness models, the term ‘offspring’ is often used in a broad sense. An individual that survives from one point in time to the next, for example, is considered to be its own ‘offspring.’ Survival, in other words, is treated as a form of clonal reproduction. In those cases where an individual can have different kinds of offspring (e.g. offspring produced by standard reproduction and ‘offspring’ produced through survival), it is usual to weight changes in offspring number using reproductive value. Equation (3) gives one usual case.

In a dynamic resource-allocation problem, the relevant differences among kinds of ‘offspring’ may be so slight that they cannot, strictly speaking, be placed into the discrete categories required by eqn (3). That said, the resource-allocation problem can still be solved using a modified version of the standard direct-fitness count.

Consider, for example, a focal individual that exhibits non mutant behaviour at time t during a given season of T time steps. If this individual is also surrounded by two non mutant neighbours, then it produces some ‘normal’ number of narrow-sense offspring (they, themselves, colonize new groups at time 0 of the next season). The individual also produces another ‘offspring’ by surviving to time t + 1 of the current season. The ‘offspring’ is, of course, the focal individual itself. Although it may be slightly larger than it was at time t, it is still the product ‘normal’ survival and is still surrounded by non mutant neighbours of a ‘normal’ size.

inline image

Now consider a focal individual that exhibits mutant behaviour at time t. The deviant behaviour of the focal individual, as well as the possible deviant behaviour of its neighbours, will certainly contribute to a change in fitness.

inline image

First, let us assume that the focal individual, in this case, produces a lower-than-normal number of narrow-sense offspring. Second, let us assume that, because of the reproductive restraint, the focal individual produces a larger-than-normal ‘offspring’ at time t + 1 that is surrounded by abnormally sized neighbours.

inline image

The second category of mutant offspring is not found in a non mutant group of neighbours. It seems that mutant behaviour creates a new class of offspring that differs in a continuous way from the comparable non mutant class.

To arrive at a direct-fitness count of the fitness change experienced by the focal individual, one first subtracts the number of offspring in scenario B with the number of offspring in A belonging to the same class. In pictures, the change in narrow-sense offspring could be conceived of as:

inline image

Although the cartoon above suggests otherwise, eqn (6) shows that the change it represents is small – in the order of ɛ.

The change in remaining offspring numbers can be similarly depicted as:

inline image

inline image

Note that I have included a ± 1 in the cartoons above to remind the reader that, although the condition of neighbours determines the class to which the focal individual belongs, it is only the focal individual itself that gets counted (see eqn 7).

In standard direct-fitness models, there is always a non zero ‘normal’ number of offspring that can be subtracted from a non zero ‘abnormal’ number to produce a small fitness change (see eqn 2). These small changes in fitness get weighted by neutral reproductive values (see eqn 3; see also Gardner et al., 2011 for a simple discussion of reproductive value and its significance in social evolution). Because changes to the reproduction component of mutant fitness in the present model are also small, I can weight these in the standard way: note that I have weighted the change in the first line of eqn (8) with a neutral reproductive value.

In contrast to the standard situation, the fitness changes associated with survival components of fitness in the present model are not small. When it comes to weight survival-fitness changes with reproductive value, then, I cannot neglect the small changes in reproductive value that occur as a consequence of deviant allocation decisions. Consequently, I must weight the fitness change listed in the second line of eqn (8) by a non-neutral reproductive value (as suggested by the absence of a hat on the x′ variable). It turns out that the non-neutral reproductive value used in the model is an approximation that adds a small change (i.e. marginal reproductive value) to a neutral reproductive value. As shown in eqn (17), marginal reproductive value is, itself, composed of changes in production of narrow-sense offspring, and changes to the future condition of the focal individual and its neighbours.

Once all fitness changes have been appropriately weighted with reproductive values, a simple sum (eqn 11) yields the overall effect of mutant-allocation decisions – this, despite the fact that certain components of fitness belong to a continuum of classes.

Example

  1. Top of page
  2. Abstract
  3. Introduction
  4. Direct fitness
  5. Direct fitness for dynamic resource allocation
  6. Box 1 – Illustrated summary of the approach
  7. Example
  8. Discussion
  9. Acknowledgments
  10. References
  11. Appendix

In this section, I investigate a discrete-time version of a simple, continuous-time example studied by Day & Taylor (2000) to illustrate how the ideas from the previous section are put into practice.

Assumptions and analysis

Assume that during any time period t, the ith individual of a group receives

  • image(19)

units of energy, where α ∈ (0,1) is constant. Overall, improved condition means more energy, and from (19) one can see that 100 × (1 − α) per cent of the increase is because of absolute improvements in condition, whereas 100 × α per cent is because of improvement measured relative to the group-average condition.

Consider time period t and assume that no prior allocation decisions in a focal group have been deviant. In that case, if the focal individual in the focal group allocates a deviant fraction, u(t;ɛ), of the energy it receives during time period t to reproduction, then its reproductive output during that time period is

  • image

It follows that the deviant-allocation strategy changes the reproductive output of the focal individual by approximately

  • image

units.

If the fraction (1 − ui(t;ɛ)) of an individual i’s energy not allocated to reproduction contributes to improvement in that individual's condition, then

  • image

With this expression, and the assumption that Ri(0) = r for i ≠ •, we determine that

  • image

where the subscript ○ is any index that is not used to label the focal individual.

With the above calculations in hand, I see that the approximate overall change in the fitness of the focal individual during time step t is equal (up to multiplication by a positive constant) to

  • image(20)

It follows that the direction of selection at time t will be determined by the size of

  • image(21)

relative to 1. Biologically speaking, the expression in eqn (21) represents the marginal residual reproductive value of the focal individual's genetic lineage (itself and its N − 1 neighbours). When this expression is < 1, selection favours ε > 0; when this expression is > 1, selection favours ε < 0. Selection is, of course, indifferent when the expression is zero.

Using eqn (18), I find that

  • image(22)

with λ••(T) = λ•○(T) = 0.

Because λ••(T) = λ•○(T) = 0, line (20) immediately tells us that the selection always favours those mutant-allocation strategies that, relative to the non mutant, increase reproductive effort during time step T − 1. Thus, any sequence of decisions favoured by selection, called u*(0),  … , u*(T − 1), will involve complete allocation to reproduction during time step T − 1 (i.e. u*(T − 1) = 1).

As long as there exists complete allocation of energy to reproduction (i.e. as long as inline image), the recursions in (22) admit a very simple solution:

  • image

Working backwards in time (or forwards in τ, from τ = 0) under the assumption that inline image, I note that the sequence

  • image(23)

cannot exceed 1 until inline image (‘⌈X⌉’ means ‘the smallest integer that is not less than X’), or when time is t* = T − τ*, or simply

  • image

(‘⌊X⌋’ means ‘the largest integer that is not greater than X’).

Tabulating the information accumulated so far (Table 1), I find that selection favours complete allocation to reproduction from time t* onwards; in the event that t* < 0, one would say that selection always favours complete allocation to reproduction.

Table 1.   A partial summary of the adaptive sequence of allocation decisions, backwards in time. Given that complete allocation to reproduction is adaptive at time t = T − 1 (see text), it is easy to see that complete allocation is also adaptive at all previous times back to time t* = T − τ*. In the time step just prior to t*, complete allocation to improvement in condition is adaptive. This information sets the stage for our investigation of adaptive decisions for time steps prior to T − τ* − 1.
t(λ••(t + 1)+(N − 1)λ•○(t + 1)r)(1 − λ••(t + 1) − (N − 1)λ•○(t + 1)r)u*(t)
  1. When the inequalities in this row hold as equalities all allocation decisions are equally adaptive. We can, in this case, choose to include any one decision as part of a candidate stable policy; for convenience, we choose inline image

T − 1=0 < 1=1 > 0, (∴ɛ > 0 always favoured)1
T − 2< 1> 0, (∴ɛ > 0 always favoured)1
T − τ*< 1> 0, (∴ɛ > 0 always favoured)1
T − τ* − 1≥ 1≤ 0, (when strict ɛ < 0 favoured)0
T − τ* − 2???
0???

It remains to determine adaptive allocation levels for times t < t*. For t = t* − 1, I find that the expression in (21) cannot be < 1, and so selection favours (or at least does not disfavour) complete allocation to improvements in condition inline image. As long as there exists complete allocation to improvements in condition (i.e. as long as inline image) prior to t* = T − τ*, the solution to recursions in eqn (22) is

  • image

and so

  • image(24)

where inline image expresses the relatedness between the focal individual and the average member of the focal group. The sequence in eqn (24) increases in τ– in other words, as one moves backwards in time. Once λ•• + (N − 1)λ•○r reaches a height of 1 greater, then, it continues to increase. It follows that selection will continue to favour complete allocation to improvements in condition. Rather than completing Table 1, I can now simply state that

  • image(25)

is an adaptive sequence of allocation decisions. Figure 1 provides a complete summary of the reasoning behind this result.

image

Figure 1.  An illustration of the procedures used to determine u*(t) in eqn (25). Working backward in time from time t = T, the sequence λ••(t) + (N − 1)rλ•○(t) increases (top panel). The optimal u*(t) (bottom panel) is determined by considering the value of the t + 1st element of the sequence in the top panel: if the t + 1st element is above 1, then u*(t) = 0; if the t + 1st element is below 1, then u*(t) = 1. Notice that the sequence in the top panel is described by eqn (24) up to and including time inline image (open circles), at which time it switches to a form based on eqn (23) (filled circles).

Download figure to PowerPoint

The result in (25) makes intuitive sense when one realizes that any u < 1 in this example translates into increased local competition for energy. As an individual's relatedness to the average member of its group increases, one would expect selection to favour decreased local competition, a more ‘cooperative’ kind of behaviour. This expectation is clearly reflected in eqn (25): larger inline image means selection favours an earlier switch to pure reproduction, or rather an earlier shift from increasing competitive tendencies to purely cooperative ones.

Simulation

The emphasis that the direct-fitness perspective places on individual fitness (rather than individual inclusive fitness) makes it easier to use this perspective when designing simulations intended to confirm (or even establish) the effect of kin selection on the evolution of dynamic traits. Here, we use the direct-fitness perspective to inform the design of a genetic algorithm (Bounds, 1987) intended to confirm that the result presented in eqn (25) can be considered to be a best strategic response to itself.

Following Wilson (2000), I treat individuals in our simulated population as virtual chromosomes. Each chromosome has T loci, and each locus stores a different allocation phenotype u(t). Unlike Wilson (2000), though, I furnish each individual/chromosome with an auxiliary chromosome that stores another sequence of T phenotypes, u(t) = (1 − r)u*(t) + ru(t). The auxiliary chromosome simulates the allocations made by the average non self neighbour of the focal individual in the model above.

The information stored in each chromosome–auxiliary pair allows me to track changes in condition using the equations

  • image(26)

Of course, x(t) represents the condition of the focal individual during time step t and x(t) represents the condition of its average neighbour during that same period of time. With the sequence of conditions in hand, I am able to determine the fitness of any chromosome as its lifetime reproductive success (LRS):

  • image(27)

Chromosomes are chosen to produce offspring for a descendant population according to LRS. Offspring are produced sexually by allowing two chromosomes to cross-over (crossing-over, alongside mutation, ensures that new phenotypic variation is created over the course of the simulation). Once produced, the offspring generation replaces the parental one. The entire process is repeated many times until fitness LRS is maximized.

In the parlance of constrained optimization theory, the genetic algorithm described above finds a sequence u(0),  … , u(T − 1) that maximizes LRS subject to (26). Figure 2 shows that the constrained maximization problem is solved when u = u*, and so the genetic algorithm confirms that the result in (25) is a best response to itself. A Matlab script that executes the genetic algorithm and produces Fig. 2 is appended.

image

Figure 2.  Results from iteration of a genetic algorithm indicate that u*(t) is a best response to itself. In (a–c), the left figure panel shows that average lifetime reproductive success (average LRS) of virtual chromosomes used in the genetic algorithm increase over the course of 10 000 simulated generations to match that enjoyed by an individual using u*(t) in a population dominated by the same strategy (horizontal dashed line, left panel). The right figure panel in (a–c) shows that the policy identified by the genetic algorithm (the best response to u*(t), indicated by filled circles) is u*(t) itself. Parameters were N = 5, T = 10, and in (a) r = 0.25, α = 0.1, (b) r = 0.9, α = 0.8, (c) r = α = 0.9. Results illustrate a range of switching times; additional parameter combinations can be investigated using the Matlab code provided in the Appendix.

Download figure to PowerPoint

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Direct fitness
  5. Direct fitness for dynamic resource allocation
  6. Box 1 – Illustrated summary of the approach
  7. Example
  8. Discussion
  9. Acknowledgments
  10. References
  11. Appendix

The direct-fitness approach to modelling the evolution of social traits is a popular alternative to the classical inclusive-fitness-based approach introduced by Hamilton (1964). In addition to its popularity, the direct-fitness method is, itself, quite versatile. Direct fitness can be used in place of more complicated population genetic models (Rousset & Billiard, 2000; Rousset, 2003; Wild, 2010), or in place of modern game-theoretic models (Taylor & Frank, 1996; Ajar, 2003), even when its cousin, the inclusive-fitness method, fails (Frank, 1997; Day & Taylor, 1998).

In this paper, I expand the scope of the direct-fitness approach to include dynamic characters usually studied using optimal-control theory (Perrin & Sibly, 1993). By doing so, I provide a robust alternative to the inclusive-fitness-based approaches already available (Day & Taylor, 1997, 2000).

Unlike the inclusive-fitness-based approaches that emphasize the use of the so-called Hamiltonian function for modelling, the direct-fitness approach used here emphasizes fitness and reproductive value. Although Hamiltonian functions can be interpreted biologically, the interpretation is cumbersome (see for e.g. Day & Taylor, 1997), and in an evolutionary context, the interpretation is bound to be less straightforward than the interpretation of familiar concepts like fitness and reproductive value. The relative ease with which evolutionary success is described using direct-fitness approach, then, could be counted as an advantage of adopting this approach instead of inclusive fitness.

A second, more practical advantage to the direct-fitness perspective is because of the focus it places on individuals’ fitness, rather than the fitness of an individuals’ kin groups. Simply put, by placing emphasis on individual fitness, one can more clearly see how to use simulation of competitive systems to verify analytical treatment of models. As these analytical treatments deal with necessary, rather than sufficient, conditions for optimal or stable sequences of behaviours (e.g. Day & Taylor, 1997, 2000), the importance of simulation should not be underestimated.

One important limitation to the direct-fitness approach (also a limitation to existing inclusive-fitness-based approaches) is that it only provides us with a way of determining whether one sequence of decisions (namely u(0),  … , u(T − 1)) will proliferate in a population dominated by a different one (namely inline image). On its own, the approach does not provide us with a sure-fire way of identifying candidate-stable policies. The fact that we were successful in determining a candidate-stable policy in the example stems from simple forms taken by the functions used therein. The determination of candidate-stable policies, in general, will require the development of existing computational techniques that find stable policies by iterating a dynamical system based on the so-called best-response function, until a Nash equilibrium is found (McNamara et al., 1997). Best-response functions based on an overall change in direct fitness – like that described in the previous section – will likely contribute to necessary development of computational tools.

Of course, the direct-fitness approach described above could be generalized further. For example, one could cope with situations in which neighbours are not necessarily in the same initial state, or situations in which different kinds of reproductive success are possible by constructing weighted averages of expressions like that in (11), where weights are simply the relative frequencies with which the various neighbour states are encountered (Taylor et al., 2007). One might also investigate the effects of temporal changes in group composition by modifying direct-fitness-based tools that deal with temporal effects of social behaviour (Lehmann, 2007, 2010).

An important issue remains outstanding. In my introduction to the paper, I mentioned that the direct-fitness method can be applied in a systematic manner to study kin selection. Taylor & Frank (1996), for example, describe how one can begin with a fitness function, differentiate that function using the Chain Rule, replace a derivative term with a relatedness coefficient and ultimately end up with a kin-selection model. Although I have outlined how direct fitness works in dynamic resource-allocation problems, I have not provided the reader with the level of step-by-step instruction offered by Taylor & Frank (1996). Those readers looking for a step-by-step way of applying the direct-fitness method to dynamic resource-allocation problems might consider defining a fitness function in terms of another kind of function, namely the Dirac delta function. This function takes a value of zero for every real number except zero, itself, where it takes a value of + ∞. The properties of the Dirac delta function mean that fitness functions that make use of it can be differentiated in the ‘Taylor-Frank’ way to produce models of overall fitness change like that found in eqn (11). I am currently working on a more detailed description of this delta function approach to model construction that includes a procedure for implementation using a computer algebra package.

Acknowledgments

  1. Top of page
  2. Abstract
  3. Introduction
  4. Direct fitness
  5. Direct fitness for dynamic resource allocation
  6. Box 1 – Illustrated summary of the approach
  7. Example
  8. Discussion
  9. Acknowledgments
  10. References
  11. Appendix

Funding was provided by the Natural Sciences and Engineering Research Council of Canada. I am grateful to Troy Day, Laurent Lehmann and two anonymous reviewers whose comments on various earlier drafts led to substantial improvements.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Direct fitness
  5. Direct fitness for dynamic resource allocation
  6. Box 1 – Illustrated summary of the approach
  7. Example
  8. Discussion
  9. Acknowledgments
  10. References
  11. Appendix

Appendix

  1. Top of page
  2. Abstract
  3. Introduction
  4. Direct fitness
  5. Direct fitness for dynamic resource allocation
  6. Box 1 – Illustrated summary of the approach
  7. Example
  8. Discussion
  9. Acknowledgments
  10. References
  11. Appendix
A genetic algorithm

In this appendix, I provide the Matlab script used to generate the panels of Fig. 2. As I mentioned in the main text, the genetic algorithm found in the script is based on one presented by Wilson (2000). It maximizes the expression in (27), subject to the constraints in (26); in doing so it solves a discrete version of a problem introduced by Day & Taylor (2000). As written, the script completes in approximately 2000 s when run on a standard desktop computer.

Because the analysis of the problem presented in the main text assumes that ɛ is small, I could neglect the possibility that condition x became negative. In the simulation, though, negative condition can occur; to avoid this one's choice of α is constrained as a result (α cannot be too large relative to inline image). The problem is attributed to the biologically artificial nature of the example and is not a problem with the algorithm per se.

I have also found that Matlab's command mnrnd sometimes returns something other than a random number (NaN in Matlab) and produces an error as a result. In my experience, this problem can be fixed by writing one's own multinomial random number generator (one that does not have problems with probability mass functions whose elements sum to something other than 1 owing to small rounding errors).

Matlab M-file

% Matlab script that executes a genetic algorithm based on Wilson (2000).

% Considers example found in the main text (see also Day & Taylor, 2000).

%——–Required for simulation—————————-

m = 100; %Size of chromosome population

TtlGens = 10000; %Number of generations to simulate

avgLRSdata = zeros(1,TtlGens); %keeps track of history of avg LRS

%——–Parameters for example—————————–

T = 10; %there are T time units t = 0, ..., T 1

alpha = 0.1;

xhat0 = 0.5;

N = 5;

r = 0.25;

%——Simulation——————————————

%Compute the adaptive sequence of allocations using eqn 25

tstar = floor(T−1/(1−alpha*(1/N + r*(N−1)/N)));

ustar = zeros(m,T);

ustar(:,tstar + 1:end) = 1; %u(tstar) is (tstar + 1)st element of this Ttuple

% Initialize population of m chromosomes

ubullet = rand(m,T);

% Simulate TtlGens generations

tic

for gen = 1:TtlGens

 % solve for sequence of conditions numerically by iterating eqn 26

 xbullet = zeros(m,T);

 xcirc = zeros(m,T);

 xbullet(:,1) = xhat0;

 xcirc(:,1) = xhat0;

 for tidx = 1:(T−1)

  xbullet(:,tidx + 1) = xbullet(:,tidx) + ...

  (1ubullet(:,tidx)).*(xbullet(:,tidx) ...

  alpha*(xbullet(:,tidx)/N+xcirc(:,tidx)*(N−1)/N));

  xcirc(:,tidx+1)= xcirc(:,tidx)+...

  (1−(1−r)*ustar(:,tidx)r*ubullet(:,tidx)).*(xcirc(:,tidx)...

  alpha*(xbullet(:,tidx)/N+xcirc(:,tidx)*(N−1)/N));

end

%calculate LRS using eqn 27

LRSperStep = ubullet.*(xbulletalpha*(xbullet/N+xcirc*(N−1)/N));

LRS = sum(LRSperStep,2);

avgLRS = mean(LRS);

avgLRSdata(1,gen) = avgLRS;

% cull population of chromosomes based on avg LRS

LRStmp = LRS(LRS>avgLRS);

ubullet = ubullet(LRS>avgLRS,:);

size(ubullet);

SizeAfterCull = ans(1);

LRS = LRStmp;

% reproduction to establish new population

p = (1/sum(LRS))*LRS;

ubulletNext = zeros(m,T);

for idv = 1:m

 mom = mnrnd(1,p)*(1:SizeAfterCull)’;

 dad = mnrnd(1,p)*(1:SizeAfterCull)’;

 %crossing over

 tX = unidrnd(T,1);

 baby = [ubullet(mom,1:tX), ubullet(dad,(tX + 1):T)];

 %mutate baby

 for tidx = 1:T

  baby(tidx) = baby(tidx) + random(‘normal’,0,0.001);

  if baby(tidx) > 1

  baby(tidx) = 1;

  end

  if baby(tidx) < 0

  baby(tidx) = 0;

  end

  end

  %add to new population

  ubulletNext(idv,:) = baby;

 end

 ubullet = ubulletNext;

end

results = mean(ubullet);

toc

%———Present results graphically————————–

%Calculate LRS of ustar to compare to result of simulation

xstar = zeros(1,T);

xstar(1,1) = xhat0;

for tidx = 1:(T−1)

 xstar(1,tidx + 1) = xstar(1,tidx)...

 + (1−alpha)*(1−ustar(1,tidx)).*xstar(1,tidx);

end

LRSstar = sum(ustar(1,:).*xstar*(1−alpha),2);

%Create figure

figure

subplot(1,2,1)

hold on

plot(1:TtlGens, avgLRSdata, ‘−k’, ‘LineWidth’,2);

plot([1,TtlGens],[LRSstar, LRSstar],‘–k’);

ylabel(‘average LRS’,‘FontName’, ‘Arial’, ‘FontSize’, 18)

xlabel(‘generation’,‘FontName’, ‘Arial’, ‘FontSize’, 18)

box on

subplot(1,2,2)

hold on

plot(0:(T1),results,‘ok’, ...

  ‘MarkerEdgeColor’,‘k’,...

  ‘MarkerFaceColor’,‘k’,...

  ‘MarkerSize’,10)

plot([tstar−0.5,tstar−0.5],[0,1],‘–k’)

ylabel(‘best response to, u*’,‘FontName’, ‘Arial’, ‘FontSize’, 18)

xlabel(‘time, t’,‘FontName’, ‘Arial’, ‘FontSize’, 18)

box on

hold off

% axes may have to be logscaled by user using the ‘Plot Tools’ window

% data generated may be saved using the ‘save’ command