The paradoxical persistence of heritable variation for fitness-related traits is an evolutionary conundrum that remains a preeminent problem in evolutionary biology. Here we describe a simple mechanism in which social competition results in the evolutionary maintenance of heritable variation for fitness related traits. We demonstrate this mechanism using a genetic model with two primary assumptions: the expression of a trait depends upon success in social competition for limited resources; and competitive success of a genotype depends on the genotypes that it competes against. We find that such social competition generates heritable (additive) genetic variation for “competition-dependent” traits. This heritable variation is not eroded by continuous directional selection because, rather than leading to fixation of favored alleles, selection leads instead to allele frequency cycling due to the concerted coevolution of the social environment with the effects of alleles. Our results provide a mechanism for the maintenance of heritable variation in natural populations and suggest an area for research into the importance of competition in the genetic architecture of fitness related traits.

Explaining the maintenance of heritable genetic variation for fitness-related traits remains a fundamental unresolved problem in evolutionary biology (e.g., Bürger 2000; Rice 2004). Such variation is paradoxical because it should be quickly eroded by selection, leading to low levels of standing heritable variation (discussed in Mousseau and Roff 1987; Bulmer 1989). Despite this, empirical patterns reveal that traits closely related to fitness often exhibit substantial amounts of heritable variation in natural populations (e.g., Mousseau and Roff 1987) and therefore can evolve (but see Houle 1992; Burt 1995). Among explanations for the maintenance of genetic variation in natural populations are: mutation–selection balance (Lande 1975), heterozygote advantage (or heterosis, e.g., Berger 1976), fertility selection (Feldman et al. 1983), environmental or temporal heterogeneity (Tomkins et al. 2004; Byers 2005), epistasis (Wolf et al. 2000), multilocus frequency-dependent selection (Clarke 1979; Slatkin 1979), and others (Bürger 2000). Although all of these explanations have some theoretical or empirical support, none has emerged as a general solution (see Bürger 2000; Rice 2004 for reviews). More importantly, mechanisms that have been shown to maintain allelic variation do not always provide an explanation for the maintenance of heritable variation for traits at equilibrium (i.e., additive genetic variation), especially in the face of ongoing directional selection (Rice 2004).

Here we present a simple model demonstrating that social (intraspecific) competition (sensu West-Eberhard 1983) can lead to the long-term maintenance of heritable genetic variation for fitness-related traits, even under continuous directional selection. We illustrate this using a simple model in which the expression of a fitness-related trait is determined by the relative success of a given genotype in competition with other genotypes in a population. We assume that the pattern of competitive success of genotypes is nontransitive, such that no single genotype outcompetes all other genotypes (e.g., Weisbrot 1966). We assume that simple directional selection acts on this “competition-dependent trait,” as may be the case for many condition-dependent traits; for example those involved in mate choice and sexual selection (Bonduriansky and Rowe 2005). We begin the presentation of the model with a biological rationale for our assumptions and then discuss the implications of our results.

The Model


Our model is built upon two main assumptions: (1) that competition among genotypes is nontransitive, meaning that the competitive success of a genotype depends entirely on the genotype it competes against and (2) that expression of a fitness-related trait depends upon success in social competition. Support for the first assumption is twofold. First, it has long been known that the outcome of competitive interactions can be strongly dependent upon nonadditive genotypic interactions among competitors (e.g., Lewontin 1955; Parsons 1958; Roy 1960; Park et al. 1964; Weisbrot 1966; Wade 2000), where the competitiveness of a genotype may be largely determined by the genotypes of its competitors. Second, we assume selection should act to eliminate most genetic variation relating to a linear hierarchy of competitors (because the most successful competitive alleles conferring success in linear hierarchies should become fixed: West-Eberhard 1979), leaving only complex (i.e., nonlinear and nontransitive) competitive interactions to persist in evolutionary time (excepting special circumstances, e.g., Byers 2005).

Our assumption that competition influences trait expression is generally well supported by available empirical evidence. The effect of competition has often been measured through its affect on the expression of some trait (e.g., Griffing 1989; Tonsor 1989; Stevens et al. 1995; Wolf et al. 1998; Bonduriansky and Rowe 2005; Muir 2005), which implies that the expression of such a trait is “competition-dependent.” More generally, variation in the resource pool available to an individual may be a major determinant of overall condition (Rower and Houle 1996) and, therefore, success in competition for limited resources could be an important source of variation in “condition-dependent” traits. The idea that condition dependence mediates the expression of fitness-related traits has been most well explored in the context of sexual selection (e.g., see Sheldon 1997; Merila 1999; David et al. 2000), where it is hypothesized that sexually selected traits are costly and therefore are honest indicators of male genetic quality (Andersson 1994). Such condition-dependent expression has been documented in a number of systems (e.g., see Rowe and Houle 1996; David et al. 2000; Cotton et al. 2004a), but the generality of such results have been questioned and there remains a lack of evidence that such variation is heritable (Cotton et al. 2004b). Thus, given the available data it is logical to assume that competitive success may often be a major determinant of resource availability and condition in natural populations and, as a result, will affect the phenotypes of individuals. Consequently, the expression of traits affected by competitive success should directly reflect the relative competitive ability of individuals, implying that genetic variation for competitive success should underlie expression of competition or condition-dependent traits.


Here we develop a single-locus, three-allele model to examine the genetics and evolution of complex competitive interactions. This is meant to represent the simplest model of competitive interactions between genotypes. Although simpler, two-allele models are constrained because two-allele systems have limited evolutionary “degrees of freedom” (i.e., they have a single “evolutionary degree of freedom” with one allele frequency parameter, see Gomulkiewicz and Kirkpatrick 1992). In contrast, a three-allele system has a critical extra evolutionary degree of freedom that allows more complex behavior and makes it a better approximation of natural systems. Therefore, we expect our model to approximate the dynamic behavior of more complex systems and, although we do not allow for mutation in this model, we suggest that a model that includes mutational input of alleles may show similar evolutionary dynamics.

We assume random mating in a diploid population is large enough to ignore drift and other stochastic processes (we discuss the potential influence of drift below in the Discussion). We designate the three alleles at the locus affecting competitive interactions (locus A) as A1, A2, and A3, which have frequencies p1, p2, and p3, respectively. The frequencies of diploid genotypes prior to selection conform to Hardy–Weinberg proportions, where diploid genotype frequencies can be calculated from the trinomial expansion of (p1+p2+p3)2. The frequencies of the diploid genotypes are designated fij, with the subscripts designating the two alleles that make up the genotype (i.e., subscripts corresponding to: 1 =A1, 2 =A2, and 3 =A3), where fij=fji. To simplify the presentation of the model throughout, parameters are defined for all nine diploid ordered genotypes despite the fact that we assume that parent of origin of alleles is unimportant.

Genetics of competition

We assume that locus A determines an individual's success in competitive interactions with other genotypes and that success in competition determines the resources available to the individual. We denote the phenotypic values associated with the success of a genotype in competition with another genotype as xijkl, where the phenotypic value is a measure of the amount of resources gained by a genotype AiAj competing with genotype AkAl and where primes indicate that the second genotype is that of a competitor (nb., the value of xijkl is the success of the AiAj genotype, whereas the AkAl genotype would have the value xklij from this interaction). The phenotypic values of this “competitive success” trait (xijkl,) are defined as the amount of resources gained or lost in competition.

We assume that competition conforms to a “zero-sum” game (Maynard Smith 1982), where individuals compete for limited resources and divide the total among themselves (i.e., the competitive success of the two interacting genotypes must be equal and of opposite sign, where xijkl+xklij= 0,). We assume that the outcome of these competitive interactions is nontransitive such that the competitive success of a genotype depends on the genotype of its competitor. We implement this model of competition by assuming an antitransitive competitive hierarchy. This fitness assumption is akin to that of the game theory model “rock–scissors–paper,” (see Maynard Smith 1982), where A1 is competitively superior to A2, A2 is superior to A3, and A3 is superior to A1, however here we explicitly examine the population genetic dynamics of this system and assess the effect of such a pattern of competition on heritable variation. Although competition between genotypes is nontransitive, we assume that the competitive success of a genotype is determined by the additive, independent effects of the alleles that constitutes it (i.e., we assume no dominance). In the most general case, we assume that the relative amount of resources won and lost in competition between each pair of alleles may differ, making the game of rock–scissors–paper a special case in which all competitive outcomes result in the same gain or loss of resources (see Maynard Smith 1982; Hofbauer and Sigmund 1998). We denote the phenotypic value of the “winner”+rij and the “loser”−rji, where rij (where the first subscript denotes the focal genotype and the second denotes the competitor's genotype) is a parameter that measures the proportional difference in resources acquired by competitors and rij is assumed to equal −rji. We denote the success of each allele against every other allele with the parameters r12, r13, and r23. For example, the success of A1 versus A2 is r12 whereas the success of A2 versus A1 would be −r12. When competition is between genotypes containing identical alleles we assume that resources are divided equally (i.e., xijkl= 0 for two individuals of the same genotype). The values of rij can be considered as the “payoffs” from competition, and the full set of competitive success of each allele or genotype against each other allele or genotype may be thought of as a payoff matrix (e.g., see Hofbauer and Sigmund 1998).

Because competitive success is determined by the average of the independent effects of the alleles that make up a genotype, the success of a homozygote competing against another homozygote is equal to the success defined above for an allele. For example, in competitive interactions between the homozygotes A1A1 and A2A2, A1A1 would gain +r12 resources, whereas A2A2 would lose −r12 resources (i.e., x1122=+r12 and x2211=−r12). This is because each A1 allele “beats” each A2 allele, giving the winner (r12+r12)/2 resources whereas the loser loses this same amount. Competition between heterozygotes is more complex because the payoff from competition between each pair of alleles is different and, consequently, the net payoff for competition between genotypes is the average of the four pairwise allelic payoff values. For example, success of A1A2 against A1A3 (x1213) is calculated as the average of the pairwise success of each allele in one individual against each allele in the other: [(A1 vs. A1) + (A1 vs. A3) + (A2 vs. A1) + (A2 vs. A3)]/4 = (0 −r13− r12+r23)/4 =x1213. In the case in which all values of rij are the same, that is, all rij= r, this value would correspond to −14r, but when considering more complex scenarios in which competition differs between different alleles, the success of A1A2 against A1A3 (x1213) would simply have the value (r13− r12+r23)/4. Although alternative assumptions about the pattern of competitive success of genotypes are possible, we found that the model results are not particularly sensitive to assumptions about how we construct the success of diploid genotypes given a pattern of competitive success of alleles, including assumptions about dominance effects of alleles.

We assume that individuals interact at random, such that the frequencies with which each genotype interacts with each other genotype are determined solely by the frequencies of the genotypes. We designate the frequencies of interactions between genotypes Fijkl (following xijkl above). Under the assumption of random interactions these frequencies are equal to the products of the frequencies of the genotypes of competitors (i.e., Fijkl=fijfkl).

The expected phenotypic value of success in competition for each genotype is calculated by averaging across its success in competitive interactions (xijkl) with all other genotypes (the results of the model are not sensitive to this assumption and the results are essentially the same with different assumptions, e.g., assuming pairwise interactions of genotypes). Thus, the average success of the genotype ij is taken as xij··, where the dot subscripts denote averaging over all values of a subscript. We denote these average phenotypic values for each genotype Xij (which are equal to xij··), and are calculated as:


Note that, although the values of xijkl are not frequency dependent, because they are assigned directly to a genotype in competition with another genotype, the values of Xij are dependent on allele frequencies as they are averaged over all competitive interactions of a given genotype. For example, X11 simplifies to (r12p2r13p3), which shows that the competitive success of the A1A1 genotype is determined by the relative abundance (p2) of the A2 allele (which A1 wins against) versus the frequency (p3) of the A3 allele (which A1 loses to) each weighted by the proportion of resources to be won or lost in that interaction (r12 and r13, respectively).

Expression of competition-dependent traits

We assume that the phenotypic value of a competition-dependent trait (denoted cij for the genotype ij) is directly determined by the outcome of competition for resources (which determines the resource pool available to an individual). We assume resources affect the expression of the competition-dependent trait during development such that the expected phenotypic value of a genotype is defined as


where ψ defines the relationship between resources gained or lost through competition and the expression of the competition-dependent trait and μc gives the mean expression independent of the effect of competition.

Genetic variances

The mean value of the competition-dependent trait inline image is calculated as the sum of the expected phenotypic value of each genotype multiplied by its frequency (i.e., inline image). Because we assume that competition is a zero-sum game inline imagec regardless of allele frequencies or the relative effects of competition, (i.e., values of rij). Consequently, the mean trait value for the competition-dependent trait does not evolve as allele frequencies change.

The total genetic variance for the competition-dependent trait (VG) is the sum of the squared deviations of the average phenotypic values of the genotypes from the mean multiplied by the frequency of the corresponding genotype (i.e., VG=inline image). The total phenotypic variance of the competition-dependent trait (VP) depends on assumptions about the number of competitive interactions experienced by genotypes and the nonsocial environmental variance. Therefore, because the model imposes no assumptions about the number of interactions experienced by genotypes, the relative importance of random environmental sources of variation or the relative contribution of the competition-dependent effects compared to other sources of trait variation, we necessarily focus on the expected pattern (rather than the amount) of heritable (i.e., additive) genetic variation. Consequently, we leave the question of the relative importance of sources of variation as an empirical and biological problem that requires further investigation.

To calculate the additive genetic variance we first calculate the average effects of the three alleles (technically, we calculate the average excess, which is equivalent to the average effect under our assumed model, Templeton 1987). The average effect of each allele can be calculated as the average phenotype associated with the allele measured as a deviation from the population mean (cf. Crow and Kimura 1970) inline image:


The additive genetic variance (Va) is calculated as twice the variance in average effects of the alleles (Crow and Kimura 1970) [Va= 2(α21p122p223p3)]:


Although we do not present the results here, we have also derived the additive genetic variance using parent–offspring regression and find that the results of the two approaches are qualitatively the same (the two do not yield identical results because allele frequencies differ between the parental and offspring generations). Here we use the analytically simpler approach of twice the variance in average effects.

Selection and evolution

We assume positive directional selection on the competition-dependent trait, where fitness is a linear function of expression of the competition-dependent trait (cij), with larger trait values having higher fitness. Such directional selection could occur for many reasons, for example when females prefer to mate with males that have larger trait values (Andersson 1994). Thus, we define the fitness of a particular genotype (wij) as its phenotypic value for the competition-dependent trait (cij) multiplied by the strength of selection (s) (with s being > 0 under directional selection for larger values):


where μ_w is expected fitness independent of the trait values of individuals. Given the linear relationship between trait values and fitness, the strength of selection acting at the level of the locus (rather than the level of the trait) is sψ. That is, ψ translates selection on the trait into selection on the alleles, and therefore, s is always multiplied by ψ to determine the strength of selection on alleles.

To calculate the evolutionary change in allele frequencies we first calculate population mean fitness inline image from the fitness values of each genotype weighted by its frequency


Thus, equation (6) demonstrates that population mean fitness does not depend on allele frequencies.

We model the change in allele frequencies in continuous time as the allele frequency at one point in time minus the expected fitness of the allele in each genotype weighted by the frequencies of the different genotypes, divided by population mean fitness. The rate of evolution in continuous time is determined by the value of the constants in the expression for allele frequency change inline image. This constant is the relative strength of selection on the locus (i.e., sψ scaled by mean fitness,inline image) and determines the unit of time of this system (i.e., the strength of selection is taken per unit time). This constant inline image is denoted by κ. By defining the changes in allele frequencies per unit time we can produce a set of equations that define the evolutionary change in the frequencies of the three alleles through time


These equations yield four equilibria. There are the three “trivial” equilibria in which one of the alleles is fixed (i.e., where p1, p2, or p3= 1) and one neutral equilibrium point where the three alleles have frequencies p1=r23/(r12+r13+r23), p2=r13/(r12+r13+r23), and p3=r12/(r12+r13+r23), respectively (see the Appendix for details). We also find that pr231pr132pr123 is a conserved quantity (i.e., does not change as the system evolves; see the Appendix for details). Thus, we find that the system exhibits neutrally stable allele frequency cycles around a neutrally stable equilibrium point (see the stability analysis in the Appendix). This stability is not a consequence of a cost or payoff for competing against oneself (because we incorporate no such effect)—the latter is known to produce cycling in some game theory versions of rock–scissors–paper (Maynard Smith 1982). Note also that the four equilibria correspond to the points where the additive genetic variance is zero (being nonzero for all other allele frequencies, see Fig. 2).

Figure 2.

Additive genetic variance for the competition-dependent trait as a function of allele frequencies for the case in which the strength of competition equal between all pairs of alleles such that r12=r13=r23. As in Figure 1, allele frequencies are defined by the ternary with the elevation of the surface giving the additive genetic variance corresponding to a particular frequency of the three alleles.


To explore the implications of the model we divide discussion of our results into two conceptual sections. In the first section, we discuss why the nontransitive pattern of competition results in heritable genetic variation; in the second section, we discuss how this variation is maintained through evolutionary time.

Genetic variance of competition-dependent traits

Our model demonstrates that, despite assuming a nonadditive model of competition among alleles with variable strengths of competition between pairs of alleles, competition ultimately contributes to additive differences between the effects of the alleles at the level of the competition-dependent phenotype (i.e., making the average effects of alleles nonzero). This can be seen in Figure 1, where we show the average effect of the A1 allele as a function of the frequencies of the three alleles (illustrated for the case in which r12=r13=r23). We see that the average effect changes from positive to negative depending on allele frequencies. Similar plots for the average effects of the other two alleles show this same pattern (eq. 3), but the magnitudes and signs of the effects differ for the three alleles at nearly all allele frequencies (having the same average effect only when all three are at the neutrally stable equilibrium).

Figure 1.

The average effect of the A1 allele (α1) on the competition-dependent trait is shown as a function of allele frequencies (the average effect is equal to the average excess, which is calculated as the mean phenotype of an allele measured as a deviation from the population mean). Shown is the case in which the strength of competition is equal between all pairs of alleles, such that r12=r13=r23. Each point in the space defined by the three axes of the ternary (triangular) plot corresponds to a particular set of frequencies for the three alleles (the space is constrained such that p1+p2+p3= 1). The elevation of the surface above the ternary defines the average effect of the allele at a particular set of allele frequencies in the ternary. The scale for the average effect is arbitrary and the surface is simply defined in terms of whether the allele has a positive or negative effect on the expression of the trait.

The differences between the average effects of alleles contribute to the additive genetic variance. This is demonstrated by equation (4) and illustrated in Figure 2 (which shows the additive genetic variance for the competition-dependent trait as a function of allele frequencies, again illustrated for the case in which r12=r13=r23). The additive genetic variance contributes to parent–offspring resemblance because alleles conferring large trait values in one generation also confer large values in subsequent generations due to temporal viscosity in allele frequencies (i.e., rare alleles in one generation are also rare in the following generation; see Figs. 1 and 3).

Figure 3.

Figure 3.

Evolutionary dynamics of allele frequencies due to directional selection on the competition-dependent trait for two patterns of competition between alleles. In both cases we show three neutrally stable cycles with arrowheads indicating the direction of allele frequency change. (A) the case in which the strength of competition is equal between all pairs of alleles, where r12=r13=r23, (B) the case in which the strength of competition between alleles is variable, with r12= 1.0; r13= 0.75; r23= 0.5.

Figure 3.

Figure 3.

Evolutionary dynamics of allele frequencies due to directional selection on the competition-dependent trait for two patterns of competition between alleles. In both cases we show three neutrally stable cycles with arrowheads indicating the direction of allele frequency change. (A) the case in which the strength of competition is equal between all pairs of alleles, where r12=r13=r23, (B) the case in which the strength of competition between alleles is variable, with r12= 1.0; r13= 0.75; r23= 0.5.

Maintenance of genetic variation

Despite assuming constant directional selection on the competition-dependent trait we find that heritable genetic variation persists indefinitely. Directional selection on competition-dependent traits is translated into selection on alleles affecting competitive ability through the genotype–phenotype relationship. Here, the genotype–phenotype relationship is frequency dependent because the effect of an allele on competitive success depends on the frequencies of the other alleles that it competes against (Fig. 1). This frequency-dependent genotype–phenotype relationship leads to evolutionary cycling of allele frequencies. These evolutionary dynamics are shown in Figure 3, which illustrates how allele frequencies change through time due solely to directional selection on the competition-dependent trait. Figure 3A shows the special case in which r12=r13=r23 and the cycling is symmetrical around the neutrally stable equilibrium point (which corresponds to the point where all three alleles are at equal frequency), which is analogous to the evolutionary dynamics of rock–scissors–paper (Hofbauer and Sigmund 1998). Figure 3B shows the case in which the relative competitiveness among alleles is asymmetrical (Fig. 3B corresponds to r12= 1; r13= 0.75; r23= 0.5), which results in asymmetric cycles. Indeed, for all values where all rij values are greater than zero, allele frequencies cycle indefinitely despite constant directional selection. Although populations are cycling in allele frequency space (Fig. 3), additive genetic variance is nonzero (Fig. 2), illustrating that the maintenance of allelic variation equates with the maintenance of heritable variation.

This allele frequency cycling is different from “classic” models of frequency dependence (e.g., Clarke 1979; Christiansen 1988; Asmussen and Basnayake 1990; Sinervo and Lively 1996) because we assume that selection on the trait is directional and independent of allele frequencies. Here frequency dependence arises as an emergent property of competition-dependent trait expression. For example, we find that the A1 allele has a positive effect on the competition-dependent trait when the A2 allele is common, a negative effect when the A3 allele is common, and a neutral effect when the A1 allele is common (Fig. 1). Frequency-dependent evolution arises from these effects due to the concerted coevolution of the effects of the three alleles with allele frequencies. For example, when the A1 allele is rare, selection favors the A2 allele because it outcompetes the A3 allele without losing to the rare A1 allele. Selection will then lead to the A2 allele becoming common, which in turn will favor the A1 allele, which outcompetes the A2 allele. Once the A1 allele becomes common selection then favors the A3 allele, which outcompetes the A1 allele and so on. As a consequence of these dynamics, it is generally the rare alleles that confer high competitive success and large values for competition-dependent traits whereas common alleles generally confer low competitive success and small trait value. Thus, the system generates a rare allele advantage for almost all allele frequency space, which is expected to actively maintain allelic variation, preventing fixation of alleles due to both selection and genetic drift (see below).


We show that additive genetic variation for a trait that reflects success in competition for limited resources persists indefinitely under conditions in which competitive success is nontransitive. Furthermore, this is true even under situations in which the relative competitive success between genotypes is variable. This “competition-dependent” genetic variation differs from “ordinary genetic variation” because loci involved in competition create the social environment experienced by conspecifics whereas, at the same time, the effects of alleles at these loci themselves depend on the current social environment provided by conspecifics. This reciprocal relationship between the effects of the alleles and the social environment that they create alters the nature of the genotype–phenotype relationship and, thereby, fundamentally alters the evolutionary dynamics of competition-dependent traits.

Nontransitive competition results in additive differences between the effects of alleles (i.e., different average effects of alleles; see Fig. 1) because the effect of an allele is determined by averaging across competitive social environments experienced by the allele, essentially “testing” the alleles against the current genetic background in a population. Consequently, alleles that do well against the current competitive genetic background in a population will have positive effects on the competition-dependent trait whereas alleles that compete poorly against the current background will have negative effects on the expression of such traits. Thus, the testing of alleles against the current social environment creates additive differences between the effects of alleles leading to heritable (additive) genetic variation.

The extent to which selection acts to remove linear components of dominance hierarchies (where a single allele wins), leaving nonlinear components, will determine how common such a scenario might be. There is evidence, both theoretical and empirical, the latter arising from research on competitive diallels (i.e., genotype-by-genotype competitive interactions), suggesting significant variation among genotypes to be nonadditive in the competitive environment (e.g., Pèrez-Tomè and Toro 1982; Asmussen and Basnayake 1990; De Miranda et al. 1991; Colegrave 1993; Bürger and Gimelfarb 2004). However, the degree to which such interactions are prevalent in natural populations remains an empirical problem and the generality of whether competitive hierarchies evolve to be transitive or nontransitive in nature is not well known. For example, there is experimental evidence that three-way genetic competitive interactions can be significant and complex (e.g., Castro et al. 1985; Hemmat and Eggleston 1989), and may exhibit negative frequency dependence (Adell et al. 1989). In contrast, there is a much larger body of literature examining competitive transitivity in interspecific interactions, which has demonstrated that such a pattern of nontransitivity may be relatively common, but it is obviously unclear the extent to which patterns from interspecific competition are mirrored at the intraspecific level. Thus, more data are required to ascertain the importance of complex genotypic interactions for determining the outcome of competition.

The reciprocal relationship between the effects of alleles and the social competitive environment leads to concerted coevolution of the two. This maintains variation because it leads to neutrally stable allele frequency cycles, where the orbit in allele frequency space is determined by the starting point (see Fig. 3). The evolutionary dynamics suggest that drift will randomly push allele frequencies into different orbits, but will not directly lead to loss of variation as a consequence. Indeed, the system is expected to buffer the loss of allelic variation by preventing the drift fixation of alleles through a process akin to negative-frequency dependence, where drift fixation is prevented because the cycles are deflected away from regions of allele frequency space in which a single allele would be fixed (see the stability analysis in the Appendix). However, because drift is a stochastic process, drift fixation is, of course, still a possible outcome, but its likelihood should be reduced due to this frequency-dependent process. Although the evolutionary dynamics should push the system away from the fixation of an allele due to drift in the three-allele system, selection is not expected to push the system away from the loss of an allele due to drift as the system orbits in allele-frequency space. In the three-allele space shown in Figure 3, this process is visualized as the population randomly drifting between orbits; cycling continues because selection pushes the system away from the corners (where drift fixation would occur). Loss of an allele due to drift will occur if, as the system randomly drifts between orbits, it collides with one of the axes. Once an allele is lost due to drift, we would be left with a two-allele stochastic system with one allele being favored over the other and the outcome being determined by the interplay of drift and selection.

The type of cyclical dynamics seen in the system we describe are rare in simple population genetic models and almost unknown in models with directional selection (Bürger 2000). Here, competitive social interactions give rise to such cyclical dynamics because of the coevolution of the effects of alleles with the social environment. That is, as allele frequencies change through time so does the average competitive environment experienced by genotypes and, as a result, they change the effects of alleles. Thus, the fitness of individual alleles changes gradually leading to slow cycling of allele frequencies and alleles favored by selection slowly change as the success of alleles in the contemporary social environment evolve. As a result, we find that the genotype underlying the largest (and fittest) phenotype changes through time due to evolution of the social environment, despite the fact that the mean of the competition-dependent trait remains constant. This evolutionary “treadmill of competition” (Dickerson 1955) maintains heritable variation of the competition-dependent trait, even under constant directional selection. This result is novel because most evolutionary models examining the genetic consequences of simple directional selection predict rapid loss of variation unless there is constant input of variation from mutation (see Bürger 2000). Here we find persistence of variation without mutational input and expect that the presence of mutational input would simply further enhance genetic variation in the system. This finding is robust in that, regardless of the relative competitive ability of each allele against the other two, the system will cycle indefinitely as long as the assumptions hold that resources are partitioned among competitors and that there is a nonlinear hierarchy for competitiveness, where no allele beats all other alleles in competition (i.e., competitive success is nontransitive).

Because allele frequencies change slowly, even under strong selection, allelic effects change only a small amount from one generation to the next. This means that fit individuals (e.g., with large trait values), which are competitively successful in the current competitive environment, will also have fit offspring because the competitive environment will be similar to that experienced by parents. This also implies that a genotype conferring high fitness in one generation may not necessarily do so in other generations or populations that are at very different allele frequencies.

Because competition can produce heritable variation it can play a role in processes for which such heritable trait variation is important, such as the lek paradox (see Andersson 1994; Tomkins et al. 2004). Condition dependence has been suggested as a means of generating heritable variation for traits that are the target of mate choice (Rowe and Houle 1996) and has emerged as the favored theory to resolve the lek paradox (Tomkins et al. 2004). However, models of condition dependence do not actually make the explicit prediction that condition-dependent trait expression maintains heritable variation, they simply predict that condition-dependence provides a large mutational target, thereby leading to a higher level of genetic variation at mutation–selection balance (Rowe and Houle 1996; Tomkins et al. 2004). Here, we suggest something fundamentally different in that, when condition-dependent traits rely on the outcome of competition, they will reflect genetic quality directly because they will reflect relative competitiveness of individuals and genetic variation for such traits will be maintained actively by a frequency-dependent process. Thus, mate choice decisions based on these traits will be adaptive yet will not lead to the rapid loss of genetic variation for either the traits themselves or for mate quality.

Despite the fact that competition can lead to real heritable variation contributing to parent–offspring resemblance, this component of heritable variation does not contribute to the evolutionary response to selection. That is, the mean of the competition-dependent trait is not frequency dependent, and therefore, never changes. As a result, the additive genetic variation associated with competition-dependent expression of a trait is not variation that contributes to realized heritability, as would be seen in the response a population shows to selection on such traits. Likewise, despite the presence of heritable variation that contributes to additive genetic differences in fitness, we find that population mean fitness is not frequency dependent and therefore does not change (see eq. 6). This phenomenon was alluded to by Fisher in his discussion of why mean fitness does not continue to increase when describing his “Fundamental Theorem of Natural Selection” (Fisher 1930). Fisher suggested that “As each organism increases in fitness, so will its enemies and competitors increase in fitness; and this will have the same effect, perhaps in a much more important degree, in impairing the environment, from the point of view of each organism concerned.” (Fisher 1930, pp 41–42). In other words, the competitive environment evolves in a way that offsets any potential increase in trait values and, as a result, the phenotypic population mean never changes. This has been referred to as the “treadmill environment” (Dickerson 1955) and is akin to an intraspecific Red Queen process (Lively 1996; Rice and Holland 1997) where the competitive environment evolves to offset any potential increases in the competition-dependent trait (see Wolf 2003). Thus, we find the surprising outcome that competition can lead to long-term maintenance of heritable variation for traits under constant directional selection, but those traits may never show an actual evolutionary response to selection. We suggest that competition dependence of fitness-related traits may be common in nature because of evidence for both genetic interactions for competitive success (e.g., Hemmat and Eggleston 1989; De Miranda et al. 1991; Colegrave 1993) and condition-dependent expression of traits (e.g., Bakker et al. 1999; David et al. 2000; Holzer et al. 2003). However, empirical investigation is required to confirm this.

Finally, it is interesting to note that the phenomenon of nonadditive interactions between genotypes is somewhat akin to epistasis (i.e., interactions between loci within genotypes) and has been referred to as “genotype-by-genotype” epistasis (Wade 2000; Wolf 2000) to reflect this similarity. In the former case, the social environment provides the genetic background that a locus experiences (which also means that a locus can interact with itself, because it both creates and interacts with the social environment) whereas with the latter it is other loci within the genome that provide the genetic background. However, despite their similarity, the two phenomena have important differences, because under directional selection epistasis alone maintains no additive genetic variation (e.g., see Gimelfarb 1989; Hermisson et al. 2003). In contrast, here we show that competition can maintain additive variation under directional selection, even in the absence of mutational input. The differences come about because with social competition the interactions occur between alleles in different individuals, rather than between alleles within individuals. As a result, the unit of selection (the individual genotype) does not contain the entire genotype determining individual fitness (because individual fitness is determined by multiple genotypes).

Associate Editor: R. Snook


We thank P. X. Kover, A. J. Moore, J. J. Mutic, N. J. Royle, and the Wolf lab group for insightful discussions that helped shape this work. This work was funded by grants from The Natural Environment Research Council (UK), The Biotechnology and Biological Sciences Research Council (UK), and The National Science Foundation (USA).



We begin by finding the internal equilibrium point. First we note that, there are no equilibria points (aside from the trivial equilibria) when there are just two alleles in the system because, in all two allele cases, one allele would be competitively dominant to the other allele, and therefore, one allele would move to fixation (this is also indicated by the linear stability analysis below). Therefore, the equilibrium point must occur when p1, p2, and p3 are greater than zero. From equation (7) we can see that, at equilibrium, r12p2=r13p3, r23p3=r12p1, and r13p1=r23p2. Therefore, p2= (r13/r23)p1, p3= (r12/r23)p1, and p1[ 1 + (r13/r23) + (r12/r23)]= 1 and we find that an equilibrium exists when






We denote these internal equilibria with an asterisk (i.e., the equilibrium frequencies are denoted p*i).

Linear stability analysis of equilibrium points

To examine the stability of the four equilibrium points we start by examining the stability of one of the trivial equilibrium points (when one allele is fixed), which we will then extend to examine the other two trivial equilibria. We start with the stability near the fixed equilibrium point where p1 is close to 1 (i.e., close to fixation of the A1 allele). We denote the frequencies of the other two alleles as p2=m and p3=n, where m and n are small such that the system is close to fixation of A1. The frequency of A1 is, therefore given as 1 −mn. Substituting these into the expressions for allele frequency change (eq. 7) and keeping only linear terms (because higher order terms are assumed small) we can express the change in allele frequencies as




which makes


This demonstrates the behavior of the system seen in Figure 3, where the system moves away from the area close to p1= 1 (or, more generally, away from the point where the system cycles closest to the corner where p1= 1) toward increasing values of p3 and decreasing values of p2. Similar analyses demonstrate that the system moves away from p2= 1 toward increasing values of p1 and decreasing values of p3 and away from p3= 1 toward increasing values of p2 and decreasing values of p1. In other words, the system is deflected from the areas close to the corners of the simplex as shown in Figure 3.

At the equilibrium point where all three alleles are present in the population the frequencies of the alleles are given by equation (A1). To examine the stability of that equilibrium point we consider the case in which the system is moved from the equilibrium in the direction where p1 decreases whereas the other two allele frequencies increase by small amounts, which we denote by m and n for the increase in frequencies of p2 and p3, respectively. This makes the three allele frequencies: p1=p*1mn, p2=p*2+m, and p3=p*3+n. Taking these frequencies, we can examine how the values of m and n will change near the equilibrium point, which can be written in matrix form as


The matrix in equation (A5) has purely imaginary eigenvalues that imply neutral stability.

The conserved quantity

Here we give a simple proof that the system has the conserved quantity, pr231pr132pr123, which we label C (see also Frean and Abraham 2001) To do so, we derive an expression for the simultaneous change in the three terms in the conserved quantity (pr231, pr132, and pr123) (i.e., an expression for the change in the conserved quantity through time) to show that this is zero. The change in C is a function of the changes in allele frequencies given in equation (7). Differentiating C with respect to time yields


which demonstrates that, as the system evolves, the conserved quantity does not change.