Phenotypic variation within populations has two sources: genetic variation and environmental variation. Here, we investigate the coevolution of these two components under fluctuating selection. Our analysis is based on the lottery model in which genetic polymorphism can be maintained by negative frequency-dependent selection, whereas environmental variation can be favored due to bet-hedging. In our model, phenotypes are characterized by a quantitative trait under stabilizing selection with the optimal phenotype fluctuating in time. Genotypes are characterized by their phenotypic offspring distribution, which is assumed to be Gaussian with heritable variation for its mean and variance. Polymorphism in the mean corresponds to genetic variance while the width of the offspring distribution corresponds to environmental variance. We show that increased environmental variance is favored whenever fluctuations in the selective optima are sufficiently strong. Given the environmental variance has evolved to its optimum, genetic polymorphism can still emerge if the distribution of selective optima is sufficiently asymmetric or leptokurtic. Polymorphism evolves in a diagonal direction in trait space: one type becomes a canalized specialist for the more common ecological conditions and the other type a de-canalized bet-hedger thriving on the less-common conditions. All results are based on analytical approximations, complemented by individual-based simulations.

Explaining the amount of phenotypic variation observed in nature is one of the defining problems of evolutionary theory. The classification of ecological conditions that favor phenotypic variation is a primary issue in this debate. Another part is the question how much phenotypic variation can be attributed to genetic variation and how much to environmental variation. One mechanism that can result in increased levels of phenotypic variation is fluctuating selection, but the conditions for the evolution and maintenance of variation are different for the genetic and the environmental component.

Several models showed that genetic variation erodes under density- and frequency-independent fluctuating selection (e.g., Cohen 1966; Bull 1987; Seger and Brockmann 1987; Frank and Slatkin 1990). Under such conditions genetic variation can only be maintained by genetic constraints or recurrent mutation, similar to the case of constant selection. In particular, heterozygote advantage with respect to geometric mean fitness facilitates the maintenance of genetic variation (Gillespie 1973; Karlin and Liberman 1975). The effect of fluctuating selection on the amount of genetic variation maintained at mutation–selection balance depends on the nature of the fluctuations. Randomly fluctuating selection does not, or only slightly, increase genetic variation at mutation–selection balance (Lande 1977; Turelli 1988) whereas periodically changing selection can significantly increase genetic variation if the selection cycle is sufficiently long and the mutation rate is sufficiently high (Bürger and Gimelfarb 2002).

In an ecological model for species coexistence by Chesson and Warner (1981), which became known as the “lottery model,” it was pointed out by Seger and Brockmann (1987) and elaborated by Ellner and co-workers (Ellner and Hairston 1994; Sasaki and Ellner 1995; Ellner 1996; Ellner and Sasaki 1996; Sasaki and Ellner 1997) that fluctuating selection can lead to the maintenance of genetic polymorphism. The crucial feature of the lottery model with generation overlap is that fluctuating phenotype-dependent viability selection only acts on juvenile survival whereas long-lived adults (or a persistent dormant stage) are not affected by environmental fluctuations. Together these ingredients result in negative frequency dependence and allow for the coexistence of genotypes that are specialized on different sets of the stochastically occurring environmental conditions. Ellner and Hairston (1994) showed for this model that under certain conditions genetic dimorphisms are not only protected from extinction on the ecological time scale but also protected on the evolutionary time scale in the sense that no single phenotypically monomorphic genotype exists that could invade and replace both genotypes present in the dimorphism. Although Ellner and Hairston did not phrase it this way, they essentially proved the existence of an evolutionary branching point in the sense of Metz et al. (1996) and Geritz et al. (1998).

Environmental variation, the fact that a single genotype produces different phenotypes, is favored under fluctuating selection by two mechanisms. First, with adaptive phenotypic plasticity a single genotype can produce different phenotypes, such that the realized phenotype is adapted to the realized environment (e.g., Schlichting and Pigliucci 1998; West-Eberhard 2003). Adaptive phenotypic plasticity relies on the existence of cues that reliably predict the future selective environment, a sensory machinery that allows a developing organism to perceive the cue and a developmental switch, such that the appropriate phenotype can be produced. The second mechanism is known as bet-hedging (Slatkin 1974; Seger and Brockmann 1987; Philippi and Seger 1989). In unpredictably fluctuating environments a genotype can increase its long-term growth rate by simultaneously producing offspring with a range of different phenotypes because in this way a genotype increases the chance that at least some of its offspring will enjoy high reproductive success under future environmental conditions. In this article, we will be interested in the second mechanism but not in the first one.

Classical models of bet-hedging were developed to explain within-genotype variation in germination strategies (Cohen 1966; Brown and Venable 1986; Evans and Dennehy 2005), pupations date (Hopper 1999), and diapause length (Hopper 1999; Menu et al. 2000). More recently this idea received attention in the context of alternative phenotypes in bacteria (Kussel et al. 2005; King and Masel 2007; Malik and Smith 2008; Veening et al. 2008) and viruses (Stumpf et al. 2002). In all these models, the distribution of offspring phenotypes is assumed to vary in a discrete manner with few alternative realizations. And indeed, for the lottery model Sasaki and Ellner (1995) showed that a discrete distribution of offspring phenotypes is usually the optimal bet-hedging strategy, even if the distribution of environmental conditions experienced by the population is continuous. Just like in the case of phenotypic plasticity, however, discrete phenotype distributions require the existence of complicated developmental switches. Several recent studies report such switches in bacteria (reviewed in Dubnau and Losick 2006; Veening et al. 2008) where they might be more common than previously thought. In multicellular organisms, there are only few well-supported examples for discrete phenotypic distributions due to bet-hedging and these examples almost exclusively deal with occurrence of some type of resting stage (Hopper 1999; Evans and Dennehy 2005). It therefore seems that the distributions predicted by Sasaki and Ellner (1995), despite of their theoretical optimality, are usually difficult to realize. However, there is continuous variation in the expression of any quantitative trait and if environmental variance effects many steps in a developmental pathway the natural expectation is that it is of Gaussian shape. The Gaussian shape of environmental variance is also a standard assumption of quantitative genetics. For fluctuating selection, also such a continuous shape of environmental variation can increase long-term fitness due to bet-hedging, as has been pointed out by Bull (1987), Haccou and Iwasa (1995), and Simons and Johnston (1997).

The question about the relative importance of genetic and environmental variance is particularly interesting under conditions where these are favored simultaneously because this raises questions about their relative advantage and their interaction. A model that allows to investigate these questions is the above-mentioned lottery model with generation overlap (Chesson and Warner 1981; Warner and Chesson 1985). Both Seger and Brockmann (1987) and Ellner and Hairston (1994) showed for this model that a genetic dimorphism can always be invaded and replaced by a single bet-hedging genotype capable of producing the mixture of the phenotypes present in the genetic polymorphism. Sasaki and Ellner (1995) generalized these findings and showed that any genetic polymorphism can be invaded and replaced by a single genotype producing the optimal distribution of phenotypic variance. Thus, at least under the conditions of the lottery model, the optimal polymorphic bet-hedging genotype (if it exists) outperforms any protected polymorphism of phenotypically monomorphic genotypes. Leimar (2005) showed that in a certain sense also the converse is true: in the lottery model the strength of selection for a mutation that introduces bet-hedging by changing a phenotypically monomorphic genotype into a phenotypically dimorphic genotype always exceeds the strength of disruptive selection favoring a genetic polymorphism. Thus, in the presence of a complex optimal bet-hedger, genetic variance in the model cannot be maintained beyond mutation–selection balance. This leads us to the question treated in this paper: How do environmental and genetic variation evolve if we do not allow for elaborate genetic switches, that is, if phenotypic diversity can only evolve due to changes in the genetic composition of the population and due to changes in the width of the Gaussian phenotype distribution produced by a single genotype.

The structure of this article is as follows. In the next section, we introduce the lottery model and adapt it for our purpose. Specifically, we assume that at the phenotypic level individuals are characterized by a quantitative character that is under stabilizing selection with respect to an optimal value that fluctuates in time. At the genotypic level individuals are characterized by their Gaussian offspring distribution with heritable genetic variation for the mean and variance. An analytical treatment of the full mutation–selection dynamics with a large number of co-segregating alleles is not possible. To gain insights into the long-term evolutionary dynamics, we will take a double approach. First, we study the model in the adaptive dynamics framework, which relies on invasion analysis in the limit of rare mutations with small effect. In a second step, the analytical predictions and insights are tested and complemented by individual-based simulations of the full model. We analyze two versions of the model. First we take the perspective that the amount of environmental variance is not evolvable but results from a fixed constraint. Here we ask how a given level of environmental variance affects the existence of evolutionary branching points and thus the potential for the evolution of genetic polymorphism and increased genetic variance. We show that increased environmental variance changes the condition for evolutionary branching to become more stringent. In the second version of the model, we assume that both the mean and the variance of the offspring distribution are subject to heritable genetic variation and ask how the potential for genetic polymorphism is affected if the environmental variance evolves toward its optimal value. We show that genetic polymorphism can still evolve if the distribution of selective optima is sufficiently different from a Gaussian distribution, either because of skewness or because it is leptokurtic.

The Model


Our model closely follows the lottery model introduced by Chesson and Warner (1981). Consider a population of organisms with a two-stage life cycle: a short-lived “juvenile” stage and a long-lived “adult” stage (Fig. 1). Time is measured in discrete steps, where one time step (or season) corresponds to the duration of the juvenile stage, that is, the time for a newborn to reach its reproductive age. Census takes place just before reproduction when the population entirely consists of adults. We assume that the population dynamics has settled at a stable nontrivial equilibrium of inline image individuals. At this equilibrium, adults survive from one season to the next with probability γ and thus persist on average for 1/(1 −γ) seasons. We assume that all adults are identical with respect to survival, and that γ (at the equilibrium) is constant in time. There is thus no selection at the adult state. Each season, inline image newborns are recruited to the adult population. Recruitment takes place in two steps in analogy to models of soft selection (Levene 1953; Wallace 1975). First, phenotype-dependent viability selection determines the frequency of phenotypes in the offspring pool. Second, phenotype-independent density-dependence reduces the number of newborns in the offspring pool to k. (It is this “lottery” with k prizes that earned this model family its name.)

Figure 1.

Life-cycle graph. The adult stage can be entered in two ways: First, by adults surviving with probability γ from one time step to the next. Second, by newborns that first undergo a phase of phenotype-dependent selection where survival depends on the match between phenotype and ecological condition and second a phase of phenotype-independent density regulation.

All selection in the model is thus due to differential viability of juveniles. Newborn individuals are characterized by a quantitative trait z. In any given season, this trait is under Gaussian stabilizing selection according to


where 1/σ2s determines the strength of selection and θt denotes the trait value that maximizes juvenile survival in season t. Variation in the selective optimum θt among seasons represents the fluctuating environment in the model. We assume that the random process generating the selective optima is ergodic and converges toward a stationary distribution with probability density function inline image, where the vector inline image contains the necessary parameters to describe the distribution. Note that the requirements on ergodicity are not very restrictive and that our analytical results are independent of possible temporal autocorrelations in the occurrence of selective optima given that the autocorrelations decay sufficiently fast over time (Tuljapurkar 1990).

Our main results are independent of the exact distribution inline image. All that is needed are the first four central moments of inline image, or, more precisely, the mean μθ, variance σ2θ, skewness gθ1, and kurtosis gθ2. To illustrate our results we will use four example distributions inline image. (1) A Bernoulli distribution f(θ; p) with only two selective optima at θ1= 1 and θ2= 0, which occur with frequency p and 1 −p; (2) a Gaussian distribution f(θ;μθ, σ2θ) with mean μθ and variance σ2θ; (3) a Poisson distribution f(θ; λ) with rate parameter λ; and (4) a leptokurtic Laplace distribution f(θ; μθ, b) = exp (− |μθ−θt|/b)/2b, where large deviations from the mean are overrepresented relative to a Gaussian. The characteristics of the four distributions are summarized in Table 1.

Table 1.  Four example distributions f(θ; y) of the selective optimum and their mean (μθ), variance (σ2θ), skewness (gθ1θ33θ), and excess kurtosis (gθ2θ44θ− 3). Positive (negative) values of gθ1 indicate distributions with a long right (left) tail. The excess kurtosis is a measure of the peakedness of the distribution. Leptokurtic distributions with g > 0 have a sharper peak and longer, fatter tails than the normal distribution (e.g., Poisson or Laplace). Distributions with g < 0 are called platykurtic. They have a more rounded peak and shorter, thinner tails than the normal distribution (e.g., Bernoulli with p= 1/2). The critical generation overlap γcrit can be calculated from these characteristics using equation (9). Thumbnail image of


We assume that the quantitative trait z decomposes additively into a heritable (genetic) and a nonheritable (environmental) component,


where the genotypic value μz can take any value in the real numbers and the environmental component e follows a Gaussian distribution with mean zero and variance σ2z. Note that e does not depend on the external environment that determines θt, but rather expresses the developmental (or micro-environmental) noise that differs among individuals. The inverse of the width of the distribution, σ−1z, measures the faithfulness of the developmental program while exposed to this noise. It is thus a measure of environmental robustness or canalization, which is itself a quantitative trait under genetic control and can take any value in the positive real numbers. Each individual is thus characterized by a two-dimensional vector inline image and produces offspring whose phenotypes z are drawn from a Gaussian distribution with mean μz and variance σ2z. Although inline image does not characterize the phenotype of a single individual but the distribution of phenotypes produced by a single individual, we will in this article, refer to inline image as “trait vector” and to the space spanned by inline image and σz∈[0, ) as “trait space.” For most part of our work, we assume clonal inheritance where, except for mutations, genotypes inline image are faithfully inherited from a single parent to its offspring. Finally, both genotypic components are subject to mutation (see section Simulation Methods). Population-wide phenotypic variation Vz can be decomposed in the usual way,


where Vg and Ve denote the genetic and environmental variation, respectively. In our notation Vg= Var(μz) and Ve2z.

For each reproducing individual with genotype inline image, the expected number of juveniles surviving viability selection in season t is given by


where C is the number of juveniles before selection (fecundity). The expected contribution Yit of an individual with genotype inline image to the k juveniles that are recruited in each season is proportional to inline image. With n different genotypes in the population,


where Njt denotes the abundance of the jth genotype at time step t. The dynamics of genotype inline image is then given by


In the lottery model, fluctuations in the selective optimum in combination with overlapping generations lead to negative frequency-dependent selection that can promote the coexistence of several species or genotypes. This phenomenon was called “storage effect of generation overlap” by Chesson (1983) and Warner and Chesson (1985). Qualitatively, the negative frequency-dependent selection can be understood as follows: Individuals of a rare genotype have a relative advantage compared to if their own type was frequent in years favorable to them because the majority of the population is unfit and thus their offspring faces less competition. Conversely, individuals of a rare genotype have a relative disadvantage in years unfavorable to them compared to if their own type was frequent because most of the other individuals have many more surviving offspring. Without overlapping generations these two effects cancel over time. With overlapping generations the offspring of favorable years are “stored” in the persistent stage and this storage effect favors rare over common types. The resulting negative frequency dependence causes elevated long-term average growth rates of rare genotypes compared to common genotypes and therefore makes protected polymorphism possible.


In our presentation of the lottery model, we equated the short-lived life stage with juveniles that are recruited to the adult population, which represent the long-lived life stage. This is close to the original design of the model by Chesson and Warner (1981) and Warner and Chesson (1985). As pointed out by Ellner and Hairston (1994) and Sasaki and Ellner (1995), an alternative interpretation equates the short-lived stage with adult individuals that reproduce once at the end of a time step and then die. Its offspring can either immediately develop into a reproducing adult or persist with probability γ in a resting stage. Prime examples for this interpretation are annual plants with seed bank and Daphnia resting eggs. Note that in the context of life cycles with resting stage it is usually the variable duration of this stage due to γ > 0 that is considered a bet-hedging strategy. In contrast, here we consider bet-hedging as achieved by producing variable phenotypes in the trait under selection, as detailed in the next paragraph.

Two alternative interpretations also exist with respect to selection. Above we assumed that fluctuating stabilizing selection acts on a quantitative trait z that is expressed in the juveniles and affects their viability. In this case environmental variation affects the juvenile production of a single adult. However, as already noted by Warner and Chesson (1985), the model only requires that fluctuating selection affects seasonal recruitment (rather than adult viability). In particular, the mathematical model remains unaltered if we interpret z as an adult trait value determining its fecundity with an optimum that fluctuates from season to season. In this interpretation, environmental variation affects an adult trait: a single adult contributes adults to subsequent generations with a range of different phenotypes. We note that there is a slight difference in the way how mutation should be implemented in these two interpretations. In the case of juvenile viability, selection should act on the newly mutated genotypes (before density regulation) whereas in case of adult fertility selection, mutation will only affect selection in the next season (after density regulation). Because the latter mechanism is computationally somewhat simpler (i.e., only k recruits undergo potential mutation each season), we have used this variant in our simulation model. (The difference in the results should however be negligible.)

Under certain conditions the model presented here is also equivalent to a consumer-resource model (H. Svardal et al., unpubl. data). This equivalence holds if we interpret the traits μz and σz as the mean and variance of the Gaussian resource utilization function of a consumer, the parameter σ2s as the variance of a normally distributed continuous resource whose abundance is not affected by the presence of the consumer, and the value of the selective optimum θt as the mean of the resource distribution. Thus, in this interpretation fluctuating selection follows from a Gaussian resource spectrum that shifts in time.

We conclude this section with a remark on the population dynamics. Here, we merely stated that the dynamics of the number of recruits per season reaches a nontrivial fixed point. Many implementations of density dependence will do this job, ranging from competition for a limited amount of space, as assumed in the original formulation of the lottery model (Chesson and Warner 1981), the saturating yield model due to Levin et al. (1984) and used by Ellner and Hairston (1994) to logistic dynamics (Chesson 1984). Importantly, even the assumption of a fixed point is by no means decisive for the existence of the storage effect but rather a requirement for mathematical tractability. Chesson (1984) showed that the storage effect continues to exist if juvenile recruitment is described by Lotka–Volterra dynamics, indicating that qualitatively our results should also hold for a much larger family of population dynamical models.

Analytical Methods: Invasion Analysis

We are interested in two main aspects of the model: (1) the long-term evolution of enhanced levels of environmental variance, and (2) the potential for the evolution and maintenance of genetic variation due to balancing selective forces (i.e., even in the absence of recurrent mutation) in the presence of elevated levels of environmental variance. Both these aspects depend primarily on properties of the (variable) fitness landscape of the model. This fitness landscape can be explored effectively by assuming a simplified mutational process where rare mutations change the genotypic trait values from inline image to inline image. Following the adaptive dynamics approach (Dieckmann and Law 1996; Metz et al. 1996; Geritz et al. 1998), we will also assume that mutational effects inline image (which can occur in all direction of the two-dimensional trait space) are small. If we assume that the population is initially monomorphic for a “resident” genotype inline image, the evolutionary dynamics can then be determined by following a series of mutation-substitution events. The fundamental tool to predict this dynamics is the so-called invasion fitness inline image, which is defined as the logarithm of the expected long-term growth rate of an infinitesimally small mutant subpopulation with genotype inline image in the resident population (Metz et al. 1992; Metz 2008). The long-term growth rate of a population in a time-varying environment is given by the product of its seasonal growth rates, and the average growth rate corresponds to the geometric mean. Using that the geometric mean is turned into an arithmetic mean on the log scale, and that time averages can be replaced by averages over the distribution of selective optima (ensemble averages) under our ergodicity assumptions on the time series of the selective optimum θt, we can derive the invasion fitness inline image for our model from equation (2) as (Appendix A),


In sufficiently large populations, a mutant has a positive probability to invade if inline image and is doomed to extinction if inline image. Furthermore, if mutations have small phenotypic effect, successful invaders will go to fixation and replace the resident, unless the mutant (as new resident) can itself be invaded by the former resident (Dercole et al. 2002; Geritz et al. 2002; Geritz 2005; Dercole and Rinaldi 2008).

Points of special interest for the evolutionary dynamics as it results from a trait substitution sequence are trait vectors inline image where the selection gradient inline image with entries


equals zero, inline image, because they are equilibria of the monomorphic evolutionary dynamics. Such points are called “candidate ESSs” (e.g., Ellner and Hairston 1994), “potential ESSs” (e.g., Otto and Day 2007) or “evolutionarily singular points” (Metz et al. 1996; Geritz et al. 1998). Singular points can be classified according to two independent properties: “invadability” and “convergence stability” and we will now discuss these two properties in more detail.

As by definition at a singular point the selection gradient equals zero, the fitness landscape locally around a singular point is described by the Hessian matrix H=[hkl] of invasion fitness with entries


If the Hessian matrix is negative definite, then the singular point is a local maximum and therefore uninvadable by nearby mutants. Conversely, if the Hessian matrix is positive definite, then the singular point is a local minimum of the fitness landscape and can be invaded by all nearby mutants. Of particular importance to us will be the case where the Hessian matrix is indefinite. Then the singular point is a saddle point of the fitness landscape and can be invaded by some nearby mutants but not by others. Figure 2 shows an example of a contour plot of the fitness landscape where the Hessian matrix is indefinite. At a saddle point in a two-dimensional space, two directions exist in which the height of the fitness landscape does not change. These directions are given by the vectors (1, v1) and (1, v2) with (1, vi) H (1, vi)T= 0 and delimit regions in (μz, σz)-space of mutants with positive invasion fitness from regions of mutants with negative invasion fitness.

Figure 2.

Contour plot of the fitness landscape locally around a singular point (μ*z, σ*z) as it emerges if the resident type is located at the singular point. In this example the Hessian matrix of invasion fitness is indefinite and thus the fitness landscape is a saddle point. The coordinate axes give the difference between mutant genotypes (μ′z, σ′z) and the singular strategy located at the origin. Mutants in the gray region have positive invasion fitness and mutants in the white region have negative invasion fitness. The hatched line gives the direction of strongest disruptive selection, corresponding to the dominant eigenvector of the Hessian matrix. αd denotes the angle between the μz-axis and the dominant eigenvector and Δα denotes the angle of the sector with positive invasion fitness. The symmetry in the fitness landscape reflects the fact that the Hessian matrix is symmetric.

Convergence stability indicates whether a singular trait vector is an attractor of the gradual evolutionary dynamics (Eshel 1983; Abrams et al. 1993; Metz et al. 1996; Geritz et al. 1998). In a two-dimensional trait space convergence stability is determined by the Jacobian matrix J=[jkl] of the selection gradient with components


(Leimar 2009). A singular point is convergence stable independently of possible genetic correlations (“strongly convergence stable,”Leimar (2009)) if J is negative definite. If J is indefinite the point is stable if genetic variation satisfies certain conditions, and otherwise it is not. Finally, if J is positive definite then the point is an evolutionary repellor. A more rigorous account on this can be found in appendix B.

A singular point that is both convergence stable and uninvadable is called a “continuously stable strategy” or CSS (Eshel 1983). It is a final stop of evolution. Singular points that are convergence stable but invadable by nearby mutants are called “evolutionary branching points” (Metz et al. 1996; Geritz et al. 1998). Selection initially acts in the direction toward such points, but once the trait value of the population is sufficiently close to the singular point selection turns disruptive and favors an increase in phenotypic variance (Rueffler et al. 2006). In the case of clonal organisms this increase can be realized by the emergence of two independent lineages and it is this scenario that earned branching points their name.

Simulation Methods

Extensive individual-based simulations are performed for three reasons. First, all analytical results are tested with respect to their robustness if the assumptions of the adaptive dynamics approximation are violated. Second, the polymorphic evolutionary dynamics after evolutionary branching cannot be investigated analytically and are therefore explored by means of simulations. Third, simulations are employed to investigate evolution in sexually reproducing diploid populations. Simulations are performed using Matlab R2009b (Mathworks 2009). More information on the computational implementation is given in the Supporting information.

Mutations affect μz and σz separately and occur with per-capita probability u for each component. Each mutation adds an increment ±δ to the genotypic value. The mutational parameters u and δ vary between 10−1 and 10−3 and population size N varies between 1000 and 10, 000. The exact choice of parameters is given in the figure legends.

To investigate the evolutionary dynamics of diploid sexually reproducing populations, we use an infinite alleles model and assume that the traits μz and σz are each coded by one additive locus. Adults produce haploid juvenile offspring (gametes) through meiosis with recombination rate r∈[0, 0.5]. Selection and density regulation act on haploid juveniles. Random pair formation of two surviving haploids results in diploid adult individuals.

Results: Fixed Environmental Variance

In this section, we investigate the scenario that the environmental variance does not evolve. Thus, we consider σz as a fixed parameter. For this one-dimensional model we show in Appendix D that μ*zθ is a singular point. This singular point is always convergence stable irrespective of the value of σz. It is invadable if and only if


If condition (7) is fulfilled, μ*zθ is an evolutionary branching point. Here selection is disruptive allowing different alleles to coexist in a protected polymorphism. If the inequality sign in condition (7) is reversed, then μ*zθ is uninvadable and therefore a CSS. Condition (7) is a straightforward generalization of a result by Ellner and Hairston (1994) who showed that a polymorphism of phenotypically monomorphic genotypes (σz= 0) is evolutionary stable if γ > σ2s2θ. Thus, genetic diversification is favored by a large generation overlap and strong selection against phenotypes deviating from the selective optimum (small σ2s) relative to the variance in the distribution of selective optima. Increasing environmental variation σ2z reduces the potential for genetic diversification by effectively weakening within-time step stabilizing selection on a genotype in the expected way (e.g., Bürger 2000, p. 160).

We investigate the evolutionary dynamics once genetic variation has been build up at a branching point by means of individual-based simulations. Figure 3 shows representative runs for the case that f(θ;μθ, σ2θ) is a Gaussian distribution for different combinations of σs and σz. The simulations show that not only the branching condition but also the potential for secondary branching depends on the sum σ2s2z. In the terminology of competition models, increased environmental variation increases the effective niche width and thereby decreases the number of possibly coexisting phenotypic cluster (cf. Bolnick 2006). Qualitatively the same simulation results can be found if inline image is a Poisson distribution.

Figure 3.

Branching for a fixed amount of environmental variance for the case of normally distributed selective optima. Time in generations (where a generation lasts 1/(1 −γ) time steps) is plotted on the x-axis and black dots indicate the distribution of individuals with genotype μz (on the y-axis) in the population. (A) The evolutionary dynamics for σs= 0.1 and σz fixed to zero. The population branches into two temporally stable sub-populations that branch again into further unstable sub-populations. (B) Increasing environmental variance to σz= 0.2 lets the secondary branching disappear while the pattern of primary branching remains qualitatively similar. (C) A combination of σz= 0 (like in (A)) and weaker selection σ2s= 0.12+ 0.22 gives similar results as in (B). We conclude that the main effect of positive environmental variance is to weaken selection. The set of selective optima are Gaussian pseudo-random numbers drawn once and then applied to all of three runs to rule out stochastic differences other than due to drift. Parameters: γ= 0.5, N= 10, 000, u= 0.01, δ= 0.01.

Results: Evolvable Environmental Variance

For the two-dimensional model, where both the mean and the standard deviation of the phenotypic offspring distribution, μz and σz, can evolve, we find that two different singular strategies can exist (Appendix C),


where σ2θ > σ2s is required for inline image. Figure 4 shows the position of the singular points in the two-dimensional trait space. The following conditions for convergence stability and uninvadability can be derived using the criteria described above (see Appendix E for derivations):

Figure 4.

Location of singular points x*= (μ*z, σ*z) and direction of the monomorphic evolutionary dynamics as a function of σ2θ and σ2s. (A) For σ2θ < σ2s the only singular point is x*= (μθ, 0) which is an attractor of the evolutionary dynamics. (B) For σ2θ > σ2s two singular points exist: x*1= (μθ, 0) and inline image with x*1 being a saddle point and x*2 being an attractor of the monomorphic evolutionary dynamics.

  • • The singular point inline image, corresponding to a fully canalized genotype, is convergence stable and uninvadable, and thus a CSS, if σ2θ < σ2s. For σ2θ > σ2s, inline image is not strongly convergence stable, but a saddle point of the evolutionary dynamics. Then the invasion of mutants with increased σz-values is possible and the evolutionary dynamics approach inline image (Fig. 4).
  • • The singular point inline image is convergence stable whenever it exists, thus whenever σ2θ > σ2s. It is invadable and thus an evolutionary branching point if
    Then inline image it is a saddle point of the fitness landscape and allows for evolutionary branching in at least some directions of the trait space, and thus for the emergence and maintenance of genetic variation. The critical generation overlap γcrit can be calculated explicitly for any distribution inline image and is given for the four distributions introduced in Section The Model in Table 1. If condition (9) is not fulfilled, inline image is uninvadable and thus a CSS.

We note that the “optimal” environmental variance σ*2z as it is found at the convergence stable equilibrium coincides with the optimal environmental variation reported by Bull (1987) for a model that is similar to the lottery model with γ= 0. However, because there is no generation overlap in Bull's model, genetic variation cannot be maintained. From equation (9) several conclusions can be drawn.

  • 1Adaptive genetic diversification is favored by a large generation overlap γ.
  • 2Adaptive genetic diversification is favored by a small value of the critical generation overlap γcrit, and thus by large values of |gθ1| and gθ2, measuring the asymmetry and leptokurtosis of the distribution inline image of the optimal phenotype across seasons.
  • 3For a symmetric distribution (gθ1= 0), we obtain γcrit= 4/(4 +gθ2+ |gθ2|) and γcrit < 1 if and only if inline image is leptokurtic (gθ2 > 0). In particular, for the Gaussian distribution with gθ2= 0, condition (9) becomes γ > γcrit= 1 which can never be fulfilled.
  • 4For asymmetric distributions (gθ1≠ 0), however, γcrit < 1 regardless of the kurtosis. Then genetic diversification is possible if the generation overlap is sufficiently large.
  • 5In the limit γ= 1, we have γ≥γcrit and genetic diversification is favored regardless of the details of the distribution inline image.

These results allow for the following interpretation. Genetic polymorphism can evolve if the Gaussian distribution of phenotypes produced by a single genotype deviates from the distribution of selective optima, either because the latter is asymmetric (producing an excess of extreme conditions in one direction), or because it is leptokurtic (where extreme conditions in both directions are more frequent than under a normal distribution). The extent of this deviation determines the minimum amount of negative frequency dependence (and thus the minimal generation overlap) that is needed for the emergence of genetic diversity.

Let us compare condition (7) for uninvadability in the model with fixed variance σz with the newly derived condition (9). By inserting the singular value inline image into condition (7), we find γ > 1, which is never fulfilled. In other words, condition (7) suggests that at inline image evolutionary branching is never possible. This apparent contradiction is resolved by noting that in the one-dimensional model genetic diversification is restricted to the μz-direction of the two-dimensional trait space. Because inline image is a saddle point of the fitness landscape for γ > γcrit (rather than a minimum), branching is only possible in some, but not all, directions of the trait space. In particular, branching only in the μz-direction will never be possible. In terms of Figure 2, the gray area where mutants have positive invasion fitness never extends in the region horizontally to the right and left of inline image. Instead, evolutionary branching can occur in a compound direction, simultaneously changing the mean and variance of the offspring distribution. In such a dimorphism, one type becomes a canalized specialist for the more common ecological conditions and the other type takes the role of a de-canalized bet-hedger thriving on the less-common ecological conditions. In Appendix F, we describe in more detail how the shape of the fitness landscape at inline image depends on the details of the model given that condition (9) is fulfilled.


The analytical results suggest that genetic diversification evolves if γ > γcrit where γcrit is given by the right-hand side of inequality (9). In computer simulations we find that this prediction is highly accurate for small mutation rates and small mutational step sizes. However, branching occurs more easily if either of them is large (Fig. 5). Then it can occur already for γ < γcrit and at points in trait space that are far away from the singular point. Thus, our analytical results are conservative concerning the potential for the emergence of genetic diversity. The deviations result from the fact that adaptive dynamics is a local analysis, applying to monomorphic resident populations and small mutational steps. More detailed explanations for the deviations can be found in Appendix G.

Figure 5.

Condition for evolutionary branching at the singular point x*2 for the case of Bernoulli-distributed selective optima as a function of the frequency of selective optimum 1, p, and the generation overlap, γ. Based on the adaptive dynamics approximation (cf. Table 1) branching occurs for combinations of p and γ corresponding to points above the white hatched line. Gray scales indicate whether branching occurred in a simulation run within one million time steps for different values of the mutational step size δ (A) and the mutation rate u (B). In panel (A) black areas correspond to combinations of p and γ where branching occurs for δ= 0.005 and u= 0.01. Increasingly lighter shades of gray correspond to combinations of p and γ where branching occurs for δ= 0.0075, 0.01, 0.025, 0.05, respectively, but not for smaller values. In panel (B) black areas correspond to combinations of p and γ where branching occurs for u= 0.001 and δ= 0.01. Increasingly lighter shades of gray correspond to combination of p and γ where branching occurs for u= 0.005, 0.01, 0.05, 0.1, respectively, but not for smaller values. White areas correspond to combinations of p and γ where branching did not occur for any for the tested mutation parameters. The adaptive dynamics approximation predicts the onset of branching well for small mutational step size and low mutation rates. Larger mutational step sizes and higher mutation rates result in branching occurring for combinations of lower p- and γ-values than predicted by the adaptive dynamics approximation. Parameters: N= 10, 000, σs= 0.1.

Due to the stochastic occurrence of selective optima in the simulations, frequently one of the branches that emerged at an evolutionary branching point dies out. We therefore discuss the long-term evolution after branching in two steps, first, for those simulation runs where extinction did not occur and then for runs with extinction.

The endpoint of the polymorphic evolutionary dynamics depends on the distribution of selective optima inline image. For Bernoulli-distributed selective optima, the two subpopulations emerging at a branching point evolve the genotypes inline image and inline image, but do so via different routes (Fig. 6A–C). Branching occurs in the direction of the dominant eigenvector of the Hessian matrix of invasion fitness (cf. Fig. 6A). Thus, the branch that is closer to the more frequently occurring selective optimum evolves a phenotype that matches this selective optimum while simultaneously canalization increases, that is, environmental variance decreases. The branch that is closer to the rare selective optimum also evolves a phenotype that matches this selective optimum but simultaneously becomes a de-canalized bet-hedger by increasing its environmental variance. The transient increase in environmental variance creates a path from the singular point to the phenotype matching the less-frequent selective environment that bypasses the fitness valley that blocks the direct path.

Figure 6.

Clonal evolutionary dynamics before and after branching for Bernoulli-, Poisson-, and Laplace-distributed selective optima. Panels in the left column show the dynamics in (μz, σz)-space where time proceeds from lighter toward darker gray. Panels in the right column show identical simulation runs but with time plotted on the x-axis and values for μz and σz on the y-axis. The cross in the panels in the left column correspond to the singular point x*2. The simulations show that for the chosen parameters the adaptive dynamics approximation accurately predicts the location of this singular point. (A)–(C) With Bernoulli-distributed selective optima, the population splits into two discrete clusters, each of which specializes on one selective optimum and becomes fully canalized. (D)–(F) With a Poisson distribution and larger values of λ, typically only a single branching event takes place. In this case, both branches maintain positive levels of environmental variation σz. (G)–(I) With a Poisson distribution and intermediate values of λ, typically more then one branching event takes place. The resulting subpopulations evolve to become specialists for the most frequently occurring selective optima θt= 0, 1, …, except for the subpopulation characterized by the highest μz-value, which maintains a positive environmental variance σ2z. (J)–(L) With Laplace-distributed selective optima, branching occurs only in the component σz. The resulting subpopulations are thus arranged vertically above each other in trait space. Parameters: N= 10, 000, σs= 0.1, u= 0.001; (A)–(C) γ= 0.95, p= 0.8, δ= 0.005; (D)–(F) γ= 0.975, λ= 1, δ= 0.01; (G)–(I) γ= 0.95, λ= 1.5, δ= 0.01; (J)–(L) γ= 0.85, inline image, δ= 0.01.

Also with Poisson-distributed selective optima branching takes place in a diagonal direction (Fig. 6D,G) such that the branch that matches the more common selective optima, the one with the lower value of μz, decreases its environmental variance whereas the branch evolving higher values of μz increases its environmental variance. Following branching, two main types of long-term evolutionary dynamics are observed. First, for larger values of λ, resulting in more symmetric distributions, only one branching event occurs and the two emerging branches reach endpoints where they both maintain positive environmental variance (Fig. 6D–F). For smaller values of λ, resulting in more asymmetric distributions, more than one branching event can be observed (Fig. 6G–I). The subpopulations become completely canalized (σz= 0) and evolve phenotypes that match the most common environmental conditions (μz= 0, 1, 2, …). This is true for all subpopulations except for the subpopulation that is characterized by the largest μz-value, which maintains positive environmental variance (Fig. 6D–F).

For Laplace-distributed selective optima, branching occurs only in σz. Simulations with one or two branching events have been observed, with larger generation overlap favoring two branching events. In the case of a single branching event, dimorphic evolution proceeds to a coalition x1= (μ1, σ1) and x2= (μ2, σ2) with μ1θ2 and 0 < σ1 < σ*z < σ2 (Fig. 6J–L). In this polymorphism one subpopulation mainly exploits the dense region around the mean whereas the other can persist by also thriving on the substantial tails of the distribution of selective optima. In the case of two branching events, three subpopulations evolve, differing only in their level of positive environmental variation.

In many simulations, we observed that one of the subpopulations went extinct before the equilibrium described above could be reached (Fig. 7). This leads to a characteristic pattern of transient genetic dimorphism with recurrent branching and extinction. Extinction typically happens after a series of environmental conditions that is unfavorable for the individuals in one of the branches. Consequently, we observe that with negative autocorrelation in the sequence of selective optima extinction occurs less frequently. Almost always extinction affects the branch with μz-values matching the less-frequent environments. The reason is that this branch consists on average of fewer individuals and is therefore more susceptible to stochastic fluctuations. Extinction of branches becomes more likely with decreasing generation overlap γ, increasing skewness gθ1, stronger selection σ−1s and decreasing population size N. All these changes increase the stochastic component in the population dynamics. Among these factors, variation in γ has the strongest effect (cf. Fig. 7 where we show characteristic simulation runs for four different values of γ). Interestingly, while increasing skewness facilitates branching (cf. condition 9), such an increase has negative effects on the stability of the emerging branches in the face of stochasticity.

Figure 7.

Polymorphic evolutionary dynamics with extinction for the case of Bernoulli-distributed selective optima for different values of the generation overlap γ. For each value of γ, two panels are shown, one depicting the dynamics of μz (upper panel) and one depicting the dynamics of σz (lower panel). Time is plotted on the x-axis as number of generations, where one generation equals 1/(1 −γ) time steps. Hatched lines indicate the value of μz and σz corresponding to x*2. In all cases γ > γcrit such that x*2 is a branching point. However, the subpopulation evolving to the rare environmental condition frequently goes extinct resulting in characteristic patterns of transient genetic dimorphism with recurrent branching and extinction. (A) For γ= 0.6, the waiting time until branching is relatively long and emerging branches are short-lived. (B) Increasing the generation overlap γ shortens the waiting time until branching and branches on average diverge further before extinction takes place. Note that in both (A) and (B) after extinction the remaining branch evolves back to the singular point before re-branching takes place. Panel (C) illustrates how stochasticity affects the branching location. For γ= 0.8 the remaining branch only evolves back to the singular point in some cases while in other cases branching takes place while the remaining branch is still located at (μz, σz) = (0, 0), corresponding to a canalized specialist for the more common environment. (D) Increasing the generation overlap even further stabilizes the evolutionary dynamics and branches become very long-lived. Even larger generation overlap results in persistent sub-populations as shown in Figure 6. Parameter values: N= 10, 000, p= 0.8, σs= 0.1, u= 0.001, δ= 0.005.

Many organisms do not reproduce clonally, but sexually, and it is therefore important to evaluate to what extend our results carry over to diploid sexually reproducing populations. The dynamics of the traits μz, σz and of the genetic variance Vg is expected to depend strongly on the details of the genetic architecture. Thus, a full exploration of these dynamics in sexual populations is beyond the scope of this article. Here, we analyze the case that each trait is characterized by a single diploid additive locus with a continuum of alleles. As in the asexual case, allelic values can change through mutations of small effect in the individual-based simulations (cf. Section Simulation Methods). Populations evolve to the predicted singular point as long as mutation rates and effects are sufficiently small. If γ > γcrit, such that inline image is a branching point, an increase in genetic variation is observed also in the diploid case. Two qualitatively different outcomes can be distinguished and their appearance shows a clear correlation with the recombination rate. First, with high recombination rates the population often remains a single connected cloud in trait space and genetic variation increases only slightly. Second, with low recombination rates branching occurs more frequently and discrete diverging clusters appear (Fig. 8). This result can be explained in the following way. In diploid sexual populations interbreeding and recombination result in up to nine different trait combinations, of which not all lie in the sector with positive invasion fitness (cf. Fig. 2). At high recombination rates, individuals with maladapted trait combinations occur at high frequency and selection against them overrides the disruptive selection acting on individuals with favorable trait combinations. Disruptive selection acting on some trait combinations then results only in a slightly increased genetic variance at mutation–selection balance. In contrast, if the frequency of maladapted trait combinations is sufficiently low, more extreme phenotypes are favored, such that discrete diverging clusters in trait space can appear.

Figure 8.

Diploid evolutionary dynamics before and after branching for Bernoulli-distributed selective optima. Panel (A) shows the dynamics in (μz, σz)-space. The simulation is initialized at (μz, σz) = (0.5, 0) from where the dynamics proceed to the singular point x*2 (indicated by the “+”). The dynamics after branching is represented by three snapshots in time (squares: generation 21 × 103, triangles: 24.5 × 103, dots: 60 × 103). Panels (B) and (C) show the identical simulation run but with time plotted on the textit x-axis and values for μz and σz on the y-axis. Note that from generation 55 × 103 onwards genetic variation in σz is lost and that the evolutionary endpoint is identical to the one predicted by the clonal dynamics (cf. Fig. 6A–C). Panel (B) shows that also individuals heterozygous in μz disappear when the variation in σz is lost. This is due to the fact that diploid adults are formed after selection on the gametes and in each step only one of the specialist genotypes has a notable amount of offspring. Parameter values: recombination rate=0.25, N= 2000, p= 0.8, γ= 0.95, σs= 0.1, u= 0.001, δ= 0.01.

The genetic architecture of most quantitative traits is polygenic. What can we expect for such a more realistic genetic architecture? If μz and σz are determined by many loci of similar effect size, we expect that discrete clusters cannot evolve. However, two previous studies have shown that under frequency-dependent disruptive selection a genetic architecture can evolve where allelic effects are concentrated at few (or even single) loci (Kopp and Hermisson 2006; van Doorn and Dieckmann 2006). Based on these results, we suggest that the scenario investigated here might also capture the long-term behavior of traits with more complicated genetic architectures.


Assume that natural selection favors phenotypic diversity within a population. What, then, are the evolutionary mechanisms that underlie this variation? In particular, under which conditions is (part of) this variation heritable, that is, due to the maintenance of genetic variation? In this article, we investigate the coevolution of genetic and environmental variation under fluctuating selection. Our analysis is based on the lottery model (Chesson and Warner 1981; Warner and Chesson 1985), which is characterized by a life cycle with a short-lived and a long-lived stage. Only the short-lived stage is sensitive to fluctuating environmental conditions: the optimal phenotype maximizing recruitment to the long-lived stage varies according to some distribution. In contrast, individuals in the long-lived stage are not affected by the fluctuating selection pressure and experience a constant mortality rate. This two-stage life cycle is decisive because under these conditions both, environmental and genetic variation, can be selectively favored. Increased environmental variation can be favored because of bet-hedging: genotypes that produce a range of different phenotypes increase their chance that at least some of their offspring enjoy high reproductive success under each realized environmental condition (Bull 1987; Seger and Brockmann 1987). Genetic variation can be maintained because of the “storage effect” of the two-stage life cycle: Genotypes can survive adverse environments in the unaffected long-lived stage, and rare genotypes contribute disproportionally to the recruitment pool in generations with their favored environment. As a result selection becomes negatively frequency-dependent leading to the emergence and maintenance of genetic polymorphism (Chesson and Warner 1981; Ellner and Hairston 1994).

In nature, the key assumptions of the lottery model apply to a wide range of organisms. In many perennial plant and animal species, major selection components affect only a short juvenile stage, whereas long-lived adults are relatively insensitive. Alternatively, the short-lived stage can be interpreted as reproductively active individuals, which produce a long-lived persistent stage such as seeds or resting eggs (see the last part in Section Genetic Model for a discussion of different model interpretations). An organism with a probability γ to survive as an adult (or seed) from one generation to the next lives on average 1/(1 −γ) generations. Organisms surviving, for example, 10 reproductive periods, corresponding to γ= 0.9, are no rarity. Often cited examples include trees, fish, sessile marine organisms, animals with resting eggs, and plants with seed banks (Chesson and Warner 1981; Ellner and Hairston 1994; Sasaki and Ellner 1995; Warner and Chesson 1985).

Our model assumptions concerning the environmental fluctuations are almost completely general: the value of the selective trait optimum can be picked from an arbitrary discrete or continuous distribution, with or without autocorrelations between generations. The example distributions that we have chosen for a detailed study represent different plausible types of ecological variation: (1) The Bernoulli distribution with two discrete selective optima can be a useful approximation if the strength of selection depends on whether a particular event (epidemic disease, forest fire, thunderstorm) happens at all during the short-lived phase, rather than on quantitative details; (2) a Gaussian distribution can be expected whenever the selective optimum is determined by many independent contributions of small effect; (3) a Poisson distribution with rate parameter λ expresses the probability for a given number of events (such as predator attacks) occurring in a fixed period of time if these events are independent and occur with an average rate λ. Also the number of independent events in other specified intervals such as distance, area, or volume are Poisson-distributed (e.g., amount of rain drops on a square meter); (4) the last example is chosen more for theoretical reasons: the Laplace distribution allows for variation in the kurtosis without introducing skew.

Past research has shown that any genetic polymorphism in the lottery model can be invaded and displaced by a genotype producing an optimal distribution of environmental variation and that the phenotype distribution produced by this “optimal bet-hedger” is generically discrete rather than continuous (Seger and Brockmann 1987; Ellner and Hairston 1994; Sasaki and Ellner 1995). The punch line of this research is that under fluctuating selection genetic variation beyond mutation–selection balance can only be maintained if constraints exist that prevent a single genotype from producing environmental variation of the optimal kind. This leads to the central assumption of our model. Our starting point is the observation that relatively few examples of discrete bet-hedgers are known in nature, at least if compared with the prevalence of unstructured continuous environmental variation, which is ubiquitous across traits and taxa. We therefore constrain the environmental variation on the focal trait z to follow a Gaussian distribution—as it is assumed throughout in models of quantitative genetics. Bet-hedging can still evolve, but only by increasing the width σz of this distribution. Modeling the environmental variance as an evolvable trait in this way, relates our approach to the theory of environmental (de)canalization, where environmental variance results from incomplete buffering of the developmental system against the effects of environmental fluctuations (Gavrilets and Hastings 1994; Wagner et al. 1997; de Visser et al. 2003). We note that our theory also applies to more general shapes of continuous environmental variation, as long as this shape can be transformed into a Gaussian by an appropriate choice of scale. The distribution of the selective optima needs to be transformed accordingly. A shape of the environmental variation that closely matches the initial optimum distribution will therefore result in an optima distribution that is close to a Gaussian after the transformation. Whether genetic variation can evolve then depends on the mismatch between the transformed distributions.

Our main results are all based on analytical work and can be summarized as follows. Fluctuating selection in the lottery model will often favor a mixture of genetic variation and Gaussian bet-hedging, and both factors combine to produce the observed phenotypic diversity. Depending on the genetic and environmental details, we find: (1) If the environmental variance σ2z is fixed (no heritable variation), selection on the trait z turns disruptive, and genetic variance can evolve if the generation overlap γ exceeds a critical value γcrit. This critical overlap increases linearly with σ2z (cf. eq. 7). (2) If σz is free to evolve, it evolves under fluctuating selection to positive values if and only if σ2θ > σ2s, where σ2θ denotes the variance in the distribution of selective optima and σ−1s denotes the strength of Gaussian stabilizing selection within each season. The optimal width of the phenotype distribution equals inline image. (3) If the environmental variance is at its optimal level σ*2z, genetic diversification due to disruptive selection is still possible for a sufficiently large generation overlap if the environmental distribution of the selective optima is either asymmetric or leptokurtic (cf. eq. 9 for the precise condition). We find that genetic diversification occurs never purely in the direction of the mean trait value μz of a genotype, but generally along a diagonal direction in the two-dimensional (μz, σz)-trait space: a genotype with reduced σz that specializes on common environments coexists with a genotype with increased σz, exploiting a wide variety of rarer environments via bet-hedging.

Finally, we have used computer simulations to test the robustness of the analytical predictions and to study the long-term consequences of disruptive selection and genetic differentiation. With a sufficiently small mutation rate and mutational step size both the clonal and the diploid simulations confirm that populations evolve an optimal environmental variance σ*2z. However, in particular increasing mutational step size results in the evolution of genetic polymorphism under much wider conditions than predicted by the analytical theory (cf. Fig. 5) and at points in trait space far away from a branching point. (Thus, by choosing a rather small mutational step size in most simulations we somewhat constrain the emergence of genetic polymorphism relative to the emergence of environmental variance.) Our analysis of the long-term polymorphic evolution focuses on clonal reproduction, where distinct clusters in phenotype space can be easily be formed. Although a Bernoulli distribution with only two selective optima favors at most a single split into two phenotypic clusters, other types of environmental fluctuations (e.g., Poisson) can result in secondary or even higher order branching events. On the other hand, we frequently observe that phenotype clusters die out after a series of adverse environmental conditions, leading to a dynamics of recurrent branching and extinction events in phenotype space (cf. Fig. 7). The prediction of evolutionary equilibria is much more difficult for diploid sexual populations, where our study gives only a preliminary account. Here, the formation of heterozygotes and recombinants can prevent clustering and the long-term fate strongly depends on constraints on the genetic system. Indeed, because under disruptive selection intermediate phenotypes have lower fitness than homozygotes, any mechanism preventing the production of the intermediate heterozygotes will be selectively favored (Rueffler et al. 2006) and it has been shown that discrete phenotypic clusters can emerge, for example, through the evolution of assortative mating (e.g., Dieckmann and Doebeli 1999; Pennings et al. 2008), through the evolution of sexual dimorphism (Bolnick and Doebeli 2003; Van Dooren et al. 2004), or through the evolution of dominance modifiers (e.g., Van Dooren 1999; Peischl and Schneider 2010).

We can interpret these results in various ways. First, we confirm predictions by Bull (1987) that Gaussian environmental variance can act as a selectively favored bet-hedging strategy. As a consequence, elevated levels of environmental variance are predicted for populations experiencing large variation in the selective optima between reproductive periods. Second, in contrast to Sasaki and Ellner (1995), our results show that genetic variation can be maintained by fluctuating selection even in the face of evolving environmental variation. Genetic variation is favored by two main factors: (1) A large generation overlap γ due to the long-lived stage in the life cycle which strengthens frequency-dependent selection; (2) environmental fluctuations leading to a non-Gaussian distribution of the selective trait optimum with a strong asymmetry or large leptokurtosis. This second factor lends a crucial advantage to genetic variation over environmental variation, because the latter is constrained to a Gaussian shape and as such less flexible. We therefore predict that long-lived organisms facing environments with asymmetric or leptokurtic distribution should have higher heritabilities than short-lived organisms in more normally distributed environments. Third, stochasticity is expected to cause fluctuations in the amount of genetic variation. In particular, it can also lead to the temporary extinction of phenotype clusters. These fluctuations are expected to increase with decreasing generation overlap and population size, and with increasing strength of selection and increasing positive autocorrelation in the sequence of environmental optima and increasing skewness in their distribution. Fourth, if the distribution of selective optima is strongly asymmetric, genetic variation should be structured such that phenotypes matching more common environmental conditions on average show a higher degree of canalization than phenotypes matching less-common environmental conditions. We note that the phenomenon that disruptive selection typically exists in a compound direction in trait space (selecting simultaneously for variation in μz and σz) while selection is stabilizing in each separate component has also been described in a recent study by Ravigné et al. (2009) in a model studying the coevolution of habitat specialization and habitat choice. Even more recently, based on an analysis of Lotka–Volterra competition models, Doebeli and Ispolatov (2010) proposed as a general principle that the likelihood of disruptive selection increases with increasing dimension of the trait space. Our results support this proposal.

How can we interpret these results in the larger context of the adaptive maintenance of phenotypic variation? There are three ecologically important mechanisms under which phenotypic diversity is truly adaptive, that is, not due to genetic constraints (such as overdominance, where the optimal genotype cannot breed true) or due to mutation–selection balance. These are (1) negative frequency dependence, (2) temporally fluctuating selection, and (3) heterogenous selection in a spatially structured population. However, these mechanisms affect the genetic and the environmental component of phenotypic variation in different ways: Under negative frequency-dependent selection genetic and environmental variation are in principle equivalent. In contrast, under temporally fluctuating selection without frequency dependence only environmental variation can evolve, and under spatially heterogeneous selection without frequency dependence only genetic polymorphism can evolve. In the first case, environmental variation serves as a bet-hedging mechanism, whereas in the latter case genetic variation reflects local adaptation.

The lottery model combines two of these mechanisms: temporally variable selection and negative frequency dependence. The parameter γ for the generation overlap allows for a continuous transition between two extremes: purely temporal variation without frequency dependence for γ= 0, and pure negative frequency dependence for γ→ 1, where all individuals face the same distribution of environmental challenges for their offspring during their infinitely long life times, and temporal fluctuations are averaged out (cf Appendix A).

If both genetic and environmental variance are unconstrained, selection for them is equally strong for γ= 1. Any value of γ < 1 introduces a time-dependent selection component, leading to an advantage of the environmental variance component due to its bet-hedging capacity as indeed observed by Sasaki and Ellner (1995). With a Gaussian constraint on the environmental variance, however, a nontrivial trade-off results, as expressed by condition (9) for the maintenance of genetic polymorphisms. In particular, genetic variation is always favored due to this constraint in the limit γ→ 1 (unless the distribution of selective trait optima is exactly Gaussian, results not shown). We expect that such a trade-off is a generic feature that should be observed also beyond the limits of our particular model. Like the lottery model (Sasaki and Ellner 1995), also competition models typically lead to phenotypic clustering (Doebeli et al. 2007; Pigolotti et al. 2007), where the “optimal” phenotype distribution that emerges in the long-term equilibrium is discrete rather than continuous. It is this complex shape of the phenotype distribution and the limitations of nonheritable variation to produce this shape that lend the more flexible genetic component its crucial advantage.

Two previous studies compare the coevolution of environmental and genetic variation under frequency-dependent competition, thus in a scenario similar to γ= 1 in our model. The results from both studies are fully consistent with our interpretation. For a two-patch Levene model of soft selection, Leimar (2005) finds that the condition where a genetic polymorphism can arise is identical to the condition where a mutant producing two alternative phenotypes can invade a genetically monomorphic population with zero environmental variance. Moreover, also the strength of selection in both scenarios is identical, suggesting that in this model genetic and environmental variation are, in principle, equivalent. Importantly, Leimar imposes no constraints on the shape of either type of variation: reproduction is clonal with a continuum of genotypes, and the environmental variation directly comes in its optimal shape, with the two alternative phenotypes each favored in one of the patches. This is different in a second study by Zhang and Hill (2007). They investigate a classical competition model, where a quantitative trait under Gaussian stabilizing selection also mediates competition among individuals with similar phenotypes, resulting in frequency-dependent disruptive selection whenever competition outweighs the stabilizing component of selection. Like in the present study, environmental variance is constrained to a Gaussian shape with evolvable width. Zhang and Hill find that Gaussian environmental variation can only be adaptively maintained if the genotypic values are restricted to a narrow interval and, hence, the genetic variance is even more severely constrained than the environmental variance.

The third mechanism that can maintain adaptive phenotypic diversity, spatially heterogeneous selection, is not part of the lottery model. It is studied by Leimar (2005) who extends his analysis of the Levene model to restricted migration. Decreasing migration allows for a continuous transition between the pure negative frequency dependence and pure spatial structure. In accordance with the above expectation, he shows that decreasing migration progressively favors the genetic over the environmental variance component. We note that many scenarios in nature will contain elements of all three mechanisms. Selection frequently varies over space and time and soft selection with gene flow entails negative frequency dependence. The scope for the evolution and maintenance of the genetic and environmental variation components under these conditions, in the presence of constraints, remains a promising field for future studies.


The authors thank M. Doebeli, O. Leimar, and an anonymous reviewer for comments on the manuscript and gratefully acknowledge funding from the Vienna Science and Technology Fund (WWTF).


APPENDIX A: Invasion Fitness

Invasion fitness inline image is given by the logarithm of long-term per capita growth rate of an infinitesimal small mutant subpopulation with genotype inline image in a resident population with genotype inline image at population dynamical equilibrium (Metz et al. 1992; Metz 2008).

The growth rate of the mutant subpopulation Nt from one year to the next is given by


Inserting inline image gives


and invasion fitness can be calculated as


Using that time averages can be replaced by ensemble averages given that the random process generating the selective optimum is ergodic and converges toward a stationary distribution, inline image can be written as in equation (3).

Based on a Taylor approximation up to first order and in the limit γ→ 1 invasion fitness inline image, as written in equation (3), becomes


Thus, in this limit invasion is determined by the arithmetic mean of the year-to-year recruitment, sampled over an infinitely long life. As a result, in this limit the lottery model becomes equivalent to a Levene-type soft selection model where invasion fitness is determined by the arithmetic mean of the recruitment from all patches and where the per patch recruitment can be written as inline image, where θi is the optimal phenotype in the ith patch.

APPENDIX B: Two-dimensional convergence stability

The mathematical characterization of convergence stability is straightforward in one-dimensional trait spaces (Geritz et al. 1998), but for two-dimensional trait spaces the situation is more complicated because then the evolutionary dynamics is not only governed by the shape of the fitness landscape but also by the distribution of mutational effects. We follow Leimar (2009) and use the concept of “strong convergence stability.” This concept is based on the observation that in the limit of small mutational effects and large population size the path of the evolutionary dynamics in the two-dimensional trait space can be described by an equation of the form


where inline image is a positive number related to the production of new mutants and C=[cij] the mutational variance-covariance matrix (Dieckmann and Law 1996). The last factor inline image is the jth element of the selection gradient inline image with entries given by the right-hand side of equation (4). We note that this equation has the same form as the one derived for the change in the mean phenotype under the weak selection limit of quantitative genetics (Lande 1976; Iwasa et al. 1991).

A singular point inline image is then defined as strongly convergence stable if the singular point is asymptotically stable under the dynamics described by equation (B1) independent of the mutational variance–covariance matrix. Strong convergence stability is determined by the Jacobian matrix J=[jkl] of the selection gradient with components


Leimar (2009) showed that a singular point inline image is strongly convergence stable for any positive definite covariance matrix inline image, if the Jacobian matrix is negative definite. If the Jacobian matrix is positive definite, then inline image is not convergence stable but repelling for any positive definite covariance matrix inline image. In cases where the Jacobian matrix is indefinite the singular point is a saddle point of the evolutionary dynamics in the absence of covariance and also for most cases with covariance.

APPENDIX C: Location of Singular Points

The invasion fitness gradient (eq. 4) is given by


After exchanging integration and differentiation we get


Executing the differentiation and using that inline image evaluated at inline image equals 1, we find


Using that inline image, inline image, and inline image, the integrals can be calculated exactly. For inline image we find


Solving inline image for μz, we find that the singular trait value μ*z equals μθ. Using this result we get


Solving inline image for σz, we find the two singular strategies given in equation (8) with σ*z1= 0 and inline image.

APPENDIX D: Fixed Environmental Variance

For the version of our model where σz is a fixed parameter and where only μz can evolve (Section Results: Fixed Environmental Variance) we find analogously to the two-dimensional case above that μ*zθ is a unique singular point. This singular point is invadable if and only if (Geritz et al. 1998)


Using the same ideas as in Appendix C, this condition can shown to equal


Solving the inequality for γ results in condition (7). The singular point is convergence stable if and only if (Geritz et al. 1998)


It can be shown that this condition equals


which is always fulfilled.

APPENDIX E: Evolvable Environmental Variance

Invadibility of singular points in a trait space with more than one dimension is determined by the Hessian matrix H=[hkl] of invasion fitness with entries given by equation (5). After exchanging differentiation and integration, hkl can be written as


We denote the second derivative in the expressions hkl as inline image. These can be explicitly calculated as


Using the definitions for the third and fourth central moment of probability distributions, that is, inline image and inline image, the integrations can be computed to give


A two-dimensional matrix A=[akl] is negative definite if and only if


that is, if the principal minors of A are positive. Inserting the equalities (E2) into inequality (E3) reveals the invadibility conditions reported in Section Results: Evolvable Environmental Variance.

Leimar (2009) showed that strong convergence stability of singular points is determined by definiteness of the Jacobian matrix J=[jkl] of invasion fitness with entries given by equation (B2). Leimar defines the Jacobian matrix as negative definite/positive definite/indefinite if its symmetric part (with elements (jkl+jlk)/2) is negative definite/positive definite/indefinite. For our model the Jacobian matrix can be shown to be symmetric, such that its definiteness can be inferred directly.

The first term on the right-hand side of equation (B2) equals the entries hkl of the Hessian matrix. The second term on the right-hand side, which we will denote qkl, after exchanging integration and differentiation equals


We denote the second derivative in the expressions qkl as inline image. These can be explicitly calculated as


where inline image and inline image again denote the singular strategies (μz, σz) = (μθ, 0) and inline image, respectively. The entries of the Jacobian matrix of the selection gradient jij=hij+qij calculate to


Inserting the equalities (E5) into inequality (E3) reveals the conditions for strong convergence stability reported in the result-section on evolvable environmental variance.

APPENDIX F: Fitness Landscape at Branching Points

In this appendix, we describe the fitness landscape at the singular point inline image in more detail. Here we are interested in the case γcrit > γ where the point inline image is a saddle point of the fitness landscape created by a population characterized by this trait vector. We analyze two aspects of these saddle points, the angle of the sector with positive invasion fitness, Δα, and the angle between the dominant eigenvector and the μz-axis, αd (cf. Fig. 2). Figure F1 shows for Bernoulli and Poisson-distributed selective optima how these properties depend on the generation overlap γ and the parameters p and λ, respectively.

Figure F1.

Characteristics of the fitness landscape at the singular point x*2 as a function of the generation overlap γ and a shape parameter of the distribution of selective optima. (A) & (B) Bernoulli distribution with parameter p; (C) & (D) Poisson distribution with parameter λ. (A) & (C) Angle of the sector with positive invasion fitness, Δα. (B) & (D) Angle between the dominant eigenvector and the μz-axis. In case of the Bernoulli distribution results are only shown for 0.5 < p < 1. Graphs for 0 < p < 0.5 are identical if the p-axis decreases from 0.5 toward 0. In case of the Poisson distribution, for better comparison with the results for the Bernoulli distribution the axes for λ is reversed such that asymmetry increases further along the λ-axis. Plots are shown for σs= 0.1.

Figure F1A,C shows that for both distributions Δα increases with increasing asymmetry (increasing deviation of p from 0.5 in case of the Bernoulli distribution and decreasing λ in case of the Poisson distribution) and with increasing generation overlap. For extreme asymmetry Δα goes back to zero because once p(1 −p) < σ2s or λ < σ2s, respectively, the singular point inline image ceases to exist. Branching is not possible for γ < γcrit, corresponding to a sector with width zero (flat part in Fig. F1A,C). As expected, the range of γ-values where branching is possible increases with increasing asymmetry.

The angle between the dominant eigenvector and the μz-axis, αd, strongly depends on the parameter p and λ, respectively. As noted in the section Results: Evolvable Environmental Variance, selection at inline image is never disruptive in the horizontal direction where only μz is varied. Instead, selection is disruptive in a compound direction, simultaneously changing the mean and variance of the offspring distribution. Figure F1 B,D show that for asymmetric distributions of selective optima the dominant eigenvector is tilted such that the type that is becoming a specialist for the environments in the fatter tail of the distribution is also decreasing its environmental variance. Conversely, the type that improves on the environments in the thinner tail increases its environmental variance. The tilt of the eigenvector increases with increasing asymmetry of the distribution. This can best be understood by considering the case of Bernoulli-distributed selective optima. With increasing (decreasing) p the selective optimum 1 occurs with increasing (decreasing) frequency and μθ=μ*z2 decreases (increases). Thus, μ*z2 is shifted toward the more common selective optimum. As a consequence, at inline image selection for a genotype adapted to selective optimum 1 corresponds to a increasingly steep dominant eigenvector.

For the Laplace distribution the situation is somewhat different. From equation (E2) we can directly deduce that the dominant eigenvector at inline image equals (0, 1). First note that h11 < 0 and h12= 0. From gθ2θ44θ4− 3 = 3 follows μθ4= 6σ4θ. By inserting this into h22 we see that h22 > 0 ⇔γ > 2/5 which is exactly the condition for inline image to be a branching point. This can lead to a genetic polymorphism purely in the environmental variance σ2z.

APPENDIX G: Accuracy of the analytical predictions

Figure 5 shows the accuracy of the analytical prediction for evolutionary branching, as given by equation (9), when compared to individual-based simulations for different values of the mutational step size and the mutation rate. We find that if both are relatively small, then γcrit predicts the onset of genetic diversification with high accuracy. Substantially increasing the mutational effect size (Fig. 5A) or the mutation rate (Fig. 5B) facilitates genetic diversification by lowering the critical generation overlap where a population starts splitting into two. These findings can be explained as follows. Uninvadability of a singular point as determined by the Hessian matrix is a local result that need not hold true for mutants at some distance. Numerical calculations show that in some directions in trait space an uninvadable singular point is indeed only separated by a narrow fitness valley from regions where mutants have positive invasion fitness. Simulations show that such invaders coexist with the resident type at the singular point and that subsequently these two types evolve further apart. Both increased mutational step size and increased mutation rate make the appearance of mutants on the other side of the fitness valley surrounding an uninvadable singular point more likely. Larger mutational steps allow to cross the fitness valley with a single mutational step whereas a higher mutation rate results in larger genetic variance around the singular point at mutation–selection balance. The waiting time until branching increases with decreasing values of the mutation rate, mutational step size, population size and generation overlap γ. Each of these factors decreases the amount of genetic variation in the neighborhood of a singular point, resulting in smaller fitness differences and therefore slower evolutionary dynamics. For population size this phenomenon that has been noted previously (Claessen et al. 2007, 2008), and for γ this phenomenon is illustrated in panels (A)–(C) of Figure 7.

Simulations show that for small values of the mutation parameters not only the onset of evolutionary branching is predicted relatively accurately by the analytical results but also the location in (μz, σz)-space where branching occurs (Fig. 6A,D,G, and J). However, as with the critical generation overlap allowing for branching, the accuracy of this prediction is sensitive to the mutation rate and the mutational effect size. If the mutation rate and in particular the mutational effect size are sufficiently high (but in a still biologically realistic range), branching can be observed at points in trait space far away from the singular point inline image. This phenomenon can be explained in the following way. In some regions of trait space, selection is directional in one direction and disruptive in another one. Locally, the first-order directional selection exceeds the second-order disruptive selection. However, with mutants sufficiently far away from the resident type, second-order terms will dominate the first-order terms, and the effect of disruptive selection surpasses the effect of directional selection. This is for instance the case for the line segment connecting inline image and inline image. For simulations that are initialized on this line segment we frequently observe that branching occurs without the population evolving to inline image (no figure) for a wide range of mutation parameters. A similar situation occurs for example with Bernoulli-distributed environmental optima if one of the optima occurs far more often than the other. Under this condition, a population that is initially characterized by (μz, σz) = (θi, 0), with θi being the more frequent selective optimum, experiences directional selection in μz toward μθ and disruptive selection in σz. In this situation branching is frequently observed immediately without prior evolution toward the singular point as illustrated in Figure 7C,D. Last but not least, whenever parameters are such that no branching occurs, the evolutionary dynamics settle at the point inline image as predicted by the analytical theory.