Abstract
 Top of page
 Abstract
 Introduction
 Theory
 Examples
 Discussion
 Acknowledgments
 References
 Appendix
An adaptive topography is derived for a large randomly mating diploid population under weak densityindependent selection in a fluctuating environment. Assuming a stationary distribution of environmental states with no temporal autocorrelation, a diffusion approximation for population size and allele frequency, p, reveals that the expected change in p involves the gradient with respect to p of the stochastic intrinsic rate of increase (the densityindependent longrun growth rate), , where r is the mean Malthusian fitness in the average environment and is the environmental variance in population growth rate. The expected relative fitness of a genotype is its Malthusian fitness in the average environment minus the covariance of its fitness with population growth rate. The influence of fitness correlation between genotypes is illustrated by an analysis of the Haldane–Jayakar model of fluctuating selection on a single diallelic locus, and on two loci with additive effects on a quantitative character.
Introduction
 Top of page
 Abstract
 Introduction
 Theory
 Examples
 Discussion
 Acknowledgments
 References
 Appendix
Wright's adaptive topography is one of the conceptual foundations of evolutionary biology (Wright, 1932, 1937, 1969; Simpson, 1953; Lande, 1976, 1979, 1982; Kauffman, 1993; Arnold et al., 2001; Gavrilets, 2004). Representing a population as a point on the surface of mean fitness as a function of allele frequencies at one or more genetic loci, assuming random mating, weak selection and loose linkage (promoting approximate Hardy–Weinberg equilibrium within loci and linkage equilibrium among loci), natural selection in a constant environment causes a population to evolve uphill on this adaptive topography, increasing the mean fitness in the population. The utility of this concept has been questioned, particularly in fluctuating environments when the population appears at best to be chasing a continually moving optimum (Fisher, 1958, pp. 44–45; Crow & Kimura, 1970, p. 236; Gillespie, 1991, p. 305).
The theory of population growth in a random environment was developed by demographers ignoring genetic variation. For a large densityindependent population subject to a stationary distribution of environmental states, the expected rate of increase in ln N, the natural logarithm of population size, is known as the longrun growth rate (Cohen, 1977, 1979). With small fluctuations in agespecific vital rates of reproduction and mortality, this can be approximated as , in which r is the population growth rate in the average environment, and is the environmental variance in population growth rate (Appendix 1). Under these conditions, all population trajectories eventually achieve the same slope, (Cohen, 1977, 1979; Tuljapurkar, 1982; Caswell, 2001; Lande et al., 2003).
Combining the above genetic and demographic theories, I previously demonstrated for allele frequency in a singlelocus haploid (asexual) population, p, and for the mean phenotype of quantitative characters in a sexual population, , that the expected evolution is governed by the gradient of the longrun growth rate of the population, , with respect to p or (Lande, 2007). Stochastic demographic theory describes densityindependent growth of an agestructured population, but the main theories of fluctuating selection (reviewed in Gillespie, 1991) do not incorporate age structure. For simplicity, the present theory describes a population without age structure undergoing densitydependent population growth, but assuming that selection does not depend on population density. Extensions to densitydependent selection and agestructure will appear elsewhere.
For diploid genetic systems in a randomly mating population, I demonstrate here that the longrun growth rate again provides an adaptive topography governing the expected evolution of allele frequencies. The evolution of such a population represented as a point on this surface is expected to produce a stochastic maximization of , which therefore constitutes an adaptive topography that does not change with time despite environmental fluctuations. However, the actual dynamics and stability of the process depend on the genetic system, the form of selection, and the amount of stochasticity in particular cases (Turelli, 1981; Gillespie, 1991; Lande, 2007). This is established by the theory developed below, and illustrated by particular examples including the Haldane–Jayakar model of fluctuating selection on a single diallelic locus, and on two loci with additive effects on a quantitative character.
Theory
 Top of page
 Abstract
 Introduction
 Theory
 Examples
 Discussion
 Acknowledgments
 References
 Appendix
Wright's adaptive topography assumes a large randomly mating diploid population in a constant environment, as well as weak selection, and for multiple loci approximate linkage equilibrium. In deterministic models, when the strength of epistatic selection is much less than the recombination rate, random mating quickly produces a nearly constant linkage disequilibrium (called quasilinkage equilibrium), after which gene frequency evolution increases the mean fitness (Kimura, 1965; Nagylaki, 1993). But under nonrandom mating, or with frequencydependent fitnesses, evolution generally does not maximize mean fitness (Wright, 1949; see also Fisher, 1941; Moran, 1964; Wright, 1967, 1988; Crow & Kimura, 1970, pp. 195–236).
Continual changes in environment cause both selection and population size to fluctuate. Theories of stochastic selection and stochastic demography can be unified by developing a genetic demography for populations in fluctuating environments. Consider a continuoustime model of a single genetic locus with two alleles, A_{0} and A_{1} at frequencies p_{0} = p and p_{1} = 1 − p, or multiple alleles, with 1 − p the total frequency of all alleles other than A_{0}. Random mating and weak selection are assumed so that the population is approximately in Hardy–Weinberg equilibrium. The ordered genotype A_{i}A_{j} therefore has frequency p_{i}p_{j}, with a Malthusian fitness at low population density that is symmetric, r_{ij} = r_{ji}. The Malthusian fitness of each genotype is assumed to be reduced by the same amount, −f(N), with increasing population density, N, so that population growth may be density dependent but selection is not.
Classical theories of deterministic population dynamics and evolution reveal that in a constant environment, under random mating and weak selection
 ( (1a))
 ( (1b))
where r = ∑_{i}∑_{j}p_{i}p_{j}r_{ij} is the mean fitness in the population and r_{i} = ∑_{j}p_{j}r_{ij} is the mean fitness of allele A_{i} (Crow & Kimura, 1970, chaps 1, 5.9). The final form of eqn (1b) is the continuoustime version of Wright's adaptive topography, demonstrating that allele frequency evolution in a constant environment always occurs in the direction of increasing mean fitness (Kimura, 1958; Crow & Kimura, 1970, chap. 5.9).
Instead of using the abundance of the ordered diploid genotype A_{i}A_{j}, denoted as N_{ij} = p_{i}p_{j}N, it is sufficient to obtain results from the dynamics of allele abundance, denoted for allele A_{i} as 2N_{i} where N_{i} = p_{i}N. Employing eqns (1a) and (1b), this obeys
Environmental stochasticity is introduced to this classical deterministic model by adding white noise to the Malthusian fitness of genotype A_{i}A_{j}, which then becomes
Assuming a stationary distribution of environmental states with no serial correlation, let E[dB_{ij}(t)] = 0 and E[dB_{ij}(t)dB_{kl}(t)] = c_{ijkl} dt (Karlin & Taylor, 1981, pp. 340–343, 347). The covariance is symmetric among diploid genotypes, c_{ijkl} = c_{klij}, and among alleles within genotypes, c_{ijkl} = c_{jikl} = c_{ijlk}.
The stochastic differential equation for N_{i} is then
 ( (2a))
Using the Ito stochastic calculus (Turelli, 1977), the corresponding diffusion process has infinitesimal mean and infinitesimal covariance
 ( (2b))
 ( (2c))
where N denotes the vector of allele abundances.
The method of analysis relies on the Ito transformation formulas (Karlin & Taylor, 1981; Gillespie, 1991, pp. 157–158) which can be applied to derive a coupled pair of diffusion processes for the natural logarithm of total population size, ln N = ln ∑_{i}N_{i}, and allele frequency, p_{i} = N_{i}/∑_{j}N_{j}. Infinitesimal moments of the transformed diffusion are calculated from those of the original diffusion for the N_{i} and the derivatives of ln N and p with respect to N_{i}. The resulting dynamical equations can be expressed in a compact and revealing form.
First, I describe the genetic components of the longrun growth rate and the environmental variance of the population, as these appear in all of the infinitesimal moments of the transformed diffusion process. The expected rate of increase in ln N at low population density is the longrun growth rate of the population, (eqns 3a and 4a), which has two components. The mean Malthusian fitness in the average environment, r is a quadratic function of p, which can be written using the symmetric matrix of Malthusian fitnesses r with elements r_{ij} and the column vector of allele frequencies, p^{T} = (p,1 − p), where the superscript ‘T’ denotes transposition (eqn 3b). The environmental variance in population growth rate, , is a quartic function of p, which, however, can also be written as a quadratic form using the matrix C with element C_{IJ} being the covariance between Malthusian fitnesses of unordered diploid genotypes I and J, and the column vector of Hardy–Weinberg frequencies for the three (unordered) diploid genotypes A_{0}A_{0}, A_{0}A_{1}, A_{1}A_{1} in proportions P^{T} = [p^{2},2p(1 − p),(1 − p)^{2}], indexed by the number of A_{1} alleles, I and J ∈ {0,1,2} (eqn 3c),
 ( (3a))
where
 ( (3b))
 ( (3c))
The infinitesimal mean of p (eqn 4c) obtained directly from the Ito transformation formulas is proportional to the difference in the expected relative fitnesses of the alleles, , as in the deterministic model (Crow & Kimura, 1970, chaps 1, 5.9),
 ( (5a))
where
 ( (5b))
The expected relative fitness of an allele, , is a weighted average of the expected relative fitnesses of diploid genotypes containing it, . Thus, the expected relative fitness of a diploid genotype is its Malthusian fitness in the average environment minus the covariance of its fitness with the population growth rate.
Differentiation of the longrun growth rate also produces the selection gradient acting on allele frequency, , which with eqn (5a) yields the gradient formula (4c) for the infinitesimal mean of p. This demonstrates that the expected evolution of allele frequency is in the direction of increasing as a function of p. With multiple alleles eqn (4c) applies to the frequency of each one, provided that the partial derivatives with respect to p are interpreted as holding constant the relative frequencies of all other alleles (Wright, 1969; Crow & Kimura, 1970). The longrun growth rate of the population therefore constitutes an adaptive topography for allele frequency. Remarkably, this adaptive topography, depicted as a function of allele frequency, does not change with time despite environmental fluctuations, assuming the latter have a stationary distribution.
Environmental fluctuations cause the expected relative fitness of a genotype to be frequency dependent (eqn 5b). Wright (1949, 1969) noted that with frequencydependent fitnesses, the mean fitness in a population generally is not maximized by evolution. This accords with the observation that under fluctuating selection, the expected evolution does not maximize the mean expected relative fitness, , but rather the longrun growth rate of the population (eqns 3a and 4a).
The infinitesimal variance of p (eqn 4d) involves a nonnegative quadratic form based on the covariance matrix of genotypic fitnesses, which can be identified as the variance of the selection coefficient acting on allele frequency. The infinitesimal mean and variance of p constitute an autonomous system, independent of N, because selection is assumed to be density independent. The dynamics of allele frequency are nevertheless coupled to those of the population size through the impact of genetic variation on the expected population dynamics (eqns 3a–3c, 4a and 4b) and through the infinitesimal covariance between ln N and p (eqn 4e).
With multiple loci, assuming epistatic selection much smaller than the recombination rate (Kimura, 1965), approximate linkage equilibrium will prevail and eqns (4a)–(4e) apply simultaneously for each locus, if the diploid genotypic fitnesses are averaged over the genetic background given the current allele frequencies at other loci (Wright, 1937, 1967, 1969). For example, with two diallelic loci, the genotype A_{i}A_{j}B_{k}B_{l} has Malthusian fitness in the average environment r_{ijkl} with i,j,k,l ∈ {0,1}. Letting p be the frequency of allele A_{0} and q be the frequency of allele B_{0}, we can construct the vectors of allele frequency and Hardy–Weinberg genotype frequencies for the second locus as for the first (above eqns 3a–3c), q^{T} = (q,1−q) and Q^{T} = [q^{2},2q(1−q), (1−q)^{2}], elements of which appear in the following formulas
 ( (6a))
 ( (6b))
Averaging across genotypes at the second locus eliminates the third and fourth subscripts in eqn (6a), whereas in eqn (6b) it eliminates the second and fourth subscripts. In the expected Malthusian fitness r_{ijkl}, the subscripts ij designate alleles at the first locus and subscripts kl represent alleles at the second locus; by contrast, C_{IKJL} is the covariance of fitness between twolocus diploid genotypes IK and JL. The indices I, J, K, L ∈ {0,1,2} count the number of A_{1} and B_{1} alleles in the genotype, e.g. C_{0112} is the covariance between genotypes A_{0}A_{0}B_{0}B_{1} and A_{0}A_{1}B_{1}B_{1}. The components of the longrun growth rate of the population, , are then
 ( (7a))
 ( (7b))
The present theory of fluctuating selection is incomplete for multiple alleles at a locus, and for multiple loci, in that the infinitesimal covariance of allele frequencies has not been specified, only the infinitesimal mean and variance of each allele frequency (eqns 4c and 4d). Describing a selection gradient for multiple alleles at a locus requires the application of nonlinear geometry (Akin, 1979; Shahshahani, 1979), which will be carried out elsewhere. The current results are nevertheless useful for analysing fluctuating selection in systems of one or more diallelic loci, as illustrated for two classical examples below.
Examples
 Top of page
 Abstract
 Introduction
 Theory
 Examples
 Discussion
 Acknowledgments
 References
 Appendix
The complexity of fluctuating selection on a diploid genetic system precludes a general analytic solution; so, results must be derived for particular cases (Turelli, 1981; Gillespie, 1991, p. 154). Models of fluctuating selection on a single genetic locus showed that with discrete nonoverlapping generations the geometric mean fitnesses of genotypes, or in continuous time their longrun growth rates, determine whether a rare allele will invade a population, regardless of the correlations in fitnesses among genotypes (Dempster, 1955; Haldane & Jayakar, 1963; Gillespie, 1973, 1991, p. 147; Tuljapurkar, 1982; Metz et al., 1992; Charlesworth, 1994; Caswell, 2001). At a diallelic locus, if the heterozygote has a higher longrun growth rate than either homozygote, then each allele invades when rare, and polymorphism persists. In the language of diffusion theory, if both 0 and 1 are entrance boundaries, then a stationary distribution of p exists (Karlin & Taylor, 1981, pp. 241–242).
However, the longrun growth rates of genotypes do not determine the expected change in allele frequency between successive generations, which defines natural selection and the expected relative fitnesses of genotypes in a polymorphic population. The present theory reveals that the expected relative fitness of a genotype within a population should be defined by its Malthusian fitness in the average environment minus the covariance between its fitness and the population growth rate (eqn 5b). Evolution in a fluctuating environment may therefore involve tradeoffs among genotypes in their means and variances (Haldane & Jayakar, 1963; Tuljapurkar, 1982), and also their correlations (Gillespie, 1973, 1991).
The behaviour of a stochastic process near its boundaries, although important, does not entirely describe the process. Thus, an invasion analysis using the longrun growth rates of rare genotypes can give a misleading or incomplete picture of the evolution in a polymorphic population. Furthermore, sample paths of diffusion processes tend to accumulate in regions of low stochasticity. The expected selection gradient, and the stochasticity of selection, as functions of allele frequency (eqns 4c and 4d), are both necessary to fully describe evolution in a fluctuating environment. To illustrate these points, I analyse some particular one and twolocus models.
Haldane–Jayakar model
Consider a symmetric version of the Haldane & Jayakar (1963) model of fluctuating selection at a diallelic locus. All three genotypes are assumed to have the same Malthusian fitness in the average environment, r_{0}. The heterozygote fitness remains constant, whereas the fitnesses of the two homozygote genotypes have variance c and correlation, ρ. A correlation of ρ = −1 between homozygote fitnesses describes random directional selection, whereas ρ = +1 describes random overdominance and underdominance. Previous analyses of the Haldane–Jayakar model were restricted to conditions for the maintenance of a polymorphism, rather than the form of a polymorphism that does persist (Haldane & Jayakar, 1963; Gillespie, 1991, p. 229; Nagylaki, 1992, pp. 65–71). Here, I derive the form of the stationary distribution of allele frequency for the Haldane–Jayakar model.
In this model, r = r_{0} is constant and the environmental variance is
 ( (8a))
The infinitesimal mean and variance of p, abbreviated as M_{p} and V_{p}, obtained from eqns (4c) and (4d), after rearrangement are
 ( (8b))
 ( (8c))
A stationary distribution of allele frequency exists for this model because the longrun growth rate of the heterozygote exceeds that of either homozygote, so that each allele invades when rare (see Table 1). The stationary distribution can be derived from Wright's formula,
where ‘const’ is a constant such that the distribution integrates to 1. Solution is facilitated by first transforming the diffusion process for allele frequency to the logit scale,
(see Appendix). Using the Ito transformation formulas, solving for the stationary distribution of y, and transforming this back to allele frequency gives
 ( (8d))
Table 1. Expected relative fitnesses and densityindependent longrun growth rates of genotypes in the Haldane–Jayakar model. Genotype  Expected relative fitness  Longrun growth rate 


A_{0} A_{0}  r_{0} − [p^{2} + ρ(1 − p)^{2}]c  r_{0} − c/2 
A_{0} A_{1}  r_{0}  r_{0} 
A_{1} A_{1}  r_{0} − [ρp^{2} + (1 − p)^{2}]c  r_{0} − c/2 
For ρ = −1, the stationary distribution of allele frequency is simply the uniform distribution, φ(p) = 1. This is a special case of Gillespie's stochastic additive scale–concave fitness function (SASCFF) model for additive effects between two alleles with equal variance and perfect negative correlation (Gillespie, 1991, p. 190). For higher correlations, the stationary distribution becomes more concentrated around p = 1/2, as depicted in Fig. 1. With a perfect positive correlation between homozygote fitnesses, ρ = +1, the stationary distribution is the Dirac delta function, δ(p − 1/2), an infinite spike with unit area at p = 1/2, which acts as an absorbing barrier eventually reached by all trajectories. The correlation between homozygotes thus has a profound influence on the dynamics of polymorphism in the Haldane–Jayakar model. This occurs mainly because a larger value of ρ decreases the infinitesimal variance at intermediate frequencies, which for ρ = +1 vanishes at p = 1/2 (see Fig. 1).
Because of its utility in determining the outcome of selection, such as whether a polymorphism can be maintained, many authors have interpreted the geometric mean fitness or longrun growth rate as a surrogate for genotypic fitness, or as fitness itself. In the Haldane–Jayakar model, where the longrun growth rate of the heterozygote exceeds that of either homozygote, this interpretation suggests that a net heterozygote advantage maintains the polymorphism. However, this is not necessarily correct, depending on the value of ρ. To understand how selection in a fluctuating environment operates one should instead examine the expected relative fitnesses of the genotypes obtained from the expected change in allele frequency (eqn 5b), as in Table 1.
Figure 2 illustrates that when ρ = −1, the expected relative fitnesses do not display a heterozygote advantage at any allele frequency, and that of the heterozygote lies exactly between the two homozygotes at every frequency, with the rarest homozygote always having the highest expected relative fitness. In this case, the polymorphism is maintained solely by frequencydependent selection favouring rare genotypes and the rare allele. A higher correlation between homozygotes produces a stronger tendency to maintain the polymorphism, due to a combination of heterozygote advantage and low stochasticity at some allele frequencies. When homozygote fitnesses vary independently, ρ = 0, the expected relative fitnesses display heterozygote advantage at every allele frequency except p = 0 or 1. When the homozygotes are positively correlated, for example ρ = +1, a heterozygote advantage is expected at every frequency.
Twolocus quantitative character model
An adaptive topography of fluctuating selection on quantitative characters, describing the expected evolution of the mean phenotype, was previously derived by Lande (2007), assuming normal distributions of phenotypes and breeding values with genetic variance maintained by mutation and recombination. Here, I apply a similar method at a more detailed level to investigate whether fluctuating directional selection, or fluctuating stabilizing and disruptive selection, can maintain genetic variance at multiple loci in a quantitative character.
Consider two diallelic loci with equal additive effects on a quantitative trait, assuming random mating and approximate Hardy–Weinberg equilibrium and linkage equilibrium. This model was often used by Wright (1935a,b, 1956, 1967, 1969) to analyse stabilizing selection in a constant environment, and is employed here to analyse fluctuating selection as shown in Table 2. All genotypes are assumed to have the same Malthusian fitness in the average environment; so, r = r_{0} is constant. At any time, the fitness function in Table 2 is a straight line with a random slope. This corresponds to the Haldane–Jayakar model of random directional selection (ρ = −1). The covariance between the fitnesses of any pair of genotypes is
 ( (9a))
Substituting this in eqns (7a) and (7b) gives the longrun growth rate of the population as a function of the frequencies of the A_{0} and B_{0} alleles at the two loci, denoted as p and q respectively (Fig. 3),
 ( (9b))
If, instead, the two halves of the fitness function, on either side of the intermediate phenotype, have opposite slopes, then in Table 2 the minus signs on the white noise terms for fitnesses of the two smallest phenotypes become plus signs. This describes random stabilizing and disruptive selection, analogous to the Haldane–Jayakar model of random overdominance and underdominance (ρ = +1). The covariance between the fitnesses of any pair of genotypes is then
 ( (10a))
which with eqns (7a) and (7b) gives the longrun growth rate of the population,
 ( (10b))
Inferences from the longrun growth rates of the genotypes can be contrasted with those gained from the adaptive topography, the longrun growth rate of the population as a function of allele frequencies.
Longrun growth rates of the genotypes show a pattern of stabilizing selection, being r_{0} for the intermediate phenotype including the double heterozygote and the two mixed double homozygotes (underlined in Table 2), r_{0} − c/2 for singly heterozygous phenotypes, and r_{0} − 2c for the extreme homozygotes (underlined in Table 2). Because the two mixed double homozygotes with intermediate phenotype have a longrun growth rate higher than that of any single heterozygote, they cannot be invaded by alternate alleles at either loci; they therefore constitute alternative stable equilibria with no polymorphism. Likewise, the two extreme double homozygotes can be invaded by alternative alleles at either locus, and so are unstable equilibria. This still allows the possibility of invasion from an edge equilibrium, where a rare allele at one locus invades, while the second locus is polymorphic. Even if one could determine the stability of all edge equilibria, which could be a daunting task in two or more dimensions, its relationship to the dynamics in the interior of the allele frequency space may be unclear. Longrun growth rates of the genotypes therefore have limited practical utility for analysis of multiplelocus genetic systems.
The adaptive topography for random directional selection (eqn 9a, Fig. 3) features a ridge connecting the two mixed homozygotes which can therefore be seen to be neutrally stable in two dimensions, not stable as indicated by invasion analysis for rare alleles at one locus at a time. Simultaneous changes in allele frequency at both loci along the ridge (p + q = 1) maintain the mean phenotype at the intermediate value and the longrun growth rate of the population also remains constant along the ridge. Stochasticity in selection will eventually drive both loci toward complementary quasifixations at one end of the ridge or the other.
Under random stabilizing and disruptive selection (eqn 9b, Fig. 3), the adaptive topography resembles that for stabilizing selection in a constant environment (Wright, 1935b, 1956, 1969, pp. 105–110). Two peaks representing alternative doubly homozygous genotypes, both corresponding to the optimum phenotype, are separated by a shallow saddle point. As with stabilizing selection in a constant environment, random stabilizing and disruptive selection thus acts to deplete additive genetic variance in a quantitative character.
The adaptive topographies in these symmetric twolocus models lack a peak in the interior of the allele frequency space, demonstrating in both cases that fluctuating selection on a quantitative character cannot maintain polygenic variation. Of course, if the expected selection were asymmetric, such that the genotype with the highest longrun growth rate was heterozygous at a single locus, then fluctuating selection could maintain polymorphism at one locus as in the Haldane–Jayakar model.
Discussion
 Top of page
 Abstract
 Introduction
 Theory
 Examples
 Discussion
 Acknowledgments
 References
 Appendix
The present theory reveals a general principle for the evolution of Mendelian populations in a fluctuating environment, extending Wright's adaptive topography for gene frequency evolution in a constant environment. Like Wright's theory, it assumes random mating, weak selection and loose linkage (to produce approximate Hardy–Weinberg and linkage equilibrium). Further, assuming a stationary distribution of environmental states, I have shown that the expected evolution of allele frequencies maximizes the longrun growth rate of a population, . Under these conditions, the surface as a function of allele frequencies remains constant in time, although selection continually fluctuates. Representing the population as a point on this adaptive topography, the expected rate and direction of allele frequency evolution is governed by the gradient of the longrun growth rate of the population with respect to allele frequency, (eqn 4c), and fluctuating selection causes stochastic perturbations around the expected evolutionary trajectory.
Longrun growth rates of the genotypes determine the invasion of rare alleles and the existence of a stationary distribution of allele frequency, that is, whether fluctuating selection can maintain a genetic polymorphism. However, the longrun growth rates of genotypes do not predict the expected change in allele frequency between generations in a polymorphic population, which is described by the expected relative fitnesses of the genotypes. The general diploid model agrees with previous haploid and continuous quantitative character models (Lande, 2007) in revealing that the expected relative fitness of a genotype is its Malthusian fitness in the average environment minus the covariance between its fitness and the population growth rate (see eqns 5a and 5b). Expected relative fitnesses of genotypes generally are frequency dependent, that is, functions of allele frequency. In the Haldane–Jayakar model of fluctuating directional selection (with ρ = −1), the expected selection favours rare genotypes and hence also the rare allele (Figs 1 and 2); this occurs because the distribution of selection coefficients is symmetrical with a mean of zero, and a rare allele will increase more in frequency under positive selection than it decreases under negative selection of the same magnitude. A similar stabilizing effect is expected from fluctuating directional selection on any genetic system, but this may not be sufficient to maintain a genetic polymorphism, depending on the mode of inheritance.
For a singlelocus haploid (or asexual) population, the genotype with the highest longrun growth rate will eventually prevail; so, fluctuating selection cannot maintain a polymorphism (Gillespie, 1991, chap. 4.2; Lande, 2007). For the Haldane–Jayakar model of a single diploid locus in a randomly mating population, fluctuating selection can maintain a polymorphism when the longrun growth rate of the heterozygote exceeds that of the homozygotes, allowing each allele to invade when rare. However, under fluctuating directional selection (with ρ = −1), the polymorphism is maintained purely by frequencydependent selection, as shown by the expected relative fitnesses of the genotypes, which do not display a heterozygote advantage at any allele frequency (Table 1, Fig. 2 top panel). The infinitesimal mean and variance of allele frequency, influenced by the fitness correlation between genotypes, together determine the form of the stationary distribution of allele frequency (eqns 8a–8d, Fig. 1).
Gillespie (1991) explains the fundamental difference between singlelocus haploid and diploid models in more technical terms. He developed the concept of chaploid models by which the dynamical properties of simple diploid systems can be approximated by multiplying the infinitesimal mean and variance of the classical haploid model, respectively, by c and c^{2}. For diploid models with additive fitnesses, where heterozygote fitness equals the mean fitness of the two corresponding homozygotes, this mapping is exact with c = 1/2 (Gillespie, 1991, p. 155). The ratio of the stochastic component to the deterministic component of allele frequency change is then smaller by a factor of 1/2 in the diploid model compared with the haploid model, allowing a stationary distribution of allele frequency to exist in the additive diploid model when it cannot in the haploid model.
Gillespie's SASCFF diploid and chaploid models constitute a special and somewhat peculiar mechanism of fluctuating selection. Motivated in part by classical quantitative genetics and partly by consideration of enzyme function, Gillespie's models postulate a constant functional relationship between phenotype and fitness, but a changing relationship between phenotype and genotype. In otherwords, SASCFF models produce fluctuating selection not by changing fitnesses directly, but through genotype–environment interaction (Gillespie, 1991, pp. 144–145, 155–156, 189–190). By contrast, the present theory and that of Lande (2007) generate fluctuating fitnesses directly by changing the fitness function acting on phenotypes, and for normally distributed polygenic characters a direct effect of environment on phenotype is incorporated as phenotypic plasticity without genotype–environment interaction. Thus, except for the Haldane–Jayakar model with ρ = −1, none of the present diploid models are equivalent to those of Gillespie.
For a quantitative character influenced by two loci with equal additive phenotypic effects subject to fluctuating directional or stabilizing and disruptive selection, polygenic variation cannot be maintained (Table 2, eqns 9a, 9b, 10a and 10b). This last example resembles deterministic stabilizing selection on a quantitative character in the classical models of Wright (1935b, 1956, 1969, pp. 105–110) and Fisher (1958, pp. 118–121), which does not maintain genetic polymorphism except perhaps at a single locus. Based on analogy with deterministic models of stabilizing selection, asymmetry of the genetic system (markedly unequal phenotypic effects among loci or displacement of the midrange phenotype from the expected optimum), or tight linkage between loci, may promote multilocus polymorphism but with limited additive genetic variance in the character (Gale & Kearsey, 1968; Barton, 1986; Bürger & Gimelfarb, 1999; Bürger, 2000, chap. VI; Willensdorfer & Bürger, 2003). In an agestructured population, strongly fluctuating selection can maintain polygenic variation but with a discontinuous phenotypic distribution (Sasaki & Ellner, 1997).
The theory and specific examples developed here for Mendelian genetic systems demonstrate the utility of the longrun growth rate of the population as an adaptive topography for the expected evolution of allele frequencies in a stochastic environment. The gradient of the longrun growth rate of the population with respect to allele frequencies indicates the expected direction of selection, and the expected relative fitnesses of the genotypes reveal the mechanism of frequencydependent selection acting on genetic polymorphism. Understanding the evolutionary dynamics of alleles at intermediate frequency requires simultaneous analysis of the expected selection gradient and the stochasticity in selection, incorporated, respectively, in the infinitesimal mean and variance of the diffusion process for p. The present succinct description of the process, and the methods provided for deriving the infinitesimal mean and variance of allele frequency change, facilitate this analysis.