Intraindividual variability in behavior shapes fitness landscapes

Abstract Although intraindividual variability (IIV) in behavior is fundamental to ecological dynamics, the factors that contribute to the expression of IIV are poorly understood. Using an individual‐based model, this study examined the effects of stochasticity on the evolution of IIV represented by the residual variability of behavior. The model describes a population of prey with nonoverlapping generations, in which prey take refuge upon encountering a predator. The strategy of a prey is characterized by the mean and IIV (i.e., standard deviation) of hiding duration. Prey with no IIV will spend the same duration hiding in a refuge at each predator encounter, while prey with IIV will have variable hiding durations among encounters. For the sources of stochasticity, within‐generation stochasticity (represented by random predator encounters) and between‐generation stochasticity (represented by random resource availability) were considered. Analysis of the model indicates that individuals with high levels of IIV are maintained in a population in the presence of between‐generation stochasticity even though the optimal strategy in each generation is a strategy with no IIV, regardless of the presence or absence of within‐generation stochasticity. This contradictory pattern emerges because the mean behavioral trait and IIV do not independently influence fitness (e.g., the sign of the selection gradient with respect to IIV depends on the mean trait). Consequently, even when evolution eventually leads toward a strategy with no IIV (i.e., the optimal strategy), greater IIV may be transiently selected. Between‐generation stochasticity consistently imposes such transient selection and maintain high levels of IIV in a population.


| 2839
OKUYAMA IIV is therefore important, even when IIV is not the main focus of interest.
Patterns observed in the expressions of IIV suggest that IIV has important fitness consequences. For example, some individuals consistently express greater IIV than others (Biro & Adriaenssens, 2013;Briffa, 2013;He, Pagani-Nunez, Chevallier, & Barnett, 2017;Highcock & Carter, 2014;Stamps et al., 2012). The expression of IIV may not be constant within an individual (e.g., an individual may adjust the level of IIV across contexts) (Jolles, Briggs, Araya-Ajoy, & Boogert, 2019;Mathot & Dingemanse, 2014), which implies IIV can be expressed as behavioral plasticity. When IIV is a heritable trait (Henriksen, 2019), these observed patterns are results of natural selection on IIV. However, optimal behavior models typically consider only the mean expression of behavior (Charnov, 1976;Cooper & Frederick, 2007), and relationships between behavioral IIV and fitness are largely unknown.
Stochasticity is a possible factor that influences the relationship between IIV and fitness. One source of stochasticity is within-generation stochasticity that is the stochasticity realized within a generation (e.g., within the life span of individuals). Such within-generation stochasticity and behavioral stochasticity (IIV) may interact and influence fitness. Another source of stochasticity is between-generation stochasticity. A good behavioral strategy in the current generation may be a poor strategy in the offspring's generation when environmental conditions change over generations (i.e., the optimal strategy depends on environmental conditions). Previous studies examined how environmental changes influence the evolution of adaptive traits and subsequent population dynamics (Abrams, 2001(Abrams, , 2003Abrams & Matsuda, 1997), but IIV was not considered in them.
The study used an individual-based model to examine the effects of the two types of stochasticity on IIV. For the within-generation and between-generation stochasticity, predator encounters and resource availability, respectively, were considered. The behavior considered is a hiding behavior of prey. Various prey species hide in a refuge such as a burrow or shell when they encounter predators (Everett & Ruiz, 1993;Kramer & Bonenfant, 1997;Martin, Lopez, & Cooper, 2003;Mima, Wada, & Goshima, 2003). When a prey experiences multiple predator encounters, the duration of hiding may vary with each encounter, which is an expression of IIV.

| THE MODEL
The model discussed in this study is similar to exiting prey refuge models (Cooper & Frederick, 2007;Martin & Lopez, 1999) in which the optimal strategy is derived by balancing the cost and benefit of hiding (described below). The model is a discrete generation individual-based model in which prey can live for a generation, and all the individuals in the following generation are the offspring of the current generation individuals. Reproductions take place at the end of each generation by prey that survived till that time. The model assumes IIV is a heritable trait. Because successful individuals will reproduce more offspring that share the traits of successful individuals, simulations will impose natural selection on the behavior. The effects of natural selection on both the mean and variability (IIV) of the behavior can be examined by the distributions of behavioral traits over generations.

| Intraindividual variation
The individual-based model describes a situation in which prey survive predator encounters and are subsequently able to reproduce.

| Survival and reproduction
Hiding has both benefits and costs. As the duration of hiding increases, the probability of survival after an encounter with a predator increases. However, hiding reduces the time available for other activities, such as forging for food or other resources. For a prey individual which has remained in a refuge for a duration t, its survival probability s(t) is described by.
in which a and b are the parameters that determine the relationship.
For example, b > 0 indicates that the longer a prey remains in a refuge, the greater the possibility of survival.
To reproduce, the prey must survive until the end of the season by successfully escaping p predators. For example, if a prey experiences three predator encounters (p = 3), it must survive all three encounters. The total hiding duration t tot of the prey is the sum of the three hiding durations, such that t tot = t 1 + t 2 + t 3 where t i is the hiding duration from ith predator encounter. For a prey that has survived all encounters with t tot , its reproductive potential r(t tot ) is also modeled as the logit function.
where α and β are the parameters that determine the relationship.
In Eq. 2, β > 0 indicates the cost to reproduction of hiding. β is (1) logit (s (t)) = a + bt (2) logit r t tot = − t tot negatively correlated with resource availability, and hiding duration (t tot ) has a greater cost when resource level is low. Because r(t tot ) is not a probability, the use of the logit function is not necessary, but the same functional form is used for the benefit (Eq. 1) and cost (Eq. 2) of hiding, simply for consistency. Using different functions such as r(t tot ) = αe −βt tot can also give equivalent results, and the results are not sensitive to the functional form of r(t tot ). The relationship between reproductive potential r(t tot ) and actual reproduction is described below (Eq. 3).
The optimal hiding duration is determined by the benefit of hiding, s(t), and the cost of hiding, 1−r(t tot ); both increase with hiding time. For a specific case, the optimal hiding duration can be easily derived. Figure 1 shows the result for a situation where prey encounters three predators (p = 3) and no IIV (hiding duration for each encounter is t; t tot = pt). The optimal hiding duration is t that maximizes the fitness function s(t) p r(pt).

| Performance of individuals
The behavioral strategy of ith prey in a population is represented by (µ i and IIV i ). In each generation, the following steps are taken for each prey individual to determine its reproductive potential: (1) determine the number of predator encounters, p; (2) generate p hiding durations (i.e., t 1 , …, t p ) from a gamma distribution with mean µ i and standard deviation IIV i ; (3) determine whether the prey survives p encounters in which each encounter is a Bernoulli process in which the survival probability is determined by Eq. 1; and (4) if the prey survives, calculate its reproductive potential based on Eq. 2. If the prey dies in a predator encounter, its reproductive potential is 0. The distributional assumption for p, which represents within-generation stochasticity, is described below.

| Fitness landscapes
Because the simulation (section 2.2.1) is a stochastic simulation, the resulting value for the reproductive potential for the same strategy (i.e., a particular combination of μ and σ IIV ) is variable each time it is simulated. The expected reproductive potential for a specific trait can be obtained by running the simulation many times and taking the average (1 million simulation runs were used to compute an average). Because a behavioral strategy consists of two components (μ and σ IIV ), computing the expected reproductive potential for various combinations of μ and σ IIV will describe the relationship between behavioral strategies and fitness on a two-dimensional space, which is referred as fitness landscape. Because prey lives only one generation, the optimal strategy is the strategy that results in the highest expected reproductive potential in the simulation (section 2.2.1). Between-generation stochasticity does not affect the optimal strategy.

| Evolution
The model assumes that the carrying capacity of the environment is K, and there is K prey in each generation at the beginning of the simulation. For example, the combined reproduction of all members of the population is more than K offspring, assuming that at least one prey survives to the end of the generation, but in each generation, a random set of K offspring will survive to begin the next generation. The number of offspring from prey individual i that survives to begin the next generation is a random variable X i such that X 1 + X 2 + … + X K = K. In particular, X = (X 1 , X 2 , …, X K ) follows a multinomial distribution, where π = (π 1 , π 2 , …, π K ) is the probability vector, and . Therefore, prey with higher reproductive potentials are expected to leave more offspring.
Each offspring produced by a parent prey with a strategy (μ and σ IIV ) will inherit the traits of the parent. An offspring's mean trait is μ + q µ and IIV trait is σ IIV + q σ where q µ and q σ are random numbers generated from a normal distribution with mean 0 and standard deviations s µ and s σ , respectively. s µ and s σ are regarded as surrogates of heritability, although they are negatively correlated with heritability (e.g., s µ = 0 and s σ = 0 when a parent and its offspring all have the same traits, and heritability decreases as s µ and s σ increase). When a generated trait value becomes negative (e.g., σ IIV + q σ < 0), the value is set to 0, because both the mean and standard deviation of the hiding time cannot be negative. This study assumed small values of s µ and s σ (Table 1) F I G U R E 1 Effects of hiding time on survival, reproduction, and fitness when prey encounter three predators (p = 3). Survival is determined by Eq. 1 (i.e., s(t) p ), and reproduction is determined by Eq. 2 (i.e. b(pt)). The resulting fitness is the product of the two,

| Parameters and stochasticity
The model described above is complete when the parameter values are determined (Table 1). The survival parameters (a and b) were set so that when a prey does not hide (t = 0), the probability of survival is approximately 0.12. One unit of hiding duration increases the odds of survival by 1.5 times. The reproductive parameters (α and β) were set such that the reproductive potential is approximately 1 for prey that did not waste any time on hiding, given that they survive, and by setting β = b, the cost (β) and benefit (b) of hiding were on a similar scale in an average environment. The expected number of predator encounters is not independent of the survival parameters (a and b) in the model. That is, a high expected number of encounters with a high per-encounter survival rate, and a low expected number of encounters with a low per-encounter survival rate will give similar outcomes. For a given set of a and b, as the number of predator encounters increase, eventually no individuals can survive and reproduce and the population collapses. When the number of encounters is too few, there is very weak selection on hiding traits.
The number of encounters was set to balance these factors.
To examine the effect of within-generation stochasticity, the expected number of predator encounters was set at 3, and results from two levels of variability (i.e., standard deviation is 0 and 4) were compared. Zero standard deviation indicates a constant (i.e., all individuals encounter 3 predators), and the stochastic predator encounters were simulated by a negative binomial distribution. Between-generation stochasticity was represented by variability in β (in Eq. 2) among generations. In each generation, the value of β was generated from a gamma distribution with a standard deviation of 0.3 (Table 1). Greater stochasticity will enhance patterns that will be shown in the Results.

| Effects of within-generation stochasticity
Stochastic predator encounter does not qualitatively change the fitness landscape (Figure 2). Both in the static condition (i.e., all prey encounter three predators) and in the stochastic environment (i.e., the number of predator encounters is variable), the optimal strategy is associated with a strategy with no IIV, σ IIV = 0. The main difference is that the fitness landscape becomes flat (i.e., a wide range of strategies perform equivalently well with respect to the optimal strategy) in the stochastic environment.  (Figure 2). When predator encounter is stochastic, some prey does not encounter any predators.
Consequently, those lucky prey will leave many offspring regardless of their strategies. Similarly, prey that encounters unusually many predators (by chance) are likely die and leave no offspring regardless of their strategies. Consequently, the chance events (rather than strategy) becomes a dominant factor in determining fitness, making the fitness landscape flat. In both cases, expressions of IIV (e.g., individuals with σ IIV > 0 in Figure 3) can be interpreted as spill-overs from the optimal strategy (σ IIV = 0) due to imperfect heritability and chance events. This study does not focus on this trivial mechanism (weak selection), and the following results will assume the static number of predator encounters to examine the evolution of IIV under an assumption selection strongly acts on hiding strategies. TA B L E 1 Parameter definitions and values. When a symbol represents a realization of a random variable, its distribution is shown. For example, t follows a gamma distribution with mean μ and standard deviation (SD) σ IIV . When SD is 0, it is constant regardless of the distribution

| Evolutionary trajectory
When the strategies of individuals in a population are suboptimal, natural selection will eventually lead to the optimal strategy over generations. However, the average strategy of the population will not approach the optimal strategy by its shortest path.
The gradient of a fitness landscape (e.g., contour in Figure 4)   OKUYAMA evolutionary trajectories take long detour. Because the fitness contour is generally tilted to the right (Figure 4), when environmental condition becomes worse (e.g., β changes from 0.1 to 0.8), selection tends to favor individuals with greater σ IIV . In the presence of between-generation stochasticity, when the environmental condition is worsened, individuals with high IIV may be transiently selected even when the optimal strategy in that condition is associated with σ IIV = 0, which maintains individuals with high IIV in the population. Figure 6 shows average traits over generations, and the corresponding movie that shows the traits of all individuals when the initial population is characterized by the strategy (µ = 25, σ IIV = 0) is provided as Supporting Information.

| D ISCUSS I ON
This study examined the effects of two types of stochasticity on the evolution of IIV. First, within-generation stochasticity has little effect on the optimal strategy but still can influence the evolution of IIV by weakening selection. Second, between-generation The first condition, the dynamic optimal strategy, is commonly acknowledged, and has been intensively studied. In any optimality models, the optimal strategy will change according to the environmental variables considered in the models. For example, optimal patch residence time in the marginal value theorem can change, for example, with patch quality (Charnov, 1976). Optimal prey choice depends upon the density of profitable prey species (Pulliam, 1974).
As such, this condition is rather trivial and is likely satisfied in various ecological scenarios.
The second condition, that mean behavioral expression and IIV do not independently influence fitness, is also likely to be valid in many situations. To express the idea clearly, an example of fitness contour when µ and σ IIV do not interact is shown in Figure 7. The reason why µ and σ IIV interact can be illustrated with a simple generic example. Suppose that the relationship between behavior x and fitness w can be represented by w(x) = −(x−1) 2 + 1 (when 0 ≤ x ≤ 2) and w(x) = 0 (when x > 2). A specific w(x) is used here but any unimodal functions (e.g., Figure 1) will work in the same way. The optimal strategy for this example is (µ = 1, σ IIV = 0), and any strategies with (µ > 2, σ IIV = 0) have 0 fitness. However, strategies (µ > 2, σ IIV > 0) (e.g., µ = 2.1, σ IIV = 1) have a positive expected fitness because IIV (σ IIV > 0) can produce x between 0 and 2 by chance. Therefore, the effect of σ IIV on fitness is not independent of µ, and the fitness contours will not be true circles as in Figure 7. Although unlikely, when the second condition is not satisfied, transient selection for greater σ IIV will not occur even when the first condition is satisfied because the selection gradient with respect to σ IIV is always directed toward Theoretical studies that examine the evolution of traits in dynamical systems typically assume that a trait evolves to the direction of the selection gradient (e.g., Abrams, Matsuda, & Harada, 1993), which is the same as the result of this study in which evolution tracks the fitness gradient ( Figure 6). However, in those studies, only the gradient with respect to mean trait w∕ μ is considered, and the gradient with respect to variability, w∕ σ IIV is not considered. This assumption (ignoring IIV) may be valid when inflexible traits (e.g., some morphological traits) are considered but may not be appropriate for traits with IIV (e.g., behavior). An important conclusion of this study is that even when the optimal strategy is associated with σ IIV = 0, the selection gradient with respect to IIV, w∕ IIV , cannot be neglected.
Weak selection can have substantial influence on IIV traits of individuals in a population (Figure 3). When selection is weak, random effects can overrule selection. In other words, fitness is largely influenced by luck rather than strategies. Although trivial, it does not necessarily mean it is unimportant. In case of hiding duration, there may be indeed a substantial proportion of individuals that may never encounter predators in natural conditions. Or predation risk per encounter may be generally very low regardless of the strategy. When we focus on the optimal strategy, we can still find a unique optimal strategy no matter how flat a fitness landscape may be ( Figure 2), but the strength of selection must be carefully examined in each case.
One immediate prediction of the model presented here is that higher levels of IIV are expected in populations that experience greater environmental variability regardless of within-generation variability (i.e., weak selection) or between-generation (i.e., transient selection). To better examine the evolutionary mechanism presented in this study, we must be able to quantify fitness landscapes. We currently know little about how IIV influences fitness, and less is known about how µ and σ IIV jointly influence fitness. Figures 2 and 4 show that IIV can have both positive and negative effects, depending on mean expression, and thus a particular experimental result that simply shows a positive or negative effect of IIV may be incomplete.
Similarly, a study that only focused on IIV may give misleading conclusions when the mean expressions are not accounted for in the experimental design and data analysis. A simultaneous consideration of both mean and IIV may provide a better understanding of the effect of IIV on individual fitness, as well as how it is maintained in populations.

ACK N OWLED G M ENTS
I thank three anonymous reviewers and the handling editor for their constructive comments. This study was supported by the Ministry of Science and Technology of Taiwan through research grants 105-2311-B-002-019-MY3 and 108-2311-B-002-017-MY3.

CO N FLI C T O F I NTE R E S T
None declared.

AUTH O R CO NTR I B UTI O N
TO performed all work presented in this study.

DATA AVA I L A B I L I T Y S TAT E M E N T
This study is not based on data.

O RCI D
Toshinori Okuyama https://orcid.org/0000-0002-9893-4797 F I G U R E 7 An example of fitness contours when μ and σ IIV do not interact. For any suboptimal strategies with σ IIV > 0, the fitness gradient with respect to σ IIV is negative (smaller σ IIV is selected)