EVOLUTION OF TRANSCRIPTION NETWORKS IN RESPONSE TO TEMPORAL FLUCTUATIONS

Authors


Abstract

Organisms respond to changes in their environment over a wide range of biological and temporal scales. Such phenotypic plasticity can involve developmental, behavioral, physiological, and genetic shifts. The adaptive value of a plastic response is known to depend on the nature of the information that is available to the organism as well as the direct and indirect costs of the plastic response. We modeled the dynamic process of simple gene regulatory networks as they responded to temporal fluctuations in environmental conditions. We simulated the evolution of networks to determine when genes that function solely as transcription factors, with no direct function of their own, are beneficial to the function of the network. When there is perfect information about the environment and there is no timing information to be extracted then there is no advantage to adding pure transcription factor genes to the network. In contrast, when there is either timing information that can be extracted or only indirect information about the current state of the environment then additional transcription factor genes improve the evolved network fitness.

The conceptualization of genomes as gene interaction networks has generated a wave of research at multiple levels of biological structure (Watts and Strogatz 1998; Kitano 2004; Proulx et al. 2005) using a variety of computational approaches (Kirkpatrick et al. 1983; Tomshine and Kaznessis 2006). Several groups have followed the evolution of gene networks under variation in selective pressure (Kashtan and Alon 2005; Kussell and Leibler 2005), whereas others have looked directly for the optimal gene network to respond to specific challenges (Tomshine and Kaznessis 2006). Although these studies have addressed certain aspects of the evolutionary process on the one hand and examined the optimal networks on the other, there is still little evidence that gene networks will evolve increased complexity because of natural selection on variation in network topology.

In a series of papers, Soyer et al. studied the properties and evolution of signal transduction networks and applied these to the evolution of bacterial chemotaxis (Soyer and Bonhoeffer 2006; Soyer et al. 2006a, 2006b). They found that networks with additional proteins were able to improve their chemotaxis performance as measured by selection for an optimal tumbling rate (Soyer et al. 2006a). They also used a generic model of signal transduction to study selection for a specific response to a temporal signal. Again they found that increased network size allowed evolved networks to more closely match the target response (Soyer and Bonhoeffer 2006). The performance features of biochemical networks have also been found to show improvement as network size increase (Ziv et al. 2007). Another approach has been to consider networks that define the dynamical systems responsible for developmental changes. This has been studied principally by defining target patterns and evolving networks to match the target (Francois and Siggia 2012). These studies clearly show that aspects of network performance depend on network size and complexity, but do not directly address how populations of individuals can evolve networks of increased size and complexity.

We consider how an imposed network topology affects the evolution of network parameters in response to fluctuations in the environment and whether topology itself is expected to evolve in a consistent way. Our framework uses simple transcription control networks that are capable of responding to temporal variation in environmental conditions. Because we assume that fitness depends on the relationship between protein concentrations and the external environment we are able to model total organismal fitness without resorting to proxy measures of fitness based on a simple, perhaps arbitrary, measure of performance. The fluctuating environmental states could represent changes in the availability of nutrients for single celled organisms or represent changes in the physiological or hormonal state for cells in a multicellular organism (Proulx and Smiley 2010). The regulatory network describes gene expression and gene expression is controlled by genetic components (e.g., genes or protein interactions), gene interactions and an environmental component (Ideker et al. 2001).

We contrast several qualitatively different types of environmental fluctuations, including periodic waiting time, Gamma distributed waiting times, and exponentially distributed waiting times. Under the periodic waiting time there is a fixed waiting time until the environment switches. Under the exponential waiting time we draw an exponential random variable to determine when the environment will next switch. We also considered environmental waiting times that are intermediate between periodic and exponential; in particular, we considered Gamma distributed waiting times. We compare these different waiting time regimes while maintaining the same average waiting time, T. Our focus is on the way that differences in the way cells get information about the environment and the utility of this information alter the outcome of the evolutionary process.

Gene networks have been described using a variety of modeling approaches. One simplification is to consider ordinary differential equations (ODEs). ODEs can be used to describe the time course of gene product concentrations. This formalism requires the parameterization of specific kinetic reactions. Dynamic models can also include stochastic effects on the production of gene products (mRNA and protein) and introduce another layer of complexity (Smolen et al. 2000). We use ODEs to describe gene regulatory networks responding to sudden stochastic changes in the environment. The environment is described by a stochastic process that is external to the organism, whereas ODEs with time-dependant parameters describe the response of the organism. This type of model is called a stochastic hybrid system and has been studies in the engineering literature (Singh and Hespanha 2010).

Even though many insights have come from modeling the regulatory dynamics of specific gene networks, the evolutionary construction of complex gene networks is still a puzzle (von Dassow et al. 2000; Ideker et al. 2001; Kitano 2004; Klipp et al. 2005; Proulx et al. 2005). One possible benefit of increasing the number of genes/proteins in a network is that subtle environmental signals can be interpreted in ways that simply cannot be achieved in smaller networks. In effect, additional regulatory genes may act as the system's memory to infer the environments’ current and future states. When the information about the current environmental state is delayed or incomplete, additional regulatory genes may allow the network to process the temporal information, for example by measuring the derivative of the environmental signal. In periodic waiting time regimes, a network that has memory can potentially predict the time at which the environment will switch states.

These observations lead to the conjecture that more complex regulatory networks will evolve when additional information about the current and future environment can be inferred by tracking the history of environmental fluctuations. Based on this premise we predict that there will be little selection for complex networks in environments where the waiting time for fluctuations is exponentially distributed and direct information is available. In contrast, when the timing of environmental fluctuations is more structured (e.g., periodic) or when there is only indirect information available then we expect more complex networks to evolve.

We modeled the evolution of gene networks that could respond to an environmental signal by adjusting protein production of a gene involved in a physiological response to the environment. We studied this both by deriving general optimal control models and by simulating the evolutionary trajectories of specific gene networks. The optimal control models can be used to identify an upper bound on the fitness that could be achieved, either by a biologically constrained system or by an arbitrarily complex controller. This gives an upper bound on achievable fitness and can be used to estimate the maximum strength of selection that can act to increase the processing ability and complexity of a gene network. We also derived constrained optimal control models that allow us to probe the limitations of realistic network architectures.

In our evolutionary simulations, we constrained populations to have networks with a fixed number of genes and compared the fitness between evolved networks of different size. Networks were increased in size by the addition of transcription factor genes that have no direct physiological function other than transcription regulation. We used this modeling framework to examine the hypothesis that larger networks are beneficial when the signal coming from the environmental fluctuations contains information that can be extracted by processing through the network.

The Models

DYNAMIC MODELS

We attempted to construct the simplest scenario where different strategies of gene regulation might be under selection. We considered a scenario where two environmental states are possible and the environment occasionally shifts between states (Proulx and Smiley 2010; Smiley and Proulx 2010). The environmental state is detected by the cell and can directly affect transcription of one or more genes in the network. Fitness depends on the match between expression of target-genes and their environment-specific optima integrated over the life of the individual. This reflects a model where individuals gather resources over an amount of time that is long compared to the rate of environmental fluctuations and then reproduce in proportion to the resources they have gathered.

We built a Direct signal model to represent the simplest case and then expanded our model to include networks with additional regulatory genes. We compared the evolved networks to see if networks were able to evolve higher fitness when they had additional regulatory genes. Because the type of environmental fluctuations and the amount of information about the environmental state are each expected to influence the optimal regulatory response, we contrast scenarios where information about the environment is perfect with those with imperfect information.

We developed several scenarios that challenged the gene regulatory network in different ways. We developed the Direct signal model that has one target gene (red circle) and direct information about the environment (blue square; Fig. 1). The target gene is a gene that has some biochemical action and determines the fitness of the gene network. To the Direct signal model, we added additional regulatory genes (pink circle) that do not have direct biochemical activities but contribute to the regulation of the target gene. These additional regulatory genes have no direct bearing on fitness and therefore selection acts on regulatory genes solely through their effect on target-gene expression trajectories.

Figure 1.

A schematic diagram of our modeling framework where environmental waiting time complexity, gene interactions, and the existence of an intermediary environmental signal are shown. The red circles indicate target genes, the pink circles indicate additional factors such as transcription factors, the light blue triangles represent intermediary environmental signals, and the blue squares represent environmental signals. The numbers with arrows correspond with equations.

The nature of the environmental fluctuations determines how much information can be gained by tracking the history of environmental shifts. We designed three different regimes for the environmental fluctuations: periodic (fixed) waiting time, gamma distributed waiting time, and exponentially distributed waiting time. In the scenario where switching times are exponentially distributed, no information can be extracted from the history. That is to say, a cell that has been in environment 1 for x time units has the same expected waiting time until the environment shifts as cell that has been in environment 1 for y time units. This led us to expect that under exponential waiting times there would not be selection pressure to evolve networks that could, in effect, track history. Under purely periodic environmental fluctuations, in contrast, a network that has the ability to track history could potentially exactly predict the time of a future environmental shift and optimally time the dynamics of gene expression. Because this opens the possibility of accurate but nonrobust control strategies, we also included a regime where waiting times follow a Gamma distribution with a tight peak around the mean waiting time. This still allows the possibility that the future can be inferred, but a strategy that simply tracks internal time will drift off the optimal solution and fail.

We also considered scenarios where the cell has imperfect information about the environment. This could occur if, for example, molecules diffuse into the cell or the cell nucleus and therefore represent a delayed integration of previous signals. In both bacterial and yeast systems, some nutrient transporters also act in a signal cascade (Thevelein and Voordeckers 2009). It is likely that these systems evolved from ancestral states where the presence of nutrients in the cell served as the only means of signaling, which would necessarily be a delayed signal. In our indirect model we explicitly track the dynamics of this intermediary environmental signal (light blue triangle) where the strength of environmental signal is a continuous parameter analogous to the diffusion rate of the molecule into the cell (Fig. 1).

We used an ODEs framework to model levels of the gene product and the intermediate environmental signal (when appropriate). For simplicity we used a one-step model to relate gene expression to protein production (Thacik and Walczak 2011). That is, we assumed that no additional posttranscriptional regulation occurred and that protein production was relatively fast so that a single ODE could describe protein levels based on transcription rates (see Smiley and Proulx 2010) for a formal analysis). We used Hill functions to describe both activation and repression similar to that of (von Dassow et al. 2000). We also allowed the state of the environment, or the intermediate environmental signal, to influence gene expression via Hill functions as well.

Hill functions can accurately describe the combinatoric interaction of binding sites where there is cooperatively at the level of binding or at the level of the logical integration of bound sites (Murray 1993). We assumed that, regardless of the number of transcription factors regulating a gene, there is an upper bound on the transcription rate that occurs when RNA polymerase is maximally recruited to a gene. The total rate of protein production, given maximum RNA polymerase recruitment is set at Max V. This ensures that there is no intrinsic advantage to having more regulators that could lead to mistaken conclusions about the benefit of having more complex regulatory networks.

In our direct signal model, the environment is assumed to directly affect transcription. This is biologically unrealistic, but represents a limiting case where the environmental signal is instantly transduced into an effect on transcription rates. The environmental signal could be mediated through changes in concentration of a cofactor, changes in concentration of a transcription factor, or even through changes in the configuration of DNA near the focal genes. The differential equation for gene i is

image(1)

where Gi represents the concentration of the protein produced by gene i, Max Vi represents the maximum rate of protein production, VE(t) is the time-dependent direct effect of the environment, inline image represents Hill functions that translate concentrations of all genes in the network into an effect on gene i, and inline image is the protein degradation rate. This model assumes that all proteins are passively degraded at a gene specific rate inline image and that expression is the product of interacting regulatory effects that maintain the total rate of protein production between 0 and Max Vi.

The Hill functions can be expressed as

image(2)

When i=j, we set Hj= 1. The Hill function for the Direct signal model is

image(3)

where Ki is the ith gene dissociation constant, inline image is the maximal rate of repression/facilitation and ni is the Hill coefficient (sometimes represented as a measure of the cooperativity between transcription factor binding sites). For the networks with more than one gene we allow arbitrary interactions between genes, ranging from strict repression to strict activation. To model this range of dynamics we take expression to be a product of Hill functions,

image(4)

where Gl represents the concentration of gene product l, Kil represents the disassociation constant for gene product l on the promoter of gene i, and inline image is a parameter that determines the direction of the effect of gene l on gene i. If inline image then we have strict repression while if inline image we have strict activation. The functional response smoothly transitions between the two endpoints as inline image varies.

For the Direct signal models, the environment is directly and accurately sensed so the model above has direct access to the signal

image(5)

For the indirect signal models, the cell senses the environment through an intermediate process. We adopt a formalism where the signal that the cell receives is a continuous variable that becomes closer to the true environmental state as the amount of time in the current environment increases. Mechanistically, this could occur if a signal molecule diffuses into the cell and the external concentration is assumed to be 0 or 1 and the rate of diffusion into the cell is vE. The parameter inline image represents the rate at which the signal molecule is degraded within the cell. The concentration of the environmental signal follows

image(6)

The response to this signal is also modeled as a Hill function,

image(7)

where Kie is the dissociation constant and nie is the Hill coefficient for the environment signal. Ec is now an environmental indicator signal and not the actual value of the environment.

In the indirect signal network, the transcription control of each gene may be determined both by the absolute level of the environmental signal (eq. 5) and by an interaction between the environmental signal and the level of other transcription factors (eq. 7). We chose to include both because many genes are combinatorically regulated and because the difference in concentration between two dynamically changing variables potentially allows the network to infer the derivative of change in the environmental signal. This can occur when a repressor and activator bind to the same DNA sequence (Theil et al. 2004) or when an activator and repressor act as cofactors and bind the same regulatory protein (Darieva et al. 2010). For the target gene in multigene indirect signal networks, we replace VE with inline image where

image(8)

where Klag and nlag are the dissociation constant and the Hill coefficient, inline image determines the direction of the effect and inline image scales the effect of Gl expression.

Fitness is assumed to depend on the amount of resources successfully gathered and metabolized by the organism over its life time and is calculated by integrating the instantaneous fitness over time. Fitness depends on both the concentration of the target gene and the state of the environment and is given by

image(9)

where inline image is the optimal level of G1 in environment e and inline image measures the strength of selection. We assigned parameter values of inline image, inline image, and inline image. With these parameters, environment 0 favors low expression of a gene 1 whereas environment 1 favors high expression of a gene. Thus, the optimal gene expression profile is a step function. So long as inline image generalist strategies are disfavored (Proulx and Smiley 2010), making the conditions for the evolution of gene regulation are more restrictive.

Theoretical Results and Predictions

OPTIMAL CONTROL SOLUTIONS

To better understand the possible causes of the evolution of the transcription control network, we have developed models of the optimal control of gene expression. To do this, we first consider a model where the target gene can be arbitrarily controlled, subject only to the constraints on the maximum rate of gene transcription and a constant rate of gene product degradation. In this situation, the dynamics of the target gene are given by

image(10)

where G is the concentration of the gene product, Max V is the biological constraint on transcription rate, and inline image is the degradation rate. The output of the controller is specified by inline image because the controller may depend explicitly on time (t), and on both the current an past values of the environmental state (E) and the concentration of the gene product (G). This notation allows for a controller function that is time-dependent in the sense that the current response to conditions is affected by previous observations of conditions. Storage of information also allows the controller to integrate and differentiate dynamic variables. Given this most flexible definition for a controller, we can examine the features of the optimal control strategy and ask whether additional constraints on the capabilities of the controller would cause a reduction in performance.

The optimal control strategy represents the upper bound on fitness that can be achieved by any gene network. We expect that our evolved networks will always have fitness that is worse than or equal to the optimal control strategy. The most fit evolved network is subject to the biologically imposed constraints of the Hill function type regulatory responses and the constraint of the imposed network architecture that we use throughout this article. There are constraints in terms of independent optimization of the feedback law that emerge because of the Hill functions (Martins and Swain 2011). The difference between the most fit evolved network and the optimal control strategy represents the maximum amount of natural selection that could be brought to bear on improving the network control. If the strength of selection is small, then the network architecture and biochemical feedback mechanisms assumed are sufficient to gain most of the possible benefits of controlling gene expression. Further improvement in the network will be slow and subject to population size–based limitations imposed by drift (Lynch 2007).

We also wanted to investigate how specific network architectures constrained the control strategies that could evolve. For example, when a network is restricted to containing a single gene then there are limitations on how information can be stored and on how complex the response to the external inputs can be. If the optimal controller fitness is much better than that of the evolved network, then we expect that the imposed network structure and assumed biochemical feedback mechanisms are not sufficient to process input in a similar way to the optimal controller. To better understand how constrained network architecture relates to the type of control that can be produced we constructed constrained controllers that are limited in the type of feedback or information storage and processing that they can perform. For example, these may be constrained to have no memory and use only the current environmental state. Comparing the fitness of the constrained controller with that of the optimal controller gives us an indication of the magnitude of the benefit that networks can capture by adding circuits that can serve as memory or provide more complex processing.

PERIODIC WAITING TIMES, DIRECT SIGNALS

Under periodic waiting times with a direct environmental signal, the optimal controller can be described as a static function of the time elapsed since the last environmental change. For example, a cell with a cyclical response entrained by the environmental change could perform optimally without further processing or auto-regulation. Furthermore, the optimal controller cannot be described by a static function of the environmental state and the concentration of the gene product (in this context, “static function” means time and history independent). Instead, it can be described as a function of the time elapsed from the last time the environment changed from 0 to 1 (or vice versa). The optimal controller can anticipate the change in the environment, and so long as a change in expression, near the optimum expression level, has only a small effect on fitness then some anticipation of the environmental shift must be included in the optimal strategy.

Clearly, a controller with a fixed periodic response suffers from a lack of robustness to environmental variation because it depends on the exact periodicity of the environment and has no mechanism for entrainment. However, if fluctuations in the timing are small then the optimal controller should be entrained to the most recent environmental transition and also anticipate the next transition.

The switching nature of the environment suggests a piece-wise constant optimal control signal (Bemporad et al. 2002) that is characterized by a set of time parameters and associated expression levels. Moreover, the periodic nature of the environment imposes a periodic control law. This reduces the optimal control problem to a parametric optimization that we solved numerically. Optimal values were obtained for four time parameters and four expression levels that are repeated periodically with the same period as the environment. Thus, the putative optimal control solution is characterized by four absolute time points at which expression levels change and can be made tractable by noting that the gene-expression trajectories will have a periodic form. This formalization reduced the control problem to a much simpler eight-parameter optimization.

We found that the optimal control strategy involves anticipation of the change in the environment, and the optimal control strategy achieves fitness higher fitness as the period gets longer (Table 1 and Fig. S2 in Supplementary Materials Appendix).

Table 1.  Fitness of optimal control strategy.
  T= 20 T= 10 T= 5
Fitness0.96990.95360.9052

PERIODIC WAITING TIMES, INDIRECT SIGNALS

We modeled the indirect signal model as a diffusion process into the cell/nucleus, which means that there is no longer a one-to-one relationship between the signal level and the environmental state. This means that the environmental state cannot be inferred from the signal level alone, and therefore additional information needs to be used to recreate the optimal control strategy found in the direct signal model.

Several strategies could be used to achieve high fitness. For example, the control strategy could be based on the derivative of the signal that completely reveals the current state of the environment (but does not unambiguously reveal the current time). Perhaps unexpectedly, it may be possible to construct the optimal controller without specifically storing information. This is because, taken together, knowledge of the control strategy, the concentration of the gene product, and the signal can reveal the state of the environment, the concentration of the gene product, and the time since the last environmental transition. Because this information reduces the control problem to the case of a direct signal, the same optimal controller (but based on a different source of information) would be optimal. This means that a controller could be constructed out of a single gene that is auto regulated by a feedback law that depends on both the environmental signal and gene product concentration.

One type of constrained controller that might be biologically realistic is a system that determines the current transcription rate based on the current signal level but does not store information or compare relative concentrations of the signal molecule and the target gene product. Such a system can be described as a static function of the signal level, U(Ec). The optimal static function can be found by discretizing the state space and using any standard optimization algorithm to find the optimal transcription level for each possible level of the signal. We performed optimizations using the fmincon function in Matlab 2011b. The constrained control solution no longer shows anticipation, but instead has spikes in the transcription rates due to the conflicting demands on the controller in different environments (Fig. S2 in Supplementary Materials Appendix). For comparison, we plotted the expression of the target gene from an evolved two-gene network and from an evolved one-gene network (1-gene without autoregulation).

EXPONENTIAL WAITING TIMES

Under exponential waiting times there can be no advantage to tracking time because of the memoryless property. This means that an optimal controller can be constructed based on inferring the probability that the system is currently in each of the possible states and choosing the response that maximizes the expected future fitness. In the case where there is direct information about the environmental state, the optimal controller can therefore be based directly on the combined information of the current level of gene expression and the state of the environment (Bertsekas 1995). This does require auto-regulation, but does not require any additional memory or complex processing. This leads us to predict that simple auto-regulatory networks should capture most of the fitness.

When the environmental state is only detected indirectly then an optimal controller must have some method for inferring the current state. As in the periodic case, this could be unambiguously accomplished by measuring the derivative of the signal. In this scenario we expect that a simple auto-regulatory network will not be able to perform the inference of the current state and that more complex networks will be required.

RELATIONSHIP BETWEEN OPTIMAL CONTROL SOLUTIONS AND SIMPLE GENE NETWORKS

Based on our analysis of the optimal controllers and the Hill function formulation of our gene regulatory networks, we can make some predictions about which conditions will lead to a benefit of having more complicated gene regulatory networks. In all cases, some amount of auto-regulation is required to achieve robust control. In the exponential case with direct information, we expect single gene networks with auto-regulation to capture most of the available fitness. Under periodic conditions, there is a possibility to extract and respond to timing information, so networks that are able to do this should have higher evolved fitness. This suggests that networks with additional transcription factor genes can process and store such information and evolve higher fitness. In our indirect signal model, regardless of the specific waiting time, there is an advantage to inferring the current state of the environment. This suggests that networks with more genes (i.e., transcription factors) would be able to evolve higher fitness.

Simulation Results

EVOLVED FITNESS OF GENE NETWORKS

Fitness calculations are the output of a dynamic process involving both the fluctuations in the environment and the dynamics of gene expression, making fitness calculations costly in terms of computational time. Like other models of gene network evolution, our models have many parameters, making an exhaustive exploration of parameter space infeasible. It is also difficult to do population genetic simulations with large populations because each new set of network parameters would require calculating a new fitness value. To get around this issue, we have adopted a simulated annealing (SA) approach that utilizes the strong selection–weak mutation assumption (Li 1987; Gillespie 1991). This approach allows us to follow the evolution of large populations as they make transitions between states characterized by a monomorphic set of network parameters (see the Supplementary Materials for more details on this approach).

We simulated the evolution of network parameters with a predetermined network architecture. For the direct signal model, the cell has perfect information about the state of the environment. Figure 2 shows the evolved fitness and the effect of including auto-regulation or an additional gene. Under the periodic waiting time regime, the single gene without auto-regulation achieved 94 % of the optimal control fitness (based on the simulation with the highest fitness). Including auto-regulation improves fitness considerably, giving up to about 98 % of the optimal control. Although two-gene networks on average achieved higher fitness than the one-gene networks, the maximum obtained in our simulations was slightly lower (97.5 %). We also examined the evolved fitness of networks experiencing a Gamma distributed waiting time. This allowed us to have conditions where there was considerable timing information available but where a preprogrammed control strategy would perform very poorly. We used parameters for the Gamma distribution that gave the same mean waiting times as in the periodic environment, but allowed the standard deviation to be on the order of the amount of time it takes for expression levels to go between 0 and 1 (SD = 1.0373). As in the periodic case, allowing auto-regulation had a large effect. The mean of the 1-gene network without auto-regulation network is 0.7751 and the mean of 1-gene with auto-regulation network is 0.8920. Adding a transcription factor only increased the mean fitness by 2.29%.

Figure 2.

Evolved network fitness for the model and model. The columns show box plots that compare three different network topologies based on 60 runs of the SA procedure. The first rows are results from the model and rest of four rows from the model (inline image). The left column shows results for the periodic waiting times, whereas the right column shows results for the Gamma distributed waiting times. Results for the model are shown for a range of mean waiting times, T. When T= 20, the shape parameter and the scale parameter of Gamma distribution are 100 and 0.2 respectively, and the variance is 4. When T= 10, the shape parameter and the scale parameter of Gamma distribution are 100 and 0.1 respectively, and the variance is 1. When T= 5, the shape parameter and the scale parameter of Gamma distribution are 100 and 0.05 respectively, and the variance is 0.25. When T= 2.5, the shape parameter and the scale parameter of Gamma distribution are 100 and 0.025 respectively, and the variance is 0.0625. The box plots show the median as the central line and the top and bottom of each box indicates the 25th and 75th percentile. Whiskers extend to the most extreme data points not considered outliers. Also shown for the periodic waiting times are the fitness of an optimal control solution which has a complete knowledge and may depend on time (upper dashed line) and of a constrained control solution based only on the current level of the environmental signal (lower dashed line).

Next we considered the indirect signal model. Under periodic waiting times we found that the network without auto-regulation evolved lower fitness than even our constrained optimal controller. However, including auto-regulation and a transcription factor gene allowed the network to evolve fitness higher than the constrained optimal controller, but lower than the unconstrained optimal controller. For environments that fluctuate more quickly the fitness achieved by one-gene networks was quite low, and the proportion of the optimal control obtained by the two-gene networks decreased from about 92% when the mean waiting time was 20 to about 68 % when the mean waiting time was 2.5. The benefit of adding auto-regulation decreased as the waiting time got smaller, Although the benefit of adding a transcription factor increased. A similar pattern was obtained for the Gamma waiting times. Note that a network that fixed the expression of G1 at 0 or 1 would achieve mean fitness of 0.5. Because evolved fitness in regimes with T= 2.5 were very close to the 0.5, we did not analyze these networks further. We were also interested in determining how the dynamics of the environmental signal altered the evolved fitness of larger networks. We simulated a range of diffusion rates for the environmental signal and found that the benefit of adding genes increased as the diffusion rate decreased (Fig. S1 in Supplementary Materials Appendix). Because the optimal control solutions show anticipation of the change in the environment we examined whether our evolved networks also showed anticipation. We measured how the network anticipated changes in the environment. The anticipation of the highest fitness network in both periodic and Gamma waiting time regimes is shown and the exponential waiting time does not show any anticipation as expected (Figs. S3 and S4 in Supplementary Materials Appendix).

We further quantified this effect by examining how long it takes for the network to achieve an instantaneous fitness of 85% of the maximal fitness. We found that the 3-gene network crosses this threshold first, followed by the two-gene network and one-gene network (Fig. S5 in Supplementary Materials Appendix). It is difficult to predict what the expression profile of the optimal network should be, but the unconstrained optimal control strategy does show anticipation of the environmental switch. In our simulations, only the more complex gene networks show anticipation and resemble the optimal control strategy. We explored the effect of changing the mean waiting time in the exponential waiting regime (Fig. 3). Under the direct signal model there is a large benefit of adding auto-regulation, but the mean fitness of evolved two-gene networks is not larger than that of evolved one-gene networks. In contrast, under the indirect signal model there is an increase in the mean evolved fitness with both the addition of auto-regulation and of a transcription factor.

Figure 3.

Evolved network fitness under exponential waiting times. The columns show box plots that compare three different network topologies based on 60 simulations for a range of average waiting time, T. The first column represents the results from the direct signal model and the second column shows the results from the Indirect signal model with inline image.

TRANSITIONS BETWEEN NETWORK TOPOLOGIES

Our simulations track the evolutionary change in fitness and network parameters for models where the number of genes in the network are fixed. To investigate how transitions between network states might occur we calculated the probability that rare mutants with different network configurations would invade using equation (S2). We considered a scenario where isolated populations that were monomorphic for a particular network size evolved following our SA routine and then exchanged rare migrants. The probability that a pair of populations will end up in a particular network state is simply

image(11)

where U(j, i) is the invasion probability for alone i individual in a population of j individuals. We calculated this probability by resampling fitness values from our SA runs 10, 000 times for a range of population sizes. We found that the pattern was similar for all population sizes of 1000 or more, and so we only report the results for populations of size 1000.

We found that for the periodic and Gamma waiting times there is always a benefit of including auto-regulation and adding a transcription factor (Fig. 4). The benefit of adding a transcription factor is higher in the direct signal model as compared to the indirect signal model with the same waiting time. In the model, the benefit of adding a transcription factor gene increases as the waiting time decreases. This is most extreme in environments where the fluctuations are very rapid and the benefit of auto-regulation decreases to be almost neutral (Fig. 4).

Figure 4.

Proportion of populations that are expected to become fixed for larger networks under regular environmental shifts. The bars are calculated using equation (11) so that values greater than 0.5 indicate that the larger network is favored. The green bar (the left bar) shows the outcome when one-gene networks without auto-regulation and one-gene networks with auto-regulation are competed. The blue bar (the middle bar) is fore competition between one-gene networks with auto-regulation and two-gene networks. The red bar (the right bar) is for competition between one-gene networks without auto-regulation and two-gene networks. The model which has T= 10 is shown on left and the indirect signal model is shown on right for a range of T.

A different picture emerges under exponential waiting times (Fig. 5). Under the direct signal model, there is always a benefit for including auto-regulation and never a net benefit for adding a transcription factor. Under the indirect model, adding a transcription factor is always beneficial, and the benefit of adding complexity to the regulatory network, through either auto-regulation or additional genes, increases as the mean waiting time gets smaller.

Figure 5.

Proportion of populations that are expected to become fixed for larger networks under exponential waiting time. The blue bar (the left bar) represents the results from the Direct signal model and the red bar (the right bar) shows results from the indirect signal model.

MUTATIONAL ROBUSTNESS OF NETWORK COMPONENTS

Because we found a larger increase in network fitness when the network size was increased in the indirect signal model as compared with the direct signal, model we wanted to assess whether or not these networks differed in how robust the evolved networks are to mutations. Even though the direct signal model did not show an increase in fitness with the addition of regulatory genes, it could still be the case that the extra regulatory genes became intwined in the network in a way that would preclude their evolutionary loss or evolutionary capture by other processes.

To investigate how robust different components of the network were to point mutations, we separated parameters into three groups: G1 only, interaction, and G2 only (see Supplementary Materials Appendix). We sampled 100 point mutations each for 100 evolved parameter sets (for a total of 104 mutations) and compared the fitness values before mutation and after mutation. Figure 6 shows that for both the direct signal and indirect signal models, most mutations in the G1 only group are deleterious. Mutations in the G2 only and interaction group are characterized by a high variance distribution relative to the mean effects, but were on average deleterious. In both the direct signal and indirect signal model, the average deleterious affect of mutations in the interaction group was larger than for mutations affecting the G2 only group.

Figure 6.

Frequency distribution of the fitness effects of mutation. The distribution was created by sampling 100 point mutations from each of 100 evolved parameter sets for direct signal and indirect signal models with two genes and each type of parameter (note the log transformation of frequency). For the indirect signal model inline image is used. The blue bar (the left bar) represents mutations that alter parameters related only to the target gene, G1. The green bar (the middle bar) represents mutations that affect parameters governing the interaction between G1 and G2. The red bar (the right bar) represents mutations that alter parameters related only to the transcription factor gene, G2.

CHARACTERIZATION OF NETWORK FUNCTION: EXPONENTIAL WAITING TIMES

We examined the evolved network parameters in each regime to look for patterns in terms of the combinations of parameter values that evolved. We generally found that each parameter was highly variable (see Supplementary Materials Appendix), with the exception of the degradation rates which all tended toward the maximum value. In general, the evolved network is a dynamic controller, meaning that the utility of the response may depend on the state of internal variables. This makes it particularly difficult to define the properties of the controller without reference to the actual dynamic responses that are generated. One exception to this is the one-gene controller under exponential waiting times.

In this case, the optimal controller is expected to be time independent and feedback can only occur through the concentration of the functional gene itself and the environmental signal. We characterized the evolved control response by plotting the rate of expression divided by the rate of degradation as a function of the concentration of the focal gene with the level of the environmental signal held at the maximum value. This gives the feedback law in the case where the environmental signal has reached its maximum level and has an equilibrium where the curve crosses a line with slope 1. Figure 7 shows the feedback law for a range of waiting times. The basic picture is that protein concentration increases quickly when the current concentration of the protein is low and reaches an equilibrium near the optimum value of 1.

Figure 7.

The feedback law for the target gene in the high-expression environment. From 30 evolved networks for the case of one-gene networks with exponential waiting time, we plot the rate of expression divided by the rate of degradation for a fixed (maximal) level of the environmental signal as a function of the concentration of the target gene. If the environment remained in the same state, the system would reach equilibrium where the curve crosses a line with slope 1. Most networks have an environments-specific equilibrium near the optimal expression level of 1 and show high rates of transcription for lower levels of the target gene. The upper box plot shows the distribution of the environment-specific equilibrium as a function of the mean waiting time. The lower box plots shows the rate of expression divided by the rate of degradation just after the environment switched (measured empirically from simulations).

To get a view of how the networks respond when far from equilibria we simulated the trajectories over the exponential waiting times and calculated the change in protein concentration over a small time window immediately following the change in the environment (from 0 to 1). We used this measure to estimate the half-life of the change toward equilibrium by calculating the proportion of the distance between the concentration at the switch-point that was covered during this time period. If the rate of gene expression and protein decay is constant, the half-life can be estimated as

image(12)

where inline image is the time interval and x is the proportion of the difference between the initial concentration and the equilibrium concentration that is traversed during the interval. This is sort of an instantaneous half-life and does not reflect the long-term dynamics because transcription levels are not static. It does, however, capture the speed with which each network responds to a shift in the environment.

We expect that in environments characterized by longer waiting times there is more selection on achieving an accurate equilibrium Although in environments with short waiting times there should be more of a premium on fast responses. Furthermore, we expect equilibria to be biased toward values lower than 1 because there is a symmetric cost to deviations from 1 but a gain in response time for values less than 1. In environments with short waiting times, however, the cell will often not approach equilibrium before the environment shifts another time, again placing increased pressure on the speed of response as opposed to an accurate equilibrium.

We found that for long waiting time environments (T= 20), most simulations evolved networks with equilibria near but below 1, whereas networks evolved under T= 5 had longer waiting times that were relatively symmetric around 1. The estimated half-life decreases as the waiting time goes down, indicating that the evolved networks have a quicker relative response in short waiting time environments.

Discussion

We considered two different scenarios regarding the way that the network gets information about the environment. In our direct signal model, information is instantly and accurately available, whereas the indirect signal model involves an intermediate step that obscures the relationship between the signal and the environment. We aimed to test the idea that the information available to the network and the predictability of the environmental transitions would determine whether networks with additional feedbacks and genes would evolve higher fitness. Under the direct signal model, we expected that under exponential waiting times there would not be a benefit of having transcription factor genes because there is no need to process the signal or track time. In contrast, under the direct signal model with periodic waiting times we expected that there would be an advantage to having a more complex regulatory structure because there is an advantage to tracking time. Our simulated populations bear this out both in terms of the absolute differences in evolved fitness (Figs. 2, 3) and in the probability of transition between one- and two-gene networks (Figs. 4, 5). We found that under periodic waiting times there is always an advantage to two-gene networks over one-gene networks and that this advantage is larger under the direct signal model.

For the exponentially distributed environmental transitions, we expected that there would be increased benefit of adding transcription factor genes when the environmental information was filtered through another process, such as diffusion of molecules into the cell. In the indirect signal model, we found that additional transcription factor genes were beneficial, whereas this was not the case for the direct signal model. Similarly when the linkage between the environment and the signal was tight there was little selection for additional regulatory genes (Fig. S1 in Supplementary Materials Appendix). We also found that when the waiting time is large there is more selection to achieve the correct equilibrium within each environment and less selection on the speed of the initial response.

Under both the direct signal and indirect signal model, we found that the transcription factor gene was functionally integrated into the network. This can be seen by the large deleterious effect that mutations altering the interaction between the two genes had on fitness (Fig. 6). In the indirect signal model, there was a much larger negative affect of mutations that altered the interaction between the genes than of mutations that only altered parameters specific to the transcription factor. This may indicate the expression dynamics depend most on having a particular type of feedback loop rather than on the specific features of the transcription factor expression levels.

The SA algorithm that we used captures many features of the evolutionary process and likely is an upper estimate on the degree of adaptation that can be obtained by evolving populations. The SA procedure does capture differences between network configurations in their ability to climb local fitness peaks and to explore new fitness peaks by drifting through fitness valleys. However, the procedure does discard one important component of the evolutionary process in that mutation selection balance is largely removed in the final stages of the procedure. Because the probability of accepting low-fitness mutations is decreased toward the end of the SA algorithm, differences between network structures in the size of their mutational target are neglected. In real populations, networks that have more genes are expected to provide a larger mutational target. The larger mutational target typically leads to an increase in the mutation load (Proulx and Phillips 2005), even if the larger network is more epistatic. Our results should be interpreted with this in mind, meaning that when two different sized networks evolve similar fitness levels, we would expect the larger network to have a higher mutation load than the smaller network.

Our simulation approach applied SA to find genotypes with high fitness using a complex gene network responding to environmental fluctuations. Because the temperature parameter in the SA algorithm is analogous to population size, performing the SA algorithm with fixed temperature roughly captures the behavior of an evolving population under the assumption that new mutations do not typically arise Although a mutation destined to fix is still segregating (Li 1987; Champagnat et al. 2006). We have verified that in our system the SA model modified to follow the population genetic fixation probabilities does reach similar end states as the standard SA model. This allows us to improve the computability of the evolutionary process without sacrificing realism. As our knowledge of genetic interaction improves and researchers develop increasingly complex parameterized networks, such computational shortcuts will be required to improve our understanding of evolutionary transitions.


Associate Editor: J. Hermission

ACKNOWLEDGMENTS

The authors thank Alexey Yanchukov for discussing these concepts and providing useful criticisms and insights. Several reviewers provided useful feedback on a previous version of this manuscript. Kyoungmin Roh was supported by National Science Foundation (NSF) grant EF-0742582 to S. R. Proulx. Support for Farshad Pour Safaei and Joao Hespanha was provided by NSF grant EF-1137835 to S. R. Proulx and J. P. Hespanha.

Ancillary