Maximizing batch fermentation efficiency by constrained model-based optimization and predictive control of adenosine triphosphate turnover

We present a constrained model-based optimization and predictive control framework to maximize the production efficiency of batch fermentations based on the core idea of manipulating adenosine triphosphate (ATP) wasting. In many bioprocesses, enforced ATP wasting — rerouting ATP use towards an energetically possibly sub-optimal path — allows increasing the metabolic flux towards the product, thereby enhancing product yields and specific productivities. However, this often comes at the expense of lower biomass yields and reduced volumetric productivities. To maximize the overall efficiency, we formulate ATP wasting as a model-based optimal con-trol problem. This allows for balancing trade-offs between different objectives such as product yield and volumetric productivity for batch fermentations. Unlike static metabolic control, one obtains a higher degree of flexibility, adaptability

have been extensively considered by now, [2][3][4] they have not found widespread industrial application for a multitude of reasons.Often, no models or models of only minimal descriptive power for the design and control of fermentation processes are available. 2,4Furthermore, the operation might be subject to significant disturbances and possible changes in process dynamics, which are difficult to reflect in the optimization and models used.While disturbances can sometimes be minimized by precise and well-defined process conditions, this is often economically challenging or impossible, for example, considering processes involving biological waste products with changing properties.6][7] For example, a high titer or product concentration often enables efficient and cost-effective downstream processing.On the other side, a high product volumetric productivity rate might be desirable for minimizing capital and production costs, especially when dealing with bulk chemicals (high volume/low value).Additionally, the yield of product on substrate is often of interest, mainly when the substrate cost is an economic constraint. 8,9 tackle the outlined challenges, we propose a model-based moving horizon optimal control framework to maximize the production efficiency of batch fermentations based on the core idea of manipulating adenosine triphosphate (ATP) wasting.The approach allows considering and balancing different process objectives and exploits constraint-based dynamic modeling to capture the fermentation dynamics.The moving horizon repeated solution of the optimal control problem allows handling large disturbances and online adjustments to changing process conditions.

| Towards efficient bioprocesses: Enforced ATP wasting
Although having high product yields and volumetric productivities is desirable, there is a well-known trade-off between these two parameters.Due to the intrinsic nature of cell metabolism, maximizing the product yield is often linked to less substrate flux directed towards biomass.In turn, this often translates into slower growth rates and lower volumetric productivities in batch fermentations given the role of biomass as the "catalyst" of the process. 8,10There are several approaches to address this trade-off, for example, two-stage fermentations. 8,11rthermore, ATP supply plays a major role as an energy provider for the cell and in regulating cellular processes. 12[14][15][16][17][18][19] Basically, enforced ATP wasting imposes a certain ATP drain in the cell metabolism, for example, by introducing an ATP futile cycle 14 or expressing an ATP-hydrolyzing enzyme such as the F 1 portion of the ATPase. 15,20The cell needs to respond to this drain with an increment in the product flux to counteract the ATP loss.Hence, increasing ATP wasting or turnover should result in increased product yields and specific productivities.4][15][16][17][18][19] This leads to the question on how to address this trade-off in a structured, adaptable, and flexible way.To do so, we propose combining model-based control with constraint-based models, namely dynamic enzyme-cost flux balance analysis (deFBA).For details see Section 2.

| Modeling, optimization, and control of ATP wasting
5][16][17][18] In two-stage processes, cells are grown first without ATP wasting, followed by a production phase catalyzed by growth-arrested cells with ATP wasting.For one-stage processes, ATP wasting is enhanced during the entire process.Naturally, modeling and simulating ATP wasting in fermentations would allow one to understand and optimize the production process.Previous efforts on mathematical modeling of ATP wasting have involved unstructured, somewhat simplified equations with very generic (lumped) parameters.
Most of the approaches have focused on simulating one-stage and twostage scenarios to deepen the understanding. 8The obtained precision of the derived models, however, is so far often limited.
Fine-tuning the ATP turnover in the cell by online optimization and control could provide more flexibility and new opportunities to the biotechnology industry.With the help of computer-aided optimization techniques and tunable gene-expression systems, [21][22][23][24][25] one could adjust the ATP wasting online to achieve desired user-defined trade-offs between product yield and volumetric productivity.As such, this idea belongs to the concept of dynamic metabolic engineering 10 because different levels of ATP wasting would render different metabolic flux distributions in time.

| Tackling process uncertainties, model-plant mismatch, and disturbances via moving horizon control
Many factors in industrial bioreactors (e.g., mass/heat transfer, mixing, etc.) can limit and influence growth and conversion rates. 26Even for highly stable and well-controlled conditions, there remains stochasticity in gene expression and consequently in the observed cell phenotype and process dynamics. 27To account for possible F I G U R E 1 Principle effect of adenosine triphosphate (ATP) wasting on the metabolic flux distribution with net ATP production linked to product formation.Left-hand side: no ATP wasting; growthdominant scenario.Right-hand side: high level of ATP wasting; production-dominant scenario.The red arrow indicates the induced ATP drain model-plant mismatch, uncertainty, and disturbances, we propose introducing feedback control by repeated solution of the optimal control problem in form of model predictive control (MPC) with shrinking time horizon. 28Unlike autoregulation of cellular metabolism, 29 this offers expanded flexibility in the form of in silico feedback control.
The cell/bioreactor is connected to the controller, which is implemented externally in silico, allowing for a higher degree of flexibility, online adjustment, and improved overall control. 30,31C using constraint-based models has been considered, for example, in Reference 32, where a predictive control strategy with online parameter and state estimation based on a deFBA model was developed.
The overall efficiency of this approach has been validated by maximizing ethanol volumetric productivity via adjusting the oxygen uptake rate online in a glycerol fermentation with Escherichia coli.In this regard, the main novelty of our work involves the application of dynamic manipulation of enforced ATP wasting, a generalizable concept that can be employed by biotechnologists towards maximizing batch fermentation efficiency in bioprocesses where the product pathway is coupled to ATP synthesis.We underline the concept considering the batch anaerobic lactate production from glucose by an engineered E. coli strain with knock-out of competitive pathways. 14The ATP wasting activity in the cell is captured by simulating the ATP-hydrolyzing effect of the ATPase enzyme (F 1 -subunit) 20 within the deFBA.To the best of our knowledge, this is the first time that dynamic ATP wasting is tackled and formulated as a model-based optimal control problem.
The contribution of this work is threefold.We present an optimal and predictive control framework of ATP turnover for dynamic metabolic control in batch fermentations.We highlight the applicability of MPC of ATP turnover for dealing with model-plant mismatch, disturbances, and uncertainties.To do so, we use constraint-based modeling in the form of deFBA to better capture the effect of temporal manipulations of the cellular ATP turnover on the fermentation dynamics.The results are in our eyes highly relevant for the bioprocess sector, an industry that has been historically strongly recipe driven.It often operates with predefined control action policies based on trial and error or often oversimplified dynamic models, thus frequently leading to suboptimal process performance. 33e remainder of the article is structured as follows.Section 2 focuses on the constraint-based modeling framework, which provides the required flexibility and descriptive power.In Section 3, we outline our optimal control formulation to maximize process efficiency considering ATP turnover.Section 4 focuses on the MPC formulation regarding ATP turnover to counteract disturbances and model-plant mismatch.We highlight the potential of our optimization and control strategy using the lactate fermentation by E. coli as a case study in Section 5.

| CONSTRAINT-BASED MODELING
We utilize constraint-based models as a base for optimizing the fermentation process.5][36] They combine conservation relations in metabolism based on the stoichiometric matrix of a given metabolic network on a genome-scale level and can also include phenomenological relations describing the overall fermentation process dynamics.
Typically, constraint-based models are underdetermined if one considers only the known balance constraints due to limited insight into the process.To eliminate the undetermined degrees of freedom, an optimization or dynamic optimal control problem with a biologically meaningful objective function is often formulated, and additional constraints are included to further limit the solution space.This allows obtaining sound predictions of the unknown variables, such as metabolic fluxes and, in the case of dynamic versions, of the change of extracellular concentrations in time.One of the key steps herein is the selection of a biological reasonable optimality criterion the cell might optimize.This could be, for example, maximum growth. 37 focus on deFBA, a specific constraint-based modeling framework. 32,38,39It allows capturing changes in the biomass composition due to temporal metabolic adaptions and considers the "cost" of producing such biomass components.deFBA modeling assumes that the biomass is composed of enzymes, ribosomes, and so-called quota compounds whose states are combined in the vector p.We denote by b the vector containing the corresponding molecular weights of the biomass components.Hence, expresses the total biomass dry weight in g=L.Extracellular metabolites z and the biomass components are collected in the molar vector x, leading to the mass balances: Here, S x denotes the stoichiometric matrix of the species in the vector x. Reaction fluxes for exchange, metabolic, and biomass production reactions are collected in the flux vector V.
For simplification, one often assumes quasi-steady-state conditions for the intracellular metabolites m: where S m is the stoichiometric matrix of the species in the molar vector m.
Moreover, metabolic fluxes are constrained by the amount of enzyme and its corresponding turnover number or catalytic constant k cat : where cat i ð Þ accounts for all reactions that an enzyme p i from the set of enzymes  catalyzes.
Additionally, metabolic fluxes can be limited by biologically feasible lower and upper bounds: A minimal fraction φ Q 0, 1 ½ of the biomass dry weight is forced to correspond to quota compounds p Q , which comprises all macromolecules needed by the cell to function properly but that are not explicitly modeled: Depending on the available insight, the set of Equations ( 1)-( 6) is typically underdetermined.To overcome this problem, one can assume that cells evolved in a way that they are always optimizing a certain cost functional, for example, given by: max where F V is the cell optimized objective function over a time horizon t 0 , t deFBA ½ .Note that t deFBA can in principle span to infinity.Frequently, the objective function of the cell in deFBA models is chosen as the maximization of the biomass integral, 32,[38][39][40] which is said to work generally well as long as the cells are not under starvation conditions. 41This leads to the dynamic optimization/optimal control problem subject to constraints: Note that Equation ( 8) is an optimal control problem aiming at determining a function V instead of a vector, which is in general difficult to solve, especially if many constraints and variables are involved. 42

| MAXIMIZATION OF THE FERMENTATION EFFICIENCY IN TERMS OF OPTIMAL CONTROL OF ATP TURNOVER
To optimize the process efficiency, one often considers a subpart of the fluxes as directly manipulable.We assume that the ATPase flux V ATPase can be directly influenced as manipulated variable.We can see from Equation (4) that different V ATPase values can in principle be reached by rational manipulation of the ATPase enzyme concentration level.This can be achieved in reality, for example, by putting the ATPase genes under the control of an inducible promoter.In the context of synthetic dynamic control circuits, different input signals can be used to modulate enzyme expression, ranging from exogenous stimuli (e.g., chemical inducers and light) to process parameters such as pH, oxygen level, and temperature. 10,22,25Naturally, depending on the selected genetic module, a suitable actuator shall be applied that translates the desired optimal ATPase flux V Ã ATPase À Á into an optimal input u Ã ð Þ to the plant.To keep the analysis limited to the general effect of dynamically controlling the ATP turnover, it is assumed in this study that a proper genetic module and actuator are available and well-characterized to achieve the desired flux levels at the specific control actions.
Changing ATP turnovers will result in increasing and decreasing patterns of the product and biomass yields.We formulate the quest for an optimal flux strategy that maximizes the efficiency of the batch process defined with the cost function J x ð Þ assuming, for simplicity, that the batch time t f is fixed: One could consider, for example, maximizing the yield of product on substrate, the volumetric productivity, or a multi-objective cost function. 6,7,32The decision variable in the optimal control problem corresponds, for simplicity of presentation, to the manipulated or regulated metabolic flux V ATPase .Note that in principle one can exploit more manipulated fluxes.
The optimal control problem will determine the optimal ATPase flux.
The deFBA model will consider the ATPase flux as given.Note that in principle there will be couplings between the maximum allowed ATPase flux and the variables appearing in the deFBA model.Those can be integrated via additional constraints in the optimal control formulation.
Note that Equation ( 9) is a bilevel optimal control problem, involving an inner and outer optimization, which is in general difficult to solve. 43ten this is overcome utilizing a discretization of the "manipulated" variables in the upper and lower level, or by approximating the lower-level optimal control problem, for example, by letting t deFBA go to infinity and finding an approximated solution of the stationary problem.

| Remark on state measurements
In formulation (9), initial conditions z 0 , p 0 ð Þ are required to solve the optimization problem.From a practical side, this implies that the states of both z and p need to be known always.However, getting measurements of the intracellular biomass composition is not straightforward.
We do not focus on this aspect here.One might for example use suitable estimators/observers, cf. for example, Reference 32 which exploits a tailored estimator.We use in the example section a tailored estimator.

| SHRINKING HORIZON MPC OF ATP WASTING
The outlined open-loop control of ATP wasting allows, depending on certain economic requirements, to achieve user-defined trade-offs between product yield and volumetric productivity that would guarantee the profitability of the plant.However, as mentioned before, the dynamics of biological systems can be affected by many factors, ranging from intrinsic stochasticity in gene expression to several processrelated variables.To counteract these, often a priori unknown effects, we propose to re-evaluate the optimal control problem during the run of the process.Specifically, we propose to use a shrinking horizon MPC formulation of ATP turnover for dealing with uncertain biological systems.
Basically, MPC is the repeated application/solution of an optimal controller, where the open-loop optimal input is applied only partially.After a certain time, the control action is re-evaluated, considering updated process plants, thus accounting for disturbances and process plant mismatch.Repeated solution of the optimal control problem thus leads to closing the loop and provides the desired feedback. 28Consequently, the manipulated variable trajectory adapts in time, and can thereby reduce the effect of shortterm unknown disturbances, model-plant mismatch, and uncertainties.For the ATP turnover application, we considered the following optimal control problem to be solved at the sampling times t k , where k is the sampling time.We assume for simplicity that t k ¼ kh, k ¼ 0, …, n À 1, that is, equidistant sampling times, where h is the sampling distance.Note that one could allow for non-equal sampling, which is avoided here for simplicity of presentation.For simplicity we also consider that the prediction horizon in the MPC problem spans to the final process time t f , leading to a shrinking MPC horizon spanning from t k to t f , which guarantees in the nominal case: Figure 2 summarizes the resulting shrinking horizon optimal control strategy considering ATP turnover as the manipulated variable.The optimal control problem is solved and the first control action from the predicted optimal trajectory is applied to the plant through a suitable actuator system.Then, state and parameter estimates based on measurements of the process such as biomass dry weight e B and extracellular metabolite concentrations e z are used at each control (sampling) instance to resolve the optimal control problem.We consider the use of an observer or state estimator for the intracellular components b p from the measured states.This cycle is repeated over a shrinking time horizon.

| APPLICATION EXAMPLE: LACTATE FERMENTATION
We consider the lactate synthesis by E. coli, which is coupled with ATP formation under anaerobic conditions making it amenable for the ATP wasting strategy.During the fermentation, ATP is not produced directly along the product pathway (pyruvate to lactate), but rather via glycolysis (glucose to pyruvate).We focus on E. coli KBM10111, which is an engineered strain that has undergone gene deletions of adhE (aldehyde-alcohol dehydrogenase), ackA (acetate kinase), and pta (phosphate acetyltransferase), blocking synthesis of alternative fermentation products.With these modifications, the pathway from glucose to lactate becomes essential to regenerate the redox cofactors required in the glycolysis, thus product formation is necessary for ATP synthesis.Using glucose as substrate, this strain can produce lactate as the main fermentation product under anaerobic conditions, while by-product formation (ethanol, formate, acetate, and succinate) is very low or fully blocked. 14The good performance of the strain makes it favorable from an application point of view.,20

| Modeling
The deFBA framework requires a suitable resource allocation model that encompasses both a metabolic and a biomass-producing part (Table 1).The model derivation for the lactate fermentation followed a similar protocol as suggested by Reference 44; therefore, only the specific assumptions or details will be mentioned.First, a reduced metabolic network of the fermentation was constructed; in this case, using the NetworkReducer algorithm. 45The E. coli genome-scale model ECGS 46 and the phenotype of the strain (growth and extracellular glucose and lactate rates) 14 were given as main inputs for the network reduction.This resulted in the reactions listed under the metabolic part of the model which are also depicted in Figure 3.Note that a reaction for amino acid (AA) production is present since AAs are needed as reactants in the biomass-producing section.Glutamine was taken as a reference AA for the sake of simplicity.Additionally, a reaction (no.9) to simulate the ATP-hydrolyzing activity of the ATPase enzyme (F 1 -subunit) was explicitly included.Catalytic constants were established with information from BRENDA 47 and SABIO-RK 48 databases.In general, k cat s were estimated as median values from the available data.Some corrections were introduced in the catalytic constants of certain enzymes (namely, ptsGHI_crr, pfkA_fbaA, gapA_pgk, gpmA_eno, ppc, and gdhA_glnA) to improve the model fitting.For these enzymes, instead of taking the median value, we selected individual k cat entries from the databases that in our opinion fitted well the experimental data.
To derive the quota reaction, it was roughly estimated from the BioNumbers database 49 that 67% of biomass dry weight was made up of quota elements (DNA, lipids, carbohydrates, non-catalytic proteins plus other small molecules), while the remaining 33% corresponded to catalytic enzymes and ribosomes.Thus, φ Q was set to 0.67 in Equation ( 6).Catalytic constants for biomass-producing reactions were calculated based on the rate of translation by ribosomes (12 AAs/s). 49Mass values of gene products to build the b vector were retrieved from UniProt. 50e deFBA model for the lactate fermentation was validated using experimental data (Figure 4).One should note that the validated scenario does not cover enforced ATP wasting.We used resource balance analysis (RBA) 32 to estimate the initial p 0 ð Þ vector from the known initial biomass dry weight concentration.RBA is based on cellular resource allocation theory. 51It works by maximizing biomass growth while allocating the biomass components accordingly, assuming that the metabolism is in a quasi-steady state for each e B value.
Overall, the glucose profile predicted by the deFBA model shows good agreement with the experimental data.Lactate and biomass concentrations are also well-described, especially at the beginning and end of the fermentation, although the model shows a slight overestimation during the mid-term of the process.Taking into account the reported standard deviations, the model fitting shows sufficient good performance and is used to analyze the effect of the dynamic manipulation of cellular ATP wasting.

| Open-loop maximization of the fermentation efficiency in terms of the ATP turnover
In a first step, we maximized the fermentation efficiency via the openloop optimal control problem as outlined in Section 3 to explore the properties of ATP wasting.
The objective function considered for the optimal control problem was the maximization of the final lactate titer, To solve the bilevel dynamic optimization problem we used collocation based on Lagrange interpolation polynomials. 38The inner-level problem was considered as optimistic and integrated via the resulting Karush-Kuhn-Tucker conditions into an overall mathematical program with complementarity constraints. 52To do so, CasADi 53 and IPOPT 54 were used.The time course of the batch was discretized given a step size h so that h ¼ t f =n.Since the predictive horizon of the optimal control problem spans until t f , we expect that the controller will try to consume all substrate available to maximize lactate concentration in the given time frame.For simplicity, we used t deFBA ¼ t f so that the deFBA model would avoid covering time frames (beyond t f ) with substrate starvation for which, as mentioned before, the biomass integral objective function is not deemed reliable.Eight control actions were considered (n ¼ 8), which turned out to be sufficiently fine discretized.

| Effect of the batch duration time on the average product yield and volumetric productivity
We first examine the influence of the fermentation time, which has a direct effect over the volumetric productivity in batch processes.For a certain final product concentration (amount per volume, e.g., mM), the longer the fermentation takes to reach that concentration, the lower the product volumetric productivity (concentration per time, e. g., mM/h) will be.Additionally, we know that the ATP wasting mechanism increases the product yield at the expense of biomass yield, thereby decreasing the volumetric productivity in batch problem to find a "palette" of process trade-offs between average product yield (Y PS,batch ), biomass yield (Y XS,batch ), and product volumetric productivity (r P,batch ).This resulted in various sets of Y PS,batch , Y XS,batch , and r P,batch for selected t f values (Figure 5).Note that the fermentations metrics were calculated as average estimates for the entire batch and should not be confused with temporal values throughout the process.As expected, all time scenarios presented in Figure 5 corresponded to fermentations with 100% substrate consumption efficiencies.Remark that RBA 32 was used to estimate the initial p 0 ð Þ vector from the given initial biomass concentration in the open-loop optimizations.
Not surprisingly, the maximum volumetric productivity possible was predicted for the scenario with no enforced ATP wasting (V ATPase ¼ 0 at all control action instances).This is indicated in orange in Figure 5.Further t f increments allowed obtaining different non-zero V ATPase sequence actions (as will be shown in later examples) that projected higher batch product yields at the expense of biomass yield and hence volumetric productivity.As was foreseeable, the predicted product yield tended to the maximum theoretical value of 2 mol lactate per mol glucose with increasing t f , matching the non-growth scenario whereby all substrate flux is directed towards the product pathway.In other words, the longer the batch time, the less critical biomass concentration becomes to achieve maximum lactate titer since the low fermentation rates are compensated by higher product yields.Overall, these results highlight the fact that using the ATP turnover as manipulated variable allows obtaining different trade-offs between product yield and volumetric productivity, thus providing flexibility and adaptability to the fermentation operation.

| Fermentation dynamics under enforced ATP wasting
Now, we take a deeper look at the fermentation dynamics under the effect of enforced ATP wasting.For exemplification let us first analyze the scenario for a batch time of 16 h (Figure 6).Later in Section 5.3, we will show the open-loop dynamics for a batch time of 13 h for comparison purposes.The corresponding fermentation metrics for t f ¼ 16h are Y PS,batch ¼ 1:90 mol=mol, r P,batch ¼ 16:5 mM=h, and Y XS,batch ¼ 4:9 g=mol.Compared to the V ATPase OFF scenario (i.e., V ATPase ¼ 0 at all control instances), optimal manipulation of V ATPase in the batch process of 16 h represented, as expected, an 11% enhancement in the process product yield at the expense of about 48% drop in volumetric productivity.Furthermore, the optimization predicted gradual increments of V ATPase over time.Due to the ATP wasting effect, this led to increasing and decreasing temporal values of Y PS and Y XS , which becomes evident when analyzing the biomass and product concentration profiles.For example, biomass had a higher growth rate at the beginning compared to later stages of the fermentation, hence the loss in volumetric productivity.On the other hand, the product concentration surpassed the maximum possible achievable with the V ATPase OFF scenario (for the same initial conditions) as a result of the enforced ATP turnover rates.In summary, the optimizer It is worth stressing that the optimization predicted a gradual increase of V ATPase instead of a two-stage fermentation scenario with a fully OFF-ATP wasting (growth-dominant) followed by a fully ON-ATP wasting (production-dominant) phase.We modified the optimal control problem in order to force the controller to apply a step-like (OFF-ON) V ATPase trajectory aimed at comparing the performance against a gradual V ATPase increase.However, doing so always led to infeasible solutions.Such a two-stage scenario is infeasible with the deFBA framework because the cell cannot instantaneously reallocate resources to allow the required ATPase enzyme accumulation needed to enable the desired V ATPase flux values during the V ATPase ON-stage.
In contrast, we think that more gradual V ATPase increments, as predicted by the controller, were feasible because the cell can slowly produce and accumulate the required ATPase enzyme without compromising resources for the production of other vital enzymes and biomass components.
To support the previous argument, we plot in Figure 7

| Scenario 1 nominal case
We first assumed that there is no model-plant mismatch, that is, that the model exactly describes the fermentation, and that states are known during the complete process.As expected, in this case, the optimal V ATPase sequence trajectory resulting from the open-loop optimization matches the closed-loop MPC one as we use a shrinking horizon approach, following Bellman's principle of optimality. 55The fermentation dynamics is shown in Figure 8.
In the context of these new results, it is worth highlighting that the predicted optimal V ATPase changes with different selected batch times to maximize lactate titer; for example, compare the open-loop predictions in Figure 6 (batch time of 16 h) and Figure 8 (batch time of 13 h).The observed trends are more or less similar; however, the delivered V ATPase for the process with t f ¼ 13h is, for example, comparatively less intense at the beginning which allows for more growth.
This means that the process with t f ¼ 16h discussed in Section 5.2 can favor production sooner via enforced ATP turnover because the resulting lower biomass accumulation, and thus slower fermentation rates, can be compensated by the allowed longer process time.

| Scenario 2 model-plant mismatch
The deFBA model assumes that enzymes are working under substrate saturation conditions.That is why, as captured by Equation ( 4), metabolic fluxes can be modeled based on the product of the catalytic constants and the enzyme amounts.However, if this assumption does not hold for certain enzymes, the controller might choose suboptimal control actions.To introduce model-plant mismatch due to, for example, substrate-unsaturated enzymes, k cat s for reactions 3, 5, 6, 12, and 13 were scaled-down by a factor of 0.75.Furthermore, the minimum quota fraction requirement by the cell was increased to φ Q ¼ 0:69.
Overall, this slows down the fermentation rates but does not affect the model stoichiometry.While the controller uses the nominal parameters as listed in Table 1, the plant simulation includes the modified parameters which, for simplicity, are fixed during the full process Figure 9 shows the performance of the MPC subject to the model-plant mismatch assuming full state information during the operation (scenario 2).In real applications, however, having online measurements of the intracellular biomass composition is not an easy task with the current state-of-the-art technologies.

| Scenario 3 model-plant mismatch and state estimation
Having the same model-plant mismatch as in scenario 2, we estimate the biomass composition using RBA 32  In summary, MPC showed potential for feedback control of the cellular ATP turnover and provides additional robustness with respect to model uncertainty.One option to make the metabolic model more accurate could be to use kinetic relations 57 that describe metabolic fluxes based on the change in intracellular metabolite concentrations over time m t ð Þ.This would allow, for example, covering different substrate saturation levels for the catalytic enzymes.However, this would also come at the expense of introducing non-linearities and increasing the complexity of the model due to the introduced kinetic functions and the need for an explicit description of the m-dynamics which currently the deFBA assumes for simplicity to be in quasi-steady-state conditions.
Moreover, the computational cost of solving the optimal control problem could increase considerably, thus hindering its applicability.

| CONCLUSION
We presented an optimal and predictive control strategy for the Unlike static metabolic engineering, considering ATP wasting in a model-based optimal control formulation allows one to consider trade-offs between product yield and volumetric productivity.We believe that the outlined approach can contribute to making industrial biotechnology more flexible and adaptable.Furthermore, metabolic coupling of product and biomass synthesis (e.g., by balancing energy cofactors) is a widely adopted design principle of microbial cell factories.For such cases, our control strategy of ATP wasting should be applicable.For these reasons, optimal dynamic manipulation of cellular ATP turnover may become a powerful add-in tool for the bioprocess and metabolic engineering sectors.
Future work focuses on kinetic and constraint-based modeling frameworks that consider the dynamics of tunable gene expression systems for the control of regulated enzymes such as the ATPase.This allows one to directly consider the enzyme expression which in turn changes the target metabolic flux.Similarly, we are exploring continuous and fed-batch configurations for the dynamic control of the ATP turnover.Finally, we are interested in the design of an observer for the biomass composition with improved performance compared to the RBA.

F I G U R E 4
Simulation of the dynamic enzyme-cost flux balance analysis (deFBA) against experimental data 14 of glucose, lactate, and biomass concentrations.Initial conditions: B 0 ð Þ¼0:05 g=L; z GLC 0 ð Þ¼20:8 mM; z LAC 0 ð Þ¼0 mM.Batch time: t deFBA ¼ 9:7 h F I G U R E 5 Effect of different batch times (t f ) on the average batch yields (Y PS,batch , Y XS,batch ) and product volumetric productivity (r P,batch ).Predictions from open-loop optimizations (no model-plant mismatch); 100% substrate consumption.Orange: non-ATP-wasting scenario.Initial conditions: B 0 ð Þ¼0:59 g=L; z GLC 0 ð Þ¼139 mM; z LAC 0 ð Þ¼0 mM was able to find a good balance between temporal improvements in product yield and reductions in biomass yield to drive metabolism towards the maximization of lactate titer.
the change in enzyme distribution as a percentage of biomass over time for the open-loop scenario presented in Figure6.One can clearly observe the gradual accumulation of the ATPase enzyme needed to enable the predicted V ATPase fluxes by the controller.Furthermore, using the case with no enforced ATP wasting as a reference, we can see how the enzyme pool of the cells with manipulated V ATPase changes comparatively different over time.This happens because the cells need to dynamically reallocate resources from other enzymes so that they can F I G U R E 6 Open-loop optimization (OLO) results for a batch time t f ¼ 16 h against the case with no manipulation of V ATPase (NC: no control, i.e., V ATPase ¼ 0 at all control instances).The dotted lines extend the end-points of the NC fermentation for ease of comparison.Initial conditions: B 0 ð Þ¼0:59 g=L;z GLC 0 ð Þ¼139 mM; z LAC 0 ð Þ¼0 mM.No model-plant mismatch F I G U R E 7Enzyme allocation dynamics for the scenarios presented in Figure6.Refer to Table1for nomenclature cope with the required ATPase production.For example, the concentrations of enzymes such as pfkA_fbaA and gpmA_eno decrease considerably with increasing accumulation of the ATPase enzyme.Mathematically speaking, the flux values in the deFBA model have an upper bound dictated by the product of the enzyme concentrations and the catalytic constants (Equation (4)).Therefore, one cannot demand that the cells reach a sudden high V ATPase flux right after a fully OFF-ATP wasting stage, as would be the ideal concept with a two-stage fermentation, because there would not be sufficient enzyme present to catalyze the subsequent fully ON-ATP wasting phase.In other words, enzyme allocation is a dynamic rather than an instantaneous phenomenon.In a future work, we will focus on how exactly one can fine-tune the expression and thus the amounts of catalytic enzymes in the cell to reach the predicted flux values for dynamic metabolic control applications.5.3 | Closed-loop maximization of fermentation efficiency using shrinking horizon predictive control casesThe outlined shrinking horizon MPC strategy allows accounting for model-plant mismatch and disturbances, which is not possible if the optimal input is applied open-loop.To evaluate the scheme, we consider three scenarios.For all MPC simulations, a batch time t f ¼ 13h is used which, according to the open-loop optimal control analysis in Figure 5, corresponds to Y PS,batch ¼ 1:87 mol=mol, r P,batch ¼ 20 mM=h, and Y XS,batch ¼ 6:3 g=mol.This underlines the flexibility of the proposed approach as it allows one to select a priori trade-offs between product yield and volumetric productivity under different economic contexts.Note that RBA 32 was used to estimate the p 0 ð Þ vector from the corresponding biomass concentration to initialize all control cases.

F I G U R E 8
Performance of the MPC scenario 1 (S1).V ATPase sequence profile for both MPC S1 and the open-loop system (OLO); and concentrations of glucose, lactate, and biomass for MPC S1.Trends of the OLO system were identical.Initial conditions: B 0 ð Þ¼ 0:59 g=L; z GLC 0 ð Þ¼ 139 mM; z LAC 0 ð Þ¼ 0 mM.Batch time: t f ¼ 13 h.No model-plant mismatch duration.
as state estimator.It requires the measured biomass concentration and applied V ATPase .The estimated p vector is used as the initial condition p t k ð Þ for the MPC shrinking horizon.Note that other estimators, such as moving horizon estimation 56 can be used.The results are shown in Figure 9, scenario 3. The MPC schemes in scenarios 2 and 3 were able to account for the introduced slower growth rate by delivering a less intense but still increasing V ATPase sequence trajectory in relation to the open-loop system.As already outlined, higher ATPase fluxes are linked to higher product yields and lower biomass yields.Therefore, we see more biomass accumulation in the middle-to-end term of the batch due to the less intense ATP wasting strength applied.Because of the presence of more biocatalyst in the processes with MPC, all glucose in the medium was consumed.On the contrary, there was only 90% glucose consumption efficiency with the open-loop system (i.e., without applying corrective actions to address model-plant mismatch).Alternatively stated, the increased product yields with the open-loop system did not compensate for the decreased biocatalyst concentration.The RBA estimation in scenario 3 caused the controller to choose a less intense V ATPase sequence trajectory compared to scenario 2. As a consequence, there was more biomass accumulation and slightly less product formation.
dynamic manipulation of the cellular ATP turnover.As manipulated variable for the control problem, the metabolic flux through an ATPconsuming mechanism was considered.Constraint-based modeling, namely deFBA, was used to describe the fermentation dynamics, while model-plant mismatch was handled employing a shrinking F I G U R E 9 Performance of the MPC scenarios 2 (S2) and 3 (S3) against the open-loop system (OLO).V ATPase sequence profile; and concentrations of glucose, lactate, and biomass.Initial conditions: B 0 ð Þ¼0:59 g=L; z GLC 0 ð Þ¼139 mM; z LAC 0 ð Þ¼0 mM.Batch time: t f ¼ 13 h.Model-plant mismatch considered horizon MPC approach.Lactate fermentation from glucose by E. coli underlines, as an industrially relevant case study, the performance and properties of the approach.
Reactions of the resource allocation model and relevant parameters for the lactate fermentation case study Model predictive control strategy of the adenosine triphosphate (ATP) turnover T A B L E 1 G3P, glyceraldehyde 3-phosphate; G6P, glucose 6-phosphate; GLC, glucose; LAC, lactate; MAL, malate; NAD, nicotinamide adenine dinucleotide; NADH, NAD-reduced; OAA, oxaloacetic acid; PEP, phosphoenolpyruvate; PYR, pyruvate; Q, quota; R, ribosome; SUCC, succinate.a It is assumed for Equation (4) that ribosomes catalyze all biomass-producing reactions.b Gene IDs used for referring to the enzymes.Underscore symbols are used to indicate which enzymes have been lumped in the model.c Reaction 34 already given in terms of g=L of Q; therefore, b Q ¼ 1 for simplification purposes.