Multi‐objective Bayesian algorithm automatically discovers low‐cost high‐growth serum‐free media for cellular agriculture application

Abstract In this work, we applied a multi‐information source modeling technique to solve a multi‐objective Bayesian optimization problem involving the simultaneous minimization of cost and maximization of growth for serum‐free C2C12 cells using a hyper‐volume improvement acquisition function. In sequential batches of custom media experiments designed using our Bayesian criteria, collected using multiple assays targeting different cellular growth dynamics, the algorithm learned to identify the trade‐off relationship between long‐term growth and cost. We were able to identify several media with >100% more growth of C2C12 cells than the control, as well as a medium with 23% more growth at only 62.5% of the cost of the control. These algorithmically generated media also maintained growth far past the study period, indicating the modeling approach approximates the cell growth well from an extremely limited data set.


INTRODUCTION
In this work we applied an active learning approach to design serum-free media, which is a necessary precondition to the development of cellular agriculture. In cellular agriculture, animal products are grown in bio-reactors [1] in an attempt to be more resource efficient and ethical than traditional animal agriculture. Because of the enormous developed an albumin-enriched B8 formula for the proliferation of bovine satellite cells. A shortcoming of this approach, which is acknowledged, was that they only looked at a very limited set of variables that may affect cell growth. One way of sifting through this large experimental design space is active learning, where data is collected to maximally inform the next decision. This process repeats itself until the user desires to stop. Previous work by [5] (bacteria culture) and [6,7] (C2C12 cell culture) demonstrate that active learning techniques are very resource-efficient in optimizing experimental systems with many variables. In this manner an active learning algorithm mimics how real scientists observe data and alter future experiments based on updated expectations.
The serum-free medium itself must contain vitamins, trace elements, carbohydrates, amino acids, and salts, with additional proteins that replace serum [8]. These serum-replacing components are particularly expensive and militate for a multi-objective optimization (MOO) approach to optimizing cell culture media in order to explore the trade-off between cost and cell growth. In MOO problems, there is often no single point that dominates the entire design space, so becomes a matter of finding sets of points that fall on the trade-off curve. Cell culture media design, particularly for cellular agriculture, is inherently a MOO problem because improved growth is often found with expensive components [9] used central composite designs to evaluate the effect of several components on a desirability function parameterization of lipid content, carbohydrate consumption and biomass accumulation. In work done to optimize cytokine dosing [10], trained a regularized polynomial model and used a derivative-free optimizer to find the conditions that maximized a desirability function of cell populations. In work by [11], genetic algorithms vector evaluated genetic algorithm (VEGA) and strength Pareto evolutionary algorithm (SPEA) were used to maximize chemical conversion while maintaining biomass of the cyanobacteria organism [12] used a genetic algorithm MOGA to maximize plant culture biomass and minimize system cost.
Here, we meld MOO with active learning using Bayesian optimization (BO) in a multi-objective Bayesian optimization (MOBO) approach to design media experiments. Specifically, we utilized the noisy expected hypervolume improvement function described in [13] to rank sets of experimental conditions that could be done in batches in a wet lab. We have previously used a multi-information source Gaussian process model described in [14] to successfully optimize cell culture media with multiple assays to robustly describe long-term cell proliferation [7]. We will extend this model again to model long-term cell growth in our serum-free system and use the models predictions to solve the MOBO problem. In Section 2 we will discuss the laboratory materials needed to solve our media design problem, including the cells and chemicals needed, as well as the mathematical derivation of the acquisition function used to solve the MOBO problem. Then in Section 3 the results will be presented, followed by Section 4 with the discussion of the implications of the results.

Serum-free cells
Recent work by [15] shows that merely seeding cells in serum-fre e media without additional preparation is insufficient in optimizing serum-free media, as the cells do not have time to adjust to the environment. A more robust approach is to slowly adapt a cell line to serum-free conditions over multiple passages [16].

Media components
The media design space was based on the E8/B8 formulation [3] comprised of basal medium, FGF2, TGF 1, insulin, transferrin, ascorbic acid, and sodium selenite. We chose to supplement this with nine growth factors which have either been found to improve cell proliferation in [7] or by expert opinion. Because the basal component is comprised of > 30 individual components it was broken down into groups based on function in cell culture. These component groups (essential and non-essential amino acids, vitamins, salts, trace metals, DNA precursors, fatty acids) were TA B L E 1 Serum-Free Medium Design Space|all components were stored as per manufacturers instructions in stock solutions described in Appendix 1. The concentration (mg/mL) of all media was between the minimum and maximum listed. varied during the optimization campaign by the algorithm which we discuss in later sections. Components believed to have significant effects on growth (carbohydrates, ascorbic acid, sodium selenite) were individually varied as well. NaCl was separated from the general salts group because it has a large effect osmolarity. All components and groups of components are shown in detail in Table 1 and Appendix 1.

Cell growth experiments and assays
We utilized a multi-information source Bayesian model to combine "cheap" measures of cell biomass with more "expensive" but higher quality measurements in order to predict long-term medium performance. We refer to the simpler and cheap assays as "low-fidelity", and more complex and expensive assays as "high-fidelity". In this work, low-fidelity experiments were conducted with Alamar-Blue and LIVE stain, common indirect chemical indicators of cell proliferation. High-fidelity experiments were conducted by cell counting with trypan blue. To start a set of experiment, vials of adapted C2C12 cells were thawed to 25 • C and the freezing medium was removed by centrifugation at 1500 × g for 4 min. The centrifuged cell pellet was resuspended in 17 mL of E8 (Gibco) and placed on 15 cm sterile plastic tissue culture dishes (Cellstar, Greiner Bio-One). Cells were incubated at 37 • C and 5% CO 2 for 48 h. Cells were harvested using tripLE solution (Gibco), diluted in PBS, and counted using a Countess II with trypan blue exclusion and disposable slides (Invitrogen).
With the known concentration of cells, 96 well plates (for the low-fidelity IS) were seeded at 2000 cells/well (25 L of PBS/cell inoculum and 75 L of test medium) and 6 well plates (for the high-fidelity IS) were seeded at 60,000 cells/well (750 L of PBS/cell inoculum and 2250 L of test medium). The final density of both formats was 20,000 cells / mL of PBS and medium.
After 72 h, all wells were measured using the different fidelity methods. AlamarBlue (Invitrogen) and LIVE stain (Biotium) assay required addition of a chemical indicator into the 96 well plates followed by collection of absorbance or fluorescence signals (this was done using Molecular Devices, ImageXpress Pico) according to manufacturer instructions. A Passage 1 cell count was also done in the 6 well plate format using the automatic cell counter. However, these three ways of measuring growth may not represent the long-term proliferation required in cellular agriculture because (1) they were only exposed to the media for 72 h and (2) were not exposed to an additional passage in their custom media which may affect growth. Therefore, a high-fidelity experiment is conducted where the Passage 1 cells are re-seeded for an additional 72 h and counted again to create a fourth and final Passage 2 metric of proliferation. This additional 72 h period is why it is considered a long-term cell growth metrics, but also why it is more tedious to use to optimize a complex media. We now move on to how to model the high-fidelity growth data using low-fidelity interactions as well as how to rank the quality of a medium.

Basics of BO
In standard BO, a function is modeled using a Gaussian Process (GP) [14], characterized by a mean 0 and covariance Σ where ( ) ∼ ( 0 , Σ). This "prior" influences the function through the covariance Σ and prior mean 0 , which models the relationship between any two points and ′ . We have chosen the squared exponential kernel because it performed well in our previous work [7].
If we collect data points of inputs = [ 1 … ] and outputs = [ 1 … ] from the process ( ) = ( ) + we get the posterior distribution ( )| , ∼ ( ( ), Σ( , )) where the mean and variance of ( ) are given by Equations (2) and (3) respectively for homoscedastic noise Σ = 2 × with process noise variance 2 . Notice that this prior is parameterized by hyper-parameters 0 , , , and . These are "learned" from the data by maximizing the log-likelihood function (which will not be discussed here for brevity).
We introduce fidelity information by modifying Equation (1) with an indicator function 1 ≠0 1 = ′ , where = 0 indicates an experiment is high-fidelity. Now, an experiment not only contains media concentrations , but a fidelity code . By applying Equations (4) to (2) and (3), we can make predictions using concentration and fidelity . We use the same kernel architecture as Equation (1) to model low-fidelity Σ ( , ′ ) and high-fidelity covariance Σ 0 ( , ′ ).

MOBO acquisition function
Turning now to how to rank a set of experiments, we have chosen hypervolume metric ( ) to rank the quality of media combinations based on = 2 criteria, predicted growth ( ) and cost ( ). To compute cost use where is a scaled marginal cost of each media component whose coefficients can be found in Table 1. If the th output (to maximize) is ( ) (use Equation (2) to solve this at = 0 if references growth and − ( ) if references cost) relative to a minimum reference point then ( ) is the product of ( ) − for each output [17]. The "+" operator in Equation (5) sets ( ) = 0 if ( ) − ≤ 0 (this acts as a threshold).
To compute the "improvement" in Equation (5) we reformulate the above expression into the product of the minimum between a max-value called , and ( ) [13] where ( ) = { , ( )}. As discussed in the cited Daulton papers, a box decomposition algorithm can be used to quickly compute ( ) by breaking down the above computation into a piece-wise integration across rectangles defined by upper and lower vertices and . We numerically integrate over the rectangles to get the approximation of the hyper-volume improvement function ( ).
Because we can run multiple experiments in a single batch, we can again reformulate ( ) into the "multi-point" ( ) where we wish to predict the best set of experiments . This can be done using the inclusion-exclusion principle for overlapping sets. In practice, this means summing across points ∑ =1 (−1) +1 and modifying the improvement calculation to incorporate all subsets of the proposed candidate pool of size for = 1 ⋯ . This additional calculation prevents double counting of any overlapping hyper-volume sets. Note that MC involves generating a fixed set of normal random numbers ∼ (0, ) and sampling the random normal process using the "reparameterization trick" [18] where the prediction is sampled as = ( ) + ( ) with Cholesky Decomposition of the covariance matrix Σ( , ) = ( ) ( ) . Pushing these samples through ( ) and ∇ ( ) allows us to solve * = ( ) using the multi-start L-BFGS-B optimization algorithm. This is an optimizer that uses function evaluations ( ) and gradients ∇ ( ) to approximate the Hessian matrix of double derivatives ∇ 2 ( ). This approximation speeds up solving * = ( ) and is commonly used in MOBO and machine learning methods. As we wish to constrain our experiments to achieve some minimum level of growth so as not to waste experimental effort in regions of the design space that cannot support cells, we modify with temperature parameter = 10 −3 . We finally arrive at Equation (6) which will be the acquisition function to be optimized throughout this work.
An example of ( ) and ( ) is plotted in Figure 1 for glutamine and pyruvate (data collected in this work). The GP model was used to predict the cell growth of the glutamine-pyruvate design space, which was then used to predict ( ) and the feasibility score ( ). Notice optimizing ( ) may not correspond to maximizing the feasibility of the experiment. This is because cost and uncertainty reduction is considered in ( ) and not ( ).

MOBO algorithm
The MOBO algorithm that designs optimal experiments is shown in Figure 2. After collecting some initial data from a variety of information sources, the model was trained and * found using multi-start L-BFGS-B for some maximum allowable number of experiments. Because we want to optimize at the high-fidelity (Passage 2) all calculations in the MOBO algorithm are done using = 0 (see Section 2.4). With * , we now find the optimal fidelity to sample. We started by defining the number of highfidelity samples we are willing to measure 0 < . ( ) was calculated using Equation (6) for all combinations ( 0 ) in * , and the dominant combination of experiments was allocated to the high-fidelity metric. The remaining − 0 experiments were allocated to the low-fidelity set of metrics.
We started our MOBO algorithm in the serum-free experiments by initialization with 10 Latin Hypercube designs [19]. The algorithm then allocated = 15 experiments with 0 = 3 high-fidelity and − 0 = 12 lowfidelity experiments using the combinatorial heuristic described above. This was repeated for 12 batches of experiments, where a batch is defined as a single group of experiments designed by the MOBO algorithm. Because of the enormous time-cost of measuring biological replicates of 0 cell counts for two passages individually, it was assumed that an averaged technical replicate would capture the underlying trends of the system. As the results will show, this did not appreciably detract from the quality optimal media found even over multiple passages. To further bias our experiments towards high growth regions of the design space, after nine batches of experiments F I G U R E 2 MOBO Algorithm|this loop describes the MOBO algorithm to maximize ( ) that describes the value of a given set of experiments given 0 high-fidelity and − 0 low-fidelity IS samples per batch of experiments. After each batch, the process is repeated until the process is optimized or resources are exhausted. Notes: To increase the presence of high growth conditions, after batch four and nine was increased from 0.5 to 0.75 and 1.0, respectively. The minimum standardized variance was thresholded at 2 = 0.02 but this needed to be changed to 0.05 to reduce numerical stability issues with optimizing ( ). IS, information source; MOBO, multi-objective Bayesian optimziation.
we ran an additional high-fidelity experiment solving * = ( ) as outlined in [20] where This is equivalent to maximizing the expected improvement of a single experiment of a noisy function without consideration of cost.

Computational validation of MOBO method
The MOBO algorithm was tested on the computational test problems introduced in [7] with an additional linear cost function that turned the single-objective optimization problem into a MOO problem. The results are in Appendixes 3 and 4. ( ) performed as well as previous metrics (a desirability function). Because the hypervolume function has fewer controllable design parameters, it was chosen for this novel system. Furthermore, empirical studies of the hyper-volume [17] and noisy-hypervolume [13] acquisition function indicate that it is superior to a wide variety of MOO and MOBO solvers on synthetic and data-based optimization problems.

Experimental validation of MOBO method
The most prominent result from the application of the MOBO algorithm to the serum-free experimental system was the steady improvement in both hyper-volume and the Passage 2 metric in Figure 3A. Figure 3B shows the trade-off between cost and growth which is important for designers that need multiple options to consider. Some of the interesting media designs are highlighted in Table 2. Only one medium (CBCB-0) dominated the control medium in both growth and cost, resulting in 23% more growth at 62.5% of the cost of the control. CBCB-0 had notably lower concentrations of major growth factors like insulin, transferrin, FGF2, and TGF 1 and higher concentrations of progesterone, estradiol, IL6, and LIF. CBCB-1 was another interesting medium that had 78% more growth at only 25% additional cost compared to the control. This was due to higher concentrations of the growth factors that CBCB-0 lacked. Finally, CBCB-2 and CBCB-3 had a 112% and 184% improvement in growth at an increase in cost of 62% and 71% over control respectively. CBCB-2 and CBCB-3 had even higher concentrations of both the insulin, TA B L E 2 Optimal Media|groups of media that lie on or near the trade-off curve. Only CBCB-2 was found by maximizing ( ) rather than ( ). The concentration (mg/mL) of all media was between the minimum and maximum listed in Table 1. The cost is a unitless metric of relative economic cost of each component or group. transferrin, FGF2, and TGF 1 growth factors, while also elevating the concentration of all factors from progesterone to Pigment epithelium-derived factor (PEDF).

Long-term proliferation
We then tested all the highlighted media for five passages to assess the ability of our Passage 2 high-fidelity metric to mimic longer-term effects of the media on growth (the effect of passaging and attachment that are difficult or impossible to evaluate with the low-fidelity data sources). Figure 4 shows the fold-increase in cells (counted using the automatic cell counter) over the initial 60,000 cells seeded over five sequential passages. CBCB-2 performed the most robustly over time but all designed media performed as well as or better than the control. We also compared our control to commercial E8 and found no significant differences in growth over five passages (data not shown).

Sensitivity analysis
We now use sensitivity analysis to understand what highfidelity behavior is being captured given the data. One way of doing this is to run Step 2 of the MOBO algorithm described in Figure 2 to find a single = 1 optimal medium combination at random starting locations (optimizers like L-BFGS-B require a starting point) and observing the results from several runs. First, we maximized ( ) under the constraint that ( ) ≥ in order to observe what sets of media combinations are predicted to produce more growth than the control while penalizing high cost. Next, we maximized only ( ) in order to observe what media might maximize growth regardless of cost. The two distributions are plotted in Figure 5. The max growth condition had generally higher concentrations of most growth factors but not all basal components. This confirms the previous section where higher growth was achieved through higher growth factor concentrations, particularly transferrin, FGF2, TGF 1, EGF, TGF 3, and PDGF. Figure 5 only tells us what the model thinks are the best conditions and not the relative magnitude of each factor on growth. As a further means of quantifying this, we computed an "integrated variogram"-metric using the VARS technique described in [21,22] (Figure 6) for each component. This VARS sensitivity analysis suggests that FGF2, IL6, TGF 1, and several basal components had significant effects on growth. This mostly confirms the previous section that FGF2, TGF 1, and several other growth factors had a large effect on growth, but it is impossible to say anything more suggestive than that.

DISCUSSION
The MOBO algorithm was successful because a robust, long-term data set was built over time, improving the model as more data was collected. Additionally, ( ) was tailored to generate high-value experiments near the tradeoff curve between cell growth and media cost. A separate constraint function ( ) translated our need to primarily search for high-growth designs into a mathematical function, as we expected most of the design space to not support cell growth. Some shortcomings of this work are that (i) we did not compare our MOBO method to an equivalent design of experiments (DOE) method, though we have previously shown similar methods are significantly more efficient than traditional DOEs such as response surface methods [5,7]. Additionally, (ii) Figure 4 indicates media performance tended to decrease over time. This could be due to morphological changes, physical damage due to passaging, or accumulation of toxins and waste products. Clearly, our Passage 2 metric was not enough to fully predict the changing dynamics of cell growth over greater than two passages, though it did so reasonably well given the significant savings in experimental time and resources.
(iii) Separate experiments with bovine satellite cells (data F I G U R E 6 Mid-level VARS variogram|ordered by highest to lowest for the ℎ = 0.3 integrated variogram. The height of the bar indicates the relative importance of that component between randomized synthetic media sampled at most with 30% Euclidean distance-differences in concentration. The plot was generated using 1000 random samples in the design space (whereas the methods described in [21,22] utilize the STAR method).
not shown) also indicated that none of the highest performing media supported growth over multiple passages, thus limiting the generalizability of our media to only our C2C12 cells adapted to commercial Essential 8 without Matrigel. However, such a result does indicate the need to re-optimize media and environmental conditions when studying new cell types or cells with significant genetic or metabolic changes, as such our methods could prove even more useful. This is particularly the case in industrial biotechnology applications such as cellular agriculture where new products and processes are being introduced at a rapid pace. Finally, (iv) C2C12 cells are much easier to culture over multiple passages compared to cells that might be used in cellular agriculture, such as animal satellite cells, therefore our exact methodology for preparing the cell banks (such as the preconditioning portion of the study) would have to be customized for less robust cell lines. Attachment factors or Matrigel might be required to allow the cells to proliferate. Different cell lines also have different metabolic needs which might be stimulated with different signaling pathways, and thus require growth factors not described in this study. Industrial applications would also require many more cells than what is generated from two passages, militating for a higher "high-fidelity" assay that might preclude our MOBO method. These facts underscores the necessity to pair this kind of black-box approach with more traditional "white-box" where cellular processes that cannot be easily modeled for learned using BO methods are understood using traditional biological techniques (such as cell isolation, cloning, and omics analysis). In general, the MOBO algorithm was able to design media according to the objective function we picked for this system. We were able to identify several media with > 100% more growth relative to the control, as well as a medium with 23% more growth at only 62.5% of the cost of the control (CBCB-0). This allows for further, more principled, experiments to be made to accompany and expand on the discoveries made in this study such as spent-media analysis and morphological studies of the cells after exposure to our media. The MOBO algorithm was also able to discover novel media, such as CBCB-2, well-suited to long-term C2C12 growth far beyond the limits of this study. This would make it an interesting candidate for large scale production such as in cellular agriculture. Further work should be performed on correlating biomarkers and morphological attributes to cell differentiation and proliferation, both to improve the robustness of predictions and to simultaneously optimize proliferation and differentiation. Even without these improvements, this work is still relevant to those interested in quickly optimizing their media formulations, generally in the serum-free case, and particularly in the case of difficult-to-measure objectives such as long-term cell growth.

PRACTICAL APPLICATION
The field of cellular agriculture, whereby animal products are grown in bioreactors for food production, has the potentially to reduce the negative environmental and ethical externalities of meat production. In this work, we design a serum-free medium for such applications using an active learning BO method for the low-cost proliferation of C2C12 mammalian muscle cells. The algorithm automatically discovers optimal combinations of media components by allocating a battery of growth assays within a closed learning loop. Through this process, several promising novel media designs were discovered which demonstrated robust cell growth at low cost.

A C K N O W L E D G M E N T S
The authors have nothing to report. Open access funding enabled and organized by Projekt DEAL.

C O N F L I C T O F I N T E R E S T S TAT E M E N T
We have no conflicts of interest to disclose.

D ATA AVA I L A B I L I T Y S TAT E M E N T
The data that support the findings of this study are openly available in at https://github.com/ZacharyCosenza/ GradStuff_Cosenza.