Machine Learning‐Driven Bioelectronics for Closed‐Loop Control of Cells

From the simplest unicellular organisms to complex animals, feedback control based on sensing and actuation is a staple of self‐regulation in biological processes and is a key to life itself. Malfunctioning of this control loop can often lead to disease or death. Bioelectronic devices that interface electronics with biological systems can be used for sensing and actuation of biological processes and have potential for novel therapeutic applications. Due to the complexity of biological systems and the challenge of affecting their innate self‐regulation, closing the loop between sensing and actuation with bioelectronics is difficult to achieve. Herein, bioelectronic proton‐conducting devices are integrated with fluorescence sensing using machine learning to provide closed‐loop control of bioelectronic actuation in living cells. Proton‐conducting bioelectronic devices control pH in a microfluidic system that houses pluripotent mammalian stem cells. This pH control affects the membrane voltage (Vmem) of the cells that is measured using genetically encoded fluorescent Vmem reporters. In this fashion, proof‐of‐concept real‐time control of Vmem toward a desired set‐point is demonstrated. Given the importance of Vmem in cell function, differentiation, and proliferation, this proof‐of‐concept opens up many possibilities in bioelectronic closed‐loop control of cell systems.

From the simplest unicellular organisms to complex animals, feedback control based on sensing and actuation is a staple of self-regulation in biological processes and is a key to life itself. Malfunctioning of this control loop can often lead to disease or death. Bioelectronic devices that interface electronics with biological systems can be used for sensing and actuation of biological processes and have potential for novel therapeutic applications. Due to the complexity of biological systems and the challenge of affecting their innate self-regulation, closing the loop between sensing and actuation with bioelectronics is difficult to achieve. Herein, bioelectronic proton-conducting devices are integrated with fluorescence sensing using machine learning to provide closed-loop control of bioelectronic actuation in living cells. Proton-conducting bioelectronic devices control pH in a microfluidic system that houses pluripotent mammalian stem cells. This pH control affects the membrane voltage (V mem ) of the cells that is measured using genetically encoded fluorescent V mem reporters. In this fashion, proof-of-concept real-time control of V mem toward a desired set-point is demonstrated. Given the importance of V mem in cell function, differentiation, and proliferation, this proof-of-concept opens up many possibilities in bioelectronic closed-loop control of cell systems.
capacitors have achieved transient V mem control in oocyte cells. [5] The ability to control V mem for long term is an essential enabling step to the modulation of cell function, metabolism, and morphogenesis, especially in bioengineering, regenerative medicine, and synthetic morphology applications. [17] Controlling cells with bioelectronics is difficult due to the complexity of cellular response to changing environmental conditions. Cellular processes are hard to model due to noise and uncertainty. [18] This complexity limits most examples of biocontrol to simplified systems such as synthetic gene networks engineered with feedback to achieve noise rejection and adaptation. [19] As the complexity of biological systems scale up, the challenge of feedback design increases. Without a model, a standard approach to bioelectronic control would require information a priori of how the biological variable will respond to an electronic stimulus. [20] This information is often not available. In addition, such an approach does not account for potential adaptation in system response or unknown changes in the environment or device response. For example, the commonly used controller in practice, known as the proportional-integralderivative (PID) controller, might not function properly when the system under control is exposed to the uncertainties, nonlinearities, and unknown disturbances caused by undesired phenomena such as varying environmental conditions. [21] Machine learning (ML) has successfully controlled biological systems when offline training and large datasets are available a priori. [22] However, deep-learning ML methods such as deep neural networks, deep recurrent neural networks, and generative adversarial networks are slow to update in response to system changes. Real-time ML-based methods, such as adaptive ML algorithms, in combination with tools from control theory learn and respond to different operational conditions [23,24] and are able to work with nonlinearities, noise, and system uncertainty that are typical of cells or bioelectronic devices. Here, we leverage a single-layer neural network with a "sense and respond" [25] composed of radial basis functions (RBFs). RBFs act as universal approximators [25] and are fast converging, thus ideal for real-time control applications. [24,26] Here, we demonstrate an array of bioelectronic proton pumps [27] merged with an adaptive ML-based controller that can provide control of V mem for an extended amount of time ( Figure 1). In brief, a bioelectronic proton pump array adds or removes H þ from solution and changes [H þ ] in proximity of human-induced pluripotent stem cells (hiPSCs) ( Figure S1, Supporting Information). In hiPSCs, an increase in extracellular [H þ ] results in cell depolarization (lower V mem ) and a decrease in extracellular [H þ ] results in cell hyperpolarization (higher V mem ). [28] We measure relative changes in V mem of the cells using ArcLight, a fluorescent reporter for membrane voltage that we expressed on the cell membrane. [29] To monitor and control the dynamics of V mem in real time, fluorescent images are taken at regularly spaced time intervals. To set and maintain a specific V mem value for a given cell, we control the system using a ML-based algorithm that maps changes in V mem to prior H þ stimuli from the proton pump. Using this information, the ML algorithm decides whether the [H þ ] surrounding a specific cell should be increased or decreased to achieve the desired V mem value and sends a control voltage (V Hþ ) to the proton pump in the array, thus closing the control loop. The algorithm is not trained on any data a priori and makes no use of a model for either the bioelectronic device or the fluorescently tagged hiPSCs. Based on the target goal and current state, the parameters of the neural network are updated in between the time-lapse fluorescent images such that the "learning" happens in real time and the target V mem is ultimately achieved.
Each individual pump within the array is made of a palladium/ palladium hydride (Pd/PdH) contact that is able to transfer H þ to and from the solution depending on an applied voltage V Hþ (Figure 2A). [12] Previously described proton pump arrays [27] were fabricated on transparent substrates with integrated microfluidic channels to enable on-chip cell culture and monitoring of device performance and cell responses via fluorescence microscopy ( Figure S2, Supporting Information). The reservoir and target chambers consist of two microfluidic channels with 40 μm tall side walls and a ceiling of transparent microfluidic tape. At the closest point, the reservoir and target chambers run parallel to each other with a distance of 1 mm, with the ion bridge (brown lines) connecting the two solutions ( Figure 2B). A 2 mm diameter window is excised from the microfluidic tape and aligned to the array of 100 μm Â 100 μm proton pump array pixels aligned in a 5 Â 4 grid with a 250 μm pitch in the target chamber prior to sealing the fluidics to allow for interfacing with removable polydimethylsiloxane (PDMS) slides through clamping ( Figure S3, Supporting Information). The microfluidic channels allow for distinct electrolyte solution to be flown within the reservoir and target channels. Silver/silver chloride (Ag/AgCl) electrodes located in the reservoir and target channels are paired with the Pd/PdH electrodes located beneath the ion bridge at each pixel in the proton pump array to control ionic flux in and out of the device. A positive value for V Hþ applied between the array pixel and the Ag/AgCl electrode in the target channel will release H þ into the target channel, thus increasing [H þ ] of www.advancedsciencenews.com www.advintellsyst.com the solution ( Figure 2C); a negative value for V Hþ will absorb H þ from the solution into the contact, thus decreasing [H þ ] of the solution ( Figure 2D). A voltage V RES between the array pixel and the reservoir can be used to move H þ from reservoir to Pd contact forming PdH, which increase the efficiency of the proton pump by localizing H þ to the array pixel although for the work presented here only V Hþ is used. [27] Given that cells and their associated V mem are highly sensitive even to very small changes in [H þ ], we first demonstrate that we can precisely set [H þ ] in a desired location at a given time so that a specific [H þ ] value is regulated in the extracellular fluid ( Figure 3). To do so, we drive the proton pump with a ML-based control algorithm ( Figure S4, Supporting Information) consisting of a RBF artificial neural network ( Figure S5, Supporting Information). In this case, [H þ ] response from an area of interest is mapped using a dye whose fluorescence is inversely proportional to [H þ ] (SNARF). [30] The algorithm leverages a neural network composed of an input layer, a hidden layer, and an output layer ( Figure 3A). The input layer receives the error value between the desired and the measured [H þ ] values, information on prior [H þ ] stimuli, as well as current and previous [H þ ] response to the applied V Hþ . The desired output consists of desired values at time k þ 1, k, and k À 1, and the measured output consists of measured values at time k À 1, k À 2, and k À 3. The hidden layer converges to a mapping that allows it to discern which value of V H þ should be applied to the individual proton pump surrounding the area of interest to achieve the desired [H þ ].
Open-loop tests of the proton pump with SNARF dye in 0.1 M Tris buffer show pH gradients forming locally around actuated pixels ( Figure S6 ] results in an error value, which, in turn, triggers V Hþ actuation of the specific pixel (blue line). We show the ability of the proton pump to perform temporal control over the pH in the form of ramp functions with both positive and negative slopes; these ramp functions combined to form a triangle wave ( Figure 3B). To test repeated stimulation over time, we demonstrate the ability of the proton pump to perform temporal control over the pH in the form of a sine wave over two periods ( Figure 3C). In addition, the sine function displays the ability of the ML-based controller to track a function with a gradually changing slope-a challenging feature to achieve with open-loop control. Finally, we use a square wave to step to and maintain certain pH ( Figure 3D). In addition to temporal control, we achieve spatial control by independently setting [H þ ] for two separate areas using two sets of proton pumps ( Figure 3E-G). To this end, we set for [H þ ] to increase as function of time in the area on the left of Figure 3E (green boxes), while we set for [H þ ] to decrease as a function of time in the selected area on the left of Figure 3E (green boxes). These [H þ ] set points correspond to the downward sloping black trace in Figure 3F and the upward sloping black trace in Figure 3G, respectively. With this strategy, we are able to independently increase and decrease [H þ ] in two areas that are separated by only 450 μm from each other ( Figure 3F,G). Diffusion of H þ across the boundaries and cross talk of the electric fields generated by the contacts make this a remarkably challenging task. This challenge is confirmed by the relatively large swings of V Hþ for the proton pumps on both sides that are being actuated trying to control [H þ ] ( Figure 3G). These large swings cause the actual [H þ ] (red trace) to oscillate visibly around the target value (black trace) until it reaches its set point toward the end of the measurement. With little a priori knowledge, the ML-based controller adapts quickly to unknown and achieves the target value.
Having demonstrated that our proton pump array precisely set [H þ ] as a function of time in specific locations, we used this ability to demonstrate proof-of-concept control of V mem in hiPSCs ( Figure 4). The V mem of proliferative cells such as www.advancedsciencenews.com www.advintellsyst.com hiPSCs, embryonic cells, and cancer cells has a large effect on their proliferative state. [31] Voltage responsive hiPSCs were generated by using a PiggyBac transposase system to integrate a genetically encoded voltage indicator called ArcLight Q239 with codon optimization for mammalian expression and sequences for membrane trafficking. [32] To integrate the reporter hiPSCs with the proton pump array, we first grow the hiPSCs on a PDMS slab that is then placed on the top of the proton pump array with the cells facing the proton pump contacts. We select a specific area for [H þ ] actuation upon visual inspection of cells and their responsiveness to V mem stimulation (yellow box) Figure 4A. We monitor V mem of cells by measuring the fluorescence intensity of the membrane reporter ArcLight within the red box of Figure 4A. In brief, higher fluorescence intensity corresponds to a hyperpolarized V mem which is sensitive to changes in pH between 7.1 and 8.8 ( Figure S7, Supporting Information).
To demonstrate short-term control, we first attempt to set V mem for the selected hiPSCs along a triangle waveform (black trace) for a period of 1400 s ( Figure 4B). We set imaging of ArcLight and subsequent ML-based control of [H þ ] based on measured V Hþ to occur every 10 se to avoid photobleaching. As a result, V mem as measured by ArcLight fluorescence (red trace) oscillates around the target function (black). For long-term V mem control, we adopt a different sequence for stimulating the hiPSCs in which we alternate periods in which the proton pumps are active with periods during which the proton pumps are left floating ( Figure 4C). We adopted this sequence after observing stimulating the cells for longer than 30 min at a time makes the cells less responsive to changes in [H þ ]. With this sequence, we were able to stepwise increase the V mem of the hiPSCs in the selected locations and maintain this for over 1 h ( Figure 4D). We estimate that we were able to increase V mem by about 10 mV with respect to the resting V mem by measuring the change in ArcLight fluorescence. [33] These changes are consistent with what we measured with patch clamping on cells exposed to solutions of different pH ( Figure S7, Supporting Information). During this time period, the ML-based control algorithm holds V Hþ negative, which corresponds to a lower [H þ ] concentration than neutral www.advancedsciencenews.com www.advintellsyst.com in solution and it is consistent with what we have observed using electrophysiology ( Figure S8, Supporting Information). To confirm that V Hþ is having an effect on V mem , we notice that when the cells are not being stimulated, V mem slowly drops toward its initial value likely because [H þ ] returns to its neutral level due to the diffusion of ions from the surrounding areas ( Figure 4D). This diffusion is slower than the timescale of our sequence so that we are able to keep V mem to a set value with minimal oscillations. We demonstrate proof-of-concept V mem control for a period of 10 h by setting the target ArcLight fluorescence 2 fluorescence units higher than the average measured V mem of the 30 min period prior ( Figure 4E,F). We observe that the V mem of the cells becomes more polarized during the 30 min of stimulation and depolarizes during the floating periods (red trace). Over the course of the trial the extent of polarization during the pumping period is greater than the depolarization in the floating period resulting in a gradual trend toward polarization for the hiPSCs. By calibrating the fluorescence change in the ArcLight reporter, we estimate that we were able to achieve an increase V mem of the selected hiPSCs by 10 mV in the 10 h period. Bioelectronic actuators are an important part of realizing two-way communication with living systems. Inherent challenges in actuating biological tissues such as information carrier mismatch and system dynamics have limited the field to a point where the number of examples of bioactuators is relatively few compared with that of biosensors. Here, we show that it is possible to combine proton pumps as bioactuators with ML-based control approaches into biohybrid system that can address these issues to achieve precise closed-loop control of pH, which, in turn, allows long-term proof-of-concept closed-loop control of membrane voltage in cells. To the best of our knowledge, this is the first proof-of-concept control of V mem on nonelectrically excitable cells using closed-loop bioelectronic devices. Our ML approach did not experience failure modes dealing with uncertainties associated with overstimulation of the biology, device variability, and saturation of the system. To this end, we can manipulate V mem for periods as long as 10 h opening the opportunity for long-term control of cell function and proliferation. This ability to induce V mem changes on nonexcitatory cell types is of interest for applications in regenerative medicine and synthetic biology. In the future, a combination of additional monovalent and divalent ion species, and small ionic molecules, can increase the capabilities of our system. Importantly, closedloop biohybrid systems, such as this one, are highly anticipated due to their broad applicability across the fields of biology and engineering. Thus, new technology in synthetic biology and bioelectronics with closed-loop control could result in unprecedented spatiotemporal control over nature. [3] Experimental Section Device Fabrication: Ion pump arrays were fabricated on borosilicate glass wafer substrates which were cleaned via sonication for 10 min in acetone and isopropanol prior to a rinse with water. Au contacts and traces were patterned photolithographically with positive photoresist (S1813; Micro-Chem Corp.) and e-beam evaporation (10 nm Ti, 100 nm Au) followed by liftoff in acetone. Subsequent patterning with S1813 photoresist defined electrode regions for electrodeposition (see "Electrodeposition" section). After electrodeposition, a 1.4 μm insulating layer of parylene-C was deposited (Specialty Coating Systems Labcoter 2 system) in the presence of A174 adhesion promoter in the deposition chamber. The parylene was etched with an oxygen plasma at 200 V with the regions over the electrodes and contact pads exposed, and the interconnecting regions protected by a thick positive photoresist (SPR220-4.5; Micro-Chem Corp.). Prior to deposition of polymer coatings, wafers were treated with a 5% solution of (3-glycidyloxypropyl)trimethoxysilane (GOPS) in ethanol via spin coating at 1000 rpm for 30 s and then baked at 110 C for 5 min to promote adhesion of the ion bridge. A mixture of 8 wt% polyvinyl alcohol (PVA) and 2% polystyrene sulfonic acid (PSS) in water was then passed through a 0.8 μm mixed cellulose esters (MCE) syringe filter and spin coated on the wafers at 3000 rpm for 30 s followed by baking at 120 C for 2 h. The PVA:PSS cation exchange membrane was etched with an oxygen plasma at 200 V with the desired pattern defined with SPR220-4.5 photoresist. A second 1.4 μm coating of parylene was then deposited with the same protocol as above to insulate and protect the ion bridge from the subsequent steps. To promote adhesion between the parylene and SU8 photoresist, GOPS was again deposited using the aforementioned process prior to SU8 patterning. SU8 was spun onto the wafer at a speed of 1250 rpm for 30 s and a ramp of 300 rpm; microfluidic channels of height 40 μm were patterned forming the sidewalls reservoir and target chambers. The parylene insulation layer protecting the ion bridge openings in the reservoir and target channels was defined and etched using the same process as the previous parylene etch. A mixture of 2.5 wt% PVA and 5 wt% poly(diallyldimethylammonium chloride) (PDDA) in water was passed through a 0.8 μm MCE syringe filter and spin coated on the device at 6000 rpm for 30 s prior to baking at 120 C for 1 h prior to etch features being patterned in SPR220 photoresist and etching with O 2 reactive ion etching. Devices were then diced from the wafers prior to sealing the microfluidics. Single-sided microfluidic transparent diagnostic tape (3M 9964) was used to seal the microchannels on the proton pump array after the devices were diced. Features in the tape layer were punched out with 1 and 2 mm diameter biopsy punches for fluidic inlets and openings for cell clamping, respectively. The tapes were then aligned to features on the device and pressed to seal by hand.
Electrodeposition: Pd nanoparticles were deposited onto the electrodes using a 1 wt% palladium nitrate solution which was made by diluting a 10 wt% PdNO 3 solution with deionized (DI) water. The nanoparticles were electrochemically deposited by applying a DC voltage of -0.3 V for 4 s using an AgCl reference electrode and Pt counter electrode. An Autolab Potentiostat was used for the electrodeposition.
Stimulation Board: The stimulation unit is a custom designed modular circuit board for stimulating up to 64 electrodes simultaneously. The design consists of single-board computer, adjustable voltage sources, ammeters, and an analog switch. The adjustable voltage source can be controlled in a range of À4 to þ4 V with a resolution of 1.95 mV and able to source current up to 30 mA. Stimulate current can be measured in a range of À1650 to þ1650 nA with a resolution of 0.8 nA using built-in ammeters. Ultra-Low-Leakage CMOS Analog switches allow the stimulation unit to connect/disconnect individual electrodes, resulting in the addition output stage, floating. The stimulation unit can operate in two different modes, standalone and network. The standalone mode is where the output stages (voltage) of the stimulation unit are preprogrammed and run sequentially. On the contrary, in network mode, the stimulation unit will be listening to an incoming User Datagram Protocol packet, and output stages can be programmed in real time via a wireless connection. The network mode allows the ML-based control algorithm, which runs in a different machine to perform closed-loop control using feedback from both fluorescent images and measured current.
[Hþ] Measurement: We used microscope-based real-time imaging over the Pd electrodes to monitor [H þ ] change. We used 50 μM SNARF dispensed in 0.1 M Tris buffer as a fluorescent [H þ ] indicator, which was flowed into the target chamber via a sealed microfluidic channel. Thereafter, the device was monitored by BZ-X710 fluorescence microscope with 10Â Nikon objective (excitation: 560/40 nm, emission: 630/75 nm), and data were collected every 2 s in real time. Each set of experiments was repeated a minimum of 3 times; representative recordings are presented. Data were analyzed using ImageJ software.
www.advancedsciencenews.com www.advintellsyst.com Cells: hiPSCs were a generous donation from David Kaplan's group (ND418566, NINDS Human Genetics and DNA Cell Line Repository). [34] Stem cells were maintained at 5% CO 2 in StemFlex Medium (A3349401; ThermoFisher) on hESC Qualified Matrigel Matrix (354277; Corning) coated plates with the dilution recommended by the manufacturer. Cells were disassociated with TrypLE Select (12563029; ThermoFisher). Transgenic cells were made by transfecting 500 ng of plasmid via 1 μL of lipofectamine 3000 (L3000008; ThermoFisher) per well of 24 well plate containing 500 μL of Opti-MEM (31985062; ThermoFisher) with 1X RevitaCell (A2644501; ThermoFisher). Reagent was removed after 4 h and replaced with StemFlex. Cells were allowed to recover for 3 days before being selected with 50 μg mL À1 G418 in StemFlex with 10 μg mL À1 ROCK inhibitor Y-27632 (72304; STEMCELL Technologies). After selection, a stable, clonally pure line was made by serial dilution using StemFlex with 1X RevitaCell. Clones were expanded in StemFlex and then assayed for alkaline phosphatase activity (SCR004; Millipore). The clone with best membrane localization and expression of ArcLight was chosen for further experimentation and frozen down using NutriFreez D10 (05-713-1E; Biological Industries). Prior to being used in experiments, hiPSC ArcLight cells were plated in mTeSR1 no phenol red (05876; STEMCELL Technologies) and allowed to expand for two passages prior to seeding on PDMS slabs that were coated in hESC Qualified Matrigel Matrix. Slabs were used in bioelectronic chips 1-3 days after seeding. Cells were perfused with mTeSR1 no phenol red during experiments.
HiPSC ArcLight pH Response Assay: A PDMS microfluidic chip with multiple inputs was used to sequentially add mTeSR1 no phenol red media set at different pH by adding 5 N NaOH or 1 N HCl to large colonies of hiPSC ArcLight cells. The pH of the media was tested before and after the experiment to make sure that it had stayed constant during the experiment. Each pH solution was flowed over the cells for 5 min followed by the control media and images were taken of the cells every minute using an EVOS FL Auto2 imaging system with a standard GFP filter. Each solution was tested twice to make sure that the genetically encoded reporter was still responsive even after high pH media was applied.
Electrophysiology: hiPSC ArcLight cells were grown in mTeSR1 with no phenol red for two passages. Cells were then seeded on hESC Qualified Matrigel Matrix coated cover slips at a very low density. Cells were patched using mTeSR no phenol red media as the external media with or without 8.31 mM NaOH. The internal solution was 130 mM KCl, 10 mM NaCl, 0.5 mM MgCl 2 , 10 mM HEPES, 1 mM EGTA, 2 mM Mg-ATP, adjusted to pH 7.2 with NaOH. Resting membrane potential recordings were obtained by current clamp at 0 pA for 10 sweeps for 10 s each.
ML-Based Controller Algorithm: ML-based controller performs in real time and updates its parameters online. The technique could roughly be considered as supervised learning because we know our target response for the cellular system and actively correct bad behavior by the algorithm. There was no training of the ML-based algorithm prior to the initiation of the experiments. Hyperparameters were tuned on the fly at the beginning of the experiment (if needed) and the neural network parameters were randomly initialized. The "training" happened in real time by learning each time a new data point was generated through the course of the experiment. To provide an appropriate real-time ML-based controller, the total instantaneous error energy function of the system (i.e., EðkÞ ¼ 1 2 e 2 ðkÞ) is considered as a Lyapunov candidate to be utilized for updating the ML-based controller's parameters for guaranteed convergence. Here, e(k) is the system error (i.e., eðkÞ ¼ RðkÞ À YðkÞ), R(k) is the target of the system, and Y(k) is the output of the system. The updates to the adaptive ML-based controller's parameters are made by computing the appropriate gradients which are all derived utilizing the aforementioned Lyapunov candidate. The ML-based controller updates its centers' c i , weights' w i , and bias term w 0 by using equations (S5, S6, and S7, Supporting Information) in supplementary materials, respectively. Algorithm S1, Supporting Information is summarizing the overall real-time adaptive ML-based control methodology used in this article as pseudocode.

Supporting Information
Supporting Information is available from the Wiley Online Library or from the author.