Recent progress on motion control of swimming micro/nanorobots

Swimming micro/nanorobots attract considerable attention due to their considerable promises in various fields ranging from environmental remediation to biomedicine. However, the control system for optimal trajectory and precise localization is an exciting yet challenging for the existing swimming micro/nanorobots. Researchers have used different actuation systems allowing multiple degrees of freedom to actuate swimming microrobots with various motion control methods from well‐controlled physicochemical processes to achieve autonomous movement along the gradient of various chemical or physical fields. Furthermore, with the advance of intelligent technology, machine learning strategies have been considered for intelligent control of swimming microrobots. In this paper, we summarized directly manual control and autonomous tactic motion of swimming micro/nanorobots. The application of reinforcement learning in controllable motions of swimming microrobots was also elaborated.

tic or light fields. [8][9][10][11][12][13][14][15][16] Furthermore, to realize their practical utilizations, swimming micro/nanorobots are required to be navigated toward a targeted destination or regulating their speed or orientation in a complex environment. The navigation of swimming micro/nanorobots with temporal and spatial precision is critical for fulfilling the demand of applications. Nevertheless, the navigation of swimming robots with micro/nanoscale scale is particularly challenging due to the strong Brownian motion and low Reynold numbers. [17][18][19] Once going into the micro/nanoscale, the navigation strategies in the macroscale world would fail to function due to the absence of inertial forces.
In nature, the diverse microorganisms have developed the capability of motile and navigation through million years of evolution for adoption in the complex environment without inertia. For example, E. coli can not only exhibit efficient propulsion in various complex fluids though the rotation of its flagella, but also autonomously locomote close or away from targets to seek nutrition or evade the outside attacks. 20,21 Inspired from the performances of motile microorganisms, the various control strategies to modulate the movement behavior of swimming micro/nanorobots have been substantially studied over the past decade. [22][23][24] As instances, increasing the concentration of fuels would enhance the velocity of chemically powered micro/nanorobots, 25 and the directionality of swimming micro/nanorobots would be navigated through manipulation of external physical forces. 26 However, the precise spatiotemporal control as the natural microorganisms, including the tactic movement that autonomously approaches or leave targets along the gradient of the chemical or physical field, particularly the motion with intelligence that enable sense the surrounding environment, generalize past experience to new situations, and navigate to object in the optimal path and energy-efficient mode, is still required to develop innovative methods to add the intelligence into swimming micro/nanorobots. In recent, advances in machine learning techniques have led to significant improvements in obtaining solutions to optimization problems, which also inspire the artificial intelligent navigation, path-planning, and efficient swimming strategies of micro-/nanomotors in complex remote work. 27 Here, we present an overview of the versatile approaches to control of swimming microrobots. In the following sections, the navigation of swimming micro/nanorobots with external field, tactic motion along the gradients of various fields, and autonomous navigation using artificial intelligence (AI) will be discussed.

MOTION CONTROL USING EXTERNAL FIELD
Manipulation of swimming microrobots not only realizes the multi-functional navigation, but also lays the foundation for exploring the diverse applications. As the most mature control method, manual mobility control has been widely used to stimulate swimming microrobots to complete various tasks due to its high reliability and flexibility.

Magnetic
Magnetic fields have been widely used as a propulsion power, in which the forces and torques generated by mag-netic fields can be applied without any barrier and perturbation by the complex environment. As a non-contact remote control strategy, magnetically propelled swimming microrobots mainly rely on boundary surface, 28 asymmetric deformations of elastic flexible filament, [29][30][31] and rotational behavior 9,32,33 to break the spatial symmetry and produce the nanoscale propulsion. The application and removal of the magnetic field can realize the on/off motion of these swimming microrobots in time, and motion performance can be precisely modulated by adjusting the type and parameters of an external magnetic field. As shown in Figure 1A, versatile motion strategies of peanut-shaped colloidal motor that used rotating magnetic fields to obtain a rolling or wobbling mode and enabled precisely controllable motion in predefined tracks and climbing over steep slopes. 34 Similarly, a type of Janus microdimer surface walkers can be precisely steered by the oscillating and rotating magnetic field to through complicated structures and complete pre-programmed trajectory in various experimental conditions. 35,36 Besides acting as a driving force, the magnetic field is also exploited to guide the direction of self-propelled swimming microrobots and achieve the versatile capacity of functioning in dual modes. [37][38][39] Figure 1B shows a ferromagnetic micromotor controlled in aqueous solution with an external magnetic stimulus. 40 The rotation on different planes (X-Y plane and X-Z plane) and directed motion of this swimming microrobots in a preset route were demonstrated. Moreover, helical propulsion can get rid of the surface and fuel demand. Based on this method, an artificial swimming microrobots bioinspired from bacteria was demonstrated structural and magnetic properties with the adaptive locomotion properties. 41 The swimming microrobots performed various and efficient corkscrew motility controlled by the magnetic frequency, viscosity and body plan (Figure 1c). While the magnetic fields enable the collective behavior of multiple nanorobots, individual control of nanorobots cannot be achieved with the global actuation fields. Besides, a complex magnetic field generator occupies a large working space, which limits the application range of magnetic drive swimming microrobots.

Light
Light-navigated swimming microrobots could convert light energy into mechanical energy to achieve noninvasively controlled at highly precise spatial and temporal resolutions. To date, light-controlled micro/nanomotors based on bubble propulsion, thermophoresis, diffusiophoresis, and electrophoresis driven, etc. have been investigated. 10 swimming microrobots by combining the photoelectrochemical and electrochemo-mechanical energy conversion processes was developed to explore determining factors for the improvement of propulsion efficiency ( Figure 2A). A group of ferrocene-based (Fc) reversible shuttles were identified with adjustable motion and efficient propulsion. 43 Apart from conventional Pt-based catalytic micromotors powered by toxic H 2 O 2 fuel, photocatalytic decomposition of glucose is also an important and efficient approach. 44 In light of this, a highly efficient glucose-fueled cuprous carbon nanotube (Cu 2 O@N-CNT) swimming microrobots was demonstrated outstanding propulsion with light-modulated three-dimensional motion, 45 as shown in Figure 2b. Furthermore, swimming microrobots made from photoactivated semiconductors also present several excellent advantages. The hybrid photoactivated swimming microrobots made from two different semiconductors (TiO 2 /Cu 2 O) exhibited wavelength-dependent modes of motion due to the disparate responses of each photocatalyst ( Figure 2c). 46 While substantial advances have been made in the previous decade towards efficient propulsion and accurate control of light-driven nano/micromotors, some issues have to be resolved. For example, consumption of material remaining in the swimming microrobots in the oxidation reactions limits its velocity and life-time.

Other control approaches
As an attractive external field source, acoustic source can be easily coupled to actuation system to achieve on-demand control over the trajectory and velocity of micro/nanomotors. 14,47 The controllable manipulation can be achieved by modulating the performance of ultrasound, such as switching on/off, frequency and direction. In essence, the acoustic radiation force (ARF) directly induced acoustic control of swimming microrobots' motion. 48,49 Based on ARF in a standing wave, several control methods were proposed for trapping and moving the microparticles. 50 Recently, an acoustically powered bubble-based swimming microrobots, does not require operation at acoustic pressure nodes, was capable of autonomous motion in three dimensions and climbing vertical boundary ( Figure 3a). 51 Moreover, acoustic fields hold considerable promise in regulating the collective behavior of catalytic nanomotors. 52 As shown in Figure 3b, reversible assembly of catalytic nanomotors was induced using acoustic fields. In this study, controlled swarm movement and separation of different nanomotors was demonstrated relying on the interaction between individual nanomotors and the acoustic field. 53 The chemical control methods also offered simple ways to excite behavior responded to chemical reactions without the need of external control systems. [54][55][56][57][58] The direction of the self-propelled were controlled by environmental gradient, external field, and confined space while the magnitude regulatory is achieve by adjusting chemical reaction rate. 59,60 One handy method for accelerating the chemical reaction is to increase the fuel concentration. 61 The moving velocity of a rolled-up microtube with an inner catalytic surface has been demonstrated to be almost equal to the product of the bubble radius and frequency, and circular trajectories of the microjets were deterministically controlled by tuning the jet velocity which depended on H 2 O 2 concentrations. 62,63 Based on the same principle, the velocity of Pt-microtubes and Pt-covered silica particles benefits strongly from the addition of surfactants by regulating bubble formation. 64 Recently, an active hybrid microcapsule motor powered by the biocatalytic decomposition of urea at physiological concentrations was presented ( Figure 3C). The moving velocity of the microcapsule can be controlled by chemically inhibiting and reactivating the enzymatic activity of urease. 65 However, these bubble microrobots also suffer from limited manipulation force capabilities, thus restricting their applications.
In addition, swimming microrobots' motion control using AC electrokinetics has also been demonstrated. Various Janus particles and nanowire have been reported to achieve regulatable manipulation using electrophoretic and dielectrophoretic forces. 66,67 Recently, the advantage of dielectrophoretic (DEP) also can be programmed into individual parts by taking advantage of shape-and material-specific force responses under external electric fields. [68][69][70] As shown in Figure 3d, shape-encoded reconfigurable assembly of swimming microrobots with self-propelled microactuators for frequency-tunable locomotion was designed. Based on self-dielectrophoresis and induced-charge electrophoresis, the metallodielectric Janus microparticles can actively propel themselves forward and backward responding to different external electric field frequencies. 71 Recent advances in manual mobility control based on fuel concentration and external field have proved to be an excellent candidate for precise, continuous and real-time manipulation of swimming microrobots. As more control mechanisms and combinatorial strategy are identified in manual mobility control, it can be expected that swimming microrobots would be expanded to more practical applications. Despite the outstanding performance of the manual mobility control method, there are still major challenges to be addressed for autonomous movement by interacting and responding to the surrounding environment, which is crucial for the non-invasive application in uncertain or dynamically changing environments.

TACTIC MOTION
In nature, motile microorganisms such as sperm or E. coli bestow approach or leave a target along the gradient of chemical or physical triggers, more widely known as a tactic movement. [72][73][74] For example, E. coli will actively approach the nutrients and flying moth will dart to the light source. Inspired by these natural behaviors, many methods for inducing controlled swimming microrobots motion have been explored, including using concentration gradient, 75 light gradient, 76 magnetic gradient and flow gradient. 77 In these cases, the swimming microrobots possess the ability of self-navigation and self-targeting responding to surrounding gradient.

Chemotaxis
Organisms are able to autonomously toward or away from a particular chemical attractant or toxin along their concentration gradients (so-called chemotaxis), stimulating the development of versatile synthetic micro/nanoswimmers with mobility and chemotaxis. 78,79 The chemotaxis phenomenon has been studied extensively since the 1880s discovered firstly and multifarious chemotactic swimming microrobots (such as bimetallic rods, 80 83 Furthermore, the mechanism for chemo-taxis in isotropic catalytic swimmers 75 and active Janus particles 84 have been investigated. A simple and intuitive description of the underlying mechanisms for positive or negative chemotaxis chemotaxis is shown in Figure 4A. Controllable chemotactic or anti-chemotactic collective behavior of swimming microrobots was also explored. 85,86

Phototaxis
Inspired by the natural phenomenon of phototactic microorganisms, intelligent photo responsive swimming microrobots with positive or negative phototaxis response to gradient light sources (eg, UV light 87 and laser beam 88 ) have been demonstrated. Motion speed and direction can be swiftly and precisely controlled by adjusting the intensity and angle of the incident light. 89 Various inorganic swimming microrobots based on positive phototaxis (toward a light source) or negative phototaxis (away from a light source) have been demonstrated. Recently, an artificial swimming microrobot was developed containing a nanostructured photocathode and photoanode at opposite ends that exhibit either positive or negative phototaxis by controlling zeta potential of the photoanode and program swimming microrobots ( Figure 4B). 90 Unlike asymmetric structures design, an intelligent lightsteered micromotor based on simple isotropic semiconductor particles also allowed light-induced motion, and even positive or negative phototaxis. 91 Moreover, a polarotactic swimming microrobots with a significant dichroic ratio was also achieved and exhibited strong dichroic swimming behavior. 92

Magnetotaxis
Natural magnetotactic bacteria could sense Earth's magnetic field to reorient themselves and transport minerals in aquatic ecosystems. 93,94 Inspired by this, a stomatocyte supramolecular nanomotors was designed with possibility of gaining directional control by integrating magnetic segments, 95 as shown in Figure 4c. Furthermore, remote magnetotactic control of the swimming microrobots integrated the microalgal cell with magnetic microbead under a low gradient magnetic field was realized. 96 Facing the stable on-demand navigation regardless of variation in morphologies, a reconfigurable micromachines based on self-folding hydrogel bilayer structure showed the ability of actively sensing magnetic fields and coordinating their movement in response, without waiting for reprogramming magnetic anisotropy. 97

Rheotaxis
In addition, swimming microrobots also exhibit rheotaxis that enable to actively reorient and swim against an imposed flow in biological or microfluidic systems. 98 Biomimetic colloidal system 99,100 and swimming microrobots 77,101 have been demonstrated excellent rheotaxis behavior. More recently, a hybrid sperm swimming microrobots that can actively swim against continuous or pulsatile flowing blood was presented and serves for magnetic guidance and cargo transport ( Figure 4D). 102 The relevance of the rheotaxis swimmers' actuation mechanism has been further studied. The coupling of hydro-electrodynamic fluid flows and electrostatic interactions with nearby surfaces induces a directional propulsion by experimentally and computationally studying the rheotaxis of self-propelled gold-platinum nanorods in microfluidic channels. 103 It was found that the ability to robustly move upstream in an imposed background flow depends strongly on the Au/Pt ratio by explore the behavior of micron-scale autophoretic Janus (Au/Pt) rods. 104

Gravitaxis
Up to now, the majority of manipulation control demonstrations of swimming microrobots have focused on area on or near a planar surface instead of moving away from this plane. Equipping swimmers gravitactic responses to control motion in all three dimensions is essential in practical application. Taking advantage of asymmetric mass distribution caused by their shape, spherical polystyrene/Pt Janus swimming microrobots 105 and Lshaped photothermal swimmers 106 were manufactured with three-dimensional motion performance resulting from a strong enough upward driving force. In these cases, the gravitactic performance can be adjusted by modulating their own size and structure. In addition, the inhomogeneous physical and chemical field also enable swimming microrobots with controllable negative-gravitactic behavior. As shown in Figure 4E, one kind of synthetic photochemically active swimming microrobots was demonstrated to swim against gravity and lift off from the wall when the light intensity was increased. 107 As described above, micro/nanomotors with taxis have been developed toward actuators in dynamically changing and complex environments. Taxis mobility control enables swimming microrobots to sense the environment and executing independently. Nevertheless, it is difficult for this autonomous motion based on environmental gradients to achieve targeted movement tasks with high efficiency and high hit rate. Another challenge comes from the need for accessible and energy-efficient paths when performing remote tasks in complex environments. For instance, following the shortest path and crossing various obstacles in the human environment is crucial for a drug-loaded swimming microrobot. Considering all these challenges, a more intelligent control strategy that can analyze the environment in real time and plan optimal path is requisite.

MOTION CONTROL WITH ARTIFICIAL INTELLIGENCE
The empirical behavior gained through the interactions with environment is a crucial factor for living systems to sense the surrounding situation and conduct ideal action. However, on-demand motion in complicated, uncontrollable, and dynamic changing environments without human assistance is still urgently need for artificial micro/nanomachines. In the past decade, AI technology has been widely used to promote major developments in industrial processing, precision medicine, intelligent transportation, and Internet security. A branch of AI known as reinforcement learning, have been implemented into microrobot systems to disentangle the complexity of biological swimming microrobots and find energy-saving navigation strategies in complex environments. 108,109

AI navigation and path-planning
Advances in AI techniques and computational facilities have led to significant improvements in solving the optimal solution and path-planning problems. 110 A pathplanning algorithm called optimal Bidirectional RRT * to build an autonomous system that consists of 3-D path planning and 3-D path following. Various paths including straight line, sinusoidal path, and helical path helical swimming microrobots have been performed to verify the effectiveness of this control strategy ( Figure 5A). [111][112][113] Based on navigation consist of artificial intelligence planner and visional feedback control, a smart microvehicle achieved precise autonomous manipulation in complicated environments and traffic scenarios. 114 As shown in Figure 5b, the microvehicle was steered by autonomous navigation to pass through the complex micromaze after optimized path planning. Moreover, considering the impact of shape and gravity on swimming strategy, the Q-learning algorithm was used to obtain approximately optimal swimming strategies and analyze the shape and gravity effects on the characteristics of learned strategies. 115 It was found that shape effect is significant for particles with large reorientation timescale.

Optimization of pathway
Reinforcement learning also allow swimming microrobots indeed learn nearly optimal strategies to find nontrivial paths and reach a target on average in less time. Efficient navigation and precise localization of swimming microrobots promising wide high-tech applications involving drug delivery, environmental remediation and energy utilization. Figure 5C displays the controlled trajectories of different motors navigating through channels filled with square, which was realized using a model-free deep reinforcement learning algorithm based on bio-inspired neural networks. 116 Furthermore, machine learning algorithms were introduced into the motion of thermophoretic artificial swimming microrobots. As can be seen from Figure 5D, the number of steps required to reach the goal and the persistence of the trajectories were continuously reduced through the learning process. 117 In addition, reinforcement learning algorithm was applied for smart particles to finding efficient swimming strategies to escape from fluids traps in two/three-dimensional chaotic flow. 118,119

Multi swimming microrobots cooperative
Multi-agent reinforcement learning enables swimming microrobots to learn cooperation to emerge collective sensing for scalar gradients (flow or light intensity). 120 For example, different dynamical phases in swarming bacteria and energy-efficient swimming simulated school of fish. 121 Combining high-fidelity flow simulations and deep reinforcement learning algorithm, the optimal collective energy savings swimming strategies was found in a simulated school of fish ( Figure 5E). 122 Similarly, swimmers can also learn to adapt their motion so as to optimally reach the target by integrating reinforcement learning algorithm with flow solver. 123 Furthermore, Turing Learning was utilized to investigate the swarm behaviors of simulated robots and found that collective behaviors can be directly inferred from the motion trajectories of individuals in the swarm. 124

CONCLUSION AND OUTLOOK
In summarize, manual mobility control based on fuel concentration and external field, autonomous taxis behaviors control relied on surrounding gradient, and intelligent self-navigation control guided by reinforcement learning algorithm are presented and discussed. Various motion manipulation strategies based on fuel concentration and external fields, including magnetic fields, light, ultrasound, and electric fields achieve on-demand motional regulation of swimming microrobots, such as their speed, motional direction, and ultimate regulation of their collective behavior. Nevertheless, swimming microrobots must possess the capability of autonomous motion and self-targeting in application of dynamically changing scenarios. The taxis behaviors enable swimming microrobots to sense the surrounding environmental gradient induced by signal and autonomous move toward or away from specific locations. Furthermore, recent success of machine-learning approaches in studies of swimming microrobots provides a glimpse to meet further intelligent application demand in more unknown and non-equilibrium environment.
However, there are still some additional challenges that should be addressed. First, swimming microrobots that combine multiple propulsion means and taxis behavior may bring unexpected motion and promising applications. Second, more intelligent mechanisms or approaches are wishful to solve the problems in artificial swarm swimming microrobots. In the future, we expect new breakthroughs from the field of intelligent mobility control of swimming microrobots.

C O N F L I C T O F I N T E R E S T
The authors declare no conflict of interest.