Design and Optimal Pose‐Constrained Visual Servoing of a Novel Active Flexible Endoscope Holder System for Solo Laparoscopic Surgery

Laparoscopic surgery (LS) has become an effective and widely accepted therapy for patients. However, performing LS requires advanced training and skills, as it demands high precision and dexterity due to visual feedback depending on the collaboration between the surgeon and endoscope assistant, restricted field of view, and limited dexterity inside the body. Thus, a flexible endoscope holder system with an automatic control scheme is proposed to assist LS performance, allowing the surgeon to perform solo LS. A DNA‐inspired helix‐based structure with wide‐angle and constant curvature bending is presented and made of printable nylon with selective laser sintering. The proposed flexible mechanism reduces the cost of manufacturing and assembling. Also, it has a hollow design for embedded sensing and actuation. Then, an optimization method is proposed, which provides a reference for the endoscope installation, robot placement, and pose selection. In addition, aiming at the misorientation problem, a pose‐constrained visual servoing control scheme for the endoscope system is developed. Both simulation and physical experiments are conducted to verify the effectiveness and feasibility of the proposed control scheme. Experimental results demonstrate that the proposed visual servoing scheme can maintain the optimized pose and reduce misorientation during automatic view steering.

Laparoscopic surgery (LS) has become an effective and widely accepted therapy for patients.However, performing LS requires advanced training and skills, as it demands high precision and dexterity due to visual feedback depending on the collaboration between the surgeon and endoscope assistant, restricted field of view, and limited dexterity inside the body.Thus, a flexible endoscope holder system with an automatic control scheme is proposed to assist LS performance, allowing the surgeon to perform solo LS.A DNA-inspired helix-based structure with wide-angle and constant curvature bending is presented and made of printable nylon with selective laser sintering.The proposed flexible mechanism reduces the cost of manufacturing and assembling.Also, it has a hollow design for embedded sensing and actuation.Then, an optimization method is proposed, which provides a reference for the endoscope installation, robot placement, and pose selection.In addition, aiming at the misorientation problem, a pose-constrained visual servoing control scheme for the endoscope system is developed.Both simulation and physical experiments are conducted to verify the effectiveness and feasibility of the proposed control scheme.Experimental results demonstrate that the proposed visual servoing scheme can maintain the optimized pose and reduce misorientation during automatic view steering.
endoscope holders to provide stable images under the surgeon's direct control, allowing the surgeon to perform solo LS and providing a promising method to overcome the shortcomings of the traditional LS.
Endoscope holder systems (EHS) can be divided into active and passive endoscope holders. [4]The passive holder system comprises several bars and ball joints, whose base is usually anchored to the operating table.The endoscope grasped by a clamp can be manually controlled by the surgeon and usually is single-handedly operable.However, it can cause temporary interruption of the surgeon and increase the surgeon's workload for adjusting the endoscope angle. [5]Thus, the active EHS is proposed, [6][7][8][9][10][11][12][13][14] whose endoscope is motorized by machine equipment.Accordingly, various user interfaces have been developed to operate the endoscope instead of manual control, such as head/eye/face movements, [6] voice-based control, [7] finger/footactivated switches, [8] etc.Although these operating methods are convenient, they are less intuitive and efficient than passive EHS.Therefore, the automatic control strategy [9][10][11][12][13][14] based on visual servoing method has been proposed and applied to active EHS to automatically track the tip of surgical instruments, which is a more intuitive and convenient control method.
Considering the direct contact between the active EHS and the human body, safety is critical in designing an EHS.Since the movement of the endoscope is constrained by a tiny incision, the first consideration to keep safety is how to ensure the incision constraint of the endoscope, and this constraint is also called the remote center of motion (RCM) constraint.From the implementation mechanism of RCM constraints, the EHS can be reclassified into design-based and control-based EHS. [15]Design-based EHS realizes RCM constraints by designing special mechanical structures, like parallel mechanisms, [16] arc connection mechanisms, [17,18] etc.However, since the systems with mechanical RCM usually have complex structures and a fixed RCM location, they are normally specific for a certain type of surgery.Instead, control-based EHS comprises a redundant manipulator and an endoscope is proposed, which has a flexible RCM position setting by adequately tuning the control algorithm.Various algorithms have been proposed to achieve RCM constraints, such as the augmented Jacobian, [19] projection in the null space method, [20] and multiobjective optimization with RCM constraint. [21,22]n addition, the traditional active EHS is designed for rigid endoscopes.It commonly has an obvious external motion space while providing visual feedback, thus increasing the frequency of collision between the endoscope and other equipment.Thus, some active EHSs have been developed, which utilize a machine body to manipulate a flexible endoscope with a bendable mechanism within the abdominal cavity.Lee et al. [23] developed a 3-degree of freedom (DoF) laparoscopic assistant robot system with motorcontroller bending and zooming mechanisms for sufficient safety features.It should be noted that the zooming mechanism limited its workspace adjacent to the initial position and thus has a limitation in more complex LS.Ma et al. [9,24] developed a flexible endoscope system based on the da Vinci Research Kit (dVRK) and validated that the proposed system has less motion space than the traditional rigid EHS, with much more safety.Considering the high cost of dVRK, the flexible EHS based on the commercial robot arm is proposed in our previous works.In the study of Zhang et al. [25] the flexible endoscope adopted a helical hollow strand (HHS) tube to design the flexible section.Its maximum bending angle was 60°and was constrained by the HHS tube's bending stiffness.To increase the bending angle of the flexible section, a 3D-printed nylon helix structure was developed in the study of Zhang et al. [11] Although the maximum bending angle of this design can reach 180°, holes reserved in the helix structure to pass through the wires affected the isotropy of the structure in terms of bending stiffness, which was not conducive to precise control of the endoscope.
The eye-hand coordination of the surgeon has been pointed out in many works, [26,27] showing its importance in LS.However, in our previous work, in the process of automatic tracking EHS, the eye-hand coordinates of the tester appeared inconsistent.Although the consistency of eye-hand coordination was manually adjusted before the tracking operation, the inconsistency will be after a period of time.Also, this inconsistency became increasingly obvious with time, as shown in Figure 1.The main reason for this phenomenon is that only the position factor is considered in the tracking process, making the direction factor uncertain.An investigation by Wentink et al. [27] validated that endoscopic camera rotation can eliminate misorientation and improve eye-hand coordination.Based on this foundation, the intuitive virtual plane [28] and minimized rotation constraint [29] were proposed to preserve orientation during the surgical procedure.However, both approaches required placing a marker in the camera view, which was not applicable in clinical practice.Therefore, this article proposes a novel active flexible EHS to tackle the abovementioned issues.The proposed EHS comprises a UR5 robot arm and a flexible endoscope.The flexible mechanism adopts a novel DNA-inspired helix-based structure.An optimization method suitable for EHS is proposed to optimize the installation configuration of the endoscope, layout, and pose of the EHS.Finally, visual servoing with optimized pose constraints is proposed to eliminate the misorientation during the tracking process.Therefore, the main contributions of this article are as follows: 1) We propose a novel helix-based continuum mechanism whose stiffness is reinforced by a series of rings.It has a large central lumen for passing cables and wires.Thus, no extra hole is required.In addition, its bending curve confirms the assumption of constant curvature bending; 2) A reachability map (RM) generation algorithm suitable for the EHS is proposed.Based on the generated RM, the surgical robot's optimal configuration, layout, and pose can be calculated; and 3) Pose constraint is first considered in the visual servoing, which will improve the safety of the surgical procedure.Moreover, the effectiveness of the visual servoing scheme is validated by simulated and physical experiments.
The remainder of this article is organized as follows.The novel helix-based continuum mechanism and the flexible endoscope prototype are introduced in Section 2. Section 3 details the configuration, layout, and pose optimization method.In Section 4, a visual servoing scheme with RCM constraints, physical limits, and pose constraints is formulated as a quadratic programming (QP) problem and solved with an recurrent neural network (RNN) solver.Section 5 and 6 present the simulation and physical experiments to validate the theoretical results and verify the visual servoing scheme.Finally, the conclusion is shown in Section 7.

DNA-Inspired Continuum Mechanism
DNA structure contains two chains coiling around each other, which is one typical helix structure, as shown in Figure 2a.The base pairs between two strands enhance the stability of the structure.Inspired by this structure, a helix structure reinforced by a slender shaft has been introduced in our present work. [11,30]Although the slender shaft can imitate the base pairs to increase the stiffness of the double helix structure, it also limits the diameter of the central cavity.Unfortunately, the endoscope must have a cavity with a suitable size to pass through the cables and tubes.Hence, a new reinforced helix-based continuum structure is proposed.The basic structure of this structure is also a double helix, which is limited by a series of rings.The 3D model of this mechanism is shown in Figure 2b.The constrained rings increase the helix structure's stiffness, especially the compressional and torsional stiffness, and maintain the hollow design.In addition, ring constraints ensure that the continuum mechanism can bend with equal curvature, which is validated in Section 5 and is beneficial to the precise control of the mechanism.

Finite Element Analysis
The effect of reinforced rings on the stiffness is analyzed with the aid of the Abaqus.To do this, the pure double helixes structure and the DNA-inspired continuum structure are designed.Two structures have the same helix strand.The only difference between those two structures is that the DNA-inspired structure has some reinforced rings.The design parameters of the two structures are shown in Figure 3a,b.The length of the flexible sections of the two structures is 60 mm.In Abaqus, Young's modulus and Shear modulus are predefined as E = 1500 MPa and G = 510 MPa, which are defined as the selected printing material nylon.Then, the same compressional force, twisting, and bending moment are applied to those two continuum mechanisms.After that, the corresponding deformation with the external load is recorded.Since the stiffness of the continuum mechanism is the resistance against deformation in response to an external load, the corresponding deformation is used to analyze the stiffness of the continuum mechanism.As shown in Figure 3c, when an axial force ranging from 0 to 0.6 N is applied to the simple helix structure and DNA-inspired structure, the axial deformation of the simple helix structure is 3.5 times that of the DNA-inspired structure.When a torque ranging from 0 to 3 N mm is applied to the two structures, the torsional deformation of the simple helix structure is about 2.9 times that of the DNA-inspired structure, as shown in Figure 3d.The bending deformation of two flexible structures with a bending moment ranging from 0 to 3 N mm is shown in Figure 3e.The result shows that the bending deformation of the simple helix structure is about 1.7 times that of the DNA-inspired structure.
Then, the dimension of the simple helix structure is redesigned to have the same bending stiffness as the DNA-inspired structure, as shown in Figure 3b.After that, their compressional and torsional stiffness are compared, as shown in Figure 4.The result shows that with the same bending stiffness, the DNAinspired structure has a 2.5-time larger compressional stiffness and 1.7-time larger torsional stiffness compared to the simple helix structure.In conclusion, the stiffness of the helix structure is obviously improved.The improvement of compressional and torsional stiffness is more significant than that of bending stiffness, which means the new design has larger compressional and torsional stiffness than the simple helix structure while preserving good flexibility.In addition, the large central lumen of the DNA-inspired structure can pass through some electronic wires and tubes for the distal functional unit.

Flexible Endoscope Design
To facilitate disinfection, the flexible endoscope is designed into two parts, as shown in  is the actuation unit, which contains four FAULHABER minimotor groups, as shown in Figure 5b.Two motors control the bending of the flexible section, one motor drives the motion of the constrained tube to change the bending length, and the final motor is responsible for the rotation of the flexible section.
One motor group contains a motor of 1727U024CXR, a gear ratio of 68:1, and a rotary encoder with 1024 counts per revolution.Each motor is equipped with an MC5005S motor driver to drive the motors, which communicates with the PC by EtherCAT.The motion of the four motors will be transferred to the flexible unit with the mechanical transmission unit, as shown in Figure 5c.

Configuration, Layout, and Pose Optimization
To optimize the installation configuration of the endoscope on the robot, a RM generation method suitable for the surgical robot system is proposed in this section, which contains the manipulability and reachable information.To do this, the EHS's possible workspace and task space are defined first.Then, RM can be generated by traversal of all the workspace and task space points.Finally, the robot's optimized configuration, layout, and pose can also be determined with the generated RM.

Workspace and Task Space
In LS, the endoscope should enter the abdomen cavity through an RCM point.To provide the FOV, the exposed rigid shaft of the endoscope would be pivoted around this point.Since the proposed endoscope can be rigid or flexible by moving the constrained tube, the task space for the endoscope system can be simplified as a cone whose vertex is the trocar position, as shown in Figure 6a.h t is the distance between RCM point to the target plane.The target plane T represents the plane where the distal point of the rigid shaft can expose the whole abdominal plane in the FOV, which can be determined according to the patient's abdominal size as the radius r t .
For one EHS, the workspace should be the dataset of trocar points through which the surgical robot can access all the points in the target plane.Without considering the application scenario, the possible workspace of the UR5 robot is theoretically encapsulated by a sphere with the r = 850 mm radius arm length of the robot.Then, this possible workspace can be further reduced when the UR5 is used to hold an endoscope.In LS, the patient usually has supine and lateral positions.The robot base is usually settled in the middle plane between the supine and lateral planes to cover the patient both in the supine and lateral positions.Thus, the possible workspace for the EHS can be reduced to the space ℛ, as shown in Figure 6b.

Inverse Kinematics and Manipulability
Both forward and inverse kinematics have been used to generate the RM. [31]In this work, the RM with the reachable and manipulability information is generated by using inverse kinematics.First, the direction vector I e of the robot end-effector in the world frame is determined according to the position of RCM and endoscope tip.Then, the rotation angle between I e and z ¼ 0, 0, 1 ½ in the world coordinate system can be described as: The rotation axis between the world coordinate and endeffector coordinate also can be obtained by According to the Rodrigues' formula, the rotation matrix of I e and z can be calculated as: where c θ and s θ represent cos(θ) and sin(θ), accordingly, Since the length of the endoscope is fixed, when the position of the endoscope's tip and trocar position are known, the position of the robot endeffector P e can be calculated based on the existing information.Then, the transformation matrix of the robot end-effector can be described as: With the transformation matrix T e , the joint configurations of UR5 q r = [q 1,⋯, q 6 ] can be calculated using inverse kinematics.The manipulability index is spanned by the singular values of the Jacobian, which is an important index to measure the performance of the robotic system and has been applied in several robotic systems. [32]When the joint configurations of the robot are determined, the manipulability index of the UR5 robot can be calculated as [33] : where J r is the Jacobian of the robot in joint space q r .Since the EHS is constrained by the RCM point, when the position of the robot's end-effector is defined, the posture of the robot's endeffector is determined.It should be noted that the manipulability J r only involves the position factor and J r ∈ ℝ 3Â6 .

RM Algorithm
To build RM, the first point is to judge whether the endoscope can reach all points in the target plane through the trocar point in the workspace.Then, if the endoscope can reach all the points, the point will be added to RM, and the average manipulability of the robot will be calculated and stored on the map.The pseudocode algorithm of RM generation is shown in Algorithm 1, and related function parameters are shown in Table 1.First, the sampling points are uniformly set in the workspace, which is used as the candidate points of the trocar (line 1 of Algorithm 1).Then, the endoscope will traverse all the discrete points in the task plane with the trocar constraint (line 2 of Algorithm 1).Once the trocar position and task point are obtained, the transformation matrix T e of the endoscope can be calculated according to Equation (1-4) (line 3 of Algorithm 1).Then, eight configurations of the robot can be obtained by using inverse kinematics (line 4 of Algorithm 1).It should be noted that the same configuration usually exists in eight configurations.The redundant configurations will be ignored.Also, some unavailable joint configurations can be eliminated by using self-collision detection, and the manipulability index of the rest of the possible joint configurations is calculated and stored to m i (line 5-12 of Algorithm 1).When the element number of m i is equal to or larger than one, it means that the surgical robot can reach the specific target position, and the number of reachable target point n c is added up to one.Then, the maximum element of m i will be stored to the m t (line 13-16 of Algorithm 1).After traversing all the target points, if the n c is equal to the number of target points, which means the surgical robot can reach all the target points under the current RCM constraint, or this point is unreachable.Then, the RCM point with the mean manipulability index is stored in the RM (line 18-21 of Algorithm 1).After traversing all the RCM points in the possible workspace, the final RM is constructed.

Optimization Method
The RMs of two common installation configurations are generated with the proposed RM generation algorithm, shown in Section 6.The results show that the endoscope system has a larger workspace and higher manipulability when the endoscope direction is vertical to the end-effector's axis of the robot.Therefore, this installation configuration is chosen as the favored configuration in this work, as shown in Figure 7.
Selecting the robot base position should obey the following principles: the robot should reach all the points in the task plane T with the trocar constraint; the robot should avoid collision with other obstacles in the operation room as much as possible.In the LS, the trocar position is usually determined by the surgeon.After selecting the trocar point, the robot base can be placed theoretically at the points on the RM, which is centered on the trocar point.To ensure the safety of the surgery, the points where collisions may occur will be removed.Finally, the layout with maximum manipulability in the rest area is chosen as the final optimized layout.
Before the surgical procedure, the trocar position and endoscope's initial posture are determined by the surgeon, and the endoscope's installation configuration and robot base's layout can be selected with the above method.Then, all the possible poses of the robot can be calculated with inverse kinematics.The pose with maximum manipulability is selected as the final optimized pose, which will be considered as a constraint condition added to the visual servoing control scheme.

Automatic Control Scheme of EHS
With the optimization method mentioned above, the final installation configuration of the endoscope on the robot is shown in Figure 7.In this section, the optimized initial pose recognized as another constraint condition, together with RCM constraint and physical limit, is unified as a visual servoing control scheme formulated as a QP problem.Most existing control schemes are based on the Jacobian matrix and its pseudoinverse. [19,34]hen dealing with the physical limits, the method based on the pseudoinverse needs to design an implicit performance parameter, which is tedious and laborious.Recent research [22,35] show that the QP problem can be solved more favorably based on

Visual Servoing Control Scheme Considering Repetitive Motion Planning
For the EHS, the velocity-level kinematic mapping between the robot pose and image space with the RCM constraints can be described as: where J m ∈ ℝ 5Â11 is Jacobian matrix of the flexible EHS with the RCM constraint; q ¼ q1 , • • • , q6 , q7 , ˙lc , ψ, θ Â Ã T ∈ ℝ 10 denotes the joint-velocity vector; q1 to q6 represent the joint velocity of the UR5; q7 expresses the rotation velocity of the endoscope; ψ and θ denote the bending direction and bending angle of the flexible joint, respectively, as shown in Figure 7; ˙lc represents the moving velocity of the constraint tube; T are the middle point in the FOV; p rcm is the position of the RCM point in the world frame.Then, taking into account the physical, RCM, and pose constraints, the visual servoing control scheme is readily described as: where 11, is a weight matrix for achieving weighted motions of the EHS; b q ¼ γ q m À q m 0 ð Þ ð Þ∈ ℝ 11 and q m ¼ q, β ½ T are the current pose of the joint-angle vector, q m 0 ð Þ is the optimized initial pose of the joint-angle vector, the design parameter γ can scale the magnitude of the robot response to the joint displacement; J m ∈ ℝ 5Â11 is Jacobian matrix of the flexible endoscope system with the RCM constraint [25] ; , and χ denote a positive constant, and x and q m with superscripts þ and À denote the upper and lower of joint-angle and joint-velocity limit, respectively.

Neural Network Controller for Visual Servoing
, the Hessian of f can be obtained as is positive definite.Thus, the objective function ( 8) is convex.
According to the Karush-Kuhn-Tucker conditions, [36] x is an optimal solution to the bound-constraint QP problem ( 8), (9), and (10) if and only if there exists the decision vector l ∈ ℝ 5 and m ∈ ℝ 11 such that and 8 > < > : The complementary condition ( 14) can be described as an equivalent piecewise linear equation where the projector operator.
projects the z i onto the projection set It should be noted that W is invertible.Its inverse matrix is defined as M ¼ W À1 .With (12), the following equation can be deduced as: where Then, l can be described as by substituting the above equation into (12), it is readily to have where 19) into (15), one can have Inspired by the previous studies, [22,35] an RNN is designed to solve m with the following dynamical equation: Since decision vector x and matrices U and V are related to the inverse of matrix H and H may be singular, the following dynamics is applied to solve the matrix inversion problem: where X ∈ ℝ 5Â5 convergences to inverse of matrix H, ℱ ⋅ ð Þ represents mapping array which is composed of activation functions (AFs) f ⋅ ð Þ.To increase the convergence of DNN, the following sign-bi-power AF [22] is adopted: where sign ⋅ ð Þ is the sign function, parameter m ∈ 0, 1 ð Þ.Then, the RNN model used to solve the QP problem ( 8), (9), and ( 10) is finally described as and output vector

Optimized Result
The RM algorithm is implemented on MATLAB, and the robotic toolbox is used to check self-collision.Before generating RMs, the dimension and sampling intervals of the possible workspace and target plane should be defined first.The mean chest breadth and depth for the normal adult are 31.04AE 3.3 cm and 20.53 AE 1.7 cm. [37]Therefore, the radius and height of the possible workspace for the EHS are defined as r = 850 mm and l 1 = l 2 = 50 mm, respectively.The sampling intervals of the radius and height of the possible workspace are set as 50 and 20 mm.The sampling target points are distributed on the circular plane T with radius r t = 150 mm.The sampling intervals of the radius and angle of the circular plane are 30 mm and 10°.The distance between the trocar position and the end-effector position is 392 mm.Then, two endoscope systems with different installation configurations are implemented to transverse the sampling points in the possible workspace to reach all the sampling points in the target plane.The manipulability of the robot when traversing the target point is calculated using the method, as shown in Algorithm 1.It should be noted that the calculated manipulability can only be shown in RM when the robot can reach all target points with the trocar constraint.RMs for two endoscope systems are generated with the proposed RM-generated algorithm.The result is shown in Figure 8.The workspace's radius of configuration I is from 150 to 850 mm, while configuration II is from 200 to 700 mm.The result shows that when the direction vector of the endoscope is perpendicular to the robot end-effector's, the robot has a larger workspace and higher manipulability.Figure 8a also shows that when the distance between the trocar point and robot base is changed from 550 to 750 mm (yellow area), the EHS has higher operability.Therefore, this area should have a higher priority when selecting the layout position.In this article, this distance is set as 680 mm, and the initial pose of the endoscope is perpendicular to the target plane.After traversing all the possible poses of the robot to catch the perpendicular endoscope, the pose with maximum manipulability is q r 0 ð Þ ¼ À3.02, À 2.16, À 1.65, 0.67, 1.76, π ½ T rad and will be selected as the optimized pose.

Manipulability Comparison
To validate the effectiveness of the proposed optimized method, the robot with different initial poses is used to finish the same visual servoing task under RCM constraints, physical limit, and initial pose constraint in the virtual robot experimentation platform (V-ERP), as shown in Figure 9a,b.The initial poses containing optimized pose and randomly selected poses are shown in Table 2.The initial pose of the endoscope system is set as q m 0 ð Þ ¼ q r 0 ð Þ, 0, 0, 0, 0.95 ½ T .The joint-angle and joint-velocity limits are set as q þ m ¼ 2π, 2π, 2π, 2π, 2π, 2π, π, 60, 10 6 , π, ½ 0.99 T , q À m ¼ À2π, À 2π, À 2π, À 2π, À 2π, À 2π, À π, 0, ½ À 10 6 , À π, 0.05 T , and qþ respectively.W ¼ diag 50, 50, 50, 50, 50, 50, ð 500, 500, 1, 1, 500Þ is the weight matrix.The green ball is the tracking target that follows a sequence on a numbered board, as shown in Figure 9c.The green ball moves from number 1 to number 9, then moves again to number 1.During the simulation, the endoscope system can track the moving green ball and keep the green ball in the center of the FOV.The corresponding manipulability of the UR5 robot is recorded and shown in Figure 9d.The result shows that although the robot with different poses can accomplish the visual servoing tasks, the robot with the optimized pose has the largest manipulability (red line in Figure 9d).

Pose Constraint Validation
To validate the superiority of the flexible endoscope system, four comparative scenarios are implemented in the V-ERP, as shown in Figure 9a.In the first case, the robotic flexible endoscope system is performed to track a green ball, which follows a sequence on a numbered board, with a pose constraint visual servoing control scheme.The scale parameter γ is set as 100.When the green ball moves away from the view center, the pixel errors u-u 0 and v-v 0 will increase abruptly.With the visual servoing control scheme, the endoscope system changes its pose to track the green ball, and pixel errors converge fast to zero, as shown in Figure 10a.The trocar position almost coincides with the RCM point, and position errors are rounded between À2 Â 10 À4 and 2 Â 10 À4 m, as shown in Figure 10b.The robot joint angles containing the robot arm and flexible joint have been shown in Figure 10c,d.The endoscope is performed to track the green ball with three sequences.After three sequences, the final camera view and pose of the robot coincide with the initial camera view and robot pose, as shown in Figure 11a,c.During the tracking sequences, the pixel trajectory stays roughly the same, as shown in Figure 11b.In the second case, the pose constraint is released to check the performance of the endoscope system in the same tracking task.The result shows that the visual field of the endoscope is rotated during the tracking sequences.After finishing three tracking sequences, the rotation angle is up to 12°, as shown in Figure 11d,e.This will increase the misorientation of the surgeon's eye-hand coordination.In addition, the EHS's final pose is obvious different from the initial pose, which Table 2. Initial pose configurations of the robot.
q r ð0Þ q 1 q 2 q 3 q 4 q 5 q 6 Optimized pose  robot arm keep consistent with the initial pose as much as possible in the tracking process, which effectively inhibits the joint drift of the robot arm, thus reducing the misorientation of the surgeon's eye-hand coordination and further improving the safety of the EHS.

Deformation Capability Validation
As concluded in Section 2, the proposed flexible mechanism has the advantage of high compressional and torsional stiffness compared to the pure helix-based structure, which has been validated with finite element analysis.In this experiment, the bending capability of the flexible section is tested with a physical experiment.The prototype of the flexible section is made of nylon using SLS.Its design diameters are shown in Figure 3.The experiment is shown in Figure 12.The bending of the flexible section is controlled by four cables.When an external force acts on one of the ropes, the flexible section begins to bend, and the bending angle increases with the increase of rope force.The result shows that the maximum bending angle of the flexible section can be up to 240°, showing good flexibility.When the external force gradually decreases, the flexible section gradually returns to its initial state.The results also reveal that the flexible section achieves a constant curvature when the compressive forces act on the joint, as shown in Figure 12, which is conducive to the precise control of joints.

Control Infrastructure
To verify the physical feasibility of the visual servoing scheme ( 8)- (10) and RNN solver (24), the experimentation of the method is conducted on the proposed flexible endoscope system.A surgical instrument with a colored marker is recognized as a tracking target.The programs for the experiments containing the visual servoing control scheme, marker detection, and motor control section are written as Robot Operating System (ROS) nodes coded in Cþþ on the computer with Ubuntu and ROS installed.
The motor controller section is based on the open-source Simple Open EtherCAT Master Library to perform the PC and motor driver communication.In the following experiments, the endoscope system with and without pose constraints is conducted to track the moving instrument, whose setup is similar to pose constraint validation in simulation.After that, the tracking speed of the endoscope system is evaluated.Finally, the feasibility of the endoscope system in settings similar to LS is tested.All the videos of the physical experiments have been uploaded online.

Object Tracking Performance
To validate the effectiveness of pose constraint on the visual servoing process, two cases similar to constraint validation in the simulation are conducted on the physical flexible endoscope system.Two cases have the same optimized initial poses, which are the same as poses set in the simulation, as shown in Figure 13a and 14a.In the first case, the instrument detected by a colored marker moves manually to follow a sequence on a numbered board.This instrument moves from number 1 to number 9, then to number 1 again.It moves to the next number only when the system tracks the target to the center of the camera view.During visual servoing process, the endoscope's FOV is rotated slowly, and the robot gradually deviates away from the initial pose, as shown in Figure 13.The rotation angle is about 15°after four tracking cycles.In the second case, the addition of the pose constraint greatly reduces the misorientation between the eye and hand coordination in the visual servoing process, as shown in Figure 14b,c.Meanwhile, the coincidence of the robot's initial and final pose means the joint drift phenomenon has also disappeared, as shown in Figure 14a,d.The effectiveness of the pose constraint for visual servoing on the physical endoscope is thus validated.
Figure 12.Bending motion of the flexible section.a-e) Deformation with free bending phase.f-j) Deformation with unbending phase.

Tracking Speed Test
To evaluate the tracking ability of the physical endoscope system, the system is conducted to track a motorized target.The experimental setup is shown in Figure 15a,b.A colored marker is attached to a moving platform, which is driven by a stepper motor, as shown in Figure 15b.The marker moves from the view center to the right for 120 mm, then to the left for 240 mm, and then back to the view center.The moving velocity is set from 10 to 30 mm s À1 with a 5 mm s À1 increment.When the robot successfully repeats the tracking task 3 times, the moving velocity increases 5 mm s À1 .During the tracking process, the endoscope system can track the target successfully at 10-20 mm s À1 every time, fails one time at 25 mm s À1 , and always fails at 30 mm s À1 , as shown in Figure 15d.The result shows that the system can track the moving target at a velocity 20 mm s À1 over at least 240 mm distance.Note that the mean velocity of the instrument in the surgical procedure ranges from 22 to 26 mm s À1 for experienced surgeons.This test demonstrates the certain tracking ability of the flexible endoscope system.In addition, the robot's initial and final pose are recorded in the tracking process, as shown in Figure 15c.The result also shows that the two poses are almost coincident, which also validates the effectiveness of the pose constraint.

Ex Vivo Experiment
To test the practicality of the flexible endoscope system, the experimentation of the method is conducted in settings similar to LS.The experimental setup is shown in

Conclusion
Aiming at the limitations of the endoscope in the LS, a novel flexible EHS is proposed in this article.The proposed EHS consists of a UR5 robot arm and a flexible endoscope.The flexible section of the endoscope adopts a novel reinforced helix-based structure, whose helix strands are constrained by serials of rings.
Compared to the pure helix structure, the proposed structure has higher compressional and torsional stiffness while preserving good compliance.Its maximum bending angle can be up to 240°.Thus, the proposed flexible endoscope can provide a broader view.Also, its bending deformation is consistent with the constant curvature assumption, which is conducive to precise deformation control.In the structure can be manufactured of nylon with SLS technology, thus requiring fewer assembly works.To optimize the installation configuration of the endoscope on the robot, an RM-generated approach is proposed.With the RMs for two common installation configurations, the configuration of the endoscope and robot base position can be optimized.Then, the optimized pose of the robot can be calculated with inverse kinematics.Thus, the optimized method can provide a reference for configuration, layout, and pose selection.The optimized pose is considered a constraint condition and collaborated with the visual servoing control scheme to eliminate the misorientation caused by joint drift of the robot arm.In the validated experiments, the better maneuverability of the optimized pose is first verified.Then, the constraint-validated simulative and physical experiments are performed to evaluate the influence of the pose constraint on the tracking process.
The results show that adding pose constraint can effectively reduce the joint drift and misorientation in the visual servoing process.All those observations demonstrate that the safety of the system is enhanced.The ex vivo experiment shows the potential of the endoscope system in the LS.However, the detection based on the colored marker is not stable in the LS process since it is easily influenced by illumination condition change and has a risk of contamination by blood, fat, etc.The instrument segmentation or detection based on the deep learning method could provide an alternative method in our future work.Except that, the proposed visual servoing scheme only achieves 2-DoF (u and v) control in the image plane.The zooming of the endoscope is also important in the surgical process.Therefore, the visual servoing with 3-DoF control will be developed in the future, such as based on the image moment method.

Figure 1 .
Figure 1.The illustration of misorientation problem: although the operator's hand coordinate has not changed, the eye coordinate has been deflected in the FOV, and deflection is more obvious with time.Frame {e} and {h} are the operator's eye and hand coordinates.a) Excellent eye-hand coordination is preset by surgeon before the surgical operation.b) Eye-hand coordination is a slight inconsistency during a period of operation.c) Poor eye-hand coordination appears after a long operation.

Figure 2 .
Figure 2. Design of the DNA-inspired continuum mechanism.a) DNA structure.b) 3D model of the helix-based continuum mechanism with reinforced rings (blue).

Figure 5a .Figure 3 .
Figure 3. a) The simple helix structure with double helixes.b) The DNA-inspired continuum mechanism.c-e) The compressional, torsional, and bending deformation with the axis force, twisting, and bending moment in the Abaqus.

Figure 4 .
Figure 4. Stiffness comparison of simple helix structure and DNA-inspired structure.a) Axial deformation with compression force.b) Torsional deformation with torsional moment.c) Bending deformation with bending moment.

Figure 5 .
Figure 5. 3D model of the flexible endoscope.a) Flexible unit.b) Actuation unit.c) Mechanical transmission unit.

Figure 6 .
Figure 6.a) The task space T definition.b) The possible workspace ℛ definition.

Figure 8 .
Figure 8. RMs of two surgical robot systems.a) Configuration I: endoscope direction is perpendicular to robot end-effector's axis.b) Configuration II: endoscope axis is parallel to robot end-effector.

Figure 9 .
Figure 9. Manipulability comparison of flexible endoscope with optimized pose and random pose.a) Optimized initial pose.b) Random initial poses.c) Tracking sequence.d) Manipulability.

Figure 10 .Figure 11 .
Figure 10.Visual servoing of the robotic endoscope system with the RCM, physical, and pose constraints handled.a) Pixel error e. b) RCM error.c) Robot joint angle.d) Flexible joint angle.

Figure 16a .Figure 13 .
Figure 13.Visual servoing with pose constraint.a) Experimental setup.b) Initial camera view.c) Final camera view after four tracking cycles.d) Initial and final pose comparison.

Figure 14 .
Figure 14.Visual servoing without pose constraint.a) Experimental setup.b) Initial camera view.c) Final camera view after four tracking cycles.d) Initial and final pose comparison.

Figure 15 .
Figure 15.Tracking speed test.a) Experimental setup.b) Motorized platform.c) Initial and final pose comparison.d) Pixel errors u-u 0 and v-v 0 during tracking process.e) Tracking failed point.

Figure 16 .
Figure 16.Ex vivo experiment of the flexible endoscope for visual servoing.a) Experimental setup.b-g) Snapshots of flexible section motion and camera views.

Table 1 .
Function description of Algorithm 1.
online optimization techniques.Therefore, a solver based on RNN is developed to solve this QP problem.