Admittance-Controlled Robotic Assistant for Fibula Osteotomies in Mandible Reconstruction Surgery

— This paper proposes a semi-autonomous robot control system for mandible reconstruction surgery. To reconstruct a segmental defect of the mandible caused by cancerous tissue, a piece of matched fibula bone is often segmented and used to replace the removed mandible section. In this paper, to provide guidance to the surgeon during fibula segmentation according to the reconstruction surgical plan and to improve the fibula

patient's mandible, a mandibulectomy surgery to remove all or part of the mandible and the tumor tissue around it may be performed.Specifically, for segmental mandibulectomy, where an entire segment of the mandible is removed, a bony autograft is necessary so as to reconstruct the length and the form of the mandible to produce the best functional and aesthetic results.Fibula free-flap reconstruction is used widely by taking a fibula bone segment, soft tissue from the calf and the fibular vascular pedicle which will be transferred and anastomosed to cervical arteries and veins for the blood supplies of the graft [4][5][6].The fibula bone segment should be shaped to match, as closely as possible, the piece of the mandible that was removed.The main difficulty of this segmentation is to be able to make the fibula bone which is straight into a curved bone piece so as to match the mandibular resection.
Traditionally, the mandible resection and fibula osteotomies are performed without guidance.This is unguided freehand surgery.The advent of preoperative virtual surgical planning with rapid prototyped models of the patient's bony anatomy and patient-specific cutting templates has led to significant refinements in mandible reconstruction [7][8][9][10] in the form of template-guided surgery.However, manufacturing a surgical toolkit is time-consuming and modification of the cutting and reconstruction templates is difficult once the surgical toolkit is printed.
Despite the costs, preparation times, and lack of intraoperative flexibility, template-guided surgery remains an attractive approach for mandible reconstruction with fibula flaps by improving the surgical outcomes [11], [12].An alternative method is image-guided surgical navigation based on preoperative imaging data [13].Using such an approach, by depicting the cutting plane and osteotomy lines in the skeletal anatomy, the systems provide real-time visual guidance to the surgeon.Notwithstanding the clinical benefits of image-guided surgical navigation, this method increases the difficulty to follow the position and orientation of the cutting plane precisely.This has motivated the development of robot-assisted (haptics-enabled) and image-guided systems for mandible resection and fibula harvesting.Compared to the template-guided surgery and (non-robotic) image-guided surgery approaches, robot-assisted image-guided surgery has significant advantages in terms of accommodating last-minute or intraoperative adjustments to the surgical plan, time and cost savings, and high precision [14][15][16].In this paper, we are interested in using a robotic assistant to achieve accurate fibula osteotomies for fibula free flap mandible reconstruction.Different from a fully-autonomous surgical robot, a semi-autonomous robotic assistant is developed to ensure safety [17], [18].To execute 3D fibula osteotomies, a surgeon manipulates a robot end-effector, which is attached to a surgical saw, to achieve collaboration between the surgeon and the robot.This co-manipulated robotic system separates the osteotomies tasks into two parts.One is the accuracy-critical part of the procedure with respect to controlling the position and orientation of the surgical saw blade according to the planned cutting plane, which will be aided by the robot.The other is the safety-critical part of the procedure (bone cutting) which will be performed under the direct control of the surgeon.
In our previous work [19], guidance virtual fixtures (VF) were used to constrain the motion of the surgical saw in the planned cutting surface.For human-robot cooperation, virtual fixtures provide an excellent balance between full autonomy and direct human control [20].However, this force guidance cannot meet sufficient bone cutting accuracy according to our previous work.This approach also has difficulty with visualizing 3D fibula osteotomies.Motivated by these shortcomings, in this study augmented reality (AR) guidance is proposed to in addition to the haptic VF guidance for increasing the saw positioning and orientating accuracy in fibula osteotomies.Recently, AR has been widely used in healthcare fields such as rehabilitation therapy [21] and brachytherapy [22] to assist surgeon operations.
In this paper, a novel semi-autonomous surgical robot control system for fibula osteotomies in mandible reconstruction surgery is proposed.It utilizes an admittance-controlled robot to provide a reference trajectory to the surgical robot by designing a desired relationship with measured interaction forces [23][24][25][26][27][28][29].The admittance controller is used to make the surgical robot compliant and enables the surgeon to adjust the position and orientation of the surgical saw when the saw is out of the cutting plane region.The admittance controller is working with four approaches as follows to implement fibula osteotomies and the experimental results will be compared to obtain the optimal approach.

1) Image-guided surgery; 2) AR-guided surgery; 3) VF-guided robot-assisted surgery; 4) AR-and VF-guided robot-assisted surgery.
The first approach only uses the admittance-controlled robot to move the surgical saw freely and track its motion.In this case, to provide the surgeon with visual feedback, two orthogonal views of the cutting scene (including the desired cutting planes) will be displayed to the user in a TV screen.The second approach is to employ an AR setup to directly project the planned 3D cutting planes onto the physical environment for visualization.The third approach involves haptic guidance using VF to provide kinesthetic guidance along desired trajectories.A combination guidance of AR and VF is proposed in the fourth approach.The hypothesis is that the fourth strategy involving visual-haptic assistance works best among the four control approaches in terms of both improving the accuracy of fibula osteotomies and reducing operative times.
To the best knowledge of the authors, this is the first work combining guidance of VF and AR for fibula osteotomies.The novel visual-haptic assistance is proposed to provide the operator with both 3D visual guidance and haptic guidance so that the task operation will be easier and more accurate compared to other three methods.
The rest of this paper is organized as follows.Section II provides an overview of the proposed semi-autonomous surgical control approaches.Section III outlines the system design and control algorithms for the surgical robot.Section IV describes the experimental protocol, results, and discussion.Finally, Section V presents concluding remarks and future directions.

II. OVERVIEW OF PROPOSED OPERATIONAL FRAMEWORK
For fibula osteotomies in fibula free flap mandible reconstruction surgery, firstly, preoperative high-resolution CT scans of a patient's fibula and mandible are obtained.As discussed above, in this study, we will focus on four methods to execute fibula osteotomies.In the image-guided surgery, the CT image is only for visual inspection.In the other three surgical guidance methods, the preoperative CT image data is processed to generate a 3D digital fibula model.Fibula cutting planes are designed and translated to AR software and/or robot software.An overview of the operational workflows of implementing fibula osteotomies using the proposed four methods is presented in Figure 1.
Method 1: Image-guided surgery.Only the preoperative CT image data will be used during operation.In order to provide visual feedback to the operator, a forward and a top view (two 2D planes) of the real-time actual and desired position and orientation of the saw blade will be displayed on a TV screen in front of the operator.Method 2: AR-guided surgery.The planned fibula cutting planes are registered to the AR system software so that the AR display can provide the operator with 3D information about the position and orientation of the planned cutting planes.Specifically, the cutting plane is calibrated to a plane containing the main axis of the fibula bone, and the images are reconstructed as a 3D object.The displayed virtual planned cutting planes will appear to float in front of the surgical scene (including the physical fibula and surgical saw) during the fibula bone cutting procedure.Therefore, before execution of fibula osteotomies, registration of stereotactic navigation between fibula and the AR setup should be performed.Method 3: VF-guided robot-assisted surgery.In a similar manner to the image-guided approach using AR, in the surgery with haptic guidance using VF, the planned fibula cutting planes are translated to the robot software to kinesthetically cue the position and orientation of the planned cutting planes to the operator.To this end, registration of stereotactic navigation between fibula and the robot coordinate system should be performed.Method 4: AR-and VF-guided robot-assisted surgery.To further provide visualization to the haptic-guided method, a combined strategy of Methods 2 and 3 is done.
Note that, in this study, the goal is to investigate the performance of the proposed four methods for fibula osteotomies.As proof-of-concept experimentations, we will focus on the system design and robot control algorithms shown in the dashed boxes in Figure 1.Experimental demonstration will depend on the simulated fibula bone and simulated cutting planes to represent the clinical operation.

III. SYSTEM DESIGN AND ROBOT CONTROL ALGORITHMS
To implement the four fibula osteotomies methods, specific system design and robot control algorithms should be implemented.Three reference frames are used in this paper: {R} indicates the robot base frame, {B} indicates the fibula bone frame, and {U} indicates the unity frame of AR.Transformation matrices relating to the three frames are determined.

A. Admittance Controller
The controller is designed to allow for the operator to collaborate with the robot assistant smoothly and to minimize the operator's hand tremor and the vibration caused by the surgical saw operation.By designing a desired relationship with measured force, admittance controllers could generate a desired motion for the robot.Alternatively, we could implement a robot impedance controller for generating torque commands to achieve the same objectives.However, in this case, the inverse dynamics of the robot would typically be required but they are by and large unavailable due to the complexity in estimating the dynamics of robots.
The controller design is outlined in Figure 2. Here, R h W is the interaction force-torque (wrench) between the robot end-effector and the operator, which is measured directly through a 6-DOF force sensor, as expressed in the frame {R}.The pre-programmed admittance model receives inputs, The desired admittance model in this study is designed as where C and M are the virtual damping and inertia (6-by-6 diagonal) matrices of the admittance model.In order to avoid restoring forces in free-space, the stiffness term is set to be zero matrix.Also, f k is a force scaling factor.
The parameters of the desired admittance model (diagonal variables of C and M) are chosen to be small values when the surgical saw is in free-space motion and larger values when the saw is in contact motion.A surface parallel to the ground is set as the divisional plane to split the free space (above) and workspace (below).The goal of this parameter adjustments is to guarantee the saw attached to the end-effector of the robot can be flexible enough to move freely in free space and rigid enough when performing cutting tasks.

B. Image-Guided Surgery
For image-guided surgery, the system displays two 2D images, a top and front view, and shows the current pose (position and orientation) of the surgical saw and projections of the desired cutting plane in both views.Thus, this image-guided surgery task is comparable to conventional surgical tool tracking, where a live image of the surgical tool is overlaid on top of X-ray or CT patient images.For visualization, a standard computer monitor was placed near the bone phantom and showed real-time cutting performance by displaying desired and actual tool positions and orientations.During the image-guided operation, the goal for the operator is to manipulate the saw blade in such a way that the actual saw pose, shown in the two 2D projections, matches the desired saw blade pose.The fibula phantom and surgical saw were modeled and displayed to the user using the Unity Engine (Version 2019.2.21 Unity Technologies, San Francisco, CA, USA).To track the pose of the saw blade, in real-time, the surgical saw was attached to the admittance-controlled robot with the controller tuned such that the saw feels to the user as if it is in free-space and the saw pose data was streamed to Unity for display.

C. AR-Guided Surgery
An AR display system will be used to evaluate the proposed system in the second surgical task.As with the 2D image-guided surgical task, there was no haptic feedback or guidance provided to the user.The AR display uses the reflection of a monitor in a half-silvered mirror to overlay a projection (in the same manner as a Pepper's ghost illusion) of the fibula, cutting planes, and surgical saw on top of the physical fibula phantom and saw.Thus, the semi-transparent mirror is used to present a virtual image of the projected surgical scene on the computer monitor (rendered using Unity) so that the operator sees both the virtual image and the surgical scene when looking through the mirror.The half-silver mirror AR setup used in this paper is an advanced  version of the prototype device presented by our group in [22].
While the AR display shows a 2D projection of the same Unity scene used for the image-guided task, the user perceives the projected virtual environment in 3D due to the parallax-from-motion effect [30], where the display uses head-tracking to update the projected image to match the position of the center of the user's eyes.Using the parallax-from-motion technique, the AR display gives the user a sense of a 3D virtual environment to show the user the planned 3D fibula cutting planes in the surgical scene.To track the user's head and update the projected image, a Micron optical tracker (ClaroNav, Toronto, ON, Canada) is used.The optical tracker ensures that the projected image moves in synchrony with the operator's head.The head tracking algorithm works with an oblique projection matrix to match the projected components of the virtual scene directly over their physical counterparts, such that the overlay of the virtual fibula, for instance, will match the pose of the physical fibula.

D. VF-Guided Robot-Assisted Surgery
Compared to the image-guided approach using AR for fibula osteotomies, an alternative is haptic guidance using VF.The advantage of haptic guidance is that it provides easier angular positioning compared to the image-guided AR approach.Specifically, guidance VF assists the operator in moving the admittance-controlled robot manipulator along the desired fibula cutting planes in the workspace so as to implement fibula osteotomies easily and potentially more accurately.
The overall controller design is shown in Figure 3.The force-torque guidance generated by VF is R v W which is transmitted to the admittance model.Note that R X is the actual pose of the center point of the saw blade, and R des X is the pose of the intersection between the planned cutting plane and its normal vector, which passes through the center point of the saw blade.
The desired robot admittance model when haptic feedback is provided to the user can be expressed as where diag( , , , , , ) is the stiffness matrix of the virtual fixtures model.The haptic feedback is designed with two objectives.First, for maximum cutting stability, the orientation of the saw blade is expected to be regulated with high feedback gains.Therefore, the gains for correcting angles ( ,, k k k    ) should be set as large values.Such orientation guidance will keep the saw blade parallel to the planning cutting plane.Second, forces generated by VF should provide kinesthetic clues as to the desired cutting point and enable the operator to collaborate with the robotic assistant.To this end, the gains for position guidance ( ,, x y z k k k ) should be moderate.

E. AR-and VF-Guided Robot-Assisted Surgery
In the haptic guidance surgery using VF, there was no visual feedback of the cutting accuracy.To further investigate if the visualization of planned 3D cutting planes would increase the operator's performance, a combined guided surgery is proposed by integrating above-described AR and VF guidance.

IV. EXPERIMENTS
Simulated experiments are implemented with the proposed four methods.A usability study emulating fibula osteotomies is carried out.The task is cutting along five planned cutting planes to evaluate the performance of cutting accuracy and operative time.

A. Experimental Setup
In this study, we employ a Panda Robotic Arm (Franka Emika GmbH, Munich, Germany) equipped with an Axia80-M20 force/torque sensor (ATI Industrial Automation, Inc., Apex, NC, USA) as the surgical robot (Figure 4).During the human-robot collaboration, the system provides an error margin of 2.61 x 10-4 m.A rotary saw (Dremel 4300-5/40, Toluca, Mexico) is attached to the robot end-effector through custom 3D-printed attachments.The feasibility of the proposed methods is verified through proof-of-concept experimentation by performing fibula osteotomies on several simulated fibula bones which are wood dowels with diameter of half inch.Although the stiffnesses of the wood dowel and the fibula bone have slight difference, considering both are rigid objects, the effect on robot control is trivial and negligible.In clinical application, the parameters of the admittance controller may need to be slightly adjusted based  on the stiffness of the fibula bone, and ideally, the cutting task performance will be the same.As discussed in Section III-C, the designed AR setup is presented in Figure 4.The TV screen is used to display front and top view of the workspace for image-guided surgery.A Unity Engine is utilized to develop a virtual environment for displaying in both AR monitor and TV screen.
MATLAB/Simulink R2019a (The MathWorks Inc, Natwick, MA, USA) was used to implement the admittance controller for the robot and haptic virtual fixtures.The MATLAB/Simulink R2019a software was installed on a PC running Ubuntu 16.04 LTS, containing an Intel Core i5-8400 running at 4.00 GHz (Intel Corporation, Santa Clara, CA, USA).The robot velocity controller was coded in C++ and implemented in Robot Operating System (ROS) making use of the Franka Control Interface library for the Panda robotic arm.The UDP blocks were used to communicate between the Simulink based models and the C++ based velocity controller for the robot with a sampling frequency of 1 kHz.
To achieve desired admittance control and haptic feedback performance and system stability, the parameters in Equations 1-3 are adjusted empirically and listed in Table I.In free space, the goal is to move the robot/saw to the workspace quickly and easily.Therefore, a small admittance of the robot and a relatively low values (compared to the value used for workspace) of the mass matrix were chosen.In workspace, the goal is to move the robot/saw accurately and safely and keep the system to be stable while the saw is working at a super high rotation rate.To guarantee the robot/saw work at a steady status, we increased the values of M and C. As a result, we chose the values listed in Table I through trial and error.

B. Experimental Protocol
For fibula free flap mandible reconstruction surgery, the number of segments needed varies.In this study, five planned cutting planes were considered for evaluation of the proposed four methods.
The planned fibula cutting planes are shown in Figure 5 (a) (front view).For the sake of brevity, the five planned cutting planes are described in the fibula bone frame {B}.All cutting planes are designed to be vertical to the front view and are indicated by a point i B and an angle i  .Here, i B is the position of the intersection between the cutting plane and the horizontal axis of frame {B}, and i  is the angle between the cutting plane and the horizontal axis of frame {B}.Specifically, the five planned cutting planes are designed as (0.13 m, 90  ), (0.11 m, 80  ), (0.09, 100  ), (0.07, 100  ), and (0.054, 90  ).Due to the size limitation of the rotary saw, the planned surgical planes cannot be cut randomly but have to be cut sequentially from right side to left. Figure 5 (b) shows the mirrored imaged displayed in AR monitor.This mirrored image is the same as the one seen by the operator through the mirror.Figure 5 (c) and (d) are the front and top view of the workspace, which are displayed on a TV screen for image-guided surgery.
Three operators (the first, second, and last author of the paper who are university post-doctoral fellows and professor without disability, two male and one female, two right-handed and one left-handed) performed the usability study involving implementing the simulated fibula osteotomies using the designed system and proposed four methods.For each cutting task (using one proposed method), the experiment was repeated five times randomized in any way by the three operators (the first author performed three times and the other operators performed one respectively).To investigate the accuracy of the osteotomized segments, the fibula segments were measured using digital calipers and compared to the planned cutting planes.

C. Experimental Results and Analysis
The usability study emulates the fibula cutting procedure.Three task-relevant measures including cut length accuracy, cut angle accuracy, and operative time were recorded.To measure the lengths and angles of the fibula segments effectively, the segments were placed in front view and only the top and bottom lengths and angles are measured.The   experimental results (shown as mean absolute error (MAE)  standard deviation) are listed in Table II.The results show that when guidance of AR and VF is provided simultaneously, both cutting accuracy and operative time are the lowest compared to the other three methods.
To present the obtained fibula segments using guidance of AR and VF, an example is shown in Figure 6 (b). Figure 6 (a) is the planned cutting planes from the front view.For each bone cutting task, 8 linear and 10 angular measurements are obtained and recorded.Among five repeated experiments, the residual length error is 1.04  0.79 mm, angle error is 1.83  1.85  , and operative time is 4.06  1.51 mins.The results of usability study present the dominant advantage of the proposed admittance-controlled system with guidance of AR and VF over the other three methods.
Statistical analysis involving a paired two-sided t-test is used to obtain the probability of the null hypothesis for the trials.Means are considered significantly different when P-value is less than 0.05.Table III shows the statistical analysis results for linear and angular measurements and operative times with respect to several groups of two cases.The P-values indicate that there is a significant difference between methods using VF and using AR & VF with respect to the length error and angle error.Also, the P-values promise a statistically significant difference between the AR-guided surgery and the AR-and VF-guided robot-assisted surgery with respect to angle error and operative time.Moreover, the P-values show that there is significant difference between image-guided surgery and AR-guided surgery with respect to operative time.

D. Discussion
The proposed admittance-controlled robot system cooperating with AR and VF is designed to increase accuracy of fibula osteotomies in fibula free flap mandible reconstruction.In our previous work [19], only haptic feedback using VF was employed.However, the system accuracy was not yet sufficient for clinical use.As shown in Table II, the residual length error using VF is 3.10  2.25 mm and the residual angle error using VF is 2.40  2.21  , which reconfirmed the system accuracy with haptic feedback was not enough for clinical operation.Two significant reasons for the large length error are that during operation the operator has no visual guidance to find the ideal cutting points, and simultaneously the cuing force generated by VF close to the desired cutting points is small.These lead to the difficulty for the operator to positioning the cutting point precisely.In contrast, the residual angle error using VF is not very large compared to that using both AR and VF.It is because VF can provide accurate angle positioning during operation.Therefore, it can be concluded that the presence of AR-based visual feedback assists the operator to enhance the cutting point positioning.
When experiments were performed with image guidance such as TV display and AR display, the residual length errors are less than that of using VF, but the angle errors and operative times of the former two methods are larger than those of the latter.It is probably because during operation the operator paid much attention to find the desired positions and orientations of the planned cutting planes.Although the angle errors using TV display and AR display are not very large, the cutting planes are not as smooth as those using VF as it is difficult for the operator to keep the saw blade in a fixed orientation during operation solely based on AR guidance.
Compared to TV display, both length accuracy and operative time with AR display are improved.It is because the AR image is projected directly on the fibula bone instead of on a separate screen.The operator only needs to focus on the AR image to execute the cutting task.In image-guided surgery using TV display, to obtain perfect cutting position and orientation the operator should pay attention to two views of the virtual image.If the operator would like to have a look at the cutting performance, he/she has to change his/her view from the TV screen to the real fibula bone, which may affect the cutting performance due to fast rotation of the saw.Additionally, following guidance in 3D is easier than 2D for human operators.
The comparative usability study demonstrates two points as follows: (a) The proposed admittance-controlled robotic assistant with AR and VF could significantly improve fibula osteotomies accuracy (both length and angle) and operative time; and (b) image-guided surgery using AR display could achieve higher cutting length precision and lower operative time than the surgery using TV display with two 2D view of the cutting planes.The contributions of this paper lay the foundation for the future implementation of even more complex fibula free flap mandible reconstruction experiments using surgical equipment such as surgical saw, 3D-printed fibula bone, and preoperative virtual surgical planning, etc.

V. CONCLUSION
A semi-autonomous admittance-controlled robot system incorporating haptic virtual fixtures (VF) and 3D augmented reality (AR) was proposed for fibula osteotomies in fibula free flap mandible reconstruction surgery.A usability study was carried out to show the feasibility and effectiveness of the proposed method by comparing the experimental results to the other three methods: image-guided surgery, AR-guided surgery, and VF-guided robot-assisted surgery.The proposed method was found to improve both the operative time and the precision of the osteotomized segments with lower average linear variation of 1.04  0.79 mm and lower average angular variation of 1.83  1.58  compared to the virtual preoperative plan.
The preclinical study demonstrated the feasibility of the proposed admittance-controlled robotic assistant with AR and VF for fibula osteotomies in mandible reconstruction.This method could effectively increase the accuracy of performing fibula osteotomies and reduce operation times compared to the other three methods investigated in the study.Future work will focus on translating the proof of concept experimentation to the clinic following a comprehensive user study.

Figure 1 .
Figure 1.Surgical workflows of robot-assisted fibula osteotomies for fibula free flap mandible reconstruction surgery.

Figure 2 .
Figure 2. Admittance controller for the robot.

W
, and generates reference Cartesian (both translational and angular) velocity for the robot, R ref V , to track.As the saw is mounted on the robot end-effector, for the sake of brevity, we use the position and orientation of the robot end-effector to indicate the position and orientation of the saw blade.A velocity controller is used for the robot and outputs control signals, R U , to the robot.The actual Cartesian velocity of the robot end-effector denotes as R V .

Figure 3 .
Figure 3. Admittance controller for the robot in addition to haptic virtue fixtures.

Figure 5 .
Figure 5. Planned cutting planes.(a) Coordinate systems and planned cutting planes.(b) 3D image viewed by operator in AR-guided surgery.(c) 2D image from the front view displayed on the TV screen in image-guided surgery.(d) 2D image from the top view displayed on the TV screen in image-guided surgery.

Figure 6 .
Figure 6.(a) Desired fibula segments from the front view.(b) Measured results of fibula segments implemented by using guidance with AR and VF from the front view.Unit is mm.

Table .
II. EXPERIMENTAL RESULTS