Randomized controlled trial of virtual reality and hybrid simulation for robotic surgical training

Authors

  • Andrew Feifer,

    Corresponding author
    1. McGill University Health Center, Division of Urology, Department of Surgery, Montreal, QuebecBernstein Center for Minimally Invasive Surgery, McGill University Health Center
    2. Department of Surgery (Urology), Memorial Sloan Kettering Cancer Center, New York, NY, USA
    Search for more papers by this author
  • Adel Al-Ammari,

    1. McGill University Health Center, Division of Urology, Department of Surgery, Montreal, QuebecBernstein Center for Minimally Invasive Surgery, McGill University Health Center
    Search for more papers by this author
  • Evan Kovac,

    1. McGill University Health Center, Division of Urology, Department of Surgery, Montreal, QuebecBernstein Center for Minimally Invasive Surgery, McGill University Health Center
    Search for more papers by this author
  • Josee Delisle,

    1. McGill University Health Center, Division of Urology, Department of Surgery, Montreal, QuebecBernstein Center for Minimally Invasive Surgery, McGill University Health Center
    Search for more papers by this author
  • Serge Carrier,

    1. McGill University Health Center, Division of Urology, Department of Surgery, Montreal, QuebecBernstein Center for Minimally Invasive Surgery, McGill University Health Center
    2. Department of Surgery (Urology), Memorial Sloan Kettering Cancer Center, New York, NY, USA
    Search for more papers by this author
  • Maurice Anidjar

    1. McGill University Health Center, Division of Urology, Department of Surgery, Montreal, QuebecBernstein Center for Minimally Invasive Surgery, McGill University Health Center
    2. Sir Mortimer B. Davis Jewish general Hospital, Montreal, Quebec, Canada
    Search for more papers by this author

Andrew Feifer, Memorial Sloan-Kettering Cancer Center, 1275 York Avenue, New York, NY 10065, USA. e-mail: feifera@mskcc.org

Abstract

Study Type – Therapy (outcomes research)

Level of Evidence 2c

OBJECTIVE

  • • To evaluate if two commonly used laparoscopic simulators could be adapted and used successfully for the robotics platform in a laparoscopic and roboticnaïve medical student population.

MATERIALS AND METHODS

  • • We identified two widely validated laparoscopic simulation programs, LapSim® (Surgical Science Sweden AB), and ProMIS® (Haptica, Ireland)for inclusion in the study. The McGill Inanimate System for Training and Evaluation of Laparoscopic Skills® task set was used for ProMIS, and adapted for the DaVinci® console (Intuitive Surgical, Inc., Sunnyvale, CA, USA) robotic platform.
  • • We then randomized 20 naïve medical students to receive training on either LapSimor ProMIS, both or neither, and evaluated them beforeand aftertraining.

RESULTS

  • • When the groups were compared at baseline, there were no statistical differences in mean scores amongst the groups in univariate analysis (α= 0.05).
  • • Whencomparing mean scores within groups before and after training sessions, statistically significant performance enhancement in all four robotic tasks were identified in the groups receiving dual training.

CONCLUSION

  • • We have shownthat the use of ProMIS hybrid and LapSimvirtual reality (VR) simulators in conjunction with each other can considerable improve robotic console performance in novice medical students compared with hybrid and VR simulation alone.
Abbreviations
MISTELS

McGill Inanimate System for Training and Evaluation of Laparoscopic Skills

VR

virtual reality

INTRODUCTION

The paradigm shift from open to robotic-assisted surgery in the management of prostate cancer has prompted renewed focus on training and simulation. Whereas comparative effectiveness research is dominated by the applicability of robotics [1], technical feasibility [2] and oncological efficacy [3], few studies have investigated training-related educational obstacles. While the steep learning curve is well documented [4], the transmission of laparoscopic simulation tasks into robotic-oriented simulation has not been thoroughly evaluated. There are various robotic simulation models [4], but few have attained a level of validation meriting widespread incorporation into training programmes, despite extensive marketing.

The incorporation of robotics into urological practice and training has occurred in several ways. Fellowship-trained robotic surgeons have been largely responsible for the dissemination and continual technical improvement. Industry-sponsored seminars [5] or expert mentorships have aided open and pure laparoscopic surgeons to migrate to robotics if desired, nevertheless, this option is not generally available to resident and fellow cohorts. Trainees are thus faced with familiarizing themselves with the robotic platform in the shadow of their mentors’ own learning curve [6]. This understandably results in poor console time, small caseloads, and incomplete preparation for the current robotic-inclined environment [7]. Medical-legal and ethical concerns [8] serve as natural barriers preventing trainees from learning all tasks in the operative suite, despite residency objectives including robotic proficiency [9]. Several programmes have advocated a step-wise integration of robotics into residency programmes[10]. Nevertheless, training programmes continue to migrate to animal and computer-based simulation models for both the establishment of training benchmarks, as well as for skill grading and resident evaluation [11].

Laparoscopic simulation can be broadly divided into two technologies virtual reality (VR) and box-trainer. These are low-fidelity simulators compared withanimal and cadaver models. While these simulators are less life-like, their unique training utility has been demonstrated in isolation and repeatedly in combination with more realistic models. Non-animal or cadaver-based high-fidelity models have recently been studied as well. The Uro-scopic Endo-urology simulator (Limbs and Things, Bristol, UK) is a relatively expensive but life-like model. While advantages exist, studies by Matsumoto and others have shown very little training enhancement withsignificant start-up and maintenance requirements compared withlow-fidelity trainers [10], which are of lower cost and equivalent efficacy. Aside from recent validation studies involving the Mimic DaVinci-trainer [12–14], the use of more expensive simulation models over low-fidelity simulators for robotic training is still debated. Partly, this stems from an undercurrent of uncertainty onthe roles of haptic feedback, unclear task sets as well as the role of visual cues on the robotic platform. Despite differing opinions on the utility for laparoscopic training prior to robotics [15], the rate of acceptance of laparoscopic simulation currently exceeds that of robotic simulation, largely due to the lack entirely validated instruments. As a result, we assessed if two commonly used laparoscopic simulators could be adapted and used successfully for the robotics platform in a laparoscopic and robotic naïve medical student population.

MATERIALS AND METHODS

COHORT

After Institutional Review Board approval, 20medical students at identical training levels without prior laparoscopic or robotic experience (defined as <1 h of ‘hands on’ experience) were identified and randomized into fourgroups representing different practice sessions using a randomized block design [ProMIS® (Haptica, Ireland), LapSim,® (Surgical Science Sweden AB),ProMIS and LapSim, or no training;fivestudents per group). Baseline performance scores, followed by identical 3-month training periods, and a final performance measurement were completed from February to May 2009, at the Jewish General Hospital, Montreal, Quebec. Each trainee had five discrete training sessions, which were standardized by time throughout the 3-month period. There were equal periods between each session, and both the before and after testing session on the DaVinci robot. Each student underwent an identical number of training sessions. Those students who were trained on both simulators did so also in fiveequally distributed sessions within the 3 months. Each training session, including those with both LapSimand ProMIS were of identical time, and no double-training was performed in this cohort.

SIMULATION MODELS

We identified two widely validated laparoscopic simulation programs, LapSim, and the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills® (MISTELS) utilizing a hybrid augmented reality trainer, ProMIS. Previous investigations have shownface, construct and internal validity of MISTELS in the translation of laparoscopic skill from the simulator to the operative suite [16]. We have previously validated MISTELS in a urology cohort [17], as well as withProMIS[18], and it currently serves as the basis of the Fundamentals of Laparoscopic Skills course, given internationally by the Society of American Gastrointestinal and Endoscopic Surgeons and the American College of Surgeons [16]. The use of metrics in simulation has also been correlated with improved operative time [19], which is the primary reason we chose to use ProMIS and not to include a simple box-trainer with time measurement. While LapSimlacks the tactile feedback of box-trainers [20]using the MISTELS, the VRmodality provides several advantages for laparoscopic training [21], e.g. enhancing visual cues, and improving motion in a virtual environment [22]. The randomized students were evaluated before and after training using the MISTELS system adapted for the DaVinci® (Intuitive Surgical, Inc., Sunnyvale, CA, USA Intuitive surgical) robotic platform.

MISTELS TASKS AND TRAINING MODULES

For the performance measurement on the robot, all students were shown a 5-min demonstration video beforeusing the console for both sessions.The MISTELS tasks have been extensively described and validated elsewhere [17], but briefly consist of fivediscrete laparoscopic tasks, which were completed in this case with the robotic arms (Fig. 1); peg transfer, pattern cutting, extracorporeal/intracorporeal suturing and cannulation. As extracorporeal suturing is not applicable for robotics and the lack of data supporting its role in robotic technique education, we excluded this task and limited the MISTELS tasks to four. The identical MISTELS task set was used before and after each training session.

Figure 1.

MISTELS tasks: peg transfer, intracorporeal suturing, cutting, and cannulation.

LAPSIM

The LapSim (Fig. 2) is a computer-generated environment that mimics real surgical situations. Its VR simulation platform uses two instruments with position sensors in 6 d.f., allowing for continuous motion tracking, while providing objective measurement. Our analysis consisted of six discrete VR tasks that were practiced at the same level of difficulty by all students. The sixtasks used were instrument navigation, grasping, lifting and grasping, cutting, and clip applying.

Figure 2.

LapSim VR simulator.

The LapSim recorded performance on 6–10 parameters per task including precision, speed, smoothness, and degree of damage to surrounding tissue. Some performance parameters were common between tasks, whereas some tasks had only specific parameters.While the LapSim tasks differed from ProMIS, they are a validated and the underlying basic skill acquisition is comparable [23,24]. The average cost of LapSim in dollars (USA) is estimated to be $25 000.

PROMIS

The ProMIS hybrid simulator (Fig. 3) consists of a Toshiba® computer as well as alaparoscopic interface composed of a plastic mannequin 73.66 cm long × 50.8 cm wide × 22.86 cm deep with a black Neoprene® cover. The mannequin contains threecamera tracking systems arranged to identify any instrument inside the simulator from threeangles. The FlexCamTM iCam malleable camera captures instrument motion in the x, y and z coordinate planes at a rate of 30 frames/s. Precise measures of time, instrument path length and instrument smoothness, as detected by changes in instrument velocity, are recorded for each instrument ambidextrously during all simulated tasks. Four MISTELS tasks were completed in this platform; intracorporeal suturing, precision cutting, cannulation and peg transfer. The average cost in dollars (USA)of the entire ProMIS setup is estimated at $50 000.

Figure 3.

ProMIShybrid simulator.

STUDENT EVALUATION AND STATISTICAL ANALYSIS

The evaluation of students on the DaVinci console and ProMIS module both used the MISTELS task set, as described by the Fundamentals of Laparoscopic Skill course. For this, the DaVinci was docked in routine fashion to identical training modules to the ProMIS simulator tasks. The scoring at each session was accomplished via the universal MISTELS scoring system, which incorporates both an ideal time limit, as well as task-specific penalties for aberrant smoothness, and pathness.LapSimscoring was computerized, and the mean performance scores were used. Performance characteristics from the robotic sessions before and after training, as well as baseline characteristics of students were compared using SAS®V9.2 (SAS InstituteInc., NC, USA). The scores for the fivetraining sessions were tabulated and compared similarly using the Mann–Whitney U test.

RESULTS

All 20 medical students enrolled in the study completed the preliminary performance analysis, the five training sessions (if randomized in LapSim, ProMIS or LapSim and ProMIS groups) as well as the final performance analysis.The statistical results of the beforeand aftertest analyses are givenin Table 1. When the groups were compared at baseline, there were no statistical differences in the mean scores amongst the groups (α= 0.05). Whencomparing the mean scores within groups between beforeand aftertraining sessions, statistically significant performance enhancement in all four robotic tasks were identified in the groups receiving dual training (LapSim and ProMIS, P < 0.05; Table 1). The comparison of the beforeand aftertraining scores for those students trained on the LapSimalone showed performance improvement in the cannulation task, but there was no overall score improvement, consistent with improved performance. Although score improvement in the ProMIS alone group was evident overall, this was expected due to enhanced repetition and familiarity with the tasks. Students who received no training faired much worse than other groups, and did not show performance enhancement improvement.

Table 1.  Statistical analysis: before and after training results for MISTELS tasks performed with the DaVinci robot
GroupsNo practiceLapSimMProMISLapSimand ProMIS
  • *

    Statistically significant (alpha < 0.05).

TasksBaselineAfter trainingPBaselineAfter trainingPBaselineAfter trainingPBaselineAfter trainingP
Mean (sd) score
Peg transfer51.0 (31.3)69.4 (4.9)0.1261.4 (13.6)75.0 (14.5)0.2159.4 (9.0)74.2 (12.2)0.0655.0 (17.6)83.4 (10.3)0.02*
Cutting50.0 (12.9)45.6 (29.4)0.9229.0 (20.2)50.6 (14.5)0.1239.0 (14.6)49.2 (23.1)0.2930.8 (20.4)64.2 (13.3)0.03*
Intracorporeal knot40.4 (10.7)47.6 (21.6)0.2531.2 (32.4)42.6 (28.0)0.4648.8 (29.7)73.6 (11.3)0.0843.6 (12.6)72.8 (10.8)0.02*
Cannulation93.4 (9.6)100.4 (5.3)0.1888.0 (9.7)103.2 (4.3)0.01*96.6 (2.7)102.8 (3.1)0.009*91.8 (5.9)105.6 (1.1)0.009*
Total score234.6 (40.7)262.8 (45.8)0.46209.6 (55.4)271.4 (48.1)0.18236.2 (43.4)300.4 (41.3)0.03*221.2 (35.7)326.4 (23.0)0.009*
Total normalized score58.6 (10.3)65.6 (11.4)0.4652.6 (13.9)68.0 (12.0)0.1858.8 (10.9)75.2 (7.8)0.03*55.4 (9.2)81.6 (5.5)0.009*

DISCUSSION

The establishment of a robotic training program continues to be difficult amongst many urology residency programs across North America [25]. The cost as well as the current general lack of accepted robotic-assisted training modules underscores the preliminary nature of those endeavours. How to appropriately and safely educate future urologist about new and far-reaching surgical technology, while concomitantly adhering to the strictest standards of medical ethics, patient safety and legality is a daunting task [8]. The training-related scheduling restraints are also vital considerations [26].The previously accepted Halstedian surgical method is not sustainable for robotic training, and thus focus has shifted to inanimate simulation environments and animal models. Efforts to create sustained performance advancement and diminished learning curves prior to live surgical training have gained tremendous popularity [27].

The presentresults have shownthat the combination of classic laparoscopic box-training ‘motor skills’, the use of which is enhanced via motion analysis technology, with the visual cued-based advantages of VR technology [28] offer previously naïve students tremendous skill enhancement towards robotic movement efficiency. Although there was score improvement in the ProMIS alone group, the magnitude of score improvement was significantly greater in the group trained with both ProMIS and LapSim, suggesting an additive mechanism for complementary skills, which may reliably facilitate console work on the robotic platform.The differences in the nature of VR and MISTELS may serve to better prepare for DaVinci then either alone. If familiarization with MISTELS tasks also used in ProMIS were solely responsible for performance enhancement, these subjects would have shownbetter performance in all tasks in the ProMIS alone group, and this was not evident in the presentinvestigation. ProMIS training may provide the realism needed for novice trainees, while VR complements the basic skill acquisition with the ability to transfer the skills to an environment devoid of haptic feedback, similar in principle and practice to the robotic platform.

Others have studied robotics and simulation, although none have used or adapted previously validated and widely available laparoscopic simulators for this purpose.Research by Lendvay et al. [29] supports further exploration of the novel DaVinci simulator (DaVinci Trainer surgical system from MIMIC Tech or Equal). Kenney et al. [13] recently reported face, content and construct validity of the DaVincisimulator. In this analysis, the investigators showed that the trainer allowed for the objective differentiation of novice and experienced robotic surgeons, and also that most participants thought the simulator was helpful. While this series does go through the initial step of exploring simulation utility delineated elsewhere [30], the preliminary nature of this investigation with the lack of concurrent or predictive validity, coupled with poor suturing module performance do not support the swift acceptance and incorporation of this simulator into training programmes. Other publications by Sethi et al. [12] support the initial findings by Kenney et al. [13], but also do not address predictive and concurrent validity.

Sun et al[31]. devised a novel computer-based simulator for the robotics platform. Although the VR model showed applicability to training for robotics, the authors recognised the preliminary nature of the work, and are anticipating future validation studies.Katsavelis et al. [32].have similarly reported on the investigation of a novel VR DaVinci simulator. The authors hoped to provide construct validity by correlating VR and real-time movement in a set of manual carrying and suturing tasks. Although good correlations existed in the bimanual carrying task between real and virtual spaces based on electromyogram muscle readouts and movement metrics, the suturing tasks did not correlate well. Thatstudy nicely showed the obstacles that are faced when creating a virtual environment mimicking real tasks purpose, realism and reproducibility. Lastly, Mehrabi et al. [33]. have devised a step-wise animal practice model for use with the DaVinci system, which confirmed the shortening of learning curves amongst trainees, with the ability to constantly monitor training progress. Such animal models might serve as complementary training modules for inanimate robotic simulation.

In the present study, we adapted two validated and accepted methodologies for laparoscopic simulation to the DaVinci trainer, acknowledging their positive features, and widespread acceptance. While the lack of direct robotic console involvement may be viewed as a drawback, there is currently no evidence to support pure robotic-oriented training, without focus on basic laparoscopic skills. Many authors support the notion that robotic tasks are facilitated by pure laparoscopic expertise [7]. We feel that the familiarization of novice students with laparoscopic skills sets, and the ability to manipulate real instruments in two and three dimensional environments, while concomitantly attuning their visual acuity using VR, offers superior additive simulation for surgical training in general, as well as optimizing skills for the robotic platform. This can be used as a stepping stone towards robotic training at the senior resident level, or as a potential adjunct to robotic experience.This benefit is more than can be explained by acclimatization to MISTELS tasks alone.A side benefit of using ProMIS with LapSim is the cost savings, as the average start-up and maintenance costs of these systems is far less than comparative products. These simulators are widely available and can be used in robotics training modules at a fraction of the cost of newer, more expensive and non-validated robotic-only simulators. Although ProMIS did show its ability to improve performance alone, we believe in the additive benefit of VR education in preparation for a comparable robotic environment. The higher cost of these more complex simulation modules may preclude their widespread application and limit their use to large academic centres, at least until a pure robotic simulator is completely validated, and priced more competitively.

Importantly, the small sample size and group numbers are limitations of the present study. Although some may feel that the useof medical students is a limitation, we disagree, and feel that their use serves to highlight simulation training differences by elimination significant biases and experiences that would surface if residents, fellows or staff were used for this exploratory analysis, Certainly, the inclusion of these secondary groups can be considered at later stages of validation. Lastly, while we have shownthe ability of combination simulation training to improve robotic skills, secondary investigations merit consideration after establishing standards that can be universally applied towards the education of novice surgeons inpreparation for robotic procedures.

In conclusion, we have shownthat the use of ProMIS hybrid and LapSim VR simulators in conjunction with each other can improve robotic console performance in novice medical students. Similarly, the use of MISTELS tasks can also be adapted for the DaVinci platform. Until pure robotic simulators are both validated and cost-effective, the utility of ProMIS and LapSim in the further training of surgical trainees cannot be understated.

ACKNOWLEDGEMENTS

The authors would like to thank Mrs Andrea Senay (research assistant) as well as Claire Deland, the head nurse of the Jewish general Hospital robotic surgery programme.

CONFLICT OF INTEREST

This study was supported by an unrestricted educational grant from Tyco Healthcare Canada and Abbott Pharmaceuticals. Josee Delisle declares a financial interest with Abbott Oncology.

Ancillary