Get access

Objective assessment in residency-based training for transoral robotic surgery§


  • This work was supported in part by NIH grant #1R21EB009143-01A1 and Johns Hopkins Hospital internal funds. A collaborative agreement with Intuitive Surgical, Inc. with no financial support provided the da Vinci robotic instruments and access to the application programming interface data. The sponsors played no role in study development, data collection, interpretation of data, or analysis.

  • Data collection was supported by donated surgical instruments, and a collaborative agreement with Intuitive Surgical, Inc. provided access to the application programming interface motion data. Neither agreement included any financial support. The study sponsors played no role in collection, analysis, or interpretation of data. Jeremy Richmon, MD provided proctoring services for robotic surgery training to trainee surgeons. Such (compensated) proctoring is unrelated to the presented work and does not create any conflicts of interest. Joseph Califano, MD, is the Director of Research of the Milton J. Dance Head and Neck Endowment. The terms of this arrangement are being managed by the Johns Hopkins University in accordance with its conflict of interest policies.

  • §

    Authors Martin Curry, DO, Jeremy Richmon, MD, and Rajesh Kumar, PhD, designed the study and were involved in data acquisition, developed the analysis methods, and performed the evaluation. Anand Malpani, BTech, Amod Jog, BTech, and Thomas Tantillo, BS, were involved in the data collection. Authors Anand Malpani, BTech, and Rajesh Kumar, PhD, performed the automated data analysis. Authors Jeremy Richmon, MD, Rajesh Kumar, PhD, Ray Blanco, MD, Patrick K. Ha, MD, and Joseph Califano, MD were involved in clinical interpretation of the data and analysis.

  • The authors have no other funding, financial relationships, or conflicts of interest to disclose.



To develop a robotic surgery training regimen integrating objective skill assessment for otolaryngology and head and neck surgery trainees consisting of training modules of increasing complexity leading up to procedure-specific training. In particular, we investigated applications of such a training approach for surgical extirpation of oropharyngeal tumors via a transoral approach using the da Vinci robotic system.

Study Design:

Prospective blinded data collection and objective evaluation (Objective Structured Assessment of Technical Skills [OSATS]) of three distinct phases using the da Vinci robotic surgical system in an academic university medical engineering/computer science laboratory setting.


Between September 2010 and July 2011, eight otolaryngology–head and neck surgery residents and four staff experts from an academic hospital participated in three distinct phases of robotic surgery training involving 1) robotic platform operational skills, 2) set up of the patient side system, and 3) a complete ex vivo surgical extirpation of an oropharyngeal tumor located in the base of tongue. Trainees performed multiple (four) approximately equally spaced training sessions in each stage of the training. In addition to trainees, baseline performance data were obtained for the experts. Each surgical stage was documented with motion and event data captured from the application programming interfaces of the da Vinci system, as well as separate video cameras as appropriate. All data were assessed using automated skill measures of task efficiency and correlated with structured assessment (OSATS and similar Likert scale) from three experts to assess expert and trainee differences and compute automated and expert assessed learning curves.


Our data show that such training results in an improved didactic robotic knowledge base and improved clinical efficiency with respect to the set up and console manipulation. Experts (e.g., average OSATS, 25; standard deviation [SD], 3.1; module 1, suturing) and trainees (average OSATS, 15.9; SD, 3.9; week 1) are well separated at the beginning of the training, and the separation reduces significantly (expert average OSATS, 27.6; SD, 2.7; trainee average OSATS, 24.2; SD, 6.8; module 3) at the conclusion of the training. Learning curves in each of the three stages show diminishing differences between the experts and trainees, which is also consistent with expert assessment. Subjective assessment by experts verified the clinical utility of the module 3 surgical environment, and a survey of trainees consistently rated the curriculum as very useful in progression to human operating room assistance.


Structured curricular robotic surgery training with objective assessment promises to reduce the overhead for mentors, allow detailed assessment of human-machine interface skills, and create customized training models for individualized training. This preliminary study verifies the utility of such training in improving human-machine operations skills (module 1), and operating room and surgical skills (modules 2 and 3). In contrast to current coarse measures of total operating time and subjective assessment of error for short mass training sessions, these methods may allow individual tasks to be removed from the trainee regimen when skill levels are within the standard deviation of the experts for these tasks, which can greatly enhance overall efficiency of the training regimen and allow time for additional and more complex training to be incorporated in the same time frame. Laryngoscope, 2012