The effect of three‐dimensional visualisation on performance in endoscopic sinus surgery: A clinical training study using surgical navigation for movement analysis in a randomised crossover design

Abstract Objectives Endoscopic imaging techniques and endoscopic endonasal surgery (EES) expertise have evolved rapidly. Only few studies have assessed the effect of three‐dimensional (3D) endoscopy on endoscopic sinus surgery (ESS). The present study aimed to objectively and subjectively assess the additional value of 3D high‐definition (HD) endoscopy in ESS. Design A randomized crossover study of endoscopic surgery performance, using five ESS tasks of varying complexity, performed on Thiel embalmed human specimens. Setting Simulated surgical environment. Participants Thirty participants, inexperienced in ESS. Main outcome measures Performance was assessed using video imaging, surgical navigation and questionnaires. Main outcome measures were as follows: efficiency (defined by time to task completion), distance covered inside the nose, average velocity towards target, accuracy (measured by error rate), and subjective assessment of endoscope characteristics. Results During ESS tasks, both efficiency and accuracy did not differ significantly between 2D HD and 3D HD endoscopy. Subjectively, imaging characteristics of the 3D HD endoscope were rated significantly better. Conclusions ESS performance of inexperienced participants was not significantly improved by the use of 3D HD endoscopy during ESS tasks, although imaging characteristics of the 3D HD endoscope were rated significantly better. Surgical field characteristics and surgical techniques are likely to influence any additional value of 3D HD endoscopy.


| INTRODUC TI ON
Over the past decades, endoscopic imaging techniques and endoscopic surgery expertise have improved continuously. Today, a wide range of sinonasal and skull base pathology can be treated successfully with endoscopic endonasal surgery (EES). 1,2 In EES, two-dimensional (2D) high-definition (HD) endoscopes are widely used, and although they provide good quality images of the surgical field, depth perception is limited compared with direct sight or use of a surgical microscope. 3,4 Specifically during extended EES procedures, when detailed display of anatomical relations is required, the limited depth perception is thought to limit surgical performance. 5,6 The development of three-dimensional (3D) endoscopes, that provide surgeons with a stereoscopic view of the surgical field, has proved challenging, especially with optics suitable for endonasal surgery. Early 3D endoscopes had large diameter optics, offered limited image quality and caused side effects to its users. 7,8 Only in recent years, reliable 3D endoscopes with 4-5 mm diameter optics became available. [8][9][10][11][12] After being used predominantly in transnasal neurosurgical procedures, 3D endoscopes have started to find their way into endoscopic sinus surgery (ESS). 5,9,13,14 Literature specifically addressing the effect of 3D endoscopy on surgical performance in ESS is sparse. 5,9,14 Most studies assessing the influence of 3D endoscopic imaging on surgical performance focused on laparoscopic surgery or neurosurgical procedures. 15,16 In laparoscopic surgery, the use of 3D endoscopic imaging reduced the duration of procedures and the amount of error rates. 16 During neurosurgical procedures in clinical studies, no significant differences in procedure duration or tumour resection were found, while depth perception, spatial orientation, identification of anatomical relations and critical structures subjectively improved with the use of 3D endoscopy. 15 In a laboratory setting, 3D endoscopy decreased the time to task completion in both laparoscopic and neurosurgery procedures. 11,12,14,[17][18][19][20][21][22] Both expert and novice endoscopic surgeons benefit from 3D visualisation, 15 although various studies suggest a specific benefit for inexperienced surgeons. 12,14,23 This can be explained by experienced surgeons already having acquired the ability to effectively translate 2D visual information to 3D perception of the surgical field, thereby reducing the additional value of 3D endoscopy.
It is questionable to what extent the results of neurosurgical and laparoscopy studies of 3D endoscopy are applicable to ESS because surgical performance and the benefits of 2D HD and 3D HD endoscopy are likely to be related to surgical field properties and surgical technique. 6,14,24 The present study aimed to assess the additional value of 3D HD endoscopy in ESS. A 3D HD and a 2D HD endoscope were used by inexperienced surgeons in a realistically simulated clinical setting. Performance was assessed during various ESS tasks, using both objective and subjective outcomes measures.
It was hypothesised that the use of 3D HD endoscopy in ESS would significantly and objectively improve surgical efficiency and accuracy, in addition to the subjective performance.

| Ethical considerations
Ethical approval for the performance of the present study was obtained from the Medical Ethical Committee of the University Medical Center Groningen (UMCG). No specific approval was deemed necessary before commencing.

| Subjects
This study was performed at the skills centre of the Wenckebach Institute of the UMCG, Groningen, the Netherlands. Medical students of the University of Groningen and residents otorhinolaryngology, neurosurgery and general surgery of the UMCG were invited to participate. Participants were included if they were able to read and write Dutch. Those failing to pass the Titmus circle test (Stereo Optical Company) were excluded. Subjects fulfilling the inclusion criteria were randomised into two groups, either performing endoscopic surgical tasks first with 2D HD and second with 3D HD endoscopy (2D first-3D second), or in the opposite (3D first-2D second) order.

| Specimen
Two Thiel-embalmed human cadaver heads were used. 25

Keypoints
• Use of three-dimensional (3D) high-definition (HD) endoscopy does not significantly improve surgical performance of inexperienced surgeons compared to two-dimensional HD during endoscopic sinus surgery tasks, even though 3D HD endoscopy is subjectively rated significantly better.
• When choosing a specific endoscope type, it is recommended to take the specific circumstances in which endoscopic endonasal surgery will be performed into account.
• Surgical navigation can be used to assess surgical performance in endonasal endoscopic (sinus) surgery tasks.

| Navigation system
A KICK electromagnetic (EM) navigation system (Brainlab AG) was used to track the location, and calculate the distance covered and the velocity of the surgical instrument used. The system contained a standard pointer, and an additional EM sensor that could be attached to the other instruments used. The systems reference sensor was fixed with sutures to the anterior scalp. The location of the pointer or instrument tip within the surgical field was registered seven times per second. The system was installed in accordance with the user manual.

| Endoscopic equipment and display
For 3D imaging, a Visionsense VSiii endoscopic system with 4 mm diameter, 0° angled optics (Visionsense, LtD), was used. This endoscope has an interpupillary distance of 1.2 mm and a dynamic focus range of 8-80 mm. It uses "insect eye" technology-incorporating an array of microlenses integrated into 4 mm optics-to project two separate endoscopic images onto dedicated sections of the camera chip. [9][10][11]28 The images were displayed on a 24-inch HD (1920 × 1080 pixels) stereoscopic LED flat screen that offered true stereoscopic images when passive polarising glasses were worn.
For 2D imaging, a Karl Storz 2D HD 0° endoscope (Karl Storz GmbH & Co.) was used. The camera of the endoscope had a maximum resolution of 1920 × 1080 pixels, making it full HD. Images were displayed on a 23-inch full HD LCD flat screen monitor.

| Study design
Five endoscopic surgical tasks (described below) were designed, based on the validated, low-cost sinus surgery task trainer developed by Steehler et al. 11,12 Although well designed and easy to use, the simulator provides a simplified model of the surgical environment in which endoscopic surgery is performed. The tasks used in the present study were modified to be performed on a human specimen. To minimise the bias of differences in anatomical knowledge, all participants watched instruction videos showing the actual surgical field and examples of the task execution, before the start of each task.

| Task 1) Identification of anatomical landmarks
Six anatomical landmarks were identified by placing the navigation system probe under direct endoscopic view on the appropriate landmark.
Landmarks were identified in the following sequence: head of the middle turbinate, uncinate process, choana upper border, head of the inferior turbinate, ethmoid bulla and most superior point of the semilunar hiatus ( Figure 1).

| Task 2) Approaching targets
The navigation system probe was placed exactly in the centre of each of the five screw heads. The following sequence was used: anterior ethmoidal artery, sphenoid sinus, maxillary sinus, posterior ethmoid artery and the nasal entrance of the sphenopalatine artery ( Figure 2).

| Task 3-5) Grasping and retrieving objects
The following objects, differing in visibility and spatial layout, were retrieved from the maxillary and sphenoid sinus using a straight, grasping forceps: coloured sponge discs, translucent tubes and ring-

| Outcome measures
Primary outcome measures were as follows: surgical efficiency (defined as the time to task completion for all tasks), the total distance covered and average velocity towards the target (task 2), the total distance covered and average velocity inside the nasal cavity (task 3), and the surgical accuracy measured by total errors per task (all tasks).
Errors were defined as identifying a wrong structure or making nonfunctional contact with non-target structures (task 1), pointing out or grasping an incorrect structure or object (tasks 2-5), missing the correct object during an attempt to grasp or hook it or losing an object after grasping or hooking it (tasks 3-5). To assess time scores and error rates, two researchers (EtD and HMH) individually reviewed video images of all tasks. A secondary, joint review was deemed necessary whenever the scores assigned differed by more than three errors per subtask. The average of the two scores was used as the final error score. Observers were unaware of participants' identity.
The differing video output of the endoscopes, however, impeded blinding for endoscope type. Secondary outcome measures were as follows: endoscope characteristics and perceived task difficulty per endoscope type. These were assessed following each round of tasks using questionnaires. The following endoscope characteristics were assessed using a ten-point Likert scale from 1 = "completely insufficient" to 10 = "perfect": image sharpness, depth representation, colour representation, field of view size, discriminatory quality, suitability for accurate maneuvres and manoeuvrability. Perceived task difficulty per endoscope was measured using a five-point Likert scale from 1 = "not difficult" to 5 = "very difficult."

| Statistical analysis
Demographic data of all participants were collected, including any experience considered relevant for endoscopic surgery (>30 endoscopic endonasal surgical procedures and/or rigid nasal endoscopies, carried out autonomously for more than half the procedures, 29 or experience with action video games (playing more than 7 h/wk on average during their most active period of gaming)). 30 To allow paired-intrauser-analyses of performance scores, the results of both groups were combined for each endoscope type, thereby minimising the effect of the order in which the endoscopes were used. Analysis was performed using the Wilcoxon signed-ranks test for paired results, after testing for homogeneity of distribution and equality of variances. After logarithmic conversion, the assumptions were met for all time and error scores, except the error scores of sponge removal. Results per task round were F I G U R E 3 Third task: participant grasping one of the six sponge discs with a straight forceps F I G U R E 4 Fourth task: participant grasping one of the four translucent tubes F I G U R E 5 Fifth task: participant retrieving one of three ringshaped objects with the hooked instrument compared using a Mann-Whitney U test for unpaired data. Because of the small sample size of this study, and the potential of type 2 errors, only descriptive statistics were used for comparison of results per endoscope type per task round. Statistical significance was assumed with a one-tailed P value < .05. All statistical analysis was conducted using IBM SPSS Statistics version 22.0 (SPSS IBM, Inc).

| Subjects
In total, 31 subjects were included: 12 (39%) residents (otorhinolaryngology [10], neurosurgery [1], general surgery [1]) and 19 (61%) medical students. One student withdrew from the study before starting. All subjects passed the Titmus circle test. Demographics of the 30 remaining participants are summarised in Table 1 first-2D second. The groups did not differ significantly in age, function, handedness or relevant experience.
The total distances covered during the "target approach" (P = . 38) and "grasping and retrieving coloured sponge discs" (P = .26) task were not significantly different. The average velocity towards target in "approaching targets" (P = .03) was significantly faster using 3D HD endoscopy, whereas the average velocity on the whole intranasal trajectory in "grasping and retrieving coloured sponge discs" (P = .20) did not differ significantly between the endoscopes (Table 3).
During the second task round, all tasks, except for "retrieving translucent tubes," were completed significantly faster than during the first task round (P = .00-.02). Users of 3D HD endoscopy completed all tasks the fastest, when 3D HD endoscopy was used during the second task round (2D first-3D second). In contrast, users of 3D HD endoscopy needed the most time to complete each of the tasks, when 3D HD endoscopy was used in the first task round (3D first-2D second).
The distance covered and mean velocity in "pointing accuracy" (P = .01 and .04) and distance covered in "sponge retrieving" (P = .02) improved significantly between the consecutive rounds. Again, participants using 3D HD endoscopy in the second round covered less distance and reached a higher average velocity than the other subgroups, although not reaching significance.

| Endoscope quality
Evaluation of endoscope characteristics showed significantly better scores for the 3D HD endoscope on image sharpness (P < .01), colour quality (P < .01) and depth representation (P = .02; Table 4).
In addition, discriminatory power (P < .01) and the suitability to perform precise procedures (P < .01) were rated significantly better for the 3D HD endoscope, while no significant differences were found in the subjective field of view (P = .70) and manoeuvrability (P = .36).

| Task difficulty
For all five tasks, task difficulty was rated mild to moderate, with no significant differences between endoscope types. A trend favoured 3D HD endoscopy, with the most notable difference in difficulty rating in the retrieval of transparent tubes and ring-shaped objects ( Table 5).
The data that support the findings of this study are available from the author (HMH) upon reasonable request.

| Key findings
The present study tested the hypothesis that the use of 3D HD endoscopy improves surgical performance in endoscopic endonasal sinus surgery. The objective results show no significant differences in surgical efficiency and accuracy between 2D HD and 3D HD endoscopy. Subjectively, the imaging quality of the 3D HD endoscope was rated significantly better.
In endoscopically naïve participants, the use of 3D HD endoscopy neither significantly nor consistently increased endoscopic surgery efficiency. Only during the target approach task, the average velocity towards a target-one of the outcome measures of efficiency-was higher using 3D HD endoscopy. This was not interpreted as an increased surgical efficiency, since the higher velocity was associated with a significant increase in error rate.

| Strengths of the study
The present study was the first study comparing 2D HD and 3D HD endoscopy specifically in ESS. Objective outcome measures were used, and a realistic surgical environment was offered by using a Thiel-embalmed human specimen as surgical field. In contrast to more simplified models, in which participants can rely on relatively few depth cues, the use of Thiel-embalmed specimens increased the availability of haptic feedback. This is important because surgical performance depends on the availability and quality of sensory feedback. 18,[31][32][33][34][35] Also this more complex surgical environment offered valuable additional depth cues such as tissue texture and shading of TA B L E 2 Time and error scores per endoscope type (combined results of first and second round) shown as absolute score and standard deviation Task 2D 3D 2D vs 3D

| Comparison with other studies
In recent past, various studies with a design similar to the present study assessed the influence of 3D endoscopy on EES performance. 11,14,17,18,22,40 A number of these studies report improved performance with the use of 3D endoscopy, in contrast with results of the present study. These differing results can be explained by differences in study setup. It should be noted that the results of our study should only be framed in the context of ESS, since key aspects of skull base surgery were not included in our study design.
In general, prior studies assessed performance in simplified surgical environments, such as box trainers and dry anatomical models, and mostly used low complexity pointing tasks. 11,14,17,18,40 In more complex surgical environments, endoscopic surgery tasks require active manoeuvring through a complex visual environment. The result is an increased availability of monocular depth cues, reducing the dependence on stereopsis. [36][37][38] For example, Shah et al 14 found that significantly more nerve hook placement tasks were successfully performed with 3D endoscopy in a box trainer model, using success or failure as outcomes. The authors found no differences in time to task completion or error rates during ring transfer and incision tasks.
Fraser et al 40 compared EES performance between 3D endoscopy and 2D endoscopy during endoscopic sellar floor removal and biopsy tasks, performed in a custom build test box. Cutting efficiency was significantly higher with 3D endoscopy during the second round of tasks, whereas time and error scores were comparable. In a third and more recent study by Rampinelli et al, 18 participants performed a grasping and dissection movement task in a 3D printed skull model.
No significant benefit of 3D endoscopy was found for inexperienced participants, which is confirmed by the present studies results. In the expert group, however, task completion time was significantly reduced. 18 The results of these three studies indicate that both surgical environment, task complexity and endoscopic surgery experience influence performance and its relation to endoscopic imaging technique.

An additional explanation for the difference in results between
prior studies and the present study is that the endoscope is used statically, rather than dynamically in most neurosurgical EES procedures. 14 The ability to move the endoscope through the surgical field provided participants with dynamic depth cues, available in both 2D and 3D endoscopy. 6,14,41 Up till now, only two previous studies used instrument tracking as objective outcome measure for endoscopic surgery performance. This could well be the result of relative inability of inexperienced users to process the more complex visual information of the 3D HD endoscope. It could also indicate that 3D HD endoscopy provides a TA B L E 5 Subjectively rated task difficulty, rated on a scale from 1-5, with 5 being the highest difficulty. Scores expressed as mean ± standard deviation (SD) false sense of confidence to inexperienced users, although the latter was not supported by results of our study. Only positive correlations were found between time to task completion and error rates.
The subjective superiority of 3D endoscopy in EES has been widely reported by both clinical and laboratory studies and is confirmed by the subjective results of the present study. 5,[9][10][11][12]17,28,40 The present study showed significantly better rating of depth perception, discriminatory power and the value in performing delicate procedures using the 3D HD endoscope. These results specifically confirm studies of 3D endoscopy in ESS by Manes et al 5 and Albrecht et al. 9 The former found enhanced depth perception and better orientation with 3D SD endoscopy compared with 2D HD endoscopy, whereas the latter found improved depth perception and comfort of use with 3D HD endoscopy and enhanced image sharpness compared with that of 2D HD. 5

| Clinical applicability
The present results indicate that in a realistic surgical environment, when performing complex surgical tasks, surgical performance is comparable with 2D HD and 3D HD endoscopy. The choice for either endoscope type should depend on the specific demands of a surgical procedure, and preference of the surgeon. Decisions on which endoscopy type to use can also be made based on the general difference in field of view between 2D and 3D endoscopes. The field of view of the 3D HD endoscope used in the present study was 31% smaller in the horizontal and 18% in the vertical direction than that of the 2D HD endoscope. The larger field of view of the 2D endoscopes could be preferable during procedures that require overview, whereas the addition of stereopsis can be preferred in procedures where the endoscope is used statically and overview is of lesser importance. 7,24 In inexperienced surgeons, training in ESS should be focused both on overcoming the difficulties of manoeuvring the endoscope through the complex surgical field, and on using the abundant visual and haptic feedback, regardless of the endoscope type used.

| CON CLUS ION
The use of 3D HD endoscopy in ESS did not significantly improve surgical performance of inexperienced surgeons compared with the use of 2D HD endoscopy, despite subjective superiority of the 3D HD endoscope. When choosing a specific endoscopic imaging technique, it is recommended to take the specific circumstances in which EES will be performed into account. Surgical field characteristics and surgical techniques are likely to influence any additional value of 3D HD endoscopy. To further assess current and future endoscopic imaging techniques, surgical navigation can be used as a specific and objective performance measure.

DATA AVA I L A B I L I T Y S TAT E M E N T
The data that support the findings of this study are available from the author (HM.H) upon reasonable request.