Validation of computer-based training in ureterorenoscopy

Authors


Thomas Knoll, Department of Urology, University Hospital Mannheim, 68135 Mannheim, Germany.
e-mail: thomas.knoll@uro.ma.uni-heidelberg.de

Abstract

OBJECTIVES

To evaluate the outcome of training both urological novices and experts, using the recently developed UroMentor (Simbionix Ltd, Israel) trainer, that provides a realistic simulation of rigid and flexible ureterorenoscopy (URS).

SUBJECTS AND METHODS

Twenty experienced urologists (total number of previous flexible URSs 21–153) were monitored during simulated flexible URS for treating a lower calyceal stone, and the outcome was correlated with individual experience. A score was compiled based on the variables recorded, including total operation time, stone contact time, complications such as bleeding or perforation, and treatment success. A further five urological residents with no endourological experience were trained on the UroMentor in rigid URS for ureteric stone treatment. Their acquired clinical skills were subsequently compared to those of five urological residents who received no simulator training.

RESULTS

All 20 experienced urologists disintegrated the stone on the simulator, and the score achieved was related to their personal experience; there was a significant difference in performance in those with < 40 and > 80 previous flexible URSs. For the five urological residents with no endourological experience, simulator training improved their skills, and comparison with urological residents who had received no simulator training showed advantages for the trained residents. After being trained on the simulator, the group performed better in the first four URSs on patients.

CONCLUSIONS

Individual experience correlates with individual performance on the simulator. Simulator training was helpful in improving clinical skills. Although the distribution of computer-based simulators is limited by high prices, virtual reality-based training has the potential to become an important tool for clinical education.

Abbreviations
URS

ureterorenoscopy

INTRODUCTION

Although endourological procedures are minimally invasive and have low morbidity, they carry a risk of major complications like ureteric perforation or avulsion. Safe and effective performance of diagnostic and therapeutic endourological procedures demands long practical experience and continuous training. Long waiting lists in some countries, expensive operating-room time or insufficient availability of trainers make adequate education difficult. Training systems have been developed to overcome this problem. Cadaver organs were the first endoscopic training model, and were followed by various synthetic models [1–4] used primarily for resection of the prostate and secondarily for ureterorenoscopy (URS) [5,6]. Although these models offered training close to the clinical situation, they were inappropriate for the upper urinary tract, with drawbacks such as the lack of blood flow and fluoroscopic control. Rapid developments in computer technology have allowed computer-based systems to simulate endoscopic procedures more realistically, including such complications as bleeding or perforations [7]. The first computer simulators were designed for TURP [8,9], and the first virtual simulator for rigid and flexible URS was recently developed [10].

Although these simulators are now available in many centres throughout the world, little evidence of the effect on clinical skills and the reliability of simulator training has been published. Only one study evaluated the improved skills of medical students practising semi-rigid URS on the simulator [11], and it is still unclear whether this improvement is upheld in clinical practice. The objectives of the present study were to evaluate whether already acquired clinical skills equalled the skills shown on the simulator, and to evaluate the actual improvement in clinical endourological surgery after simulator training.

SUBJECTS AND METHODS

We used a commercially available computer-based simulator for semi-rigid and flexible URS, as previously described (UroMentor, Simbionix Ltd, Israel) [10]. Twenty experienced urologists (total flexible URSs, 21–153) were monitored during flexible URS on the simulator for treating a lower calyceal stone (Fig. 1). Before assessing their performance each urologist was allowed to train on the simulator for 10 min to become accustomed to it; none of them had previous experience with virtual URS. A holmium laser was used for stone disintegration and a tip-less Nitinol basket for stone extraction. Each trainee was allowed one 20-min practice session to become familiar with the computer device. The outcome of training was correlated with individual experience. Recorded variables included total operation time, X-ray exposure, guidewire insertion time, time of progression from the orifice to the stone, stone contact time, number of perforations, bleeding events, laser misfiring, scope damage and treatment success (Fig. 2).

Figure 1.

Flexible ureterorenoscopic access to lower calyx.

Figure 2.

Consecutive analysis of treatment performance.

A second group comprised five urological residents with no endourological experience who were given 10 simulator training sessions in semi-rigid URS for treating a distal ureteric stone (treatment time per case 16–28 min). A virtual pneumatic lithotripter probe was available with optional forceps or baskets for extracting fragments. Like ‘real’ URS, all procedures were supervised by an experienced urologist; supervision included assistance in technique and use of stone disintegration or extraction tools. The variables recorded included total operation time, X-ray exposure, guidewire insertion time, time of progression from the orifice to the stone, stone contact time, number of perforations, bleeding events, lithotripter misfiring, ‘push-back’ and treatment success. The clinical skills of this group were subsequently compared to those of the same number of residents of a similar level of experience, with no former simulator training, during endoscopic procedures in five comparable cases. The variables recorded were the same as in simulator training. Both groups were supervised by certified urologists. The procedures did not differ from routine teaching URS and the resident was not allowed to make significant mistakes, i.e. ureteric perforations. Supervisors were aware of whether the resident had had simulator training before.

The results were analysed statistically using the Mann–Whitney U-test, and expressed as the means (sd), with P < 0.05 considered to indicate statistically significant differences.

RESULTS

Although all 20 experienced urologists disintegrated the stone only the most experienced completely cleared it (all those with >80 flexible URS, half with 60-80, two-thirds with 40–60, a third with 20–40 and a fifth with <20). The values of the variables assessed were equivalent to their clinical experience (significant difference between 1–40 and > 80 URSs, P < 0.01). These findings were confirmed by comparing mean operation times; urologists with >40 URSs were significantly faster than those with <40, with a mean operation time of 12.3 (2.2) vs 18.5 (1.8) min (P < 0.05; Table 1). Except for X-ray exposure, time of progression to stone contact and bleeding events, there was a distinct improvement, corresponding to the degree of clinical experience (Table 1).

Table 1.  Overview of performance variables achieved by subgroups with different clinical experience
MeanPrevious experience (number of previous flexible URSs)
<2020–4040–6060–8080–100100–120>120
Total operation time, min19.817.213.313.514.59.011
X-ray exposure, min 5.3 4.8 4.1 4 3.94.23.8
Time to introduce guidewire, min 7.6 5.5 3.2 3.1 2.82.51.6
Time to progress from orifice to stone, min 2.2 2.1 1.8 1.9 1.51.20.9
Perforation per operation, n 1.3 0.5 0.3 0 000
Bleeding events, n 4.2 4 2.1 1.6 0.501
Laser misfire, n23141211 896
Scope damage per operation, n 3.2 1.8 1.1 0.3 000

Both resident groups successfully disintegrated and extracted the stone under the supervision of an experienced urologist, but the simulator-trained residents were faster than untrained residents. There were statistically significant advantages for the first four URSs in favour of the simulator-trained residents (P < 0.05) (Fig. 3). Complications were negligible in both groups.

Figure 3.

The reduction in operating time with the number of procedures undertaken for residents trained on the simulator (green circles) or untrained residents (red squares).

DISCUSSION

The number of endourological procedures accomplished affects the rate of complications. Similar problems are encountered in all centres where URS is used; how to acquire the required level of competency despite a possible shortage of endourological procedures, insufficient endourological training devices, no time for teaching or more trainees than available cases. Several models have been developed to overcome these problems [8–10]. Bench models for URS are favoured for their relatively low costs and easy transportation. Brehmer et al.[12] reported that the skills acquired on these models are equivalent to clinical skills. Regular, standardized training improves performance on the bench model [13]. The latest generation of computer-based simulators could, for the first time, offer standardized ‘real’ training. Although these machines are currently becoming more popular worldwide, information on the validity of virtual reality training is rare. Only one Canadian group published a report on the benefit of simulator training among medical students [11]. Although it was foreseeable that regular training further improved performance on the simulator, information on the effect of these skills on surgery was missing. We used the same computer-based simulator for semi-rigid and flexible URS, and found that simulator training skills were comparable to the skill needed in clinical practice. The results achieved by urologists highly experienced in flexible URS were clearly better than those of the novices. Conversely, standardized simulator training enabled inexperienced residents to become competent in semi-rigid URS. In the present study, the simulator-trained residents used simple ureteroscopy for removing a distal ureteric stone, and were faster than their untrained colleagues. That the difference was compensated after only five procedures, and the equal outcome in terms of safety, are explained by the simplicity of the chosen scenario. Advantages of simulator training might be more pronounced for more difficult cases.

The present results indicate that a high level of simulation is achieved by the latest virtual-reality technology. Although such devices are costly (up to 50 000 Euro), they have several advantages over bench or cadaver models. First, these systems offer a realistic simulation of endourological cases encountered in clinical practice, real instruments, and complications such as bleeding or ureteric perforation. Objective data acquisition is a further advantage for the trainee and the trainer; the simulators continuously record performance variables, e.g. total operation time or error rates, and effectiveness and improvement are easy to compare. Adversely, current systems lack haptic feedback to mediate the sense of touch. This point is more relevant for semi-rigid URS than for flexible URS, which has limited tactile feedback and is a relatively straightforward procedure to simulate on the computer. Several scientific and commercial groups are working on this issue, and the latest prototypes presented at urological symposia were promising. Nevertheless, a comparison between bench and computer models would be interesting.

The present results support the value of simulator training for URS on clinical performance. Virtual reality-based training has the potential to become a standard tool for clinical education. Currently, high prices limit the availability of computer simulators to large endourological centres; these costs are expected to decline in the near future, and a wider distribution will allow multicentre trials to evaluate the effects of simulator training.

CONFLICT OF INTEREST

The UroMentor simulator, in the development of which the authors were substantially involved, was provided by Simbionix for academic purposes. The authors have no financial interest in the simulator or the company.

Ancillary