SEARCH

SEARCH BY CITATION

Keywords:

  • assessment;
  • simulation;
  • training;
  • urology

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. Conflict of Interest
  9. References

What's known on the subject? and What does the study add?

  • A competent urologist should not only have effective technical skills, but also other attributes that would make him/her a complete surgeon. These include team-working, communication and decision-making skills. Although evidence for effectiveness of simulation exists for individual simulators, there is a paucity of evidence for utility and effectiveness of these simulators in training programmes that aims to combine technical and non-technical skills training.
  • This article explains the process of development and validation of a centrally coordinated simulation program (Participants – South-East Region Specialist Registrars) under the umbrella of the British Association for Urological Surgeons (BAUS) and the London Deanery. This program incorporated training of both technical (synthetic, animal and virtual reality models) and non-technical skills (simulated operating theatres).

Objectives

  • To establish the feasibility and acceptability of a centralized, simulation-based training-programme.
  • Simulation is increasingly establishing its role in urological training, with two areas that are relevant to urologists: (i) technical skills and (ii) non-technical skills.

Materials and Methods

  • For this London Deanery supported pilot Simulation and Technology enhanced Learning Initiative (STeLI) project, we developed a structured multimodal simulation training programme.
  • The programme incorporated: (i) technical skills training using virtual-reality simulators (Uro-mentor and Perc-mentor [Symbionix, Cleveland, OH, USA], Procedicus MIST-Nephrectomy [Mentice, Gothenburg, Sweden] and SEP Robotic simulator [Sim Surgery, Oslo, Norway]); bench-top models (synthetic models for cystocopy, transurethral resection of the prostate, transurethral resection of bladder tumour, ureteroscopy); and a European (Aalborg, Denmark) wet-lab training facility; as well as (ii) non-technical skills/crisis resource management (CRM), using SimMan (Laerdal Medical Ltd, Orpington, UK) to teach team-working, decision-making and communication skills.
  • The feasibility, acceptability and construct validity of these training modules were assessed using validated questionnaires, as well as global and procedure/task-specific rating scales.

Results

  • In total 33, three specialist registrars of different grades and five urological nurses participated in the present study.
  • Construct-validity between junior and senior trainees was significant. Of the participants, 90% rated the training models as being realistic and easy to use.
  • In total 95% of the participants recommended the use of simulation during surgical training, 95% approved the format of the teaching by the faculty and 90% rated the sessions as well organized.
  • A significant number of trainees (60%) would like to have easy access to a simulation facility to allow more practice and enhancement of their skills.

Conclusions

  • A centralized simulation programme that provides training in both technical and non-technical skills is feasible.
  • It is expected to improve the performance of future surgeons in a simulated environment and thus improve patient safety.

Abbreviations
CRM

crisis resource management

STeLI

Simulation and Technology enhanced Learning Initiative

Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. Conflict of Interest
  9. References

The principal aim of a urology training programme is ‘to produce trained surgeons with the knowledge, skills and attitudes required to be a competent and safe Consultant Urologist’ [1]. Urology remains at the forefront of surgical evolution, particularly in the field of minimally-invasive surgery including laparoscopy and robotics. As the nature of practice is changing, so are the definitions and requirements of competency in urology [2]. To ensure that patient safety is not compromised, it is vital that urology training constantly adapts to these changes in practice. With a reduction in training hours, volume-based traditional Halstedian model of training (‘see one, do one, teach one’) will probably not play a major role in future training [3]. Novel methods of training that are valid and reproducible are constantly being sought. With the increase in litigation, as well as higher expectations of public and quality assurance targets, it is desirable to acquire a degree of technical competence before operating on patients [4]. This has led to increasing interest in the field of simulation, which has the potential to supplement workplace training and shorten the initial phases of the learning curve without compromising patient safety [5]. A competent urologist should not only have technical skills, but also other attributes that would make him/her a complete surgeon. These include team-working, communication and decision-making skills. With this objective in mind, we devised our simulation programme aiming to encompass both technical and non-technical skill aspects of training [6-8]. There are two areas that are most relevant to urologists: (i) procedural skills and (ii) decision-making, communication and team-working skills. The programme aims to establish the feasibility and acceptability of a centralized, simulation-based training programme.

Materials and Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. Conflict of Interest
  9. References

Simulation and Technology enhanced Learning Initiative (STeLI) in Urology was a pilot project supported by London Deanery and School of Surgery. It was conducted at the Simulation and Interactive Learning (SaIL) Centre, Guy's Hospital, King's College London, from November 2009 to October 2010. This programme incorporated: (i) technical skills training through validated virtual-reality simulators (Uro-mentor and Perc-mentor [Symbionix, Cleveland, OH, USA], Procedicus MIST-Nephrectomy [Mentice, Gothenburg, Sweden] and SEP Robotic simulator [Sim Surgery, Oslo, Norway]), bench-top models and a European (Aalborg, Denmark) wet-lab training facility (Table 1) and (ii) non-technical skills/crisis resource management (CRM), using SimMan (Laerdal Medical Ltd, Orpington, UK) to teach team-working, decision-making and communication skills. Participants were urology trainees from the South East London regional training programme. Nurses taking part were from the Urology Department, Guy's Hospital London. In total, 33 specialist registrars and five nurses were recruited using convenience sampling. A counterbalanced design was used where each urology trainee performed defined stages of a simulated procedure on virtual simulators and bench-top box trainers (Olympus, London, UK). The sessions for technical skills were run for 14 half-days, whereas CRM sessions were conducted as seven full-day programmes (alternate Fridays during term) to cover a series of clinical scenarios encountered in urological practice (Fig. 1).

  1. Technical skills sessions: cystoscopy, TURP, transurethral resection of the bladder tumour, rigid and flexible ureteroscopy, percutaneous nephrolithotomy, laparoscopy, laparoscopic nephrectomy, robotics, single-port laparoscopy and robotics.
  2. CRM sessions: using an interactive human patient mannequin to run four to six scenarios per course as a basis to explore team-working, decision-making and communication skills. Each 15-min scenario consisted of one or two trainees in a high-fidelity simulation environment using SimMan 2 or 3G. They were provided with colleague support appropriate to the scenario (e.g. anaesthetist, junior doctor, nurse) and a video relay to the debriefing room allowed full observation by other course members and also provided film footage for playback in the debrief. The scenarios were followed by 30–40-min structured, student-centred debriefs led by experienced faculty (Table 2).
figure

Figure 1. A summary of the methods used in the present study. STeLI, Simulation and Technology enhanced Learning Initiative; TURBT, transurethral resection of bladder tumour; VR, virtual reality.

Download figure to PowerPoint

Table 1. Models for the training programme
Simulated proceduresModels
TURPBench top models set up with irrigation and real instruments
Transurethral resection of the bladder tumourBench top models set up with irrigation and real instruments
TRUSBench top models
Botulinum toxin A (Botox)Bench top models with injection kit
Single-port surgeryBench top models and da Vinci robot
UreterorenoscopyVirtual reality and bench top models
Percutaneous nephrolithotomyVirtual reality and bench top models
LaparoscopyVirtual reality and bench top models
RoboticsVirtual reality simulator and da Vinci robot
Table 2. Clinical scenarios for training and assessment of non-technical skills
Scenario numberScenario description
1Urological sepsis in an elderly patient
2Complication after laparoscopic nephrectomy. Discussion with family about complications after surgery for scenario 1
3Ward-based chest pain after cystectomy with progression to ventricular fibrillation arrest
4Transurethral resection syndrome
5Transfusion reaction in a patient bleeding after nephrectomy
6Anuric patient secondary to catheter misplacement leading into opiate toxicity

Data Capture

The feasibility, acceptability and construct validity of procedural skills and CRMs (checklists) were evaluated.

Face/content validity

Participants evaluated the realism using multi-item questionnaires that were specifically developed for this simulation (face validity and content validity). Questionnaire items addressed aspects of the simulated environment, including ‘background environment’, ‘equipment’, ‘model’, etc. Participants indicated their agreement with individual items on five-point Likert scales (1 = not at all, 5 = very much).

Trainee perceptions

Each trainee also participated in an individual semi-structured interview after the simulation (using a topic guide) by the trainers.

Assessment of technical performance

There were two experts/faculty who assessed the technical performance of surgeons using the validated Objective Structured Assessment of Technical Skills scale. The global scale consists of seven generic components of operating skill that are marked on a five-point Likert scale. Each surgeon can thus achieve a minimum score of 7 and a maximum score of 35 [9].

Statistical Analysis

Questionnaire scores were cumulated as means and a parametrical analysis was used (independent-sample t-test) to compare the ratings of experts and novices. Cronbach's α was calculated for scale reliability (with a coefficient of 0.70 being regarded as acceptable).

There were two researchers (urologists) who independently coded all non-structured questions based on the topic guide. Items were grouped together and any emergent themes were identified.

Performance rating

Performance ratings were cumulated as medians. Inter-rater reliability between experts was established using the intra-class correlation coefficient (with a correlation of 0.70 being regarded as acceptable). The non-parametrical Mann–Whitney U-test was used to compare the technical performances of senior and junior trainees in the procedural simulation settings. The Wilcoxon signed rank test was used to compare the performance of surgeons between the virtual simulator, bench-top box trainer and a European wet-lab training facility, as well as in the setting of the human patient simulation training in CRM.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. Conflict of Interest
  9. References

Technical Performance

The technical performance of both junior (speciality trainee level 3/4/5/6; ST3–6) and senior surgeons (ST7 and fellows) was rated. Construct-validity between junior and senior groups was established. Seniors performed significantly better than the junior trainees in all the simulation sessions: mean (SD), 26.3 (5.9) vs 15.2 (3.9), respectively (P < 0.001) (Fig. 2).

figure

Figure 2. Overall construct validity of participants. ST, speciality trainee level.

Download figure to PowerPoint

Senior trainees scored significantly better on virtual reality simulators, bench-top box trainers and the European wet-lab training facility, as well as in human patient simulation training in CRM using SimMan, than junior trainees: mean (SD), 19.8 (5.2) vs 16.3 (3.8), respectively (P < 0.01). Acceptable inter-rater reliability was obtained for Objective Structured Assessment of Technical Skills performance ratings (r = 0.73, P < 0.01).

Quantitative and Qualitative Findings

The training models were rated as being realistic and easy to use by 90% of trainees; 90% considered that they serve as a good training format; 95% recommended simulation during training; 90% rated that sessions were well organized; 95% approved the format of teaching by the faculty; and 60% would like to have more time and easy access in the future being specifically dedicated to simulation-based training (Figs 3, 4).

figure

Figure 3. Feedback evaluating feasibility and acceptability of the programme. TURBT, transurethral resection of bladder tumour.

Download figure to PowerPoint

figure

Figure 4. Training session for technical and non-technical skills.

Download figure to PowerPoint

There were two main themes that were identified from interviews carried out after the simulation.

  1. The behaviours of trainees illustrated the value of different aspects of the CRM training, including team interaction, communication and leadership skills.
  2. Trainees are aware of an increased cognitive load on the virtual reality simulator, bench-top box trainer and the wet-lab facility, as well as in the CRM, as a result of pressure/anxiety about the unknown and the interplay between technical and non-technical skills, perceiving this as added value.

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. Conflict of Interest
  9. References

The present study shows the feasibility and acceptability of a central simulation programme in urology. Construct-validity between junior and senior groups was established. Senior trainees performed equally well on the virtual reality simulators, bench-top box trainer and wet-lab training facility, as well as CRM. Novices performed better on the virtual reality simulator, bench-top box trainer and wet-lab training facility than in CRM. These findings suggest that, although junior trainees may have gained skills in the technical domains, they have not yet gained the ability to fully integrate the range of non-technical skills required for independent higher surgical practice.

As highlighted in the semi-structured survey (interviews/questionnaires), some junior trainees felt increased pressure and anxiety from the need to multitask during CRM. Therefore, we postulate that this may have placed additional cognitive load on the participants, leading to the weaker performance of some non-technical skills in juniors. Stress is known to cause a change in the attention focus of the surgeon from a team focus to a narrowed self focus [10, 11]. This ultimately leads to loss of cohesion of the surgical team, which has a direct impact on patient safety [11, 12]. We consider that, for the novice, repeated CRM training and practice is necessary so that they can develop the non-technical cognitive skills practiced by experts to manage stress earlier in their careers. Simulation can address these factors in an environment where patient care is not compromised and thus contributes to a significant improvement in patient safety.

It has been reported in the literature that simulators are effective during the initial phase of training [8]. Various studies have individually validated both bench-top and virtual reality simulators. However, these simulators have never been assessed with respect to the effectiveness of a central training programme established in adjunct to the workplace training. The present study was designed to establish the feasibility, acceptability and construct validity of these simulators, when used collectively for a programme. Future work entails an evaluation of the educational impact of these simulators.

The cost-effectiveness of these simulators also needs to be studied. Current literature suggests a difference in opinion regarding the cost-effectiveness of simulation [13]. Low-fidelity bench models are considered to be more cost-effective when training novices than high-fidelity models [8].

The main limitation of the present study is the small size of the participants and the restricted location where the study was carried out. That being said, it allowed trainees from South London and South East England to have structured training in a regional facility.

The findings of the present study highlight important implications for urological education. First, the simulation programme was sufficiently reliable to trigger strongly positive responses and high levels of perceived realism. It is also suggested that CRM cues may be sufficient to recreate an experience that is representative of the clinical environment, provided that such cues are encountered in daily practice. Second, trainee surgeons agreed that simulation would be a valuable training and assessment tool for wider use. Challenges within the non-technical skills area could be introduced in the CRM environment (human patient simulation) without compromising patient safety. The human patient simulation environment provides an acceptable means of exploring non-technical skills without compromising patient safety (as well as the opportunity to challenge trainees in this area). Finally, centralized urology simulation offers great potential as a research tool. Factors influencing technical and non-technical skills performance, such as stress, teamwork, decision-making and situation awareness, are examples of possible areas of investigation. Based on these findings, the BAUS has launched the first national simulation program of its kind. SIMULATE (Simulation in Urological Learning and Teaching; http://www.surgical-simulation.com) goes live in 2012 and will include a combination of technical and non-technical skills.

In conclusion, a centralized simulation programme that provides training in both technical and non-technical skills is feasible and acceptable. We are continually striving to improve the performance of tomorrows' surgeons. Future work entails an evaluation of the efficacy of simulation-based training in the real clinical environment.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. Conflict of Interest
  9. References

We acknowledge support from Isabel DeAbreu (SAiL Centre Manager), James Gaydon (technical support), Trevor Marsh (Olympus), Ali Bhasoun (medical student) and all the trainers who contributed to make this pilot programme effective and successful. Prokar Dasgupta acknowledges financial support from the Department of Health via the National Institute for Health Research (NIHR) comprehensive Biomedical Research Centre award to Guy's & St Thomas' NHS Foundation Trust in partnership with King's College London and King's College Hospital NHS Foundation Trust. He also acknowledges the support of the MRC Centre for Transplantation, Olympus and a project grant from the Guy's and St Thomas' Charity. The project is supported by project grants from The School of Surgery (Professor Nigel Stanfield) and London Deanery. The authors are grateful to the consultant faculty and trainees for their kind participation.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. Conflict of Interest
  9. References