Learning curves for urological procedures: a systematic review

Authors


Abstract

Objective

  • To determine the number of cases a urological surgeon must complete to achieve proficiency for various urological procedures.

Patient and Methods

  • The MEDLINE, EMBASE and PsycINFO databases were systematically searched for studies published up to December 2011.
  • Studies pertaining to learning curves of urological procedures were included.
  • Two reviewers independently identified potentially relevant articles.
  • Procedure name, statistical analysis, procedure setting, number of participants, outcomes and learning curves were analysed.

Results

  • Forty-four studies described the learning curve for different urological procedures.
  • The learning curve for open radical prostatectomy ranged from 250 to 1000 cases and for laparoscopic radical prostatectomy from 200 to 750 cases.
  • The learning curve for robot-assisted laparoscopic prostatectomy (RALP) has been reported to be 40 procedures as a minimum number.
  • Robot-assisted radical cystectomy has a documented learning curve of 16–30 cases, depending on which outcome variable is measured.
  • Irrespective of previous laparoscopic experience, there is a significant reduction in operating time (P = 0.008), estimated blood loss (P = 0.008) and complication rates (P = 0.042) after 100 RALPs.

Conclusions

  • The available literature can act as a guide to the learning curves of trainee urologists. Although the learning curve may vary among individual surgeons, a consensus should exist for the minimum number of cases to achieve proficiency.
  • The complexities associated with defining procedural competence are vast.
  • The majority of learning curve trials have focused on the latest surgical techniques and there is a paucity of data pertaining to basic urological procedures.

Introduction

The surgical learning curve is the period during which a surgeon in training finds the procedure more difficult, takes longer to complete, there is higher rate of complications and lower efficacy because of inexperience [1, 2]. There are no evidence-based definitions pertaining to the term ‘learning curve’ or how to measure it, but it can be thought of as an improvement in performance over time. This improvement is at its maximum early on in the learning process then tails off over time.

The learning curve is influenced by surgeon-related factors such as attitude, self-confidence or experience in other procedures. On the way to mastery of a procedure, the curve represents the initial challenges in competence, and the change in technical proficiency and efficiency with increasing experience [3]. Patient factors such as case-mix can also influence the learning curve. The surgical learning curve often differs depending on the outcome measure being investigated. For instance, achieving high potency rates at 12 months after a surgical procedure may require 150 cases, whereas obtaining mean estimated blood loss (EBL) of <100 mL might be achieved sooner. Case volume has to be considered in the context of case frequency, mix and selection. A surgeon who has been working for 25 years and performs five prostatectomies a year compared with another who has just graduated but performs two prostatectomies a week will have different outcomes. Often being ‘past’ the learning curve is taken to imply expertise in that skill, but given that one surgeon's definition of expertise may differ from another's, learning curve discrepancies may arise.

In urology new surgical technologies and operating techniques are constantly evolving. With the emergence of new procedures, there is a period in which, regardless of preclinical training, the inexperience of the surgeon makes the operation more difficult or lengthy. This period is also referred to as the ‘procedure development learning curve’. This should be differentiated from the learning curve of a surgeon who is learning an established procedure.

Before embarking on a new surgical technique it is imperative to know how many cases a surgeon is required to perform to be competent and safe at a new technique. Higher-volume centres have been associated with more favourable outcomes across a wide range of procedures and conditions, with the most consistent absolute differences in peri-operative mortality rates between high- and low-volume hospitals being observed for cancer surgery of the pancreas, the oesophagus and paediatric tumours and surgery for unruptured abdominal aortic aneurysm [4]. With regard to survival rates after major oncological surgery, there are associations between surgical volume and long-term cancer-specific survival after surgery for rectal and lung cancer [5, 6]. This has significant implications on both training and the adoption of new techniques. Furthermore, there are implications for patients, where the surgeon's workload might become part of the information required by the patients at the time of preoperative counselling so they can make an informed decision about their care.

The aim of the present study was to provide a systematic review of the published literature on urological learning curves and make recommendations to establish standardized definitions of procedural competency.

Materials and Methods

Study Selection

This review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement (http://www.prisma-statement.org/) [7].

A broad search of the literature was performed in December 2011 using the MEDLINE (from 1950 to December 2011), EMBASE (from 1980 to December 2011) and PsychINFO (from 1966 to December 2011) databases. The following keywords were used during the search: ‘urology’, ‘urological surgery’ and ‘learning curve’. The Cochrane database and the Database of Abstracts of Reviews of Effectiveness were reviewed. The selection was limited to English-language articles only. Empirical studies describing the evaluation of learning curves, in the operating theatre and on simulators, in urological surgery were included. Review articles, studies describing models, letters, bulletins, comments and studies describing non-technical skills were excluded from the analysis.

The full text of each article was obtained and further screened for inclusion if it had information pertaining to learning curves of urological procedures. We included conference abstracts as well. Studies assessing learning curves within a virtual reality setting were also included. We excluded editorials, letters and bulletins and studies not related to learning curves.

Two reviewers (K.A. and H.A.) independently identified potentially relevant articles. Conflicts between reviewers were subsequently discussed, such that agreement was >0.85 (Cohen coefficient).

Data Collection and Analysis

An electronic data collection form (Microsoft Excel 2007, Redmond, WA, USA) was used to extract data including name of the procedure, statistical analysis, number of surgeons who contributed to the development of the learning curve, previous experience of the surgeons, the procedure setting, the variables or outcome measures used to measure the learning curve and the learning curve itself. Disagreement in the assessment and data extraction were resolved by consensus.

Because of heterogeneous study designs and lack of comparative variables, direct comparisons or meta-analysis of data were not feasible; however, if identical tools or outcome measures were used in different studies the results for the different items of the framework used were summarized. Where possible, the collated data were divided into ‘novice’ and ‘expert’ for statistical analysis. Novice subjects were defined as those on the initial phase of their learning curve and experts were defined as those subjects who had reached the plateau phase of their learning curve.

Results

Study Selection

A total of 1439 potentially relevant publications were identified by the search, of which 1355 were excluded from analysis after the abstract review. Of the remaining 84 studies we excluded a further 40 after reviewing the full text because of repetition, incomplete data and lack of relevance to the present study; thus, 44 studies were finally included in the systematic review (Fig. 1). The following procedures had learning curve evaluations: robot-assisted laparoscopic prostatectomy (RALP); open retropubic, perineal and laparoscopic radical prostatectomy (LRP); percutaneous nephrolithotomy (PCNL); other upper urinary tract procedures; and robot-assisted radical cystectomy (RARC).

Figure 1.

Study selection.

Robot-Assisted Laparoscopic Prostatectomy

Seventeen studies investigated the learning curve of RALP (Table 1 [1, 8-23]). The number of participating surgeons ranged from 1 to 13. All studies evaluated outcomes in real patients. A range of variables were included in the calculation of the learning curve including; operating time (OT), EBL, positive surgical margin (PSM) rate, urethrovesical anastomosis time, complications, length of hospital stay (LOS), transfusion rate, early continence, potency and conversion rate.

Table 1. Learning curve studies on RALP
Study*No. of participating surgeonsPrevious experienceOutcome measuresStatistical analysisLearning curve No. of cases: outcome measure
  1. *The procedure setting for all studies was real patients. (A), abstract; VUAT, vesico-urethral anastomosis time; C, complications; TR, transfusion rate; EC, early continence; CR, conversion rate; LOS, length of stay; CRT, catheter removal time; PSM, positive surgical margin rate; EBL, estimated blood loss.
Herrell and Smith 2005 [1]1>2500 RRPsOT, EBL, LOS, TR, continence, potency, PSM 250
Gumus et al. 2011 [8]1Laparoscopically naïveOT, EBL, LOS, PSM, EC, potency 80–120
O'Malley et al. 2006 [9]2Laparoscopically naïveOT, VUAT, PSM 40: OT, 10: VUAT, 200: PSM
Gyomber et al. 2010 (A) [10]  OT, EBL, TR, PSM, CR, C 50: OT, 150: PSM
Sooriakumaran P et al. 2011 (A) [11]3 OT, PSM rate 750: OT, 1600: PSM
Doumerc et al. 2010 [12]1 OT, PSM, C, ECOne-sample t-test, joinpoint regression, chi-squared with Yates correction, anova110 : OT; 140 : PSM (pT2);170: PSM (pT3); 200: EC
Tabata et al. 2011 (A) [13]1 OT, PSM, C 100: PSM; >200: OT
Kim et al. 2010 (A) [14]  OT, LOS, EBL, pad free continence rate, potency <200: LOS, OT, EBL, PSM, pad-free continence rate; >200: potency
Gyomber et al. 2010 (A) [15]  OT, EBL, PSM, LOS, early postoperative complications 50: PSM (pT2)
Gyomber et al. 2011 (A) [16]13 PSMLogistic regression and weighted means50: PSM
Sanchez-Salas et al. 2011 (A) [17]3>300 LRPsPSM 100: PSM (pT2)
Jung et al. 2010 (A) [18]8Laparoscopic surgeonsPSM 200
Chang et al. 2011 (A) [19]8Four robotic surgeons, four laparoscopic surgeonsPSMChi-squared test, multivariate analysisIndividual laparoscopic surgeons = robotic surgeons at 40 cases. laparoscopic surgeons group = robotic surgeons after 300 cases
Yen-Chuan Ou et al. 2011 [20]1 OT, console time, EBL, TR, PSM, node positive rate, CMann–Whitney U-test, Fisher's exact test, Yates correction150
Sharma et al. 2011 [21]2Extensive open and laparoscopic experienceOT, EBL, PSM, C, potencyMultivariable logistic regression, multivariable linear regression, chi-squared test>500
Giberti C et al. 2010 (A) [22]  OT, TR, CR, CRT, PSM, EC, potency 200
Linn et al. 2010 (A) [23]1 OT, EBL, LOS, TR, PSM, CR >20

In the study by Herrell and Smith [1], the learning curve for a surgeon who had performed >2500 retropubic radical prostatectomies (RRPs) was achieved after 250 cases. Gumus et al. [8] investigated the learning curve in laparoscopically naïve surgeons, which ranged from 80 to 120 cases. The study by O'Malley et al. [9] subdivided the learning curve of laparoscopically naïve surgeons into mean OT (40 cases), vesico-urethral anastomosis time (10 cases), and margin status (200 cases). In 10 studies, the previous experience of the participating surgeon/s was not documented. The mean OT reached a plateau after 50–200 cases [9-14] and the PSM rate reached a plateau after 50–1600 cases, depending on the authors definition of acceptable PSM rates [9-16]. After 200 cases the learning curve was reached for LOS (mean 1.13 days), EBL (mean 147.8 mL), pad-free continence rate and potency (at 12 months) [14]. The remaining four studies investigated the learning curve of experienced laparoscopic surgeons. In the study by Sanchez-Salas et al. [17], three surgeons with >300 LRP cases in their logbook were able to achieve acceptable positive surgical margin rates after 100 cases in pT2 cancers, whereas another study on the robotic surgery outcomes of eight experienced laparoscopic surgeons showed they achieved an acceptable positive surgical margin rate after 200 cases [18]. Chang et al. [19] reported that, after 300 cases, a group of four laparoscopic surgeons could achieve equal positive surgical margin rates to those of four experienced robotic surgeons.

The transition from a novice state to an experienced surgeon is evident by the improved trend in OT, EBL, complication and PSM rates (Figs 2, 3). Surgeons on the initial learning curve of RALP generally have longer OTs and higher mean EBL and complication rates [8, 20-22]. This transition depends on a number of factors, such as previous laparoscopic experience. The above-mentioned studies show that, irrespective of a trainee's previous laparoscopic experience, a significant transition is reached after 100 cases; the OT and EBL are significantly reduced (P = 0.008 for both) after this transition point. In addition, as the level of experience increases and the learning curve reaches its plateau, overall complications reduce significantly (P = 0.042).

Figure 2.

Line Plot for estimated blood loss (mL), complications and operative time (min) in RALP. C1 denotes novice surgeons on their initial learning curve and C2 denotes experienced surgeons who have reached a plateau on their learning curves.

Figure 3.

Positive surgical margins in prostatectomies performed by surgeons of different levels of expertise. Groups are divided based on the number of cases performed: <50, 50–99, 100–249, 250 to >1000. With increasing level of experience, the PSM rate decreases.

Open RRP, Perineal Prostatectomy and LRP

Three studies investigated the learning curve for both open and LRP, while one study evaluated the learning curve for perineal prostatectomy (Table 2 [24-30]). Vickers et al. [26] evaluated the learning curve of RRP in terms of oncological outcomes in a series of >7700 patients operated on by 72 surgeons at four major US academic medical centres between 1987 and 2003. Specifically, they defined patients with biochemical recurrence as those with a PSA level >0.4 ng/mL who then experienced a further PSA increase, then evaluated the 5-year recurrence-free survival probabilities, observing that a plateau in prostate cancer recurrence rates was reached after 250 cases. The 5-year probability of recurrence was 17.9% for patients treated by surgeons with limited experience (10 previous operations) and 10.7% for those treated by surgeons who had performed 250 previous operations, with the observed differences being statistically significant [24, 26].

Table 2. Open RRP, perineal prostatectomy and LRP learning curve studies
StudyProcedure*No of participantsPrevious experienceOutcome measuresStatistical analysisLearning curve No. of cases: outcome measure
  1. *The procedure setting for all studies was real patients. C, complications; RR, recurrence rate; TR, transfusion rate; OT, operative time; EBL, estimated blood loss; PSM, positive surgical margin rate.
Vickers et al. 2007 [24]Open prostatectomy7220–102 previous casesProstate cancer recurrence, PSA (>0.4 ng/mL)Multivariable survival time regression models<50: 36% RR; 50–99: 29% RR; 100–249: 23% RR; 250–999: 22% RR; >1000: 11% RR
Saito et al. 2011 [25]Open prostatectomy5 OT, EBL, TR, PSManova, Fischer's exact test29: TR, 20: OT
Vickers et al. 2010 [26]Open prostatectomy7220–102 previous casesPSMMultivariable survival time regression models10: 40% PSM, 250: 25% PSM
Eliya et al. 2011 [27]Perineal prostatectomy2Experienced open surgeonsOT, LOS, C, PSM, capsular penetration statusanova, chi-squared testNo clear learning curve after 96 cases, but PSM improved.
Hruza et al. 2010 [28]LRP5<20 previous LRPsCPearson x2 test and Fisher's exact test, logistic regression model was used for multivariable analysisThird-generation surgeons learning curve shorter than first- and second-generation
Secin et al. 2010 [29]LRP51

LRP

22; <50 cases

4; 50–99 cases

13; 100–249 cases

12; 250 or >

ORP

20; 0

5; 1–10

13; 11–99

13; 100 or >

PSMLogistic regression model, multivariable models, permutation test200–250
Vickers et al. 2009 [30]LRP29 RRMultivariable models10: 17% RR; 250: 16% RR; 750: 9% RR

Eliya et al. [27] were the only authors to investigate the learning curve for perineal prostatectomy as depicted by two experienced open prostatectomy surgeons. To determine a learning curve, the patients were divided into four groups: group 1 cases 1–22; group 2 cases 23–53; group 3 cases 54–74; and group 4 cases 75–96. Although no clear learning curve was attained after the 96 cases were reviewed, the PSM rate gradually improved with increasing experience of the technique, with only two positive margins obtained in the fourth group compared with six in the first group and 11 in both the second and third groups.

Hruza et al. [28] analysed complications in 2200 consecutive patients who underwent LRP between 1999 and 2008 at a single institution. Complications were classified according to the modified Clavien system. Five surgeons were divided into three groups; first-generation surgeons, with vast open surgical experience with no laparoscopic training, second-generation surgeons with experience in open surgery who were trained by first-generation surgeons, and third-generation surgeons with no or limited experience in open surgery who were trained by first- or second-generation surgeons using a special training programme. The learning curve of third-generation surgeons reached a plateau earlier compared with that of the first-generation surgeons (250 vs 700 cases).

Secin et al. [29] analysed records from 8544 consecutive patients with prostate cancer treated laparoscopically by 51 surgeons at 14 academic institutions in Europe and the USA. The probability of a PSM was calculated as a function of surgeon experience with adjustment for pathological stage, Gleason score and PSA. A second model incorporated previous experience with open RRP and surgeon generation. PSMs occurred in 1862 patients (22%). There was an apparent improvement in PSM rates up to a plateau at 200 to 250 surgeries. Changes in PSM rates once this plateau was reached were relatively minimal relative to the CIs. The absolute risk difference for 10 vs 250 previous surgeries was 4.8% (95% CI 1.5, 8.5). Previous open radical prostatectomy experience was not statistically significant when added to the model.

In 2009, Vickers et al. [30] conducted a retrospective cohort study of 4702 patients with prostate cancer treated laparoscopically by 29 surgeons in seven institutions in Europe and North America between January 1998 and June 2007. Multivariable models were used to assess the association between surgeon experience at the time of each patient's operation and prostate-cancer recurrence, with adjustment for established predictors. The 5-year risk of recurrence decreased from 17% to 16% to 9% for a patient treated by a surgeon with experience of 10, 250 and 750 previous LRPs, respectively. The learning curve for LRP was slower than the previously reported learning curve for open surgery (P < 0.001). Interestingly, surgeons with previous experience of open RRP had significantly poorer results than those who started the procedure using laparoscopy with no previous open prostatectomy experience. Although the PSM rate can be influenced by a number of preoperative factors (PSA level, a patient's BMI, Gleason score and clinical stage of disease), a surgeon's level of experience can affect the PSM rate and thus oncological outcomes (Fig. 3). Data from the above studies [25-27, 29, 30] suggest there is an improved PSM rate as the surgeons become more proficient in performing prostatectomies.

Percutaneous Nephrolithotomy

A total of six studies evaluated the learning curve for PCNL and renal access (Table 3 [31-36]). In addition to total OT, complications and LOS the variables measured included stone extraction rate, stone-free rate (SFR) and fluoroscopy time. Ziaee et al. [31] reviewed 105 consecutive PCNL operations performed by a fellow in endourology, with no previous experience in performing independent PCNL. The OT decreased from a mean of 95.4 min in the first to 15th patients to 78.3 min in the 31st to 45th patients, and then remained unchanged. Minor complications were only observed in the first to 45th patients. Stone extraction percentage increased from a mean of 88.3% in the first to 15th patients to 99.3% in 91st to 105th patients. The percentage of patients with no residual fragments decreased from 53% in the first to 15th patients to 6.7% in the 91st to 105th patients.

Table 3. Percutaneous nephrolithotomy and percutaneous renal access learning curve studies
AuthorProcedure*No of participantsPrevious experienceOutcome measuresStatistical analysisLearning curve No. of cases: outcome measure
  1. *The procedure setting for all studies was real patients. (A), Abstract; C, complications; SER, stone extraction rate; SFR, stone free rate; OT, operative time; LOS, length of stay.
Ziaee et al. 2010 [31]PCNL10OT, C, SER, SFR, No of access, tubeless casesChi-squared test, anova45: C and OT; 105: SER
Tanriverdi et al. 2007 [32]PCNL10 PCNL, extensive experience in other endourology procedures i.e. transurethral resection and ureterorenoscopyOT and fluoroscopy time screeninganova, chi-squared test, Mann–Whitney U-test, or t-test60
Negrete-Pulido et al. 2010 [33]Percutaneous renal access/puncture10Time to correct puncture, fluoroscopy time screeningDescriptive analysis, anova and Markov chain50
Jang et al. 2011 [34]Flank PCNL1 OT, C, SFR, drop in haemoglobin level, LOS, need for additional procedures after surgeryanova35
Godbole et al. 2010 (A) [35]PCNL  OT, C, success of 1st pass puncture & track formation, complete stone clearance 12
Bucuras et al. 2011 (A) [36]PCNL  OT, LOS, SFR, C 40

Tanriverdi et al. [32] reviewed 104 procedures in a PCNL-naïve surgeon. PCNL procedures were analysed in seven sets of 15 cases regarding the OT and fluoroscopy time, stone size, SFR, blood transfusion rate and EBL. The mean OT was 2.4 h for the first 15 cases, which decreased to a mean of 1.5 h for cases 46 to 60. No further decrease in the OT was observed after case 60. The fluoroscopic screening time reached a peak of 17.5 min in the first 15 cases, and dropped to a mean of 8.9 min for cases 46 to 60. The decline in the mean fluoroscopy screening time continued in cases 61 to 104, but the decline was not significant. There was no significant difference in stone size, SFR, blood transfusion rate or EBL among each set of cases. A similar study by Negrete-Pulido et al. [33] reviewed 92 percutaneous renal accesses in patients with renal stones or PUJ obstruction performed by an endourology fellow without previous experience in percutaneous surgery. The rate of success increased from 82.5 to 97.6% after the first 40 punctures. Puncture time and fluoroscopy time decreased as the number of procedures increased. The incidence of complications was 30% for the first 20 cases, decreasing to 10% in the next 20 cases and 3.7% in the last 33 cases.

Jang et al. [34] calculated the learning curve for 53 PCNL procedures performed in the flank position by one experienced endourological surgeon. The mean OT for the 53 patients was 97.3 ± 43.1 min. The mean OT gradually decreased as the surgeon accumulated experience. From the 36th case, the mean OT showed a significant decrease to 72.2 ± 24.1 min (P = 0.003). The overall (range) SFR was 64.2 (61.1–76.5)% for all procedures. There were no significant differences in the drop in haemoglobin level, SFR, re-treatment rate, LOS or complication rate. There was no injury to the bowel or renal vessels, and no other major complications occurred.

Other Upper Urinary Tract Procedures

A total of 12 studies investigated the learning curve for upper urinary tract procedures (Table 4 [37-48]). These consisted of robot-assisted laparoscopic partial nephrectomy (RALPN) [37-42], laparoendoscopic single-site (LESS) donor nephrectomy [43], retroperitoneal laparoscopic donor nephrectomy [44], laparoscopic pyeloplasty [45], retroperitoneal laparoscopic partial nephrectomy [46], LESS pyeloplasty [47] and robot-assisted paediatric pyeloplasty [48].

Table 4. Learning curve studies for upper urinary tract procedures
AuthorProcedure*No of participantsPrevious experienceOutcome measuresStatistical analysisLearning curve No. of cases: outcome measure
  1. *The procedure setting for all studies was real patients. (A), abstract; C, complications; WIT, warm ischemia time; OT, operative time; EBL, estimated blood loss; LOS, length of stay.
Lavery et al. 20011 [37]RALPN1>100 RALPs and 15 robot-assisted pyeloplasty'sOT, WITChi-squared and Student's t-test5 to convert from laparoscopic to robotic approach
Pierorazio et al. 2011 [38]RALPN1Robot-naïveOT, WIT, EBLt-test, chi-squared test, anova25 to convert from laparoscopic to robotic approach
Finnegan et al. 2010 (A) [39]RALPN1 OT, WIT, LOS, C, EBL Significant difference in WIT, LOS and C in 1st 75 vs last 75 cases
Jacobsohn et al. 2011 (A) [40]RALPN1 WIT >72, no plateau
Tufek et al. 2011 (A) [41]RALPN1Extensive previous robotic surgery experienceOT, WIT, EBL 32
Oh et al. 2011 (A) [42]RALPN1 OT, WIT, LOS, CLinear regression analysis, multivariate analysis20: WIT and C, 50: OT
Choi et al. 2011 (A) [43]LESS donor nephrectomy1 OT, WIT, LOS, C, change in donor GFR 50: OT
Ye et al. 2011 (A) [44]Retroperitoneal laparoscopic donor nephrectomy  OT, EBL, LOS, C 40
Naya et al. 2011 (A) [45]Laparoscopic pyeloplasty1 OT, C intraoperative, C postoperative 51
Ma 2010 (A) [46]Retroperitoneal laparoscopic partial nephrectomy1 OT, WIT, EBL, postoperative outcome 65
Best et al. 2011 [47]LESS pyeloplasty1Experienced laparoscopic surgeon30 days complication rate 11
Sorensen et al. 2011 [48]Robot-assisted paediatric pyeloplasty2No robotic surgery experience, 20 laparoscopic cases/yearOT, C, postoperative pain, LOS, surgical success 15–20: OT

Robot-Assisted Radical Cystectomy

Two studies investigated the learning curve for RARC (Table 5 [49, 50]). Both papers used total OT, cystectomy time, pelvic lymph node dissection time, EBL, PSM rate, complications and LOS to deduce a learning curve. In the study by Hayn et al. [49], 496 patients who underwent RARC by 21 surgeons at 14 institutions from 2003 to 2009 were prospectively studied. The mean OT was 386 min, the mean EBL was 408 mL and the mean lymph node yield (LNY) was 18. Overall, 34 of 482 patients (7%) had a PSM. Using statistical models, it was estimated that 21 patients were required for OT to reach 6.5 h and 8, 20 and 30 patients were required to reach lymph node yields of 12, 16 and 20, respectively. For all patients, PSM rates of <5% were achieved after 30 patients. For patients with pathological stage higher than T2, PSM rates of <15% were achieved after 24 patients.

Table 5. Robot-assisted radical cystectomy learning curve studies
AuthorProcedure*No of participantsPrevious experienceOutcome measuresStatistical analysisLearning curve
  1. *The procedure setting for all studies was real patients. LNY, lymph node yield; OT, operative time; EBL, estimated blood loss; PSM, positive surgical margin rate.
Hayn et al. 2010 [49]RARC21 OT, LNY, EBL, PSMFisher's exact test, Kruskal–Wallis non-parametric test, nonlinear mixed model21: OT, 30: LNY, 30: PSM
Guru et al. 2009 [50]RARC RALP experienceOT, LNY, PSM 16: OT, 11: EBL, 12: LOS, 30: LNY

Guru et al. [50] divided 100 RARC procedures into four groups. Overall OT decreased from 375 min in group 1 to 352 min in group 4, with <1% change in OT after case 16. Time from incision to bladder extirpation decreased from 187 min in cohort 1 to 165 min in cohort 4. Time for pelvic lymph node dissection increased from 44 min in cohort 1 to 77 min in cohort 4. Lymph node yield increased from 14 nodes in cohort 1 to 23 nodes in cohort 4. PSMs decreased from four patients in cohort 1 to no patients in cohort 4. There was no change in complication rate which was nine patients in cohorts 1 and 4.

Quality Evaluation of Included Studies

A tool to evaluate educational articles describing learning curves is not available in the literature. A number of statistical methods were used to demonstrate learning curves in operating procedures. While 24 of the 44 studies did not document the statistical methods, in the remaining 20 studies, a range of tools were used such as: the Mann–Whitney U-test, Fisher's exact test, one-sample t-test, joinpoint regression, the chi-squared test, anova, multivariable logistic regression, multivariable linear regression, Pearson's test and the Kruskal–Wallis non-parametric test. While anova and the t-test are relatively simple methods of showing the learning curve, the performance means are analysed in arbitrary groups, and one cannot therefore identify where specifically the learning curve reaches a plateau.

The Friedman test and regression analysis as used by eight studies [12, 16, 21, 24, 26, 28, 29, 42], are more sophisticated methods, looking at learning curves through the analysis of repeated measures across time, which allows the identification of significant changes in performance from repetition to repetition and enables investigators to identify learning plateaus on the learning curve.

Although not used by any of the studies identified in the present review, cumulative sum (CUSUM) charts are a useful way of identifying when performance begins to decline or improve and, therefore, when remediation is necessary. CUSUM charts represent a sequential analysis that can identify small changes to the means of a process. CUSUM involves summing differences between measured values and a benchmark value to determine the extent of deviation away from the benchmark. A threshold for change in both positive and negative value away from the benchmark is calculated [51].

Discussion

To our knowledge this is the first systematic review of learning curve studies within urological surgery (Fig. 4). The number of procedures required varied from procedure to procedure, and was often affected by the surgeon's previous experience. Broadly speaking, the outcome measures used to calculate learning curves were either surgery-related (OT, EBL), oncological, (PSM rate), or related to quality of life (pain, erectile dysfunction, incontinence).

Figure 4.

An overview of learning curves for urological procedures. LC, learning curve; RR, recurrence rate; C, complications; TR, transfusion rate; SER, stone extraction rate; OT, operative time; UVT, vesico-urethral anastamosis time; PSM, positive surgical margin; SFR, stone free rate.

The results of the present review provide a guide for trainees, trainers and experienced surgeons alike. In a surgical generation where minimally invasive techniques have superseded open approaches, the thought of learning how to perform a RALP or LRP without previous laparoscopic or even open experience is daunting; however large multicentre studies investigating, in particular, the learning curve for LRP showed that the learning curve for complications reached a plateau earlier for third-generation surgeons compared with first- generation surgeons (250 vs 700 cases) [28]. When PSM rate was used as the outcome variable for defining the learning curve, previous open experience and surgeon generation did not improve the PSM rate; on the contrary, poorer results were seen in surgeons with previous experience compared with those whose first operation was laparoscopic, suggesting that the rate is primarily a function of specifically laparoscopic training and experience [29, 30]. Similar observations can be made with respect to RALP. Surgeons with open and laparoscopic experience have a learning curve of 250 and 100–300 cases, respectively [1, 17, 19], whereas surgeons without such experience require 40 cases to reach similar OTs, 10 cases to perform the vesico-urethral anastamosis at an equivalent time and 200 cases to reach acceptable PSM rates [8, 9]. Converting from laparoscopic to RALPN requires a learning curve of 5–25 cases [37, 38], while learning robot-assisted paediatric pyeloplasty is associated with a learning curve of 15–20 cases [48]. These represent relatively short learning curves and, given the benefit of robot-assisted laparoscopic surgery, this is a potentially worthwhile investment for high-volume centres.

One of the inherent flaws in the documented learning curves is the absence of a standardized definition of the optimum method of calculating a learning curve. Even within the same procedure authors have measured different variables and set different endpoints within a given variable. For example, in the study by Gumus et al. [8], a 6% PSM rate was deemed to be acceptable to attain competence, whereas in the studies by Gyomber et al. [15], Sooriakumaran et al. [11] and Doumerc et al. [12] the overall PSM rates were 19.3, 19.7 and 21%, respectively, as the benchmark in their learning curves. This represents a large variation and a potential bias in the documented learning curves. In addition, the trainer is often required to take over when the trainee is not progressing during a case, but this has not been accounted for in the studies and is inevitably a difficult variable to control for, unless part of a trial. Furthermore, the complexity of each case will play a large role in the learning curve outcomes but there is a paucity of data on the learning curves in this regard. Within any given procedure there is a separate learning curve for individuals, which will not follow the learning curve calculated by analysing a group of surgeons. To minimize the adverse outcomes associated with an individual's learning curve, experienced supervision should be sought [52].

The above-mentioned discrepancies make comparisons between studies inaccurate and confusing to the learner. For each procedure there needs to be a consensus about which variables are most important to measure when defining competence, e.g. patient outcomes, survival and/or surgical processes such as OT and EBL, as well as about fixed endpoints within each variable and what statistical model to use [53].

Extensive laparoscopic training literature has shown that it is not possible to calculate how many procedures have to be performed before somebody is competent in a certain procedure [54]. The learning curve for very basic laparoscopic exercises differs between subjects, ranging from 5.5 to 21 h and 171 to 782 repetitions [55]. This indicates that no single number of surgeries can be identified that applies to all surgeons because of immense inter-individual differences in technical aptitude and previous experience. It is more accurate to define proficiency levels, which have to be reached before a surgeon is deemed competent in a certain procedure, i.e. maximum time to perform circumcision or maximum EBL during RALP. Furthermore analysing the individual steps involved in a procedure may provide a more accurate depiction of the learning curve. Certain aspects of a procedure are more technically challenging than others, therefore, these will require particular attention during training. The factors that affect progression within a certain procedure need to be examined, in particular, the non-technical as well as technical attributes of the trainee.

The majority of the learning curve studies have focused on complex, innovative procedures with a paucity of data pertaining to the routine work of a urological surgeon. Junior trainees and trainers would value learning curve data on scrotal surgery, circumcisions, flexible and rigid cystoscopy, TURPs, transurethral resection of bladder tumours, orchidectomies and other procedures which make up the majority of the workload of most urologists, in particular those in training.

The most important single factor to consider when developing new surgical techniques is patient safety. With large studies showing that a surgeon's case volume equates to better oncological outcomes [24, 26, 30], there is a lack of studies examining the learning curve in a simulated setting. Given that we know that if a surgeon has performed only 10 RRPs the 5-year probability of prostate cancer recurrence is ∼17.9%, while 250 previous procedures reduces the risk of recurrence by 7.2% to 10.7%, is it ethical to train in the real-life environment? Wet- and dry-laboratory specimens, courses and fellowships have traditionally been used to overcome the early stages of the learning curve and to refine existing skills and techniques; however, with the expansion in surgical simulation, inanimate trainers are available in many forms and provide an exciting avenue for future surgical trainees. Direct comparison between these and the traditional wet-and dry-laboratory methods have yet to be assessed, and perhaps a combination of training environments will lead to the most effective training. Mechanical and virtual reality simulators have been validated as training and assessment tools and have been shown to improve a surgeon's performance in the operating theatre, suggesting that simulation training may contribute to the acquisition of the experience necessary to advance along the learning curve [56-60]. Moreover, simulation has been shown to shorten the length of the learning curve for real procedures in the operating theatre when compared to no simulation training [61]. For effective simulation-based education, proficiency-based curricula should be used to ensure individual competency. An example of one such curriculum is the structured curriculum of skill acquisition on the virtual reality simulator the MIST-VR, which was developed by investigating the learning curves of 20 medical students on 12 tasks. Trainees work through all 12 tasks at easy then medium difficulty. The skills acquired at easy and medium difficulty over repeated practice should then translate to specific performance targets at the hard difficulty level where the assessment of skill can occur against an expert-derived performance standard [62]. Such performance-based criteria for skill proficiency take into account that different learners will progress at different rates along the learning curve. Other validated simulator curricula include those developed by LapSim and LapMentor [63, 64]. A systematic training and assessment of technical skills framework addresses technical skills training, from early acquisition to final granting of privileges for practice, by providing a roadmap to guide the development of a curriculum to address education along the learning curve [61]. Early education focuses on knowledge-based learning and then progresses to providing an understanding of the deconstructed portions of a procedure that can be practised on a validated model. These skills can then be assessed as they translate to the real operating theatre, culminating in final assessment to advance the trainee to independent practice. Such approaches minimize the inherent risks with learning fundamental principles of new procedures in the operating theatre. Cost-effectiveness analyses are still awaited.

The present review has several limitations. The poor quality and study design of included articles reflects the lack of well-designed learning curve studies, and led to challenges in summarizing the procedural learning curves. Within any given procedure, a variety of different outcome measures were used with different endpoints, making comparisons between studies difficult. Often endpoint targets were used for the calculation of the learning curve as opposed to the rate of achieving the outcome itself. Because of these inconsistencies of data reporting, it was not feasible to correlate the various outcome measures for the given procedures. Medical subject heading (MeSH) terms on learning curves are lacking in the literature databases, which leads to the possibility of missing some articles. We attempted to overcome this bias by searching for keywords and common authors on the topics. Most of the studies lacked descriptions of appropriate methodology, outcome measures and use of statistical tests, thus making a meta-analytical approach impossible.

Our findings relate to other studies which have shown that it is possible to quantify learning curves using literature reviews, even though the process is often hindered by insufficient reporting which then limits formal statistical estimation. Standardized reporting and collation of data should be developed for each procedure to improve overall reporting, level of consistency and allow more accurate calculations of learning curves [65].

Conclusions

The current studies can act as a guide for developing urologists to follow during their training. The complexities associated with defining procedural competence are vast. The majority of learning curve trials have focused on the latest surgical techniques with a paucity of data pertaining to basic urological procedure information that would be useful for junior trainees. Future work should include the identification of outcome measures for individual procedures and validation of the learning curves with a systematic approach. Learning curves can either be based on outcome evaluation or observation of practice by trainers until proficiency is gained.

Acknowledgements

We would like to acknowledge Prof Craig Ramsey, healthcare assessment programme director at the University of Aberdeen for his contribution to the review of this manuscript. P.D. acknowledges financial support from the National Institute for Health Research (NIHR) Biomedical Research Centre based at Guy's and St Thomas' NHS Foundation Trust and King's College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. P.D. also acknowledges the support of the MRC Centre for Transplantation, London Deanery, London School of Surgery and Olympus. P.D., S.K. and K.A. acknowledge funding for the SIMULATE project from the Urology Foundation (TUF) and the BAUS.

Conflict of Interest

Khurshid Guru is a Board Member of Surgical Simulated Services. No other conflicts declared.

Abbreviations
RALP

robot-assisted laparoscopic prostatectomy

LRP

laparoscopic radical prostatectomy

PCNL

percutaneous nephrolithotomy

RARC

robot-assisted radical cystectomy

OT

operating time

EBL

estimated blood loss

PSM

margin status

LOS

length of hospital stay

RRP

retropubic radical prostatectomy

RALPN

robot-assisted laparoscopic partial nephrectomy

LESS

laparoendoscopic single-site

CUSUM

cumulative sum

Ancillary