SEARCH

SEARCH BY CITATION

Keywords:

  • prostatectomy;
  • robot;
  • learning;
  • learning rate;
  • learning curve

Abstract

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS, SUBJECTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGEMENTS
  8. CONFLICT OF INTEREST
  9. REFERENCES

Study Type – Therapy (case series) Level of Evidence 4

OBJECTIVE

To adapt an industrial definition of learning-curve analysis to surgical learning, and elucidate the rate at which experienced open surgeons acquire skills specific to robot-assisted radical prostatectomy (RARP) at a community-based medical centre.

PATIENTS, SUBJECTS AND METHODS

The total procedure time (TPT) of the first 75 RARPs, performed by three surgeons experienced with retropubic RP, was analysed to determine the point at which their learning rate stabilised. Operative characteristics were compared before and after this point to isolate the plateau of learning rate as a mark of acquiring surgical skill. The operative characteristics examined were TPT, estimated blood loss (EBL), bladder neck contractures (BNC), positive margins (PM) and length of hospital stay (LOS).

RESULTS

The mean rate of TPT decrease, for procedures 1–75, was 13.4% per doubling of RARPs performed. After the first 25 procedures the TPT decreased at a rate of 1.8% per doubling, not significantly different from 0 (P > 0.05). There was no significant difference between procedures 1–25 and 26–75 in rates of EBL, BNC and PM. There was a significant change for all surgeons in TPT, with a mean of 303.1 min (RARPs 1–25) vs 213.6 min (26–75) (P < 0.001), and LOS, of 2.1 days (1–25) vs 1.4 days (26–75) (P < 0.001).

CONCLUSIONS

An industrial definition of learning-curve analysis can be adapted to provide an objective measure of learning RARP. The average learning rate for RARP was found to plateau by the 25th procedure. Also, the learning rate plateau can serve as an objective measure of the acquisition of surgical skill.


Abbreviations
(R)(RA)(L)RP

(retropubic) (robot-assisted) (laparoscopic) radical prostatectomy

TPT

total procedure time

EBL

estimated blood loss

BNC

bladder neck contractures

LOS

length of stay

PM

positive margins

ITI

intraoperative interval

LCA

learning curve analysis.

INTRODUCTION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS, SUBJECTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGEMENTS
  8. CONFLICT OF INTEREST
  9. REFERENCES

Robot-assisted radical prostatectomy (RARP) has rapidly become an accepted method for the surgical management of organ-confined prostate cancer in many regions of the USA. As of 2004, there were »200 daVinci Surgical Systems (Intuitive Surgical, Sunnyvale, CA, USA) in use in the USA, and by June 2009 this had increased to >900 [1]. Many of these systems have been purchased by community-based medical centres, as centrepieces of multidisciplinary minimally invasive surgical programmes. Of the total number of RPs performed in 2005, an estimated 20% were completed by RARP [2], and by 2007 this proportion was reported, by Intuitive Surgical, to be >50%[3]. Although many studies have reported the short-term benefits of RARP, particularly for blood loss and length of stay (LOS), long-term functional and oncological benefits have not been definitively confirmed. Despite this lack of evidence supporting improved long-term outcomes, RARP is rapidly replacing retropubic RP (RRP) in regions of the USA where the daVinci Surgical System is available. One of the keys to this rapid and widespread dissemination is the relative ease with which open surgical skills are transferred to the laparoscopic environment with the aid of the daVinci System.

Although the actual dissection performed in RARP is anatomically quite similar to RRP, it is technically dissimilar, requiring surgical skills that are both equipment-specific and procedure-specific. As urological residency and fellowship programmes train the next generation of urologists in the surgical management of prostate cancer, and as experienced open surgeons embrace this emerging new technology, the issue of appropriate training remains paramount.

Studies have proposed a ‘learning curve’ for RARP of 12–150 procedures [4,5] for the experienced open surgeon, with most studies proposing 18–45 [6–8]. This variability in analysis is, in no small part, due to the lack of agreement when defining a ‘learning curve’. For RARP the learning curve has been described primarily subjectively, based on perceptions of competence [5,7]. It has also been described as the time to ‘4 h’ proficiency, i.e. to complete the surgery in <4 h [4,8], and perhaps most objectively, as the point at which the linear regression curve of total operative time for RARP crosses that of traditional laparoscopic RP (LRP) [4].

The concept of the learning curve was first presented in an industrial context by T.P. Wright in 1936, in his seminal work ‘Factors affecting the costs of airplanes’[9]. He discovered that unit costs decrease exponentially as a function of cumulative output. When cost data were plotted on a scattergram, the line of best fit took the form of a power function. This line was called the ‘learning curve’. Over the past 70 years learning-curve analysis (LCA) has been used extensively by manufacturing industries [10,11] and increasingly by energy industry analysts [12–14] to model their previous experiences and project future costs.

The term ‘learning curve’ has also been loosely used to describe the time course required to acquire the skills necessary to satisfactorily perform a surgical procedure. The aim of the present study was to provide a more rigorous application of LCA, presenting a simple, objective and verifiable method of analysing the rate at which experienced open surgeons learn RARP.

PATIENTS, SUBJECTS AND METHODS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS, SUBJECTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGEMENTS
  8. CONFLICT OF INTEREST
  9. REFERENCES

The three urological surgeons included in this study underwent residency training between 1988 and 2000, and had performed >150 RRPs each. Surgeons 1 and 3 had formal laparoscopic experience in residency, while surgeon 2 did not. Each surgeon had a 2-day training course with the daVinci laparoscopic system, which included robot-assisted porcine cholecystectomy and nephrectomy, as well as two cadaveric RARPs. Each surgeon was mentored for their first two procedures by an experienced robotic laparoscopic surgeon. The first 75 consecutive RARPs by each of the three surgeons were included in this study. The overall study period was 3 years, with surgeons 1 and 2 completing their first 75 cases in 2 years, and surgeon 3 completing them in 3 years, beginning in November 2003. At the beginning of the study period surgeons 1–3 were 39, 45 and 34 years old, respectively. All procedures were performed at the Maine Medical Center, Portland, Maine, USA.

The first 20 patients were included using the following selection criteria: a PSA level of <10 ng/mL, a body mass index of <30 kg/m2, and a Gleason score of ≤7. All 20 patients were impotent before surgery and none had undergone previous intraperitoneal surgery. Erectile function was assessed before surgery using a standardized patient-reported questionnaire [15]; patients with erectile function adequate for intercourse were initially managed with RRP. After the first 20 cases all patients were offered RARP. Of the first 20 patients, 13 (65%) were operated by surgeon 1, four (20%) by surgeon 2 and three (15%) by surgeon 3. Subsequently RARP was offered to all medically fit candidates with no restriction.

The RARP was performed using modifications of the Vattikuti Institute Prostatectomy technique [16], as described by Ahlering et al.[4], with no significant inter-surgeon changes to the operation over the study period. In particular, no changes were made in the management of the dorsal vein complex, as all were suture ligated; nor were changes made to the single-layer urethrovesical anastomosis. Operative characteristics (summarized in Table 1) included estimated blood loss (EBL), bladder neck contractures (BNC), positive margins (PM) and LOS.

Table 1.  The operative characteristics from before and after the learning curve plateau; statistically significant characteristics are shown in bold
CharacteristicsSurgeon 1PSurgeon 2PSurgeon 3P
Procedure no.Procedure no.Procedure no.
1–2526–751–2526–751–2526–75
Mean TPT, min345.4247.7<0.001298.0203.7<0.001265.8189.5<0.001
EBL, mL 1491820.169 252 2030.103 122 1390.579
BNC, %  4  41.000   4   21.000   4  41.000
LOS, days  2.2  1.66 0.050  2.12  1.38<0.001  2.04  1.18<0.001
PM, %   8  240.122  24  221.000  28  160.236

Intraoperative times were recorded on a project specific time sheet by the operating room circulating nurse. The surgery was divided into five parts and each of these intraoperative intervals (ITIs) was evaluated, along with total procedure time (TPT). Two of the ITIs were peri-robotic: ‘incision to robot docking’ and ‘robot undocking to procedure end’; three intervals concerned times using the robotic interface: ‘robot docking to undocking’, ‘dissection time’ (i.e. robot docking to anastomosis start) and ‘anastomosis’.

Learning rates are described in energy-industry reports as the percentage at which the unit cost decreases with every doubling of cumulative production [14]. For the purpose of our study we defined the learning rate as the percentage decrease in operative time (in minutes) per doubling of cumulative procedure number.

The conventional form of the learning curve is a power function [9]y=ax−b, where y is the time (minutes) required to perform the xth procedure, a is the time required to perform the first procedure, x is the cumulative number of procedures and –b is a parameter measuring the rate by which procedure length is reduced as cumulative experience is gained, and is called the ‘learning rate exponent’[12].

When data are plotted on a scattergram, as in Fig. 1, a line of best fit can be generated, using nonlinear regression. This line takes the form y=ax−b. Each doubling of cumulative experience results in an operative time reduction of 1 – 2-b. An example showing the derivation of this equation is as follows:

image

Figure 1. A scattergram of the TPT for procedures 1–75 for all three surgeons. To the right of each line of best fit is its equation. The exponent is termed the ‘learning rate exponent’ and can be used to solve the equation for each surgeon’s unique learning rate. The learning rates derived from these best fit lines are given in bars 2–4 of Fig. 3.

Download figure to PowerPoint

TPT data of surgeon 1 for procedures 1–75. The equation of the line of best fit and learning rate exponent can be seen on Fig. 1. Procedure nos 2 and 4 are used to represent a doubling of cumulative experience.

Learning rate

  • image

This gives a learning rate of 11.73%.

Note that regardless of whether the doubling of procedure number is 2–4 or 400–800, the equation will reduce in the same manner, producing the same learning rate.

When the log of both sides of the learning rate equation is taken, it gives the equation of a linear function providing a more familiar representation of the steepness of a learning curve. The equation is log y=log a − b(log x) and is represented in Fig. 2.

image

Figure 2. A scattergram of TPT on log-scaled axes, showing the ‘steepness’ of each surgeon’s learning curve. Note that with more rapid reduction in TPT (as seen with surgeon 2) the learning curve is steeper. Surgeon 2’s learning rate, as seen in Fig. 3, is the fastest.

Download figure to PowerPoint

To ascertain learning rates for all of the ITIs examined, individual data sets were established for each of the three surgeons for each of the five ITIs and TPT. The data points were plotted on scattergrams and trend lines were generated for each. From the trend lines the learning rate was calculated, for each of the ITIs for incremental procedure intervals 1–75, 11–75, 16–75, 21–75, 26–75 and 31–75. After developing the learning curves, the learning rates of each incremental procedure interval for the ITIs and TPT can be represented in bar graph form. These data are represented in Fig. 3.

image

Figure 3. Learning rates calculated from the lines of best fit of scattergrams at incremental procedure intervals for: A, The first four bars show the learning rates derived from the graph in Fig. 1, similar graphs (not shown) were made for the subsequent procedure intervals. The vertical dashed line represents the point after which the aggregate learning rate has reached a plateau. B, the perirobotic ITIs; there is no meaningful plateau of learning for these portions of the surgery; C, the robotic-specific portions of the surgery; note the distinct plateau that follows the pattern of TPT.

Download figure to PowerPoint

To determine the plateau of the learning curve, a t-test was used on the linear regression lines of the TPT data at the six incremental procedure intervals above. We considered that a plateau had been reached by a given procedure number if the slope of the linear regression line for TPT after that point was not significantly different from zero by the t-test (P > 0.05).

On determining the procedure number by which the learning rate reached a plateau, we examined the perioperative characteristics, comparing TPT, EBL, BNC, PM and LOS. BNC was defined as any patient requiring office-based dilatation or surgical intervention within a year of RARP. These variables were chosen as markers of surgical proficiency. By comparing operative characteristics from before and after the learning rate plateau, we could ascertain whether gains in efficiency could be related to gains in proficiency.

Both surgical intervals and clinical outcome data were gathered prospectively, while study design and model fitting were retrospective. For the statistical analysis we used standard software. Categorical data were compared using Fisher’s exact test, continuous variables using the two-tailed t-test, with P < 0.05 considered to indicate statistical significance.

RESULTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS, SUBJECTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGEMENTS
  8. CONFLICT OF INTEREST
  9. REFERENCES

Figure 1 presents the TPT of the first 75 cases for each of the three surgeons; the TPT for surgeon 1 was, on average, 1 h longer than for surgeon 3 and remained consistently longer throughout the first 75 procedures, while the TPT curve for surgeon 2 was about midway between surgeons 1 and 3. While in absolute terms surgeons 1, 2 and 3 varied by as much as 25% in their TPT (Fig. 1), Fig. 3 shows that their learning rates were similar, with <3% separating their learning rates over 75 procedures.

Figure 3 presents the collected data from numerous scattergrams. The first set of bars are the mean of the individual surgeon’s learning rates as derived from their best fit lines in Fig. 1. Subsequent sets represent the learning rates for the incremental procedure intervals. This shows the point after which the learning rate has either reached a plateau or approaches zero. For the TPT the plateau is reached between the 21st and 25th procedures. A mean of the three surgeon’s learning rates show this clearly; between the 21st and 75th procedure the improvements in TPT were at 4.07% per doubling of experience, whereas between the 26th and 75th procedure the learning rate was 1.80% and remained almost constant from that point forward. This point was validated using the t-test for the line of best fit generated on the mean data from the three surgeons. For procedures 21–75 the P value for the line of best fit was <0.05, while that for procedures 26–75 was P > 0.05. A statistically significant improvement in TPT is conferred by P < 0.05.

Figure 3B,C present learning rate data for the ITIs, which are subsets of the TPT. Figure 3B presents data of the portions of the surgery before and after robot involvement, and Fig. 3C while the robot is engaged. We assessed the learning rates of the anastomosis and dissection time, as they represent technically distinct aspects of the surgery.

The robotic-specific ITIs, as shown in Fig. 3C, show a dramatic reduction in learning rate between the 21st and 26th procedure. After the 25th procedure the learning rate approaches zero, similar to the TPT. The average learning rate for anastomosis was somewhat faster than that of the TPT or dissection, with the learning rate plateau achieved by the 20th procedure.

While TPT and the robotic-specific ITIs followed a consistent pattern of learning-rate plateau, the same was not true for the peri-robotic ITIs. The learning rates for the intervals represented in Fig. 3B show great variability between surgeons and failed to produce a meaningful plateau.

Table 1 provides the operative characteristics of the three individual surgeons at procedure number intervals 1–25 and 26–75. For all three surgeons there was no significant difference in BNC, PM and EBL before and after the plateau of the learning rate; TPT and LOS were significantly different for all surgeons.

DISCUSSION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS, SUBJECTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGEMENTS
  8. CONFLICT OF INTEREST
  9. REFERENCES

It is a fundamental human characteristic that a person engaged in a repetitive task will improve his or her performance over time [17]. T.P. Wright first formally codified this phenomenon in his observations of the aircraft industry during the build-up to World War II, leading to the development of LCA [9]. Wright noticed that when decreasing aircraft costs were plotted on a scattergram against numbers of aircraft produced, the line of best fit, determined by least mean squares methods, was a power function, y=ax−b.

Such derived ‘learning curves’ show the decline in unit cost of production as experience is gained. In the case of RARP this can be viewed as the decline in TPT (or one of the ITIs) as the number of procedures increases. Learning curves take the form of a power function, where unit costs (TPT) decrease exponentially as a function of cumulative output (procedure number) [13]. When the curves of these data are plotted on a logarithmically scaled axis, as in Fig. 2, it is a straight line. The slope of this line represents the ‘learning rate’, which is the percentage decline in costs (TPT) per doubling of cumulative production (number of surgical procedures) [14]. Simply stated, the percentage decline in TPT between the fourth and eight procedures will equal the percentage decline in TPT between the 400th and 800th procedures, assuming the learning rate is constant, which is not the case for RARP.

Analysts of emerging technologies, particularly energy analysts, use LCA when attempting to project the reduction of costs with increased experience [14]. Learning rates have been found to range between 10% and 30% across a wide variety of mature technologies. Energy analysts have also noted that there are alterations in the learning rate as an industry matures, due to the widespread acquisition of a given technology [13]. Acquiring skills particular to the da Vinci Surgical system parallels the acquisition of technical skills in industry, and was accompanied by a similar change in the learning rate, shown in Fig. 3, representing a plateau. Further adapting the language of industry analysts, the first phase has been called the ‘initial’ or ‘start-up’ phase, while the second phase, after the plateau, has been called the steady state [11].

The concept of a ‘steep’ learning curve has been misrepresented in much of the medical literature. Because the word ‘steep’ is commonly associated with a slope that is difficult to climb, an erroneous understanding of learning curves has prevailed. Contrary to common usage, a task with a ‘steep’ learning curve requires less time/effort to learn because gains in efficiency are made at a higher rate. For example, in Fig. 2, surgeon 2 has a steeper learning curve than surgeon 1. The rate of learning for surgeon 2 was 14.5% vs 11.7% for surgeon 1 over the entire series. The steeper learning curve for surgeon 2 represents faster learning. While this might seem a purely semantic distinction we would suggest the term ‘prolonged’ learning curve for a task which is difficult to learn. This would help to reduce ambiguity in discussions of learning curves.

In this study we identified the point at which a significant rate of decrease in TPT could no longer be detected for RARP, by identifying the plateau of the learning curve. This approach is transparent and has face validity, but has limitations, particularly in its inability to assess intra-operator variability. The selection of an optimal correlation structure in these analyses is uncertain. Because of our study design, formally assessing the correlation structure within surgeons was not a primary concern, as all data analysed came from one clinical practice, patients were essentially assigned randomly to surgeons, and the overall sample size was small. However, these limitations might be balanced by the ease of reproducibility, overall simplicity and transparency of our approach.

Although the TPT is far from a perfect marker of surgical learning, its plateau identifies a limit in the acquisition of surgical skill. While average results for TPT and robotic-specific ITIs show that the 25th procedure is the point of the learning rate plateau, there remains a 3% intersurgeon variability in the learning rate. We attribute this variability to the relative time course of the experience of the three surgeons. Surgeon 1 completed 75 procedures before surgeon 2, who in turn completed 75 procedures in advance of surgeon 3. It is possible that insights from surgeon 1’s experience influenced surgeon 2’s learning rate, and their combined experience influenced the learning rate of surgeon 3. Also, more of surgeon 1’s early cases were done with a less ‘robotically experienced’ operative team. Surgical density did not appear to be a factor in the learning rate difference, with surgeons 1 and 2 having higher surgical density (completing 75 cases in 2 years, vs 3 years for surgeon 3), but more prolonged learning curves. Surgeon age might have contributed to the observed differences in the learning rate, as surgeon 3 was the youngest.

In this study there was a statistically significant difference between LOS in the procedure intervals 1–25 and 26–75. We attribute this change in LOS to the surgeons’ changing expectations of patient recovery speeds rather than to actual differences in postoperative course.

Learning in the ITIs before and after engaging the robotic interface showed unexpected inconsistency, with no apparent learning rate plateau achieved in the first 75 procedures. This suggests that while the robotically engaged surgeon becomes more efficient at a reliable rate, systemic ‘team’ inefficiencies are improved less consistently. These peri-robotic inefficiencies might therefore serve as good targets for improving the overall procedure efficiency for both new and established robotic programmes.

It can be argued that TPT is an aggregate endpoint representing the cumulative learning of many smaller tasks required to complete RARP. Furthermore, it is likely that TPT is the aspect of the surgery least likely to affect the patient in the long term. Indeed, this study does not evaluate any cancer-specific or quality-of-life outcomes as measures of surgical proficiency. We recognize that a more detailed analysis of PM rates referenced to pathological stage, grade, tumour diameter, prostate volume, and nerve-sparing might have resulted in a different assessment of surgical learning. Furthermore, this study does not address biochemical recurrence rates as a measure of surgical proficiency which, from an oncological perspective, might be the most meaningful endpoint. With a longer follow-up we hope to address this issue.

Nevertheless, TPT remains an important measure of the acquisition of surgical skill to the surgeon, as well as hospital administrators responsible for the management of operating room costs and use. We showed a large variability in absolute operative time among the three surgeons in this study but have also shown a very consistent learning rate. If by the 25th procedure a surgeon is performing the surgery within 4 h, it is unlikely that a 2-h operative time will be achieved in the near term.

In conclusion, our experience using LCA shows promise as an objective measure of surgical learning and skill acquisition. Through our analysis of TPT we showed that on average a surgeons’ learning curve for RARP reaches a plateau by the 25th procedure.

While LCA is useful in determining proficiency in adapting to RARP, it can also be used for the evaluating the acquisition of surgical skill more generally. We hope that this study will serve as a model for others in providing an objective assessment of surgical learning. It is also our hope to elucidate what a surgical ‘learning curve’ represents. A ‘steeper’ learning curve actually represents faster learning and a quicker adaptation to the robotic interface. If a learning curve takes longer to overcome it might be better described as ‘prolonged’.

Because we evaluated experienced surgeons converting to RARP, our analysis is most applicable to those adopting RARP after completing residency training. Identification of the learning rate plateau has implications for predicting operating room use for physicians, practices and hospitals considering RARP. LCA might also have broader application for the assessment of surgical trainees in residency and fellowship. We also hope that hospitals and physician practices can benefit from this analytical technique when considering an investment in minimally invasive robotic equipment or future new technologies.

ACKNOWLEDGEMENTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS, SUBJECTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGEMENTS
  8. CONFLICT OF INTEREST
  9. REFERENCES

We thank Nananda F. Col, MD, MPP, MPH and the staff of Maine Medical Center’s, Center for Outcomes Research and Evaluation for their outstanding editorial support.

REFERENCES

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS, SUBJECTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGEMENTS
  8. CONFLICT OF INTEREST
  9. REFERENCES