Validation of a novel robotic-assisted partial nephrectomy surgical training model

Authors


Andrew J. Hung, USC Institute of Urology, 1441 Eastlake Ave, Suite 7416, Los Angeles, CA 90089, USA. e-mail: ajhung@gmail.com

Abstract

Study Type – Therapy (case series)

Level of Evidence 4

What's known on the subject? and What does the study add?

One area of particular growth for robotic surgery has been partial nephrectomy. Despite a perceived notion that robotic-assisted partial nephrectomy is more easily adaptable compared to laparoscopic partial nephrectomy, there is nonetheless an associated learning curve.

Validated training models with a corresponding assessment method for robotic-assisted partial nephrectomy were previously unavailable. We have designed and validated a RAPN surgical model appropriate for resident and fellow training.

OBJECTIVE

  • • To evaluate the face, content and construct validities of a novel ex vivo surgical training model for robotic-assisted partial nephrectomy (RAPN).

METHODS

  • • We prospectively identified participants as novice (not completed any robotic console cases), intermediate (at least one robotic console case but <100 cases), and expert (≥100 robotic console cases). Each participant performed a partial nephrectomy using the da Vinci Si Surgical System on an ex vivo porcine kidney with an embedded Styrofoam ball that mimics a renal tumour. Subjects completed a post-study questionnaire assessing training model realism and utility. Participants were anonymously judged by three expert reviewers using a validated laparoscopic assessment tool. Performance between groups was compared using the tukey–kramer test.

RESULTS

  • • The 46 participants recruited for this study included 24 novices, nine intermediates, and 13 experts. Overall, expert surgeons rated the training model as ‘very realistic’ (median visual analogue score 7/10) (face validity). Experts also rated the model as an ‘extremely useful’ training tool for residents (median 9/10) and fellows (9/10) (content validity), although less so for experienced robotic surgeons (5/10). Experts outscored novices on overall performance (P= 0.0002) as well as individual metrics, including ‘depth perception,’‘bimanual dexterity,’‘efficiency,’‘tissue handling,’‘autonomy,’‘precision,’ and ‘instrument and camera awareness’ (P < 0.05) (construct validity). Experts similarly outperformed intermediates in most metrics (P < 0.05).

CONCLUSION

  • • Our novel ex vivo RAPN surgical model has demonstrated face, content and construct validity. Future development of this model should include simulation of haemostasis management and renal reconstruction.
Abbreviations
LPN

laparoscopic partial nephretomy

NSS

nephron sparing surgery

RAPN

robot assisted partial nephrectomy

WIT

warm ischaemia time.

INTRODUCTION

Nephron sparing surgery (NSS) is becoming the standard of care for the majority of T1a renal tumours. An increasing body of evidence suggests that decreasing renal function may have a significant adverse effect on cardiovascular morbidity, non-cardiovascular morbidity and overall longevity [1–4]. Laparoscopic partial nephrectomy (LPN) has been demonstrated to have equivalent oncologic and functional outcomes compared with open partial nephrectomy, while the minimally invasive approach additionally offers the benefit of lower blood loss and reduced hospital stay [5]. LPN is technically challenging and hence limited to centres with expertise in the laparoscopic arena. The technical difficulties associated with LPN have also contributed to an increased use of radical nephrectomy for tumours amenable to NSS, adversely impacting long-term renal function in these patients.

With the advent of the da Vinci Surgical System (Intuitive Surgical, Sunnyvale, CA) for urological procedures, one area of particular growth has been robotic-assisted partial nephrectomy (RAPN). Robotic surgery may allow an urologist without extensive laparoscopic experience a shorter learning curve to performing partial nephrectomy compared with LPNs [6].

Several training models have been developed for LPN [7,8], but none has yet been developed for RAPN. One recent study found that the learning curve of RAPN for an experienced robotic surgeon is 20 to 30 cases to reach consistently a warm ischaemia time (WIT) <20 min and acceptable overall perioperative outcomes [9].

Herein, we present a validation study for a surgical training model for RAPN, to our knowledge, the first of its kind, assessing its realism (face validity), usefulness as a training model in the eyes of experts (content validity) and ability to differentiate levels of surgical experience (construct validity).

MATERIALS AND METHODS

The validation study was performed in a prospective manner after obtaining Institutional Review Board approval. Participants were categorized as robotic novice (completed no robotic console cases), robotic intermediate (at least one robotic console case but <100 console cases) or robotic expert (≥100 robotic cases (any procedure); threshold defined a priori). Participants first completed a pre-study self-reported questionnaire on demographics and surgical experience. Each participant then reviewed a brief orientation video demonstrating the task of the training model.

Our novel partial nephrectomy model consisted of an ex vivo porcine kidney where a 1.5 inch (∼3.8 cm) diameter Styrofoam ball (Dow Chemical Company, USA) was embedded in the parenchyma of the kidney to mimic an exophytic renal tumour. The porcine kidneys were purchased from a commercial vendor (Fig. 1A). A melon scooper (Oxo, Chambersburg, Pennsylvania, USA) with a 1 inch (∼2.5 cm) diameter was utilized to score the renal capsule at the upper or lower pole of the kidney (Fig. 1B). A 15-blade scalpel was then used to complete sharply the defect, with particular care not to enter the collecting system (Fig. 1C). Super glue (Gorilla, Cincinnati, Ohio, USA) was applied to the complete surface of the divot, and the foam ball was affixed to the divot (Fig. 1D). Manual pressure was applied uniformly in all directions on the renal tissue around the foam ball for 5 min to allow complete fixation (Fig. 1E). Cost per porcine kidney with Styrofoam tumour model was ∼$15. Construction time per kidney averaged 7 min.

Figure 1.

RAPN training model: materials and construction process. A) equipment used in model concstruction, B) a melon scooper was used to score the renal capsule, C) a 15-blade scalpel was used to complete the defect, D) superglue was applied to the divot, E) manual pressure was applied to affix the foam ball and F) excision of foam tumour.

The model task consisted of excising the Styrofoam tumour while maintaining a clear margin of renal parenchyma. (Figs 1F and 2). Available robotic instruments included Prograsp forceps and curved scissors. Cautery and a fourth robotic arm option were not provided. This initial exercise consisted of tumour excision only and renal reconstruction was not included.

Figure 2.

RAPN training model: excision of tumour with clear margin.

After completion of the exercise, expert participants filled out a post study questionnaire to assess realism of the training model (face validity) and its utility as a training tool (content validity) based on a 1 to 10 visual analogue scale (1 being worst and 10 being best). All performances were recorded on DVD through the da Vinci's video output, and later scored in a blinded fashion by three expert robotic surgeons (M.M.D, M.A., C.N.), each having completed over 300 robotic console cases. In addition to the expert scoring, objective parameters such as time to task completion, number of robotic instrument collisions, tumour margin status and closest tumour margin if margin was negative (measured with specimen bi-valved) were prospectively recorded for each participant. Tumour margin was considered negative if no Styrofoam tumour was grossly visible on the resection undersurface of the specimen.

Performance scoring was based on Global Operative Assessment of Laparoscopic Skills (GOALS), a validated assessment tool for laparoscopic surgery [10]. These metrics included ‘depth perception,’‘bimanual dexterity,’‘efficiency,’‘tissue handling,’ and participant ‘autonomy’ to complete said task. Additionally, ‘instrument and camera awareness’ (ability to keep instruments in view and move camera effectively to field of interest) and ‘precision’ of instrument action were two novel metrics designed by our group to assess specifically robotic surgery performance and utilized to evaluate the study participants. Each scorer could award a maximum of 5 points for each of the seven metrics (maximum 35 points). Performance metrics of experts, intermediates and novices were compared (construct validity) using the Tukey–Kramer test for multiple comparisons. Spearman's analysis assessed correlation between validated GOALS metrics and novel metrics. Significance was considered at 0.05 and all P values were two-sided.

RESULTS

A total of 46 participants were recruited for this study, including 24 novices, nine intermediates, and 13 experts (Table 1). The novice cohort consisted of urology residents and surgeons with a median 4 years (range 1–40 years) of overall surgical experience, limited robotic experience (median 0 years (range 0–2 years), and who had not completed any robotic console case. The intermediate group consisted of urology residents, fellows and surgeons with a median surgical experience of 9 years (range 4–35 years) and who had completed at least one robotic console case with a median of nine robotic console cases performed (range 1–61). The expert cohort had performed a median of 500 robotic cases (range 100–1500), including 300 (90–1100) robotic prostatectomies and 100 (0–250) RAPNs.

Table 1.  Participant demographics and surgical experience (median [range])
 ExpertIntermediateNovice P value
  1. RARP, robotic-assisted radical prostatectomy; RAPN, robotic-assisted partial nephrectomy.

n 13924 
Age, years42 (36–52)39 (30–69)32 (28–65)<0.001
Surgical experience, years19 (13–25)9 (5–35)4 (1–40)<0.001
Laparoscopic experience, years11 (3–22)4 (1–8)2 (0–13)<0.001
Robotic experience     
 Years6 (3–10)2 (1–7)0 (0–2)<0.001
 Total console cases500 (100–1500)9 (1–61)0<0.001
 Console cases–RARP300 (90–1100)8 (1–60)0<0.001
 Console cases–RAPN100 (0–250)0 (0–4)0<0.001

FACE AND CONTENT VALIDITY

Overall, expert surgeons rated the training model as ‘very realistic’ (median score 7/10 range [6–9]). Expert surgeons also rated the model as an ‘extremely useful’ training tool for residents (9/10 [7.5–10]) and fellows (9/10 [7–10]), although less so for experienced robotic surgeons (5/10 [3–10]).

CONSTRUCT VALIDITY

Experts completed the RAPN training task in significantly less time compared to intermediates and novices (P < 0.008, P= 0.01, respectively) (Table 2). However, no statistical difference was observed between the three groups in number of robotic instrument collisions, positive margin rate or distance of closest margin (P > 0.05).

Table 2.  Construct validity: performance (median [range]) on RAPN training model among groups
 ExpertIntermediateNovice P value (Expert vs. novice) P value (Expert vs. intermediate) P value (Intermediate vs. novice)
n101224   
Time, s300 (187–450)608 (314–1457)555 (130–900)0.010.0080.2
Collisions, number0 (0–3)1 (0–5)0 (0–8)0.80.60.9
Positive margin, %3/10 (30%)4/12 (33%)12/24 (50%)0.30.30.3
Closest margin, mm1 (1–3)2 (1–2)3 (1–6)0.30.30.4
Overall score (35 points) 30 (2034) 24 (1832) 23 (1729)0.0020.0031.0
 Depth perception4.3 (2.7–4.7)3.3 (2.0–4.5)3.3 (2.0–4.0)<0.0040.0051.0
 Bimanual dexterity4.0 (3.0–4.7)3.3 (2.7–4.5)3.2 (1.7–4.3)<0.0020.0090.9
 Efficiency4.0 (3.0–4.7)3.0 (2.0–4.0)3.0 (1.7–4.0)<0.001<0.0021.0
 Tissue handling4.0 (2.7–5.0)3.0 (2.3–3.7)2.7 (2.0–4.3)<0.0010.0031.0
 Autonomy4.7 (3.7–5.0)4.0 (3.3–5.0)4.0 (3.3–5.0)0.020.040.9
 Precision4.0 (2.7–4.7)3.3 (2.7–4.3)3.0 (1.7–4.3)0.020.40.6
 Instrument and camera awareness4.7 (3.7–5.0)3.7 (2.7–4.5)3.7 (3.0–4.7)0.003<0.0090.4
Self-reported difficulty level 3.5 (1–8)6 (2–8)7 (5–9)<0.0010.0030.5

Expert participants outscored both intermediates (P= 0.003) and novices (P < 0.002) in the overall score as evaluated independently by expert judges. In terms of individual metrics, experts outperformed novices in ‘depth perception’ (P < 0.004, ‘bimanual dexterity’ (P < 0.002), ‘efficiency’ (P < 0.001), ‘tissue handling’ (P < 0.001), ‘autonomy’ (P= 0.02), ‘precision’ (P= 0.02), and ‘instrument and camera awareness’ (P= 0.003).

Similarly experts outscored intermediates on ‘depth perception’ (P= 0.005), ‘bimanual dexterity’ (P= 0.009), ‘efficiency’ (P < 0.002), ‘tissue handling’ (P= 0.003), ‘autonomy’ (P= 0.04), and ‘instrument and camera awareness’ (P < 0.009). Only with ‘precision’ did the expert judges not detect a difference between expert and intermediate groups (P= 0.4).

There were no significant differences between the intermediate and novice cohorts in overall score as well as individual metrics (P > 0.05).

Paralleling performance outcomes, expert participants rated the exercise as significantly easier (median 3.5/10 [range 1–8]) than intermediates (6/10 (2–8) (P= 0.003) or novices (7/10 (5–9) (P < 0.001). There was no difference in self-reported difficulty level between novice and intermediate groups (P > 0.05).

VALIDATION OF NOVEL METRICS

The novel metrics, ‘precision’ and ‘instrument and camera awareness,’ highly correlated with scores from the validated GOALS assessment tool (r =0.8 and 0.9, respectively [P < 0.001]).

DISCUSSION

RAPN may present urologists with a more easily adaptable minimally invasive approach to performing NSS compared with LPN. The initial development of this training model for RAPN allows a trainee to achieve the experience of resecting a renal mass without compromising the oncological success of a live case. Gaining appreciation of the three-dimensional aspect of tumour resection is critical for the success of maintaining a negative margin while maximally preserving healthy surrounding parenchyma. Additionally, handling renal parenchyma delicately to avoid trauma, while at the same time providing adequate traction required for tumour resection, is a critical component of the procedure. As we have demonstrated, all the above aspects of RAPN can be practiced on a model that involves a tumour or comparable surrogate in kidney tissue.

Expert participants of this study rated the training model highly for both its realism (face validity) and utility as a training tool for both residents and fellows (content validity). However, expert participants conferred upon it a more limited training role for expert surgeons, probably due to the lack of reconstruction and haemostasis management that are also critical aspects of RAPN. Our novel model lends itself to further development including varying locations of the tumour (upper pole, hilar etc), depth of penetration as well as including reconstruction. The renal artery vein and collecting system can easily be connected to a pulsatile pressurized irrigation system where adequacy of haemostasis and collecting system repair can also be practiced and evaluated. Our institution is currently refining the present model to include renorrhaphy and haemostasis control.

Although our novel model was relatively inexpensive and easy to create, we did find some drawbacks. The Styroform adhesiveness to the tissue is short-lived, and if the procedure is not performed within 30 min of adherence, it has a tendency to peel off. The angles of the working robotic instruments to the kidney may not simulate accurately the clinical scenario. Lastly, there is no availability of retraction by an assistant.

With incorporation of robotic-assisted surgery into our surgical practices, it is paramount that the training paradigm shifts accordingly. This is particularly relevant for procedures such as partial nephrectomy where the procedure relies not just on accuracy but speed. Even while some show that experienced surgeons can adopt RAPN quickly [9], there is nonetheless a learning curve that must be addressed. This study validates one training model for surgeons to practice and assimilate with the da Vinci controls and manoeuvrability as well as practice a key aspect of RAPN without setting foot in the operating room.

Presently, there is no validated curriculum for robotic training or consensus of the process through which robotic surgeons should be accredited [11]. While several surgical simulator platforms have been introduced to alleviate the robotic learning curve and have been validated in the literature [12–17], none can yet simulate realistic anatomy and organic tissue handling that tissue models like the present one can offer. Live animal models offer a realistic perspective for a given procedure but the associated costs may be prohibitive While ex vivo tissue models still require a dedicated robot in a wet-lab, they at least obviate the need for live animals and yet allow exposure to a validated step-wise practice of a procedure such as partial nephrectomy.

Equally important to a feasible training model is the evaluation method. The GOALS assessment tool has been validated and widely utilized in laparoscopic training [10,18–20], and it has been adopted by our institution for the assessment of robotic trainees using ex vivo tissue models including the one presented in this study [21]. We adopted GOALS for this robotic training model and further developed two novel metrics we believe are relevant in robotic surgery: ‘awareness of instrument and camera’ and ‘precision’. We have shown in virtual reality simulation that robotic surgeons are more economical and precise with their robotic instrument movements and ‘clutch’ their console master controllers more often [17]. The ability to demonstrate differing levels of performance on this surgical model between robotic experts and other participants (construct validity) confirms for the first time the translatability of the validated GOALS metrics from laparoscopic to robotic surgery. We found that expert scoring for both novel metrics correlated very strongly with the validated GOALS metrics.

In conclusion, validated training models with a corresponding assessment method for RAPN were previously unavailable. We have designed a RAPN surgical model specific for tumour resection appropriate for resident and fellow training that demonstrates face, content and construct validities utilizing a validated scoring tool along with novel assessment metrics. Future development of this model should simulate the management of haemostasis and renal reconstruction.

ACKNOWLEDGEMENT

We would like to thank Drs Erik Castle, Michael Stifelman, Ashok Hemal, Surena Matin, James Porter and Steven Shichman for their insightful feedback after hands-on evaluation of the training model. Jie Cai performed the statistical analyses for this study. Intuitive Surgical provided the da Vinci Surgical System for this study. The funding for this study was institutional.

CONFLICT OF INTEREST

None declared.

Ancillary