Artificial Intelligence‐Supported Video Analysis as a Means to Assess the Impact of DROP‐IN Image Guidance on Robotic Surgeons: Radioguided Sentinel Lymph Node versus PSMA‐Targeted Prostate Cancer Surgery

The introduction of the tethered DROP‐IN gamma probe has enabled targeted robot‐assisted radioguided prostate cancer (PCa) resection of pelvic sentinel lymph nodes (SLNs) and prostate‐specific membrane antigen (PSMA)‐positive lesions. While both procedures use 99mTc‐isotopes, the two vary in signal and background intensity. To understand how the different levels of image guidance impact surgical decision‐making, computer‐vision algorithms are used to extract the DROP‐IN probe kinematic form clinical videos. 44 PCa patients undergo SLN (25) and PSMA‐targeted (19) resections. PSMA‐PET/CT and SPECT/CT create preoperative roadmaps, and intraoperative probe signal intensities are recorded. Using neural network‐based software, probe trajectories are extracted from videos to extract multiparametric kinematics and generate decision‐making and dexterity scores. PSMA‐targeted resections yield significantly lower nodal signal intensities in preoperative SPECT‐CT scans (three‐fold; p = 0.01), intraoperative probe readouts (eight‐fold; p < 0.001), and signal‐to‐background ratios (SBR; two‐fold; p < 0.001). Kinematics assessment reveal that the challenges encounter during PSMA‐targeted procedures converted to longer target identification times and increase in probe pick‐ups (both five‐fold; p < 0.001). This results in a fourfold reduction in the decision‐making score (p < 0.001). Reduced signal intensities and intraoperative SBR values negatively affect the impact that image‐guided surgery strategies have on the surgical decision‐making process.


Introduction
For decades, radioguided surgery has provided the global standard in image-guided surgery, initially for sentinel lymph node (SLN) procedures in prostate cancer (PCa) [1] and more recently for prostate specific membrane antigen (PSMA) receptortargeting procedures. [2]As laparoscopic gamma probes poorly support robotic resections, a robotic-tailored DROP-IN gamma probe [3] has been introduced to facilitate robotic surgery radioguidance procedures.The DROP-IN gamma probe technology utilizes the maneuverability of the steerable robotic instruments and provides the operating surgeon with autonomy with regard to detector use.
The DROP-IN technology has improved image-guided robotic SLN, [4] and PSMA surgery. [5]While both procedures make use of the same detector and gamma-emitting 99m Tc radioisotopes, there are distinct differences related to the tracer, its target structures, the route of administration and The introduction of the tethered DROP-IN gamma probe has enabled targeted robot-assisted radioguided prostate cancer (PCa) resection of pelvic sentinel lymph nodes (SLNs) and prostate-specific membrane antigen (PSMA)-positive lesions.While both procedures use 99m Tc-isotopes, the two vary in signal and background intensity.To understand how the different levels of image guidance impact surgical decision-making, computer-vision algorithms are used to extract the DROP-IN probe kinematic form clinical videos.44 PCa patients undergo SLN (25) and PSMA-targeted (19) resections.PSMA-PET/CT and SPECT/CT create preoperative roadmaps, and intraoperative probe signal intensities are recorded.Using neural network-based software, probe trajectories are extracted from videos to extract multiparametric kinematics and generate decision-making and dexterity scores.PSMA-targeted resections yield significantly lower nodal signal intensities in preoperative SPECT-CT scans (three-fold; p = 0.01), intraoperative probe readouts (eight-fold; p < 0.001), and signal-to-background ratios (SBR; two-fold; p < 0.001).Kinematics assessment reveal that the challenges encounter during PSMA-targeted procedures converted to longer target identification times and increase in probe pick-ups (both five-fold; p < 0.001).This results in a fourfold reduction in the decision-making score (p < 0.001).Reduced signal intensities and intraoperative SBR values negatively affect the impact that image-guided surgery strategies have on the surgical decision-making process.
resulting pharmacokinetics, as well as the oncological and anatomical aspects of the approaches.The SLN technology can be used to identify nodal micrometastases in PSMA-PET/CT N0M0 patients [6] and relies on colloidal nanoparticles, for example, indocyanine green (ICG)-99m Tc-nanocolloid that, following deposition in or around the primary tumor, is transported to the SLN(s). [7]The amount of tracer dispersing into the nodes is generally relatively high (a 2 h postinjection of the median percentage of the injected dose (%ID) in the SN is 0.7 %ID (IQR 0.18-1.457c] In contrast, PSMA-receptor targeting surgery is achieved following intravenous injection of 99m Tc-labeled small-molecule inhibitors (e.g., 99m Tc-PSMA I&S), which can bind to PSMA-expressing PCa cells that are connected to the vascular system (nodal accumulation is roughly around 0.002 %ID at 5-6 h postinjection). [8]Hereby the expression of PSMA receptors and the often low tumor volume are limiting factors for the signal accumulation, even taking internalization into account. [9]Both approaches suffer from background signals originating from different organs, prostate and liver versus urine, blood vessels and intestines, for SLN and PSMA, respectively. [10]onsidering the above, an open question that remains is as follows.To what extent does the chosen tracer impact the surgeon's ability to identify the target?A question that can only be answered using performance assessments.Today's surgical performance assessments, as implemented during for example robotic training, tend to rely on error-based scoring via timeconsuming expert assessments. [11]Since this approach is not scalable and is not ideal for the evaluation of new technologies, there is a need for automated means to gather high-end surgical performance data.Thereby creating a need for automated means to gather high-end surgical performance data.For example, using mechanical instrument tracking during robot-assisted radical prostatectomies, Hung et al. drew parallels between the surgeons instrument kinematics and surgical outcomes. [12]Ideally, the use of kinematic measures extends to the evaluation of emerging image guidance strategies.Unfortunately, as of now access to mechanical instrument tracking recorders is still limited, making it difficult to scale the approach.An alternative route to extract kinematic data is to explore instrument tracking in the video data output that is inherent to minimally invasive robotic surgeries.To this end, marker-based video tracking of da Vinci instruments has been exploited in the form of patterned- [13] and fluorescent markers. [14]14b,16a,19] To make video-based instrument tracking accessible for a global audience of robotic surgeons, markerless detection algorithms are needed that can automatically extract the required data out of endoscopic videostreams. [20]20b] Despite practical challenges, deep-learning algorithms, due to their capability in feature extraction and expression, are showing potential in such surgical indications. [21]e hypothesize that through AI-supported video analysis of state-of-the-art surgical procedures, a direct connection can be made between innovative design features and the actions of the surgeon.More specifically, we retrospectively assessed key performance indices during two different procedures, that is, DROP-IN gamma-probe-guided, robot-assisted nodal dissections in PCa patients that were in either targeting SLNs or PSMA-positive lesions.We trained a neural network algorithm that allowed automated digitization of the DROP-IN gamma probe trajectory from endoscopic videos.The extraction of multiparametric kinematic indices from these tracks allowed us to set up performance assessments that could be used to objectively score the impact that the tracer had on the surgical decision-making (Figure 1).

Patient Inclusion and Characteristics
In total, 44 patients with PCa underwent robot-assisted radioguided surgery procedures between 2018 and 2021.SLN in primary prostate cancer patients were included in the trial nr.NL57838.031.16 (n = 25) [4a] and salvage PSMA-targeted dissections in recurrent prostate cancer patients were included in the trial nr.5a] Both studies were approved by the local ethical committee.After written informed consent was obtained from all patients, data analysis and collection were completed.

Preoperative Imaging
In the SLN group (primary surgery setting), patients were injected with ICG-99m Tc-nanocolloid (intraprostatic injection; %200MBq; 5-6 h prior to surgery).Following preoperative lymphoscintigraphy, a 3D SPECT/CT was performed 2 h after tracer injection and was used to provide a roadmap for surgery.The patients in the PSMA-group (salvage surgery setting) received a 68 Ga-PSMA-11 or 18 F-DCFPyl 3D PSMA PET/CT within 60 d before salvage surgery according to local protocol. [6,22]This image was used as a primary roadmap.Salvage surgery patients had a maximum of three soft-tissue lesions visible on preoperative PSMA PET/CT and received an intravenous injection of 99m Tc-PSMA I&S (%550MBq; 22-26 h prior to surgery).At %17 h after injection, preoperative SPECT/CT was performed to achieve a secondary surgical road map.
The location and number of suspect nodes were evaluated by trained nuclear medicine physicians.The total signal intensity of the lesions at preoperative SPECT images was calculated using MIM (MIM Software Inc., OHIO, USA) in both groups.To this end, the lesion location specified by nuclear medicine physicians were delineated by thresholding or using PET Edge (a semi-automatic delineation tool running on MIM). [23]The total SPECT counts (sum of the counts) in the target lesion were considered signal intensity.For the PSMA-group the maximum standardized uptake value (SUVmax) was calculated on PSMA PET images using Osirix (OsiriX Imaging Software, Pixmeo, Bernex, Switzerland).

Intraoperative Imaging
In all surgeries, the imaging roadmaps were used for macroscopic guidance and nodal identification was performed using a DROP-IN gamma probe prototype (Eurorad S.A, Strasbourg, France).Surgeries were performed at a single tertiary care referral center through a six-port transperitoneal approach using a da Vinci Si, X or Xi system (Intuitive Surgical Inc., Sunnyvale, US).
The DROP-IN probe was inserted into the abdominal cavity either through or next to a 12-mm trocar within the Alexis port (Applied Medical Corp., Rancho Santa Margarita, US).The operating surgeon was able to autonomously grasp and maneuver the DROP-IN probe using the da Vinci surgical console and the ProGrasp Forceps.SLN and PSMA-targeted node were traced using the acoustic and numeric gamma probe feedback.In vivo radioactive signal-to-background ratios (SBRs) were recorded.To this end, the maximum probe counts before final resection of the lesion (SLN or PSMA-targeted node) were considered as signal value and the average counts on the direct surrounding tissue (<2 cm) were considered as background value.In the SLN group, the fluorescence component of the tracer (i.e., ICG) was also used to complement DROP-IN radioguidance with fluorescence imaging.The output of one "eye" of the firefly endoscope was recorded to document the surgical procedure in 2D.In all cases, the radioactive signal intensities were also measured ex vivo and histopathological evaluation was performed.

Automated DROP-IN Gamma Probe Tracking
Images preprocessing was performed using the Albumentations library. [24]The dataset contained images with two resolutions (1920 Â 1080 or 1280 Â 720) that were resized with h = w = 299 pixel and used as input.Data augmentation included random horizontal (50%) and vertical (50%) flip, brightness, and contrast change (20%).A 34-layer residual networks (Resnet) architecture [25] was trained and employed to segment out the DROP-IN probe tip location in each individual video frame (Figure 2).As the endoscopic recordings were in 2D, only xand ymovements could be defined.The implementation was realized using Python (3.8.5) and PyTorch libraries.The output from the instrument detection model contained the predicted bounding box and the x and y coordinates of the tip location.The Adam optimizer was used for training with an initial learning rate of 0.0001, reducing by 0.95% every epoch and training batch size of 8. Mean squared error (squared L2 norm) was used as a loss and test set (577 images from 2 surgeries).For validation of the network performance, 100 randomly selected images from two different surgeries were annotated also for probe diameter.Using the known diameter of the gamma probe (12 mm) and tip location, the distance between predicted tip location and ground truth was converted from pixels to millimeter.The mean squared error (squared L2 norm) of the result was then calculated.

Extraction Kinematic Metrics
Registered coordinates of the DROP-IN Probe tip allowed us to reconstruct its movement trajectory over time.In line with our previous works, [9,14b,16a,26] the digitized track allowed us to use custom algorithms (MATLAB, the MathWorks Inc., MA, USA) to extract kinematic metrics such as straightness index, angular dispersion, and adjusted speed.To eliminate the effect of camera location and zooming, the adjusted speed was considered as percentage of position change in each time point.The mean values of metrics/lesion were calculated.The number of probe pick-ups and total search time for each lesion were manually extracted from the recorded endoscopic videos as well.

Scoring Surgical Dexterity and Decision-Making
16a,19] The overall dexterity (Dx) was represented by the normalized jerkiness over the time and traveled pathlength (Equation ( 1)), wherein a low dexterity index indicated more smooth instrument movements. [27]The decision making (DM) index was calculated by a correlation between the intentional movements: extreme peaks in jerkiness, the number of probe pickup, searching time, and straightness index.A low DM index indicates a more focused procedure.The weight factors (i.e., wf 1 , wf 2 , wf 3, and wf 4 ) were determined as described in other studies [16a,19] Dx ¼

Statistics
Descriptive statistics were reported as the median and interquartile range (IQR) or the frequency and proportion according to established guidelines. [28]Statistical significance between the metrics was established via a two-sampled t-test or Mann-Whitney test, using a confidence interval of 95%.Pearson correlation coefficient was used to establish correlations between metrics.All statistical analyses were performed using SPSS statistics (IMB Corp., New York, USA).

Patient Characteristics
Patient information for the SLN and PSMA-groups is provided in Here, all trajectories were evaluated on a per lesion basis.

Target Identification
In the SLN group, the preoperative SPECT/CT roadmaps revealed 97 target lesions (average 3.8 per patient; Figure 2).In this group, MIM indicated that the median estimated tracer uptake was 0.19 %ID per lesion converting to an uptake of 0.188 MBq.In the PSMA group, the diagnostic PET/CT roadmaps indicated 21 PSMA-avid lesions (average 1.1 per patient).
Owing to the inferior spatial resolution of 99m Tc-PSMA SPECT/CT, this secondary imaging approach only supported the visual identification of 13 target lesions (62%) (average 0.7 per patient).Here, the SPECT/CT images indicated that even 17 h postinjection background signals can be seen in, among others, bowel, bladder, and urine (see Figure 2C). [29]The median estimated SPECT tracer uptake in PSMA-avid lesions was 0.01 % ID converting to 0.0055 MBq, which is significantly lower than that in the SLN group (p < 0.001).MIM showed that signal counts in SPECT were three times higher in the SLN group as compared to the PSMA groups, 51 942 versus 16 885 counts, respectively (p = 0.01).Surgically, out of 92 SLNs identified on SPECT, 47 SLNs were pursued using the DROP-IN probe, for example, nodes outside the ePLND field.Nodes located in difficult anatomical locations were left in situ.In the PSMA-group 19/21 of the PSMA-avid lesions could be successfully identified intraoperatively.One lesion could not be pursued due to extensive intestinal adhesions, and one small LN (2 mm on PSMA PET/CT) located in the pararectal fat could not be detected due to high background signal in the rectum.During SLN surgery, the median DROP-IN gamma count rates (in vivo) were about 7 times higher than those found during PSMA-targeted surgery (i.e., median 1040 vs. 137 counts s À1 , respectively; p < 0.001).As result of the rotational freedom of the DROP-IN probe, the SLNs could be detected without much interreference of background signals.In contrast, the PSMA-targeted surgery suffered from background signals that originated from the bowel, blood, vessels, bladder, and urine.This combination of relatively low signal intensity and relatively high background converted to a two-fold reduction in SBR for the PSMA-targeted procedures compared to the SLN procedures (i.e., median 1.8 vs. 3.8, respectively; p < 0.001).
To further investigate the impact of preoperative imaging, the signal intensities identified in preoperative images were related to the intraoperative counts recorded (see Figure S1, Supporting Information).For the SLN group, there was a correlation between SPECT signal intensities and intraoperative signal counts (p = 0.003) (Figure S1A, Supporting Information).The SUVmax calculated on PSMA PET/CT for targeted lesions again revealed a correlation (p = 0.013) (Figure S1B, Supporting Information).For the PSMA SPECT intensities (Figure S1C, Supporting Information), the correlation was only a visible trend not statistically significant, being that for these images preoperative signal intensities were generally low.
Pathology identified 9 tumor bearing lesions in the SLN group and 21 tumor bearing lesions in the PSMA group.This converted to a sensitivity and specificity of 100% for the SLN group and 86% sensitivity and 100% specificity for the PSMA group.

Assessment of Surgical Performance
By applying our tracking algorithm, we were able to digitize the path traveled by the DROP-IN gamma probe in individual procedures with a 7 mm accuracy (regression-based mean squared error) (Figure 3 and 4).These digital tracks allowed us to extract a variety of kinematic metrics to set up DM and Dx performance  scores.A Pearson correlation coefficient analysis revealed that the in vivo SBR and signal intensity was negatively correlated with time and number of probe pickup as well as DM and Dx index (Figure 5).Positive correlations with SBR and signal intensity, in which the two variables move in tandem, have been found for straightness index and angular dispersion.The SPECT signal shows a weak negative correlation with Dx and positive correlation with straightness index.Correspondingly, the search time was extended by nearly five-fold in the PSMA group (75 sec per SLN vs. 368 sec per PSMA lesion, p < 0.001; Figure 6A).In addition, for PSMA-lesions, five probe pickups were necessary before the surgeon was able to localize the target lesion (1 vs. 5, p < 0.001; Figure 6D).The angular dispersion was higher for SLN procedures (0.62 vs. 0.56, p < 0.001).
Overall, the extracted kinematic metrics converted into a fourfold higher decision-making index (8.43AU SLN vs. 33.12AU PSMA; p < 0.001) and 280-fold higher dexterity index for the PSMA group (0.05 AU SLN vs. 14.02AU PSMA; p < 0.001).Lower DM and Dx numbers are considered to indicate better performance.In line with Pearson correlation, these values were highly correlated to SBR values, whereby an SBR < 2 value resulted in inferior DM and Dx values (Figure 6B,E). [19]Relating the preoperative SPECT signal to DM and Dx revealed a similar but less strong trend (Figure 6C,F).The findings were not influenced by differences in the pelvic distribution of the lesions (Figure 2D,E) but prior surgery in the salvage cases could have negatively affected the patient anatomy.

Discussion
We have been able to use automated computer-vision strategies to elucidate the practical impact of robotic image-guided surgery strategies.By extracting kinematic metrics from raw endoscopic video footage, we have been able to come up with quantitative and objective performance measures.Specifically, this yielded a more profound understanding of the interaction between man and machine, namely, the procedural relation between the signal intensity and SBR value on the DROP-IN gamma probe kinematics.
Preoperative SPECT/CT was highly efficient in SLN identification (100%), but was only able to visualize 62% of the PSMApositive lesions.The ability of the preoperative SPECT/CT scans to act as a preoperative surgical roadmap was obvious for SLNs, but questionable for PSMA-positive lesions.It seems that the PSMA-SPECT/CT scan merely help assess whether 99m Tc-PSMA I&S shows aberrant tracer uptake compared to the PSMA-PET/ CT. [30] Nevertheless, involved surgeons indicated for both SLNand PSMA-targeted procedures that the preoperative SPECT/CT scan was considered indicative for a successful procedure.7c] While we found a significant correlation between SPECT signal intensities and in vivo signal counts for SLN procedures, we could not establish a direct correlation between preoperative SPECT imaging and intraoperative signal intensities in the PSMA-targeted procedures (see Figure S1, Supporting Information).Based on this data it is, however, highly likely that larger data sets, which are less influenced by outliers, will show a correlation with the SPECT signal.In spite of the small dataset, the SUVmax in PSMA-PET/CT was already positively correlated with in vivo counts.Moreover, while requiring more extensive data collection and processing, it seems well worth the effort to further investigate to what extent preoperative findings can guide patient selection and can predict surgical performance.Such insights could ultimately help ensure that image-guided surgery procedures are only executed when the surgeon, and ultimately also the patient, are likely to benefit from them.
Regarding the retrospective analysis of recorded surgical procedures, the use of marker-less video-based DROP-IN probe tracking (x-and y-axis) has been crucial to set up the presented performance-based comparison.This allowed us to identify kinematic metrics that relate to the type of the utilized tracer.We found that the intraoperative SBR provide the most critical metric.The SBR is related to the tracer pharmacokinetics and was predictive for both DM and Dx (see Figure 6B,E).SBR values below 2 proved to be detrimental, as this directly converted to a five-fold increase in time of probe use and the number of probe pickups.Here, it is important to note that these findings also reflect the personal experience of the surgeons.The SLN group benefitted from complementary fluorescence guidance (hybrid SLN tracer) and the PSMA group underwent (extensive) previous treatment, features that could have influenced both DM and Dx.Furthermore, all involved surgeons were considered SLN experts but had limited experience in performing the newer and more challenging PSMA procedures.
By focusing on instrument kinematics, we have been able to realize a short-term means to assess how a technology reflects on surgical performance.20a,21a,33] Ultimately, such data should help identify best practices that could in turn be used to establish certified training programs.Initial data from Hung et al. suggest kinematic data also complements and even relates to traditional outcome measures such as positive surgical margin, urinary continence recovery, and hospital length of stay. [12,34]his correlation is an area we are determined to investigate in future studies.

Conclusion
We found that lower signal intensities and intraoperative SBR values directly negatively impacted the practical utility of image guidance strategies.Hereby the intraoperative measurements directly related to kinematic metrics and surgical decision-making.

Figure 1 .
Figure 1.Schematic overview of the study setup.Surgeries are performed based on preoperative roadmaps.Surgical instruments are tracked and digitized kinematic features extracted.These features are then used to determine DM and Dx scores in relation to SBRs.

Figure 2 .
Figure 2. Preoperative imaging.Sentinel lymph node and higher echelon node identified on preoperative SLN SPECT imaging A).An example of a metastatic lesion identified on preoperative PSMA PET and PSMA SPECT imaging in a patient that underwent PSMA-RGS B,C).Locations of identified lesion used for kinematic comparison of SLN D) and PSMA-targeted E) procedures.(Co = common; Ex = external; In = internal; L = left; Ob = obturator; P rec = pararectal; P sac = presacral; P ves = paravesical; R = right; SN = sentinel node.).

Figure 3 .
Figure 3. Overview of the tracking of DROP-IN gamma probe using an AI network in laparoscopic view.The output is the bounding box and the tip location of the DROP-IN gamma probe providing the basis for the movement trajectories during the tumor-targeted procedures.

Figure 4 .
Figure 4. Digitized DROP-IN probe trajectory; x-y location of the tracked DROP-IN gamma probe overlaid on surgical videos and plotted over the time in SLN procedure A,C) and PSMA-targeted surgery B,D) respectively.The tracks are color-coded based on the probe pickups during the surgery.

Figure 5 .
Figure 5. Feature analysis.Cluster analysis of Pearson correlation applied on the kinemetric metrics extracted from the traveled paths by DROP-IN gamma probe (# Pickup: number of probe pickup per lesion, Time: search time per lesion, AD: angular dispersion, ST: straightness index, and speed value during search for each lesion), performance scores (DM: decision making, Dx: dexterity), and signal feature (SPECT signal: total counts in SPECT lesion, SBR: intraoperative signal-to-background ratio).The color bar indicates the correlation strength between the features (red positive and blue negative correlation).

Figure 6 .
Figure 6.Kinematic and performance metrics.Comparing search time and number of probe pickups per lesion between SLN and PSMA-targeted lesions A,D).The decision-making and dexterity index plotted against intraoperative SBR values B,E).The decision-making and dexterity index plotted against total SPECT signal intensities corrected for radioactive decay C,F) for each lesion.

Table 1 .
The target identification rate defined using preoperative imaging was based on 25 SLN and 19 PSMA patients.SPECT signal intensities were calculated in 15 SLN patients (23 lesions) and 17 PSMA patients (17 lesions).In some of the procedures, the quality of video recordings was not sufficient for DROP-IN probe tracking, which led to comparing probe trajectories of 23 SLN lesions (in 15 patients) with 12 PSMA lesions (in 12 patients).