Study of capsule endoscopy delivery at scale through enhanced artificial intelligence-enabled analysis (the CESCAIL study)

Aim: Lower gastrointestinal (GI) diagnostics have been facing relentless capacity con - straints for many years, even before the COVID-19 era. Restrictions from the COVID pandemic have resulted in a significant backlog in lower GI diagnostics


S TRENG TH S AND LIMITATI ON S OF THE S TU DY
A limitation of this study is the focus on artificial intelligence (AI) polyp detection only.The detection rate of other colonic conditions such as inflammatory bowel disease, diverticulosis, angiodysplasia and infection were not included in this study.In the future, detection of these other significant colonic diseases will be included in the AI system to prevent further unnecessary resources being spent on follow-up colonoscopy for subsequent diagnostic confirmation.
If the results show a comparable accuracy and better efficiency for AI-enabled analysis, this will help address the service backlog through more efficient allocation of senior clinicians' time, rapid review and treatment initiation (if indicated), and a reduction in patient anxiety by reducing the overall pathway time.Along with the minimally invasive nature of colon capsule endoscopy compared with optical colonoscopy, this will have a profound impact on lower gastrointestinal diagnostics and improve the overall standard of care for our patients.

INTRODUC TI ON
Colorectal cancer is the third most common cancer and fourth most common cause of cancer-related death in the United Kingdom [1].
Early diagnosis has a significant positive impact on long-term survival rates.Diagnostic services have been under enormous capacity constraints to meet an ever-rising demand; the average growth in colonoscopy diagnostic activity was 5.3% per annum from 2014 to 2019.Between 2017 and 2019 (the pre-COVID period) there was also a marked increase in the number of patients waiting more than 6 weeks for a diagnostic test [2].
Amid the first wave of the COVID pandemic there was a dramatic reduction of 90% in the diagnostic service capacity, along with a 97%-99% reduction in bowel cancer screening.This consequently led to a 58% reduction in cancer detection per week on the national endoscopy database [3] and created a significant backlog in an already strained service.Delayed diagnosis can substantially diminish 10-year survival in patients with colorectal cancer [1].For instance, a 6-month delay could cause a reduction of more than 29% in survival [4,5].
To address the current backlog, following an evaluation of colon capsule endoscopy (CCE) in NHS Scotland [6], NHS England implemented a pilot project in 2021 utilizing CCE to investigate those with suspected colorectal cancer of a low-risk category.The risk was stratified by low levels of blood in the stool measured by a faecal immunochemical test (FIT).This project was further extended to include postpolypectomy surveillance in 2022 due to the continuing backlog pressure.

Capsule endoscopy technology has been available in small
bowel investigation for over a decade and has been demonstrated as minimally invasive and safe.As imaging quality and battery technology continue to mature, the shift of this novel technology toward colonic application has become possible.A colon capsule is an imaging device with at least two cameras, one on either end, to allow direct antegrade and retrograde vision of the colon.These wide-angle cameras (172°, providing a near 360° view) are programmed to operate for up to 10-12 h, recording videos of the colonic mucosal wall [7].A recent meta-analysis demonstrated that CCE has a sensitivity and specificity of 87% and 95%, respectively, for the detection of polyps > 10 mm and 86% and 88%, respectively, for polyps >6 mm, compared with the gold standard colonoscopy [8].The study concluded that CCE is safe and accurate for colonic polyp and cancer screening.As a 12-h video containing up to 400 000 images often takes 50-70 min to read and generate a report, the time-consuming analysis remains burdensome and inefficient compared with other standard investigation modalities such as optical colonoscopy or CT virtual colonography.There is an additional risk of overlooking lesions due to reader fatigue [9].
From the perspective of improving efficiency, artificial intelligence (AI) has shown great popularity and promise in both diagnostic and therapeutic endoscopy in recent years.In addition to AI-supported solutions for optical colonoscopy using real-time signalling of endoscopic findings while performing an endoscopic procedure [10], AI has also been used to identify a range of different small bowel conditions including bleeding lesions, inflammatory bowel disease and coeliac disease on capsule endoscopy.These advances became possible due to the development of convolutional neural networks (CNNs) [11].A recent study using AI-assisted analysis claimed to reduce reading times from 60 to 8 min and improve sensitivity from 75% to 99% compared with conventional clinician interpretation in small bowel capsule video analysis.The application of AI improves both the performance and accuracy of small bowel capsule endoscopy [12].A systematic review also confirmed a pooled sensitivity and specificity for ulcer detection of 95% and 94%, respectively, while the pooled sensitivity and specificity for bleeding were 98% and 99%, respectively [13].A similar result was also demonstrated recently in a different Japanese study led by Aoki et al. that also suggested a reduction in reading time from 12 to 3 min in the expert's review and 20 to 5 min in the trainee's analysis [14].
The use of AI in CCE has been motivated after the reported success of AI applications in small bowel capsule endoscopy.A particular focus in recent years has been the application of AI in polyp detection.A relatively small retrospective study showed a sensitivity and specificity of a CNN in mucosal lesion and blood detection of 96% and 98%, respectively, with reading times reduced to 13 min [15].These impressive results led to the proposal and design of this large-scale multicentre study.
The aim of this study is to determine the feasibility, accuracy and productivity of a CNN based AI-enabled analysis tool (AiSPEED™) for polyp detection compared with the 'gold standard' conventional clinician analysis care pathway.

Patient and public involvement
Prior to this study, extensive work with public and patient stakeholders has been conducted by AGEM CSU as part of an Innovate UK-funded project AID-GI which, led by CHI, developed the first version of AiSPEED™.Through a mixed-methods qualitative and quantitative study to understand patient and clinician attitudes about implementing CCE and AI-enabled reporting, we know that 75% of patients select it as their preferred delivery method if it is as effective as colonoscopy, with 77% indicating confidence in the use of AI-enabled reporting [16].
Input from patients has been strengthened by further patient involvement through a patient and public research advisory group, that reviewed and provided feedback on this study's proposal.
We have further engaged with clinicians nationwide as well as relevant charities, who have demonstrated enthusiasm for CCE and AI-enabled reporting and have stated their support for this study.
In addition, the study documents have been reviewed by our patient and public involvement (PPI) co-applicant and other PPI representatives, who will also be involved at all stages of the research, providing important input into the study, giving the public a voice in decisions and ensuring that dissemination activities can be delivered effectively.

Eligibility criteria
Participants must be aged 18 years or above, be able to provide informed consent and meet NHS England criteria, including symptomatic patients with a negative FIT or FIT between 10 and 100 μg/mL and the addition of postpolypectomy surveillance following the change in the NHS England criteria in 2021, for undergoing CCE as part of their standard care [2,3].
Prospectively consented participants will be identified from the colorectal referral and postpolypectomy surveillance lists.They will be approached via the phone before the patient information packs are sent out a few days before their allocated CCE procedure date [17,18].They will be consented to in person, over the phone or by electronic consent via email before undergoing CCE following their bowel preparation as per the local trust care pathway.Pillcam Colon 2 capsules (Medtronic) will be used, with the captured video images stored in a Pillcam recorder (DR3) and transferred to the standard reporting software (RAPID) provided by Medtronic.The colon capsule video of each patient will be made available on two different pathways: (1) the AI-enabled video analysis and (2) the conventional clinician analysis pathway.
For retrospectively consented participants, their previous CCE videos will be available from the conventional clinician analysis pathway.Therefore, the videos will be directly used in the AI-enabled video analysis pathway after obtaining the retrospective consent, which is done in person, over the phone or by electronic consent via email.Depending on the local site approvals, for example Caldicott Guardian approval, retrospective direct consent from the participant may or may not be required.The retrospective AI-enabled video analysis pathway is the same as the prospective cohort described below.The report from the retrospective AI-enabled analysis pathway will be compared with the report previously generated by the clinician when the CCE was first carried out.
In the AI-enabled video analysis pathway, the video will be deidentified using the RAPID software and securely transferred to CHI for the AI-enabled prereading processing, which consists of a trained nurse prereader using AiSPEED™ and documenting results in RAPID.The results of this analysis will be downloaded back onto the NHS computer system for a CCE-trained clinician to validate and generate the final report.Figure 1 illustrates the data flow in the AI-enabled analysis pathway between the NHS and CHI systems.As the AiSPEED prereading service is provided by CHI to support the AI-enabled video analysis pathway, all the relevant AiSPEED components are the intellectual property of CHI.
The standard clinician analysis pathway consists of a clinician's direct analysis of the video and reporting in RAPID.The final diagnosis is based solely on the standard clinician analysis, ensuring that patient care is entirely independent of the AI-enabled analysis pathway.The participant will be notified of their final diagnosis from the standard clinician's report.Each video will be interpreted by two different clinicians covering the two pathways.This avoids potential experimenter bias that might occur when the same clinician reviews a video more than once in both pathways.With the awareness of high interobserver variation in CCE reading, the focus of this study is mainly on the two pathways rather than the readers.The reports from each pathway are then compared for accuracy.The productivity will only be compared

Outcomes
The primary objective is to determine if the AI prereading workflow, which includes the AI-enabled video analysis tool (AiSPEED) operated by a trained nurse prereader for analysis and interpretation of the video, is comparable (noninferior) to the workflow of conventional clinician analysis as an alternative method of reading the CCE.The performance of each pathway is measured using sensitivity, specificity and accuracy, by comparing it with the conventional clinical analysis.A comprehensive statistical analysis plan will be created before recruitment ends, with the aim of providing a detailed description of the intended analysis plan.
The secondary objectives include assessing the accuracy of detecting polyps stratified by FIT and the size of the polyps as well as comparing the time taken for AI-enabled reading against conventional clinician reading.However, the time taken for conventional clinician video analysis and the comparison in productivity might not be available in some of the retrospective subpopulations if these data were not recorded or available during the original conventional clinician video analysis.
A predetermined change control plan for AiSPEED™ has already been developed to govern the introduction of improvements into production.Throughout the clinical trial, no automatic or continuous change of the neural network that underpins the AI system will occur.
If the accuracy is comparable to the clinician enable analysis and the processing timelier for the AI system, this will not only help address the service backlog caused by the COVID pandemic but allow relocation of clinicians' time to therapeutic procedures or an improvement in service provision and patient care.

Sample size
The sample size is based on the accurate (true) or inaccurate (false) detection of polyps in the AI-enabled CCE analysis, where we define the gold standard CCE clinician assessment as the true detection of polyps.To show that the AI-enabled CCE analysis is at least as accurate as the current gold standard CCE reporting, we power the study based on an equivalence or noninferiority analysis with a set noninferiority limit of 7.5%; a difference greater than this would be significant in practice.
We assume that both AI-enabled and conventional CCE reporting have a minimum accuracy of 80% in polyp detection.Under this assumption, at a 5% level of significance, 90% power and 7.5% noninferiority limit, a sample size of 597 participants will be required.
With an expected 11% loss to follow up, we plan to recruit 674 participants in total.

Recruitment
Patients referred to secondary care with lower gastrointestinal symptoms will be reviewed and assessed.Patients who have opted for CCE as per their local site's standard pathway or have already had their CCE based on the NHS England CCE criteria will be invited to enter the CESCAIL study.These patients will be screened and identified based on the study eligibility criteria prior to being consented and enrolled prospectively into the study at their colon capsule appointment.Depending on the local site approvals, for example Caldicott Guardian Approval, retrospective consent may or may not be required after screening and identifying these patients retrospectively with the same criteria.

Data collection
Demographic details of the participants, date of referral and CCE, past medical history, medications and biochemical and haematological parameters (including FIT results) will be collected as part of the initial assessment.After the participants have undergone the CCE procedure, the time spent analysing, time for breaks and pauses, quality of bowel preparation, number of polyps, classification of polyps, number of colorectal cancers and all other data will be inputted onto the electronic data capture (EDC) system.These data will be collected from the reports in both the AI-enabled and conventional pathways separately.

Data management and storage
All the data will be stored on the online-validated, good clinical practice (GCP) compliant, EDC system, that has log-in access limited to CESCAIL study team members.In addition screening and recruitment logs of all participants approached to take part and enrolled in the study will be held at each site and stored on password-protected NHS computers.Data entered onto the online EDC database will be pseudonymized using each participant's unique study ID.CCE footage uploaded for AI-enabled prereading will also be pseudonymous throughout the pathway.The process of pseudonymity will be either achieved by de-identifying any admissible patient details through the Medtronic RAPID software or at the point of registration and the data are securely maintained on a master list of de-identifiers.This ensures that no patient identifiable data will be stored with the capsule videos.
Accurate records of all participating patients, all original signed consent forms and all copies of case report form pages will be secured in a site file.Paper forms with identifiable information will be stored in secure, locked filing cabinets.Personal data collected during the trial will be in accordance with General Data Protection Regulations (GDPR).The handling of data will be clearly documented in the obtained participant information sheet and consent.The final trial dataset will only be made available to those who require it for final analysis.The uploaded anonymous videos will be kept and used for AI system improvement by CHI after the trial.

Statistical outcomes
The study population will be described by summaries of participant age, sex, body mass index and other reported study outcomes including polyp detection rate and time taken for CCE reporting; means and standard deviations will be used for continuous variables and proportions for dichotomous and categorical measures.Participant flow through the study will be tabulated and presented graphically.
In the context of possible selection bias, the key characteristics of the study population will be compared with the expected characteristics available in the published literature.
A difference in baseline data between the prospectively and retrospectively consented subpopulations is not anticipated.However, it may be that for some unexpected reasons the populations do differ.Therefore, summary statistics will be calculated for both subpopulations in addition to the full population and tests undertaken (i.e.t-tests and chi-square tests for continuous and categorical data, respectively) to identify if any or all the baseline data differ significantly between the methods of consent.

Primary analysis
The primary analysis will compare the diagnostic accuracy of the AI-enabled video analysis alongside the gold standard analysis by trained clinicians for the detection of colorectal polyps.The accuracy of the AI-enabled polyp video analysis will be quantified by the sensitivity (true positive rate) and specificity (true negative rate).
The overall test performance will be measured by the area under the receiver operating characteristic (ROC) curve, utilizing bootstrapping for the estimation of confidence intervals.
The primary analysis will utilize the full study population.
However, the full analysis will also be reported for the prospectively and retrospectively consented subpopulations.Results from the subpopulations are not anticipated to be different.However, if the baseline data differ in any important characteristic, the sensitivities and specificities could also be different between the subpopulations.All analyses will be undertaken in R [19] using the pROC package [20].

Secondary analysis
All secondary diagnostic outcomes will be analysed and reported in a similar manner to the primary outcomes, where data are available.
A subgroup analysis will assess the accuracy of the test treatment within groups, defined by the FIT result and the size of the identified polyps.
Secondary analyses will also be performed for time taken for CCE reporting, utilizing a regression model with time taken as the response variable (after logarithmic transformation) and the CCE method as the explanatory variable, after adjusting for baseline age and sex.

Interim analysis
The study has been designed into two distinctive stages (stage 1 and stage 2).Stage 1 will be used to test the study and data capture processes, potentially leading to some modification of the AI algorithm prior to the more formal testing of the performance in stage 2.
In stage 1, 118 retrospective participants will be recruited through a purposive, yet arbitrary, sampling approach based on the study timelines and anticipated recruitment rates at the selected centre for the study, without additional convoluted participant selection criteria.After data collection has been completed on these participants, an 'interim' analysis will be undertaken using the methodology described here for the definitive study analysis.Due to the small sample size, and likely lack of precision in parameter estimates, stage 1 data will not be used to make decisions about whether to stop or continue the study into the main out-of-sample testing phase in stage 2.
Stage 1 data may be used to initiate the device certification.To maintain the integrity of the trial, all stage 1 analyses will remain confidential within the study team and will not be disseminated more widely until after all stage 2 participants have been recruited.

Missing data
Missing data are not expected to be a problem in this study.The primary and secondary analyses will be based on complete case

14631318, 0 , 3 LEI
Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/codi.16575 by Test, Wiley Online Library on [25/05/2023].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License | et al.ME THOD Participants, interventions and outcomes Study design and setting The Capsule Endoscopy delivery at Scale through enhanced AI anaLysis (CESCAIL) study is a combined prospective and retrospective multicentre diagnostic accuracy study sponsored by University Hospitals Coventry and Warwickshire and funded by the National Institute for Health and Care Research (NIHR) AI in Health and Care Award, with collaborators CorporateHealth International UK Limited (CHI), NHS Highland, NHS Arden & GEM Commissioning Support Unit (AGEM CSU) and the University of Barcelona.CESCAIL compares the diagnostic accuracy and productivity of AI-enabled video analysis with that of video analysis by trained clinicians.It uses only centres conducting CCE in patients as part of their standard care pathway, with more than one trained CCE clinician at each centre.Participants are recruited retrospectively and prospectively when referred routinely for CCE (e.g.postpolypectomy surveillance), or urgently with lower gastrointestinal symptoms as part of their standard care pathway.

14631318, 0 ,
Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/codi.16575 by Test, Wiley Online Library on [25/05/2023].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License in the prospective cohort.Figure 2 shows a flow diagram of the participant timeline throughout the study.

5 LEI
More importantly, this could potentially reduce the cost of the CCE procedure by decreasing senior clinicians' reporting times.This will undoubtedly contribute to training inexperienced CCE readers in the future by F I G U R E 1 Data flow between National Health Service (NHS) and CorporateHealth International UK Limited (CHI) systems.14631318, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/codi.16575 by Test, Wiley Online Library on [25/05/2023].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License | et al.
identifying and revealing the abnormalities to the trainee as part of the pattern recognition learning.

F I G U R E 2
Flow diagram of participant timeline (AI, artificial intelligence; CCE, colon capsule endoscopy; GI, gastrointestinal).

-
Standard Care Study specific activity -Additional research related activity