The optimal cut‐off value in fit‐based colorectal cancer screening: An observational study

Abstract Background Colorectal cancer (CRC) screening programs using fecal immunochemical test (FIT) have to choose a cut‐off value to decide which citizens to recall for colonoscopy. The evidence on the optimal cut‐off value is sparse and based on studies with a low number of cancer cases. Methods This observational study used data from the Danish Colorectal Cancer Screening Database. Sensitivity and specificity were estimated for various cut‐off values based on a large number of cancers. Traditionally optimal cut‐off values are found by weighting sensitivity and specificity equally. As this might result in too many unnecessary colonoscopies we also provide optimal cut‐off values for different weighting of sensitivity and specificity/number of needed colonoscopies to detect one cancer. Results Weighting sensitivity and specificity equally gives an optimal cut‐off value of 45 ng Hb/ml. This, however, means making 24 colonoscopies to detect one cancer. Weighting sensitivity lower and for example, aiming at making about 16 colonoscopies to detect one cancer, gives an optimal cut‐off value of 125 ng Hb/ml. Conclusions The optimal cut‐off value in an FIT population‐based screening program is 45 ng Hb/ml, when as traditionally sensitivity and specificity are weighted equally. If, however, 24 colonoscopies needed to detect one cancer is too huge a burden on the health care system and the participants, 80, 125, 175, and 350 ng Hb/ml are optimal cut‐off values when only 19/16/14/10 colonoscopies are accepted to find one cancer.

test (FIT), which detects tiny amounts of human hemoglobin in a stool sample.
Screening should sort out apparently healthy persons who probably have a disease from those who probably do not. 2 In FIT screening the sorting is done by allocating participants with an FIT level above a certain cut-off value (positive screening test) to the group who probably have CRC, leaving those with an FIT level below a certain cut-off value (negative screening test) in the group who probably do not have CRC.
The cut-off value should be chosen so that the odds of having a positive screening test given that you have the disease (sensitivity) is as high as possible, while at the same time maximizing the odds of having a negative screening test given that you do not have the disease (specificity). Decreasing the cut-off value to get a higher sensitivity will though inevitably decrease specificity. The optimal cut-off value, therefore, depends upon how important an increase in sensitivity is weighted compared to the inevitable following decrease in specificity.
Six studies have previously reported sensitivity and specificity for various cut-off values. [3][4][5][6][7][8] However, all of these studies are based on a low number of CRCs and none of the studies are from population-based screening programs.
This register-based study aims at estimating sensitivity and specificity for various cut-off values in the prevalence round of a population-based FIT screening program with a large number of CRCs, and thereby provide optimal cut-off values for various weighting of sensitivity and specificity.

| The Danish CRC screening program
Population-based CRC screening with FIT started in Denmark in March 2014 for citizens aged 50-74 years. In order to gradually build up the needed capacity in hospitals, the first screening round took almost 4 years and ended in December 2017. All citizens aged 50-74 years at some point in 2014-2017 were invited to participate in the first screening round. 9 Citizens already enrolled in a surveillance program after a CRC or adenoma diagnosis, were advised in the invitation letter not to participate. Citizens with inflammatory bowel disease were advised to discuss with their treating physician whether participation was relevant. Returned FITs were analyzed using the OC Sensor Diana (2013-2017) (Eiken Chemical Company), yielding how much hemoglobin the sample contains. A test was considered positive if the stool-sample contained more than 100 ng hemoglobin (Hb)/ml (20 µg Hb/g feces as the sample bottle collects 10 mg feces and contains 2 ml buffer). Citizens with a positive FIT were informed by letter and referred to colonoscopy at a hospital-based screening endoscopy unit.

| Study design and population
Our study population consists of all participants in the Danish CRC screening program who submitted an analyzable stoolsample between 1 March 2014 and 31 December 2015. The study population was followed for CRC diagnoses for 2 years after the sample was analyzed.

| Data
Information on invitation date, date when returned sample was analyzed and the amount of blood in the sample was retrieved from the Danish CRC Screening Database (DCCSD), 10 a clinical quality database that was established to monitor the quality of the national Danish CRC screening program. FIT-values below 35 ng Hb/ml were only known as ≤35 ng Hb/ml and FIT-values above 1000 ng Hb/ml were only known as ≥1000 ng Hb/ml. DCCSD gets individual data on all invitations and participation from the administrative database IAM. 11 Information on CRCs, adenomas, and colonoscopies was obtained from the Danish National Pathology Data Bank and the Danish National Patient Register. Data were linked using the unique Danish personal registration number issued to all Danish residents. The completeness and reliability of the Danish National Pathology Data Bank and the Danish National Patient Register is deemed to be very high and good. 12,13

Lay Summary
Studies with relatively few cancers have estimated the optimal cut-off value in FIT-based CRC screening to be 50-150 ng Hb/ml when, as traditionally, weighting sensitivity, and specificity equally. It can be discussed whether it is acceptable to weight sensitivity and specificity equally as this results in many unnecessary colonoscopies to detect one cancer.
No studies have estimated optimal cut-off value without weighting sensitivity and specificity equally. This study used different weighting of sensitivity and specificity and found that given it is acceptable to make 24/19/16/14/10 colonoscopies to detect one cancer, the optimal cut-off value is 45, 80, 125, 175, and 350 ng Hb/ml.
We defined a participant as a citizen who sent in an analyzable stool-sample in the study period (1 March 2014 and 31 December 2015). A positive FIT is a FIT where the measured amount of blood is above the cut off level. We estimated the number of true positive/false positive FITs by the number of positive FITs where the participant is diagnosed/ not diagnosed with CRC 0-24 months after the test. Similarly the number of false negative/true negative FITs was estimated by the number of nonpositive FITs where the participant is diagnosed/not diagnosed with CRC 0-24 months after the test. We defined Sensitivity for the FIT test as the number of true positives divided by the number of true positives plus the number of false negatives. Specificity was defined as the number of true negatives divided by the number of true negatives plus the number of false positives.

| Analyses
In order to avoid that the number of cancers within each group of FIT values were too low, the span of the analyzed cut-off values was 5 ng Hb/ml for cut-off values between 35 and 75 ng Hb/ml; 10 ng Hb/ml for cut-off values between 80 and 100 ng Hb/ml; 25 ng Hb/ml for cut-off values between 100 and 500 ng Hb/ml and 100 ng Hb/ml for cut-off values between 500 and 1000 ng Hb/ml.
In the optimal study design all participants would have had a subsequent colonoscopy. This was, however, not the case in this observational study. As many of the cancers are at an early stage when discovered by screening it is not likely that all true positive cases with a cut-off value between 100 ng Hb/ml and 100 + x ng Hb/ml would have emerged as cancers in 2 years after the test, had the cutoff value been 100 + x ng Hb/ml. As we estimate number of false negative cases by number of negative test, which emerges as cancer in 2 years after the test, we did not know how many false negative cases there would have been, had the cut-off value been above 100 ng Hb/ml. We neither know how many true positive cases there would have been, had the cut-off value been below 100 ng Hb/ml, as these participants had no colonoscopy. There are 18.04 true positive cases per 1000 tests among tests with a cut-off value of 100-149 ng Hb/ml, whereas there are 6.35 false negative cases per 1000 tests among tests with a cut-off value of 80-99 ng Hb/ml. Close to 100 ng Hb/ml the ratio between number of true positive cases and number of false negative cases is, therefore, approximately 2.8 (18.04/6.35). In our main analysis, we assumed that the ratio between number of true positive cases and number of false negative cases was 2.8 for all cut-off values. The ratio between the number of true positive cases and the number of false negative cases might though be lower for high FIT values than for low FIT values. We, therefore, made two sensitivity analyses, one assuming the ratio to decrease linearly (dotted line, Figure 1) and one assuming the ratio to follow a hyperbola (dashed line, Figure 1).
A receiver operating characteristic curve (ROC curve) was used to show the trade-off between sensitivity and specificity. 14 The Youden index, that is, sensitivity + specificity, is a commonly used measure of overall diagnostic effectiveness. 15 Traditionally the optimal cut-off point is found by putting equal weight on sensitivity and specificity and thereby maximizing sensitivity + specificity (the Youden index). We not only calculated the optimal cut-off value using this method, but also recognized that it might not be appropriate in CRC screening to weight sensitivity and specificity equally. is a decision to make in each screening program. If the burden of unnecessary colonoscopies on the health care system and the participants should be weighted higher then specificity should be given more weight than sensitivity. We, therefore, also calculated the Youden index giving different weighting to specificity versus sensitivity.
To visualize how these different weighting of sensitivity versus specificity affects number of screen-detected CRC and number of needed colonoscopies, we for each optimal cut-off value also calculated percentage CRCs missed/ detected and percentage less/extra colonoscopies needed compared to a cut-off of 100 ng Hb/ml.
The 10-year average risk that an advanced adenoma transition to CRC has been reported to be 25-40%. 16 Although it is important to detect adenomas at CRC screening it is not equally important as finding CRCs. It is, therefore, not appropriate to include the number of screen-detected adenomas when calculating sensitivity and specificity. To show how the number of detected adenomas is affected by choice of cut-off value, we calculated percentage missed adenomas compared to a cut-off value of 100 ng Hb/ml for each optimal cut-off value.
Data were processed using SAS (version 9.4).

| RESULTS
In total 532,316 citizens accepted the invitation to CRC screening and sent in a stool sample before 31 December 2015. Of these 488 only sent in un-analyzable stool samples and were, therefore, excluded from the analyses (Figure 2). Among the remaining 531,828 eligible participants 35,582 had a FIT value above 100 ng Hb/ml and were, therefore, invited to colonoscopy. Of these 90.1% (32,057) had a colonoscopy within 6 months (Figure 2). At the current cut-off value of 100 ng Hb/ml, screening of 531,828 citizens resulted in 2086 screen detected cancers (true positive cases), 33,496 false positive cases, 495,785 true negative cases and 461 interval cancers (false negative cases). Table 1 shows that Increasing the cut-off value from 100 to 200 ng Hb/ml will decrease the sensitivity from 81.9% The ROC curve (Figure 3) shows that a cut-off value of 45 ng Hb/ml result in a point on the ROC curve closest to (0,1), that is, maximizes the Youden index (0.873 + 0.898 = 1.771). The present cut-off value of 100 ng Hb/ml results in a slightly lower Youden index (0.819 + 0.937 = 1.756)/further distance to (0,1). Lowering the cut-off value to 45 ng Hb/ml will, however, result in 24 colonoscopies needed to detect one cancer ( Table 2). If this is considered to be a too high burden on the health care system and the participants, sensitivity needs to be weighted lower than specificity. Table 2 shows the optimal cut-off value for different weighing of sensitivity and specificity. Weighting sensitivity by 40% and specificity by 60% (i.e., maximizing 0.4 × sensitivity+0.6 × specificity) and thereby accepting to make 19 colonoscopies to detect one cancer gives an optimal cut-off value of 80 ng Hb/ml. Using 80 ng Hb/ ml as cut-off value instead of 100 ng Hb/ml will detect 4% extra cancers at the cost of 13% extra colonoscopies. Weighting sensitivity by 35% and specificity by 65% and thereby accepting to make 16 colonoscopies to detect one cancer gives an optimal cut-off value of 125 ng Hb/ml. The effect of using 125 ng Hb/ml as cut-off value instead of 100 ng Hb/ml is 11% less colonoscopies needed as well as 8% less adenomas and 3% less cancers detected. As seen in Table 2, the two sensitivity analyses resulted in unchanged optimal cut-off values except when sensitivity is weighted by only 10-20%.

| DISCUSSION
The study provided optimal cut-off values for FIT-based CRC screening for various weighing of sensitivity versus specificity. Sensitivity analyses revealed that the provided optimal cut-off values were quite robust toward changes to the assumptions.

| Strengths and weaknesses
The major strengths of this study are the large number of participants and subsequent cancers as well as the use of high validity Danish registers to follow all participants on an individual level for 2 years.
We only included cancers diagnosed up to 24 months after the FIT to decide whether the test was true positive, false positive, false negative or true negative. This can have caused some misclassification, for example, citizens with a positive FIT and a CRC not found at colonoscopy nor within the next 2 years, will be falsely categorized as false positive although they are true positive. This number is though low, as number of interval cancers after a clean colon colonoscopy in the Danish CRC screening program is low. 17 As seen in our sensitivity analyses small changes to number of true/false positive and true/false negative cases only result in minor changes to the optimal cut-off values, wherefore the effect of these misclassifications is expected to be minimal.
As only participants with a cut-off value above 100 ng Hb/ ml had a subsequent colonoscopy, we had to estimate number of false negative cases had the cut-off value been above 100 ng Hb/ml and number of true positive cases had the cutoff value been below 100 ng Hb/ml. Our sensitivity analyses though showed that the results are quite robust toward major  7 Our results, when weighting sensitivity and specificity equally, are in line with results from all of these studies, suggesting that they are generalizable to other types of FIT. Brenner et al. calculated sensitivity and specificity both for detecting CRC and advanced neoplasms and found the same optimal cut-off value. 8 To show results based on real-life data from an organized CRC screening program, we also included citizens who chose not to have the recommended colonoscopy after a FIT value ≥100 ng Hb/ml. In the Danish CRC screening program about 90% had a colonoscopy after a positive FIT. Programs with a lower colonoscopy participation rate might experience "Number needed to scope to find one cancer" different from those reported here.
Different cut-off values males and females might be an appropriate consideration for future studies as, for example, the Danish CRC screening program has shown a higher positivity rate and increased detection of CRC among males as compared to females. 17,18

| CONCLUSION
The optimal cut-off value in an FIT population-based screening program is 45 ng Hb/ml when, as traditionally, sensitivity, and specificity are weighted equally. If, however, 24 colonoscopies needed to detect one cancer is too huge a burden on the health care system and the participants, 80, 125, 175, and 350 ng Hb/ml are optimal cut-off values when only 19/16/14/10 colonoscopies are accepted to find one cancer.

ETHICS STATEMENT
According to the Consolidation Act on Research Ethics Review of Health Research Projects, Consolidation Act number 1083 of 15 September 2017 section 14 (2) notification of questionnaire surveys or medical database research projects to the research ethics committee system is only required if the project involves human biological material. Therefore, this study may be conducted without an approval from the committees. According to EU's General Data Protection Regulation (article 30), the project was listed at the record of processing activities for research projects in Central Denmark Region (J. no: 1-16-02-68-20).