Evaluation of an immunochemical fecal occult blood test with automated reading in screening for colorectal cancer in a general average-risk population

Abstract

Colorectal cancer screening is a high public health priority in all industrialized countries. However, the low sensitivity of the common guaiac screening test (HemoccultII) makes practitioners and public health decision makers reluctant to set up a national screening program. In recent years, immunochemical tests based on the use of a specific antibody have been found to be more sensitive than the HemoccultII test. However, for screening purposes, any gain in sensitivity is of interest only if specificity and positive predictive value are satisfactory. Our aim was to assess the performance of an immunochemical test with an automated reading technique (Magstream 1000) for different hemoglobin content cut-off points. The study was carried out in the general population aged 50–74 years in the geographic area of Cotentin (Normandy, France). From 1 January 2001 to 31 December 2002, 7,421 one-time screening tests (Magstream) were administered by general practitioners and occupational physicians to patients at the end of regular consultations. Colonoscopy was proposed to the 434 people with a positive test. All cancers occurring in the study population between 1 January 2001 and 31 December 2003 were collected by general practitioners, gastroenterologists and the local registry. At the usual positivity threshold (20 ng hemoglobin/ml), screening sensitivity and specificity at 2 years of follow-up with 95% CIs were, respectively, 0.85 (0.72–0.98) and 0.94 (0.94–0.95). If the hemoglobin content cut-off point had been set at 50 ng/ml instead of the usual cut-off, positivity would have been 3.1% and positive predictive value for a cancer or a large adenoma would have been 0.49, with sensitivity of 0.68–0.83 and specificity of 0.97. Our results suggest that use of an immunochemical test with an automated reading technique could improve the prospects for mass-screening for colorectal cancer since it offers a promising alternative to guaiac tests. © 2005 Wiley-Liss, Inc.

Colorectal cancer is a major cause of death in all industrialized countries. In France, where more than 33,000 cases are reported annually,1 colorectal cancer screening is, as in the majority of industrialized countries, a high public health priority. The efficacy of the fecal occult blood test in screening for colorectal cancer has been established in 3 randomized controlled trials using the HemoccultII test, the most common guaiac test.2, 3, 4 Previously, a nonrandomized controlled trial suggested that biennial screening with the fecal occult blood test could lead to a 16% reduction in mortality after 11 years of follow-up in France.5 Despite these encouraging results, very few countries have carried out a biennial screening program with the HemoccultII test in large populations. The reluctance on the part of those in charge of public health programs is largely due to the poor sensitivity of the guaiac test, estimated to be approximately 50% in population-based studies,6, 7, 8 which limits the benefit of screening. This low sensitivity also partly accounts for the low adherence of practitioners to screening campaigns and, as a consequence, for the low participation rate of the target population in France. Immunochemical fecal occult blood tests are based on the use of a specific human hemoglobin antibody and do not require any specific diet restriction. In recent years, screening with immunochemical tests has been found to be more sensitive than with the Hemoccult test.9, 10 However, for screening purposes, any gain in sensitivity is of interest only if specificity and positive predictive value are satisfactory. A new automated reading technique for the immunologic test Magstream 1000 uses a quantitative outcome as the positivity cut-off point, making it possible to search for the optimal balance between sensitivity and specificity. A first evaluation of this promising technique has been published.11 However, this evaluation was conducted on individuals referred for colonoscopy, and its results were thus not applicable to the general average-risk population. Therefore, our aim was to assess the performance of screening with an immunochemical fecal occult blood test with an automated reading technique (Magstream 1000) for different hemoglobin content cut-off points in a general average-risk population based on preliminary results from a screening program carried out in Normandy (France).

Material and methods

A screening program was implemented in January 2001 using the Magstream test in the geographic area of Cotentin (Normandy, France). One hundred and sixty-four practitioners were supplied with test kits and invited to administer tests to patients aged 50–74 years at the end of their regular consultations. Patients were asked to obtain 2 fecal samples at home on 2 different days using the collection kit comprised of 2 sticks, 2 tubes and a plastic bag provided by the manufacturer (Fujirebio, Tokyo, Japan). Samples were then sent in a prestamped envelope to the central analysis center (Institut Régional pour la Santé). Directions for use were clearly indicated on an attached sheet. No specific dietary restriction was stipulated.

All tests (2 samples per person) were processed at the central laboratory with the Magstream 1000 device. Fecal occult blood is detected using immunologic indirect agglutination. Magnetic gelatin particles attached to antihuman hemoglobin placed in a magnetic field are used to quantify the level of hemoglobin with an optical reader. When a plate with 80 samples is tilted 60 degrees from the horizontal position, magnetic particles can slide down the slope of the well if they are free and thus form a measurable line. The higher the presence of human hemoglobin, the more the particles are prevented from sliding down and the shorter the line. The hemoglobin level is thus expressed as a quantitative outcome. According to the manufacturer's instructions, the test is considered positive when at least one of the samples contains at least 20 ng/ml hemoglobin, with a corresponding line measuring <3 mm.

The patient and practitioner are informed of the test result within 5 days. In case of a positive result, the patient is invited to consult the practitioner. During the consultation, a colonoscopy is prescribed.

This analysis was focused on all tests performed from 1 January 2001 to 31 December 2002 in the study population (n = 7,421 patients) and all cancers occurring in the study population between 1 January 2001 and 31 December 2003. Table I shows the distribution of age and sex in the study population. The test was positive for 434 people (5.8%). Although all patients with positive tests were invited by their practitioner to undergo a colonoscopy, only 366 (84.3%) complied. The 66 patients who did not undergo colonoscopy were excluded for estimation of positive predictive value.

Table I. Distribution of Age and Sex in the Study Population (n = 7,421)

	Age (years)
	50–54	55–59	60–64	65–69	70–74	Total
Males	697 (21.7%)	674 (21.1%)	642 (20.1%)	654 (20.5%)	530 (16.6%)	3,197
Females	854 (20.2%)	835 (19.8%)	831 (19.6%)	987 (23.4%)	717 (17.0%)	4,224

All cancers occurring in the study population, whatever the test result, were collected by general practitioners, gastroenterologists and pathologists who were routinely contacted (over a 6-month period) regarding occurrence and by the local cancer registry of the geographic area of la Manche, which forms part of the French network of cancer registries (FRANCIM). This follow-up, available for 2 years in most cases, enabled us to estimate sensitivity and specificity. The sensitivity for a given period was estimated by Se_p = a/(a + c), a being the number of cancers occurring within this period after a positive test (irrespective of whether a colonoscopy was performed or not) and c being the number of cancers occurring within this delay after a negative test. The specificity for the same delay was estimated by Sp_p = d/(d + b), d being the number of subjects free of cancer after a negative test and b being the number of subjects without cancer after a positive test (false-positive). Follow-up for the 6,987 patients with negative tests was at least 12 months. In this population, it was at least 18 months for 6,526 patients (93.4%) and at least 24 months for 5,597 (80.1%). After 12 months, cancers identified after a negative test were known only for patients with a sufficient follow-up period. Therefore, after 12 months, c and d were estimated by applying the incidence calculated as below to the total number of negative tests. As described in a previous study conducted in similar conditions,12 the incidence of cancers occurring among subjects with a negative test was estimated with the actuarial method.

Results

All patients with >20 ng hemoglobin/ml were invited to undergo a colonoscopy. Results of screening and the occurrence of cancers by applying this threshold are summarized in Figure 1. Of the 366 patients who underwent a colonoscopy after a positive test, 22 (6.0%) had a cancer, 102 (27.9%) had at least one adenoma ≥1 cm and 79 (21.6%) had an adenoma <1 cm. Of the 68 patients who did not undergo a colonoscopy after a positive test, 2 had a cancer within 2 months following the test.

Figure 1. Results of screening and occurrence of cancers (2-year follow-up) using the usual threshold positivity of 20 ng hemoglobin/ml.

Four cancers were diagnosed in patients with a negative test, with delays between test and cancer occurrence of 13, 16, 21 and 23 months. Table II shows the incidence of cancers after a negative test and the corresponding screening sensitivity at 12, 18 and 24 months after the test, sensitivity at 18 and 24 months being estimated as above. Considering the number of cancers occurring within this delay after a positive test (irrespective of whether a colonoscopy was done), screening sensitivity at 2 years was 0.85 [95% confidence interval (CI) 0.72–0.98] and specificity was 0.94 (95% CI 0.94–0.95).

Table II. Incidence of Colorectal Cancer in Subjects with a Negative Test According to Delay Since Test and Corresponding Screening Sensitivity

Delay sincetest (months)	Patients at riskduring period1	Number ofcancers observed	Cumulated incidence(per 100,000)	Number ofcancers calculated	Screening sensitivity atend of period (95% CI)
1 Patients at risk = patients at beginning of period − [(patients at beginning of period − patients at end of period)/2].
0–12	6,987	0	0	—	1
13–18	6,757	2	29.6	2.07	0.92 (0.82–1.0)
19–24	6,262	2	61.5	4.30	0.85 (0.72–0.98)

Table III shows the results of colonoscopy according to the hemoglobin content of the fecal sample. For each patient, the content reported was the higher of the 2 samples. The positive predictive value for cancer and large adenoma regularly and significantly increased with the hemoglobin content (χ² for trend, p < 0.001). In patients with a positive test and no colonoscopy, hemoglobin content was <50 ng/ml in 30, 50–74 in 14 and > 74 in 24. The 2 cancers occurred in individuals with a hemoglobin content of 53 and 67 ng/ml. Table IV presents positivity rate, positive predictive value, specificity and sensitivity for the usual hemoglobin content cut-off (20 ng/ml) and 2 alternative cut-off points: 50 and 75 ng/ml. By increasing cut-off points, the positivity rate was expected to be lower, sensitivity to be lower and specificity to be higher. For the 2 alternative cut-off points, sensitivity and specificity were both estimated by 2 values. One corresponds to the case where cancers not detected with a given hemoglobin content cut-off point and detected by colonoscopy at higher cut-off points had occurred within the 2 years following the test. The other corresponds to cases that had not occurred within this period. If the hemoglobin content cut-off point had been set at 50 ng/ml instead of the usual level, the positivity rate would have been 3.1% (227/7,421), i.e., almost 2-fold lower than with the usual cut-off; positive predictive value for a cancer or a large adenoma would have been almost 0.5 (93/189); and sensitivity would have ranged from 0.68 (19/28) to 0.83 (19/23), the corresponding specificity being 0.97 in both cases (7,190/7,398 or 7,185/7,393). Similarly, if the cut-off point had been 75 ng/ml, the positivity rate would have been 2.0% (148/7,421), i.e., almost 3-fold lower than with the usual cut-off; positive predictive value for a cancer or a large adenoma would have been 0.54 (67/124); and sensitivity would have ranged from 0.61 (17/28) to 0.81 (17/21), the corresponding specificity being 0.98 in both cases (7,269/7,400 or 7,262/7,393).

Table III. Colonoscopy Outcomes After a Positive Test According to Hemoglobin Content (n = 366)

Hemoglobincontent (ng/ml)	Cancer n (%)	Large adenoma1n (%)	Small adenoma1n (%)	Neither cancernor adenoma n (%)	Total n (%)
1 Larger or smaller than 1 cm.
20–29	1 (1.4)	11 (15.3)	21 (29.2)	39 (54.1)	72 (100)
30–39	2 (3.9)	8 (15.7)	13 (25.5)	28 (54.9)	51 (100)
40–49	2 (3.7)	7 (13.0)	13 (24.1)	32 (59.2)	54 (100)
50–74	1 (1.5)	25 (38.5)	14 (21.5)	25 (38.5)	65 (100)
75–99	12 (11.9)	40 (39.6)	17 (16.8)	32 (31.7)	101 (100)
≥100	4 (17.4)	11 (47.8)	1 (4.3)	7 (30.5)	23 (100)
Total	22 (6.0)	102 (27.9)	79 (21.6)	163 (44.5)	366 (100)

Table IV. Screening Sensitivity, Specificity and Positive Predictive Value and Positivity Rate According to Hemoglobin Content Cut-Off Point (One-Time Test)

Hemoglobin content (ng/ml)	Positivity rate1	Positive2 predictive value for cancer	Positive2 predictive value for large adenoma	Sensitivity1 at 2 years [(min − max) for alternative cut-off points]	Specificity1
1 Estimated including patients with positive test and no colonoscopy (2 additional cancers diagnosed within 2 months after positive test with hemoglobin content of 53 and 67 ng/ml). 2 Estimated excluding people with a positive test and no colonoscopy.
>20 (usual cut-off)	5.8%	0.06	0.28	0.85	0.94
>50 (alternative cut-off)	3.1%	0.09	0.40	(0.68–0.83)	0.97
>75 (alternative cut-off)	2.0%	0.13	0.41	(0.61–0.81)	0.98

Discussion

In agreement with an increasing number of reports, our study suggests that the use of an immunochemical fecal occult blood test could mean an improvement in screening for colorectal cancer in the future, particularly when used with automated reading. In France and several other countries, an effective operational national program for colorectal cancer screening is still far away despite evidence for the efficacy of fecal occult blood testing at reducing mortality. The reluctance on the part of those responsible for public health programs and of numerous practitioners is largely due to the poor sensitivity of the HemoccultII test. Rehydration of the HemoccultII test prior to processing can increase sensitivity, but the associated decrease in specificity and positive predictive value and the high positivity rate render its value in mass screening debatable. Annual testing would detect missed cancers, but the number of colonoscopies and the screening program cost would be much higher. Moreover, satisfactory compliance with an annual program could not possibly be achieved, particularly in France. Interest has thus increasingly grown in recent years in the use of immunochemical tests to screen for fecal occult blood. Young et al.9 emphasized the need for more studies assessing the performance of such tests.

All studies conducted in specific populations, high-risk or average -risk individuals, consistently show that the sensitivity of screening with immunochemical tests is higher than with guaiac tests.9, 10, 11, 13, 14 As in an Italian study,10 our study was conducted in an area covered by a cancer registry, making the collection of cancer data easier. In our study, the incidence of cancers after a negative immunochemical test was almost half that obtained after the HemoccultII test with very similar conditions and methodology.12 However, with the lack of an available control group, our study did not permit a direct comparison. Using the usual threshold of 20 ng/ml, screening sensitivity 2 years after the automated test was estimated at 0.85, an estimation very close to that found in the Italian population-based study10 and lower than that found in a Chinese study conducted in a population referred for colonoscopy.11

Despite the indisputable gain in sensitivity, the advantages of the lack of dietary and medication restriction and of the standardization of results, there are still 2 putative obstacles before using immunologic tests in large-scale populations: the relatively high positivity rate (approx. 6–7%) and revelation costs. Automated reading is likely to remove these obstacles. Regarding costs, we previously conducted a cost–effectiveness analysis, which suggested that use of an immunologic test with automated reading for 20 years of biennial screening in France instead of the HemoccultII test would correspond to an incremental cost–effectiveness ratio of 2,980 euros per year of lives saved.15 Concerning the high positivity rate, quantitative measurement of hemoglobin content permits establishment of a positivity cut-off point within the ideal balance between sensitivity and specificity. At the usual hemoglobin content cut-off of 20 ng/ml, the positive predictive value for cancer and for large adenoma was close to that obtained with the nonrehydrated HemoccultII test used in European studies2, 3 despite a relatively high positivity rate of 5.9%. However, screening specificity (0.94) was relatively low.

Not surprisingly and in accordance with the Italian study,16 which did not report sensitivity according to different hemoglobin content cut-off points, a higher cut-off was associated with a decrease in positivity rate and an increase in positive predictive value. Furthermore, in agreement with the Chinese study,11 a higher hemoglobin content cut-off point was associated with a decrease in sensitivity, allowing for differences in study population, as previously mentioned. The ideal balance between sensitivity and specificity is likely to vary from one country to another, depending on health-care organization. In France, where the HemoccultII test is used as a reference, our results suggest that a cut-off of 50 ng hemoglobin/ml could be associated with a gain in sensitivity and positive predictive value, with specificity and positivity rate very close to those obtained with the guaiac test.

The main limitation of our study is the lack of data on sensitivity and specificity of screening for adenomas. Although data collection on disease occurring after a negative test was possible due to the local cancer registry, these data were focused on cancer. Therefore, we were not able to estimate sensitivity and specificity of screening for adenomas. We could only estimate the positive predictive value for adenomas, which varied with the hemoglobin content cut-off point in the same manner as the positive predictive value for cancer. Since earlier detection and surgical removal of premalignant adenomas after a fecal occult blood test contributes to a gain in specific mortality,17 detailed studies are required to assess the potential gain in sensitivity for adenomas detected by immunologic tests. One other limitation was due to the low rate of colonoscopy acceptance among people with a positive test. This issue is regularly of major concern in colorectal cancer screening campaigns in France. Despite repetitive specific targeting of information, the rate of people refusing a colonoscopy in spite of a positive fecal occult blood test is regularly about 15%.5, 12

To be successful, any national screening strategy has to be agreed upon by all the participants involved, i.e., general practitioners, gastroenterologists, public health specialists and public health decision makers. Our findings, which are consistent with those in Chinese and Italian studies, present a promising alternative to guaiac tests, which have difficulty in gaining the approval of practitioners and individuals in charge of screening policy in France as in some other countries. Our results suggest that the use of an immunochemical test with automated reading could improve the prospects of mass screening for colorectal cancer.

Acknowledgements

The authors thank all general practitioners, gastroenterologists and pathologists involved in the study.