Referral population studies underestimate differences between human papillomavirus assays in primary cervical screening

We studied how representative cytologically abnormal women (“referral populations”) are with respect to uncovering differences between human papillomavirus (HPV) assays in the primary screening where most women are cytologically normal.


| INTRODUCTION
Human papillomavirus (HPV) testing is going to replace cytology in primary cervical screening. Randomised trials comparing the two testing modalities in screening showed that women with negative HPV test results have a lower risk of developing cervical cancer than women with normal cytology. 1 Cervical screening laboratories can now choose between more than 100 commercially available HPV assays. The randomised trials were, however, undertaken either with one commercially available HPV assay, Hybrid Capture 2 (HC2) or with an in-house polymerase chain reaction (PCR) assay using GP5+/6+ primers. Other, newer, HPV assays will most likely not be submitted to similarly rigorous but costly and time-consuming randomised trials. Instead, their accuracy of detecting high-grade cervical intraepithelial neoplasia (CIN), a treatable screening endpoint, has been compared to that of HC2 or GP5+/6+ in several smaller, predominantly split-sample, studies. [2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20] The caveat is that split-sample testing literature is dominated by studies of referral populations. 8,9,[11][12][13][14][15][16][17][18][19][20] As referral populations typically include women with abnormal cervical cytology who have a high risk of ≥CIN2, such studies have several practical advantages over primary screening studies. The high risk of ≥CIN2 means that adequately powered referral population studies can be smaller than adequately powered primary screening studies where the prevalence of ≥CIN2 is much lower. Furthermore, women with abnormal cytology are routinely offered a follow-up, so CIN lesions detected through abnormal cytology can be traced in routine pathology registrations. In contrast, follow-up of cytology-normal/HPV-positive women at present necessitates ethical approval and an additional research infrastructure.
In primary screening, most HPV-positive women have normal cytology, and the main reason for replacing cytology with HPV testing is to detect also the CIN missed by cytology (ie, the cytologically normal high-grade CIN). Referral populations, therefore, represent only selected subgroups of women undergoing primary screening, and it remains unknown whether they are an adequate substitute for primary screening populations when HPV assays are being compared.
To shed light on the use of referral populations in studies evaluating HPV assays for primary screening, we compared HPV infections and ≥CIN2 detected by four commercially available HPV assays in women with normal vs abnormal cytology.

| Study design
We used data from the Horizon study, the design of which was described in detail previously. 2  high-risk genotypes plus genotype 66 in combination, with a signal threshold of ≥0.5 signal to a cut-off (S/CO) value. In these three assays, low-risk genotypes return a negative test result (except in cases of cross-reactivity, described in detail previously). 28 Among the four studied assays, CLART was the only assay that allowed individual detection of 35 HPV genotypes (including all 13 defined as high-risk by IARC) and was used in the analysis to discriminate between single and multiple infections. Within the context of primary cervical screening, we considered samples to have a positive CLART test result if the assay detected one or more of the 13 high-risk HPV genotypes. All other infections detected by CLART were considered to represent negative, non-actionable, screening findings. In the analysis, these infections could only contribute to multiple infections in the presence of at least one high-risk genotype.
Women with abnormal cytology were managed according to routine Danish recommendations (Appendix). Women with cytology-normal/HPV-positive test results on one or more HPV assays were invited, for study purposes, for repeated cytology and HPV testing in 18 months. All colposcopies were undertaken under routine conditions either by a hospital or privately practicing gynaecologists. In Denmark, it is recommended to take directed biopsies from all suspicious areas after application of acetic acid, and a random biopsy from all quadrants if lesions are not visible. The most severe follow-up testing results in 2.5 years after the baseline were determined through linkage to the national Danish Pathology Register (Patobank). 29

| Statistical analysis
Samples with inadequate cytology (N=25, 0.5%) were excluded, as were samples with invalid CLART test results (N=12, 0.2%). The remaining 4997 samples were included in the analysis. They represent a typical collection of samples handled by a cytology laboratory during its routine operations. Their screening histories were determined from the Patobank from 1 January 2000 until the baseline testing date. These screening histories were used to determine the reason for taking each sample. The reasons were categorised as either primary screening or follow-up of recent abnormalities (see Appendix for detailed definitions). In the present analysis, we limited the primary screening population to samples taken at age 30-  Finally, we evaluated the differences in the assays' ability to detect HPV infections and ≥CIN2 by determining the degree of their concordance in baseline testing results. As previously, 2 concordance was calculated as the conditional probability that all four assays returned a positive test result if at least one of the four was positive, and discordance as [100%-concordance]. The 95% confidence intervals (CI) for relative discordance comparing the various populations were calculated assuming lognormal distribution.

| Ethical approval
Baseline testing on the residual material was undertaken as a quality

| Signal strengths in detected HPV infections
A comparison of central tendency measures showed that signal strengths, particularly for HC2, were not distributed normally (Table 2).
Median signal strengths for positive HPV test results in all 4997 samples were weaker in normal than in abnormal cytology (Table 3).
For HC2, the median RLU/CO in normal cytology was 11.0 (IQR:   | 421 signal strengths that were observed for all 4997 samples when comparing normal vs abnormal cytology (Table 3 above) were also seen in primary screening, regardless of the assay (Table 3 below).
Next, we compared the complete screening population (N=2846; of which 4% of samples were associated with abnormal cytology) with the referral population with abnormal cytology (N=367). The median signal strengths of HPV-positive samples were weaker in the primary screening than in the referral population (P<.001; Table 4).
In the referral population with normal cytology (N=516), the median signal strengths (Table 5) appeared similar to those in samples associated with normal cytology from primary screening (N=2719; Table 3 below).

| Signal strengths in detected ≥CIN2
When considering all 4997 samples, HPV-positive ≥CIN2 with normal cytology had weaker median signal strengths than HPV-positive ≥CIN2 with abnormal cytology (P<.001 for HC2 and cobas, and P<.002 for APTIMA; Table 3  The differences in HPV signal strengths between normal and abnormal cytology associated with ≥CIN2 were also seen for samples from primary screening but were significant only for APTIMA (P=.02; Table 3 below). The primary screening population as a whole (Table 4), where the majority of ≥CIN2 in our population had abnormal cytology, also showed weaker signal strengths than the referral population with abnormal cytology, although the differences were not significant. Referral population samples with normal cytology (Table 5) were fairly similar to primary screening samples with normal cytology (Table 3 below).

| Concordance between HPV assays in detecting HPV infections and ≥CIN2
Among all 4997 samples, 1343 (29%) of 4630 samples with normal cytology had a positive test result on one or more HPV assays, but in 902 (67%) of the 1343 samples, the assays showed discordance ( Discordance between the four assays was less frequent in samples associated with ≥CIN2, 26% in normal and 15% in abnormal cytology when all 4997 samples were included. In primary screening, discordance was 33% in normal cytology and 11% in abnormal cytology. Although assay discordance in detecting ≥CIN2 was two to three times as frequent in normal than in abnormal cytology, the difference did not reach statistical significance in our data.

| CONCLUSION S
In our study, HPV infections had weaker median signal strengths in normal than in abnormal cytology. Being in the range of approximately 100 RLU/CO on HC2 and ca. 6 Ct on cobas, the absolute differences in the median signal strengths were not trivial. Abnormal cytology samples also showed a higher degree of assay concordance than normal cytology samples in detecting HPV infections.
HPV testing detects more ≥CIN2 than cytology. 30 At least part of ≥CIN2 in normal cytology is clinically important. In the two-round screening data from randomised trials, treatment of cytology-normal/HPV-positive ≥CIN2 reduced the population's risk of developing cervical cancer. 1 In our study, most ≥CIN2 were detected by all HPV assays. Nonetheless, the lower median signal strengths and assay discordance in women with normal cytology suggested that cytology-normal ≥CIN2 are not only difficult to detect through cytology but might also be more difficult to detect with HPV assays than is the case with cytology-abnormal ≥CIN2.
HPV assays utilise various molecular targets and methodologies, which result in different test dynamics. Differences in assay characteristics and calibration may, as discussed previously, 2 explain why the assays return discordant HPV test results. Higher amounts of the target viral input in a sample and the associated stronger signals may, however, make all assays more likely to return a positive test result.
We observed this pattern in women with cytological and histological abnormalities, who, in turn, also showed a higher likelihood of concordance between HPV assays than women without abnormalities.
Based on these data, we can infer that referral population studies focusing predominantly on women with abnormal cytology underestimate the differences between HPV assays that would become apparent in primary screening. With a view on the practicality of undertaking the necessary assay validation studies, an international expert group has developed non-inferiority testing guidelines for primary screening. These guidelines require that new assays be compared to HC2. 31  CI, confidence interval; CIN, cervical intraepithelial neoplasia; HPV, Human Papillomavirus; RD, relative discordance. Note: In women with normal cytology undergoing follow-up for a recent abnormality (N=516), 200 (39%) tested positive on at least one HPV assay. Of the 200 women, 143 (72%) did not test positive on all four assays. The concordance between the four assays in detecting HPV infections was similar as in women with normal cytology undergoing primary screening, relative discordance for normal cytology in follow-up vs primary screening: 0.92 (95% CI: 0.84-1.02).
HPV status was not limited to the results of a single assay, as women qualified for repeated testing if one or more HPV assays were positive. This allowed for a more comprehensive evaluation of the differences in detecting HPV-related abnormalities between the four assays than has been the case in several previous studies.
Nevertheless, this is a post-hoc analysis, and the study was not powered to detect differences between cytology-normal and cytology-abnormal CIN. In primary screening, only 12 out of 50 ≥CIN2 were associated with normal cytology. With larger numbers of screened women, the observed differences in the ability of the assays to detect ≥CIN2 could, plausibly, reach statistical significance.
Studies from other populations are warranted to corroborate our findings.
In conclusion, differences between HPV assays tend to be under-

DEFINITION OF PRIMARY SCREENING SAMPLES
In the Patobank, 29 the reason for taking the sample is not registered systematically. We used an algorithm to determine which samples were most likely taken for primary screening, ie, not for follow-up of a recent abnormality. Primary samples were defined as those without a: • previous histological diagnosis of cervical cancer, • histologically-confirmed cervical intraepithelial neoplasia (CIN) of any grade in ≤3 years, • ASCUS cytology or non-CIN cervical histology in ≤15 months, • more severe cytological abnormality, inadequate cytology or a positive HPV test result in ≤12 months.
Reflecting routine practice, primary samples included a small proportion of samples taken for investigation of symptoms. The Patobank data were retrieved from 1 January 2000 onwards.

Processing of samples and assay instrumentation
The study protocol, sample storage, and assay testing protocols were agreed upon with all manufacturers before the study. All instrumentation and software were used as supplied and maintained by the manufacturers.

Baseline
These testing protocols were published, with minor textual revisions, previously. 2

HC2
We used cytology post-quot material that remained from the cytology procedure. As part of the cytology processing, post-quot material was diluted approximately 1:1 in SurePath. DNA was either denatured prior to testing by pre-treating manually in line with the manufacturer's CE-IVD protocol, or was isolated and purified using the DSP AXpH DNA kit on QIASymphony SP (Qiagen, Hilden, Germany). Testing was undertaken on automated Rapid Capture System (RCS; Qiagen, Gaithersburg, MD, USA). A minority of samples used for routine HC2 triage of women with ASCUS at age ≥30 years were denatured and tested manually.    Follow-up testing for women with cytology-normal/ HPV-positive test results at baseline Testing on HC2 for all follow-up samples was performed on the cytology post-quot material with manual DNA denaturation followed by testing on the RCS.