- Top of page
Objective To evaluate new techniques in primary cervical cancer screening programmes.
Design Cross sectional pilot study.
Setting Department of Obstetrics and Gynaecology, Helsinki University Hospital.
Population Consecutive 2032 human papillomavirus (HPV) DNA and Pap smear samples were taken. Histological diagnoses were obtained from 460 patients.
Methods We compared the validity of the high risk (HR) HPV DNA detection test to automation-assisted and conventional Pap smear screening.
Main outcome measures Specificity and sensitivity of screening methods.
Results Twenty-three percent of women were HPV positive. Forty-five of 46 had high grade lesions and cancers were HR HPV DNA positive, whereas 72/93 of low grade and more severe lesions were HR HPV DNA positive. When histologically verified high grade lesions were observed, the relative sensitivity of HR Hybrid Capture 2 (HR HC2) test was 98% compared with conventional Pap smear and Papnet tests, which performed 54%versus 58%, 83%versus 86% and 93%versus 98% relative sensitivity respectively, using cytological diagnoses HSIL (high grade squamous intraepithelial lesion), LSIL (low grade squamous intraepithelial lesion) or ASCUS (atypical squamous cells of undetermined significance) as the cutoff. The specificity of HC2 test (77–79%) was comparable with the ASCUS+ (ASCUS and more severe) cytology (68–79%), but lower when compared with LSIL+ (91–95%) or especially HSIL+ (97–99%) Pap smear results.
Conclusion Pap smear, as a screening test, is very different from HPV DNA detecting test HR HC 2. If cutoff LSIL or more severe lesions is used, primary Pap smear is clearly more specific than HR HC2, but markedly less sensitive. Due to high relative sensitivity of the HPV, only very few histologically confirmed high grade lesions would be detected among HPV negatives using simultaneous cytology. On the other hand, using HPV DNA test alone would lead to multifold amounts of referrals for colposcopy. A posterior Pap smear assessment among HPV positives might be helpful in increasing sensitivity and specificity of screening and defining those who need an immediate referral or treatment. We plan to incorporate primary HR HPV DNA test with posterior Pap smear reading of HPV positives into our ongoing randomised prospective multiarm trial evaluating new techniques in organised screening for cervical cancer in Finland.
- Top of page
Cervical cancer is still a major problem with nearly half a million new cancer cases annually in the world, in spite of a good tool for preventing this fatal cancer—the Pap smear. Oldest organised mass screening programmes, based on Pap smear, have been working for 40 years (e.g. in Finland, Sweden and in British Columbia, Canada), effectively reducing the incidence of cancer 70–80%, which seems close to the maximal effect to be gained with this method.1–6 But there has been also unsuccessful screening programmes with clearly smaller or no effect.2 Most of these have been based on opportunistic screening. Especially the failures are found in developing countries, where the cancer problem is biggest. These failures have been claimed to be results of low quality of screening, low sensitivity and specificity of conventional Pap smear or low quality colposcopy, treatment and follow up practices. Those factors may have, naturally, some role, but a successful screening is based on high coverage and attendance rates of the total female population.6
New problems in screening in future, especially in some Western countries, will be the shortage of cytotechnicians. More smears should be screened by fewer people. In developing countries, a lack of trained cytotechnicians, in addition to other infrastructure problems, also exists. This has generated a development of automation-assisted screening devices and also of better readable Pap smears (e.g. liquid Pap systems) to decrease the workload used per slide, allowing more slides to be screened with the same staff.
During the last decade, the role of human papillomavirus (HPV) in development of cervical cancer and cervical precancerous lesions has been confirmed and a great number of articles concerning HPV detection in cervical cancer screening has been published. These studies show that HPV detection has a high sensitivity in detecting especially high grade precancerous lesions, but the specificity of HPV test alone is quite low for mass screening purposes.7–10 Many HPV tests have been developed based on hybridisation and multiplication techniques, but only few are commercially available or otherwise practical to be used in mass screening settings.
As all these new techniques may become routine screening programmes throughout the world, their effectiveness should be evaluated. In Finland, we have an ongoing multiarm study focussing on new screening techniques for cervical cancer. New competing techniques are compared with conventional Pap smear screening in randomised prospective setting within the organised screening programme. The ultimate endpoint will be the development of cervical cancer incidence and mortality in Finland. An automation-assisted (Papnet) screening arm is ongoing since 1999 and the results on process measures have been published elsewhere.11 The next evaluation arm will be a routine HPV screening compared with conventional Pap smear screening and automation-assisted (Papnet) Pap smear screening. To be able to construct the best possible way to use HPV test in the cervical cancer screening arm, we run a cross sectional pilot study, where we compared consistency and some key validity parameters of the conventional Pap smear to those of the HR HC2 HPV test (Digene, Gaithersburg, Maryland) and of automation-assisted screening.
- Top of page
Altogether, 2032 consecutive cervical smear samples were taken from 1999 women visiting the outpatient clinics of the three different Departments of Gynaecology of the Helsinki University Central Hospital (i.e. Naistenklinikka, Jorvi and Kätilöopisto hospitals) in Finland from October 2000 to March 2001. These outpatient clinics did not do normal routine screening, but were referral places for various gynaecological disorders. The patients included women referred for colposcopies, operations, having bleeding disorders, ovarian masses, incontinence problems etc. All women were informed of the study and a written consent was obtained. The ethical committee of the hospital approved the study protocol.
During the visit, a Pap smear was taken from these women as a routine test. The mean age of the women was 38.3 years (range 15–86 years). A normal, conventional vaginal–cervical–endocervical Pap smear was first taken (one sample with a spatula from the vaginal fornices, one with a second spatula from the ectocervix and one with a endobrush from the endocervical canal); all three samples were spread on the same objective glass. The endobrush device was then placed and left in the HC2 transport vial. The HR HPV DNA HC2 samples were analysed by the manufacturer, Digene, in the United States. The conventional smears were prepared, stained and analysed routinely and with the automation-assisted method (Papnet) as described in the study protocol published earlier.11 None of the smears was technically inadequate. Endocervical glandular cells are not present in 4.8% of the smears.
The cytological diagnoses obtained originally from the conventional Pap smears from the three hospitals were used. The same slides were also scanned using Papnet device. Two cytotechnicians (both over 15 year experience) screened the scanned Pap smears. Their experience with Papnet was four years and over 25,000 smears each. After screening by the cytotechnicians, the smears were analysed and diagnosed by an experienced cytopathologist (P.N.). All the screening and diagnosing of the smears was done without the data of the other methods, including HPV results.
Colposcopy was performed and biopsies were taken when clinically or cytopathologically indicated. The referrals were based on the results of the conventional Pap smear results. High grade squamous intraepithelial lesion (HSIL) was always an indication for colposcopy, low grade squamous intraepithelial lesion (LSIL) also generally (about 30% were only followed cytologically), atypical squamous cells of undetermined significance (ASCUS) indicated cytological follow up.
Distributions of the test results, as well as histologically confirmed findings, were drawn into the main categories (HSIL+, LSIL, ASCUS, normal). The consistency of cytological results in these groups, obtained with the two different methods, were compared each others using simple and weighted κ coefficients. The relative test sensitivity and specificity were also calculated, using the histologically confirmed outcome as the gold standard. Ninety-five percent confidence intervals were estimated for sensitivity and specificity with a binomial test. It was also tested with a pairwise McNemar's test, whether there were differences in the sensitivity estimates of the tests. The pairwise testing between the conventional and automation-assisted cytology and HPV DNA test was performed with various cutoff values in histology, but using ASCUS+ (leading to follow up activity at minimum) as the cutoff in cytology. A crude graphical approximation was also done comparing the different specificity levels in the screening test cutoffs.
- Top of page
We obtained results from 2032 HPV DNA samples among 1999 women. Twenty-three percent of them were HR HPV DNA positive (1 rlu/co cutoff). The distribution of HPV results is shown in Fig. 1. Detailed cytological diagnoses of the conventional Pap smear and automation-assisted cytology (Papnet) are shown in Table 1. LSIL or more severe lesions were found in 7.5% and 9.2%, respectively.
Table 1. Cytological diagnoses of the different Pap smear methods. Values are given as n (%).
|Cancer||2 (0.1)||2 (0.1)|
|HSIL||50 (2.5)||55 (2.8)|
|LSIL||101 (5.0)||132 (6.7)|
|AGUS||25 (1.2)||41 (2.1)|
|ASCUS||316 (15)||400 (20)|
|Benign or normal||1528 (75)||1340 (68)|
|Not detectable||5 (0.2)||0 (0.0)|
|All above||2027 (100)||1970 (100)|
The histological diagnoses were obtained from 460 patients who were referred for colposcopy because of a original referral or an abnormal Pap smear detected in this study. We found 188 (9.3%) low grade lesions (HPV changes or mild dysplasia, i.e. cervical intraepithelial neoplasia 1 [CIN1]), 41 (2.0%) high grade lesions (moderate and severe dysplasia and cancer in situ, i.e. CIN2–3) and 5 (0.3%) cancers.
Nearly half of the HR HPV positive persons (221/472 for conventional and 192/472 for automation-assisted screening) were cytologically benign or normal. HPV DNA positivity compared with histological diagnoses is shown in Table 2. Forty-five of 46 (98%) of all high grade lesions and cancers were HR HPV DNA positive, whereas 136/234 (58%) were borderline or low grade and more severe lesions were HR HPV DNA positive.
Table 2. HPV DNA results compared with the histologically confirmed findings.
|HPV DNA (HR HC2)||Histologically confirmed outcomes||Borderline||Benign||Normal/no coded status||ND/other||Total|
Consistency of the two cytological methods was fairly good (κ= 0.68, weighted κ= 0.75. Kappa estimates for histologically verified CIN1+ cases were only slightly better (Table 3).
Table 3. Comparison of the conventional and automation-assisted cytological results.
|Conventional||Automation-assisted||ASCUS or AGUS||Benign or normal||Total|
|HSIL or cancer||LSIL|
|Whole data (both of the cytological analyses performed)|
|HSIL or ca||44||1||0||0||45|
|ASCUS or AGUS||2||30||256||41||329|
|Benign or normal||5||7||181||1293||1486|
|Simple κ coefficient 0.68, 95% CI 0.65–0.72, weighted κ coefficient 0.75, 95% CI 0.72–0.78.|
|Among verified histological CIN1 and more severe cases (and both of the cytological analyses performed)|
|HSIL or ca||27||0||0||0||27|
|ASCUS or AGUS||2||3||16||2||23|
|Benign or normal||1||0||2||7||10|
|Simple κ coefficient 0.77, 95% CI 0.66–0.88, weighted κ coefficient 0.80, 95% CI 0.70–0.91.|
Relative sensitivity and specificity of each screening method are shown in Table 4 and Fig. 2. Assuming a similar specificity level (Fig. 2), the relative sensitivity of the HR HC2 test was higher when histologically confirmed high grade and cancer lesions (CIN2+ or CIN3+) were observed. The differences were not statistically significant. When using the ASCUS+ cytological cutoff (leading at minimum to follow up activity), the difference in CIN2+ and CIN3+ sensitivity between HPV and either of the two cytological tests was statistically not significant however. The relative sensitivity of the HR HC2 test was slightly lower; however, when any histological CIN1+ outcomes were included in the analysis (P < 0.05 for the comparison with either of the two cytological methods using ASCUS+ as the cutoff), the difference was of a borderline significance when adjusting for the specificity level (Fig. 2).
Table 4. Sensitivity (Se) and specificity (Sp) of the three cytological screening methods with 95% confidence intervals (CI) using various cutoff points, grouped by the severity of the histologically confirmed findings. Values are given as % [95% CI].
|Conventional cytological screening|
| ASCUS+||80 [28–99]||95 [77–100]||93 [82–99]||86 [77–92]|
| LSIL+||60 [15–95]||86 [65–97]||83 [69–92]||60 [50–70]|
| HSIL+||60 [15–95]||68 [45–86]||54 [39–69]||33 [24–44]|
|Automation-assisted cytological screening|
| ASCUS+||100 [40–100]||95 [76–100]||98 [87–100]||89 [80–94]|
| LSIL+||100 [40–100]||90 [70–99]||85 [71–94]||55 [44–66]|
| HSIL+||100 [40–100]||76 [53–92]||59 [42–74]||38 [28–49]|
|HPV DNA HR HC2, 1+ rlu/co||100 [48–100]||100 [85–100]||98 [88–100]||77 [68–85]|
|Conventional cytological screening|
| ASCUS+||76 [74–78]||76 [75–78]||77 [75–79]||79 [77–81]|
| LSIL+||93 [91–94]||93 [92–94]||94 [93–95]||95 [94–96]|
| HSIL+||98 [97–98]||98 [98–99]||99 [98–99]||99 [98–99]|
|Automation-assisted cytological screening|
| ASCUS+||68 [66–70]||69 [67–71]||69 [67–71]||71 [69–73]|
| LSIL+||91 [89–92]||91 [90–92]||92 [91–93]||93 [92–94]|
| HSIL+||97 [96–100]||98 [97–99]||98 [98–99]||99 [98–99]|
|HPV DNA HR HC2, 1+ rlu/co||77 [75–79]||78 [76–79]||78 [77–80]||79 [78–81]|
Figure 2. Sensitivity and specificity estimates of the two cytological tests and of the HPV DNA test using various histologically confirmed outcomes. Cutoffs of the cytological results were HSIL, LSIL and ASCUS. Cutoff of the HPV DNA test was 1 rlu/co. Regarding the HPV DNA test, 95% confidence limits of the test sensitivity and specificity are indicated.
Download figure to PowerPoint
Specificity of the cytological methods for any low and high grade histological lesions was clearly over 90% when cytological diagnosis LSIL+ was used (leading to referral) and nearly 100% when HSIL+ was used. All these specificity estimates were superior compared with those of the HR HPV DNA test. The specificity of the cytological tests decreased with about 20% to 30% when ASCUS+ was the cutoff. At cutoff, ASCUS+ specificity estimates of automation-assisted cytology were lower when compared both with conventional cytology or HR HPV DNA test (P < 0.05). Specificity of HR HC2 was comparable or only slightly better than with the conventional ASCUS+ cytology.
- Top of page
The results showed that Pap smear, as a screening test, whatever the precise method of using it is, is very different from HR HPV DNA detecting test HR HC2. With cutoff LSIL and more severe lesions (leads to referral for colposcopy), Pap smear is clearly more specific than HR HC2, but markedly less sensitive. With cytological cutoff ASCUS+, these differences between tests diminish. In practice, when these tests are used alone as a single test in mass screening for cervical cancer, one could put it very strongly: Pap smear misses some of the cancerous or precancerous cases, while HR HC2 test (or probably any other HPV DNA test) will produce a lot of false positive cases (i.e. women with HPV infection who do not have or never will get any precancerous lesion, let alone cervical cancer). If these HR HPV DNA positive women would be referred to colposcopy, all the gynaecological outpatient clinics would be filled in short time. Our results confirmed also that the amount of HR HPV DNA positivity does not correlate strongly enough to the severity of the lesion to be used in practice, as shown in some earlier articles.12,13
Verification bias was one of the limitations of the present study. The cross sectional sensitivity estimates, particularly of the cytological methods, are in this study likely too high, because not all of the women under the study were referred to colposcopy. Opposite to the verification bias, there is some tendency to under-estimate the relative sensitivity of the HR HPV DNA test in the present study compared with the cytological screening methods, because among those who are HPV positive (n= 472), 38% (n= 179) had a lower cytological classification than ASCUS in both of the two cytological tests. These women were thus not subject to any confirmational investigations as to their precancerous status. There might have been precancerous lesions in some of them. The HR HPV DNA test still appeared slightly more valid for CIN2+ and CIN3+ findings than the two cytological tests, as it was slightly more sensitive if similar specificity values were assumed.
A follow up design of cancers after the screening visit could have offered a more reliable gold standard than the cross sectional histologically confirmed information used in this study.14 One could avoid then errors in the validity parameters potentially arising also from false positive or overdiagnosis of the histologically confirmed pre-cancerous lesions. The follow up information is not available yet.
The HR HC2 HPV DNA test was positive for each of the CIN3+ cases and all except one of the CIN2 cases detected through the two cytological tests. High relative sensitivity of the HPV test (a small number of sensitivity errors) suggests that among HPV negatives, only very few CIN2+ cases would be detected using simultaneous independent cytology and that simultaneous double testing (independent HPV and cytology) does not add meaningfully to the cross sectional sensitivity compared with HPV testing alone. There is a fairly good consistency in this notion when compared with results of a review including several cross sectional studies using a double-testing design15 and meta-analysis on triage studies.16 Follow up activity after the screening visits (e.g. a new test after one year for those with HPV positive and cytology negative) might have revealed some more cancer or pre-cancer cases. However, one should be aware that in the follow up screening activity, some of the cancers that possibly need rather immediate treatment would be detected in a later period. One may not reach the optimal sensitivity in a follow up screening because of non-attendance in the later screening or follow up visits.17 Irrespective of the limitations of the cross sectional design used in this study, one can argue that a posterior cytological testing in connection with a positive HPV result, combined with the longitudinal follow up, might lead to optimal sensitivity and also specificity in the screening activity.
Cervical cancer develops slowly, as a rule in 10–15 years via precancers. Taking advantage of this, organised cervical cancer screening and its preventive effect is based on repeated Pap tests with 3–5 year intervals, although one single Pap test already gives increased protection against cervical cancer.11 When Pap smear test is repeated in a screening programme, the sensitivity is increasing up to 80–90%,5 which has also been shown in the Finnish organised mass screening programme, where the age-adjusted incidence of cervical cancer in Finland decreased 80% in 30 years screening history between 1962 and 1992.3 The use of HR HPV DNA test alone, using the standard positivity cutoff point (>1 rlu%) recommended by the manufacturer, would probably have the same result, but with much higher costs. In Finland, the test itself costs at least twice the price of a Pap smear and it yields three to five times more colposcopies according to our study (LSIL in Pap smear as a threshold point for colposcopy and HPV positivity with HR HPV DNA test). Using a higher cutoff level (e.g. 5 rlu/co) like in this study decreases the sensitivity clearly, but the specificity remains lower than that of a Pap smear.
The fact that HPV test has high sensitivity and low specificity and Pap smear low sensitivity and high specificity could be used to support each other. As a result to our pilot trial, we are setting up a new arm in our ongoing prospective, randomised multiarm trial11: one-third of the Finnish female population is randomised (using the population registry) to HPV arm. An HR HC2 HPV DNA test will be taken simultaneously with a Pap smear. The Pap smear is fixed and stained and covered with a glass but not screened if the HC2 test is negative. If the HC2 test is positive, the Pap smear will be screened and diagnosed. With negative Pap result and positive HPV result, the women will be in placed in the risk group and called to new tests after a year. If the Pap smear result is positive (LSIL+), the woman is referred to colposcopy. With this algorithm, the number of colposcopies and the costs will be kept reasonably low. Simultaneously, the sensitivity of this single screening round will increase.
In that kind of a randomised, prospective trial, the amount of yearly found and histologically confirmed precancerous lesions is only a surrogate marker. The endpoint will be the development of the cervical cancer incidence after the screening visit in these trial arms and especially the number of interval cancers (i.e. the cancers emerging between screening rounds). This trial may not have direct impact on screening policies in other countries, but certainly it will do it indirectly.
The results obtained with Papnet are in concordance with our results from the large mass screening study.11 Automation-assisted screening performed with a slightly higher sensitivity but a little lower specificity compared with the conventional screening. Numerical differences were small. Differences observed between conventional and automation-assisted cytology match within the range of variability in the criteria or practices between the units.