Fax: (617) 724-6564
Human papillomavirus testing using hybrid capture II with surepath collection†
Initial evaluation and longitudinal data provide clinical validation for this method
Version of Record online: 7 NOV 2006
Copyright © 2006 American Cancer Society
Volume 108, Issue 6, pages 468–474, 25 December 2006
How to Cite
Ko, V., Tambouret, R. H., Kuebler, D. L., Black-Schaffer, W. S. and Wilbur, D. C. (2006), Human papillomavirus testing using hybrid capture II with surepath collection. Cancer, 108: 468–474. doi: 10.1002/cncr.22285
This study was presented at the 2006 United States and Canadian Academy of Pathology Annual Meeting, Atlanta, Georgia.
- Issue online: 11 DEC 2006
- Version of Record online: 7 NOV 2006
- Manuscript Accepted: 7 AUG 2006
- Manuscript Revised: 1 AUG 2006
- Manuscript Received: 13 MAR 2006
- cervical cytology;
- human papillomavirus testing;
- Hybrid Capture II
Testing for human papillomavirus (HPV) is an integral part of equivocal cervical cytology triage. Clinical validation of non-FDA (Food and Drug Administration)–approved methods is therefore important because of the high volume of such tests and the implications for missed high-grade lesions if test performance is not optimal.
A preinitiation study and 17 months of follow-up data using Hybrid Capture II (HC II) HPV detection with SurePath (SP) sample collection were analyzed. Results of HPV tests on abnormal cytology samples were collected and compared with follow-up results. HPV-positive rates were determined in cases of low-grade squamous intraepithelial lesion (LSIL) and high-grade squamous intraepithelial lesion (HSIL), and follow-up rates of cervical intraepithelial neoplasia (CIN) were determined in HPV-positive and -negative cases of atypical squamous cells of unknown significance (ASC-US). Rates were compared with published data using FDA-validated methods.
The preinitiation study showed the test method to be 100% sensitive for the detection of LSIL (20 cases) and HSIL (8). The ASC-US follow-up study (2319 cases with 625 having biopsy results) showed that the rate of CIN III+ in HPV +/− cases was 7.8%/1.4%, and of CIN II+ was 17.5%/4.3%, respectively. The positive predictive values/negative predictive values (PPV/NPVs) (CIN II+) for the test were 17.5%/95.7%, respectively.
Published FDA-validated HPV testing follow-up data show that the expected rates of CIN III+ and CIN II+ in the HPV-negative ASC-US population are 1.4% and 5%, respectively, with PPV/NPVs (CIN II+) of 20%/99%, respectively. By comparison, the present data using HC II with SP show strong similarity, indicating clinical validity for the use of this method. Cancer (Cancer Cytopathol) 2006; © 2006 American Cancer Society.
High risk types of the human papillomavirus (hrHPV) are known to be the main causative agents associated with the development of cervical cancer. hrHPV is detected in nearly 100% of cervical carcinomas and their precursor lesions, while hrHPV-negative carcinomas are extremely rare.1 At the molecular level, hrHPV E6 and E7 genes and overexpression of related oncoproteins have been shown to promote a variety of effects in cells, including immortalization, alteration of the cell cycle, and the promotion of cell growth and mutations, all of which predispose to the development of neoplasia.2 Given that hrHPV is necessary in the pathogenesis of virtually all cases of cervical cancer, testing for hrHPV infection occupies a central role in cervical cancer screening.
Since 2001, hrHPV testing has been recommended as the standard of care for managing women with atypical squamous cells of unknown significance (ASC-US) Pap smears collected by a liquid-based method.3 These recommendations are largely based on the findings of the ALTS (ASCUS/LSIL Triage Study) trial. The ALTS trial was a large multicenter clinical trial, a portion of which studied 3488 women referred for atypical squamous cells of undetermined significance (ASC-US) using 3 management strategies: 1) immediate colposcopy, 2) cytology triage, and 3) hrHPV triage. On the basis of results obtained in the trial, the utility of hrHPV testing was confirmed as a triage for ASC-US cases because of its ability to identify an equivalent number of high-grade lesions with fewer referrals to colposcopic examination.4–6
The 2004 American College of Obstetrics and Gynecology Guidelines, which are based on data from the ALTS trial, recommend that patients with either a ≥LSIL+ cytology or ASC-US with a positive hrHPV test should have immediate colposcopy. Patients with a negative cytology and positive hrHPV test need to be retested by both methods in 6-12 months. Patients with ASC-US and a negative hrHPV should have a second cytology in 1 year. In addition, patients aged 30 years or older with a negative for intraepithelial lesion or malignancy (NILM) cytology and negative hrHPV test can have their screening interval lengthened to 3 years, because these 2 tests, when used in combination, have a negative predictive value (NPV) of 99.9% (at a threshold of CIN II+ (cervical intraepithelial neoplasia II+)), leaving only a 0.1% chance of a missed high-grade lesion.7
In addition to its role in ASC-US, hrHPV testing may also show utility in the triage of atypical glandular cell (AGC) cases, although no formal recommendations have been made to date. One study of 187 AGC cases showed that hrHPV testing had a sensitivity of 83%, a specificity of 78%-82%, a positive predictive value (PPV) of 56%-61%, and a NPV of 91%-95% for high-grade cervical disease (squamous and endocervical).8
Given the importance of a patient's hrHPV status in cervical cancer screening and the high prevalence of ASC-US samples in the typical screening population, erroneous test results could lead to inappropriate management of patients and the potential for missed high-grade lesions. Therefore it is imperative that new testing methods for hrHPV detection undergo rigorous clinical validation studies prior to widespread use. Currently, there is only 1 FDA (Food and Drug Administration)-approved hrHPV test: the Hybrid Capture II test (HC II) (Digene, Gaithersburg, MD) used in conjunction with the Standard Transport Media (STM) (Digene) kit or the ThinPrep (TP) (Cytyc, Marlborough, MA) collection system. The United States FDA premarket approval process ensures that these methods are valid through rigorous clinical studies. Non-FDA approved testing must also meet significant standards via in-house validation. The use of such a validation method is very common; however, the FDA does not oversee this process.9 Many experts would agree that it is not enough to test an assay's analytical sensitivity in terms of limit of detection for measuring viral load; rather, the use of more clinically relevant benchmarks such as the sensitivity, specificity, PPV, and NPV for disease are required, all of which require patient follow-up data.9
At Massachusetts General Hospital (MGH), the SurePath (SP) (TriPath Imaging, Burlington, NC) liquid-based system is used as 1 method of collection because of its use of ethanol fixation, lower cost, lower unsatisfactory rate, more reliable quantity available for HPV testing, and because at the time of adoption, it was the only liquid-based preparation method with available automated scanning. The SP system for routine processing of cervical specimens is currently FDA approved, and is widely used for cervical cytology evaluation. However, the use of residual cells from this method for hrHPV testing using HC II is not FDA approved and therefore requires, at a minimum, laboratory validation. In addition, further validation using clinical follow-up would enhance the credibility of the test. The present study compares published data obtained from FDA-approved methods with the results of similar studies using this new method.
HC II for hrHPV (which includes types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68) is a nucleic acid hybridization assay, and uses an HPV RNA probe cocktail to hybridize with target DNA in the patient specimen. RNA:DNA hybrids are captured by the surface of a microplate well coated with antibodies specific for RNA:DNA hybrids. Alkaline phosphatase conjugated antibodies against RNA:DNA hybrids are then reacted with the captured hybrids, and the addition of a chemiluminescent substrate results in emitted light that is measured as relative light units (RLUs) by a luminometer. An RLU greater than the cutoff value indicates the presence of hrHPV DNA in the specimen.
The SP Pap test method combines liquid-based specimen collection with preprocessing centrifugation steps through a sucrose density gradient media. This method is designed to diminish blood, inflammation, and acellular debris, thereby improving the overall adequacy and quality of the collected sample.
A preinitiation analytical validation study was done in order to show primary performance data of HC II/SP, compared with an FDA-approved method (STM), on known abnormal SP cytology samples before general introduction of the method for ASC-US triage. This study is essentially a pilot done in order to illustrate rough comparable performance before large-scale use. The type of study performed is similar to preinitiation studies commonly performed before use of other types of non-FDA–approved methods. The test population consisted of prospectively obtained patients from a high-risk gynecology clinic with additional clinician selection of patients most at risk for high-grade lesions based on clinical history. This study comprised a concurrent dual collection of patient specimen using the SP method and the STM collection kit. The STM sample (preinitiation study only) was processed for HC II testing according to the FDA-approved directions provided by Digene. The SP sample (for both preinitiation and follow-up studies) was processed according to a protocol adapted from the Yale-New Haven Hospital (Schofield K, personal communication). First, the entire 10-mL SP sample undergoes routine processing for liquid-based cytology (vortexing and density gradient centrifugation) in order to prepare a Papanicolaou-stained slide. After this, the Yale protocol begins by adding 4 mL of Cytorich fluid (TriPath) to the entire remaining cellular sample. The specimen is then centrifuged at 2900 g for 15 min. The supernatant is decanted, and 250 μL of a 2:1 mixture of Specimen Transport Medium (STM) and denaturation mixture (Digene) is added according the manufacturer's specifications. The samples are vortexed and then denatured at 65°C in a water bath for 90 min. This differs from the STM/TP method, which requires denaturation at 65°C for 45 min in the water bath. After denaturation, samples are allowed to reach room temperature. For the remaining steps, the Digene HC II manufacturer's protocol is followed. The sample may be stored in the refrigerator at 2°C–8°C overnight for processing the next day, or frozen at −20°C for future testing according to the Digene package insert. The results of hrHPV testing were reported as either positive or negative, as determined by the sample's RLU, compared with the cutoff value.
The follow-up study population consisted of all patients who received a SP Pap test with hrHPV testing ordered between July 1, 2004, and December 1, 2005. The population from which specimens were received was considered an overall normal to medium risk group. To show efficacy of the SP method in clinical use, all cases that had HC II using the SP collection method were checked for concurrent or follow-up biopsies and hysterectomy specimens. Histologic results within 1 year of the hrHPV test, up to December 31, 2005, were then collected and the diagnoses were classified as negative, CIN I, CIN II, CIN III, or carcinoma. Routine clinical practice determined treatment and overall management for this population. For this study, the analysis was focused on the cases with an original cytologic interpretation of ASC-US, although results of hrHPV tests performed on cases of low-grade squamous intraepithelial lesion (LSIL), high-grade squamous intraepithelial lesion (HSIL), and carcinoma were also analyzed.
Operating characteristics of the HC II/SP method were calculated, including sensitivity, specificity, NPV, and PPV at appropriate histology thresholds and compared with published data for FDA-validated methods. The procedures followed here were per approved protocol no. 2005-P-001842/1 of the IRB of MGH.
The preinitiation validation study consisted of 108 cases, of which 28 were either LSIL (20) or HSIL (8). The remaining cases were categorized as negative (58), ASC (21), and unsatisfactory (1). Comparison of results on LSIL and HSIL cases showed close similarities between the SP and STM methods (no differences by statistical analysis), although the SP method was numerically more sensitive at 100% (vs. 82.1% for STM) for all SIL cases. For LSIL cases, HC II/SP was positive in 100% (20/20), while HC II/STM was positive in 80% (16/20). For HSIL cases, HC II/SP was positive in 100% (8/8), while HC II/STM was positive in 87.5% (7/8). At a threshold of ASC+ the sensitivity of SP was 89.8%, compared with that of STM at 75.5%. At a threshold of LSIL+, the sensitivity of SP was 100%, compared with 82.1% for STM. At a threshold of HSIL+, the sensitivity of SP was 100%, compared with 87.5% for STM (Table 1).
|Diagnosis||SurePath||STM||McNemar test with Yates correction|
|HSIL+||8/8 (100)*||7/8 (87.5)||P = 1.0000|
|LSIL+||28/28 (100)||23/28 (82.1)||P = .0736|
|ASC-US+||44/49 (89.8)||37/49 (75.5)||P = .0233|
The ongoing follow-up study initially included 3479 cases that had HC II/SP assays, of which 2319 (66.7%) cases were interpreted as ASC-US. Of the remaining cases, 16 were AGC (0.5%), 147 were ASC-H (4.2%), 21 were HSIL (0.6%), 134 were LSIL (3.9%), 838 were NILM (24.1%), 3 were unsatisfactory (<0.1%), and 1 was carcinoma (<0.1%). The following hrHPV-positive rates were obtained for each cytologic interpretive category: carcinoma, 100%; HSIL, 100%; LSIL, 77.6%; ASC-H, 70.1%; ASC-US, 40.1%; AGC, 25.0%; and NILM, 14.0%. By comparison, the ALTS hrHPV-positive rates by cytologic categories6 are as follows: HSIL+, 92.3%; LSIL, 83.3%; ASC-US, 48.9%; and NILM, 31.0% (Table 2).
|Cytologic diagnosis||All cases||Biopsied cases||ALTS cases*|
|n||Number of hrHPV+||Average age, y||n||Number of hrHPV+||n||Number of hrHPV+|
|Carcinoma||1 (<0.1)†||1 (100)†||52||1 (100)‡||1 (100)§|
|HSIL||21 (0.6)||21 (100)||33.9||17 (81.0)||17 (100)||246||227 (92.3)†|
|LSIL||134 (3.9)||104 (77.6)||30.2||84 (62.7)||68 (81.0)||630||525 (83.3)|
|ASC-H||147 (4.2)||103 (70.1)||35.7||94 (63.9)||75 (79.8)|
|ASC-US||2319 (66.7)||930 (40.1)||36.4||625 (27.0)||485 (77.6)||1134||555 (48.9)|
|AGC||16 (0.5)||4 (25.0)||40.9||11 (68.8)||4 (36.4)|
|NILM||838 (24.1)||117 (14.0)||35.7||75 (8.9)||22 (29.3)||1460||453 (31.0)|
|Total||3479||1532 (44.0)||35.9||907 (26.1)||672 (74.1)|
Of the 2319 ASC-US cases, 930 (40.1%) cases were hrHPV-positive, and 1389 (59.9%) cases were hrHPV-negative (Table 2). There were 625 (27%) biopsies available in the follow-up period. In 485 cases (77.6%) the biopsy followed a positive hrHPV result and in 140 cases (22.4%) the biopsy followed a negative hrHPV result. In the hrHPV-positive biopsy group, the following histologic results were obtained: Negative, 329 (67.8%); CIN I, 71 (14.6%); CIN II, 47 (9.7%); CIN II+, 85 (17.5%); CIN III+, 38 (7.8%). In the hrHPV-negative biopsy group, the following histologic results were obtained: Negative, 112 (80%); CIN I, 22 (15.7%); CIN II, 4 (2.9%); CIN II+, 6 (4.3%); CIN III+, 2 (1.4%) (Table 3). Both hrHPV-negative CIN III+ specimens were from a single patient with squamous cell carcinoma of the cervix.
|ASC-US biopsy diagnosis||hrHPV+||hrHPV−|
|Negative||329 (67.8)†||112 (80)|
|CIN I+||156 (32.2)||28 (20)|
|CIN II+||85 (17.5)||6 (4.3)|
|CIN III+||38 (7.8)||2 (1.4)|
Using a threshold of CIN II+, the sensitivity of the HC II/SP test is 93.4% (86.4%-96.9%), the specificity is 25.1%, the PPV is 17.5%, and the NPV is 95.7% (Table 4). In addition, 21 of 21 (100%) HSIL specimens, 104 of 134 (77.6%) LSIL cases, and 4 of 16 (25.0%) AGC cases tested for hrHPV were found to be positive during this period (Table 2).
|< CIN II+||400||134|
By comparison, the ALTS trial (enrollment data) using HC II/TP showed a sensitivity of 95% (92%–97%), PPV of 20%, and NPV of 99% using the same threshold of CIN II+ (Table 5). A specificity calculation was not identified in the ALTS study articles reviewed.
|MGH ASC-US||ALTS ASC-US|
|Sensitivity, %||93.4 (86.4-96.9)||95 (92-97)|
|Specificity, %||25.1 (21.6-28.9)||Not given|
hrHPV has been shown to be a necessary cause for virtually all cases of cervical cancer and its precursor lesions,1 and hrHPV testing has become an integral component of the standard of care for screening and patient management.3, 7 The hrHPV test has an important complimentary role in conjunction with the Pap test in guiding the appropriate triage of patients with highly prevalent ASC-US interpretations, and with NILM interpretations in the population older than 30 years. Given the potential increase in cervical cancer prevalence that might be caused by false-negative hrHPV results, and the potential for over treatment with false-positive hrHPV results under such management strategies, it is vital for laboratories to clinically validate their testing methods. The current standard for comparison for new hrHPV testing methods is the data from the ALTS trial because it uses an FDA-validated testing method (HC II/TP) and was a large, rigorous, multicenter study with comprehensive patient follow-up and pathologic adjudication.
Our preinitiation study showed 100% sensitivity for the detection of both LSIL and HSIL. Data from ALTS (and from our own ongoing larger study) suggest that a figure in the range of 80%-85% might be expected for LSIL. We have no specific explanation for the 100% sensitivity in this population, other than statistical “luck,” meaning we had a run of hrHPV-positive LSIL cases. Data from the overall study, in which hrHPV prevalence in the LSIL population was 78%, do not suggest increased cross-reactivity with low risk HPV types, or a nonspecific false-positivity, as a reason for the preinitiation study LSIL results.
A portion of the ALTS trial followed 3488 women referred for ASC-US, 1161 of whom were assigned to colposcopic triage by HC II/TP testing. On the basis of enrollment data, which is more comparable with the data in this study than is the longitudinal data portion of ALTS, HC II/TP using a threshold of CIN II+ for ASC-US cases has a sensitivity of 95.0% (CI of 92%-97%), a PPV of 20%, and a NPV of 99%. Using a threshold of CIN III+ slightly increases the sensitivity to 96.3%. On the basis of ALTS longitudinal data (2-year follow-up), the probability of having a CIN III+ biopsy after a negative hrHPV test was 1.4%.4–6 This parameter was not presented for the ALTS enrollment data, and is hence the only ALTS CIN III+ data point that can be utilized as a comparator to the present study findings.
By comparison, the present data examined 2319 cases with ASC-US using HC II/SP method, of which 625 had follow-up biopsies or hysterectomies. Using a threshold of CIN II+ for ASC-US cases, the present study shows strikingly similar results for sensitivity (93.4%), NPV (95.7%), and PPV (17.5%) for ASC-US (Tables 4 and 5). In particular, the probability of having a CIN III+ biopsy after a negative hrHPV test was identical to the ALTS longitudinal data point (1.4%) (Table 3). In addition, hrHPV tests performed on cases interpreted as either LSIL or HSIL showed high detection sensitivity in both the preinitiation and follow-up studies, indicating excellent analytic sensitivity for known abnormal cases.
Two of the false-negative hrHPV tests for CIN III+ were from a single patient with squamous cell carcinoma. Since carcinomas are more frequently found to have technical false-negative hrHPV results when compared with precursor lesions, this is not surprising.10 Of note, no documented CIN III cases were missed by HC II/SP in this study.
Our data are not exactly comparable to the ALTS data, since unlike the patients in the ALTS trial, our patients were in routine clinical management settings and hence, not all patients underwent colposcopy. In our study, 52.2% of hrHPV-positive patients underwent biopsy, and only a small portion of hrHPV-negative patients underwent the same (10.1%). The biopsy rate probably corresponds to the same percentage of colposcopic examinations, since the protocol at MGH includes an endocervical curettage on all negative colposcopies. In routine practice, however, given a negative hrHPV test, only the highest risk patients on the basis of history and symptoms are likely to receive a colposcopic examination and biopsy. Note that the prevalence of CINII+ in ASC-US patients is 14.6% in this study, compared with 11.4% in the ALTS trial, consistent with a “higher risk” biopsy pool population. The hrHPV-positive rate of 77.6% within the ASC-US cases that were ultimately brought to biopsy also supports this higher risk population (Table 2). Such a selection bias would exaggerate the apparent proportion of CIN identified within the hrHPV-negative group and enhance the significance of the validation data as follows: if all hrHPV-negative ASC-US patients had undergone colposcopy, as was the case in the ALTS trial, the number of double negative hrHPV/biopsy patients would be increased at the CIN II+ threshold, and the prevalence would therefore be decreased. This, in turn, would increase the NPV of the HC II/SP method above the present value of 95.7% (relative to the NPV of 99% in ALTS).
Similarly, if more hrHPV-positive patients had been biopsied in this study, the PPV might have increased, but this parameter is not as clinically significant as the NPV. A high NPV allows one to confidently rule out high-grade disease in a negative test,9 and is critical because in the newest screening guidelines, double negative Pap and hrHPV tests permit a 3-year interval until the next screening in women older than 30 years. A PPV greater than the current rate of 17.5% might theoretically increase its usefulness in confirming hrHPV infection; however, although HPV infection is necessary to cause cervical cancer, it is not sufficient for the development of a high-grade lesion, and further testing such as colposcopy would in any case be required. Thus, increasing the PPV would potentially increase the cost-effectiveness of current patient management guidelines, but it would not be expected to actually improve cancer prevention.
Another characteristic of this study to note is that most of our data are compared with the ALTS enrollment data, because the ALTS study design is more similar to the present one, which being retrospective, focused on the follow-up of specimens. The longitudinal data portion of ALTS, on the other hand, was a prospective study and examined the cumulative risk over 2 years for patients, and is therefore not directly comparable.
Mindful of the above-noted differences between this study and ALTS, the preinitiation validation data showing similar results between SP and STM collection methods, and the strikingly similar sensitivity, NPV, and PPV of our follow-up study results when compared with the ALTS data (1.4% CIN III+ and 4.3% CIN II+ rate in the hrHPV-negative ASC-US population, and 100% hrHPV-positive rate for all tested cases of cytologic HSIL), provide substantial validation for the use of HC II by the SP method.
Laboratories that use non-FDA–approved hrHPV tests, including in situ hybridization and polymerase chain reaction (PCR) based methods, need to perform clinical validation of their assays and should achieve substantial comparability of their results to the benchmarks of the ALTS trial. If such measures show equivalent clinical performance, then accepted triage guidelines using hrHPV results can similarly be implemented for all patients. However, any method shown to perform in a less than equivalent manner, particularly with regard to sensitivity or NPV, would require specific alteration of the triage guidelines to prevent tests with different performance characteristics from inappropriately triaging patients in either direction. In addition, cost-effectiveness data generated from ALTS data would require modification for test methodologies with substantially different performance parameters. In the present case, because SP is a less expensive method than is TP, using HC II with SP should maintain the cost-effectiveness of hrHPV testing that has been shown previously.11
Note Added in Proof
Additional clinical review of the ASC-US/HPV-negative study population that did not receive an initial colposcopic examination and biopsy was performed via chart review. The follow-up period was up to 2 years and all additional cytology and biopsy results obtained were recorded. As expected, the preponderance of follow-up was cytologic. This review yielded additional cervical results on 876 patients (70% of the 1249 patients not having immediate colposcopy and biopsy). In this review there were 5 additional cases of CIN2+ identified (2 CIN3 and 3 CIN2). This represents a 0.6% prevalence of CIN2+ in this population, compared with the 4.3% prevalence in the biopsied ASC-US/HPV-negative population reported in this study. This finding supports the assertion made in the discussion that the biopsied ASC-US/HPV negative population represents a higher clinical risk pool based on other factors known to the treating clinicians. Modeling the overall data to a lower CIN2+ prevalence (0.6%) in the entire. ASC-US/HPV negative population not receiving an initial colposcopy and biopsy (1249 patients) yields a NPV of 99.1%. Modeling more conservatively with a rate 50% higher (0.9%) yields a NPV of 98.8%. Both of these NPV results (based on the extended data collection and assumptions as noted) are equivalent to data from ALTS.
- 92005: 1.Making a valid point about HPV tests. CAP Today, Sept