SEARCH

SEARCH BY CITATION

Keywords:

  • gold standard;
  • acetic acid-aided visual inspection;
  • colposcopy;
  • cervical neoplasia

Abstract

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References
  7. Appendix

As acetic acid-aided visual inspection (VIA) and colposcopic-directed biopsy miss small ≥cervical intraepithelial neoplasia (CIN) 2, inflation of sensitivity of VIA may occur when colposcopic-directed biopsy is the gold standard for ≥CIN 2. To determine whether such inflation occurs, we reviewed 375 women with ≥CIN 2 from the Shanxi Province Cervical Cancer Screening Study II. These women had positive self or physician-collected tests for high-risk human papillomavirus or abnormal cervical cytology and had VIA followed by colposcopy with directed biopsy and endocervical curettage (ECC). If a cervical quadrant had no lesion, a random biopsy at the squamocolumnar junction within that quadrant was obtained. Sensitivity of colposcopic-directed biopsy was higher for ≥CIN 2 involving 3–4 cervical quadrants (81.3%) than for ≥CIN 2 involving 0–2 quadrants (49.0%, p < 0.001). Sensitivities of VIA, cytology of ≥ASC-US, ≥LSIL, and ≥HSIL were higher for ≥CIN 2 involving 3–4 quadrants than for ≥CIN 2 involving 0–2 quadrants. When a colposcopic-directed biopsy gold standard was compared with that of a 5-biopsy standard (which included ≥CIN 2 from colposcopic-directed biopsy, random biopsy, or ECC), the sensitivity for ≥CIN 2 of VIA was inflated by 20.0% (65.9% vs. 45.9%, p < 0.001). Sensitivities of other screening tests were not affected. Similar inflation of sensitivity of VIA was found with an endpoint of ≥CIN 3 (70.4% vs. 52.0%, p = 0.0013). Inflation of sensitivity of VIA depended upon agreement between colposcopic-directed biopsy and the screening tests as measured by kappa. Studies of VIA that used colposcopic-directed biopsy as the gold standard require reevaluation. © 2007 Wiley-Liss, Inc.

Though loop excision of the transformation zone is occasionally employed,1 biopsy of colposcopic-detected lesions2, 3, 4 is the most common gold standard for the diagnosis of cervical intraepithelial neoplasia (CIN) 2 or worse utilized in cervical cancer screening studies. In the Shanxi Province Cervical Cancer Screening Studies (SPOCCS) I and II,5, 6 we conducted population-based cervical cancer screening studies utilizing a 5-biopsy gold standard for CIN 2 or worse. In these studies, at colposcopy, the cervix was divided into 4 quadrants by 2 perpendicular lines drawn from 12 to 6 o'clock and from 9 to 3 o'clock. Each cervical quadrant was graded separately by colposcopic impression as negative (no lesion), low (suggestive of human papillomavirus (HPV) or CIN 1), high (suggestive of CIN 2 or 3), or cancer. Colposcopic-detected lesions were biopsied. Cervical quadrants without colposcopic-detected lesions had random biopsies at the squamocolumnar junction at 2, 4, 8 or 10 o'clock. An endocervical curettage (ECC) was then obtained. A bronchoscopic biopsy instrument was used to obtain virtually painless 2-mm biopsies. With this protocol, patients undergoing colposcopy had at least 4 cervical biopsies (some cervical quadrants with colposcopic-detected lesions had more than 1 biopsy) and an ECC.

In SPOCCS I, 1,997 women were screened with acetic acid-aided visual inspection (VIA), cervical cytology, self-test for high-risk HPV with specimen collected with Dacron swab and physician-collected test for high-risk HPV (specimen obtained from liquid-based cytology transport media). All 1,997 women underwent colposcopy with biopsy. Of 83 women with CIN 2 or worse diagnosed by the 5-biopsy gold standard in SPOCSS I, 62 (74.7%) were diagnosed by a colposcopic-directed biopsy and 21 (25.3%) were diagnosed by random biopsy in a colposcopically negative cervical quadrant and/or by a positive ECC.7 In SPOCCS II, 8,497 women were screened with VIA, cervical cytology, self-test for high-risk HPV with specimen collected by a conical-shaped brush, and physician-collected high-risk HPV with specimen collected from the endocervix with a conical-shaped brush.6 A total of 3,139 women had colposcopy with the 5-biopsy standard described above. Of 364 women in SPOCCS II with adequate colposcopy and CIN 2 or worse diagnosed by the 5-biopsy standard, 208 (57.1%) were diagnosed by a colposcopic-directed biopsy and 156 (42.9%) were diagnosed by a random biopsy in a colposcopically negative cervical quadrant and/or by a positive ECC.8

The use of colposcopic-directed biopsy as the gold standard in cervical cancer screening studies may underestimate the prevalence of CIN 2 or worse. However, it is reasonable to use colposcopic-directed biopsies if diagnosis is conditionally independent of the screening test results, given disease status (presence or absence of CIN 2 or worse).9 In January of 2006, in a review of the SPOCCS I data, we reported a higher sensitivity of different diagnostic tests (i.e., gold standard colposcopic-directed biopsy, VIA and cytology of high grade squamous intraepithelial lesion (HSIL) or greater) for the detection of CIN 2 or worse involving 3 or 4 quadrants of the cervix than for CIN 2 or worse involving 0–2 cervical quadrants (zero quadrants is diagnosis solely by ECC or by directed biopsy of vaginal vault lesion).7 Given that the gold standard of colposcopic-directed biopsy and the screening tests of VIA and cytology of ≥HSIL appear to be interdependent, it was expected that the sensitivity of VIA and cytology of ≥HSIL for CIN 2 or worse would likely be inflated due to diagnostic inaccuracies when colposcopic-directed biopsy was used as the gold standard.10

In SPOCCS I, the sensitivity of VIA for the detection of 62 cases of CIN 2 or worse with a colposcopic-directed biopsy gold-standard (79.0%) was not significantly different from the sensitivity for detection of 83 cases of CIN 2 or worse with the 5-biopsy gold-standard (71.1%) (p > 0.25).7 Similarly, the sensitivity of cytology of ≥HSIL for CIN 2 or worse with a colposcopic-directed biopsy gold-standard (85.5%) was not significantly different from that found with the 5-biopsy gold-standard (75.9%) (p > 0.1).7 This failure to achieve significance in difference of sensitivities for CIN 2 or worse was attributed to the low power associated with the small number of women with CIN 2 or worse (N = 83). Further, due to the limited number of women with CIN 3 or cancer (N = 42) in SPOCCS I, CIN 2 or worse was used as the study endpoint despite CIN 2 being a generally poor-defined entity, which may or may not progress to CIN 3 or invasive cervical cancer.1

To address the problems of low power and low numbers of CIN 3 or cancer in our prior publication, we review here the larger SPOCCS II study results to determine whether colposcopy or the screening tests missed CIN 2 or worse involving 0–2 quadrants of the cervix and whether any of the screening tests had significant inflation of sensitivity for CIN 2 or worse when a gold standard of colposcopic-directed biopsy was compared with that of a 5-biopsy standard. The analysis was repeated using CIN 3 or cancer as the endpoint.

Methods

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References
  7. Appendix

The 375 women within the SPOCCS II that were diagnosed with CIN 2 or worse were reviewed. The study population and methods are described in detail in a previous publication.6 In brief, 8,497 women residing in Shanxi Province, China were screened with 4 different screening tests for cervical neoplasia: self and physician-collected tests for high-risk HPV, cervical cytology and VIA. The self-tests for high-risk HPV were obtained in the local villages with a conical-shaped brush (Cervical Sampler®, Digene Corp., Gaithersburg, MD) that was placed high in the vagina and rotated 3–4 times. The brush was then placed in Special Transport Media (STM®, Digene Corp.) and tested for a mixture of 13 intermediate and high-risk types of HPV with the second-generation hybrid capture microplate-based HPV test (HC II test, Digene, Corp.). Women were recalled to a central clinic where they had physician-collected high-risk HPV, cervical cytology, VIA, and in selected cases, colposcopy and biopsy. Physician-collected cervical specimens were obtained from the endocervix with a conical-shaped brush like that used to obtain the self-tests for high-risk HPV and then tested for high-risk HPV with HC II. A cutoff value of 1.0 pg HPV DNA was defined as positive. Cervical specimens for liquid-based cytology were obtained with the Rovex brush and placed in CytoRich® transport media. Liquid-based cytology slides were prepared by the AutoCyte® Prep method (AutoCyte®, TriPath Imaging, Burlington, NC) and interpreted using the Bethesda Classification system by cytopathologists in Beijing who were blinded to the results of the other tests. Visual inspection was performed 1 min after 5% acetic acid was applied to the cervix. VIA was defined as positive when there was a white lesion that might or might not abut the squamocolumnar junction or a friable mass with an irregular surface.

Colposcopy and biopsy with the 5-biopsy gold standard (as described in the introduction) was performed on 3,139 women. Three thousand and sixty of the 3,139 (97.5%) women had positive self or physician-collected high-risk HPV tests or abnormal cervical cytology; 16 (0.5%) had negative self and physician-collected high-risk HPV tests and normal cervical cytology but had abnormal VIA; and 63 (2.0%) had negative self and physician-collected HPV tests, negative cervical cytology and normal VIA. Biopsies were processed and interpreted by pathologists without knowledge of the results of all other diagnostic tests. The final diagnosis was based on the worst biopsy (colposcopic-directed, random in a cervical quadrant without a lesion, or ECC) obtained. Although some subjects underwent cervical conization or hysterectomy as treatment for cervical neoplasia, results of these procedures were not considered in determining the final diagnosis. The Institutional Review Board for human subjects of The Cleveland Clinic Foundation approved this review of previously collected data.

The sensitivities of the self and physician-collected tests for high-risk HPV, VIA, cervical cytology with abnormal defined as ≥atypical squamous cells of uncertain significance (ASC-US), cervical cytology with abnormal defined as ≥low grade squamous intraepithelial lesion (LSIL), cervical cytology with abnormal defined as ≥HSIL, and colposcopy were calculated for CIN 2 or worse involving 0–2 quadrants of the cervix and for CIN 2 or worse involving 3–4 quadrants of the cervix. Differences in proportions were tested with the χ2 statistic utilizing STAT 1 statistical software (Hewlett–Packard, Palo Alto, CA). Sensitivities of the screening tests were calculated and compared for 2 chosen gold standards: (i) colposcopic-directed biopsy and (ii) the 5-biopsy standard. The correlation between colposcopic-directed biopsy and the screening tests was tested with the kappa statistic utilizing SAS version 9.1.3 (SAS Institute, Cary, NC). The analysis was then repeated using CIN 3 or cancer as the endpoint.

Results

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References
  7. Appendix

In the Shanxi Province Cervical Cancer Screening Study II, the mean age of the 8,497 screened women was 40.9 years (range 27–56). The mean number of pregnancies was 3.05 (range 0–13). Though the SPOCCS II protocol was to recall women to a central clinic within 3 months of their self-test for high-risk HPV, there were delays secondary to funding and transportation such that the time between the self-test and the other screening tests and colposcopy was between 4 and 16 months.

The distribution of colposcopy and biopsy and the rates of CIN 2 or worse within subgroups based on result of physician-collected high-risk HPV test and cervical cytology within SPOCCS II are shown in Table I. Three thousand five hundred and sixty-two of 8,497 screened women had positive self or physician-collected high-risk HPV tests or cervical cytology of ≥ASC-US and, by protocol should have had colposcopy and biopsy. Of these 3,562 women, 2,810 had positive physician-collected high-risk HPV tests or cervical cytology of ≥ASC-US and 752 had positive self-test for high-risk HPV but had negative physician-collected high-risk HPV tests and normal cervical cytology. Of the 3,562 that should have had colposcopy and biopsy, 3,060 had colposcopy with biopsy and 502 did not. Four hundred and sixty of the 502 women that failed to have colposcopy with biopsy were evaluated late in the study and had negative self and physician-collected high-risk HPV tests and cervical cytology of ASC-US. The remaining 42 women that failed to have colposcopy and biopsy were self-test high-risk HPV negative with positive physician-collected high-risk HPV or abnormal cervical cytology that failed to return to the clinic for a second visit. Seventy-nine women with negative self and physician-collected high-risk HPV tests and normal cervical cytology, who by protocol should not have had colposcopy with biopsy, had the procedure. Sixteen of these 79 women had colposcopy solely because their VIA was positive. The remaining 63 women with all 4 negative screening tests were randomly selected for colposcopy and biopsy. None of these 79 women with negative self and physician-collected high-risk HPV tests and normal cytology had CIN 2 or worse.

Table I. Distribution of Colposcopy and CIN 2 or Worse Among 8,497 Women in Shanxi Province Cervical Cancer Screening Program (SPOCCS) II
 ColposcopyPathology with 5-biopsy gold standardTotal
# No colpo# colpo# CIN 1 or less# CIN 2 or worse
  1. Colpo is colposcopy, CIN is cervical intraepithelial neoplasia, VIA is acetic acid-aided visual inspection, HPV is physician-collected high-risk human papillomavirus, ASC-US is atypical squamous cells of uncertain significance, Neg is negative, and Pos is positive.

HPV Pos, Cytology ≥ ASC-US
VIA Neg8776622162784
VIA Pos4271116159275
Total121,0477383211,059
HPV Pos, Cytology Normal
VIA Neg2584884033873
VIA Pos18274983
Total2693091442956
HPV Neg, Cytology ≥ ASC-US
VIA Neg4403007346740
VIA Pos243151455
Total46433178510795
HPV Neg, Cytology Normal
VIA Neg4,5327595,28925,291
VIA Pos324723960396
Total4,8568315,68525,687
Total5,3583,1398,1223758,497

As shown in Table I, the rate of CIN 2 or worse among women that had colposcopy and biopsy was 321/1047 (30.7%) for positive physician-collected high-risk HPV tests and positive cervical cytology, 42/930 (4.5%) for positive physician-collected high-risk HPV tests with negative cervical cytology, 10/331 (3.0%) for negative physician-collected high-risk HPV tests with cervical cytology of ≥ASC-US and 2/831 (0.2%) for negative physician-collected high-risk HPV tests and normal cervical cytology. Three hundred and fifty-three of 809 VIA positive women did not have colposcopy and biopsy. Three hundred and twenty-four of these 359 VIA positive women that did not have colposcopy and biopsy had negative physician-collected high-risk HPV and normal cervical cytology, 24 had negative physician-collected high-risk HPV and cervical cytology of ≥ASC-US, 1 had positive physician-collectedhigh-risk HPV and normal cervical cytology and 4 had positive physician-collected high-risk HPV and cervical cytology of ≥ASC-US. By multiplying the number of VIA positive women that did not have colposcopy by the rate of CIN 2 or worse in each of the 4 subgroups and then adding, we estimate that about (324)(0.002) + (24) (0.03) + (1)(0.045) + 4(0.307) = 2.06 women with CIN 2 or worse and positive VIA were not detected.

Three hundred and seventy-five of the 8,497 (4.4%) women participating in SPOCCS II had CIN 2 or worse. The distribution of CIN 2 or worse, type of biopsy (colposcopic-directed with or without positive ECC vs. random with or without positive ECC or solely by positive ECC) and the number of cervical quadrants involved with CIN 2 or worse are presented in Table II. Of the 375 women with CIN 2 or worse, 173 had CIN 2, 181 had CIN 3 and 21 had invasive cervical cancer. Cervical intraepithelial neoplasia 2 or worse involved only the ECC in 19 women, the vaginal wall in 1 woman, 1 quadrant of the cervix in 177, 2 quadrants in 103, 3 quadrants in 47 and 4 quadrants in 28 women. Involvement of 3–4 cervical quadrants was found in 10 of 173 (5.8%) women with CIN 2, 51/181 (28.2%) women with CIN 3 and 14/21 (66.7%) women with invasive cancer (5.8% vs. 28.2% vs. 66.7%, p < 0.001). Colposcopic-directed biopsy with or without ECC showing CIN 2 or worse resulted in a diagnosis of CIN 2 in 86/173 (49.7%) women, CIN 3 in 106/181 (58.6%) women and invasive cancer in 16/21 (76.2%) women (49.7% vs. 58.6% vs. 76.2%, p < 0.05). Two hundred and eight of 375 (55.5%) women with CIN 2 or worse were diagnosed by a colposcopic-directed biopsy with or without an ECC showing CIN 2 or worse, 148 were diagnosed by a random biopsy from a cervical quadrant without a colposcopic-detected lesion with or without an ECC showing CIN 2 or worse and 19 were diagnosed solely by an ECC showing CIN 2 or worse. Of the 19 women with CIN 2 or worse diagnosed solely by a positive ECC, 7 had CIN 2, 11 had CIN 3 and 1 had invasive cancer.

Table II. Type of Biopsy and Number of Cervical Quadrants Involved Among 375 Women with CIN 2 or Worse
Pathology diagnosisTypeNumber of cervical quadrants involved with CIN 2 or worse
01234Total
  • 1

    One woman had CIN 2 diagnosed on a colposcopic-directed vaginal vault biopsy. CIN is cervical intraepithelial neoplasia, ECC is endocervical curettage, Colpo is diagnosis by colposcopic-directed biopsy with or without positive ECC, Random/ECC is diagnosis by random biopsy with or without positive ECC or solely by positive ECC.

CIN 2Colpo1*55235286
Random/ECC762152187
Total81173873173
CIN 3Colpo025402318106
Random/ECC1134209175
Total1159603219181
SCCColpo0127616
Random/ECC103105
Total1158621
TotalColpo1181653526208
Random/ECC199638122167
Total201771034728375

Two hundred and two of the 8,497 (2.4%) women were diagnosed with CIN 3 or cancer. The type of biopsy and number of quadrants involved with CIN 3 or cancer are presented in Table III. When the endpoint is CIN 3 or cancer, cervical quadrants and ECCs that were interpreted as normal, CIN 1 or CIN 2 were coded as negative. Cervical intraepithelial neoplasia 3 or cancer was present only in the ECC in 24 women, only in a vaginal vault biopsy in 1 woman, involved 1 quadrant of the cervix in 97, 2 quadrants in 53, 3 quadrants in 16 and 4 quadrants in 11 women. One hundred and fifteen of 202 (56.9%) women with CIN 3 or cancer were diagnosed by a colposcopic-directed biopsy with or without an ECC showing CIN 3 or cancer, 63/202 (31.2%) were diagnosed by a random biopsy with or without an ECC showing CIN 3 or cancer, and 24/202 (11.9%) were diagnosed solely by an ECC showing CIN 3 or cancer. Among the 24 women with CIN 3 or cancer diagnosed solely by a positive ECC, there are 11 with CIN 3 and 1 with invasive cancer that appear in the 19 women with ECCs showing CIN 2 or worse (see first column of Table II). In addition, there are 7 women that had an ECC of CIN 3 with a random biopsy of CIN 2 and 5 women with an ECC of CIN 3 and a colposcopic-directed biopsy of CIN 2.

Table III. Type of Biopsy and Number of Cervical Quadrants Involved Among 202 Women with CIN 3 or Cancer
PathologydiagnosisBiopsy typeNumber of cervical quadrants involved with CIN 3 or worse
01234Total
  • 1

    One woman had CIN 2 in a cervical quadrant with negative ECC and was diagnosed with CIN 3 solely on the basis of a vaginal vault biopsy. CIN is cervical intraepithelial neoplasia, ECC is endocervical curettage, Colpo is diagnosis by colposcopic-directed biopsy with or without positive ECC, Random/ECC is diagnosis by random biopsy with or without positive ECC or solely by positive ECC.

CIN 3Colpo1*50347799
Random/ECC2344114082
Total249445117181
SCCColpo0255416
Random/ECC113005
Total1385421
TotalColpo1152391211115
Random/ECC2445144087
Total2597531611202

As shown in Table IV, colposcopic-directed biopsy is more likely to detect CIN 2 or worse when the CIN 2 or worse involves 3–4 quadrants of the cervix (81.3%) than when the CIN 2 or worse involves 0–2 quadrants of the cervix (49.0%) (p < 0.001); similar results were found for an endpoint of CIN 3 or cancer [85.2% for 3–4 quadrants vs. 52.6% for 0–2 quadrants, (p = 0.0014)]. With an endpoint of CIN 2 or worse, the sensitivities of VIA, cervical cytology of ≥ASCUS, cervical cytology of ≥LSIL and cervical cytology of ≥HSIL are higher for lesions involving 3–4 quadrants than for those involving 0–2 quadrants. In contrast, the sensitivities for CIN 2 or worse of self-test and physician-collected test for high-risk HPV are not functions of lesion size. With an endpoint of CIN 3 or cancer, the sensitivity of cervical cytology of ≥HSIL is higher for lesions involving 3–4 quadrants than for lesions involving 0–2 quadrants while the sensitivities of the other screening tests are not functions of lesion size.

Table IV. Sensitivity for High-Grade (CIN 2 or Worse or CIN 3 or Cancer) Lesions by Different Screening Methods, Stratified by Number of Cervical Quadrants Involved
Screening testCIN 2 or worseCIN 3 or cancer
0–2 quadrants of cervix involved (N = 300) (%)3–4 quadrants of cervix involved (N = 75) %p0–2 quadrants of cervix involved (N = 175) %3–4 quadrants of cervix involved (N = 27) %p
  1. CIN is cervical intraepithelial neoplasia, ASC-US is atypical squamous cells of uncertain significance, LSIL is low-grade squamous intraepithelial lesion, HSIL is high-grade squamous intraepithelial lesion, HPV self-test is high-risk human papillomavirus obtained from the upper vagina, and HPV physician is high-risk human papillomavirus obtained from the endocervix.

Colposcopic directed biopsy49.081.3<0.00152.685.20.0014
Visual inspection positive41.762.7<0.00152.051.90.99
Cytology
 ≥ASC-US85.396.00.01393.796.30.60
 ≥LSIL74.394.7<0.00186.996.30.16
 ≥HSIL49.076.0<0.00164.688.90.01
HPV physician positive96.398.70.3097.196.30.81
HPV self-test positive86.392.00.1886.388.90.71

As shown in Table V, the sensitivity for CIN 2 or worse of VIA is 65.9% when determined by a gold standard of colposcopic-directed biopsy but only 45.9% when determined by the 5-biopsy gold standard. This 20.0% difference in sensitivity for CIN 2 or worse of VIA is significant (p < 0.001). Similarly, the sensitivity for CIN 3 or cancer of VIA is 70.4% when the gold standard is colposcopic-directed biopsy but only 52.0% with the 5-biopsy gold standard. This 18.4% difference is significant (p = 0.001). With neither endpoint (CIN 2 or worse or CIN 3 or cancer) was inflation of screening tests other than VIA found when the gold standard was changed from colposcopic-directed biopsy to the 5-biopsy standard. As shown in Table VI, though some of the differences in specificity of screening tests are statistically significant, even the greatest difference of 1.4% seen for cytology of ≥LSIL with an endpoint of CIN 2 or worse is not of clinical significance.

Table V. Comparison of Sensitivity of Screening Tests and Colposcopy with 5-Biopsy and Colposcopic-Directed Biopsy Gold Standards with Endpoints of CIN 2 or Worse and CIN 3 or Cancer
Screening testCIN 2 or worseCIN 3 or cancer
Sensitivity with 5-biopsy gold standard (≥CIN 2, N = 375) (%)Sensitivity with colposcopic-directed biopsy standard (≥CIN 2, N = 208) (%)pSensitivity with 5-biopsy gold standard (≥CIN 3, N = 202) (%)Sensitivity with colposcopic-directed biopsy standard (≥CIN 3, N = 115) (%)p
  1. CIN is cervical intraepithelial neoplasia, ASC-US is atypical squamous cells of uncertain significance, LSIL is low-grade squamous intraepithelial lesion, HSIL is high-grade squamous intraepithelial lesion, HPV self-test is high-risk human papillomavirus obtained from the upper vagina, and HPV physician is high-risk human papillomavirus obtained from the endocervix.

Colposcopic directed biopsy55.5100.0<0.000156.9100.0<0.0001
Visual inspection positive45.965.9<0.000152.070.40.0013
Cytology
 ≥ASC-US88.290.40.2694.195.70.55
 ≥LSIL78.482.70.2488.190.40.53
 ≥HSIL54.458.70.3167.872.20.42
HPV physician positive96.897.60.5797.097.40.85
HPV self-test positive87.589.40.4586.692.20.14
Table VI. Comparison of Specificity of Screening Tests with 5-Biopsy and Colposcopic-Directed Biopsy Gold Standards with Endpoints of CIN 2 or Worse and CIN 3 or Cancer
Screening testCIN 2 or worseCIN 3 or cancer
Specificity with 5-biopsy gold standard (≤CIN 1, N = 8,122) (%)Specificity with colposcopic-directed biopsy standard (≤CIN 1, N = 8289) (%)pSpecificity with 5-biopsy gold standard (≤CIN 2, N = 8295) (%)Specificity with colposcopic-directed biopsy standard (≤CIN 2, N = 8382) (%)p
  1. CIN is cervical intraepithelial neoplasia, ASC-US is atypical squamous cells of uncertain significance, LSIL is low-grade squamous intraepithelial lesion, HSIL is high-grade squamous intraepithelial lesion, HPV self-test is high-risk human papillomavirus obtained from the upper vagina, and HPV physician is high-risk human papillomavirus obtained from the endocervix.

Visual inspection positive92.291.90.9791.591.30.65
Cytology
 ≥ASC-US81.379.90.0379.979.10.23
 ≥LSIL93.291.8<0.0191.991.10.06
 ≥HSIL98.797.7<0.0197.997.2<0.01
HPV physician positive79.778.10.0278.177.30.23
HPV self-test positive77.276.00.0675.975.30.39

We examined the relationship between the concordances of the screening tests with diagnosis by colposcopic-directed biopsy (as determined by the kappa statistic). The 2-by-2 tables correlating the screening tests with colposcopic-directed biopsy using CIN 2 or worse as the endpoint are shown in Table VII; similar data using CIN 3 or cancer as the endpoint is shown in Table VIII. When the endpoint is CIN 2 or worse, VIA (p < 0.0001) and cytology of ≥LSIL (p = 0.024) have significant correlation with diagnosis by colposcopic-directed biopsy, while with an endpoint of CIN 3 or cancer, VIA (p < 0.0001) and self-test for high-risk HPV (p = 0.008) have significant correlations with diagnosis by colposcopic-directed biopsy.

Table VII. Correlation of Screening Tests and Diagnosis by Colposcopic-Directed Biopsy using CIN 2 or Worse as Endpoint
Screening testTest posTest negKappap
Colpo posColpo negColpo posColpo neg
  1. Acetic acid-aided visual inspection is visual inspection, colpo is diagnosis of CIN 2 or worse by colposcopic-directed biopsy with or without endocervical curettage of CIN 2 or worse, cervical cytology of ≥atypical squamous cells of uncertain significance is ≥ASC-US, cervical cytology of ≥low-grade squamous intraepithelial lesion is ≥LSIL, cervical cytology of ≥high-grade squamous intraepithelial lesion is ≥HSIL, positive is pos, negative is neg, and HPV is high-risk human papillomavirus.

Visual inspection13735711320.440<0.0001
≥ASC-US18814020270.0710.057
≥LSIL17212236450.1020.024
≥HSIL1228286850.0950.065
HPV physician-collected pos203160570.0200.329
HPV self-test pos18614222250.0470.202
Table VIII. Correlation of Screening Tests and Diagnosis by Colposcopic-Directed Biopsy using CIN 3 or Cancer as Endpoint
Screening testTest posTest negKappap
Colpo posColpo negColpo posColpo neg
  1. Acetic acid-aided visual inspection is visual inspection, colpo is diagnosis of CIN 3 or worse by colposcopic-directed biopsy with or without endocervical curettage of CIN 3 or worse, cervical cytology of ≥atypical squamous cells of uncertain significance is ≥ASC-US, cervical cytology of ≥low-grade squamous intraepithelial lesion is ≥LSIL, cervical cytology of ≥high-grade squamous intraepithelial lesion is ≥HSIL, positive is pos, negative is neg, and HPV is high-risk human papillomavirus.

Visual inspection812434630.423<0.0001
≥ASC-US11080570.0410.271
≥LSIL1047411130.0590.242
≥HSIL835432330.1040.128
HPV physician-collected pos11284330.0100.728
HPV self-test pos106699180.1400.008

As the kappa statistic measuring correlation of screening tests and colposcopic-directed biopsy and inflation of sensitivity secondary to a correlated gold standard are both functions of the 4 numbers in the 2-by-2 tables seen in Tables VII and VIII, a mathematical relationship between the 2 exists. The equation for the ratio of inflation of sensitivity of screening tests secondary to a correlated gold standard divided by kappa for the correlation of the screening tests with diagnosis by colposcopic-directed biopsy is derived in the Appendix.

Discussion

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References
  7. Appendix

In our previous report, we concluded that, as cytology of ≥HSIL and VIA missed CIN 2 or worse involving 0–2 quadrants of the cervix and colposcopic-directed biopsy missed CIN 2 or worse involving 0–2 quadrants of the cervix, inflation of sensitivity for CIN 2 or worse of these 2 screening tests was likely to occur when a gold standard of colposcopic-detected biopsy was employed.7 We attributed our failure to detect significant inflation of sensitivity for CIN 2 or worse of these 2 screening tests to a lack of power. Our current review of roughly 4 times as many women confirms that significant inflation of sensitivity of correlated screening tests occurs when colposcopic-directed biopsy is the gold standard for CIN 2 or worse and provides more detail in explaining why this occurs. In contrast to our prior conclusion, lesion size is not the important factor in determining inflation of sensitivity of CIN 2 or worse (or CIN 3 or cancer) when colposcopic-directed biopsy is the gold standard. This is shown (Tables IV and V) for CIN 2 or worse in which, like colposcopy, 4 of the screening tests missed more CIN 2 or worse involving 0–2 cervical quadrants, yet only 1 of the 4 (VIA) had significant inflation of sensitivity when the gold standard of colposcopic-directed biopsy was compared to a 5-biopsy standard. It is even clearer for CIN 3 or cancer where colposcopy and cytology of ≥HSIL missed more CIN 3 or cancer involving 0–2 cervical quadrants, yet only VIA had significant inflation of sensitivity when the gold standard of colposcopic-directed biopsy was compared to that of a 5-biopsy standard.

As shown by the derivation of a mathematical relationship, agreement between the screening test and colposcopic-directed biopsy (as measured by kappa) is the significant factor in predicting inflation of sensitivity of CIN 2 or worse secondary to using colposcopic-directed biopsy as the gold standard. It is not enough that the gold standard of colposcopic-directed biopsy and the screening test both miss more lesions involving 0–2 quadrants of the cervix. For inflation of sensitivity to occur, the screening test and the gold standard must miss at least some of the exact same (whether small or large) lesions.

Repeating the analysis to determine the factors responsible for inflation of sensitivity of screening tests secondary to inaccurate gold standard with an endpoint of CIN 3 or cancer was done not only because CIN 2 is not a reliable indicator of neoplsia,1 but also because lesion size increases with increasing grade of CIN. As shown in Table II, 3–4 quadrants of the cervix were involved with CIN 2 or worse in only 5.8% of women when the worst biopsy was CIN 2 compared with 28.1% when the worst biopsy was CIN 3 and 66.7% when the worst biopsy was invasive cancer. As our initial hypothesis was that inflation of sensitivity of screening tests secondary to the inaccurate gold standard of colposcopic-directed biopsy would be seen when the gold standard and the screening test both missed small lesions, testing the hypothesis with an endpoint of CIN 3 or cancer seemed prudent. When the analysis was repeated with an endpoint of CIN 3 or cancer, there was significant correlation of VIA with colposcopic-directed biopsy (p < 0.001) and there was significant inflation of sensitivity of VIA when a colposcopic-directed gold standard was compared with a 5-biopsy standard (18.4%, p = 0.0013).

As 3,139 of the 8,497 participants in SPOCCS II and 456 of the 809 women with positive VIA in SPOCCS II had colposcopy and biopsy to verify the diagnosis, the possibility of verification bias must be considered. The magnitude of verification bias in a cervical cancer screening study is low if the number of women with CIN 2 or worse among those that did not have colposcopy and biopsy is small. To limit cost and still minimize verification bias, in SPOCCS II, colposcopy and biopsy was planned for the 3,562 women that had positive self or physician-collected high-risk HPV tests or abnormal cervical cytology. Verification bias could have been minimized without referring self-test high-risk HPV positive yet physician-collected high-risk HPV negative and cytology negative women for colposcopy and biopsy as virtually all women with CIN 2 or worse have either a positive physician-collected high-risk HPV test or an abnormal cervical cytology. Data to support this assertion comes from the current trial in which there were only 2 cases of CIN 2 or worse among 831 women that had negative physician-collected high-risk HPV tests and normal cervical cytology, from SPOCCS I where there were no cases of CIN 2 or worse among 1,280 women with negative physician-collected high-risk HPV tests and normal cervical cytology,5 and from the recent review conducted by Arbyn et al.11 Verification bias in SPOCCS II is greater than originally planned as 502 of the 3,562 women that should have had colposcopy and biopsy did not have this evaluation. Though most (460) of the 502 women that did not have colposcopy came from the subgroup that had negative self and physician-collected high-risk HPV tests and had ASC-US cytology in which the risk of CIN 2 or worse is about 1%,12 42 of these women had other constellations of positive high-risk HPV tests and/or abnormal cervical cytology in which the risk of CIN 2 or worse is higher. For the analysis of inflation of sensitivity of VIA in particular, there must be few cases of CIN 2 or worse among the 353 women that were VIA positive but did not have colposcopy and biopsy. Using the rates of CIN 2 or worse in the 4 subgroups based on result of physician-collected high-risk HPV and cervical cytology, we estimated that 2 VIA positive women with CIN 2 or worse were not detected in SPOCCS II.

We conclude that, despite the verification bias inherent in SPOCCS II, inflation of sensitivity of VIA for CIN 2 or worse (and CIN 3 or cancer) is found when colposcopic-directed biopsy is used as the gold standard in cervical cancer screening studies. This inflation is secondary to correlation between VIA and colposcopic-directed biopsy and might have been expected as VIA and colposcopy both use acetic acid enhancement to detect CIN by visual means. The magnitude of the inflation of sensitivity of VIA in a given study cannot be determined without the results of a second, more accurate, gold standard.

Inflation of sensitivity of VIA associated with correlation of VIA with the gold standard of colposcopic-directed biopsy could be one explanation for the high values of apparent sensitivity of VIA reported by Sankaranarayanan et al.13 We suspect that other screening tests that use optical methodology (e.g., cervicography14 and speculoscopy15) are also likely correlated with colposcopy and hence also likely to have inflated sensitivity when colposcopic-directed biopsy is the gold standard for CIN 2, CIN 3, or cancer.

We advise a 5-biopsy gold standard for diagnosing CIN not only in cervical cancer screening studies but also in evaluation of women with abnormal cervical cytology. As noted previously, the multiple biopsies performed in SPOCCS I and II were obtained with a bronchoscopic biopsy instrument that takes 2-mm biopsies. In our experience, women have less discomfort with 4 cervical biopsies performed with the 2-mm instrument than they have with 1 biopsy performed with the usual biopsy instrument (e.g., baby Tischler). Though processing and interpretation of 2-mm biopsies may appear difficult, in neither the Cleveland Clinic where these biopsies have been the standard colposcopic biopsies for over 20 years nor in Beijing during SPOCCS I and II has this been a significant problem. SPOCCS I and II were performed in the same population and though the sensitivity of colposcopic-directed biopsy in SPOCCS II (56.3%) was lower than that in SPOCCS I (75.6%), utilizing the 5-biopsy gold standard resulted in virtually identical risks of CIN 2 or worse in the 2 studies.5, 6 We conclude that utilizing the 5-biopsy gold standard in colposcopy clinics would avoid missed diagnosis of CIN 2 or worse secondary to variation in expertise of colposcopy. As fewer cases of CIN 2 or worse would be missed at the initial colposcopic evaluation, adopting the 5-biopsy standard would also likely decrease the need for follow-up after a colposcopic diagnosis of CIN 1 or less.16

If the 5-biopsy standard is adopted for evaluation of women with abnormal cervical cytology, the rules for treatment of CIN may need to be changed. As shown in Table II, the CIN 2 or worse diagnosed by random biopsy or solely by positive ECC is smaller and of lower grade than that diagnosed by colposcopic-directed biopsy. Given that CIN 2 is a poorly defined entity and may resolve without treatment,1 removal or ablation of the transformation zone in young women with small CIN 2 may not be indicated.17 Ablation or excision of the transformation zone in women with CIN 2 is advised in older women that have large lesions, are unlikely to return for follow-up, or when the diagnosis of CIN 2 is uncertain (e.g., when the cervical cytology is HSIL, the ECC is positive for CIN 2, or the pathologic interpretation is uncertain).

References

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References
  7. Appendix
  • 1
    ASCUS-LSIL Triage Study (ALTS) Group. Results of a randomized trial on the management of cytology interpretations of atypical squamous cells of undetermined significance. Am J Obstet Gynecol 2003; 188: 138392.
  • 2
    Sankaranarayanan R, Wesley R, Thara S, Dhakad N, Chandralekha B, Sebastian P, Chithrahara K, Parkin DM, Nair MK. Test characteristics of visual inspection with 4% acetic acid (VIA) and Lugol's iodine (VILI) in cervical cancer screening in Kerala, India. Int J Cancer 2003; 106: 4048.
  • 3
    Wright TC,Jr, Denny L, Kuhn L, Pollack A, Lorincz A. HPV DNA testing of self-collected vaginal samples compared with cytologic screening to detect cervical cancer. JAMA 2000; 283: 816.
  • 4
    Herrero R, Hildesheim A, Bratti C, Sherman ME, Hutchinson M, Morales J, Balmaceda I, Greenberg MD, Alfara M, Burk RD, Wacholder S, Plummer M, et al. Population-based study of human papillomavirus infection and cervical neoplasia in rural Costa Rica. J Natl Cancer Inst 2000; 92: 46474.
  • 5
    Belinson JL, Qiao YL, Pretorius R, Zhang WH, Elson P, Li L, Pan QJ, Fischer C, Lorincz A, Zahniser D. Shanxi province cervical cancer screening study. A cross sectional comparative trial of multiple techniques to detect cervical intraepithelial neoplasia. Gynecol Oncol 2001; 83: 43944.
  • 6
    Belinson JL, Qiao YL, Pretorius RG, Zhang WH, Rong SD, Huang MN, Zhao FH, Wu LY, Ren SD, Huang RG, Washington MF, Pan QJ, Li, L, et al. Shanxi province cervical cancer screening study II: self-sampling for high-risk human papillomavirus compared to direct sampling for human papillomavirus and liquid based cervical cytology. Int J Gynecol Cancer 2003; 13: 81926.
  • 7
    Pretorius RG, Kim RJ, Belinson JL, Elson P, Qiao YL. Inflation of sensitivity of cervical cancer screening tests secondary to correlated error in colposcopy. J Low Genital Tract Dis 2006; 10: 4550.
  • 8
    Pretorius RG, Zhang WH, Belinson JL, Haung MN, Wu LY, Zhang X, Qiao YL. The relative importance of colposcopically directed biopsy, random cervical biopsy, and endocervical curettage in the diagnosis of cervical intraepithelial neoplasia II or worse. Am J Obstet Gynecol 2004; 191: 4304.
  • 9
    Pepe MS. Incomplete data and imperfect reference tests. In: PepeMS, ed. The statistical evaluation of medical tests for classification and prediction. Oxford: Oxford University Press, 2004. 168213.
  • 10
    Phelps CE, Hutson A. Estimating diagnostic test accuracy using a “fuzzy gold standard.” Med Decis Making 1995; 15: 4457.
  • 11
    Arbyn M, Buntinx F, Van Ranst M, Paraskevaidis E, Martin-Hirsch P, Dillner J. Virologic versus cytolgic triage of women with equivocal Pap smears: a meta-analysis of the accuracy to detect high-grade intraepithelial neoplasia. J Natl Cancer Inst 2004; 96: 28093.
  • 12
    Pretorius RG, Peterson P, Novak S, Azizi F, Sadeghi M, Lorincz AT. Comparison of two signal-amplification DNA tests for high-risk HPV as an aid to colposcopy. J Reprod Med 2002; 47: 2906.
  • 13
    Sankaranarayanan R, Basu P, Wesley RS, Mahe C, Keita N, Mbalawa CC, Sharma R, Dolo A, Shastri SS, Nacoulma M, Nayama M, Somanathan T, et al.; IARC Multicentre Study Group on Cervical Cancer Early Detection. Accuracy of visual screening for cervical neoplasia: results from an IARC multicentre study in India and Africa. Int J Cancer 2004; 110: 90713.
  • 14
    De Vuyst H, Clacys P, Njiru S, Muchiri L, Steyaert S, De Sutter P, Van Marck E, Bwayo J, Temmerman M. Comparison of pap smear, visual inspection with acetic acid, human papillomavirus DNA-PCR testing and cervicography. Int J Gynaecol Obstet 2005; 89: 1206.
  • 15
    Lonky N, Mann WJ, Massad LS, Mutch DG, Blanco JS, Vasilev SA, Finan MA, Scotti RJ. Ability of visual tests to predict underlying cervical neoplasia. Colposcopy and speculoscopy. J Reprod Med 1995; 40: 5306.
  • 16
    Pretorius RG, Peterson P, Azizi F, Burchette RJ. Subsequent risk and presentation of cervical intraepithelial neoplasia (CIN) 3 or cancer after a colposcopic diagnosis of CIN 1 or less. Am J Obstet Gynecol 2006; 195: 12605.
  • 17
    American College of Obstetricians and Gynecologists. Evaluation and management of abnormal cervical cytology and histology in the adolescent: ACOG Committee Opinion No. 330. Obstet Gynecol 2006; 107: 9638.
  • 18
    BishopYMM, FienbergSE, HollandPW, eds. Discrete multivariate analysis: theory and practice. Cambridge, MA: The MIT Press, 1975. 3956.

Appendix

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. References
  7. Appendix

Figure 1 is a plot of inflation of sensitivity of the screening tests when a colposcopic-directed gold standard is compared with a 5-biopsy gold standard versus kappa for the correlation of the screening tests with diagnosis by colposcopic-directed biopsy. The points in Figure 1 appear to be on a straight line (r = 0.998) and the ratio of inflation of sensitivity secondary to the intermediate gold standard of colposcopic-directed biopsy to kappa appears to be 0.49. The r value is so close to 1.0 that we suspected a mathematical relationship between the 2 variables.

thumbnail image

Figure 1. Plot of inflation of sensitivity secondary to gold standard of colposcopic-directed biopsy versus kappa for correlation between screening tests and colposcopy with endpoints of CIN 2 or worse (open circles) and CIN 3 or cancer (solid triangles).

Download figure to PowerPoint

In the present case, the intermediate gold standard is colposcopic-directed biopsy. The cells in Table AI subdivide the positives of the universal “gold standard,” in this case, the 5-biopsy standard. The sensitivity of a screening test determined by the 5-biopsy gold standard is (A + B)/(A + B + C + D). The sensitivity of a screening test determined by the intermediate, colposcopic-directed biopsy gold standard is A/(A + C). The difference between screening test sensitivity determined by an intermediate gold standard and the screening test sensitivity determined by the universal gold standard (defined in this article as inflation of sensitivity) is [A/(A + C)] − [(A + B)/(A + B + C + D)]. Kappa is 2(ADBC)/[(A + B) (B + D) + (A + C) (C + D)].18

Table AI. Model of a 2-by-2 Table Correlating a Screening Test and an Intermediate Gold Standard Within a Populationknown to be Positive by a Yet More Universal “Gold Standard”
 Intermediate gold standard positiveIntermediate gold standard negativeUniversal gold standard positive
Screening test positiveABA + B
Screening test negativeCDC + D
 A + CB + DA + B + C + D

The ratio of inflation of sensitivity secondary to a correlated intermediate gold standard divided by kappa is 0.5 − [(A + B)/2] [2/(A + B + C + D) −1/(A + C)]. In the situations described in this manuscript, the second term is always small compared to 0.5, so the ratio of inflation of sensitivity secondary to an inaccurate intermediate gold standard to kappa is always close to 0.5. The r value for the plot seen in Figure 1 is not exactly 1.0 and the ratio of inflation of sensitivity secondary to a correlated gold standard divided by kappa is not constant.