We thank the Screening Effectiveness and Research in Community-Based Healthcare (SEARCH) investigators, project managers, data managers, and chart abstractors for the data they have provided for this study.
Papanicolaou (Pap) testing has transitioned from conventional preparations (CPs) to liquid-based preparations (LBPs) because of the perceived superiority of LBPs. Many studies conclude that LBPs reduce unsatisfactory Pap tests; however, some believe that the evidence substantiating this claim is weak. The authors studied the effect of the transition from CPs to LBPs on the proportion of unsatisfactory Pap tests in 4 health care systems in the United States participating in the National Institutes of Health-funded Screening Effectiveness and Research in Community-Based Healthcare (SEARCH) project.
The study cohort consisted of 548,174 women ages 21 to 65 years who had 1443,725 total Pap tests between 2000 and 2010. Segmented regression analysis was used to estimate the effect of adopting LBPs on the proportion of unsatisfactory Pap tests after adjusting for age.
Three sites that implemented SurePath LBP experienced significant reductions in unsatisfactory Pap tests (estimated effect: site 1, −2.46%; 95% confidence interval [CI], −1.47%, −3.45%; site 2, −1.78%; 95% CI, −1.54%, −2.02%; site 3, −8.25%; 95% CI, −7.33%, −9.17%). The fourth site that implemented ThinPrep LBP did not experience a reduction in unsatisfactory Pap tests. The relative risk of an unsatisfactory Pap test in women aged ≥50 years increased after the transition to LBPs (SurePath: relative risk, 2.1; 95% CI, 1.9-2.2; ThinPrep: relative risk, 1.7; 95% CI, 1.5-2.0).
Recently in the United States, Papanicolaou (Pap) tests have largely shifted from conventional preparations (CPs) to liquid-based preparations (LBPs) because of the perceived shortcomings of CPs. LBP marketing highlights several putative advantages, including increased sensitivity for detecting precancerous cervical lesions, fewer unsatisfactory specimens, the ability to test for human papillomavirus (HPV) from the same specimen, and the facilitation of computer-assisted screening.[1-4] Currently, there are 2 commercially available LBP systems, ThinPrep (Cytyc Corporation, Boxborough, Mass) and SurePath (Tripath Imaging Inc., Burlington, NC), approved for Pap testing by the US Food and Drug Administration (FDA).
LBPs are prepared by rinsing the sampling tool into a liquid-based preparatory media producing, a cell suspension from which a slide is prepared that contains a monolayer of cells. The liquid-based media allows for immediate cell fixation with preservation of nuclear detail and dissolution of obscuring elements. CPs, conversely, are prepared by directly smearing the cells onto a slide with the collection instrument with an immediate spray application of or immersion into a fixative solution to avoid air-drying artifact. Preparation of CP slides, thus, requires technical proficiency of the operator. It is for these reasons that LBPs are believed to reduce unsatisfactory Pap tests compared with CPs.
Unsatisfactory Pap test results are undesirable for several reasons. Repeat tests increase costs and require recalling the patient. Various studies also have pointed out that a substantial fraction of false-negative Pap tests were identified as unsatisfactory on retrospective review[5-7] and that smear quality correlates with a higher rate of detection of squamous intraepithelial lesions. In consideration of this evidence, the 2001 Bethesda System (TBS) for Pap test reporting included semiquantitative criteria, which arbitrarily defined an “estimated minimum of approximately 8000 to 12,000 well preserved and well visualized squamous cells” as adequate for evaluation with conventional smears. Other studies since 2001 TBS publication have reported that unsatisfactory results after applying the more stringent criteria were not associated with increased squamous intraepithelial lesions relative to satisfactory and benign studies, suggesting the cutoff point of 10,000 cells may be too high.[10, 11]
Studies comparing LBPs to CPs have produced conflicting reports regarding unsatisfactory test results. Although most studies have reported that LBPs lowered the percentage of unsatisfactory studies, others have reported no difference.[12-15] A recent systematic review examining the relative performance of LBPs and CPs concluded that the design of the majority of those studies had serious methodological flaws and that pooled analyses of the proportion of unsatisfactory results revealed strong evidence of heterogeneity. More recent randomized controlled trials, however, have reported decreased unsatisfactory studies with LBPs compared with CPs.[1, 17] The objective of our current study was to compare the proportion of unsatisfactory specimens of CPs and LBPs at multiple community practices before and after the transition from conventional to liquid-based tests.
MATERIALS AND METHODS
We conducted this study within a National Institutes of Health/National Cancer Institute collaborative multicenter project, “Screening Effectiveness and Research in Community-Based Healthcare” (SEARCH). The 4 health care systems participating in SEARCH were members of the Health Maintenance Organization (HMO) Cancer Research Network (CRN), which, at the time of this study, consisted of 14 integrated health care delivery systems that use their administrative and research infrastructures to conduct cancer prevention and care research. Four CRN systems contributed data for this study: Group Health, Washington; Reliant Medical Group, Massachusetts; and Kaiser Permanente Northwest (Oregon/southwest Washington); and Kaiser Permanente Hawaii. Within each of these 4 health care systems, multiple clinic sites contributed Pap tests. The institutional review boards of participating sites and the University of Massachusetts Medical School approved the study protocol.
Eligible patients included women ages 21 to 65 years who were enrolled in 1 of the participating health plans during periods that varied by site: from 1996 to 2010 (2 sites), from 1997 to 2007 (1 site), and from 2001 to 2010 (1 site). Women who had a history of total hysterectomy or known gynecologic cancer before they received a Pap test were excluded.
Data were collected on women's demographics information, cancer and hysterectomy history, enrollment, and Pap and HPV testing. When possible, these data were collected using the HMO Research Network's Virtual Data Warehouse, which defines a data model for enrollment, use, diagnoses, procedures, pharmacy, laboratory, and vital signs data developed and is maintained by CRN member institutions.
Data that were unavailable within the Virtual Data Warehouse were extracted from operational laboratory information systems or other data resources at each site. Pap test dates, diagnoses, reason(s) for reports of unsatisfactory smears, and cytologic methods were collected from free text cytology reports at sites 1, 3, and 4; at site 2, this information was extracted from a coded cytology data set. Results from Pap tests were classified using the coding scheme provided in Table 1, and extracted data were mapped programmatically using decision rules that were developed by 2 investigators (A.K. and C.L.O., a cytopathologist). Reasons provided for unsatisfactory Pap smears were mapped to insufficient cellularity, obscuring elements (blood or inflammation), air-drying artifact, insufficient clinical information, and unspecified. Pap tests were classified by processing method as either conventional, ThinPrep, or SurePath. All laboratories were certified and maintained standards established by Clinical Laboratory Improvement Amendments and processed specimens according to the manufacturers' recommendations. Sites 1, 2, and 3 processed Pap tests at a central laboratory (unique for each site), and site 4 used multiple laboratories, although most Pap tests at site 4 (>95% of Pap tests) were processed at a single laboratory. Sites 1, 2, and 3 implemented SurePath methodology, and all laboratories from site 4 implemented ThinPrep methodology. At site 1, all unsatisfactory SurePath specimens were reprocessed, and a second slide was prepared. If the specimen remained unsatisfactory, then both slides were reviewed by a second cytotechnologist. At site 2, all unsatisfactory SurePath slides were reviewed by a second cytotechnologist, and, in some cases, a second slide was prepared at the discretion of the cytotechnologist or cytopathologist. At site 3, all unsatisfactory SurePath specimens were reprocessed, and a second slide was prepared. If the specimen remained unsatisfactory, then a second cytotechnologist and a cytopathologist reviewed both slides. At site 4, all unsatisfactory ThinPrep slides were reviewed by a second cytotechnologist. Bloody specimens were reprocessed with glacial acetic acid at the discretion of the cytotechnologist and cytopathologist. Data from site 4 were pooled for analysis.
Table 1. Cytology Diagnosis Codes
Atypical squamous cells of undetermined significance
Atypical squamous cells, cannot exclude a high-grade squamous intraepithelial lesion
Atypical glandular cells
Endocervical adenocarcinoma in situ
Low-grade squamous intraepithelial lesion
High-grade squamous intraepithelial lesion
Squamous cell carcinoma malignancy (not in situ)
Any malignancy other than squamous cell
Unsatisfactory or inadequate sample
The time of implementation of LBP for each site was defined as the quarter in which the majority of specimens (>50%) were processed as LBPs. LBP tests before the implementation of LBP and CP tests after the implementation of LBP were excluded from analysis. Table 2 indicates that the time to intervention ranged from the second quarter (Q2) of 2004 at site 2 to Q2 of 2006 at site 1. Each site had a minimum of 17 quarters before the intervention and 15 quarters after the intervention.
The intervention was considered the implementation of liquid-based cytology at each site.
We calculated the proportions of unsatisfactory Pap smears stratified by type of Pap test, women's age group, and health plan. We conducted an intervention analysis using segmented regression. The intervention was considered the implementation of liquid-based cytology at each site. By including an indicator variable for the intervention, the segmented regression models divided the time series at each site into 2 segments: a preintervention segment beginning at the start of the study period (Q1 of 2000) and an intervention segment starting at the quarter of implementation of LBP. The models controlled for time since the start of the study period, which included the uptake of TBS semiquantitative criteria for adequacy. The models also controlled for time after the implementation of LBP and the average age of women receiving Pap tests each quarter. This allowed us to estimate changes in the level and trends in the quarterly proportion of unsatisfactory Pap smears after the implementation of liquid-based cytology at each site.
Diagnostic testing of the initial models, which used ordinary least squares, indicated homoskedasticity (constant variance), but the Durbin-Watson test statistics indicated that autocorrelation was present. We used the Cochrane-Orcutt iterative procedure to correct for the correlated errors and obtain generalized least squares estimates of the parameters. After adjustment for autocorrelation, diagnostic testing indicated homoskedasticity, normally distributed residuals, and no autocorrelation with Durbin-Watson test statistics close to 2.
In Q2 and Q3 of 2006 at site 1, the type of cytologic preparation used could not be ascertained, and these quarters were excluded from the analysis. For a sensitivity analysis, we also included in the regression analysis those quarters that had missing values. The impacts on the estimated effects and confidence intervals (CIs) were not substantial.
To investigate the relation between age and unsatisfactory Pap smear rates, we grouped women into groups ages <50 years and ≥50 years to approximate premenopausal and postmenopausal women. The proportion of unsatisfactory Pap tests was then calculated according to the type of Pap test in each age group. We calculated risk ratios, with the group aged <50 years considered the referent group.
In total, 1,406,967 Pap tests were included in the study. The quarterly unsatisfactory proportions over time at the 4 study sites are summarized in Table 3 and in Figure 1. At site 1, with conventional smears, the unsatisfactory proportion increased significantly at a rate of 0.10% per quarter (95% CI, 0.05%-0.15%) from 0.9% unsatisfactory Pap test results in Q1 of 2000 to 4% unsatisfactory Pap tests results in Q1 of 2006. After the implementation of SurePath at site 1, the unsatisfactory proportion decreased significantly. The estimated effect of the switch to LBPs was a decrease in the unsatisfactory proportion of −2.46% (95% CI, −1.47%, −3.45%) (Table 4). At site 2, the estimated effect of the switch to LBPs was −1.78% (95% CI, −1.54%, −2.02%); at site 3, the estimated effect was −8.25% (95% CI, −7.33%,−9.17%); and, at site 4, the switch to LBPs resulted in a small but statistically significant increase in the proportion of unsatisfactory Pap tests of 0.78% (95% CI, 0.19%-1.38%).
Table 3. Quarterly Unsatisfactory Rate by Screening Site
No. of Pap Tests (%)
Abbreviations: Pap, Papanicolaou; Q, quarter; Unsats, unsatisfactory.
This was the first quarter with a majority of liquid-based cytology Pap tests.
Table 4. Age-Adjusted Estimate of the Effect of Switching to Liquid-Based Cytology on the Rate of Unsatisfactory Papanicolaou Tests
Estimate (95% CI), %
Abbreviations: CI, confidence interval.
0.10 (0.05, 0.15)
Change in trend
−0.13 (−0.23, −0.03)
Effect on unsatisfactory rate
−2.46 (−3.45, −1.47)
0.11 (0.09, 0.12)
Change in trend
−0.09 (−0.12, −0.06)
Effect on unsatisfactory rate
−1.78 (−2.02, −1.54)
0.46 (0.29, 0.63)
Change in trend
−0.39 (−0.64, −0.15)
Effect on unsatisfactory rate
−8.25 (−9.17, −7.33)
0.00 (−0.03, 0.04)
Change in trend
0.01 (−0.06, 0.07)
Effect on unsatisfactory rate
0.78 (0.19, 1.38)
Women aged ≥50 years were significantly less likely to have an unsatisfactory Pap test with conventional smears relative to women aged <50 years at the sites that switched to SurePath (relative risk [RR], 0.61; 95% CI, 0.59-0.64) and at the site that switched to ThinPrep (RR, 0.81; 95% CI, 0.68-0.97). After the switch to liquid-based methods, the RR in women aged ≥50 years versus women aged <50 years for an unsatisfactory Pap test increased at the sites that switched to SurePath (RR, 2.1; 95% CI, 1.9-2.2) and at the site that switched to ThinPrep (RR, 1.7; 95% CI, 1.5-2.0) (Table 5).
Table 5. Proportion of Unsatisfactory Papanicolaou Tests by Age
No. of Pap Tests
Abbreviations: CI, confidence interval; Pap, Papanicolaou; RR, relative risk; SurePath, liquid-based preparation from Tripath Imaging Inc. (Burlington, NC); ThinPrep, liquid-based preparation from Cytyc Corporation (Boxborough, Mass).
Age <50 y
Age ≥50 y
Age <50 y
Age ≥50 y
Age <50 y
Age ≥50 y
Age <50 y
Age ≥50 y
Insufficient squamous cellularity was the predominant cause of unsatisfactory studies after the switch to liquid-based methods at the sites that switched to SurePath and the site that switched to ThinPrep. Before the switch, the reasons for unsatisfactory Pap tests varied (Table 6).
Table 6. Summary of Reasons for Unsatisfactory Papanicolaou Tests
Reason for Unsatisfactory Pap Test
No. of Pap Tests (%)
Conventional, n = 5280
SurePath, n = 2618
ThinPrep, n = 521
Abbreviations: Pap, Papanicolaou; SurePath, liquid-based preparation from Tripath Imaging Inc. (Burlington, NC); ThinPrep, liquid-based preparation from Cytyc Corporation (Boxborough, Mass).
This multisite study of greater than 1 million Pap tests compared unsatisfactory results before and after the transition from conventional methods to LBPs in 4 health care systems. We observed that 3 of 4 systems implementing liquid-based technologies demonstrated significant reductions in the proportion of unsatisfactory studies. The fourth site did not experience a reduction in unsatisfactory studies; however, as discussed below, the lack of a reduction in unsatisfactory studies in that system may have been related to the slow uptake of TBS criteria for adequacy with conventional smears. We observed that the reasons for unsatisfactory studies differed before and after the transition. The reason for an unsatisfactory study in the overwhelming majority of cases after the transition was “insufficient squamous cellularity.” Before the switch, at all sites, the reasons for an unsatisfactory study were more heterogenous. At all 4 sites, we observed that the RR of an unsatisfactory Pap test among women aged ≥50 years increased after the transition to liquid-based methods.
The idea that liquid-based Pap tests reduce the rate of unsatisfactory studies relative to conventional methods has become accepted by many, and both FDA-approved LBPs, ThinPrep and SurePath, are allowed by the FDA to claim a reduction in unsatisfactory tests when marketing their products. Problematically high numbers of unsatisfactory studies were the impetus to switch to liquid-based technology in some countries, and many studies have concluded that liquid-based cytology reduces the rate unsatisfactory studies. Davey and colleagues, however, in their systematic review of 56 studies comparing liquid-based with conventional cytology, concluded that there is insufficient evidence to conclude that liquid-based cytology lowers the fraction of unsatisfactory Pap tests. This conclusion was controversial and other investigators have questioned this finding.[22, 23] In our study, the 3 sites that switched to SurePath had significantly lower proportions of unsatisfactory studies, and the site that switched to ThinPrep did not significantly reduce the proportion of unsatisfactory studies. However, the laboratory that switched to ThinPrep had a relatively low fraction of unsatisfactory tests with CPs and, as discussed below, was the least affected by the publication of TBS semiquantitative criteria for adequacy. Thus, the lack of a reduction in unsatisfactory tests observed at the ThinPrep site may reflect slow uptake of the TBS criteria. Our findings are in line with the College of American Pathology self-reported rates of interpretative categories in Pap testing, which have consistently demonstrated lower unsatisfactory rates for SurePath compared with ThinPrep and similar rates of unsatisfactory studies for ThinPrep compared with conventional methods.
Before publication of the 2001 Bethesda semiquantitative criteria for adequacy (in early 2002), the proportion of unsatisfactory tests was similar at all 4 sites using conventional methods. After that publication, we observed significant variation in the proportion of unsatisfactory Pap tests with CPs. This finding suggests variability in the uptake of TBS criteria for unsatisfactory results with conventional smears during this time.
Not surprisingly, the causes of unsatisfactory smears with CPs were heterogeneous. Liquid-based smears determined unsatisfactory were almost exclusively caused by insufficient squamous cellularity. Predictably, liquid-based smears effectively eliminated air-drying artifact as a cause of unsatisfactory studies and substantially reduced obscuring elements as a cause. We hypothesize that the increased RR for unsatisfactory studies in women aged >50 years after the transition to LBPs (both SurePath and ThinPrep) is related to the finding that LBPs effectively remove obscuring elements (reasons for unsatisfactory Pap tests more common among younger women) and that LBPs have little or no effect on cervical atrophy (a reason for unsatisfactory Pap tests observed almost exclusively among older women).
Our study has limitations, including the relatively small number of sites and the retrospective, nonrandomized design. The cytologic diagnoses were not subjected to central review, and we do not know the fraction of total LBPs that were reprocessed, so we cannot control for regional variability in diagnostic criteria or variability in the threshold to reprocess. Our study consisted of many different clinical sites within 4 health care systems, and we do not have details on the nature or duration of training for clinical and laboratory personnel transitioning to LBPs, so we do not know whether the training was similar across sites. We also concede that the uptake of TBS semiquantitative criteria seems to vary among our participating sites; and this, along with the timing of the transition to LBP, has an impact on the estimate of the effect of reducing unsatisfactory results in our model.
In conclusion, the impact on the rate of unsatisfactory results from transitioning to a liquid-based Pap test depends on the proportion of unsatisfactory studies produced in a particular laboratory using CPs and the specific type of liquid-based test that is used. Laboratories that have a high proportion of unsatisfactory studies using CPs may experience a significant reduction in unsatisfactory studies, and those that have a low proportion of unsatisfactory tests may experience little or no effect.
Data collection for this work was supported by an award from the National Cancer Institute at the National Institutes of Health (Cancer Screening Effectiveness and Research in Community-Based Healthcare [SEARCH]; UC2 CA148576; to Drs. Buist and Doubeni [coprincipal investigator]). The content is solely the responsibility of the authors and does not represent the official views of the National Institutes of Health. Note that elements of the data infrastructure were developed for the HMO Cancer Research Network Virtual Data Warehouse (U19 CA79689; to E. H. Wagner, M. C. Hornbrook, and L. Kushi, who developed the Virtual Data Warehouse within the research network).