SEARCH

SEARCH BY CITATION

Keywords:

  • gynecologic cytology;
  • automated cytology;
  • FocalPoint;
  • cervical screening;
  • Papanicolaou test

Abstract

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. Acknowledgements
  8. FUNDING SOURCES
  9. REFERENCES

BACKGROUND:

Studies of the performance of the automated FocalPoint Guided Screening (FPGS) imaging system in gynecologic cytology screening relative to manual screening have yielded conflicting results. In view of this uncertainty, a validation study of the FPGS was conducted before its potential adoption in 2 large laboratories in Ontario.

METHODS:

After an intense period of laboratory training, a cohort of 10,233 current and seeded abnormal slides were classified initially by FPGS. Manual screening and reclassification blinded to the FPGS results were then performed. Any adequacy and/or cytodiagnostic discrepancy between the 2 screening methods subsequently was resolved through a consensus process (truth). The performance of each method's adequacy and cytodiagnosis vis-a-vis the truth was established. The sensitivity and specificity of each method at 4 cytodiagnostic thresholds (atypical squamous cells of undetermined significance or worse [ASC-US+], low-grade squamous intraepithelial lesion or worse [LSIL+], high-grade squamous intraepithelial lesion or worse [HSIL+], and carcinoma) were compared. The false-negative rate for each cytodiagnosis was determined.

RESULTS:

The performance of FPGS in detecting carcinoma, HSIL+, and LSIL+ was no different from the performance of manual screening, but the false-negative rates for LSIL and ASC-US were higher with FPGS than with manual screening.

CONCLUSIONS:

The results from this validation study in the authors' laboratory environment provided no evidence that FPGS has diagnostic performance that differs from manual screening in detecting LSIL+, HSIL+, or carcinoma. Cancer (Cancer Cytopathol) 2013;121:189–196. © 2013 American Cancer Society.


INTRODUCTION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. Acknowledgements
  8. FUNDING SOURCES
  9. REFERENCES

In the Canadian province of Ontario, most screening gynecologic cytology specimens are processed and interpreted by community laboratories. In 2011, these laboratories processed 1.67 million Papanicolaou (Pap) tests. For the last decade, the vast majority of these Pap tests have been performed using SurePath liquid-based cytology (BD Diagnostics, Franklin Lakes, NJ).1 Later, the FocalPoint Imaging System Profiler was added to this basic liquid-based platform.2 The prevalence rate of unsatisfactory Pap tests in 2008 was 0.5%. The prevalence and abbreviations for each of the gynecologic cytodiagnoses in satisfactory Pap tests in the Ontario cervical screening program using the Bethesda System are provided in Table 1.3

Table 1. Prevalence of Screening Cytodiagnoses in the Ontario Cervical Screening Program 2008
CytodiagnosisAbbreviationPrevalence, %e
Negative for intraepithelial lesion or malignancyNILM95
Atypical cells of uncertain significanceASC-US2.3
Atypical squamous cells, high-grade squamous intraepithelial lesion cannot be excludedASC-H0.1
Atypical glandular cellsAGC0.1
Low-grade squamous intraepithelial lesionLSIL2.1
High-grade squamous intraepithelial lesionHSIL0.3
Carcinoma and malignancy0.015

The FocalPoint Guided Screening (GS) imaging system (FPGS) is a further development in the FocalPoint system.4 The FPGS selects up to 10 images of the most significant abnormality for review by a cytotechnologist. If these images are judged to be within normal limits, then the case is signed out by the cytotechnologist provided that high-risk criteria based on clinical history are not present and the case was not selected for quality-control (QC) review by the GS software. Alternatively, if a significant abnormality is suspected, then a full manual slide review must be performed. For Ontario laboratories, the FPGS presented an opportunity to improve both the quality and efficiency of gynecologic cytology screening.

On reviewing the literature, however, only 2 clinical studies have compared the performance of the FPGS with manual screening, and those studies provided conflicting evidence regarding the efficacy of the FPGS. The initial study of the FPGS did identify significantly improved sensitivity for detecting both low-grade squamous intraepithelial lesion or worse (LSIL+) (9.8%) and high-grade squamous intraepithelial lesion or worse (HSIL+) (19.6%) compared with routine manual screening, and any decreases in specificity were small.4 Subsequently, an independent clinical trial provided no support for this finding of a superior performance of FPGS over manual screening.5 The investigators from that major trial (the United Kingdom—Manual Assessment Versus Automated Reading in Cytology [MAVARIC]) concluded that FPGS was less sensitive than manual reading for the detection of cervical intraepithelial neoplasia (CIN) grade 2 to 3 or worse (CIN2+) lesions. Faced with this information, 2 Ontario laboratories considered the desirability of performing a validation study of the FPGS.

Whereas the process of verification provides objective evidence that instrument performance meets specified criteria and performance parameters, validation defines prerequisites that must be adequately met before they are implemented for their intended use. Good laboratory practice and Ontario laboratory regulations require validation of new laboratory technology. Specifically, before the implementation of new gynecologic liquid-based instruments and new automated screening instruments, a laboratory is obliged to validate and document the functioning of the instrument in its own specific laboratory environment and its capability to replace existing procedures.6 The laboratories undertook a review of the current recommended framework for validation procedures7 and recognized that the scope of evaluation is determined by the parameters and nature of the evaluation and that this may require an extensive undertaking. The 2 laboratories concluded that an extensive Ontario validation trial comparing the effectiveness of the FPGS with manual screening, which is widely regarded as the appropriate comparator, was necessary.

MATERIALS AND METHODS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. Acknowledgements
  8. FUNDING SOURCES
  9. REFERENCES

The current validation study was conducted at 2 clinical laboratories using an identical study protocol that consisted of 3 phases. Both clinical laboratories are fully accredited by Ontario Laboratory Accreditation and are fully compliant with the recommended cytology quality-assurance practices required by the Quality Management Program-Laboratory Services of Ontario, which include the monitoring of multiple laboratory parameters. Greater than 50% of Ontario's screening Pap tests are processed in these 2 clinical laboratories.

In the first phase, cytotechnologists (CTs) were trained in the operation of the BD FPGS and field-of-view (FOV) screening procedures by the manufacturer's personnel followed by a slide test. After this initial training, a 17-week period of practice was instituted, during which time CTs were assigned to assess both routine and seeded abnormal cases for up to 7 hours daily to develop confidence and proficiency in the use of the FPGS technology.

The second phase of the study consisted of an analysis of a cohort of slides from current routine cases that were randomly selected from the daily slide volume and seeded abnormal cases (repeat preparations of abnormal cases reported earlier in the year) to ensure an adequate representation across all diagnostic categories. The cohort size was determined to ensure statistically significant differences in the screening performance of manual screening versus FPGS in specificity, sensitivity, and positive and negative predictive values at 3 diagnostic thresholds—HSIL+ (HSIL, adenocarcinoma in situ [AIS], and invasive carcinoma), LSIL+ (LSIL, HSIL, AIS, and invasive carcinoma), and atypical squamous cells (ASC) of undetermined significance (ASC-US) or worse (ASC-US+) (ASC-US, ASC-cannot exclude HSIL [ASC-H], LSIL, HSIL, AIS, or invasive carcinoma).

All samples were taken from women of any age, and slides were prepared using the BD PrepStain (BD Diagnostics). Any current and seeded abnormal slides were excluded if physical characteristics were present that would exclude screening of the slides by either the manual or FPGS method. Specifically, if coverslips were broken, cracked, or overhanging; were part of a multiple slide case; revealed obscuring factors or bubbles, or were incompatible with processing on the FPGS, then these slides were excluded from the study. Complete cases required the review of a slide by both arms of the study. All study cases were accompanied by all available clinical information during screening. Generic barcodes were placed over the laboratory slide labels to mask the identity of the case during both screens. The screening results from both arms were not available to the other arm. Initially, cases were screened by the FPGS, and this was followed by the control arm of manual screening. The FPGS arm consisted of a 100% FocalPoint Slide Profiler slide processing and GS initial review. The CT reviewed all FOVs (maximum of 10 plus locator FOV) even if an abnormality was identified in a previous FOV. Slides were selected for full slide review (FSR) based on detection or suspicion of an abnormality and were reviewed immediately by the same CT, and a single diagnosis was recorded from the combined FOV and FSR. FSR by the primary CT also was performed if there were no FOVs provided by the slide profiler and in cases of process review or rerun cases. Partial microscopic review—that is, a rapid manual scan of no more than 30 seconds—was performed if there was a discrepancy between the FPGS and the FOV with respect to the presence of endocervical cells to confirm an FPGS report of insufficient cellularity or for cases in which the CT deemed that additional screening was needed to formulate an interpretation. Individuals functioning as QC CTs were responsible for rescreening the slides from targeted review cases (based on clinical/historic information) and the cases flagged for QC by the GS software (at least 15% of slides that initially were interpreted as negative were flagged based on a quintile score for QC review). This QC review consisted of a full FOV review followed by FSR by a QC CT.

After the FPGS arm was completed, all markings on the slides were removed, and the slides proceeded to the manual screening arm. All interpretations in the manual screening arm were rendered blinded to the FPGS results. To avoid bias, the slides were not distributed to CTs who were involved in the FPGS arm. An attempt was made to avoid sending slides to the same pathologist(s) who had reviewed the case in the FPGS arm.

The third phase of the study was an adjudication process for both adequacy and diagnostic interpretations. Slides that were interpreted as unsatisfactory in 1 arm but were satisfactory in the other arm were subjected to an adjudication panel consisting of 3 CTs who each examined the slide. A majority opinion established “truth.”

Slides with different diagnostic interpretations between the 2 arms of the study were subjected to an adjudication panel for “truth” resolution. The adjudication panel consisted of 3 cytopathologists. Two panel members individually examined each slide with 2 sets of interpretations, and provided an independent assessment result for each evaluated slide. If the 2 members' interpretations differed, then a third cytopathologist's interpretation was sought. Resolution required agreement among 2 of the 3 members or consensus after a conference.

According to local laboratory practice, verification, validation, and quality-assurance studies of approved and available laboratory technologies do not require ethics board approval. Patient reports were issued on the basis of the manual screening. Amended reports based on new consensus findings were issued as required.

Five parameters were chosen to compare the performance of the 2 methods. The initial parameter was the evaluation of adequacy compared with the consensus determination. Next, the relative performance of each method in cytodiagnostic interpretation compared with the consensus interpretation (“truth”) was identified. Finally, the sensitivity, specificity, and false-negative rates at 4 diagnostic thresholds (carcinoma, HSIL+, LSIL+, and ASC-US+) for each method were derived and compared. Sensitivity is the probability that a sample with a lesion will test positive for that lesion or worse and is defined as the ratio of the number of positive tests divided by the number of positive samples (true-positives plus false-negative). Specificity is the probability that a sample without a lesion will test negative for that lesion or worse and is defined as the ratio of the number of negative tests divided by the number of negative samples (true-negative plus false-positive). The false-negative rate is the probability that a cytodiagnosis, as determined by the consensus process, will have a negative test (negative for intraepithelial lesion or malignancy [NILM]) using either screening method and is defined as false-negative/false-negatives plus true-positives.

Agreement of rates (sensitivity, specificity, false-negative rate) for the various levels of dysplasia were tested statistically using a conventional 2-sample test for independent proportions. Statistical significance was examined with a conventional 2-tailed P value for the test statistic (Z).8 A 2-tailed test is conducted when there is no previous expectation for whether 1 proportion will to be higher than the other; that is, this is a test for differences in proportions, not whether 1 proportion is larger or smaller. Statistical P values ≥ .05 are reported as “not significant,” and P values (significance levels) are reported for statistically significant differences. This test of proportions was done first to compare rates from the 2 different laboratories separately for manual screening and GS and for differences between manual and GS results in each laboratory to ensure that it was acceptable to combine the proportions. The tests were then conducted on the combined results from the 2 laboratories to test for differences between manual screening and GS results in the combined data. It is important to note that the statistical significance test for different rates is affected not only by the difference in the rates but also by the number of cases on which the rates are based. Therefore, large differences in rates may not be statistically significant when based on few cases (eg carcinoma and HSIL+), and smaller differences can be highly significant when based on many cases (eg ASC-US+).

In the fourth and final phase of the study, all FOV images for atypical granular cells (AGC), ASC-H, LSIL, and HSIL cases that were interpreted as NILM by the FPGS arm (that is, false-negative cases) were reviewed for the presence or absence of abnormal cells with full knowledge of the final consensus cytodiagnosis.

RESULTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. Acknowledgements
  8. FUNDING SOURCES
  9. REFERENCES

In the initial phase of the study, 8 CTs reviewed 21,097 slides over a 17-week period. After the elimination of slides for physical characteristics or misplacement from the initial selected cohort, 10,233 slides, consisting of 9545 current routine slides and 688 seeded abnormal slides, were reviewed by both the FPGS method and the manual method.

There were no consistent differences in the mix of samples or statistical rates in the 2 laboratories; therefore, the results were combined to better investigate differences between manual screening and GS. There was a high degree of agreement between the FPGS and manual screening arms with respect to the consensus adequacy interpretation (Table 2). For example, in total, 45 unsatisfactory Pap tests were identified after FPGS review and adjudication, and 39 cases were identified by the FPGS method.

Table 2. Comparison of FocalPoint Guided Screening and Manual Screening Adequacy Interpretations Versus Consensus Interpretation
 Consensus Interpretation 
Screening ArmSatisfactoryUnsatisfactoryTotal
FocalPoint Guided Screening   
 Satisfactory10,188 10,188
 Unsatisfactory63945
 Total10,1943910,233
Manual screening   
 Satisfactory10,192410,196
 Unsatisfactory23537
 Total10,1943910,233

The performance of both the FPGS and manual screening arms, compared with the truth determination for each Bethesda System diagnostic category, is presented in Table 3. Both methods were similar in their performance for all diagnostic categories. However, in the FPGS arm, 73 of 560 LSIL cases were classified as ASC or NILM (13%) compared with 59 LSIL cases classified as ASC or NILM (10.5%) in the manual screening arm. However, this difference was not statistically significant (Z = 1.30; P = .19).

Table 3. Comparison of FocalPoint Guided Screening and Manual Screening Cytodiagnostic Interpretations Versus Consensus Interpretation
 Consensus Interpretation 
Screening ArmUSPNILMASCASC-HAGCLSILHSILCATotal
  1. Abbreviations: AGC, atypical glandular cells; ASC-H, atypical squamous cells, high-grade squamous intraepithelial lesion cannot be excluded; ASC-US, atypical cells of uncertain significance; CA, carcinoma; HSIL, high-grade squamous intraepithelial lesion; LSIL, low-grade squamous intraepithelial lesion; NILM, negative for intraepithelial lesion or malignancy; USP, unsatisfactory Papanicolaou test.

FocalPoint Guided Screening         
 USP39600000045
 NILM086341261742308813
 ASC0253266203130555
 ASC-H05923076050
 AGC010001600026
 LSIL0153930474201552
 HSIL0147261472169
 CA00001022023
  Total39892444436265601812310233
Manual screening         
 USP35200000037
 NILM4875774236208848
 ASC0146333635360547
 ASC-H02323037038
 AGC08001701127
 LSIL083310495170554
 HSIL0114031460155
 CA00003022227
  Total39892444436265601812310,233

Data on the sensitivity of both methods at 4 diagnostic thresholds are provided in Table 4. There was no statistical difference between the FPGS and manual screening methods with respect to the detection of carcinoma, HSIL+, or LSIL+ lesions. However, the sensitivity of manual screening for ASC-US+ lesions (93.1%) was statistically superior to that of FPGS (85.9%; Z-score, −6.00; P < .001).

Table 4. The Sensitivity of FocalPoint Guided Screening and Manual Screening at 4 Diagnostic Thresholds
Screening MethodCarcinomaHSIL+LSIL+ASC-US+
  • Abbreviations: ASC-US+, atypical cells of uncertain significance or worse; FPGS, FocalPoint Guided Screening; HSIL+, high-grade squamous intraepithelial lesion or worse; LSIL+, low-grade squamous intraepithelial lesion or worse.

  • a

    This score indicates a statistically significant difference.

FPGS, %8783.88885.9
Manual, %95.783.389.793.1
Difference, %−8.70.5−1.7−7.2
Z-score−1.060.13−1.06−6.00a

Data on the specificity of both methods at 4 diagnostic thresholds are provided in Table 5. There was no statistically significant difference in specificity to carcinoma, HSIL+, or LSIL+ lesions between the FPGS and manual screening methods. However, the specificity manual screening for ASC-US+ lesions (98.2%) was statistically superior to that of FPGS (96.8%; Z-score, −5.69; P < .001).

Table 5. The Specificity of FocalPoint Guided Screening and Manual Screening at 4 Diagnostic Thresholds
Screening MethodCarcinomaHSIL+LSIL+ASC-US+
  • Abbreviations: ASC-US+, atypical cells of uncertain significance or worse; FPGS, FocalPoint Guided Screening; HSIL+, high-grade squamous intraepithelial lesion or worse; LSIL+, low-grade squamous intraepithelial lesion or worse.

  • a

    This score indicates a statistically significant difference.

FPGS, %10099.899.296.8
Manual, %10099.999.598.2
Difference, %00.1−0.3−1.4
Z-score0.71−1.57−1.90−5.69a

The false-negative rates of both methods for 6 cytodiagnostic categories are provided in Table 6. There were no statistically significant differences in false-negative rates for carcinoma, HSIL+, AGC, or ASC-H lesions between the FPGS and manual screening methods. However, the false-negative rates of manual screening for LSIL and ASC-US lesions (1% and 16.7%, respectively) were statistically lower than those for FPGS screening (7.5% [Z-score, 5.378] and 28.4% [Z-score, 4.22], respectively).

Table 6. False-Negative Rates of FocalPoint Guided Screening and Manual Screening for Cytodiagnosesa
Screening MethodCarcinomaHSILLSILAGCASC-HASC
  • Abbreviations: AGC, atypical granular cells; ASC, atypical squamous cells; ASC-H, atypical squamous cells, high-grade squamous intraepithelial lesion cannot be excluded; FPGS, FocalPoint Guided Screening; HSIL, high-grade squamous intraepithelial lesion; LSIL, low-grade squamous intraepithelial lesion.

  • a

    False-negative indicates negative for intraepithelial lesion or malignancy by manual or FPGS methods on a cytodiagnosis by consensus process of ASC, ASC-H, AGC, LSIL, HSIL, or carcinoma.

  • b

    This score indicates a statistically significant difference.

FPGS: No./total (%)0/22 (0)3/181 (1.7)42/560 (7.5)7/26 (26.9)1/36 (2.8)126/444 (28.4)
Manual screening: No./total (%)0/22 (0)2/181 (1.1)6/560 (1)3/26 (11.5)2/36 (5.6)74/444 (16.7)
Difference, %10.66.415.4−2.811.7
Z-score0.000.455.38b1.44−0.594.22b

There were 53 false-negative AGC, ASC-H, LSIL, and HSIL cases in the FPGS arm; and, on review of the FOV images from these cases, abnormal cells could be detected in 45 of them. In 9 of these 45 cases, the abnormal cells in the FPGS FOV images were restricted to the periphery of the FOV.

DISCUSSION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. Acknowledgements
  8. FUNDING SOURCES
  9. REFERENCES

In the clinical pathology laboratory, the validation of new assays through comparison with current methods is widely practiced before implementation to ensure that implementation of the novel technology in the local laboratory environment maintains current diagnostic capabilities. In contrast, in anatomic pathology, there may be a tendency to inadequately validate new assays in comparison with existing methods.9 How often rigorous validation is done for automated screening devices in particular is uncertain, because it is almost certain that most validation studies are performed “in-house” and do not enter the published literature. Despite the resource-intensive nature of a comprehensive validation study for the FPGS, the laboratories participating in our study concluded that rigorous validation must be performed for any prospective automated cytology system to be in compliance with laboratory standards and accreditation and to maintain the integrity of Ontario's cervical screening program.

The objective of the current validation study was to determine whether there were any significant differences between the FPGS and manual screening methods in gynecologic cytology. It was conducted after a lengthy period of training to achieve the best possible performance of the FPGS. A comparison of our current results with the formal clinical trial data on the FPGS indicates that our study did achieve good performance of the FPGS.10 The sensitivity and specificity for carcinoma achieved in this study (87% and 100%, respectively) exceeded those reported in the industry-sponsored clinical trial (69.4% and 99.7%, respectively). The sensitivity and specificity of the current study for HSIL+ (83.8% and 99.8%, respectively) were similar to those from the clinical trial data (85.3% and 95.1%, respectively). Our current study's sensitivity and specificity for LSIL+ (88% and 99.2%, respectively) surpassed those in the clinical trial (86.1%, and 88.7%, respectively) as did the sensitivity and specificity for ASC-US+ (85.9% and 96.8%, respectively) versus the clinical trial (81.1% and 84.5%, respectively).

Although the FPGS performance in terms of adequacy detection was acceptable (Table 2), the FPGS performance in cytodiagnosis vis-a-vis the consensus cytodiagnosis was similar to the manual method (Table 3). Although manual and FPGS methods classified 10.5% and 13% of LSIL cases as NILM or ASC, respectively, this apparent difference was not statistically significant.

There were no significant differences in the sensitivity of the 2 methods with respect to the most abnormal cytodiagnostic categories (carcinoma, HSIL+, and LSIL+), although the manual screening method for ASC-US+ lesions was statistically superior to that of the FPGS method (Table 4). Similarly, there were no significant differences in the specificities of the 2 methods in the cytodiagnostic categories of carcinoma, HSIL+, or LSIL+, but the specificity of the manual screening method for ASC-US+ lesions was superior to that of the FPGS method (Table 5). Finally, the false-negative rates of manual screening for LSIL and ASC-US lesions were less than those of FPGS (Table 6).

In summary, several measures indicate that there are no differences between the performance of FPGS and manual screening with respect to carcinoma, HSIL+, and LSIL+, but other measures do indicate that the detection of lower grade lesions (LSIL and ASC-US) by FPGS is significantly lower. This finding is dissimilar to the industry-sponsored trial, which identified a similar proportion of false-negative rates for FPGS compared with manual screening for detecting adjudicated cases of LSIL (39 vs 37 of 575 LSIL cases, respectively).10

Is the low sensitivity of this study an inherent limitation or flaw of the FPGS, or is it a product of less than optimal CT FOV review? The re-examination of FPGS FOV images of the HSIL, LSIL, AGC, and ASC-H false-negative cases suggests that the major cause for FPGS insensitivity is human review, because >80% of reviewed FOVs did reveal abnormal cells. It is possible, then, that the sensitivity of FPGS may be enhanced with improved CT experience and performance in detecting abnormal cells.

Only a single, independent, comprehensive validation study of the FPGS has been published to date: the MAVARIC trial, as noted above. In contrast to our current findings, MAVARIC investigators observed the FPGS was less sensitive than manual screening in detecting CIN2+ lesions.5 Although both the MAVARIC trial and this Ontario trial are rigorous, unbiased, prospective comparisons of manual and automation-assisted reading of Pap tests, there are significant differences between these 2 validation studies with respect to both the laboratory environments and the trial methods. Four differences in particular may have contributed to the various conclusions. First, screening in the United Kingdom is performed by cytoscreeners whose sole responsibility is screening of Pap tests.11 The United Kingdom Code of Practice describes specific procedures to limit visual fatigue and proscribes daily maximal hours and anticipated annual productivity that are much lower than the permitted levels for an Ontario CT. In Ontario and most of North America, CTs screen Pap tests and may have other responsibilities in the laboratory and nongynecologic cytology. The licensing body for Ontario CTs recommends that a CT with no other duties or distractions screen no more than 60 slides in an individual 6-hour shift.12 In the current study, most CTs in the 2 laboratories screen between 50 and 60 slides during their working day. Given these differences, it is possible that the sensitivity of manual screening in an academic laboratory in the United Kingdom devoted only to gynecologic cytology is superior to jurisdictions working under different regulations.

Second, the MAVARIC trial also differs from our Ontario trial with respect to postscreening QC. MAVARIC adopted postscreening rapid review only as a QC measure for the primary screening. In contrast, the current Ontario trial followed routine QC procedures of rescreening high-risk cases based on clinical/historic information. In addition, the FPGS software selected at least 15% of slides that initially were interpreted as negative based on quintile score. If the FPGS QC review procedure is superior to rapid review, then the sensitivity of the overall FPGS relative to manual screening would be enhanced.

Third, the pretrial training of MAVARIC cytoscreeners may have been suboptimal and may have led to poor performance of the FPGS. Our Ontario study required 17 weeks of CT training on the FPGS to ensure adaptation to the new review technique and to maximize CT FOV detection capability. Even then, FOV review false-negatives still occurred. In contrast, the MAVARIC trial required only a 5-day period of training. After the initial training, cytoscreeners reviewed an additional test set consisting of only 100 slides. This brief training period may have led to a lack of skill in FOV review in the MAVARIC study.13 Finally, MAVARIC used histopathologic confirmation as the “gold standard” rather than a consensus cytodiagnosis that was used in the current study.

Another study did address the performance of the FPGS relative to FocalPoint Slide Profiling using an historic comparison. In that study, the FPGS was adopted after 8 years of use of the FocalPoint Slider Profiler in primary screening. A higher proportion of LSIL and ASC-US cytodiagnoses were rendered by the adopted FPGS compared with the historic FocalPoint Slide Profiler results.14 The authors concluded that the FPGS was a “promising screening tool.”

The validation study performed by the 2 laboratories that participated in the current study does not support substitution of the FPGS for current practice, which consists of the manual screening method, assisted by the FocalPoint Slide Profiler. To date, neither laboratory has undertaken to substitute FPGS for this current practice. Recently, the National Health Service Cervical Screening Program issued guidelines for the implementation of “no further review” (NFR) using the FocalPoint Slide Profiler.15 The current Ontario practice consists of processing all SurePath slides on the FocalPoint Slide Profiler, 3.5% of which are rejected by the system based on physical and/or staining characteristics. A proportion of FocalPoint reviewed slides are identified as NFR. In 2011, this NFR proportion constituted 23% of all FocalPoint reviewed slides, and the vast majority of these cases (94%) were reported without manual review. The remaining 77% of FocalPoint reviewed slides had higher quintile scores and were subjected to manual screening review by a CT. Manually screened slides that had an assessment of abnormal were forwarded for pathologist review and sign-out. At least 10% of the manually reviewed slides that were assessed as “negative” and 6% of the NFR cohort, or approximately 16% of the entire FocalPoint reviewed cohort, were subjected to a second manual screening QC review before sign-out. The performance of the FocalPoint Slide Profiler is verified on an ongoing basis. Annually or whenever software or optical components are replaced, the performance of each FocalPoint instrument is compared with manual screening using a 1000-case comparison cohort. Second, positive, targeted, retrospective reviews of prior negative tests based on current HSIL+ findings are divided into 2 major categories—those caused by CT screening and those identified in the NFR fraction of the FocalPoint reviewed slides. Finally, the false-negative rate of each instrument is monitored through routine QC rescreening.

In summary, in this validation study in the Ontario laboratory environment, there was no evidence that FPGS has diagnostic performance that is different from manual screening for carcinoma, HSIL+, or LSIL+; however, the performance of FPGS in detecting lower grade lesions (LSIL and ASC-US alone) was less than that of manual screening. The FPGS performance could be enhanced through improved CT detection of abnormal cells in FOV images.

Acknowledgements

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. Acknowledgements
  8. FUNDING SOURCES
  9. REFERENCES

We thank Mr. Dan Tholen, MS, for his insights and statistical analysis and cytotechnologists L. Bilodeau, G. Brujan, R. Costa-Correa, A. Gracie, H. Lee, Z. Nausedas, J. Nowakowski, and T. Sochirca.

FUNDING SOURCES

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. Acknowledgements
  8. FUNDING SOURCES
  9. REFERENCES

This validation study received no extramural funding and was supported by the clinical laboratory quality-assurance programs of LifeLabs Medical Laboratory Services and Gamma-Dynacare Medical Laboratories of Ontario.

CONFLICT OF INTEREST DISCLOSURES

The authors made no disclosures.

REFERENCES

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. Acknowledgements
  8. FUNDING SOURCES
  9. REFERENCES