On-site adequacy evaluations performed by cytotechnologists

Correlation with final interpretations of 5241 image-guided fine-needle aspiration biopsies

Authors

  • Oname O. Burlingame MD, PhD,

    1. Department of Pathology, Brigham and Women's Hospital, Boston, Massachusetts
    2. Harvard Medical School, Boston, Massachusetts
    Search for more papers by this author
  • Kessé O. Kessé CT (ASCP),

    1. Department of Pathology, Brigham and Women's Hospital, Boston, Massachusetts
    Search for more papers by this author
  • Stuart G. Silverman MD,

    1. Harvard Medical School, Boston, Massachusetts
    2. Department of Radiology, Brigham and Women's Hospital, Boston, Massachusetts
    Search for more papers by this author
  • Edmund S. Cibas MD

    Corresponding author
    1. Department of Pathology, Brigham and Women's Hospital, Boston, Massachusetts
    2. Harvard Medical School, Boston, Massachusetts
    • Department of Pathology, Brigham and Women's Hospital, 75 Francis Street, Boston, MA 02115

    Search for more papers by this author
    • Fax: (617) 739-6192


  • We acknowledge Kelly H. Zou, PhD, for her assistance with the statistical analysis.

Abstract

BACKGROUND:

Practice patterns regarding on-site assessment of the adequacy of image-guided fine-needle aspiration biopsies (FNABs) vary among laboratories, but in many laboratories primary responsibility rests with the cytotechnologists. On-site evaluation provides feedback on the need for additional passes and facilitates triaging of the specimen for time-sensitive ancillary studies. Prior studies have suggested that cytotechnologists can assess the initially obtained specimens correctly, but they are few in number and limited by small size. The purpose of this study was to assess the frequency with which our cytotechnologists were able to correctly assess specimens as adequate using a large-scale database that included a wide range of image-guided FNABs.

METHODS:

The frequency that on-site adequacy assessments of 5241 image-guided FNABs were correct was determined by correlating the cytotechnologists' assessments of adequacy with the final cytologic interpretation. An adequacy assessment was considered correct if the FNAB was ultimately reported as satisfactory and unequivocally benign or malignant. An adequate reading on a case that was ultimately reported as unsatisfactory, atypical, or suspicious was deemed “incorrect.” The effect of imaging modality was also analyzed.

RESULTS:

Of 5241 FNABs, 2784 (53%) were interpreted as adequate on site. Of these, 2637 (95%) were correctly considered adequate. Of the common biopsy sites sampled, the adequacy assessments for liver FNABs demonstrated the highest frequency for being correctly considered adequate (97%) and those for kidney FNABs showed the lowest (90%). Imaging modality had no effect on accuracy.

CONCLUSIONS:

Cytotechnologists are almost always correct when assessing initial FNAB samples as adequate. Cancer (Cancer Cytopathol) 2012;. © 2011 American Cancer Society.

Fine needle aspiration biopsy (FNAB) plays an important role in the evaluation of patients across many medical specialties. The well-documented accuracy of FNAB is associated with lower cost and morbidity compared with other biopsy techniques.1 Since the development of advanced imaging methods like computed tomography (CT), ultrasonography (US), and magnetic resonance imaging (MRI), image-guided FNAB has become a common method for sampling lesions in a wide variety of superficial and deep-seated body sites. Over the past decade, the number of FNABs has increased substantially, and image guidance is used to perform an increasing proportion of them.2, 3

A common part of many image-guided FNAB protocols is the microscopic evaluation of rapidly stained smears during the procedure to assess adequacy. Adequacy evaluations are performed by cytopathologists, cytotechnologists, and pathologist trainees and less commonly by clinical personnel without formal pathology training.4 Practice patterns regarding on-site adequacy evaluation of FNABs vary among laboratories, but in many laboratories primary responsibility rests with the cytotechnologists.

On-site evaluation increases the sensitivity, accuracy, and utility of FNABs.5 Attempts to replace on-site evaluation with supplemental methods, such as large needle biopsies, gross visual assessment, or cell block material, have proven unsuccessful.6-8 A few studies, however, have concluded that, given certain circumstances, on-site evaluation is only marginally useful9, 10 or not useful at all11-13 when incorporating variables like cost effectiveness, and it can add to the length of the procedure. To improve on the effectiveness of the procedure, many laboratories employ cytotechnologists on the front line of adequacy readings, with or without backup by a cytopathologist in difficult cases.

The purpose of this study was to assess the frequency with which our cytotechnologists were able to correctly assess specimens as adequate using a large-scale database that included a wide range of image-guided FNABs.

MATERIALS AND METHODS

During the 7-year period 2003 through 2009, 5241 image-guided FNABs were performed. Image-guided FNABs were included in the study if they were performed on patients at our institution with an on-site adequacy assessment. FNABs were performed by staff radiologists or radiology fellows and residents under the direct supervision of staff radiologists. FNAB samples were obtained using 25-20-gauge needles and aspiration technique, typically using 5-10 cc of suction. A nonaspiration technique was used if the specimens were hemorrhagic.14

Of 5241 image-guided FNABs performed, 4729 (90.2%) had an on-site adequacy assessment by 1 of 10 board-certified cytotechnologists. Typically, the first 1 or 2 specimens (including both the needle and syringe) were handed to a cytotechnologist, who expressed the material onto glass slides and prepared 1 or more air-dried and alcohol-fixed smears using standard techniques. The needle was rinsed in RPMI Medium 1640 (Invitrogen, Carlsbad, CA) and the resulting cellular suspension later sedimented for cell block sections. To determine adequacy, the cytotechnologist stained selected air-dried smears with a rapid Romanowsky-type stain (Hemacolor, Harleco, EM Science, Gibbstown, NJ). The wet slide was immediately examined under a microscope without a coverslip. After evaluating the stained smears, the cytotechnologist rendered an opinion about the adequacy of the sample and communicated this to the radiologist. If not adequate, the radiologist then determined if additional passes could be safely obtained. Not infrequently, a procedure was terminated before the sample could be confirmed as adequate because of patient discomfort or concern for untoward complications with additional passes. A cytopathologist provided on-site consultation to the cytotechnologist for the adequacy assessment on an as-needed basis (approximately 13% of cases).

To assess adequacy, the cytotechnologist examined the rapidly stained air-dried smears for the quantity of lesional cells and the quality of cell preservation. Sparsely cellular smears and/or those with poor cellular preservation or obscuring elements like blood and clotting artifact were considered “not adequate.” Smears with relatively abundant and well-visualized lesional material (eg, presumptive neoplastic cells, organisms) were considered adequate. The needle rinse was triaged for ancillary testing and cell block preparation per laboratory policy. Samples with abundant lymphoid cells were routinely triaged for flow cytometry; those with abundant inflammation for microbiologic studies, and presumptive soft tissue and renal neoplasms for cytogenetic analysis. In complicated cases, the cytotechnologist conferred with a pathologist regarding the triage of the needle rinse portion of the sample.

The frequency with which on-site adequacy assessments of 5241 image-guided FNABs were correct was determined by comparing the cytotechnologists' assessments of adequacy with the final cytologic interpretation. Adequacy assessments were considered correct if the biopsy result was ultimately reported as satisfactory and unequivocally benign or malignant. An adequate reading on a case that was ultimately reported as unsatisfactory, atypical, or suspicious was recorded as “incorrect.”

For sample group sizes greater than 30, statistical analysis was performed using a 2-tailed Z test to compare proportions with a threshold confidence interval of 95%. When 1 or both group sizes numbered 30 or below, a Pearson's chi-square test with Yates continuity correction was performed. For both forms of statistical analysis, P < .05 was considered statistically significant.

RESULTS

Of 4729 preliminary assessments, 2784 (53%) were assessed as adequate during the on-site evaluation. Two thousand six hundred and thirty-seven of these (94.7%) resulted in a definitive final cytologic interpretation that was unequivocally either benign or malignant. Thus, cytotechnologists had an overall error rate of 5.3% when evaluating an on-site FNAB as adequate. Multiple body sites were sampled over the study period, and the accuracy of adequacy evaluations for different sites is summarized in Table 1. The most common general sites (abdomen, retroperitoneum, lung, and bone) comprised 83% of the FNABs in this study. For the most commonly biopsied sites, cytotechnologists were most accurate with on-site adequacy evaluations of abdominal FNABs (95.9%) and least accurate with on-site adequacy evaluations of retroperitoneal FNABs (92.7%), which were significantly different (P = .0064). Of the specific organs sampled, liver FNABs comprised 21% of the FNABs and yielded the most accurate on-site adequacy evaluations by cytotechnologists (97.4%). Kidney FNABs comprised 8% of all FNABs and yielded the least accurate on-site adequacy evaluations by cytotechnologists (89.7%) for a specific organ.

Table 1. Frequency of Correct Adequacy Assessments of Image-Guided Fine Needle Aspiration Biopsy Procedures, Stratified by Body Part
Body SiteNo. of FNABs% Adequate (n)Correct (%)
  1. FNAB, fine needle aspiration biopsy; n, number.

Abdomen167564.2% (1075)95.9%
Liver110166.6% (733)97.4%
Pancreas22052.3% (115)92.2%
Retroperitoneum112752.0% (586)92.7%
Kidney43642.2% (184)89.7%
Lung79451.1% (406)94.8%
Bone77038.8% (299)95.3%
Neck14344.8% (64)82.8%
Thyroid5048.0% (24)58.3%
Axilla13863.8% (88)98.9%
Lower extremity13030.8% (40)92.5%
Mediastinum10352.4% (54)94.4%
Groin8756.3% (49)95.9%
Chest wall8550.6% (43)95.3%
Spine6043.3% (26)100.0%
Upper extremity4240.5% (17)88.2%
Buttock3138.7% (12)100.0%
Head1915.8% (3)100.0%
Pleura1250.0% (6)100.0%
Back1172.7% (8)100.0%
Breast862.5% (5)80.0%
Perineum560.0% (3)100.0%
Not specified10 (0)N/A
Total524153.1% (2784)94.7%

The annual results are summarized in Table 2. The accuracy of on-site adequacy evaluations by cytotechnologists ranged from 93.2% to 96.5%, with no definitive trend over the course of the study. In contrast, the number of FNABs performed increased steadily over the study period, with an average annual increase of 11%. Despite the increasing volume, the accuracy of the on-site adequacy evaluations did not differ significantly between the start (95.3%) and the end (93.8%) of the study (P = .49). Comparison of the annual accuracy of on-site adequacy evaluations and annual volume of FNABs is shown in Figure 1.

Table 2. Frequency of Correct Adequacy Assessments of Image-Guided Fine Needle Aspiration Biopsy Procedures, Stratified by Year
 Year
 2003200420052006200720082009Total
  1. FNAB, fine needle aspiration biopsy; n, number.

No. of FNABs5346607037377998349745241
% Adequate (n)52.2% (279)48.9% (323)52.6% (370)56.7% (418)58.3% (466)51.3% (428)51.3% (500)53.1% (2784)
% Correct95.3%93.2%94.3%94.3%95.5%96.5%93.8%94.7%
Figure 1.

FNAB volume and correctness of adequacy assessments over time.

A variety of guidance methods were used to guide the FNABs in this study; CT represented the majority. The accuracy of the on-site adequacy evaluations by cytotechnologists ranged from 92.9% (PET) to 96.3% (MRI) depending on the imaging modality used, but these differences were not statistically significant (Table 3).

Table 3. Frequency of Correct Adequacy Assessments of Image-Guided Fine Needle Aspiration Biopsy Procedures, Stratified by Guidance Modality
Guidance MethodFNABs (n)% Adequate (n)Accuracy (%)
  1. FNAB, fine needle aspiration biopsy; n, number; CT, computed tomography; MRI, magnetic resonance imaging; PET, positron emission tomography.

CT scan448652.2% (2344)94.9%
Ultrasound31070.0% (217)94.9%
Endoscopy23546.0% (108)93.5%
MRI9755.7% (54)96.3%
PET2751.9% (14)92.9%

The accuracy by individual cytotechnologists is displayed in Table 4, along with the number of adequacy evaluations performed during this time and the percentage of cases interpreted on site as adequate by the cytotechnologist. Accuracy ranged from 89.9% to 97.5% (median, 95.5%; mean, 94.7%; SD, 2.4%). This is shown in graphic form in Figure 2.

Table 4. Frequency of Correct Adequacy Assessments of Image-Guided Fine Needle Aspiration Biopsy Procedures, Stratified by Individual Cytotechnologist
 Cytotechnologist
ABCDEFGHIJ
Total cases (n)465539404401978705454224151297
% Adequate46.2%48.1%50.5%56.6%54.1%50.9%57.3%62.9%52.3%50.8%
Accuracy (%)92.1%95.8%97.5%96.9%95.5%95.8%95.4%92.9%89.9%95.4%
Figure 2.

The percentage of correct adequacy assessments stratified by individual cytotechnologists.

DISCUSSION

FNAB is accurate, cost effective, and safer than more invasive diagnostic procedures. Image guidance has increased the number and variety of lesions that can be safely sampled. Because of its safety and versatility, the number of image-guided FNABs continues to increase.2 Still, image-guided FNABs are invasive and not without risk. Methods that increase the effectiveness of FNAB procedures enhance the utility of this increasingly popular diagnostic procedure.

On-site evaluation of FNABs helps to determine if the quantity and quality of material are sufficient for diagnosis and if time-sensitive special studies such as microbial cultures or flow cytometry should be initiated. Such measures increase the accuracy of the diagnosis. Other potential benefits of on-site evaluation include fewer passes and thus fewer complications, optimized utilization of imaging facilities, improvement of FNAB skills (based on feedback on adequacy), and increased adequacy of concurrent histologic samples.15-17

The time and expense of an on-site evaluation, however, can limit its use at some institutions. Although the clinical value of on-site adequacy assessment—principally, the reduction in unsatisfactory specimens—has been well documented,5, 6, 10, 18, 19 there has been little investigation into how often assessments rendered by cytotechnologists or cytopathologists are correct. In a study of 5688 preliminary diagnoses rendered by cytopathologists, Nasuti et al reported 85% agreement between the preliminary and final diagnoses. The preliminary diagnosis was deferred in 12% of cases, with disagreement in 3%.20 Because preliminary diagnoses instead of adequacy evaluations were reported and only a subset of biopsies were obtained using image guidance, we cannot directly compare these results with those in our study. In a study of endoscopic ultrasound-guided FNABs, the authors showed that on-site adequacy assessments by cytotechnologists improved the diagnostic yield compared with no adequacy assessment at all.18 Savoy et al compared the accuracy of 1 cytotechnologist with 3 endosonographers in providing adequacy readings for endoscopic ultrasound-guided FNABs. Accuracy was measured against the final determination by a cytopathologist. The accuracy of the cytotechnologist's readings was 82%, higher than that for the 3 endosonographers. The adequacy evaluations were not made on site but rather in a simulated environment, blinded, and without clinical information, as in a proficiency testing encounter.4 Redman et al directly compared the accuracy of adequacy evaluations of thyroid FNABs performed by cytopathologists and cytotechnologists and found that both were highly accurate (97% vs 93%, respectively), although there was a statistically significant difference in favor of the cytopathologists.19 The reasons for this are not apparent, because cytotechnologists are highly skilled in rapid screening, and adequacy criteria are clearly defined for thyroid FNABs.

Our data on abdominal and liver FNABs are generally in keeping with those in the literature, given differences in study design. We found the highest accuracy for FNABs of the abdomen (as a general category) and liver (as a specific organ site)—96% and 97%, respectively. Silverman et al reported the accuracy of a preliminary diagnosis for liver and “abdominal” biopsies at 97% and 100%, respectively.21 A study of on-site adequacy evaluations by cytology fellows reported an accuracy of 74% for abdominal FNABs.12 Nasuti et al found 87% agreement between preliminary and final diagnoses for liver FNABs in their study.20

In this study, cytotechnologists demonstrated the lowest accuracy for retroperitoneal FNABs (93%). Kidney FNABs were the most common retroperitoneal site, and cytotechnologists were the least accurate (90%) at evaluating the adequacy of this organ compared with all other specific organ sites. Silverman et al reported a similar accuracy of 89% for kidney FNABs.21 So did Andonian et al, who found that the accuracy of cytotechnologist-performed adequacy evaluations of 118 kidney FNABs was 88%. A much higher proportion of specimens was interpreted as adequate on site than in our study (75 vs 42%), which suggests that the percentage of adequate FNABs does not affect the accuracy of adequacy evaluations.22, 23 In addition, logistic regression analysis demonstrated no relationship between accuracy and percentage adequacy at a particular body site in our study (data not shown). Other studies of on-site evaluations of retroperitoneal FNABs have been published, but specific data about these evaluations have not been reported.24-27

The accuracy of our cytotechnologists was 95% for lung FNABs. This is in keeping with results reported by others, who found 84% and 97% agreement between a preliminary and final diagnosis rendered by cytopathologists for lung FNABs.20, 21 When adequacy was reported by cytology fellows, accuracy was 92% for thoracic FNABs.12 We found that our cytotechnologists were accurate in their adequacy assessments in 95% of bone FNAB adequacy evaluations, similar to the 89% reported for on-site preliminary diagnoses of bone FNABs.21 Our cytotechnologists demonstrated 92% accuracy for pancreatic FNAB compared with 82% and 77% in prior studies.20, 21 Interestingly, the on-site evaluations were least accurate for the pancreas in both prior studies, whereas on-site evaluations were least accurate for the kidney in our study.

Although on-site adequacy assessment is performed for the majority of deep-seated lesions at our institution, adequacy evaluation is performed for only a selected minority of thyroid FNABs, usually for patients who have had a prior nondiagnostic result, which likely accounts for the low percentage of adequate specimens (48%). Cytotechnologists performed adequacy evaluations for only 50 thyroid FNABs during the study period, with an accuracy of 58%. Larger studies of unselected on-site thyroid FNAB adequacy evaluations showed higher accuracy rates of 82% and 95%.10, 28 In 1 study, the evaluations were performed by cytopathologists only10; in the other, the qualifications of the person performing the on-site evaluations were not specified.28 A larger percentage of thyroid FNABs were evaluated as adequate (68% and 99%) in those studies compared to ours (48%), but this is likely due to selection bias as described above.28, 29 Regarding the accuracy of an on-site preliminary diagnosis, others have found 86% and 83% agreement with the final diagnosis for thyroid and other head and neck FNABs, respectively; 100% agreement was found in a small study of 6 thyroid FNABs.20, 21

Generating accuracy data by organ can be a useful tool for quality improvement. Efforts at educational enhancement can be focussed on those areas that appear weakest. In our hospital, there are 2 monthly interdepartmental cytology-radiology conferences, 1 with the abdominal radiology division and the other with the thoracic radiologists. There is also a “director's rounds” for the cytotechnologists, begun in 2006, during which criteria for adequacy readings are reinforced, pitfalls discussed, and discrepant cases reviewed. Given the lower accuracy rates for kidney FNABs, recent sessions have been devoted to a review of the cytopathology of kidney tumors, and we hope to demonstrate improvement in the months and years to come. Similarly, generating data by individual cytotechnologist permits targeted feedback.

Data regarding the accuracy of on-site adequacy evaluations of FNABs using a specific imaging technology are rare. Although many studies have examined on-site evaluations for procedures guided by CT, US, fluoroscopy, mammography, or endoscopic ultrasound, only a subset utilized cytotechnologists exclusively.4, 5, 18, 30-32 In a study comparing MRI with CT and US for liver FNABs, with cytotechnologists for on-site adequacy evaluation, the authors did not include data on the accuracy of the evaluations, but they found no difference in the diagnostic yield among the 3 imaging techniques.33

An important limitation of this study is that the accuracy of the adequacy assessment was measured against only a broad (albeit definitive) general category for the final cytologic interpretation (positive or negative for malignancy). In many cases, the final interpretation rendered provided sufficient diagnostic information for patient management. It is likely, however, that in some instances the final cytologic diagnosis, although definitive with regard to malignancy, was not sufficiently specific, and further diagnostic procedures would be required. This would apply to cases called “positive for malignancy” but lacking further essential subtyping (eg, specific type of carcinoma, lymphoma, or sarcoma) because there was insufficient material for immunohistochemistry. With the increasing application of molecular testing of FNABs for therapeutic decision making, it may not be sufficient for a sample to be “positive for malignancy” if there are insufficient cells for EGFR, BRAF, or K-RAS testing. Determining the accuracy of the adequacy assessment for such circumstances would require a more complex analysis than that provided here.

The current payment structure in the United States reimburses laboratories for evaluations performed by cytopathologists but not cytotechnologists. Our study did not address whether reimbursement is adequate for the effort expended. Layfield et al determined that the costs of on-site evaluation of FNABs by cytopathologists exceeded compensation. In their study, the cytopathologist provided a preliminary diagnosis that was used for immediate therapeutic decision making, similar to a frozen section diagnosis.34 An immediate diagnosis is not needed for most FNAB procedures because therapeutic interventions are rarely performed at the time of the procedure, but important decisions are made regarding the number of passes performed, the need for passes obtained with larger needles, and the optimal processing method. Very few if any of the FNABs in this 7-year study required an on-site preliminary diagnosis for therapeutic purposes. If a FNAB requires “merely” an on-site adequacy assessment as opposed to a diagnosis, cytotechnologists are a potential alternative source of technical skill for on-site evaluations. In our laboratory, cytotechnologists are on the front line of adequacy interpretations, but for difficult cases backup is provided by cytopathologists. Assigning primary responsibility to the cytotechnologists (with cytopathologist support) harnesses the cytotechnologist's skills at slide preparation, sample allocation, and microscopic evaluation.4 A cytopathologist may not be cost effective (or even available) in some settings. Cytotechnologists represent a good marriage between skill and resource allocation in many settings where assistance for image-guided FNABs is needed.

FUNDING SOURCES

No specific funding was disclosed.

CONFLICT OF INTEREST DISCLOSURES

The authors make no disclosures.

Ancillary