Mandatory review of outside pathologic material is intended to detect interpretive errors that may have a clinically significant impact on patient care. Prior to definitive treatment of referred patients, the University of Iowa Carver College of Medicine requires a review of pertinent pathologic material previously obtained at outside institutions. The aims of this study were to determine if this local standard of practice has a measurable impact on patient care.
The pathologic diagnoses of 499 second opinion cytology cases seen at the University of Iowa Carver College of Medicine were studied. Each second opinion was classified as “no diagnostic disagreement”, “minor disagreement”, or “major disagreement” with respect to the originating institution's interpretation. The clinical impact of major disagreement cases was determined by pathologic and clinical follow-up via chart review.
Second opinion cytology resulted in 37 cases (7.4% of total cases) with major diagnostic disagreements. Clinical and pathologic follow-up was available in 30 of the major disagreement cases; second opinion diagnosis was better supported in 22 of these cases compared to the outside diagnosis. The second opinion in 6 major disagreement cases prompted changes in clinical management.
In 1993, the Association of Directors of Anatomic and Surgical Pathology recommended a standard practice of second opinion review of pathologic material for referred patients before definitive treatment, and many healthcare institutions have since adopted such a policy.1 A survey by Gupta and Layfield published in 2000 found that 50% of participating hospitals had a mandatory second opinion policy, and 38% encouraged second opinion review but did not have a formal policy in place.2 Several studies have examined the impact of second opinion review in surgical pathology since the early 1990s, with diagnostic disagreement rates ranging from 0.25% to 24%, and several of these studies have demonstrated that patient care is indeed directly altered by the process of second opinion review.3-10 There have been relatively few studies investigating the impact of second opinion review of cytologic specimen material. Of these, most are limited to specific organs or use cytologic specimens as a subset of broader-scope studies examining all referred anatomic pathology material.4, 5, 11-15 To our knowledge, there are no studies that have examined the clinical consequences or validation of second opinion review of cytologic material independent of anatomic site.
In this study, we sought to classify diagnostic disagreements in referred cytopathology material based on the level of pathologic disagreement and the potential for change in clinical care based on the second opinion interpretation, followed by chart review validation. We then attempted to ascertain the impact on patient care of clinically significant interpretative disagreements by examining the rate at which medical, surgical, and prognostic decisions were changed based on the second opinion interpretation. Our results indicate clinically significant disagreements in a relatively high percentage of all cases reviewed; much higher, in fact, than the rates of clinically significant disagreements in many surgical pathologic second opinion reviews reported in the literature. In addition, there was a small but important fraction of cases with clinically significant interpretive disagreements that directly prompted a change in patient care. Clinical and pathologic follow-up of these major disagreements validated the second opinion interpretation most of the time. However, the originating institution's interpretation was validated in a minority of cases. Therefore, second opinion review of cytopathology should be considered a useful patient care practice in academic medical centers, with the recognition that second opinion does not necessarily reflect the gold standard in all cases.
MATERIALS AND METHODS
Outside Slide Review at the University of Iowa Hospitals and Clinics
At the University of Iowa Hospitals and Clinics (UIHC), our institution mandates “when tissues have been removed at other institutions…to be used as a basis for developing, recommending, or continuing a treatment plan…the tissues shall be sent to the Pathology Laboratory for a formal examination before implementing the treatment plan…” (University of Iowa Hospitals and Clinics Bylaws, Rules, and Regulations, Article VIII, Section 8). Therefore, many patients seen by our clinicians for referral care or second opinion have outside pathology studies (OSS) that are reviewed by our pathologists in the division of anatomic pathology.
The requisite outside pathology materials reviewed are the glass slides and the pathology report(s). Outside cytopathology cases are distributed to the cytopathologists in our division, and an interpretation is rendered. Our cytopathologists are all board certified by the American Board of Pathology (ABP), and all practice both surgical pathology and cytopathology in our institution. With the exception of one, our cytopathologists are also subspecialty board certified in cytopathology by the ABP.
Review of OSS Reports and Definition of Level of Agreement
We retrieved 499 OSS cytology reports at UIHC dating from January 2005 to December 2005 and January 2007 to July 2007. All reports were reviewed by 2 of the authors (N.L. and J.A.W.); neither reviewer was the pathologist of record (ie, signout pathologist) of these second opinion cases. The originating institution's report (outside interpretation) and the second opinion report from UIHC were compared, and cases were classified as no interpretive disagreement, minor interpretive disagreement, and major interpretive disagreement. Two methods were used to classify the level of interpretive agreement or disagreement. If both the outside pathologist and the second opinion pathologist used diagnostic terms in the spectrum of benign/normal, atypical, suspicious, or malignant, we used a 2-step discrepancy rule to define a major disagreement. For example, if a lung nodule aspirate was interpreted as “atypical cells” on the outside, but was called “carcinoma” on second opinion, the disagreement would be considered major. A minor interpretive disagreement was defined as a 1-step difference between the 2 opinions on the benign-to-malignant scale. Although most case interpretations used the nomenclature of the benign/atypical/suspicious/malignant classification, interpretations of cytologic material from some anatomic sites—most notably thyroid and uterine cervix—did not fit well into this scheme. In those cases, major interpretive disagreements were defined as a difference in opinion with the potential for significant change in treatment or prognosis as determined by the authors, using the prevailing local standards of practice within our institution. For example, the outside fine needle aspirate interpretation of a thyroid nodule was diagnosed as follicular neoplasm, whereas our second opinion was benign nodule. Note that for the purposes of this study, the interpretation of follicular neoplasm and follicular lesion were considered equivalent. The designation of follicular neoplasm/lesion does not fit well into the benign-to-malignant scale, but the difference between the 2 interpretations may have an impact on surgical management, as “follicular neoplasm/lesion” would certainly be triaged to a surgical resection at our institution, whereas “benign nodule” would allow for a recommendation of nonsurgical follow-up. Similar challenges in classification were encountered in screening cervical cytology tests. Although the level of interpretative difference between high-grade squamous intraepithelial lesion and low-grade squamous intraepithelial lesion may be seen as a 1-step discrepancy, the difference in subsequent management—frequently the difference between an excisional procedure and careful clinical follow-up—necessitated classification of such differences as major disagreements.
Classification of Second Opinion Data
The numbers of minor and major disagreements were enumerated and segregated into generic specimen types including: fluids (ie, washes, brushes, and scrapings), needle aspiration biopsies, and exfoliative cervical-vaginal cytology (CVC). Furthermore, CVC cases were separated into those performed as a screening procedure and those performed as a diagnostic procedure; a specimen taken from a postmenopausal woman for uterine bleeding would be an example of a diagnostic, or complaint-specific, procedure. This categorization of CVC was necessary, as similar interpretative differences in each setting (screening vs diagnostic) may not have similar clinical impact on therapy and follow-up. Review of any accompanying histologic material by the second opinion cytopathologist at the time of cytology slide review was documented. In some instances, cytologic material from >1 anatomic site or cytologic material from different dates of collection from the same patient was reviewed concurrently. If cytologic specimens from different anatomic sites were collected on the same day—for example, bronchial brush specimens from the right and left bronchus obtained during the same bronchoscopic procedure—those specimens would be enumerated as a single case. However, if cytologic specimens from the same site were collected on different days—for example, 3 voided urine specimens collected at 6-month intervals—each case was considered separately.
Chart Review Investigation of Major Disagreements
Chart review of OSS cases characterized as major interpretive disagreements was performed to determine: 1) the clinical impact of the revised diagnosis; 2) the number of cases that prompted changes in treatment as a result of the changed diagnosis; and 3) validation of the second opinion. For purposes of second opinion validation, the interpretation closest to the diagnosis based on subsequent histologic material was deemed correct. For example, an outside institution's interpretation of a bronchial brush was “atypical,” and the second opinion was “suspicious”; carcinoma was detected in subsequent endobronchial biopsies; therefore, the second opinion would be supported as the more correct interpretation.
There was no interpretive disagreement in 407 of 499 (81.6%) cases. Minor disagreements comprised 55 cases (11.0%), and major disagreements comprised 37 cases (7.4%). Of the cases with major disagreements, 33 came from community hospitals and laboratories, 1 case came from a community hospital after obtaining consulting opinions from another academic center, and 3 cases came directly from academic health centers. Clinical and/or pathologic follow-up was available for 30 of the 37 major disagreement cases; the second opinion interpretation was better supported in 22 cases (73.3%), the outside original opinion was better supported in 4 cases (13.3%), and both the outside and second opinions were equally supported in 4 cases (13.3%). The second opinion interpretation in 6 of the 37 cases with a major disagreement (1.2% of total cases reviewed) prompted changes in the clinical management of the patient. Pathologic or clinical follow-up was available for 5 of the 6 cases that prompted a change in the clinical management as a result of the second opinion (Fig. 1). The second opinion was validated in all 5 of these cases. In 10 of 37 (27.0%) cases with major disagreements, the second opinion cytopathologist concurrently reviewed histologic material. However, histologic material was not concurrently reviewed in any of the 6 cases in which changes in clinical management were made. Tables 1 to 3 summarize the number of major interpretive disagreements with respect to specimen type, anatomic site, and whether there was a change in clinical management.
Table 1. Major Disagreement Cases, Fluids
Specimen Type and/or Anatomic Site
Second Opinion Diagnosis
Documented Change in Management
N/A indicates not applicable.
Denotes cases in which histologic material was concurrently reviewed with cytologic material.
Of the 37 cases with a major interpretive disagreement, thyroid fine needle aspirate, CVC (screening procedures only), and urine cytology were the most common specimens to yield a major interpretive disagreement (9 of 37 [24.3%], 7 of 37 [18.9%], and 6 of 37 [16.2%], respectively). Other anatomic sites and specimens that displayed significant disagreements in 3 or fewer cases included: parotid aspirate, lung aspirate, liver aspirate, pancreas aspirate, retroperitoneal aspirate, neck aspirate, bronchial brushing, pleural fluid, and peritoneal scraping. Of the 6 cases with major disagreements that prompted changes in clinical management, 3 were thyroid fine needle aspirations, 2 were cervical-vaginal cytology (screening procedures), and 1 was a parotid fine needle aspiration (Table 4).
Table 4. Cases Prompting a Change in Clinical Management
Parotidectomy and neck dissection: squamous cell carcinoma
Patient underwent staging procedure
Biopsy: CIN I
Patient avoided excisional procedure
Biopsy: CIN I
Patient avoided excisional procedure
The subject of diagnostic error has been a point of emphasis in cytopathology for over 2 decades, due in part to the public's perception of deficiencies in pathologic interpretation of cervicovaginal screening samples.16 More recently, the controversial mandate by the Center for Medicare and Medicaid Services for mandatory cytology proficiency testing has also added vigor to the debate regarding the most appropriate manner to ensure delivery of the best patient care in the diagnostic cytopathology realm. Nongynecological cytopathology, although a bit more hidden from the public eye, is a rapidly growing area within cytopathology because of the increasing availability of new image-guided aspiration procedures, such as endoscopic and bronchoscopic ultrasound. In this context, it is important to consider ways to detect and minimize interpretative diagnostic errors on the part of the pathologist, and thereby improve patient care in a clinically relevant manner. To reduce diagnostic error, one needs a reasonable definition of error; a universal definition has proven rather elusive in the context of anatomic pathology. An assessment of diagnostic accuracy—meaning how close to the truth a diagnostic interpretation may be—is often quite difficult.17, 18 Subsequent pathologic and clinical follow-up are the best available gold standards to determine accuracy of cytopathologic diagnosis; however, it is not a practical way to reduce real-time errors in the delivery of healthcare to an individual patient. Second opinion pathology, in contrast, is a method to assess precision, or interobserver reproducibility, in diagnostic cytopathology, and thus represents a potential surrogate for accuracy. The advantage of second opinion cytopathology is that it can be performed before definitive clinical management, with the caveat that reproducibility of a diagnosis is not always equivalent to diagnostic accuracy.
In the present study, we attempted to determine whether it is possible to reasonably assess cytopathologic interpretative precision in the context of mandatory second opinion, and whether second opinion in cytopathology is a worthwhile patient care endeavor. We found major interpretive disagreements in 7.4% of cases reviewed. This rate is similar to Layfield et al's study, which found an 8% major discrepancy rate in a review of 146 outside cytology cases; this rate is much lower than the 21% discordant diagnosis rate in cytology cases reported by Abt et al.4, 11 Rates of clinically significant disagreements upon second opinion review range widely in surgical pathology investigations, but tend to be lower than those reported in cytopathology.3-15 A recent review of surgical pathology second opinion cases within our own institution found a major disagreement rate of 2.3%, similar to other major diagnostic disagreement rates for surgical pathology material reported in the literature.10 The reasons for higher discrepancy rates in cytopathology versus surgical pathology are likely multifactorial. First, assessment of diagnostic interpretative precision in cytopathology is more difficult than in surgical pathology. Cytologic interpretations can certainly be definitive, endpoint anatomic diagnoses, but pathologists often use equivocations and descriptive nomenclature to convey an interpretative cytopathologic impression. Cytologic interpretations frequently use modifiers such as “atypical” and “suspicious for” that are less prevalent in surgical pathology diagnoses. These modifiers are rarely standardized or well defined, and can be open to different interpretations among referral pathologists and clinicians. Furthermore, attempts to assign cytologic interpretations to limited, discrete categories such as benign, atypical, suspicious, or malignant may artificially simplify interpretations that were purposely made open-ended. Consequently, assessment of concordance in studies similar to our own may result in a higher rate of perceived major disagreements that, in reality, were only minor disagreements. As an illustration of the difficulty in defining concordance based on a 4-tiered scale within cytopathology, we can examine 2 cases of bronchial brushings that were included in our study. In 2 bronchial brushing cases classified as major disagreements, the outside interpretations were within the “malignant” category, whereas the corresponding second opinions were within the “suspicious” category, a 1-step difference that a priori would only have been coded as a minor disagreement. However, because of the prevailing local standards of practice within our institution, this difference is best classified as a major disagreement, as “suspicious” in this particular clinical context would have prompted a repeat diagnostic procedure to firmly establish a cancer diagnosis, whereas a “malignant” interpretation on cytology at our institution would allow for the option of definitive surgical and/or medical management.
Second, in many instances and anatomic sites, cytology is not intended to be the final diagnostic threshold before definitive treatment. The cytologic interpretation may used as a triage tool, directing clinicians to watch and wait, treat, or obtain histologic material depending on the clinical scenario. Cytopathology is also used as a screening test, and therefore higher disagreement rates may be both expected and acceptable. Finally, in our study, outside referral cases at our institutions were only reviewed by pathologists with specialty training in cytology. In most instances, the outside cases were from community pathologists whose general practice may not routinely encounter nongynecologic cytology specimens. Although community pathologist exposure to surgical pathology material may also be variable, it is likely the average community pathologist has more experience with surgical pathology specimens than cytology specimens. Lower disagreement rates may have been encountered if referral material was originally reviewed by specialists in cytopathology.
Of the 37 cases with a major interpretive disagreement, clinical management was changed based on the second review opinion in 6 cases (1.2% of total cases reviewed). Three of these cases were thyroid fine needle aspirates. In each case, the outside interpretation was “follicular neoplasm,” or “cannot exclude follicular neoplasm,” and the second opinion was “benign nodule.” As a result of the second opinion, in each of these cases, a total thyroidectomy or lobectomy was not performed. In 2 of 3 cases, the patients have been followed up for >2 years with normal exams, laboratory studies, and serial imaging, as well as repeat benign fine needle aspiration in 1 case; in these 2 cases, then, the second opinion was validated. In the third case, the patient had not yet followed up clinically, so validation of the second opinion cannot be assessed. It should be noted that in 1 thyroid aspirate case with a major diagnostic disagreement, the outside diagnosis was follicular neoplasm, and the second opinion review was benign nodule with cystic change. In most cases, this interpretive difference at our institution would result in close clinical follow-up instead of surgical management. However, because of the size of the nodule, a thyroidectomy was performed, and subsequent histologic examination revealed a microscopic focus of follicular variant of papillary carcinoma separate and distinct from the large cystic nodule. Although the originating institution in this case was credited as having the more correct interpretation, this was almost certainly a sampling issue. The cytologic material reviewed sampled the dominant nodule, which on histologic follow-up was confirmed as a cystic benign nodule. This case again demonstrates the difficultly in truly determining the correct interpretation, particularly in thyroid aspirates, where sampling may play an enormous role in diagnosis and subsequent treatment.
There were 2 screening cervicovaginal cytology cases in which clinical management was changed based on the second opinion. In each case, the outside interpretation of a Papanicolaou (Pap) test was high-grade squamous intraepithelial lesion, and subsequent cervical biopsy material showed mild squamous dysplasia (cervical intraepithelial neoplasia I). Had the Pap test not been reviewed, an excisional procedure of the cervix would be indicated, as a high-grade lesion may have been missed on colposcopy. Instead, on second opinion review both Pap tests were interpreted as low-grade squamous intraepithelial lesions, and the patients were spared an excisional procedure as a result of concordant Pap and biopsy results. In the last case, in which clinical management was changed based on the second opinion, a fine needle aspiration of a parotid mass was interpreted as a possible pleomorphic adenoma by the originating institution and as carcinoma at our institution after review. After additional fine needle aspirations of the mass and neck lymph nodes were performed at our institution showing squamous cell carcinoma, a parotidectomy with neck dissection was performed.
In each of the cases described above, there was an evident change in clinical management, but why were there only 6 examples in which clinical management was changed, when our study revealed 37 cases with a major interpretive disagreement? First, there were only 30 of 37 cases with follow-up documented in our institution's computer medical record. Second, a change in clinical management was only recorded if the clinician documented a clinical plan before review of the cytologic material and then documented a change in that plan after review. In many instances, review of the cytologic material had occurred before the patient's clinic appointment, and any changes in clinical management that may have been prompted by interpretive disagreements between the outside interpretation and second opinion were not documented in the patient's medical record. Third, referral cytologic material may have corresponding histologic material that is more diagnostically definitive. For example, bronchial brushing and endobronchial biopsies from a patient may both be sent for second opinion review. Although there may be a major disagreement between the originating institution's interpretation of benign and a review interpretation of suspicious or malignant on bronchial cytology, that disagreement is moot and unlikely to change clinical management if the endobronchial biopsy carries a diagnosis of carcinoma on both the outside pathology report and the referral report. Fourth, there were several instances in which there was a delay in diagnosis because of missed malignancies by outside review of cytologic material that were detected and subsequently diagnosed as malignant. For example, a patient with cytologic evaluation of multiple voided urine samples had carcinoma cells missed on the first urine analysis that were detected on second opinion review, but a subsequent specimen collected 6 months later was diagnosed as carcinoma by the originating institution. In cases such as these, no formal change in treatment plan was made at our institution, despite a significant time in delay of diagnosis. Finally, the outside material was reviewed or the second opinion interpretation was rendered after initiation of treatment in a small minority of cases.
Interestingly, the rate at which clinical management changes were made based on second opinion of cytology material review (1.2%) was nearly identical to a recent review of surgical pathology second opinion cases within our own institution (clinical management change in 1.21% of total cases reviewed), but there are important differences leading to these final percentages.10 In the surgical pathology cases, there was a lower rate of major disagreements (2.3%), but of those cases with a major disagreement, there was a higher likelihood that clinical management would be changed, such that >50% of the cases with a major disagreement resulted in a change in clinical management. Conversely, in the cytology cases, there was a much higher rate of major disagreements (7.4%), but a smaller percentage of major disagreements lead to changes in clinical management (16.2%). This again highlights the inherent differences between cytology and surgical pathology. Cytologic specimens are more likely than surgical pathology material to be used as a triage or screening tool to guide clinicians in a treatment algorithm.
Second opinion pathology is costly in terms of pathologists' time. One approach to reduce the number of cases reviewed would be to limit review to problematic sites. Indeed, our study would seem to suggest that, based on the rate of change in clinical management, thyroid and cervicovaginal cytology would be problematic sites and are critical for review, whereas review of other specimen types and anatomic sites would be less critical. However, prospective review and real-time discussions with clinicians at the time of cytologic material review may result in a higher rate of major disagreements that were not uncovered by our investigation. In turn, other specimen types or anatomic sites may be revealed to be just as critical for review. Until those prospective studies are carried out to definitively categorize low-risk cytologic material, it seems prudent not to restrict mandatory review to select anatomic sites, despite increased burden on pathologists' time.
In summary, we have shown that mandatory second opinion in an academic center is a useful patient care tool. We were able to demonstrate that clinicians do change clinical management based on second opinion review, albeit not uniformly. A continued review of outside material of all anatomic sites is also supported by our findings.