SEARCH

SEARCH BY CITATION

Keywords:

  • cytopathology;
  • correlation;
  • medical error;
  • quality assurance;
  • quality improvement

Abstract

  1. Top of page
  2. Abstract
  3. Medical Error
  4. Methods of Error Detection
  5. Limitations in the Methods of CH Correlation
  6. RCA of Correlation-Detected Errors
  7. Frequency of CH Correlation Discordance and Calculated Performance Measures
  8. Outcomes of Correlation-Detected Errors
  9. QI on the Basis of Errors Detected by CH Correlation
  10. FUNDING SOURCES
  11. REFERENCES

The process of cytologic-histologic correlation is highly valuable to the fields of both cytopathology and surgical pathology, because correlation provides a wealth of data that may be used to improve diagnostic testing and screening processes. In this study, overall improvement appeared to be driven largely by improvement in preanalytic Papanicolaou (Pap) test sampling, because longer institutional participation also was associated with improved sampling sensitivity. The authors hypothesized that Pap test sampling may have improved secondary to the introduction of liquid-based technology, which was implemented in many laboratories during the study time frame. Through the performance of continuous data tracking and retrospective root cause analysis to identify factors that may have influenced any observed changes in performance indicators, institutions may learn which initiatives are successful or unsuccessful. The future of correlation lies in the standardization of methods, the development of more formal and rigorous root cause analysis processes to determine system components underlying correlation discrepancies, and the active use of correlation data to redesign testing and screening processes for quality and patient safety improvement. Cancer (Cancer Cytopathol) 2011;. © 2011 American Cancer Society.

Cytologic-histologic (CH) correlation is a method of medical error detection1 that is used most frequently by cytopathology personnel to evaluate failures in cytologic screening or diagnostic tests. Simply put, CH correlation is the process by which cytologic and histologic interpretations are compared, generally from the same anatomic site, to determine whether they are concordant or discordant.2-9 In most scenarios, a discordant diagnosis meets the Institute of Medicine (IOM) definition of medical error.1 In this article, we review the methods by which CH correlation is performed and the results of CH correlation root cause analysis (RCA) that determine the sources of testing and screening failures. The use of these RCA data to drive system redesign for quality improvement purposes also will is discussed.

Medical Error

  1. Top of page
  2. Abstract
  3. Medical Error
  4. Methods of Error Detection
  5. Limitations in the Methods of CH Correlation
  6. RCA of Correlation-Detected Errors
  7. Frequency of CH Correlation Discordance and Calculated Performance Measures
  8. Outcomes of Correlation-Detected Errors
  9. QI on the Basis of Errors Detected by CH Correlation
  10. FUNDING SOURCES
  11. REFERENCES

The IOM defined a medical error as the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim.1 This definition encompasses all types of error, including those occurring in diagnostic testing and screening, and does not link patient outcome to error. Pathology laboratories traditionally have considered 2 types of error: errors of accuracy and errors of precision.10, 11, 12 Accuracy is the closeness of a measure to its true value. For example, a cytologic diagnosis is accurate if that diagnosis corresponds to the disease process in the patient. Precision is the degree to which repeated measures produce the same results; and, in screening and diagnostic testing, precision is often referred to as diagnostic reproducibility. For example, a diagnosis is precise if 2 cytopathologists independently examine the specimen slides and make the same diagnostic interpretation. A lack of reproducibility is problematic on several levels, because it may reduce trust among clinicians and pathologists and, at times, it may create uncertainty medical decision making. An error detected by CH correlation is usually an error of accuracy, although interpathologist disagreement about the cause of a correlation error is an example of an error in precision.11 Both of these error types are discussed in more detail below.

Medical errors, as described in the IOM report To Err Is Human, medical errors permeate all levels of health care.1 Patient safety researchers believe that most human errors are made in faulty systems; medical errors generally happen because of active events occurring in a system with latent conditions that lead to active failures.1 A surgeon amputating the wrong leg is an example of an active error.1 Latent conditions leading to this error may include the lack of mandatory time-out checks to prevent an error13-15 or an operating room that is excessively busy.

A specific concept that has an important bearing on medical error, especially errors detected by CH correlation, is the step-wise process of diagnostic testing and screening. This process has been described as the total testing process (TTP), which is a system-based framework for examining all possible interactions and activities that may affect the quality of cytologic and surgical pathology tests.16-18 For the process of CH correlation, the steps of interest occur in the preanalytic phase and the analytic phase for both tests. These 2 phases are defined by the activities that align with the clinical workflow internal and external to the laboratory as 1) the preanalytic phase (patient identification, specimen procurement, and transport) and 2) the analytic phase (specimen processing and interpretation). Errors that occur in individual steps19 may lead to an error detected by the CH correlation process.3

Methods of Error Detection

  1. Top of page
  2. Abstract
  3. Medical Error
  4. Methods of Error Detection
  5. Limitations in the Methods of CH Correlation
  6. RCA of Correlation-Detected Errors
  7. Frequency of CH Correlation Discordance and Calculated Performance Measures
  8. Outcomes of Correlation-Detected Errors
  9. QI on the Basis of Errors Detected by CH Correlation
  10. FUNDING SOURCES
  11. REFERENCES

Medical errors may be detected using a variety of methods, which often are described in a dichotomous manner, such as active or passive, retrospective or prospective, and self-reporting or third-party reporting, to name a few.20, 21 The method of detection determines the frequency and type of error and the severity of clinical outcomes associated with the error. For example, active error-detection methods, such as performing direct observations, detect a higher frequency of error than passive methods, such as reviewing charts or CH correlation diagnoses.11, 22 Active methods seek out errors in progress, and passive methods detect errors after they happen.

CH correlation most commonly is performed in a retrospective and passive manner and through third-party reporting.2-9 Because of this process, the determination of cause may be difficult because of the general inability to evaluate all steps in the testing process to determine where failures occurred.3 Because correlation evaluates only cases in which both a cytologic and a surgical pathology specimen have been procured, the method does not evaluate the majority of either all cytologic or all surgical pathology cases and, thus, cannot provide a frequency of all cytologic or surgical pathology errors.3 Because correlation generally evaluates cytologic specimens that are antecedent or concurrent to surgical pathology specimens, the method focuses more on detecting cytologic rather than surgical pathology errors.

The process of CH correlation is regulated by the Clinical Laboratory Improvement Amendments of 1988 (CLIA 1988) and is viewed as part of a laboratory's overall quality-control program.23 The CLIA 1988 regulations stipulate that laboratories must establish and follow policies and procedures written for a program that is designed to detect errors in both the performance of cytologic examinations and the reporting of results. This program must include laboratory comparison of the available clinical information with cytology reports as well as comparison of gynecologic cytology reports of high-grade squamous intraepithelial lesion (HSIL), adenocarcinoma, or other malignant neoplasms with the histopathology report, when available in the laboratory (either on site or in storage), and a determination of the causes of any discrepancies.23

In some laboratories, CH correlation is performed on a prospective basis, often on surgical pathology specimens, before specimen sign-out, with antecedent cytology. This process is a form of quality improvement and lowers the error frequency for surgical pathology specimens.11, 24, 25

Limitations in the Methods of CH Correlation

  1. Top of page
  2. Abstract
  3. Medical Error
  4. Methods of Error Detection
  5. Limitations in the Methods of CH Correlation
  6. RCA of Correlation-Detected Errors
  7. Frequency of CH Correlation Discordance and Calculated Performance Measures
  8. Outcomes of Correlation-Detected Errors
  9. QI on the Basis of Errors Detected by CH Correlation
  10. FUNDING SOURCES
  11. REFERENCES

North American laboratories are not standardized in the performance of CH correlation.26 A specific method for the performance of correlation for Papanicolaou (Pap) test-histologic specimen pairs is not outlined in the CLIA 1988 regulations.23 The performance of CH correlation for nongynecologic specimens is not governmentally regulated.

The lack of a nationally standardized method for the performance of CH correlation is problematic for several reasons. First, because CH correlation is performed in a nonstandard way, it is difficult, if not impossible, for laboratories to compare data.26 Second, the lack of standardization results in laboratories performing CH correlation with variable rigor, which affects error detection frequencies. Because errors often have a negative connotation, a bias exists to not perform correlation in a manner to detect error.26-29 Third, the lack of correlation guidelines does not allow for laboratories to employ “best practices” to use correlation data for self-improvement.30

The ability to perform CH correlation depends on the laboratory information system. Electronic laboratory information systems vary in their ability to track cytologic and histologic correlation pairs.26, 30, 31 Vrbin et al conducted a survey of 162 American laboratories on current CH correlation practices26 to obtain evidence regarding the existing level of variability in correlation methods. The respondent laboratories sent tools that were classified into the categories of forms, logs, or tally sheets. Those authors constructed a list of variables to compare the amount of specific CH correlation information recorded by each institution. The list of variables was derived from the College of American Pathologists (CAP) Commission for Laboratory Accreditation Cytopathology Checklist and from the materials sent by the individual laboratories.32, 33 On the basis of the CAP checklist, 15 items (Table 1) were considered key variables when performing CH correlation, and Vrbin et al classified these variables as the minimum expected set of variables that could be collected. The response rate was 32.1% (52 laboratories responded), and a total of 84 CH correlation tools were obtained.26

Table 1. Fifteen Minimal Variables for the Performance of Cytologic-Histologic Correlation
1. Cytology case number
2. Sign-out cytology diagnosis
3. Sign-out cytologist
4. Original cytotechnologist diagnosis (for gynecologic cases)
5. Sign-out cytotechnologist (for gynecologic cases)
6. Review cytology diagnosis
7. Review cytologist
8. Surgical pathology case number
9. Sign-out surgical pathology diagnosis
10. Sign-out surgical pathologist
11. Review surgical pathology diagnosis
12. Review surgical pathologist
13. Significance of discrepancy (ie, effect on patient care or presumed impact on patient care)
14. Action taken (ie, what occurred as a result of identification of the discrepancy)
15. Reason for correlation (ie, if correlation was part of normal cytologic-histologic correlation, as a result of clinician concern, etc)

The only minimum expected variables listed by >50% of laboratories were cytology case number, sign-out cytology diagnosis, surgical pathology case number, and sign-out surgical pathology diagnosis. No laboratory listed all 15 of the minimum expected variables, and the majority of laboratories recorded data on <50% of the minimum expected variables. In summary, Vrbin et al demonstrated that the collection and recording methods of laboratory CH correlation data were nonstandard in the United States.26

In 2002, the United States Agency for Healthcare Research and Policy (AHRQ) funded 4 laboratories to evaluate the process and outcomes of CH correlation error detection.3, 30, 34 The investigators reported on several methodological issues related to CH error detection: definition of discordance gradient, look-back time period, institutional practice variation, nonstandard practices of evaluation of multiple specimens obtained from the same anatomic sites, and nonstandard practices of evaluation of evaluation of specimens from different anatomic sites.

First, the definition used for a discrepant cytologic and histologic diagnosis markedly affected the discrepancy frequency. In determining whether a diagnosis is discrepant, cytologic and histologic diagnoses must be compared, and these diagnoses are categorized using different scales. Some laboratories use semiquantitative scales, and other laboratories use more descriptive interpretations.3 Laboratory use of indeterminate diagnoses (eg, atypical, suspicious, etc) also varies. Many cytology laboratories use a semiquantitative scale for Pap tests and nongynecologic specimens in which standard diagnoses are associated with a graded probability of disease.3 The 2001 Bethesda System classification of Pap test diagnoses is a typical example of a semiquantitative scale.35 In many laboratories, cervical histologic specimens are graded using the cervical intraepithelial neoplasia (CIN) system.36 A challenge in performing gynecologic CH correlation is that the 2001 Bethesda System and the CIN system do not exactly correspond (eg, a Pap test diagnosis of atypical squamous cells does not have a correlate in the CIN system). Nongynecologic cytologic and histologic diagnostic classification systems exhibit even less correspondence, and many histologic nongynecologic classification systems are simply descriptive.3

In the AHRQ-funded project described above, the 4 institutions reached consensus on gynecologic and nongynecologic CH correlation diagnostic gradation scales, and these are listed in Table 2.3 The degree of difference between scaled cytologic and histologic diagnostic categories (ie, the number of steps) that were considered discordant affected the number of correlations performed. The institutions elected to evaluate only 2-step or greater discrepant diagnoses. The evaluation of 1-step diagnostic differences would have resulted in a far larger number of diagnostic discordant pairs. The variation in the number of discordant cases identified by a single institution based on using this scale compared with a less rigorous scale is shown in Figure 137; 1 of the major differences was in evaluating the number of discordant low-grade squamous intraepithelial lesion (LSIL) cases. That is, the higher the step-difference, the greater the probability of clinical consequences; for example, a benign/malignant discrepant pair had a greater likelihood of clinical consequences compared with a benign/atypical case pair. By using the IOM definition of error,1 all discordant case pairs are errors, although the choice of the number of step differences to evaluate in correlation analysis determines the amount of work that will be performed and the frequency of finding discordant cases with higher clinical impact.

thumbnail image

Figure 1. This chart illustrates variation in the number of discordant cases depending on the step difference in nonconcordance. The number of discrepant gynecologic cases detected by the cytologic-histologic (CH) process is plotted against the year (1998-2004) in which correlation was performed. The gynecologic cases are plotted according to the original Papanicolaou (Pap) test diagnosis (ie, no evidence of intraepithelial lesion [NL], atypical squamous cells of undetermined significance [ASCUS], atypical squamous cells-cannot rule out high-grade squamous intraepithelial lesion [ASC-H], atypical glandular cells [AGC], low-grade squamous intraepithelial lesion [LSL], high-grade squamous intraepithelial lesion [HSL], dysplasia not otherwise specified [Dysp, NOS], and glandular cancer/adenocarcinoma in situ [Gland CA/AIS]). At this institution, the grading scale for determining the presence of a CH discrepancy changed over time; consequently, the number of detected discrepancies changed. In 1998 and in 2003, the grading scale was more rigid (ie, more correlations were performed), resulting in the detection of more discrepancies; and, in 1999, 2000, and 2001, the grading scale was less rigid. These data indicate that the choice of scale in determining a nonconcordance determines the number of discrepancies detected. Source: Grzybicki DM. Assessment for best practices for standardized quality assurance activities in pathology. Paper presented at: Agency for Healthcare Research and Quality Improving Hospital and Laboratory Safety Meeting; May 19–20, 2006; Pittsburgh, PA.37

Download figure to PowerPoint

Table 2. Diagnostic Steps for Gynecologic and Nongynecologic Specimensa
 Gynecologic SpecimensNongynecologic Specimens
StepCytology DiagnosisSurgical DiagnosisCytology DiagnosisSurgical Diagnosis
  • Abbreviations: ASC-US, atypical squamous cells of undetermined significance; CIN, cervical intraepithelial neoplasia; HSIL, high-grade squamous intraepithelial lesion; LSIL, low-grade squamous intraepithelial lesions.

  • a

    Atypical and suspicious diagnoses are used in surgical pathology diagnoses although at a lower frequency than used in cytopathology diagnoses.

0No evidence of intraepithelial lesion or malignancyBenignBenignBenign
1ASC-USNo equivalentAtypicalAtypical infrequently used
2LSILCIN 1SuspiciousSuspicious infrequently used
3HSILCIN 2 or CIN 3MalignantMalignant
4Invasive carcinomaInvasive carcinoma  

The look-back time period between the initial cytologic diagnosis and the follow-up histologic diagnosis also affected the frequency of discordant cases examined in correlation analysis. Grzybicki demonstrated that the number of Pap test-cervical biopsy/curettage correlation cases varied based on the time interval between the antecedent Pap test and the follow-up histologic specimen (Table 3).37 In this instance, a cumulative 2-month look-back period detected only 69.8% of correlation case pairs compared with the number detected by a 6-month or greater look-back period.

Table 3. Proportion of Total Cytologic-Histologic Discrepancies Identified According to the Number of Months of Look Back After Acquiring the Surgical Specimen
Time of Look Back, moDiscrepancies, %
<120.9
128.8
220.1
310.2
47.1
54.8
63.5
>62.2
Surgical specimen accessioned before cytology specimen2.2
Indeterminate based on recorded data0.1

Several institution-specific factors affected the frequency of detecting discordant cases. For example, patient preference and institutional or individual clinic scheduling patterns affected the timeliness of follow-up colposcopic examination visits;3, 34 if the time interval between cytologic and histologic sampling was long, then short laboratory look-back periods did not detect some discordant case pairs. Institutions also displayed variable practices in procuring a histologic specimen if no lesion was identified on the colposcopic examination, which, of course, affected discrepancy frequency. For some nongynecologic specimen types, longer look-back intervals from surgical pathology specimen procurement were necessary to capture the appropriate clinical management follow-up intervals.37 For example, procedures to obtain histologic tissue after a thyroid gland fine-needle aspiration (FNA) generally were performed later compared with procedures after a lung FNA.

In some scenarios, laboratories were not standardized in correlation methods for multiple specimens obtained from the same anatomic site.3, 26 For example, histologic follow-up of an atypical Pap test may involve a cervical biopsy as well as a cone excision; some laboratories correlated both histologic specimens with the original Pap test, and other laboratories correlated only 1 of the histologic specimens.

Laboratories also were not standardized in their performance of correlation for cytologic and histologic specimens from different anatomic sites.3 For example, a woman who had metastatic ovarian cancer may have undergone exploratory laparotomy with an omental excision and a thoracentesis. Although the clinical question is the same for both the thorax specimen and the abdominal specimen (ie, is cancer present?), only some laboratories correlated the omental and pleural fluid specimen diagnoses.

RCA of Correlation-Detected Errors

  1. Top of page
  2. Abstract
  3. Medical Error
  4. Methods of Error Detection
  5. Limitations in the Methods of CH Correlation
  6. RCA of Correlation-Detected Errors
  7. Frequency of CH Correlation Discordance and Calculated Performance Measures
  8. Outcomes of Correlation-Detected Errors
  9. QI on the Basis of Errors Detected by CH Correlation
  10. FUNDING SOURCES
  11. REFERENCES

Cytology personnel use different RCA methods to determine the cause of errors in discordant cases.3, 26 On the basis of finding that the cytologic and histologic diagnoses are different, a general assumption is that a false-negative or a false-positive diagnosis has been rendered for 1 and/or the other specimen.

RCA methods using dichotomous assessment

Although the CLIA 1988 regulations23 do not specifically stipulate slide review, many laboratories perform RCA by initially re-examining the cytologic and histologic slides.3, 26 The secondary review forms the basis of establishing the cause of an error. In many laboratories, a cytopathologist performs the secondary review; and, in some laboratories, a cytotechnologist rescreens the cytologic slides, especially for Pap tests.26 Some training laboratories have cytopathology fellows solely perform the correlation. Secondary review is a commonly used method in anatomic pathology to detect error, and a weakness in this method of RCA is that it overemphasizes the active component of cognitive failure.11

In the most commonly used method of CH correlation RCA (ie, the traditional method), the cause of error is classified dichotomously into the categories of interpretation or sampling.2-9, 11, 26, 38-40 Although several biases (eg, hindsight bias)41, 42 limit the traditional approach, the cytopathologist and cytotechnologist slide review involves both technical skills (eg, moving the slide) and cognitive tasks in reaching a diagnostic interpretation.43 The second interpretation is compared with the original diagnostic interpretation, and differences are handled using several methods (eg, adjudication with input from the original pathologist, additional pathologists, outside expertise, etc).11 There has been little study of the optimal adjudication method.11

In the traditional method of secondary review, the cytology and surgical pathology diagnoses are considered reproducible or not reproducible. The cause of the discrepancy is classified as a sampling error if diagnostic material is present on either the cytologic or surgical pathology specimen (and was interpreted appropriately), but not the other (and also was interpreted appropriately). Sampling errors generally are considered false-negative errors and, conceptually, are believed to be failures that occurred in the TTP steps during or after specimen procurement and before diagnostic interpretation. A recent study by Hearp et al44 provided supporting evidence justifying sampling error as a valid cause of noncorrelation in women with HSIL who were followed by cervical biopsy alone. This conclusion was generated as a result of performance of a retrospective medical record review of patients with HSIL on Pap test and a noncorrelating cervical histologic specimen on initial follow-up. The authors then identified the subgroup of these patients who underwent a second follow-up cervical sampling procedure to validate or not the HSIL diagnosis. Examination of the final diagnoses for these specimens revealed that the majority of them confirmed a high-grade diagnosis. Because the definition of a sampling error is not standard, some cytopathologists consider the identification of specimen limitations (eg, abundant blood or obscuring inflammation) as sufficient evidence for the commission of a sampling error; more discussion of this practice is provided below.

The cause of the discrepancy is classified as an interpretation error if diagnostic material is or is not present on a slide and was not appropriately interpreted. Interpretation errors may be false-positive or false-negative errors and, conceptually, are believed to be failures that occurred in the TTP steps of diagnostic interpretation. In this traditional model, both the cytologic diagnosis and the histologic diagnosis may be classified as erroneous (eg, an under-called Pap test and an over-called histologic specimen).3

This traditional RCA method emphasizes the active component of error and de-emphasizes latent components. In other words, it is determined that the error was caused by a failure in human action, such as cognition (eg, cytopathologist interpreting a slide) or technical skill (eg, Pap test procurement), and not by underlying system failures. This dichotomous classification scheme has a connotation of blame, because it indicates that either clinical personnel or cytopathology personnel failed in a particular task.3 Multiple steps are involved in the procurement, processing, and interpretation of a specimen, as discussed above, and the classification of an error simply as either a sampling error or an interpretation error generally does not yield sufficient information in its consideration of the TTP to allow for improvement through process redesign.

For example, some errors traditionally classified as sampling may not represent true failures in tissue procurement or processing. The failure to observe tumor or preneoplastic cells on a slide actually may be attributed to patient-related factors or biologic variation, such as CIN regression, rather than a failure in diagnostic or screening steps. For example, a Pap test diagnosed as LSIL may be followed by a histopathologic cervical diagnosis of benign; on review, both the Pap test and the cervical biopsy diagnosis may be confirmed. The discordant pair may be secondary to a failure in clinical sampling or disease regression in the interval time period. The ability for the traditional form of RCA to truly determine the cause of discrepancy is limited. In this example, some may legitimately question whether an error has occurred, depending on the interval time between the Pap test and the follow-up colposcopic examination. However, some level of clinical uncertainty is unavoidable, and this example illustrates a system problem related to the entire process of Pap test screening/colposcopic examination follow-up.

Although the CH correlation process focuses on detecting errors of accuracy, the traditional dichotomous method of classifying root cause is associated with errors of precision.3 In a study that assessed reproducibility, cytopathologists from 4 institutions performed RCA on the same set of 40 previously identified, discrepant gynecologic and nongynecologic cases using the traditional dichotomous classification system.3 The pairwise kappa statistic ranged from .02 to .74, and 60% of pairwise kappa statistics were <.40, indicating overall poor agreement.3 In a similar study involving the dichotomous classification of 40 discrepant pulmonary case pairs, the pairwise kappa statistic ranged from −.154 to 1.0.45 The authors of those studies concluded that the lack of agreement in assigning root cause precluded confident targeting of these errors for quality improvement (QI) interventions with prospects of success.45

The less than optimal precision in assessing error root cause is attributable in part to the diagnostic discordance between the review and original pathologists and the review pathologists examining the same case. The frequency of secondary/original diagnostic disagreement most likely is greater in discordant case pairs compared with nondiscordant case pairs because of specific characteristics, such as biologic variation, issues of sampling quality, and challenges in interpretation (eg, that may be related to issues like experience).45 In other words, discordant case pairs are more difficult to interpret, and the high degree of difficulty is secondary to system factors, which are not identified using a dichotomous RCA method.

RCA methods using continuous assessment of specimen components

It is evident from the examples discussed above that dichotomous assessment of error root cause has several major weaknesses. Alternative methods consider multiple causes of error or shift the traditional binary assessment to a continuous assessment of factors.46-48 The No-Blame Box method of RCA is an example of this shift and involves a continuous assessment of 2 specimen factors affecting interpretation (ie, specimen quality and amount of neoplastic/preneoplastic or tumor tissue present).46, 47 Similar to the traditional dichotomous method, the No-Blame Box method involves the examination of cytologic and histologic slides, often retrospectively.46

A No-Blame Box is depicted in Figure 2. In the No-Blame Box, the amount of tumor is depicted vertically, increasing from no tumor at the top of the box to abundant tumor at the bottom. The overall quality is depicted horizontally, increasing from a poor-quality specimen at the left to an excellent-quality specimen at the right. Specimen quality reflects the ability to interpret the slide, and specimens of lower quality (eg, obscuring artifacts, thickness of the preparation, staining and air drying effects, etc) receive a lower quality grading. The right edge of the box depicts specimens of optimal interpretability.

thumbnail image

Figure 2. The No-Blame Box is illustrated with assessments of interpretive error. Although the No-Blame Box represents a continuous assessment of 2 variables, the box was divided arbitrarily divided into 4 quadrants, which represent 4 general categories of the level of tumor and specimen quality. In the traditional dichotomous assessment, Boxes A and B generally represent “sampling error,” Box D represents “interpretive error,” and Box C represents “sampling and interpretive error.” However, individual observers may use different criteria when assessing the components of interpretive and sampling failure for each case. The forms in each box represent different theoretical assessments of an interpretation (false-negative) error. The smallest form (ie, a dashed-line circle in the lower right corner) represents a strict adjudication or an interpretation error in which an observer requires a large amount of tumor in an excellent-quality specimen. The largest form (ie, a dotted-line rectangle extending into Boxes A and B) represents the least strict adjudication of an interpretation error in which an observer included cases in which a minimal amount of atypical cells in a poor-quality specimen are considered a false-negative. The other forms represent other adjudication processes that require more or less tumor (or atypical cells) and degrees of specimen quality. Source: Raab SS, Stone CH, Wojcik EM et al Use of a new method in reaching consensus on the cause of cytologic-histologic correlation discrepancy. Am J Clin Pathol. 2006;126:836-842.46

Download figure to PowerPoint

The No-Blame Box is a visual tool and does not involve the evaluation of individual practitioner performance per se. After examining the slides, the pathologist places a mark in the box at the location that best describes the amount of tumor and the specimen quality.46 The forms drawn on the box in Figure 2 reflect theoretical assessments of an interpretation error made by different pathologists. If the original cytologic specimen was classified as benign, then the smallest circle represents an assessment that a large amount of tumor in a high-quality specimen is necessary to identify that an error was secondary to an interpretation failure. The largest form indicates the criteria (a small amount of tumor even in a poorer quality specimen) that a second pathologist may use to classify an error as interpretive.

By using the No-Blame Box method, Raab et al measured the interobserver agreement of error cause adjudication in 40 bronchial cytologic and histologic specimens among 5 cytopathologists from different institutions.46 Each pathologist assessed whether the discrepancy was caused by sampling, interpretation, or both based on the location of the mark, with the upper 2 boxes representing sampling error and the lower 2 boxes representing interpretation error. Use of the No-Blame Box for cytologic specimens is illustrated in Figure 3, which indicates the 40 responses of 1 cytopathologist. All pairwise kappa statistics were .400, indicating acceptable to excellent agreement.

thumbnail image

Figure 3. Boxes A through D illustrate use of the No-Blame Box to assess error in 40 pulmonary cytologic-histologic correlation 2-step discrepancies. Source: Raab SS, Stone CH, Wojcik EM et al Use of a new method in reaching consensus on the cause of cytologic-histologic correlation discrepancy. Am J Clin Pathol. 2006;126:836-842.46

Download figure to PowerPoint

Cytopathologists may use data from the No-Blame Box method to understand more effectively the interplay between specimen quality and diagnostic interpretation. For example, if errors in pulmonary CH correlation pairs are categorized predominantly in the 2 left-hand boxes, reflecting poor specimen quality, then error reduction strategies would focus on improving specimen quality and assisting cytology personnel in interpreting poor-quality specimens.46 These QI efforts reflect a focus on targeting system problems rather than active human failures.

RCA methods using system assessment

Several methods of error RCA involve the assessment of system components of failure, although there are only a few published examples of the use of these methods to determine the cause of correlation errors. One such method is the Eindhoven Classification Model for the Medical Event Reporting System for Transfusion Medicine (ECM) (Table 4).14, 49-52 This method focuses on 3 domains: technical, organizational, and human. These domains are useful in classifying contributing factors and organizing causes of error, and they allow for error investigation to focus on system factors rather than entirely on human factors.

Table 4. Root Causes of Error Using the Eindhoven Classification Model
CodeCategoryDefinition
Errors that result from underlying system failures
Latent errors
Technical: Physical items such as equipment, physical installations, software, materials, labels and forms
TEXExternalFailures beyond the control of the investigating organization
TDDesignInadequate design of equipment, software, or materials; can apply to the design of workspace software packages, forms, and label design
TCConstructionDesigns that were not constructed properly; examples include incorrect set-up and installation of equipment in an inaccessible area
TMMaterialsMaterial defects found; examples could be the weld seams on blood bags, defects in label adhesive or ink smears on preprinted labels or forms
Organizational
OEXExternalFailures beyond the control and responsibility of the investigation organization
OPProtocols/ProceduresQuality and availability of protocols that are too complicated, inaccurate, unrealistic, absent or poorly presented
OKTransfer of knowledgeFailures resulting from inadequate measures taken to ensure that situational or site-specific knowledge or information is transferred to all new or inexperienced staff
OMManagement prioritiesInternal management decisions in which safety is relegated to an inferior position when there are conflicting demands or objectives; this is a conflict between production needs and safety
OCCultureA collective approach, and its attendant modes, to safety and risk rather than the behavior of just one individual; groups might establish their own modes of function as opposed to following prescribed methods
Active errors Errors or failures that result from human behavior
HEXExternalFailures originating beyond the control and responsibility of the investigation organization
Knowledge-based behaviors
HKK The inability of an individual to apply his or her existing knowledge to a novel situation
Rule-based behaviors
HRQQualificationsThe incorrect fit between an individual's qualification, training, or education and a particular task
HRCCoordinationA lack of task coordination within a health care team in an organization
HRVVerificationThe incorrect or incomplete assessment of a situation, including related conditions of the patient/donor and materials to be used before beginning the task
HRIInterventionFailures that result from faulty task planning and execution; this would be selecting the wrong rule or protocol (planning) or executing the protocol incorrectly (execution)
HRMMonitoringFailures that result from monitoring of process or patient status
Skill-based behaviors
HSSSlipFailures in the performance of highly developed skills
HSTTrippingFailures in whole-body movement; these errors are often referred to as “slipping, tripping, or falling”
Other factors
PRFPatient-related factorsFailures related to patient/donor characteristics or actions that are beyond the control of the health professional team and influence treatment
Unclassifiable Failures that cannor be classified in any of the current categories

The most effective time to perform RCA is immediately after an error has occurred; short time intervals help to identify latent components of error. Most cytology personnel perform CH correlation in a retrospective manner, as mentioned above, limiting the ability to assess these latent components. Raab et al used the ECM to classify errors detected by CH correlation in thyroid gland FNA with surgical pathology follow-up by first examining overall FNA performance data and then performing chart reviews for individual discrepant cases.52 In summary, those authors identified sources of error in all domains and observed that all errors had several latent and active components. The 2 main types and causes of error were 1) false-negative diagnoses caused by the “interpretation” of poor samples as non-neoplastic and 2) false-positive diagnoses caused by the interpretation of poor samples as neoplastic or nondefinitive.52-54 The cause of the interpretation and sampling errors involved several latent systems components. Understanding these components for the entire thyroid gland FNA service served as the basis for QI redesign efforts.

Frequency of CH Correlation Discordance and Calculated Performance Measures

  1. Top of page
  2. Abstract
  3. Medical Error
  4. Methods of Error Detection
  5. Limitations in the Methods of CH Correlation
  6. RCA of Correlation-Detected Errors
  7. Frequency of CH Correlation Discordance and Calculated Performance Measures
  8. Outcomes of Correlation-Detected Errors
  9. QI on the Basis of Errors Detected by CH Correlation
  10. FUNDING SOURCES
  11. REFERENCES

Because main error detection method in cytology is CH correlation, much of the published cytology patient safety literature is based on this detection method.2-9, 25, 55-58 Weaknesses in this body of literature stem from the lack of standardization of protocols, the lack of descriptions of underlying systems, and the limitation of single-institutional reporting.

Much of the CH correlation literature reports discordant case pair frequency data, although some studies use these data to calculate test performance measures, such as sensitivity and specificity. A few of these studies report multi-institutional data. For example, in a CAP Q-Probes study of 22,439 Pap test-histologic follow-up correlations from 349 laboratories, Jones and Novis reported sensitivity of 89.4% and specificity of 64.8%.2

On the basis of 1 year of data, the aggregated frequency of gynecologic CH correlation discordant case pairs (using a denominator of total number of case pairs correlated) was 4% (individual institutional range, 1.8%-9.4%).3 The aggregated frequency of nongynecologic CH correlation discordant case pairs was 10.8% (individual institutional range, 4.9%-11.8%). For some institutions, more than 1 in every 10 patients who had a correlating CH specimen pair had an error in diagnosis. Contingency tables indicated that gynecologic and nongynecologic error frequencies, regardless of the denominator used (eg, the number of correlating case pairs or the number of total cytology specimens), were institution-dependent (P < .001). For some correlation error types, some institutions had statistically significant higher error frequencies compared with other institutions (P < .001). Because these institutions standardized data-collection processes, these findings most likely represented differences in system processes and pathways. Laboratories that reported higher CH correlation discordant frequencies actually may have been better at detecting errors rather than being of poorer quality.

The institutional nongynecologic error frequency by anatomic site varied.3 For example, the frequency of CH lung discrepancies was 6% and 17% at 2 institutions. Contingency tables revealed a statistically significant association between institution and error cause for both gynecologic and nongynecologic errors (P < .001). Institutions A and B reported significantly fewer interpretation errors, and Institution C reported significantly more interpretation errors than expected. The majority of errors were attributed to cytology, rather than surgical, sampling or interpretation.

Outcomes of Correlation-Detected Errors

  1. Top of page
  2. Abstract
  3. Medical Error
  4. Methods of Error Detection
  5. Limitations in the Methods of CH Correlation
  6. RCA of Correlation-Detected Errors
  7. Frequency of CH Correlation Discordance and Calculated Performance Measures
  8. Outcomes of Correlation-Detected Errors
  9. QI on the Basis of Errors Detected by CH Correlation
  10. FUNDING SOURCES
  11. REFERENCES

Until relatively recently, the effect of errors detected by CH correlation have been unknown in larger datasets. Although medical errors affect several quality metrics, such as efficiency, timeliness, and patient safety,59-61 most published data examine patient safety. The assessment of patient harm requires the collection of clinical follow-up information. The degree of harm is classified using 1 of several standardized scales.62-65 A harm scale for outcomes associated with pathology diagnostic testing errors was generated by modification of several of the clinical severity scales described above and is provided in Table 5.3

Table 5. Error Clinical Severity Categories
No harm
 The clinician acted regardless of an erroneous diagnosis
  Example: A patient had a lung mass, and the clinician performed a bronchial washing and biopsy at the same time; the washing was diagnosed as malignant, and the biopsy was diagnosed as benign (sampling error); the clinician acted on the malignant cytology diagnosis regardless of the surgical diagnosis
Near miss
 The clinician intervened before harm occurred or the clinician did not act on an erroneous diagnosis
  Example: A patient had a lung mass, and a bronchoalveolar lavage was obtained and diagnosed as benign (sampling error); the surgeon proceeded with a therapeutic surgical procedure, because the radiologic evidence supported the diagnosis of malignancy; the diagnosis on the surgical specimen was malignant
Significant event
 Minimal harm (grade 1)
  a. Unnecessary, noninvasive further diagnostic test(s) performed (eg, blood test, noninvasive radiologic examination)
  b. Delay in diagnosis or therapy ≤6 mo
  c. Minor morbidity because of (otherwise) unnecessary further diagnostic effort(s) or therapy (eg, bronchoscopy) predicated on presence of (unjustified) diagnosis
 Moderate harm (grade 2)
  a. Unnecessary, invasive further diagnostic test(s) (eg tissue biopsy, re-excision, angiogram, radionuclide study, colonoscopy)
  b. Delay in diagnosis or therapy for ≥6 mo duration
  c. Major morbidity lasting ≤6 mo because of (otherwise) unnecessary further diagnostic efforts or therapy predicated on the presence of (unjustified) diagnosis
 Severe harm (grade 3)
  Loss of life, limb, or other body part or long-lasting morbidity (>6 mo)

The frequency of no harm, near miss, and harm events varied by institution. For example, for nongynecologic errors, the frequency of no harm events ranged from 32% to 74.3%, and the frequency of grade 2 harm events ranged from 0% to 45%.3 Examination of the outcome data for significant associated factors revealed that, for both gynecologic and nongynecologic errors, a statistically significant association existed between institution and assignment of error-associated clinical severity (P < .001). These findings illustrate the high level of variability in individual assessments of harm. Analysis of the data in aggregate revealed a harm severity assignment for gynecologic errors of 46% no harm, 8% near miss, and 45% harm events.3 The harm severity assignment for nongynecologic errors was 55% no harm, 5% near miss, and 39% harm events. If harm occurred, then it generally was assessed as grade 1 or 2 (minimal or mild).

Individual institutional error-associated harm also has been reported. For example, Clary et al reported that an individual laboratory's overall nongynecologic CH correlation discrepancy rate was 2.26% and 0.44% of all cytologic and histologic cases, respectively, and that 23% of discrepancies resulted in patient harm.6

In another example of an analysis of harm associated with nongynecologic CH discrepancies, Raab et al evaluated urine cytologic discrepancies for specimens obtained for the early detection and surveillance of urothelial carcinoma.66 CH discrepancies were observed in 208 urine specimens (40.9%) with histologic follow-up, and the cause of discrepancy was assessed as interpretation and sampling in 35.1% and 63%, respectively. Of all discrepancies, 101 (48.6%) resulted in either minimal or mild harm, consisting mainly of repeat testing and/or diagnostic delays; no cases were assessed as being associated with severe harm. The authors concluded that current screening and surveillance methods incorporating urine cytology were accurate in diagnosing cancer. However, the current protocols resulted in potentially reducible errors that lead to unnecessary testing and diagnostic delays.

Raab et al measured the frequency and outcome of cervical cancer prevention failures that occurred in the Pap and colposcopy testing phases involving 1646,580 Pap tests in 4 American hospital systems over a 6 years.58 By defining a screening failure as a 2-step or greater discordant Pap test result and follow-up histologic diagnosis, the authors recorded a total of 5278 failures (0.321% of all Pap tests), and 48% and 52% of failures occurred in the Pap test and colposcopy phases, respectively. Missed squamous cancers (1 in 187,786 Pap tests), glandular cancers (1 in 19,426 Pap tests), and high-grade lesions (1 in 6870 Pap tests) constituted 4.1% of all failures. Harm generally was classified as minimal or mild in degree. Unnecessary repeated tests or diagnostic delays occurred in 70.8% and 63.9% of failures involving high-grade and low-grade lesions, respectively. The authors concluded that cervical cancer prevention practices were highly successful in preventing squamous cancers, although a high frequency of failures resulted in low-impact negative outcomes.

QI on the Basis of Errors Detected by CH Correlation

  1. Top of page
  2. Abstract
  3. Medical Error
  4. Methods of Error Detection
  5. Limitations in the Methods of CH Correlation
  6. RCA of Correlation-Detected Errors
  7. Frequency of CH Correlation Discordance and Calculated Performance Measures
  8. Outcomes of Correlation-Detected Errors
  9. QI on the Basis of Errors Detected by CH Correlation
  10. FUNDING SOURCES
  11. REFERENCES

Several recently published studies have reported on the effectiveness of QI initiatives based on RCA findings for errors detected by CH correlation. These studies describe the use of both passive and active methods of QI. Passive methods have involved continuous data tracking of numbers of CH correlation errors, and active methods have involved process redesign using QI methods such as Lean.

Passive methods of QI: Data tracking

The CAP developed the Q-Tracks program for the self-reporting and continued tracking of several quality metrics, including CH correlation data. Raab et al evaluated the impact of continuous monitoring of correlation data on Pap test screening performance in 213 laboratories using a mixed linear model to determine whether the length of participation in the Q-Tracks program was associated with improved performance.67 During this period, institutions evaluated 287,570 paired Pap test-histologic correlation specimens and identified 98,424 (34.2%) true-positive Pap test correlations, 19,006 (6.6%) false-positive Pap test correlations, and 6575 (2.3%) false-negative Pap test correlations. The mean predictive value of a positive Pap test was 83.6%, sensitivity was 93.7%, screening/interpretive sensitivity was 99.2%, and sampling sensitivity was 94.2%. Longer participation was associated significantly with a higher predictive value of a positive Pap test (P = .01) (Fig. 4), higher Pap test sensitivity (P = .002), higher Pap test sampling sensitivity (P = .03), and a higher proportion of positive histologic diagnoses for a Pap test diagnosis of atypical squamous cells (P < .001).

thumbnail image

Figure 4. This chart illustrates participation in the Q-Tracks cytologic-histologic correlation program and reflects the predictive value of a positive Papanicolaou (Pap) test according to the mean increase in Pap sensitivity (%). Institutions are categorized according to the number of years they participated (eg, 7-8 years, 5-6 years, 3-4 years, or 1-2 years). Those institutions that participated longer had a higher mean increase in Pap test sensitivity. Source: Raab SS, Jones BA, Souers R, Tworek JA. The effect of continuous monitoring of cytologic-histologic correlation data on cervical cancer screening performance. Arch Pathol Lab Med. 2008;132:16-22.67

Download figure to PowerPoint

In that study, overall improvement appeared to be driven largely by improvement in preanalytic Pap test sampling, because longer institutional participation also was associated with improved sampling sensitivity. The authors hypothesized that Pap test sampling may have improved secondary to the introduction of liquid-based technology, which was implemented in many laboratories during the study time frame. Through performance of continuous data tracking and retrospective RCA to identify factors that may have influenced any observed changes in performance indicators, institutions may learn which initiatives are successful or unsuccessful.

The example presented above illustrates a passive method for determining causes of error reduction. Laboratories also may actively use CH correlation continuous data tracking to investigate the effectiveness of QI implementation strategies. Raab et al reported using statistical process-control charts to track CH correlation data after the implementation of QI initiatives.68 The use of statistical process control for the monitoring of pertinent quality-control indicators in industry and health care has been described extensively elsewhere.69, 70 One challenge in patient safety work is being able to demonstrate a postintervention change in error rates. The use of statistical process-control charts may be very helpful under these circumstances for revealing clinically significant trends in postintervention repeated measures.

In Figure 5, statistical process-control charts illustrate gynecologic CH correlation discrepancy data from 6 institutions.68 The arrow in 4 of the charts indicates the time at which a patient safety intervention, involving the standardization of correlation data-collection processes, was introduced and the detected number of errors before and after implementation. Project sites E and F in Figure 5 did not perform the same standardization. Analysis of the data using this method revealed a variable postimplementation error detection response among institutions. For only some institutions, the intervention resulted in improved error detection with the number of errors increasing, as would be expected.

thumbnail image

Figure 5. Statistical process control charts of gynecologic discrepancies at are shown from 6 institutions. Source: Raab SS. Improving patient safety by examining pathology errors: projected results. Paper presented at Agency for Healthcare Research and Quality Summary Meeting; August 24–25, 2007; Pittsburgh, PA.68

Download figure to PowerPoint

Several authors have evaluated test performance and errors detected by the CH process using likelihood ratios and receiver operating characteristic (ROC) curves. For example, using ROC curves, Raab et al demonstrated that pathologists interpreted bronchial brush specimens with higher levels of diagnostic accuracy when clinical history was provided compared with situations in which clinical history was absent.71 Raab et al also demonstrated that the likelihood ratio for individual diagnostic categories varied among pathologists, resulting in different clinically malignant probabilities.71 Renshaw et al reported that ROC curves could be used to determine the diagnostic thresholds of cytopathologists in gynecologic cytopathology; these data could then be used to drive diagnostic standardization processes.72

Active methods of QI: Process redesign

Because most errors are secondary to a variety of system failures, optimal efforts to improve patient safety focus on improving these systems, often through process redesign. The cytopathology literature describes the findings from several formal or informal RCAs of CH correlation error, but studies reporting the effectiveness of process design to reduce these errors are relatively scarce. Several examples of the effectiveness of process redesign are described below.

In 1 study, Raab et al reported that RCA of Pap test errors detected by CH correlation indicated that failures occurred during several main steps of the procurement process.73, 74 The authors developed checklists to evaluate for their utility process in assisting clinicians to improve the sampling quality of Pap tests. Checklists were provided to clinicians to focus on obtaining better samples and to cytotechnologists, who provided feedback of specimen quality to the clinicians on individual cases in a timely manner. Many of the processes that were changed (and measured by the checklists) were based on the use of Lean methods of redesign, such as creating “1-by-1” workflow.73-77

In another study, the investigators sought to determine whether the implementation of checklists and Lean methods resulted in improved Pap test quality and diagnostic accuracy in 5 clinician practices.73 The investigators performed a 1-year case-control study that included 5384 women in a control group (preintervention) and 5442 women in a case group (postintervention) who had a Pap test procured by 1 of 5 clinicians. The investigators compared the case and control Pap test quality and accuracy measures using the proportion of Pap tests that lacked a transformation zone component, the proportion of unsatisfactory Pap tests, and the frequency of newly detected CIN after a previous benign Pap test.

After the intervention, there was a statistically significant decrease in the mean proportion of Pap tests that lacked a transformation zone component (P = .011). Two of 5 clinicians had a statistically significant decrease in their unsatisfactory Pap test frequency. The case group had a 114% increase in newly detected CIN after a previous benign Pap test (P = .004). The dissemination of checklists and Lean methods resulted in improved Pap test quality and diagnostic accuracy.

Recently, surgeon and patient safety researcher Atul Gawande, MD, reported on the utility of checklists in a variety of medical fields for reducing errors, and the checklist described above for Pap testing is available online at: http://www.projectcheck.org/checklists.html (January 30, 2011).78, 79 In a second study, Raab et al reported on the effectiveness of the redesign of a thyroid gland FNA service based on the RCA of CH correlation errors discussed above.52 The redesign of practice resulted in the implementation of 2 interventions.

Intervention 1

A standardized diagnostic terminology scheme was developed and used. One source of error was the use of nonstandardized diagnostic criteria in assessing the adequacy of FNA specimens, which led to the making of definitive interpretations on poor-quality specimens.80 The cytopathologists developed and adopted standardized diagnostic terminology, including a nonspecific specimen category. Patient specimens that did not meet specific benign criteria and also lacked neoplastic features were diagnosed as nonspecific. The cytopathologists also developed a specimen-adequacy scoring system to assist cytopathologists in not making definitive diagnoses on noninterpretable specimens.53, 54 The scoring system was based on the grading of 3 criteria: cellularity, background, and preservation.

Intervention 2

The cytopathologists further expanded an immediate interpretation service. A second source of error was the lack of immediate feedback for a radiology service that performed thyroid gland FNAs. Immediate interpretation improves specimen quality by simultaneously targeting the procurement and interpretation processes and providing immediate information for the aspirator.

The effect of the 2 interventions on the outcome measures of individual cohorts was examined. The prestandardization cohort consisted of 1343 patients and 1543 FNAs, and the poststandardization cohort consisted of 1081 patients and 1176 FNAs.80

For the terminology standardization cohorts, there was a statistically significant difference in diagnostic category use (P < .001), with more nonspecific diagnoses and fewer atypical diagnoses in the post-terminology standardization cohort. The post-terminology standardization cohort had a higher noninterpretable specimen rate, because 155 specimens (13.2%) were classified as nonspecific, a category that had not been used previously. For the post-terminology standardization cohort, significantly fewer patients underwent surgery, had a false-negative diagnosis, underwent a repeat FNA, or had an atypical diagnosis.80 The sensitivity increased significantly, because fewer false-negative diagnoses were made.

Compared with patients who did not have an immediate interpretation, patients who had an immediate interpretation had a significantly lower noninterpretable specimen and repeat FNA rate. The greatest improvement in FNA performance was observed in patients who had an immediate interpretation using the standardized terminology; these patients had a high interpretable specimen rate and improved sensitivity. Compared with the preterminology standardization cohort, the poststandardization cohort had a statistically significant higher diagnostic accuracy (P = .05).80

In conclusion, the process of CH correlation has evolved over the past decade. The process of correlation is highly valuable to the fields of both cytopathology and surgical pathology, because correlation provides a wealth of data that may be used to improve diagnostic testing and screening processes. The future of CH correlation lies in the standardization of methods, the development of more formal and rigorous RCA processes to determine system components underlying correlation discrepancies, and the active use of correlation data to redesign testing and screening processes for quality and patient safety improvement.

FUNDING SOURCES

  1. Top of page
  2. Abstract
  3. Medical Error
  4. Methods of Error Detection
  5. Limitations in the Methods of CH Correlation
  6. RCA of Correlation-Detected Errors
  7. Frequency of CH Correlation Discordance and Calculated Performance Measures
  8. Outcomes of Correlation-Detected Errors
  9. QI on the Basis of Errors Detected by CH Correlation
  10. FUNDING SOURCES
  11. REFERENCES

Several of the published articles discussed in this review article were based on the authors' research funded by the Agency for Healthcare Quality and Research (1 RO1 HS 13321-01).

CONFLICT OF INTEREST DISCLOSURES

The authors made no disclosures.

REFERENCES

  1. Top of page
  2. Abstract
  3. Medical Error
  4. Methods of Error Detection
  5. Limitations in the Methods of CH Correlation
  6. RCA of Correlation-Detected Errors
  7. Frequency of CH Correlation Discordance and Calculated Performance Measures
  8. Outcomes of Correlation-Detected Errors
  9. QI on the Basis of Errors Detected by CH Correlation
  10. FUNDING SOURCES
  11. REFERENCES
  • 1
    Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: The National Academies Press; 1999.
  • 2
    Jones BA, Novis DA. Cervical biopsy-cytology correlation: a College of American Pathologists Q-Probes study of 22,439 correlations in 348 laboratories. Arch Pathol Lab Med. 1996; 120: 523-531.
  • 3
    Raab SS, Grzybicki DM, Janosky JE, et al. Clinical impact and frequency of anatomic pathology errors in cancer diagnosis. Cancer. 2005; 104: 2205-2213.
  • 4
    Joste NE, Crum CP, Cibas ES. Cytologic/histologic correlation for quality control in cervicovaginal cytology. Experience with 1,582 paired cases. Am J Clin Pathol. 1995; 103: 32-34.
  • 5
    Joste NE, Wolz M, Pai RK, Lathrop SL. Noncorrelating Pap tests and cervical biopsies: histological predictors of subsequent correlation. Diagn Cytopathol. 2005; 32: 310-314.
  • 6
    Clary JM, Silverman JF, Liu Y, et al. Cytohistologic discrepancies: a means to improve pathology practice and patient outcomes. Am J Clin Pathol. 2002; 117: 567-573.
  • 7
    Raab SS, Nakhleh RE, Ruby SG. Patient safety in anatomic pathology: measuring discrepancy frequencies and causes. Arch Pathol Lab Med. 2005; 129: 459-466.
  • 8
    Renshaw AA. Measuring and reporting errors in surgical pathology. Lessons from gynecologic cytology. Am J Clin Pathol. 2001; 115: 338-341.
  • 9
    Jones BA, Novis DA. Follow-up of abnormal gynecologic cytology: a College of American Pathologists Q-Probes study of 16,132 cases from 306 laboratories. Arch Pathol Lab Med. 2000; 124: 665-671.
  • 10
    Valenstein P. Quality Management in Clinical Laboratories. Promoting Patient Safety Through Risk Reduction and Continuous Improvement. Northfield, IL: College of American Pathologists; 2005.
  • 11
    Raab SS, Grzybicki DM. Quality in cancer diagnosis. CA Cancer J Clin. 2010; 133: 139-165.
  • 12
    National Oceanic and Atmospheric Administration (NOAA). NOAA website. http://celebrating200years.noaa. gov/magazine/tct/accuracy_vs_precision.html. Accessed December 19, 2010.
  • 13
    Reason J. Understanding adverse events: human factors. Qual Health Care. 1995; 4: 80-89.
  • 14
    Kaplan HS, Battles JB, Van der Schaaf TW, et al. Identification and classification of the causes of events in transfusion medicine. Transfusion. 1998; 11-12: 1071-1081.
  • 15
    Kalra J. Medical errors: an introduction to concepts. Clin Biochem. 2004; 37: 1043-1051.
  • 16
    Wolcott J, Schwartz A, Goodman C, The Lewin Group. Laboratory Medicine: A National Status Report, May 2008. https://www.futurelabmedicine.org/reports/laboratory_ medicine_-_a_national_status_report_from_the_lewin_group. pdf. Accessed January 18, 2011.
  • 17
    Boone J. Introductory remarks. Paper presented at: Institute on Critical Issues in Health Laboratory Practice: Managing for Better Health; September 23-26, 2007; Atlanta, GA.
  • 18
    Lundberg GD. How clinicians should use the diagnostic laboratory in a changing medical world. Clin Cim Acta. 1999; 280: 3-11.
  • 19
    Stroobants AK, Goldschmidt HM, Plebani M. Error budget calculations in laboratory medicine: linking the concepts of biological variation and allowable medical errors. Clin Chim Acta. 2003; 333: 169-176.
  • 20
    Meier FA, Zarbo RJ, Varney RC, et al. Amended reports: development and validation of a taxonomy of defects. Am J Clin Pathol. 2008; 130: 238-246.
  • 21
    Zarbo RJ, Meier FA, Raab SS. Error detection in anatomic pathology. Arch Pathol Lab Med. 2005; 129: 1237-1245.
  • 22
    Roy JE, Hunt JL. Detection and classification of diagnostic discrepancies (errors) in surgical pathology. Adv Anat Pathol. 2010; 17: 359-365.
  • 23
    Department of Health and Human Services, Health Care Financing Administration. Clinical laboratory improvement amendments of 1988: final rule. Fed Regist. 1992. 57 7146: §493.1-493.25; codified at 42 CFR §493.
  • 24
    Bewtra C, Pathan M, Hashish H. Abnormal Pap smears with negative follow-up biopsies: improving cytohistologic correlations. Diagn Cytopathol. 2003; 29: 200-202.
  • 25
    Ibrahim SN, Krigman HR, Coogan AC, et al. Prospective correlation of cervicovaginal cytologic and histologic specimens. Am J Clin Pathol. 1996; 106: 319-324.
  • 26
    Vrbin CM, Grzybicki DM, Zaleski MS, Raab SS. Variability in cytologic-histologic correlation practices and implications on patient safety. Arch Pathol Lab Med. 2005; 129: 893-898.
  • 27
    Cantaboni A, Pezzotta MG, Sironi M, Porcellati M. Quality assurance in pathology. Cytologic and histologic correlation. Acta Cytol. 1992; 36: 717-721.
  • 28
    Tan KB, Chang SAE, Soh VCH, Thamboo TP, Nilsson B, Chan NHL. Quality indices in a cervicovaginal cytology service. Before and after laboratory accreditation. Arch Pathol Lab Med. 2004; 128: 303-307.
  • 29
    Dodd LG, Sneige N, Villarreal Y, et al. Quality-assurance study of simultaneously sampled, non-correlating cervical cytology and biopsies. Diagn Cytopathol. 1993; 9: 38-44.
  • 30
    Raab SS. Improving patient safety by examining pathology errors. Clin Lab Med. 2004; 24: 849-863.
  • 31
    Grzybicki DM, Turcsany B, Becich MJ, Gupta D, Gilbertson JR, Raab SS. Database construction for improving patient safety by examining pathology errors. Am J Clin Pathol. 2005; 124: 500-509.
  • 32
    Hamlin WB. Requirements for accreditation by the College of American Pathologists Laboratory Accreditation Program. Arch Pathol Lab Med. 1999; 123: 465-467.
  • 33
    College of American Pathologists Commission on Laboratory Accreditation. Laboratory Accreditation Program: Cytopathology Checklist, 2007. Available at: http://www.cap.org/apps/docs/laboratory_accreditation/checklists/cytopathology_sep07.pdf. Accessed January 29, 2011.
  • 34
    Raab SS, Grzybicki DM, Janosky JE, et al. The Agency for Healthcare Research and Quality Hospital Labs Studying Patient Safety (AHRQ-HLPS) Consortium and Patient Safety. Am J Clin Pathol. 2006; 126( suppl 1): S12-S20.
  • 35
    Solomon D, Davey D, Kurman R, et al. The Bethesda System 2001: terminology for reporting the results of cervical cytology. JAMA. 2002; 287: 2114-2119.
  • 36
    Prendiville W. The treatment of CIN: what are the risks? Cytopathology. 2009; 20: 145-153.
  • 37
    Grzybicki DM. Assessment for best practices for standardized quality assurance activities in pathology. Paper presented at: Agency for Healthcare Research and Quality Improving Hospital and Laboratory Safety Meeting; May 19-20, 2006; Pittsburgh, PA.
  • 38
    Anderson MB, Jones BA. False positive cervicovaginal cytology. A follow-up study. Acta Cytol. 1997; 41: 1697-1700.
  • 39
    Tritz DM, Weeks JA, Spires SE, et al. Etiologies for non-correlating cervical cytologies and biopsies. Am J Clin Pathol. 1995; 103: 594-597.
  • 40
    Sidawy MK, Siriaunkgul S, Frost AR. Retrospective analysis of non-correlating cervical smears and colposcopically directed biopsies. Diagn Cytopathol. 1994; 11: 343-347.
  • 41
    Foucar E. Errors in anatomic pathology. Am J Clin Pathol. 2001; 116( suppl): S34-S46.
  • 42
    Thompson SC, Armstrong W, Thomas C. Illusions of control, underestimations, and accuracy: a control heuristic explanation. Psychol Bull. 1998; 123: 143-161.
  • 43
    Donovan T, Manning DJ. The radiology task: Bayesian theory and perception. Br J Radiol. 2007; 80: 389-391.
  • 44
    Hearp ML, Locante AM, Ben-Rubin M, Dietrich R, David O. Validity of sampling error as a cause in noncorrelation. Cancer Cytopathol. 2007; 111: 275-279.
  • 45
    Raab SS, Meier FA, Zarbo RJ, et al. The “big dog” effect: variability assessing the causes of error in diagnosis of patients with lung cancer. J Clin Oncol. 2006; 24: 2808-2814.
  • 46
    Raab SS, Stone CH, Wojcik EM, et al. Use of a new method in reaching consensus on the cause of cytologic-histologic correlation discrepancy. Am J Clin Pathol. 2006; 126: 836-842.
  • 47
    Raab SS, Thomison JB. Evaluating errors in cytologic-histologic correlation: a practical application of root-cause analysis. ASCP Check Sample. Cytopathology. 2009; 37: 41-53.
  • 48
    Koen TM, Mody DR, Scheiber-Pacht M, Fairley T, Thrall MJ. Limiting the use of atypical/inconclusive as a category in nongynecologic cytology specimens. Arch Pathol Lab Med. 2010; 134: 1016-1019.
  • 49
    Aspden P, Corrigan J, Wolcott J, et al. Patient Safety: Achieving a New Standard of Care. Washington, DC: The National Academies Press; 2003.
  • 50
    Simmons D. Sedation and patient safety. Crit Care Nurs Clin N Am. 2005; 17: 279-285.
  • 51
    Nodit L, Balassanian R, Sudilovsky D, Raab SS. Improving the quality of cytology diagnosis: root cause analysis for errors in bronchial washing and brushing specimens. Am J Clin Pathol. 2005; 124: 883-893.
  • 52
    Raab SS, Vrbin CM, Grzybicki DM, et al. Errors in thyroid gland fine needle aspiration. Am J Clin Pathol. 2006; 125: 873-882.
  • 53
    Poller DN, Ibrahim AK, Cummings MH, et al. Fine-needle aspiration of the thyroid. Cancer Cytopathol. 2000; 90: 239-244.
  • 54
    Poller DN, Stelow EB, Yiangou C. Thyroid FNAC cytology: can we do it better? Cytopathology. 2008; 19: 4-10.
  • 55
    Rasbridge SA, Nayagam M. Discordance between cytologic and histologic reports in cervical intraepithelial neoplasia. Results of a 1-year audit. Acta Cytol. 1995; 39: 648-653.
  • 56
    DiBonito L, Falconieri G, Tomasic G, Colautti I, Bonifacio D, Dudine S. Cervical cytopathology. An evaluation of its accuracy based on cytohistologic comparison. Cancer. 1993; 72: 3002-3006.
  • 57
    Cioc AM, Julius CJ, Proca DM, Tranovich VL, Keyhani-Rofagha S. Cervical biopsy/cytology correlation data can be collected prospectively and shared clinically. Diagn Cytopathol. 2002; 26: 49-52.
  • 58
    Raab SS, Grzybicki DM, Zarbo RJ, et al. Frequency and outcome of cervical cancer prevention failures in the United States. Am J Clin Pathol. 2007; 128: 817-824.
  • 59
    Committee on Quality and Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: The National Academies Press; 2001.
  • 60
    Richardson WC, Corrigan J. The IOM Quality Initiative: A Progress Report at Year Six. IOM Newsletter. 2002; I: 1-7. http://health.usf.edu/medicine/educationalaffairs/pace_files/IOM%20 Quality%20Initiative.pdf. Accessed January 28, 2011.
  • 61
    Institute of Medicine. Envisioning the National Health Care Quality Report. Washington, DC: The National Academies Press; 2001.
  • 62
    Battles JB, Kaplan HS, Van der Schaaf TW, Shea CE. The attributes of medical event reporting systems: experience with a prototype medical event reporting system for transfusion medicine. Arch Pathol Lab Med. 1998; 122: 231-238.
  • 63
    Dovey SM, Meyers DS, Phillips RL Jr, et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care. 2002; 11: 233-238.
  • 64
    Rubin G, George A, Chinn DJ, Richardson C. Errors in general practice: development of an error classification and pilot study of a method for detecting errors. Qual Saf Health Care. 2003; 12: 443-447.
  • 65
    Fernald DH, Pace WD, Harris DM, West DR, Main DS, Westfall JM. Event reporting to a primary care patient safety reporting system: a report from the ASIPS collaborative. Ann Fam Med. 2004; 2: 327-332.
  • 66
    Raab SS, Grzybicki DM, Vrbin CM, Geisinger KR. Urine cytology discrepancies: frequencies, causes, and outcomes. Am J Clin Pathol. 2007; 127: 946-953.
  • 67
    Raab SS, Jones BA, Souers R, Tworek JA. The effect of continuous monitoring of cytologic-histologic correlation data on cervical cancer screening performance. Arch Pathol Lab Med. 2008; 132: 16-22.
  • 68
    Raab SS. Improving Patient Safety by Examining Pathology Errors: Projected Results. Paper presented at Agency for Healthcare Research and Quality Summary Meeting; August 24-25, 2007; Pittsburgh, PA.
  • 69
    Baker RA, Newland RF. Continuous quality improvement of perfusion practice: the role of electronic data collection and statistical control charts. Perfusion. 2008; 23: 7-16.
  • 70
    Knapp RG, Miller MC 3rd. Analytic procedures for evaluating health care. Statistical control charts. Eval Health Prof. 1983; 6: 345-362.
  • 71
    Raab SS, Thomas PA, Lenel JC, et al. Pathology and probability. Likelihood ratios and receiver operating characteristic curves in the interpretation of bronchial brush specimens. Am J Clin Pathol. 1995; 103: 588-593.
  • 72
    Renshaw AA, Dean BR, Cibas ES. Receiver operating characteristic curves for analysis of the results of cervicovaginal smears. A useful quality improvement tool. Arch Pathol Lab Med. 1997; 121: 986-975.
  • 73
    Raab SS, Andrew-Jaja C, Grzybicki DM, et al. Dissemination of Lean methods to improve Pap testing quality and patient safety. J Low Genit Tract Dis. 2008; 12: 103-110.
  • 74
    Raab SS, Andrew-Jaja C, Condel JL, Dabbs DJ. Improving Pap test quality and reducing medical errors by using Toyota Production System methods. Am J Obstet Gyn. 2006; 194: 57-64.
  • 75
    Raab SS, Grzybicki DM, Condel JL, et al. Effect of Lean method implementation in the histopathology section of an anatomic pathology laboratory. J Clin Pathol. 2008; 61: 1193-1199.
  • 76
    Zarbo RJ, Tuthill JM, D'Angelo R, et al. The Henry Ford Production System: reduction of surgical pathology in-process misidentification defects by bar code-specified work process standardization. Am J Clin Pathol. 2009; 131: 468-477.
  • 77
    Condel JL, Jukic DM, Sharbaugh DT, Raab SS. Histology errors: use of real-time root cause analysis to improve practice. Pathol Case Rev. 2005; 10: 82-87.
  • 78
    Gawande A. The Checklist Manifesto: How to Get Things Right. New York: Metropolitan Books, LLC; 2009.
  • 79
    Gawande A, Boorman D. A Checklist for Checklists. http://www.gawande.com. Accessed January 30, 2011.
  • 80
    Raab SS, Grzybicki DM, Sudilovsky D, et al. Effectiveness of Toyota process redesign in reducing thyroid gland fine-needle aspiration error. Am J Clin Pathol. 2006; 126: 585-592.