A central tenet in the Patient Protection and Affordable Care Act is the increased use of information technology (IT) to improve patient care. Although there are many suggestions for how IT can improve performance in clinicians' offices and in managing patients through the electronic medical record, areas in which IT can lead to improvements in cytology and patient care have not to the best of our knowledge been well defined. We have previously suggested that changes in laboratory information systems (LIS) are necessary for cytology laboratory managers to comply with the requirements of reporting critical values/critical diagnoses/urgent diagnoses/unexpected diagnoses. Without these changes, critical values in cytology can be no more than a recommendation, because without them there is no way a laboratory manager can perform quality assessment to ensure compliance. Herein we describe another opportunity for IT to improve patient care in cytology by allowing real-time quality assessment of screening sensitivity in gynecologic cytology.
A central tenet in quality assessment for any laboratory test is to ensure the performance of that test using controls. These controls should include every type of result that is reported for that test. If a laboratory test reports positive and negative findings, then positive and negative controls should be included. If the laboratory test also reports indeterminate results, than indeterminate results should also be included in the controls that are run.
A major exception to this rule is anatomic pathology. In surgical pathology and cytology, including gynecologic cytology, controls to ensure the correct screening and interpretation of the slide are not run. Although there are many possible reasons for this, historically it has been very difficult to deidentify controls. Surgical pathologists and cytologists are smart, and they are very adept at identifying cases that are seeded into their work. In surgical pathology and nongynecologic cytology, this deficiency has not proven to be a problem. Although there are individual studies that have identified large error rates in very specific specialty areas,[2, 3] and there may be wide variations in performance from laboratory to laboratory, at least some studies have suggested that for “good” surgical pathologists and nongynecologic cytologists, error rates in real life are <5%, usually <2%, and may be as low as 0.1%. However, error rates in screening and interpreting gynecologic cytology range from 0% to 40%, depending on the method and workload used, and in the United States, all of the laboratories that have participated in trials sponsored by the US Food and Drug Administration of automated assisted screening devices have had error rates of at least 15%. It is this difference in performance for gynecologic cytology that has driven the efforts to find alternative and better screening methods and to examine workload more closely.
This problem with gynecologic cytology is well known, and cytologists have tried to respond to it. The most well-known method to identify errors in gynecologic cytology is the review of negative slides that is mandated as part of Clinical Laboratory Improvement Amendments of 1988.6 However, because this is a nonblind review of negative cases alone, there are no controls involved in this process, and measures of performance are unreliable. Although it is true that some laboratories do a very good job reviewing negative cases, most often in settings with a very low workload, the only way they can document this performance is by performing an alternative quality assurance technique that does incorporate controls.
However, over 30 years ago, some investigators, most notably Hindman,[9-11] did the work of incorporating controls into gynecologic cytology by seeding unknown cases into the daily workload. Although this method worked, it was labor-intensive and required identifying abnormal cases, removing the dots, deidentifying them, renumbering them, and then incorporating them into the daily workload. Because to the best of our knowledge very little has been written about seeding slides since that time, it is not unreasonable to assume that the effort involved in seeding is labor-prohibitive in the majority of laboratories.
But is this still true? Is there a way to achieve the same results of seeding without the labor involved? We believe the answer is yes; we can do seeding and we can have controls in gynecologic cytology. To achieve this, there are 2 important questions that must be answered. First, what cases should be seeded? If we only include easy cases, the results are going to appear better than they should be. If we only include hard cases, the results are going to appear worse than they should be. Given the tenet expressed in our second paragraph, we know we must include all the types of abnormal results that are reported in the laboratory, including atypical squamous cells, atypical glandular cells of undetermined significance, etc. The result is that if one is going to do seeding one should include a random sample of all the abnormal cases that are seen in the laboratory, and there should be no chance for selection bias to influence which cases are included and which are not in the seeded cases.
The second question is the more difficult. How should we go about the nuts and bolts of actually seeding these cases? We believe the work involved in removing dots, deidentifying cases, and renumbering them are cost- and labor-prohibitive, and that other alternatives are more feasible. Specifically, a percentage of the laboratory's cases could be prescreened to identify abnormal cases that can be used as controls for routine screening. This does not involve “rapid” prescreening, although there are tremendous advantages to that technique.[12-15] What we are proposing is that a random percentage of the laboratory's workload is prescreened in the usual fashion by cytotechnologists to identify abnormal cases, but no marks are made on the slides. Instead, the cytotechnologist goes to the LIS and enters in a prescreen diagnosis that then remains hidden until the time the case is signed out. The slides are then returned to the regular pool of cases and screened in a normal fashion. At the time of sign out, if a cytologist tries to sign out a case as negative for intraepithelial lesion or malignancy that was identified as being abnormal on the prescreen (or whatever algorithm the cytology laboratory manager desires), the LIS will stop him or her before the case is released, and the case will be sent for further rescreening before it is signed out. The LIS will keep that prescreen diagnosis hidden until the very last step, and will then be able to calculate the sensitivity of screening, the sensitivity of prescreening, and specificities as well. In this way, by simply prescreening a random sample of cases, applying no dots, and entering the results in the computer, every laboratory can have a random sample of abnormal findings “seeded” into their routine screening, and every laboratory can have appropriate controls with which to perform quality assessment. A possible workflow for this proposal is presented in Figure 1.
Alternatively, the widespread adoption of liquid-based preparations for gynecologic cytology facilitates a protocol that is more similar to that of Hindman. In this approach, multiple slides would be prepared from a subset of cases representing the full spectrum of cytologic interpretations. The slides would be labeled with fictitious names, and corresponding fictitious patients would need to be created in the LIS. These cases would of course need to be checked to make sure that cells representative of the desired diagnosis were present on the slide. These slides would then be distributed with the routine workload as “prospective controls” or “knowns” in a manner similar to that described by Hindman (Fig. 2).[9-11] Again, the LIS plays a key role in enabling such a system via creating the fictitious labels and patient histories, flagging discrepant results, and keeping track of the performance statistics of the individuals examining slides. Although the logistics of managing such a system are complex, if it could be accomplished in the past with conventional preparations and in an environment with relatively little computer support, it certainly could be accomplished with today's information management tools.
In laboratories that use automated assisted prescreening devices, the methods we are proposing herein can also be used and provide a more rigorous, objective process for assessing the performance of the screening protocol as it has been implemented by the individual laboratory. Prescreening a percentage of cases (typically manually) before they are evaluated by the screening devices would be just as effective as prescreening cases before manual review. Similarly, seeded cases can also be run through these devices in a manner that is similar to how they are used in manual screening. Given recent concerns regarding false-negative rates with automated assisted screening devices if workloads are increased significantly over what is achievable with manual screening, more rigorous, real-time measures of the performance of these devices would provide much needed data to this debate.
This vision will not happen on its own. For it to happen, cytologists and the organizations that represent them need to start asking their LIS manufacturers for it. Although prescreening can work without being incorporated into the LIS, it is more labor-intensive, and the history of gynecologic cytology suggests that it simply will not be done, at least here in the United States. However, we can change this if we want. There is a real need for better quality assessment in gynecologic cytology. And there is real funding available for manufacturers who can demonstrate that their IT solutions can improve patient care. The cytology community 's job is to ask these manufacturers to provide them with the tools they need to get it done.