Fax: (501) 296-1184
New applications for old assays and the importance of validation†
Article first published online: 17 OCT 2011
Copyright © 2011 American Cancer Society
Volume 120, Issue 1, pages 3–6, 25 February 2012
How to Cite
Hunt, J. L. (2012), New applications for old assays and the importance of validation. Cancer Cytopathology, 120: 3–6. doi: 10.1002/cncy.20196
See referenced original article on pages 18-25, this issue.
- Issue published online: 13 FEB 2012
- Article first published online: 17 OCT 2011
- Manuscript Accepted: 13 AUG 2011
- Manuscript Revised: 8 AUG 2011
- Manuscript Received: 3 AUG 2011
The article by Bishop et al1 in this issue of Cancer reports on the use of the molecular hybrid capture assay for identifying human papillomavirus (HPV) in cytology samples from head and neck squamous carcinomas. In gynecologic cytology specimens, of course, the hybrid capture technique is used in 1 of the common, standard, US Food and Drug Administration (FDA)-approved commercial assays for HPV. Other HPV detection technologies have been reported both for gynecologic cytology specimens and for tissue samples, and several are commercially available, FDA-approved assays.2 At first blush, a reader may think that the study by Bishop et al is an obvious extension of an accepted and standard technology being used in a slightly different sample type. Of course, the same assay likely will work on cytology from another tissue site or tumor type! However, Bishop and colleagues present a lot more than just reasonably valuable research results. They importantly highlight several critical aspects of assay validation in anatomic pathology practice that often are overlooked and deserve special emphasis: 1) Clinical validity is an important consideration in new assay validation; 2) many analytes will have several different detection methodologies, and there may be considerable controversy around the “best” approach; and 3) quality control, assay validation, and standardization for any new laboratory-developed test (LDT) are absolutely critical.
To begin, some still may be questioning whether there really is a clinical need to test head and neck squamous cell carcinomas for HPV and whether the HPV status will impact clinical decision making. These questions relate directly to what has been referred to as the “clinical validity” of an assay, and they are very important to ask before introducing new assays. It is my experience that most clinicians working with head and neck cancer patients have started to view HPV testing in certain squamous cell carcinomas as the standard of care. This is particularly true in tumors that involve the oropharynx, which includes tonsil, tongue base (lingual tonsil), and sometimes the soft palate. Many clinician colleagues have asked for HPV testing in oropharyngeal carcinomas to be done as a reflex testing before oncology treatment decisions are made, and they consider the HPV status in therapy selection. In contrast to concerns that have been raised in gynecologic samples, there is very little risk of false-positive results, because high-risk HPV does not appear to be present in normal tissues from the oropharynx.3 There is now a fairly remarkable body of literature about the importance of the association between HPV and squamous carcinomas related to both treatment responsiveness and prognosis.4 In fact, patients who have HPV-associated squamous cell carcinoma will have a much better prognosis than those who have squamous cell carcinoma secondary to tobacco use, and they also tend to be radiation responsive.5, 6 The article by Bishop et al also points out that it may be essential to test for HPV in patients with head and neck squamous cell carcinoma enrollment in some current clinical trials.1
Pathologists may find additional value in testing for HPV as a diagnostic assay in certain situations. In the setting of a squamous cell carcinoma to the neck with an unknown primary tumor, the pathologist can help direct clinicians to the primary site through HPV testing. Multiple studies have now demonstrated that detection of HPV in specimens from squamous cell carcinoma metastasizing to neck lymph nodes is strongly suggestive of a primary oropharyngeal site.7 Of course, not every metastatic squamous cell carcinoma in the neck is from head and neck mucosal sites. Rare cases of metastasis from distant organs, including the uterine cervix and lung, have been reported, and there would be potential for confounding HPV testing in some rare clinical scenarios for unknown primary tumor in the neck.8, 9
The literature about HPV in head and neck squamous cell carcinomas has been muddied by controversy surrounding the detection method for the virus. Various methods have been reported for all types of tissue sources, including cytology samples and formalin-fixed, paraffin-embedded tissues. The most common assays have included polymerase chain reaction (PCR), tissue-based in situ hybridization, and immunohistochemical makers (either for direct detection or with surrogate markers). Although some studies have demonstrated reasonable concordance between these methods, like the study by Bishop et al, in which only 1 case had discordant situ hybridization and PCR results,1 others have demonstrated a dissatisfying low concordance between these assays. A recent report from Schlecht et al demonstrated in situ hybridization sensitivities of only 21% and 58% using 2 different commercially available probes compared with a PCR-based assay.10 In contrast, p16 immunohistochemistry, which is strongly associated with the presence of HPV, appeared to have higher sensitivity (52% and 93%, again, using 2 different commercial antibodies).10
A very practical approach for detecting an HPV-associated squamous carcinoma in paraffin-embedded tissue samples is to perform an immunohistochemical stain for p16. Because of the viral cellular actions, p16 will be overexpressed in the presence of oncogenic HPV. Initially, there were issues related to the different clones of antibody having different performance characteristics. However, today, commercially available p16 antibodies can be validated in the laboratory and correlates very well with the HPV status. Despite the controversy about current direct viral detection methodology, many studies have now demonstrated that p16 immunohistochemistry is a very reasonable and reliable method for detecting HPV-associated squamous carcinomas.11 There is also some evidence that p16 can be used in cytology preparations as a surrogate marker for HPV, especially when performed on cell block material.12 There is still the possibility that p16 can be overexpressed in head and neck squamous carcinomas that are not related to HPV. Because p16 is the protein product of the cyclin-dependent kinase 2A (CDKN2A) tumor suppressor gene and this gene can be altered by mutation or genetic loss, altered expression secondary to mutation may be an unusual cause for false-positive results.
A final major issue that is highlighted by the Bishop et al report is the importance of assay validation and quality control are important especially for LDTs. In anatomic pathology, there has been a tendency to incompletely or inadequately validate new assays, especially when it is a marker that has been around but is being applied to a new type of lesion or tumor. Similar to the physician who uses a medication for “off-label” treatments, the pathologist must be particularly cautious of the ramifications of adapting an FDA-approved, commercially available assay for “off-label” uses. I personally think this is a terrific way to use our limited resources, applying well validated and often simple technology for the good of our patients. But, it is important to recognize that using technology in this manner can lead to performing a different assay, which is essentially an LDT. Currently, there is much debate about LDTs at the national level, but there are general expectations about validation and quality control in the laboratory that should be considered for new applications.13
Indeed, most currently reported PCR and in situ hybridization assays for HPV in squamous cell carcinoma of the head and neck represent laboratory-developed assays that are not marketed commercially as FDA-approved assays. This means that there is little standardization between laboratories, that control material is not consistent from laboratory to laboratory, and that performance characteristics may be largely unknown for the assays.14 It may seem perfectly reasonable to take a standard, FDA-approved assay for HPV detection in cervical cytology specimens and apply it to head and neck cancer cytology specimens. Technically, there should be very little difference between the sample preparations and the tumor cells themselves in these 2 different types of cancer. Although it may be tempting to simply adopt the gynecologic cytology FDA-approved assays to test for HPV in head and neck cancer specimens, this is not necessarily an acceptable leap, because new applications for existing tests often should be accompanied by validation in the laboratory.
Bishop et al have demonstrated the correct way to bring in a new assay, with a full validation that includes comparison of the assay with others reported in the literature or currently in use. They reported that the hybrid capture HPV detection approach in head and neck cytology preparations, either fine-needle aspirate samples or brush preparations from fresh tumors, was very reliable and comparable to both PCR and in situ hybridization approaches.1 Although they have a relatively small number of samples, their results demonstrate a reassuring lack of false-negative and false-positive results, and their data set would serve as excellent documentation for the validation of the assay for clinical use an LDT in their practice.
A final concerning question arises: Do we really need to revalidate every molecular assay when it is applied to a new organ site or tumor type? The answer depends on the test type, the clinical scenario, and other factors, including the clinical and specimen indications if it is a commercially available, FDA-approved test. In the case of the commercial assays for HPV in cervical cytology samples, there are fairly strict criteria and guidelines, and these assays should probably not be adapted without some form of in-house validation. Other commercially available assays may not designate the source tissue or such specific indications for the assay, and these are more easily used on any tumor or sample type. Finally, in the setting of an LDT, it is possible to design a validation set that includes different organ and tissue sites to create a more generic in-house test.
One potential impracticality to fully adopting the approach of Bishop et al, in which HPV testing is performed expressly on cytologic preparations (even from surgical cases), might be encountered when trying to obtain fresh brush cytology samples from resected tumors. Those of us who have participated in banking of fresh or frozen samples from tumors, for research or clinical testing, know that the disruption to the workflow in the grossing room can be challenging. However, the benefits that the authors report from a utilization point of view may overcome the workflow issues. For example, the authors reported efficiency gains through performing the assay batched with the other higher volume cytology samples. They report that cases could be interpreted on the same day. They also report reduced cost by using the hybrid capture technique, again through batching with the cytology workload.
As we move into an era in which we will focus heavily on containing the cost of health care, there also will be a need to critically evaluate the adjunctive assays that we use in anatomic pathology for diagnostic, prognostic, and therapeutic purposes. Assessment of why and how we do these tests will only become more important. Thus, there probably will be an even greater emphasis in anatomic pathology on clinical validity, assay validation, quality control and standardization, and cost and efficiency. The study by Bishop et al provides an excellent example of how to strategically and appropriately adopt an assay for use in the clinical anatomic pathology laboratory. Finally, as health care changes, so will our specimens. More and more tumors are being diagnosed on minute tissue or cytology samples, with adjuvant therapy before surgery or sometimes even replacing surgery. Cytology samples are a wonderful source of cells and DNA for molecular testing, and we should continue to develop assays for the precious material obtained from lower risk, minimally invasive procedures, such as fine-needle aspiration.
No specific funding was disclosed.
CONFLICT OF INTEREST DISCLOSURES
The authors made no disclosures.
- 7Detection of human papillomavirus-related squamous cell carcinoma cytologically and by in situ hybridization in fine-needle aspiration biopsies of cervical metastasis: a tool for identifying the site of an occult head and neck primary. Cancer. 2008; 114: 118-123., , .