Flow cytometry and other technologies of cell-based fluorescence assays, such as confocal imaging and image capture morphometrics, are required to pass through regulatory review processes using decision criteria usually defined with a soluble analyte in plasma or serum samples in mind. The U.S. Food and Drug Administration (FDA) has entered into a public dialogue in the U.S. regarding their regulatory interest in laboratory developed tests (LDTs) or so-called “home brew” assays performed in most midsized and nearly all large university and regional reference laboratories. The absence of well-defined guidelines for validation of cell-based assays using fluorescence detection to quantitate molecules on cells has thus become a subject of concern for the International Council for Standardization of Haematology (ICSH) and International Clinical Cytometry Society (ICCS). Accordingly, a group of over 40 international experts in the areas of test development, test validation, and clinical practice of a variety of assay types using flow cytometry and/or morphologic image analysis were invited to participate in a two-day workshop to define how best to organize a set of practical guidelines useful to in vitro diagnostic (IVD) innovators, clinical laboratories, regulatory scientists, and laboratory inspectors. The focus of the group was restricted to fluorescence reporter reagents, although some common principles are shared by immunohistochemistry or immunocytochemistry techniques and noted where appropriate. The members attending the workshop, those writing various sections of the guidelines and those editing the documents volunteered their time. The workshop costs were supported by unrestricted educational grants made to ICSH and ICCS by a mixture of commercial reference laboratories, diagnostic companies, and pharma-related entities with an interest in quality clinical trials.
Why is this document needed by everyone in the cell-based diagnostics field? First and most important there is no such guideline for proper validation of measurements made on or within cells, in contrast to those for soluble analytes. Second, most of the assays performed by flow cytometry in clinical practice are performed as LDTs due in large part to the dearth of commercially available diagnostic assay kits. LDTs exist because scientific advances provided clinically valuable “esoteric” tests, but that at test volumes typically too low for a commercial vendor to be able to recoup the development and regulatory submission costs. It thus falls on experienced flow cytometry laboratories to develop and implement diagnostic tests that will be used in very small subsets of patients. This void of assay validation guidelines for cell-based clinical tests has caused regulators and laboratory professionals to look to chemistry-oriented guidelines, most commonly from the Clinical Laboratory Standards Institute (CLSI), for the validation of cell-based assays. While some standard validation procedures can be readily transferred from assays of analytes in solution to cell-based assays, others are more difficult to directly apply. Cell-based assays principally differ from clinical chemistry and immunoassay methods in that stable reference preparations are unavailable for the cell-based measurements. Traceability to analytes in solution is usually straightforward, but traceability of an analyte associated with a particular cell population is presently not feasible.
Nonetheless, by adopting standardized methods and quality control (QC) procedures, accurate and precise results can be obtained both between and within cell-based analytical laboratories. Yet some concepts used for soluble analytes are not readily adopted to cell-based measurements. For example, the concept of limit of blank is difficult to apply to samples with different cells in the mixture having potentially different levels of autofluorescence or background fluorescence. How can one practically look at repeatability over a 20-day period when no stabilized material exists and the viability of leukocytes is typically no greater than 72 h in most assays? Even the concept of repeatability to determine within-day assay imprecision differs from a chemistry assay, where replicates of 20 single measurements from separate aliquots are easily performed by fully robotic instruments requiring small specimen volumes. Conversely, each flow cytometric assay “measurement” actually represents the mean or median of 1,000 – >50,000 unique cellular measurements, but may not be repeated many times because of limited sample availability and stability. Linearity assessments of many flow cytometric assays are more challenging than solute measurements. For liquids it is predictable what mixtures of solutions will create. However, for cellular measurements, samples containing cells with high expression of an analyte mixed with others having low analyte expression, do not create a cellular population with intermediate expression, but retain the coexistence of distinct cellular subsets with high and low analyte expression. These important aspects of assay validation, which affect the determination of sensitivity, linearity, precision, and stability of cell-based fluorescence assays, lack appropriate guidance from European directives, CLSI documents, FDA guidances, or other laboratory standards from global regulatory bodies.
A measurement of molecules on cells is different from solute measurements. Cells also contain molecular entities that can be influenced by specimen processing and fixation, not only is there heterogeneity between cell subtypes, but also within defined cell types with cell activation. Moreover, many cell-based fluorescence assays involve the use of monoclonal antibodies that contain Fc portions binding to cell subsets, which express one or more specific Fc receptors. Cells can not only bind the reporter reagent through Fc receptors, but the monoclonal reagents can induce cell activation through the crosslinking of these receptors. Furthermore, Fc binding of monoclonal antibodies is but one source of so-called nonspecific cellular fluorescence inherent in most cell-based fluorescence assays. Additionally, the common use of intracellular staining introduces other factors that influence staining such as the intracellular availability of illcharacterized low affinity binding sites that are capable of binding monoclonal antibodies and the “free” fluorochromes or those fluorochromes attached to the reporter reagent.
Before envisioning a picture of insurmountable assay limitations, one needs the perspective that even with the added challenges of cell-based quantitative measurements; flow cytometry can be sensitive to below 1,000 molecules per cell with a high degree of precision. Many diagnostic assays have evolved from manual, labor-intensive, low precision, subjective assays to flow cytometric assays that are now the recommended methods used in clinical practice throughout the industrialized world. Table 1 lists some of the diagnostic assays performed in clinical laboratories throughout the world. Their frequency of use and rate of adoption is influenced by differences in national healthcare reimbursement policies and regulatory processes restricting access to approved products. These assays represent a new generation of testing, offering improved sensitivity, accuracy, reproducibility, and most importantly, enhanced clinical utility. ICSH and ICCS have recognized the lack of clear, scientifically based guidelines for validation, defining the expected performance of cell-based fluorescent diagnostic assays. Thus, jointly ICSH and ICCS have cooperated in the production of guidelines that should serve as a benchmark for such assays, irrespective of whether they are being used in patient sample testing, clinical trials (such as those in support of investigation into the performance of a new diagnostic or therapeutic product) or the development of a new IVD device using this technology.
Table 1. Cell-Based Fluorescence Assays in Current IVD Practice with Clinical Flow Cytometry Technology
|T cell subsets||✓|| || || || || |
|HLA-B27||✓|| || || || || |
|Leukemia/ Lymphoma/ MDS evaluations||✓||✓||✓||✓||✓||✓|
|CD34 stem cell counts|| || || || || ||✓|
|Genetic immunodeficiency assays||✓||✓||✓|| || || |
|Hematoflowimmunodifferential kit||✓||✓||✓|| || ||✓|
|Infection/sepsis (CD64, HLA-DR)|| ||✓||✓|| || ||✓|
|Anti-PMN titer (ANCA)|| ||✓|| || || || |
|PNH screen|| ||✓||✓||✓|| || |
|Chronic granulomatous disease and other genetic cause of PMN dysfunction|| ||✓||✓|| || || |
|Reticulocytes, including IRF|| || || ||✓|| || |
|FMH by anti-HbF or anti-RhD|| || || ||✓|| || |
|Allogenic transfusion detection|| || || ||✓|| || |
|Hereditary Spherocytosis and related defects (EMA test)|| || || ||✓|| || |
|Immunoplatelet count (CD61,CD42, CD41)|| || || || ||✓|| |
|Reticulated platelets or IPF|| || || || ||✓|| |
|HIT assay|| || || || ||✓|| |
|Genetic causes of bleeding or thrombo-cytopenia|| || || || ||✓|| |
ICSH and ICCS present this expert-driven guideline of the Lucerne, Maine workshop held during March 11–12, 2011, including subsequent iterative writings and editorial review by other experts in the field of diagnostic flow cytometry and assay development, as a practical guidance for laboratories seeking assistance in proper assay validation of cell-based fluorescence assays. The document should also be of indirect value to manufacturers of IVD devices for flow cytometric or similar cell-based assays, as well as to regulators of IVD assay approval and registration. In particular, we present this guidance recognizing the on-going evolution of clinical diagnostic laboratory practice, regulation of IVD devices and laboratory accreditation policies throughout the world. We believe the dearth of expert guidance has hindered efficient regulatory approval of flow cytometric diagnostic assays and instrumentation, due in large part to the frustration involved in inappropriate application of chemistry principles to cell-based measurements. We offer this two-year work product as a practical guideline for laboratories and a proposed guideline for regulatory bodies to build future guidance documents for IVD diagnostic cell-based assays.
ICSH and ICCS would like to acknowledge the generous financial support in the form of unrestricted educational grants and contributions of their expert employees of the following institutions: Abbott Diagnostics, Biogen, BD Biosciences, Beckman Coulter, Genoptix Laboratories, Horiba Medical, LabCorp, Mayo Medical Laboratory, Covance, Trillium Diagnostics, PhenoPath, Arista, and Verity Software House. In keeping with the spirit of support for international pursuit of laboratory excellence through quality practices and scientifically validated standardized procedures, ICSH and ICCS have arranged through the cooperation of the publisher, Wiley Blackwell, to have the entirety of this guideline for cell-based fluorescence assay validation freely available for internet download. Downloads are available at the ICSH web site, accessed through www.hemestandards.org or www.haemstandards.org. Links will also be available at the ICCS web site, www.cytometry.org, and the publisher's web site, www.onlinelibrary.wiley.com.
Figure 1. Attendees at the ICSH/ICCS Workshop on Practice Guidelines for Validation of Cell-based Fluorescent Assays. Dedham, Maine, March 2011. (left to right): Annalee Estrellado, Jin-Yeong Han, Curtis Hanson, Shabnam Tanqri, Patrick Jacobs, Anna Porwit, Dragan Jevremovic, Marie C Béné, Bruce H Davis, Ben Hunsberger, Bob Hoffman, David Barnett, Norman Purvis, Virginia Litwin, Horacio Vall, Teri Oldaker, Raul Louzao, Patrick O'Neil, Amar Dasgupta, Brent L Wood, Peter Gambell, Jitakshi De, Ming Yan, Steven Kussick, David Kaplan, Not pictured: T. Vincent Shankey (photographer).
Download figure to PowerPoint