Multiplatform comparison of multiplexed bead arrays using HPV genotyping as a test case

Authors

  • Simon R. Corrie,

    1. HPV Research Laboratory, Department of Pathology, School of Medicine, University of Washington, Seattle, 98109-4104, Washington
    2. Australian Institute for Bioengineering and Nanotechnology, Center for Biomarker Research and Development, The University of Queensland, Brisbane 4072, QLD, Australia
    Search for more papers by this author
  • Qinghua Feng,

    1. HPV Research Laboratory, Department of Pathology, School of Medicine, University of Washington, Seattle, 98109-4104, Washington
    Search for more papers by this author
  • Tiffany Blair,

    1. HPV Research Laboratory, Department of Pathology, School of Medicine, University of Washington, Seattle, 98109-4104, Washington
    Search for more papers by this author
  • Stephen E. Hawes,

    1. HPV Research Laboratory, Department of Pathology, School of Medicine, University of Washington, Seattle, 98109-4104, Washington
    Search for more papers by this author
  • Nancy B. Kiviat,

    1. HPV Research Laboratory, Department of Pathology, School of Medicine, University of Washington, Seattle, 98109-4104, Washington
    Search for more papers by this author
  • Matt Trau

    Corresponding author
    1. Australian Institute for Bioengineering and Nanotechnology, Center for Biomarker Research and Development, The University of Queensland, Brisbane 4072, QLD, Australia
    • Australian Institute for Bioengineering and Nanotechnology, Center for Biomarker Research and Development, The University of Queensland, Brisbane 4072, QLD, Australia
    Search for more papers by this author

Abstract

While previous studies have investigated the utility of Luminex technology in comparison to other standard techniques, there have been few studies directly comparing different bead-based assays. A key barrier to establishing Luminex technology in research or clinical laboratories is the apparent need to purchase not only encoded bead sets but also the Luminex instrument. However, as flow cytometry instrumentation continues to improve in sensitivity and in the number and diversity of detection parameters, a diverse range of bead-based assays is likely to emerge. Human papillomavirus (HPV) genotyping requires multiplexed analysis of 10–100 individual genotypes per sample, which is well suited to bead-based assays whilst technically challenging and costly for related technologies (e.g., qPCR). Here we performed an unbiased technical comparison between Luminex technology and our in-house 3-mercaptopropyl trimethoxysilane (“MPS”) bead platform, which has been designed for integration with generic cytometry instruments. In genotyping 200 clinical samples, we compared the two bead assays against the goldstandard Roche Line Blot (RLB) assay, and both performed well in receiver-operator characteristic (ROC) curve analysis. We also show instrument-based differences are a significant factor in comparing the methods, which needs to be considered in future comparative studies. These multi-platform analyses are important in establishing the validity of new methods, as well as highlighting specific advantages anddisadvantages of the assays for specific applications. © 2011 International Society for Advancement of Cytometry

Bead-based assays are emerging in semi-quantitative and mutiplexable format, with a myriad of potential applications in molecular diagnostics. Increased reaction kinetics and three-dimensional mass diffusion combined with the analytical power of particle-by-particle flow cytometry analysis has driven the development of multiplexed bead assay (MBA) equivalents of traditional ELISA, DNA hybridization, PCR, and even ChIP assays (1–8). Whilst there have been numerous comparisons between Luminex MBA technology and relevant solution-phase techniques (ELISA, microarray, qPCR), there have not, to our knowledge, been any comparisons drawn with other bead-based flow cytometric techniques.

Cross-platform analysis highlights advantages and disadvantages of specific assays relative to utility, cost, reagent availability and future development potential. Whilst previous studies have found strong correlations when comparing different commercial MBA platforms (9), MBA against traditional ELISA (8, 10), and also MBA against emerging multiple cytokine analysis platforms (11, 12), to our knowledge there has not been comparison drawn between MBA platforms for genotyping applications. A key issue in establishing Luminex technology in a research or clinical laboratory is the need for a Luminex cytometer which currently is designed exclusively for Luminex-specific bead-based assays. However, flow cytometric instruments have significantly improved detection capabilities in terms of excitation sources, sensitivity, and number of detection parameters for encoding and reporter analysis purposes. However, independent validation studies need to be completed when pairing MBAs with different cytometers due to the different sensitivities and technical specifications. Therefore, cross-platform analysis comparing Luminex technology against bead-based systems which can be used in combination with generic flow instrumentation may expand the accessibility of bead-based systems for specific applications.

HPV genotyping is a particular example in which bead-based assays can drastically reduce the complexity of assay development. While PAP smears can indicate the presence of pre-cancerous lesions, samples presenting often require further testing to estimate the likelihood of disease progression (13, 14). The Digene Hybrid Capture (HC) technique, which can identify the presence of high risk (HR) HPV via a pooled probe approach (15–17) is FDA-approved for identification of clinical samples harboring 13 HRHPV subtypes and is useful as an adjunct test to PAP testing to assess the need for those with minimally abnormal PAP results to undergo further testing and follow-up (18). However, the HC test is limited, as it does not determine the specific HPV type associated with the infection. Information regarding the genotyping of HRHPV infection is important to identify those cases in which a persistent infection of a single HRHPV subtype is present over a period of time, rather than once-off infections (19, 20). This requires repeated detection of individual viral subtypes which is currently beyond the capabilities of the FDA-approved tests.

An alternative approach is subtype-specific HRHPV genotyping, in which the presence of 13–18 HRHPV subtypes is detected in a single assay (19, 20). The most common techniques first involve PCR amplification of all HPV DNA in a sample using a common (or “degenerate”) set of L1-specific primers and optimized thermocycling conditions. Following the PCR, the amplified products are then hybridized to a two-dimensional array which is functionalized with viral subtype-specific DNA probes. These Reverse Line Blot (RLB) assays represent the most widely used platform for HRHPV genotyping (21)—however they are costly and not amenable to automation. A number of bead-based HPV genotyping assays have now been developed with the Luminex platform using different combinations of primers, PCR conditions and statistical analyses (14, 22–26). Recently we published a comprehensive study comparing a Luminex assay to HC, RLB, and PAP-based tests showing excellent comparisons (14). However, not every laboratory has access to a Luminex instrument, which currently can only be used to analyze Luminex bead sets combined with a single target fluorophore. Flow cytometry, on the other hand, is a common platform in clinical laboratories and has been adapted to analyzing generic multiplexed bead arrays (MBAs) (6, 27–30). We therefore developed a new bead-based platform (termed “MPS,” named after the key constituent—3′-mercaptopropyl-trimethoxysilane) designed for application with generic flow cytometric instrumentation (27)—for which there exists significantly higher multiplexing potential. While the Luminex 100S uses two encoding channels and can encode ∼100 bead sets (10 bead sets per channel), the BD-LSR2 uses up to 17 channels (31) which could theoretically encode up to 108 distinguishable particle sets if enough spectrally distinct fluorescent dyes are available (32). Incremental advances have already been made in this area, with Luminex's FLEXMAP 3D® instrument using three encoding dyes to produce 500 distinguishable bead sets, however current efforts to further improve multiplexing levels is complicated by interaction effects between encoding dyes or encoding and analyte dyes. On the reporter side, more channels overall could lead to multiple reporter channels providing the ability to multiple targets at once [e.g., “two-color” gene expression (6)].

The MPS bead assay presented in this study is based on thiol-functionalized organosilica particles developed in our laboratory. The beads are formed via a novel condensation-emulsion process (33), which proceeds to form microporous structures with both propyl-thiol and primary silanol groups available for reaction at the solid/liquid interface (34). Fluorescent encoding of these beads was achieved by reacting maleimide-functionalized molecular fluorophores to the thiol groups to form stable thioether bonds which can withstand harsh organic synthesis conditions including DNA phospharamadite or peptide synthesis chemistry (35). This is a significant advantage over polystyrene-based beads in which fluorophores are not covalently attached to the matrix and can be released upon swelling in organic solvents [e.g., those typically used for DNA purification (35) from tissue samples] and some aqueous conditions. The silanol groups are used for probe attachment (oligonucleotide in this case) as a separate functional pathway. While the MPS bead system has been applied to several emerging applications using generic flow equipment (28, 36–38), there has not previously been any direct comparison with gold standard techniques.

The purpose of this study was to perform a multi-platform analysis comparing the MPS beads with Luminex technology, using HRHPV genotyping as a relevant clinical case study. HPV genotyping using a Reverse Line Blot (RLB, Roche Molecular Diagnostics, Pleasanton, CA) assay was considered as the gold standard (21), and we used receiver-operator characteristic (ROC) curve analyses to compare the quantitative Luminex and MPS assays. We further examined platform-specific technical differences which need to taken into account when considering multi-platform assays and cross-platform comparisons.

Materials and Methods

Sample Preparation

Cervical swab samples were a subset chosen from among 739 women enrolled between March 2003 and December 2006 into a study of biomarkers for cervical cancer in HIV infected and uninfected women Senegal, West Africa (39). Written informed consent was obtained according to procedures approved by the Human Subjects Committee of the University of Washington. In all, 200 samples were chosen for the current study, with RLB assays performed at the time of sample collection and MBA assays performed between October and December 2008. In order to increase the efficiency of the current study, samples from women with HPV infection were oversampled so that ∼80% of samples were from women with HPV infection and 20% were from women without HPV infection. Clinical samples, cell lines (ATCC) and HPV plasmid samples were prepared using standard methods as described in previous reports (14, 40).

PCR Protocols

A two-step PCR protocol was employed for maximum sensitivity. In the first step, 3 μL of template was combined with 47 μL PCR buffer [30 μM My9, My11 HPV51 primer each, 3.75 U AmpliTaq Gold DNA polymerase, 25 mM dNTPs each, 5 μL 10× buffer, 6 mM MgCl2, using cycling conditions: 94°C for 9 min, 40 cycles of (95°C for 30 s, 55 for 60 s 74 for 60 s)] and the products purified using a Qiagen PCR Purification Kit (Qiagen). The second step was identical to the first except that a Cy5-labelled My11 primer was the only primer used, the number of cycles was reduced to 20 and no post-PCR purification was required. A PTC-200 Peltier Thermocycler (MJ Research) was used for all PCR reactions, primers were purchased from Integrated DNA Technologies and other PCR reagents were purchased from Applied Biosystems.

Real-time PCR (ABI7900, Applied Biosystems) for the E7 gene of HPV39 was performed using Quantace Sensimix (5 μL 2× PCR buffer, 0.067 μL of 30 μM primer mix, 5′-ACCATGCAGTTAATCACCAACATC-3′, 5′-TTGTGTGACGCTGTGGT TCA-3′, 4.8 μL template). The cycling conditions were: 95°C for 10 min, 40 cycles of 95°C for 10 s, 60°C for 1 min and a final extension step at 72°C for 5 min. Samples were amplified in triplicate and B-actin controls were also run for each sample (primers: 5′- TGGTGATGGAGGAGGTTTAGTAAGT-3′, 5′-AACCAATAAAACCTACTCCTCCCTTAA-3′). Only those samples for which B-actin was positive were considered for HPV analysis.

Bead Synthesis, Encoding, and Functionalization

MPS beads encoded with AT488/AT550 (Attotech, Seigen, Germany) were used for all reactions. Details of particle synthesis, encoding, functionalization and DNA coupling are described elsewhere (34). Luminex beads were purchased (Qiagen) encoded with terminal carboxylic acid groups. Amine-labeled HPV probe sequences [same as those used in RLB assay (41, 42)] were attached to bead sets via EDC (Pierce Biotechnologies) chemistry. Briefly, 106 beads were washed thoroughly in 0.1M MES/0.01% triton (MES free acid, pH ∼4.5, Sigma) and resuspended in a final volume of 50 μL 0.1M MES. Nearly 2 μL of a 100 μM probe solution was added and the suspension vortexed thoroughly before adding 2.5 μL of a 50 mg mL−1 solution of EDC dissolved in water. Beadsuspensions were rotated for 30 min in the dark and then washed in 0.5 mL tween 0.02%, 0.5 mL SDS 0. 1%, and finally in 50 μL TE buffer pH 8.0. Probe sequences for HPV genotyping are the same as those used in a previous study (14).

Hybridization Reactions

For each hybridization reaction, 33 μL of bead mixture (containing ∼1,000 each bead type) in 1.5× TMAC buffer (4.5M TMAC (tetramethyl ammonium chloride), 0.15% sarkosyl, 1M Tris-HCl pH 8.0, 0.5M EDTA pH 8.0) were combined with 17 μL PCR product in 0.2 mL PCR tubes. The samples were incubated for 10 min at 95°C to denature PCR products and then hybridization proceeded at 55°C for 30 min. K562 negative controls were included in triplicate for each PCR protocol per plate. After the reaction the samples were transferred to a Qiagen LiquiChip filter plate (Qiagen) and rinsed three times in 1× TMAC buffer and three times in 6×SSPE buffer (0.9M NaCl, 60 mM sodium phosphate, 6 mM EDTA). Samples were then analyzed directly on a Becton Dickinson LSRII (Becton Dickinson) using a 488-nm laser to excite encoding dyes and a 633-nm laser to excite the Cy5-labeled products. For each 96-well plate, eight controls were analyzed [K562 (x3), SiHa, HeLa, Caski, NTC] plus 80 clinical samples. For Luminex assays, the washing step was replaced by a SAPE incubation reaction for signal detection (according to manufacturer's instructions). These assays were analyzed on the Luminex 100 system. Roche Line Blot (RLB) assays were performed and analyzed as described previously (14).

Data Analysis

Data from MPS assays was analyzed using a simple batch analysis program as previously described by Corrie et al. (27). Briefly this program allows the user to set the gating scheme for the beads (i.e., gates set in scatter and encoding parameters—see Supporting Information Table S2 and Fig. S1 for typical encoding data and gating schemes) for a single file, extract descriptive statistics and then apply that scheme to all files in the user-defined folder (e.g., 96-files for a 96-well plate). Cy5-channel data for each bead set is then extracted and fluorescence intensity values further than three standard deviations from the median are removed (typically ∼1% of data) and the results exported to a spreadsheet. Luminex assay data was exported from the Luminex 100 IS software also into a spreadsheet unless otherwise noted. To determine the presence or absence of each HPV sub-type and to maintain similar analysis techniques for both bead systems, Cy5 or SAPE fluorescence intensity values for K562 DNA were subtracted from each bead set in the samples rather than applying an arbitrary cut-off. ROC curves were constructed for each assay to compare overall sensitivity and specificity and to compare against the RLB assay as the gold standard.

Table 1. MPS bead assay is able to detect plasmids diluted ∼10 copies in initial PCR reaction
SampleFluorescence signal intensities—BD-LSR2  
HPV typeHPV18HPV35HPV45HPV52HPV59Max CHaP value
  • a

    Maximum cross-hybridization signal across remaining 12 probes.

185947.3−115.3−424.751.8−545.0520.80.0074
3547.127234.1−263.0156.7−5.9600.00.0053
45−0.9−19.57690.3126.0241.0454.80.0026
5247.134.5−98.320535.3−248.21004.60.0065
59−0.9−79.069.3551.438328551.40.0021
Table 2. Statistical comparisons between MPS and Luminex assays in comparison to RLB standard
HRHPV subtypePositives (RLB)Negatives (RLB)LuminexMPSDifference
AUCP-valueAUCP-valueP-value
16411571.00<0.00010.98<0.00010.065
18251751.00<0.00010.98<0.00010.084
3191890.98<0.00010.98<0.00010.829
3391890.98<0.00010.89<0.00010.1412
35121861.00<0.00010.93<0.00010.2531
39111870.81<0.00010.740.00670.2049
45141840.99<0.00010.95<0.00010.3011
51131850.95<0.00010.730.00070.0001
52221760.99<0.00010.97<0.00010.2264
56151830.99<0.00010.84<0.00010.0408
58221760.99<0.00010.88<0.00010.0053
5971910.96<0.00010.97<0.00010.8499
6891891.00<0.00010.87<0.00010.1011
Table 3. Comparison of CVs for negative samples: Luminex beads analyzed by Luminex 100 or BD-LSR2
TypeStatL/L/LL/L/FL/BD/FTypeStatL/L/LL/L/FL/BD/F
  1. L = luminex; BD = Becton Dickinson LSR2 cytometer F = flowjo software 1/2/3 = beads/instrument/software.

HPV 16CV43.443.629.8HPV 51CV51.452.229.9
Mean34.532.5969Mean36.834.8976
Min1211292Min1714.2474
Max1091062202Max1601532577
HPV 18CV66.566.835.4HPV 84CV70.471.534.2
Mean49.947.41202Mean39.337.51069
Min2221.9497Min1816.5482
Max2642503474Max2462383291
Table 4. Average S:N as measured by Luminex and BD-LSR2 machines
SubtypeAverage S:Nn positive
MPSLuminex
1653.0520.647
1828.1413.4212
3936.0214.806
4522.9610.397
5124.4610.4912
5632.2717.586

Results and Discussion

The MBA methodology in this study was kept identical to that for the Luminex assay, except for the replacement of a biotin-labeled detection primer with a direct fluorophore-labeled sequence in the case of the MPS assay. As described previously (14) and shown schematically in Figure 1, purified DNA samples were subjected to exponential PCR amplification followed by linear amplification with a single labeled primer. At the end of the reaction, an excess of a labeled strand was present to bind to the probe-functionalized beads without competition from the complementary sequence. The amplified DNA was mixed with a suspension of the 13 individually “bar-coded” bead mixtures representing the 13 most common HRHPV subtypes and after 30 min the reaction was stopped and excess DNA removed. For the Luminex assays, an extra step mixing the suspension with PE-labeled streptavidin (SAPE) was performed to bind the biotinylated PCR products. These suspensions were then analyzed by either the Luminex machine (Luminex beads) or the BD-LSR2 (MPS beads).

Confirmation of Cy5-Labelling Strategy for MPS Beads

Luminex-based assays for HPV genotyping have proven to be highly sensitive, detecting just 1–10 template copies of plasmid per PCR reaction (14). Prior to clinical sample analysis, the two-step PCR protocol was used to amplify positive control DNA sequences extracted from plasmids to validate the Cy5-labelling strategy for the MPS assay. In order to determine the sensitivity of this technique, several relevant type-specific plasmids were diluted down to ∼1–10 copies/PCR reaction in the 1st step. Table 1 shows that all tested plasmids were detected well above K562 background (DNA extracted from K562 cells which contain no HPV sequences) down to ∼10 copies HPV DNA per sample with minimal cross-hybridization to unrelated probes. In terms of assay optimization, we found that a final washing step using a stringent buffer (6XSSPE) significantly reduced signals from the K562 negative control targets thus increasing the signal-to-noise ratio for both bead types. The high sensitivity obtained with the MPS assay suggests that the fluorophore-labeled primers were sufficient for detection purposes without the need for the additional SAPE incubation step.

Clinical Sample Analysis

Two hundred clinical samples were tested in a blinded study to determine the assay performance for each of the 13 HRHPV types for both bead assays. When choosing the samples to be tested, we ensured that ∼80% of the samples (154/200) were positive for HPV; of these 200 samples, 131 (65.5%) of the samples also tested positive for at least one HRHPV subtype, 41 (20.5%) contained HPV16, 25 (12.5%) contained HPV18 and 65 (32.5%) contained HPV16 or HPV18. It is estimated that up to 80% of cervical cancer is caused by HPV16/18 infections (43), it is clearly most important that the assay respond with high sensitivity and specificity for these types. In order to compare the two bead methods a priori, we produced ROC curves for both Luminex and MPS data, for each HPV subtype, standardized in each case to the LB assay as a gold standard. Comparison between the shape of the curves, the area under the curves (AUC) and optimal sensitivity/specificity values provided a basis for technical comparison between the methods. Figure 2 shows the ROC curves and distribution of the HRHPV16/18 data sets based on the optimal limits for each assay. As shown in Table 2, the Area Under Curve (AUC) test statistics suggest that both MBAs perform extremely well (AUC > 0.7 in all cases) with respect to the RLB standard. In comparing the MBA platforms against each other, there was no statistically significant difference in performance for 10/13 bead types tested; in three cases (Types 51, 56, 58) the Luminex AUC values were significantly higher than the corresponding values for the MPS assays.

Figure 1.

Assay methodologies—following genomic DNA purification from cervical samples, exponential PCR targeting the L1 gene was performed to amplify all HRHPV DNA in an unbiased manner. PCR product was purified and amplified in a linear PCR reaction using a labeled primer. Samples for Luminex assays were prepared using biotinylated primers as per the manufacturer's instructions, requiring an extra step to incorporate the SAPE reporter. MPS assays used direct Cy5-labelled primers such that PCR products could be hybridized, washed and analyzed directly without SAPE incubation. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 2.

HPV 16 and 18 ROC curves (A/D) and respective genotyping results from 200 clinical samples stratified by LB. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Instrument and Software Bias

As the Luminex and MPS assays require different analysis instruments (Luminex 100 and BD-LSR2, respectively) and respective software, we investigated the effect of analysis instrument on the results from a model batch of Luminex samples. A microtitre plate containing 96 samples was examined by both instruments within 60 min of assay completion. Data sets were exported to FloJo to remove any source of software bias post-analysis; it is clear that by comparing “L/L/F” against “L/L/L” columns in Table 3 that the file export process did not affect the data, whereas the different cytometer predictably yielded very different raw data. We hypothesized that the assay with lower overall variability would yield lower CVs among the negative samples—analysis of assay CV values suggests that the BD-LSR2 instrument yielded consistently lower CVs for negative samples in comparison to the Luminex instrument. However, the signal-to-noise ratio (S/N—measured as average positive/negative signal for each subtype) was significantly higher for all viral subtypes when analyzed using the Luminex 100 (Table 4). These results suggest that, aside from any assay-related differences observed between the MPS and Luminex platforms, the analytical equipment used for measuring signal intensities had a reproducible impact on the results.

The key technical differences between the assays lay in the analysis instrument used and the detection fluorophore. First, we found that the Luminex 100 instrument consistently showed a higher signal/noise ratio in comparing the same set of samples (stratified by the RLB assay) in comparison the BD-LSR2. This was surprising however it is possible that while the Luminex instrument detection parameters are fixed by the manufacturer for optimal performance, generic cytometers require flexibility for multiple users and hence may not have been optimized for our application. One key difference between the instruments is that for the Luminex 100s machine, a 532-nm laser is used to excite the PE reporter fluorophore, whereas for the LSR2, a 488-nm laser was used, which would provide ∼10% lower excitation energy (estimated using Becton Dickinson's Spectra Viewer application). Second, the use of SAPE in molecular detection assays is reported to increase assay sensitivity 5- to 10-fold in comparison to the corresponding dye conjugate. However, this also requires an extra assay step and the significant cost of SAPE. For the MPS system we instead utilized Cy5-conjugated primers, which reduced the number of steps in the assay apparently without significantly altering sensitivity based on plasmid detection (Table 1).

It is important going forward that a range of compatible bead reagents and analysis equipment exists to satisfy the needs of different user groups—be they research groups or clinical laboratories. This study highlights some important points for future work. First, it should come as little surprise that Luminex particles can be analyzed on flow cytometry equipment not designed specifically for beads, however to our knowledge we have not seen any such results in the literature to date. Second, the reverse is not so simple—Luminex cytometers, understandably, do not easily recognize the encoding properties of non-Luminex particles. Thirdly, the use of multi-parameter, high resolution cytometers should facilitate both (a) significantly higher encoding potential and (b) multiple reporter analysis—over and above that currently provided by the dedicated Luminex instrument. Finally, the differences between instruments appear to account for much of the variation between assays. Comparing the distributions of raw data for the MPS/LSR2 assay versus the Luminex/Luminex assay, we found very little difference in performance based on ROC curves—however we observed significant differences in signal and variability when comparing the same Luminex particles on the Luminex versus the LSR2 platform.

Conclusions

We have completed, to our knowledge, the first multi-platform comparison for a multiplexed, bead-based genotyping assay. Our results suggest that there is little difference between Luminex and MPS bead assays based on the distribution of results from 200 clinical samples detecting 13 HRHPV subtypes. Importantly, a higher S/N was observed for beads analyzed using the Luminex 100S in comparison to the BD-LSR2, likely due to optimal design of the optical detection system in the Luminex 100s.

Ancillary