The characterization and certification of a Legionella DNA quantitative reference material as a primary measurement standard for Legionella qPCR.
The characterization and certification of a Legionella DNA quantitative reference material as a primary measurement standard for Legionella qPCR.
Twelve laboratories participated in a collaborative certification campaign. A candidate reference DNA material was analysed through PCR-based limiting dilution assays (LDAs). The validated data were used to statistically assign both a reference value and an associated uncertainty to the reference material.
This LDA method allowed for the direct quantification of the amount of Legionella DNA per tube in genomic units (GU) and the determination of the associated uncertainties. This method could be used for the certification of all types of microbiological standards for qPCR.
The use of this primary standard will improve the accuracy of Legionella qPCR measurements and the overall consistency of these measurements among different laboratories. The extensive use of this certified reference material (CRM) has been integrated in the French standard NF T90-471 (April 2010) and in the ISO Technical Specification 12 869 (Anon 2012 International Standardisation Organisation) for validating qPCR methods and ensuring the reliability of these methods.
Legionnaire's disease (LD) is a bacterial infection that is characterized by severe pneumonia. Legionella, the causative agent of Legionnaires' disease, is ubiquitous in both natural and man-made aqueous environments. A total of 11 867 cases of this disease were reported by 34 countries in Europe between 2007 and 2008 (Joseph et al. 2010), and 22 418 cases of the disease were reported in the USA between 2000 and 2009 (Hicks et al. 2011). Although at least 50 Legionella species have been described to date, L. pneumophila is responsible for more than 90% of LD cases. LD is acquired by the inhalation of aerosols from Legionella-contaminated environmental sources, such as hot-water systems and cooling towers. Because the transmission of Legionella from one human to another human has never been observed, prevention efforts must concentrate on the elimination of this pathogen from water and aerosol producing systems. Thus, the regular monitoring of potentially contaminated water sources is recommended in most countries in Europe for cooling tower installations and hot-water systems.
In most European countries, national regulations or guidelines include the culture-based detection and enumeration of Legionella in accordance with the international ISO 11731 standard (Anon 1998). Recently, however, efforts to improve the quantification of Legionella in water by developing molecular tools based on quantitative real-time PCR have been very active (Wellinghausen et al. 2001; Joly et al. 2006; Behets et al. 2007; Yaradou et al. 2007; Dusserre et al. 2008; Morio et al. 2008; Lee et al. 2011). Real-time PCR allows for the quantification of DNA in a few hours with high sensitivity and specificity. Although several suppliers have proposed the use of validated and certified Legionella spp. and L. pneumophila qPCR methods for many years, in-house PCR assays continue to be regularly employed. In France, the need for the standardization of the Legionella real-time qPCR methods led to the publication of the XP T 90-471 experimental standard (Anon 2006a) by the Association Française de Normalisation (AFNOR). In addition, external quality assays revealed high between-laboratory variability in the results of Legionella qPCR amplifications. Because this lack of measurement accuracy could primarily originate from the absence of a common quantitative reference standard for qPCR assay calibration, the French Health Ministry tasked the National Reference Centre for Legionella (NRCL) with the mission of developing a primary standard for Legionella DNA to improve the consistency of Legionella qPCR analysis for regulatory applications.
Thus, the objective of this work was to establish a CRM that synchronized both commercially available and in-house standards. The CRM will contribute to inter-laboratory comparisons and improve the reliability of these comparisons. In 2007 and 2008, a preliminary test was conducted to evaluate three candidate Legionella DNA standards and to choose one of these standards by consensus. This study describes the subsequent work that has been performed on the chosen candidate material to enable its characterization and certification according to ISO Guide 35 (Anon 2006b), focusing on the inter-laboratory certification campaign that was organized in 2009. In 2010, AFNOR released a newly revised version of the French standard (NF T90-471) that includes the use of this CRM to help ensure the equivalence of results that are obtained by different qPCR systems (Anon 2010).
A batch of 1500 tubes of the candidate DNA was produced in January 2009. The Legionella DNA was extracted from the L. pneumophila reference strain Philadelphia (WDCM 00107, which is equivalent to ATCC 33152 or CIP 103854) by mechanical cell lysis: 8 ml of culture broth was added to 500 μl Chelex 13% (in ultrapure water) and centrifuged at 5500 g for 5 min, the supernatant was discarded, and 1·5 ml of Tris 10 m mol l−1 buffer pH 7·0 was added. The tube was placed in a sonicator bath for 20 min at 250 watts and finally heated 10 min at 100°C. The purification of extracted DNA was performed by adding RNase 10 mg ml−1 for 30 min at 37°C then using Nucleospin Plant L kit (Macherey-Nagel, Dueren, Germany). The DNA extract is paced on the silica column (Macherey-Nagel) and then bound and washed as recommended in the kit instruction for use. The DNA quantity was estimated by OD measurements (at 260 nm) and PCR-based limiting dilution assays; subsequently, all of the genetic material was aliquoted into 2-ml microtubes (with an estimated quantity of 8·33 × 106 GU per tube), dehydrated under vacuum conditions and stored at −20°C.
The purity of the DNA was studied by the measurement of its absorbance at 260, 230 and 280 nm. For a sample to qualify as pure DNA, the ratio of its absorbance at 260 to its absorbance at 280 nm must be approximately 1·8, and the ratio of its absorbance at 260 to its absorbance at 230 nm must be between 2·0 and 2·2.
The homogeneity of the candidate material has been assessed on samples that have been rehydrated to an estimated concentration of 250 000 GU/PCR (C1). Two sources of data have been used: (i) the measurements that were performed in a single laboratory on 15 tubes and (ii) the measurements that were conducted by the laboratories during the certification campaign. For the former data source, the standard deviation of the 15 values provided an estimate of the homogeneity of the candidate material. For the latter data source, the homogeneity of the candidate material was assessed by examining the inter-laboratory standard deviation, which was determined through the analysis of variance method (anova).
The long-term stability of the candidate material was assessed by testing groups of 3 samples every 3 months for 1 year and thereafter once a year for 3 years. Each of these samples was rehydrated to an estimated concentration of 250 000 GU/PCR; dilutions of these samples were then created and analysed by qPCR to establish a standard curve. The parameters of the standard curve (which has slope and intercept values that were determined by linear regression) were validated, and the quantity of DNA in the nondiluted standard was monitored over time to assess the stability of the samples that were stored at −20°C. In terms of log (GU), the uncertainty of the qPCR method that was used is ±0·25; therefore, we considered a variation greater than ±0·50 to be critical.
Additional stability measurements were performed; in particular, the protocol described above was applied to assess samples of the candidate material that had been stored at different temperatures than the storage temperature. The purpose of this analysis was to anticipate any potential change in the samples that could be caused by stress conditions (which could be encountered during the transport or storage of samples by their users). A total of 9 samples were stored at +5°C and analysed in groups of 3 after 1, 2 and 4 months of storage, and 6 samples were stored at +35°C and analysed in groups of 3 after 1 and 2 months of storage. These two data series were compared with the data that were obtained at month 0 of the experiment.
Twelve French laboratories participated in the June 2009 certification campaign that sought to determine the quantity of DNA that was contained in the candidate reference material. Two samples were shipped to each of the participating laboratories in refrigerated packaging, accompanied by sufficient diluent (10 m mol l−1 Tris, pH 8·3) to perform all of the requested dilutions and PCRs. The study participants were asked to store the samples at −20°C and to analyse these samples according to the protocol that is described below within 2 weeks after the samples were shipped.
All of the laboratories followed the same analytical protocols. In particular, from the samples of the candidate material, the participants in each laboratory created 4 levels of tenfold serial dilutions (C1 = 250 000 GU per well to C5 = 25 GU per well), the last of these serial dilutions was then used to prepare 5 independent levels of limit dilutions. For each of these limit dilution levels, 4 independent dilutions were created. Each dilution step is described in the protocol that is shown in Fig. 1. Six replicates of each dilution (C6 to C10: 5 levels × 4 repetitions) were assessed. In these tests, laboratories utilized their particular PCR systems, which target different genes and/or use different probes and primers but are all designed to specifically amplify L. pneumophila genes (Table 1). All of the PCR amplifications were conducted in accordance with the typical PCR protocols and specific loading schemes of each laboratory. Several quality controls were utilized in these analyses. These quality controls included the controls from the commercial PCR kits (an inhibition control and an external positive control): a minimum of 10 negative controls, which were created from the diluent that was used to rehydrate the DNA, and five positive controls, which consisted of Legionella DNA that was used at 250 GU per well (without any further dilution). The five levels of tenfold dilutions (C1–C5) were also analysed to establish a standard curve for assessing the efficacy of the PCR system.
|1, 6, 9, 11, 12||GeneDisc Cycler, Pall-GeneSystems||L. pneumophila GeneDisc Pack Premium, Pall-GeneSystems|
|2, 7||LightCycler 480, Roche||LightCycler® FastStart DNA Master HybProbe, primers and probes designed by Roche and Sigmaa|
|3||StepOnePlus, Applied Biosystems||iQ-Check L. pneumophila, Bio-Rad|
|4||LightCycler 2 0, Roche||LightCycler® FastStart DNA Master HybProbe, primers and probes designed by Roche and Sigmaa|
|5||Rotor-Gene 3000, QIAGEN||iQ-Check L. pneumophila, Bio-Rad|
|8||Mx300P, Stratagene||Adiacontrol Pneumo Realtime, AES Chemunex|
|10||iCycler-iQ, Bio-Rad||iQ-Check L. pneumophila, Bio-Rad|
Deviations in the results of quantification analyses may be produced by differences in the preparation of each dilution level. Therefore, the metrological properties of the pipettes that were used by the laboratories that participated in this study were analysed according to the NF EN ISO 8655-2 standard (Anon 2002). The uncertainties of these pipettes were determined according to the method prescribed in the GUM (Anon 2008) and were then propagated to each dilution level. The overall uncertainties that were obtained for each laboratory were then compared and could be used to discard quantification results that might suffer from high levels of imprecision.
The results for each tested sample are expressed in terms of the number of positive replicates (out of 6). These results were validated by analyses of the negative controls, which must demonstrate no amplification; the positive controls, which must produce results that are equivalent to the expected value; and the standard curve efficiency, which must be validated in accordance with the AFNOR standard (a calibration function is established from linear regressions that use the results from samples of the C1 to C5 dilutions, and the slope of this resulting line must be between−4·115 and −2·839) (Anon 2010).
Limiting dilution assays are designed to estimate the frequency of positive events in a population. The number of positive/negative results has been totalled over the 4 samples for each dilution level between C6 and C10 (inclusive). This analysis is possible because the samples for different dilution levels are independently created. To account for the probability of false positives and false negatives, the quantity of GU is determined by the minimum chi-squared method, which was initially proposed by C. Taswell (Taswell 1981) and improved further by Rodrigo et al. (1997). In this study, the probability of false positives has been determined by the results of negative controls (samples without Legionella DNA) and is zero. The probability of false negatives was unknown but was assumed to be low due to the high sensitivity of the PCR method. The calculations for the statistical analysis are performed using the Newton method of the Quality program (version 1·1·4, University of Washington, Seattle, WA, USA). The output of this program for each iteration of the analysis is the minimum value of the chi-square, the p-value associated with this chi-squared value (the probability of observing this chi-squared value), a quantity of GUs in the C5 dilution tube (c) and the uncertainty that is associated with this GU estimate.
To estimate the value of the candidate material, c was calculated individually for each laboratory. The validity of the results was controlled by a comparison of the p-values, the minimum chi-squared values and the uncertainties that are associated with the c estimates. The value that was attributed to the candidate material was then estimated by calculating the mean of the c values that were obtained by the various laboratories that participated in this study.
The global uncertainty of c was estimated by examining the dispersion of the c values from these participating laboratories. Because the c value that was attributed to the DNA standard was a mean, its uncertainty was equal to s(c)/sqrt(p), where s(c) is the standard deviation of the c values from the laboratories, p is the number of participating laboratories, and sqrt(p) is the square root of p.
The mean absorbance ratios for the tested samples were 1·82 (260 nm / 280 nm) and 2·04 (260 nm / 230 nm), confirming that these samples contained only pure DNA (free of organics or proteins). These absorbance ratios are consistent with the level of purity that is required for the proposed CRM.
The results of the different measurements are displayed in Table 2. The intra-laboratory standard deviation for the single laboratory that assessed 15 samples is 0·11 cycle threshold (Ct). Among the other laboratories that participated in this study, the observed intra-laboratory standard deviation ranged from 0·03 to 1·98 Ct. This dispersion reflects differences in the repeatability of the measurements and the precision of the intermediate steps for each laboratory. The intra-laboratory standard deviations are also dependent on the PCR-measuring system that was used. In particular, the standard deviations were most homogeneous for PCR system 1; therefore, an initial estimation was obtained that only used the data from laboratories that used system 1. An anova examination of the data from laboratories that used this system produces an inter-laboratory standard deviation of 0·32 Ct and an intra-laboratory standard deviation of 0·23 Ct. The statistical analysis of the other PCR systems produces much greater laboratory effects that cannot be attributed to the nonhomogeneous characteristics of the standard measurements but rather depending on the thermocycler or the operator. In addition, the repeatability and reproducibility of the measurements by the participating laboratories are greater than the dispersion caused by the nonhomogeneous characteristics of the standard measurements.
|Measuring system||Tubes||Average Ct for C1 (250 000 GU)||Intra-laboratory standard deviation|
|Measuring system||Laboratories||Average Ct for C1 (250 000 GU)||Intra-laboratory standard deviation||Standard deviation of the averages|
At 42 months after the production of the samples, the DNA standard demonstrated no significant variations from its initial concentration (Fig. 2a). All of the measurements of the stored samples fall within the limits of ±0·25 of variation in terms of log (GU), and the regression has a slope of 0·0001, confirming the stability of the DNA under the tested storage conditions.
The parameters of the standard curves (slope and intercept) did not show significant variations either.
Neither the samples that were stored for 4 months at +5°C nor samples that were stored for 2 months at +35°C demonstrated significant variations from initial concentrations (Fig. 2b).
All of the laboratories except for one obtained validated results. However, the results of laboratory 8 were discarded immediately because there were no positive results; this occurrence was most likely generated by a dilution error. Therefore, the statistical analysis was conducted on a total of 1320 replicates. Figure 3 provides a graphical representation of the validated results. The effect of pipette uncertainties on the uncertainty of the overall results of the laboratories was relatively low (a maximum of 1% of the concentration). The homogeneity analysis of the validated results indicated that the results from laboratories 2 and 9 were outliers; therefore, the data from these two laboratories were not included in the calculation of the certified value. The replicate C6a from laboratory 5 was also found to be aberrant and was excluded before the results for each dilution level were consolidated. Thus, the c value was computed from the results of only 9 laboratories.
The certified value was obtained after a statistical analysis of the results, which was performed as described in the ‘Statistical Analysis' portion of the ‘'Materials and methods'' section. The estimated UG value and the associated uncertainty obtained by 9 laboratories are graphically represented in Fig. 4.
The certified value corresponding to the mean of 9 estimated values was 10·6 × 106 ± 1·7 × 106 GU. The associated uncertainty U(c) for this value is the expanded uncertainty, which was calculated as 2*u(c); in this expression, the standard uncertainty u(c) is one-third of the standard deviation of the estimated values from the 9 laboratories.
Although many reference materials are certified in terms of CFU or number of bacteria, few bacterial DNA CRMs exist for molecular-based assays (Philipp et al. 2007, Wolfgang et al. 2007), and these CRMs are largely used for presence/absence assays (IRMM 448 Campylobacter jejuni genomic DNA, IRMM 449 Escherichia coli O157 genomic DNA, IRMM 447 Listeria monocytogenes genomic DNA, IRMM 331 intact genomic DNA of Bacillus licheniformis). However, the need for quantitative nucleic acid CRMs for the standardization of qPCR methods is steadily growing in both bacteriology and virology. The only examples of quantitative CRMs are human DNA from NIST (Kline et al. 2009) and certain viral CRMs, such as HBV DNA from NIST or HIV-1 RNA from the WHO (Heermann et al. 1999; Holmes et al. 2001).
We decided to use the limiting dilution assay (LDA) methodology (qualitative PCR), which is used for viral DNA certifications (Heermann et al. 1999; Holmes et al. 2001), to characterize a quantitative Legionella DNA standard. The main advantage of LDA is its independence from pre-existing standards that are used to calibrate qPCR techniques. LDA approaches are simple to perform, and the statistical analysis of LDA results is straightforward; moreover, this methodology is applicable to any micro-organism and can be used with any PCR detection system, as long as it is validated.
Other techniques that could have been used to certify the DNA standard include OD measurements (Kline et al. 2009) and qPCR (Heermann et al. 1999; Holmes et al. 2001). However, UV spectrometry instruments are not very accurate and require a high DNA concentration. The disadvantage of qPCR approaches is that they require a calibration curve; therefore, quantification results from this method are dependent on the standard DNA that is used, generating additional variability. Recently, another methodology (Holden et al. 2007) has been developed to certify quantitative DNA standards by measuring the mass of phosphorus that is present in DNA samples (Yang et al. 2004). However, this approach requires knowledge of the DNA sequence and a significant sample size of highly purified DNA.
The certification campaign included 12 laboratories, each of which used its own PCR system to analyse one sample of the candidate DNA standard. This method ensured that the results represent the wide diversity of analytical procedures (both commercial and in-house) that can be utilized. There was good agreement between the results that were obtained by all of the participating laboratories; in terms of log (GU), the difference between the highest and lowest values of the final estimation was 0·30, with a standard deviation of 0·09. Quantitative assays for the HIV-1 RNA standard (Holmes et al. 2001) produced similar results, with differences (in terms of log (GU)) that varied between 0·27 and 0·40 depending on the samples that were tested.
This assay provided an estimate of 1·06 × 107 GU for each sample of the candidate DNA standard, with a range from 8·9 × 106 to 1·23 × 107 GU. In comparison, from hybridization approaches, samples of the HBV DNA standard (Heermann et al. 1999) were estimated to contain 2·7 × 109 GU, with a range of 2·1 × 109 to 3·4 × 109 GU; from LDA assays, an estimate of 1·04 × 109 genomes per ml and a range of 6·0 × 108 to 1·5 × 109 genomes per ml were obtained. Thus, the results of our assay are consistent with acceptable expectations for DNA CRM with respect to associated uncertainty.
The results from certain laboratories were excluded prior to the statistical analysis; however, these exclusions do not appear to reflect PCR system deficiencies, given that other laboratories that utilized the same systems obtained validated results. In total, 1079 results were entered into the statistical program to estimate the number of GU in the candidate DNA standard. This high number of results contributes to reducing the uncertainty of the estimations. The other components of the uncertainty are pipetting errors and problems with the homogeneity of the DNA samples. Pipetting errors were minimized during this study through the use of a plethora of precautions, such as the gravimetrically measured preparation of the dilutions and the analysis of the pipette calibration certificates to verify compliance with ISO standards (Anon 2002). The probabilities of false-negative or false-positive results were controlled by including numerous positive and negative samples for each PCR. The possibility of false-negative results for low concentration levels near the detection limit of the PCR systems is a more probable source of uncertainty; therefore, we designed the limit dilutions to be representative of a number of points that were near the estimated extinction point. This design allows for the analysis of the results for each laboratory to obtain a better estimate of the extinction point. The uncertainty should incorporate all of the factors that could cause imprecision, such as the homogeneity and stability of the DNA samples, the preparation of the dilutions, the statistical method that is used and laboratory effects. With respect to laboratory effects, the overall values of c are more scattered than the uncertainty that is associated with the values that are generated by any single laboratory. We assumed that all sources of uncertainty are included in this scattering of the values of c that were obtained.
The final result that we generated is the successful tracing of the CRM to a certified GU quantity rather than to a particular mass of DNA. Because the objective of developing the CRM is to provide a common measurement standard for qPCR (which generates results that are expressed in GU per litre) that any laboratory can use, it is most convenient to express the certified value in GU because this format obviates the need for calculating the GU from a particular value for the mass of the DNA in a sample.
The primary standard for Legionella detection and quantification by qPCR that is described in this study has been available since 2009 and can be used to validate new analytic methods and to assign values to in-house quality control materials. NF T90 471 (April 2010) and TS 12869 (Anon 2012) require the use of this Legionella DNA standard for creating a working calibration solution and specify a connecting protocol that ensures the correct calibration of PCR systems. This standard contributes to the reduction in within- and among-laboratory variability and bias during the quantification of Legionella DNA.
To reduce even more the variability of the quantification method described in NF T90 471, the DNA extraction step should also be controlled with a reference material, because high variability between laboratories can also be explained by differences in the extraction efficiency. There is no such reference material, but the method we described can be used to certify one in the future.
We would like to thank all of the laboratories that participated in the certification campaign and the experts who provided assistance with the preliminary study of this topic and contributed to the choice of the candidate material. This work received financial support from the Ministère chargé de la Santé.