Comparison of Bacterial Culture and Real-Time PCR for the Detection of Salmonella in Grow–Finish Pigs in Western Canada Using a Bayesian Approach
Wendy Wilkins, DVM. Department of Large Animal Clinical Sciences, Western College of Veterinary Medicine, University of Saskatchewan, 52 Campus Drive, Saskatoon, Saskatchewan S7N 5B4, Canada.
The study objective was to evaluate the accuracy of a real-time polymerase chain reaction (RT-PCR) and a culture protocol used to detect Salmonella in the faeces of grow–finish pigs using a Bayesian approach. The RT-PCR was invA-gene-based assay, while the culture protocol included pre-enrichment in buffered peptone water, selective enrichment in tetrathionate and Rappaport-Vassiliadis broths, and isolation on semi-solid (modified semi-solid RV) or solid (XLT4, Rambach) agar plates. Bayesian analysis was performed using a two-test, two-population model with dependence between culture and RT-PCR and compared to a second model with conditional independence between these two tests. Two hundred and ninety three individual faecal and 294 pooled pen samples from grow–finish pig collected from 10 farms were tested and results were divided into two groups according to herd size (five herds <250 sows, five herds with >400 sows). In the dependence model, RT-PCR sensitivity (Se) and specificity (Sp) were estimated to be 90% (95% probability interval 74, 97) and 99% (98, 99), respectively. Culture Se was 92% (75, 99), while culture Sp was considered 100% as all culture-positive samples were confirmed by serotyping. In the conditional independence model, RT-PCR Se and Sp, and culture Se, were 96% (93, 98), 99% (98, 100) and 97% (94, 100), respectively. The dependence model resulted in posterior estimates of Se that were lower and with broader probability intervals than the independence model, indicating that when RT-PCR and culture are evaluated relative to each other, the correlation between these tests is an important source of bias and should be adjusted for during analysis. The RT-PCR evaluated in this study performed almost comparably to culture; given the cost savings associated with using this test and more timely results, the RT-PCR may be a useful alternative to culture for screening large numbers of samples, particularly when Salmonella prevalence is low.
- • Using a Bayesian model specifying dependence between tests, RT-PCR sensitivity (Se) and specificity (Sp) were estimated to be 90% (95% probability interval 74, 97) and 99% (98, 99), respectively, while culture Se was 92% (75, 99).
- • The correlation between RT-PCR and bacterial culture was found to bias estimates of test accuracy; researchers should adjust for this correlation during analysis.
- • The RT-PCR evaluated in this study performed comparably to culture and may be a useful alternative to culture for screening large numbers of samples, particularly when Salmonella prevalence is low.
Traditionally, the evaluations of the sensitivity (Se) and specificity (Sp) of diagnostic tests are done by comparison to a gold standard, a test (or tests) which accurately determines the true disease state of an animal (Greiner and Gardner, 2000). Test Se and Sp can be directly estimated from observed results when a true gold standard is used. As is often the case, though, evaluation of tests for detecting Salmonella infection in pigs is complicated by the lack of a gold standard and test Se and Sp are usually assessed relative to other imperfect tests, resulting in relative Se and Sp estimates.
Salmonella infections are prevalent in pigs, with most infections being sub-clinical (Schwartz, 1999). Acutely infected pigs may shed Salmonella; infection may be cleared, or pigs may progress to a chronic carrier state with intermittent shedding of Salmonella in the faeces (Wood and Rose, 1992; Gray et al., 1995) and contribute to the persistence of Salmonella within the herd. In these latter situations and when no gold standard is available, the Bayesian approach is particularly useful (Branscum et al., 2005) as none of the competing tests are treated as the gold standard and the diagnostic error rates are estimated for all studied tests (Hui and Zhou, 1998). Prior information (such as that generated by previous research or expert opinion) is combined with the observed data to obtain posterior distributions of test parameters (Branscum et al., 2005). Knowledge of the true disease or infection status of the animal is therefore not necessary, and instead this unknown information is incorporated into the model as a latent variable (Enoe et al., 2000).
The study objective was to use Bayesian methods to evaluate the accuracy of a bacterial culture protocol and a real-time polymerase-chain reaction assay (RT-PCR), previously used to evaluate Salmonella status in a range of sample matrices (Bohaychuk et al., 2007), for assessing Salmonella in western Canadian pigs under field conditions.
Materials and Methods
Farms were purposely selected based on presumed Salmonella status to ensure a sufficient number of Salmonella-positive samples for statistical analysis. The number of herds and samples was a function of logistical and financial constraints. Ten farrow-to-finish swine herds (herd size n > 100 sows) from Alberta (seven farms) and Saskatchewan (three farms), Western Canada, were selected by swine veterinarians, based on presumed Salmonella positive (n = 7) or Salmonella negative status (n = 3) and the producer’s willingness to participate in the study. Herds were presumed positive at the time of herd selection if either the herd veterinarian or producer observed clinical salmonellosis within the previous 12 months, if Salmonella species were identified during routine testing, or if replacement breeding stock were purchased from known Salmonella-positive farms. Herds were presumed negative if none of these criteria were met.
Samples were delivered to the laboratory either within 2 h of leaving the farm, or held on ice overnight and delivered the following day. On each farm, 30 grow–finish pens were selected according to a computer-generated random number list. One individual rectal faecal sample (minimum 10 g) and one pooled pen floor faecal sample was collected from each grow–finish pen. Pooled samples consisted of a minimum of 5 g of faecal material taken from each of five different floor locations within each pen (minimum 25 g total).
Bacteriologic culture for Salmonella was performed by the Agri-Food Laboratories Branch (AFLB), Food Safety and Animal Health Division of Alberta Agriculture and Rural Development. All samples were refrigerated and cultured within 24–48 h of receiving samples at the laboratory. Faecal samples were thoroughly mixed prior to culture. The culture protocol has been previously described in detail (Wilkins et al., 2010). Briefly, faeces were pre-enriched in buffered peptone water then selectively enriched using Rappaport Vassiliadis (RV) and tetrathionate (TT) broths in parallel. The RV and TT were streaked onto XLT4, Rambach (RAM) and modified semi-solid RV (MSRV) agar plates; halos of growth occurring on MSRV plates were streaked to XLT4 and RAM. Suspect colonies were screened using triple sugar iron, urea and lysine iron agar slants and sero-agglutination (Poly O and Poly O1 antisera; Denka Seiken Co. Ltd, Tokyo, Japan). One isolate from each Salmonella positive sample was frozen at −70°C prior to being sent for serotyping at the Public Health Agency of Canada, Laboratory for Foodborne Zoonoses, Guelph, ON, as previously described (Rajić et al., 2005).
RV and TT enrichment broths (150 μl each) from each sample cultured as described above were mixed together then analyzed using a previously published RT-PCR assay with primers and hybridization probes to the Salmonella invA gene (Bohaychuk et al., 2007). DNA extraction was done using a semiautomated magnetic particle processor (KingFisher mL; Thermo Electron Corporation, Vantaa, Finland) and DNA extraction kit (Magnesil KF genomic system kit; Promega, Madison, WI, USA) according to the manufacturers’ instructions. RT-PCR was performed using the LightCycler RT-PCR machine (Roche, Germany). Internal control and target DNA both were amplified by the same set of primers to monitor each RT-PCR reaction for inhibition and accuracy of reagent preparation. Results were interpreted as positive or negative depending on the shape of the curve and the crossing point provided by the LightCycler analysis software.
The overall agreement between tests (kappa statistic) was first calculated using Stata/SE v9.2 (StataCorp LP, College Station, TX, USA). A Bayesian approach was then used to estimate the Se and Sp of bacterial culture (Sec, Spc) and RT-PCR (Sep, Spp). When estimates of test accuracy are based on a model which assumes independence between test outcomes (conditional on the disease status of the animal), the resulting estimates of Se and Sp could be misleading if the test outcomes for a given animal are correlated (Branscum et al., 2005). Since both culture and the RT-PCR were conducted using the same pre-enriched sample, we assumed that the results of these two tests would be correlated. Therefore, a two-test, two-population model with dependence between culture and RT-PCR was specified using the WinBUGS software (http://www.mrc-bsu.cam.ac.uk/bugs/). Both individual and pooled faecal samples were included in the analysis. Because there are eight parameters to be estimated (Se and Sp for both tests, two population prevalences and two covariances) and only six degrees of freedom, the inclusion of informative prior information for some parameters was necessary (Georgiadis et al., 2003; Branscum et al., 2005). Therefore, prior information for Spc and Salmonella prevalence in each population were specified for all models.
Spc was considered to be 100% as all isolates were confirmed as Salmonella spp. through serotyping; however, technical errors may occasionally occur and therefore the beta distribution for Sp was modelled as a = 999 and b = 1 (i.e. 1 error in 1000).
The dataset was divided into two populations based on herd size, with the assumption supported in previous primary research that Salmonella prevalence would differ in these populations (Baggesen et al., 1996; Farzan et al., 2006; Mejia et al., 2006). Population 1 (Pop1) consisted of five herds with <250 breeding sows per herd (mean = 195, range 130–240); Population 2 (Pop2) of five herds with >400 breeding sows per herd (mean = 866, range 426–2070). Our initial bacteriological results also indicated that the expected prevalence in Pop1 was substantially lower than in Pop2. Diffuse prior distributions for the prevalence in Pop1 (PrA) and Pop2 (PrB) were based on reported estimates from two different studies conducted in Canada, one reported a prevalence of 14% (Rajić et al., 2005) and the other reported a prevalence of 35% (Anonymous, 2006). The resulting beta parameters for Model A were α = 3.4 and β = 15.2 for Pop1, and α = 7.3 and β = 12.7 for Pop2.
To check for potential influence of prior test information, Model B was specified with prior information for Sec ≥ 40% with a mode of 80%, as described by Mainar et al. (2008), with corresponding beta parameters of α = 5.3 and β = 2.4. In the conditional dependence model, prior information for test 2 (in this case the RT-PCR) is a function of other model parameters and cannot be specified a priori. A two-test, two-population model with conditional independence between tests was also specified using the same prior information as Model A to evaluate possible difference in posterior estimates when correlation between tests was not accounted for. To check the assumption of constant test accuracy across populations separate Bayesian analyses of the two populations were run using a two-test, one population model as described by Branscum et al. (2005).
Inferences were based on 100 000 iterations after discarding an initial burn-in of 5000 iterations. Convergence was assessed by running multiple chains from dispersed starting values (Gelman and Rubin, 1992), checking the standard errors, and visually checking of the kernel density and trace plots for each parameter. Analyses were also run separately for each population to verify the assumption of constant Se and Sp across populations (Georgiadis et al., 2003).
More positive samples (38%) and a broader range of serovars (12 typeable serovars) were detected from 294 pooled pen samples than 293 individual faecal samples (25% positive, 8 typeable serovars); however, overall serovar distribution between Pop1 and Pop2 was similar with S. derby and S. typhimurium var. Copenhagen as the most prevalent serovars (Wilkins et al., 2010). The bacteriological prevalence was 21% (60/287) in Pop1 and 42% (125/300) in Pop2, thus satisfying the model requirements for two populations with different prevalences. The overall agreement between the two tests measured as kappa was 0.96 (95% CI 0.95, 0.98).
The cross-classified test results are shown in Table 1. The RT-PCR failed to detect 3/60 culture-positive samples in Pop1 and 4/125 culture-positive samples in Pop2. Six additional positive samples were identified by RT-PCR in Pop2, while no additional positives were identified in Pop1. The results of the individual population analyses showed that the Se and Sp of each test agreed within 1–3%, with considerable overlap in probability intervals (PI), satisfying the assumption of constant test accuracy across populations.
Table 1. Two-way classification of test results for culture and RT-PCR for the detection of Salmonella in 587 faecal samples from grow–finish pigs from 10 farms in Alberta and Saskatchewan, 2004
The mean posterior estimates resulting from each model are shown in Table 2. The correlation among test outcomes for infected animals was 0.49 (95% PI 0.002, 0.88) and among test outcomes for non-infected animals was 0.21 (95% PI 0.004, 0.71). Under the assumption of conditional dependence (Model A), posterior means and respective 95% PI were: Sec 92% (95% PI 75, 99), Sep 90% (95% PI 74, 97), Spp 99% (95% PI 98, 99), PrA 22% (95% PI 17, 28) and PrB 45% (95% PI 38, 55). When tests were assumed to be conditionally independent (Model C), poster estimates for Sec and Sep increased by 5–6%, while 95% PIs narrowed considerably to within ±3% of the mean. Specifying informative prior information for Sec (Model B) decreased posterior estimates for Sec and Sep by 4–5% as compared to Model A. Posterior estimates for Spp, PrA and PrB differed little between the various models.
Table 2. Bayesian posterior estimates of the sensitivity and specificity of culture and RT-PCR used to detect Salmonella in 587 faecal samples from grow–finish pigs from 10 farms in Alberta and Saskatchewan, 2004
|Model Aa||92 (75, 99)||100||90 (74, 97)||99 (98, 99)||22 (17, 28)||45 (38, 55)|
|Model Bb||87 (71, 97)||100||86 (70, 96)||99 (98, 99)||24 (18, 31)||47 (39, 58)|
|Model Cc||97 (94, 100)||100||96 (93, 98)||99 (98, 100)||21 (16, 25)||43 (37, 48)|
The RT-PCR used in this study was previously evaluated in a variety of different sample matrices, and the results were published elsewhere (Bohaychuk et al., 2007). As compared to culture of pig faeces, the estimates of Sep and Spp obtained through traditional methods were 99% and 97%, with confidence intervals within 3–5% of the mean. These results are similar to the posterior estimates (Sep 96%, Spp 99%) resulting from Model C, which assumed conditional independence between tests. However, the conditional dependence models (Models A and B) yielded posterior estimates for Sep which were 6–10% lower and with much wider 95% PIs. This discrepancy can be attributed to the correlation between test outcomes for infected animals. Although the posterior estimates of Sep under the conditional dependence model were lower than the Sep reported by Bohaychuck et al. (2007) it is important to note that in our analysis the posterior estimates of Sec were also similarly decreased. The kappa statistic was therefore very similar in their study and our current analysis (κ = 0.96 and 0.95, respectively).
From a biologic perspective, two tests that detect similar responses are likely to be somewhat dependent (Branscum et al., 2005). Although it can be argued that the biological basis of culture (which detects viable Salmonella) is different from PCR (which detects Salmonella DNA), both tests were conducted on the same selectively enriched samples. Therefore, it is not surprising to find evidence of dependency between these two tests. In contrast to the current study, Mainar-Jaime et al. (2008) found no significant correlation between tests when they used Bayesian methods to evaluate culture and PCR for Salmonella in pig caecal content, even though both tests were done using the same pre-enriched samples. A possible explanation may be that potential correlation due to similar biological basis was masked by the relatively poor performance of culture in that study, which only detected 42% (28/67) of the PCR-positive samples.
In general, the performance of PCR (as compared to culture) to detect Salmonella in the faeces of healthy pigs has been infrequently reported in the existing literature. Performing PCR assays directly on faeces is impractical, due to low numbers of bacteria present in subclinical or transient infections and the likely presence of inhibitors coextracted with DNA from samples (Sibley et al., 2003). The available information indicates that PCR of pre-enriched samples tends to be quite sensitive though specificity may be more variable (Sibley et al., 2003; Bohaychuk et al. 2007; Mainar-Jaime et al., 2008). The need for selective enrichment of samples limits the usefulness of PCR as a ‘rapid’ assay for detecting Salmonella in faeces, though results may still be available 24–48 h sooner than when only traditional culture methods are utilized. Other studies have evaluated PCR for detecting Salmonella in pork and pork carcasses (Croci et al., 2004; Patel and Bhagwat, 2008; Lofstrum et al., 2009) with results often available the same day as the absence of high numbers of competing bacteria and inhibitors precludes a need for selective enrichment steps. A number of major Danish slaughterhouses have taken advantage of the rapid PCR, using it to screen meat and carcass swabs in order to shorten the cold storage time and facilitate the export of fresh pork (Lofstrum et al., 2009).
The real benefit of using RT-PCR to screen pig faeces for Salmonella may be cost savings. Bohaychuk et al. (2007) estimated the cost (including labour and materials) of the RT-PCR to be 62% that of conventional culture. Regardless of differences in test accuracy between the models assuming dependence or independence between tests, the accuracy of the RT-PCR evaluated in this study was comparable to that of culture. Identification of presumptive colonies of Salmonella is the most time, labour and cost intensive part of Salmonella culture. Therefore, the RT-PCR may be especially useful for screening large numbers of samples in situations where Salmonella prevalence is expected to be low. Examples of areas for potential use as a cost-effective screening tool are repeated testing to confirm Salmonella-free status of farms or live animal exports, or to distinguish between high- and low-prevalence herds. This test could also be used to screen carcasses for Salmonella contamination at slaughter, although further study is needed to determine if pre-enrichment steps could be minimized in this situation to provide results more rapidly.
The Bayesian approach used in this study found evidence of a correlation between test outcomes in infected animals which impacted the posterior estimates of test sensitivity and in particular their associated 95% PIs, suggesting that the correlation between these tests is an important source of bias and should be adjusted for during analysis. Failure to account for this correlation may result in underestimation of actual Salmonella prevalence when these tests are applied in practice. The Bayesian conditional dependence model incorporates the uncertainty regarding the accuracy of culture and adjusts for this correlation, resulting in more realistic posterior estimates of Se and Sp for both culture and RT-PCR. The RT-PCR evaluated in this study performed comparably to culture; given the cost savings associated with using this test and more timely results, the RT-PCR may be a useful alternative for screening large numbers of samples, complemented by bacterial culture of RT-PCR-positive samples when isolates are required for serotyping or other analysis.
The authors wish to acknowledge R. King, P. Lu, W. Lazaroff and D. Patterson from Molecular Biology, Food Safety and Animal Health Division, Alberta Agriculture and Rural Development, Edmonton, Alberta, and the Veterinary Microbiology technicians, as well as L. Cole, E. Wilkie and K. Mistry at the Laboratory for Foodborne Zoonoses, Guelph, ON. Thanks also to the veterinarians and pork producers who participated in this project. This project was funded by Agriculture and Rural Development, the Alberta Livestock Industry Development Fund and Saskatchewan Pork.