Diagnostic sensitivity and specificity of different methods used by two laboratories for the detection of Phytophthora ramorum on multiple natural hosts

Authors

  • A. M. Vettraino,

    1. University of Tuscia, S. Camillo de Lellis snc, 01100 Viterbo, Italy
    Search for more papers by this author
  • S. Sukno,

    1. Department of Plant Pathology and Microbiology, Texas A&M University, 2132 College Station, TX 77845, USA
    2. Centro Hispano-Luso de Investigaciones Agrarias (CIALE), Departamento de Microbiología y Genética, Universidad de Salamanca, Calle Del Duero 12, 37185 Villamayor, Salamanca, Spain
    Search for more papers by this author
  • A. Vannini,

    1. University of Tuscia, S. Camillo de Lellis snc, 01100 Viterbo, Italy
    Search for more papers by this author
  • M. Garbelotto

    Corresponding author
    1. Department of Environmental Science, Policy and Management, University of California, Berkeley, 137 Mulford Hall, Berkeley, CA 94720-3114, USA
      *E-mail: matteo@nature.berkeley.edu
    Search for more papers by this author

*E-mail: matteo@nature.berkeley.edu

Abstract

Five detection methods were comparatively tested on putative Phytophthora ramorum field samples from 41 wild plant species. The tested methods included two culture-based assays, a DAS-ELISA-based polyclonal assay, a nested PCR-based assay, and a TaqMan real-time PCR assay. Diagnostic values including sensitivity, specificity, positive predictive value and negative predictive value were calculated for each method. The effects of host species, seasonality and host location were analysed and compared between two laboratories. Significant effects of season, host species and laboratory were detected. It is concluded that a combination of either culturing and molecular diagnosis or of two molecular assays is the most promising approach to diagnose this pathogen. Based on the results of this and other studies, diagnosis should occur as much as possible during wet and warm periods favourable to the pathogen, and proficiency tests should be performed to compare results obtained with molecular approaches in different laboratories. Furthermore, length of time lapsed between sample collection and processing strongly affected the diagnostic sensitivity of culture-based methods, and therefore needs to be taken into account when comparing results from different laboratories.

Introduction

Phytophthora ramorum is the causal agent of a devastating disease of Californian oak species (Quercus agrifolia and Q. kelloggii) and tanoak (Lithocarpus densiflorus). The disease is commonly known as sudden oak death (SOD) and causes high tree mortality in California and Southern Oregon, on the west coast of the USA (Rizzo et al., 2002). This pathogen is also frequently detected in nurseries both in the USA and Europe, where it is reported to cause a variety of symptoms, including leaf spots, branch and twig dieback, and stem necrosis, on a wide range of hosts (Davidson et al., 2003, Brasier et al., 2004). Since its first detection in Europe (Werres & Marwitz, 1997; Werres et al., 2001) on Rhododendron and Viburnum, and in the USA on Q. agrifolia, Q. kelloggii and L. densiflorus (Rizzo et al., 2002), the number of new hosts and reports of P. ramorum has increased dramatically in both continents, including a few reports of its spread in European wild areas (Brasier et al., 2004; De Gruyter & Steeghs, 2006). Quarantine regulations have been issued in Europe and the USA to control movement of the pathogen (Anonymous, 2000, 2002); however, the effectiveness of these regulations depends on an exhaustive knowledge of host range and accurate and sensitive detection methods. Several studies have been carried out and others are still ongoing in order to assess the potential host range of P. ramorum (http://rapra.csl.gov.uk). Even excluding hosts exclusively limited to the ornamental trade, the list of natural hosts already includes more than 72 species (Anonymous, 2008) and is far from being complete because of the high number of new hosts that are continually reported. Identification of P. ramorum based upon symptom recognition may not be accurate since the symptoms this pathogen causes are highly variable and overlap with similar symptoms caused by other pathogens and physiological conditions (Davidson et al., 2003). A number of diagnostic methods are routinely used to detect P. ramorum from diseased tissues, including classical biological detection, polyclonal antibody assays based on serological techniques, and conventional and real-time PCR techniques (MacDonald et al., 1990; Bilodeau et al., 2003, 2007; Martin et al., 2004; Hughes et al., 2005, 2006a,b; Hayden et al., 2006; Schena et al., 2006; Tooley et al., 2006; Kox et al., 2007; Tomlinson et al., 2007). None of the methods are optimal, but all incur errors of incorrect identification of the pathogen where it is not present (false positive) or failed identification of the pathogen where it is present (false negative). In an attempt to minimize false negative and false positive diagnoses, the US Animal and Plant Health Inspection Service (USDA-APHIS) and the European Plant Protection Organization (EPPO) have issued diagnostic protocols for P. ramorum that use a serological pre-screening followed by an analysis based on a biological and/or molecular method (Anonymous, 2004, 2006).

A comparison of accuracy among detection methods can be made by evaluating their diagnostic values, namely sensitivity, specificity, predictive positive value and predictive negative value (Alberg et al., 2004). Currently, comparative information among methods regarding diagnostic values of assays is available for only a few ornamental host plants, including Rhododendron, Viburnum, Camellia and Pieris (Hughes et al., 2006a,b; Lane et al., 2006; Kox et al., 2007), whilst such information is basically missing for natural hosts in forest settings. Furthermore, a thorough analysis of the comparative efficacy of different detection methods performed by different laboratories on a range of hosts collected in different periods from different areas is still lacking. The aim of the present study was to compare diagnostic values of culture-based, serological and molecular P. ramorum detection methods on plant material with symptoms sampled from a range of plant species at two different times in multiple field sites, and processed in two different laboratories in the USA.

Materials and methods

A woodland survey was conducted in 2005 in Marin, Monterey and St Cruz counties in California. A total of seven sites (Big Sur, Angel Island, China Camp, Bolinas Ridge, Samuel P. Taylor, St Cruz-Harpin and St Cruz Bean Creek) were investigated for the presence of P. ramorum in the late spring (June) and in early autumn (September). At least 30 samples showing symptoms characteristic of P. ramorum infection were collected along four 100-m transects at each site. Symptoms included foliar necrosis, twig dieback and seeping trunk lesions. In June, all sampled plants were identified, tagged and mapped with a GPS unit (Trimble Corp.) in order to be re-sampled in September. Five leaves with symptoms were arbitrarily collected per tree from hosts showing symptoms, while 3 cm2 of necrotic tissue were collected from L. densiflorus trunk lesions. A total of 428 plants with symptoms were sampled each in the spring and autumn surveys, including leaf samples from 233 Umbellularia californica trees, 175 leaf samples from 39 other host species, and trunk tissue samples from 20 L. densiflora trunk lesions.

Samples were taken to the Forest Pathology and Mycology Laboratory at the University of California, Berkeley (UCB). Each of the five samples from each host were split in half using a sterilized scalpel so that each half contained equal amounts of material with symptoms. The 10 segments were then mixed together and randomly divided into two equal sets. One set was shipped to the laboratory of Plant Pathology and Microbiology at the University of Texas (TAMU), whilst the other was immediately processed at UCB. The former samples were shipped to TAMU using a 3-day service in an insulated container with ice-packs, stored at 4°C immediately upon arrival and processed within 24 h of receipt. As a consequence, samples were processed at TAMU approximately 4 days later than at UCB. Within each laboratory, the sample fragments were randomly assigned to individual diagnostic methods, including two direct plating assays, one immunological assay and two PCR assays. PCR assays were performed on the same extract, allowing the use of the fifth sample as a spare.

Diagnostic methods

Five distinct methods were employed for the diagnosis of P. ramorum, including direct plating on two semi-selective media, one immunological and two PCR-based molecular methods. Infected tissue was randomly assigned to each method.

Culturing

Leaf and bark tissues (approximately 0·1 cm2) were transferred to either V8-PARP (vegetable juice 50 mL; CaCO 3·5 g; agar 15 g L−1) or PARP (17 g corn meal agar L−1), amended with 5 mg pimaricin, 250 mg ampicillin, 10 mg rifampicin and 50 mg pentachloronitrobenzene L−1 (Anonymous, 2004). Samples were incubated in the dark at room temperature (20–22°C) for 10 days. Agar plates were checked for growth of P. ramorum after 7–10 days of incubation and P. ramorum was identified by microscopic examination (Werres et al., 2001).

DAS-ELISA

Serological assays were carried out using the Pathoscreen kit for Phytophthora (Agdia). This kit is based on DAS-ELISA technology and on polyclonal antibodies developed for the Phytophthora genus. Samples were processed according to the manufacturer’s protocol and as follows. Each sample was placed in a 2-mL microcentrifuge tube and a similar volume of sterile 0·5-mm glass beads (BioSpec Products Inc.) was added, along with general extraction buffer (GEB2-Agdia) at a sample-to-buffer ratio of 1:10 (g mL−1). The microfuge tubes were then sealed with a screw-cap lid containing an O-ring, and shaken in a MiniBeadbeater-8 homogenizer (BioSpec Products Inc.) for 3 min on quarter power in order to macerate the sample. Each tissue sample was assayed in duplicate. In each assay, positive controls from the Agdia kit, and negative controls comprised of 100 μL extraction buffer were used.

Molecular analyses

Prior to DNA extraction, all plant material was processed and stored as previously described by Hayden et al. (2004). Briefly, the plant tissue was lyophilized for 48 h, pulverized with glass beads and the DNA extracted using a cetyltrimethylammonium bromide (CTAB) extraction protocol (Hayden et al., 2004). Wood and bark samples were processed using a QIAmp DNA stool mini kit (Qiagen Inc.) according to the manufacturer’s instructions. For each sample, DNA concentration and quality were determined with a NanoDrop ND-1000 spectrophotometer (NanoDrop Technologies) and all samples were stored neat at −20°C prior to testing. All tissue samples were tested in duplicate by performing two DNA extractions.

The nested SYBR-green (SYBR) (Hayden et al., 2004) and the CSL TaqMan real-time assays (Hayden et al., 2006; Hughes et al., 2006 a,b) were performed using the same DNA extract. The nested SYBR-green assay consisted of two rounds of PCR. The first round amplification was a conventional PCR reaction using the primer set Phyto1/Phyto4 and the second round of amplification used a SYBR-Green PCR kit (Applied Biosystems) with the primer set Phyto2/Phyto3. These primers are designed to specifically amplify a fragment of the P. ramorum nuclear ribosomal RNA gene. Reaction parameters and reagents for both the first and second rounds of amplification were as described by Hayden et al. (2004). The CSL TaqMan real-time PCR assay for specific amplification of P. ramorum DNA was performed using primers Pram114Fc/Pram-190R and a TaqMan real-time probe (Pram probe) designed to amplify a fragment of the ITS1 region of the nuclear ribosomal RNA gene. The CSL TaqMan real-time assays were performed in a multiplex fashion by including control primers (COX-F and COX-RW) and probe (cox), designed to detect the plant cythocrome oxidase (COX) gene. Amplification conditions using the TaqMan real-time Universal PCR master mix Kit (Applied Biosystems) were as described by Hughes et al. (2006a & b).

All of the methods, parameters, reagents and protocols used at the UCB and TAMU laboratories were exactly the same except for the thermalcyclers. The first round of the nested SYBR-green assay was performed in an iCycler thermalcycler (Biorad) at UCB and in a TC-412 Flexigene thermalcycler Techne ® Inc. at TAMU. Both the CSL TaqMan real-time assay and the second round of the nested SYBR-green assay were performed in an iCycler IQ thermalcycler at UCB and in an ABI Prism 7700 Sequence Detection System (Applied Biosystems) at TAMU. Each plant tissue sample was assayed in duplicate. If a sample tested negative in either duplicate, then the assay was repeated for both duplicates. For each PCR run, DNA from lettuce leaves was extracted during DNA extraction of field samples and served as negative controls, following the assay design of Hayden et al. (2004). Lettuce is not known as a host of P. ramorum and never yielded positive indication of infection in any of the assays associated with this study. As a positive control, DNA from the reference strain of P. ramorum 418 (ATCC MYA-3676) was used to monitor the performance of the PCR. Determinations of the threshold cycle (Ct) were made using the icycler image analysis program (BioRad) at UCB, and the sequence detection software v.1·3·1 (Applied BioSystems) at TAMU. Samples with a Ct < 36 were scored as positive.

Diagnostic values

Raw results from any diagnostic test can be interpreted as an estimate of the apparent prevalence of a disease, but do not equate to the actual level of infection in the analysed set of samples. Better estimates of the true prevalence of a pathogen can be made by taking into account test sensitivity and specificity (Alberg et al., 2004). These values provide information on the proportion of true positives and true negatives identified by a diagnostic method. Diagnostic sensitivity is the proportion of true positive samples correctly identified by the test, while diagnostic specificity is the proportion of true negative samples correctly identified by the test (Kox et al., 2007). Estimation of the above diagnostic values thus requires the use of an appropriate reference or ‘gold standard’ to determine which samples can be considered true positives. A gold standard test is the diagnostic method or combination of methods that unambiguously determine whether a pathogen is present in any given sample (Kox et al., 2007; Gharabaghi et al., 2008). According to the diagnostic protocol of the EPPO (Anonymous, 2006) and USDA-APHIS recommendations (Anonymous, 2004), positive culture results combined with positive CSL TaqMan PCR results may be considered to represent the range of samples actually infected by P. ramorum. The use of positive results from both assays combined as ‘gold standard’ combines the high specificity of culturing (absence of false positives) with the high sensitivity of the TaqMan assay (reduced number of false negatives). A combined culturing–molecular gold standard is further necessary because in the case of co-infection of P. ramorum with other Phytophthora spp., culturing can fail to detect P. ramorum-positive samples (Kox et al., 2007).

The accuracy of any diagnostic method can be determined by calculating its positive predictive value (PPV) and its negative predictive value (NPV): PPV indicates the probability that a positive test result correctly identifies the presence of infection, while NPV indicates the probability that a negative test result correctly identifies the absence of infection. Predictive values are functions of prevalence (disease incidence based on the gold standard) and of the sensitivity and specificity of each test. The effect of prevalence on predictive values was calculated according to Bayes’s theorem (Alberg et al., 2004), using the following equations: PPV = diagnostic sensitivity × prevalence/[(diagnostic sensitivity × prevalence)] + [(1 − diagnostic specificity) × (1 − prevalence)] and NPV = diagnostic specificity × (1 − prevalence)/[(1 − diagnostic sensitivity) × prevalence] + [diagnostic specificity × (1 − prevalence)]. Differences in results between tests were assessed using Fisher’s exact test (Colin et al., 2001; Frenk et al., 2006; Smedley et al., 2007). The confidence interval of the estimates was computed using the ‘exact method’ developed by Clopper and Pearson (graphpad prism package 2·01; Graph-Pad). Graphical presentation was performed using the graphpad prism package 2·01.

Results

Comparison of diagnostic methods

During the June 2005 survey, 489 plants with symptoms were identified as described above. In September 2005 each plant was revisited and tissue samples again collected. However, some plants could not be re-sampled because they had died, had been removed, or otherwise could not be located. In total, tissue samples were collected from 428 plants in both sampling seasons, representing 41 different species. Because of this difference, and because of some missing or failed results in both laboratories, different subsets of the entire collection were used for different analyses (Table 1). Subsets were always chosen so as to try to maximize the number of samples fully processed either in the two seasons, or by the two laboratories. Species, sampling sites and number of samples are reported in Table 2. Phytophthora ramorum was detected at all sites in both sampling periods. In total, 25 of 41 species (60·1%) tested positive for P. ramorum in at least one of the five assays during the June survey (Table 3). Of those, 16 (43·9%) were already known as hosts of P. ramorum in the USA and/or Europe (http://rapra.csl.gov.uk). The remaining nine species, Artemisia tridentata, Cryptacantha torreyana, Cytisus scoparius, Lilium sp., Marah fabaceus, Oxalis sp., Populus deltoides, Rubus ursinus and Vinca sp., are reported for the first time as putative hosts for P. ramorum. Among these species Lilium sp., M. fabaceus and R. ursinus can be considered infected, according to the criteria used for the gold standard. However, all putative host species need to be confirmed as P. ramorum hosts by performing Koch’s postulates.

Table 1.   Number of plant samples used for statistical analyses of Phytophthora ramorum assay accuracy
Statistical analysisUCBTAMU
JuneSeptemberJuneSeptember
Total number of collected plants489428  
Sensitivity and specificity of each assay method428   
Effect of different plant taxa on accuracy of tests428   
Effect of sampling period and site on accuracy of tests346346  
Collection site effect on the efficacy of each assay233   
Effect of laboratories on accuracy of tests135135135135
Table 2.   Plant samples showing characteristic symptoms of Phytophthora ramorum infection collected both in June (n = 428) and September (n = 428) 2005, by host species, source and site
SpeciesSourceNo. samplesSitea
ABCBOCHDDGH
  1. aA = Angel’s Island, BC = St Cruz Bean Creek, BO = Bolinas Bridge Road, CH = China Camp, D = Big Sur, DG = Samuel, H = St Cruz-Harpin.

Acer macrophyllum leaf10 42  4 
Adiantum jordaniileaf1     1 
Aesculus californicaleaf3 1   2 
Arbutus menziesii leaf13 216 13
Arctostaphylos sp.leaf1   1   
Artemisia tridentata leaf11      
Baccharis pilularis var. consanguinealeaf2  1  1 
Corylus cornuta leaf196361 21
Cryptantha torreyanaleaf1     1 
Cytisus scopariusleaf21   1  
Eucalyptus sp.leaf22      
Fragaria vesca ssp. californicaleaf1      1
Grass (unidentified)leaf1    1  
Heteromeles arbutifolialeaf15 1363 2
Juglans californica var. californicaleaf1 1     
Lilium sp.leaf5    5  
Lithocarpus densiflorusleaf29102  152 
Lithocarpus densifloruscanker209  92  
Lonicera hispidulaleaf42    2 
Marah fabaceusleaf4  1 21 
Melilotus sp.leaf1     1 
Mimulus guttatusleaf2  1   1
Oxalis sp.leaf1 1     
Pinus muricata leaf11      
Polystichum munitum leaf11      
Populus deltoides leaf1 1     
Pseudotsuga menziesiileaf3 2    1
Quercus agrifolialeaf3     21
Quercus chrysolepisleaf1  1    
Rhamnus purshianaleaf6  2  4 
Rhododendron sp.leaf1 1     
Ribes californicumleaf1 1     
Rosa californicaleaf6 11  22
Rubus ursinus leaf3 1   2 
Salix caprealeaf51    4 
Sambucus callicarpa leaf1      1
Sequoia sempervirens leaf17 31 823
Smilacina racemosaleaf1     1 
Symphoricarpos sp.leaf2      2
Umbellularia californicaleaf23333272936442935
Vaccinium ovatum leaf2     11
Vinca sp.leaf1 1     
Total 42870544959816554
Table 3.   Detection of Phytopthora ramorum in field plant samples using culture-based, DAS-ELISA, TaqMan and SYBR-green assays. Species already reported as P. ramorum hosts are indicated with an ‘x’. Data are from samples obtained during the June survey and were processed at UCB
HostSpeciesSourceNo. samples testedaAssay
Culture-basedDAS-ELISATaqManSYBR
  1. aNumber of positive samples detected in PARP or V8-PARP assay.

xAcer macrophyllum leaf105645
xAdiantum jordaniileaf11111
xAesculus californicaleaf32201
xArbutus menziesii leaf137811
xArctostaphylos sp.leaf10000
 Artemisia tridentata leaf10100
 Baccharis pilularis var. consanguinealeaf20000
xCorylus cornuta leaf1951026
 Cryptantha torrreyanaleaf10001
 Cytisus scopariusleaf20002
xEucalyptus sp.leaf20000
 Fragaria vesca ssp. californicaleaf10000
 Grass (unidentified)leaf10000
xHeteromeles arbutifolialeaf155836
 Juglans californica var. californicaleaf10000
 Lilium sp.leaf52121
xLithocarpus densiflorusleaf2921262323
xLithocarpus densifloruscanker204161613
xLonicera hispidulaleaf41102
 Marah fabaceusleaf41333
 Melilotus sp.leaf10000
 Mimulus guttatusleaf20000
 Oxalis sp.leaf10100
 Pinus muricata leaf10000
 Polystichum munitum leaf10000
 Populus deltoides leaf10100
xPseudotsuga menziesiileaf32222
xQuercus agrifolialeaf33322
xQuercus chrysolepisleaf10000
xRhamnus purshianaleaf63334
xRhododendron sp.leaf10000
 Ribes californicumleaf10000
xRosa californicaleaf63224
 Rubus ursinus leaf31303
xSalix caprealeaf50000
 Sambucus callicarpa leaf10000
xSequoia sempervirens leaf176646
xSmilacina racemosaleaf10011
 Symphoricarpos sp.leaf21011
xUmbellularia californicaleaf233207218228228
xVaccinium ovatum leaf20000
 Vinca sp.leaf10100
 Total 428280323298316

The results of the June survey were used to compare the accuracy of each assay method. During the June survey, 356 (83·2%) samples tested positive in at least one assay, including 241 samples (56·3%) that tested positive in all assays, whilst 72 (16·8%) were negative in all tests. The culture-based assays had the lowest proportions of P. ramorum-positive samples (65%). No significant difference was found between the proportion of positive samples detected by plating on V8-PARP and on PARP (Fisher’s exact test; > 0·05), so all further analyses were performed using only the V8-PARP assay results. Phytophthora ramorum was detected in 75·4, 69·6 and 73·8% of the samples tested by the DAS-ELISA, TaqMan and SYBR assays, respectively. The accuracy of each test was evaluated for 428 samples, analysed in June at the UCB laboratory, by calculating diagnostic sensitivity, diagnostic specificity, PPV and NPV (Table 4) in relation to the gold standard. The diagnostic sensitivity of the culturing assay was the lowest (86·7%), and in pairwise comparisons it was significantly lower than all the other methods (Fisher’s exact test, < 0·05). The diagnostic specificity value of DAS-ELISA was not significantly different from that of SYBR PCR (Fisher’s exact test, > 0·05). The diagnostic specificity values of both DAS-ELISA and SYBR PCR were significantly lower than the corresponding values of culturing and TaqMan assays (Fisher’s exact test, < 0·05). PPVs and NPVs valid for this experimental population characterized by a prevalence of 75% are shown in Table 4. Since PPV and NPV are dependent on disease prevalence, they were compared over a range of hypothetical prevalence values (Table 5). Results indicated that overall as prevalence increases, PPVs also increase, whilst NPVs decrease; rates of increase/decrease differed with method (Table 4).

Table 4.   Diagnostic values of culturing, enzyme-linked immunosorbent assay (DAS-ELISA), TaqMan assay and SYBR-green assay for Phytophthora ramorum detection in 428 plant field samples (prevalence 75%)
MethodPositiveNegativeFisher P-valueaDiagnostic valueb
TrueFalseFalseTrueSe % (CI)Sp % (CI)PPV % (CI)NPV % (CI)
  1. aFisher’s P-value of interaction of each method with the gold standard.

  2. bSe, diagnostic sensitivity; Sp, diagnostic specificity; PPV, predictive positive value; NPV, predictive negative value; CI = 95% confidence interval.

  3. Gold standard = those samples that tested positive in both the culture assay and the CSL TaqMan PCR assay.

Culturing280043105<0·000186·7 (82·4–90·2)10010070·9 (62·9–78·1)
DAS-ELISA298252580<0·000192·3 (88·7–94·9)76·2 (66·8–83·9)92·3 (88·7–94·9)76·2 (66·8–83·9)
TaqMan298025105<0·000192·3 (88·7–94·9)10010080·8 (72·9–87·1)
SYBR-green302142191<0·000193·5 (90·2–95·9)86·7 (78·6–92·5)95·6 (92·6–97·5)81·3 (72·8–88·0)
Table 5.   Positive predictive values (PPVs) and negative predictive values (NPVs) of culturing, enzyme-linked immunosorbent assay (DAS-ELISA), TaqMan assay (TAQ) and SYBR-green assay (SYBR) for Phytophthora ramorum detection in wild plant samples at different disease prevalences
PrevalencePositive predictive valuesNegative predictive values
CulturingELISATAQSYBRCulturingELISATAQSYBR
  1. PPV = the probability that a positive test result correctly identifies the presence of infection; NPV = the probability that a negative test result correctly identifies the absence of infection.

0·011·000·041·000·071·001·001·001·00
0·051·000·171·000·270·990·991·001·00
0·101·000·301·000·440·990·990·990·99
0·201·000·491·000·640·970·970·980·98
0·401·000·721·000·830·920·930·950·96
0·601·000·851·000·920·840·860·890·91
0·801·000·941·000·970·660·700·760·78
0·901·000·971·000·980·460·510·580·62

Effect of plant taxa on accuracy of tests

For this analysis, a total of 233 samples from U. californica and 195 samples from other hosts were analysed according to UCB results. The frequency of P. ramorum detection in the U. californica samples was higher than for the other hosts with all the methods used (Fisher’s exact test, < 0·0001) (Fig. 1). Accuracy of diagnostic methods varied by host. Diagnostic sensitivity of molecular and immunological assays was higher for U. californica than for other hosts (Fisher’s exact test, < 0·0001, < 0·0001 and < 0·005 for TaqMan, SYBR and DAS-ELISA, respectively). Conversely, the diagnostic sensitivity of the culture-based assay did not differ between U. californica and other hosts (Table 6) (Fisher’s exact test, > 0·05). Diagnostic specificity did not differ significantly among assays (Fisher’s exact test, > 0·05).

Figure 1.

 Frequency of Phytophthora ramorum detection in samples of Umbellularia californica (white bars) and other species (black bars), through culturing, enzyme-linked immunosorbent assay (DAS-ELISA), TaqMan and SYBR-green assays. All samples were collected in June and processed at the University of California, Berkeley (UCB).

Table 6.   Diagnostic sensitivity and specificity values of culturing, enzyme-linked immunosorbent assay (DAS-ELISA), TaqMan assay and SYBR-green assay for Phytophthora ramorum detection in 233 Umbellularia californica samples and 195 samples from other plant species
SamplesCulturingDAS-ELISATaqManSYBR-green
Se % (CI)Sp % (CI)Se % (CI)Sp % (CI)Se % (CI)Sp % (CI)Se % (CI)Sp % (CI)
  1. Se = diagnostic sensitivity; CI = 95% confidence interval; Sp = diagnostic specificity.

U. californica90·8 (86·3–94·2)10095·2 (91·5–97·6)80 (28·4–99·5)100100100100
Other species90·1 (82·0–95·3)96·185·3 (76·5–91·7)76 (66·4–83·9)73·7 (63·6–82·2)10077·9 (68·2–85·7)86·6 (77·6–92·1)

Effects of sampling period and site on accuracy of tests

A total of 692 samples were analysed at UCB to study the effect of sampling time and location on the accuracy of each assay. These 692 samples represented those that were tested using all four diagnostic assays both in June (346 samples) and September (346 samples). The diagnostic values of different assays performed in two sampling periods were analysed using a ‘spring gold standard’ for June samples based on combined culturing and CSL TaqMan data of the June samples, whilst the results of the September samples were analysed using an ‘autumn gold standard’ based on combined culturing and CSL TaqMan data of the September samples. Using two standards ensured the results were not prejudiced by new infections occurring in between the two sampling periods or by local extinction of spring infections. Diagnostic values of methods were calculated using the actual disease prevalence measured at each sampling time. The percentage of positive samples significantly decreased from June to September both for culturing and TaqMan assays (Fischer’s exact test; < 0·0001), but remained unchanged for the other techniques. Diagnostic sensitivity, but not diagnostic specificity, of culturing was significantly affected by sampling period (Fisher’s exact test,  0·01, data not shown), but no effect of seasonality was detected on the diagnostic sensitivity of the other molecular and immunological assays (Fisher’s exact test, > 0·05).

The effect of the samples’ provenance on the efficacy of each assay was investigated using only collections of U. californica. This was done because this host was the only one present in significant numbers at all study sites. For all assays, diagnostic accuracy was not affected by the collection site of the samples (Fisher’s exact test, > 0·05).

Effect of laboratory on accuracy of tests

Although all samples were processed in both laboratories, results for one assay were not available for a certain number of samples processed at TAMU. A total of 135 samples (115 foliar and 20 bark samples) were processed both at the TAMU and UCB laboratories using all five assays and thus were considered for this analysis. When comparisons were made among those assays employed on the entire sample size, results did not change. Diagnostic values of culturing, ELISA, TaqMan and SYBR-green analysis from June and September were evaluated for both laboratories using a spring gold standard (based on the combined culturing and TaqMan results of spring samples tested at UCB) and an autumn gold standard (based on the combined culturing and TaqMan results of autumn samples tested at UCB), respectively. The choice of UCB for the determination of the gold standard for both laboratories was necessary because of the loss in viability of samples shipped to TAMU (see below).

The percentage of positive samples detected by culturing at TAMU was significantly lower than that detected at UCB both in June (Fisher’s exact test, < 0·004) (Fig. 2) and September (Fisher’s exact test, < 0·002) (Fig. 3), whilst the percentage of positive samples detected by TaqMan was lower at UCB in September (Fisher’s exact test, < 0·02) (Fig. 3). However, only the diagnostic sensitivity of culturing was significantly different between UCB and TAMU, both in June and September for foliar and bark samples (Fig. 4).

Figure 2.

 Percentage of Phytophthora ramorum-positive foliar samples (collected from woodland plant species in California) detected by culturing, ELISA, TaqMan and SYBR-green at the University of California, Berkeley (UCB) (white bars) and Texas A&M University (TAMU) (grey bars) in June. Bars capped with the same letter within a specific method are not significantly different according to Fisher’exact test (< 0·05).

Figure 3.

 Percentage of Phytophthora ramorum-positive foliar samples (collected from woodland plant species in California) detected by culturing, ELISA, TaqMan and SYBR-green at the University of California, Berkeley (UCB) (white bars) and Texas A&M University (TAMU) (grey bars) in September. Bars capped with the same letter within a specific method are not significantly different according to Fisher’exact test (< 0·05).

Figure 4.

 Sensitivity values for diagnosis of Phytophthora ramorum of culturing at the University of California, Berkeley (UCB) (white bars) and Texas A&M University (TAMU) (grey bars), in June and September, for foliar (plain bars) and bark samples (dashed bars) collected from woodland plant species in California.

Discussion

Diagnostic values have been calculated for diagnostic assays specifically designed for P. ramorum, and have been used either for the comparison of two such assays on a few host species (Hughes et al., 2006a,b; Lane et al., 2006; Tomlinson et al., 2007), or for the comparison of multiple methods on a single host species (Kox et al., 2007). The present study used diagnostic values from two laboratories to compare culturing, molecular and immunological methods for in planta detection of P. ramorum in 41 different plant taxa sampled in two seasons from seven non-cultivated sites in California. The results presented here provide new information on the application and accuracy of diagnostic methods for the detection of P. ramorum in natural environments at the host community level.

The choice of the most effective diagnostic approach for the detection of P. ramorum, including the combination of more than one diagnostic assay when necessary, is relevant to properly enforce quarantines and to provide the best tools for epidemiological studies. Proper diagnosis of P. ramorum can be problematic because of its wide host range, the variability of symptoms it may cause on different hosts, and its cyclical life cycle. In the present study, 25 plant taxa were found to harbour P. ramorum. Nine plant species were reported as potential hosts of P. ramorm for the first time (Anonymous, 2008), even if their susceptibility needs to be further confirmed through the completion of Koch’s postulates. Umbellularia californica was the prevalent co-dominant and understory tree species at all of the study sites; hence it is not surprising that the bulk of the samples that tested positive for P. ramorum were of this host species.

Overall, about 63 and 43% of field samples with symptoms were infected with P. ramorum in June and September, respectively, based on culturing results. These values are higher than those previously reported in literature. Hayden et al. (2004) reported the detection of P. ramorum in 36% of samples taken from different hosts with symptoms with molecular and culture-based methods, whilst assays from the Central Science Laboratory (UK) did not exceed a 30% level of detection (Lane et al., 2006). The high proportion of positive samples reported in the present study is probably the result of the increased knowledge regarding appearance of symptoms caused by P. ramorum in California and by the favourable wet weather conditions during spring and autumn 2005. The proportion of positive samples does not provide a measure of accuracy of a detection assay, since false positives and negatives are not considered. Instead, values of diagnostic sensitivity and specificity need to be used to estimate the validity of any given method. Values of diagnostic sensitivity and specificity of molecular (sensitivity: TaqMan = 92·3; SYBR-green = 93·5; specificity: TaqMan = 100 and SYBR-green = 86·7) and immunological methods (sensitivity: DAS-ELISA = 92·3; specificity: DAS-ELISA = 76·2) calculated using the total number of samples were high and comparable with those reported by Kox et al. (2007) (sensitivity/specificity: DAS-ELISA = 91·2/77·5 and TaqMan = 83·8/100) for Rhododendron samples. The DAS-ELISA and SYBR-green had lower diagnostic specificity than TaqMan and are therefore more likely to result in false positive results (Lane et al., 2007). Diagnostic sensitivity of the culture-based assays was significantly lower than that of molecular and immunological methods, as already reported for leaf samples by Kox et al. (2007). Success of culturing can be affected, as discussed below, by the amount of time that has lapsed between sample collection and culturing, by sample handling resulting in desiccation or deterioration of the sample, and also by the physiological state of the pathogen at the time of sampling. For instance, it has been hypothesized that lower culturing success in the autumn may be caused in part by a dormant state of the overwintering pathogen, and in part by its death from desiccation (Hayden et al., 2004). Diagnostic values of culture-based assays did not significantly differ between U. californica and other hosts, suggesting that the accuracy of currently accepted isolation methods in professional pathology laboratories is independent of host, although some hosts may be recalcitrant to culturing and may require some modification of isolation techniques. Conversely, diagnostic sensitivity of the molecular and immunological methods reached 100 and 95·2% respectively, when applied to U. californica, but resulted in lower values for other species. The pooling of non-U. californica hosts does not allow for exact identification of those responding differently to the DNA-based assay. Nonetheless, the presence of significant differences between U. californica and other hosts, in spite of opposite and potentially neutralizing responses of individual hosts, is a strong indication that there are differences in the results driven by host species.

The data support the hypothesis that the type of host tissue being sampled is a critical issue for pathogen detection when using molecular and immunological methods. This effect may be caused by the varying intensity of pathogen colonization in plant tissue from different hosts, and/or by a reduction in the efficacy of molecular assays caused by the presence of inhibitors. The effect of PCR inhibitors has been previously documented by showing that the addition of plant DNA extracts can dramatically reduce the sensitivity of PCR-based assays for P. ramorum (Martin et al., 2004; Hayden et al., 2006). Likewise, interaction between plant and pathogen proteins has been reported to affect the sensitivity of viral detection on hibiscus (Kamenova & Adkins, 2004).

The effect of seasonality on the proportion of samples positively detected with culture-based and TaqMan assays mirrors data in the literature. Hayden et al. (2004) reported the decreasing efficacy of a nested PCR assay from spring to winter on canker exudates from oak species and leaf samples from different hosts. Similarly, a seasonal effect on isolation of P. ramorum from U. californica leaves was reported by Davidson et al. (2003), with a reduction in isolation success from 90% in June to 25% in October. In the present study, the reduction of the proportion of positive samples between the June and September surveys was less marked but still significant. The drop in number of positive samples between seasons may be explained in two ways, probably not mutually exclusive, but both affecting pathogen viability: (i) the infection on some plants may die out because of unfavourable environmental conditions and, (ii) the pathogen may be dormant but still alive on some plants because of unfavourable environmental conditions. Unfortunately, because diagnostic values are calculated against a gold standard based on methods that may be highly susceptible to seasonality, this and other similar studies cannot fully resolve the issue. One solution would be to determine the infection status of the same plant hosts a third time when environmental conditions become favourable once again during a wet spring, but somehow excluding the pathogen from re-infecting the same host. Based on the assumptions above, the significant drop in number of positive samples detected when employing the CSL Taqman assay may either signify that a number of infections died out in between the two samplings, or, alternatively, that this assay may not be optimal to detect small amounts of target DNA, such as those expected for a dormant live pathogen embedded in host tissue. It should be noted that although there was a significant reduction in the percentage of positive samples detected by the Taqman assay between June and September, the diagnostic sensitivity of the assay was not statistically different in the two sampling periods.

The lower number of positive samples detected by culturing in the TAMU laboratory can be explained by the delay in processing as a result of shipping. Variation in lag times between sample collection and processing in the laboratory is an inevitable consequence intrinsically associated with the regulatory diagnostic process at the national or even state level. These results are an indication that culturing may not be a diagnostic assay which produce results that can be easily compared among laboratories, even when technical proficiency is optimal and identical among them. The results, in terms of positive detection and diagnostic sensitivity, of both SYBR-green and DAS-ELISA, were identical in the two laboratories, indicating that these techniques are robust and can be easily used among different operators providing identical results. Both techniques displayed lower specificity than the other assays tested. The lower specificity of the DAS-ELISA assay used is not surprising because the polyclonal antibodies currently available cross-react with a number of Phytophthora and Pythium spp. (MacDonald et al., 1990; Themann & Werres, 1997). Surprisingly, a lower number of positive samples was obtained by CSL Taqman at UCB than at TAMU, even though there were no statistically significant differences in diagnostic sensitivity between the two laboratories using this assay. The differences in the number of positive samples may have been caused either by operator error or by the use of different equipment in the two laboratories. In either case, the results are indicative that CSL Taqman may be more sensitive to operator and/or machinery differences than other diagnostic assays, thus reinforcing the need to periodically compare results from different laboratories through proficiency panels. As a future direction, the optimization of molecular protocols to a wide range of hosts and tissue types, and assays producing identical results in different laboratories, may reduce the use of culture-based assays, and therefore reduce the time and cost of analysis.

This study provides useful comparative information on the most commonly used detection methods for P. ramorum and puts forth several questions regarding how comparable these methods may actually be. In particular, the current study allows for a true analysis of the effect of season and of provenance of samples on diagnostic assays. These two effects cannot be properly studied in plant nurseries because of the artificial environmental conditions imposed by horticultural practices, and because of trading of plants among facilities. Significant effects of seasonality and of time between collection of samples and isolations on culture-based assays are reported here, indicating that culturing should not be used in surveys that are carried out in different seasons, or when lag time between collection and plating cannot be standardized. Conversely, when frequency of infection needs to be analysed among different plant taxa and if host-specific baseline data on sensitivity of molecular methods is missing, the culture-based assay may be the best choice in order to avoid unreliable estimations of disease abundance caused by the presence of inhibitors. Provenance of samples did not seem to affect any of the tested assays, indicating that once parameters are defined for a host species, these may be used without excessive concerns regarding within-species variation. This observation may be extremely relevant, not only for surveys of natural ecosystems on a large geographic scale, but also for the regulation of the trade of ornamental plants on a planetary scale.

Overall, the data presented here indicate that optimal detection of P. ramorum will occur when sampling is performed in wet and warm periods: operators should be aware that the efficiency of all techniques will decrease significantly when surveying in dry conditions. Because of the marked effect of seasonality, it may be inappropriate to compare infection levels among seasons or even among years characterized by different amounts of rainfall without correcting for the effect of seasonality.

Considering the advantages and disadvantages of each method, the application of more than one assay appears to guarantee the best accuracy in P. ramorum detection. Although current regulations recommend the use of culturing and of a molecular assay less prone to produce false negatives, this study shows that outcomes will vary depending on the molecular assay employed. Significant changes in numbers of samples positively detected by an assay when performed in different laboratories or at different sampling times warrant further research, even if no significant differences in sensitivity and or specificity were found among assays. In fact, significant differences in numbers of samples positive for P. ramorum obtained by different assays may be indicative of intrinsic properties of the tested assays, and failure to detect significant changes in sensitivity and specificity may be linked to the inevitably limited sampling size of most studies. Furthermore, lack of statistical significance when comparing assays could be a consequence of the current choice of gold standard, heavily relying on culturing success. Because the purpose of using molecular techniques is not only to make results more comparable through time and space, but also to improve the detection rates obtained by culturing, the selection of a molecular assay that is comparable in sensitivity to culturing is clearly not fully justifiable. For instance, this study shows that in absolute terms, the DAS-ELISA and SYBR assays surpass CSL TaqMan in sensitivity. Although more research is needed to differentiate between false negatives and true positives when using high-sensitivity methods, and to design new assays that will increase both sensitivity and specificity, an effort towards increasing the sensitivity of assays should be pursued

Finally, it should be noted that the choice of a method or of a combination of methods also depends on the prevalence of the disease in host populations. The data indicate that as prevalence increases, the rate at which PPV increases or NPV decreases varies by method: whilst at low prevalence, there are no differences in predictive values among methods, at high prevalence, values obtained by molecular methods appear to be more accurate. This information may be important in directing a different choice of diagnostic assays during outbreaks vs. periods of low disease spread.

Acknowledgements

This work was supported by a grant from the United States Department of Agriculture-Animal and Plant Health Inspection Services to MG. We would like to thank David Appel for helpful comments and suggestions and Daniel Hüberli for the excellent assistance in the field. We also thank Dr. M. Thon for review of this manuscript.

Ancillary