Plant–soil feedback: experimental approaches, statistical analyses and ecological interpretations

Authors

  • E. Pernilla Brinkman,

    Corresponding author
    1. Department of Terrestrial Ecology, Netherlands Institute of Ecology (NIOO-KNAW), Boterhoeksestraat 48, 6666 GA Heteren, The Netherlands
    Search for more papers by this author
  • Wim H. Van der Putten,

    1. Department of Terrestrial Ecology, Netherlands Institute of Ecology (NIOO-KNAW), Boterhoeksestraat 48, 6666 GA Heteren, The Netherlands
    2. Laboratory of Nematology, Wageningen University, PO Box 8123, 6700 ES Wageningen, The Netherlands
    Search for more papers by this author
  • Evert-Jan Bakker,

    1. Biometris (Mathematical and Statistical Methods Group), Wageningen University, PO Box 100, 6708 PD Wageningen, The Netherlands
    Search for more papers by this author
  • Koen J.F. Verhoeven

    1. Department of Terrestrial Ecology, Netherlands Institute of Ecology (NIOO-KNAW), Boterhoeksestraat 48, 6666 GA Heteren, The Netherlands
    2. Laboratory of Nematology, Wageningen University, PO Box 8123, 6700 ES Wageningen, The Netherlands
    Search for more papers by this author

Corresponding author. E-mail: p.brinkman@nioo.knaw.nl

Summary

1. Feedback between plants and soil organisms has become widely recognized as a driving force of community composition and ecosystem functioning. However, there is little uniformity in quantification and analysis of plant–soil feedback effects. Meta-analysis suggested that the various experimental methods tend to result in different feedback values. Yet, a direct comparison of the different experimental approaches and their statistical analyses is lacking.

2. We used currently applied methods to calculate plant–soil feedback value ranges and compared their statistical analyses to those based on actual biomass data. Then, we re-analysed a case study to compare plant–soil feedback values obtained under the same environmental conditions, but using different experimental approaches: soil sterilization, addition of soil inoculum, and soil conditioning by ‘own’ vs. ‘foreign’ plant species.

3. Different measures to calculate plant–soil feedback values were more variable in positive than in negative feedback values. Analysis of calculated feedback values that are based on treatment averages can lead to a serious inflation of type I errors.

4. In our case study, both the strength and the direction of the feedback effects depended on the experimental approach that was chosen, leading to diverging conclusions on whether feedback to a certain soil was positive or negative. Soil sterilization and addition of soil organisms yielded larger feedback than comparison of own and foreign soil.

5.Synthesis. The ecological interpretation of plant–soil feedback effects strongly depends on the experimental procedure. When the research question focuses on the strength and the sign of plant–soil feedback, soil sterilization (presumed that the side effect of increased nutrient availability can be controlled) or addition of soil inoculum is to be preferred. When the research question concerns the specificity of soil feedback effects, plant performance can be better compared between own and foreign soil. We recommend that when using calculated feedback values, the original data need to be presented as well in order to trace the cause of the effect.

Introduction

Plants alter abiotic and biotic soil properties and resulting processes, which in turn influence plant development, productivity and competitiveness (Ehrenfeld, Ravit & Elgersma 2005). There is general consensus that these plant–soil interactions are major drivers of vegetation diversity and functioning of ecosystems (Wardle 2002; Bever 2003; Reynolds et al. 2003; Van der Putten 2003). The term ‘plant–soil feedback’ has been coined to name mutual interactions between plants and soil organisms (Bever 1994; Bever, Westover & Antonovics 1997) and has been adopted by many ecologists. Indeed, plant–soil feedback has become an ecological concept and a number of reviews have focused on how to interpret negative and positive plant–soil feedback effects and their roles in community organization and ecosystem functioning (Bever, Westover & Antonovics 1997; Bever 2003; Reynolds et al. 2003; Van der Putten 2003; Ehrenfeld, Ravit & Elgersma 2005; Reinhart & Callaway 2006). Although the concept of plant–soil feedback has become widely recognized, there is little uniformity in experimental quantification and statistical analysis of feedback effects. There is evidence that the type of experimental approach that is used to establish plant–soil feedback effects influences the direction and size of the effect (Kulmatiski & Kardol 2008; Kulmatiski et al. 2008). As for the analysis, to date no study has yet compared the strengths and weaknesses of the different methods that are used.

The basic idea of plant–soil feedback experiments is that plants first influence the composition of the soil community, which is called soil conditioning. Then, the effects of conditioning are evaluated by assessing soil effects on subsequent plant growth. Although this idea appears quite simple, the design of plant–soil feedback experiments is complex. For example, proper controls are required in order to assess if effects of soil conditioning indeed are due to altered soil community composition and not to changes in the nutritional status of the soil. It is also important to properly ensure independence of replicates by using separate soil samples. If replicates are not independent, for instance when a soil sample has been split and one half sterilized, this should be accounted for in the analysis (Garrett et al. 2004; see e.g. Littell et al. 2006 for an in-depth discussion of mixed models).

The earlier experiments testing plant–soil feedback effects were started from natural field-sampled soil (Oremus & Otten 1981; Augspurger & Kelly 1984; Van der Putten, Van Dijk & Troelstra 1988; Van der Putten, Van Dijk & Peters 1993). The strength of this method is that plants have influenced the soil for a long period of time under natural conditions. A weakness of this approach is that soils may differ not only in the composition of the soil community, but also in abiotic properties and that these effects are difficult to disentangle. In addition, unless plants have been growing in monoculture for a long time period, several different plant species may have influenced the soil community. Nevertheless, these field-based experiments can yield important insight into plant–soil feedback interactions both when based on sampled soil from natural vegetation (Van der Putten, Van Dijk & Peters 1993; Packer & Clay 2000), or from long-term field experiments (Bezemer et al. 2006).

A next step in studies on plant–soil feedback interactions has been that plant species were grown in living soils to develop a soil community, followed by a test phase in which the growth response on changed biotic conditions was tested (e.g. Bever 1994; Klironomos 2002). The strength of this two-phase plant–soil feedback approach is that the effects are less dependent on possible side effects of local differences in abiotic soil conditions and that the plant species influencing the soil is controlled. However, a weakness is that in pots under greenhouse conditions a different soil community may develop than in the field. This is due to the fact that in the greenhouse, growth conditions like temperature, soil moisture and soil texture are likely to be different and that the soil biota are exposed to the plant roots for a shorter time period than in the field. In general, effect sizes of experiments with experimentally conditioned soils are greater than with natural field-collected soil (Kulmatiski et al. 2008). This suggests that conditioning in the greenhouse may exaggerate effects of soil biota when compared to field-collected soil samples. Another weakness of these feedback experiments is that the growth responses in the feedback phase can be due to nutrient depletion in the first phase. The possibility of nutrient depletion can be relatively easily excluded when the biomass in the first (conditioning) phase does not negatively correlate with that in the second (test) phase (Kardol, Bezemer & Van der Putten 2006). However, if they do correlate, the feedback effects may be due to changes in soil community composition, as well as to changes in soil nutrient availability.

In two-phase plant–soil feedback experiments, in the second phase of the experiment, new plants are grown in the conditioned soil and their performance is determined by assessing biomass or ontogenetic changes (Bever 1994). The feedback effect can be quantified with a variety of methods (Fig. 1): soil sterilization (comparing growth of phase-2 plants on non-sterilized vs. sterilized phase-1 soils; Van der Putten, Van Dijk & Peters 1993; Brinkman, Troelstra & Van der Putten 2005; Van der Putten et al. 2007), addition of soil inoculum to sterilized background soil (comparing growth of phase-2 plants on phase-1 soil with vs. without soil organisms; Callaway et al. 2004; Nijjer, Rogers & Siemann 2007; Van der Putten et al. 2007) and soil conditioning by ‘own’ vs. ‘foreign’ plant species (comparing growth of phase-2 plants on soil that was conditioned by the same species vs. by different species; Bever 1994; Peltzer 2001; Klironomos 2002; Callaway et al. 2004; Bezemer et al. 2006; Nijjer, Rogers & Siemann 2007; Van der Putten et al. 2007; Engelkes et al. 2008). The approaches of soil sterilization and addition of soil inoculum both compare plant performance between soils with and without soil organisms. In contrast, the comparison of ‘own’ vs. ‘foreign’ soil is based on the assumption that soil organisms with a specific relationship to the plant are more abundant in ‘own’ than in ‘foreign’ soil. Therefore, this approach addresses the specificity of the interactions between plants and soil organisms in combination with the abundance of these specific soil biota. This may explain why experiments comparing performance in ‘own’ vs. ‘foreign’ soil typically yield weaker feedback effects than experiments using soil sterilization (Kulmatiski & Kardol 2008).

Figure 1.

 The three approaches to assess plant–soil feedback that were used in the case study with, as an example, Cenchrus biflorus as a test plant. Soil samples were taken from the root zone of Eragrostis lehmanniana, Aristida meridionalis and of C. biflorus. In phase 1 (the conditioning phase), plant species were grown on soil with inoculum originating from their own root zone. Sterilized background soil was mixed with 3% root zone soil that was either sterilized or left non-sterilized, creating soil without or with soil organisms, respectively. In phase 2 (the test phase), one half of the conditioned soils with soil inoculum with biota was sterilized as a control. In phase 2, C. biflorus was planted on all conditioned soils. For the soil inoculum approach (inline image), plant biomass was compared between inoculated soils with or without biota in phase one. For the sterilization approach (inline image), plant biomass was compared between soils that were sterilized or non-sterilized in phase 2. For the own vs. foreign soil approach (*), plant biomass was compared between non-sterilized soils that in phase 1 were conditioned by different plant species.

A strength of comparing non-sterilized to sterilized soil is that in the non-sterilized soil, soil organisms are present in normal densities from the beginning of the experiment. A weakness of sterilization is that, particularly in nutrient rich soils, the concentration of nutrients increases due to decomposition of the killed soil organisms (Powlsen & Jenkinson 1976), so that both biotic and abiotic soil conditions are changed at the same time. The increased nutrient availability may unintentionally enhance plant growth (Jakobsen & Andersen 1982). This side effect can be reduced, but not always completely ruled out, by additional nutrient supply to the non-sterilized and/or the sterilized soil (Troelstra et al. 2001). However, nutrient addition to the non-sterilized soil can decrease the level of mycorrhizal colonization (Jensen & Jakobsen 1980) and of pathogenic effects (Van der Putten & Peters 1997), which makes it less likely to observe significant plant–soil feedback effects. Second, mainly in the sterilized soil, an artificial soil community may develop due to accidental contamination and colonization with soil organisms originating from the air or water supply. Often, the first colonizers will be opportunistic saprophytic soil biota that feed on easily decomposable carbon resources that have become available due to sterilization, and possibly also other bacterivorous and fungivorous soil organisms.

Differences in effects of nutrient release by soil sterilization are largely avoided with the addition of soil inoculum, where small amounts of living and sterilized soil are added to a common sterilized background soil. However, when adding a soil inoculum, the density of soil organisms is smaller than in the field and hence, density-dependent effects on the plant will be smaller or take time to develop. The composition of the soil is a balance between the amount of soil inoculum that is needed to observe realistic effects of soil biota on the plants and the amount of sterilized background soil that is needed to dilute abiotic differences between the soil inocula. An inoculum density between 1 and 15% already can cause a decrease in plant biomass (Van der Putten, Van Dijk & Troelstra 1988; see Appendix S1 in Supporting Information). Preferably, the sterilized background soil should have the same abiotic properties as the added soil inoculum. In addition, as greenhouse conditions differ from field conditions, the conditioned soil communities may not necessarily reflect the communities in the field soil. Therefore, the conditioning phase should be long enough to develop a soil community, but repetition of the conditioning phase leading to three- and four-phased feedback experiments may diminish, rather than enhance, the resemblance of the conditioned soil community with that from the field.

Another relevant issue concerns the analysis of the data. Plant–soil feedback effects have been calculated and statistically analysed in a variety of ways (Table 1). In relatively simple plant–soil feedback experiments with only one plant species subjected to two treatments, biomass data usually have been compared directly by a t-test or anova. More complex experiments analysing feedback of different plant species have either tested soil effects on plant biomass for each plant species separately (Klironomos 2002), analysed soil effects on plant biomass in a full factorial experiment with all possible combinations of plant and soil origins (Bever 1994; Macel et al. 2007), or calculated feedback ratios and compared those among different plant species (Van der Putten, Van Dijk & Peters 1993).

Table 1.   Feedback calculations and statistical tests used in published literature and in the present study to simulate two treatments in simple soil feedback experiments. The combination ‘own’ (O) and ‘foreign’ (F) may also be substituted by the combinations ‘NS’ and ‘S’ or ‘+ biota’ and ‘- biota’, respectively. FB1, FB2, FB3 and FB4 are abbreviations of the four feedback calculation methods that are used in the text and in Figs. 4 and 5. An index ‘i’ refers to an individual observation, whereas an index ‘average’ refers to a treatment average
CalculationVariable modelledNumber, abbreviation and statistical test in the simulation
  1. Calculations as used by e.g. 1(Bever 1994; Bartelt-Ryser et al. 2005; Macel et al. 2007; Nijjer, Rogers & Siemann 2007), 2(Klironomos 2002), 3(Troelstra et al. 2001; Brinkman, Troelstra & Van der Putten 2005), 4(Kardol et al. 2007), 5(Van der Putten, Van Dijk & Peters 1993), 6(Engelkes et al. 2008), 7(Petermann et al. 2008), 8(inline image, where n = the number of different ‘F ’ treatments; Bezemer et al. 2006), 9(Markham & Chanway 1996).

O vs. F (no ratio calculation)1Biomass1One-way anova or paired t-test
Oi – Fi (pairwise)2Biomass2Paired t-test (deviation from 0)
Oi/Faverage3Feedback3 (FB1)One-sample t-test (deviation from 1)
(Oi – Faverage)/Faverage4Feedback One-sample t-test (deviation from 0)
Oi/Fi (pairwise)5Feedback4 (FB2)One-sample t-test (deviation from 1)
(Oi – Fi)/Fi (pairwise)6Feedback One-sample t-test (deviation from 0)
ln(Oi/Fi) (pairwise)7Feedback One-sample t-test (deviation from 0)
(Oi–grand average) / grand average8Feedback5 (FB3)One-sample t-test (deviation from 0)
(Oi - Fi)/max(Oi,Fi)9Feedback6 (FB4)One-sample t-test (deviation from 0)

Plant–soil feedback effects often are presented as the ratio of biomass obtained in test soil relative to control soil. Alternatively, to simplify interpretation in terms of positive and negative feedback, the biomass difference between test and control soils (that is the soil effect size) can be expressed as a fraction of biomass in the control soil (De Deyn et al. 2003). An advantage of calculating and presenting feedback ratios instead of analysing original biomass data is that relative values facilitate simple comparisons between species (or environments). However, some general caveats exist with respect to the analysis of ratios. For instance, they can obscure underlying patterns that are present in the original (numerator or denominator) variables, and their significance testing can have reduced power due to inflated variances of ratios compared to the variances of the original variables (Jasieński & Bazzaz 1999).

Here, we compare and analyse the different experimental and arithmetic methodologies that have been used to asses plant–soil feedback effects and discuss their ecological interpretations. We use the terminology of positive and negative feedback from the perspective of the effect on the plant, as is the common usage in literature on plant–soil feedback (Bever 1994; Bever, Westover & Antonovics 1997; Ehrenfeld, Ravit & Elgersma 2005). First, we compare possible value ranges of some commonly used feedback calculation methods. Then, we carry out a simple simulation in order to evaluate how different arithmetic approaches to capture the soil feedback effect are affecting the statistical analysis. Finally, we re-analyze a published study illustrating how different experimental setups for determining plant–soil feedback effects may influence the conclusions on the sign and direction of the plant–soil feedback. In our final conclusions, we propose which experimental methods are most appropriate to answer specific ecological questions.

Materials and methods

In Table 1, we give an overview of commonly applied measures to quantify and analyse feedback effects. Using the formulae in Table 1, feedback values can be calculated for individual replicates in the experiment; for instance, one ‘own’ (O) and one ‘foreign’ (F) plant yield one feedback score, which then is used to calculate an average feedback effect. Calculations that require pairing of individual ‘own’ to individual ‘foreign’ plants are indicated as ‘pairwise’ in Table 1. Such pairs can be defined naturally by the experimental design, for instance following blocking effects in the experiment. Some of the calculations can easily be derived from other calculations. For example, (O-F)/F is equivalent to (O/F)-1 and is applied to simplify interpretation of feedback effects in terms of positive and negative values.

Feedback calculations

We evaluated the range of outcomes of several commonly applied plant–soil feedback measures, for example for own vs. foreign, assuming a biomass in ‘foreign’ soil (F) of 1, and varying the biomass in ‘own’ soil (O) from 0.001 to 1000. It is assumed that one O treatment is compared with one or more F treatments. Obviously, the treatments can also be substituted by other soil treatments, for example ‘non-sterilized’ and ‘sterilized’, or ‘addition of soil inoculum with biota’ and ‘addition of soil inoculum without biota’. We calculated the outcomes for the following feedback calculation methods (Table 1): (– F)/F; ln(O/F); (– F)/(maximum biomass value of either O or F); (- (grand average of biomass in O and 4F))/(grand average of biomass in O and 4F), and (- (grand average of biomass in O and 10F))/(grand average of biomass in O and 10F)). The latter two measures contrast O with the overall mean of the O and F treatments and are used in the case of 4 and 10 foreign treatments, respectively. Division by the maximum biomass is derived from plant competition studies (Markham & Chanway 1996) and has the advantage that the resulting feedback values are constrained between –1 and 1.

Simulation

We simulated a simple experiment with one plant species and two soil treatment levels (‘own’ and ‘foreign’) in order to test how different analysis approaches affect statistical power to detect the plant–soil feedback effect. Simulations were performed in sas (version 8.02, The SAS Institute, Cary, NC, USA). Simulated data sets consisted of ten plants on ‘own’ soil that were randomly drawn from a normal distribution with mean = 10 and SD = 1 and ten plants on ‘foreign’ soil that were randomly drawn from a normal distribution with mean = (10 + effect size) and SD = 1. The effect size mimicked a hypothetical soil treatment effect. Four effect sizes were tested: 0, 0.5, 1 and 2, corresponding to units of SD. The effect size 0 represents the absence of a soil effect, so that values for plants on ‘own’ and ‘foreign’ soil were drawn from the same distribution. For each effect size, 1000 random data sets were generated. Each simulated data set was subjected to six tests: anova and paired t-test of original biomass values, and one-sample t-test of values generated with four different feedback calculations (FB1, FB2, FB3 and FB4; see Table 1). For the paired t-test and feedback calculations, individuals in ‘own’ and ‘foreign’ soils were paired randomly, resulting in data sets that consist of 10 replicates with one ‘own’ and one ‘foreign’ individual per replicate (yielding 10 feedback values following the calculations of Table 1).

Case study

Experimental design

We re-analysed a published study (Van der Putten et al. 2007) and results of an unpublished side experiment (W.H. Van der Putten) to illustrate differences in feedback effects created by the following experimental approaches: soil sterilization, addition of soil inoculum, and own vs. foreign soil. Originally, the experiment was performed to compare plant–soil feedback of the grass Cenchrus biflorus Roxb., which is invasive in southern Africa, with two native grasses, Aristida meridionalis Henr. and Eragrostis lehmanniana Nees. In the first (conditioning) phase of the experiment, the three plant species were grown in sterilized background soil inoculated with 3% soil originating from their own root zone (Fig. 1). There were two treatments: inoculation with non-sterilized soil (live inoculum) in order to let the plants develop a soil community with plant-species specific organisms, and inoculation with sterilized soil (dead inoculum) as a control. Thus, each of the three plant species conditioned soils with and without added soil organisms. In the second phase of the experiment, the feedback effect was tested in three ways (Fig. 1). (i) Soil sterilization: plants were grown in soil that in phase 1 contained live inoculum (conspecific soil) and in phase 2 was sterilized or left non-sterilized. (ii) Addition of soil inoculum: plants were grown in soil that in phase 1 contained either live or dead inoculum (conspecific soil) and in phase 2 was left non-sterilized. (iii) Own vs. foreign soil: plants were grown in soil that in phase 1 contained live inoculum and had been conditioned by their conspecifics, or by each of the two heterospecifics. The soil was left non-sterilized in phase 2.

Analysis approach

Because several statistical approaches exist to test for soil feedback effects, and because none may be advocated as the best one for all situations, we use this case study to illustrate and compare different analysis approaches. For convenience, we performed separate analyses for the ‘soil sterilization’ and for the ‘addition of soil inoculum’ feedback approaches (thus: separate analyses for each panel in Fig. 2).

Figure 2.

 Case study: dry biomass (g) ± SE of Cenchrus biflorus (cen) and Eragrostis lehmanniana (era) when grown in soil conditioned by either of the two species or by Aristida meridionalis (ari). Two different experimental approaches were used. Upper panel: addition of soil inoculum with biota in phase 1, followed by sterilization or no sterilization in phase 2. Lower panel: addition of soil inoculum with (+) or without (-) biota in phase 1, followed by no sterilization in phase 2.

A first analysis approach is to subject biomass data to anova, modelling the effects of test plant species, conditioning plant species, soil treatment and all possible interactions on plant biomass. As a general rule, it is recommended to log-transform biomass data prior to analysis. One advantage of log-transformation, which transforms multiplicative models into additive models, is that the soil treatment effect on plant species of different size can be fairly compared. For example, a growth reduction in ‘own’ compared to ‘foreign’ soil of 25% will result in similar soil treatment effects irrespective of absolute plant size after log-transformation (log(0.75Y) – log(Y) = log(0.75), irrespective of the value of Y), which is not true for non-transformed data. In this particular study, non-independence between data points is introduced because soil from individual phase-1 plants was used for multiple phase-2 pots (Fig. 1). This can be accounted for by including phase-1 pots as a random factor in the statistical model. In the analysis of the biomass data, the treatment effect (‘Sterilization’ or ‘Addition of soil inoculum’) captures the main soil feedback effect while ‘Treatment × Test plant’ and ‘Treatment × Conditioning plant’ test whether soil treatment effects differ between test plant species and conditioning plant species, respectively (Table 2). The anova approach provides much flexibility, and through custom tests (‘contrasts’) the model residual error can be used to test the significance of soil treatment effects in individual species. A second analysis approach is to first calculate feedback values and subsequently subject those to anova, testing the effects of ‘Test plant’, ‘Conditioning plant’ and their interaction on the calculated feedback values. An advantage of this approach is that the feedback ratio (and not biomass per se) is the most relevant and informative measure from the biologist’s perspective. In this experiment, natural pairs existed of non-sterilized and sterilized samples (‘sterilization approach’), because samples of the same replicate pair shared conditioned soil from the same individual phase-1 pot. In contrast, in the ‘addition of biota approach’ no such natural pairs existed and samples of ‘soil inoculum with biota’ and ‘soil inoculum without biota’ were therefore randomly paired to calculate feedback ratios. Thus, each pair yielded one feedback value, the resulting feedback values were independent and there was no need to include phase-1 pots in the model. Unfortunately, feedback values that were calculated as the biomass ratio in ‘non-sterilized : sterilized soil’ and ‘soil inoculum with biota : soil inoculum without biota’ treatments turned out to be difficult to analyse by anova because of severe violations of data assumptions for anova, which could not be solved by simple transformations (no homogeneity of variances and no normally distributed residuals; data not shown). We therefore analysed feedback scores following method FB4 = (O – F)/max(O,F) (see Table 1). In this calculation, soil treatments refer to ‘soil inoculum with biota’ vs. ‘soil inoculum without biota’, or to ‘non-sterilized’ vs. ‘sterilized’ phase-1 soil. The resulting feedback data did meet standard assumptions for anova analysis (data not shown). A drawback of this approach is that biological interpretation may be less straightforward because the soil treatment effect is not expressed as a ratio of the same variable for all data points. However, if a clear soil effect does exist (e.g. ‘sterilized’ has consistently larger biomass than ‘non-sterilized’), then this method yields the same results as fixed ratios. In the analysis of feedback values there is no overall test for a feedback main effect; ‘Test plant’ and ‘Conditioning plant’ test whether feedback differs between soil-conditioning species and test species, respectively. Feedback of individual species was tested for deviation from the no-feedback expectation using multiple t-tests. For this purpose, we followed the often-used approach of sub-setting the data by species and performing t-tests on these reduced single-species data sets (see for instance Klironomos 2002; Van der Putten et al. 2007; Van Grunsven et al. 2007). Note that it is also possible, but not typically used in soil feedback studies, to perform t-tests for individual species using an error term in the calculation of the t-statistic that is estimated across all species simultaneously and not just based on the sample variation in the focus species that is tested against the null expectation value (comparable to contrast testing in anova as described above).

Table 2.   Case study: test results for fixed effects on biomass or on calculated feedback ratios, for the ‘sterilization’ and the ‘biota addition’ approaches. Effect of test plant (T; Cenchrus biflorus, Eragrostis lehmanniana), conditioning plant (C; C. biflorus, E. lehmanniana, Aristida meridionalis) and soil treatment (S). Soil treatment is sterilization in phase 2 (yes, no) for the ‘sterilization’ approach, and addition of soil inoculum with or without biota in phase 1 (+, -) for the ‘addition of soil inoculum approach. See Fig. 1 for details of the experimental design
Effectd.f.numSterilizationAddition of soil inoculum
Biomass*Feedback values†Biomass*Feedback values†
FpFpFpFp
  1. *Biomass data were ln-transformed prior to analysis.

  2. †Feedback ratios were calculated for each replicate pair as (Ai – Bi)/maximum(Ai, Bi); where A and B are: biomass obtained in non-sterilized and sterilized soil, respectively (soil sterilization approach); or biomass obtained in soil treatments with ‘inoculum with biota’ and ‘inoculum without biota’, respectively (soil inoculum approach).

Test plant (T)15.30.0276.10.02233.2<0.00113.00.001
Conditioning plant (C)21.20.314.00.0320.60.587.30.003
T × C20.20.820.50.612.70.090.30.76
Soil treatment (S)117.4<0.001  0.110.74  
S × T14.00.053  28.7<0.001  
S × C25.10.012  5.20.013  
S × T × C20.30.76  0.180.84  

Results

Feedback calculations

Most of the presented calculations based on feedback ratios, except the logarithmic transformation, resulted in comparable negative feedback values with a lower limit of –1. However, the different calculation methods resulted in a wide variety of positive feedback values with different upper limits (Fig. 3). Calculation methods that use an average value in the denominator and/or numerator result in potential feedback values that depend on the number of treatments. For example, the calculation of ((O– grand average)/grand average) results in a potential feedback value range from -1 up to [the number of treatments –1] (Table 1 and Fig. 3). Note that only the logarithmic transformation of the ratio between ‘own’ and ‘foreign’ biomass [ln(O/F)] and the [(– F)/maximum of (O;F)] ratio provide feedback scores that are symmetrical around the no-effect point (where ‘own’ and ‘foreign’ are equal). For instance, a twofold biomass increase and a twofold biomass decrease after logarithmic transformation or after division by the maximum value lead to the same numerical feedback score of opposite sign (Markham & Chanway 1996; Petermann et al. 2008).

Figure 3.

 Potential value range of several feedback calculation methods. In the calculation of the feedback value, the biomass in foreign soil(s) (F) was set at one, whereas the biomass in own soil (O) was allowed to vary between 1000 and 0.001. F(4) is the grand average of the biomass in own and four foreign soils; F(10) is the grand average of the biomass in own and 10 foreign soils; max(O;F) is the maximum value of O and F. When O:F≤ 1, feedback values calculated with (O - F)/F and [(O - F)/max(O;F)] are identical. Note the logarithmic scale on the x-axis.

Simulation

The power of the anova on biomass data was slightly higher than the power of the paired t-test, FB2 and FB4 (analysis of calculated feedback values; Table 1 and Fig. 4). Note that the paired t-test, FB2 and FB4 required arbitrary pairing of ‘own’ and ‘foreign’ plants, because no pair or replica effects were simulated and thus no natural pairs existed. The small difference is entirely due to the difference in degrees of freedom of the t-test (9 for paired, 18 for independent samples) and is an artefact due to the independence of the data, where dependence is assumed in the paired-samples t-test. In real experiments such natural pairs might exist, for instance due to blocking effects in the experimental design or because conditioned soil from a single replicate pot is used to test growth of one ‘own’ and one ‘foreign’ plant. In such cases, paired t-tests or a one-sample t-test for FB2 is appropriate. FB1 seemed to have higher power, while FB3 had lower power than anova (Fig. 4). The differences were large: a soil effect size of 1 (unit SD) was significant only in about 25% of all experiments when analysed via FB3, in about 50% of all experiments when analysed via anova or paired t-test, FB2 and FB4, whereas 75% of all experiments resulted in a significant effect when using FB1. However, further inspection of the P-values for the true null cases (effect size = 0) indicated inappropriate behaviour of the FB1 and FB3 tests, showing that these approaches should not be used. Under proper significance testing conditions these true null cases should show the following characteristics: about 5% of the tests should give a significant result at a significance threshold of α = 0.05 (about 50 out of 1000 iterations), and their P-values should follow a uniform distribution. This was true for anova, paired t-test, FB2 and FB4, but not for FB1 and FB3 (Fig. 5). In contrast, the histogram for FB1 showed a strong bias towards low P-values, even though in reality there was no effect of soil treatment in the simulated experiment, and this method thus leads to a serious inflation of type I errors. The method presumably magnifies random small differences between ‘own’ and ‘foreign’ plants by artificially reducing the variance of the calculated feedback scores, relative to the variance of the original biomass scores. The histogram of FB3 showed a clear bias against low P values (<2% of all P-values below 0.05), which suggests unnecessary low power to detect effects in general.

Figure 4.

 Power of six analysis approaches to detect soil feedback effects. Proportion of the simulated data sets in which a significant (at alpha = 0.05) soil treatment effect was detected at effect sizes of 0, 0.5, 1 and 2 (expressed in units SD). See Table 1 for abbreviations.

Figure 5.

 Distribution of P-values observed when simulated effect size is zero (true null cases, 1000 random data sets), resulting from six approaches to testing plant–soil feedback. See Table 1 for abbreviations.

Case study

The re-analysis of the case study shows how the interpretation of plant–soil feedback effects may differ, depending on the experimental approach. For both C. biflorus and E. lehmanniana, the approaches of adding soil inoculum and applying soil sterilization yielded very different results (Tables 2 and 3). Both the direction of the feedback effects and the conditioning plant species that caused the feedback effects were very different for the two approaches (Fig. 2). For example, in the sterilization approach, soil conditioned by A. meridionalis caused a significant negative feedback effect to C. biflorus, whereas in the addition of soil inoculum approach, A. meridionalis did not cause a significant feedback effect. In contrast, in the addition of soil inoculum approach, C. biflorus caused a significant positive feedback effect to C. biflorus, which was not the case in the soil sterilization approach (Fig. 2 and Table 3). A comparison of biomass in own and foreign soil did not result in a significant feedback effect for either of the test species in either of the experimental setups (data not shown).

Table 3.   Case study: test effects (P-values) of sterilization and addition of soil inoculum for specific experimental groups, using contrasts for original biomass data and t-tests for calculated feedback values. Phase-2 plants are ‘Cen’ = Cenchrus biflorus and ‘Era’ = Eragrostis lehmanniana, phase-1 plants are C. biflorus, E. lehmanniana and ‘Ari’ = Aristida meridionalis. See Fig. 1 and Table 2 for details of experimental approach and analyses
Experimental groupSterilization effectAddition of soil inoculum-effect
Phase-2 plantPhase-1 plantBiomass*FeedbackBiomass*Feedback
  1. *Biomass data were ln-transformed prior to analysis.

CenCen0.910.910.004<0.001
CenAri0.0210.0420.360.66
CenEra0.700.990.0220.14
EraCen0.520.0060.930.69
EraAri<0.0010.0030.0030.039
EraEra0.0170.0290.640.64

In general, the results of using the original biomass data in the analysis or analysing calculated feedback values were very similar. However, some differences are apparent, for instance in the species-level tests for presence of a feedback effect (Table 3). These differences may be attributed to log-transformation in the biomass analysis but not in the feedback analysis, and/or to using an error variance that is estimated across the entire experiment (contrast tests in biomass analysis) or that is estimated for the particular subset of data only (t-tests in feedback analysis).

Discussion

Feedback calculations

Calculation of plant–soil feedback values simplifies the presentation of results and the structure of the statistical analysis. However, plant–soil feedback effects can be quantified by a variety of methods, each having their strengths and weaknesses. Basically, many calculations result in non-symmetry of negative and positive feedback effects (see Markham & Chanway 1996; Armas, Ordiales & Pugnaire 2004) and the exact values can only be interpreted after back-calculation. In addition, different calculation methods result in different ranges of feedback values, which hampers comparisons among studies. Feedback calculations based on average treatment values should be avoided, because they either increase the rate of type I errors or cause a bias against detecting true effects (Fig. 2). When using calculated feedback ratios, the original data still need to be presented as well to show where the actual feedback effect originates from.

Experimental approach

In our re-analysis of the case study originally published by Van der Putten et al. (2007), supplemented with their unpublished data, the significance, as well as the direction of the feedback effects depended on whether the soil was sterilized or a soil inoculum was added. Apparently, these two experimental approaches influence different soil processes in either the first or the second phase of the experiment. The net result depends on the balance between effects caused by symbionts and soil pathogens, as well as on effects caused by increased nutrient availability due to sterilization (Powlsen & Jenkinson 1976; Jakobsen & Andersen 1982). Also in the addition-of-soil-inoculum approach, nutrient release due to soil sterilization will affect the soil community. Moreover, the initial density of soil organisms is smaller than in field soil, so that effects of symbionts and soil pathogens may be less prominent. If effects of symbionts are less density-dependent than are pathogen effects, the addition-of-soil-inoculum approach will produce results tending towards less negative, or more positive, feedback effects than the sterilization approach. In our case study, biomass of plants grown in own soil was not significantly different from biomass of plants grown in foreign soil. This agrees with a comparison of results from different experiments, where soil sterilization resulted in stronger feedback on plants than when plants were grown in own vs. foreign soil (Kulmatiski & Kardol 2008). It could indicate that the soil organisms that caused the feedback effects were not very species-specific, or that they were not present in higher densities in ‘own’ as opposed to ‘foreign’ soil.

We propose that experimental approaches should be chosen carefully depending on the type of ecological question (Table 4). When comparing plant–soil feedback among different plant species, the interest either may be in general soil effects or in plant species-specific effects. General soil effects consider the feedback of a whole set of plant species in a vegetation type, community or successional stage, as opposed to feedback effects of specific plant species, for example of invasive species, on each other. When the interest is in general soil effects, performance in own soil can be compared to that in foreign soil, which may be composed of a mixture of soil of different origins or plant species. The analysis then involves paired-comparisons anova or calculation of the own : foreign ratio of paired replicates, followed by anova. Most methods to calculate plant–soil feedback ratios produce positive and negative values that are disproportional to their original effect sizes. However, division by the maximum value of the two treatments or logarithmic transformation of the own : foreign ratio yields a perfect comparison of the size of positive and negative feedback effects (Markham & Chanway 1996; Petermann et al. 2008; our Fig. 3). When the aim is to unravel plant species-specific effects in plant communities, e.g. to study the role of plant–soil feedback effects in plant community assembly processes, a full factorial setup is required where performance of all plant species is tested in soils conditioned by all plant species. The analysis will then involve an anova followed by calculation of contrasts of own vs. foreign soil.

Table 4.   Summary of strengths and weaknesses of different experimental approaches
Soil treatmentSoil communityAbiotic effectsReplication
Sterilized (S) vs. non-sterilized (NS)Natural in density and compositionIn S soil release of nutrients; this effect can be reduced, but not removed, when nutrients are added to all treatmentsSeparate field sample for each replicate provides insight in spatial variation. Often lack of independence of S and NS; this should be accounted for in the analysis
Addition of soil inoculum with or without biotaDensity depends on development time; composition may differ from naturalDiluted when small amounts of inoculum are added to a (natural) common S background soilSeparate field sample for each replicate provides insight in spatial variation. Often lack of independence of S and NS; this should be accounted for in the analysis
Own vs. foreign soilInterest in abundance and/or presence of soilorganisms with specific interaction with the plantDiluted when small amounts of inoculum are added to a (natural) common S background soilSeparate field sample for each replicate

When the research question also focuses on the kind of organisms that cause the plant–soil feedback effect, a first step would be the addition of soil inoculum or the sterilization approach (Table 4). The feedback effect indicates the type of soil organisms that are predominantly present in the soil. When the feedback effect is positive, the effect can be due to arbuscular or other mycorrhizal fungi, symbiotic or free-living nitrogen fixing microbes, or growth-promoting rhizobacteria. In contrast, when the feedback effect is negative, mainly soil organisms with a pathogenic (bacteria, fungi, viruses), parasitic (root-feeding nematodes) or herbivorous (insect larvae) nature are affecting plant performance. In both cases, the feedback is a net effect of soil organisms with a positive and with a negative influence on the plant.

Plant–soil feedback experiments are a first step towards identifying causal agents. To elucidate the contribution of different groups of soil biota to plant performance, sterilized soil can be inoculated with different fractions of soil biota, such as a microbial filtrate, specific microbes that have been cultured on growth media, mycorrhiza, nematodes or microarthropods (e.g. De Rooij-van der Goes 1995; Westover & Bever 2001; Klironomos 2002; Agrawal et al. 2005; Brinkman, Duyts & Van der Putten 2005; Van der Putten et al. 2007). It is, however, difficult to attribute certain soil organisms to different filtrate sizes and the contents of the filtrates should be validated. For example, microbial filtrates may also contain eggs of nematodes, which themselves belong to a larger fraction size (Wurst et al. 2008). Plant–soil feedback effects can be caused by single species (e.g. Packer & Clay 2000) or by a mixture of species of soil organisms (e.g. De Rooij-van der Goes 1995; Van der Stoel, Van der Putten & Duyts 2002). When a specific species causes (most of) the feedback effect, it needs to be isolated, cultured and re-introduced into the soil, and finally isolated again, applying Koch’s postulates (e.g. Packer & Clay 2000). However, when the feedback effect is not clearly caused by a specific soil organism, it is much more difficult to find out which groups of organisms account for the effect, as there are many testable combinations. These numbers of combinations may be narrowed down by field surveys preceding inoculation trials (De Rooij-van der Goes 1995). Most likely, in natural ecosystems the chance of finding single major pathogens is much smaller than in agriculture or horticulture, where plants have been artificially bred for high yields and grown in monocultures.

More complex designs

Our simulations described a simplified experiment and highlighted that basic analysis choices can have implications for the statistical detection of plant–soil feedback effects. Experiments become complicated when multiple species or other experimental treatments are included. Such experiments are frequently analysed by calculating feedback values and analysing differences in feedback among species with anova (Van der Putten, Van Dijk & Peters 1993; Bezemer et al. 2006). The deviation from the neutral feedback expectation is then tested separately per species using t-tests (e.g. Klironomos 2002; Kardol et al. 2007; Van Grunsven et al. 2007). Alternatively, original biomass data can be analysed using anova models that incorporate the entire experimental design and that are supplemented with specific custom hypothesis tests (contrasts) to capture soil feedback effects for individual species (Bever 1994). Compared to the analysis of log-transformed biomass data, the analysis of feedback ratios has the advantage that these ratios are usually the basic measure of interest. However, a typical disadvantage of ratio testing is that they tend to be more difficult to analyse than original biomass data, due to violation of data assumption for anova (as observed in our case study). Also, ratios can mask interesting patterns that are visible only in the original numerator and denominator variables. However, analysis of feedback ratios is an appropriate alternative when natural pairs exist of ‘own’ and ‘foreign’ plants to feed into the feedback calculation.

Conclusions

Plant–soil feedback interactions get increasing attention. We point at several methodological aspects that need to be carefully considered when planning and analysing results of plant–soil feedback studies. Meta-analysis has already shown that soil sterilization yields greater plant–soil feedback effects than comparing performance in ‘own’ to ‘foreign’ soil (Kulmatiski & Kardol 2008). Our re-analysis of a case study confirms that this difference in effect strength also emerges within a study. In addition, we showed that the approaches of adding soil inoculum and applying soil sterilization yielded very different feedback results. These results can be due to confounding effects of nutrient release after soil sterilization, or to selective effects of nutrient supply on plant–soil biota interactions. However, these effects may also be due to changes in soil community composition following inoculum addition. Therefore, we conclude that a more balanced view on plant–soil feedback effects requires a variety of approaches that need to be carried out in parallel. Specific methods can be applied under specific conditions. For example, comparing individual plant–soil feedback effects in sterilized vs. non-sterilized soil can be done best in low-fertile systems, whereas in richer soils adding sterile vs. non-sterile soil inoculum will be more preferable. Comparing own vs. foreign soils is a method well applicable when unravelling effects of plant–soil feedback in plant community dynamics involving multiple plant species.

The interpretation and comparison of calculated feedback values among studies should be done cautiously. First, many ratios are not symmetric for negative and positive plant–soil feedback, so that in most calculation methods a positive value of for example +0.5 indicates a lower feedback strength than a negative value of –0.5. Second, some calculation methods yield significant effects more or less easily than others. Especially feedback calculations based on average treatment values should be avoided, because they either lead to many false discoveries (when the average of the control treatment is used) or are biased against detecting significant effects (when the grand average is used).

This does not mean that all previous plant–soil feedback studies should be considered unreliable. The previous studies have shown their value in demonstrating that plant–soil feedback effects may have strong impacts on plant community processes and that most of these effects are negative, rather than positive (Bever 2003). With the present analysis, we hope to create awareness that some experimental approaches are more suited than others to address specific ecological questions. For some questions, a variety of approaches may be required to get a realistic impression of the direction and strength of the plant–soil feedback effects. Further studies are required to get a more complete picture of how the direction and strength of plant–soil feedback effects may relate to plant traits and ecosystem characteristics. It will remain a challenge for quite a while to relate plant–soil feedback effects to agents that may cause these effects.

Acknowledgements

We thank Arjen Biere for useful discussions during the preparation of the manuscript and Frank Menting for his contribution to Fig. 1. We greatly appreciate the comments of two anonymous reviewers on an earlier version of the manuscript. W.v.d.P. was financially supported by an ALW-VICI grant and K.J.F.V. by an ALW-VENI grant. Publication 4781 Netherlands Institute of Ecology (NIOO-KNAW).

Ancillary