Körner (Körner, 2003) recently provided a critique of meta-analysis in ecological CO2 research, specifically focussing on our work (Jablonski et al., 2002). This meta-analysis was the first attempt in many years to provide summary statistics from a large body of literature covering both domesticated (crop) and nondomesticated (wild) plant species’ reproductive responses to elevated atmospheric CO2. Our goal was to calculate the mean CO2 treatment effect on several measures of reproductive quantity and quality and to explore possible causes for variation in CO2 effect size across studies. Another result of our work was to document gaps in the literature as related to specific research areas or study species. However, meta-analysis is not an appropriate tool for establishing firm causal relationships among variables – rather, it provides a statistically defensible means for evaluating the degree to which published data obtained from independent sources support specific hypotheses (Gurevitch et al., 2001). As the scientific domain in which meta-analytic tools are applied continues to expand, the statistical basis for these new applications must keep pace (Lajeunesse & Forbes, 2003) and be understood both by meta-analysts and their critics.
One of the surprising results in Jablonski et al. (2002) was the relative insensitivity of CO2 effects on reproduction to interacting environmental stress factors (see Table 1 in Jablonski et al. (2002)). The most common stress factors present in our dataset and hence examined closely were high temperature, low nutrients, high ozone, and drought. Of the eight response variables we reported on, only one, fruit number, showed a significant effect attributable to low nutrient stress, and this result was based on only four studies, all conducted with crop plants. More typical was the CO2 effect on total seed number, in which plants grown with some stress factor in combination with elevated CO2 were no different in their reproductive output than unstressed plants grown at high CO2. Whether this insensitivity was a result of real underlying biological causes or was an artifact either of the nature of the published data, or our sampling of it, was not resolvable. However, given this result and that taxonomic and functional group identity had highly significant effects on the magnitude of the CO2 response, we had no statistical justification for first stratifying the data based on plant nutritional status. Put another way, our data did not support the hypothesis that interacting stress factors (many of which could affect plant resource status) were important drivers of plant reproductive responses to elevated CO2. The same was true for annual vs perennial life history (sink strength?) or duration of CO2 exposure (plant age?).
We share Körner's assessment of the database on plant reproduction under elevated CO2 as being heavily biased towards crop plants or to annual wild plants grown like crops (i.e. not in their native habitat). Relevant experiments with perennial species (crop or wild) remain both very rare and highly idiosyncratic, rendering them of questionable value in a meta-analysis. The first condition could be corrected with greater levels of funding for long-term elevated CO2 experiments in natural ecosystems. The second condition unfortunately is a fact of life for those integrating ecological data and is likely to compromise many quantitative research syntheses. Meta-analysts desire replication at the level of the publication. The more similar two independent experiments are to each other in design and execution, the better. Meta-analysis has proven invaluable in the biomedical sciences where many-fold replication of experimental findings are required (and supported) before clinical decisions based on treatment effect size can be made. Ecologists are rewarded for proposing experiments that are different from any previously done, and novelty of results is a prime factor for manuscript acceptance in front-line journals (including this one). This framework for conducting ecological science may need to be rethought as ecologists become increasingly involved in providing policy-relevant data, such as quantitative assessments of the biological consequences of global change, to problems of substantial societal concern.