Global climate change is one of the most pervasive human transformations of the Earth, and it represents one of the greatest threats to current ecological function and human socio-economic interests (Rosenzweig et al., 2007). This relatively recent realisation led to a rapid increase in the production of papers concerned with marine climate change from an average of <15 per year in the preceding two decades, to >80 per year during the first decade of the new millennium (Hoegh-Guldberg & Bruno, 2010). Studies aiming to identify climate change as a driver of marine ecological change, either through observational analyses of time-series (Brown et al., 2011) or experimental manipulations of climate change factors (this study), have increased similarly, although only accounting for 18% and 13% of all marine climate change papers, respectively. It is clear that MCCEs were published in higher impact journals compared with marine studies in general. The high incidence of identifiable limitations in experimental design suggests that this publication pattern has been driven by the topicality of the subject and broad scientific interest in establishing mechanistic relationships between climate change variables and biotic responses, rather than a ‘higher-than-usual’ quality of the experiments. A prominent feature for all MCCE classifications was a strong dominance of a few categories (Fig. 2a–c), demonstrating that the current mechanistic evidence for possible links between climate change and ecological changes is highly biased towards a limited subset of environmental and ecological conditions.
Physical and ecological scope
Acidification experiments (>60% of MCCEs) were more common than temperature experiments (~40%), and this is interesting because the evidence for physical change and ensuing biological consequences is much stronger for ocean warming than for ocean acidification (Harley et al., 2006; Poloczanska et al., 2007; Hawkins et al., 2008; Rosenzweig et al., 2008; Wootton et al., 2008; Wernberg et al., 2011a). The ‘over-representation’ of acidification experiments probably reflects the fact that ocean acidification is a novel, climate change-specific stressor, whereas temperature is a well-known driver of species distributions and interactions (Clarke & Gaston, 2006; Tittensor et al., 2010), where there is a large body of mechanistic knowledge not explicitly linked to climate change. Acidification clearly affects marine organisms (e.g. Hall-Spencer et al., 2008; Wootton et al., 2008) and the scarcity of documentation for ongoing ocean acidification and associated biological impacts probably reflects a simple lack of data (Richardson & Poloczanska, 2008; Wernberg et al., 2011a). Nevertheless, in contrast to the pervasive impacts of warming, impacts of acidification are idiosyncratic: virtually, all temperature experiments showed significant effects of warming, whereas many studies of acidification showed only subtle effects and 13 (18% of all ocean acidification experiments) found no effects at all (Table S1).
Single-factor experiments accounted for as many as 65% of all MCCEs. Yet, extensive meta-analyses of both marine (Crain et al., 2008) and non-marine (Darling & Côté, 2008) experiments have shown that concurrent impacts of multiple stressors are predominantly non-additive and therefore cannot be understood or predicted in isolation from one another. In addition to the multiple physical manifestations of climate change, humans also have substantial non-climate impacts on their natural environment: introduced species, eutrophication, over-fishing and sedimentation caused by dredging, land run-off and marine infrastructure are also causing dramatic impacts and global transformation (Jackson et al., 2001; Lotze et al., 2006; Airoldi & Beck, 2007). Clearly, impacts of climate change are not isolated from these diverse stressors or their local environmental and biological context (Wernberg et al., 2011a). The 38 MCCEs testing effects of more than one factor confirm the importance of multiple concurrent stressors as at least 82% found interactive effects.
Almost 60% of all MCCEs were studies of a single species. Single-species studies cannot consider ecological effects through changes to species interactions. Yet, all single-species populations are embedded in communities of multiple interacting species where ecological effects mediated by shifts in species interactions might be as strong as, or stronger than, autecological effects driven by species tolerances (Hawkins et al., 2008; Kordas et al., 2011).
The overall implication of the apparent biases in physical and ecological scope of MCCEs is that we have a poor understanding of how physical and biological changes combine in their direct and indirect effects. Consequently, there is a great need for studies targeting interactions between multiple stressors and multiple species, particularly under climate change specific scenarios. To address this knowledge-gap, future experiments must include more than one test-factor and change the focus from assessing the performance of individuals (metabolism, growth, reproduction etc.) to assessing effects on species interactions such as the strength of competition, predation or herbivory. This will necessitate multispecies experiments combining currently co-occurring species as well as species that might only co-occur in the future.
Organisms and environments
Most studies tested effects on temperate species, and this probably reflects that most marine laboratories are located on temperate shores in Europe, North America and Australasia. This geographical bias is unfortunate because tropical and polar organisms are likely to be under severe threat from climate change, having adapted to climatically extreme, but relatively stable environments, where the difference between optimal and lethal conditions can be small (e.g. Peck et al., 2004; Hoegh-Guldberg et al., 2007). Compounding these biological limitations are physical constraints on dispersal for polar organisms which, in contrast to tropical and temperate organisms that can and do shift polewards (e.g. Parmesan & Yohe, 2003; Precht & Aronson, 2004; Greenstein & Pandolfi, 2008; Wernberg et al., 2011b), have no-where to go to escape warming waters.
Benthic invertebrates such as sea urchins, mussels and crabs were by far the most studied groups of organisms, presumably because of their wide distribution, high diversity and ease of collection and experimentation. These animals have a long history as experimental models, and they continue to form the basis of our understanding of ecological response to marine climate change. However, this bias is a concern, as planktonic organisms play a crucial role in global biochemical cycles, including accounting for about half of the biosphere's primary production and exhibit high sensitivity to climate variability (Boyd & Doney, 2002; Doney, 2006). Moreover, marine macrophytes are some of the most productive primary producers in the world (Mann, 1973), and they are particularly important because of their critical contribution to the ecological function of many ecosystems through their diverse roles as the primary habitat providers, food sources and ecosystem engineers (e.g. Dayton, 1985; Wernberg et al., 2005; Thomsen et al., 2010). It is particularly important to understand how habitat-providing and -modifying species will be affected by climate change because habitat-mediated environmental amelioration is thought to become increasingly important to maintaining ecological function in the future (Halpern et al., 2007).
It was apparent that for most organisms, only a single life stage had been considered – typically larval or juvenile stages for large organisms (e.g. fishes) and large or adult stages for organisms with very small propagules (e.g. seaweeds). However, juvenile and adult stages often have different tolerances to environmental stress (e.g. Gilman, 2006; Fredersdorf et al., 2009) and without knowing which ontogentic stage is most vulnerable to a particular stressor, there is a risk of substantially under-estimating the potential ecological consequences (Russell et al., 2012). Although logistically challenging, an increasing experimental effort is required for both small and large organisms (plankton, fishes, etc.) and on organisms from marginal or particularly vulnerable environments, to ensure a balanced understanding of how marine organisms might respond to climate change. Future experiments should focus on organisms that condition the existence of associated communities, and in particular on how climate change might influence the functions they provide. A focus on ecological function is particularly important where experiments and access to experimental organisms are limiting (e.g. polar regions, deep sea). Similarly, future experiments should explicitly contrast the vulnerability of different life stages to identify bottlenecks for population persistence under future environmental and biological conditions.
The physical forcing of climate change operates at regional scales, and interacts with processes at multiple spatial and temporal scales to impact local biota (Helmuth et al., 2006; Wernberg et al., 2011a). Global-change variables are therefore difficult to manipulate, particularly in situ. In the marine environment, the biophysical properties of water and the general inaccessibility of the underwater environment exacerbate these logistic constraints on experimentation. Still, experimental venue, selection of treatment levels, assignment to experimental units and the distribution of replicates among treatments (i.e. experimental design) have fundamental implications for the inferences that can be drawn from any experiment (Hurlbert, 1984; Underwood, 1997). In particular, it is critical that treatment levels capture the range and magnitude of variation that is relevant to the context, and that experimental units and replicates are independent, and integrate an appropriate level of ‘random’ non-treatment variation.
MCCEs had a high incidence of pseudoreplication (>40%) and this is perhaps surprising, given the strong traditions for experiments in marine ecology (Underwood, 1997) and the time since the issue was brought to the attention of ecologists (Hurlbert, 1984). However, the problem is not an isolated phenomenon of MCCEs as previous reviews of experiments in aquatic ecology have found up to 51% of experiments to be affected (Hurlbert, 1984; Hurlbert & White, 1993). Pseudoreplication is a problem because it limits the inference space and the ability to extrapolate the results. Technically, pseudoreplication covers a broad range of issues which influence the power structure of statistical tests by inflating the degrees of freedom in favour of the proposed hypothesis, thereby increasing the risk of type 1 error (Hurlbert, 1984; Hurlbert & White, 1993). Commonly this occurs by subsampling the same treatment thus failing to incorporate an appropriate amount of ‘random’ background variation (simple pseudoreplication) or by pooling treatments thus conflating ‘random’ background and ‘treatment’ variation (sacrificial pseudoreplication) (Hurlbert & White, 1993). Conceptually, pseudoreplication is akin to failing to incorporate autocorrelation into space- and time-series analyses, and this has been identified as one of the most prevalent issues with the analysis of observational evidence for impacts of climate change (Brown et al., 2011). It can perhaps be argued from a precautionary principle that pseudoreplication is not a serious problem because it implies that conclusions that climate change will have no effect are conservative. Nevertheless, it obscures an objective assessment of impact and may contribute to unnecessary spending on mitigation and adaptation.
Several MCCEs (6%) tested effects of climatic variables manipulated far beyond projections for 2100. While testing extreme values of relevant factors can be useful to delimit their impact-domain and identify worst case scenarios, or identify the impacts of discrete events such as heat waves or cyclones (reviewed in Jentsch et al., 2007), the outcomes of such studies are arguably of limited ecological relevance in relation to understanding impacts of overall climate change in the foreseeable future.
The overwhelming majority of MCCEs were ex situ studies (~90%), typically conducted in aquaria and small mesocosms. That these experiments have provided valuable information is beyond question (Benton et al., 2007). However, what makes these experiments useful and informative is also their Achilles heel: the confined and highly controlled nature of the physico-chemical and biological environment in ex situ experiments reduces realism and limits the inference space to a highly artificial world (Carpenter, 1996). Species, populations and individuals in nature experience a constantly changing environment, where physio-chemical and biological influences fluctuate both predictably (i.e. over the cycle of a day or a year) and randomly. Moreover, biological communities are often connected across a range of spatial and temporal scales, which extend beyond the confines of an aquarium in a laboratory (e.g. Caley et al., 1996; Borthagaray et al., 2009). Consequently, ecological outcomes of selection and species interactions have been shown to differ fundamentally between highly controlled experiments and those with a greater similarity to natural conditions (Skelly, 2002; Van Doorslaer et al., 2010).
The lack of field-based MCCEs is a serious limitation because it exposes the artificial nature of the current experimental understanding. This shortcoming has undoubtedly been driven by difficulties with controlled manipulations of climate change factors in situ. However, other approaches such as ‘opportunistic and natural experiments’ (e.g. Schiel et al., 2004), ‘comparative experiments’ (e.g. Wernberg et al., 2010) and ‘mensurative experiments’ (e.g. Hall-Spencer et al., 2008) have long been advocated (Underwood, 1996; Menge et al., 2002; Dunne et al., 2004). Natural and opportunistic experiments can rarely be planned and are therefore not an efficient tool for systematic use in climate change studies. In contrast, comparative experiments, where identical manipulative experiments are carried out in different places characterised by different climates, are particularly useful (Menge et al., 2002; Dunne et al., 2004), especially to test how climate might modulate the impacts of additional factors (e.g. Wernberg et al., 2010). Comparative experiments can be criticised because they, strictly speaking, do not manipulate the climate factor and because it is impossible entirely to avoid confounding climate and non-climate factors. However, in carefully planned and cautiously interpreted experiments, this is no greater limitation to inference and extrapolation than the highly artificial conditions of enclosures and aquaria. It is important to recognise that these approaches can provide unique insights that are complementary in scope and scale to the prevalent ex-situ approaches, and to appreciate that even experiments with limitations on manipulation and replication can provide tests of hypotheses (Hurlbert, 2004). Future small-scale ex-situ experiments should focus on increasing their inference space through appropriate replication (i.e. avoid pseudoreplication), both in terms of the physical design of experimental units and in terms of increasing the power of subsequent analyses (see, for example, the detailed instructions in Riebesell et al.,2010). A more pragmatic approach is warranted for field experiments where it may be necessary to accept some level of pseudoreplication and confounding to gain the advantage of a substantially less artificial experimental venue (Hargrove & Pickering, 1992; Oksanen, 2001). In some cases, regression-based methods can alleviate difficulties of replication, and they should be used more widely in MCCEs particularly because they have the added benefit of a better parameterisation of the cause–effect relationship, which will facilitate projections into the future, with no loss of power (Cottingham et al., 2005) or ability to weight the relative importance of climate and non-climate drivers (see Brown et al., 2011 for a discussion relating to observational studies). As artificiality and limitations are inevitable, and probably more pronounced in MCCEs than in other experiments, a weighted evidence approach, where conclusions are driven by multiple pieces of independent evidence pointing in the same direction (Cleland, 2001), is a more productive approach towards understanding biological impacts of marine climate change than a quest for perfectly executed decisive experiments (sensu Platt, 1964).