- Top of page
- Supporting Information
Human alteration of natural flow regimes (flow magnitude, frequency, duration, timing, rate of change; Poff et al., 1997) is ubiquitous, and research on ecological effects of flow regime alteration is being published at an increasing rate globally (Stewardson & Webb, 2010). However, the observational and opportunistic nature of much of this research (Webb, Stewardson & Koster, 2010) makes it inferentially weak compared to manipulative experiments (Johnson, 2002). Thus, while general principles of flow alteration are well accepted, there has been little success in deriving specific predictions about how degraded biota will respond to flow restoration (Souchon et al., 2008). Against this backdrop, Poff & Zimmerman (2010) synthesised the findings of 165 published studies, attempting to derive quantitative flow alteration–response relationships. They employed review methods commonly used in ecology, but also extended these by attempting both qualitative and quantitative systematic syntheses of the literature. However, their approach was informal, in that they did not employ a previously published systematic review method (e.g. CEBC, 2010). There were also no formal tests of the underlying hypotheses. They specifically noted the difficulty of synthesising a diverse set of studies that differed in experimental designs, modes of reporting, causes of flow alteration and species studied. Overall, their results supported those of earlier, less comprehensive, narrative literature reviews (Poff et al., 1997; Bunn & Arthington, 2002; Lloyd et al., 2003), but did not extend these earlier findings with quantitative results. In this study, we seek to extend the results of Poff & Zimmerman (2010) by applying a formal systematic review method.
Systematic reviews concisely summarise and synthesise the literature on a research question, providing insights beyond those possible with single studies. In contrast to the narrative reviews more common in ecology and environmental science, systematic reviews explicitly treat the literature as data and conduct analyses to test hypotheses (Khan et al., 2003). Systematic reviews are common in several research fields that must deal with complex, multivariate cause–effect relationships, most notably medical research, where they are a key component of ‘evidence-based medicine’ (Pullin & Knight, 2009). Recent calls to embrace evidence-based methods in environmental management (Sutherland et al., 2004; Pullin, Knight & Watkinson, 2009) argue that systematic reviews should be used more widely and have led to the establishment of a foundation to promote their use (Pullin & Knight, 2009). Systematic reviews that supply environmental managers with the synthesised findings of research in an easy to understand form may be able to improve the input of science into policy development (Skinner et al., 2012) and thus help managers to fulfil legislative requirements to use ‘best available science’ in policy development (Ryder et al., 2010). However, they have yet to be widely adopted in environmental management (Pullin & Stewart, 2006).
The inferential strength of individual studies in environmental science is limited by an inability to randomise treatments, confounding environmental variables, and little or no replication. Systematic reviews of such studies need to take these limitations into account. Such problems of weak inference are also faced in epidemiological research. In response, epidemiologists developed ‘causal criteria analysis’, a formal method for combining multiple individually weak pieces of evidence to reach strong conclusions about cause–effect relationships (Hill, 1965; Susser, 1991; Tugwell & Haynes, 2006). The recently published Eco Evidence method for systematic review (Norris et al., 2012) is modelled on causal criteria analysis and uses the literature, an underexploited source of evidence, for assessing questions of causality. The method provides a rule set that assists reviewers to interpret the results of individual studies, weighting them by the strength of their experimental design. This promotes consistency and repeatability when large numbers of studies are being reviewed. Evidence from multiple studies is then synthesised using a standardised framework to test individual cause–effect hypotheses and overall research questions. Hypotheses may be retained or may be falsified. The method is supported by freely available software (Webb et al., 2011, 2012a), which automates the synthesis of evidence across studies, banks the evidence in a reusable online database and provides a standard report to maximise transparency of the process. To date, Eco Evidence has been successfully used in a number of topic-specific systematic reviews of river, wetland and floodplain environments (Harrison, 2010; Greet, Webb & Cousens, 2011; Grove et al., 2012; Webb, Wallis & Stewardson, 2012b). Here, we use the Eco Evidence method and software to synthesise literature on ecological responses to human-altered river flow regimes, demonstrating its ability to synthesise the literature on a management issue of global significance (Dudgeon et al., 2006; Arthington et al., 2010). More specifically, we compare the Eco Evidence results to those of Poff & Zimmerman (2010; hereafter PZ2010) demonstrating the particular benefits and promise of this standardised approach compared to a more traditional literature review.
We used the Eco Evidence method for weighting and combining evidence to reanalyse the same 165 studies as PZ2010. We conducted four analyses that tested whether different features of Eco Evidence led to demonstrably stronger conclusions than the original review. While supporting the general findings of PZ2010, our analyses reached more definitive conclusions about the ecological effects of flow alteration. We were able to test hypotheses at scales directly relevant to management (i.e. effects of directional changes in flow components on specific taxa) and to identify where evidence was consistent, conflicting, or insufficient to reach a conclusion. We use this reanalysis not only as a synthesis of current knowledge in freshwater ecology, but as a case study to demonstrate the utility of the Eco Evidence approach to inform a range of complex environmental management issues.