Weed risk assessment: a way forward or a waste of time?



1. Many alien weeds pose significant environmental and/or economic threats across the globe, and methods to assess the potential risk of species introductions are key components in the management of plant invasions. Three broad approaches have been adopted in weed risk assessment: quantitative statistical models, semi-quantitative scoring and qualitative expert assessment. Yet, the effectiveness of these different approaches is rarely evaluated. By bringing together perspectives drawn from statistics, complexity theory, bioeconomics and cognitive psychology, this review presents the first interdisciplinary appraisal of whether weed risk assessment is a valuable tool in the management of plant invasions.

2. Problems in obtaining an objective measure of the hazards posed by weeds, challenges of predicting complex hierarchical and nonlinear systems, difficulties in quantifying uncertainty and variability, as well as cognitive biases in expert judgement, all limit the utility of current risk assessment approaches. The accuracy of weed risk assessment protocols is usually insufficient, given inherent low base-rates even when the costs and benefits of decisions are taken into account, and implies that the predictive value of weed risk assessment is questionable.

3. Current practices could be improved to address consistent hazard identification, encompass a hierarchy of spatio-temporal scales, incorporate uncertainty, generate realistic base-rates, and train risk assessors to limit cognitive biases. However, such refinements may still fail to predict weed risks any better than a knowledge of prior invasion history and quality of climate match.

4. Alternative approaches include scenario planning that seeks qualitative inputs regarding hypothetical events to facilitate long-range planning using multiple alternatives each explicit in their treatment of uncertainty. This represents a change from prevention towards adaptive management where the difficulty in prediction is acknowledged and investment targets early detection, mitigation and management.

5.Synthesis and applications. Scenario planning may be particularly suitable for weeds as they can be rapidly surveyed and have sufficiently long lag phases between naturalisation and invasion that early detection is often feasible. If integrated with assessments of ecosystem vulnerability to invasion and interventions to improve ecosystem resilience, it would deliver a robust post-border approach to invasive plant management. This approach would address threats from new introductions as well as ‘sleeper weeds’ already present in a region.

Weed risk assessment: from AWRA to Z-score

Introduced plants may account for as much as 40% of the economic losses to agriculture (Pimentel et al. 2001) as well as impose a major financial burden on the resources available to manage natural areas (Williams & Timmins 2002; Williamson 2002; Vilàet al. 2010; Williams et al. 2010; Oreska & Aldridge 2011). In the light of these impacts, there is an increasing need to develop robust tools to identify which plant introductions are likely to cause harm (Groves, Panetta & Virtue 2001). The last 20 years has seen a burgeoning of approaches that use species traits, climate relationships and/or introduction pathways to discriminate between alien plants that pose a high or low risk to the recipient region (Heikkilä 2011; Roberts et al. 2011). These tools have been applied in pre-border screening of proposed new plant introductions as well as in post-border assessments of existing naturalised species (e.g. taxa having established self-maintaining populations). However, there has been no evaluation as to whether the ability to predict the risks of introducing harmful alien plants (i.e. weeds) has reliably improved as a result of the efforts to develop weed risk assessment tools.

Three broad approaches have been adopted in both pre- and post-border weed risk assessment: quantitative statistical models, semi-quantitative scoring and qualitative expert assessment. Quantitative statistical models apply linear models, discriminant functions or decision trees across a large number of species to assess the extent to which one or more life-history traits account for observed variation in one or more measures of invasiveness (Pyšek & Richardson 2007). For example, the Z-score (a discriminant function derived from mean seed mass, mean interval between large seed crops and minimum juvenile period) can be used to distinguish between invasive and non-invasive conifers (Richardson & Rejmánek 2004). Semi-quantitative scoring involves the assessment of individual species against a fixed set of criteria and tallying responses (often binary yes/no answers) to generate an overall count that is compared with a pre-defined threshold for accepting or rejecting a species (Williams & Newfield 2002). The most widely applied approach is the Australian Weed Risk Assessment (AWRA) protocol that uses the answers to 49 questions concerning the species’ biology, biogeography, and behaviour elsewhere, to classify a plant species according to its risk of becoming invasive (Pheloung, Williams & Halloy 1999). Qualitative expert assessment is usually undertaken by decision panels who use their experience to answer broad questions regarding likelihoods of introduction, establishment, impact and management on a qualitative scale (negligible, low, medium and high) and then summarise the overall risk based on these answers (FAO, 2004; Fowler 2004). Guidance is provided in assigning scores and these can be given numeric values so that overall measure of risk can then be calculated by averaging, summing or multiplying, and thus, the approach becomes semi-quantitative (Baker et al. 2008). The comparative performance of these three approaches will not be the same as they differ in the weighting they give to the probabilities of introduction, establishment, dispersal, economic and environmental impact, and management (Hulme 2011a). While each approach has its strong advocates and also suffers from biases, this review aims not to dwell on the specifics of individual schemes (e.g. Roberts et al. 2011) but to address broader concerns of more general relevance to weed risk assessment as a whole. In addition to summarising recent literature relevant to weed risk assessment, data from these publications are used to explore concepts and ideas in more detail. By bringing together perspectives drawn from statistics, complexity theory, bioeconomics and cognitive psychology, the aim is to present an interdisciplinary appraisal of weed risk assessment. The unfortunate conclusion is that, to date, most weed risk assessments have probably been a waste of time, and that a different, adaptive, approach to the management of plant invasion risks is needed.

Hazard definition

Risk assessment aims to assess the likelihood of a particular consequence or hazard. As such, a clear definition of the hazard is needed because the more nebulous the target, the more difficult it will be to predict its likelihood with any accuracy. For alien plants, the terms casual, naturalised, invasive and weed reflect different sequential stages in the progression from initial introduction to a species eliciting economic and environmental impacts (Pyšek, Richardson & Williamson 2004). A further term, noxious weed, is often used in the USA, Australia and New Zealand, for a subset of particularly harmful weeds whose control/eradication is mandatory (Pyšek, Richardson & Williamson 2004). In such a scheme, only weeds present an actual hazard, as none of the other stages are assumed to have any negative environmental or economic impact (Richardson et al. 2000). Unfortunately, weeds (noxious or otherwise) are poorly characterised in floras, and thus, risk assessments have often only examined the transitions between casual and naturalised status or the distinctions between local and widespread naturalised plants (Pyšek & Richardson 2007; Dawson, Burslem & Hulme 2009a; Gassó, Basnou & Vilà 2010). Yet, only a small proportion of naturalised plants are recorded as having impacts. In Europe, over 3700 alien plant species have naturalised, but only around 9% are recorded as having either economic or ecological impacts (Vilàet al. 2010). The proportion of naturalised species having known impacts is much lower for plants than for other taxonomic groups and may simply reflect the absence of evidence rather than evidence of absence regarding harmful effects. As a consequence, impacts have often been assumed to scale with the geographic distribution of a species (Parker et al. 1999), and thus, risk assessments have aimed to distinguish between widespread and more localised naturalised plants (Lloret et al. 2005; Hanspach et al. 2008; Dawson, Burslem & Hulme 2009b). However, at least for one measure of impact, financial cost, there is little evidence that species distribution is equivalent to impact (Fig. 1). This reflects that species differ considerably in their per capita impact and that many widespread alien plants have negligible impacts. Only relatively rarely is risk assessment undertaken to distinguish species with known impacts from those with no known impact (Pheloung, Williams & Halloy 1999; Daehler et al. 2004; Nishida et al. 2009; McClay et al. 2010). In these analyses, weeds are usually classified using expert knowledge which may not always be objective, accurate, consistent or reproducible. Since the relative importance of plant species traits varies across the invasion sequence from introduction to impact (Kuster et al. 2008; Dawson, Burslem & Hulme 2009b), we might expect those traits important in naturalisation to be quite different from those determining impact. Yet, our current perceptions of the importance of different pathways, taxonomic groups and life-history traits in invasions is largely shaped by detailed knowledge of naturalisation rather than impacts (Vilàet al. 2011). Thus, it is hard not to conclude that given the complexity of impacts and the difficulty in defining weeds, a framework for predicting such hazards remains a distant goal.

Figure 1.

 Lack of relationship between how widespread an alien plant species is in Great Britain (number hectads or 10 × 10 km squares occupied) and its estimated economic cost (data from Table 1). Data are presented separately for each of three sources of economic data. No significant relationship exists across all data combined (R2 = 0·022, F(1,28) = 0·63, = 0·435) or for any one of the individual data sources.


The processes that lead an introduced plant to become a weed share several features in common with complex systems, in particular they usually exhibit nestedness, nonlinear relationships, positive feedbacks, temporal lag effects as well as non-stationarity. The determinants of weed naturalisation act across a nested spatial hierarchy that ranges from local to global scales. At a local scale, weed risks are shaped not only by species life-history traits but also their spatio-temporal interactions with the biotic and abiotic features of the ecosystems they colonise (Richardson & Pyšek 2006). At regional scales, extrinsic processes such as propagule pressure are important correlates of plant naturalisation (Pyšek, Krivánek & Jarošík 2009) and these are often influenced by human variables such as local economic activity (Taylor & Irwin 2004) and human population density (Pyšek et al. 2010b). Regional dynamics are increasingly influenced by global drivers that include import volumes and trade origins (Hulme 2009). At each of these hierarchical levels, nonlinear relationships appear important and include Allee effects at local scales (Elam et al. 2007), range expansion dynamics at regional scales (Pachepsky & Levine 2011), and macroeconomic correlates of alien plant richness at continental scales (Vilà & Pujadas 2001). Positive feedbacks on weed naturalisation and spread have largely been observed at local scales and include intraspecific relationships where a single species modifies the environment (e.g. fire regime, nitrogen cycling) that subsequently assists further population expansion (Laungani & Knops 2009) and interspecific interactions where one weed species may facilitate the establishment of another (Tecco et al. 2007). Time-lags are known to occur between introduction and local naturalisation (Daehler 2009), naturalisation and regional spread (Aikio, Duncan & Hulme 2010a), as well as macroeconomic drivers and species introductions (Essl et al. 2011). Finally, non-stationarity refers to a complex system that shifts systematically over time such that past conditions may not be a guide to future dynamics. This appears to be the case with plant naturalisation where annual rates of naturalisation are not constant and appear to have progressively increased since the early 20th century (Hulme et al. 2009). Climate change may also lead to an increase or decrease in the susceptibility of ecosystems to invasion such that past patterns of weed establishment may not hold in the future. Furthermore, weeds may undergo rapid phenotypic and/or genotypic adaptation following introduction into a new region, which may influence the magnitude of their impacts and response to environmental drivers (Hulme 2008; Whitney & Gabler 2008). The implications of complexity on weed risk assessment are twofold. First, current knowledge of weed risks is firmly based on information drawn from local, regional or global perspectives, but a hierarchical approach that integrates across scales would appear essential because the drivers of alien plant distribution are scale dependent (Collingham et al. 2000). Even in situations where risk assessment considers processes occurring at more than one scale, the links among different scales are rarely considered. Such integration is particularly important in these vertically nested systems where the higher levels of system aggregation are thought to set the boundaries to behaviour while the lower levels determine the range of possible behaviours (Holling 2001). Second, while further quantification of these hierarchical relationships will facilitate greater understanding of weed risks, the behaviour of such complex systems (and thus the likelihood of an introduced plant becoming a weed) will be notoriously difficult to predict (Pilkey & Pilkey-Jarvis 2007).

Uncertainty, variability and overconfidence

Given the complexity of the relationships between the many potential explanatory variables, current understanding of weed risks is imprecise. Imprecision stems not only from emergent properties of complex systems but also from both the variability and uncertainty in the parameter estimates. Variability and uncertainty may be easily confused, but they are distinct concepts defined within a decision-making context. Variability is a property of a parameter such as a life-history trait, dispersal rate or susceptibility to herbicide and reflects temporal, spatial, or individual differences in the value of that parameter. The existence of variability in the population implies that a single action or strategy may not emerge as optimal for each individual in a population, and consequently, any decision made will go too far for some and not far enough for others (Thompson 2002). Uncertainty, on the other hand, is a property of the analyst and reflects ignorance about a poorly characterized phenomenon such as lag phases in population growth, propagule pressure, or impact on ecosystem function. Uncertainty implies that decisions may be non-optimal because the outcomes may not be as expected. The integration of variance and uncertainty into weed risk assessment is still in its infancy (e.g. Caley, Lonsdale & Pheloung 2006; Diez et al. 2008). For example, ensemble approaches to modelling alien plant species distributions usually find considerable variation among competing models in terms of goodness of fit or projected distribution range (Stohlgren et al. 2010; Lemke et al. 2011). Yet, this variation is usually ignored and either the best fitting and/or a combined ‘average’ model is used in prediction without further accounting for the observed uncertainty in model outputs.

An increasing number of probabilistic risk assessment tools have been developed to help integrate estimates of variability and uncertainty in risk assessment (Vose 2008). These usually require prior estimates of the range and distribution of variation or uncertainty in parameters to be specified in the model (Burgman 2005). For variance, the values can be estimated objectively from data but for uncertainty the specification depends on the availability of prior information such as differences in performance of species distribution models. However, in many cases, there are no prior data upon which to base measures of uncertainty and values depend upon subjective choices. Alternatively, an ordinal ‘uncertainty score’ is associated with each parameter (Baker et al. 2008). However, a major limitation to incorporating uncertainty in risk assessment methods is that, when asked for subjective estimates of prior distributions for quantities that can be verified, researchers and decision makers often produce distributions that are far too tight, with actual values lying outside an expert’s estimated 98% confidence limits up to 45% of the time (Hammitt & Shlyakhter 1999). Overconfidence becomes more of a problem as the availability of information increases, as parameters become more difficult to estimate, and the fewer parameters are used to characterise uncertainty (Soll & Klayman 2004). A narrower prior distribution results in a smaller expected value of information which leads to the tendency to underestimate the probability of extreme events. Underestimating extreme events, such as high-impact and fast-spreading species, is a major concern in weed risk assessment, as management costs for a region may be largely determined by only a handful of particularly problematic species (Table 1). A single plant species, Fallopia japonica, accounts for one-third of the total annual alien weed-management costs in Great Britain (Williams et al. 2010). Although such underestimates may be reduced by adopting prior distributions designed to mitigate the effect of neglecting potential surprises, such as a long-tailed compound distribution (Franklin et al. 2008), extreme cases will, by definition, be hard to predict.

Table 1.  Estimated economic costs of 30 alien plants in Great Britain
SpeciesCommon nameCost (£) Source
  1. Key to sources: 1: Williams et al. (2010), 2: Williamson (2002), 3: Oreska & Aldridge (2011)

Fallopia japonica Japanese knotweed165 609 0001
Avena fatua Wild oat58 235 1682
Veronica persica Common field-speedwell36 397 1122
Hydroctyle ranunculoides Floating pennywort25 467 0001
Avena sterilis Winter wild oat12 736 7242
Elodea canadensis Canadian pondweed11 640 5793
Rhododendron ponticum Rhododendron8 621 0001
Elodea nuttallii Nuttall’s pondweed4 369 7113
Heracleum mantegazzianum Giant hogweed2 362 0001
Lagarosiphon major Curly waterweed1 173 2143
Myriophyllum aquaticum Parrot’s feather1 131 5503
Aegopodium podagraria Ground elder1 000 0002
Impatiens glandulifera Himalayan balsam1 000 0001
Buddleja davidii Buddleia961 0001
Matricaria discoidea Pineapple weed798 1082
Azolla filiculoides Water fern444 7713
Galinsoga parviflora Gallant soldier337 7292
Crassula helmsii Australian swamp stonecrop132 4403
Ludwigia grandiflora Water primrose24 0001
Centranthus ruber Red valerian16 9842
Conyza canadensis Canadian fleabane16 9842
Cymbalaria muralis Ivy-leaved toadflax16 9842
Veronica filiformis Slender speedwell16 9842
Crepis vesicaria Beaked hawksbeard9 7992
Epilobium brunnescens New Zealand willowherb9 7992
Epilobium ciliatum American willowherb9 7992
Senecio squalidus Oxford ragwort9 7992
Mimulus guttatus Monkeyflower9 7992
Allium triquetrum Three cornered leek6 7682
Smyrnium olusatrum Alexanders1 8812

Ignoring the silent evidence of failure

The analyses of existing aliens in floras that form the foundation of current knowledge on weed risks represents a strongly biased subset of the species introduced to a region and ignores the multitude that have failed to naturalise. The step between introduction and establishment is a critically important filter in biological invasions and one for which we have little information (Puth & Post 2005). For example, marked differences exist in the representation of different plant families in alien floras either when compared with the global (Pyšek 1998) or regional native species pool (Vilà & Muñoz 1999). However, for Australia and New Zealand, the taxonomic composition of the alien flora appears to be strongly determined by the frequency with which species in different families were introduced (Fig. 2). Only when knowledge of the introduction effort is taken into account can outlying families that exhibit higher naturalisation rates be seen, which for both countries include the Poaceae. The relatively high proportion of the variance explained by introduction effort and the similarity in relationships in both countries are indicative that much of current perceptions of what underpins weed success may be confounded by different patterns of species introduction (Lambdon, Lloret & Hulme 2008). In Australia, about 80% of noxious weeds stem from ornamental plants used in gardening (Virtue, Bennett & Randall 2004), and this fact is often interpreted as ornamental species pose the highest risk of becoming noxious weeds. Although garden ornamentals represent the major pathway by which alien plants have been introduced to Australia, of the 25 360 species introduced via this route, only around 1% is classed as noxious weeds (Virtue, Bennett & Randall 2004). Indeed, compared with other major introduction pathways, garden ornamentals have the lowest probability either of naturalising or becoming noxious weeds. Thus, for any risk assessment, it is important to consider the prior knowledge of the probability that a particular event will occur and assess these risks correspondingly.

Figure 2.

 Significant positive relationships between the number of alien species introduced and number naturalised (both log scales) for the top 20 alien plant families naturalised in New Zealand and Australia (data from Diez et al. 2009). Neither the slopes nor intercepts of this relationship differed between New Zealand (y = 0·674x + 0·097, R2 = 0·660, F(1,18) = 37·88, < 0·0001, solid line) and Australia (y = 0·764x− 0·092, R2 = 0·802, F(1,18) = 78·00, < 0·0001, dotted line).

Insufficient accuracy to account for low base-rates

It is widely acknowledged that only a small proportion of plant species introduced into a region successfully become weeds, but actual values are remarkably hard to come by largely because, as described earlier, such information requires data on failures. However, this information is essential to guide risk assessment because the lower the proportion of successes or base-rate (also termed prevalence), the more difficult it becomes to make robust predictions regarding the likelihood of invasion (Hastie & Dawes 2010). Where this has been undertaken for regional floras, the percentage of species naturalising is close to 10%, e.g. British Isles – 13% (Williamson 1996); Australia and New Zealand each 9% (Diez et al. 2009). Comparable estimates for weed numbers (species with known impacts) are complicated by different weed classifications in these three countries (i.e. in the extent they distinguish between environmental and agricultural weeds) but can be estimated as: British Isles 0·9% (Williamson 2002), Australia 1·3% (Virtue, Bennett & Randall 2004) and New Zealand 1·3% (Popay, Champion & James 2010). In general, the lower the base-rate, the higher the accuracy (i.e. success in correctly predicting a pest or weed) required for predictions to be better than a random guess (Smith, Lonsdale & Fortune 1999). This is because for a given level of accuracy, the lower the base-rate the higher the corresponding rate of false positives, and thus, the discriminatory power of any prediction is reduced. For example, to generate predictions better than random (e.g. having a probability >0·5 of correctly classifying a species as a weed) where the base-rate is only 2%, a test would need to have an accuracy of 98%. Yet, reported accuracies using the semi-quantitative AWRA protocol are often <90% (Gordon et al. 2008) and suggest that given the typically low base-rates found in plant invasions, weed risk assessments are likely to have too low an accuracy to be valuable tools.

Unfortunately, most weed risk assessments suffer from base-rate neglect, where analyses do not account for realistic weed prevalence. The average base-rate across seven AWRA analyses was 32·7 ± 8·1% (Gordon et al. 2008), an order of magnitude higher than might be expected for realistic values of weed prevalence. Such high base-rates indicate a biased species pool upon which the analyses were undertaken. These species pools tended to include species already naturalised in the region for which life-history data were available or represent specifically selected similar sample sizes of weeds and non-weeds, approaches that would both lead to over representation of weeds in the analyses. Such tests should sample weeds in relation to their prevalence in the species pool, even when sample sizes available for screening are small. It is instructive to note that weed risk assessment studies using a more objective measure of the species pool tend to find base-rates closer to 10% (Křivánek & Pyšek 2006; Dawson, Burslem & Hulme 2009a). Receiver-Operating Characteristics analyses are increasingly used to assess the performance of weed risk assessment, as they provide a measure of goodness of fit that is independent of base-rate (Gordon et al. 2008; Dawson, Burslem & Hulme 2009a). However, the overall accuracy, calculated as the proportion of correct decisions, depends on the proportions of cases and controls in a data set, and so is not an inherent property of the diagnostic test but rather a property of its application in a particular context (Hughes & Madden 2003). Thus, at best, assuming an unbiased species pool, current weed risk assessment analyses probably have potential to estimate the risk of a naturalised species becoming a weed rather than a new species introduction becoming naturalised. However, at worst, where comparison groups are biased towards weeds, weed risk assessment may not tell us anything useful about plant invasions at all.

Cost–benefit analyses and the challenge of balance-sheet risk assessment

The low base-rates in plant invasions considerably limit the predictive power of weed risk assessment (Williamson 1996). However, if the relative benefits of preventing the introduction of harmful species versus the costs of not importing valuable species are taken into account, this may mean weed risk assessment can be a suitable screening tool, even where prediction is poor (Smith, Lonsdale & Fortune 1999; Hughes & Madden 2003; Keller, Lodge & Finnoff 2007). While this bioeconomic perspective has been used to emphasize the regulatory value of weed risk assessments, the underlying assumptions of such modelling have perhaps been accepted too uncritically (Weber et al. 2009). First, outcomes of bioeconomic modelling are sensitive to the underlying assumptions of costs and benefits, the discount rate (i.e. the net present value in relation to future interest rates), and time-scale over which costs and benefits are experienced. Thus, two bioeconomic assessments of the AWRA protocol drew quite different conclusions regarding the robustness of decision-making (Smith, Lonsdale & Fortune 1999; Keller, Lodge & Finnoff 2007). Second, costs (such as those arising from lost production or management expenditure) are notoriously hard to quantify and estimates for the same species and region can differ considerably. Recent estimates of the costs of floating pennywort Hydrocotyle ranunculoides in Great Britain range over two orders of magnitude from £375 088 (Oreska & Aldridge 2011) to £25 467 000 (Williams et al. 2010). Given these difficulties in estimating actual costs, predicting potential costs of a species appears rather fanciful. Third, in the absence of reliable predicted costs, averages of the historical costs of weeds are often used instead (e.g. Keller, Lodge & Finnoff 2007). Unfortunately, this approach ignores the strongly skewed distribution of costs, such that median values are likely to be significantly lower than means. In Britain, while the mean annual cost for 30 alien weeds is £11 085 556, the median value is two orders of magnitude smaller at only £621 440 (Table 1). Fourth, benefits are also likely to be skewed. Weeds pose a threat to one-third of all New Zealand nationally threatened plant species, and estimates suggest that without action, weeds could potentially degrade 7% of the conservation estate within a decade, corresponding to a loss of native biodiversity equivalent to $1·3 billion (Williams & Timmins 2002). However, this decadal value is little more than the export income generated by kiwi fruit Actinidia deliciosa in 2009 (Statistics New Zealand 2010). Kiwi fruit spreads from cultivation and is a significant high-risk weed in sub-tropical forests of New Zealand (Williams & Newfield 2002), yet the outcome of balancing the costs to the environment against the economic benefits would not have prevented the introduction of this species. This is likely to be true of other high-value high-risk introductions. Finally, even if consistent performance is found across the globe, the bioeconomic value of weed risk assessment will vary across different regions depending on the underlying base-rate and the expected environmental and economic cost of weeds. Thus, extrapolating the benefits accrued to Australia in implementing weed risk assessment may not translate to the numerous other parts of the world where similar AWRA protocols have been applied to screen alien plants (Gordon et al. 2008).

Cognitive biases and the problem of expert judgement

Cognitive biases are inherent in all decision-making processes, and even the most objective statistical approaches can be influenced by overconfidence and base-rate neglect, as discussed previously. Similarly, while semi-quantitative scoring methods might be seen as objective, bias can actually exist in the particular questions asked. This includes ‘focusing bias’ where too much importance may be placed on certain preconceived attributes associated with invasiveness. These preconceived attributes often stem from knowledge of specific case studies or personal experience rather than reflect general phenomena. Such bias can be impervious to new evidence that contradicts the established paradigm. For example, the AWRA strongly weighs climate-match and evidence of invasiveness elsewhere (Weber et al. 2009). However, an increasing number of introduced plants are either the first representatives of their genera to become weeds of any sort anywhere in the world (Williams & Newfield 2002) and/or may sometimes occupy distinct climate niches in their introduced range (Broennimann et al. 2007; Gallagher et al. 2010). However, cognitive biases come into their own in qualitative expert assessments and form a major discipline (heuristics) in experimental psychology. Most cognitive biases would be relevant to expert panels addressing weed risks (Koehler & Harvey 2004). There is no scope to cover all potential cognitive biases here but framing, anchoring and confirmation bias are illustrative of the challenges facing expert-based risk assessment.

Risk assessors may draw different conclusions from the same information, depending on how that information is presented. Such a framing bias can be illustrated by the following two statements regarding cogongrass Imperata cylindrica:

  • 1 Cogongrass is a perennial rhizomatous grass native to east and southeast Asia, India, Micronesia, Australia, and eastern and southern Africa. It grows from 0·6–3 m tall. The leaves are about 2 cm wide near the base of the plant and narrow to a sharp point at the top; the margins are finely toothed and are embedded with sharp silica crystals. http://en.wikipedia.org/wiki/Imperata_cylindrica.
  • 2 Cogongrass is an aggressive, rhizomatous, perennial grass that is distributed throughout the tropical and subtropical regions of the world. It has become established in the southeastern United States within the last fifty years, with Alabama, Mississippi, and Florida having extensive acreage of roadway and pasture infested with cogongrass. http://plants.ifas.ufl.edu/parks/cogon_grass.html.

Of the two statements, the latter is certainly more value laden but undoubtedly paints a more vivid picture of cogongrass. Given that many weed compendia or species factsheets often portray species in relation to the most dramatic impact, framing on such information may be a frequent limitation of expert judgement. Such framing may also occur if particular alien plant species are presented as reference examples to guide experts in answering specific questions. Unfortunately, once such a picture is perceived, it can be hard to erase from memory and can contaminate subsequent assessments as experts adjust their scoring to match this mental picture resulting in potentially a higher risk rating than is warranted. Subsequently, experts may anchor on such information and allow an initially formed opinion to shape subsequent judgement, even if contrary information comes to light (Hastie & Dawes 2010). Such anchoring may occur as assessors build a mental picture of the target species as they progressively work through individual questions of a risk assessment. If initial answers indicate a high probability of introduction and establishment, subsequent scores regarding spread and impact may become inflated to match the growing expectation of a high-risk species. This can lead to confirmation bias, where experts tend to search for, or interpret information in a way that corroborates initial preconceptions. These inherent cognitive biases suggest using known invasive alien species to test the robustness of qualitative expert assessments may not be valid because it is almost impossible for experts to be objective and, almost without fail, such tests conclude that qualitative expert assessments work well (e.g. Baker et al. 2008).

A way forward?

The foregoing paints a rather negative picture of the prospect of predicting weed risks. Why then do researchers and regulators persevere in this task? First, hindsight bias results in decision makers filtering their memories of past events through present knowledge, so that those events look more predictable than they actually were at the time they occurred (Hastie & Dawes 2010). The tendency to generate hindcasts to ‘predict’ past naturalisation and invasion events compounds this hindsight bias. Hindcasts have questionable predictive value because it is hard to be objective about past events (because good explanations can often be imagined) while they also erroneously assume the past is a perfect guide to the future (Pilkey & Pilkey-Jarvis 2007). Second, the recent vogue for invasive species risk assessments has been largely driven by the demands of regulatory authorities for tools to address trade-related biosecurity issues (Hulme 2011a). Historically, these tools were initially developed to address risks of phytosanitary pests to agriculture (Baker et al. 2008). Compared with the risk posed by environmental weeds, those from phytosanitary pests are more easily estimated because the target (e.g. crop) is known, the potential pool of pests of a specific crop is relatively small and well recognised, and the costs (both in terms of production losses and management interventions) are usually large and quantifiable in monetary terms. The perceived success of phytosanitary measures in agriculture has spurred their development to address invasive plants (Brunel & Petter 2010), but it is likely that an approach that works for crop pests may not work for the more complex scenario of environmental weeds. Indeed, weeds may be particularly intractable because of their generally low base-rates and long lag phases compared with other taxa (Williamson 1996). Third, there is considerable inertia in regulatory authorities to appraise the effectiveness of their risk assessments. Thus, the weed risk approaches adopted by the European and Mediterranean Plant Protection Organisation (based on FAO 2004) and the Australian and New Zealand biosecurity agencies (based on Pheloung, Williams & Halloy 1999) are relatively unchanged from original protocols developed over a decade ago.

Given these difficulties, is there a future for weed risk assessment? Clearly current practices could be improved to address consistent hazard identification, encompass a hierarchy of spatio-temporal scales, incorporate uncertainty, account for the evidence of failures to generate realistic base-rates, and train risk assessors to limit cognitive biases. However, such refinements may still fail to predict weed risks any better than a knowledge of prior invasion history and quality of climate match. Field trials aiming to examine species performance in a new region have been proposed as a means to reduce uncertainty in the prediction of weed risk. These studies can result in quite different predictions than those based on weed risk assessment tools (Davis et al. 2011). However, such trials will need to assess different sizes of founder populations and the extent and character of cultivation (intentional or unintentional) that the species might receive (Minton & Mack 2010). As a result, this approach is only likely to be suitable for screening a small number of high-value introductions e.g. biofuel or genetically modified crops. For horticultural species, unintentional field trials may already exist in the living collections of botanic gardens and data on the performance of introduced species might be a valuable indicator of behaviour outside the cultivated environment (Hulme 2011b).

Alternative approaches such as scenario planning could be adopted. Scenario planning brings together diverse stakeholders to develop a small number of hypothetical futures for the system of interest. The narrower the scope of strategic decision, the easier will be the scenario construction. Each scenario is developed to capture key ingredients of the uncertainty about the system’s future, and to provide insight into drivers of change and implications of current trends within the system. Decision-makers can then analyse how robustly alternative management strategies might perform across these different storylines (Pilkey & Pilkey-Jarvis 2007; Alcamo 2009; Hulme 2011a). The approach has not been frequently used for weed management, and its more widespread adoption would represent a philosophical change from prevention towards adaptive management. Under these circumstances, decision-makers would acknowledge the difficulty in prediction and, as a consequence, target investment towards early detection, mitigation and management. Such a strategy may be particularly suitable for weeds that can be rapidly surveyed and have sufficiently long lag phases between first naturalisation and invasion (Aikio, Duncan & Hulme 2010a) that early detection is a feasible option. Early-detection and rapid-response (EDRR) systems for weed surveillance are currently being developed and trialled around the world (e.g. Westbrooks 2004). There remains considerable scope to optimise EDRR for more effective weed detection and control (Giljohann et al. 2011). Furthermore, if EDRR can be integrated with assessments of ecosystem vulnerability to invasion (Pyšek et al. 2010a) and interventions to improve ecosystem resilience (Hulme 2006), it would deliver a robust post-border approach to invasive plant management. This approach would address threats from both intentional and unintentional introductions as well as plants that have yet to have noticeable impacts but have already been naturalised in a region for several decades (sleeper weeds). Targeting sleeper weeds is seen as an alternative management approach when predictions regarding new introductions are too uncertain to be valuable and the eradication of widely naturalised weeds is no longer a cost-effective option (Groves 2006). While it has been suggested that tools such as the AWRA protocol should be used to identify sleeper weeds, the success of such approaches will again depend on their accuracy and the underlying base-rate. Additional information drawn from herbaria on the spatio-temporal dynamics of naturalised plants may help to identify changes that might correspond to sleeper weeds emerging from their slumbers and thus reduce the rate of false positives (Aikio, Duncan & Hulme 2010b).

In conclusion, current weed risk assessment approaches on their own are rarely sufficiently reliable to predict future weeds but if combined with experimental or survey data may help screen intentional introductions or naturalised sleeper weeds. However, in the face of increasing number of unintentional plant introductions (Hulme et al. 2008), managing future weeds requires a different approach. Scenario planning relating to EDRR would help identify future guilds of target weeds, guide surveillance strategies, pinpoint the ecosystems most at risk and explore options for mitigation. To date, resistance to such an approach has focused on the high public costs, but the private sector could also play a significant role through graduated licence fees, cost-sharing instruments and environmental bonds (Touza, Dehnen-Schmutz & Jones 2007). It may now be time to acknowledge the high cost of poor weed risk prediction and invest in the development of alternative approaches to weed management.


The ideas in this paper have benefited from discussion with many colleagues through projects funded by the European Union 6th and 7th framework propgramme, particularly DAISIE (SSPI-CT-2003-511202); ALARM (GOCE-CT-2003-506675) and PRATIQUE (KBBE-212459).