The future direction of ecological risk assessment in the united states: Reflecting on the U.S. environmental protection agency's “examination of risk assessment practices and principles”



As part of an agency evaluation of the development of risk assessment tools, the U.S. Environmental Protection Agency (U.S. EPA) issued a staff paper in March 2004 describing the current state of practices and policies regarding risk assessment from the agency's perspective. The staff paper provided a rationale for the agency's positions regarding technical topics frequently the subject of comments or complaints. Considerations relevant to ecological risk assessment were included, primarily in a single chapter focused on this topic. This commentary highlights four technical issues important to the advancement of ecological risk assessment practice and discusses how some of the points raised by the U.S. EPA in the staff paper have significance to ecological risk assessors. Discussion regarding incorporating population- and community-level analysis into risk assessments and the apparent reluctance of the U.S. EPA to adopt advances in this area is provided. Also, continuing inconsistencies in the body weight scaling of toxicity values between human health and ecological risk assessment practices are discussed. Ideas for refining the calculation of exposure point concentrations for ecological receptors and the progression between screening steps and comprehensive risk assessment are also discussed.


The U.S. Environmental Protection Agency (U.S. EPA) released a staff paper, “An Examination of EPA Risk Assessment Principles and Practices” (USEPA 2004), intended to present the state of practice and policy regarding risk assessment from the agency's perspective and to encourage dialogue with outside practitioners and interested nonspecialists. The staff paper is neither a new guidance manual for carrying out risk assessments nor a watershed policy memorandum. Rather, the staff paper attempts to provide historical context to help explain current practices and policies. It also provides explanations for agency positions regarding technical topics that draw frequent or contentious comments. The review that resulted in the staff paper was carried out by a broad group of agency staff representing headquarters and the regional offices. The review was completed in response to agency management's desire to evaluate practices and policies relating to risk assessment, and the staff report represents documentation of current positions. As anticipated, and in fact encouraged by the U.S. EPA, this type of review also provides an accessible starting point for external review, analysis, and feedback regarding agency practices and rationales.

This commentary focuses on ecological risk assessment (ERA) topics in conjunction with what is said (and not said) in the U.S. EPA staff paper. The goal is to present a different perspective on several significant technical positions taken by the agency and to highlight key technical areas where further dialogue, research, and scientific analysis will help advance the state of agency practice. This commentary includes an overview of the staff paper, including chapter 6 (the chapter focused specifically on ERA).


Chapters 1 through 5 of the staff paper focus primarily on the human health risk assessment perspective; however, several of the concepts are generally applicable to ERA as well. Chapter 1 presents an introduction to U.S. EPA risk assessment, including background on the evolution of risk assessment guidance and how the U.S. EPA uses risk assessment in decision-making. Chapter 1 explains that risk assessments are expected by the U.S. EPA to analyze and describe the nature and extent of potential environmental risks, explain adverse outcomes that could be associated with the risk, and describe the confidence and uncertainties in the risk characterization.

Chapter 2 explains how the U.S. EPA intends to protect human health and the environment by requiring selection of input factors to ensure that risk is not likely to be underestimated. The U.S. EPA recognizes that bounds are necessary to this directive because without limits, inputs and scenarios can be infinitely extended to more and more protective hypothetical scenarios. Consistent with years of U.S. EPA statements, assessments are supposed to avoid “gross overestimation of risk” (USEPA 2004) and input values selected are expected to be “scientifically plausible given existing uncertainties” (USEPA 2004). Although the staff paper clearly directs that assessment assumptions be constrained in order to provide a useful tool, this is often not the case in agency application. Individual decision-makers are less motivated to push assessors for refinements toward realism and plausibility than they are to push for assurances that receptor-dependent input values capture the protective end of the range of scenarios. The line separating scientific plausibility from scientific implausibility is not always clear, so ongoing dialogue is necessary to achieve a balance consistent with guidance directives. This is particularly challenging for ERA given the number of input factors needed for multipathway, multispecies assessments.

Chapter 3 addresses uncertainty and variability and the related precision that can be achieved in risk estimates calculated using the U.S. EPA assessment approaches. Discussion applicable to ERA on uncertainties in parameter specification (e.g., relating to the measurement of input values and the selection of surrogate values) is presented. Also applicable to ERA, uncertainties in model specification (e.g., relating to the use of measurement endpoints as indirect surrogates and oversimplified representations) are characterized. The staff paper specifies that ratio-based risk estimates (hazard indices) should only be presented with one significant figure. This is significant for ERAs. The hazard index is the sum of hazard quotients (HQs) calculated for each separate chemical as the ratio between an exposure estimate and corresponding Toxicity Reference Value (TRV). It has been common practice for some regulatory staff to view the hazard indices threshold value of 1 as a bright line at 1.0, with calculated hazard indices of 1.1, 1.2 or 1.3 requiring additional explanation or progression to the next level of more complex assessment. Such interpretations assume more precision in the calculation than is plausible, and the staff paper makes it clear all of the example values listed above are correctly interpreted as being equivalent to hazard indices of 1.

Chapter 4 focuses on information gaps and the use of default values and extrapolation assumptions relevant specifically to human health risk assessment. However, chapter 5 provides some insight into issues broadly applicable to ERA, including a discussion regarding the priority given to chemical/site-specific information. The staff paper explains the U.S. EPA's practice of giving chemical/site-specific information preference over generic default inputs where such information is available, relevant, and supported by peer review. This makes clear that the U.S. EPA does expect risk assessors to use regionally relevant and site-relevant receptor inputs where possible, rather than standardized default receptor characteristics. Since the background information compiled in the Wildlife Exposure Factors Handbook (USEPA 1993) includes regionally distinct options for some parameters, the staff paper substantiates the selection of regionally relevant values, even with respect to certain default values. With regard to peer review, however, it is not clear whether the U.S. EPA intends to require that unique, site-derived values must be submitted for peer review before they can be used in ERA, or only that values selected from the literature must be from peer-reviewed sources. Overall, although the staff report defends the U.S. EPA's basis for selecting default inputs, it does reinforce the importance of specifying chemical/site-specific inputs where relevant.


Chapter 6 of the staff paper addresses topics specific to ERA, beginning with an overview of ERA guidance and the current U.S. EPA approach. The staff paper then addresses the following topics: organism-level versus population-level ERAs, strategies to maintain protectiveness in ERA, two methods for calculating Water Quality Criteria, and uncertainty in pesticide assessments.

Several of the points made by the U.S. EPA warrant broader discussion and serve as the key technical points for this commentary. These include (1) the U.S. EPA's rationale regarding the strength of individual organism-based assessment tools compared with population- or community-level approaches, (2) an unsettled technical choice relating to TRV estimation, (3) broader aspects of site-specific exposure characterization, and (4) refinements to preliminary risk characterization that provide for more early decision points in the ERA process.

Organism-level versus population-level ERAs

The current U.S. EPA approach to ERA relies predominantly upon comparisons between environmental concentrations and criteria values derived to protect against toxic responses in individual receptor organisms. Guidance directs that environmental risks generally be managed to protect higher organizational levels (populations and communities), except where statutory protection is provided for individual organisms (e.g., endangered species). The focus on protecting populations and assemblages of populations is sensible and reasonable because individual organisms die while populations are the lowest ecological unit expected to persist through time. However, the assessment tools prepared for risk managers using the current U.S. EPA approach are intrinsically related to predicting potential risks to individuals. Currently, decision-makers are simply directed to factor this limitation of the assessments they receive into their interpretation. Updated approaches linking population- or community-level assessment tools directly to decision goals could improve the information base available to decision-makers. The U.S. EPA has heard comments to this effect frequently. The staff paper outlines responses and the rationale for the agency's continued reliance on individual-based assessment tools.

In response to comments about the use of organism (individual)-level attributes to assess ecological risk, the staff paper points out that the protection of organisms is generally interpreted as being protective of a population. After limited discussion, the following conclusion is stated:

[T]he bottom line is that, although methods exist for predicting the effects of chemicals at levels of organization higher than the organism, they are still in the development phase and have not been shown to be reliable. On the other hand, assessment of ecological risks using measures of organism-level effects is justified by experience, policy, and judicial decision. (USEPA 2004)

The staff paper also defends individual-based HQ values, stating they can be used to characterize population-level risks if they are based on effects such as mortality or offspring production and can be correlated with population dynamics or other population-level indicators. The staff paper also states that although Maltby et al. (2001) described hypothetical approaches for such extrapolation, such strategies for getting beyond the individual implications of the HQ have not been implemented because they are “impractical” and “undefinable.”

To many experienced risk assessors, the overall protection provided by decisions based on organism-level ERA results is not as clear as the staff paper suggests. Although decisions intended to provide protection at the level of individuals can be protective of populations, there are examples where communities have been harmed and populations extirpated by remedial actions conducted to address perceived potential risks. Large-scale, in-stream sediment removal or capping efforts using sediment screening guidelines or projected food chain-based risks to birds or mammals as removal triggers provide an obvious example; such remediation activities can cause practically a 100% mortality risk for all of the benthic species in the short term, resulting in whole-scale disruption of community structure and the food web. Removal activities in wetlands areas can also be highly disruptive, not only from the actual removal of the substrate, but because of the infrastructure often necessary to perform the remedial action. Wetland excavations resulting in the elimination of observable, established benthic communities often occur on the basis of comparisons to generic screening values. Clearly, responsible environmental management cannot be saddled with a “do no harm” imperative, but these examples demonstrate that remedial actions intended to protect individuals can harm the very populations and communities the ERA guidance directs should be the focus of protection.

The position presented in the staff paper that population- and community-level assessment tools are not adequately developed is a convenient, but not correct, generalization. As discussed below, although existing approaches are still being refined in conjunction with the lessons of practical implementation, there are specific methods that have been published and subjected to broad scientific consideration. The staff paper does not include any statements about research needs or specific efforts to bring population- and community-level tools online for the future. The views reflected in the staff paper appear to downplay the advances that have been made in the field and the significant efforts that are currently underway (including those of U.S. EPA staff), which are described below.

The topic of organism-level versus population-level decision-making and corresponding assessment tools is actively being investigated and rapidly advanced, not just in the United States, but also in Canada, Europe, and Asia/Pacific countries. Population-level assessment has been the topic of extensive presentations and debate at scientific meetings (Society of Environmental Toxicology and Chemistry [SETAC] 2002, 2003, and SETAC-Europe 2004; Pastorok et al. 2002). In fact, the U.S. EPA was among sponsors of a Pellston Workshop held on the topic in 2003 (Barnthouse et al. forthcoming). In all of these forums, risk assessors from regulatory agencies, industry, and academia, along with risk managers, have presented ideas and research and continue to build through direct interactions the foundation establishing how, where, and when different tools for assessing risks at the population level or community level can be effective.

Further dialog on the topic of individual- versus population-level ERA must acknowledge the widely recognized fundamental limitations on projecting estimates of risk for individuals to population-level impacts using the HQ approach (Tannenbaum 2003; Tannenbaum et al. 2003; Sorensen et al. 2004). Principal among these are the intentionally protective (i.e., biased) assumptions used to estimate exposure, uncertainties in comparing environmental exposures to laboratory-derived criteria, the inability to account for prey/habitat substitution in food chain uptake predictions, and limitations on accounting for episodic or seasonal fluctuations in exposure.

Although there are technical challenges and uncertainties in addressing populations, there are many examples where field-level studies have focused on population-level end- points using empirical studies in addition to (or in lieu of) chemical-specific HQs for individuals (e.g., Fontenot et al. 1998; McGee et al. 1999; Nacci et al. 1999; Fontenot et al. 2000; Boonstra and Bowman 2003; Meyer and DiGiuliuo 2003). Ecologists were successfully describing the effects of stressors on populations and interactions affecting community structure long before ERA as a formalized regulatory application was developed.

Although U.S. EPA guidance clearly indicates that interpretation of the ecological significance of quantitative results and uncertainties must be provided in order to reasonably characterize the relevance of individual-based HQs, this step requires in-depth consideration and frequently lessens the importance of individual quantitative results. Also, substantiating decisions based on this type of synthetic interpretation is often perceived to be a less objective basis than simply pointing to the numbers arising out of analytical chemistry-based comparisons. Accordingly, guidance directives notwithstanding, individual decision-makers are frequently hesitant to make decisions based on weight-of-evidence evaluations that balance quantitative results with more direct information and context for the subject ecosystem.

The limitations of relying only on individual-based approaches are much more significant than characterized in the staff paper. Attempting to protect individuals representing a few receptor species does not protect all relevant populations in all cases, as illustrated by the examples of using only individual-based HQs to select remedial strategies. Calculating a full set of HQs on sites with habitat areas that are observably too small or too isolated to affect populations can be a waste of resources. Even current individual-based approaches need continuing review and refinement, and there is no reason to close the door to population-level approaches. Population-level assessment approaches are already being used at a variety of sites, and there is good reason to believe we will continue to see advances in this area. Improved population assessment tools, when applied appropriately, can lead to better informed risk management decision-making.

Continuing inconsistencies in TRV calculation

The U.S. EPA did not use the staff paper as an opportunity to weigh in and clarify confusion and inconsistency pertaining to another technical detail significant to ERA, allometric scaling of TRVs based on body mass differences between test species and receptor species. Currently, ecological risk assessors struggle to keep up with the specific scaling approaches expected, or accepted, by agency staff in different regions. There is confusion about whether interspecific conversions should be used at all and, if so, between what groups of species and using what numerical scaling values. There is a considerable technical heritage regarding body mass scaling familiar to many ecological risk assessors leading to strongly held opinions and corresponding challenges in reaching agreement on particular assessments. Accordingly, this is a topic where broad-based consideration by the agency and discussion in the staff paper would have been useful to practitioners.

Allometric relationships describe how various physiological and morphological factors change in conjunction with changes in the size of organisms. Defining the specific relationship relevant for a particular factor, dose response, for example, allows predictions to be made about how dose responses change with increasing organism size. Ecologists have made effective use of allometric relationships in multiple areas, and various allometric conversions have been used in ecological risk evaluations to describe physiological rates as a function of body mass (Peters 1983; Calder 1984; Davidson et al. 1986).

With regard to ERA, one of the uses of allometric relationships is for interspecies scaling of toxicity testing results. TRVs can be corrected to reflect equivalent doses based on body mass differences between the species used in the test from which the TRV was derived and the receptor species needed for particular risk assessments. Accounting for the effect of body size on dose-response characteristics allows TRVs to be calculated for receptors of different sizes on a dose-equivalent basis. TRV differences resulting from scaling conversions result in directly proportional changes in calculated HQs (e.g., a 20% change in TRV causes a 20% change in HQ). Accordingly, significant swings in quantitative results and interpretations can depend on the conversion.

Agency guidance used in pesticide programs (Urban and Cook 1986) and sources of TRVs used by U.S. EPA regions (Sample et al. 1996) specify the need to convert TRVs based on receptor species body mass. At the same time, some regional and site-level staff object to toxicity value conversions. The latest Superfund program guidance (USEPA 1997) fails to give direction regarding when conversions are appropriate and the preferred specific scaling exponents. The argument in favor of allometric scaling is that dose responses are well recognized to be best described by a power function of body mass and, therefore, a scaling exponent should be applied to allow TRVs to be expressed on a dose-equivalent basis for each receptor in a risk assessment. The argument against using scaling factors is that the relative differences in body mass between many species—for instance, rats and mink—are too small to warrant the additional effort required to convert each toxicity value and that such conversions would not often change the outcome of the ERA. Also, the adjustment of “standard” TRV numbers and the use of multiple numerical TRV values due to the conversions for different receptors is perceived to be more demanding upon reviewers and more difficult to explain to nonspecialists.

Other groups are attempting to clarify the expected handling of scaling factors, with a California state agency taking the interesting compromise position in guidance that allometric scaling conversions to TRVs should be used when the body mass differences between the test species and receptor species are at least 2-fold, but these conversions should not be used when the body mass difference is less than 2-fold (DTSC 1999). The new complications created by this policy (e.g., differences of opinion about whether and how to determine when species actually differ in body mass by a factor of 2) illustrate the challenges faced by ecological risk assessors.

TRV conversions, where used, are typically done for mammalian and avian receptors, but not for fish and other aquatic receptors. For mammals, the scaling exponent commonly assigned for ERA purposes is 0.75 (Sample and Arenal 1999). Historically, similar body mass scaling factors were often assumed to apply for birds as well, and the same scaling exponent has been used for birds and mammals in many ERA applications. A specific evaluation of avian dose-response characteristics, however, indicates an interesting and significant difference. For birds, scaling factors may often be above, rather than below, a value of 1 (Mineau et al. 1996), implying that smaller avian species are relatively more sensitive than larger species per unit body mass. This is the opposite of the rule of thumb assumption related to mammals, where larger species are expected to have more sensitive dose responses. Broad-based consideration regarding the applicability of a revised avian scaling exponent by the agency could be helpful.

Site-specific information to derive exposure concentrations

The U.S. EPA staff paper touches on the significance of adequately representing site-specific exposure point concentrations (see section 6.3.4). However, the staff paper does not go beyond acknowledging that spatial and temporal elements of habitat use are important and that distribution characteristics for environmental parameters can be used to refine exposure estimates. Also, the staff paper places such refinements in the category of “site-specific” information, missing the opportunity to point out that the same type of distributional considerations may be relevant in product/chemical-specific assessments and in non-site-specific assessments done in support of generic water quality criteria.

The current approach to characterizing exposure point concentrations carries uncertainties related to its non- ecological heritage. The limited discussion in the staff paper suggests that improved approaches to account for the uniqueness and complexities of defining exposure point concentrations relevant for ecological receptors are not currently a priority for the U.S. EPA. Future dialogue on ERA should include consideration of additional approaches for characterizing exposure point concentrations that account for ecological factors. For example, exposure can be influenced by the amount of dissolved organic matter present in water (Kim et al. 1999); by mineral characteristics (e.g., acid volatile sulfide/simultaneously extractable metals; Ankley 1996); types of organic carbon in sediments (Ghosh et al. 2003); and aging of chemicals in the environment (Alexander 2000). This points out the importance of understanding site conditions when selecting information for ERA use.

The relevance of defining spatial characteristics specific to ecological exposure has been discussed for several years in the literature (Suter 1993; Hope 2000; Landis and McLaughlin 2000). Also, computerized mapping tools and spatial analysis have been applied to ecological risk topics to such an extent that a useful evaluation incorporating spatial refinements into guidance has been published (Woodbury 2003). These types of tools allow receptor-specific exposure point concentrations to be calculated based on their use of selected areas. There is clearly adequate scientific groundwork to begin discussing how and when in-depth spatial analysis could be added to comprehensive ERA.

Another refinement that could be considered is calculating exposure point concentrations differently for chemicals that bioaccumulate versus those that reflect predominantly direct exposure risks. Currently, exposure estimates for bio-accumulative chemicals do not typically account for varying conditions over the life cycle of receptors. Exposure is assumed to occur at a sensitive life stage at a site-relevant exposure point and to continue consistently. For bioaccumulative compounds, this is clearly a simplification of how long-term exposure accrues. The typical rationale for this approach is that a protective assessment must consider that the sensitive life stage could occur while receptors are at the exposure point. An approach that “accumulates” long-term exposure accounting for concentrations in different areas at different life stages would represent a substantial technical refinement and could lead to improvements in the predictive qualities of ecological risk assessments. Consistent with typical tiered approaches to ERA, ecologically optimized exposure estimation could be identified as an available refinement tool for specific assessments where this level of detail warranted.

Bridging the gap between screening and comprehensive ERA approaches

Per the U.S. EPA directed 8-step ERA process (USEPA 1997), screening-level comparisons are carried out at step 2, prior to describing the site-relevant food web, habitat characteristics, and other site use/disturbance factors that affect receptor use. Screening typically entails comparing the maximum detected concentration of each constituent analyzed in relevant environmental media to generic, default criteria (e.g., Water Quality Criteria). The decision structure incorporated in the guidance calls for progression to a comprehensive characterization of ecological factors if a maximum-detected site concentration exceeds the corresponding screening value. The result is that ecological risk screening is completed without any real consideration of ecological information.

The screening step per se, however, is not the cause of frustration. Rather, there is no clear provision for revisiting the screening level comparisons after site-relevant characteristics such as habitat quality, home range size, or human disturbance are incorporated. The next chance to conclude the ERA process does not come until after comprehensive exposure analysis, prey characterization, and risk characterization have been completed. This leaves risk assessors spending unwarranted time and effort characterizing “postage stamp” sites and habitat that is clearly human controlled (e.g., drainage features). Many risk assessors have expressed frustration at dealing with these types of situations and have sought a refinement to the process that identifies an additional pathway to a decision point early in “Problem Formulation—Step 3.”

The U.S. EPA Region 4 staff provide a useful clarification of the 8 step ERA process (Simon 2000). Although readily available on the Internet and used throughout U.S. EPA Region 4, this clarification has not received broad discussion among practitioners, and many agency staff in other regions appear not to be aware of it. U.S. EPA Region 4 (Simon 2000) specifies subdividing step 3 of the ERA process so that screening level calculations can be revised based on parameters such as site-specific area use factors that may indicate only a small proportion of foraging could reasonably occur at the site. The refinements also allow consideration of the nature or quality of habitat present and provide for submittal of an assessment encompassing screening plus additional considerations prior to the most intensive aspects of comprehensive ERA:

The U.S. EPA Problem Formulation [i.e., step 3] is commonly thought of in two parts: step 3a and step 3b. Step 3a serves to introduce information to refine the risk estimates from steps 1 and 2. For the majority of sites, ecological risk assessment activities will cease after completion of step 3a. At many sites, a single deliverable document consisting of the reporting of results from steps 1, 2, and 3 a may be submitted. At those sites with greater ecological concerns, the additional problem formulation is called step 3b.

It is very important at this stage to perform a “reality check.” Sites that do not warrant further study should not be carried forward. (Simon 2000)

The U.S. EPA Region 4 approach provides a mechanism for efficiently addressing sites where emerging problem formulation information makes it clear that remedial decisions will not be based on ecological risk. The staff paper could have acknowledged and supported this helpful and effective initiative by agency staff.


The overall positive influence of the U.S. EPA and its technical staff in the development of a scientific process for characterizing potential risks to nonhuman receptors and the advances that have marked the last 15 years should not be discounted. Were it not for the impetus provided by the U.S. EPA, assessment methods in the United States would be advancing more slowly. Promoting the development of advanced risk assessment methodologies will support more sensible, better targeted environmental management decisions.

With regard to ERA, the staff paper is clearly not a culminating chapter. However, it does serve to consolidate agency positions in one document and motivate technical dialogue. Follow-up, scientific findings, and regulatory refinements now require input and participation from interested parties beyond just the agency staff and management. For ecological risk assessors, topics such as incorporating population- and community-level assessment tools, improving exposure estimation, and expanding the ERA screening process represent areas where further advances could promote more effective, and cost-effective, site management.