SEARCH

SEARCH BY CITATION

Keywords:

  • Ecological risk assessment;
  • Ecological impact;
  • Risk;
  • Hazard quotient;
  • Field-truthing

Abstract

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES

A frank assessment of present-day ecological risk assessments (ERA) for managed contaminated sites reveals that fundamental concepts regarding the receptors that are considered and the chemical exposures they experience are commonly misapplied. As a consequence, environmental managers are not being supplied with the information needed for proper decision making. The stepwise review of ecological risk issues provided here suggests that the ERA process needs to be severely revamped. Further, what is likely hindering the development of a refined ecological assessment process that is better suited to environmental problem solving and land management is the unwillingness of stakeholders to agree that much of the current ERA practice and convention is flawed.


ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES

Ecological risk is the probability that a receptor of concern will develop or may have already developed one or more toxicological endpoints of concern as a result of its interacting with one or more contaminated environmental media (e.g., soil) (USEPA 1998). Considering that the overwhelming majority of sites where ecological risk assessments (ERAs) are conducted have been contaminated for decades (Tannenbaum 2002, 2003a), it is fair to ask if we are truly out to estimate the chances that the site ecological receptors might develop health problems sometime in the future. At the heart of this question, it is recognized that ecological receptors in the main have lived for 10 or more generations (and in many instances, more than 100 generations) at the affected site by the time an ERA is proposed. This is quite unlike the case for human-health risk assessments (HHRAs), where the objective is to evaluate the chemical exposures that newcomers to a site will experience and that they could continue to experience for a considerable portion of their lives (Tannenbaum 2003a). The ERAs and HHRAs differ radically for a more dramatic reason as well. For HHRAs, where contaminant exposures are estimated to be unhealthful, we interrupt the exposure pathway in some fashion (e.g., ground-water production wells are taken off line) with the hope that an unwanted toxicological response can be averted within the very lifetime of the actual or anticipated receptor.

In retrospective ERAs (Callow and Forbes 2003), where multiple generations have already lived amid the contaminated condition, clearly, there is no objective of averting unwanted outcomes within individual lives (Tannenbaum 2003a). It's far too late for that. This striking difference, the duration of chemical exposures that are considered in the two types of assessments, gives us pause to ask several other questions:

  • How legitimate is an expressed concern that the rabbit, deer, or songbird populations at a 40-year-old contaminated site, e.g., are potentially going to become harmed, if not severely reduced or decimated, from the soil contaminants that remain to this day?

  • If it is suspected that soil contaminants have the potential to trigger health effects, then shouldn't those effects have already come about?

  • If it is suspected that the site contaminants have the potential to wreak severe effects on the animal populations, then why are the populations still present after multiple decades of continuous exposure to the contaminants?

There do not appear to be any instances where birds or mammals at contaminated sites demonstrated overt signs of stress or impact as a result of chronic chemical exposures (although they may have demonstrated evidence of chemical exposure) (Tannenbaum 2002, 2003a). If this is what the scorecard has to show after nearly two decades and hundreds, if not thousands, of ERAs conducted, the chances of stumbling onto a first instance of demonstrated harm at a terrestrial site should be next to nil. It would seem prudent, at this time, for ecological risk assessors and risk managers to acknowledge the apparent pointlessness in endeavoring to forecast the likelihood of toxicological effects arising at contaminated sites, and further, to rethink the question they believe needs to be answered.

Acknowledging a concept of a statute of limitations for the development of toxicological effects could be a starting point for revamping the ERA process. In other words, we should recognize that sites with multiple decade-old contamination have had more than ample opportunity to evoke toxicological effects in site receptors, and we should admit that the prospect of effects first arising at some point still off in the future is unrealistic. Seemingly, such acknowledgements would suggest that a paradigm shift from ecological risk assessment to ecological effects assessment or ecological impact assessment is sorely needed.

ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES

The U.S. Environmental Protection Agency's (U.S. EPA's) Superfund guidance for risk assessment (USEPA 1989b), the quintessential HHRA guidance document and the one upon which ERA practice is patterned, clearly notes that the hazard quotient (HQ; the ratio of an animal's estimated chemical intake relative to an effect- or no-effect-level dose) is not a risk measure. There is no shortage of references in the open literature that also acknowledge this point (Kolluru 1996; Tannenbaum et al. 2003b). Nevertheless, the HQ is routinely computed today with its values of varying magnitude said to correspond to different levels of ecological risk (Menzie et al. 1992). Aside from the mystery of why the U.S. EPA's acknowledgement of this critical HQ limitation goes unheeded by professionals in the ERA field (including those within the U.S. EPA), there are dramatic consequences for the ERA process (more so than for the HHRA process) (Tannenbaum et al. 2003b). The most far reaching of these is that ERA, from its inception until the present day, remains devoid of a risk assessment tool. A greater tragedy may be that ecological risk assessors and risk managers do not seem to have this realization.

What are the effects of failing to recognize that the HQ does not measure risk? Let us begin with a rather benign effect and then advance to the more severe. Individuals employed under the title of ecological risk assessor are, in fact, not assessing ecological risk. The statistics they generate are not probabilities of birds or mammals displaying toxicological endpoints in the future. (The proverbial HQ of five that the assessor might report does not mean that there is a 5% chance that a receptor will develop an endpoint of concern.) A more appropriate job title for these individuals would seem to be hazard quotient calculators or level-of-concern measurers (USEPA 1989b).

We should be concerned with this matter of wrongly terming our professional working title because it deludes us into thinking that we are largely doing all that we could regarding ERAs at contaminated sites. To date, we have never produced an ERA for a contaminated site, although the rank and file of assessors think otherwise. To date, we have only screened contaminated sites, and it appears that every ounce of effort and every research dollar continues to be applied to only do more of the same. Thus, because we fail to note that the HQ is only a measure of a level of concern (USEPA 1989b), our thinking is that all that remains to be done to make ERAs more robust is to refine our system of computing HQs.

A rather clear demonstration of this is the Army Risk Assessment Modeling System (USACE 2004). The Army Risk Assessment Modeling System boasts an ultra-user-friendly desktop application, where, with the ability to access a multitude of databases, one can easily assemble all the elements of a conventional food-chain model and produce a customized version on a computer screen within a matter of minutes. The versatility of the application, coupled with simulations that take only a matter of seconds to run, fosters confidence that the model's singular output, the HQ, is all that is needed to assess ecological risk.

True inroads into recognizing that we are falling short regarding ERAs at contaminated sites might proceed from the regular inclusion of an expanded treatment of HQ method limitations in the uncertainty sections of ERAs. Acknowledging in uncertainty sections that HQs may be under- or overestimates ignores the seminal point that the HQ is far from the ideal statistic. Such treatments would more than identify the limitations by name, but would also explain how ill-prepared the decision maker is in the aftermath of HQ calculation because of the method limitations. A consideration of some HQ limitations that are not commonly known or acknowledged in the literature would further underscore how far removed from actual risk assessment we are when we assess sites and base remedial decisions on HQs.

First, the hazard quotient is so named for a good reason; it measures hazard and not risk. If it truly were a risk-measuring device, it would be called a risk quotient. One would think that this difference in terminology would drive home the critical point.

Second, the HQ is not linked to a temporal scheme. A HQ of 5 means the same thing for a site with contamination that is 5 years old as it does for a site with contamination that is 5,000 years old.

Third, although risk assessors do acknowledge that HQs can often produce values that are too large, the possibility of HQs being underestimated is never mentioned. A consideration of our limited abilities to evaluate the chemical exposures of a red fox, for example, when applying the HQ method, illustrates how potentially nonrepresentative a HQ may be in articulating the ratio of the actual chemical intake to an assumed safe dose. The toxicity reference value for the fox assessment will more than likely derive from a rodent study that was conducted in a laboratory, on a syngenic strain of animal, and where the form of the chemical tested almost assuredly was not that which is present in the fox's diet. In addition, the lab study's test concentration will be different than that in the fox's diet. Finally, where the controlled lab study will probably not have run for more than 270 d, the fox population has been exposed in a highly variable environment (e.g., ambient temperature) for multiple generations. Considering these extreme differences, it would appear to be irresponsible to conclude that HQs, at their worst, can only be overestimations.

Given the above analysis, we would do well to reassess our overextended allegiance to the HQ. When we are at our most obstinate, we often adopt the position that the only way to demonstrate that a receptor is protected is with a desktop analysis that produces a HQ of 1.0 or minimally above 1.0. We must realize that we are thwarting the development of the ERA field when we belittle all other attempts to demonstrate receptor protection, particularly when these attempts involve innovative field-based methods.

ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES

In HHRA or ERA, preliminary remediation goals (PRGs) are those contaminant-specific concentrations in a medium (such as soil) that should confer a level of protection for the receptor that contacts that medium. In theory, PRGs need to be developed after it has been demonstrated that the current contaminant concentrations will likely pose unacceptable health risks to the receptor. PRGs are intended to indicate how extensive a cleanup may need to be, and often, in practice, PRGs are taken to be the site's actual recommended cleanup numbers. An unavoidable complication for ERA follows, however, from the fact that HQs are not risk measures. Given that we do not know that receptors are at risk when an HQ exceeds 1.0, that HQs do not provide a justification for site remediation (Tannenbaum et al. 2003b), and that ERAs do not ordinarily progress beyond the HQ calculation, we are never in a position to calculate so-called eco-PRGs. Further, until ERAs do progress beyond the HQ to a point where we are in fact expressing risk, every instance of eco-PRG development today is a premature one.

Those who proceed to develop eco-PRGs based on HQs that exceed 1.0 are committing mathematical errors in addition to believing that they have a valid demonstration that the contaminated site requires remediation. The HQs are not linearly scaled values (USEPA 1989b; Kolluru 1996; Tannenbaum et al. 2003b) except as they occur in fortuitous cases. Thus, there is no technical basis for the commonly employed approach of generating safe soil concentrations through back-calculation. In the simple case where a soil contaminant has a HQ of 100 for a given receptor, dividing the soil concentration by 100 to arrive at the soil concentration that corresponds to the HQ-of-1.0 condition is an erroneous computation (even in those instances where contaminant transfer factors or other concentration-dependent multipliers are not used in the food-chain model). Back-calculation is also founded on a false assumption, namely that only a HQ of 1.0 is safe. There are countless examples of contaminated sites with exorbitantly high HQs (i.e., those of 4, 5, or 6 digits) and where the site ecology has been described as a healthy, thriving community. Finally, HQ back-calculation overlooks one other key HQ method limitation, namely, its tendency for generating unrealistic values. As a cautionary note when faced with a HQ of 500 at a contaminated site, sooner than setting out to derive a PRG, one should realize that a receptor could not possibly be chronically consuming a chemical at 500 times the safe dose.

ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES

Organisms exposed to contaminated sites are likely to manifest contaminant concentrations in their bodies, and there are several exposure routes that allow this to happen. Moreover, it is a reasonable expectation that, if one were to collect specimens of a given species from both a contaminated site and its matched reference location, the higher chemical detections would be found in the biota of the contaminated site. (If such an arrangement is not found, one might want to question the abilities of the analytical laboratory involved.) Although species such as earthworms and small rodents are commonly collected for the purposes of highlighting potentially significant body burden differences where they might exist, we should stop to ask ourselves if such information is providing any value to ERAs.

First and foremost, if the intention is to suggest that the onsite receptor, with its greater body burden, is at risk if not already seriously harmed, then there is a grave flaw in our pretense. The fact remains that, presently, there is no ability to substantiate such claims because nearly all toxicity testing is of the administered dose genre. Consequently, the current toxicological databases almost never permit one to establish linkages of body burden and effect (although to a very limited extent, we are beginning to relate the two in certain aquatic species [USACE/USEPA 2004]).

Insisting that higher body burdens of receptors at affected sites are clear-cut indications of compromised health is nothing less than establishing guilt by association. One is arguing that, because the compound is physically present somewhere in the body, the receptor is being harmed. But is guilt by association the standard being employed to clean up sites? Certainly not! When a site with unquestionably contaminated soil is encountered, it is not automatically concluded that there must be a cleanup. Instead, a risk assessment process is employed, and it is only after doing so and after identifying a potential problem that a site remedy is implemented (Tannenbaum 2003b).

Recalling that virtually every site where an ERA is conducted is a historically contaminated one puts the body burden matter into perspective. If, in fact, significantly higher tissue concentrations in on-site worms or deer mice were measured, it would be naive to think that the cohorts collected were the very first to manifest noteworthy concentrations. Because the elevated tissue concentrations are far from new, the body burden data could be better used to indicate that, despite the atypical tissue levels, the animals are otherwise functioning normally. Until a certain chemical concentration in a rodent's blood, femur, liver, or whole-body analysis is known to equate with an increased chance for the development of a harmful effect, body burden data are unnecessarily being collected.

In ERA, chemical body burden analysis is conducted for reasons other than the intention of linking tissue concentration with effect in the carrier organism. The most common reason for pursuing such data is to support food-chain modeling that intends to assess the chemical exposures of higher trophic-level species (e.g., fox, hawk). Although it is true that the chemical intake to the worm or rodent consumer is better known when the dietary items are directly analyzed for chemical content (than by modeling up from a soil concentration), it must be recalled that food-chain modeling exercises terminate only in HQs, which, as mentioned above, are not risk measures. Thus, the net gain of estimating or directly measuring body burdens in dietary items is questioned.

Sometimes the stated objective for measuring chemical tissue burden is to support inferences about chemical uptake in deer. The curiosity here is that comprehensive studies of deer contaminant uptake already exist and are commonly cited in ERAs. These studies prominently note, e.g., that metals and explosives do not manifest themselves in organs, muscle, and other tissues (USAEHA 1994; USACHPPM 1995). Every opportunity to advise against what will otherwise be unnecessary data collection at the expense of harvesting field biota should be sought.

A final concern is that case where no apparent chemical uptake has occurred based on a comparison of tissue levels in specimens collected at both the site of interest and the matched reference location—a situation that is ripe for misinterpretation. One should not be so naive as to conclude, in such instances, that on-site animals are healthy. The possibility that chemicals of interest do not bioaccumulate in tissues should not be discounted. Conceivably, certain chemicals of interest have relatively short half lives in the body, and, though not manifesting themselves to significant concentrations, may nevertheless cause toxic effects that are of concern.

ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES

Although ecological risk assessors do not like to admit it, small mammals (rodents and insectivores) such as mice, rats, voles, and shrews do not drive cleanups. This is true despite the near guarantee that one or more species of this grouping will be present at each terrestrial site of interest and, invariably, at least one small rodent species will be evaluated in every conventional HQ-based ERA. The fact remains that anticipated ill health in small mammals rarely, if ever, triggers site remediation, a situation no different from that of the earthworm. The only exception to this would be that extremely rare case where a protected-status small rodent species at a contaminated site is anticipated to be experiencing adverse effects.

What about larger mammals that could drive a cleanup if the relevant information suggested that they were being imperiled? A matter very much overlooked is that many, or most, of these routinely evaluated species are not spatially relevant to the site in question due to either their natural density, their lateral movements (homing), or both. To appreciate this, the sizes of managed sites that go through an ecological evaluation process, regardless of the regulatory program governing them, need to be considered. One statistic that puts the matter into perspective is that, in the United States, 59.5% of National Priority List sites are smaller than 20 acres (USEPA 1989a). In conjunction with this fact is the qualifier that not all of a listed site's acreage reflects contamination that is in soil (e.g., a migrating ground-water plume may account for much, if not all, of the site's potentially or actually contaminated acreage). It is equally important to note that, although some sites (whether National Priority List-listed or not) may be enormous in size, such as Department of Defense installations that cover thousands of acres, these sites are often compartmentalized such that terrestrial ERAs occur at land parcels that are orders of magnitude smaller than the overall site.

If hypothetical 50- and 100-acre sites are considered and the readily available literature for values of average animal density are used, we will not often be able to justify the inclusion of many of the commonly evaluated mammals (as opposed to special-status species) in ERAs (Table 1). This exercise makes it apparent that, too often, there will be an insufficient number of animals (of not just one species, but perhaps, all those mammal species described for the region where the contaminated site is located) to warrant an ERA. Recalling that a typical terrestrial site is much smaller than 50 acres makes the demonstration that much clearer.

Through the use of readily available information, it can be demonstrated in an alternate way that most mammals (i.e., other than special-status species) are probably not sufficiently contacting contaminated sites to warrant an evaluation. As Table 2 demonstrates, based on the average literature-reported home range, direct contact with an affected site is marginal at a 50-acre site. Thus, the justification for evaluating these mammals at the commonly encountered 5-, 10-, or 20-acre site is not demonstrated. In short, is there any reason to evaluate a higher mammal where the site of interest constitutes 1 or 2% of the animal's home range? Could a case be made that an area contacted by a higher mammal less than 5% of the time is one that should be remediated?

What emerges from the cursory analysis above is that, with minor exception (e.g., certain rabbit species), there are no mammal species that are suited to terrestrial ERAs; justification for not including mammals in receptor-of-concern lists reflects either too few species representatives being present or the site constituting an infinitesimal portion of the animal's home range. It would seem that a spatially relevant screening step is sorely needed at the beginning of the ecological evaluation process. By incorporating this necessary step, appropriate mammal receptor-of-concern lists in ERAs are likely very short or nonexistent. Unfortunately, this necessary screening step alone will not allow for improvements in our ERAs. Stakeholders also need to be informed that there is no absolute requirement that every terrestrial ERA include at least one mammal. It is a perfectly reasonable outcome that a proper spatial relevance screening may reveal that not a single mammal species qualifies as legitimate for evaluation at a given site.

Table Table 1.. Number of animals expected to be present, based on average reported densitiesa
Mammal50-acre site100-acre site
  1. a Sources used: Burt and Grossenheider (1980), Damath (1987), Chapman and Feldman (1992), USEPA (1993), CH2MHill (2001).

Black-tailed jackrabbit2.55
Coyote< 11
Long-tail weasel1.53
Mule deer24
Raccoon< 1< 1
Red fox< 1< 1
White-tailed deer25

Until this critical education occurs, we will continue to see rather preposterous ERAs. One extreme example was where the house mouse (Mus musculus) was selected for evaluation because it was the only site mammal species present. Although the site boasted an active mouse eradication program involving setting out poison, clearly demonstrating that the house mouse did not satisfy the terms of an assessment endpoint (i.e., a valued receptor that is to be protected [USEPA 1998]), the mouse was selected because the stakeholder was unfamiliar and uncomfortable with conducting an ERA where not a single mammal would be considered.

ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES

Ecological risk assessment guidance recommends comparing model results with field data as a means of providing a check on whether our understanding of the system under evaluation was correct (USEPA 1998). This recommendation acts as a reminder that model results alone cannot prove that ecological receptors are being harmed. Commonly, though, the approach taken to implement the recommendation is to collect field data to be used in modeling (with the use of HQs) a receptor's exposure.

Table Table 2.. Percentage of an animal's average home rangea that is occupied by a site
Mammal50-acre site100-acre site
  1. a Source used: Damath (1987).

Coyote0.270.54
Kit fox612
Long-tail weasel18.236.4
Mule deer2550
Red fox5.210.4
White-tailed deer10.320.6

A familiar example would be improving a rabbit's estimated dietary intake of a contaminant by analyzing plant matter for tissue burden, whereas previously, the plant's contaminant concentration was estimated by modeling from contaminant concentrations in the soil. In fewer instances, the recommendation is taken to mean that standard U.S. EPA-sanctioned and American Society for Testing and Materials-approved toxicity tests should be run using site soils. Rather routinely, earthworm tests that evaluate growth as an endpoint or plant tests that evaluate seed germination or root elongation as endpoints are conducted. Although these tests have a field character to them, we are hard pressed to say that they satisfy the guidance by “providing a check on whether the understanding of the site was correct.”

The following example demonstrates this point. If a cottontail rabbit population has an HQ well above 1.0 for a reproductive effect, an unresolved question remains, “Is the rabbit population reproductively impacted?” What should be clear is that standard toxicity tests such as those mentioned here cannot shed any additional light on the unresolved question. How could these tests possibly supply the needed weight-of-evidence information to address the unresolved question when they involve species of the wrong phylum (earthworms are invertebrates) and the wrong phylogenetic kingdom (plants)?

It would seem that ecological assessments can only get on track, meeting the guidance's recommendation of “checking to see that our understanding of the site was correct,” by directly assessing receptors in the field. Presently, this is a prospect that the regulatory community has yet to embrace (Tannenbaum et al. 2003a). Consequently, the only efforts to refine ERAs involve making adjustments to desktop modeling and incorporating bioavailability studies. Presumably, the perception that appropriate field verification endeavors are too difficult and/or too costly to devise and conduct and the lack of available guidance on how to devise an appropriate field method are dissuading stakeholders from making first attempts. The U.S. Army's recent development of an inexpensive yet rather definitive field-truthing test, however, indicates that the task is not nearly so daunting (Tannenbaum 2001, McDonald and Wilcockson 2003).

ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES

Ecotoxicologists and risk assessors are primed to think in terms of two groups of animals vis-à-vis toxicological responses. In a laboratory setting, the focus is on the treated animals and the controls. In a field study, the focus is on the animals at the contaminated site and the matched reference location. The fact is that, where a statistically significant difference is observed, we are allowed to claim that the two groups actually represent different populations. A complication almost universally overlooked, however, is that, although the animals exposed to the toxins are statistically different from their counterparts, it is unknown if they are health compromised in any way. Thus, if a treated group of rats has a statistically higher enzyme level, a larger liver, a shifted blood pH, or shifted phenotypic frequency of a certain hair color, it is unknown if the exposed group will live a shorter life, run more slowly, select inadequate food, fail to find a suitable mate, or fail to produce as many viable young as the nonexposed rats (Tannenbaum 2001).

Although laboratory study-based discoveries of differential toxicological responses are appropriate matter for publication in the peer-reviewed scientific literature, such findings may not be assisting in ecological assessments at all. The findings can potentially lead risk assessors far astray because they give rise to the situation where observed differences are labeled as adverse responses to toxins (Tannenbaum 2001). Putting aside the HQ method's many limitations, is there really a need to know that there is a possibility that the rodents running around at a contaminated site have livers that are statistically larger than those of their reference location counterparts? Should a site be remediated because on-site animals are assumed to have statistically larger livers?

The above leads to the question, “How much of an absolute difference in a biological measure is meaningful within an ERA framework? “ To date, the question remains essentially unanswered, although the figure of 20% (with minimal empirical supports) is often suggested (Suter et al. 2000). This figure is considered to be the difference that supersedes the natural variability that will exist for any dataset of biological information (e.g., wing length or respiration rate) and that can be attributable to the site's stressors. The matter for ecological risk assessors to squarely address becomes one of how much trust to put behind a measured difference of 20% or more. The challenge is extreme because, for virtually every biological measure, it is not known how much of a difference truly correlates with a health-compromised condition.

A classic exception to this, and one that has given rise to the only existing field-truthing ecological assessment method, rodent sperm analysis (Tannenbaum et al. 2003a), is the three sperm parameters established as barometers of reproductive success in rodents and other mammals. Further, the rodent sperm analysis test demonstrates how misleading the commonly applied 20% guideline can be. Specifically, an 80 to 90% sperm count reduction is needed before a claim of compromised reproductive success can be made (Bucci and Meistrich 1987; Gray et al. 1992; Meistrich et al. 1994), and conversely, an increase in abnormally shaped sperm of only 4% over the control rate allows for the same claim. Considering these great deviations from the 20% guideline, it would seem prudent to exercise extreme caution when interpreting absolute differences for all other biological measures. Seemingly, a very necessary task for the ecotoxicologist is to develop rather precise effects benchmarks, akin to those of the rodent sperm analysis method, for the actual species that inhabit contaminated sites of interest.

ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES

Imagine the case of a dedicated field team wanting to know if the red fox population at a sizable 50-year-old contaminated property is suffering any ill effects from its chemical exposures. Based on the high-quality data collected over 11 years, the team compiles what is reputedly the most comprehensive fox population study known to mankind involving some 40-odd conventional measures and a half dozen novel metrics. The results convincingly show that the population is healthy and stable and possibly even faring slightly better than the populations studied at several exquisitely matched reference locations. All stakeholders readily agree that their concern over the fox can be put to rest.

Two years later, a doctoral student working on his dissertation is drawing blood samples from red fox at the contaminated site and also from fox that occupy some of the reference locations used in the earlier study. His analysis shows that the red blood cell count is markedly reduced in the site animals and that the cell nuclei of the site animals are dramatically deformed. Further, three of the five enzymes he assays are notably higher in the blood samples of the site foxes than in the reference animals. The other two enzymes are vastly underproduced in the site foxes. The student's findings become known to the stakeholders of the site that was closed out on the basis of the earlier comprehensive population study. Should the red fox health assessment at the contaminated site be reopened? Would this question be any different if the doctoral student had conducted his work 10 years after the otherwise convincing population study was concluded and not two years after?

I believe the correct answer is that the fox health assessment should not be reopened, although certain readers almost assuredly will answer differently. This case highlights a critical point, namely that stakeholders do not have a common focus in their valuation of data. The regulating community, e.g., in classic fashion, would pounce on the new discovery, insisting that only now is a fuller picture of fox population health emerging. The responsible party would argue that the newfound information does not impact in any way on fox population health. They would also argue that the tissue and enzyme findings are not new phenomena. If someone had bothered to evaluate the cells and the enzymes at the time of the population study, then the differences would have been apparent at that time.

To improve ERAs, I suggest that we learn to discriminate between findings that concern the inside of the animal and those that concern the outside (such as physical appearance and population features), and further that we realize that it is only the outside of the animal that matters. In this case, if the fox population is doing everything it should despite the contaminated site condition (and indeed it is, with the data clearly indicating that the species is being perpetuated), then the enzyme levels of on-site animals or the cell nuclei not being shaped as they would appear in a textbook are unimportant. These measures, although undeniably atypical, are at best indicators of exposure. They are not indicators of impact and cannot be so until such time as we become aware of how much of a reduced red cell count or how much of an altered cell nucleus shape is needed in the specific species we're evaluating to confer an assessment of poor health in that animal.

What if the dissertation never took place? Is the true picture being missed because each and every tissue measure one could study is not being assessed?

There is every chance in the world that animals roaming contaminated sites have altered behaviors that are not studied or compromised endocrine systems for which definitive tests are lacking. It is also possible that every internal organ is deformed. The reality is that we will never have the opportunity to conduct internal examinations that could identify such changes in the receptors that concern us. The critical point here is that the greatest outer (outside) manifestation of the health of a species is its reproductive performance. As long as reproduction is not compromised in contaminated site receptors (the anticipated outcome), there is no need to be concerned with any other endpoint. A population that is reproducing normally is demonstrating openly that animals are able to live to the age of reproduction, find mates, go through a courtship ritual, and produce viable young. In such a case, there is no need for additional assessment information.

Perhaps the best way to demonstrate the superior value of reproductive success data is with another hypothetical case. Imagine that there is an exquisitely working diagnostic test for neurotoxic effects and that, at a site that has lead (a known neurotoxin) present among several other contaminants of concern, the test has clearly indicated that the receptors are not neurologically compromised. With this knowledge, the assessment work is not complete. Invariably, stakeholders will comment to the effect that, although there are no neurotoxicity issues over which to be concerned, reproduction (reputedly the toxicological endpoint of greatest concern in ERAs) has been overlooked. Lead and several other site contaminants of concern are reproductive toxins, and further, even in the absence of known reproductive toxins, any two compounds could act synergistically to trigger reproductive effects. The salient point is that, whereas the outcome of the neurotoxicity testing was insufficient to close out the site (but rather required reproduction to be evaluated), the situation is reversed where a valid reproduction assessment is first applied. If reproduction can be shown to be unimpaired, it is highly unlikely that a stakeholder will subsequently ask that another endpoint, such as neurotoxicity, be assessed.

A practical and daily demonstration of reproduction's dominant role is evident in the selection of assessment endpoints within ERAs, in concert with certain ERA outcomes relative to those endpoints. Almost always the stated endpoint is “a reproductively successful population.” Although by no means an endorsement for HQ usage on the part of this author, it is true that, in that case where a reproduction-based HQ is below 1.0, an ERA can be put to bed without a stakeholder request that a different endpoint-based HQ exercise proceed.

SUMMARY

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES

In this commentary, several misapplied concepts and erroneous assumptions that are common to ERAs are presented. Following the established ERA guidance will impede sites from moving through the process they need to follow to satisfy regulatory requirements. Complacency with the ERA process, as it stands today, is a formidable deterrent to the development of (impact) assessment schemes that can (truly) indicate the health status of site receptors and that can better ensure that decision making is technically supported. A willingness on the part of the regulating community to recognize that ERA practices need to change, and perhaps in a radical fashion, will allow the discussions provided here to set the stage for further ERA improvements.

REFERENCES

  1. Top of page
  2. Abstract
  3. ISSUE 1. “RISK” IS THE ENTIRELY WRONG MEASURE TO BE PURSUING
  4. ISSUE 2. EVEN IF RISK WERE THE APPROPRIATE MEASURE, WE ARE NOT CALCULATING IT
  5. ISSUE 3. THERE HAS YET TO BE A NEED FOR ECO-PRELIMINARY REMEDIATION GOALS AND WE HAVE NO WAY TO DERIVE THEM ANYWAY
  6. ISSUE 4. A CHEMICAL BODY BURDEN DOES NOT INDICATE THAT A RECEPTOR IS AT RISK
  7. ISSUE 5. IN ALMOST ALL INSTANCES, THERE ARE NO SPATIALLY RELEVANT MAMMALS TO EVALUATE
  8. ISSUE 6. WE RARELY USE FIELD STUDIES FOR THE VALIDATION OF SUSPECTED EFFECTS, AND WHEN WE DO GO TO THE FIELD, IT IS TO CONDUCT TESTING THAT WILL NOT ADDRESS THE CONCERN
  9. ISSUE 7. WE TEND TO OVERSTATE STATISTICALLY SIGNIFICANT FINDINGS
  10. ISSUE 8. PURSUING TOXICOLOGICAL ENDPOINTS OTHER THAN REPRODUCTION IS LIKELY A WASTED EFFORT
  11. SUMMARY
  12. Acknowledgements
  13. REFERENCES
  • Bucci LR, Meistrich ML. 1987. Effects of busulfan on murine spermatogenesis: Cytotoxicity, sterility, sperm abnormalities, and dominant lethal mutations. Mutat Res 176: 259268.
  • Burt WH, Grossenheider RP. 1980. A field guide to the mammals. Boston (MA), USA: Houghton.
  • Callow P, Forbes VE. 2003. Does ecotoxicology inform ecological risk assessment? Environ Sci Technol 37: 146151.
  • CH2MHill. 2001. Development of terrestrial exposure and bioaccumulation information for the army risk assessment modeling system (ARAMS). Aberdeen Proving Ground (MD), USA: U.S. Army Center for Health Promotion and Preventive Medicine.
  • ChapmanJA, FeldhamerGA, editors. 1992. Wild mammals of North America: Biology, management, and economics. Baltimore (MD), USA: Johns Hopkins University Press. 1147 p.
  • Damuth J. 1987. Interspecific allometry of population density in mammals and other animals: The independence of body mass and population energy-use. Biol J Linn Soc 31: 193246.
  • Gray LE, Marshall JO, Setzer R. 1992. Correlation of ejaculated sperm numbers with fertility in the rat. Toxicologist 12: 433.
  • Kolluru R. 1996. Health risk assessment: Principles and practices. In: KolluruR, BartellSM, StricoffS, PitbladoR, editors. Risk assessment and management handbook for environmental health, and safety professionals. New York (NY), USA: McGraw-Hill. p 10.30.59.
  • McDonald BG, Wilcockson JB. 2003. Improving the use of toxicity reference values in wildlife food chain modeling and ecological risk. Hum Ecol Risk Assess 9: 15851594.
  • Meistrich ML, Kasai K, Olds-Clarke P, MacGregor GR, Berkowitz AD, Tung KSK. 1994. Deficiency in fertilization by morphologically abnormal sperm produced by azh mutant mice. Mol Reprod Dev 37: 6977.
  • Menzie CA, Burmaster DE, Freshman JS, Callahan CA. 1992. Assessment of methods for estimating ecological risk in the terrestrial component: A case study at the Baird & McGuire Superfund site in Holbrook, Massachusetts. Environ Toxicol Chem 11: 245260.
  • Suter GW, Efroymson RA, Sample BE, Jones DS. 2000. Ecological risk assessment for contaminated sites. Boca Raton (FL), USA: Lewis. 464 p.
  • Tannenbaum LV. 2001. What's so bad about weight loss, blood chemistry effects, kidney toxicity, etc. in a modeled ecological receptor? Hum Ecol Risk Assess 7: 17651767.
  • Tannenbaum LV. 2002. Learned discourse: Terrestrial ecological risk assessment: Are we missing the forest for the trees? SETAC Globe 3: 3839.
  • Tannenbaum LV. 2003a. Can ecological receptors really be at risk? Hum Ecol Risk Assess 9: 513.
  • Tannenbaum LV. 2003b. The role of background in hazardous waste site risk assessments: Getting back to basics. Human Ecol Risk Assess 9: 15991607.
  • Tannenbaum LV, Bazar M, Hawkins M, Cornaby BW, Ferguson EA, Carroll LC, Ryan PF. 2003a. Rodent sperm analysis in field-based ecological risk assessment: Pilot study at Ravenna Army Ammunition Plant, Ravenna, Ohio. Environ Pollut 123: 2129.
  • Tannenbaum LV, Johnson MS, Bazar M. 2003b. Application of the hazard quotient method in remedial decisions: A comparison of human and ecological risk assessments. Hum Ecol Risk Assess 9: 387401.
  • [USACE] U.S. Army Corps of Engineers. ARAMS Army Risk Assessment Modeling System home page [online]. 2004. Vicksburg (MS): USACE. http://www.wes.army.mil/el/arams. Accessed 14 July 2004.
  • [USACE/USEPA] U.S. Army Corp of Engineers/U.S. Environmental Protection Agency. 2004. The Environmental Residue-Effects Database (ERED) http://www.wes.army.mil/el/ered/. ccessed 14 July 2004.
  • [USACHPPM] U.S. Army Center for Health Promotion and Preventive Medicine. 1995. Health risk assessment of consuming deer from Aberdeen Proving Ground, Maryland. Aberdeen Proving Ground (MD): USACHPPM. Field study 75–23-YS50–94.
  • [USAEHA] U.S. Army Environmental Hygiene Agency. 1994. Health risk assessment for consumption of deer muscle and liver from Joliet Army Ammunition Plant, Joliet, Illinois. Aberdeen Proving Ground (MD): USAEHA. Project 75–51-YF23.
  • [USEPA] U.S. Environmental Protection Agency. 1989a. NPL characterization project: Region II results. Washington DC: USEPA.
  • [USEPA] U.S. Environmental Protection Agency. 1989b. Risk assessment guidance for Superfund. Volume I: Human health evaluation manual (Part A). Washington DC: USEPA. EPA/540/1–89/002.
  • [USEPA] U.S. Environmental Protection Agency. 1998. Guidelines for ecological risk assessment. Washington DC: USEPA. EPA/630/R-95/002F.
  • [USEPA] U.S. Environmental Protection Agency. 1993. Wildlife exposure factors handbook, Vol I. Washington, DC: USEPA. EPA/600/R-93/187a.