• Decision criteria;
  • Environmental risk assessment;
  • Site remediation


  1. Top of page
  2. Abstract
  9. Acknowledgements

Decision making in environmental projects can be complex and seemingly intractable, principally because of the inherent trade-offs between sociopolitical, environmental, ecological, and economic factors. The selection of appropriate remedial and abatement strategies for contaminated sites, land use planning, and regulatory processes often involves multiple additional criteria such as the distribution of costs and benefits, environmental impacts for different populations, safety, ecological risk, or human values. Some of these criteria cannot be easily condensed into a monetary value, partly because environmental concerns often involve ethical and moral principles that may not be related to any economic use or value. Furthermore, even if it were possible to aggregate multiple criteria rankings into a common unit, this approach would not always be desirable because the ability to track conflicting stakeholder preferences may be lost in the process. Consequently, selecting from among many different alternatives often involves making trade-offs that fail to satisfy 1 or more stakeholder groups. Nevertheless, considerable research in the area of multicriteria decision analysis (MCDA) has made available practical methods for applying scientific decision theoretical approaches to complex multicriteria problems. This paper presents a review of the available literature and provides recommendations for applying MCDA techniques in environmental projects. A generalized framework for decision analysis is proposed to highlight the fundamental ingredients for more structured and tractable environmental decision making.


  1. Top of page
  2. Abstract
  9. Acknowledgements

Environmental decisions are often complex and multifaceted and involve many different stakeholders with different priorities or objectives—presenting exactly the type of problem that behavioral decision research has shown humans are poorly equipped to solve unaided. Most people, when confronted with such problems, will attempt to use intuitive or heuristic approaches to simplify the complexity until the problem seems more manageable. In the process, important information may be lost, opposing points of view may be discarded, and elements of uncertainty may be ignored. In short, there are many reasons to expect that, on their own, individuals (either lay or expert) will often experience difficulty making informed, thoughtful choices in a complex decision-making environment involving value trade offs and uncertainty (McDaniels et al. 1999).

Moreover, environmental decisions typically draw on multidisciplinary knowledge bases, incorporating natural, physical, and social sciences and medicine, politics, and ethics. This fact, and the tendency of environmental issues to involve shared resources and broad constituencies, means that group decision processes are often necessary. These may have some advantages over individual processes; in particular, more perspectives may be put forward for consideration, the probability of benefiting from the presence of natural systematic thinkers is higher, and groups often learn to rely on more deliberative, well-informed members. However, groups are also susceptible to the tendency to establish entrenched positions (defeating compromise initiatives) or to prematurely adopt a common perspective that excludes contrary information. This tendency has been termed by some as “groupthink” (McDaniels et al. 1999).

For environmental management projects, decision makers often receive 4 generalized types of technical input: the results of modeling and monitoring studies, risk assessment, cost or cost-benefit analysis, and stakeholder preferences (Figure 1a). However, current decision processes typically offer little guidance on how to integrate or judge the relative importance of information from each source. Also, information comes in different forms. While modeling and monitoring results are usually presented as quantitative estimates, risk assessment and cost-benefit analyses may incorporate a higher degree of qualitative judgment by the project team. Structured information about stakeholder preferences may not be presented to the decision maker at all and may be handled in an ad hoc or subjective manner that exacerbates the difficulty of defending the decision process as reliable and fair. Moreover, where structured approaches are employed, they may be perceived as lacking the flexibility to adapt to localized concerns or faithfully represent minority viewpoints. A systematic methodology to combine both quantitative and qualitative inputs from scientific or engineering studies of risk, cost, and benefit, as well as stakeholder views and values to rank project alternatives, has yet to be fully developed for environmental decision making. As a result, decision makers are prevented from identifying all plausible alternatives and from making full use of all available and necessary information in choosing between identified project alternatives.

In response to current decision-making challenges, this paper reviews the efforts of several government agencies and scientists to implement new concepts in decision analysis and operations research for complex, environmental projects. Recent (published over the past 10 y) literature on environmental applications of multicriteria decision theory and regulatory guidance developed by the United States and international agencies is summarized. The general trends in the field are reflected in Figure 1b. Multicriteria decision analysis (MCDA) tools can be applied to assess value judgments of individual decision makers or multiple stakeholders. For individuals, risk-based decision analysis quantifies value judgments, scores different project alternatives on the criteria of interest, and facilitates selection of a preferred course of action. For group problems, the process of quantifying stakeholder preferences may be more intensive, often incorporating aspects of group decision making.

thumbnail image

Figure Figure 1.. Current and evolving decision-making processes for contaminated sediment management.

Download figure to PowerPoint

One of the advantages of an MCDA approach in group decisions is the capacity for calling attention to similarities or potential areas of conflict between stakeholders with different views, which results in a more complete understanding of the values held by others. The evolving decision-analytic approaches are applicable to the full range of environmental projects, but for purposes of focusing subsequent discussion, more specific attention is given to decision making at contaminated sites. This paper has 4 objectives: (1) to summarize the most common MCDA methods in practice, (2) to discuss the use of MCDA methods within selected regulatory agencies in the United States and the European Union, (3) to review the use of MCDA methods within selected environmental challenges, and (4) to identify the components of an appropriate decision-analytic framework.


  1. Top of page
  2. Abstract
  9. Acknowledgements

Figure 2 illustrates decision dilemmas for a contaminated sediment management project summarized from Kane Driscoll et al. (2002). The decision makers must select a management alternative that minimizes human health and ecological risks and cost while maximizing public acceptance. Three remediation alternatives (A, B, and C) are identified for consideration by stakeholders and/or project team. Criteria are established to aid decision makers in judging the relative strengths of the alternatives. For ecological risk, 2 criteria are selected, specifically, the number of complete exposure pathways (i.e., distinct exposure route that leads to a receptor) and the maximum calculated hazard quotient from all the pathways. For human health risk, 2 similar criteria are selected, including the number of complete human exposure pathways and the maximum cancer risk calculated from the complete pathways. Decision makers use dollars per cubic yard of sediment to represent the cost criteria. The impacted area (i.e., amount of surface or subsurface area required to manage the sediment) is used as a measure of public acceptance. Quantitative estimates for these criteria are developed through research, monitoring, and survey studies or through expert judgment elicitation. The resulting data are used to parameterize the decision table or matrix depicted in Figure 2.

A decision matrix in a form similar to Figure 2 is usually the final product of feasibility studies for Superfund projects in the United States and similar investigations conducted elsewhere. Decisions are typically based on an informal, ad hoc comparison of the considered alternatives. For example, the decision matrix developed in Figure 2 clearly shows that alternative C is most efficient from risk reduction point of view but is also most costly. Depending on the decision maker's sensitivity to the budget, a decision maker may select alternative C or a cheaper alternative B, which may still result in a significant risk reduction.

The methods of MCDA evolved as a response to the observed inability of people to effectively analyze multiple streams of dissimilar information. There are many different MCDA methods, and a detailed analysis of the theoretical foundations of these methods and their comparative strengths and weaknesses is presented in Belton and Steward (2002). The common purpose of MCDA methods is to evaluate and choose among alternatives based on multiple criteria using systematic analysis that overcomes the limitations of unstructured individual or group decision making.

thumbnail image

Figure Figure 2.. Example decision criteria and matrix.

Download figure to PowerPoint

Almost all decision analysis methodologies share similar steps of organization in the construction of the decision matrix. Each MCDA methodology synthesizes the matrix information and ranks the alternatives by different means (Yoe 2002). Different methods require diverse types of value information and follow various optimization algorithms. Some techniques rank options, some identify a single optimal alternative, some provide an incomplete ranking, and others differentiate between acceptable and unacceptable alternatives.

Within MCDA, elementary methods can be used to reduce complex problems to a singular basis for selection of a preferred alternative. Competing decision criteria may be present, but intercriteria weightings are not required. For example, an elementary goal aspiration approach may rank the dredging alternatives in relation to the total number of performance thresholds met or exceeded. While the analysis can, in most applications of elementary approaches, be executed without the help of computer software, these methods are best suited for problems with few alternatives and criteria, a condition that is rarely characteristic of environmental projects.

Multiattribute utility theory or multiattribute value theory (MAUT/MAVT) and the analytical hierarchy process (AHP) are optimization methods. They employ numerical scores to communicate the merit of 1 option in comparison to others on a single scale. Scores are developed from the performance of alternatives with respect to an individual criterion and aggregated into an overall score. Individual scores may be simply summed or averaged, or a weighting mechanism can be used to favor some criteria more heavily than others. The goal of MAUT/MAVT is to find a simple expression for decision-maker preferences. Through the use of utility/value functions, the MAUT method transforms the diverse criteria in Figure 2 (such as cost, risks, and stakeholder acceptance) into 1 common dimensionless scale (typically 0–1) of utility or value. Utility functions for each criteria convert the criteria units into the 0-to-1 utility scale and are combined with weighting functions of the criteria within the overall decision to form a decision score for each alternative. MAUT also relies on the assumptions that the decision maker is rational (e.g., more utility is preferred to less utility), preferences do not change, and the decision maker has perfect knowledge and is consistent in his or her judgments. The goal of decision makers in this process is to maximize utility/value, which makes this a compensatory optimization approach.

Table Table 1.. Applications of decision support tools in environmental managementa
Application areaMethodDecision contextFunding agencyCitation
  1. a PROMETHEE = Preference Ranking Organization METHod for Enrichment Evaluations; ELECTRE = Elimination Et Choix Traduisant la Realite; AHP = analytical hierarchy process; GIS = geographic information system; MAUT = multiattribute utility theory; MCDA = multicriteria decision analysis; EIA = environmental impact assessment; USDA = U.S. Department of Agriculture; USDOE = U.S. Department of Energy; SMART = simple multiattribute rating technique.

Prioritization of sites/areas for industrial/military activityAHP + GISLand condition assessment for allocation of military training areasU.S. Army Engineering Research and Development CenterMendoza et al. (2002)
 AHP + GISSelection of boundaries for national parkInternational Institute for Geoinformation Science and Earth Observation, The NetherlandsSharifi et al. (2002)
 PROMETHEEWaste management activities in CanadaNatural Sciences and Engineering Research Council of CanadaVaillancourt and Waaub (2002)
 ELECTRE + GISLand management: develop a land suitability map for housing in SwitzerlandSwiss National Foundation for Research (FNRS)Joerin and Musy (2000)
 AHP + GISLandfill sitting Siddiqui et al. (1996)
 MAUT + GISSelection of park boundariesUSDOEKeisler and Sundell (1997)
Environmental/remedial technology selectionSMARTChoosing a remedial action alternative at Superfund siteU.S. Army Corps of EngineersWakeman (2003)
 MAUTSelection of management alternative Missouri RiverUniversity of Missouri—Columbia, USAPrato (2003)
 MAUT + AHPRegulation of water flow in a lake–river systemAcademy of FinlandHamalainen et al. (2001)
 MAUTOffsite emergency management following a nuclear accident (such as the Chernobyl accident)European Commission, UkraineEhrhardt and Shershakov (1996); Hamalainen et al. (2000)
Environmental impact assessmentReviewReview of MCDA use for EIAs in NetherlandsVrije University, The NetherlandsJanssen (2001)
 AHPSocioeconomic impact assessment for a construction project in IndiaIndira Gandhi Institute of Development Research, IndiaRamanathan (2001)
 ELECTREHighway environmental appraisal in IrelandDublin Institute of Technology; University College Dublin, IrelandRogers and Bruen (1998)
 AHP and MAUT/SMARTEnvironmental impact assessment of 2 water development projects on a Finnish riverFinnish Environmental Agency; Helsinki University of TechnologyMarttunen and Hamalainen (1995)
 PROMETHEEPrioritization of EIAs in JordanStaffordshire University, United KingdomAl-Rashdan et al. (1999)
Natural resource managementAHPNatural park managementUSDA Forest ServicesSchmoldt et al. (1994); Peterson et al. (1994); Schmoldt and Peterson (2001b)
 AHPManagement of small forest in North Carolina, USAUSDA Forest ServicesRauscher et al. (2000)
 MAUTManagement of spruce budworm in Canadian forestsNational Science and Engineering Research Council of CanadaLevy et al. (2000)
 AHP, MAUT, and outrankingForestry planning in FinlandFinnish Academy of Sciences; Finnish Forest Research InstituteKangas et al. (2001)
 MAUTImprovement of habitat suitability measurementsFinnish Forest Research InstituteStore and Kangas (2001)
 AHPEnvironmental vulnerability assessment for mid-Atlantic regionUSEPA/USDOETran et al. (2002)
 WeightingManagement of marine protected areas in TobagoU.K. Department of International DevelopmentBrown et al. (2001)
 MAUTFisheries management: select among alternative commercial fishery opening daysFisheries and Ocean, CanadaMcDaniels (1995)
 AHP, MAUT, and outrankingFisheries management Mardle and Pascoe (1999)

Like MAUT, AHP is a compensatory optimization approach. However, AHP uses a quantitative comparison method that is based on pairwise comparisons of decision criteria rather than utility and weighting functions. All individual criteria must be paired against all others and the results compiled in matrix form. In the contaminated sediment example, the AHP method would require the decision maker to answer questions such as, “With respect to the selection of a sediment alternative, which is more important, public acceptability or cost?” The user uses a numerical scale to compare the choices, and the AHP method moves systematically through all pairwise comparisons of criteria and alternatives. The AHP technique thus relies on the supposition that humans are more capable of making relative judgments than absolute judgments. Consequently, the rationality assumption in AHP is more relaxed than in MAUT.

Unlike MAUT and AHP, outranking is based on the principle that one alternative may have a degree of dominance over another (Kangas et al. 2001) rather than the supposition that a single best alternative can be identified. Outranking models compare the performance of 2 (or more) alternatives at a time, initially in terms of each criterion, to identify the extent to which a preference for one over the other can be asserted without using a prescribed scale such as the AHP method. In aggregating preference information across all relevant criteria, the outranking model seeks to establish the strength of evidence favoring the selection of one alternative over another, for example, by favoring the sediment alternative that performs the best on the greatest number of criteria. Therefore, outranking models are partially compensatory and most appropriate when criteria metrics are not easily aggregated, measurement scales vary over wide ranges, and units are incommensurate or incomparable.


  1. Top of page
  2. Abstract
  9. Acknowledgements

Decision process implementation is often based on the results of physical modeling and engineering optimization schemes. Even though federal agencies are required to consider social and political factors, the typical decision analysis process does not provide specifically for explicit consideration of such issues. Comparatively little effort is applied to engaging and understanding stakeholder perspectives (including the general public as well as potentially responsible parties and natural resource trustees) or to provide for potential learning among stakeholders. A result of this weakness in current and common decision models is that the process tends to quickly become adversarial whereby there is little incentive to understand multiple perspectives or to share information. However, a review of regulatory and guidance documents reveals several programs in the United States where regulatory agencies involved in environmental issues are beginning to implement formal decision-analytic tools (such as MCDA) in their decision-making process.

U.S. Army Corps of Engineers

Historically, the U.S. Army Corps of Engineers (USACE) has used essentially a single-measure approach to civil works, planning decisions through its Principles and Guidelines (P&G) framework (USACE 1983). The USACE has primarily used net national economic development (NED) benefits as the single measure to choose among different alternatives. The P&G method makes use of a complex analysis of each alternative to determine the benefits and costs in terms of dollars and other nondollar measures (e.g., environmental quality and safety); the alternative with the highest net NED benefit (i.e., with no environmental degradation) is usually selected. The USACE uses a variety of mechanistic and deterministic fate and transport models to provide information in quantifying the various economic development and ecological restoration accounting requirements as dictated by P&G procedures. The level of complexity and scope addressed by these models is determined at the project level by a planning team. Issues such as uncertainty and risk are also addressed through formulation at the individual project management level.

Table Table 2.. Applications of decision support tools for stakeholder involvementa
MethodApplication formatDecision contextFunding agencyCitation
  1. a MAUT = multiattribute utility theory; USDOE = U.S. Department of Energy; NSF = National Science Foundation; USEPA = U.S. Environmental Protection Agency; USDA = U.S. Department of Agriculture; AHP = analytical hierarchy process.

MAUTIndividual surveys performed under supervisionGeneric radiologically contaminated siteUSDOE/NSFArvai and Gregory (2003)
MAUTWorkshop and individual interviewsConsensus building for water resource managementSocial Sciences and Humanities; Research Council of Canada; NSFMcDaniels and Roessler (1998); Gregory et al. (2001)
MAUTWorkshopDeveloping alternatives for coal mine exploration in MalaysiaSocial Sciences and Humanities Research Council of Canada; NSFGregory and Keeney (1994)
MAUTValue elicitation in workshop settingsElicitation of value judgments for wilderness preservationSocial Sciences and Humanities; Research Council of Canada; NSFMcDaniels and Roessler (1998)
MAUTWorkshopDeveloping alternatives for coal mine exploration in MalaysiaNSF; USEPAGregory and Wellman (2001)
MAVTIndividual surveysRegional forest planningLa Trobe University, AustraliaAnanda and Herath (2003)
MAUTIndividual surveysAir quality valuation in KoreaKorea UniversityKwak et al. (2001)
MAUTIndividual surveysRisk attitudes by farmers in SpainEuropean Union; Spanish governmentGomez-Limon et al. (2003)
AHPWorkshopForest fire managementUSDASchmoldt and Peterson (2001b)
MAUTIndividual surveysWater use planning Gregory and Failing (2002)
Mental modelingIndividual surveys, workshopWatershed managementUSEPAFocht et al. (1999); Whitaker and Focht (2001)
Mental modelingIndividual surveys, workshopEnergy policy Gregory et al. (2003)

While the P&G method is not specifically required for planning efforts related to military installation operation and maintenance, regulatory actions, or operational or maintenance dredging, it presents a general decision approach that influences many USACE decisions. The USACE planning approach is essentially a monocriterion approach where a decision is based on a comparison of alternatives using 1 or 2 factors. Cost-benefit analysis, for example, is a monocriterion approach. The P&G approach has its challenges in that knowledge of the costs, benefits, impacts, and interactions is rarely precisely known. This single-number approach is limiting and may not always lead to an alternative or decision process satisfactory to stakeholders.

In response to a USACE request for a review of P&G planning procedures, the National Research Council (NRC 1999) provided recommendations for streamlining planning processes, revising P&G guidelines, analyzing cost-sharing requirements, and estimating the effects of risk and uncertainty integration in the planning process. As an integration mechanism, the NRC (1999) recommended that further decision analysis tools be implemented to aid in the comparison and quantification of environmental benefits from restoration, flood damage reduction, and navigation projects. In addition, new USACE initiatives, such as the Environmental Operating Principles within USACE civil works planning, dictate that projects adhere to a concept of environmental sustainability that is defined as “a synergistic process, whereby environmental and economic considerations are effectively balanced through the life of project planning, design, construction, operation and maintenance to improve the quality of life for present and future generations” (USACE 2003a, p. 5).

In addition, revised planning procedures have been proposed to formulate more sustainable options through “combined” economic development and ecosystem restoration plans (USACE 2003b). While still adhering to the overall P&G methodology, USACE (2003b) advises project delivery teams to formulate acceptable, combined economic development and ecosystem restoration alternatives using MCDA and trade-off methods (Males 2002). Despite the existence of new guidance and revisions on the application of MCDA techniques to environmental projects, there remains a need for a systematic strategy to implement these methods within specific USACE mission areas (e.g., navigation and restoration) as well as linkage with existing risk analysis and adaptive management procedures.

U.S. Environmental Protection Agency

Stahl et al. (2002) and Stahl (2003) recently reviewed the decision analysis process in the U.S. Environmental Protection Agency (USEPA) and observed that although USEPA has a mandate to make decisions in the public interest pertaining to the protection of human health and welfare, there are barriers in current USEPA decision processes that may discourage stakeholder participation, integration of perspectives, learning about new alternatives, and consensus building. Similar to the USACE, the USEPA uses a variety of modeling tools to support its current decision-making processes. The majority of these tools are quantitative multimedia systems that assess benefits and risks associated with each proposed alternative with the objective of selecting the best option (Stahl 2003).

Several USEPA guidance documents introduce decision-analytic tools and recommend their use. Multicriteria integrated resource assessment (MIRA) has been proposed as an alternative framework to existing decision analysis approaches at USEPA (Stahl et al. 2002; USEPA 2002; Stahl 2003). MIRA is a process that directs stakeholders to organize scientific data, establishes links between the results produced by the research community, and organizes applications in the regulatory community. MIRA utilizes AHP-based trade-off analysis to determine the relative importance of decision criteria.

Multiattribute product evaluation is inherent in the nature of life-cycle assessment, which has rapidly emerged as a tool to analyze and assess the environmental impacts associated with a product, process, or service (Miettinen and Hamalainen 1997; Seppala et al. 2002). Further, the USEPA has developed the “Framework for Responsible Environmental Decision-Making” to assist the Office of Pollution Prevention and Toxics in their development of guidelines for promoting the use of environmentally preferable products and services (USEPA 2000). The “Framework for Responsible Environmental Decision-Making” decision-making method provides a foundation for linking life cycle indicator results with technical and economic factors for decision makers when quantifying the environmental performance of competing products.

U.S. Department of Energy

Similar to the USACE and USEPA, the U.S. Department of Energy (USDOE) uses a variety of multimedia models to support its decision-making process. A recent review (Corporate Project 7 Team 2003) concluded that even though there are a significant number of guidance documents, systems, and processes in use within the USDOE to determine, manage, and communicate risk, there is a great need for comparative risk assessment tools, risk management decision trees, and risk communication tools that would allow site managers to reach agreement with their regulators and other stakeholders while achieving mutual understanding of the relationship between risk parameters, regulatory constraints, and cleanup. Several USDOE models have been developed specifically for dealing with radiologically contaminated sites and sites with dual (chemical and radiological) contamination. Several of the current models are deterministic, although probabilistic multimedia models have also been developed and used (USDOE 2003).

Several USDOE guidance documents introduce decision-analytical tools and recommend their use. Technical guidance developed for a wide range of USDOE decision needs (Baker et al. 2001) segregate the decision process into 8 sequential steps: defining the problem, determining the requirements, establishing the goals of the project, identifying alternative methods and products, defining the criteria of concern, selecting an appropriate decision-making tool for the particular situation, evaluating the alternatives against the criteria, and, finally, validating solution(s) against the problem statement. Guidance also focuses on how to select a decision-making tool from among 5 recommended evaluation methods. These methods include pros-and-cons analysis, Kepner-Tregoe (K-T) decision analysis, AHP, MAUT, and cost-benefit analysis.

The USDOE has developed a standard paradigm for selecting or developing a risk-based prioritization (RBP) system (USDOE 1998). The paradigm describes the issues that should be considered when comparing, selecting, or implementing RBP systems. It also identifies characteristics that should be used in evaluating the quality of a RBP system and its associated results. The USDOE (1998) recommends the use of MAUT as an RBP model because it is a flexible, quantitative decision analysis technique and management tool for clearly documenting the advantages and disadvantages of policy choices in a structured framework. The MAUT merits special consideration because it provides sound ways to combine quantitatively dissimilar measures of costs, risks, and benefits, along with decision-maker preferences, into high-level, aggregated measures that can be used to evaluate alternatives. The MAUT allows full aggregation of performance measures into 1 single measure of value that can be used for ranking alternatives. However, USDOE (1998) cautions that the results of MAUT analysis should not normally be used as the principal basis for decision making because decision making will generally require accounting for factors that cannot be readily quantified (e.g., equity). Furthermore, USDOE (1998) guidance states that no technique can eliminate the need to rely heavily on sound knowledge, data, and judgments or the need for a critical appraisal of results.

The USDOE has used a multiattribute model as the core of its Environmental Restoration Priority System for prioritizing restoration projects developed in the late 1980s (Jenni et al. 1995). Although the Environmental Restoration Priority System was designed to operate with any specified set of values and trade offs, its use was limited to values that were elicited from USDOE managers, including those based on risk analysis. The USDOE has not applied the Environmental Restoration Priority System because of stakeholder opposition, although similar decision support systems have since been adopted for use at various USDOE sites (CRESP 1999). The USDOE has attempted to use simple weighting to aid program planning and budget formulation processes (CRESP 1999).

European Union

A detailed review of the regulatory background and use of decision-analytic tools in the European Union (EU) was recently conducted within the EU-sponsored Contaminated Land Rehabilitation Network for Environmental Technologies project (Bardos et al. 2002). The review found that environmental risk assessment, cost-benefit analysis, life cycle assessment, and MCDA were the principal analytical tools used to support environmental decision making for contaminated land management in 16 EU countries (Austria, Belgium, Denmark, Finland, France, Germany, Greece, Ireland, Italy, the Netherlands, Norway, Portugal, Spain, Sweden, Switzerland, and the UK). Similar to the United States, quantitative methods such as risk assessment and cost-benefit analysis are presently the dominant decision support approaches, while MCDA and explicit trade offs are used less frequently.

Pereira and Quintana (2002) reviewed the evolution of decision support systems for environmental applications developed by the EU Joint Research Center. The concept of environmental decision support has evolved from highly technocratic systems aimed at improving understanding of technical issues by individual decision makers to a platform for helping all parties involved in a decision process engage in meaningful debate. Applications developed in the group include water resources management, siting of waste disposal plants, hazardous substance transportation, urban transportation, management, and groundwater management.


  1. Top of page
  2. Abstract
  9. Acknowledgements

The MCDA applications are relevant to environmental management, stakeholder involvement, and the management of contaminated lands. Recent publications present more comprehensive reviews of studies relevant to management of terrestrial sites (Linkov et al. 2004) and contaminated sediments (Linkov et al. 2005). The use of MCDA is more strongly evident within the broad areas of environmental management and stakeholder involvement. Fewer efforts have been made to apply MCDA to the management of contaminated lands and risk analysis. It should be noted that MCDA has also been applied in many other related policy development areas, such as manufacturing and services; medical, military, and public policy (Keefer et al. 2002a, 2002b); climate change (Bell et al. 2003); industrial facility siting (Larichev and Olson 2001); energy policy (Hobbs and Meier 2000; Keefer et al. 2002a, 2002b); agricultural resource management (Hayashi 2000); and life-cycle assessment (Seppala et al. 2002).

Application to general environmental management

The MCDA methods have been extensively applied to a range of environmental management challenges (Table 1). Each of the examples identified in the course of this review were classified into 1 of 5 application areas: (1) prioritization of site/areas for industrial/military activity, (2) environmental/remedial technology selection, (3) environmental impact assessment, (4) stakeholder involvement, and (5) natural resource planning.

Prioritization of sites/areas for industrial/military activity—Management of contaminated sites often requires site zoning for remediation, restoration, or other uses. Even though applications of MCDA methods for contaminated site zoning could not be found in this review, the MCDA methods described in this study (e.g., MAUT, AHP, and outranking) have been used, in conjunction with geographic information system, for selection of site boundaries and the identification of geographical areas for related uses (e.g., industrial or military).

Mendoza et al. (2002) used AHP for allocating areas for military training exercises at Ford Hood, Texas, USA. Keisler and Sundell (1997) and Sharifi et al. (2003) proposed a framework that integrates MAUT and spatial analysis to determine national park boundaries. Joerin and Musy (2000) developed a generic method to integrate multiple considerations, such as impacts, air quality, noise, accessibility, climate, utility networks (e.g., water, electricity), and aesthetics related to land management. Vaillancourt and Waaub (2002) used outranking and a geographic information system framework to select a site for a new waste management facility in Montreal, Quebec, Canada.

Environmental/remedial technology selection—The selection of a feasible remedial action is usually the final stage of a contaminated site investigation (e.g., as required under the U.S. Superfund program). This review identified several instances in which MCDA methods were used to select the best technology or remedial method.

For example, a MAUT-based method was applied to compare current and alternative water control plans in the Missouri River, USA (Prato 2003). A related problem of regulating water flow in a river–lake system was addressed by Hamalainen et al. (2001) from the perspectives of group decision theory and stakeholder consensus building. Wakeman (2003) used the simple multiattribute rating technique (SMART) (Edwards 1977) to decide which action alternative to implement in handling the contaminated river sediment at Milltown Dam, Montana, USA. Factors considered by Edwards (1977) included availability of materials and services, ability to construct, and reliability. One of the most advanced applications of MCDA techniques in this area was implemented for nuclear accident emergency management as a part of the EU-RODOS project, which used a MAUT analysis for strategy selection for population protection after a nuclear accident (Ehrhardt and Shershakov 1996).

Environmental impact assessment—Environmental impact assessments (EIA) are routinely conducted for all major projects in the United States with the potential to affect the environment. The assessment of site contamination is often an integral part of EIA. Janssen (2001) reviewed 21 EIAs conducted in the Netherlands in the period 1992–2000. Most of the EIAs reviewed by Janssen (2001) used weighted summation methods, although a few projects used either the AHP or a MAUT-based approach.

Marttunen and Hamalainen (1995) reviewed MAUT/SMART and the AHP methods used for decision analysis in EIAs for the assessment of environmental impacts of a water development project in Finland. SMART was chosen over AHP because the AHP procedure proved to be too time consuming for stakeholders (Marttunen and Hamalainen 1995). Ramanathan (2001) recommended the use of AHP for considering multiple criteria and multiple stakeholders in EIA as well as to assess the socioeconomic impact of a proposed liquefied petroleum gas recovery plant in an industrial area in India. Rogers and Bruen (1998) used Elimination Et Choix Traduisant la Realite (ELECTRE) III (outranking) methodology in evaluating thresholds for noise impacts from a highway project in Ireland. Al-Rashdan et al. (1999) used Preference Ranking Organization METHod for Enrichment Evaluations (PROMETHEE) (outranking) methodology to rank environmental impact assessments related to wastewater projects in Jordan; the methodology was found to be very useful in solving problems with conflicting criteria.

Natural resource management—The management of natural resources has involved the application of MCDA. Steiguer et al. (2003) developed an annotated bibliography that includes 124 examples of the application of MCDA to projects ranging from theoretical studies to real-world forest and natural resource management situations. Steiguer et al. (2003) indicate that MCDA constitutes a newer and, perhaps, more acceptable method for quantifying and evaluating public preferences. Nevertheless, few studies included empirical testing of MCDA utility or its feasibility and, in most of the studies, researchers have used hypothetical data or, at best, simplified decision situations; few studies were designed to implement an MCDA-generated management strategy.

The AHP approach within MCDA has received the most attention in natural resource management applications (Steiguer et al. 2003). The application of AHP in natural resource planning is summarized in Schmoldt et al. (2001) with Table 1 listing some representative publications. Schmoldt et al. (1994) and Schmoldt and Peterson (2001b) used AHP to address different aspects of natural park management, including developing inventory and monitoring programs, as well as strategic management plans. Pavlikakis and Tsihrintzis (2003) evaluated the utility of MAUT and AHP in selecting a technically suitable and socially acceptable management plan for a national park in eastern Macedonia and Thrace in Greece.

Methods of MCDA have been extensively applied to a wide range of projects in forest management; AHP was applied for a project-scale forest management problem by Rauscher et al. (2000), and MAUT analysis was applied to identify policy alternatives to manage a budworm outbreak in a local site in Canada (Levy et al. 2000). Kangas et al. (2001) tested the application of several MAUT and outranking methods for large-scale forest policy planning in Finland. Store and Kangas (2001) used MAUT-based methods to conduct a habitat suitability evaluation over large forested areas. Finally, Tran et al. (2002) used AHP to assess the environmental vulnerability of forests across the mid-Atlantic region in the United States.

The MCDA has also been applied to manage aquatic resources. Simon and Pascoe (1999) reviewed applications of MCDA in fisheries management. Brown et al. (2001) used weighting-based trade-off analysis to select a management option for Buccoo Reef Marine Park in Tobago; criteria evaluated included ecological, social, and economic factors. McDaniels (1995) used a MAUT approach to select among alternative commercial fishery openings involving conflicting long-term objectives for salmon management.

Application to stakeholder involvement

Most of the examples presented here attempt to represent the value judgments of a single decision maker and incorporate these value judgments into the overall decision-making process. Stakeholder values are often considered as 1 attribute, along with others, such as costs or risk reduction. The MCDA can also be used as a framework that permits stakeholders to structure their views about the pros and cons of different environmental and remedial management options. Applications of MCDA for group decision making in other areas have been reviewed by Bose et al. (1997) and Matsatsinis and Samaras (2001).

Arvai and Gregory (2003) was the only study identified that addressed the application of decision-analytic tools to include stakeholder involvement at contaminated sites (Table 2). Arvai and Gregory (2003) compared 2 approaches for involving stakeholders in identifying radioactive waste cleanup priorities at USDOE sites, (1) a traditional approach that involved communication of scientific information that is currently in use in many USDOE, USEPA, and other U.S. federal programs and (2) a values-oriented communication approach that helped stakeholders make difficult trade-offs across technical and social concerns. The 2nd approach has strong affinity to the MAUT-based trade-offs discussed earlier in this paper. Arvai and Gregory (2003) concluded that the incorporation of value-based trade-offs information leads stakeholders to making more informed choices.

Table 2 summarizes several other representative stakeholder involvement studies in the areas related to management of natural resources and technology selection. This review addresses studies that involve local communities at action-specific levels rather than broad-based public involvement efforts.

Several studies propose the use of MCDA tools for consensus building and advocate the utility of this application or illustrate the value-oriented approaches that are based on MAUT. In general, applications may include individual surveys and workshops designed to elicit value judgment and construct decision alternatives. Specific applications include water resource management (McDaniels et al. 1999; Gregory et al. 2001), mining (Gregory and Keeny 1994), wilderness preservation (McDaniels and Roessler 1998), and estuary management (Gregory and Wellman 2001). The McDaniels et al. (1999) study concludes that value-based approaches result in a higher level of comfort for participants and are useful in developing consensus-based management decisions. The MAUT-based applications appear to be used in stakeholder value elicitation for regional forest planning (Ananda and Herath 2003), air quality valuation (Kwak et al. 2001), and agricultural applications (Gomez-Limon et al. 2003). In addition, Schmoldt and Peterson (2001a) advocated the use of AHP as a decision support tool in workshop settings for forest resource management.

The examples presented previously used MCDA to facilitate consensus building. An alternative application of MCDA is in the organization of diverse interests instead of seeking consensus-based middle ground. Gregory and Failing (2002) argue that a clear expression of difference facilitates development and acceptance of management plans. Another approach to ranking risk involves soliciting the views of participants both as individuals and in a group setting (Morgan et al. 2000; Florig et al. 2001). In this manner, decision makers can obtain information on the rankings of options that involve multiple objectives by weighing the attributes identified by individuals and groups developed from the 2 methods.

Mental modeling (Morgan et al. 2002) may be a promising tool for assessing individual judgments. It involves individual, 1-on-1 interviews, leading participants through a jointly determined agenda of topics. The method allows free expression and encourages elaboration on topics in order to reveal individual perspectives at considerable depth. When effectively done, analysts can identify what people believe and why they believe it. They are also able to compare analyses over time and provide insights into why beliefs change. Environmental applications of mental modeling include management of the Illinois River basin in eastern Oklahoma, USA, (Focht et al. 1999; Whitaker and Focht 2001) and in energy policy development (Gregory et al. 2003).

Table Table 3.. Applications of decision support tools for contaminated sitesa
MethodSite typeDecision contextCriteriaFunding agencyCitation
  1. a AHP = analytical hierarchy process; USDOE = U. S. Department of Energy; MAUT = multiattribute utility theory; INEEL = Idaho National Environmental Engineering Laboratory; USDOD = U. S. Department of Defense.

AHP + MAUT + fuzzyHazardous chemical waste landfill located at a U.S. national laboratorySelection of remedial action alternativeProgrammatic, life cycle cost, socioeconomic impact, environmental, human health and safetyUSDOEAccorsi (1999a, 1999b); Bonano et al. (2000); Apostolakis (2001)
AHP + linear programmingSavannah River Site (nuclear and chemical industries)Optimal budget/resource allocation for remediationCost of remediation, mortgage cost reduction, technical feasibility, annual funding constraintsUSDOEDeschaine et al. (1998)
MAUTINEEL subsurface disposal areaSelection of remedial action alternativeImplementability, short-term effectiveness, long-term effectiveness, reduction of toxicity/mobility/volume, costUSDOEGrelk (1997); Parnell et al. (2001)
MAUTINEEL landfillSelection of remedial technologyCost, time, and riskUSDOERalston et al. (1996)
MAUTINEEL landfillSelection of remedial technologyRisk of successful development and risk of successful field implementationUSDOETimmerman et al. (1996)
AHPContaminated USDOD installations in KoreaSelection of brownfield management technologiesResource requirements, data quality, method limitations, compliance with policy, input requirements, and outputUSDODHartman and Goltz (2001)

Application to management of contaminated lands

Review of the recent literature reveals few studies that use MCDA techniques to facilitate decision making for the management of contaminated sites (Table 3). The absence of more examples of the application of MCDA at contaminated sites emphasizes the need to provide formal training on complex decision-making analysis.

Most applications of MCDA have been conducted by USDOE to develop decision models to evaluate specific criteria for the selection of remediation technologies. Grelk (1997), Grelk et al. (1998), and Parnell et al. (2001) have developed a decision analysis value model that is based on the process required by the legislation in the United States supporting the Superfund program. The USDOE has also sponsored a series of studies designed to develop decision models used to perform analysis of remedial alternatives for a mixed-waste subsurface disposal site at Idaho National Environmental Engineering Laboratory, USA. Ralston et al. (1996) developed a generic model that incorporates life cycle cost and technological risk assessment for landfill waste site remediation. Timmerman et al. (1996) proposed the use of MAUT by USDOE for selecting technology judged to pose the lowest level of failure or development risks. Deschaine et al. (1998) used a MCDA simulation model based on AHP to select the most promising remediation projects from a 114 radiological site remediation portfolio at the USDOE Savannah River Site. Accorsi et al. (1999a, 1999b), Bonano et al. (2000), and Apostolakis (2001) developed a methodology that uses AHP, influence diagrams, MAUT, and risk assessment techniques to integrate the results of advanced impact evaluation techniques with stakeholder preferences. Most of the studies presented in Table 3 focus on evaluation of technical risk and comparison of alternative technologies; environmental risk assessment and stakeholder opinions were not usually quantified.


  1. Top of page
  2. Abstract
  9. Acknowledgements

Successful environmental decision making in complex settings will depend on the extent to which 3 key components are integrated within the process: people, process, and tools. Based on this review of MCDA concepts and applications, a systematic decision framework is proposed in Figure 3. This framework is intended to give a generalized road map to the environmental decision process.

The correct combination of people is the 1st essential element to the overall decision process shown in Figure 3. The involvement of 3 groups of stakeholders (i.e., decision makers, scientists and engineers, and the general public) are symbolized by dark lines for direct involvement and dotted lines for less direct involvement. While the actual membership and the function of these 3 base groups may intersect or vary, the roles of each group are essential to collecting the largest amount of information to support the decision process. Each group has its own way of viewing the world, its own method of envisioning solutions, and its own societal responsibility. Policymakers and decision makers spend most of their effort defining the problem context and the overall constraints to the decision. In addition, they may have responsibility for the selection of the final decision and its implementation. Stakeholders may provide input to defining the problem but have the highest degree of interaction in helping formulate success criteria and contributing value judgments for weighting the various success criteria. Depending on the problem and regulatory context, stakeholders may have some responsibility in ranking and selecting the final option. Scientists and engineers have the most focused role because they provide the measurements or estimations of the desired criteria that determine the success of various alternatives. While scientists and engineers may take a secondary role as stakeholders or decision makers, their primary role is to provide the technical details as requested by the decision process.

thumbnail image

Figure Figure 3.. Synthesis of decision-making ingredients.

Download figure to PowerPoint

In Figure 3, the framework places process in the center of the overall decision process. While it is reasonable to expect that the decision-making process may vary in specific details among regulatory programs and project types, emphasis should be given to designing an adaptable structure such that participants can modify aspects of the project to suit local concerns while still producing a structure that provides the required outputs. The process depicted in Figure 3 follows 2 basic themes: (1) generating management alternatives, success criteria, and value judgments and (2) ranking the alternatives by applying the criteria levels and value weights. The 1st part of the process generates and defines choices, success levels, and preferences. The latter section methodically prunes nonfeasible alternatives by 1st applying screening mechanisms (e.g., overall cost, technical feasibility, and general societal acceptance) followed by more detailed ranking of the remaining options using decision analysis techniques (e.g., AHP, MAUT, and outranking) that adopt the various criteria levels generated by environmental tools, monitoring, or stakeholder surveys.

As shown in Figure 3, the tools used within group decision making and scientific research are essential elements of the overall decision process. Similar to people, the applicability of the various tools is symbolized by solid lines (representing direct, or high, utility) and dotted lines (representing indirect, or lower, utility). Decision analysis tools help generate and guide the preferences of stakeholder groups, as well as individual value judgments, into organized structures that can be linked with the other technical tools from risk analysis, modeling/monitoring, and cost estimations. The decision analysis software also provides useful graphical techniques and visualization methods to express the gathered information in understandable formats. When changes occur in the requirements or decision process, decision analysis tools can respond efficiently to reprocess and iterate with the new inputs.

The framework depicted in Figure 3 provides a focused role for the detailed scientific and engineering efforts invested in experimentation, environmental monitoring, and modeling, which, together, provide the rigorous and defendable details for evaluating criteria performance under various alternatives. This symbiotic relationship between decision and scientific or engineering tools allows each of the 3 components to have a unique and valued role in the decision process.

As with most other decision processes, it is assumed that the framework presented in Figure 3 is iterative at each phase throughout the course of complex decision making. A 1st-pass effort may efficiently point out challenges that may occur, key stakeholders to be included, or modeling and analysis studies that should be initiated. As the challenges become more apparent, the process repeats itself again to explore and adapt to more subtle aspects of possible decisions and their outcomes.


  1. Top of page
  2. Abstract
  9. Acknowledgements

Effective environmental decision making requires an explicit structure for coordinating joint consideration of the environmental, ecological, technological, economic, and sociopolitical factors relevant to evaluating and selecting among management alternatives. Each of these factors includes multiple subcriteria, making the process inherently multiobjective. Integrating this heterogeneous information with respect to human aspirations and technical applications demands a systematic and understandable framework to organize the people, processes, and tools for making a structured and defensible decision.

Stakeholder involvement is increasingly recognized as an essential element of successful environmental decision making. The challenge of capturing and organizing that involvement as structured inputs to decision making alongside the results of scientific and engineering studies and cost analyses can be met through application of the tools reviewed in this paper. The current environmental decision-making context limits stakeholder participation within the decide-and-defend paradigm that positions stakeholders as constraints to be tested rather than the source of core values that should drive the decision-making process. Consequently, potentially controversial alternatives are eliminated early, and little effort is devoted to maximizing stakeholder satisfaction with either the decision process or the outcome. Instead, the final decision may be something to which no one objects too strenuously. Ultimately, this process does little to serve the needs or interests of stakeholders who must live with the consequences of an environmental decision.

The increasing volume of complex and, often, controversial information generated to support environmental decisions and the limited capacity of any 1 individual decision maker to integrate and process that information emphasizes the need for developing tractable methods for aggregating the information in a manner consistent with the values of the decision maker. The field of MCDA includes methods that can help develop a decision-analytic framework useful for environmental management, including the management of contaminated sites. The purpose of MCDA is not always to single out the correct decision but to help improve understanding in a way that facilitates a decision-making process involving risk, multiple criteria, and conflicting interests. The MCDA visualizes trade offs among multiple conflicting criteria and quantifies the uncertainties necessary for comparison of available remedial and abatement alternatives. This process helps technical project personnel, as well as decision makers and stakeholders, to systematically consider and apply value judgments to derive a most favorable management alternative. The MCDA also provides methods for participatory decision making in which stakeholder values are elicited and explicitly incorporated into the decision process.

Different MCDA methods have strengths and limitations. No matter which analytical decision tool is selected, implementation requires complex, often impossible trade offs. This complexity is probably one of the main reasons why MCDA is still not widely used in practical applications. However, explicit and structured approaches will often result in a more efficient and effective decision process as compared with the often intuition- and bias-driven decision processes that regulatory agencies are often accused of using in decision making.

Formal applications of MCDA in the management of contaminated sites are rare at present. Applications in related areas are more numerous; however, to date, MCDA has remained largely an academic exercise with some exceptions involving the use of AHP-based methods in natural resource planning. Nevertheless, the positive results reported in the studies reviewed in this paper, as well as the availability of recently developed software tools such as Expert Choice (, Criterim Decision Plus (, and Decision Lab (, provide more than an adequate basis for recommending the use of MCDA in contaminated site management.

Environmental decision making also involves complex trade offs between divergent criteria. The traditional approach to environmental decision making involves valuing multiple criteria based on a common unit, usually monetary, and thereafter performing standard mathematical optimization procedures. Extensive scientific research in the area of decision analysis has exposed several weaknesses in this approach (Belton and Steward 2002). At the same time, new methods that facilitate a more rigorous analysis of multiple criteria have been developed. These methods, collectively known as MCDA methods, are increasingly being adopted in environmental decision making.

In summary, this paper reviewed MCDA methods currently in use and cited numerous environmental applications of these methods. While MCDA offers demonstrable advantages, choosing among MCDA methods is a complex task. Each method has strengths and weaknesses; while some methods are better grounded in mathematical theory, others may be easier to implement. The availability to evaluate and/or support a decision also may act as a constraint on method of decision analysis. It is unavoidable that the decision maker will have to choose, on a case-by-case basis, the most suitable MCDA technique applicable for different situations.

Finally, this paper identified the components necessary to develop a decision analysis framework, to facilitate the selection of an MCDA process, and thereafter to provide guidance on the implementation of the selected MCDA method within the larger context of the people, processes, and tools used in decision making. The extensive growth over the past 30 y in the amount and diversity of information required for environmental decision making has exceeded the capacity of common, unstructured decision models. Focused effort directed at integrating MCDA principles and tools with existing approaches, including the use of risk and cost-benefit analysis, will lead to more effective, efficient, and credible decision making.


  1. Top of page
  2. Abstract
  9. Acknowledgements

Acknowledgement—The authors are grateful to Cooke, Small, Valverde, Gloria, Yoe, Florig, Sullivan, Dortch, Dyer, and Gardner for their review and suggestions. Special thanks to Stahl for making her dissertation available. Support for this study was provided by the USACE Hazard/Risk Focus Area of the Environmental Quality Technology Program and the Center for Contaminated Sediments. Additional support was provided by the National Oceanic and Atmospheric Administration through the Cooperative Institute for Coastal and Estuarine Environmental Technology. Permission was granted by the USACE Chief of Engineers to publish this material.


  1. Top of page
  2. Abstract
  9. Acknowledgements
  • Accorsi R, Apostolakis GE, Zio E. 1999a. Prioritizing stakeholder concerns in environmental risk management. Journal of Risk Research 2: 1129.
  • Accorsi R, Zio E, Apostolakis GE. 1999b. Developing utility functions for environmental decision-making. Prog Nucl Energy 34: 387411.
  • Al-Rashdan D, Al-Kloub B, Dean A, Al-Shemmeri T. 1999. Theory and methodology environmental impact assessment and ranking the environmental projects in Jordan. European Journal of Operational Research 118: 3045.
  • Ananda J, Herath G. 2003. Incorporating stakeholder values into regional forest planning: A value function approach. Ecological Economics 45: 7590.
  • Apostolakis GE. 2001. Assessment and management of environmental risks. In: LinkovI, Palma-OliveiraJ, editors. Assessment and management of environmental risks. Boston (MA), USA: Kluwer. p 211220.
  • Arvai J, Gregory R. 2003. Testing alternative decision approaches for identifying cleanup priorities at contaminated sites. Environ Sci Technol 37: 14691476.
  • Baker D, Bridges D, Hunter R, Johnson G, Krupa J, Murphy J, Sorenson K. 2001. Guidebook to decision-making methods. Washington DC: U.S. Department of Energy. WSRC-IM-2002-00002.
  • Bardos P, Lewis A, Nortcliff S, Matiotti C, Marot F, Sullivan T. 2002. CLARINET report: Review of decision support tools for contaminated land management, and their use in Europe. Vienna: Austrian Federal Environment Agency.
  • Bell M, Hobbs BF, Ellis H. 2003. The use of multi-criteria decision-making methods in the integrated assessment of climate change: Implications for IA practitioners. Socio-Economic Planning Sciences 37: 289316.
  • Belton V, Steward T. 2002. Multiple criteria decision analysis: An integrated approach. Boston (MA), USA: Kluwer.
  • Bonano EJ, Apostolakis GE, Salter PF, Ghassemi A, Jennings S. 2000. Application of risk assessment and decision analysis to the evaluation, ranking and selection of environmental remediation alternatives. J Hazard Mater 71: 3557.
  • Bose U, Davey AM, Olson DL. 1997. Multi-attribute utility methods in group decision-making: Past applications and potential for inclusion in GDSS. International Journal of Management Sciences 25: 691706.
  • Brown B, Neil-Adger W, Tompkins E, Bacon P, Shim D, Young K. 2001. Trade-off analysis for marine protected area management. Ecological Economics 37: 417434.
  • Corporate Project 7 Team. 2003. Assessment report. Corporate project 7: A cleanup program driven by risk-based end states. Washington DC: U.S. Department of Energy.
  • [CRESP] Consortium for Risk Evaluation with Stakeholder Participation. 1999. Peer review of the U.S. Department of Energy use of risk in its prioritization process. New Brunswick (NJ), USA: CRESP.
  • Deschaine LM, Breslau B, Ades MJ, Selg RA, Saaty TL. 1998. Decision support software to optimize resource allocation: Theory and case history. Society for Computer Simulation, Simulators International XV. Boston (MA), USA: Society for Modeling Simulations. p 139144.
  • Edwards W. 1977. How to use multi-attribute utility measurement for social decision-making. IEEE (Inst Electr Electron Eng) Trans Biomed Eng 7: 326340.
  • Ehrhardt J, Shershakov VM. 1996. Real-time on-line decision support systems (RODOS) for off-site emergency management following a nuclear accident. Final Report. Luxemburg: European Commission.
  • Florig HK, Morgan MG, Morgan KM, Jenni KE, Fischoff B, Fischbeck PS, DeKay ML. 2001. A deliberative method for ranking risks (I): Overview and test bed development. Risk Analysis 21: 913922.
  • Focht W, DeShong T, Wood J, Whitaker K. 1999. A protocol for the elicitation of stakeholders' concerns and preferences for incorporation into policy dialogue. In: Proceedings of the third workshop in the environmental policy and economics workshop series: Economic research and policy concerning water use and watershed management. Washington DC: Environmental Law Institute. p 124.
  • Gomez-Limon JA, Arriaza M, Riesgo L. 2003. An MCDM analysis of agricultural risk aversion. European Journal of Operational Research 151: 569585.
  • Gregory R, Failing L. 2002. Using decision analysis to encourage sound deliberation: Water use planning in British Columbia, Canada. Journal of Policy Analysis and Management 21: 492499.
  • Gregory R, Fischhoff B, Thorne S, Butte G. 2003. A multi-channel stakeholder consultation process for transmission deregulation. Energy Policy 31: 12911299.
  • Gregory R, Keeney RL. 1994. Creating policy alternatives using stakeholder values. Management Science 40: 10351048.
  • Gregory R, McDaniels T, Fields D. 2001. Decision aiding, not dispute resolution: Creating insights through structured environmental decisions. Journal of Policy Analysis and Management 20: 415432.
  • Gregory R, Wellman K. 2001. Bringing stakeholder values into environmental policy choices: A community-based estuary case study. Ecological Economics 39: 3752.
  • Grelk BJ. 1997. A CERCLA-based decision support system for environmental remediation strategy selection [thesis]. Colorado Springs (CO): U.S. Department of the Air Force, Air University, Air Force Institute of Technology.
  • Grelk B, Kloeber JM, Jackson JA, Deckro RF, Parnell GS. 1998. Quantifying CERCLA using site decision maker values. Remediation 8: 87105.
  • Hamalainen RP, Kettunen E, Ehtamo H. 2001. Evaluating a framework for multi-stakeholder decision support in water resources management. Group Decision and Negotiation 10: 331353.
  • Hamalainen RP, Lindstedt M, Sinkko K. 2000. Multi-attribute risk analysis in nuclear emergency management. Risk Analysis 20: 455468.
  • Hartman DH, Goltz MN. 2001. Application of the analytic hierarchy process to select characterization and risk-based decision-making and management methods for hazardous waste sites. Environmental Engineering and Policy 3: 17.
  • Hayashi K. 2000. Multi-criteria analysis for agricultural resource management: A critical survey and future perspectives. European Journal of Operational Research 122: 486500.
  • Hobbs BF, Meier P. 2000. Energy decisions and the environment: A guide to the use of multi-criteria methods. Boston (MA), USA: Kluwer.
  • Janssen R. 2001. On the use of multi-criteria analysis in environmental impact assessment in the Netherlands. Journal of Multi-Criteria Decision Analysis 10: 101109.
  • Jenni KE, Merkhofer MW, Williams C. 1995. The rise and fall of a risk-based priority system: Lessons from DOE's environmental restoration priority system. Risk Analysis 15: 397410.
  • Joerin F, Musy A. 2000. Land management with GIS and multi-criteria analysis. International Transactions in Operational Research 7: 6778.
  • Kane Driscoll SB, Wickwire WT, Cura JJ, Vorhees DJ, Butler CL, Moore DW, Bridges TS. 2002. A comparative screening-level ecological and human health risk assessment for dredged material management alternatives in New York/New Jersey Harbor. International Journal of Human and Ecological Risk Assessment 8: 603626.
  • Kangas J, Kangas A, Leskinen P, Pykalainen J. 2001. MCDM methods in strategic planning of forestry on state-owned lands in Finland: Applications and experiences. Journal of Multi-Criteria Decision Analysis 10: 257271.
  • Keefer D, Kirkwood CW, Corner JL. 2002a. Perspective on decision analysis applications, 1990–2001. Decision Analysis 1: (online supplement).
  • Keefer D, Kirkwood CW, Corner JL. 2002b. Summary of decision analysis applications in the operations research literature, 1990–2001. Tucson (AZ), USA: Technical Report Department of Supply Chain Management, Arizona State University.
  • Keisler JM, Sundell RC. 1997. Combining multi-attribute utility and geographic information for boundary decisions: An application to park planning. Journal of Geographic Information and Decision Analysis 1: 101118.
  • Kwak SJ, Yoo SH, Kim TY. 2001. A constructive approach to air-quality valuation in Korea. Ecological Economics 38: 327344.
  • Larichev OI, Olson DI. 2001. Multiple criteria analysis in strategic siting problems. Boston (MA), USA: Kluwer.
  • Levy J, Hipel K, Kilgour DM. 2000. Using environmental indicators to quantify the robustness of policy alternatives to uncertainty. Ecol Model 130: 7986.
  • Linkov I, Sahay S, Seager TP, Kiker G, Bridges T. 2005. Multi-criteria decision analysis: Framework for applications in remedial planning for contaminated sediments. In: ProthJM, LevnerE, LinkovI, editors. Strategic management of marine ecosystems. Amsterdam, The Netherlands: Kluwer.
  • Linkov I, Varghese A, Jamil S, Seager TP, Kiker GA, Bridges TS. 2004. Multi-criteria decision analysis: Framework for applications in remedial planning for contaminated sites. In: LinkovI, RamadanA, editors. Comparative risk assessment and environmental decision making. Amsterdam, The Netherlands: Kluwer. p 1554.
  • Males RM. 2002. Beyond expected value: Making decisions under risk and uncertainty. RMM Technical Services, under contract to Planning and Management Consultants Ltd. Prepared for U.S. Army Corps of Engineers, Institute for Water Resources. IWR Report 02-R-4. Accessed 1 August 2003.
  • Marttunen M, Hamalainen RP. 1995. Decision analysis interviews in environmental impact assessment. European Journal of Operational Research 87: 551563.
  • Matsatsinis NF, Samaras AP. 2001. MCDA and preference disaggregation in group decision support systems. European Journal of Operational Research 130: 414429.
  • McDaniels TL. 1995. Using judgment in resource management: A multiple objective analysis of a fisheries management decision. Operations Research 43: 415426.
  • McDaniels TL, Roessler C. 1998. Multi-attribute elicitation of wilderness preservation benefits: A constructive approach. Ecological Economics 27: 299312.
  • McDaniels TL, Gregory RS, Fields D. 1999. Democratizing risk management: Successful public involvement in local water management decisions. Risk Analysis 19: 497510.
  • Miettinen P, Hamalainen RP. 1997. How to benefit from decision analysis in environmental life cycle assessment (LCA). European Journal of Operational Research 102: 279294.
  • Mendoza GA, Anderson AB, Gertner GZ. 2002. Integrating multi-criteria analysis and GIS for land condition assessment: Part 2—Allocation of military training areas. Journal of Geographic Information and Decision Analysis 6: 1730.
  • Morgan MG, Florig HK, DeKay ML, Fischbeck PS. 2000. Categorizing risks for risk ranking. Risk Analysis 20: 4958.
  • Morgan MG, Fischhoff B, Bostrom A, Atman CJ. 2002. Risk communication. Boston (MA), USA: Cambridge University Press.
  • [NRC] National Research Council. 1999. New directions in water resources planning for the U.S. Army Corps of Engineers. Washington DC: National Academy Press.
  • Parnell GS, Frimpon M, Barnes J, Kloeber JM Jr, Deckro RF, Jackson JA. 2001. Safety risk analysis of an innovative environmental technology. Risk Analysis 21: 143155.
  • Pavlikakis GE, Tsihrintzis VA. 2003. A quantitative method for accounting human opinion, preferences, and perceptions in ecosystem management. J Environ Manag 68: 193205.
  • Pereira AG, Quintana SC. 2002. From technocratic to participatory decision support systems: Responding to the new governance initiatives. Journal of Geographic Information and Decision Analysis 6: 95107.
  • Peterson D, Silsbee D, Schmoldt D. 1994. A case study of resources management planning with multiple objectives and projects. Environ Manag 18: 729742.
  • Prato T. 2003. Multiple-attribute evaluation of ecosystem management for the Missouri River system. Ecological Economics 45: 297309.
  • Ralston BE, Jackson JA, Kloeber JM Jr, Deckro RF. 1996. Development of a decision support system for the Department of Energy selection of waste remediation technologies. Wright Patterson Air Force Base, USA: Center for Modeling, Simulation, and Analysis. Technical report 96-02: 1123.
  • Ramanathan R. 2001. A note on the use of the analytical hierarchy process for environmental impact assessment. J Environ Manag 63: 2735.
  • Rauscher HM, Lloyd FT, Loftis DL, Twery MJ. 2000. A practical decision-analysis process for forest ecosystem management. Comput Electron Agric 27: 195226.
  • Rogers M, Bruen M. 1998. Choosing realistic values of indifference, preference and veto thresholds for use with environmental criteria within ELECTRE. European Journal of Operational Research 107: 542551.
  • Schmoldt DL, Kangas J, Mendoza GA, Pesonen M. 2001. The analytic hierarchy process in natural resource and environmental decision making. Amsterdam, The Netherlands: Kluwer.
  • Schmoldt DL, Peterson DL. 2001a. Efficient group decision making in workshop settings. In: Name, editors. The analytical hierarchy process in natural resource and environmental decision-making. Amsterdam, The Netherlands: Kluwer. p 97114.
  • Schmoldt D, Peterson D. 2001b. Strategic and tactical planning for managing national park resources. In: Name, editors. The analytical hierarchy process in natural resource and environmental decision-making. Amsterdam, The Netherlands: Kluwer. p 6779.
  • Schmoldt D, Peterson D, Silsbee D. 1994. Developing inventory and monitoring programs based on multiple objectives. Environ Manag 18: 707727.
  • Seppala J, Basson L, Norris GA. 2002. Decision analysis frameworks for life-cycle impact assessment. J Ind Ecol 5: 4568.
  • Sharifi MA, van den Toorn W, Rico A, Emmanuel M. 2003. Application of GIS and multicriteria evaluation in locating sustainable boundary between Tunari National Park and Cochabamba City (Bolivia). Journal of Multi-Criteria Decision Analysis 11: 151164.
  • Siddiqui M, Everett J, Vieux B. 1996. Landfill siting using geographic information systems: A demonstration. Journal of Environmental Engineering 122: 515523.
  • Simon M, Pascoe S. 1999. A review of applications of multiple criteria decision-making techniques to fisheries. Marine Resource Economics 14: 4163.
  • Stahl CH. 2003. Multi-criteria integrated resource assessment (MIRA): A new decision analytic approach to inform environmental policy analysis [thesis]. Wilmington (DE), USA: University of Delaware.
  • Stahl CH, Cimorelli AJ, Chow AH. 2002. A new approach to environmental decision analysis: Multi-criteria integrated resource assessment (MIRA). Bulletin of Science, Technology and Society 22: 443459.
  • Steiguer JE, Liberti L, Schuler A, Hansen B. 2003. Multi-criteria decision models for forestry and natural resources management: An annotated bibliography. Washington, DC: U.S. Department of Agriculture, Forest Service. General technical report NE-307.
  • Store R, Kangas J. 2001. Integrating spatial multi-criteria evaluation and expert knowledge for GIS-based habitat suitability modeling. Landsc Urban Plann 55: 7993.
  • Timmerman TJ, Kloeber JM Jr, Jackson JA, Deckro RF. 1996. Selecting remediation technologies through a technical risk index: An application of multi-attribute utility theory. Wright Patterson Air Force Base, USA: Center for Modeling, Simulation, and Analysis. Technical report 96–01.
  • Tran L, Knight CG, O'Neill R, Smith E, Ritters K, Wickham J. 2002. Environmental assessment fuzzy decision analysis for integrated environmental vulnerability assessment of the mid-atlantic region. Environ Manag 29: 845859.
  • [USACE] U.S. Army Corps of Engineers. 1983. The economic and environmental principles and guidelines for water and related land resources implementation. Engineering Regulation (ER) 105-2-100.
  • [USACE] U.S. Army Corps of Engineers. 2003a. Environmental operating principles and implementation guidance. Accessed 13 January 2005.
  • [USACE] U.S. Army Corps of Engineers. 2003b. Planning civil works projects under the environmental operating principles. Circular 1105-2-404. Accessed 13 January 2005.
  • [USDOE] U.S. Department of Energy. 1998. Guidelines for risk-based prioritization of DOE Activities. Washington, DC: USDOE. DOE-DP-STD-3023-98.
  • [USDOE] U.S. Department of Energy. 2003. Washington, DC: USDOE-RESRAD Environmental Assessment Division. Accessed 13 January 2005.
  • [USEPA] U.S. Environmental Protection Agency. 2000. Framework for responsible environmental decision-making (FRED): Using life cycle assessment to evaluate preferability of products. Washington, DC: USEPA. EPA/600/R-00/095.
  • [USEPA] U.S. Environmental Protection Agency. 2002. Consistency and transparency in determination of USEPA anticipated ozone designations. Special Review. Washington, DC: USEPA, Office of Inspector General. Report 2002-S-00016.
  • Vaillancourt K, Waaub JP. 2002. Environmental site evaluation of waste management facilities embedded into EUGENE model: A multi-criteria approach. European Journal of Operational Research 139: 436448.
  • Wakeman JS. 2003. Milltown reservoir sediment/Clark Fork River superfund site-focused feasibility study. Accessed 13 January 2005.
  • Whitaker K, Focht W. 2001. Expert modeling of environmental impacts. OPS Special Issue: Environmental Policy in Oklahoma 10: 179186.
  • Yoe C. 2002. Trade-off analysis planning and procedures guidebook. Accessed 13 January 2005.