SEARCH

SEARCH BY CITATION

Keywords:

  • hydrometeorology;
  • uncertainty communication;
  • ensemble flood forecasts

Abstract

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Flow of uncertainty and risk-based decisions in hydrological forecasting
  5. 3. Case-study I: communication and use of ensemble forecasts in flood warning
  6. 4. Case-study II: risk perception and decision-making in hydroelectricity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

There is a common agreement in the scientific community that communicating uncertain hydro-meteorological forecasts to water managers, civil protection authorities and other stakeholders is far from being a resolved issue. This paper focuses on the communication of uncertain hydrological forecasts to decision-makers such as operational hydrologists and water managers in charge of flood warning and scenario-based reservoir operation. Results from case studies conducted together with flood forecasting experts in Europe and operational forecasters from the hydroelectric sector in France are presented. They illustrate some key issues on dealing with probabilistic hydro-meteorological forecasts and communicating uncertainty in operational flood forecasting. Copyright © 2010 Royal Meteorological Society


1. Introduction

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Flow of uncertainty and risk-based decisions in hydrological forecasting
  5. 3. Case-study I: communication and use of ensemble forecasts in flood warning
  6. 4. Case-study II: risk perception and decision-making in hydroelectricity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

Observed trends of increased economic losses associated with weather-climate extremes (Guha-Sapir et al., 2004) are in great part related to changes in societal behaviour and vulnerability, both in time and space: increasing exposure to extreme events, growth of valuable properties at risk, as well as increasing density of poverty and of risk-prone urbanized areas (Changnon et al., 2000). A recent study on flood damage in 31 European countries showed that if the 27 major floods observed between 1970 and 2006 were to take place under current societal conditions, the total flood losses, expressed in 2006 US$ normalized values, would amount to 140 billion (Barredo, 2009). Risk awareness, increased preparedness and investment in flood mitigation measures, as well as enhanced early warning systems are admitted to play a key role in reducing weather-related losses in vulnerable countries (WMO, 2009; Wahlström, 2009). Particularly, coupled meteo-hydrological forecasting systems are effective tools to achieve longer lead times in hydrological forecasting. To develop such systems successfully, interdisciplinary projects are crucial, as they can provide the basis for tracking uncertainty from atmospheric forcing to streamflow predictions. Uncertainty is in fact inherent in the system (Pappenberger and Beven, 2006) and single deterministic forecasts are, in a large majority of cases, insufficient.

Cascading forecast uncertainty in coupled models is an essential step to improve the quality of hydrological forecasts. However it is not a simple task as it may appear at first sight. In the last few years, several approaches have been developed to deal with the propagation of uncertainty in meteo-hydrological chains for forecasting and integrated flood risk assessment (see the review by Cloke and Pappenberger (2009) and examples in Apel et al. (2004), Pappenberger et al. (2005), Romanowicz et al. (2006), McMillan and Brasington (2008), Olsson and Lindström (2008), Block et al. (2009), Golding (2009), He et al. (2009), and Thielen et al. (2009b)). The best methodology to quantify the total predictive uncertainty in hydrology is however still debated (Todini, 2004; Beven, 2006; Mantovan and Todini, 2006; Beven et al., 2008).

Even though opinions on the best models and practices might not converge today, or in the future, there is a common agreement that one must avoid uncertainty misrepresentation and miscommunication, as well as misinterpretation of information by users. A common conclusion from reported studies is that probabilistic forecasts can potentially add value to flood forecasting and warning (e.g. Dietrich et al., 2008; Bartholmes et al., 2009; Jaun and Ahrens, 2009; Renner et al., 2009, and other case-studies reviewed in Cloke and Pappenberger (2009)). However, a number of challenges still remain. They involve: (1) improving forecast accuracy with higher-resolution data collection and assimilation; (2) tracking estimation errors using a full uncertainty analysis to improve the predictability skill of the probabilistic system; (3) constraining uncertainties and narrowing prediction bounds with model refinement; (4) implementing coupled models in real-time operational forecasting; (5) improving the human interpretation (forecaster expertize) of forecasts and the communication of probabilistic products to water managers, civil protection authorities, stakeholders and decision-makers.

These scientific concerns match in several aspects those from operational hydrologists, who are often confronted with the apparently paradoxical situation where the growing demand for risk-based information goes hand in hand with the need of precision in the probabilistic information delivered to decision makers and to the public. Several recent studies point out that uncertainty, when properly explained and defined, is no longer unwelcome among emergence response organizations, users of flood risk information and the general public (Roulston et al., 2006; Handmer and Proudley, 2007; McCarthy et al., 2007; Morss et al., 2008; Créton-Cazanave, 2009; Knight, 2009). In this context, the issues of quantification of uncertainty, as well as interpretation and communication of uncertain forecasts are topical.

The way to communicate uncertain forecasts should be in harmony with the goals of the forecasting system and the specific needs of end-users. Communication of forecast products, characteristics and metrics is presented by Buizza et al. (2007) as a key attribute to assess the functional quality of a forecast (functional and technical qualities are introduced as two distinct frameworks to structure the problem of assessing the overall forecast value of hydro-meteorological forecasts). The varied examples shown by the authors illustrate well the difficulties in measuring the functional quality (and, consequently, the usefulness) of a forecast in an objective way: how to note attributes such as usefulness of contents and appropriateness of format, forecast availability and means of distribution, training and information sessions. Besides, scientific studies have also shown that interpretation of uncertainty can be affected by the way the outcome is framed, the severity of the event being forecasted, the way it is defined and by the different notions users might have of forecast confidence (e.g. Gigerenzer et al., 2005; Broad et al., 2007; Morss and Wahl, 2007; Morss et al., 2008; Joslyn et al., 2009).

Is then effective communication of uncertainty in hydro-meteorological forecasts an impossible mission? This paper is a contribution to addressing this question. The focus on the interpretation and communication of uncertain hydrological forecasts based on (uncertain) meteorological forecasts and (uncertain) rainfall-runoff modelling approaches to decision-makers such as operational hydrologists and water managers in charge of flood warning and scenario-based reservoir operation. First, an overview of the typical flow of uncertainties and risk-based decisions in hydrological forecasting systems is presented to introduce its interactions with the communication of forecasts (Section 2). The challenges related to the extraction of meaningful information from probabilistic forecasts and the test of its usefulness in assisting operational flood forecasters are then illustrated with the help of two case studies (Sections 3 and 4). Conclusions are drawn in Section 5.

2. Flow of uncertainty and risk-based decisions in hydrological forecasting

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Flow of uncertainty and risk-based decisions in hydrological forecasting
  5. 3. Case-study I: communication and use of ensemble forecasts in flood warning
  6. 4. Case-study II: risk perception and decision-making in hydroelectricity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

In hydrology, risk is often formulated as the product of hazard by vulnerability (including exposure and consequences) (e.g. Kelman and Spence, 2003; Apel et al., 2009). This is not only an effective formulation for communicating with stakeholders and helping in territory planning and land management regulations. In effect, it can also offer a practical structure of reasoning for operational forecasters in their routinely tasks, which involve (not exhaustively):

  • real-time forecasting: dealing with different sources of information and data that need to be processed in short time during critical (under-pressure) situations (data to quality control, several interconnected model variables to analyse, blending forecasts from different models to perform combined probabilistic forecasts);

  • expert knowledge-based evaluation: how to interpret model outputs, especially when it concerns extreme conditions linked to rare events (rare events are, by definition, not commonly experienced by the forecasters and by the models, which usually have not been trained on data records containing such events during their calibration period);

  • framing effects in uncertain forecasts: the way experts use their experience and knowledge to build a scheme of interpretation and to communicate forecasts (awareness of the total uncertainties involved in the process of hydrological forecasting, effects of wording, consequences of the underestimation of the hydrological risk of outlier/extreme events, which tend to be rejected by operational forecasters).

The process of flood forecasting at river basins thus involves making and communicating decisions on possible future scenarios, usually predicted by hydro-meteorological forecasting systems, in order to increase preparedness for flood events and decrease false alerts or misses of severe, potentially devastating, floods. A sequence of actions at different levels of complexity has to be undertaken by the forecaster, which comprises the management of databases and their interconnections in real-time, the coupling of mathematical models (e.g. models in meteorology for weather prediction, models in hydrology for streamflow forecasting, and models in hydraulics for flood propagation and inundation mapping), as well as the setup of pre- and post-processing routines and visualization tools.

A schematic view of the main general interconnections present in a typical flood forecasting situation is presented in Figure 1. Weather forecast centres provide input data to the models running at hydrological forecast centres. These centres also usually receive data from local centres (discharge and meteorological observations, radar and/or river water level measurements describing the current local situation) and are responsible for providing streamflow forecasts and pre-warning covering the river basins under survey. Local centres receive these forecasts and run their models at finer resolution to provide specific at-site forecasts and/or flood inundation maps. Feedbacks in real time, as well as expert knowledge exchange between the different centres allow to constantly evaluate the potential flood situation predicted to occur. On the basis of these exchanges, it is possible to alert end-users (public, stakeholders, river basin managers, civil protection agencies or other decision makers) on the probabilities of a flood to occur at river basins under risk.

thumbnail image

Figure 1. Schematic view of the main interconnections in a meteo-hydrological flood forecasting chain and its flow of uncertainties and decisions

Download figure to PowerPoint

Sources of uncertainty in the hydro-meteorological forecast chain are many and include: the meteorological forcing (which is often seen to be the most uncertain at lead times beyond 2–3 days), corrections and downscaling procedure of the meteorological predictions, antecedent conditions of the system, observation networks (meteorological as well as hydrological), methods of data assimilation (discharge, soil moisture), geometry of the system (including flood defence structures), possibility of infrastructure failure (dykes or backing up of drains), characteristics of the system (in the form of model parameters), and limitations of the hydrological model to fully represent processes (for example surface and sub-surface flow processes in the flood generation and routing). The importance of the individual components will vary in time, depending on the dominant flow regime, and in space, as each catchment is unique (Beven, 2000). It will also depend on the interactions between the space-time scales of the predicted event, the main catchment characteristics (area and response time) and the resolution of the meteorological forcing data (e.g. Komma et al., 2007; Thirel et al., 2008). A full uncertainty analysis is needed to track all sources of uncertainty and to estimate both their relative importance in the system and the total uncertainty from the combination of each component (Krzysztofowicz, 2001, 2002; Pappenberger et al., 2005). Although the best methodology to cascade uncertainty in an operational setting still has to be explored (Cloke and Pappenberger, 2009), it is well accepted that the total magnitude of the uncertainty will influence the quality of the predictions. It will also impact on the interpretation of model output forecasts and, eventually, the decision making.

The basis for an efficient communication of final forecast products lies on the quantification of uncertainty, but also on the assessment of how users perceive and understand uncertainty, and tend to act in face of uncertain information. This will additionally support the decision on the kind of information to provide. Studies based on weather forecasts are more frequently found in the literature (e.g. Gigerenzer et al., 2005; Roulston et al., 2006; Broad et al., 2007), while they are still rare in hydrological forecasting. Emphasis should be put on the clearness of the meaning of the message to convey, on the choice of appropriate terminology (use of probabilities, frequencies or plain language with a likelihood scale for a general description), as well as on the choice of the type of information to communicate accordingly to the user's goal (e.g. exceedences of critical flood thresholds may be already enough for launching some operational warning procedures, whilst more precise quantitative information might be necessary for reservoir operation). The value of the information will not necessarily be in the information itself (i.e. more complex information is not straightforwardly more valuable) but on the use one makes of it (for some practical guidelines on communicating uncertainty, see Faulkner et al., 2007; Kloprogge et al., 2007). This is not usually clear to operational hydrologists, who are often traditionally trained in a deterministic framework and more used to working with numerical language, and might have difficulty to go further than putting figures and curves on the information support to be delivered. However, as stated by Hartmann et al. (2002), ‘…even the best forecasts can be worthless if users misinterpret them’.

As can be seen from Figure 1, the flow of uncertainty is accompanied by a flow of expertize knowledge and a flow of decisions, both of which can either add uncertainty or narrow prediction bounds to the forecasts issued by the models (raw model outputs). Forecasters usually derive their forecasts and formulate their judgements on upcoming events from a knowledge base, which includes many types of information (Murphy, 1993): observations, model output predictions, previous forecasting experience, feedback on prior forecasting performance. As noted by Atger (2001), forecasters judgements are essentially probabilistic, even though ‘this probabilistic judgement is not necessarily stated explicitly. However uncertainties are communicated, with the help of confidence indices, risk level estimations, interpercentile intervals or likelihood of occurrence of an event, it is acknowledged that ignoring uncertainty in the formulation of forecasts affect the efficiency of the forecasting process, as well as the quality and value of the forecasts (Murphy, 1993). Also, in several countries, flood forecasting is usually performed at different levels of organization: for example harmonized medium-range forecasts and pre-warning at centres operating nationwide, and more specialized short-range forecasts at local river basin centres. Therefore, additionally to the flow of data and models from different sources, and the propagation of their inherent uncertainties, it is also necessary to take into account the impact of the flow of decisions made by experts at centres with different responsibilities in flood warning and communication to the end-user.

3. Case-study I: communication and use of ensemble forecasts in flood warning

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Flow of uncertainty and risk-based decisions in hydrological forecasting
  5. 3. Case-study I: communication and use of ensemble forecasts in flood warning
  6. 4. Case-study II: risk perception and decision-making in hydroelectricity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

In this case study, results from exploratory research conducted within the pre-operational European Flood Alert System (EFAS) project at the Joint Research Centre of the European Commission (Thielen et al., 2009a) are presented. The aim of this research was to implement ensemble forecasts in the system and develop probabilistic products that could provide useful information to EFAS users (national operational forecasters in charge of flood warning). When operational forecasters have to decide, based on a set of possible future scenarios, whether or not a flood warning has to be issued to initiate an emergency procedure, the need of extracting the useful data to quantify and efficiently interpret uncertainty arises. To ensure a good use and an efficient communication of uncertain forecasts is a step forward for assessing and improving the performance of a forecasting system.

In this context, a basic framework for efficiently organizing discussions and best practices to improve the use and communication of probabilistic information in hydrometeorological forecasting is proposed (Figure 2). It highlights some important issues to be considered: a clear specification of the forecasting system's aims along with the steps taken for model development and set-up contribute to specify probabilistic products and inform users about the strengths and limitations of the forecasting system. It is also important to continuously train users during the process. Workshops and meetings can provide good opportunities to improve formulation, evaluation and communication of forecasts interactively. The set up of a network can facilitate open exchanges between developers, experts and users.

thumbnail image

Figure 2. Basic steps in the process of improving the use and communication of probabilistic information in hydrometeorological forecasting systems

Download figure to PowerPoint

The following paragraphs describe the main achievements from two studies carried out to explore the dissemination and the use of ensemble prediction information implemented in EFAS for operational flood forecasting and decision-making (Ramos et al., 2007): a workshop organized together with flood forecasters from the Member States (Thielen et al., 2005) and a survey to assess users general perception of uncertainty in forecasting.

3.1. Workshop results: dealing with uncertain forecasts for decision-making

The workshop's concept was to have a small group of forecasters from different river basins working through a number of case studies specially prepared to put them in a real-time flood forecasting situation where uncertainty was included. Participants had a limited time to analyse the situations proposed and take a decision on whether or not they would send an alert for emergency procedure. Invitations were issued to 11 forecasters from eight different European countries, representing nine different national hydrological services and resulting in a group with a wide range of experience and different cultural backgrounds.

The participants were split into four workgroups to analyse three case studies, one after the other. Each case study corresponded to the forecasting of a potential flood situation at more than 2 days in advance and in a different river basin. The groups had information on three consecutive days of forecast. After having analysed each day of forecast, they had to decide whether or not they would contact the civil protection authorities with a flood warning on the situation they were forecasting for the following days. They also had to qualify the level of severity of the situation: low, medium or high. Time to analyse the data was limited (1 h) in order to mimic the decision-making process under pressure, which is often the case in real-time forecasting situations. Before starting the case studies, an overview of technical aspects concerning the EFAS forecasting system and Ensemble Prediction Systems (EPS) in hydrometeorology was given in order to bring the participants to a similar level of knowledge.

The basic information available for each case study came from the medium-term forecasts of EFAS. Basically, it consisted of: (1) maps characterizing the river basins, including topography, river network and localization of discharge gauging stations; (2) information about meteorological conditions and precipitation forecasts from the Deutsche Wetterdienst (forecasts for the next 7 days) and the European Centre for Medium-Range Weather Forecasts, ECMWF (forecasts for the next 10 days), the latest including overview maps of the 51 EPS precipitation forecasts; (3) observed discharges at the gauging stations for the previous days; (4) discharge forecasts at the gauging stations for the next 48 h from a local model; (5) EFAS information reports summarizing the situation for the next 7–10 days in terms of exceeding EFAS alert levels, including deterministic and probabilistic flood forecasts based on EPS. A computer and a CD-ROM containing all other information participants could eventually need to base their decisions on (e.g. radar images, qualitative information on river basins' characteristics) were also provided.

In order to assess the influence of different available information on forecasters' analyses and decisions, information was distributed unequally: on a rotating basis, one control group was assigned to receive reduced information, which consisted of all information mentioned above, except the forecasted hydrographs (point 4) and information on EFAS flood forecasts and alert levels (point 5), while the other three groups had the complete deterministic and probabilistic information. The control group was thus without any hydrological uncertainty information based on EPS. Also, the case studies were selected in a way that in each case the role of the EPS forecasts was slightly different: sometimes in agreement with the results of the deterministic forecasts, sometimes contradicting them.

During the case studies, each group had an ‘observer’ (one of the organizers of the workshop), who was not supposed to interfere in the group's discussions and in the decision-making process, but to provide technical assistance, if necessary, as well as to take notes on how the groups were dealing with the uncertainty in the forecasts and on their general attitude towards the exercise. After working with all case studies, plenty of time was given to a plenary session, where participants were invited to present their decisions and discuss on the way they had used the probabilistic forecasts based on EPS, on the difficulties they had encountered during the exercise and on their views on the best way to communicate uncertainty in flood forecasting.

Some interesting results regarding the practice of forecasting under uncertainty are summarized below (quotations come from the plenary discussion):

  • In general, forecasters found EPS-based forecasts helpful and important, and missed the EPS information when it was not provided: ‘… it was a great confusion for us (to) work without EPS’

  • The majority of forecasters used EPS uncertainty information to confirm the deterministic forecast: ‘if the EPS shows the same direction (as) the other forecast, it makes us more confident’; ‘(if it doesn’t it makes us) much more confused’

  • Confidence in a forecast was basically built on: agreement between forecasts from different sources, EPS supporting one or both deterministic forecasts, and forecast persistence from 1 day of forecast to another: ‘when forecasts disagree ….I wait for the next forecast’; ‘a persistent situation can give probably more skill than a different situation’.

Additionally, the experiment showed that:

  • Forecasters were mainly conservative when issuing an alert: they showed a tendency to maintain the highest level of severity issued until they could be sure that there was no risk of achieving it anymore. In different workgroups, it was observed that when forecasters issued a high severity level in the first day of forecast, they preferred to keep it through the next few days, even if the risk decreased. They would only decrease the level they issued if, on the third day, the situation showed to be no longer severe.

  • Local expert knowledge is perceived as a key element for good flood forecasting. Forecasters highlighted the difficulties in performing flood forecasting over a region they are not used to working with. This result is in accordance with other studies arguing that familiarity with the situation and local expertize have a role to play in judging unusual situations in flood forecasting and warning (Blöschl, 2008). Local expert knowledge concerns knowledge of the behaviour of the river basin, as well as prior knowledge concerning other meteorological and hydrological situations experienced by the watershed.

  • The understanding of how to use uncertainty information to make decisions on flood warning increased with subsequent case studies. This observed ‘training effect’ highlights the importance of training in the use of probabilistic forecasts and products in operational forecasting.

3.2. Survey results: experience in forecasting and uncertainty

Together with the workshop, a survey was prepared to assess flood forecasters general perception of uncertainty. Two main issues were addressed:

  • 1.
    the respondents' work experience with flood forecasting, uncertainty, forecast communication and decision-making, as well as their experience and knowledge with ensemble prediction systems (EPS);
  • 2.
    the respondents' general perceptions of the use and communication of uncertainty in flood forecasting. This issue was assessed with the help of 43 statements to which respondents had to answer by choosing between ‘strongly agree’, ‘agree’, ‘not sure’, ‘disagree’ and ‘strongly disagree’.

The survey was conducted at the Joint Research Centre (JRC), in Italy, during winter 2005/2006. The questionnaire was distributed in person to forecasters working in different national hydrological services in Europe and the respondents had approximately 20 min to complete the questions. Twenty five questionnaires were collected. Respondents came from 14 different European countries. Germany was the country with more respondents (4 out of 25), followed by France and Italy (three each). The proportion of male and female respondents was 64 and 36%, respectively. Most of the participants (52%) had less than 5 years of experience in flood forecasting, while 32% (8 respondents out of 25) had more than 10 years of experience.

Results show that the use of meteorological products in hydrological forecasting predominates in the sample surveyed (operational forecasters involved in forecasting at relatively larger and/or transnational river basins): a large majority of respondents have experience with flood forecasting based on meteorological forecasts (92%). However, experience with uncertainty and/or ensemble predictions is much smaller: about half of the respondents answered to have experience with uncertainty in forecasting (52%) and only 44% answered to have experience with forecasting based on Ensemble Prediction Systems (EPS) (11 out of 25 respondents) (Figure 3).

thumbnail image

Figure 3. Work experience in flood forecasting: (a) Do you have experience with flood forecasting based on meteorological forecasts? (b) When a flood event is forecasted, are you in contact with the civil protection or any other decision maker? (c) Do you participate in reporting and sending flood alerts to other authorities? (d) Have you ever been involved in communicating a flood forecast or warning to the media (television, radio, newspaper)? (e) Do you have experience with uncertainty in forecasting? (f) Do you have experience with forecasting based on EPS (Ensemble Prediction Systems)? Respondents were European forecasters (n = 25) from 14 different countries. ‘Yes’ responses are shown in black, ‘no’ responses in gray

Download figure to PowerPoint

Figure 4 shows forecasters' judgments on their experience and knowledge with EPS: 80% of the respondents answered to have ‘very little’ (32%), ‘a bit’ (16%) or ‘a fair bit’ (32%) direct experience with EPS. It seems that while some forecasters may have direct contact with EPS (most probably with meteorological EPS, according to the results from Figure 3), they might not perform hydrological forecasting based on EPS. When asked how they would rate their knowledge of EPS, most of the answers were ‘okay’ (32%), followed by ‘good’ (24%) and ‘poor’ (20%). These results show that there is room for improving experience and training on the use of probabilistic products in hydrological forecasting.

thumbnail image

Figure 4. Knowledge and direct experience with EPS: (a) How much direct experience have you had working with EPS? (b) How would you rate your knowledge of EPS? Respondents were European forecasters (n = 25) from 14 different countries

Download figure to PowerPoint

The survey also showed that a large majority of forecasters (92%) agree or strongly agree with the statement that ‘ensemble predictions provide flood forecasts with valuable information’. Eighty four percent disagree or strongly disagree that ‘uncertainty analysis is not relevant for operational flood forecasting and warning purposes’. The majority (55%) said they were ‘not sure’ about the statement that ‘experienced forecasters are more skillful at predicting floods than EPS’. Fifty eight percent disagree or strongly disagree that ‘flood forecasters should not issue flood warnings until they are certain that there will be a flood’, while 33% opted for the answer ‘not sure’.

A large majority of respondents (80%) said to be in contact with the civil protection or any other decision maker when a flood event is forecasted (Figure 3). Seventy six percent answered that they participate in reporting and sending flood alerts to other authorities, while 56% of the respondents indicated they had already been involved in communicating a flood forecast or warning to the media (television, radio, newspapers). Forty six percent of the respondents said that they agree with the statement that ‘ensemble predictions cannot be understood by policy makers and the public’ (17% disagreed with the statement and 33% answered they were not sure). Only 4% disagree that policy makers and the public are confused by probabilistic statements such as ‘there is a 25% chance of flooding’, while 38% answered ‘not sure’ and 50% agreed with the statement.

In summary, from the workshop and the survey, it emerged that although forecasters have been in contact with ensemble predictions, there is still room for improving their knowledge and increasing their experience on the use of uncertainty in flood forecasting. It was seen that forecasters are usually open to new products and methods to assess and describe flood hazard in a probabilistic way, although they are still not sure on how to efficiently use the information. A large majority of forecasters are actually involved in the communication of forecasting results, which evidences the important role of communication in their work. With the increasing use of probabilistic products in forecasting, together with the progressively embracing of the concept of uncertainty by users, the effective way of communicating uncertainty in forecasts will become an even more essential topic for operational hydrological forecasters. Basically, forecasters' main concerns rely on: (1) the efficient way to disseminate probabilistic results, (2) the type and quantity of information to be provided to end-users, (3) the way to measure the usefulness of the information provided, (4) the analysis of consistence between model outputs and persistence in time-consecutive forecasts, and (5) the best way to combine forecasts from different sources.

Finally, it should be noted that, since the workshop and survey have been conducted, a number of operational hydrological services in Europe have adopted EPS in their flood forecasting chain. Therefore, a similar study carried out today could yield different results. More details on the main technical results of the workshop can be found in Thielen et al. (2005), while more insights on risk perception can be found in Demeritt et al. (2007).

4. Case-study II: risk perception and decision-making in hydroelectricity

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Flow of uncertainty and risk-based decisions in hydrological forecasting
  5. 3. Case-study I: communication and use of ensemble forecasts in flood warning
  6. 4. Case-study II: risk perception and decision-making in hydroelectricity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

Over the last 60 years, the French national electricity company (EDF) has accumulated experience on real-time hydrometeorological monitoring and operational inflow forecasting, from flood forecasting (for safety and security) to long-term reservoir management and decision-making (Lugiez and Guillot, 1960). Applied research has been particularly carried out to investigate the potential improvements on risk analysis by using information from ensemble weather predictions and calibrated probabilistic forecasting systems. It is expected that probabilistic forecasts in hydroelectric systems should not only represent a gain in safety and security, but also in the optimization of the economic value of water resources.

At EDF, 1–7 day deterministic flow forecasts are performed on a daily basis on a hundred of watersheds in France, mainly located in mountainous areas (Alps, Pyrenees and Massif Central). Seven-day probabilistic forecasts, based on analogues, i.e. historic situations analogous to current predictions (Zorita and von Storch, 1999; Obled et al., 2002) and on rainfall and temperature forecasts from the Ensemble prediction system of ECMWF (EPS, 51 members), were pre-operationally implemented in 2008. In the pre-operational probabilistic forecasting chain, meteorological forecasts are combined with a statistical model that estimates the uncertainty from the rainfall-runoff model to produce ensemble streamflow forecasts that take into account both meteorological and hydrological uncertainties (Montanari and Brath, 2004; Schaefli et al., 2007; Mathevet et al., 2009). The quality of the probabilistic forecasts is being investigated in order to improve the efficiency of the forecasts issued and enhance forecaster expertize. Forecast evaluation has focused on three main variables of interest: flood volume, peak flow and timing.

Currently, the end-to-end operational streamflow forecasting chain includes several steps. First, meteorological forecasts are retrieved, visualized and submitted to a first expert interpretation and reporting. These forecasts come from the Arpège and Aladin Météo-France weather prediction systems (deterministic forecasts), from an analogue-based forecast system (quantile-based forecasts) and from the EPS-ECMWF (ensemble forecasts). Second, based on the expert analysis and on previous meteorological forecasts, a scenario of precipitation and air temperature is constructed and used to force a hydrological model and a water temperature model. Third, streamflow and river temperature forecasts are delivered to two distinct EDF internal users: (1) the unit responsible for the optimization/trading of resources (energy purchase, production and sale) and the guarantee of energy delivery to clients, and (2) the local hydraulic centres, responsible for dam security and reservoir management.

When necessary, expert probabilistic streamflow forecasts (forecasts based on human expertize) are also produced. In this case, to cope with the uncertainties in hydrometeorological forecasts, forecasters are encouraged to modify temperature, rainfall and streamflow forecasts, given their experience and knowledge of the behaviour of the watersheds. In their predictions, forecasters have to consider the location of the hydro-meteorological event, its temporal evolution and its magnitude. They are asked to build scenarios for the 10, 50 and 90% percentiles (i.e. the median scenario and the 80% predictive interval of the probability distribution, with the range of values lying between the 90th percentile at the upper bound and the 10% percentile at the lower bound). For some case studies, it is then possible to compare these expert probabilistic streamflow forecasts to the automatic model outputs from the pre-operational probabilistic forecasting chain.

The flood event observed in May 2008 in the Durance River at the Serre-Ponçon reservoir (southeastern France) illustrates the difficulty forecasters can encounter to perform good expert probabilistic forecasts in the operational context. The flood event registered a maximum peak flow slightly exceeding the 20 year return period flood (750 m3 s−1) on the 30 May. On the 23 May, a 6 day streamflow forecast scenario was issued based on Arpège and Aladin Météo-France weather prediction systems and on the analogue forecasting system running at EDF. Forecasters built their more probable air temperature and precipitation scenarios based on their analysis of the meteorological situation. At that time, forecasters were prepared for a warm and rainy week. They were predicting that the upcoming event would most probably be the peak of the annual snowmelt-based runoff at the outlet of the catchment. Forecasters issued an expert probabilistic forecast based on their subjective construction of a low (10%) and a high (90%) scenario. Their hydrological forecast is shown in Figure 5(a): the 80% confidence interval (defined by the 10 and 90% quantiles) and the median scenario. An increase in streamflow was predicted, with a discharge peak to occur on 28–29 May. The predictions show a 50% of probability that this peak would be greater than 300 m3 s−1, and a 10% of probability that this peak would be greater than 450 m3 s−1 (approximately, the 5 year return period flood at the location). The confidence interval of the expert probabilistic forecast indicated a chance of an upcoming stronger event, comparatively to the one predicted by the forecasts issued by the models without human expertize (not shown). However, this chance being too small, this expert probabilistic forecast was not considered alarming enough. No severe warning was issued to EDF internal users. Although the group of forecasters kept the event under survey, no other expert probabilistic forecast was produced during the following days.

thumbnail image

Figure 5. Expert probabilistic streamflow forecasts in the Durance River at Serre-Ponçon for the 30 May 2008 flood event: forecast issued on (a) 23 May, and (b) 27 May. Median predictions are represented by black dots; around the median, the grey shadowed area indicates the 80% confidence interval (range of values between the 90th percentile at the upper bound and the 10th percentile at the lower bound). Observed streamflow (continuous line) and simulated streamflow with observed meteorological data (broken line) are also represented. The flood threshold corresponding to a 20 year return period is indicated (Q20)

Download figure to PowerPoint

Four days later, on 27 May, the expert forecasts issued showed a much different picture of the situation: at this time, the predictions were indicating the exceedence of the 20 year flood (Figure 5(b)). In fact, an important amount of rain (mean areal rainfall of about 40 mm in 24 h for 3700 km2), which was not forecasted by any available meteorological model the days before it happened, had been recorded on the 25 May. River flow had increased sharply (reaching the 10 year return period flood) and forecasters were alerted for the potential dangerousness of the upcoming event. On the 27 May, the forecasting system could assimilate this new data and produce better forecasts for the next days. Forecasters were able to provide a widener inter-quantile range of their probabilistic forecasts, by using not only their experience, but also model updates. The 90% quantile was now forecasted to exceed the 20 year flood (Figure 5(b)). Forecasters had even predicted the possibility of a 50 year flood event (900 m3 s−1) to occur, with a 10% probability. This second probabilistic forecast proved to be very useful for management operations, by anticipating with 3–4 days this major flood event.

Retrospectively, it can be said that the increasing trend in the temporal evolution of discharges was correctly predicted several days in advance. However, the intensity of the flood peak and the flood volume were largely underestimated, by both the models and the expert probabilistic forecasts. In the case here presented, it can also be seen that the hydrological model simulated well the evolution of flows when retrospectively forced with observed meteorological data (see dashed lines in Figure 5). Would the forecasters be able to forecast the event with an increased lead time if a better forecast of the event was issued by the hydrometeorological chain of models? Are they, somehow, guided (or constrained) by the possibilities given by the models?

The answer to the questions above is not straightforward. A study conducted at EDF by Houdant (2004) indicated that it seems that forecasters have a tendency to underestimate hydrological risks and uncertainties. An in-depth analysis of a large number of events would be necessary to better assess this issue, which is out of scope of this paper. However, in order to check, for the case-study presented here, if the deficiencies of the expert probabilistic forecasts issued could mainly be due to the meteorological predictions or to a clear overconfidence of the forecasters, we run reforecasts of ensemble streamflows for the 23 May (Figure 6(a)) and the 27 May (Figure 6(b)) based on EPS-ECMWF and rainfall-runoff model uncertainty. These data were not available to the forecasters at the time of the flood event.

thumbnail image

Figure 6. Automatic probabilistic streamflow forecasts based on EPS-ECMWF and rainfall-runoff model uncertainty in the Durance River at Serre-Ponçon for the 30 May 2008 flood event: forecast issued on (a) 23 May, and (b) 27 May. Median predictions are represented by black dots; around the median, the grey shadowed area indicates the 80% confidence interval (range of values between the 90th percentile at the upper bound and the 10th percentile at the lower bound). Observed streamflow (continuous line) and simulated streamflow with observed meteorological data (broken line) are also represented. The flood threshold corresponding to a 20 year return period is indicated (Q20)

Download figure to PowerPoint

In comparison to the expert probabilistic forecasts in Figure 5, these two forecasts are very interesting. Figure 6 shows that the 80% confidence intervals of the ensemble forecasts are in fact narrower than that of the expert forecasts. It was seen that, for the 25 May, very few members of the rainfall ensemble predicted a rainfall greater than 30 mm and no member predicted a rainfall greater than 40 mm. As a consequence, the peak flow associated with a 10% probability of exceedence in the hydrological ensemble forecasts of the 23 May was of only 400 m3 s−1 (Figure 6(a)). On the 27 May (Figure 6(b)), a significant decrease of streamflow was predicted before the peak of 750 m3 s−1 was forecast with a probability of 10%. Post-event analyses of the meteorological situation for this particular event showed that precipitation was caused by an eastward Mediterranean depression, whose speed had been underestimated by the models. All EPS members were localizing strong rainfalls on the Cevennes region, situated 200 km west of the Durance watershed. Like the ensemble predictions, expert probabilistic forecasts were not able to envisage the probability of an eastward shift of precipitation and its consequence when falling over the Durance watershed.

What made this particular forecast so difficult? Neither did the weather forecasts from the 51 members of EPS-ECMWF locate large rainfall amounts on the Durance watershed, and instead placed strong rainfalls in the Cevennes region, nor were forecasters able to imagine the geographic displacement of the rainfall event. It seems that once forecasters have built-up their most probable scenario, usually supported by successive concordant meteorological forecasts, it is difficult for them to imagine another probable spatial pattern. This is in agreement with what was reported in the study by Houdant (2004) and observed in the results of the workshop presented in Section 3. Certainly, the event was rare, and weather forecast uncertainties, as well as rainfall-temperature-runoff modelling uncertainties were considerable. These uncertainties were combined to forecasters' self-censure, and forecasters were convinced that the event, if it was to occur, would take place in the Cevennes region. As a consequence, the hydrological risk at the Durance watershed during this event was estimated to be low. The better forecasts issued on 27 May were maybe highly influenced by the large amount of rainfall that had just been recorded, as well as by the fast increase of discharge values observed in the previous 24 h. What is most certainly, however, is that by discussing model results among themselves, forecasters were able to improve their interpretation of model outputs and underpin the forecasts they delivered to their users.

A recent study presented by Önkal et al. (2009) indicates that in forecasting tasks people have a tendency to take greater account of advice when they think it has been provided by a human expert rather than a statistical method. In the case study presented here, the comparison of expert and automatic probabilistic forecasts confirmed that human expertize is an important step to efficiently improve hydrometeorological forecasts and uncertainty communication. However, it also showed that human expertise per se is not sufficient to prevent misses or false alarms. More practice with probabilistic forecast interpretation and stimulating team discussions are essential to improve the use of probabilistic forecasts in hydrology and to succeed in uncertainty communication.

5. Conclusions

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Flow of uncertainty and risk-based decisions in hydrological forecasting
  5. 3. Case-study I: communication and use of ensemble forecasts in flood warning
  6. 4. Case-study II: risk perception and decision-making in hydroelectricity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

This paper addresses the question: is communicating uncertainty in hydro-meteorological forecasts mission impossible? Two case-studies dealing with the probabilistic forecasting of discharges have been presented in an attempt to illustrate the main aspects to consider in the quest for an answer. From this experience, there is an optimistic temptation to bet on a negative answer: the mission is not impossible, at least not in its absolute terms, although the tasks to be executed might be difficult to accomplish. Efficient communication of forecast uncertainty will result in an increased preparedness, whether the event happens or not, as users will be able to understand the message conveyed and act accordingly. However, a lot of work is still necessary to show the value of such a communication process, which needs to go far beyond the traditional approach of just displaying numbers, scores and good performance measures.

Several challenges encountered by the hydrological community in probabilistic forecast and communication are not completely detached from those that correlated sciences dealing with decision-making under risk have been facing (see, for instance, the recommendations in NRC (2006) and the guidelines proposed by WMO (2008), focusing on weather and climate forecasts). Much can therefore be learnt from outside-hydrology experiences and their contextualization to the operational problems at hand. Specific guidelines to a hydro-meteorological forecasting chain need to be developed. How far this is possible, given the uniqueness of catchments in their hydrological behaviour, the different organizational structures and socio-cultural differences faced by hydrometeorological forecasters, also needs to be explored.

The approach of probabilistic forecasting in meteo-hydrology is still novel. Many methods and techniques to cascade uncertainties still have to be developed and tested in operational environments. A period of several years will be needed to build up the know-how of forecasters within the hydrological forecasting agencies and to fully incorporate the benefits of these new operational flood forecasting tools. However, the occurrence of single extreme flood events, and the responses to them, can significantly alter the course of flood policies in individual countries and catalyse scientific and operational developments (Pitt, 2008). Therefore, alternative ways to increase effectiveness in forecasting processes and communication must be thought out and tested.

As policy and decision making often rely on scientific forecasts and experts judgements, attention must be paid to initiatives that promote or reinforce the active participation of expert forecasters in the meteo-hydrological forecasting chain. The practice of face-to-face forecast briefings, focusing on sharing how forecasters interpret, describe and perceive the model output forecasted scenarios, together with continuous technical training, is essential in the communication of uncertain forecasts. Training for users should also be encouraged, so together flood forecasters and users can define and agree on the level of uncertainty acceptable for each community or problem in hand. Focus should be placed on studies that take account of forecast requirements of different types of users, with different needs in terms of forecast formulation, tolerance to false alarms and optimum probability thresholds for the detection of critical events.

Finally, integrated platforms, allowing a continuous exchange of knowledge and information in real time, are crucial for the development and implementation of useful flood forecasting and warning systems. Due to the real-time characteristic of a flood forecasting exercise, such systems can benefit greatly from an automatic (as much as possible, considering that the activity is strongly based on expert knowledge) and adaptive (i.e. robust in dealing with different situations one can find in forecasting weather-driven processes) chaining of suitable components for communication and decision support.

In summary, efficient communication of uncertainty in hydro-meteorological forecasts is not a mission impossible. The numerous other questions remaining unanswered in hydrological forecasting should not neutralize the goal of such a mission, and the suspense kept should instead act as a catalyst for overcoming the remaining challenges.

Acknowledgements

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Flow of uncertainty and risk-based decisions in hydrological forecasting
  5. 3. Case-study I: communication and use of ensemble forecasts in flood warning
  6. 4. Case-study II: risk perception and decision-making in hydroelectricity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

The authors gratefully acknowledge the members of the EFAS team, who have contributed to a better understanding of probabilistic flood forecasting and its communication to different end-users. We also thank the meteorological and hydrological services at the origin of the data used in our case-studies.

References

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Flow of uncertainty and risk-based decisions in hydrological forecasting
  5. 3. Case-study I: communication and use of ensemble forecasts in flood warning
  6. 4. Case-study II: risk perception and decision-making in hydroelectricity
  7. 5. Conclusions
  8. Acknowledgements
  9. References
  • Apel H, Aronica GT, Kreibich H, Thieken AH. 2009. Flood risk analyses—how detailed do we need to be? Natural Hazards 49: 7998.
  • Apel H, Thieken AH, Merz B, Blöschl G. 2004. Flood risk assessment and associated uncertainty. Natural Hazards and Earth System Sciences 4: 295308.
  • Atger F. 2001. Verification of intense precipitation forecasts from single models and ensemble prediction systems. Nonlinear Processes in Geophysics 8: 401417.
  • Barredo JI. 2009. Normalised flood losses in Europe: 1970–2006. Natural Hazards and Earth System Sciences 9: 97104.
  • Bartholmes JC, Thielen J, Ramos MH, Gentilini S. 2009. The European flood alert system EFAS—part 2: statistical skill assessment of probabilistic and deterministic operational forecasts. Hydrology and Earth System Sciences 13(2): 141153.
  • Beven KJ. 2000. Uniqueness of place and process representations in hydrological modelling. Hydrology and Earth System Sciences 4(2): 203213.
  • Beven KJ. 2006. On undermining the science? Hydrological Processes 20(14): 31413146.
  • Beven KJ, Smith PJ, Freer JE. 2008. So just why would a modeller choose to be incoherent? Journal of Hydrology 354(1–4): 1532.
  • Block PJ, Souza Filho FA, Sun L, Kwon HH. 2009. A streamflow forecasting framework using multiple climate and hydrological models. Journal of the American Water Resources Association 45(4): 828843.
  • Blöschl G. 2008. Flood warning—on the value of local information. International Journal of River Basin Management 6(1): 4150.
  • Broad K, Leiserowitz A, Weinkle J, Steketee M. 2007. Misinterpretations of the “Cone of Uncertainty” in Florida during the 2004 hurricane season. Bulletin of the American Meteorological Society 88(5): 651667.
  • Buizza R, Asensio H, Balint G, Bartholmes J, Bliefernicht J, Bogner K, Chavaux F, de Roo A, Donnadille J, Ducrocq V, Edlund C, Kotroni V, Krahe P, Kunz M, Lacire K, Lelay M, Marsigli C, Milelli M, Montani A, Pappenberger F, Rabufetti D, Ramos MH, Ritter B, Schipper JW, Steiner P, Thielen-Del Pozzo J, Vincendon B. 2007. EURORISK/PREVIEW report on the technical quality, functional quality and forecast value of meteorological and hydrological forecasts. ECMWF Technical Memorandum 516, ECMWF Research Department: Shinfield Park, Reading, United Kingdom; 21 pages.
  • Changnon SA, Pielke RA, Changnon D, Sylves RT, Pulwarty R Jr. 2000. Human factors explain the increased losses from weather and climate extremes. Bulletin of the American Meteorological Society 81(3): 437442.
  • Cloke H, Pappenberger F. 2009. Ensemble flood forecasting: a review. Journal of Hydrology 375(3–4): 613626.
  • Créton-Cazanave L. 2009. Warning! The use of meteorological information during a flash-flood warning process. Advances in Science and Research 3: 99103.
  • Demeritt D, Cloke H, Pappenberger F, Thielen J, Bartholmes J, Ramos MH. 2007. Ensemble predictions and perceptions of risk, uncertainty, and error in flood forecasting. Environmental Hazards 7(2): 115127.
  • Dietrich J, Trepte S, Wang Y, Schumann AH, Voß F, Hesser FB, Denhard M. 2008. Combination of different types of ensembles for the adaptive simulation of probabilistic flood forecasts: hindcasts for the Mulde 2002 extreme event. Nonlinear Processes in Geophysics 15(2): 275286.
  • Faulkner H, Parker D, Green C, Beven K. 2007. Developing a translational discourse to communicate uncertainty in flood risk between science and the practitioner. Ambio: A Journal of the Human Environment 36: 692703.
  • Gigerenzer G, Hertwig R, van den Broek E, Fasolo B, Katsikopoulos KV. 2005. “A 30% Chance of Rain Tomorrow”: how does the public understand probabilistic weather forecasts? Risk Analysis 25(3): 623629.
  • Golding BW. 2009. Uncertainty propagation in a London flood simulation. Journal of Flood Risk Management 2(1): 215.
  • Guha-Sapir D, Hargitt D, Hoyois P. 2004. Thirty Years of Natural Disasters 1974–2003: the Numbers. Centre for Research on the Epidemiology of Disasters, UCL Presses Universitaires de Louvain: Louvain, Belgium; 190page. http://www.em-dat.net/documents/Publication/publication_2004_emdat.pdf. [Access- ed April 2010].
  • Handmer J, Proudley B. 2007. Communicating uncertainty via probabilities: the case of weather forecasts. Environmental Hazards 7: 7987.
  • Hartmann HC, Bales R, Sorooshian S. 2002. Weather, climate, and hydrologic forecasting for the US Southwest: a survey. Climate Research 21: 239258.
  • He Y, Wetterhall F, Cloke HL, Pappenberger F, Wilson M, Freer J, McGregor G. 2009. Tracking the uncertainty in flood alerts driven by grand ensemble weather predictions. Meteorological Applications 16: 91101.
  • Houdant B. 2004. Contribution à l'amélioration de la prévision hydrométéorologique opérationnelle. Pour l'usage des probabilités dans la communication entre acteurs [Contribution to the improvement of operational hydrometeorological forecasting. For the use of probability in the communication between actors], PhD thesis, ENGREF, EDF, Grenoble, France; 209 page (in French).
  • Jaun S, Ahrens B. 2009. Evaluation of a probabilistic hydrometeorological forecast system. Hydrology and Earth System Sciences 13: 10311043.
  • Joslyn SL, Nadav-Greenberg L, Taing MU, Nichols RM. 2009. The effects of wording on the understanding and use of uncertainty information in a threshold forecasting decision. Applied Cognitive Psychology 23(1): 5572.
  • Kelman I, Spence R. 2003. A flood failure flowchart for buildings. Proceedings of the Institution of Civil Engineers: Municipal Engineer 156(3): 207214.
  • Kloprogge P, van der Sluijs J, Wardekker A. 2007. Uncertainty communication: issues and good practice. Report NWS-E-2007-199, Department of Science, Technology and Society, Utrecht University: Utrecht, The Netherlands; 64 page.
  • KnightL (ed.) 2009. World Disasters Report 2009—Focus on Early Warning, Early Action. International Federation of Red Cross and Red Crescent Societies, ATAR Roto Presse: Satigny/Vernier, Switzerland; 210page. http://www.ifrc.org. [Accessed April 2010].
  • Komma J, Reszler C, Blöschl G, Haiden T. 2007. Ensemble prediction of floods—catchment non-linearity and forecast probabilities. Natural Hazards Earth System Sciences 7: 431444.
  • Krzysztofowicz R. 2001. The case for probabilistic forecasting in hydrology. Journal of Hydrology 249(1–4): 29.
  • Krzysztofowicz R. 2002. Bayesian system for probabilistic river stage forecasting. Journal of Hydrology 268(1–4): 1640.
  • Lugiez F, Guillot P. 1960. Dix années de prévisions d'apports à Electricité de France. IAHS-AISH Publication 51: 558566.
  • McCarthy S, Tunstall S, Parker D, Faulkner H, Howe J. 2007. Risk communication in emergency response to a simulated extreme flood. Environmental Hazards 7(3): 179192.
  • McMillan HK, Brasington J. 2008. End-to-end flood risk assessment: a coupled model cascade with uncertainty estimation. Water Resources Research 44(3): W03419. DOI: 10.1029/2007WR005995.
  • Mantovan P, Todini E. 2006. Hydrological forecasting uncertainty assessment: incoherence of the GLUE methodology. Journal of Hydrology 330: 368381.
  • Mathevet T, Garavaglia F, Garçon R, Gailhard J, Paquet E. 2009. Operational hydrological ensemble forecasts in France. Recent development of the French Hydropower Company (EDF), taking into account rainfall and hydrological model uncertainties. Geophysical Research Abstracts, Vol. 11, EGU2009-10248.
  • Montanari A, Brath A. 2004. A stochastic approach for assessing the uncertainty of rainfall-runoff simulations. Water Resources Research 40: W01106. DOI: 10.1029/2003WR002540.
  • Morss RE, Demuth JL, Lazo JK. 2008. Communicating uncertainty in weather forecasts: a survey of the U.S. public. Weather and Forecasting 23: 974991.
  • Morss RE, Wahl E. 2007. An ethical analysis of hydrometeorological prediction and decision making: the case of the 1997 red river flood. Environmental Hazards 7: 342352.
  • Murphy AH. 1993. What is a good forecast? An essay on the nature of goodness in weather forecasting. Weather and Forecasting 8: 281293.
  • NRC (National Research Council). 2006. Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. Committee on Estimating and Communicating Uncertainty in Weather and Climate Forecasts, National Research Council, The National Academies Press: Washington, D.C.; 124 page. www.nap.edu. [Accessed April 2010].
  • Obled C, Bontron G, Garçon R. 2002. Quantitative precipitation forecasts: a statistical adaptation of model outputs though an analogues sorting approach. Atmospheric Research 63: 303324.
  • Olsson J, Lindström G. 2008. Evaluation and calibration of operational hydrological ensemble forecasts in Sweden. Journal of Hydrology 350: 1424.
  • Önkal D, Goodwin P, Thomson M, Gönül S, Pollock A. 2009. The relative influence of advice from human experts and statistical methods on forecast adjustments. Journal of Behavioral Decision Making 22(4): 390409.
  • Pappenberger F, Beven KJ. 2006. Ignorance is bliss: or 7 reasons not to use uncertainty analysis. Water Resources Research 42: W05302; DOI: 10.1029/2005WR004820.
  • Pappenberger F, Beven KJ, Hunter MN, Bates PD, Gouweleeuw BT, Thielen J, de Roo APJ. 2005. Cascading model uncertainty from medium range weather forecasts (10 days) through a rainfall-runoff model to flood inundation predictions within the European Flood Forecasting System (EFFS). Hydrology and Earth System Sciences 9(4): 381393.
  • Pitt M. 2008. The Pitt review: learning lessons from the 2007 floods. Cabinet Office, 22 Whitehall, London SW1A 2WH; 462page. http://archive.cabinetoffice.gov.uk/. [Accessed April 2010].
  • Ramos MH, Bartholmes J, Thielen J. 2007. Development of decision support products based on ensemble weather forecasts in the European Flood Alert System. Atmospheric Science Letters 8(4): 113119.
  • Renner M, Werner MGF, Rademacher S, Sprokkereef E. 2009. Verification of ensemble flow forecasts for the River Rhine. Journal of Hydrology 376: 463475.
  • Romanowicz RJ, Beven KJ, Young PC. 2006. Uncertainty propagation in a sequential model for flood forecasting. IAHS-AISH Publication 303: 177184.
  • Roulston MS, Bolton GE, Kleit AN, Sears-Collins AL. 2006. A laboratory study of the benefits of including uncertainty information in weather forecasts. Weather and Forecasting 21(1): 116122.
  • Schaefli B, Balin Talamba D, Musy A. 2007. Quantifying hydrological modeling errors through a mixture of normal distributions. Journal of Hydrology 332: 303315.
  • Thielen J, Bartholmes J, Ramos MH, de Roo A. 2009a. The European flood alert system—part 1: concept and development. Hydrology and Earth System Sciences 13(2): 125140.
  • Thielen J, Bogner K, Pappenberger F, Kalas M, del Medico M, de Roo A. 2009b. Monthly-, medium-, and short-range flood warning: testing the limits of predictability. Meteorological Applications 16(1): 7790.
  • Thielen J, Ramos MH, Bartholmes J, de Roo A, Cloke H, Pappenberger F, Demeritt D. 2005. Summary report of the 1st EFAS workshop on the use of Ensemble Prediction System in flood forecasting, 21-22nd November 2005, Ispra. European Report EUR 22118 EN, European Communities: Italy; 23page. http://floods.jrc.ec.europa.eu/efas-documents. [Accessed April 2010].
  • Thirel G, Rousset-Regimbeau F, Martin E, Habets F. 2008. On the impacts of short-range meteorological forecasts for ensemble stream flow predictions. Journal of Hydrolometerology 9(6): 13011317.
  • Todini E. 2004. Role and treatment of uncertainty in real-time flood forecasting. Hydrological Processes 18(14): 27432746.
  • Wahlström M. 2009. Disaster risk reduction, climate risk management and sustainable development. WMO Bulletin 58(3): 165174.
  • WMO (World Meteorological Organization). 2008. Guidelines on communicating forecast uncertainty. Technical document PWS-18 WMO/TD 1422; 25page. www.wmo.int. [Accessed April 2010].
  • WMO (World Meteorological Organization). 2009. Conference statement—Summary of the expert segment. World Climate Conference-3. Geneva, 31 August—4 September 2009, Conference statement, Summary of the Expert Segment, 41page. http://www.wmo.int/wcc3. [Accessed April 2010].
  • Zorita E, von Storch H. 1999. The analog method as a simple statistical downscaling technique: comparison with more complicated methods. Journal of Climate 12: 24742489.