• Open Access

Aims, challenges and progress of the Hydrological Ensemble Prediction Experiment (HEPEX) following the third HEPEX workshop held in Stresa 27 to 29 June 2007



Since several years, users of weather forecasts have begun to realize the benefit of quantifying the uncertainty associated with forecasts rather than relying on single value forecasts. At the same time, hydrologists and water managers have begun to explore the potential benefit of ensemble prediction systems (EPS) for hydrological applications. The Hydrologic Ensemble Prediction Experiment (HEPEX) is an international project that aims to foster the development of probabilistic hydrological forecasting and corresponding decision making tools. Since 2004, HEPEX has provided discussion opportunities for hydrological and meteorological scientists involved in the development, testing, and operational management of forecasting systems, and end users. Copyright © 2008 Royal Meteorological Society

1. Introduction: what is HEPEX and what are its goals?

HEPEX stands for Hydrological Ensemble Prediction Experiment and is an international effort that brings together hydrological and meteorological communities to develop advanced probabilistic hydrological forecast techniques that use emerging weather and climate ensemble forecasts (Schaake et al., 2006a,b). HEPEX was launched in 2004 as an independent, cooperative international scientific activity comprised primarily of researchers, forecasters, water managers, and users.

The overarching goal for HEPEX was defined during the first meeting as ‘to develop and test procedures to produce reliable hydrological ensemble forecasts, and to demonstrate their utility in decision making related to the water, environmental, and emergency management sectors’.

Hydrological ensemble predictions can span across large spatiotemporal ranges from short-term and very localized predictions to global climate change modeling. Figure 1 illustrates different components of a ‘hydrological ensemble prediction system (EPS) for products and services’. This system can be split into five main components: land data assimilator (hydrological, meteorological, and geographical observations), atmospheric ensemble preprocessing modules of weather forecasting data, hydrological models, hydrological ensemble processor, and product generator. Ultimately it is the interaction of these five components that determines the quality, usefulness, and applicability of hydrological ensemble products. The impact of improved products could be manifold and found for example in the economy (hydropower, agriculture), public safety (flood prevention, flood damage reductions), environmental management, and also health (reduction of diseases induced through flooding).

Figure 1.

Schematic view of different elements of a hydrological ensemble prediction system where boxes representing input are shaded gray, those representing processes shaded blue, and those representing output (products and services) shaded yellow. The arrows indicate direction of flow of information

2. HEPEX information exchange platform—the internet, workshops, and test-beds

HEPEX is sharing information through the internet, workshops and most importantly through collaborative work in test-bed projects.

2.1. The internet

Currently the HEPEX homepage (http://hydis8.eng.uci.edu/hepex/) gives access to key documents. The webpage is open access and contains announcements, documents such as the HEPEX implementation plan or information on HEPEX conference sessions, and documentation from HEPEX workshops.

2.2. Workshops

Since its foundation, three international HEPEX workshops have taken place.

The first HEPEX workshop took place in March 2004 in Reading, the United Kingdom (http://www.ecmwf.int/newsevents/meetings/workshops/2004/HEPEX/) At this meeting, scientists and experts with a background in meteorology and hydrology identified the potential of HEPEX to foster knowledge sharing and communication between these two communities.

Some fundamental science questions were identified during the first HEPEX workshop and they constituted the core of HEPEX activities since then. They are as follows:

  • - What is the influence of increased resolution for both the land-surface scheme and for the atmospheric model on the accuracy and reliability of ensemble streamflow forecasts and ensemble weather forecasts?

  • - What is the benefit of using global, medium-range, and short-range ensemble weather forecasts for hydrological modeling of reservoir inflow and outflow?

  • - What is the best method for combining forecasts with different lead times?

  • - How can skillful and reliable meteorological forcing during the forecast period be generated for seasonal hydrological forecasting?

  • - How can hydrological ensembles be generated that reflect all known uncertainties?

  • - How can climate information, such as climate model forecast, be used with confidence in seasonal hydrological forecast?

  • - How can hydrological ensembles be validated for extreme events?

Also more practically oriented and at times quite specific questions on methodologies are investigated, for example:

  • - Does ensemble forecasting require different parameter estimation approaches than deterministic forecasting, such as multiple parameter sets or tailored objective functions?

  • - How can automatic calibration aid in characterizing uncertainty?

  • - What are the effective techniques for data assimilation of snow?

  • - How can systematic over- or under-prediction of rainfall forecasts from both deterministic systems and EPS be detected and corrected for better flood forecasting?

  • - How can different types of forecast information (e.g. climate indices, climate model outputs) be effectively combined?

  • - What are the advantages and limitations of different methods for extracting information from Numerical Weather Prediction models, for the purposes of forecasting streamflow?

The second HEPEX workshop was held at the National Centre for Atmospheric Research (NCAR), in Boulder, Colorado, in July 2005 where state-of-the-art research in hydrological and meteorological ensemble forecasting, ranging from short-term flashflood to seasonal drought forecasting was discussed. The main goal of this workshop was to formulate an implementation strategy to address key science questions and to test alternative forecast approaches (Wood et al., 2006). One of the key elements of this strategy was the formation of test-bed projects to address the science questions formulated earlier. A test-bed is an archived collection of relevant hydrological and meteorological data and models for specific basins or locations, where a variety of forecast approaches and methodologies can be tested and intercompared. During the second HEPEX workshop, the first eight test-beds were defined.

The third HEPEX workshop, held in Stresa, Italy, from 27 to 29 June 2007 (Hartman and Schaake, 2007), focused on the results obtained on the science questions. Progress from the test-bed projects that started in July 2005 was reported and several new test-beds were proposed. Furthermore, new research on weather and climate forecast applications, hydrological ensemble processing, and uncertainty in hydrometeorological forecasting were presented. Discussions and outcome of the third HEPEX workshop are detailed in Section 3 of this paper.

2.3. Test-beds (HEPEX-community experiments)

The aim of the test-beds is to create a platform on which important science questions can be explored or methods can be developed that improve the performance of hydrological ensemble predictions for better decision making. The definition of test-beds is relatively broad. A test-bed can be a single basin with or without subbasins, a region containing multiple basins, or possibly a global collection of sites that facilitate experiments addressing questions over a range of scales and climates. Regardless of geographical domain, test-beds focus on one or more clearly defined HEPEX science questions and have the potential to develop data resources needed for community experiments.

A description of the eight initial test-beds, presented at the second HEPEX workshop (Figure 2), their goals and methodologies are described in short reports listed on the HEPEX webpage. A summary of the topics of the eight test-beds are listed below (for more detail see http://hydis8.eng.uci.edu/hepex/). Due to the heterogeneous nature of the HEPEX community, similar questions can be addressed in different test-beds.

  • 1.Great Lakes (Canada/USA): Importance of detailed atmospheric and hydrological modeling for medium-range atmospheric and hydrological forecasting on large basins.
  • 2.Bangladesh: Provide operational real-time forecasts of river discharge into Bangladesh at daily, weekly, monthly, and seasonal timescales.
  • 3.Rio Grande Basin (Brazil): Explore (1) the use of global ensembles, (2) the use of Regional Atmospheric Modeling System (RAMS) forecasts for extending lead times up to a month and longer, and (3) the use of short-term rainfall forecasts from the operational ETA model.
  • 4.Po Basin (Italy): (1) Removal of bias in meteorological forecasts for flood forecasting in geographically dominated terrain, and (2) test methods for flood forecasting based on threshold exceedance.
  • 5.Western Basins (USA, British Columbia, Canada): Development of hydrological forecasting techniques that are particular to the orographically complex, snowmelt-driven basins of the Western USA and British Columbia with focus on monthly to seasonal forecasts.
  • 6.Eastern Basins (USA): Research on (1) the generation of skilful and reliable meteorological forcing for seasonal hydrological forecasting, (2) the generation of hydrological ensembles that reflect the total uncertainties, and (3) how to use climate information reliably in seasonal hydrological ensembles for extreme events.
  • 7.Statistical downscaling: Identification of (1) space-time scales for which forecast skill is present for different variables, (2) Global Forecast System (GFS) output variables that can be used to provide subgrid information as input to statistical models for replication of precipitation processes, and (3) sample size required to reliably forecast precipitation, temperature, and streamflow for different thresholds.
  • 8.Hydrologic uncertainty: advantages and limitations of different methods for characterizing and reducing different uncertainties in hydrological model simulations.
Figure 2.

Eight test-bed projects initiated during the second HEPEX workshop in 2005

In addition to the existing eight test-beds, the following new test-beds focusing on key topics were proposed during the third HEPEX workshop to cover science questions not specifically addressed before or to provide data not available with the previous test-beds:

  • 1.Use of weather forecasts in operational electricity production (USA)
  • 2.Hydrologic model parameter uncertainty
  • 3.Ensemble precipitation analysis
  • 4.Seasonal forecasts within a changing climate in the Rhine River basin—an end-to-end process with end users
  • 5.Value and application of probabilistic forecasts for end users in the Tuolumne and Cedar River watersheds (USA)
  • 6.Hydrologic postprocessors
  • 7.Ensemble and probabilistic product generation for customers and partners
  • 8.End-to-end EPS in France
  • 9.Mesoscale Alpine Project (MAP)-D phase
  • 10.Verification techniques

These ten proposed test-bed projects and the original eight established in 2005 can be grouped in two types: the majority (11) focuses on scientific issues to make end-to-end ensemble forecasts for a particular river basin, whereas the others (7) address specific science issues that potentially cut across all of the basin-oriented projects.

3. Results from the third HEPEXworkshop

The third HEPEX workshop was organized in three sessions. The first session was devoted to discussing the existing test-beds (progress from six of the eight test-beds was reported). The second session featured presentations from participants on various aspects of ensemble weather and climate forecast applications, hydrological ensemble processing, best practices for analyzing and visualizing uncertainty, and user perspectives. Finally, the third session consisted of discussion groups on user-oriented issues, hydrometeorological forcing, and sources of uncertainty.

Most of the oral and poster presentations are included in the Book of Abstracts (Thielen et al., 2007) that can be downloaded from the HEPEX webpage. This special edition of Atmospheric Science Letters (ASL) contains a selection of the papers on the results presented during the HEPEX workshop.

A wide range of topics was presented on meteorological and hydrological application at different temporal scales. Recent developments in meteorological ensemble forecasts relevant for hydrological applications were presented (for example Buizza, 2007; Bonta, 2007; Diomede et al., 2007; Gebhardt et al., 2007) including analogue methods (Marty et al., 2007) that provide promising results. The increase in hydrological ensemble forecasting studies for operational flood forecasting and hydropower generation was encouraging (e.g. Csik et al., 2007; Hartman, 2007; Howard, 2007; Tucci et al., 2007). Verification and skill score studies (for instance Bartholmes et al., 2007; Pappenberger et al., 2007; Pietroniro et al., 2007) complemented the case studies both for meteorological and hydrological applications and demonstrated that the methodologies of the two communities have been approaching each other over the past years.

Thematically, the main results of the workshop can be divided into the following topics:

  • Uncertainty: scientific challenges and approaches

  • Decision making based on uncertain results

  • Additional test-bed projects

  • Future activities and workshops

3.1. Uncertainty: scientific challenges and approaches

The first challenge is to identify the sources of uncertainty in a system and to understand how these uncertainties are propagated through complex nonlinear systems. Questions that have been addressed are, for example, ‘what is the degree and impact of uncertainty associated with the data upon which our research and products are based?’ How certain are ‘actual’ measurements, in particular during severe or extreme events? How reliable are estimates from remote sensing devices? Once having identified these sources of uncertainty, we need to understand how the uncertainties are routed through the system of a hydrological ensemble prediction generator. What is the degree of modeling uncertainty arising from individual model components? How is uncertainty cascaded through nonlinear hydrological processes? Even if the different components of uncertainty have been identified for specific scales and processes, how can these uncertain information be combined across different temporal and spatial scales and still provide reliable information? And finally, even if identified and understood, how can we make sure that the information provides added value for targeted decision making processes?

It is important that a measure of the uncertainty—whether it is arising from the input, the model, or the postprocessing—is provided with any forecasts or simulations, e.g. in the form of spread, error bars, exceedances, or other representations. There was consensus amongst the workshop participants, however, that in order to represent added value for end users, the uncertainty information must be meaningful and lead to improved forecast skill and also allow to make better decisions than without it. For example, information on discharge spread ranging over several 1000 m3/s becomes meaningless for decision making.

In fact, several presentations during the workshop showed very encouraging results on the subject of increasing skill and reducing uncertainty. For example, research results from the Great Lakes test-bed suggests that the skill in probabilistic forecasting increases when EPS are combined to multi-ensembles, rather than having individual EPS only. This result is particularly interesting because with the international project TIGGE (http://www.wmo.ch/pages/prog/arep/thorpex/GIFS_TIGGE_WG.html) (the THORPEX Interactive Grand Global Ensemble project (THORPEX is The Observing System Research and Predictability Experiment programme, promoted by the World Meteorological Organization)) a large multiensemble database will soon be available for testing this hypothesis. Todini et al. (2007) proposed combining different methods and types of models to improve robustness of the forecasts and reduce uncertainty.

Postprocessing routines capable of reducing or correcting the uncertainty of the hydrological ensemble model output were discussed as the best way forward for operational applications. For example Bogner et al. (2007) presented different methods of bias correction for adjusting the ensemble traces using a transformation derived with simulated and observed flows. His results from the 2002 Danube floods case study showed encouraging results to improve the usefulness of ensembles in operational flood forecasting.

3.2. Decision making based on uncertain results

Having established that all meteorological and hydrological forecast results are invariably associated with a more or lesser degree of uncertainty, the following questions that need addressing were identified: who are finally the end users of these forecasts? If they are decision makers, how should the uncertainty results be conveyed to them? How valuable are ensemble forecasts for end users and does the value of a forecast depends on the degree of forecast uncertainty? How can sound decisions be made using ensemble forecasts? What kinds of forecast products are needed to support user-decision processes and do they already exist or need to be developed?

As mentioned earlier, the value of a forecast must be linked to the optimal decision based on that forecast. This value may depend on the accuracy (i.e. degree of uncertainty) of the forecast and on the reliability of the uncertainty information. Unfortunately this also means that forecast value depends very much on the circumstances and varies from one end user to another and from event to event. Therefore, guidance is needed for end users on how to deal with uncertainty.

Because the reliability of a probabilistic forecasting system cannot be estimated from single cases, decision makers must rely on the performance of the forecasting system over an adequately long period of time. This is not trivial, particularly when hydrological forecasting systems rely on data from weather forecasting models, which are continuously improved over time (e.g. physics, representation of subgrid scale parameterization, grid spacing, etc.). Very few meteorological services do—or can—provide 30 years of reforecasting for their recently updated models, although they are critically needed for this type of application and assessment. The validity of hydrological ensemble reforecasts is a function of the degree to which currently employed models, techniques, data, and interaction can be emulated over a recent historical period for which reliable observations are available.

Sound decision making also requires knowledge of the past performance of the system, which can be expressed through skill scores. Meteorological forecasting centers the issue that EPS are already in the habit of systematically calculating skill scores over past performance. In hydrological applications, however, where EPS are still mostly only explored and studied at case study basis, the long-term skill assessments are not yet systematically done. Also, Bartholmes et al. (2007) reviewed a number of existing skill scores for meteorological applications and illustrate that not all of them are adapted for hydrological ensemble forecasts.

3.3. Future activities

The core of HEPEX activities remain the test-beds. The newly proposed ten test-bed projects together with the original eight that began in 2005 form a very large activity. These are expected to work together to form a tightly knitted matrix of techniques and experimental applications. The cells of this matrix represent opportunities for collaboration to develop between the ‘end-to-end’ and ‘cross-cutting science’ test-beds. This collaborative process is just beginning. For example, the North American Great Lakes test-bed invites other researchers to test their forecasting models, downscaling, and verification techniques for an intercomparison study on a subbasin of the test-bed for which data are made available daily through the ftp site. The downscaling test-bed hopes to collaborate with some of the end-to-end test-beds to test alternative downscaling techniques. Furthermore, efforts will be made to enlarge the participation to HEPEX to anyone who can make a valuable contribution to the core activities. Particular efforts will be made to engage participants from, Asia, Africa and Oceania.

In addition to these core activities, HEPEX envisages to address key science issues over the next 2 to 3 years through thematic workshops, improve the understanding between the meteorological and hydrological communities, and identify end-users' needs.

The following three thematic workshops are planned in addition to the fourth international HEPEX workshop in 2009/2010:

  • Downscaling of atmospheric forecasts to produce reliable ensemble forcing for hydrological models

  • Postprocessing techniques to improve the reliability of hydrological ensemble forecasts

  • End-user applications and requirements.

The following two position papers have been proposed to foster the understanding of meteorological services of the specific needs of the hydrological forecasting community:

  • ‘The importance of meteorological model reforecasts for the hydrological forecasting and water management communities’

  • ‘The advances of the hydrometeorological community in the field of uncertainty assessment’

In addition, end user's views will be specifically addressed in a survey on hydrological ensembles and forecast uncertainty.

4. Vision for the future of HEPEX

One of the most important objectives for HEPEX is to demonstrate that its work is addressing and meeting real user needs. To achieve this goal, end users will be increasingly involved in a more systematic way in the HEPEX activities. Their involvement will not only be ‘top-down’ but also ‘down-up,’ and cutting across different steps of the generation of probabilistic forecasts and through this they will effectively create the motor of further HEPEX activities. Through competence building, it will also be increasingly playing a role or gaining impact in other important organizations such as:

  • The International Association of Hydrological Sciences (IAHS)—The IAHS formed a new Working Group on Hydrometeorological Projects at its last meeting in Perugia, Italy in July 2007 to facilitate collaboration with projects such as HEPEX. The IAHS project Prediction for Ungaged Basins (PUB) is a major potential source to meet some of HEPEX science requirements

  • World Climate Research Programme (WCRP)/Global Energy and Water Cycle Experiment/Hydrology Application Project (GEWEX/HAP) which address similar problems as HEPEX

  • Group of Earth Observations/Global Earth Observation System of Systems (GEO/GEOSS) which may benefit from HEPEX results in defining new observational products

  • United Nations Educational, Scientific, and Cultural Organization (UNESCO)

  • World Meteorological Organisation/Hydrological and Water Resources Programme (WMO/HWRP)

  • World Weather Research Program (WWRP)/The Observice System Research and Predictability Experiment (THORPEX)/THORPEX Interactive Grand Global Ensemble (TIGGE) to explore the potential of superensemble prediction systems for hydrological applications

  • Operational forecast organizations and water management professional organizations to ensure higher involvement of end users

The HEPEX User Council is envisaged to provide a stronger leadership to the whole project to achieve these goals. Ultimately HEPEX will increase knowledge about probabilistic forecasting and uncertainty in hydrological applications and also provide guidance for different user communities on how to make decisions based on probabilistic forecasts and how to communicate it to different end users.