A long-term assessment of precipitation forecast skill using the Fractions Skill Score
Article first published online: 4 OCT 2011
Copyright © 2011 British Crown Copyright, the Met Office. Published by John Wiley & Sons Ltd.
Special Issue: Progress and challenges in forecast verification. Guest editors: A. Ghelli, C. Coelho, M. Mittermaier and C. Power
Volume 20, Issue 2, pages 176–186, June 2013
How to Cite
Mittermaier, M., Roberts, N. and Thompson, S. A. (2013), A long-term assessment of precipitation forecast skill using the Fractions Skill Score. Met. Apps, 20: 176–186. doi: 10.1002/met.296
- Issue published online: 10 JUN 2013
- Article first published online: 4 OCT 2011
- Manuscript Accepted: 8 AUG 2011
- Manuscript Revised: 1 AUG 2011
- Manuscript Received: 7 FEB 2011
- high-resolution NWP;
- spatial verification methods, precipitation;
- long-term monitoring against radar
The Fractions Skill Score (FSS) is a spatial verification metric routinely computed in the operational verification suite. It enables the comparison of forecasts of different resolutions against a common spatial truth (radar rainfall analyses) in such a way that high-resolution forecasts are not penalized for representativeness errors that arise from the ‘double penalty’ problem. Officially Met Office model precipitation forecast accuracy is monitored using the Equitable Threat Score (ETS) at gauge locations. These precipitation scores form part of a basket of measures assessing six surface parameters known as the UK index, which forms the basis for making decisions regarding model upgrades (especially over the UK). It is used to monitor the impact of continuous model improvements. This framework and the methodology underlying it, is less appropriate for high-resolution forecasts for reasons as described above. For precipitation forecasts in particular, a new framework for long-term monitoring is necessary and the FSS provides such a potential framework.
This paper provides an objective critique of FSS results to date. It has been shown that the ‘convection-permitting’ (4 km) Unified Model (MetUM) forecasts are better than the 12 km MetUM (significant at the 5% level). The scale at which the models have sufficient practical skill is typically 10 km better for the high-resolution forecasts, and are better at forecasting afternoon convection exceeding 4 mm (6 h)−1. The use of frequency (percentile) thresholds is recommended because of the implicit bias removal this approach provides, as any rain in a forecast period is treated as ‘the event of interest’. © 2011 British Crown Copyright, the Met Office. Published by John Wiley & Sons Ltd.