Three recommendations for evaluating climate predictions
Article first published online: 10 JUN 2013
© 2013 The Authors. Meteorological Applications published by John Wiley & Sons Ltd on behalf of the Royal Meteorological Society.
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
Special Issue: Progress and challenges in forecast verification. Guest editors: A. Ghelli, C. Coelho, M. Mittermaier and C. Power
Volume 20, Issue 2, pages 246–255, June 2013
How to Cite
Fricker, T. E., Ferro, C. A. T. and Stephenson, D. B. (2013), Three recommendations for evaluating climate predictions. Met. Apps, 20: 246–255. doi: 10.1002/met.1409
- Issue published online: 10 JUN 2013
- Article first published online: 10 JUN 2013
- Manuscript Accepted: 10 APR 2013
- Manuscript Revised: 9 APR 2013
- Manuscript Received: 14 SEP 2012
- decadal hindcasts;
- spurious skill;
Evaluation is important for improving climate prediction systems and establishing the credibility of their predictions of the future. This paper shows how the choices that must be made about how to evaluate predictions affect the outcome and ultimately our view of the prediction system's quality. The aim of evaluation is to measure selected attributes of the predictions, but some attributes are susceptible to having their apparent performance artificially inflated by the presence of climate trends, thus rendering past performance an unreliable indicator of future performance. We describe a class of performance measures that are immune to such spurious skill. The way in which an ensemble prediction is interpreted also has strong implications for the apparent performance, so we give recommendations about how evaluation should be tailored to different interpretations. Finally, we explore the role of the timescale of the predictand in evaluation and suggest ways to describe the relationship between timescale and performance. The ideas in this paper are illustrated using decadal temperature hindcasts from the CMIP5 archive. © 2013 The Authors. Meteorological Applications published by John Wiley & Sons Ltd on behalf of the Royal Meteorological Society.