Validation of the forecast skill of the Global Modeling and Assimilation Office Observing System Simulation Experiment



A global Observing System Simulation Experiment (OSSE) framework has been developed at the National Aeronautics and Space Administration Global Modeling and Assimilation Office (NASA/GMAO). The OSSE uses a 13-month integration of the European Centre for Medium-Range Weather Forecasts (ECMWF) operational model as the Nature Run, and the Goddard Earth Observing System version-5 (GEOS-5) forecast model with Gridpoint Statistical Interpolation (GSI) data assimilation as the forecast model. Synthetic observations have been developed with correlated observation errors to replicate the observing network from 2005–2006.

The performance of the GMAO OSSE in terms of forecast skill and observation impacts is evaluated against real observational data for the same period. Metrics of anomaly correlation of 500 hPa geopotential and root-mean-square error of temperature and wind fields for 120 h forecasts are calculated for once-daily forecasts from July 2005, and an adjoint is used to measure observation impacts of different data types. The forecast skill of the OSSE is found to be slightly higher than for real data, with smaller observation impacts overall, possibly due to insufficient model error in the OSSE. While there is similar relative ranking of observation impact for most data types in the OSSE compared with real data, for individual satellite channels the agreement is not as good. Some caveats and difficulties of using the OSSE system are discussed along with recommendations of how to avoid potential pitfalls when performing OSSEs.