STORM: An empirical storm-time ionospheric correction model 2. Validation


  • E. A. Araujo-Pradere,

    1. Cooperative Institute for Research in Environmental Sciences, University of Colorado, Space Environment Center, National Oceanic and Atmospheric Administration, Boulder, Colorado, USA
    Search for more papers by this author
  • T. J. Fuller-Rowell

    1. Cooperative Institute for Research in Environmental Sciences, University of Colorado, Space Environment Center, National Oceanic and Atmospheric Administration, Boulder, Colorado, USA
    Search for more papers by this author


[1] STORM is an empirical ionospheric correction model designed to capture the changes in F region electron density during geomagnetic storms. The model is driven by the previous 33 hours of ap, and the output is used to scale the quiet time F region critical frequency (foF2) to account for increases or decreases in electron density resulting from a storm. The model provides a simple tool for modeling the perturbed ionosphere. The quality of the model has been evaluated by comparing the predictions of the model with the observed ionospheric response during the six storms in the year 2000. The model output has been compared with the actual ionospheric response at 15 ionosonde stations for each storm. The comparisons show that the model captures the decreases in electron density particularly well in summer and equinox at midlatitudes and high latitudes but is less accurate in winter. The value of the model has been quantified by comparing the daily root mean square error of the STORM predictions with the monthly mean. The results of the validation show that there is a 33% improvement of the STORM model predictions over the monthly median during the storm days and that the model captures more than half of the increase in variability on the storm days, a significant advance over climatology.


1. Introduction

[2] STORM is the first empirical model of the response of the ionosphere to a geomagnetic storm that has demonstrated a consistent and measurable improvement over climatology. The first characterization of STORM has been designed to adjust the F-region peak critical frequency (foF2) as function of geomagnetic latitude, season, and intensity of the storm [Araujo-Pradere et al., 2002; Fuller-Rowell et al., 2001; Araujo-Pradere and Fuller-Rowell, 2000]. The model is purely empirical, being based solely on an analysis of an extensive database of ionosonde observations, but the algorithms and data sorting procedure has been guided by numerical simulations from a coupled thermosphere ionosphere model [Fuller-Rowell et al., 1996a]. The intensity of the storm is characterized by a new index derived from filtering the previous 33 hours of ap. Several limited validations of the STORM predictions have been published [e.g., Araujo-Pradere and Fuller-Rowell, 2001; Araujo-Pradere, 2002]. In this paper, we will introduce an analysis of the errors on the prediction of the STORM model, and present the results from a comprehensive validation against all available F2 region critical frequency (foF2) measurement for each storm in the year 2000 with maximum ap > 150, the threshold used by the NOAA Space Weather Scales ( for geomagnetic storms with strong (G3) and higher descriptor. The storms occurred on April 5, May 23, July 13, August 10, September 15, and October 3, and for each storm, an average of 15 stations were used in the analysis. The results are shown for 5-day periods for each storm, to cover the response and recovery phase of the ionosphere. The seasonal coverage of the six storms is not complete so this first attempt at validation is a first step in quantifying the accuracy of the model. The daily root mean square error (RMSE = (Σ(model − data)2)/24)0.5)) has been chosen as the metric to quantify the model predictions, which are compared with the corresponding values using the monthly mean. During the discussion of the statistical results, we will use a storm and non-storm day classification, where storm days correspond to Dst < −100 nT, and non-storm days are when Dst never goes below −50 nT. During the storm days, the validation demonstrates a consistent improvement over the predictions using the monthly mean. For periods of geomagnetic quiet, Dst > −50, the model does not normally predict any change from the monthly mean (correction factor = 1). The main goal is to determine if the empirical storm model shows a quantitative improvement compared with the monthly mean or quiet time reference models such as the International Reference Ionosphere (IRI; Bilitza [1990], Bilitza et al. [1993]).

2. Overview of the Empirical Model

[3] A meticulous description of the STORM model has been presented in the companion paper [Araujo-Pradere et al., 2002] so only a brief review will be presented here. The model was developed by analyzing the global distribution of ionosonde data for many storms over the last 20 years. The analysis techniques were guided by new understanding from physical model simulation [Fuller-Rowell et al., 1996a, 1996b] regarding the more persistent aspects of the storm response, and the seasonal/latitude dependence. Much of this longer-term, consistent response is believed to arise from development of neutral composition changes driven by the magnetospheric sources, and their subsequent movement by the global wind field. Clearly there will be more dynamic, storm-specific responses that are much harder to capture in an empirical model, but this first version of the model already demonstrates that reasonable fraction of the ionospheric storm response is repeatable from storm to storm provided the data is sorted in a physically realistic way.

[4] The theory developed from the numerical simulations implied the need for a new storm index to quantify the integrated effect of Joule and particle heating for the duration of the storm, and to estimate the time for the composition changes to recover. It has been understood for a long time that the ionosphere takes a day or two to recover from a large geomagnetic storm, which is believed to result from the gradual diffusion of neutral species to their pre-storm distribution. The index is derived by applying a filter weighting function to the previous 33 hours of ap. The optimum shape and length of the filter was obtained by the singular value decomposition method [Detman and Vassiliadis, 1997], minimizing the mean square difference between the filter input (ap index) and filter output (foF2 ratios = foF2observed/foF2monthly mean). We refer to this index either as filtered ap or integrated ap. Using this index as a measure of the intensity of a storm, the ionosonde data were sorted by latitude and season.

[5] Including all the features, the algorithm that describes the empirical model is given by Fuller-Rowell et al. [1998]:

equation image

where Φ = foF2obs/foF2mm (foF2observed/foF2monthly mean), X(t0) = ∫F(τ)P(t0 − τ)dτ, and F(τ) is the filter weighting function of the ap index, P, over the 33 previous hours, and α corresponds to the local time of the maximum in the diurnal variation. The coefficients a0, a1, a2 and a3 have been adjusted to fit the non-linear relationship between the ionospheric response and the integral of the geomagnetic index ap, and are a function of season and latitude.

[6] As output, the model provides a Correction Factor (Φ) used to scale the IRI or any other quiet time reference (QT), such as the monthly mean, using the expression:

equation image

The model is triggered when the filtered ap exceeds 200 units, i.e.

equation image

which is equivalent to an average ap of about 9, or a Kp of 2+, for the previous time history. This threshold also usually corresponds to days when Dst never drops below −50 nT. This avoids making a correction for quiet conditions, for which the model is not designed. For quiet geomagnetic conditions, the use of the monthly mean, the global IRI model, or any other quiet time reference (Φ = 1), is adequate.

[7] The STORM model is currently running in an operational mode. A real time version of the model has been implemented, using the hourly values of the 3-hour running ap, as provided by the USAF Hourly Magnetometer Analysis Reports. Hourly updates of the model predictions, in six latitude bands, can be found at

3. Error Analysis

[8] Figure 1 shows all the ionospheric data used to produce the model sorted by season (5 intervals), latitude (4 regions), and storm intensity (filtered ap). Using this procedure, a definite trend can be seen in the data, particularly in summer and equinox, but there is also significant scatter about the fit to the distribution resulting from geophysical variability. The error bars on this figure represent the standard deviation, or scatter about the mean, due to geophysical variability, which would represent the error on the model prediction when applied to a given station for a particular storm.

Figure 1.

Sort of the storm-time ionospheric response into four geomagnetic bins (60–80, 0–20, 20–40, 40–60) and five seasonal bins (from summer to winter, including intermediates seasons). Each panel shows the relationship between the foF2 ratio and the integral of ap. The fit to the data used in the model is shown in each panel, and the error bars represent the estimated geophysical variability (standard deviation).

[9] The error on the mean (standard deviation of mean) of the distribution, which is the error on the fit to the data, is substantially less than the scatter from the geophysical variability. Figure 2 shows the same information where the error bar reflects the error on the mean, which represents the accuracy of the STORM model prediction of the average ionospheric response at a given site.

Figure 2.

Sort of the storm-time ionospheric response into four geomagnetic bins (60–80, 0–20, 20–40, 40–60) and five seasonal bins (from summer to winter, including intermediates seasons). Each panel shows the relationship between the foF2 ratio and the integral of ap. The fit to the data used in the model is shown in each panel, and the error bars represent the estimated standard deviation of the mean.

[10] In most of the intervals, the estimates of the size of the error bars, both the geophysical variability and the standard deviation of mean, were derived from the data itself. In some regions, insufficient data was available to make a reliable estimate so the values from neighboring bins were uses to estimate the error. For the highest levels of storm perturbation, where no data was available, the error of the prediction was assumed to gradually increase. In some case, therefore, the error estimates are not based on real information so clearly should be treated with caution.

4. Data Sources for the Validation

[11] The only criteria in the selection of the stations were that data was available in the NGDC Space Physics Interactive Data Resource (, and that there was reasonable continuity of the ionospheric data (foF2) for the period of interest. Table 1 shows the ionosonde stations included in this study, in each case the station code, the geographic coordinates, and the geomagnetic latitude are given. The stations cover geomagnetic latitudes from 88.8 N to 42.0 S (geographic latitudes from 77.5 N to 34.0 S), with the best coverage at midlatitudes in the north hemisphere.

Table 1. Names, Codes, Geographic Coordinates, and Geomagnetic Latitude of the Stations Used in the Study
 StationCodeLatitudeLongitudeGeomagnetic Latitude
7Millstone HillMHJ4542.6288.553.9
9Fairford, UKFF05151.7358.550.0
11Wallops IsWP93737.8284.549.2
16Point ArguelloPA83634.6239.442.3
18Eglin AFBEG93130.4273.341.1
20San VitoVT13940.617.834.4
24Port StanleyPSJ5J−51.7302.2−40.6

[12] The storms were selected under two criteria: they occurred in the year 2000 (to assure that none of the storms were in the database used to construct the model) and ap > 150. Six storms fulfill these conditions, April 6 (max. ap = 300, max. Dst = −288 nT), May 24 (max. ap = 297, max. Dst = −133.3 nT), the well-studied Bastille Day storm of July 15 (max. ap = 400, max. Dst = −287.66 nT), August 12 (max. ap = 179, max. Dst = −220.3 nT), September 17 (max. ap = 236, max. Dst = −161 nT), and October 5 (max. ap = 179, max. Dst = −185 nT). Figure 3 shows the time history of ap and Dst for the events during 2000. To obtain the foF2 ratio, Φ (foF2obs/foF2mm), a 30-day running mean centered on the storm date was used to calculate the monthly mean for each storm.

Figure 3.

Geomagnetic activity (ap and Dst indices) for each storm in the period of interest [Araujo-Pradere, 2002].

[13] The time history of the geomagnetic index ap for each interval was used as the input of the model. This includes the ap values for the 33 hours prior to the first hour of the period, which is needed to obtain the first point of the output (due to the length of the filter weighting function).

[14] In this work, foF2 hourly values for each site were used for a 5-day period of the storm (120 values), in order to see the full picture of the perturbed period, including the quiet background. In case where higher temporal resolution was available the hourly average was used.

5. Results

[15] The empirical storm-time correction model has been tested for each 5-day period of the storms. Figures 4a to 4f show the response of the ionosphere to each storm and the prediction of the empirical model for the fifteen stations. For each storm and station, the time evolution of the ratio of the hourly foF2 and the monthly mean, and the prediction from the empirical model are displayed, together with the normalized RMSE for both.

Figure 4a.

Data and output of the STORM model, as foF2 ratio, at 15 different locations for the storm of April 2000. The gray line shows the output of the STORM model, the black line is the observation. An empty square represents the daily RMSE for STORM, and the cross is for the monthly mean.

Figure 4b.

Same as Figure 4a but for the storm of May 2000.

Figure 4c.

Same as Figure 4a but for the storm of July 2000.

Figure 4d.

Same as Figure 4a but for the storm of August 2000.

Figure 4e.

Same as Figure 4a but for the storm of September 2000.

Figure 4f.

Same as Figure 4a but for the storm of October 2000.

[16] The results are presented from the earlier storm (April 6) to the latest (October 3), covering in this way from equinox to equinox through a solstice and intermediate seasons as defined by Araujo-Pradere et al. [2002]. For each storm, the stations are organized by geomagnetic latitude, from the most northerly going south.

[17] The black line represents the ratio from the data, while the thick gray line corresponds to the STORM model output. The x axis corresponds to time, from the 00:00 UT of the first day of the period up to the 120th hour (23:00 UT of the 5th day) of each storm. The y axis is the ratio of foF2 for both the observed ratio and the STORM prediction. The value Φ = 1 represents the quiet conditions (monthly mean). Also shown are the normalized RMSE for each 24 hours interval using either the STORM model ratios (empty boxes), or the monthly mean (black crosses) as the prediction. The y axis also quantifies the RMSE values, the metric used to assess the quality of the predictions.

[18] Most of the data used in this study rest on computer processing. In Figures 4a to 4f it is possible to observe a number of spikes evidently erroneous, but we decided to keep all the information to avoid the removal of real details that the model should be tested against.

[19] The clear visual message from this set of figures is the ability of the model to capture the tendency of the changes, in many of the cases. For non-storm days there are no significant difference between the prediction and the monthly mean RMSE. For storm days, the model captures the direction and magnitude of the depletion reasonably well.

[20] Table 2 shows the numerical values of the daily RMSE for each day of the 5-day period of each storm. In this table the storm days, and the corresponding averages, are shown in bold, and the % of improvement is given by the expression:

equation image

In light of the seasonal dependence in the error estimates shown in Figures 1 and 2, the RMSE have been separated by hemisphere, and so by season. Since no northern winter storms are present in the validation data set, the seasonal dependence in the accuracy of the model is manifest as a hemispheric dependence. Notice that a substantial improvement occurs in the north in all cases corresponding to equinox and summer conditions, whereas in the south in the winter months there is no significant improvement over climatology. Changes within ±10% can be considered as being essentially the same as climatology. The average over all storm days and both hemispheres is a 33% improvement.

Table 2. A Comprehensive Validation of the STORM Empirical Ionospheric Modela
RMSENorthern HemisphereSouthern Hemisphere
DaysAveragesPercent ImprovementDaysAveragesPercent Improvement
12345Five daysStorm Days12345Five daysStorm Days
  • a

    The normalized RMSE (e.g., 0.13 represents a RMSE of 13%) comparing the model and ionosonde observations are shown for five days for each of the storms in the year 2000, for northern and southern hemisphere stations separately.

April 2000
May 2000
July 2000
Aug. 2000
Sept. 2000
Oct. 2000

[21] Since quiet days also have geophysical variability, it is valuable to estimate how much of the increase in standard deviation during the storm days is captured by the model. During the quiet days preceding the storms (non-storm days), the variability of the data around the monthly mean (standard deviation of the data) is about 13%, very close to previous results [Araujo-Pradere and Fuller-Rowell, 2000]; and during the storm days the standard deviation increases to about 27%. The STORM model reduces this standard deviation to 19%, which implies that the STORM model is capturing more than half of the storm induced variability.

6. Conclusion

[22] A comprehensive validation has been performed of a new model designed to follow the changes in the F region ionosphere in response to a geomagnetic storm. The model has been tested on five significant geomagnetic storms during the year 2000. For each storm, data from 15 ionosonde stations were obtained. Visually, the prediction from the storm model follows the observed changes for many of the cases, but does particularly well in the summer hemisphere. The accuracy of the model has been quantified by evaluating the daily RMSE between the model and observations and compared with the prediction using the monthly mean. The values calculated using this metric shows the model is on average 33% improved over the monthly mean. The results indicate that the STORM model captures more than half of the increase in variability due to the storms.


[23] This work was performed under a grant from the National Science Foundation Space Weather Program.