Data assimilation for neutral thermospheric species during geomagnetic storms



[1] During a geomagnetic storm, Joule heating heats the neutral gas and drives horizontally divergent winds which force upwelling of the neutral atmosphere. The heavier molecular species N2 and O2, abundant in the lower thermosphere, are transported to high altitude where they increase the loss rate of the F region ionosphere. The “bulge” of enhanced molecular species, or depleted atomic oxygen, is long-lived, returning to equilibrium mainly through the slow process of molecular diffusion. Its longevity, of the order of a day, enables the global wind system to transport the composition disturbance over thousands of kilometers, driven by the combination of quiet and storm-time wind fields. In a stand-alone physical model the formation and subsequent movement of the composition features depend on accurate specification of the spatial and temporal distribution of the Joule heating from the magnetosphere and knowledge of the time-dependent wind fields to define the transport. Neither is sufficiently well known given current observational capability. An alternative approach is to combine the knowledge contained in a physical model with observations of the thermospheric composition. It has been demonstrated that FUV images can provide a reliable estimate of the magnitude and structure of oxygen-depleted regions on the sunlit side of Earth. A Kalman filter data assimilation method has been developed to combine FUV observations with a physical model in order to optimally define the global distribution of neutral thermosphere composition. This distribution is used as one of the important drivers in a model for Global Assimilation of Ionospheric Measurements (GAIM) in order to improve specification and forecast of the response of the ionosphere to geomagnetic storms.

1. Introduction

[2] Advances in forecasting tropospheric weather over the last two decades have been built on three pillars: Improvements in capturing physical processes in numerical models, a huge increase in the availability of data primarily from new satellite observations, and the ability to combine the two using optimal data assimilation techniques. For the upper atmosphere and ionosphere the first pillar is in place, the second is imminent, and the third is rapidly approaching. Physical models of the upper atmosphere both for the ionosphere [Schunk and Sojka, 1996; Richards and Torr, 1996; Bailey and Balan, 1996] and for the coupled thermosphere ionosphere system [Roble, 1996; Fuller-Rowell et al., 1996a; Millward et al., 1996] have matured over the years and can now simulate many of the observed features. These models are able to match the global features in comprehensive empirical models such as the International Reference Ionosphere (IRI; Bilitza [2001]) and the Mass Spectrometer and Incoherent Scatter (MSIS; Hedin [1987]) neutral atmosphere model. In addition, the physical models have the added advantage of being able to follow time-dependent changes and can be used to interpret observations by analysis of the physical processes embedded in the model. For a given season, and level of solar and geomagnetic activity, the physical models are able to describe the global distribution of ion and neutral parameters to about the same level of accuracy as the empirical models. The advances will come by combining the knowledge of the physics contained in the numerical models with the rapidly increasing data source.

[3] The volume of real-time observational data for the upper atmosphere has been limited in the past to a few ground-based ionosondes and incoherent scatter radar facilities, and perhaps one or two in situ measurements from polar orbiting spacecraft. In the future, data will be available from an ever-increasing global network of dual-frequency GPS receivers providing slant path electron content, routine imaging from a variety of polar and equatorial spacecraft, and from constellations of satellites providing a dense global distribution of occultation measurements. These new sources will increase the data availability by about an order of magnitude. The maturity of the models and the promise of increased data resources have spawned the application of data assimilation techniques in the space physics community. One of the major thrusts is proceeding under the Multi-Disciplinary University Research Initiative (MURI) Global Assimilation of Ionospheric Measurements (GAIM) initiative [see Schunk et al., 2004]. By applying these new techniques to specification and forecast of the ionosphere and neutral upper atmosphere, the accuracy of the predictions will begin to parallel the breakthroughs in meteorological weather forecasting.

[4] One of the major challenges in upper atmospheric modeling is to be able to capture the response to geomagnetic storms. Geomagnetic storms result when high-speed plasma injected into the solar wind from coronal mass ejections or coronal holes impinges upon Earth's geomagnetic field. If the arriving solar wind plasma has a southward magnetic field, energy is coupled efficiently into Earth's magnetosphere and upper atmosphere. From the perspective of the upper atmosphere it is a period of intense energy input for a period of several hours to days, primarily from Joule heating. During the course of a storm the temperature of the neutral atmosphere can rise by hundreds of degrees Kelvin. Thermal expansion of the atmosphere raises neutral density and can have significant effects on satellite drag. Ionospheric ions drift in response to the electric field, and by colliding with the atmosphere, can drive winds in excess of 1 km/s at high latitudes.

[5] The energy injection drives global thermospheric storm winds and composition changes, and many of the ionospheric changes at midlatitudes can be understood as a response to these thermospheric perturbations. Wind surges propagate from high latitudes around the globe and transport plasma along magnetic field lines to regions of altered chemical composition, changing ion recombination rates. Many of the increases in NmF2 and total electron current (TEC) are thought to result from these “traveling atmospheric disturbances.” The divergent nature of the wind causes upwelling of molecular rich thermospheric gas from lower altitudes. These regions of enhanced molecular species at F region altitudes, or composition “bulges,” can be transported by the preexisting background wind fields and by the storm winds [Fuller-Rowell et al., 1996b]; the changed composition again feeds back to the ionosphere. The regions of upwelling (increases of N2 and O2) cause the ionosphere to decay faster and create negative phases of ionospheric storms.

[6] The dynamical changes are complicated because they are driven by the highly variable magnetospheric energy sources. The wind surges propagate and interact around the globe and often appear as a random mixture of waves. Exactly where a composition bulge will be created is also difficult to determine; composition changes are created by persistent divergence of the wind field in areas of significant energy injection (mainly Joule heating). Accurate knowledge of the spatial and temporal distribution of the magnetospheric sources is required to predict where and when these composition changes will manifest themselves. With the currently available observational capability these sources are not known well enough to model the correct magnitude and location of the composition changes. An alternative approach is to measure the composition response and remove errors by using data assimilate techniques.

[7] Much of the interest in understanding the response of the upper atmosphere to geomagnetic storms has stemmed from the need to predict the ionospheric response. The need arises for practical reasons: The requirement for ground-to-ground communication via the ionosphere using HF radio propagation and from ground-to-satellite through the ionosphere at higher frequencies. The parameters that have received the most attention are the peak F region electron density (NmF2), or critical frequency (foF2), which is related to the maximum usable frequency (MUF) for oblique propagation of radio waves, and the TEC, which is significant for the phase delay of high frequency ground-to-satellite navigation signals.

2. Data Sources for Neutral Atmosphere Composition

[8] Several instruments are scheduled to go into orbit this year and will supply a large portion of the data for the GAIM project. The main data sources for the neutral atmosphere composition are the Special Sensor Ultraviolet Limb Imager (SSULI) and the Special Sensor Ultraviolet Spectrographic Imager (SSUSI). These two instruments will be carried aboard the Defense Meteorological Satellite Program (DMSP) satellites. The DMSP satellites maintain a near-polar Sun-synchronous orbit at an altitude of approximately 830 km and carry numerous other instruments for various environmental parameter sensing.

[9] SSUSI, developed by the Applied Physics Laboratory (APL) at Johns Hopkins University [Paxton et al., 1992], will measure the height-integrated thermospheric O/N2 ratio on Earth's disk and the neutral density profiles of the major species O, O2, and N2 on the limb. The neutral composition measurements are only possible on the sunlit side of Earth. On the nightside, SSUSI measures the F region ion, O+. The SSUSI instrument builds a spectrographic image by scanning across the satellite's ground track as shown in Figure 1. The device scans from horizon to horizon and onto the limb perpendicularly across the satellite flight track once every 22 s. For details on the algorithm development, see Strickland et al. [1995].

Figure 1.

The SSUSI instrument conducting a horizon-to-horizon scan.

[10] SSULI [McCoy et al., 1992; McCoy and Thonnard, 1997] is an optical remote sensor developed by the Naval Research Lab (NRL). The SSULI instrument measures vertical profiles of the natural airglow radiation from atoms, molecules, and ions in the upper atmosphere. SSULI also observes ions in the upper atmosphere and ionosphere by viewing Earth's limb. Like the SSUSI instrument, SSULI is able to observe, on the sunlit side, all of the main neutral atmospheric species, O, O2, and N2, and on the night side, O only [Meier and Picone, 1994]. The limb scanner faces in the opposite direction of the satellite flight direction, as shown in Figure 2. The instrument scans up and down in the vertical direction, scanning Earth's disk and limb every 10 s.

Figure 2.

The SSULI conducting a limb scan.

3. Data Assimilation Techniques

[11] One of the most popular data assimilation techniques is the sequential filter [Kalman, 1960; Kalman and Bucy, 1961], often known as the Kalman or Kalman-Bucy filter. Its popularity can be attributed to its ease of coding, optimized speed of convergence, and ability to provide a statistical estimate of the error in the solution. The Kalman filter has been applied based on previous research in using data assimilation techniques in meteorology and oceanography [Cohn, 1982; Fukumori and Malanotte-Rizzoli, 1995; Howe et al., 1998; Fukumori et al., 1999]. The Kalman filter uses estimates of the errors in the state and measurements to calculate a “gain.” This gain will weight accordingly the predicted state or new measurement based on their respective error estimates. For example, if the state is not well known or if the model propagating the state forward is not very accurate, then the Kalman filter will weight the measurements more heavily. However, if the measurements have a large error, and the state estimate and forward prediction model are accurate, then the measurements will be given less weight. As more measurements are taken, the measurement error can be averaged out, and knowledge of the state increases. If measurements become unavailable, the errors in the state are assumed to grow with time, and any new measurements will be weighted more heavily again.

[12] Weighting either the state or measurements on how well they are known seems straightforward, and it would appear that there is no need for a systematic calculation of the gain. However, there is a single value of the gain that is optimal; of all the gains that could be chosen, there is one that will provide the most accurate balance based on the estimated state, propagation model, and measurement error and will allow the filter to reduce the errors to a minimum and as quickly as possible. The Kalman filter, since it is derived using statistics and optimization theory, determines this optimal gain value.

[13] There are many techniques used to propagate the state forward in time, including Gauss-Markov [Liebelt, 1967] and persistence and using a physical model. The Gauss-Markov process assumes that the errors in the state grow at an exponential rate with time, allowing the state to relax back to climatology according to a specific timescale if observations have become unavailable. Persistence assumes that the state, although it may not be very predictable, is slowly changing and that the most likely future state will be the same as the current state. The physical model uses the current state as well as other parameters to forecast the most likely future state on the basis of the dynamic equations of the neutral thermosphere. Although all these methods have been tested, here we present only the results using the Gauss-Markov process.

[14] The propagation model uses the analysis field, or current estimate of the state, as the initial conditions to predict a new state, together with an estimate of the errors. Before the Kalman filter is applied, the model error variance must be statistically estimated from a set of several trial runs. A “correlation length” [Houser, 1996] is defined as 0.4 km/s, or 240 km/10 min, where 10 min is update time of the Kalman filter. The correlation length defines the time it takes for a grid point to affect another grid point for a given distance between the two grid points. This value, 240 km/10 min, consistently provides the lowest RMS and closely matches the typical wind speeds expected at high latitudes in the thermosphere. The Kalman gain matrix weights accordingly on the basis of estimates of measurement, state, and model errors to produce a new analysis field at the current time. It is important to specify the error covariances correctly for both the data and the model. The practical difficulty with the method lies in the fact that the calculation of the model forecast error covariance is prohibitive for large models. This has prompted the development of approximate or suboptimal methods.

4. Simulation Scenarios

[15] Since data are not yet available from the SSULI and SSUSI instruments, in order to test the assimilation framework, an artificial atmosphere was created by using a coupled thermosphere ionosphere model (CTIM; Fuller-Rowell et al. [1996a]). Four case studies have been constructed, each 48 hours in length, all at moderately high solar activity. For both equinox and solstice, either two quiet days are simulated, or one quiet and one storm day. The quiet days are at Kp 2, and the storms are characterized by a 12-hour period of Kp 7, commencing at the beginning of the second 24-hour period.

[16] The Kalman state used in the filter is a two-dimensional representation of the thermospheric composition covering all latitudes and longitudes with a resolution of 2° and 6°, respectively. Specifically, it is defined as the ratio of the height integrated atomic oxygen and molecular nitrogen concentrations down to a reference level defined as the altitude where the molecular nitrogen column depth is 1021 m−3. The state defined in this way is consistent with the observable from the SSUSI instrument and can be easily constructed from the SSULI observations. Since MSIS assumes diffusive equilibrium, the vertical profiles of the three major species O, O2, and N2, can be constructed reasonably uniquely from a given value of the state. This was confirmed by sorting MSIS profiles for a given O/N2 ratio over a range of season, local time, and solar and geomagnetic activity. A similar sorting of CTIM model profiles revealed a small spread in the vertical profiles indicative of the impact of departures from diffusive equilibrium. The largest spread in vertical profiles, for a given value of the state, tend to be localized to regions of high-latitude gradient where altitude dependent horizontal winds act to “mix” the profiles. A typical range of values for the state is from 0.3 to 1.2, with the column abundance of N2 changing by a factor of four over the range.

[17] The observation scenario assumes two Sun-synchronous satellites, each comprising a SSUSI and SSULI instrument. The local times of the equatorial crossings of the two orbit planes are assumed to be 0600/1800 hours and 1200/2400 hours, with observations from both instruments available only on the sunlit side of Earth. The satellites and instruments are assumed to observe the artificial atmosphere constructed by CTIM for each of the four scenarios. To simulate instrument noise, a random Gaussian error is added to the observations of 0.1, which is typically about 10 to 20% of the state value. The error estimate was based on the error analysis performed by S. Thonnard (private communication, 2002). The observation error variance is assumed to be known precisely in the Kalman filter and is also set to a value of 0.1.

[18] The choice of local time crossing has been chosen to optimize the data availability. Since nighttime observations are not available, the dawn/dusk orbit minimizes the time for which observations are unavailable. However, no account has been taken of the increase in errors introduced by applying the instrument algorithms across the steep gradients at the terminator. For the present tests of the filter the O/N2 ratio for the truth file was extracted directly from the CTIM simulation; a full simulation of the dayglow and the instruments retrieval algorithms has not been attempted.

[19] The propagation model used for the present application of the Kalman filter is the Gauss-Markov scheme, where the state is assumed to relax back to MSIS composition structure with an e-folding time of 15 hours. The choice of the recovery timescale is based on simulation results from the CTIM physical model and on empirical estimates of the ionospheric to geomagnetic activity [Araujo-Pradere and Fuller-Rowell, 2002]. The physical model, CTIM, has not been used in the Kalman filter for the present simulations because this model was used to generate the truth data set. The update cycle for the Kalman filter is every 10 min, and the state is initialized to MSIS. The root mean square error (RMSE) between the Kalman state and the CTIM truth file over the two-dimensional grid is used to estimate the accuracy of the reconstruction.

5. Results

[20] To illustrate the results, information from one of the simulations scenarios has been selected: The equinox interval with the first day quiet, followed by a 12 hour storm and 12 hours of recovery. At the start of the simulation the truth file from CTIM and the initialized Kalman state (MSIS) are significantly different with an RMSE of 18%. Figure 3 shows the results of the Kalman filter after 24 hours of the quiet day conditions. On the left-hand side of the figure, the CTIM truth file, the filter reconstruction, and their difference after one quiet day are shown. The filter has captured the structure reasonably well and has reduced the RMSE to 6%. The vertical bars show the position of the dawn and dusk terminators; at this time (1200 UT) the central area between 90 and 270° longitude is on the nightside. The right-hand side information is not relevant for the quiet day.

Figure 3.

Kalman filter results after 24 hours on the quiet day at 1200 UT.

[21] Figure 4 shows the situation at the end of the 12-hour storm when the global composition is perturbed by the high-latitude heating. The overall structure is similar except that the region of upwelling in the polar regions, where there are low values of the state, penetrates to lower latitudes. The region of downwelling and larger state values is narrower. This can be seen in the panels on the right-hand side, at the top, which shows the difference between the quiet and storm truth file, showing the impact of the storm.

Figure 4.

Kalman filter results after 12 hours of the storm at 0000 UT.

[22] The Kalman filter reconstruction has captured some of the storm-time structure, but the RMSE error has increased above the quiet day values to about 8%. On the bottom right is shown how well the Kalman filter has captured these storm time changes. Clearly there are regions, particularly at low latitudes that have been unobserved for many hours, that have a large error. In the middle panel on the right it is shown how MSIS would capture the changes due to the storm at this time. With rapid changes in the state during a storm it is a challenge for a two-satellite configuration to sample the atmosphere fast enough to capture the very dynamic changes that are occurring.

[23] Finally, Figure 5 shows the situation after 12 hours of recovery from the storm. The effects of the storm are still present as depicted in the top right panel, and the filter is still attempting to follow the recovery. The RMSE error has stayed about the same value as at the end of the driven phase of the storm around 8%.

Figure 5.

Kalman filter results 12 hours into recovery from the storm at 1200 UT.

6. Conclusions

[24] A Kalman filter data assimilation technique has been developed to follow the thermospheric neutral composition changes during a geomagnetic storm. The case shown uses a Gauss-Markov method to propagate the state forward in time. With a two-satellite configuration the quiet prestorm structure can be captured reasonably well, reducing the error to 6%. During the dynamic driven phase of a storm the scheme with two satellites cannot measure sufficiently to maintain the accuracy, and the RMSE increases to 8%. The difficulty arises because the SSUSI and SSULI instruments are not yet capable of nightside observations. During the 12 hours of the recovery from the storm the filter maintains about the same error as during the storm. Another 24 hours of quiet are required to return the filter accuracy to quiet time levels. The demonstration highlights the need for nighttime measurements either in situ or from imaging. It is also possible that the use of a physical model to propagate the state, rather than a Gauss-Markov scheme, may help estimate the changes in the state in the regions of darkness.