A physics-based data assimilation model of the ionosphere is under development as the central part of a Department of Defense/Multidisciplinary University Research Initiative (MURI)-funded program called Global Assimilation of Ionospheric Measurements (GAIM). With the significant increase in the number of ionospheric observations that will become available over the next decade, this model will provide a powerful tool toward an improved specification and forecasting of the global ionosphere, with an unprecedented accuracy and reliability. The goal of this effort will be specifications and forecasts on spatial grids that can be global, regional, or local (25 km × 25 km). The specification/forecast will be in the form of three-dimensional electron density distributions from 90 km to geosynchronous altitudes (35,000 km). The main data assimilation in GAIM will be performed by a Kalman filter. In this paper we present a practical method for the implementation of a Kalman filter using a new physics-based ionosphere/plasmasphere model (IPM). This model currently includes 5 ion species (O2+, N2+, NO+, O+, and H+) and covers the low and middle latitudes from 90 km to about 20,000 km altitude. A Kalman filter based on approximations of the state error covariance matrix is developed, employing a reduction of the model dimension and a linearization of the physical model. These approximations lead to a dramatic reduction in the computational requirements. To develop and evaluate the performance of the algorithm, we have used an Observation System Simulation Experiment. In this paper, we will initially present the physics-based IPM used in GAIM and demonstrate its use in the reduced state Kalman filter. Initial results of the filter in the South American sector using synthetic measurements are very encouraging and demonstrate the proper performance of the technique.
If you can't find a tool you're looking for, please click the link at the top of the page to "Go to old article view". Alternatively, view our Knowledge Base articles for additional help. Your feedback is important to us, so please let us know if you have comments or ideas for improvement.
 The ionosphere is a complex and dynamic medium that exhibits weather features at all latitudes and longitudes. The sources for this variability come from the couplings, time delays, and feedback mechanisms that are inherent in the ionosphere-thermosphere system, as well as from the effects of solar, interplanetary, magnetospheric, and mesospheric processes. Although the climatological behavior of the ionosphere is well understood and physics-based theoretical and/or numerical models of the ionosphere are able to reproduce many of these features, they typically fall short in reproducing ionospheric weather. The major reason for this is the lack of reliable specifications of the ionospheric driving forces, including the convection electric fields and particle precipitation at high latitudes, the low-latitude electric fields, and the global neutral winds and composition. The initial conditions are not as important as the drivers at E and F region altitudes because of the short time constants for electron density changes (≤30 min).
 Currently, the most promising ionospheric weather models are physics-based data-driven models that use data assimilation techniques to specify the ionospheric plasma distribution. These models provide the means to estimate the state of the ionosphere from incomplete and sparse measurements. The estimate is achieved by using the physical relationships embodied in the theoretical models to dynamically interpolate and extrapolate the observations, both in time and space.
 Over the past decades, data assimilation models have become a dominant tool for specifications and forecasts in meteorology and oceanography. However, in the ionosphere these models are only emerging, largely owing to the lack of suitable data to be ingested in such models in the past. However, with the significant increase in the number of ionospheric observations that will become available over the next decade, data assimilation will provide a powerful technique toward an improved specification and forecasting of the global ionosphere, with an unprecedented accuracy and reliability [Schunk et al., 2002].
 A powerful way to assimilate data into a time-dependent model is via the Kalman filter [e.g., Gelb, 1974]. Howe et al.  used a Kalman filter to reconstruct the global ionospheric Ne distribution based on synthetic slant total electron content (TEC) measurements between 51 GPS ground stations and 21 GPS satellites, as well as slant TEC measurements from one LEO satellite in occultation. In their study, a statistical model (Gauss-Markov process) was used for the evolution of the reconstructed electron density field and the error covariances. Schunk et al.  also used a Gauss-Markov Kalman filter and synthetic data to reconstruct the global Ne distribution, but in addition to the slant TEC measurements, included in situ Ne observations from two Defense Meteorological Satellite Program (DMSP) satellites and bottomside Ne profiles from 16 Digisondes. Recently, Schunk et al.  have extended their work to use real observations from a network of GPS ground receivers, Digisondes, and DMSP satellites in their Kalman filter.
 Simultaneously with the development of the data assimilation algorithms, new physics-based ionospheric and plasmaspheric models have been developed that are especially tailored for the use in the Kalman filter [Schunk et al., 2004]. However, when using these models in the Kalman filter, the computational load drastically increases owing to the complex temporal evolution of the state error covariance matrix. Recently, Fukumori and Malanotte-Rizzoli  used approximate methods in oceanographic data assimilation to evaluate the error evolution. In their study, a reduced state approximation was used that effectively reduced the size of the estimation problem and consequently reduced the computational load. In this paper, we report on the development of a physics-based Kalman filter that employs this approximation in combination with a numerical linearization of the physical model. This filter will be used in Global Assimilation of Ionospheric Measurements (GAIM) to continuously reconstruct the global three-dimensional (3-D) Ne distribution [Schunk et al., 2004]. In this paper, we first introduce our new ionosphere/plasmasphere model (IPM) that will be used in the Kalman filter. Next, the Kalman filter equations are briefly recalled and the approximations are described. Finally, an example of the Kalman filter reconstruction using synthetic data over the South American region is presented.
2. Ionosphere/Plasmasphere Model
Schunk et al. [2002, 2003] developed a Kalman filter for the ionospheric plasma density using a Gauss-Markov process for the temporal evolution of the electron density variations from a given background value. Although the Gauss-Markov filter obtains acceptable ionospheric electron density reconstructions, the replacement of this statistical transition model with a physics-based model of the ionosphere/plasmasphere system should lead to significant improvements in the 3-D plasma density reconstruction and will, in particular, be important in data sparse regions where a proper filter performance depends on the accurate evolution of the model error covariances. Therefore we developed a new IPM for GAIM, which is especially tailored for the use in our Kalman filter. This model will be global, regional, or local depending on the operational demands and the available data sources. The model solves the continuity and momentum equations for the five ions, O+, NO+, O2+, N2+, and H+, on convecting flux tubes that realistically follow the geomagnetic field. Currently, the model extends in magnetic latitude from equatorial to upper midlatitudes and extends from 100 to about 20,000 km altitude, covering the ionospheric E and F region as well as the plasmasphere. An extension of the IPM to high latitudes is currently under development. A detailed description of the IPM is given by Schunk et al. .
Figure 1 shows the latitude-altitude grid used in IPM for the plasmasphere and the ionosphere in a geomagnetic meridian plane in the American sector (285°E geographic longitude). The grid points are shown in geographic coordinates, and the offset between the geomagnetic and geographic equators can be seen. For computational efficiency and model robustness, a variable number of grid points along a field line is used, which varies from 49 to 1599 points depending on the equatorial crossing altitude of the field line. In a magnetic meridian plane, 60 flux tubes are used, evenly spaced in equatorial crossing altitude by 20 km from 100 to 600 km altitude and an exponentially increasing spacing for the upper altitudes. Figure 1 (right panel) shows the high resolution of the IPM grid near the location of the equatorial anomalies, which are typically located in this longitudinal sector near −25°S and 0° geographic latitude. The high spatial resolution in this region is necessary to capture the large Ne gradients seen in this region. At midlatitudes the latitudinal resolution is about 2.5°.
 The zonal resolution of the IPM in the global mode is set to 7.5°. In the regional or local modes, however, it can be significantly increased, and a resolution of 1° is feasible. Figure 2 shows the plasma densities obtained from the IPM at 300 km altitude. The densities correspond to geomagnetically quiet (Kp = 2) December solstice conditions with a solar decimetric flux of 158. The densities are shown along realistic geomagnetic field lines that have been used in the IPM. The use of a realistic geometry can at times be particularly important in regions of large magnetic declination angles (e.g., the American sector).
Figure 3 shows model results of altitudinal profiles of the ionospheric and plasmaspheric ion composition at the geomagnetic equator. The profiles correspond to the Peruvian sector (285°E) and local noon conditions. As expected, H+ is the dominant ion in the topside ionosphere and plasmasphere, whereas O+ is dominant in the ionospheric F region. In the E region, below about 150 km altitude, molecular ions are the major constituents. Clearly, a model extending from 100 to 20,000 km altitude has to account for all these constituents.
 The external thermospheric and electrodynamic drivers for IPM are represented by empirical models (for a detailed description, see Schunk et al. ). However, as outlined by Schunk et al. , these drivers are being replaced by data-driven assimilation models [see, e.g., Fuller-Rowell et al., 2004]. In this paper, an initial attempt in this direction is made by using the Kalman filter not only for the reconstruction of the 3-D plasma densities but also to improve the specification of the equatorial zonal electric fields (vertical plasma drifts).
 As mentioned above, the IPM solves the governing physical equations along convecting flux tubes (Lagrangian frame), which have, besides other advantages, the computational strength that plasma densities along field lines can be determined independent of the densities along the other flux tubes. As explained below, this leads to major advantages for the numerical linearization of the model. In the Kalman filter, however, a spatially fixed grid is preferable. Therefore, at each assimilation time step (15 min), the convected density field is mapped onto a fixed grid using a sinh interpolation in the magnetic dipole coordinate q along magnetic field lines and an inclination angle-dependent mapping function between adjacent field lines.
3. Kalman Filter
 In this section we give a brief introduction to the Kalman filter, which is used for the reconstruction of the ionospheric and plasmaspheric plasma densities. The Kalman filter is a well-documented [e.g., Gelb, 1974; Howe et al., 1998] technique that can be derived as a recursive algorithm that minimizes the error (i.e., finding the best estimate of the state) at a time t based on all information prior to this time. The goal is to combine the measurement data from an actual observing system with the information obtained from the system model and their corresponding statistical description of uncertainties. Formally, the filter performs a recursive least squares inversion of the observations (e.g., slant TEC) for the model variables (e.g., Ne) using a dynamical model as a constraint. In practice, a weighted average of the model estimate and of the data is performed using the relative accuracy of the two as the weights. As a result, an improved estimate of the model variables is obtained where the improvement is in a statistical sense; i.e., it has the least expected error given the observations and the model, along with their error statistics. In this approach the specification of the error covariances for both the model and the observations are of crucial importance. The model dynamics, which provide the temporal evolution of both the state vector and the model error covariance matrix, can be given, as in our case, by a first-principles physics-based model (IPM). Along with the best estimate of the state, the Kalman filter also generates a theoretical estimate of the analysis error. Since the Kalman gain is dynamic, the model evolves in time and learns what model components are producing the best estimates for any time of day and sets the weights accordingly.
Table 1 summarizes the Kalman filter equations which are the framework for the rest of the paper. Note that we will adopt the conventional notation of Ide et al. . Using equation (K1), a forecast of the state vector x is made by linearly relating the state vector at a time t + 1 to the state at the previous time step t through the transition matrix M, where η represents the error generated by the operator (model error). This error, as well as the observation error ε, is assumed to be white and have respective covariance matrices Q and R. In general, the transition model comes from the numerical integration of a system of coupled partial differential equations, which is in our case performed by the ionosphere/plasmasphere model. As shown below, a linearization of the model evolution at time t is performed to obtain the transition matrix M. Similar to the forecast of the state vector, a forecast of the model error covariance matrix Pf (K2) is performed, which depends on the previous error and the transition model error covariance Q. Observations enter the Kalman filter through equation (K3), which linearly relates the measurements yo to the state vector through the measurement matrix H. In our case the state vector constitutes of electron density variations from a prescribed “best guess” ionospheric background field. The observational database can contain line-of-sight (column integrated) optical emissions, Digisonde measurements of the bottomside electron density profile, in situ measurements of electron densities form multiple satellites, hundreds of slant-path total electron content (TEC) measurements from radio beacons and/or GPS satellites, and thousands of occultation measurements. In the case of a nonlinear relationship between the observations and the electron density (e.g., column integrated optical emissions), linearization and iteration are typically performed. The Kalman gain K (equation (K4)), which gives the optimum combination of the model state and the data given their respective error covariances, combines the model state forecast xf with the data to obtain the new model state estimate, the so-called analysis (equation (K5)). The model error covariance matrix (equation (K6)) is then reduced by the amount related to the new information entered in the system through the data. Equations (K2) and (K6) together describe the evolution of the error covariance matrix and form the so-called Riccati equations. At this point, a new forecast (equations (K1) and (K2)) is performed and the assimilation is repeated.
 Although it is theoretically straightforward to apply the Kalman filter to ionospheric data assimilation, difficulties arise when implementing the system owing to the enormous computational requirements, both in storage and CPU time, associated with the propagation of the model error covariance matrix (equation (K2)) (for a full discussion of this problem, see Fukumori and Malanotte-Rizzoli ). As a practical method, we implemented an approximation of the state error covariance matrix, employing a reduction of the model dimension and a linearization of the physical model for the propagation of the error covariance matrix. These approximations lead to a dramatic reduction in the computational requirements. Formally, the two approximations could lead to suboptimal estimations, but the uncertainty associated with the observation and model errors may lead to statistically indistinguishable differences between the truly optimal and suboptimal estimates [Cane et al., 1996]. The strength of these approximations, however, lies in the otherwise not possible use of the Kalman filter framework to objectively evaluate the model state error covariances. These errors are typically anisotropic and inhomogeneous and are difficult to specify.
4.1. Reduced State Approximation
 Over the past several years the reduced state approximation has been successfully employed in oceanographic data assimilation [e.g., Fukumori, 1995; Malanotte-Rizzoli et al., 1996]. The idea is to approximate the model error covariance matrix with one of fewer degrees of freedom that would only resolve the covariance of the larger scales. Given the distribution of measurements and our crude understanding of the accuracies of the model and the data, the errors introduced by this approximation are most likely of second-order and can formally be included in the Kalman filter. The effective dimension of the reduced state is then largely given by computational requirements and by an assessment of “reasonable” scales over which error covariances can be determined. Figure 4 shows the reduced grid used to track the error covariances in a geomagnetic meridian plane. Every second field line of the full grid was selected, resulting in an altitudinal resolution of 40 km in the equatorial F region. Along each field line, 25 unevenly spaced grid points were placed to give a reasonable height resolution in the midlatitude F region of about 50–60 km. In the zonal direction a resolution of 15° was selected. The resulting number of grid points is only about 1% of the full grid. Clearly, as computational speed and memory increase and more observations become available, this resolution can be refined. Furthermore, the spatial resolution in a regional or local mode can be significantly increased.
 Following Fukumori , the reduced state approximation is implemented as follows: Let us assume the following transformation between the full IPM state x and a reduced state x′:
where defines a reference state. Note that the reduced state x′ constitutes a perturbation on the reference state in the reduced domain. B is the transformation matrix between the state on the reduced grid and the full grid. This transformation should be continuous and smooth and is, in our case, given by a sinh(x) interpolation along field lines and a linear interpolation between adjacent flux tubes. Given this transformation the error covariance matrix on the fine grid can also be approximated by the error covariance matrix on the coarse grid:
where the nonstochastic reference state does not contribute to P. This transformation is the essential part of the approximate filter. Given the smaller dimension of the reduced grid, derivation of the statistical properties of the reduced state x′(t), which is the major computational load in the Kalman filter, will be less demanding than on the full IPM grid.
Equation (1) is only an approximation, and the exact relationship includes the null space of the transformation (features that are not resolved in the reduced state):
We make the assumption that the null space, U, is dynamically uncoupled from the reduced space and can, in the Kalman filter, be treated as noise (small-scale dynamics does not affect the larger scales). This, of course, might not always be valid, in which case a regional model with an increased spatial resolution should be utilized.
 In addition to the transformation matrix B, we also need its pseudoinverse, which defines the transformation from the full to the reduced grid:
In general, for a pseudoinverse:
This means the transformation reduced to full to reduced is exact, but the transformation full to reduced to full is inexact (subgrid information has been lost). The errors introduced by this transformation can be formally taken into account in the Kalman filter. Now other system matrices can easily be constructed in a manner similar to the calculation of the reduced error covariance matrix (equation (2)).
4.2. Model Linearization
 In the Kalman filter the evolution of the state vector and its error covariance matrix is performed via the state transition matrix M (equations (K1) and (K2) in Table 1). This matrix is obtained from a numerical linearization of our dynamical model (IPM). Note that in our case the state vector consists of perturbations from a background ionosphere. The evolution of the background field utilizes the full nonlinear model and only the perturbations along with their error covariances evolve with the linear model. To numerically linearize our dynamical model (IPM), we start with the time evolution of the full model state x(t):
where w(t) denotes the various driving forces and boundary conditions. Our goal is to find the operator M′ on the reduced grid that propagates the reduced state vector x′ forward in time over our assimilation time step of 15 min. Combining equations (3), (4), and (7) yields:
With our previously introduced assumption that the null space is dynamically uncoupled, this becomes:
where q represents the formal error due to the effects of the reduced model's unresolved physics. Notice that the IPM densities are always evaluated on the full IPM grid. However, our final result should be in the form x′(t + 1) = M′x′(t) + error.
 For a linear model the columns of M′ can be obtained by integrating the corresponding column of the identity matrix by the model operator in equation (9). However, for a nonlinear model like IPM, the linearization is performed about the reference state . From equation (9) we have:
which can be written as:
Then the ith column of M′, m′i, can be obtained by
where ei is the ith column of the identity matrix in the coarse domain and the last two terms are evaluated using the IPM model. In practice, at time t we perturb the background ionosphere at a given grid point on the reduced grid, transform it to the full grid, and then observe the evolution of this perturbation as well as the evolution of the background field as given by the IPM. The new densities are then transformed back to the reduced domain and their difference is taken. This linearization is then repeated after every assimilation time step (15 min). Obviously, there is some freedom in the choice of the background density field. In this paper, we use a so-called “best guess” ionosphere (GAIM 1), which is described below and also in detail in the paper by Schunk et al. . Other choices include the use of a climatological background ionosphere or the analysis field obtained from the Kalman filter.
 It is important to note that the numerical linearization of IPM (equation (15)) involves thousands of model runs because each grid point in the coarse domain needs to be separately perturbed. However, since the electron densities are evaluated in a Lagrangian frame, as mentioned in section 2, each model run only involves the temporal evolution of the Ne field along one field line, which is computationally extremely efficient and naturally parallel. As a result, the linearization can easily be performed in real time using a network of only several personal computers.
5. Feedback of Equatorial Vertical Drifts
 In addition to the continuous reconstruction of the 3-D ionospheric and plasmaspheric density field, GAIM also specifies the thermospheric and electrodynamic drivers that determine the plasma distribution [Schunk et al., 2004]. These drivers will be determined through separate assimilation models and, when available, will be provided to the IPM model. The introduction of these adjusted drivers into the physical model is the first step in GAIM, and the resulting ionospheric density field is called GAIM 1. Because measured drivers are used, the global Ne distribution at this step should already be an improvement over time-dependent climatology (GAIM 0), where only drivers from statistical empirical models were used. However, it is feasible to also use the output of the Kalman filter, which in our nomenclature is called GAIM 2, to further improve the specification of these drivers and feed them back into GAIM 1. In this paper, an initial attempt in this direction is presented considering variations only in the equatorial zonal electric field (vertical plasma drift). The feasibility to include other drivers (e.g., neutral winds) is the subject of further investigations.
 To determine the real time variations of the equatorial drifts from their nominal background values, the differences between the density field obtained by the Kalman filter (GAIM 2) and from our “best guess” ionosphere (GAIM 1) are analyzed. A linear model for the variations of Ne for varying equatorial vertical drifts is then employed to adjust GAIM 1 in the direction of GAIM 2. A detailed description of this technique is, however, beyond the scope of this paper and will be published elsewhere.
6. Kalman Filter Example
 Although the final outcome of the Kalman filter development in GAIM will be the specification and forecast of the true ionosphere, it is essential for the development of the physics-based Kalman filter that it can be tested in a known environment. Therefore, in our initial application of the technique, we constructed a Data Simulation System Experiment (OSSE) for the South American region using two different synthetic (model-generated) data types: slant TEC from three ground-based GPS stations and bottomside density profiles from one ionosonde. Figure 5 shows our model region as well as the locations of the GPS receivers and the ionosonde.
 The synthetic data were generated by probing the 3-D, time-dependent electron density distribution for the weather (true) simulation exactly the same way the real instruments probe the real ionosphere. For the ionosondes, observations were taken in 15-min intervals from 100 km altitude up to the height of the F2 layer, hmF2, in 10 km increments. For the GPS receivers, slant TECs were generated only for elevation angles greater than 15°. When the synthetic data were generated, noise was added to each “measurement” in order to mimic a real observation. A 5 TEC unit (TECU) level of noise was added to all simulated TEC measurements and a 10% uncertainty to the simulated ionosonde measurements. For the weather simulation we varied the equatorial vertical drift by superposing on the climatology values a constant offset of 5 m/s in addition to a random component.
 The Kalman filter assimilation procedure was implemented as follows. At 1200 UT on day 335 the Ne distribution obtained from the climatology run was taken to be the initial distribution at the start of the Ne reconstruction. Every 15 min, the evolving weather simulation was probed to obtain the two synthetic data types (with noise) as described above. Since our state vector, as mentioned above, consists of perturbations from a given background (GAIM 1), we subtracted from each data point the corresponding background value. At these time marks the ionosphere/plasmasphere model was also integrated forward in time, and the linearized transition matrix M and the transition model error covariance matrix Q were determined. Using the new data and the new transition and error matrices, the Kalman filter reconstructed an updated estimate of the Ne distribution. This update was then used to improve the estimation of the equatorial vertical plasma drifts, as briefly outlined above. The new drift velocities were fed back into the IPM background run and the assimilation was repeated at the next 15 min time mark. As time advanced, the Kalman filter produced a 3-D, time-dependent, Ne distribution that got closer and closer to the Ne distribution associated with our weather simulation.
7. Simulation Results
Figures 6–8 show results from our Kalman filter analysis along a geomagnetic meridional plane from 100 km up to 1500 km altitude after 10, 17, and 24 hours of assimilation time. The top row shows, from left to right, the weather (true) case, the climatology (GAIM 0) case, the “best guess” (GAIM 1) case which already includes an adjustment of the equatorial electric field, and the Kalman filter reconstruction (GAIM 2). In the bottom panel the height profiles at the magnetic equator for the four density fields, “truth,” GAIM 0, 1, and 2, are shown (left panel) with the percentage differences between the “truth” case and GAIM 0, 1, and 2, respectively. It is apparent that the Kalman filter was successful in capturing the ionospheric density distribution during both daytime and nighttime conditions. In particular, the filter was able to track the equatorial layer height as well as the strength of the equatorial anomaly. Note the similarity between GAIM 1 and GAIM 2, which indicates a successful reconstruction of the equatorial vertical plasma drifts. Figure 9 shows the vertical plasma drifts used in this experiment. The long dashed line represents the climatology case, as obtained from the Scherliess and Fejer  empirical model that was used for GAIM 0. The short dashed line indicates the “true” vertical drifts used for the weather simulation, and the solid line shows the plasma drift velocities obtained from the Kalman filter analysis. Our initial results indicate that the use of the Kalman filter analysis can lead to significant improvements in the specification of the equatorial vertical drifts. Initial results, using only slant TEC data from near-equatorial ground-based GPS stations also resulted in acceptable vertical drift reconstructions. However, it has to be kept in mind that in our current study using synthetic data only the vertical drift velocities were varied and the meridional wind effects were not taken into consideration. When using actual data, this assumption cannot be made and the effects are subject to further investigation.
8. Summary and Conclusions
 A physics-based data assimilation model for the ionosphere is under development as the central part of GAIM. The model is centered on a new model of the ionosphere/plasmasphere system (IPM) and will provide specifications and forecasts from 90 km to geosynchronous altitudes on a global, regional, or local grid, depending on the operational demands and available data sets. The main data assimilation in GAIM is performed by a Kalman filter. We have presented a practical method for the implementation of the filter based on a reduced state approximation and a numerical linearization of the physical model. In addition to the continuous 3-D reconstruction of the Ne density field, an initial attempt was made to use the Kalman filter analysis to also improve the specification of the equatorial vertical drifts. The initial results of this assimilation technique and the driver specification over the South American region are very encouraging.
 This research was supported by the DOD MURI program via grant N00014-99-1-0712 to Utah State University.