US-TEC: A new data assimilation product from the Space Environment Center characterizing the ionospheric total electron content using real-time GPS data

Authors


Abstract

[1] The potential of data assimilation for operational numerical weather forecasting has been appreciated for many years. For space weather it is a new path that we are just beginning to explore. With the emergence of satellite constellations and the networks of ground-based observations, sufficient data sources are now available to make the application of data assimilation techniques a viable option. The first space weather product at Space Environment Center (SEC) utilizing data assimilation techniques, US-TEC, was launched as a test operational product in November 2004. US-TEC characterizes the ionospheric total electron content (TEC) over the continental United States (CONUS) every 15 min with about a 15-min latency. US-TEC is based on a Kalman filter data assimilation scheme driven by a ground-based network of real-time GPS stations. The product includes a map of the vertical TEC, an estimate of the uncertainty in the map, and the departure of the TEC from a 10-day average at that particular universal time. In addition, data files are provided for vertical TEC and the line-of-sight electron content to all GPS satellites in view over the CONUS at that time. The information can be used to improve single-frequency GPS positioning by providing more accurate corrections for the ionospheric signal delay, or it can be used to initialize rapid integer ambiguity resolution schemes for dual-frequency GPS systems. Validation of US-TEC indicates an accuracy of the line-of-sight electron content of between 2 and 3 TEC units (1 TECU = 1016 el m−2), equivalent to less than 50 cm signal delay at L1 frequencies, which promises value for GPS users. This is the first step along a path that will likely lead to major improvement in space weather forecasting, paralleling the advances achieved in meteorological weather forecasting.

1. Introduction

[2] Advances in forecasting tropospheric weather have been built on three pillars: improvements in capturing physical processes in numerical models, a huge increase in the availability of data (primarily from new satellite observations), and the ability to combine the two using optimal data assimilation techniques. Physical models of the upper atmosphere both for the ionosphere [Schunk and Sojka, 1996] and for the coupled thermosphere-ionosphere system [Roble, 1996; Fuller-Rowell et al., 1996] have matured over the years and can now simulate many of the observed features. These models are able to match the global features in comprehensive empirical models such as the International Reference Ionosphere (IRI [Bilitza, 2001]) and the Mass Spectrometer and Incoherent Scatter (MSIS [Hedin, 1987]) neutral atmosphere model. In addition, the physical models have the added advantage of being able to follow time-dependent changes and can be used to interpret observations by analysis of the physical processes embedded in the model. For a given season, and level of solar and geomagnetic activity, the physical models are able to describe the global distribution of ion and neutral parameters reasonably well, but quantitative comparisons with empirical models and data are fairly limited [e.g., Killeen and Roble, 1994; Marcos et al., 1992]. The advances will come by combining the knowledge of the physics contained in the numerical models with the rapidly increasing data source.

[3] The volume of real-time observational data for the upper atmosphere has been limited in the past to a few ground-based ionosondes and incoherent scatter radar facilities, and one or two in situ measurements from polar orbiting spacecraft. Now, data are available from an ever-increasing global network of dual-frequency GPS receivers providing slant path electron content, and routine imaging from a variety of polar and equatorial spacecraft. In the near future, data will also be available from constellations of satellites providing a dense global distribution of occultation measurements. The maturity of the models and the promise of increased data resources have spawned the application of data assimilation techniques in the space physics community. One of the major thrusts proceeded under the GAIM-MURI initiatives [Scherliess et al., 2004; Wang et al., 2004]. By applying these new techniques to specification and forecast of the ionosphere and neutral upper atmosphere, the accuracy of the predictions will begin to parallel the breakthroughs in meteorological weather forecasting.

2. New Product

[4] Space Environment Center (SEC) is a 24/7 operational center and in FY05 transferred to the National Weather Service (NWS) to be a full National Center for Environmental Prediction (NCEP) partner. SEC's mission is to characterize the space environment in real time and provide alerts and warnings to customers, in much the same way as NWS provides tropospheric weather forecasts. SEC is now making the first steps in adopting data assimilation techniques for space weather. Through a collaboration between the National Geodetic Survey (NGS), the University of Colorado's Cooperative Institute for Research in Environmental Sciences (CIRES), and SEC, a regional ionospheric data assimilation model has been developed to specify the total electron content over the continental United States (CONUS). This first venture does not yet utilize the sophisticated physical models within the assimilation process, as proposed by Schunk et al. [2004], but it does capitalize on the recent increase in data availability. Since US-TEC does not use a physical model, it is a matter of debate if it constitutes a true data assimilation model. Since a Kalman filter is used to continuously update the state and a sophisticated empirical model is used as the background, we venture to suggest that “data assimilation” is the appropriate category to describe the process. The model relies on the real-time network of ground-based, dual-frequency, GPS receivers operated by the U.S. Coast Guard for the National Differential GPS Service (NDGPS). The NDGPS system has twin GPS receivers currently operating at approximately 80 stations across the CONUS. These data are also part of the Continuously Operating Reference Station (CORS) network of over 400 stations that are used by NGS for geodetic application. Data from about 60 stations are streamed into SEC within 5 min of acquisition.

[5] The availability of the GPS data in real time and recent model development have resulted in the first space weather assimilative model suitable for transition to SEC Space Weather Operations (SWO). With the reliable feed of GPS data, the model is capable of accurate real-time specification of vertical and slant path total electron content (TEC) over the CONUS. The product (US-TEC) is of potential value for improved single-frequency GPS positioning and more rapid dual-frequency decimeter and centimeter accuracy positioning.

[6] Figure 1 illustrates maps of the vertical TEC, the uncertainty, and the difference of the current values from the average over the previous 10 days at that particular universal time. In this case, geomagnetic activity is a little higher than average with a Kp of about 5, so the vertical TEC is slightly disturbed. The scale extends to 100 TEC units (1 TECU = 1016 el m−2) so that during severe geomagnetic storm conditions when TEC can increase dramatically, the values will not saturate. Real-time maps can be found at http://www.sec.noaa.gov/ustec.

Figure 1.

Sample map of (top) vertical TEC, (middle) its uncertainty, and (bottom) difference from a 10-day average at 1745 UT on 2 January 2005. The small circles indicate the locations of the observing sites, which number approximately 60 for a typical inversion.

[7] The software package on which the product is based, MAGIC, was developed by Spencer et al. [2004], and evolved from previous research by Mitchell and Spencer [2002]. MAGIC is a research tool designed to study various approaches to the problem of obtaining an estimate of the ionospheric structure with sufficient accuracy to provide utility for space weather and geodetic positioning applications.

3. Theory

[8] MAGIC is based on a Kalman filter, which provides a means of optimally updating a solution to a linear least squares problem by combining time-dependent observations and a prior model estimate of the solution. The unknowns, which in this case represent the ionospheric electron density field, are stored in the state vector, x. Associated with this vector is a covariance matrix, P, which is updated by the filter for each iteration. P represents the uncertainty in the state.

[9] First, the state vector x is projected into the future (the minus superscript implies prior values)

equation image

The matrices A and B are generated from model estimates M as follows:

equation image
equation image

where the matrix G defines the correlation between spatially separated terms, and α is a constant. The matrix G is defined by the radial, latitude, and longitude covariance vectors; by default a Gaussian function is used. With α set to zero, B is zero and hence the A term only exists, which defines the forward projection in terms of the relative spatial/temporal variations in the model. With α greater than zero the B term increasingly dominates, which sets the future state estimate to that of the model in an absolute sense.

[10] The next stage in the update of the filter involves projecting the error covariance matrix

equation image

where the process noise matrix Q is defined as

equation image
equation image

and where k, the process noise, is a constant, and G, as before, defines the correlation between terms. The Q matrix therefore defines the variance as being a constant fraction of an average of the model and projected state estimate.

[11] Given a set of line integral observations z with covariance R, and path integrals defined by H, the Kalman gain is given by

equation image

Finally, using the Kalman gain, the state vector and its covariance are updated

equation image
equation image

[12] One final modification has been made in the form of the inclusion of linear time evolution terms. The state vector now becomes

equation image

with terms appended to the observation matrix H as

equation image

[13] Central to the MAGIC assimilation method is the option of mapping the state vector to enable a more succinct representation of the electron density field in three dimensions. The mapping is applied to the radial profile using a set of empirical orthonormal functions, EOFs, obtained from the IRI95 model. The mapping significantly improves the ability of the filter to image variations in the electron density profile. As an added advantage, the reduction in size of the state vector will also increase performance and reduce memory requirements by one or more orders of magnitude.

[14] Example mapping functions are shown in Figure 2. These EOFs were generated by applying singular value decomposition to a set of model profiles generated by IRI95. The dominant term, EOF1, represents a mean ionospheric profile. The higher-order EOFs, which gradually decrease in significance, allow the profile to depart from the mean. Typically, 2 to 3 EOFs are sufficient when using ground-based data alone.

Figure 2.

Example empirical orthonormal functions (EOFs).

[15] Cycle slip detection and phase to pseudorange leveling must be carried out for all available data over the past few hours for each filter update. The data are then processed into a format that can be ingested by the Kalman filter. Note that the Kalman filter state error estimates are highly dependent on the process noise parameter. One of the output products offered by the real-time software is ASCII files, containing the vertical TEC and the slant line-of-sight electron content to each satellite for a grid of points across the US mainland. Data are stored for each satellite as a two-dimensional matrix of slant TEC values (in 0.1 TEC units).

[16] The US-TEC test product is currently being validated for reliability and accuracy, and is expected to transition to full operations in the near future. This venture into space weather assimilative models is a first step, and is laying the foundations for future products of this kind. The next steps will extend the coverage to global, include a wider spectrum of input data, and adopt physical models for the propagation of the state [e.g., Schunk et al., 2004; Hajj et al., 2004]. The latter will enable the assimilative models to forecast the space weather system.

4. Validation

[17] Validation of absolute TEC is a challenge because there are very few direct TEC measurements that are “unbiased.” One of the key components of a good metric is to have a reliable and accurate measurement against which to compare the model. The accuracy of the US-TEC map is expected to be in the range of 1 to 3 TEC units, so ideally the measurement requirement for the metric should be accurate to less than 1 TEC unit, in order for the validation to be meaningful. Ionosondes provide an estimate of TEC, but it is not sufficiently accurate since there are no topside data to estimate the thickness of the electron density profile. Data from the TOPEX/Poseidon and Jason satellite would be preferable, but the data are only available over the oceans where no CORS station exists, and there is a bias in the data that has to be estimated.

[18] Because of the difficulty in obtaining an absolute unbiased TEC measurement, part of the validation effort is devoted to a differential accuracy estimate. The differential method is internal, but may reflect the absolute accuracy in some cases. The absolute method utilizes data from the Los Alamos Fast On-Orbit Recording of Transient Events (FORTE) satellite [Moses and Jacobson, 2004]. The FORTE satellite at 800 km altitude includes a broadband RF receiver between 30 and 300 MHz. By recording the transmission from a simulated lightning pulse, the time delay as a function of frequency can be used to estimate the absolute TEC along the transmission path from transmitter to the satellite.

4.1. Analysis of Differential TEC

[19] If continuous GPS data are available along a given receiver–satellite link with no cycle slips, estimates of the differential TEC, from one epoch to the next, can be determined with an accuracy of less than 0.01 TEC units. This fact can be utilized to estimate the uncertainty in the US-TEC vertical and slant path maps. By integrating through US-TEC inversion maps at two different times, the difference in the slant TEC can be compared with the direct TEC difference derived from the original RINEX data file.

[20] The TEC difference between observations and US-TEC, ɛ, can therefore be expressed as

equation image

(in TEC units), where Δϕ is the combination of the differential phase between dual frequencies (L1 and L2) as expressed in TEC units, and δ(Δϕ) is the difference in the differential phase between two epochs along the same satellite-receiver link. The two epochs can be separated by subminutes to hours. The TEC estimate is relative, so that as the time difference between the rays tends to zero, the TEC difference, δ(Δϕ), will tend to zero and the TEC difference, ɛ, compared to the inversion, will also tend to zero. As the time difference between the rays increases, the relative TEC differences between the observations and model will increase.

[21] For small time separation, the TEC differences in the inversion, δ(ΔϕUSTEC), will correlate (i.e., the ionosphere may be systematically low or high for both rays) hence the errors will cancel. However, for a sufficiently large time difference, the errors in the maps will tend to decorrelate, so we suggest that the error estimate will tend to saturate to the actual error in the US-TEC inversion. The differential accuracy analysis computes a root mean square error (RMSE) between the RINEX TEC difference and inversion TEC differences for paths between a given receiver and all satellites over the whole day. Figure 3 shows an example for the ARP3 receiver at Aransas Pass, Texas. Figure 3 shows the gradual increase in the RMSE as the time difference between the rays increases. The value tends to plateau, or saturate, a little over 2 TEC units. The RMSE for the International Reference Ionosphere (IRI) is also shown, which has a consistently larger error and does not reach a plateau. Note that the RMSE for this study is representative of the uncertainty in the average slant path TEC. The equivalent error for vertical TEC is expected to be a factor of about 1.3 lower, depending on the average elevation angle of all the satellites in view. The long time separation of the rays ensures that the method samples maps at widely different times, so the error estimate does not simply reflect the smoothness in the maps. A more complete description of the method can be found in work by E. A. Araujo-Pradere et al. (Differential validation of the US-TEC model, submitted to Radio Science, 2006).

Figure 3.

Example of differential TEC validation for one of the nine reference stations.

[22] The final error estimate (from the saturation level) will be an averaged estimate for the day. If no saturation occurs (as is common with the IRI), it implies that values are systematically biased over timescales greater than a few hours, so they cannot be estimated with this method.

[23] The estimates of accuracy using the differential method have been performed for nine stations spread over the CONUS; the data for none of these sites were included in the assimilation process. At each station the daily averaged RMSE analysis has been performed every fifth day since March 2004. The stations chosen are PABH: Florida Beach, Washington, YBHB: Yreka, California, BILL: Lake Skinner, California, CLK1: Clark 1, South Dakota, HBRK: Hillsboro, Kansas, ARP3: Aransas Pass 3, Texas, WES2: Haystack Observatory, Westford, Massachusetts, VIMS: Virginia Institute of Marine Science, Wachapreague, Virginia, and CCV3: Cape Canaveral 3, Florida.

[24] Table 1 shows the RMSE statistics at a time difference of 160 min, for a three-month period for each station and day, and the averages over the previous 6 months. The values from the IRI are also shown. Note that in most cases the RMSE for IRI has not reached a plateau, so the values sampled at a time difference of 160 min (see example in Figure 3) are not expected to reflect the true uncertainty. For each day, the average number of stations available for the 96 TEC maps during that day is displayed, together with the RMSE difference between US-TEC and IRI and the daily Ap. On typical days when about 60 stations are consistently available, the uncertainty for the slant path TEC is between 2 and 3 TEC units. The average RMSE for all conditions over the 6-month period is 2.4 TEC units, which is equivalent to just less than 2 TEC units uncertainty in the vertical TEC. Note that on the storm day, 25 July 2005, the RMSE rises slightly to 3.3 TEC units, which is to be expected because of the steeper gradients existing during geomagnetic storms.

Table 1. RMSE of Differential TEC for IRI and US-TEC for Each of Nine Reference Stations Between April and September 2005
 JulyAugustSeptemberAve
510152025305101520253051015202530
IRI
pabh2.64.14.43.26.11.62.62.04.53.94.02.8 4.22.12.03.02.93.4
ybhb3.44.54.64.07.32.82.93.84.54.53.94.2 4.43.33.04.14.14.0
bill5.05.05.45.27.83.34.05.24.54.04.24.6 3.53.89.74.96.45.1
clk12.32.45.54.36.92.63.02.94.13.94.24.5 5.04.72.12.53.13.8
hbrk3.73.66.04.79.53.53.63.15.33.43.23.5 4.74.92.84.03.44.4
arp34.95.15.15.38.13.33.64.64.94.13.26.9 4.24.73.75.75.05.0
wes22.94.04.95.06.73.03.03.45.83.72.94.8 4.84.22.72.32.53.8
vims3.54.95.84.88.64.02.93.56.02.63.34.7 3.15.32.83.62.94.3
ccv3 5.96.25.17.63.63.23.26.33.4 4.3 3.14.32.84.23.14.5
Average3.54.45.34.67.63.13.23.23.55.13.74.5 4.14.23.53.83.74.2
 
US-TEC
pabh1.91.91.81.63.21.11.61.22.02.01.91.8 1.71.31.61.71.71.9
ybhb2.02.82.32.12.91.71.91.62.52.62.52.3 2.22.11.82.12.52.3
bill3.13.53.43.53.72.12.52.43.02.82.32.9 2.72.68.92.73.53.2
clk11.61.52.12.53.21.31.61.91.82.12.42.9 2.22.41.21.31.51.9
hbrk1.91.62.22.63.91.51.71.72.12.12.32.0 2.02.31.31.61.92.1
arp33.42.82.83.92.81.92.72.63.03.51.84.7 3.33.32.32.93.33.2
wes21.71.92.21.82.91.41.61.92.61.31.82.4 2.32.31.51.41.62.0
vims1.91.72.12.04.01.51.81.92.41.62.32.3 2.02.51.71.51.42.0
ccv3 2.82.43.03.11.62.02.22.82.4 2.4 2.42.62.12.32.32.7
Average2.22.32.42.53.31.61.91.92.52.32.22.6 2.32.42.52.02.32.4
 
US-TEC/IRI
pabh−0.8−2.2−2.5−1.5−2.9−0.4−1.1−0.8−2.5−1.9−2.1−1.0 −2.5−0.8−0.4−1.3−1.2−1.5
ybhb−1.4−1.7−2.4−1.9−4.4−1.1−1.0−2.2−2.0−1.9−1.3−1.9 −2.1−1.3−1.2−2.0−1.6−1.7
bill−1.9−1.6−2.0−1.7−4.0−1.2−1.5−2.8−1.5−1.2−1.9−1.7 −0.9−1.2−0.8−2.3−3.0−1.9
clk1−0.7−0.9−3.4−1.8−3.7−1.3−1.4−1.0−2.4−1.8−1.8−1.5 −2.8−2.3−0.9−1.2−1.7−1.9
hbrk−1.9−2.0−3.8−2.1−5.6−2.0−1.9−1.3−3.2−1.4−0.8−1.5 −2.7−2.6−1.5−2.4−1.5−2.3
arp3−1.5−2.3−2.3−1.5−5.3−1.4−1.0−2.0−1.8−0.6−1.4−2.2 −1.0−1.4−1.4−2.8−1.7−1.8
wes2−1.2−2.1−2.7−3.2−3.7−1.6−1.4−1.5−3.2−2.3−1.0−2.3 −2.6−1.9−1.1−0.9−1.0−1.9
vims−1.6−3.3−3.7−2.8−4.5−2.5−1.1−1.6−3.6−1.1−1.0−2.4 −1.2−2.8−1.1−2.1−1.5−2.2
ccv3 −3.0−3.8−2.1−4.5−2.0−1.2−0.9−3.5−1.0 −1.9 −0.7−1.7−0.7−1.8−0.8−1.8
Average Difference−1.4−2.1−3.0−2.0−4.3−1.5−1.3−1.6−2.6−1.4−1.4−1.8 −1.8−1.8−1.0−1.9−1.5−1.9
Ap Index7899122771471473475141354 
Number of Stations5859595858575857575349583585958575750

4.2. Analysis of Absolute TEC

[25] The purpose of the absolute validation is to confirm, or otherwise, the notion that the differential TEC validation, described above, provides a realistic estimate of the slant path TEC uncertainty. This analysis requires measurements of TEC that are accurate to less than 1 TEC unit. None of the conventional data sources, including ionosondes, TOPEX, and dual-frequency GPS, reach this criterion for a variety of reasons. Possible sources of data with sufficient accuracy are satellite-based lightning detectors. There are two satellite-based VHF lightning detection projects at Los Alamos National Laboratories, the Fast On-Orbit Recording of Transient Events (FORTE) system and the VHF Global Lightning and Severe Storm monitor (V-GLASS) system. We have performed an analysis comparing the FORTE data with US-TEC.

[26] With the FORTE broadband signal, the phase can unambiguously be connected across a wide bandwidth, and the problem of resolving the phase ambiguity in different channels does not arise. The fit to these data provides the unambiguous TEC in some appropriate units.

[27] In addition to the frequency-dependent delay, two other aspects need to be considered when interpreting or comparing with the FORTE data. At the relatively low frequencies used by FORTE, the bending of the rays adds an additional path length, and hence delay, to the signal. This additional delay depends on frequency as 1/f4, the angle from the zenith, and the total TEC. In order to estimate the additional path length, either ray-tracing algorithms must be employed to estimate the delay due to the bending of the rays, or the fit to the data must include the 1/f4 dependence. A second aspect is that the FORTE satellite is at 800 km so the observations only sample the electron content below the satellite altitude. Therefore the three-dimensional plasma distribution predicted by US-TEC was sampled just up to this altitude, and so does not include the plasmaspheric contribution above.

[28] From a collection of about 20 overpasses from different days, all at low to moderate geomagnetic activity with Kp < 3, the TEC data from the FORTE satellite were estimated from a combined 1/f2 and 1/f4 fit to the signal delay, only using data down to 40 MHz. The RMSE of the TEC between FORTE and US-TEC was 2.65 TEC units over all the slant paths, equivalent to about 2.0 TEC units for the vertical content. The uncertainty estimates were reasonably consistent with the results from the differential method.

[29] The analysis has also been performed using sophisticated ray-tracing algorithms. Figure 4 shows a comparison of FORTE and US-TEC for eight overpasses of the satellite. The RMSE for this sample of the data set is 3.9 TEC units for the slant path, which is equivalent to less than 3 TEC units for the vertical. It is clear from Figure 4 that the uncertainty is not randomly distributed, but there appears to be a bias that is dependent on the absolute magnitude of the TEC. The average bias for the slant path TEC is 3.3 TEC units and standard deviation about the mean is 2.1 TEC units. The bias may be a result of the longer path length in the FORTE data, which is difficult to eliminate entirely in the ray tracing algorithms.

Figure 4.

Comparison of slant path TEC from FORTE and US-TEC.

[30] There are three possible sources for the difference in FORTE and US-TEC.

[31] 1. Uncertainty in the FORTE observations, the fits to the data, or the ray tracing, required to extract the TEC value.

[32] 2. Errors in the US-TEC map itself.

[33] 3. Errors introduced in subsampling the US-TEC vertical profile. The latter is a result of the difference in the TEC to the altitude of the FORTE satellite and the total TEC to the GPS satellites. It is necessary to subsample the US-TEC inversion to include only the TEC to the satellite altitude. Uncertainties in the vertical profile can lead to the introduction of errors even when the total TEC is precisely known.

5. Conclusion

[34] US-TEC has been running as a test operational product at SEC for over a year. The model is driven by the available real-time, ground-based, dual-frequency, GPS stations. The analysis of the RMSE for the differential TEC indicates an uncertainty in the slant path TEC of about 2.4 TEC units over the 6-month testing period. This can be compared with 2.65 TEC units for the fits to the FORTE data, and 3.9 TEC units for the estimates using the ray-tracing algorithms. The results indicate that the differential TEC RMSE are likely to be valid and to reflect values close to the true uncertainty. The US-TEC uncertainty in the vertical TEC is therefore estimated to be about 2 TEC units during quiet geomagnetic conditions. The accuracy of the estimates of slant path electron content, provided in the data files, are expected to be between 2 and 3 TEC units. The most geomagnetically active day in Table 1 was 25 July with an Ap of 122. The uncertainty rose slightly in US-TEC by about 0.7 TEC units, whereas the IRI reference model increased substantially.

[35] The timeliness and accuracy of the product renders it useful for improved single-frequency positioning and for initializing dual-frequency integer ambiguity resolution schemes for real-time kinematic applications (RTK). Future developments include improving the temporal resolution to 5 min, increasing the number of data stations, and expanding the geographic domain of the TEC maps.

Ancillary