The Short‐Time Prediction of Thermospheric Mass Density Based on Ensemble‐Transfer Learning

Reliable short‐time prediction of thermospheric mass density along the satellite orbit is always essential but challenging for the operation of Low‐Earth orbit satellites. In this paper, three machine‐learning prediction algorithms are investigated, including the Bidirectional Long Short‐Term Memory, the Transformer, and the Light Gradient Boosting Machine (LightGBM) ensemble model of the above models. We use satellite data from CHAMP, GOCE, and SWARM‐C to evaluate the robustness and accuracy of different density variations. The comparison demonstrates that all models achieve compelling predictions and are much better than NRLMSISE‐00. The LightGBM ensemble model (LE‐model) consistently outperforms others in accuracy and stability. Furthermore, when the obtained density data from the newly launched satellites are limited, the trained LE‐model can provide a valid prediction for the new satellite orbit by transfer learning. This study offers a promising insight into the short‐time prediction of thermospheric mass density using ensemble‐transfer learning and may be advantageous to future research on space whether.

outperforms others in accuracy and stability.Furthermore, when the obtained density data from the newly launched satellites are limited, the trained LE-model can provide a valid prediction for the new satellite orbit by transfer learning.This study offers a promising insight into the short-time prediction of thermospheric mass density using ensemble-transfer learning and may be advantageous to future research on space whether.

Plain Language Summary
Low-Earth orbit (LEO) satellites play an important role in many aspects, such as navigation, aerospace, military industry, and so on.The LEO satellites suffer atmospheric drag caused by thermospheric mass density.Therefore, we present three different machine-learning algorithms to achieve a robust short-time prediction for thermospheric mass density.All models can provide effective results from testing with

Introduction
Thermospheric mass density is an essential parameter in the earth's thermosphere and can cause atmospheric drag force on LEO satellites.As a typical non-conservative force, atmospheric drag is the fundamental perturbation source of satellites, and it has a constant and considerable effect on the satellites, such as orbit determination, collision warning, and motion safety (Doornbos, 2012).Meanwhile, extreme space environment conditions can lead to complex and variable responses of thermosphere mass density, significantly impacting satellites' orbit (Lei et al., 2013;Emmert, 2015).
For instance, in February 2022, 49 Starlink satellites suffered from a moderate geomagnetic storm (Kp=5) during orbit raising.In a short time, atmospheric drag increased by 50% because of increased density and declined orbit.In the end, this event caused 40 satellites to fall out of orbits.Therefore, with the growing importance of LEO satellites in navigation, communication, aerospace, military industry, emergency response, and commercial applications, short-time prediction of thermosphere mass density along the satellite orbit is indispensable.
With the rapid development of artificial intelligence (AI) technology and the continuous increase of observations in space physics, machine learning, as one kind of data-driven methods, provides a promising way for researchers.Thereinto, deep learning technology (Lecun et al., 2021) based on neural network (NN) has shown its powerful data-learning ability and has been widely applied in various aspects of the space weather forecast, such as geomagnetic index (Shprits et  Different from the past methods, in this work, we try to present a short-time prediction with high-precision and high-fidelity for thermospheric mass density.We design three algorithms based on the Bi-LSTM, the Transformer, and the LightGBM ensemble model of the above models using ensemble learning for two kinds of multi-step prediction.The prediction performances are compared with CHAMP, GOCE, and SWARM-C to verify the robustness and accuracy.The results demonstrate that reliable predictions can be achieved in all models, and the LE-model consistently outperforms other standalone models in terms of accuracy and stability.
Subsequently, when the available density data from the newly launched satellites are limited, the trained LE-model is also suitable for neutral density prediction along new satellite orbit using transfer learning.This lightweight method saves time and computing resources while maintaining accuracy and offers beneficial assistance for satellite-borne intellectualization research.Additionally, the NRLMSISE-00 has been used in the comparative analysis of the whole verification process.This study gives a promising insight into the short-time prediction of thermospheric mass density based on machine-learning algorithms, which can be critical for practical applications to the related research of LEO satellites.This paper's structure is as follows: The descriptions of the datasets are briefly introduced in Section 2. Section 3 illustrates the model algorithms.In Section 4, the corresponding experimental results and analyses are introduced.Finally, the conclusion and directions for future work are summarized in Section 5.

Data Description
In this study, we utilize the accelerometer-derived (ACC) data from TU Delft (http://thermosphere.tudelft.nl/)as datasets, which are quasi-instantaneous along orbits and closer to the actual values than the precise orbit determination (POD) data (Doornbos, 2012;Siemes et al., 2016;March et al., 2019).The ACC data used comprise the ACC density and corresponding local parameters (altitude, longitude, and latitude).Moreover, the external physical parameters (F 10.7 , Ap, and DOY) from OMNI data (https://spdf.gsfc.nasa.gov/pub/data/omni/low_res_omni/)are added to datasets since the dynamic natural variabilities need to be considered.To verify the algorithm's usability and broader applicability, three different datasets from CHAMP, GOCE and SWARM-C are utilized.Each dataset is partitioned three times to get test-1/2/3, which there are 9 tests in total.The tests are sequential periods that keep the same test samples (100000 for CHAMP, 120000 for GOCE, and 300000 for SWARM-C).Table 1 depicts the detailed information of three satellites in the datasets used.Each test-1 period encompasses one moderate storm.The storm event is selected when the minimum Dst value is less than -50 nT during the main phase, and the Dst value of -10 nT is used as the threshold for the start of the initial phase and the ending of the recovery phase.

The Prediction Results and Discussions
The empirical model, NRLMSISE-00, is used as a unified reference during the whole model comparison process.The parameter inputs consist of the DOY, seconds since the start of the day, longitude, latitude, altitude, F 10.7 , F 10.7A (81-day sliding averaged value of F 10.7 ), and Ap.Furthermore, to accurately assess the models, the root mean square error (RMSE), and the coefficient of determination ( ) are utilized as evaluating standards in this study.They are defined in the following investigations: In detail,  is the predicted density, and  is the observed density. and  are respectively the mean density values of  and  .And  is the sample number.

The Results of Model Comparison
In this section, to verify the robustness and accuracy of the model algorithms, the this period, the solar activity is very low, which F 10.7 is from 69.5 sfu to 74.4 sfu (the average value is 71.5 sfu).However, the altitude of satellite orbit has roughly risen 200 km, which reaches the range of 433 km to 464 km.Consequently, the density values decrease several orders of magnitude compared with before.Figure 5 displays the detailed comparison.In this test, the NRLMSISE-00 gives worse results (RMSE: 1.527×10 -13 ,  : -1.948) than before in CHAMP ( : 0.109) or GOCE ( : 0.731).
Oppositely, the machine-learning models still maintain a superior level of performance (RMSE: 1.077×10 -14 -1.697×10 -14 ,  : 0.964 -0.985).In more detail, different from the above tests, the Bi-model first precedes the Trans-model when N=6 time steps.But the final comparison result has not changed that the LE-model of 6 time steps continuously outperforms other models in terms of accuracy and stability.From the qualitative analysis of three satellite datasets, the machine-learning models give satisfactory prediction results with different periods, altitudes, and response variations.Subsequently, it is also necessary to further verify the prediction performances by statistical analysis.In Figure 6, there is the RMSE decrease between each model and NRLMSISE-00 in the three different tests.It is visible that these models are much more advanced than NRLMSISE-00, in which all decrease values of RMSE are larger than 78.5%, and the best value is 93.0%.More importantly, the LE-model invariably has the best performance for the two kinds of multi-step predictions.In detail, the RMSE decreases of the LE-model are followed by GOCE (86.1%, 82.5%)<CHAMP (87.9%, 87.8%)<SWARM-C (93.0%, 91.0%) for N=6/10 time step.However, on the contrary, the LE-model performances ( as the example) from above tests are followed by SWARM-C (0.985, 0.976)<CHAMP (0.987, 0.987)<GOCE (0.995, 0.992) for N=6/10 time step.Actually, it needs to be noted that this sorting order of the RMSE decrease is under the huge influence of the NRLMSISE-00 results that are followed by SWARM-C ( : -1.948)<CHAMP ( :  To make the statistical analysis more convincible, the results of test-2 and test-3 in each satellite dataset also show in Table 3 and Table 4.In the test-2, the  is from 0.928 to 0.995, and the decrease of RMSE is from 79.0% to 92.9%.In the test-3, the  is from 0.944 to 0.997, and the decrease of RMSE is from 81.7% to 94.8%.
Importantly, the LE-model consistently has the best average prediction performance since it can outperform the other two models when the test conditions are constantly changing.

The Results of Transfer Learning
For the newly launched LEO satellites, the obtained data along the new satellite orbits are limited, and the available models could not be trained with sufficient data to perform accurate predictions.To solve the problem, based on transfer learning, we utilize the effective pre-trained machine-learning model by large amounts of data from the past satellites to retrain the obtained data from the new satellite with fine-tuning.On the one hand, this lightweight method can provide a new viewpoint for the short-time prediction of thermospheric mass density along orbits, which reduces computation complexity and dramatically saves time and computing resources while maintaining accuracy.

Conclusion
This paper aims to apply machine-learning algorithms to provide a robust al., 2019; Xu et al., 2020; Tan et al., 2018; Gruet et al., 2018), solar activity (Fang et al., 2019; Tang et al., 2021(a); Tang et al., 2021(b)), Total Electron Content (TEC) (Chen et al., 2019; Pan et al., 2020; Liu et al., 2020; Tang et al., 2020; Chen et al., 2022), electron flux (Pires de Lima et al., 2020; Tang et al., 2022), as well as NO emission (Chen et al., 2021).While related studies are far more than those listed above, these works have proved that machine-learning methods have been intensively studied in space weather forecasts.As for thermospheric mass density, NN also has been gradually used in recent years.Wang et al. (2014) utilized Artificial Neural Network (ANN) to investigate intra-annual variations at a fixed altitude for ten years.Weng et al. (2020) revisited the average variation trend during either 1967-2005 or 1967-2013 from 250 km to 575 km based on ANN.These works utilized the basic NN to focus on the long-term trend and states by using large-scale physical parameters.On the other hand, Perez et al. (2014) proposed the orbit-concerned prediction by ANN, and then Perez and Bevilacqua (2015) presented two time-delay prediction approaches based on NN.These works mainly tried to use current external parameters and density values to realize the window prediction along the satellite orbit.Recently, Wang et al. (2022) used near-real-time parameters to provide a deep-learning algorithm based on the LSTM (Long Short-Term Memory)-based ensemble learning, which paid more attention to storm-time prediction.

For
the time-step setting, the serial 10 data samples (10 time steps) are prepared as the historical data, almost 300 seconds long for CHAMP and 100 seconds long for GOCE and SWARM-C.Then, the serial N samples (N=6/10 time steps) are predicted, which roughly correspond to 180/300 seconds for CHAMP, 60/100 seconds for GOCE, and SWARM-C backward.Therefore, when the current time is T, the historical samples contain a total of 10 data samples from T-10 to T-1, while the predictions contain N data samples from T to T+N-1.Figure1depicts the detailed time-step settings for the three satellites based on their temporal resolutions. Fig

Fig 2 (
Fig 2 (a) The details of three models.(b) The process of transfer learning in this

Fig 3 Fig 4
Fig 3 The comparison of the machine-learning models (red) and NRLMSISE-00 (blue) with CHAMP observation during the period (-74 nT≤Dst≤44 nT) from July 31 to September 4, 2010.Similarly, the models display their corresponding prediction results of GOCE's

Fig 5
Fig 5 The comparison of the machine-learning models (red) and NRLMSISE-00 (blue) with SWARM-C observation during the period (-59 nT≤Dst≤14 nT) from August 24 to September 30, 2020.
0.109)<GOCE ( : 0.731) from the above tests.Although NRLMSISE-00 constantly updates space weather indices to conduct model correction, it still often gives unsatisfactory simulation results like these.

Fig 6
Fig 6 The decrease of RMSE (%) between each machine-learning model and NRLMSISE-00 for N=6/10 time steps in different satellite datasets during the corresponding test period.

a
massive reduction from about 3.5 hours to 5 minutes since transfer learning just needs a little new data to retrain, which saves a large amount of training time and computing resources.The pre-trained LE-model has an excellent prediction performance for the fundamental variation features of thermospheric mass density, especially the details of peaks and valleys.Therefore, the GOCE LE-model can rapidly learn valuable features of density variations along the new satellite orbits during quiet periods.

Fig 7
Fig 7 The detailed comparison of the LE-model prediction (red), the observation (black), and NRLMSISE-00 (blue) during a quiet period (-7 nT≤Dst≤10 nT) from March 8 to 9, 2020.The storm case in transfer learning is from February 23 to 25, 2015, and the

Table 4 The evaluation results in test-3 of three satellite datasets.
(Keller, 2018)and, it may supply helpful assistance for future satellite-borne applications of AI technologies.With the in-depth development of aerospace digitization and intellectualization, satellite-borne software systems are improving to support on-board data processing, AI algorithms, and independent adjustment and decision-making, like Project Blackjack(Keller, 2018).Moreover, the altitude range is from 434 km to 465 km.From Figure7, the LE-model (red line) achieves positive transfer results that are more consistent with the measured  : 0.985).The difference in RMSE is more than an order of magnitude, and the RMSE decrease of the LE-model is 91.4%.Meanwhile, the prediction performance has hardly any reduction since  of LE-model, in this case, is extremely close to that in the test of SWARM-C (both are around 0.985).However, the training time has variation from March 8 to 9, 2020.During this quiet period, the Dst is between a minimum value of -7 nT and a maximum value of 10 nT, and the F 10.7 is around 69 sfu.