Accuracy of Global Geospace Simulations: Influence of Solar Wind Monitor Location and Solar Wind Driving

Some space weather models, such as the Space Weather Modeling Framework (SWMF) used in this study, use solar wind propagated from the first Lagrange point (L1) to the bow shock nose (BSN) to forecast geomagnetic storms. The SWMF is a highly coupled framework of space weather models that include multiple facets of the Geospace environment, such as the magnetosphere and ionosphere. The propagated solar wind measurements are used as a boundary condition for SWMF. The solar wind propagation method is a timeshift based on the calculated phase front normal (PFN) which leads to some uncertainties. For example, the propagated solar wind could have evolved during this timeshift. We use a data set of 123 geomagnetic storms between 2010 and 2019 run by the SWMF Geospace configuration to analyze the impact solar wind propagation and solar wind driving has on the geomagnetic indices. We look at the probability distributions of errors in SYM‐H, cross polar cap potential (CPCP), and auroral electrojet indices AL and AU. Through studying the median errors (MdE), standard deviations and standardized regression coefficients, we find that the errors depend on the propagation parameters. Among the results, we show that the accuracy of the simulated SYM‐H depends on the spacecraft distance from the Sun‐Earth line. We also quantify the dependence of the standard deviation in SYM‐H errors on the PFN and solar wind pressure. These statistics provide an insight into how the propagation method affects the final product of the simulation, which are the geomagnetic indices.


Introduction
The modern society relies on space-based and ground-based technologies that are susceptible to space weather hazards (National Research Council, 2008).Accurate and timely space weather forecasts can mitigate risks associated with, for example, increased errors in positioning and navigation applications, malfunctioning satellites and their subsystems, or large-scale power failures caused by geomagnetically induced currents (GIC) (Maynard, 1995).Such models that forecast these kinds of events, be they global magnetospheric simulations or empirical models derived from observations, rely on observations in the solar wind and/or in the magnetosphereionosphere system.While the requirements for model accuracy increase, it is important to review all potential error sources in the input-output analyses.
A geomagnetic index is typically a summary of ground magnetometer observations that can be used as a proxy of the global level of geomagnetic activity.The hourly Disturbance Storm-Time index Dst and its one-minutecadence variant SYM-H are computed as a weighted average of magnetic perturbations of mid-latitude magnetic stations (Iyemori, 1990;Sugiura, 1964), and give a proxy of the ring current intensity.On the other hand, the polar cap index (PCI) is based on observations of a single ground magnetometer at the Thule station, and is used as a proxy for the electric potential across the polar cap (O. A. Troshichev et al., 1988).The Auroral Electrojet indices are aggregates of the minimum (AL) or maximum (AU) observations made by a set of northern hemisphere high-latitude ground magnetometer stations (Davis & Sugiura, 1966), and provide an estimate of the westward and eastward electrojet current intensity, respectively.The interpretability of these indices and the wide availability of the data make them useful for model validation, especially when the models can easily output corresponding time series.
Operationally, the National Oceanic and Atmospheric Administration (NOAA) Space Weather Prediction Center (SWPC) predicts geomagnetic indices such as the Auroral Electrojet (AE, AU and AL) and the Disturbance Storm-Time (Dst) by running global magnetohydrodynamic (MHD) simulations, whose results can be used to compute corresponding model indices (A.Pulkkinen et al., 2013).Generally, these models use ideal MHD coupled with modules representing processes in the ionosphere (Janhunen et al., 2012) and the inner magnetosphere (Gombosi et al., 2021;Lyon et al., 2004;Toffoletto et al., 2003).The Space Weather Modeling Framework (SWMF) Geospace configuration is a combination of models that represent the magnetosphere, inner magnetosphere and ionosphere, and uses solar wind input and the solar radio flux (F 10.7 ) to forecast geomagnetic activity (Tóth et al., 2012).
The solar wind input for the global MHD simulations is prepared from solar wind observations that are typically obtained from spacecraft such as the Advanced Composition Explorer (ACE) (Stone et al., 1998) and Wind (Wilson III et al., 2021) that orbit close to the first Lagrangian point (L1).The solar wind observations are then propagated to the magnetospheric bow shock nose (BSN) to serve as inputs into the models (A.Pulkkinen & Rastätter, 2009).This propagation is done to avoid running an MHD simulation from L1 to the BSN, over a distance that is similar to the length of the entire magnetotail.Here we focus on one particular aspect of the propagation problem, the accuracy of the empirically estimated time shift from the observing solar wind spacecraft to the magnetopause.

Model Validation Through Geomagnetic Indices
M. W. Liemohn et al. (2018) discuss guidelines for model validation using geomagnetic indices SYM-H, AL, and AU.For prediction of geomagnetic indices, the study evaluated the model accuracy using multiple criteria, which can be divided into fit metrics or event detection performance metrics.The fit metrics include statistical evaluation of the error defined as the difference between the model and the observed values.These metrics include the root mean squared error (RMSE), the mean error (ME), and the mean absolute error (MAE).Event detection performance metrics are binary evaluations of the model ability to correctly assess whether an event would exceed a chosen index threshold.These metrics include skill scores such as the Heidke Skill Score (HSS), probability of detection (POD), and probability of false detection (POFD).
Many efforts have been undertaken to validate SWMF's ability to predict geomagnetic indices (Rastaetter et al., 2013;Haiducek et al., 2017;Welling et al., 2017;M. Liemohn et al., 2018).SWMF has been shown to predict SYM-H well with an RMSE of 17-18 nT, when the ring current effects simulated by the Rice Convection Model (RCM) are included.In the case of a pure MHD magnetosphere, the RMSE is almost double, ∼29 nT (Haiducek et al., 2017).Rastaetter et al. (2013) studied the performance of 30 model configurations regarding their ability to predict SYM-H, and showed that physics-based models that include a ring current model are comparable to empirical or statistical models, indicating that models such as the SWMF can predict magnetic disturbances at the mid-latitudes associated with the ring current processes.
Several studies have also addressed the auroral latitude index predictions.The SWMF has been shown to underpredict the magnitude of AL with RMSE of 230-270 nT (Haiducek et al., 2017;Al Shidi et al., 2022;T. I. Pulkkinen et al., 2022).However, the skill scores of SWMF are relatively good (HSS of 0.4-0.6)if a threshold value of 200 nT is used (Al Shidi et al., 2022).This means although the model underpredicts the actual currents, it can still detect the times of auroral electrojet intensification during geomagnetic storms.
Finally, Morley, Brito, and Welling (2018) present robust accuracy metrics for the radiation belt electron flux predictions.An interesting outcome of that study is that they found the median absolute error (MdAE) to be a robust metric, while the MAE/RMSE in their data set was not.The signed median is of interest to this study, as it indicates whether a model is over/under-predicting, and is easy to interpret.These results led us to use the median error (MdE) for this study.

Solar Wind Propagation
Generally, the solar wind input for any input-output analyses relies on solar wind observations propagated from L1 to close to the BSN.L1 monitors such as Wind or ACE have orbits that can be as far as ∼236 Earth radii (R E )

10.1029/2023SW003747
AL SHIDI ET AL. from the Earth, and >100 R E away from the Sun-Earth line.There are multiple ways to propagate the solar wind the 200 R E to reach the vicinity of the Earth's magnetosphere-one way would be to run an MHD simulation from L1 to the Earth and use that as input.However, in practice, most applications use a time-delay technique based on 1-D propagation and the method of characteristics (Cash et al., 2016;Weimer et al., 2003).
The propagation time is determined using the solar wind speed, distance to the observing spacecraft, and by finding a phase front normal (PFN) that describes the orientation of the solar wind front that is approaching the Earth (see Figure 1).The PFN can be found using several methods, such as the minimum variance analysis (MVA) technique (Sonnerup & Scheible, 1998;Weimer et al., 2002) or the cross-product technique (Horbury et al., 2001).Typically, a combination of these techniques are used, and this is also what is used for the widely available propagated solar wind data set in the OMNI database (King & Papitashvili, 2023).
In Figure 1, we show a schematic drawing that shows the principle behind solar wind propagation from L1 to the BSN.The axes are in the Geocentric Solar Ecliptic (GSE) coordinate system with the Earth marked by a blue circle at the origin.The bow shock enveloping the Earth is shown in brown.The black cross represents the location of the L1 spacecraft observing the solar wind.The phase front normal ( n see calculation below) is shown by the red arrow and the phase front is illustrated by the red dashed line.The instantaneous solar wind measurement is propagated along the phase front line to the location marked by a red cross.The velocity of the solar wind is shown as a blue arrow at the propagated location.The position of the spacecraft relative to the BSN, r, is represented as well as its components, Δx and Δy.In this 2-dimensional case Δy would represent the perpendicular distance of the spacecraft to the Sun-Earth line.The uncertainty arises when from the assumption that the solar wind quantities at the measurement point can be propagated unchanged to the Earth and/or the satellite is in similar conditions to the location being propagated.
While the solar wind and IMF parameters have their intrinsic uncertainties related to the measurement techniques, the uncertainties related to the time shifting to the bow shock create an additional source of (timing) errors.The time shift of the solar wind is calculated by considering a solar wind front detected at time t at an upstream spacecraft (at location r spacecraft ), and assuming that it reaches the bow shock nose at time t + Δt.The transit time or timeshift Δt can be calculated by advecting the phase front normal plane using the equation where r BSN is the location of bow shock nose, V is the solar wind velocity observed at L1, and n is the shock phase front normal derived from the solar wind observations at the spacecraft location (see Figure 1 demonstrating the principle of the PFN advection).Errors of this propagation method will arise from errors in the solar wind speed, positioning accuracy of the spacecraft, as well as the evaluation of the PFN orientation.
Cameron and Jackel (2019) have shown that using a numerical simulation to propagate the solar wind can capture some features that the time shifting method cannot.Weimer et al. (2003) suggests that using the MVA technique to determine PFN is prone to some uncertainty based on the number of data points used.Several studies conclude that the errors in timing the arrival of the phase fronts can be of the order of several minutes (A.Ridley, 2000;Weimer et al., 2002Weimer et al., , 2003;;Mailyan et al., 2008;A. Pulkkinen & Rastätter, 2009;Milan et al., 2022).
This study focuses on the uncertainty and errors that are related with the solar wind propagation.Specifically, we analyze the relationship between solar wind propagation parameters (i.e., PFN, time shift, spacecraft displacement from Sun-Earth line) to the final product of the simulation (geomagnetic indices).The aim is to statistically quantify the errors and conditional distributions of the errors in order to have a better understanding of how SWMF performs based on the preparation of the measurements in the input.We find the median errors, which show the tendency of the simulation to over/underestimate geomagnetic indices, and the standard deviation of errors, which shows the spread of the distribution of the errors.The following sections will focus on how we achieve this analysis.

The SWMF Geospace Model
The SWMF Geospace configuration (Tóth et al., 2012) consists of three coupled models describing the different regimes in the global magnetosphere-ionosophere system (Gombosi et al., 2021).The Block-Adaptive-Tree-Solarwind-Roe-Upwind-Scheme (BATSRUS) code (Powell et al., 1993) solves the ideal magnetohydrodynamic equations in the solar wind and magnetosphere regions.The model uses Adaptive Mesh Refinement (AMR) to refine the grid size in regions of large gradients or of special interest.The highest resolution used for the simulations analyzed in this study was set to 1/8 Earth radii (R E ), which is mostly used in the inner magnetosphere and close to the magnetospheric boundaries.The simulation box covers the region around the Earth with the x-axis spanning from 224 R E in the magnetotail to 32 R E in the sunward direction, and the y and z-axes span from 128R E to 128R E in Geocentric Solar Magnetospheric coordinates.The Rice Convection Model (RCM) is an inner-magnetosphere model primarily used to account for the effects of the ring current (Wolf, 1983).RCM solves bounce-averaged drift kinetic equations for the particle populations on a polar grid (Toffoletto et al., 2003).RCM is two-way coupled between BATSRUS and the Ridley Ionosphere Model (RIM) ionosphere.The coupling takes place every 10 s.BATSRUS supplies the electromagnetic fields to the RCM, and RCM nudges the plasma pressure to account for the energetic particle impacts for the BATSRUS inner magnetosphere region.RIM is a potential solver for the ionosphere that solves the Poisson equation on a spherical grid (A.J. Ridley et al., 2004).It is coupled with RCM and the BATSRUS, and uses an empirical conductance model to find the electric field in the ionosphere.The coupling between BATSRUS with RIM happens every 5 s.BATSRUS supplies RIM with fieldaligned currents (FAC), which enables RIM to solve the electric potential using the conductance model.

Simulation Runs
We ran 123 storms using SWMF in the Geospace configuration for a previous study of ground magnetometer observations (Al Shidi et al., 2022;Al Shidi & Pulkkinen, 2022).In this study, we use that data set and its output of geomagnetic indices.The storms were selected based on having a minimum Dst stronger than 50 nT.Each storm was run for 54 hr in total, including 6 hr before the storm onset time and 48 hr thereafter.Thus, the data set comprises a mix of quiet time, storm main and recovery phases.

Simulation Inputs
The storms were simulated with the SWMF Geospace model using the OMNI solar wind and interplanetary magnetic field (IMF) observations (N.Papitashvili et al., 2014) as input for the simulation Sunward (upstream) boundary.The observations come from a collection of spacecraft, most of them located at or near the first Lagrangian point L1.The solar wind observations are then propagated to the BSN to represent the solar wind and IMF parameters encountered by the Earth's magnetosphere.We use the multi-spacecraft solar wind parameter data as input into the model.Also, from OMNI, we use the hourly average 10.7 cm solar radio flux (F 10.7 ) as an input to RIM which uses it for its conductance model (A.J. Ridley et al., 2004).
In this study, we use the IMF components in GSM coordinates (B x , B y , B z ), the solar wind speed (V) and density (n) to get the solar wind dynamic pressure (p), solar wind electric field (E y = VB z ) and an empirical solar windmagnetosphere coupling function (Newell et al., 2007) (2) 1/ 2 is the transverse component of the magnetic field perpendicular to the Sun-Earth line.The normalizing factor α ∼ 10 3 provides conversion to units of Wb/s, which allows interpretation of the parameter as the rate of magnetic flux conversion at the dayside magnetopause (Newell et al., 2007).The dynamic pressure was chosen as a solar wind driving parameter because
of its ability to affect SYM-H and AL (Zhao et al., 2021;O. Troshichev et al., 2022).The Newell coupling function is a good analog that combines the IMF and flow pressure effects of the solar wind.
We use the 1-min cadence OMNI data set which combines spacecraft solar wind observations propagated to the BSN as input to BATSRUS.When there are data gaps in the input, BATSRUS uses the previously given solar wind measurement.OMNI includes ACE and WIND measurements but does not include certain solar wind propagation parameters such as the PFN.Therefore, we use the separate propagated WIND measurements provided by NASA Space Physics Data Facility (SPDF) to collate missing parameters such as the phase front normal with the outputs.

Simulation Outputs
Each storm simulation was configured to output geomagnetic indices including SYM-H, AL, AU, and the northern hemisphere polar cap index PCI, which can be used to derive an estimate for the Cross-Polar Cap Potential (CPCP) (A.J. Ridley & Kihn, 2004) in the form where the time of year is specified as T = 2π(N MONTH /12) with numbering of months starting from zero (Jan = 0).
To examine the errors in the local geomagnetic field prediction, the simulation output includes magnetic field perturbations ΔB at over a hundred ground magnetometer station locations perturbations (the amount of stations depend on data availability at the time) at 1-min cadence (Al Shidi & Pulkkinen, 2022).The simulation output also includes geomagnetic indices at 1-min cadence derived from virtual ground magnetometer perturbations in their respective latitude bands.The ground magnetic field is calculated through a Biot-Savart intergral from the inner boundary of the simulation throughout the simulation box (Gombosi et al., 2021).

Error Analysis
The observations and simulation outputs are compiled to time series vectors y j,observed and y j,model , where y j is the value of the geomagnetic index j ( j = SYM-H, AU, AL, CPCP).The simulation prediction error at time t i is then defined as Using the time series of errors for each index, we then bin conditional probability distributions under different solar wind driving and propagation parameters, k = (E y , dΦ/dt, PFN, the perpendicular distance from the Sun-Earth line | x × r|, Δt), where r = r spacecraft r BSN .Statistical analyses are then performed on these errors, such as calculating the median error and standard deviation.For both median and standard deviation, when a 95% confidence interval is presented, a bootstrapping method was used.The number of resamples chosen was 100 due to the large sample sizes, n, in the data.

Results and Analysis
This study focuses on the magnitude of the errors in the simulation output as compared to the observations for given solar wind input.We note that the solar wind driver parameters used here are not independent, but reflect different characteristics of the incoming solar wind.Specifically, we examine the intensity of the driving using the solar wind electric field and the Newell et al. (2007) coupling function (E y , dΦ/dt), which both depend on solar wind velocity and magnetic field.Likewise, parameters associated with solar wind front orientation and its propagation to the bow shock nose (see Figure 1, PFN n ⋅ x, | x × r|, and timeshift Δt) are functions of magnetic field orientation and solar wind velocity.
Table 1 gives a quantitative summary of the median errors (MdE) and their 95% confidence intervals between the simulated and observed magnetospheric response parameters CPCP, SYM-H, AL, and AU.The distributions of the errors are approximately Gaussian with skewness (g 1 ) being 0.5 < g 1 < 0.5 for all but the AL index.The distributions of errors in AL is right-skewed (g 1 = 1.66), which is consistent with previous studies showing that the simulation has a tendency to underpredict the observations (model value less negative than observed) (Haiducek et al., 2017;T. I. Pulkkinen et al., 2022).A skewness test for each distribution results in a p-value of zero (p skew = 0), which confirms they are skewed from the normal distribution.

Univariate Analysis
In this section, we examine the probability distributions of the errors for each of the driver parameters.Figures 2, 5, and 6 show the error probability distributions as a functions of the driver parameters for each of the activity measures y j .The error distributions are computed for 10 ranges of the driver parameter values, and the figures show the errors normalized for each driver value (vertical bin).The error distributions were discretized into 50 bins.Normalization of each driver parameter bin means that the figures do not give the absolute value of the error, but rather a distribution of the error magnitudes within that particular range of the driver parameter.
For more quantitative assessment of the SYM-H and AL indices, which are routinely used for evaluation of model performance (M.W. Liemohn et al., 2018), we tabulate the MdE and standard deviation of errors (σ E ).The conditional bounds and limits to calculate the errors were chosen to encompass 95% of all the samples in the data set.Section 3.1.1discusses the errors related to the global geomagnetic indices, while Section 3.1.2assesses the capability of the model to predict localized errors by examining magnetic perturbations at two individual ground magnetometer stations, one at auroral and another at sub-auroral latitude.We focus mainly on the global geomagnetic indices errors.

Global Geomagnetic Indices
The four panels in Figure 2    Taking a closer look at the numbers, Table A1 shows the MdE and σ E with their 95% CI.Figures 3 and 4 provide a visual representation of the aforementioned Table .The number of samples in each bin of E y values, n, is also included (note that the bin ranges are asymmetric due to the choice of including 95% of all data in the analysis).For the SYM-H index, the median error is closest to zero for high solar wind driving, while the error increases toward lower driving and has a strongly negative bias for the strongly negative electric field (strongly positive IMF B z and/ or high solar wind speed).This means that under those conditions, the model overpredicts SYM-H, which may be an indication of the model's ability to reproduce the magnetospheric configuration under a closed magnetosphere conditions.On the other hand, the SYM-H error standard deviation increases with larger solar wind driving.
The median error MdE of the AL index increases with increasing E y , as does its standard deviation σ E : During the strongest solar wind driving (E y > 6 mV/m), the MdE is 351.91 nT with the standard deviation σ E being to 359.34 nT.This is a significant difference from the overall AL error distribution where the MdE = 38.49nT (see Table 1).A 2-sample Kolmogorov-Smirnov test (KS test) (Hodges, 1958), which will test how likely the overall distribution of errors ( ∞ < E y < ∞) and the error distribution with the strongest solar wind driving (E y > 6 mV/ m) confirms these two distributions are from different distributions (p ks = 1.1 × 10 319 ).The sample sizes are 173,269 and 5,592, respectively, and the KS statistic is 0.44.The small p-value is a reflection on how the subsampling (E y > 6 mV/m) of the parent distribution generates a distinct distribution.The error distributions in the geomagnetic indices as function of the perpendicular distance of the spacecraft to the Sun-earth line (| x × r|) show mostly consistent standard deviation throughout.However, there is a noticeable change in the medians (not shown) for each given distance whose origin remains unclear.For example, calculating Cohen's D for a SYM-H errors between the two ranges of (0, 20) Re and (80, 100) Re gives a value of 0.48 which shows a medium-sized effect of half a standard deviation between the two.Lastly, we examine the effect of the time shift Δt from the spacecraft location to the bow shock nose.Noting that most of the OMNI observations come from spacecraft at or near L1, higher values of Δt imply slower solar wind and the lower values imply high solar wind speed.Alternatively (referring to Figure 1), higher values of Δt imply close to perpendicular PFN angles and smaller propagation times imply more oblique PFN orientation.The error distributions for all parameters are slightly larger for low time shifts (high solar wind speed, oblique fronts).The polar cap potential and the AU index show very little variation across the range of values in Δt.
Table A2, Figure 3, and Figure 4 show a summary of the corresponding numerical values.The bounds were chosen to correspond to Figure 5, but including fewer bins to maintain readability.Table A2 shows the number of samples and the bin sizes, the median error MdE and standard deviation σ E with their 95% CI conditional to phase front normal (PFN n ⋅ x) and the perpendicular distance from the spacecraft location to the Sun-Earth line (| x × r|) .The SYM-H MdE and its 95% CI decreases when the PFN is closer to the Sun-Earth line, from 9.52 nT in its most oblique orientation to 1.05 nT in cases where the front is close to perpendicular to the Sun-Earth line.The uncertainty in the median at the most oblique angles could be caused by lack of samples (n = 192) (see Table A2), but the trend can still be seen with smaller CI widths of around ∼0.4 nT in the next bin 0.2 <| n ⋅ x|< 0.4, which has a much larger sample size (n = 17,664).However, the standard deviation σ E increases when the PFN projects closer to the Sun-Earth line from 13.78 to 18.19 nT.Thus, both the errors and standard deviation show a conditional dependence on the PFN, which may be related to the fact that the oblique fronts are almost in the direction of the solar wind velocity.
The next section of Table A2 shows the errors in AL conditional to the PFN | n ⋅ x|.The dependence of the errors seems relatively weak.The overall distribution MdE and its 95% CI from Table 1 is 38 .49 [37.59, 39.15] nT.The MdE conditional to PFN are relatively close to that with the closest distribution being 37. 86 [36.16, 38.81] nT, which has some overlap in their CI.Again, the uncertainty in the median (the MdE CI width) is larger than the rest when n ⋅ x < 0.2 (55.18 to ∼3 nT).This is likely due to the smaller sample size, the large uncertainty when the PFN angle is so oblique to the Sun-Earth line, or a combination of both.
We then discuss the effects of the perpendicular distance of the spacecraft to the BSN.The third section of Table A2 shows the SYM-H errors conditional to the perpendicular distance.A conditional dependence is especially noticeable when the distance is <20R E .The MdE in the <20R E case suggests that SWMF is in these The σ E stays relatively constant (∼17 nT) and becomes larger in the case of the closer distance (<20R E ), to 20.52 [20.27, 20.82] nT.
The MdE of the AL index shows a similar trend to that of SYM-H, as shown in the right most sub-figures of Figure 3 and bottom section of Table A2.In the <20R E case, the MdE becomes smaller (from ∼40 nT to 14.86 [12.14, 16.59] nT).However, unlike the SYM-H, the standard deviation also becomes smaller, reducing from ∼210 nT to 187.76 [182.43, 198.50] nT.In both SYM-H and AL cases, however, we note that the smallest distance bin has a factor of 3-5 smaller sample size.
The table and figure summaries confirm a conditional dependence between propagation parameters (PFN and perpendicular distance) and MdE and/or σ E .We note here that the errors analyzed are errors in magnitude between the observed and simulated values, not errors related to timing.Perhaps the most noteworthy result is for the smallest spacecraft perpendicular distance from the Sun-Earth line (<20R E ).In that case, both SYM-H and AL errors show a marked change in behavior.The conclusion from Figure 5 suggesting that stronger solar wind driving leads to a larger spread in errors (larger σ E ) is quantitatively confirmed by the standard deviations and their confidence intervals given in Table A2.

Local Magnetometer Recordings
The analysis in the previous section focused on global geomagnetic activity as described by the geomagnetic indices.In order to study the localized effects, in this subsection, we examine the role of each of the driver parameters in generating errors at individual magnetometer stations.We select two representative stations, Yellowknife, Canada (YKC) in the auroral latitudes recording both eastward and westward ionospheric electrojets (Lyatsky et al., 2006), and Boulder, CO, USA (BOU), at the mid-latitude range that mostly reacts to the ring current and field-aligned currents coupling the magnetosphere and ionosphere (Dubyagin et al., 2014).The errors computed use the observed and simulated ground magnetic perturbations horizontal (magnitude of northward and eastward components) to the Earth's surface.
The four panels in each row of Figure 6 shows the error probability distributions for each of the driver parameters (E y , dΦ/dt, n ⋅ x, and | x × r|) for YKC at the top and BOU at the bottom panel.Note that due to the very different scales of the signals (the auroral zone magnetometers record variations in the 100s of nT, whereas the mid-latitude stations typically record variations of the order of 10s of nT), the horizontal scales are different for the two stations.The local individual recordings show similar dependence on the driver parameters to the aggregated global indices, although the error distributions in the perpendicular component are narrower than those of the global indices (see also Table A3).Note, however, that for the solar wind driver functions the horizontal components are centered around zero error.This result is consistent with the detailed analysis of individual stations by Al Shidi et al. (2022), who concluded that the errors for the north and east components are typically larger than the total horizontal component.As the geomagnetic indices are based on the north component records, any errors in that will translate to errors in the indices.Furthermore, the stations show an increase in standard deviation for strong driving in the closed magnetosphere configuration (strongly positive E y ).Neither station show a dependence of the standard deviations on either the phase front normal or the perpendicular distance to the Sun-Earth line.
The results of this section were meant to illustrate the more localized effects of solar wind propagation and driving.The primary motivation of this study is the geomagnetic indices errors that are a summary of individual stations like these.The YKC station is a station that can be used to find AL or AU.The BOU station is a station that can be used to find Dst or SYM-H.We show that solar wind driving has a dependence on the standard deviations of the B H errors.

Regression Analysis
In order to study the relationship between the solar wind parameters and the simulation errors, we perform a linear regression analysis to examine how the errors change with the varying inputs.We determine the regression coefficients, b k , where k is the off-axis distance of the spacecraft to the BSN (| x × r|), the PFN projection on the Sun-Earth line (| n ⋅ x|) , solar wind driving electric field analogs VB y , and VB z and lastly the solar wind dynamic pressure (p).We then compute the Standardized Regression Coefficients (SRC jk ) to determine the magnitude of the effect of a solar wind quantity k on the errors for each geomagnetic index j.The SRC is given by the equation where σ is the standard deviation of the solar wind parameter k or activity index error j.The SRC is a measure of how much of the standard deviation in the errors can be attributed to the standard deviation in the inputs.This allows us to make a relational inference between the solar wind inputs and activity index errors, and to directly compare across the parameters as the SRCs are unitless.Table 2 shows the SRCs found for each solar wind input and geomagnetic index error and the SRC confidence intervals.
The SRCs show that the standard deviation in the errors of SYM-H relates strongest to the standard deviation in pressure with an SRC of 0.293.It also suggests relationships with the PFN and VB z with SRC's between ∼0.1-0.2.Interestingly, the SRC of the spacecraft off-axis distance, | x × r|, is strongest with SYM-H when compared with the other indices.The standard deviation of errors in AL have strongest dependence on the standard deviations in solar wind driving VB y and VB z with SRC ∼0.3.We note that the relation of AL and AU to the propagation parameters | x × r| and n ⋅ x are relatively weak (SRC ∼ 0.01-0.02).
Through the SRC analysis, we can conclude that the standard deviations in solar wind propagation parameter | x × r| and n ⋅ x does have a connection to the SYM-H, and n ⋅ x to the CPCP as they both have an SRC of around 0.1.This is in contrast to other indices such as AL and AU, whose SRCs are an order of magnitude smaller (∼0.01).We can also conclude that the electric field components VB y and VB z shows a strong relationship with AL (SRC 0.3).This is to be expected, as it is widely accepted that there is a correlation between solar wind electric field and AL (e.g., O. Troshichev et al., 2022).

Discussion
This study focuses on the analysis of the magnitude and probability distribution of errors between observations of geomagnetic indices and their predictions using the Space Weather Modeling Framework Geospace simulation.
The errors can arise either from inaccuracies in the measurement (measurement errors), errors in the predicted magnitude with the correct driver profile (model error), or errors in timing of the arrival of the solar wind and IMF at the bow shock nose.The data set is over-represented by storm-time data, however, this is a conscious choice to
evaluate the performance of SWMF during storm time specifically and not dominate the analysis with quiet time conditions.For a data set that includes a month long of data which includes quiet time intervals, please see M. Liemohn et al. (2018).
Our results indicate that the median errors show a dependence on the geometric parameters such as the phase front normal angle or the perpendicular distance from the Sun-Earth line of the observing spacecraft, while the widest distributions of the errors are obtained for high solar wind driving and high dynamic pressure.An earlier study (A.J. Ridley, 2000) demonstrated that the uncertainty in timing between measurements at L1 with WIND and closer to the BSN with the spacecraft IMP 8 can be on average 7.5-8.5 min.However, their study was limited to the solar wind, and did not assess the impact of these timing errors on the errors in geomagnetic indices, which have their own intrinsic time scales as they respond to the variable solar wind driving.More recently, Milan et al. (2022) found that the correlation of peaks in the cusp field-aligned currents observed by the AMPERE satellites with IMF B y deteriorates with larger perpendicular distance of the solar wind monitor from the Sun-Earth line.Furthermore, they find an average time lag of about 17 min between the solar wind front arrival at the bow shock nose and the response caused by the front in the ionosphere.The simulation is expected to also respond with the same time lag and follow the same processes which makes calculating errors on the same minute fair.BATS-R-US has been shown in the past its ability to recreate the global circulation pattern (Gordeev et al., 2011).
The nonlinear relationship between the solar wind inputs and the geomagnetic index errors poses limitations to the regression analysis performed in Section 3.2.The value of the regression analysis is in its ability to address the relationship between the standard deviations of both input and output simultaneously, characterized by the standardized regression coefficients.The fact that the SRC of the spacecraft perpendicular distance in relation to the errors is relatively small could be due to the lack of standard deviation in the spacecraft orbit.The spacecraft flies a steady path and that path's standard deviation is truly independent to the standard deviation in the errors.
We note that this study does not address errors in propagation time.We believe it is important for solar wind monitor measurements to publish error bars as that could strengthen studies like this and provide better insight as to the origins of certain errors, specifically measurement uncertainty.Inputs such as F 10.7 would provide better detail if we are provided error bars.Further investigations into the cause of the errors would require better metadata for the observational inputs and outputs: Neither the geomagnetic indices nor the space-borne solar wind measurements and propagation times come with uncertainty estimates or error bars.Such parameters would be necessary to understand how the uncertainties propagate from the input to the output measurements, and would be valuable for uncertainty quantification and validation of space weather forecasts.Lacking these, Morley, Welling, and Woodroffe (2018) attempted to quantify the uncertainty of measurement errors in the solar wind input by ensemble modeling with SWMF, and showed that using ensemble modeling can improve the mean error of SYM-H by a few nT.

Conclusion
We present the conditional dependence of the SWMF Geospace simulation errors in geomagnetic indices (as compared to observed ones) of the solar wind driver and propagation (from L1 to bow shock nose) parameters.
Based on a large simulation data set, we draw the following conclusions of SWMF behavior during geomagnetic storm times:

Space Weather
10.1029/2023SW003747 1.The standard deviation of the errors is generally dependent on the intensity of the driving, such that stronger driving (larger E y ) produces larger standard deviations in SYM-H and AL errors.This implies that the errors are known to lesser accuracy during strong geomagnetic activity.2. We show that there is a dependence of the median error (MdE) on the phase front normal angle and spacecraft perpendicular distance from the Sun-Earth line.The MdE for both SYM-H and AL decreases when the perpendicular distance is < 20R E .An interesting detail is that the SYM-H that typically is underpredicted by the model (observed values show higher activity than the simulation), shows the opposite behavior when the perpendicular distance is < 20R E .3. The standard deviations of the errors are more dependent on the intensity of the solar wind driving, E y , than on the solar wind propagation parameters, such as PFN and spacecraft perpendicular distance to the Sun-Earth line.4. Using standardized regression coefficients, we show the dependence of the error standard deviations on the inputs.We confirm that there is a relationship between the standard deviations in the phase front normal (PFN) with the standard deviations in SYM-H, and an even stronger relationship between the flow pressure and SYM-H standard deviations.5. Perhaps not surprisingly, the standard deviations in the electric field components VB y and VB z show the strongest relationship with standard deviations in AL.The standardized regression coefficients provide a (unitless) measure of the degree to which the solar wind parameters impact the errors.
For future studies, we propose a multivariate approach: The phase front normal (PFN) and solar wind monitor locations are not direct inputs into the simulation, and are not necessarily orthogonal to the real inputs.It would be interesting to see the dependence of the errors with PFN during stronger solar wind driving.Furthermore, a more 3-dimensional approach instead of only investigating projections to the Sun-Earth line might provide valuable insights of the spatial structuring of the solar wind and its impacts on solar wind-magnetosphere coupling.We believe the different components, elliptical and zenithal, of the distances can contribute to errors in the geomagnetic indices.

Appendix A: Tables of Median Errors and Standard Deviation of Errors
The numerical statistics of the errors presented in the previous figures can be seen below.See Table A1, Table A2, and Table A3 Note.The columns show the number of datapoints in each bin, the bin boundaries, the median error, confidence interval of the median error, the standard deviation σ E , and confidence interval of σ E .

Figure 1 .
Figure 1.A diagram (not to scale) showing a simplified version of how solar wind parameters are propagated from L1.The Earth is shown in blue at the origin of the GSE coordinate system, with the bow shock engulfing the magnetosphere oriented by the direction of the solar wind flow velocity V (blue arrow).The spacecraft situated upstream (black cross) observes a phase front plane (dashed red line; phase front normal n is shown with the red arrow) The position r from the bow shock to the spacecraft has GSE components (Δx) and (Δy).The distance of the front plane from the bow shock along the solar wind flow direction is marked with a red x.The propagation time from the front plane to the bow shock is then determined by the distance of x to the bow shock and the solar wind speed.
show the errors in the activity indices (from left to right) SYM-H, CPCP, AL, and AU as a function of the intensity of the solar wind electric field E y (top row) and the Newell et al. (2007) coupling parameter (bottom row).The errors are mostly centered around zero.The error distributions widen in the polar cap potential slightly both for strong driving (strongly positive E y ) and for strong electric field in a closed magnetosphere

Figure 2 .
Figure 2. From left to right: Binned probability distribution of errors in SYM-H, CPCP, AL and AU as a function of the (top) solar wind electric field and (bottom) the Newell coupling function at the bow shock nose.The driver parameters were binned into 10 bins, and the errors into 50 bins.Each driver bin is normalized to highlight the distribution of the errors rather than their magnitude.

Figure 3 .
Figure 3. From top to bottom, left to right, the blue line shows median errors calculated in bins for SYM-H and AL, with respect to E y , PFN n ⋅ x, and spacecraft perpendicular distance.The orange bars show the 95% CI of the calculated medians for each bin.

Figure 5 ,
Figure 5, similarly to Figure 2, shows probability distributions of errors in (left to right) SYM-H, CPCP, AL, and AU in relation to (top to bottom) phase front normal (PFN, n ⋅ x), spacecraft perpendicular distance (| x × r|) , and timeshift (Δt).Examination of the error distributions of the global indices as a function of PFN shows that there is less dependence of the error distributions on the phase front orientation than on the solar wind driver function magnitude: The standard deviations and the medians are relatively similar for angles perpendicular and oblique to the sun-earth line (smaller and larger n ⋅ x, respectively).

Figure 4 .
Figure 4. From top to bottom, left to right, the blue line shows standard deviation of errors calculated in bins for SYM-H and AL, with respect to E y , PFN n ⋅ x, and spacecraft perpendicular distance.The orange bars show the 95% CI of the calculated standard deviation for each bin.

Figure 5 .
Figure 5. From left to right: Binned conditional probability distribution of errors in SYM-H, CPCP, AL and AU as a function of the (top) the PFN projection to the x axis ( x ⋅ n) (middle) the perpendicular distance from the Sun-Earth line (| x × r|) , and (bottom) time shift to the bow shock nose Δt.The driver parameters were binned into 10 bins, and the errors into 50 bins.Each driver bin is normalized to highlight the distribution of the errors rather than their magnitude.

Figure 6 .
Figure 6.From left to right: Binned probability distribution of errors the horizontal magnetic field at (top) YKC and (bottom) BOU as a function of the solar wind E y , the Newell coupling function dΦ/dt, the PFN projection to the x axis (|x ⋅ n|), and the perpendicular distance from the Sun-Earth line (|x × r|).The driver parameters were binned into 10 bins, and the errors into 50 bins.Each driver bin is normalized to highlight the distribution of the errors rather than their magnitude.

Table 1 A
Quantitative Summary of the Overall Error Distributions for the Geomagnetic Indices From left to right the columns give the geomagnetic parameter, the median error (MdE), the 95% confidence interval of the MdE, and the skewness respectively.

Table 2
Table of the SRC and Their 95% Confidence Intervals in Brackets Note.From top to bottom the rows are the geomagnetic index errors of SYM-H, CPCP, AL and AU, respectively.The columns from left to right are the off-axis distance of the spacecraft (| x × r|) , the PFN projected on the Sun-Earth line ( n ⋅ x) , solar wind drivers VB y , VB z , and flow pressure p, respectively.

Table A2
Statistics of the SYM-H and AL Errors Conditional to | n ⋅ x| and the Perpendicular Distance of the Spacecraft From the Sun-Earth Line| x × r| in [R E ] n (SYM-H, n ⋅ x)