Comparison of Low‐Latitude Ionospheric Scintillation Forecasting Techniques Using a Physics‐Based Model

The temporal change in height of a specific electron density can be used as a proxy for vertical plasma drift (PVPD) at the magnetic equator. The use of PVPDs as a predictor of low‐latitude ionospheric scintillation during the subsequent evening has previously been shown to have forecasting skill when using ionosonde data. The implementation of this approach using a physics‐based model is proposed to provide greater forecast antecedence without the need for local ionosondes. For the first time, the physics‐based model PVPD method is compared to another forecasting approach that uses a physics‐based model to calculate Rayleigh‐Taylor growth rates (RTGRs). In equinoctial test cases considered, when appropriate scintillation observation thresholds are selected, PVPD forecasting is shown to have skill similar to or better than the RTGR method using the same physics‐based model. PVPD forecasting requires only electron densities and corresponding altitudes. Therefore, this approach could be applied using an ionospheric data assimilation model whereas the majority of these models do not provide output for all variables required for RTGR forecasting. The forecasting skill in these test cases, the simplicity of physics‐based PVPD forecasting, and the suitability of this method for use of ionospheric data assimilation model output make this method attractive as a forecasting tool in an operational setting if skill can be further demonstrated for a wide range of conditions. However, both PVPD and RTGR forecasting skill are shown to be limited during solstitial months with high scintillation activity. This may be improved by using a data assimilation model.

high conductivity (Farley, 1960). During the daytime, the dynamo electric fields (E) are eastward, which causes an upward E  B plasma drift (where B is the magnetic field), whilst the reverse occurs at night (Martyn, 1953;Woodman, 1970). As the ionosphere co-rotates with the Earth toward dusk, the eastward component of the F region neutral wind increases (King-Hele & Walker, 1977) as the wind blows across the terminator from day to night. The increased eastward wind dynamo component, in combination with the sharp day-night conductivity gradient across the terminator, leads to an enhancement of the vertical E  B plasma drift (Rishbeth, 1971) known as the pre-reversal enhancement (PRE). The F layer, therefore, continues to rise as the ionosphere co-rotates into darkness. A detailed description of these processes is provided by Heelis (2004).
At the same time (in the absence of sunlight), the lower ionosphere rapidly decays (through ion-electron recombination), and a steep vertical electron density gradient develops on the bottomside of the raised F layer. This produces the necessary configuration for the Rayleigh-Taylor instability in which a heavy fluid is situated above a light fluid. Under such circumstances, small perturbations in the bottom-side of the F region can grow into large "equatorial plasma bubbles" (EPBs) of depleted plasma (Ott, 1978;Sultan, 1996). These bubbles, and associated irregularities, can form plume-like structures as they rise through the denser regions above their initial location, extending to altitudes of more than 1,000 km (Woodman & La Hoz, 1976).
The occurrence of EPBs has been shown to be correlated with the magnitude of the PRE vertical plasma drift (Basu et al., 1996;Fejer et al., 1999;Kil et al., 2009). EPB occurrence has also been associated with the presence of atmospheric waves such as gravity waves (Abdu et al., 2009;Tsunoda, 2010) that can produce the perturbations required to initiate the Rayleigh-Taylor instability as they propagate upwards from lower altitudes. Anderson et al. (2004) suggested that the "seeding" mechanism for EPB generation can be considered to be present at all times, so the necessary and sufficient condition for EPB formation is a large PRE vertical plasma drift. However, later work demonstrated that while the probability of EPB generation increases with increasing PRE vertical plasma drift, large vertical plasma drifts do not guarantee the presence of EPBs (Huang & Hairston, 2015;Kil et al., 2009), possibly due to a lack of the perturbations required to initiate EPB generation.
In order to mitigate the effects of low-latitude scintillation several approaches have been developed to forecast the occurrence of plasma irregularities. Ionospheric observations have been used for regional scintillation forecasts by applying methods such as machine learning and data mining (e.g., Rezende et al., 2010) or by considering the temporal change of the observables and then propagating them forward in time using forecasts of irregularity drift (e.g., Groves et al., 1997). Observations used to forecast EPB generation/scintillation include S4 (Groves et al., 1997), total electron content (Sridharan et al., 2012;Sunda et al., 2017;Tanna et al., 2013), the virtual height associated with a specified plasma frequency (Anderson et al., 2004;Sousasantos et al., 2017), the minimum F layer virtual height (h'F;de Lima et al., 2014;Redmon et al., 2010;Rezende et al., 2010;Sridhar et al., 2017), the F2 layer o-mode critical frequency (foF2; Bagiya et al., 2014), ultraviolet spectrographic imagery (Kelly et al., 2014), and coherent scatter radar and planar Langmuir probe observations of electron densities (Costa et al., 2011). A drawback of these approaches is the absence of significant antecedence because observations for a specified region generally need to be made around the time of local sunset. An alternative approach that allows greater antecedence is the use of an ionospheric model. A model also makes forecasting possible for regions with limited ionospheric observations such as over oceans.
Empirical ionospheric models can capture the global climatology of EPB formation, structure, longevity, and decay, including the resulting scintillation effects. There is, however, considerable day-to-day variability around this climatology. Retterer (2010) noted that their ionospheric plume model would not accurately predict ionospheric variability if it was driven only by empirical models. This is because, unless the empirical models were adequately parameterized, the output would represent averages or binnings that would suppress the day-to-day variability. Several empirical models have been shown to include inaccuracies when nowcasting or forecasting scintillation due to the description of conditions in terms of statistical parameters (Forte & Radicella, 2005;Hamel et al., 2014;Priyadarshi, 2015). A physics-based model may, therefore, be more successful at representing day-to-day variability when forecasting scintillation due to EPBs.
A scintillation forecasting technique using a physics-based model to calculate integrated field line Rayleigh-Taylor growth rates (RTGRs) is described in Section 2. A second scintillation forecasting technique using ionosonde data to calculate a proxy for vertical plasma drift at the magnetic equator is described in Section 3. For the first time, the ionosonde technique is adapted to use output from a physics-based model, providing a simple and computationally cheap method for real-time scintillation forecasting which may prove attractive for operational use. In Section 4, methods for comparing forecasting skill are discussed, and in Section 5, the forecasting skill of the RTGR and the proxy vertical plasma drift (PVPD) forecasting methods are compared using output from the same physics-based model. Section 6 provides a summary of this work. Sultan (1996) developed an expression to describe the ionospheric RTGR:

Scintillation Forecasting Using Rayleigh-Taylor Growth Rates
where  E P and  F P are the field line integrated E and F region Pedersen conductivities, P V is the vertical (upwards) plasma drift speed at the magnetic equator, P L U is the integrated Pedersen conductivity-weighted neutral wind in the geomagnetic meridional and vertical plane, e g is the acceleration due to gravity for a given altitude,  eff represents the F region ion-neutral collision frequency weighted by the electron density along the field line, F K is the F region field line electron content height gradient, and T R is the recombination rate integrated along the field line.  used output from the physics-based Thermosphere-Ionosphere-Electrodynamics General Circulation Model (TIE-GCM: Richmond et al., 1992) to calculate RTGRs for field lines passing over Vanimo, Papua New Guinea (141.3°E geographic longitude, −11.4°N magnetic latitude) during March and April 2000 and for multiple locations in March and April 2011 (Carter, Rettere, Yizengaw, Groves, et al., 2014). The daily maximum RTGR was used as a predictor for whether strong scintillation would be observed at that location. Carter, Rettere, Yizengaw, Groves, et al., 2014 set the recombination rate ( T R ) equal to zero, with F K : Here, h is the height, and e N is the integrated field line electron content. RTGR field line integration was conducted between an altitude of 125 km in one hemisphere to 125 km altitude in the other hemisphere, for field lines with peak altitudes of up to 600 km.
The dominant factor when calculating the daily maximum TIE-GCM RTGR is increased vertical plasma drift speed at the magnetic equator due to the PRE Rajesh et al., 2017). Increases in geomagnetic activity limit the magnitude of TIE-GCM's enhancement of plasma drift speed thereby reducing TIE-GCM RTGR values . Days on which the maximum TIE-GCM RTGR value was low were found to correspond to days on which strong scintillation was not observed.  selected thresholds for the daily maximum RTGR and daily maximum hourly average of amplitude scintillation (S4) observations. The maximum S4 value for a chosen day was predicted to be above the S4 threshold only if the maximum RTGR value for that day was above the RTGR threshold. RTGR forecasting correctly predicted whether maximum S4 values would be above or below the S4 threshold on 48 of the 56 days considered. The Carter et al. approach demonstrated forecasting skill during seasons in which EPB generation was common. However, during seasons in which EPB generation was less frequent, RTGR forecasting performed worse than persistence forecasting at some locations (Carter, Rettere, Yizangaw, Wiens, et al., 2014).
Forecasting skill with TIE-GCM can be improved by using data assimilation (DA). This approach provides a more accurate representation of the ionospheric conditions so should produce more accurate forecasts. DA has been shown to improve the accuracy of the RTGR forecasting technique (Rajesh et al., 2017); however, results have only been published for a two day test scenario, so it is not yet well validated. A drawback of incorporating DA is the increased computational expense of the background model.

Scintillation Forecasting Using a Proxy for Vertical Plasma Drift
Another approach for forecasting scintillation is the use of PVPDs. Anderson et al. (2004) calculated PVPDs at the magnetic equator by determining the virtual height associated with a plasma frequency of 4 MHz (  11 2 10 e − /m 3 ) and investigating the rate of change of this height. PVPD speeds greater than 20 ms −1 between 1830 and 2000 local time (LT) were found to correspond to strong scintillation observations (five minute average S4 >0.5) during the subsequent evening (Anderson et al., 2004). Maximum PVPD speeds less than 20 ms −1 corresponded to days in which strong scintillation was not observed. This approach successfully predicted whether strong scintillation would occur on 84% of 300 consecutive days in 2001 and 2002. Two key limitations to this approach were that regional forecasts could not be produced until the observation window had commenced and the need for a local ionosonde.
Limited antecedence and the need for an ionosonde would be avoided if PVPD calculations used model output rather than ionosonde data. Since the change in virtual height of a 4 MHz ionosonde return signal can be represented by the change in altitude of electron density  11 2 10 e − /m 3 , PVPD calculations using a model require only electron densities, their corresponding altitudes, and the interval between simulated observations. Therefore, it is straightforward to use output from a 3D ionospheric model (e.g., International Reference Ionosphere (IRI: Rawer et al., 1978), NeQuick (Nava et al., 2008), TIE-GCM) to generate scintillation forecasts using PVPDs. In this study, the height of the   11 3 2 10 e /m electron density is determined using linear interpolation between vertically adjacent grid points. The maximum PVPD is determined by the largest change in height of the   11 3 2 10 e /m electron density between each consecutive 15 min model time step from 1830 to 2000 LT. If the maximum PVPD speed at a location on the magnetic equator is greater than a specified threshold, a forecast for strong scintillation conditions in the subsequent evening is predicted for that longitude. PVPD forecasting requires model output from one altitude profile per time step since only points at the magnetic equator are necessary for a forecast at that specific longitude. RTGR forecasting, however, needs a greater number of profiles per time step as model conditions are required at several locations along the magnetic field lines. PVPD values are calculated for 1830-2000 LT whereas RTGR forecasting calculates maximum daily RTGR values, so a larger number of time steps are required. The complexity of the individual calculations required for PVPD and RTGR forecasting are similar; however, RTGR forecasting requires a greater number of calculations and TIE-GCM output corresponding to a greater number of variables, time steps, and grid points per time step. Once TIE-GCM output has been produced, the calculation of PVPDs is, therefore, less complex and less computationally expensive than calculating RTGRs. However, the computational expense of the PVPD and RTGR calculations is significantly less than the expense of running a physics-based ionosphere-thermosphere model such as TIE-GCM or a data assimilation (DA) model. PVPD forecasting only requires electron densities and associated altitude as inputs whereas RTGR forecasting also requires Pedersen conductivity, neutral wind speed, and ion-neutral collision frequencies. Therefore, PVPD forecasting could be performed using output from an ionospheric DA model. However, the majority of currently existing DA models do not provide output for all variables required for RTGR forecasting.

S4 Calculation
Various methods can be used to determine scintillation strength for a given evening from a set of amplitude scintillation (S4) observations.  calculated the mean GPS S4 (with 5° minimum elevation) from the Vanimo Ionospheric Scintillation Monitor for each hour and used the maximum value to represent scintillation strength for that day. In later work , the 90th percentile of one minute S4 observations (30° minimum elevation) was calculated for each hour. The largest of these values (S4 90 ) represented scintillation strength for that evening. Maximum hourly average S4 values will be considered when comparing forecasting skill for Vanimo in March and April 2000 . S4 90 will be considered for Vanimo test cases and test cases used in later work by .

Performance Testing
RTGR forecasting has demonstrated forecasting skill including successfully predicting whether strong scintillation would occur at Vanimo, Papua New Guinea for 48 of the 56 days considered during March and April 2000 ). However, this success was generated by retrospectively choosing thresholds to provide the optimal results, which would not be possible in an operational setting. A useful method for evaluating forecasting skill is the receiver operating characteristic (ROC) curve. A ROC curve keeps the observation threshold (S4) fixed while stepping through possible values for the predictor threshold (PVPD or RTGR). The true positive rate (TPR) and false positive rate (FPR) are calculated for each of the values for the predictor threshold. The TPR is the proportion of positive events (i.e., strong scintillation days) which were correctly predicted. The FPR is the proportion of negative events (i.e., weak scintillation days) which were incorrectly predicted to be a positive event. A ROC curve plots TPRs against FPRs (Figure 1a). The line y = x indicates a forecasting method with no skill. A perfect model is represented by the line from (0,0) to (0,1) to (1,1). The area under the ROC curve (AUC) is equal to the probability that a model will rank a randomly chosen positive event higher than a randomly chosen negative event (i.e., the probability that a randomly chosen strong scintillation day will have a higher PVPD or RTGR than a randomly chosen weak scintillation day). Simply, if the predictor threshold is unknown, models with a larger AUC have greater forecasting skill, and a model with no skill has AUC = 0.5 whereas a model with perfect skill has AUC = 1. Since the S4 threshold is fixed for an individual ROC curve, sets of ROC curves and AUCs can be calculated for a range of S4 threshold values (e.g., Figure 2a).
A further metric to determine forecasting skill using ROC curves is Youden's index (Youden, 1950), also known as the Peirce skill score. The maximum Youden's index (YI) value indicates the point on the ROC curve for which the model forecast skill improvement over random chance is greatest. YI is given by NUGENT ET AL.
which can be observed on a ROC plot as the vertical distance of a specific point on the ROC curve from the line representing no skill (the maximum YI is displayed for each forecasting method in Figure 1a). A maximum YI of 0 represents a method with no forecasting skill (as the skill is no better than chance), and a maximum  YI 1 represents a perfect model. The maximum YI can be calculated for a range of S4 threshold values (e.g., Figure 1b) by inspection of the set of ROC curves generated using the same set of S4 thresholds.

Statistical Significance of Results
The methods described in Section 4.2 provide an assessment of the relative forecasting performance of the different approaches. It is also important to consider the absolute forecasting skill; that is, whilst one approach may outperform the other, this is not useful if both approaches have little skill. The maximum YI is calculated from a wide range of PVPD or RTGR thresholds: if the S4 threshold is set so low or high that most observations fall into one class, any PVPD or RTGR threshold which is also particularly low or high may suggest higher forecasting skill than is actually present. The AUC may also be affected by low or high S4 thresholds. To address this, 100 randomly generated sequences of numbers were used as randomly generated forecasts (RGFs). If the PVPD and/or RTGR methods outperform at least 95 of these RGFs, there is effectively 95% confidence that the method is more successful than random chance. If more than five RGFs are capable of matching the performance of all considered forecasting methods for a chosen S4 threshold, then the results for this threshold are not included in the analysis. For consistency, the same 100 RGFs are used for each data set.
Statistical significance of results can be further impacted by small data sets. Each test case corresponds to a 2-month period and includes between 56 and 61 data points. Leave-one-out jackknifing (Quenouille, 1949) has been used to test whether small changes to a data set could have a large impact on results. One data point (one day) is removed from the data set, and the model performance is calculated. This is repeatedly performed with each of the data points excluded in turn. The mean and standard deviation of these leaveone-out performance values are calculated. The mean and two standard deviations shown in Section 5 effectively provide a 95% confidence interval of forecasting skill for a specific S4 threshold when the data are subject to small variations.
NUGENT ET AL.    showed that, using hourly average S4 values at Vanimo for 56 days in March and April 2000, RTGR forecasting could successfully predict that strong scintillation would occur on 17 days, successfully predict that strong scintillation would not occur on 31 days, predict strong scintillation which did not occur on 3 days, and predict that strong scintillation would not occur on 5 days for which strong scintillation was observed, i.e., whether strong scintillation would or would not occur was correctly predicted on 85.7% of days in the test case. This was achieved by predicting S4 would be greater than 0.244 when the daily maximum RTGR was greater than    3 1 0.625 10 s . Using the same S4 threshold, PVPD forecasting successfully predicted that strong scintillation would occur on 17 days, successfully predicted that strong scintillation would not occur on 29 days, predicted strong scintillation which did not occur on 5 days, and predicted that strong scintillation would not occur on 5 days for which strong scintillation was observed, i.e., whether strong scintillation would or would not occur was correctly predicted on 82.1% of days. This was achieved using a PVPD threshold of 18 ms −1 (i.e., there is a prediction of strong scintillation when the maximum PVPD speed is greater than 18 ms −1 ). Figure 1a displays a ROC plot for PVPD (blue) and RTGR (red) forecasting using TIE-GCM and persistence forecasting (green) for March and April 2000 at Vanimo, Papua New Guinea (141.3°E geographic longitude, −11.4°N magnetic latitude). Persistence forecasts were generated using the observed daily S4 value as the predictor value for the following day. In Figure 1a, the maximum hourly average S4 threshold has been fixed using the value 0.244 that was retrospectively selected by . The maximum YI is indicated by a colored circle for each forecasting method. There is not a significant difference between the overall performance of RTGR and PVPD forecasting; however, both the RTGR and PVPD forecasting methods clearly outperform persistence. In Figure 1b, the maximum Youden's Indices (YIs) corresponding to Figure 1a are presented for the range of S4 thresholds which resulted in PVPD, RTGR, or persistence forecasting outperforming at least 95 of 100 randomly generated forecasts (RGFs). In Figure 1b, PVPD YIs do not have a significant reduction in forecasting skill compared to RTGRs for any considered S4 thresholds, although the RTGR maximum YI is slightly higher than the PVPD YI for the chosen S4 threshold of 0.244. Both PVPD and RTGR forecasting significantly outperform persistence forecasting for all considered S4 thresholds. Area under the ROC curves (AUCs) are not displayed for this data set as at least 5% of the RGFs outperformed PVPD, RTGR, and persistence forecasting for all possible S4 thresholds. This suggests that an understanding of reasonable PVPD and RTGR thresholds to use is an important factor for forecasting skill if using the maximum hourly average to determine S4 values and thresholds.

Results
In Figure 2, the AUCs ( Figure 2a) and maximum YIs (Figure 2b) are presented for the same conditions as Figure 1 when the maximum hourly 90th percentile of one minute S4 observations (S4 90 ) determines scintillation strength as used by . In Figure 2a, the horizontal black-dashed line (AUC = 0.5) indicates no skill. In Figure 2a, the range of S4 thresholds for which a forecasting approach outperforms 95% of RGFs is small. This provides further evidence that having an understanding of appropriate PVPD or RTGR thresholds to use is an important factor for the success of each approach. PVPD AUCs and YIs are not significantly worse than RTGR AUCs or YIs for all considered S4 thresholds at Vanimo whether S4 is determined using the hourly average (Figure 1 8°N). PVPD forecasting performs as well as or better than RTGR and persistence forecasting apart from AUCs at Ascension Island for S4 90 thresholds above 0.67 (Figure 3a), AUCs for very low S4 90 thresholds at Nairobi (Figure 3d), and YIs for a very small range of S4 90 thresholds at Calcutta (Figure 4b). PVPD forecasting AUCs are significantly greater than both RTGR and persistence AUCs for all S4 90 thresholds below 0.55 at Ascension Island (Figure 3a), most S4 90 thresholds at Guam (Figure 3c), and most S4 90 thresholds between 0.15 and 0.49 at Nairobi (Figure 3d). PVPD forecasting maximum YIs are significantly greater than RTGR and persistence YIs for all considered S4 90 thresholds at Ascension Island ( Figure 4a) and Guam (Figure 4c).
Forecasting skill will be improved if suitable PVPD or RTGR thresholds are known prior to forecasting. However, these thresholds (and forecast success rates) are likely to vary under differing conditions such as longitude, season, and solar activity (e.g., Fejer et al., 1999;Smith et al., 2016). The PVPD and RTGR threshold values which produced the maximum YI in the test cases considered are provided in Figure 5. There is a noticeable difference between the optimum PVPD thresholds for Vanimo when compared to the other four test cases. The Vanimo test case was for March and April 2000 whereas the other test cases were for March and April 2011. Solar activity was higher in March and April 2000 (monthly means of the daily total sunspot number (MDTSN) of 217.7 and 191.5) than in March and April 2011 (MDTSN of 78.6 and 76.1), suggesting that the optimum PVPD threshold may be dependent on solar activity. The optimum PVPD thresholds do not, however, appear to be heavily dependent on the choice of S4 threshold. In Figure 6 the maximum YIs are displayed for the Vanimo test case when the PVPD threshold is fixed at 18 ms −1 for all S4 thresholds. There is not a significant reduction in forecasting skill when compared to the RTGR forecasting skill. This behavior is also seen in the other four test cases (apart from a slight increase in the small S4 range previously discussed at Calcutta), as shown in the supporting information.
NUGENT ET AL.  These results suggest that TIE-GCM PVPD forecasting, whilst being less complex and computationally cheaper, can perform at least as well as TIE-GCM RTGR forecasting when the S4 threshold is not set so low or high that most days fall into one observed scintillation strength class.
We now consider test cases beyond those previously discussed by Carter, Rettere, Yizengaw, Groves, et al. (2014.  used output from the Vanimo Ionospheric Scintillation Monitor (ISM) in March and April 2000 to test whether RTGR forecasting had skill. Vanimo ISM data are available from August 1999to October 2000, March 2001to October 2002, March 2003to May 2004, October-December 2006, February 2007to August 2008, and October 2008 to June 2009. We will now consider PVPD and RTGR forecasting performance during each pair of equinoctial (March and April, September and October) or solstitial (June and July, December and January) months within these periods.
The day-to-day variability of S4 90 daily maximum values from the Vanimo ISM shows a clear dependence on solar activity. Figure 7 shows S4   day background values seen between 1999 and 2004. Indeed, during all days on which the Vanimo ISM data is available between 2006 and 2009, there were only 4 days on which the daily S4 90 value exceeded 0.3. Therefore, as persistent strong scintillation is not present in these years, analysis of forecasting skill for these years would be both pointless and misleading. This is also true for winter solstitial months (December and January) during high solar activity years. During the years considered (1999)(2000)(2001)(2002)(2003)(2004), there was only one daily S4 90 value greater than 0.3 in December or January (no data are available for December 2000 and January 2001or December 2002and January 2003. Therefore, further test cases to be considered will include all equinoctial and June-July pairs of months between 1999 and 2004 for which significant data gaps are not present. Table 1 provides the number of days for which S4 data is available during each of these pairs of months. Test cases in which over a third of days do not have data available will not be included in the analysis. Results for March and April 2000 have already been discussed. Therefore, we will now consider forecasting skill for March and April in 2001and 2002, September and October in 1999, 2000, 2001, and 2003, and June and July in 2000, 2001, 2002, and 2003 Note. The monthly means of the daily total sunspot number are also provided.
In Figure 8 the maximum PVPD, RTGR, and persistence forecasting YIs are shown for March and April in 2001 and 2002. As seen in the previously discussed test cases, PVPD forecasting skill is as good as or better than RTGR forecasting for all considered S4 90 thresholds. PVPD forecasting also outperforms persistence forecasting for all considered S4 90 thresholds. Figure 9 displays maximum YIs for September and October 1999, 2000, and 2001, and June and July 2002. In September and October 1999, 2000, and 2001, the PVPD and RTGR forecasting skill is very similar. In June and July 2002, PVPD forecasting outperforms RTGR and persistence forecasting. However, PVPD forecasting is only able to outperform 95% of random forecasts for a small range of S4 90 thresholds. The PVPD and RTGR thresholds used to obtain these results are shown in Figure 10. The optimal PVPD thresholds for March and April in 2001 and 2002 are lower than those used for March and April 2000 ( Figure 5). The PVPD thresholds for September and October 1999 are lower than those used for September and October 2000, which tend to be lower than those used for September and October 2001. The monthly means of the daily total sunspot numbers (MDTSN:   lower. This lends further support to the suggestion that solar activity appears to be an important factor when determining the optimal PVPD threshold to use. It also suggests that the choice of PVPD threshold may be dependent on whether the equinoctial months are in the spring or autumn as, for example, the September and October 2001 PVPD thresholds are lower than the March and April 2000 PVPD thresholds despite having slightly higher MDTSNs. In June and July 2000, 2001, and 2003, and in September and October 2003  performance of PVPD and RTGR forecasting during the June and July test cases suggest that these forecasting methods, when using TIE-GCM, may not be suitable for scintillation forecasting during solstitial months. In the September and October 2003 test case, the PVPD values are particularly low, with only one PVPD value in this test case exceeding 10.1 ms −1 . This may be due to lower solar activity during this Vanimo test case, which has suppressed the equinoctial TIE-GCM daily PVPD values. Also, it may not be a coincidence that the only equinoctial test case which does not demonstrate any obvious forecasting skill includes extremely disturbed solar and geomagnetic conditions (including the Halloween storm). One should bear in mind with these results that TIE-GCM day-to-day variability is only driven by 3 hourly Kp and daily F10.7 solar flux values. It is possible that incorporating data assimilation into TIE-GCM may provide a significant improvement in forecasting skill for PVPD and RTGR forecasting.

Summary and Future Work
The PVPD low-latitude ionospheric forecasting method uses vertical drift speeds at the magnetic equator determined using ionosondes (Anderson et al., 2004). The application of this method to output from a physics-based ionosphere model has been proposed and tested for the first time. This method has been compared to an existing RTGR forecasting approach  using TIE-GCM.
PVPD forecasting with TIE-GCM provides forecasts with greater antecedence than PVPD forecasts generated using ionosondes, does not require equipment at forecast locations, and is computationally cheaper and simpler than the RTGR forecasting approach. In equinoctial test cases, the PVPD forecasting approach has been shown to have forecasting skill which is nearly always as good as or better than RTGR forecasting when S4 thresholds are not too low or high (i.e., the classification of strong and weak scintillation days is not too heavily weighted toward one class  In all December-January solstitial test cases and in years with low solar activity (2006)(2007)(2008)(2009) during the operational lifetime of the Vanimo Ionospheric Scintillation Monitor, the presence of strong scintillation is extremely limited. However, strong scintillation is observed in June-July solstitial test cases during years with higher solar activity (1999)(2000)(2001)(2002)(2003). Neither PVPD, RTGR, or persistence forecasting are able to demonstrate significant levels of forecasting skill by outperforming 95% of random forecasts in June-July solstitial test cases, apart from PVPD forecasting for a small range of S4 90 thresholds in 2002.
When compared to PVPD forecasting, the increase in complexity and computational expense associated with RTGR forecasting does not provide an obvious improvement in forecasting skill in the considered test cases. Furthermore, since the PVPD method only requires a 3D electron density model, PVPD forecasting is able to be performed without a coupled ionosphere-thermosphere model. However, the use of a different model would produce different PVPD forecasting skill results, and the use of a climatological ionospheric model for PVPD forecasting would be ineffective.  global early warning scintillation forecast could be provided for all low-latitude regions currently post-sunset and pre-sunrise or which will experience sunset within the next (e.g.) six hours if forecasts of model drivers are available. The computational cheapness, simplicity, and skill of PVPD forecasting during equinoctial months may make this approach attractive for an operational setting. However, if PVPD forecasting or any other forecasting technique could also demonstrate further skill during other conditions, particularly during solstitial months, the likelihood of operational usefulness would be significantly increased. An improvement in forecasting skill would be likely if using a data assimilation model. This would also provide the opportunity to determine whether the poor performance during solstitial months is due to limitations of TIE-GCM or whether the underlying behavior of the ionosphere is different during these periods.
All test cases considered in this work have been for Vanimo, Papua New Guinea, or at other locations during March and April 2011. Further cases must be considered to determine forecasting skill, particularly locations other than Vanimo under different environmental conditions. The relationship between vertical plasma drifts at the magnetic equator and subsequent EPB generation has previously been shown to be dependent on factors such as location, season, and solar activity (e.g., Fejer et al., 1999;Smith et al., 2016). Future work will involve determining PVPD thresholds which provide forecasting skill for different S4 thresholds and environmental conditions. The limitation of the range of suitable S4 thresholds is, in part, due to the models running through all possible PVPD/RTGR thresholds and, for the maximum Youden's Index, selecting the threshold which provides the greatest forecasting skill. This can result in randomly generated NUGENT ET AL.

10.1029/2020SW002462
14 of 16  forecasts performing equally well in cases with few data points. With a previously determined PVPD threshold, the maximum S4 threshold for which forecasting skill can be demonstrated may increase if sufficient data are available to determine suitable PVPD thresholds for these stronger, more rare scintillation events.
Further work will include a comparison of PVPD and RTGR forecasting skill using output from TIE-GCM and the Advanced Ensemble electron density (Ne) Assimilation System (AENeAS; Elvidge & Angling, 2019). AENeAS is a data assimilation model of the ionosphere and thermosphere, which uses TIE-GCM as a background model and assimilates data using a variant of the ensemble Kalman filter. The availability of ionosonde, GPS, and radio occultation data in near-real time make it likely that PVPD forecasting skill will improve significantly when using output from AENeAS rather than TIE-GCM. The availability of data in near-real time for some of the input variables required for RTGR forecasting (e.g., ion-neutral collision frequencies, neutral wind speeds) is, however, currently severely limited. In the future, if these data are available in near-real time at many low-latitude locations, RTGR forecasting skill may surpass that of PVPD forecasting as RTGR forecasting takes into account more of the underlying processes involved in EPB generation. However, it is possible that PVPD forecasting will show a greater improvement in near-real time forecasting skill than RTGR forecasting when data assimilation models are used with currently available data sources. Prior knowledge of appropriate PVPD thresholds to use for specified longitudes, seasons, levels of solar activity, etc., will also provide the opportunity to further improve the skill of low-latitude scintillation forecasts.