Ionospheric critical frequency (foF2) is an important ionospheric parameter in telecommunication. Ionospheric processes are highly nonlinear and time varying. Thus, mathematical modeling based on physical principles is extremely difficult if not impossible. The authors forecast foF2 values by using neural networks and, in parallel, they calculate foF2 values based on the IRI model. The foF2 values were forecast 1 h in advance by using the Middle East Technical University Neural Network model (METU-NN) and the work was reported previously. Since then, the METU-NN has been improved. In this paper, 1 h in advance forecast foF2 values and the calculated foF2 values have been compared with the observed values considering the Slough (51.5°N, 0.6°W), Uppsala (59.8°N, 17.6°E), and Rome (41.8°N, 12.5°E) station foF2 data. The authors have considered the models alternative to each other. The performance results of the models are promising. The METU-NN foF2 forecast errors are smaller than the calculated foF2 errors. The models may be used in parallel employing the METU-NN as the primary source for the foF2 forecasting.
 Near-Earth space processes, including the ionospheric processes, affect navigation and telecommunication systems and the related system operation plans. In these domains, accurate and reliable modeling and forecasting of the ionospheric processes, especially in disturbed conditions, are important requirements. This fact has resulted in international projects, research, and investigations for alternative solution techniques, services, and references. As an example, International Reference Ionosphere (IRI) is an international project sponsored by the Committee on Space Research and International Union of Radio Science [Bilitza, 2001]. Forecasting the ionospheric critical frequency (foF2) is a common typical problem, which requires a solution that is capable of dealing with such a complex and highly nonlinear process.
 The 1 h in advance forecast of foF2 by the METU-NN model has been adapted for this work and employed herewith. Alternatively, the calculation of the foF2 values has been performed by making use of the IRI model, and this approach will be called the IRI-based model or just the IRI model. These approaches can be run in parallel. In this paper, these solution models and the performance results based on case studies have been presented.
Sethi et al.  presented foF2 comparison results considering the digital ionosonde observations at a low-latitude station (Delhi, 28.6°N, 77.2°E) during the solar active period in between August 2000 and July 2001. Their analysis considered diurnal and seasonal variations. In their work, it is concluded that foF2 observations and IRI-2000 model outputs showed coherent variations at daytime. On the other hand, nighttime day-to-day variations of the foF2 observations and IRI model outputs in the winter and in the equinox months were inconsistent [Sethi et al., 2003].
Zhang et al.  used monthly median values of the foF2 Digisonde observations with hourly time resolution at Hainan (19.5°N, 109.1°E) during March 2002 to February 2005 and compared the observations with the IRI model outputs. Then, Wang et al.  used digital ionosonde observations with quarter-hourly time interval resolution at Hainan in between February 2002 and April 2007. The time period covers high solar activity to low solar activity years. The variation patterns of the foF2 observations and the IRI model outputs are in good agreement. However, they observed a systematic deviation between observations and the IRI model outputs, especially during the moderate solar activity. The deviation gets smaller during the summertime at low solar activity [Wang et al., 2009].
Kim et al.  compared foF2 values at East Siberian high and middle latitudes during 2003–2006 using oblique ionosonde calculated data, vertical ionosonde experimental data, and the IRI model outputs. The calculated values follow the experimental data well. However, the IRI model outputs are not in agreement with them. They concluded that IRI model outputs are insufficient for that region of interest [Kim et al., 2007].
McKinnell and Friedrich  developed NN-based ionospheric model for the auroral zone to predict the high-latitude electron density profile. They used radar data from EISCAT at Tromso (69.58°N, 19.23°E) combined with rocket-borne measurements. The NN input space was designed considering day, time, total absorption, local magnetic K index, planetary Ap index, 10.7 cm solar radio flux, solar zenith angle, and pressure surface. They concluded that the model could be incorporated into the IRI model.
Xu et al.  presented a single-station model (SSM) using Fourier expansion. They used hourly foF2 observations obtained at Chongqing (29.5°N, 106.4°E) ionospheric observatory during 1977–1997. The years correspond to two solar cycles, i.e., 21 and 22. They evaluated the SSM and IRI model performance with the observation data. They concluded that the IRI model outputs have higher error values.
McKinnell and Oyeyemi  developed an NN-based global empirical model for foF2 using extended temporal and spatial inputs. According to their paper, their model can be a suitable replacement of the modules within the IRI model for the purpose of F2 peak electron density predictions. They demonstrated the significance of inputs of the NN-based model. The RMS error in foF2 predictions is around 1 MHz.
 In this work, considering the previous research and our background on data-driven modeling, we present a dual solution strategy for the foF2 forecasts and calculation. In order to facilitate a comparison between the METU-NN forecast foF2 values and analytically calculated IRI-based foF2 values, the geographic locations and the periods specified in Table 1 have been considered. The performance analyses are presented in section 4.
Table 1. Stations That Provided the Input Data for Both Cases
 The METU-NN has a two-layer feed forward architecture and Levenberg-Marquardt back-propagation algorithm is used in training [Haykin, 1999; Hagan and Menhaj, 1994]. Figure 1 gives the block diagram of the METU-NN. The hidden layer has six neurons in this specific METU-NN model forecasting the foF2 values.
 Each neuron in the hidden layer has links from the inputs, an activation function, a summation node, and links to the next layer. The METU-NN employs hyperbolic tangent sigmoid functions as the activation functions in the hidden layer and a linear function as the activation function in the output layer. It would be inevitable for NN to memorize the input data set if the validation set was not used in training. In the training phase of the METU-NN, terminating the training process, when the gradient of the error in ‘validation data within training’ approaches zero, prevents memorization.
2.1. Inputs and Output of the METU-NN in foF2 Forecasting
 In this work, raw data provided by the COST 251 Ionospheric Database have been used in constructing the inputs [Dick et al., 1999]. The data considered include the years 1979, 1982, 1983, 1989, 1990, and 1991.
 Slough (51.5°N, 0.6°W), Uppsala (59.8°N, 17.6°E), and Rome (41.8°N, 12.5°E) station foF2 data are extracted from the database. The time periods for training, validation within training, and validation within operation are given in Table 1.
 The inputs used in the METU-NN are summarized in Table 2. The output is the 1 h in advance forecast of the foF2: foF2(h + 1).
Table 2. Inputs of the METU-NN
foF2 observed at hour ‘h’
foF2 First Difference (FD)
foF2(h) − foF2(h − 1)
foF2 Second Difference (SD)
FD(h) − FD(h − 1)
foF2 Relative Difference (RD)
Trigonometric Sine component of the hour ‘h’
Trigonometric Cosine component of the hour ‘h’
 At solar maximum conditions, the rate of occurrence of the disturbed cases in the foF2 process is high and large foF2 fluctuations occur. These make the forecasting problem more challenging. The data set used in the work covers the time period of large solar storms as disturbed conditions as well as the quiet conditions.
 The time periods just before and after the 21st solar cycle maximum were selected in the author's previous work as well [Tulunay et al., 2001]. Since the probability of occurrence of trough is high above Slough and at higher latitudes, two of the ionospheric station locations are selected as Slough and Uppsala. In addition, data for another ionospheric station, Rome, covering the 22nd solar cycle maximum are employed as well.
2.2. Development and Operation of the METU-NN in foF2 Forecasting
 Within the development mode, training the NN optimizes the key parameters in the METU-NN. Within the operation mode, the developed model, METU-NN, is used to forecast the foF2 values 1 h in advance.
 The NN models are trained using representative data. The long-term major process conditions (e.g., solar maximum) are considered inherently in the training data. The trained NN model is then used for the selected major operation conditions.
 The ionospheric data are grouped in three sets. The first two sets are used in the development mode, and the last set is used in the operation mode. The first set covers data between 6 January 1979 and 6 May 1979 and data between 1 September 1989 and 31 December 1989. These are used in the training. The second set covers data between 6 January 1983 and 6 May 1983 and data between 1 September 1990 and 31 December 1990. They are used in the validation process during training. The last set covers data between 6 January 1982 and 6 May 1982 and data between 1 September 1991 and 31 December 1991. They are used in the operation mode.
3. Calculating foF2 by Using a Method Based on IRI Model Outputs
 The foF2 values are calculated for the same spatial and temporal coverage using a second method based on the International Reference Ionosphere (IRI) Model-95 outputs (http://modelweb.gsfc.nasa.gov/ionos/iri.html). The refraction index of an ordinary wave entering the ionospheric plasma vertically is expressed as in equation (1) [Davies, 1990]:
Here, ω is the angular frequency of the submitted wave.
Here ωpe and ωpi are the resonance frequencies of the electrons and ions, respectively.
 A wave reflects when its refraction index is zero. Thus, reflection occurs when the angular frequency of a wave is equal to the resonance frequency of the medium. The maximum value of the angular frequency of a reflected wave at a time is equal to the maximum resonance frequency at that time. This frequency is the ionospheric critical frequency, foF2, and it is expressed as in equation (3) [Davies, 1990]:
Here, e is the charge of electron, ɛ0 is the space dielectric constant, me and mi are the mass of electrons and ions, and Ne and Ni are the density of the electrons and ions, respectively.
 The ion and electron density values at the height where electron density is maximum, and at any time and geographic location coordinate, can be used to calculate ωpe2 and ωpi2 of equation (4). Then, by using equation (3), foF2 can be obtained. The electron and ion density outputs of the IRI-95 model, for the altitude where the electron density is of its maximum value, have been used to calculate the foF2 values.
 The performance results cover the operation of the models between 6 January and 6 May 1982 and between 1 September and 31 December 1991. The model results are evaluated by using some observed data. As measures of performance, we calculated the mean absolute error (MAE), root-mean-square error (RMSE) and cross-correlation coefficient (R) between the forecast or calculated and observed foF2 values. Equations (5), (6), and (7) are defining the respective measures:
In equation (5), oi is the model output results; mi is the observed data, and N is the total number of observations considered.
In equation (7), C(o, m) is the cross-covariance function and C(o, o) and C(m, m) are the autocovariance functions.
Table 3 shows the MAE, RMSE, and the R in the case studies for different ionospheric station data. The planetary 3 h–range index, Kp, is employed in the analysis (ftp://ftp.ngdc.noaa.gov/STP/GEOMAGNETIC_DATA/INDICES/KP_AP/). The performance results are presented considering different geomagnetic conditions, i.e., for whole data, for Kp ≤ 3 conditions, and for Kp > 3 conditions.
Table 3. Performance Test Results of the ‘METU-NN Forecast’ and ‘IRI-Based Calculated’ foF2 Values
For Kp ≤ 3, MAE (MHz)
For Kp > 3, MAE (MHz)
For Kp ≤ 3, RMSE (MHz)
For Kp > 3, RMSE (MHz)
R (× 10−2)
For Kp ≤ 3, R (× 10−2)
For Kp > 3, R (× 10−2)
Table 3 shows the mean performance results. In addition, the cumulative distributions of the absolute error values have also been calculated. Figures 2–4 exhibit the cumulative distributions of the METU-NN forecast and calculated foF2 error values considering the Slough, Uppsala, and Rome station foF2 data, respectively.
Figures 5 and 6 are the scatter diagrams considering Slough station data for the METU-NN forecast versus observed foF2 values, and calculated versus observed foF2 values, respectively. Similarly, Figures 7 and 8 are the corresponding scatter diagrams considering the Uppsala station data. Figures 9 and 10 are the corresponding scatter diagrams considering the Rome station data.
 To illustrate the daily performance results, some of the days within the operation time period are chosen. These are 29 January to 2 February 1982, 25–31 October 1991, and 8–10 November 1991. Figures 11–13 are the zoomed-in portions of the operation results. Figure 11 shows the foF2 variations considering the Slough data; Figures 12 and 13 show the foF2 variations considering the Rome data. Among these examples, in terms of geomagnetic conditions, 29 January 1982 is very quiet, 2 February 1982, 27–29 and 31 October 1991, and 8–9 November 1991 have strong solar storms.
 Considering the arithmetic mean of the errors of the foF2 values of the stations, as shown in Table 3, the METU-NN foF2 forecast values are within −0.51 and +0.51 MHz, whereas the arithmetic mean of the errors of the IRI-based calculated foF2 values are within −1.10 and +1,10. The METU-NN performed 46% to 63% better than the IRI-based calculated results. In general, the error values for quiet conditions (Kp ≤ 3) are smaller than the errors for disturbed conditions (Kp > 3).
 Considering the cross-correlation coefficients between the forecast and observed foF2 values, the METU-NN results are nearer to 1 than those of the IRI-based calculated foF2 values. Therefore, statistically, we checked if there is any significant difference between the two coefficients at a α = 0.05 level by using the Z distribution statistics. The computation showed that under hypothesis H0 we could not reject H0 since the Z value we obtained is greater than 1.96. Therefore, considering the cross-coefficient coefficients only, the results are not significantly different at a 0.05 level.
 The cumulative distribution of the error variation is useful in terms of displaying the overall performance. Considering Figures 2–4, the cumulative distribution of the absolute error values for the METU-NN forecast results show smaller values when compared with the errors of the IRI-based calculated foF2 results. In addition, the error values of the calculated foF2 results at Rome are smaller than the error values of the calculated results at higher latitudes (Slough and Uppsala).
 The scatter diagrams display the overall performance as well. When the scatter diagrams shown in Figures 5–10 are compared, they show that the best fit line in the METU-NN results has a slope closer to 1 and the deviations of the scatter points are small. Considering the temporal variation samples of the observed, METU-NN forecast, and IRI-based calculated foF2 values shown in Figures 11–13, we show again that the METU-NN forecast foF2 values follow the observed ones with smaller error values.
 In this work, the ionospheric critical frequency, foF2 values, obtained at the Slough, Uppsala, and Rome ionospheric stations during the years of 1979, 1982, 1983, 1989, 1990, and 1991 were considered. The foF2 values are forecast 1 h in advance by using the METU-NN, and in parallel, the foF2 values are calculated by a method based on the IRI model.
 The METU-NN forecast results show that the model learned the general shape of the inherent nonlinearity. This can be observed visually when the superimposed variations of the observed foF2 values and 1 h in advance forecast foF2 values are considered. The general variations of the forecast values are coherent with the observed values. As seen in Figures 11–13, even when there had been some extreme foF2 values in the experimental data set, the METU-NN forecast values follow the trend of the variations.
 The METU-NN was trained by considering the foF2 values sampled during the magnetically quiet and disturbed conditions. The selected time periods in this work are long enough to generalize the short-term performance results at such cases and locations. In addition, when the major operation conditions change, the model can be retrained and the updated model can be used. As an example, the model can be retrained for other solar cycles or for other locations. Training and operation of the NN are consecutive processes, which make it useful in terms of adaptation (evolution) capability.
 The reliability of the forecast data is very important in real time applications because the telecommunication and navigation applications using ionospheric parameters require future data values. Availability of the two methods may be an advantage in practice when fine tuning is necessary. In such cases the METU-NN model may be preferable. The models can be used in data postprocessing as well, especially in filling the data gaps.
 This work is partially supported by the EU Action of the COST 296 (Mitigation of Ionospheric Effects on Radio Systems).