A comparison between the hourly autoscaled and manually scaled characteristics from the Chilton ionosonde from 1996 to 2004



[1] Most monitoring ionosonde stations now rely on automatic processing rather than manual processing to provide ionospheric characteristics. A systematic assessment was made of the quality of all the key ionospheric characteristics scaled automatically from hourly ionograms from the midlatitude Chilton ionosonde in the United Kingdom, by comparing them with the definitive values produced by manual scaling. The data for the study were a nearly continuous series of hourly values covering the majority of solar cycle 23 from 1996 to the end of 2004. This period includes solar maximum and minimum conditions and quiet and disturbed intervals, and the investigation is the first comprehensive examination of the performance of automatic scaling without any data preselection over such an extended period. The accuracy of autoscaled values during storm periods was examined against the global storm index Dst for the whole 9-year data set. Geomagnetic conditions were found to have only a small effect on autoscaling performance, with the most important identifiable cause of error being the truncation of automatic layer traces due to broadcast interference. Overall, the performance of the autoscaling algorithms was found to be acceptable, with the characteristics foF2, h′E, M(3000)F2, and MUF(3000)F2 within defined error bounds for more than 90% of the time and all characteristics within these bounds more than 80% of the time.

1. Introduction

[2] Most ionosonde stations now rely on automatic scaling for obtaining regular ionospheric characteristics, and it is rare for a skilled person, or scaler, to check each ionogram routinely and manually correct or determine the values of the characteristics. Despite the complexity of ionogram data, automated processing has the obvious advantages of speed and reduced cost. This allows ionospheric sounding data to be made available on the Web in near real time, thus enabling the development of the nowcasting and forecasting capabilities required of space weather services (e.g., http://SpaceWeatherWeb.rl.ac.uk/, http://www.esa-spaceweather.net/) [Stamper et al., 2004a].

[3] The present study addresses the question of whether this change to rely on automatic scaling has been at the expense of accuracy in the scaled characteristics. The midlatitude Chilton ionosonde in the United Kingdom (51.6°N, 358.7°E) has maintained a high standard of manual scaling, with experienced scalers used during its operation over many decades. In particular, the manually scaled data used in this investigation were all produced by a single person. The ionosonde is one of the most widely used types of instrument, the Digital Portable Sounder, model DPS-1, also known as a “Digisonde” and produced by the University of Massachusetts, Lowell (UML) [Reinisch, 1996]. At all times the Chilton ionosonde has kept up with equipment and software updates, making it an ideal representative station for investigating the performance of automatic scaling algorithms, specifically the ARTIST software developed at UML [Reinisch and Huang, 1983].

[4] Unlike previous studies [Gilbert and Smith, 1988; Fox and Blundell, 1989; Igi et al., 1992; Pezzopane and Scotto, 2005], this study covers all the regularly scaled characteristics (foF2, h′F2, foE, h′E, MUF3000F2, foF1, h′F1, and fmin) for the continuous period from 1996 to the end of 2004. This encompasses sunspot minimum and maximum, storms and quiet periods, and changes in ionosonde equipment and software [McNamara, 2006].

[5] This study was conducted as a result of participation in the COST 271 Action “Effects of the upper atmosphere on terrestrial and Earth-space communications” [Zolesi and Cander, 2006] and the resulting creation of a real-time ionospheric database of European sounders, available at http://www.ukssdc.ac.uk/prompt_database.html [Stamper et al., 2004b].

2. Scope of Analysis

[6] Our analysis concentrated on eight of the more important ionospheric characteristics. These were (1) foE, foF1, and foF2, the critical frequencies of the principal ionospheric layers; (2) h′E and h′F2, the minimum virtual heights of the E and F2 layers; (3) fmin, indicative of ionospheric absorption; and (4) M(3000)F2 and MUF(3000)F2, the standard radio propagation characteristics used to indicate the maximum transmission frequency possible on a 3000-km circuit.

[7] For each of these the quantity investigated was the error in the autoscaled value, defined as the difference between the autoscaled and manually scaled values from the same sounding. This assumes the correctness of the manually scaled values [Piggott and Rawer, 1972]. For the Chilton sounder this is a reasonable assumption because all the ionograms in the study were scaled by the same person, who has more than 15 years experience of scaling ionograms.

3. Diurnal Variation of Autoscaling Errors

[8] In order to visualize such a long time series as 9 years of hourly soundings for each characteristic the data are presented as contour plots of the time of day against day number for that year. This format highlights the diurnal variations of the error in the automatic value compared with the manual value during the year. The color scale indicates the magnitude of the error, where a lighter color indicates a larger error. Figure 1 shows an example of a contour plot of the error of the autoscaled foF2 value for a year of high solar activity, namely 2001. The time of local sunrise and sunset (at ground level) is shown as a thin white line. The Chilton ionosonde makes its soundings on the hour. Figure 1 shows that the majority of the larger errors in the automatically scaled value of foF2 (light patches) consistently occur just after dusk. There are occasional larger errors during daylight but these are intermittent and rarely last longer than 2 to 3 h. There is a suggestion of marginally greater inaccuracy in the automatic value around the autumn equinox.

Figure 1.

An example of a contour plot of the annual, diurnal variation in the error in foF2 for the year 2001.

[9] Figure 2 shows multiple panels of the absolute error in automatically scaled foF2 values for the years 1996 to 2004, inclusive. The earliest year is shown in Figure 2, bottom left, with successive plots following a bottom to top then left to right order, enabling a column of panels to be readily interpreted as a single continuous plot. The contour colors are on a fixed scale from 0 to 4 MHz. Similar plots were generated for each of the other seven characteristics, and those for foE and M(3000)F2 are reproduced here in Figures 3 and 4. The most striking feature of these plots is a step change in the size of the errors near day 140 of 1999 (20 May). The size of the errors in some characteristics before this point, and the abrupt nature of the change, excludes the possibility that this change is due to natural variation of the geophysical environment. This leaves environmental or instrumental causes as the remaining possibilities.

Figure 2.

Contour plots of the annual, diurnal variation in the error in foF2 for 1996 to 2004.

Figure 3.

Contour plots of the annual, diurnal variation in the error in foE for 1996 to 2004.

Figure 4.

Contour plots of the annual, diurnal variation in the error in M(3000)F2 for 1996 to 2004.

[10] The most probable cause is instrumental change. UML released a new version of the ARTIST software in May 1999, although the engineering log book for the Chilton instrument records this as having been installed there only in January 2000. The evidence from the data presented here is that it was actually installed soon after release in May 1999. This is possible, despite the record in the log book, because at that time engineers at UML had the capability to modify parameters on the Chilton instrument remotely. Because of this instrumental effect, subsequent analysis concentrated on the data from the year 2000 onward.

4. Pointwise Analysis of Automatic and Manually Scaled Characteristics

[11] A scatterplot of automatically scaled against manually scaled foF2 values for all the hourly soundings in the year 2001 is shown in Figure 5. The fitted line, the parameters for which are shown at the top of the diagram, indicates that the fit is generally good, with a slope close to 1 and only a small intercept. Nonetheless, Figure 5 shows prominent “ledges” extending to the right of the ideal y = x line, where a frequency is scaled preferentially by the autoscaling algorithm when the true value ranges from that value up to about 3 MHz higher. Looking at the panels in other years (2000 onward in Figure 6), this phenomenon can be seen to reoccur at frequencies of roughly 6 MHz, 9.5 MHz, and 12 MHz.

Figure 5.

All the hourly autoscaled foF2 values for 2001 plotted against the true, manually verified, values.

Figure 6.

Correlation between manually scaled and autoscaled values of foF2.

[12] The cause of the problem becomes clear when the raw ionogram data are examined in more detail (see Figure 7). The cause was identified as interference from other radio sources. The Digisonde checks the noise (interference) level for each sounding frequency and retains only signals that are larger than the noise threshold. The presence of an external signal at just above 6 MHz would significantly raise the noise floor and would therefore cause a gap in the echo trace seen in Figure 7.

Figure 7.

Chilton ionogram for 1900 UT on 3 March 2003.

[13] Figure 7 shows a typical raw ionogram selected from one of the intervals where this occurred. The figure shows the swept frequency of 1 to 15 MHz on the x axis and the virtual height (corresponding to the time of flight) of the returned echo on the y axis. The red pixels indicate the O polarization, and the green pixels indicate the X trace. The pale blue line is the true height electron density profile generated by the ARTIST automatic-scaling software. It is clear that there is a gap between about 6 MHz and 6.3 MHz in both the O and X traces due to interference. Although the ionogram traces continue, it is clear that the ARTIST O-trace scaling prematurely halts at the gap. ARTIST determines foF2 as the maximum frequency on the scaled O-trace, which is clearly incorrect when that trace is ended prematurely as in this example. This is what produces the cluster of errors near specific interference frequencies on the annual plots of automatic against manual foF2. Clearly, this error also affects the h′F2 and foF1 characteristics and the HF propagation characteristics of M(3000)F2 and MUF(3000)F2, but has no effect on the lower-frequency characteristics like fmin, foE, and h′E.

[14] To assess the overall quality of the agreement between autoscaled and manually scaled values, a linear fit was carried out for each characteristic for each year. A summary plot of the correlation coefficients for these regressions for each ionospheric characteristic in each of the 9 years is shown in Figure 8. This figure demonstrates the overall improvement in the accuracy of the automatic scaling, after May 1999, for all the characteristics. It also shows that the ARTIST automatic-scaling software performs best at determining foF2 and MUF(3000)F2 and less well with fmin and M(3000)F2. The overall accuracy of ARTIST has improved considerably since 1999/2000, principally because of the instrument changes discussed previously and clearly demonstrated in Figures 2, 3, and 4.

Figure 8.

Correlation coefficients between the manually scaled and autoscaled values for each characteristic, calculated for each year.

[15] Similarly good correlations between the automatic and manual values hold for the other characteristics foE, h′F2, and MUF(3000)F2 from the year 2000 onward. The annual scatterplots for h′F2 are shown in Figure 9, and those for foE and MUF(3000)F2 show an equivalent pattern. Generally, the virtual height h′F2 is determined less reliably than the critical frequencies.

Figure 9.

Correlation between manually scaled and autoscaled values of h′F2.

5. Impact of Geomagnetic Activity

[16] In order for ionosondes to continue to be a primary source of real-time data on the state of the ionosphere for HF communications and other applications affected by space weather, the automatic-scaling software has to be able to function reliably during ionospheric disturbances throughout a solar cycle. To assess the reliability of the ARTIST software at Chilton in varying conditions, the errors in the autoscaled hourly values were compared with the geomagnetic Dst index. Classification of geomagnetic storms is frequently based on the Dst index, which can be obtained promptly from the World Data Center for Geomagnetism, Kyoto University, Japan, at http://swdcwww.kugi.kyoto-u.ac.jp/wdc/Sec3.html. This index, derived from a network of near-equatorial observatories, responds readily to the intensity of the globally symmetrical equatorial electrojet (the “ring current”) and is therefore called the “Disturbance Storm Time” (Dst) index [Mayaud, 1980].

[17] Figure 10 shows a comparison between the errors in autoscaled value against Dst value for each hour in the year 2001, at high solar activity. Similar plots for the years 1996–1999 are significantly affected by the instrumental problems discussed previously, but these do not affect the later years 2000–2004 for which the results were broadly similar to those from 2001 (see Figure 11).

Figure 10.

Errors in hourly foF2 plotted against the corresponding Dst index values for the year 2001.

Figure 11.

Errors in hourly foF2 plotted against corresponding Dst index values for the years 1996 to 2004.

[18] Inspection of the plots for each parameter for a year does not reveal any obvious relationship between the Dst index and the errors in any of the autoscaled values. In all cases the largest errors are observed with Dst values close to zero, indicating a geomagnetically quiet period. Comparing the marginal distribution of Dst values for the example year of 2001 (Figure 12) with the form of the autoscaling error distributions in the panel plots makes it clear that the error distributions are largely what one would expect to see if Dst and autoscaling errors were uncorrelated. Indeed, examination of the correlation coefficient, r, between Dst and each of the characteristics for each year reveals that in no case does the magnitude of r exceed 0.17, indicating essentially uncorrelated variables.

Figure 12.

Distribution of Dst values for the year 2001.

[19] However, to investigate the possibility that this distribution of Dst values was obscuring any dependence on Dst of the autoscaling errors, another form of plot was produced for the errors in foF2 values for 2001. To compensate for the much greater number of observations with near-zero Dst, the observations were binned by Dst value, ensuring that each bin contained at least 200 observations, and that all observations for any given Dst value lay inside a single bin. The resulting plot is shown in Figure 13, where the x axis values are the average Dst value in each bin, and the y axis values are the mean of the magnitude of the foF2 autoscaling errors; the error bars show the standard deviation of the magnitudes in each bin.

Figure 13.

Distribution of binned foF2 autoscaling errors for the year 2001. All bins contain at least 200 points.

[20] The plot suggests that there may be a correlation between the level of geomagnetic disturbance and the errors in autoscaled values, but that the correlation is weak and leads to only a small increase in errors. In the case of the 2001 foF2 values the total range of error magnitudes is only roughly 0.15 MHz, which is not significant in the context of the much larger errors shown in Figure 11 for 2001. In summary, although the geomagnetic conditions may have an impact on the accuracy of autoscaling, that impact is small and outweighed by other factors.

6. Summary of Analysis

[21] Some specific failings of the autoscaling algorithms have been identified associated with interference and early versions of the automatic scaling software. This should not be permitted to give a misleading impression of the overall accuracy of the autoscaling process. The improvement in accuracy following the change to the sounder parameters in 1999 is made very clear, and the relative stability of the performance from 2000 to 2004 is also demonstrated. It should also be noted that a new version of the ARTIST software was released in 2005 which significantly reduces the sensitivity to trace gaps [Reinisch et al., 2005].

[22] A useful indication of the overall quality of the performance of the autoscaling can be gained by examining the International Union of Radio Science (URSI) guidance on when ionospheric characteristics are too uncertain to merit quoting a numerical value [Piggott and Rawer, 1972]. Although this guidance is intended to be used at the time of scaling to indicate doubtful values, it gives a benchmark by which the seriousness of the errors in autoscaled values can be judged. The limits are quoted in terms of both absolute deviations (0.25/0.5 MHz or 2/5 km for E/F layer, respectively) and percentage deviations (20%). For simplicity we adopted a single, on average slightly more stringent, condition for each parameter as set out in Table 1.

Table 1. Maximum Acceptable Errors in Ionospheric Characteristics
Type of ParameterE Layer LimitF Layer Limit
Frequency0.2 MHz0.7 MHz (1.0 for MUF)
Virtual height20 km50 km
M factor 0.4

[23] Using the data from 2000 onward, the eight scaled characteristics studied were found to fall into two groups, according to how frequently the errors remain below these limits: (1) >90% reliable – foF2, h′E, M(3000)F2, and MUF(3000)F2 and (2) >80% reliable – fmin, foE, foF1, and h′F2. For the vast majority of soundings the accuracy of characteristics is therefore acceptable, with particularly good reliability when scaling the widely used characteristics foF2 and MUF(3000)F2. In general, the scaling is less good for any characteristics likely to be affected by difficulties in distinguishing the relevant traces in the relatively “busy” low-frequency portion of an ionogram: hence the lesser accuracy with fmin, foE, foF1, and h′F2.


[24] The authors wish to thank Rita Blake, the Chilton ionosonde scaler, for all her diligent hours scaling ionograms over the years. Thanks also to Sarah James, Chris Davis, and John Smith of the Ionosonde Group at RAL. This work was funded by the Radio Communications Agency of the U.K. Government, now part of Office of Communications, Ofcom.