On the insignificance of Herschel's sunspot correlation

Authors


Corresponding author: J. J. Love, USGS Geomagnetism Program, Box 25046, MS 966 DFC, Denver, CO 80225, USA.(jlove@usgs.gov)

Abstract

[1] We examine William Herschel's hypothesis that solar-cycle variation of the Sun's irradiance has a modulating effect on the Earth's climate and that this is, specifically, manifested as an anticorrelation between sunspot number and the market price of wheat. Since Herschel first proposed his hypothesis in 1801, it has been regarded with both interest and skepticism. Recently, reports have been published that either support Herschel's hypothesis or rely on its validity. As a test of Herschel's hypothesis, we seek to reject a null hypothesis of a statistically random correlation between historical sunspot numbers, wheat prices in London and the United States, and wheat farm yields in the United States. We employ binary-correlation, Pearson-correlation, and frequency-domain methods. We test our methods using a historical geomagnetic activity index, well known to be causally correlated with sunspot number. As expected, the measured correlation between sunspot number and geomagnetic activity would be an unlikely realization of random data; the correlation is “statistically significant.” On the other hand, measured correlations between sunspot number and wheat price and wheat yield data would be very likely realizations of random data; these correlations are “insignificant.” Therefore, Herschel's hypothesis must be regarded with skepticism. We compare and contrast our results with those of other researchers. We discuss procedures for evaluating hypotheses that are formulated from historical data.

1 Introduction

[2] William Herschel [1801] interpreted telescopic observations of the Sun in terms of solar meteorology. The photosphere, he believed, was the top of luminous clouds and sunspots were openings in the clouds. Therefore, variations in sunspots would correspond to variations in solar irradiance which might affect the heating of the Earth's atmosphere and the Earth's weather. Riccioli (Almagestum Novum, 1651) and others had proposed similar hypotheses, but Herschel took the idea one important step further: he sought quantitative evidence, even if scarce and indirect, that might support the hypothesis. The problem was that Herschel was contemplating all of this decades before Schwabe [1844] discovered the ∼11-year solar-cycle waxing and waning of sunspots and before Wolf took up his ambitious compilation of historical sunspot numbers. Still, Herschel [1801] knew that sometimes the Sun had relatively few or no spots and that this condition could persist for several years: Flamsteed saw “no spot in the Sun” from 1677 to 1684; Cassini saw “no spot” from 1686 to 1688; etc., while at other times, sunspots were clearly seen. As for terrestrial meteorological data, Herschel lacked reliable measurements, and so he considered a proxy. Reasoning that farm crop yields would be correlated with temperature and that market prices for crop products would be anticorrelated with yields, he chose to analyze the London wheat price data compiled in Adam Smith's Wealth of Nations. Herschel found that wheat prices during five durations of time with few sunspots were high (inflated), while prices during five other durations were low (deflated). From this, he suggested that diminished sunspot number might correspond to a “deficiency of the solar beams.” In publishing his ideas, Herschel hoped to motivate a broader discussion on the role played by the Sun in affecting phenomena on the Earth.

[3] Subsequent analyses by reputable nineteenth century scientists did not convincingly confirm the existence of a correlation (or anticorrelation) between sunspots and wheat prices [e.g., Carrington, 1863; Poynting, 1884]. Still, the idea persisted, partly because of Herschel's enormous reputation, partly because a rigorous philosophy for statistical hypothesis testing had yet to be developed, and partly because it was simply so enticing. A correlation, if demonstrated, would enable the prediction of crop yields and product prices, possibly for financial gain. It is, therefore, not surprising that the next influential proponent of a hypothesis similar to Herschel's was an economist: William Stanley Jevons [1879] reported a correlation between sunspots and wheat prices in India, to which he assigned elaborate interpretations. But Jevons's evident lack of objectivity was soon ridiculed [e.g., Proctor, 1880], and today, while economists sometimes discuss a “sunspot effect,” it is usually as an abstraction of the extrinsic variables that contribute to a “market psychology” of uncertainty [e.g., Cass and Shell, 1983].

[4] The first clear evidence that specific terrestrial phenomena can be affected by sunspots was obtained by Edward Sabine [1856], who found a correlation between the solar cycle and the occurrence of magnetic storms recorded at ground-based observatories. Magnetic storms result from the dynamic interaction of the solar wind with the coupled magnetospheric-ionospheric system. Many storms are caused by coronal mass ejections from active regions defined by sunspots, but they can also be driven by high-speed streams of plasma flowing from coronal holes that develop during the declining phase of each solar cycle. While the physics of the solar cycle and magnetic storms remains the subject of active research, there is no doubt about the reality of the causal relationship. Indeed, Sabine's statistical correlation has held up over the 14 solar cycles since he discovered it [e.g., Chapman and Bartels, 1962, chapter 11], and it is one of the foundational principles of modern operational “space weather” forecasting.

[5] In contrast, a relationship between the sunspot solar cycle and terrestrial weather has been much more difficult to detect [e.g., Meadows, 1975]. Herschel's analysis on this subject, if not exactly meeting modern standards, is certainly important in the historical evolution of ideas [e.g., Hoyt and Schatten, 1997; Bard and Frank, 2006; Benestad, 2006; Eddy, 2009]. Still, we know of only one recent critical analysis of Herschel's hypothesis [Krut, 2008], while numerous publications report what are seemingly statistically significant correlations (or, sometimes, anticorrelations) between sunspots and agricultural prices and crop yields [e.g., King et al., 1974; Harrison, 1976; Vines, 1977; Legrand, 1977, 1978; Currie et al., 1993; Stanhill and Cohen, 2001; Pustil'nik and Yom Din, 2004a, 2004b; Garnett et al., 2006; Pustil'nik and Yom Din, 2009, 2013]. Herschel's hypothesis is also often depicted as being essentially factual in the popular literature [e.g., Clark, 2007; Cohen, 2011]. Given the present situation, we are motivated to conduct our own significance tests of Herschel's hypothesis. Results inform the wider and controversial subject of the role played by the Sun and solar-terrestrial interaction in affecting global climate change [e.g., Moore et al., 2006; Gray et al., 2010; Love et al., 2011; Lockwood, 2012].

2 The Data and Their Preparation

[6] As Herschel understood it, sunspot number might be used as a proxy measure of solar irradiance; for review of this and other proxies, see Gray et al. [2010]. We use annual mean international (Zurich or Wolf) relative sunspot number RZ, covering years 1700–2012, or more than 28 solar cycles up to the present rise phase of cycle 24. We obtained RZ from the Royal Observatory of Belgium [e.g., Clette et al., 2007]. Prior to 1700 and during the Maunder Minimum in sunspot number when systematic counts were not always made, we use a list of year dates of solar-cycle minima and maxima estimated from monthly sunspot numbers and records of days with and without sunspots [Eddy, 1976], obtained from NOAA's National Geophysical Data Center.

[7] We compare sunspot numbers with the terrestrial data summarized in Table 1. Of these, the most straightforward comparison is between sunspots and geomagnetic activity. The aa index [e.g., Mayaud, 1980], 1868–2012, measures magnetic storm intensity and lower levels of global magnetic field disturbance. Although geomagnetic activity is not the type of solar-terrestrial effect that Herschel was contemplating in 1801, there is, today, a reasonably well-established understanding of the relationship between sunspots and geomagnetic activity and, as such, a correlational analysis of RZ and aaserves as a qualitative check of our analysis methods. The aa index is derived from British and Australian magnetic observatory data; it can be obtained from the British Geological Survey. We average the 3-h aa index values into annual means.

Table 1. Terrestrial Data, Binary Results, Correlations With Sunspots, Autocorrelations
     MinMax Binary MaxMin Binary Pearson Cross Corr Lag-1 Autocorr
• Data  Years DecIncp(D)p(I) IncDecp(I)p(D) r(•,RZ)p(N)p(Ne)p(Nes) r1(•)r1(RZ)
aa Geomagnetic Activity 1868–2012 1110.99970.0031 1120.99980.0015 0.6898<0.00010.0001<0.0001 0.72420.7786
LondonWheat Prices 1646–1700 130.93750.3125 140.96870.1875
LondonWheat Prices 1646–1801 850.29050.8665 770.60470.6047
LondonWheat Prices 1646–1880 1190.41190.7482 10110.66820.5000
LondonWheat Prices 1700–1801 720.08980.9804 630.25390.9101 −0.23450.01760.41810.3404 0.70510.8099
LondonWheat Prices 1700–1880 1060.22720.8949 970.40180.7727 −0.2741<0.00010.24610.1232 0.78840.8187
LondonWheat Prices 1802–1880 330.65620.6562 340.77340.5000 −0.25590.02280.43290.3696 0.68570.8036
AmericanWheat Prices 1842–2012 690.84910.3036 870.50000.6963 0.01600.83500.95110.9316 0.86480.7724
AmericanWheat Yields 1866–2012 670.70940.5000 1120.01120.9982 0.01920.81670.94020.9243 0.80070.7709

[8] Following Herschel [1801], we analyze the market price of wheat. We combine two London annual mean wheat price lists, that of Smith [1776, Book I, chapter XI], of prices paid at Windsor Market on Lady Day (25 March) and Michaelmas (29 September) covering years 1646–1755 and a similar list compiled by Poynting [1884, Appendix, Table I] covering years 1756–1880. We combine two lists of the monthly mean price paid for wheat of all types in the United States: one from the National Bureau of Economic Research (NBER, m04001a), 1842–1907, and one from the Department of Agriculture (USDA), Economic Research Service, 1908–2012. For comparison with the London list, we extract the March and September data values from the NBER and USDA lists and average them for each calendar year. We also analyze a time series of wheat farm yield (bushels per acre), obtained from the USDA for all types of planted wheat, 1866–2012.

[9] For the positive definite data considered here, it is convenient to analyze their logarithms, so, for example, RZ→ log(RZ) [e.g., Chambers, 1886, p. 103]. Ratios of consecutive values of the untransformed data measure relative change over time, equivalent to differences of consecutive values of the log-transformed data. Long ago, Yule [1926] noted that data sets with trends can give “nonsense” measures of correlation. Advanced methods can be used to accommodate nonstationary trends [e.g., Stern and Kaufmann, 2000], but we prefer a simpler approach: prior to calculating Pearson correlations and Lomb spectra, we subtract a linear trend from the log-transformed data. For the sunspot data, this removes a long-term drift that might be described in terms of a random walk [Love and Rigler, 2012]. For the price data, this removes a constant rate of inflation. For convenient comparison, we normalize for time variation; we calculate the long-term mean μ and variance σ2 for each log-transformed data set; we plot, for example, log(RZ)−μ/σ, which is nondimensional.

3 Qualitative Assessment

[10] In Figure 1a, we see right away that aa geomagnetic activity lags behind but is otherwise correlated with solar-cycle variation in sunspot number. This is as expected. The wheat price and farm yield data are not, however, obviously correlated (or anticorrelated) in any way with sunspot number. Consider, for example, annual mean wheat prices in London (Figure 1b). In seemingly agreement with Herschel's hypothesis, wheat prices are high when sunspot numbers are low for years 1708–1714 and 1804–1816. But selectively focussing attention on subsets of data that are consistent with a hypothesis is not objective. It is useful to intentionally look for periods of time that obviously do not support Herschel's hypothesis, such as 1723–1744 when wheat prices are more correlated with sunspot numbers. In an average sense, London wheat prices are not apparently anticorrelated with solar-cycle variation of sunspots. Similar qualitative observations pertain to American wheat prices (Figure 1c) and to American wheat farm yields (Figure 1d).

Figure 1.

Detrended and variance-normalized time series of sunspot number RZ (black) and (a) the aa geomagnetic activity index (1868–2012, red), (b) the London (1646–1880, orange) and (c) American (1842–2012, green) wheat price, and (d) American wheat farm yield (1866–2012, blue).

4 Binary Tests

[11] Herschel [1801, pp. 314–316] used binary statistics in his analysis. He formed ratios of the average price of wheat for durations of time when the Sun had relatively few spots, divided by the average price for neighboring durations when the Sun seemed to have a more typical number of spots. This simple approach is attractive because it effectively removes long-term trends in the historical price of wheat that are affected by changing economic conditions, evolving farming methods, etc. We follow Herschel's method. We compare annual means of geomagnetic activity, wheat price, and farm yield for the years of solar-cycle minimum (maximum) with annual values for the following solar- cycle maximum (minimum). In Table 1, we denote these comparisons as “MinMax” (“MaxMin”). Considering, first, aa geomagnetic activity, 1868–2012, there are 11 MinMax increases (12 MaxMin decreases) in activity but only 1 MinMax decrease (1 MaxMin decrease). These realizations are not what would be expected for random binary “coin flip” trials. More objectively, we can estimate a significance probability using a null hypothesis binomial model. Assuming, for example, that the probability of a decrease or increase is 0.5 (binary), then the aa MinMax “p(I)-value” probability that 11 or more activity increases would be realized in 11+1=12 trials is only 0.0031; the complementary p(D)-value that 1 or more activity decreases would be realized in 1+11=12 trials is 0.9997. These probabilities are consistent with the known physical relationship between sunspot number and geomagnetic activity, and they give us confidence in the validity of our evaluation method.

[12] Herschel's hypothesis would predict MinMax London wheat price decreases (MaxMin increases), but for the entire duration for which we have London data, 1646–1880, MinMax (MaxMin) has 11 decreases and 9 increases (10 increases and 11 decreases). These are about what we might expect for the random binary null hypothesis: the London wheat MinMax “p(D)-value” probability that 11 or more price decreases would be realized in 11+9=20 trials is 0.4119; the p(I)-value that 9 or more price increases would be realized in 9+11=20 trials is 0.7482. These probabilities are not small. Similarly, the London wheat MaxMin p-values are not small. Therefore, the null hypothesis of randomness cannot be rejected. Herschel's hypothesis of a statistical relationship between sunspot number and the price of wheat in London is not seemingly supported by binary tests of the data. On the other hand, there appear (at first) to be significant MaxMin increases in American wheat farm yields from solar-cycle maximum to minimum, p(I)=0.0112. Interestingly, this is actually the opposite of what Herschel hypothesized for British farms, where he thought yields would follow terrestrial temperature and be correlated with sunspot number. But before we entertain a new hypothesis, we note that MinMax decreases in American wheat farm yield are not significant, p(D)=0.7094. This inconsistency has a simple and plausible explanation: random data will occasionally give small p-value measures of statistical significance. This is why we analyze more than one data set.

5 Pearson Correlation Tests

[13] Pearson's r coefficient is a conventional metric of cross-correlation between two time series [e.g., Press et al., 1992, chapter 14.5], but the estimation of its statistical significance requires care [e.g., von Storch, 1995]. In Table 1, we list r-values for cross-correlation between sunspot number and terrestrial data. The aa geomagnetic activity time series (log transformed and detrended), 1868–2012, has r=0.6898. A p-value measure of significance can be calculated assuming that the N=145 annual mean values give r-values that have a null hypothesis Gaussian distribution. With p(N)<0.0001, the measured correlation would be an unlikely realization of the null hypothesis, but before jumping to conclusions, we should accommodate for autocorrelation in the data. In Table 1, we list 1-year lagged autocorrelations r1 for each time series. Positive r1-values correspond to annual means that are partially redundant—the data are not statistically independent. This means that the information content of the time series could be “effectively” contained in a number of data Ne that is smaller than the number of annual mean values, Ne/N=(1−r1)/(1+r1) [e.g., Priestley, 1981, chapter 5.3.2]. After averaging the effective data numbers for each time series, we recalculate the cross-correlational p-values. For aa geomagnetic activity, p(Ne)=0.0001, which is, still, a small probability. A better approximation for effective data number, appropriate for an oscillatory time series, can be obtained with a second-order autoregressive model [e.g., Thiébaux and Zwiers, 1984], but we prefer a more straightforward estimate of effective data number, one that is directly motivated by the theme of our analysis. The number of data Nes that would effectively represent possible modulation of annual mean wheat prices and farm yields across a sequence of solar-cycle minima and maxima is given by an effective Nyquist number: Nes/N=2/Ts, where Ts is the ∼11-year length of the solar cycle (measured in years). For aa geomagnetic activity, p(Nes)<0.0001, a small probability. The correlation between sunspots and geomagnetic activity is “effectively” statistically significant.

[14] In contrast to the geomagnetic data, our evaluations for correlations of sunspot number with wheat price and farm yield are very different. For 1700–1880, the wheat price data have r=−0.2741 and an unadjusted p(N)<0.0001, but the effective p(Nes)=0.1232 is not a small probability. More generally, for all of the wheat data, all of the effective p-values are relatively large. Therefore, our evaluations using Pearson r are consistent with the binary tests in section 4: an anticorrelation between sunspot number and the London price of wheat is statistically insignificant. Furthermore, independent of the sunspot correlations listed in Table 1, American wheat farm yields and wheat prices are not, themselves, anticorrelated, they are correlated, r=0.4454, although with correction for r1 autocorrelation, this is not especially significant, p(Ne)=0.1269. As Carrington [1863] and Poynting [1884] long ago emphasized, and as even Herschel [1801, p. 313] acknowledged, the price of wheat is not a simple inverse function of farm yield. Not surprisingly, a myriad of interacting factors determine the price of wheat. This means that the string of seemingly logical associations that led Herschel [1801] to hypothesize an anticorrelation between sunspots and the price of wheat is not supported by the American data.

6 Frequency-Domain Tests

[15] We have, so far, focussed on the statistical significance of correlations between sunspots and terrestrial data, but we can also consider the properties of each individual data time series. We choose to examine the discrete frequency-domain power spectrum of each data set. For this, we detrend each data set, apply a Hamming window, and use a Lomb periodogram algorithm, which normalizes spectral power by the time-domain variance of the data [e.g., Press et al., 1992, chapter 13.8]. Spectra for annual sunspot number and the terrestrial data are shown in Figure 2, in each case, calculated for a common overlap duration of 1868–2012. Both the sunspot data and the aa geomagnetic activity data show prominent (and expected) spectral peaks at about 11 years. The “significance” of these peaks can be estimated by assuming that the individual Fourier power values (one for each discrete frequency) have a null hypothesis exponential distribution [e.g., Scargle, 1982]. The probabilities for random data to have spectral amplitudes exceeding those actually observed at solar-cycle periods are small: p(RZ)≪0.0001; p(aa) = 0.0004. On the other hand, ∼11-year solar-cycle variation is not obviously present in the wheat price and farm yield data; their p-values are all approximately unity. Therefore, the fundamental time behavior of sunspot number—the solar cycle—is not detectable with any reasonable level of confidence in these data. This means that among other things, introducing time lags between the (detrended) sunspot and wheat data will not lead to statistically significant Pearson correlations.

Figure 2.

Lomb power spectra for sunspot number RZ (black), aa geomagnetic activity index (red), American wheat price (green), and American wheat farm yield (blue). Based on data for the common interval 1868–2012.

7 Contrast With Previous Analyses

[16] Our finding that Herschel's hypothesis is statistically insignificant is consistent with that of Krut [2008], who used a binomial sign test in his examination of American wheat price and farm yield data. On the other hand, our results stand in curious juxtaposition with the reports of Pustil'nik and Yom Din [2004a, 2004b, 2009, 2013]. They cited Herschel [1801] as motivation for their analyses. However, they identified a seemingly significant tendency for increases in American wheat prices from solar-cycle minimum to subsequent maximum (MinMax, cycles 15–22, years 1913–1989), the opposite of the relationship suggested by Herschel. We have reproduced the one-sided Student t test significance probability reported by Pustil'nik and Yom Din [2004b, pp. 479–480] that was based on USDA wheat price data, p=0.0335. But when we add USDA data for cycle 23, 1996–2000, and earlier NBER wheat price data, cycles 9–14, 1843–1907, we obtain a much larger probability, p=0.2907. Since Pustil'nik and Yom Din were not apparently testing a hypothesis corresponding exactly to Herschel's MinMax price decreases, but were, it seems, open to consideration of a separate hypothesis for MinMax increases, it is more reasonable to use a two-sided t test. This doubles the probability to p=0.5814, which is not small and certainly not indicative of a statistically significant relationship between sunspot number and the American price of wheat. In a different study, Pustil'nik and Yom Din [2004a, pp. 347–350] performed an interval analysis on “bursts” in wheat prices. These intervals were derived using an 11-year filter, and so their statistical comparison with the actual distribution of solar-cycle durations was, at least partially, predetermined. We regard the results of Pustil'nik and Yom Din with skepticism.

[17] With respect to other published results that seem to support Herschel's hypothesis, those of Vines [1977] and Currie et al. [1993] are certainly worthy of comment. They identified ∼11-year solar-cycle periodicities in data recording wheat, oat, and wine harvests, fish catches, rainfall, and sea level; related results have also been published and frequently cited [e.g., Currie, 1974; Vines, 1982; Currie and Fairbridge, 1985]. Our results (Figure 2) contradict theirs. We speculate that the heavy filtering that Vines [1977] and Currie et al. [1993] applied to the data prior to spectrum estimation might have predetermined their results. We regard their results with skepticism.

8 The Past and the Future

[18] In controlled laboratory settings, a hypothesis, once clearly stated, can be tested by using it to predict the future and, then, prospectively collecting new data that can be compared against the prediction. If the prediction is deemed to be statistically significant, then the null hypothesis can be rejected. This approach works in some natural settings when the phenomenon of interest evolves relatively rapidly over time and new data can be collected without having to wait too long. But for evaluation of climatological hypotheses, decades of future data might be required before prospective significance tests can be made. Alternatively, a hypotheses can be tested by reserving a subset of the available data—a subset that is not seen when the hypothesis is developed—and, then, once the hypothesis is stated, these reserve data can be used for objective testing.

[19] In this context, let us consider Herschel's hypothesis. All of the data Herschel discussed in his 1801 paper were collected prior to 1717, during the Maunder Minimum and long before his paper was published. His identification of five durations of time with few sunspots and inflated wheat prices and five other durations that might have had sunspots and which had deflated prices [Herschel, 1801, pp. 313-316] would be an unlikely realization of binary statistics, but it is not clear whether or not Herschel was inspired to state his hypothesis after inspection of these data. Having said this, Herschel acknowledged that predictions based on his hypothesis “ought not be relied on by any one, with more confidence than the arguments... may appear to deserve” [Herschel, 1801, p. 318]. Today, we have considerably more data than were available to Herschel; these were collected both before and after he stated his hypothesis, and they can be used for both retrospective and prospective testing. For London wheat prices both before 1801 and, separately, after 1802, binary significance probabilities and Pearson correlations and their effective probabilities are summarized in Table 1. None of these are indicative of statistical significance. While solar irradiance may affect global climate, from our analysis of data of the type considered by Herschel, we conclude that historical wheat prices are not demonstrably useful for inferring past sunspot numbers, and, conversely, sunspot numbers are not demonstrably useful for predicting future wheat prices.

Acknowledgments

[20] We thank C. A. Finn, L. A. Pustil'nik, E. J. Rigler, G. Yom Din, and an anonymous reviewer for reviewing a draft manuscript. We thank D. M. Perkins for the useful conversations.

[21] The Editor thanks an anonymous reviewer for his/her assistance in evaluating this paper.

Ancillary