Validation of a priori CME arrival predictions made using real-time heliospheric imager observations

Between December 2010 and March 2013, volunteers for the Solar Stormwatch (SSW) Citizen Science project have identiﬁed and analyzed coronal mass ejections (CMEs) in the near real-time Solar Terrestrial Relations Observatory Heliospheric Imager observations, in order to make “Fearless Forecasts” of CME arrival times and speeds at Earth. Of the 60 predictions of Earth-directed CMEs, 20 resulted in an identiﬁable Interplanetary CME (ICME) at Earth within 1.5–6 days, with an average error in predicted transit time of 22 h, and average transit time of 82.3 h. The average error in predicting arrival speed is 151 km s − 1 , with an average arrival speed of 425km s − 1 . In the same time period, there were 44 CMEs for which there are no corresponding SSW predictions, and there were 600 days on which there was neither a CME predicted nor observed. A number of metrics show that the SSW predictions do have useful forecast skill; however, there is still much room for improvement. We investigate potential improvements by using SSW inputs in three models of ICME propagation: two of constant acceleration and one of aerodynamic drag. We ﬁnd that taking account of interplanetary acceleration can improve the average errors of transit time to 19 h and arrival speed to 77 km s − 1 .


CMEs and CME Arrival Time/Speed Predictions
Coronal mass ejections (CMEs) are large eruptions of coronal plasma and magnetic field carrying momentum and energy from the corona into the heliosphere.Interplanetary CMEs (ICMEs), the interplanetary manifestations of CMEs, are the principle drivers of severe space weather [Gosling, 1993], which can have significant detrimental effects to ground-and space-based technology, so there is increasing need to accurately predict ICME arrivals at Earth [Hapgood, 2011].Until recently, it was only possible to routinely observe CMEs close to the Sun (≈ 30R s ) with coronagraphs, and in near-Earth space with in situ spacecraft measurements.By combining observations of CMEs in these two spatial domains, it is possible to construct empirical models of CME propagation and hence forecast CME arrival time and speed at Earth [e.g., Gopalswamy et al., 2000;Vršnak, 2001].With the advent of instruments like Solar Mass Ejection Imager on the Coriolis spacecraft [Eyles et al., 2003] and the Heliospheric Imagers (HI) [Eyles et al., 2009] on the Solar Terrestrial Relations Observatory (STEREO) [Kaiser et al., 2008], it is possible to image CMEs as they propagate through the heliosphere to 1 AU and beyond.Therefore, Earth-directed CMEs can now be imaged all the way to Earth [Davis et al., 2009].Gopalswamy et al. [2000] measured the coronal speed of Earth-directed CMEs using halo CME observations made by the Large Angle Spectroscopic Coronagraph (LASCO) [Brueckner et al., 1995], located on the near-Earth SOHO spacecraft [Domingo et al., 1995].By comparison with in situ observations of ICMEs in near-Earth space, they inferred the Sun-to-Earth transit times and average interplanetary acceleration of ejecta.They concluded that faster ICMEs were decelerated, and slower ICMEs were accelerated by interaction with the background solar wind.On the basis of a strong correlation between interplanetary acceleration and initial CME speed, they proposed an empirical forecast of ICME arrival time and speed at Earth using only the initial CME speed and assuming constant acceleration.Gopalswamy et al. [2001] improved on this by ceasing acceleration at 0.76 AU, since acceleration is, generally, maximum close to the Sun and minimum at Earth.Vršnak [2001]  with distance from the Sun and depends on ICME speed and solar wind radial speed.From this, they produced forecasts of ICME arrival time and speed at Earth.Empirical models like these are useful as quick tools to provide a reasonable forecast of CME arrival time and speed at 1 AU [Cargill, 2004;Forbes et al., 2006].
Physics-based MHD models can also model ICME propagation.WSA Enlil [Odstrcil, 2003] is one of the more complex MHD models, which may take several hours to run.These numerical models require eight variables (three components of the magnetic field, three components of velocity, density, and temperature) to be specified throughout the heliosphere [Forbes et al., 2006], before the onset of a CME.MHD models continue to provide new information about the physics involved with CMEs.Zhao and Dryer [2014] reviewed the original Fearless Forecasts made during Solar Cycle 23.Several models were used to give near real-time predictions of the arrivals of interplanetary shocks (IS) at Earth.This is further discussed in section 2.4.
In this paper, we examine the performance of real-time predictions of ICME arrival time and speed and discuss ways to improve these predictions.In section 1.2, we discuss NASA's STEREO's spacecraft and the HI data, from which time-elongation plots (section 1.3) can be constructed (elongation is the angle between the Sun, observer, and the target).In section 2, we outline the Solar Stormwatch (SSW) project, the way in which CMEs are tracked by the volunteers, and how the data are analyzed.We explain the process by which we obtained our uncertainties and elongations, and how we matched real-time predictions with arrivals of ICMEs in near-Earth space.We present and comment on our results and use skill scores to test how well the SSW predictions perform.In section 3, we use different models of ICME propagation to examine whether taking some form of acceleration into account improves the average errors in the transit time and arrival speed.We discuss these results in section 3.4.Section 4 is the discussion of our findings and conclusions.

STEREO and HI
The STEREO spacecraft [Kaiser et al., 2008], launched in October 2006, provide an invaluable platform for studying CME propagation and evolution.STEREO comprises twin spacecraft in Earth-like heliocentric orbits, one preceding Earth in its orbit (STEREO-A) and the other trailing behind (STEREO-B).Both are moving away from Earth at 22.5 • yr −1 .Figure 1 (produced using the STEREO Science Center orbit tool) shows the positions of the STEREO spacecraft relative to Earth and the Sun on 1 January 2011 00:00 UTC.Between January 2011 and May 2013, the separation angle between STEREO-A and Earth increased from 85.5 • to 137.8 • .For STEREO-B, the separation angle with Earth increased from 89.9 • to 140.9 • over the same time interval.Thus, in this period, the STEREO spacecraft were close to being in quadrature with Earth (see Figure 1).
Each spacecraft carries the same suite of in situ and remote sensing instrumentation, including the Sun Earth Connection Coronal and Heliospheric Investigation (SECCHI) [Howard et al., 2008].The Heliospheric Imager (HI), part of the SECCHI package, contains two wide-field, white-light cameras, HI1 (inner) and HI2 (outer), that are, in nominal operations, aligned in the ecliptic plane to observe the Sun-Earth line [Eyles et al., 2009].HI1 has a field of view (FOV) of 4 • -24 • , centered nominally at 14 • elongation from Sun center, and HI2 has a FOV of 18 • -88 • , centered nominally at 53.7 • elongation from Sun center.HI tracks plasma density features by detecting sunlight scattered due to free electrons via Thomson scattering within the plasma [Thomson, 1906].Detected light is maximized from the surface of a sphere of which the Sun and observer lie at either ends of a diameter such that, at all points on this sphere, the angle between the Sun, sphere surface, and observer is 90  2012] suggests that the sphere is de-emphasized to some degree and thus termed it the Thomson plateau.Despite this, during the period January 2011 to May 2013, the STEREO satellites were at the optimum location for detecting Earth-directed ICMEs.
In addition to the daily download of science-quality data, STEREO can return near real-time data, via a lossy compression format of lower spatial and temporal resolution.Although the near real-time data contain less information, it allows real-time forecasts with lead times typically greater than 24 h to be made [Davis et al., 2011].Owing to constraints on the rate of data transmission from STEREO, the near real-time data can only be downlinked when ground stations from the volunteer network are available.The science data have a dedicated schedule for downlinking via the Deep Space Network, but this only takes place once a day so is of limited use for forecasting purposes, as the data stream is not updated in real time.Sheeley et al. [1999] developed a technique for tracking ICMEs using a time-height map (commonly called a J map) of observations from LASCO/SOHO coronagraphs [Brueckner et al., 1995].The authors stacked vertical radial profiles extracted at a fixed position angle (PA) from a sequence of coronagraph images, creating a time-height plot where solar wind transients moving outward from the Sun appear as positively inclined tracks.A track's properties depend on the speed and direction of the transient.Sheeley et al. [2008] and Rouillard et al. [2008] extended this technique to take advantage of HI's elongation range.J maps (which here are time-elongation maps) are similarly constructed by extracting a strip along a fixed PA (often the ecliptic, defined by the PA of Earth) from a series of HI images, and plotting the strips as a function of elongation (y axis) and time (x axis).Outward directed solar transients appear to have positive gradients [Davies et al., 2009].In order to highlight faint transients, J maps are usually derived from difference images, so solar transients have a bright leading edge (corresponding to a density enhancement relative to the previous image) and a dark trailing edge (corresponding to a density depletion) [Davis et al., 2011].

J Maps
In this study, the appearance of the plasma density fronts was maximized by reducing unwanted signals in the HI images (e.g., the signal of sunlight scattered from heliospheric dust and the background star field) according to the methods discussed in Davies et al. [2009].By assuming constant speed and propagation direction over transient observation time, it is possible to estimate its speed and direction from its J map viewed from a single vantage point [Rouillard et al., 2008].
Figure 2 is a J map of STEREO science-quality data for April 2011 and Figure 3 shows the STEREO near real-time J map for the same period.Although the real-time data have more data gaps than the science data, and the images from which it is derived are of lower resolution, transient tracks are still visible in the J map.
The tracks do not extend as far as those in Figure 2 (≈ 10  for real-time analysis of solar transients in the early stages of propagation.Figure 2 has 2.4% missing data for HI1 and 3.9% for HI2. Figure 3 has 61.5% missing data for HI1 and 73.9% for HI2.The amount of missing data for the real-time J map is much higher than for the science J map, highlighting the challenges of working with the near real-time data to make real-time forecasts.

Solar Stormwatch
The Solar Stormwatch (SSW) project (www.solarstormwatch.com) is part of the Zooniverse assemblage of Citizen Science projects [e.g., Davis et al., 2012;Fortson et al., 2012], based on the philosophy of Smith et al.
[2011] that a majority scaling decision by a large group of amateur volunteers is equivalent to that of an expert.Davis et al. [2012] found this to be true for SSW volunteers identifying the time of entry of CMEs into the HI FOV.In analyzing J maps derived from STEREO/HI real-time ecliptic data, the SSW volunteers manually select points along a CME's time-elongation track.These tracks are analyzed to look for clusters, and then all the tracks associated with one event/cluster are averaged into one time-elongation profile for the event.This profile is used to obtain best fit estimates of the propagation direction and radial speed, which are then used to forecast if and when a CME will arrive at Earth.For ecliptic J map analysis, the propagation direction is effectively the ecliptic longitude.The SSW user interface is undergoing revision [Barnard et al., 2014].Currently, user tracks of ICMEs in J maps are subject to large variations, as the users may be tracking different events.These tracks are discarded, as they do not provide useful information.The new interface will reduce the number of discarded tracks, thus increasing the statistical robustness of the results.

SSW Analysis
Initially, speed and direction estimates were made using the fixed  fitting (FPF) method [Rouillard et al., 2008].In September 2011, this changed to the Harmonic Mean Fitting (HMF) method [Lugaz et al., 2009], which does not significantly affect the resulting predictions [Möstl et al., 2014].The FPF method assumes the ICME is a radially propagating point source with fixed radial speed.The HMF method assumes that the ICME cross section takes the form of a radially expanding circle anchored to the Sun at one point along the circumference, and propagation is at constant speed.The real-time predictions of ICME radial speed, propagation direction, and arrival time at 1 AU are posted on the SSW forum and Twitter (@solarstormwatch).Once posted, these predictions cannot be changed.Our study looks at 124 SSW forecasts made between December 2010 and May 2013.Each is categorized based on predicted proximity in longitude to Earth, as calculated from the absolute propagation direction: TUCKER-HOOD ET AL.

Space Weather
10.1002/2014SW001106 A direct hit is equivalent to a 360 • PA halo CME seen in LASCO [Gopalswamy et al., 2000].SSW predicts 60 Earth-directed transients (categories 1-3), which we cross correlate with ICMEs in near-Earth space, as identified by Richardson and Cane [2010] using in situ magnetic field and plasma observations.Typical ICME properties include decrease in proton temperature, increase in solar wind speed, increase in plasma density, and changes to magnetic field direction (see section 2.3).Assuming constant propagation speed, the SSW analyses were used to predict arrival speed and time at Earth: predicted arrival speed is hence by methodology the same as the near-Sun speed.From these predictions, we define the time that the ICME was at 4 • elongation (the inner edge of the FOV and the lower boundary of the J maps), referred to as the start time, t 0 .The positions of the STEREO spacecraft change over time so the distance corresponding to 4 • elongation for Earth-directed CMEs will change, due to changes in viewing geometry.

Calculating Errors for the SSW Predictions
When the original SSW predictions were made, they did not include uncertainty estimates.To enable us to perform a comprehensive analysis, we recalculated the uncertainties for the original SSW predictions.The original average time-elongation profiles, from which the real-time predictions were made, were not stored, only the individual time-elongation profiles, so we recalculated these (using the same methodology) to obtain the arrival time and speed uncertainties.Some SSW volunteers have added tracks after the original forecasts were produced, and this information was incorporated into our revised average tracks.Consequently, when we used the new average tracks to recalculate uncertainties in arrival time/speed, we obtained different results, due to the large uncertainties in the near real-time J maps.To overcome this, we calculated the maximum elongation, where the best match to the original SSW forecast occurs, which differs between events.For nearly all forecasts, the recalculated speeds are within 25% of the original prediction, and the recalculated arrival times are within 10 h of the original prediction.The large differences obtained when recalculating arrival times and speeds from only slightly changed data are clear limitations of using the near real-time data.For two of the Earth-directed events, the postforecast tracking resulted in CME speeds that diverged significantly from the original values and so are discounted from further analysis.

Data and ICME Identification
Identifying ICMEs from in situ magnetic field and plasma data is a subjective process, as no one characteristic signature is either necessary or sufficient [Gosling, 1997;Zurbuchen and Richardson, 2006].To objectively test the SSW forecasts, we use the published catalogue of Richardson and Cane [2010], which is available via the ACE Science Center and regularly updated (www.srl.caltech.edu/ACE/ASC/DATA/level3/icmetable2.htm).This catalogue is derived from the ACE, SOHO, and Wind spacecraft.We allow a window of 1.5-6 days from the start time t 0 (see section 2.1) for an ICME to be seen at L1.This corresponds to average ICME speeds of 300 km s −1 -1150 km s −1 [Owens et al., 2005], covering the range of arrival speeds predicted by the original SSW analysis.In our analysis, we look for the time of first disturbance.Richardson and Cane [2010] list the shock time and the start of the ICME-in the case of fast ICMEs, these are the same, and in the case of slow ICMEs, there will be a time difference between these.There is one case where there are two SSW events predicted to impact Earth 5 h apart, with only one matching ICME.As we are unable to determine which prediction is associated with the ICME, we discount both from further analysis; leaving 56 predicted Earth-directed events.

Results of SSW Predictions
Table 1 summarizes the performance of the original SSW predictions against observational data.Of the 56 SSW ICME forecasts, 20 result in subsequent ICMEs in near-Earth space (hit) and 36 are false alarms.There are 44 ICMEs observed in near-Earth space during this period, according to the Richardson and Cane [2010] catalogue, that do not have a corresponding SSW forecast (missed events).These could result from real-time data gaps; the ICME being incorrectly predicted to miss Earth or no SSW volunteers tracking that ICME.In the same time period, there were 600 days on which there was neither a ICME forecast by SSW (allowing a maximum of 6 days transit time) nor a ICME seen in the Richardson and Cane [2010] catalogue.In calculating this value, if a day is duplicated in two or more events, it is only counted once, thus assuming a maximum TUCKER-HOOD ET AL.
©2014.The Authors. of one ICME per day.There are four outcomes in a contingency table: a is the number of times a predicted event occurs (hit), b is the number of times a predicted event does not occur (false alarm), c is the number of times an event is not predicted and does occur (miss), and d is the number of times an event is not predicted and does not occur (correct nonevent).Given a fixed number of events, the numbers in a contingency table can be described by the bias, the hit rate, and the false alarm rate.The bias B compares the number of times an event was predicted with the number of times an event was observed.If B = 1, there is no bias and the event was predicted the same number of times it was observed.If B > 1, the event was predicted more times than it was observed, and if B < 1, then the event was predicted fewer times than it was observed.
The hit rate H is the fraction of observed events that were correctly predicted and ranges from 0 (poor) to 1 (perfect).The false alarm ratio (FAR) is the fraction of predicted correct events that did not occur (false alarms) and ranges from 0 (perfect score) to 1.These are given by (3) In the case of Table 1, B is 0.88 (there were fewer predicted CMEs than observed CMEs), H is 0.31 (31% of CMEs were correctly predicted), and the FAR is 0.64 (in roughly two thirds of predicted CME events, a CME did not occur).Skill scores can evaluate the performance of the predictions.The Heidke Skill Score (HSS) [Heidke, 1926] measures the fractional improvements over random chance and ranges from −∞ to 1, where a perfect score is 1 and no skill level is 0. Negative values mean chance prediction is better than the model.The Odds Ratio (OR) [e.g., Mosteller, 1968;Stephenson, 2000] measures the ratio of the odds of making a hit to the odds of making a false alarm and ranges from 0 to ∞, where 1 is no skill (the probability of a CME occurring is the same as the probability of a CME not occurring) and ∞ is a perfect score.The OR is independent of biases.The Critical Success Index (CSI) measures the fraction of observed and/or forecast events that were correctly predicted [Schaefer, 1990].These are given by Here HSS is 0.27, indicating that the SSW predictions are ≈25% more accurate than a random chance forecast would be.OR is 7.6, so the odds of a hit prediction being correct are ≈8 times greater than the odds of a hit prediction being incorrect.CSI is 0.20, showing that 20% of ICMEs were correctly forecast.accuracy to those given in Zhao and Dryer [2014].However, the other models have a large sample size, while our sample size is much smaller, which may affect the results.
The U.S. National Weather Service (NWS) uses FAR as a percentage for various events.In the case of tornadoes, the NWS has a target FAR for 2009-2014 of 72% [Bilder and Johnson, 2012].In 2011, the NWS began including space weather in its forecasts.For 2011, the target FAR for space weather was 70%, and the actual FAR was 63% [Bilder and Johnson, 2012].In the case of Table 1, the FAR is 64%, which is close to the target set by the NWS, and higher than the actual FAR.Thus, while our results show the difficulty of making real-time forecasts using the lower quality near real-time data, there is definitely some useful skill in these forecasts.and range of 630 km s −1 .By contrast, the actual arrival speeds have a mean of 425 km s −1 and range of 360 km s −1 .This clearly shows the influence of the background solar wind interacting with the ICMEs, and either decelerating or accelerating them to similar speeds by the time they reach Earth.Kim et al. [2007] compared 91 predictions of shocks made with the Empirical Shock Arrival model with their actual arrival times and found that 55 events ( 60%) were within ± 12 h of the actual arrival time.McKenna-Lawlor et al. [2006] used STOA, ISPM, and HAFv.2 to make predictions of IS arrival times at L1.The number of hits within ± 24 h were 50%, 57%, and 51%, respectively.In the case of Table 3, eight events (40%) are within ± 12 h of the actual arrival time, and 12 events (60%) are within 24 h of the actual arrival time.Although our success rate for ± 12 h is rather low, that for ± 24 h is comparable to results achieved by others.We would expect that with better quality data, we would achieve a higher success rate.
TUCKER-HOOD ET AL.

Space Weather
10.1002/2014SW001106  Burlaga et al. [1981] define a magnetic cloud (MC) as a transient ejection in the solar wind with relatively strong magnetic fields, a large, smooth rotation of the magnetic field direction, and a relatively low density and proton temperature.It follows behind the interplanetary shock region commonly associated with CMEs [Burlaga et al., 1982].The Richardson and Cane [2010] list identifies the presence of an MC within the ICME.Table 4 summarizes the identification of MCs from Richardson and Cane [2010] according to their SSW forecasts of Earth proximity for the 20 hits.The top row of Table 4 gives the catalogue classification, which can be either no MC, partial MC, or MC.The first column is the SSW forecast of Earth proximity, of which there are three categories: near miss, glancing blow, and direct hit.There is a very slight tendency for a direct hit prediction to be more likely to be associated with an MC in near-Earth space than for a glancing blow or near miss.Recent work by Yan et al. [2014] using STEREO/SECCHI and SOHO/LASCO data found that there is no tendency for a halo ICME (the equivalent of an SSW direct hit) to result in an MC at L1.The authors also examined the number of MCs encountered by STEREO-A, STEREO-B, and ACE and concluded that the chance of receiving an MC at a given point in the heliosphere is very uncertain.
Having estimated the maximum elongations to which events were tracked as part of the error analysis in section 2.2, we separated the events into three groups, according to maximum elongation, to investigate what effect this has on CME arrival time predictions.Figure 4 presents elongation ( • ) against dt (days) for low maximum elongation (12  4, suggesting that the extent to which ICMEs are tracked in real-time data does not significantly improve forecasts.In contrast, Williams et al. [2009] found that the error in estimates of speed generally decrease the further the transient is tracked.The reason for the two different results is due to the nature of the real-time data used in our study.

Hindcasts of ICME Arrival Time and Speed
We now consider how assuming constant speed affects ICME arrival time and speed predictions, by testing three models of ICME propagation.We do this to see if, without changing the near real-time data, the forecasts can be made more accurate and reliable.

Space Weather
10.1002/2014SW001106 Gopalswamy et al. [2000] examined 28 halo ICMEs from January 1996 to June 1998, using LASCO/SOHO.The authors determined the near-Sun CME speeds u using a linear fit to the height-time plots in the LASCO field of view and found that the average interplanetary acceleration of ejecta between the Sun and Earth a G00 (m s −2 ) is a function of u (km s −1 ), such that

Constant Acceleration Model of Gopalswamy et al. [2000]
If u > 400 km s −1 , then the ICME accelerates, and if u < 400 km s −1 , then the ICME decelerates, effectively asymptotically approaching a fixed speed of 400 km s −1 .This is the result of the ICME interacting with the continual background solar wind, which has an average speed of 400 km s −1 , and will either slow down or speed up the ICME.If acceleration is assumed as constant between the Sun and Earth, ICME arrival time and speed at 1 AU can be predicted using only u. Gopalswamy et al. [2000] inferred the value of u from the plane-of-sky speed of halo CMEs, which is effectively the nonradial CME expansion rate.Schwenn et al. [2005] have shown this to be a reasonable proxy for radial ICME propagation speed.When the STEREO spacecraft are in quadrature with the Earth-Sun line, it is possible to measure ICME propagation speed directly, avoiding any plane-of-sky errors.Using SSW start time and initial speed with equation ( 7), assuming constant acceleration, we calculate revised predictions of arrival time and speed at 1 AU for ICMEs in Table 3.
The results are presented in section 3.4.

Reparameterized Constant Acceleration Model of Gopalswamy et al. [2000]
We reparameterize equation ( 7) by performing a linear fit between the SSW initial speed u HI (km s −1 ) (second column in Table 3), and average interplanetary acceleration between the Sun and Earth a HI (km s −2 ), using the method of York et al. [2004].This takes into account the errors in both variables at each point, producing a weighted relationship, such that Equation ( 8) has the same units as equation ( 7).We use equation ( 8) with the assumption of constant acceleration to calculate revised estimates of arrival time and speed for the ICMEs in Table 3.The asymptotic speed is the same in both cases, but the rate of acceleration is now lower than in the original equation.By performing the same analysis as Gopalswamy et al. [2000] to obtain equation ( 8), we confirm that the linear relationship between the acceleration and initial speed still exists when using different data from a different point in the solar cycle.

Drag Force Model
Cargill [2004] argued that ICME dynamics result mainly from the competition of three forces: (a) the Lorentz force, (b) the gravitational force, and (c) the drag force resulting from the interaction of the ICME with the background solar wind outside the low solar corona.The equalization of speeds of the ICME and background solar wind is primarily due to the drag force, thus determining that the drag force is important when forecasting ICME arrival times at 1 AU.The drag acceleration a DRAG (km s −2 ) at time t was related by Cargill [2004] to the instantaneous ICME speed, v t (km s −1 ), and background solar wind speed, w (km s −1 ): where c d is the dimensionless drag coefficient and the inverse deceleration length  (m −1 ) is sw is the solar wind density,  icme is the ICME density, C is the ICME cross-sectional area, and  is the volume of the ICME.Cargill [2004] showed that c d is of order unity and varies gradually through the interplanetary medium.Equation ( 10) can be rewritten as where  a C is the ambient (background) density and M * = M + M v , where M is the mass of the ICME and M v is the "virtual mass" (see Cargill et al. [1996], for further discussion).Given the assumptions that M * and TUCKER-HOOD ET AL.
©2014.The Authors.w remain constant, density decreases with distance as  a ≈ 1 distance 2 , and the ICME dimensions are roughly proportional to heliocentric distance, the value of  is approximately constant.We take the value of  to be 4 × 10 −11 m −1 as given by Vršnak and Žic [2007], who analyzed ICMEs over the period January 1997 to August 2004.The value of w is taken as the average solar wind speed over the same period as the events in Table 3 (410 km s −1 ), calculated from ACE data.We integrate equation ( 9) to produce new predictions of arrival time and speed for the events in Table 3.

Discussion of Hindcasting Results
We have tested three empirical models of ICME acceleration to examine the possibility of improving forecasts made with the near real-time data.Figures 5 and 6 are plots of the absolute values of dt and dv for each of the models used: (1) is the original SSW model; (2) is the Gopalswamy et al. [2000] model in equation ( 7); (3) is the reparameterized model given by equation (8); and (4) is the drag model of Cargill [2004].The central line is the median, the 25th and 75th percentiles are the edges of the box, the whiskers extend to the most extreme values not considered outliers, and the outliers are plotted separately as red crosses.In Figure 5, the SSW model has the highest median value and the only outlier, suggesting that the other models produce more accurate forecasts of arrival time.It does not have the largest range, so possibly improvements in acceleration are not the only way to improve these forecasts.Figure 6 also shows that the SSW model performs poorly when compared to the other models, as it has the highest median, outlier, and largest range of absolute values of dv.The absolute average error (hours) between actual and predicted arrival times, ⟨|ΔT|⟩ and the absolute average error (km s −1 ) between actual and predicted arrival speeds ⟨|ΔV|⟩ for the SSW model, equations ( 7)-( 9) are listed below.The quoted errors correspond to ±1 standard deviation.The original SSW analysis, assuming constant speed, has the highest average error in transit time and arrival speed, suggesting that including some form of acceleration in ICME propagation reduces uncertainty.However, we suggest that these effects are minor when compared with the uncertainties introduced by the current real-time data capabilities.
1. SSW model: ⟨|ΔT|⟩ = 22 ± 18 h and ⟨|ΔV|⟩ = 151 ± 135 km s −1 2. Equation ( 7): ⟨|ΔT|⟩ = 19 ± 16 h and ⟨|ΔV|⟩ = 77 ± 74 km s −1 3. Equation ( 8): ⟨|ΔT|⟩ = 19 ± 14 h and ⟨|ΔV|⟩ = 82 ± 82 km s −1 4. Equation ( 9): ⟨|ΔT|⟩ = 19 ± 17 h and ⟨|ΔV|⟩ = 77 ± 74 km s −1 Möstl et al. [2014] tracked Earth-directed CMEs from the Sun using J maps, and the FPF, HMF, and Self-Similar Expansion methods to hindcast predictions of arrival times of speeds.They found the average absolute difference between predicted and observed ICME arrival times to be 8.1 ± 6.3 h and speeds were within 284 ± 288 km s −1 .Empirical corrections made using linear fits from the data set used led to a reduction in average absolute differences of 6.1 ± 5.0 h for arrival times, and 53 ± 50 km s −1 for arrival speeds.This agrees with our conclusion, with two provisos: 1.As well as using an empirical model based on our data set, we have also used two models of ICME propagation that do not depend on our data set, and both show the same trend in results.2. our values of average error are generally higher than those of Möstl et al. [2014], most likely due to the near real-time data we have used.
Figure 7 shows SSW initial speed against observed transit time, where the Gopalswamy et al. [2000] model (equation ( 7)) is the dotted black line; the reparameterized version (equation ( 8)) is the dashed red line; and the drag model of Cargill [2004] (equation ( 9)) is the dash-dotted green line.There is a general trend of higher initial speed resulting in shorter transit times, borne out by all three models.Similar analyses have been performed by Gopalswamy et al. [2000] (cf. Figure 3), Gopalswamy et al. [2001] (cf. Figure 6), and Owens and Cargill [2004] (cf. Figure 2).These all show the same general trend as our results, although Figure 7 has considerably more scatter.This is likely to be the result of initial speeds being estimated from near real-time data, rather than the higher-resolution and better quality scientific data.

Discussion
In this study, we have tested the skill of Fearless Forecasts of ICME arrival times and speeds, made using near real-time data from STEREO's HI instruments, and analyzed by the Citizen Science project Solar Stormwatch.
Of the 124 ICMEs tracked by SSW in the period under consideration (January 2011 to May 2013), 60 were TUCKER-HOOD ET AL.

Space Weather
10.1002/2014SW001106 predicted to arrive in near-Earth space, defined by their propagation direction being within a certain range of degrees from Earth.We discounted four events for reasons discussed in section 2. Of the remaining 56 predictions, 20 could be associated with in situ observations of an ICME in near-Earth space, as taken from the list of Richardson and Cane [2010].This means the SSW predictions resulted in 36 false alarms.
During the same period, 44 near-Earth ICMEs were missed by the SSW predictions.This could be due to data gaps in the near real-time HI data, insufficient SSW volunteers at the time of the event, or incorrect determination of the propagation direction of an event.With the present SSW data, we are unable to differentiate between these possibilities.In the same period, there were 600 days on which there was neither an SSW prediction of an ICME in near-Earth space, nor an ICME seen at L1 according to the Richardson and Cane [2010] list.We test the results of the SSW predictions using skill scores: 1. Bias B is 0.88-there are more observed CMEs than there are predicted CMEs. 2. Hit rate H is 0.31-30% of CMEs are correctly predicted.
3. False alarm ratio FAR is 0.64-in around two thirds of predicted CME cases, a CME was not seen at L1. 4. Heidke Skill Score HSS is 0.27-SSW predictions of CME arrivals at L1 are 25% more accurate than a random chance prediction would be. 5. Odds Ratio OR is 7.6-the odds of a CME prediction being right are nearly 8 times greater than the odds of a CME prediction being incorrect.6. Critical Success Index CSI is 0.20-one fifth of CMEs were correctly predicted.
Previous studies of ICME prediction [e.g., Gopalswamy et al., 2000;Owens and Cargill, 2004;Vršnak, 2001] have mainly been hindcasts, focusing on the uncertainty in arrival times and speeds of ICMEs known to arrive at Earth, rather than testing the ability to correctly forecast whether or not a CME will arrive at Earth.Zhao and Dryer [2014] detail real-time Fearless Forecasts made during Cycle 23 using several different models.Our success rate is of similar magnitude to these, although our sample size is much smaller.Advanced forecasting of ICME arrivals with current real-time data capabilities is very challenging.This could be due to a combination of technical and physical issues: 1. Technical issues: reduced spatial and temporal resolution of near real-time data when compared with scientific data, the difficulty in finding ground stations to voluntarily downlink the data, and the assumptions involved in the SSW analysis.2. Physical issues: ICMEs changing during propagation due to encounters with CIRs, preceding/following ICMEs, and current sheets which are not seen in the near real-time data.
We tested the influence of the elongation to which ICMEs were tracked on the predictions by looking at the difference between the estimated start time of the ICME at L1 and the predicted CME arrival time for the 20 CMEs that were hits.There was no noticeable trend in the relationship between maximum elongation and dt, so we consider that the maximum elongation to which a CME has been tracked has no discernible effect on real-time predictions.This contrasts with previous such work [e.g., Williams et al., 2009], where, generally, the greater the maximum elongation, the lower the uncertainties in estimates of ICME speed.This may be due to the fact that our tracks are fairly short, and we are using near real-time data.We examined the probability that the predicted CME would contain a typical magnetic cloud signature and found that there was a 50% chance that a direct hit prediction would result in a CME containing a magnetic cloud.For glancing blow, the probability was 44%, and for near miss the probability was 33%.However, the uncertainties in these results are rather large, possibly due to the low sample size.
In future work, we will examine how much prediction improvement can be achieved through hindcasts of the SSW events using science-quality data.This will provide information on the optimum telemetry rates for real-time forecasting for future satellite missions.

Figure 1 .
Figure 1.The positions of the STEREO spacecraft relative to Earth on 1 January 2011 00:00 UTC.Axes are in Heliocentric Earth Ecliptic coordinates, and the units are AU.Produced using the STEREO Science Center's orbit tool.

Figure 2 .
Figure 2.An example of a time-elongation map (J map) constructed from running differences of scientific data from STEREO-A along the PA corresponding to the ecliptic.The x axis is time, with UTC along the bottom and Carrington date along the top.The y axis is elongation angle ( • ), starting at 4 • .The HI1 FOV is plotted to 20 • , and the remainder is from the HI2 FOV.Solar transients appear as approximately linear white/black tracks with positive gradients.Data gaps are vertical black bars.The curved track is the signature of Venus.

Figure 3 .
Figure 3.An example of a J map constructed with real-time data from STEREO for the same interval, in the same format as Figure 2. Note the significant increase in the number and duration of data gaps.Such J maps are used in the real-time predictions of ICME arrival time and speed used in this study.

Figure 4 .
Figure 4. Maximum elongation (degrees) plotted against dt (days) from the fitted time-elongation profiles.The blue points are profiles tracked to lower elongations (low group), the red points are the medium group, and the green points are the high group.The circles show the mean of each group, with the standard deviations shown as error bars.

Figure 5 .
Figure 5.The absolute values of dt (hours) for each of the models.(1) is the SSW model; (2) is the Gopalswamy et al. [2000] model; (3) is the reparameterized model; and (4) is theCargill [2004] drag model.The central line is the median, the 25th and 75th percentiles are the edges of the box, the whiskers extend to the most extreme points not considered outliers, and the outliers are plotted as red crosses.

Figure 6 .
Figure 6.The absolute values of dv (km s −1 ) for each of the models.(1) is the SSW model; (2) is the Gopalswamy et al. [2000] model; (3) is the reparameterized model; and (4) is theCargill [2004] drag model.The central line is the median, the 25th and 75th percentiles are the edges of the box, the whiskers extend to the most extreme points not considered outliers, and the outliers are plotted as red crosses.

Figure 7 .
Figure 7. Transit time to Earth (days) from 4• elongation (start time) as a function of SSW initial speeds (km s −1 ).The SSW model is the solid pink line; the Gopalswamy et al. [2000] model is the dotted black line; the reparameterized model is the dash-dotted green line; and the dashed red line is the Cargill [2004] drag model.

Table 1 .
Prediction Results Contingency Table

Table 2 .
Skill Scores: SSW Forecasts Versus Zhao and Dryer [2014] a Zhao and Dryer [2014] skill scores are based on data from Solar Cycle 23. a

Table 2
Zhao and Dryer [2014]d Dryer [2014], compares these results with skill scores from several different models [seeZhao and Dryer, 2014, and references therein].The Hakamada-Akasofu-Fry version 2 (HAFv.2) model is a solar wind model that calculates the solar wind speed, density, dynamic pressure, and magnetic field, as a function of location and time.The Shock Time of Arrival (STOA) model is based on the theory of blast waves from point sources.The Interplanetary Shock Propagation Model (ISPM) depends on the theory that the key parameter for the shock is the net energy released into the solar wind.The ideal value for each skill score is given in the second column.The results in Table2suggest that our results have a similar level of TUCKER-HOOD ET AL.©2014.The Authors.

Table 3 .
SSW Predictions Versus Actual Transient Arrival Data From Richardson and Cane [2010]Catalogue

Table 3
Richardson and Cane [2010]rth arrival times and speeds, along with the corresponding in situ values from theRichardson and Cane [2010]catalogue for those 20 "hit" ICMEs.The first and second columns list the SSW predictions of Earth arrival time and speed; the third is ICME shock time at L1; the fourth is the mean ICME speed; the fifth is dt, SSW-predicted ICME arrival time at L1 subtracted from observed ICME arrival time at L1 (hours); and the sixth is dv, SSW forecast arrival speed subtracted from the mean ICME speed (km s −1 ).Columns three and four are from theRichardson and Cane [2010]list.Positive values of dt imply that predicted time is after observed arrival time(and vice versa).Positive values of dv imply predicted speed is smaller than observed speed (and vice versa).The SSW speeds have a mean of 522 km s −1

Table 4 .
Magnetic Cloud Results Table No Magnetic Cloud Partial Magnetic Cloud Magnetic Cloud