Application of mesoscale ensemble forecast method for prediction of wind speed ramps
Abstract
Sudden changes in wind speed, so‐called wind speed ramps, are a major concern for wind power system operators. The present study applies the mesoscale ensemble forecast method for the prediction of wind speed ramps at wind farms in Japan and evaluates the ability and utility of this method. The mesoscale ensemble forecast in this study (ENS21) consists of 21 members with a horizontal resolution of 10 km for a 5‐year period. The simulated results show that ENS21 produces better accuracy than the deterministic forecast with a horizontal resolution of 10 km (DET_L). On the other hand, the deterministic forecast with a horizontal resolution of 5 km (DET_H) also produces better accuracy than DET_L. From a practical perspective, however, the ENS21 is computationally expensive. Thus, the eight‐member mesoscale ensemble forecast (ENS8) with as same computational cost as a deterministic forecast with a horizontal resolution of 5 km (DET_H) is also evaluated. The simulated results show that ENS8 has almost same accuracy as ENS21 and DET_H in wind speed ramp forecasts. ENS8 has advantages over ENS21 and DET_H because ENS8 is computationally efficient and is able to benefit wind power operators with flexibility in the selection of probability thresholds for decision processes compared with a single. It can be concluded that the mesoscale ensemble forecast method is more useful for prediction of the wind speed ramp than the single deterministic forecast method with the same computational cost if the ensemble members are successfully selected.
1 INTRODUCTION
The demand for renewable energy sources in Japan, particularly wind power, has been increasing dramatically, especially after the Great East Japan earthquake and the Fukushima accident in March 2011. According to the wind vision report released by the Japan Wind Power Association,1 the total capacity of wind power installation is expected to increase from 4.3 billion kWh in 2010 to 23 billion kWh by 2020 and to 84 kWh by 2030, an increase of approximately 20 times in 20 years.
In contrast with fossil fuel energy sources, wind power production is highly dependent on wind conditions that constantly fluctuate over time and space. In particular, the sudden increases and decreases in wind speeds, which are known as wind speed ramp‐up and ramp‐down events, induce instability in wind power supplies.2 Unstable wind power supplies thus cause economic losses, as the power system must be balanced by starting up other hydro or fuel energy sources.3, 4
To increase the stability and reliability of wind power production, accurate forecasts of wind speed ramp events are therefore necessary. Generally, wind speed ramps are driven by factors at multiple temporal and spatial scales, such as the development and movement of air fronts, areas of low and high pressure, thunderstorms, boundary layer processes, mountain waves, flow channelling, and sea breeze.5, 6 These features can explain why wind speed ramp events remain poorly forecasted.7
There are two approaches for wind speed ramp forecasts. The first is based on statistical analysis of wind time series, and the second uses numerical weather prediction (NWP) models that are sometimes combined with statistical bias correction methods.8-10 The first approach provides reasonable results in the majority of cases in the estimation of the mean monthly wind speed or the wind speed on an even longer temporal scale (quarterly and annual). However, at the short‐range timescale (from a few hours to several days), as the effect of atmospheric dynamics becomes more important, the second approach using NWP models is preferred.9
Mesoscale NWP models have advantages in representing particular local weather phenomena. Recently, mesoscale NWP models have been used to predict wind speed ramp events in a country or in a region of a country.7, 9, 11, 12 However, there remain several weaknesses of mesoscale NWP models, such as the uncertainties in model schemes and initial and boundary conditions (IBCs).13
Deppe et al7 have evaluated the ability of a mesoscale model, the Weather Research and Forecast (WRF) model, to predict wind speed ramp events at the Pomeroy wind farms in Iowa, United States. They have conducted mesoscale ensemble forecasts with multiple planetary boundary layer (PBL) schemes. They show that the model performance regarding ramp events was generally poor and that the model tends to predict fewer ramp events compared with the observations. The most recent efforts that enhanced the ability to predict ramp events are those of the Wind Forecast Improvement Project (WFIP),11, 12, 14 sponsored by the Department of Energy of the United States. The WFIP consists of an ensemble of high‐resolution rapid‐update NWP models. Each of these ensemble members incorporates a variety of model configurations, physics parameterizations, and data assimilation techniques. The results show that the WFIP provides a better performance on ramp forecasts than a current forecast system.
As noted above, although few in number, the applications of ensemble forecasts with the mesoscale NWP model for wind ramps are introduced; however, most are based on an ensemble of physical schemes or time‐lagged ensemble. There remains a lack of studies on wind ramp forecasts with mesoscale NWP models but based on an ensemble of IBCs. In addition, the previous studies on ensemble forecasts are limited to cross‐member comparison, there have no studies that describes an ensemble forecast as one probabilistic system. As mentioned by Zhang et al,11 there is need to develop probabilistic wind ramp forecasts through NWP ensembles.
Japan is located in the mid‐latitudes between the Pacific Ocean and the Eurasian continent, where large synoptic‐scale atmospheric variabilities occur mainly due to developing and travelling of high/low pressure systems. The large synoptic‐scale variabilities can induce large uncertainties in the IBCs for mesoscale NWP models.13, 15 Wind speed ramps are sensitive to the variabilities of synoptic‐scale atmospheres or local atmospheric circulations due to complex terrain. Therefore, a mesoscale ensemble approach with multiple IBCs derived from a global ensemble forecast is beneficial to reduce uncertainties in predicting wind speed ramp events.
The goal of the present study is to assess the performance of the WRF model, perturbed with IBCs, in predicting wind speed ramp events along the western coast of Hokkaido, Japan. The mesoscale ensemble forecast is compared with deterministic forecasts and conducted at two horizontal resolutions for a 5‐year period from April 2011 to March 2016. Such a long verification period, which was never done in any previous studies for wind ramp forecasts, is expected to provide robust results for analysis.
2 METHODOLOGY
2.1 Model configuration and data
In the present study, version 3.5.1 of the WRF model16 was used to conduct numerical forecasts. Numerical forecasts comprise two deterministic forecasts and one mesoscale ensemble forecast (Table 1).
| Type of Forecast | Initial and Boundary Conditions | Horizontal Resolution | Offline Forecast Period | Computational Cost (With 20 Processors) | |
|---|---|---|---|---|---|
| DET_L (control forecast) | Deterministic forecast | Global forecast system (0.5°) | 30 km (d01); 10 km (d02) | April 1, 2011, to March 31, 2016 | 18 min |
| DET_H | 25 km (d01); 5 km (d02) | 59 min | |||
| ENS21 (21 members) | Mesoscale ensemble forecast | Global ensemble forecast system (2.5°) | 30 km (d01); 10 km (d02) | 18 min for one member | |
| ENS8 (eight members) |
The two deterministic forecasts have different spatial resolutions. The first with a spatial resolution of 10 × 10 km (hereafter called DET_L) was used as the control forecast; the second had a higher spatial resolution of 5 × 5 km (hereafter called DET_H). Both DET_L and DET_H used the global forecast data from the Global Forecast System (GFS), which is operated by the National Center for Environmental Prediction (NCEP), as the IBCs.
The mesoscale ensemble forecast consisted of 21 single model runs (hereafter called ENS21) that used the global prediction data from the Global Ensemble Forecast System (GEFS). The GEFS is used to address the nature of uncertainty in weather observations, which is used to initialize weather forecast models, by generating an ensemble of multiple forecasts, each perturbed, from the original observations using the ensemble transform Kalman filter method. The ENS21 had an innermost grid spacing of 10 × 10 km (Figure 1), similar to the control forecast DET_L.
The forecasts were initialized every day at 0000 UTC (0900 of local standard time) for a 5‐year period from April 01, 2011, to March 31, 2016. The forecasts at a lead time (LT) of +24 to +48 hours are used for analysis. The details of the model configuration are summarized in Table 2.
2.2 Definitions of wind speed ramps
In the present study, the forecast performance is verified with wind speed ramps rather than wind power ramps because wind power production is usually dependent on human factors and wind power ramps are sometimes not the result of wind speed ramps. The observed wind speed data (at each 30 minutes) were obtained from the Automated Meteorological Data Acquisition System (AMeDAS). AMeDAS is administered by the Japan Meteorological Agency and consists of approximately 1300 local stations covering the whole the country. In this study, data from six stations, Koetoi, Teshio, Haboro, Suttsu, Setana, and Esashi (Figure 1B), located on the western coast of Hokkaido near the wind farms of interest, are used.
Wind speed ramps are classified into two categories: ramp‐ups and ramp‐downs. Events are defined as ramp‐ups or ramp‐downs if the change in wind speed is greater than 5 m/s or lower than −5 m/s over a 6‐hour time window. The wind speed ramp detection algorithm was developed based on the procedure made by the Technological Research and Development Project for Wind Power System Output Fluctuation,23 which is supported by the New Energy and Industrial Technology Development Organization (NEDO). NEDO is the biggest governmental organization in Japan and was established in 1980 in order to promote the development and introduction of new energy technologies including wind power techniques.
Note that in this present study, model output statistics (MOS) techniques are not employed for post‐processing the forecast outputs. The purpose is to verify the ability of the WRF model itself for wind speed ramp prediction. The combination of the WRF and MOS may induce difficulty in explaining the forecast results.
2.3 Verification metrics
A contingency table (Table 3) was used to represent the forecast results and observations. In a contingency table, four combinations, also called joint distributions, of forecasts (yes or no) and observations (yes or no) are defined. These include hits (A)—ramp events forecast to occur that did occur; false alarms (B)—ramp events forecast to occur but that did not occur; missed (C)—ramp events forecast not to occur but that did occur; and correct negatives (D)—ramp events forecast not to occur that did not occur.
| Observed | |||
|---|---|---|---|
| Yes | No | ||
| Forecast | Yes | Hits (A) | False alarms (B) |
| No | Misses (C) | Correct negatives (D) |
Key quality measures derived from the contingency table are the probability of detection (POD), success ratio (SR), frequency bias (Fbias), probability of false detection (POFD), and critical success index (CSI).
3 RESULTS AND DISCUSSION
3.1 Wind speed ramps in Hokkaido
Wind speed ramps depend on atmospheric conditions, and their occurrences vary seasonally. An analysis of observation data shows that wind speed ramps occur often in October to April, and relatively rarely in the summer months (Figure 2). The increase in the wind speed ramps during winter is likely due to the seasonal strengthened north‐westerly wind associated with the East Asian winter monsoon. Overall, the WRF model shows good accuracy in the prediction of the frequency of ramp occurrences for either mesoscale ensemble forecasts or deterministic forecasts. The main trend of seasonal variation of ramps is well predicted. The lower and higher peaks of predicted ramps agreed well with those of the observation. Note that Figure 2 shows the number of the ramps in their relative frequency but not in their absolute value.
Regarding the diurnal variation (Figure 3), the ramp‐ups occur commonly in the morning (8‐9 am), whereas the ramp‐downs occur more often in the afternoon (3‐5 pm). A reason for the timing of the ramp‐up peak is associated with the vertical mixing layer that starts to form after sunrise and cause an explosive increase in the wind speed. On the other hand, the decay of the sea breeze and the boundary stabilization during the time of sunset are likely an explanation for the timing of the ramp‐down peak. In addition, the secondary peak of the ramp‐ups is seen around 9 pm. An explanation for this can be associated with the development of the land and the mountain breezes after nightfall. In contrast, the weakening land and mountain breezes in the early morning during the time of sunset is a possible reason for the secondary peak of the ramp‐downs. The models show a good performance on both the diurnal variation and the relative frequency of the ramp events (Figure 3). However, there is an early prediction on the ramp‐down peak by the models. Three models tended to predict the ramp‐down peak 1 to 2 hours earlier than the observation.
3.2 The performance of deterministic WRF forecasts
Here, the model performance is verified using a performance diagram (as illustrated in Figure 4). A performance diagram is a useful tool to summarize the model outcomes of dichotomous (yes/no) forecasts,24 such as forecasts of wind speed ramps. In the diagrams, the POD values are represented by the vertical axis, and the SR values are represented by the bottom horizontal axis. The Fbias and the CSI are represented indirectly through the geometric relationship of the POD and the SR. In the performance space, the overall forecast metrics are improved as the value moves towards the upper right of the diagram.
First, it is worthwhile to see the forecast skills of 21 individual forecasts of ENS21 (iENS21) in comparison with those of DET_L and DET_H (Figure 4). The results show that single iENS21 forecasts tend to gather together on the performance space (Figure 4) near the location of DET_L. This means that the iENS21 had the same performance in terms of ramp forecast, comparable with the control forecast DET_L. On the other hand, it appears that DET_H had relatively better forecast performance. For ramp‐up forecasts, the CSIs of iENS21 were approximately 0.24; on the other hand, the CSIs of DET_H and DET_L were approximately 0.33 and 0.26, respectively. For ramp‐down forecasts, the CSIs of iENS21 were approximately 0.18, while the CSI of DET_H and DET_L were approximately 0.26 and 0.19, respectively.
It is seen that the iENS21 and DET_L tended to under‐forecast the number of ramp events. The Fbias values of iENS21 and DET_L were approximately 0.5 for both ramp‐ups and ramp‐downs. Meanwhile, the Fbias of DET_H was approximately 1.0, indicating the congruence between the numbers of forecasted and observed wind speed ramps. The POD values of approximately 0.45 for ramp‐ups and approximately 0.35 for ramp‐downs of DET_H were much higher than those of iENS21 and DET_L. Meanwhile, the SR values did not vary between forecasts.
3.3 Probabilistic insight of mesoscale ensemble WRF forecasts
This subsection provides probabilistic insights from ENS21. First, the reliability of ENS21 was assessed using reliability diagrams (as illustrated in Figure 5)25 that summarize the correspondence of the forecast probabilities (horizontal axis) with the frequency of observed ramp occurrences (vertical axis) given the forecast. The forecast probability of ramp events is defined as the fraction of ensemble members predicting events to the total number of members.
The results show that both ramp‐up and ramp‐down forecasts are overconfident, ie, low risks are underestimated and high risks overestimated. The slopes of the regression lines are 0.56 and 0.42 for ramp‐up and ramp‐down forecasts, respectively. According to Weisheimer and Palmer,26 the reliability of the ramp‐up forecasts can be classified as “still very useful for decision‐making,” whereas the ramp‐down forecasts are identified as “marginally useful.” One interesting observation that should be noted is that the number of forecasts tended to decrease with lower probability bins (upper‐left bar graph of the reliability diagrams); however, the number increases again at the highest probability bins. This implies the uncertainty of ENS21, which is characterized by the collective positive forecasts of wind speed ramp occurrences.
On the other hand, the ability of the forecast systems to discriminate between situations leading to yes or no regarding the occurrence of wind speed ramps is verified using the relative operating characteristic (ROC) diagrams (Figure 6).27 In the ROC diagram, both the POD (vertical axis) and the POFD (horizontal axis) are considered. A perfect forecast system (no false alarms and all hits) is positioned at the upper‐left corner on the ROC space. In Figure 6, ENS21 is represented as a curve, and DET_L and DET_H are points on the ROC space. For the ramp‐up forecast, the ROC score (the area under the ROC curve) of ENS21 is 0.68, higher than that of DET_H and DET_L, which are 0.64 and 0.61, respectively. For the ramp‐down forecast, the ROC score of ENS21 is 0.64, while those of DET_H and DET_L are 0.61 and 0.58, respectively. This result implies that the ENS21, DET_H, and DET_L are able to discriminate the situation leading to yes and no ramp forecasts, but the skill is still at fair or poor levels.
Figure 7 illustrates the results of ENS21 versus DET_L and DET_H in the performance diagram space. Here, the results of ENS21 are represented as a curve at different probabilistic thresholds. At the most strict probabilistic threshold of 21/21, ie, forecast‐yes of a ramp only if it is forecasted by all ensemble members, ENS21 appears to under‐forecast the number of actual ramp occurrences. The misses are large, and the POD is almost zero. With probability thresholds reduced, the number of ramps predicted by ENS21 increases, thus remarkably increasing the chances of Hits and POD. Concurrently, the chance of a false alarm also increases, causing SR to gradually decrease. For ramp‐up (ramp‐down) forecasts, CSI reaches its largest value of approximately 0.35 (0.29) at the threshold of 2/21 (1/21). On the other hand, the CSI values of DET_L and DET_H are 0.26 and 0.33, respectively, for ramp‐up forecasts and 0.19 and 0.26, respectively, for ramp‐down forecasts.
3.4 Reduction in the ensemble size
A disadvantage of the ensemble forecast is a huge computational resource required compared with a deterministic forecast. With 20 processors, one member of ENS21 took about 18 minutes to finish a forecast. It is comparable with DET_L, which required also about 18 minutes to have the same job done. On the other hand, the higher‐resolution DET_H forecast needed about 59 minutes to finish a forecast.
The skill of the ENS8 forecast system is verified by comparing to that of ENS21. There is almost no difference in the performances between ENS8 and ENS21 in terms of the climatological seasonal and diurnal distribution of wind speed ramps (see Figure 2). In Figure 8, the performance curve of ENS8 almost overlaps with that of ENS21. The maximum CSI of ENS8 for the ramp‐up forecast can reach 0.34 at the probabilistic threshold of 1/8, comparable with the maximum CSI of ENS21, which is 0.35 at the probabilistic threshold of 2/21. The results show that there is almost no difference between ENS21 and ENS8. The ENS8 can maintain the distribution of the ensemble forecast represented in the performance diagram. On the other hand, the reliability of ENS8 and skill of ENS8 on the ROC space are also compared with those of ENS21 (not shown here). The results show that there is not an obvious difference in the reliability curves and ROC curves between eight‐member and 21‐member mesoscale ensemble forecasts.
3.5 Discussion
In the present study, the mesoscale ensemble of IBCs for the WRF forecasts is provided by the GEFS data. Note that the techniques to create perturbed initial conditions for GEFS can affect or even create systematic biases in forecast results. These biases of “mother” models can thus be inherited by mesoscale NWP models. Thus, the use of other ensemble datasets, for example, from the European Centre for Medium‐Range Weather Forecasts or the Japan Meteorological Agency, is reasonable in order to generalize the results of this study.
The performance of ramp‐ups is better than ramp‐downs (Figures 5‐7). Commonly, ramp‐ups are driven by the passage of weather systems such as fronts, low pressures, that usually provoke sudden wind increases; whereas, ramp‐downs are caused by reverse processes such as relaxation after frontal passages, boundary‐layer stabilization that occur usually in more gradual mode.28 This implies a greater uncertainty associated with ramp‐down forecasts than ramp‐up, and why they are more difficult to be predicted by the models. This result is consistent with that of Deppe et al7 that also showed the poorer performance of ramp‐downs compared with ramp‐ups.
This study attempted to bridge a study gap raised by Zhang et al11 that highlighted the need of the development of probabilistic wind ramp forecasts through NWP ensembles. In this study, the probabilistic insight into the ensemble forecast was provided with using verification tools such as the reliability and ROC diagrams. The ROC diagram in particular can represent intuitive trade‐off between POD and POFD (Figure 5), implying that it can help wind farm operators to choose the best operating probability threshold that can optimize the expected expenses,29 though further analysis on the operating cost is not within the scope of this paper. In addition, this study demonstrated that the reduction of the ensemble size of ENS21 from 21 to eight can save the computational cost, while retaining the characteristics of the probabilistic forecast. With eight members, ENS8 has the comparable computational cost with the higher resolution deterministic forecast, DET_H, in the case of perfect parallel computation. However, ENS8 is expected to have the advantages over DET_H as it can provide the probability information about the forecast. Nevertheless, still there is an open discussion on the way of ensemble‐member selection. There is no guarantee that the method provided by this study is the best way for the ensemble‐size reduction. It is valuable to discuss on this issue in future works.
In the present study, we do not use MOS techniques for post‐processing the forecast results. The reason is that we want to verify the ability of the ensemble model itself for wind speed ramp forecasts. The use of MOS could induce confusion about which one contributes to the improvement of ramp forecasts. The use of MOS for post‐processing the outcomes of ensemble forecasts is worth evaluating in future research.
4 CONCLUSIONS
This study evaluated the ability and utility of the mesoscale ensemble forecast method for wind speed ramp forecasts. The main results of the study are as follows.
ENS21 improves the forecasts of wind speed ramps compared with the single forecast—DET_L with the same horizontal resolution. The CSI of ENS21 is 0.27 for ramp‐up and 0.20 for ramp‐down forecasts. Meanwhile, the CSI values of the DET_L are 0.22 and 0.17, respectively. ENS21 is able to provide probabilistic forecasts of wind speed ramps, and thus, it can provide multiple decision levels to wind power operators through the selection of proper probability thresholds. The disadvantage of ENS21 is that it requires higher computational costs.
The visualization by multiple probability thresholds provides a useful insight into the characteristics of the ENS21 forecasts for the ramp events. It is also demonstrated that the ENS21 can provide user‐oriented forecast information regarding wind speed ramp events, thus naturally offering a multitude of decision levels for final users compared with a single, even‐higher‐resolution deterministic forecast.
The ensemble size of the ENS21 forecast can be reduced to save the computational costs while maintaining the accuracy and the beneficial characteristics of probability forecasts. In detail, the member of the mesoscale ensemble forecast is reduced from 21 to eight members so that the computational costs of the mesoscale ensemble forecast are comparable with that of the high resolution single deterministic forecast (DET_H), which has a horizontal resolution that is a half as precise and assumes a perfect parallel computation. The verification results show that the performance of the eight‐member mesoscale ensemble forecast (ENS8) does not change much from that of the EN21 and the DET_H in terms of the CSI values.
ENS8 has advantages over ENS21 and DET_H because ENS8 is computationally efficient and is able to benefit wind power operators with flexibility in the selection of probability thresholds for decision processes compared with a single. It can be concluded that the mesoscale ensemble forecast method is more useful for prediction of the wind speed ramp than the single deterministic forecast method with the same computational cost if the ensemble members are successfully selected.
ACKNOWLEDGEMENTS
This paper is based on results obtained from a project commissioned by the New Energy and Industrial Technology Development Organization (NEDO). The WRF simulations were conducted under the technical support of Ms Yoshimi Kobayashi.
