Evaluation of four numerical wind flow models for wind resource mapping



A wide range of numerical wind flow models are available to simulate atmospheric flows. For wind resource mapping, the traditional approach has been to rely on linear Jackson–Hunt type wind flow models. Mesoscale numerical weather prediction (NWP) models coupled to linear wind flow models have been in use since the end of the 1990s. In the last few years, computational fluid dynamics (CFD) methods, in particular Reynolds-averaged Navier–Stokes (RANS) models, have entered the mainstream, whereas more advanced CFD models such as large-eddy simulations (LES) have been explored in research but remain computationally intensive. The present study aims to evaluate the ability of four numerical models to predict the variation in mean wind speed across sites with a wide range of terrain complexities, surface characteristics and wind climates. The four are (1) Jackson–Hunt type model, (2) CFD/RANS model, (3) coupled NWP and mass-consistent model and (4) coupled NWP and LES model. The wind flow model predictions are compared against high-quality observations from a total of 26 meteorological masts in four project areas. The coupled NWP model and NWP-LES model produced the lowest root mean square error (RMSE) as measured between the predicted and observed mean wind speeds. The RMSE for the linear Jackson-Hunt type model was 29% greater than the coupled NWP models and for the RANS model 58% greater than the coupled NWP models. The key advantage of the coupled NWP models appears to be their ability to simulate the unsteadiness of the flow as well as phenomena due to atmospheric stability and other thermal effects. Copyright © 2012 John Wiley & Sons, Ltd.


The optimal design of a wind project and the accurate prediction of its energy production depend on having an accurate and detailed understanding of the spatial distribution of the wind resource across the project area. Currently, numerical wind flow models, combined with onsite meteorological measurements, are the preferred approach for estimating this distribution. It is consequently important to continually assess potential improvements in such models.

Linear wind flow models such as WAsP,[1, 2] MS3DJH/MsMicro[3, 4] and MSFD[5] are widely used to predict the spatial variation of the average wind speed, directional frequency distribution (wind rose), wind shear and other boundary layer characteristics. Most such models are based on the theory of Jackson and Hunt.[6] They came into wide use in the 1980s when the computing resource was very limited. They run fast while performing reasonably well where the wind is not significantly affected by steep slopes, flow separations, thermally driven flows, low-level jets and other dynamic and nonlinear phenomena.

Reynolds-averaged Navier–Stokes models (referred to as RANS or CFD) are emerging as an alternative to linear models for wind energy applications. RANS models solve the conservation of mass and momentum equations, but the conservation of energy equation is not usually included. The RANS models assume steady-state flows so they tend to run relatively fast on a standard personal computer (PC). Usually, the simulation proceeds with a constant inlet wind profile until convergence is reached. For idealized cases, i.e. 2D or 3D flow over escarpments and hills, such steady-state RANS models perform well and give a high level of detail on the turbulence characteristics of the flow.[7] Several research studies at wind farm sites show that RANS models can perform better than the industry standard WAsP model, but others show little or no improvements over WAsP, and in some cases WAsP performs better.[8-11]

On the next rung up the ladder of sophistication are mesoscale numerical weather prediction (NWP) models (e.g. MASS,[12] ARPS,[13, 14] KAMM[15]). In principle, fully compressible, non-hydrostatic NWP models can simulate a broad range of meteorological phenomena from the synoptic to the microscales. However, the required computing power is substantial and increases rapidly with finer grid spacing. To circumvent this issue, NWP models are usually coupled to a diagnostic (microscale) wind flow model to achieve a higher spatial resolution. The microscale models used for this purpose include Jackson–Hunt-type models (e.g. WAsP, MsMicro) and mass-conserving models (e.g. WindMap,[16] CALMET[17]). Two leading examples of such coupled mesoscale–microscale models are the KAMM/WAsP system developed by Risø National Laboratory[18] and the SiteWind system developed by AWS Truepower.[16] AWS Truepower's approach is to run the mesoscale model (MASS) for a sample of days in nested grids from 30 km grid spacing down to 1.2 or 0.4 km. Then, the mean wind flow is downscaled to approximately a 50 m grid spacing using the microscale model (WindMap). Previous research has suggested that this approach is generally more accurate than WAsP over wind-project-scale distances in complex terrain, especially where mesoscale circulations have a significant impact on the spatial distribution of the wind resource.[19]

The next step in sophistication is coupled NWP and RANS models. Lately, research studies on the transport and dispersion of contaminants have relied on such coupled models.[20, 21] A typical approach is to initialize the RANS model from a single point or sounding profile from the mesoscale NWP model rather than from the full 3D gridded data. Since RANS models were initially developed in the engineering discipline to study flows around objects (e.g. airfoils), they do not include a complete conservation of energy equation, if any. Therefore, a challenge is to preserve the consistency of the thermodynamic (pressure, air density, etc.) and turbulent (turbulent kinetic energy, dissipation rates) quantities between the NWP and RANS model.

The next level of sophistication is NWP models coupled to LES. LES models have their origin in meteorology and weather prediction.[22-24] They solve the unsteady, nonlinear Navier–Stokes equations with the full physics parameterization schemes (radiation, microphysics, cloud convection, land surface-atmosphere interaction, turbulence, etc.). They are run at a high resolution compared with NWP models, i.e. close to the inertial sub-range of 3D turbulence, and are therefore able to explicitly resolve the energetically important eddies of the flow while parameterizing the small ones. The validity of LES depends crucially on the quality of the chosen turbulence closure scheme because of limited grid resolution and thermal stratification effects. However, LES models are mainly used as a research tool since the necessary computing power represents a major hurdle.

The present study aims to test a range of models at four sites with different topographic and surface characteristics and wind climate conditions. One of the sites is in flat terrain, one is in a coastal area and two are in mountainous terrain. Four different numerical wind flow models are compared:

  • WAsP—a linear Jackson–Hunt wind flow model
  • Meteodyn WT—a CFD/RANS model
  • SiteWind—a coupled mesoscale NWP–mass consistent model
  • ARPS—a coupled mesoscale NWP-LES


2.1 Site description

The terrain elevation and met mast locations for the four sites in this study are shown in Figure 1. These wind farm sites were chosen because of the diversity of conditions they represent and because they offer an abundance of high-quality data. The terrain ruggedness index (RIX)[25] at the met masts is around 0%, 0–3%, 0–1% and 0–8% for sites 1, 2, 3 and 4, respectively.

Figure 1.

Terrain elevation maps with met mast locations at the four sites. Site 1 (a) is fairly flat whereas site 2 (b), site 3 (c) and site 4 (d) are more complex. The simulation domain was different for each site: 12 km × 17 km for site 1, 12 km × 12 km for site 2 and 17 km × 17 km for Sites 3 and 4.

Site 1 is located in the US Great Plains. The region consists of gently rolling hills, and the terrain in the vicinity of the wind farm is fairly flat (Figure 1, top left panel). The land cover is mostly grassland with patches of trees. During the wind resource assessment campaign, eight 60 m tall towers were installed. Each is equipped with RM Young (R. M. Young, 2801 Aero Park Dr. Traverse City, MI) prop-vane and NRG (NRG Systems, Inc., 110 Riggs Road Hinesburg, VT) #40 cup anemometers at three measurement heights between 32 and 58 m above ground. The prevailing wind directions are from the south and north.

Site 2 is located in a broad gap in the US eastern Rocky Mountains in mainly sloping terrain (Figure 1, top right panel). The mostly grassland ground cover means the surface roughness in this area is relatively low. A total of six 60 m tall towers are equipped with NRG #40, WindSensor (WindSensor, Søkrogen 9 DK-4000 Roskilde Denmark) and Vector (Vector Instruments, 115 Marsh Road RHYL, North Wales LL18 2AB, United Kingdom) cup anemometers. The wind speeds at each mast are measured at three heights ranging from 32 to 58 m above ground. The prevailing wind direction is almost exclusively from the southwest.

Site 3 is located in Canada on a peninsula overlooking one of the Great Lakes. The project area is primarily forested with trees of approximately 15 m height and clearings of 30–200 m radius around the met masts. The three 80-m tall towers are equipped with NRG #40 cup anemometers. The measurement heights range from 38 to 76 m above ground. The prevailing wind directions are from the northwest and southeast, which correspond to flows perpendicular to the coastline.

Site 4 is located in the Gaspé Peninsula in Canada on an elevated feature about 35 km inland from the St. Lawrence River. The site is considered complex because of variations in terrain elevation and the surrounding forests with tree heights of approximately 10 m. A total of nine 50 m towers are equipped with NRG #40 anemometers. The wind speeds are measured at three heights ranging from 22 to 50 m above ground. The prevailing wind directions are from the northwest and southeast.

The met mast data were quality controlled using automated routines and manually screened by an experienced meteorologist/analyst. The data were adjusted for effects on anemometer readings of the tower, turbulence and, where necessary for NRG #40 anemometers, dry friction whip (DFW). (A large proportion of NRG #40 anemometers manufactured between May 2006 and December 2008 have been found to experience DFW, a vibratory phenomenon that causes sensors to under-report wind speed. For NRG #40 sensors experiencing DFW, AWS Truepower has developed a correction to remove the bias.[26])

In addition, a long-term climate correction was applied to the data using nearby reference stations and a linear measure–correlate–predict method.[28] It is standard practice within the wind energy industry to install tall towers with anemometers and record the onsite winds for a period of a year or more. The main purpose of the climate correction was to adjust the mean wind speeds among the masts at a site to a common period (the reference station period of record). Once the met mast data were adjusted in this fashion, the met mast time series data consisting of 10 min averages were converted to a TAB file, i.e. binned by the frequency of wind speeds and directions. TAB files are used by the numerical models either as initial conditions or in post-process, depending on the type of model. Nevertheless, all four numerical models had access to the same TAB files.

2.2 Linear Jackson–Hunt wind flow model

The Wind Atlas Analysis and Application Program (WAsP) developed at Risø DTU National Laboratory is a spectral model based on the Jackson–Hunt theory. The model solves the linearized Navier–Stokes equations under several assumptions: steady-state flow, linear advection and first-order turbulence closure. In addition, the terrain is only taken into account as a first-order perturbation. Forested areas are modeled by specifying the appropriate roughness length surface based on a roughness length to canopy height ratio of 0.075 but without a zero plane displacement height. In order to have a fair comparison, we have used the same terrain elevation and surface roughness maps without displacement height for all four numerical models. Although some studies[27] have recommended to employ a displacement height in WAsP and have shown benefits at heights of one or two times the tree height, our study focused mainly on measurement heights around 60–80 m corresponding to at least five times the tree heights. The displacement ratio should have less impact on the wind speeds at higher altitudes. In addition, since the mean wind speed at the target mast is based on the one at the reference mast through WAsP's double extrapolation method, the impact of the displacement height is likely reduced when the two masts are located in a similar land cover environment.

A schematic description of the WAsP process is shown in Figure 2. The necessary inputs to WAsP include the terrain elevation and surface roughness as well as the measured mean wind speeds and frequencies by direction sectors from onsite meteorological masts. Like most diagnostic microscale models, WAsP calculates the mean wind flow for each directional sector independently. In this study, wind resource grid (WRG) files were generated with a grid-cell spacing of 50 m using WAsP version 10. In order to improve the accuracy of the predictions, some analysts have proposed applying corrections based on their experience or on RIX.[25] For comparability with the other wind flow models, however, no such corrections were applied here.

Figure 2.

Diagram of a typical microscale modeling process. The diagnostic models simulate the mean wind flow over N independent directional sectors.

2.3 Computational fluid dynamics/Reynolds-averaged Navier–Stokes model

The CFD/RANS model, Meteodyn WT, developed by Meteodyn, solves the nonlinear Navier–Stokes momentum equation with the MIGAL solver. This RANS model assumes an incompressible, steady-state flow and uses a k-l turbulence model based on Yamada[29] and Arritt.[30] The Meteodyn WT model considers the forest canopy as a volume within the simulation domain with a resistive body force. Without solving the conservation of energy equation, the model partially takes into account thermal stability through an adjusted turbulence length scale in the turbulent kinetic energy equation. There are 10 different thermal stability classes to choose from in the model. However, in the absence of surface heat flux or temperature profile measurements, we were not able to calculate the stability parameter at the sites. Therefore, the first set of Meteodyn WT simulations were carried assuming neutral atmospheric conditions (L = 10,000 m) at each site, where L is the Monin–Obukhov length. Near-neutral condition is the recommended (default) option when the atmospheric stratification is unknown according to the Meteodyn WT User's Manual.[31] A second set of Meteodyn WT simulations was designed to capture thermal stability effects. In the absence of measurements to determine the thermal stability, an equal weight of one-third was assigned to each three thermal stability classes (unstable, neutral, stable). A one-third weighting factor fits relatively well with previous analyses made by Barthelmie et al.[41] at an onshore met mast in a coastal environment. For the stable (unstable) cases, we performed Meteodyn WT simulations assuming a Monin–Obukhov length L = 500 m (L = −500 m) which is the middle point of the very stable (unstable) and stable (unstable) atmospheric classes as defined in Van Wijk et al.[40] and Barthelmie et al.[41] We are also aware that the stability parameter z/L is another metric for classifying thermal stability where z/L = 0 for statically neutral stability and a typical range of 1–5 (−5 to −1) for stable (unstable) stratification according to the American Meteorological Society glossary. Following this classification, atmospheric conditions with L = 500 m would be considered mildly stable assuming a measurement height of 60 m. However, to the best of our knowledge there are no standards (yet) in the wind energy community for classifying atmospheric stability conditions. In this study, we relied on the classification of Van Wijk et al.[40] and Barthelmie et al.[41]

The initial conditions at the inlet are set using the surface roughness, a corresponding reference logarithmic wind speed profile in the surface layer, and an Ekman wind speed profile for the remainder of the planetary boundary layer (PBL). A diagram of the Meteodyn WT process is shown in Figure 2. As for most microscale models, Meteodyn calculates the mean wind flow independently for each directional sector. In post-processing, the program calculates a directional speed ratio between every point and the mast and applies those ratios to the observed wind speeds. In this study, Meteodyn version 4.2.0 was used and WRG files were generated with a 50 m grid spacing.

2.4 Coupled mesoscale NWP and mass-consistent model

The SiteWind system developed at AWS Truepower is composed of the mesoscale atmospheric simulation system (MASS)[12] and the WindMap mass-consistent diagnostic model.[16] MASS is a fully compressible NWP model that can run in hydrostatic or non-hydrostatic mode. The PBL scheme implemented in MASS is based on Therry and LaCarrère.[32] As displayed in Figure 3, the MASS model is typically initialized by the NCEP/NCAR reanalysis[33] on the T62 Gaussian grid, which has a spatial resolution of approximately 1.875°. An optimal interpolation scheme is used to assimilate data every 12 h from radiosonde observations and surface stations, if available. Various surface fields (snow cover, soil moisture, soil type, etc.) are ingested by the NWP model to model surface–atmosphere interactions (i.e. the surface energy budget). Following a dynamical downscaling approach, MASS is run in cascade mode from a 30 km grid mesh down to 1.2 km (standard SiteWind) or 400 m (high-resolution SiteWind). An example of a nested grid configuration is shown in Figure 4. To obtain a statistically representative picture of the wind climate, a sample of 72 days within a 15 year period is modeled by MASS. The case days are chosen using a stratified sampling so that each month is equally weighted.

Figure 3.

Diagram of NWP or LES modeling process. A sample of N days is run with these unsteady models to capture the local wind climate at the site of interest. SST and NDVI refer to sea surface temperature and normalized difference vegetation index.

Figure 4.

Dynamical downscaling based on nested grids for NWP or LES modeling.

The next step is to downscale the mean wind flow modeled by MASS to a 50 m grid spacing using a mass-consistent model, WindMap.[34] The WindMap model ingests high-resolution terrain and land use data as shown in Figure 2. To model forested areas, MASS and WindMap rely on the surface roughness map which employs a ratio of 0.075 for the surface roughness length to canopy height. In stand-alone mode, WindMap (such as WAsP and Meteodyn WT) is initialized by data from a met mast. In SiteWind, it is initialized by mean wind fields generated by MASS. WindMap outputs a 50 m grid spacing WRG file. In post-process, the WRG file is adjusted to a single onsite met mast, i.e. the reference mast, using the ratio of observed over modeled wind speeds by direction sectors. One at a time, each met mast except the target mast is used as a reference mast to adjust the WRG file and to predict the mean wind speed at that specific target mast. This adjustment procedure is repeated in a round-robin fashion for all the masts as described in Section 2.7 to adjust the modeled wind speeds. In this study, SiteWind relied on MASS version 6.8 and WindMap version 1.3.8.

2.5 Coupled mesoscale NWP and LES model

The advanced regional prediction system (ARPS) was developed at Oklahoma University[13, 14] for the explicit prediction of convective and cold-season storms as well as mesoscale weather systems. ARPS includes a fully compressible non-hydrostatic NWP model. The PBL parameterization scheme follows Sun and Chang.[35] Forested areas are taken into account by specifying the appropriate roughness length based on a surface roughness length to canopy height ratio of 0.075. At sufficiently fine grid resolutions and time steps, ARPS switches from an NWP to a LES model, assuming a sub-grid scale (SGS) turbulence parameterization scheme is selected. In this study, we relied on ARPS version 5.2.10 and chose an SGS scheme based on Moeng[23] and Deardorff.[36] Since ARPS was developed to resolve cloud-scale features, it was not optimized to ingest coarse resolution reanalysis data. For high-resolution wind flow modeling studies, we initialize ARPS with the North American mesoscale (NAM) analyses, which have a horizontal grid resolution of 12.2 km. Nested-grid ARPS simulations are conducted for grid spacings ranging from 12 km down to 400 m in NWP mode and then in LES mode at 90 m (Figure 4). A grid spacing of 90 m is on the coarser end of the spectrum for large-eddy simulations[39-41] but was chosen to keep the computational burden manageable.

Since the NAM analyses are not available online prior to 2004, a sample of 72 days from a 5 year period is simulated by ARPS. As in the SiteWind process (Figure 3), the case days are chosen so that each calendar month is equally weighted. Once the ARPS simulations are completed, the LES model outputs are summarized in a WRG file. In post-process, the same adjustment procedure as described in Section 2.4 is applied to the ARPS modeled data. One at a time, each met mast except the target mast is used to adjust the WRG file and to predict the mean wind speed at that specific target mast. This adjustment procedure is repeated in a round-robin fashion for all the masts as described in Section 2.7. Needless to say, an NWP or LES model can generate time-varying 3D meteorological fields but for this comparison study we focused on the averaged wind speeds and frequencies by direction sectors contained in the WRG file.

2.6 Runtime

The numerical model runtimes were compared by running a simulation with each model over a 12 km × 12 km domain with eight met masts and creating the WRG files. As described in previous subsections, the diagnostic microscale models (Jackson–Hunt, RANS and mass-consistent) are run in a different way from the NWP-based models (Figures 2 and 3). The diagnostic microscale models, which are time independent, were run over 12 direction sectors (each 30° wide). The WAsP model has a longer runtime than Meteodyn WT in this comparison since WAsP must be run once for each mast whereas Meteodyn WT is run once independent of the number of masts. It should also be noted that WAsP was designed and optimized to calculate the mean wind speeds at discrete independent points, e.g. turbine locations, rather than over a 2D or 3D grid. The NWP models (both MASS and ARPS) are run for 72 representative case days. Each case day consists of a 36 h simulation, i.e. a 12 h spin-up and a 24 h hindcast. The diagnostic microscale model runs were performed on a single processor on a standard personal computer. The NWP and NWP-LES model runs were performed on a Linux cluster. A total of 80 dual quad-cores were available, and multiple NWP and LES model simulations were run at the same time. If the 72 case days are each run on an individual core, the wall clock time of the 1.2 km grid spacing NWP model runs is around 12 h (Table 1). The LES runs are much more computationally intensive than the NWP runs. In order to manage the computational burden, we ran each LES simulation on 16 cores using parallel processing; a 36 h simulation typically completes in 72 h of wall clock time.

Table 1. Approximate runtimes in CPU hours for the different numerical models for a typical 12 km × 12 km domain with eight met masts.
Model (resolution)Jackson–Hunt (50 m)CFD/RANS (50 m)NWP/mass-consistent (1.2 km/50 m)NWP/mass-consistent (400 m/50 m)NWP/LES (90 m)
  1. The mass-consistent, Jackson–Hunt and CFD/RANS models are run over 12 direction sectors. The NWP and LES models are run for 72 representative 24 h days. The diagnostic microscale models are run on one processor on a PC. The NWP and LES models are run on a Linux cluster.

  2. Standard PC → 1 CPU at 2.50 GHz, 6 GB of RAM; Linux cluster → 1 core at 2.66 GHz, 16 GB of RAM per nodes.

Unitper mastper directionper dayper dayper day
Total time (CPU hours)48308642304187,20

2.7 Metrics of error

The metrics of error for each site and model are the mean bias and the root mean square error (RMSE) between the predicted and observed mean wind speeds, defined as follows

display math(1)
display math(2)

where N is the number of masts and Ui,j is the predicted mean wind speed at mast i based on the reference mast j and Uiobs is the observed mean speed at mast i. Each numerical model outputs the mean wind speed and frequency by direction sector in a WRG file. At a site containing N met masts, each numerical model generates N WRG files, i.e. one for each mast. The met mast used in creating a WRG file is called the reference mast, as the modeled mean wind speed at the met mast location is equal to the measured. As can be seen from equations (1) and (2), the validation is performed in a round-robin fashion where the bias and RMSE are calculated using the difference between the predicted and observed mean wind speeds at every met mast location except the reference mast (i ≠ j). The predicted mean wind speed at the target mast i is taken from a WRG file that was adjusted to another met mast than the target mast, i.e. the reference mast j. Since there are N − 1 possible reference masts, the difference between the predicted and observed mean wind speeds at a target met mast is calculated N − 1 times, each time omitting the measurements at the target mast. This round-robin approach allows the mean wind speed to be predicted at the target mast independently of any measurements taken there.


3.1 Local near-surface wind climates

The diurnal variations in mean wind speed and wind shear suggest that the atmospheric stability is quite different between day and night at each site. The mean wind speeds and shear exponents by time of day were calculated at one met mast within each site. The mean wind speeds were calculated using the top anemometers. Following one of the most widely used methods for calculating the wind shear,[28] we define the wind shear exponent to be the α in the power law:

display math(3)

where U1 and U2 are the measured wind speeds at the corresponding heights h1 and h2 of the anemometers on a mast. The mean diurnal wind shear exponents were calculated between the bottom and top levels on the mast, i.e. between 32 and 58 m for sites 1 and 2, between 38 and 76 m for site 3 and between 30 and 48 m for site 4. Figure 5 shows that the wind shear exponents are significantly higher at night than during the day at all sites. This pattern follows the common experience that over land, radiative cooling at night tends to create a stable surface layer whereas solar heating during the day tends to create a convective layer. Unstable layers are characterized by lower wind shears compared with stable layers because of turbulent mixing. The wind roses are similar between daytime (09:00–15:00) and nighttime (21:00–03:00) at all sites. Therefore, the wind shear differences between daytime and nighttime are mainly due to the atmospheric stability. A study by Barthelmie et al.[41] indicates that even in a coastal area, the PBL is non-neutral, i.e. stable or unstable, approximately 70% of the time on average, although the proportion varies with time of day, month (season), wind speed and direction.

Figure 5.

Diurnal mean wind speeds (grey) at the tall tower's top level and diurnal wind shear exponents (black) between the tall tower's bottom and top levels at site 1 (top left), site 2 (bottom left), site 3 (top right) and site 4 (bottom right). The period of records at each met mast covered at least 1 year.

As described in Section 2.1, site 1 is located in relatively flat and homogeneous terrain. Given that grassland covers most of the area surrounding the met mast, the surface roughness is relatively low, and it would be expected that the wind shear exponent would also be relatively low on the basis of the Monin–Obukhov theory under neutral stratification. Figure 5 shows that the mean wind shear exponent (below 80 m) and wind speed (at 80 m) are significantly higher at night, reflecting frequent nocturnal low-level jets (LLJs) because of the decoupling of the PBL after sunset (i.e. stable surface layer), as well as the development of a lee trough east of the Rocky Mountains.[42] An LLJ is a thin layer of high wind speeds above the surface layer. Using a TV tower with six measurement heights located south-east of site 1, Crawford and Hudson [43] demonstrated that the nocturnal LLJ maximum in this region typically occurs from 90 m to 444.5 m above ground.

The wind shear exponent profiles at site 2 show much smaller diurnal variations but the wind speed profiles are opposite to the pattern seen at site 1. The mean wind speeds are stronger during the daytime when the wind shear exponent is low (~0.07) and relatively weaker at night. A reasonable explanation is that solar heating induces daytime convection which forces high momentum middle level air to be mixed down to the surface. Site 3, which has a very complex wind climate because of land–lake circulations, complex topography and forest, experiences higher wind speeds at night. The mean wind shear exponent averages 0.25 during the day and 0.35 during the night. From a visual inspection of the wind speed and potential temperature profiles predicted by the mesoscale model MASS, nocturnal LLJs seem to occur somewhat frequently. Finally, according to measurements at site 4, there are not much diurnal variations in wind speeds but the mean shear exponent varies from 0.35 during daytime to 0.45 during nighttime. Such high wind shear exponents are characteristic of forested areas.

3.2 Validation: error statistics

All four modeling systems described earlier were run at the four validation sites using the same met mast data, terrain elevation and surface roughness maps, with the exception of the two mesoscale NWP models, MASS and ARPS, which employed coarser resolution topographic and land cover data sets. Apart from the different set of equations at the core of each model, a major difference between SiteWind and ARPS versus WAsP and Meteodyn is that the latter needed the met mast data as input whereas the coupled mesoscale and microscale systems used it in post-process.

At each validation site, the error statistics—the mean bias and RMSE—were calculated from equations (1) and (2) using the modeled and observed mean wind speeds. The total bias and RMSE were calculated using equations (4) and (5) where k represents the site and N the number of masts at site k. The results are shown in Table 2. The RMSE values in Table 2 reflect possible errors both in the observed mean speeds (adjusted for tower effects, turbulence, height, long-term climate and, where appropriate, DFW) and the numerical model outputs. The uncertainty in the observed mean speeds at 80 m, considering all factors, is estimated to be about 4%.

display math(4)
display math(5)
Table 2. Validation of the modeled mean wind speeds at 80 m above ground level. using the bias and RMSE calculations from equations (1)(4).
  • *

    For site 3, the simulation domain was too small to include mast 2.

Land coverMixedOpenForestedForested 
Number of masts863*926
Mean distance between masts7.3 km5.0 km5.7 km6.0 km 
 Bias/RMSE (m s−1 and %)
Meteodyn WT (neutral)0.03/0.500.06/0.460.18/1.070.30/0.950.15/0.76
Meteodyn WT (unstable/neutral/stable)0.02/0.410.09/0.550.20/1.090.33/0.970.16/0.76
SiteWind (experimental)0.01/0.24−0.02/0.30−0.02/0.59−0.11/0.63−0.04/0.46

The mean bias is generally close to 0% for all numerical models, as one would expect given the round-robin approach for the validation (Section 2.7). The error in predicting the mean wind speed at mast a using mast b as a reference (i.e. inline image) should normally be offset by the error of predicting the speed at mast b using mast a as a reference (i.e. inline image).

With respect to the RMSE, which is related to the modeling uncertainty, all four numerical models performed better in flat terrain than in complex terrain. The RMSE for all models is nearly twice as large at sites 3 and 4 (complex terrain with forest) as at site 1 (flat terrain). The coupled mesoscale and mass-consistent model, SiteWind, performed almost as well as the coupled mesoscale NWP and LES model, ARPS. SiteWind has the advantage of a final horizontal grid spacing of 50 m, compared with 90 m for ARPS (Table 1). It is also worth noting that MASS and ARPS did not use the same initial and boundary conditions as mentioned in Sections 2.4 and 2.5. The SiteWind system has relied on MASS which is initialized by the NCEP/NCAR reanalysis data whereas ARPS was not optimized to ingest coarse resolution reanalysis data. When the MASS model within the SiteWind system was run at a higher resolution of 400 m instead of the default 1.2 km grid spacing, the performance varied from site to site but overall did not improve much compared with the standard SiteWind configuration. Most important, all three coupled NWP models exhibited lower error statistics than the Jackson–Hunt and RANS models. Table 2 shows that SiteWind and ARPS had similar total RMSE around 0.45–0.48 m s−1 (5.4–5.8% of the mean speed) compared with 0.62 m s−1 (8.0%) for WAsP and 0.76 m s−1 (9.4%) for Meteodyn WT. Running Meteodyn WT simulations with an equal mix of unstable, neutral and stable atmospheric conditions affected the results at individual sites but did not improve the overall error statistics compared with the standard configuration, i.e. neutral stratification.

Figure 6 shows a side-by-side comparison of the mean wind speed maps generated by MASS and SiteWind at site 3. The mean wind speed maps generated by the NWP model alone suggest that the mesoscale model captures the areas of relatively higher and lower wind speeds. However, the coarser horizontal resolution of the mesoscale model (1.2 km grid spacing) produces relatively smooth mean wind flow features in comparison with the coupled mesoscale and microscale system (50 m grid spacing). The advantage of using a coupled mesoscale NWP model with a mass-consistent model is that the latter is able to capture the fine details of the topography and surface roughness while preserving much of the information from the mesoscale simulations (e.g. low-level jets, land/sea breezes, mountain-valley circulations, etc.). A research study by Frank et al. [44] has demonstrated that a coupled mesoscale NWP and microscale model shows improvement over a mesoscale model alone.

Figure 6.

Mean wind speed maps in m/s generated by MASS (left) and SiteWind (right) at Site 3. The black lines correspond to the terrain elevation contours every 100 m. The land-water mask is defined by the white contour lines.


Four numerical wind flow models have been run at four sites having differing wind climates and surface characteristics and a total of 26 met masts. All four modeling systems used the same met mast data, terrain elevation and surface roughness maps, with the exception of the two mesoscale NWP models, MASS and ARPS, which employed somewhat coarser-resolution topographic and land cover data sets.

On-site measurements of wind speeds and wind shears suggest that thermal effects have significant impacts on the mean wind speeds within the first 100 m of the PBL. Although the mix of atmospheric stability conditions (unstable, neutral and stable) varies from site to site, daytime convection due to solar heating of the surface and nocturnal stability due to radiative cooling are common. Typically, near-neutral atmospheric conditions develop either when the winds are moderate to strong under cloudy skies or around sunrise and sunset. Given the wide range of stability conditions in the real world and their effects on both observed wind shear and wind speed, it seems important to develop wind flow models that can handle them (as well as other thermal effects).

As expected for the round-robin validation exercise reported here, the mean bias is generally close to 0%. Of much greater interest is the RMSE. SiteWind and ARPS exhibit similar total RMSEs of around 0.45–0.48 m s−1 (5.4–5.8% of the observed mean), compared with 0.62 m s−1 (8.0%) for WAsP and 0.76 m s−1 (or 9.4%) for Meteodyn WT. (The contribution of measurement uncertainties is estimated to be about 4%.) The linear Jackson–Hunt type model and nonlinear CFD/RANS model produce significantly higher RMSEs, approximately 29% and 58% above the value found for SiteWind and ARPS. For a fair comparison, none of the models employed a zero plane displacement height although it has been recommended for WAsP in some previous studies.[27] Regarding the Meteodyn WT simulations, the default configuration assumed neutral stratification. We found that the overall error statistics did not improve when using an equal mix of unstable, neutral and stable atmospheric conditions compared with the default simulations although it did affect the results at individual sites. Our choice of Monin–Obukhov lengths for the stable (L = 500 m) and unstable (L = −500 m) cases in the Meteodyn WT simulations was based on the classification of Van Wijk et al.[40] and Bartherlmie et al.[41] Although it has not been tested in this study, it is possible that a different choice of Monin–Obukhov lengths for the stable and unstable cases would have altered the overall results of Meteodyn WT.

It is interesting that the coupled NWP and mass-consistent model, SiteWind, performed almost as well as the coupled NWP-LES model, ARPS, despite the fact that the LES model is far more sophisticated than the mass-consistent model. This may be explained, perhaps, by the fact that the NWP-LES model was run at a lower resolution than the other models (90 m compared with 50 m). In theory, the accuracy of the LES model should improve substantially by conducting simulations at a higher resolution and with refinements to the turbulence scheme. In addition to using different final grid resolutions, MASS (within the SiteWind system) and ARPS did not use the same initial and boundary conditions. The SiteWind system typically relies on MASS initialized by the NCEP/NCAR reanalysis data whereas ARPS was not optimized to ingest coarse resolution reanalysis data.

In any case, the superior performance of all three coupled NWP models suggests that the correct simulation of thermal stability, as well as of other phenomena related to temperature and moisture gradients developed in dynamic mesoscale simulations, is of critical importance in understanding atmospheric wind flow even over domains of quite modest size, such as those studied here. The linear Jackson–Hunt type model and nonlinear steady-state CFD/RANS models do not have the physical equations to fully model such effects. A key disadvantage of the NWP-based modeling systems is the greater computer power required, especially for the coupled NWP-LES model. It is up to the user to decide whether the gain in accuracy obtained through these methods is worth the additional cost in computer time.


We would like to thank several colleagues at AWS Truepower for their help during this research study: Eddie Natenberg, Chuck Alonge, Dr. Jeff Freedman and Colin Rickert. The help of the ARPS support group is also gratefully acknowledged. This paper benefited tremendously from the comments of two anonymous reviewers.